2024-08-02T17:26:41.189 INFO:root:teuthology version: 0.0.1.dev379+g53ce146 2024-08-02T17:26:41.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-02T17:26:41.326 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714 branch: squid-release description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} email: yweinste@redhat.com first_in_suite: false job_id: '7832714' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid-release ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid-release sha1: 26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4941 sha1: 26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: squid-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi069.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDhpftqBfUaGTL4q1GIkPEM64iQnfpGKcsuqvR/XGjvDYGeOe+sGYesSm+i7CncPNKJjkNJK1e79wJed3X/A5gY= smithi077.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPf7Ks1sR0hs7oRCD0aTUCnzSR8Jt7uPFT13zJfEYMF3PN6BpSkoebDPjVHTguGpC3p8ZM1tFo6iPis+hophSlo= smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjQurSw8s8JJkaS3k+PaCC3cC48EM5gnTelUad/MherbxbUJ1x4PMGJoW89Z+vBRouQaj8+2qJjNOxm/6LyyX0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 53ce1462e129f6eb4071986336534c740fdebd31 timestamp: 2024-08-02_15:32:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.4074525 2024-08-02T17:26:41.326 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/qa; will attempt to use it 2024-08-02T17:26:41.327 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/qa/tasks 2024-08-02T17:26:41.327 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-02T17:26:41.328 INFO:teuthology.task.internal:Checking packages... 2024-08-02T17:26:41.350 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea' 2024-08-02T17:26:41.350 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-02T17:26:41.350 INFO:teuthology.packaging:ref: None 2024-08-02T17:26:41.350 INFO:teuthology.packaging:tag: None 2024-08-02T17:26:41.350 INFO:teuthology.packaging:branch: squid-release 2024-08-02T17:26:41.350 INFO:teuthology.packaging:sha1: 26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:26:41.351 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid-release 2024-08-02T17:26:41.487 INFO:teuthology.task.internal:Found packages for ceph version 19.1.0-1260.g26c3fb8e 2024-08-02T17:26:41.488 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-02T17:26:41.495 INFO:teuthology.task.internal:no buildpackages task found 2024-08-02T17:26:41.495 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-02T17:26:41.508 INFO:teuthology.task.internal:Saving configuration 2024-08-02T17:26:41.530 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-02T17:26:41.542 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-02T17:26:41.564 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi069.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-02 17:21:44.881206', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDhpftqBfUaGTL4q1GIkPEM64iQnfpGKcsuqvR/XGjvDYGeOe+sGYesSm+i7CncPNKJjkNJK1e79wJed3X/A5gY='} 2024-08-02T17:26:41.587 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi077.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-02 17:21:44.882872', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPf7Ks1sR0hs7oRCD0aTUCnzSR8Jt7uPFT13zJfEYMF3PN6BpSkoebDPjVHTguGpC3p8ZM1tFo6iPis+hophSlo='} 2024-08-02T17:26:41.607 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-02 17:21:44.879305', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJjQurSw8s8JJkaS3k+PaCC3cC48EM5gnTelUad/MherbxbUJ1x4PMGJoW89Z+vBRouQaj8+2qJjNOxm/6LyyX0='} 2024-08-02T17:26:41.607 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-02T17:26:41.614 INFO:teuthology.task.internal:roles: ubuntu@smithi069.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-02T17:26:41.614 INFO:teuthology.task.internal:roles: ubuntu@smithi077.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-02T17:26:41.614 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-02T17:26:41.614 INFO:teuthology.run_tasks:Running task console_log... 2024-08-02T17:26:41.694 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f063c89bac0>, signals=[15]) 2024-08-02T17:26:41.695 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-02T17:26:41.701 INFO:teuthology.task.internal:Opening connections... 2024-08-02T17:26:41.701 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:26:41.702 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-02T17:26:41.780 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi077.front.sepia.ceph.com 2024-08-02T17:26:41.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-02T17:26:41.857 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-08-02T17:26:41.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-02T17:26:41.931 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-02T17:26:41.938 DEBUG:teuthology.orchestra.run.smithi069:> uname -m 2024-08-02T17:26:41.956 INFO:teuthology.orchestra.run.smithi069.stdout:x86_64 2024-08-02T17:26:41.956 DEBUG:teuthology.orchestra.run.smithi069:> cat /etc/os-release 2024-08-02T17:26:42.013 INFO:teuthology.orchestra.run.smithi069.stdout:NAME="CentOS Stream" 2024-08-02T17:26:42.013 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION="9" 2024-08-02T17:26:42.013 INFO:teuthology.orchestra.run.smithi069.stdout:ID="centos" 2024-08-02T17:26:42.013 INFO:teuthology.orchestra.run.smithi069.stdout:ID_LIKE="rhel fedora" 2024-08-02T17:26:42.013 INFO:teuthology.orchestra.run.smithi069.stdout:VERSION_ID="9" 2024-08-02T17:26:42.013 INFO:teuthology.orchestra.run.smithi069.stdout:PLATFORM_ID="platform:el9" 2024-08-02T17:26:42.013 INFO:teuthology.orchestra.run.smithi069.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-02T17:26:42.014 INFO:teuthology.orchestra.run.smithi069.stdout:ANSI_COLOR="0;31" 2024-08-02T17:26:42.014 INFO:teuthology.orchestra.run.smithi069.stdout:LOGO="fedora-logo-icon" 2024-08-02T17:26:42.014 INFO:teuthology.orchestra.run.smithi069.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-02T17:26:42.014 INFO:teuthology.orchestra.run.smithi069.stdout:HOME_URL="https://centos.org/" 2024-08-02T17:26:42.014 INFO:teuthology.orchestra.run.smithi069.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-02T17:26:42.014 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-02T17:26:42.014 INFO:teuthology.orchestra.run.smithi069.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-02T17:26:42.015 INFO:teuthology.lock.ops:Updating smithi069.front.sepia.ceph.com on lock server 2024-08-02T17:26:42.038 DEBUG:teuthology.orchestra.run.smithi077:> uname -m 2024-08-02T17:26:42.055 INFO:teuthology.orchestra.run.smithi077.stdout:x86_64 2024-08-02T17:26:42.056 DEBUG:teuthology.orchestra.run.smithi077:> cat /etc/os-release 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:NAME="CentOS Stream" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION="9" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:ID="centos" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:ID_LIKE="rhel fedora" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:VERSION_ID="9" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:PLATFORM_ID="platform:el9" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:ANSI_COLOR="0;31" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:LOGO="fedora-logo-icon" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-02T17:26:42.113 INFO:teuthology.orchestra.run.smithi077.stdout:HOME_URL="https://centos.org/" 2024-08-02T17:26:42.114 INFO:teuthology.orchestra.run.smithi077.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-02T17:26:42.114 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-02T17:26:42.114 INFO:teuthology.orchestra.run.smithi077.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-02T17:26:42.114 INFO:teuthology.lock.ops:Updating smithi077.front.sepia.ceph.com on lock server 2024-08-02T17:26:42.130 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-08-02T17:26:42.148 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-08-02T17:26:42.149 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-08-02T17:26:42.206 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-02T17:26:42.207 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-02T17:26:42.208 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-08-02T17:26:42.229 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-02T17:26:42.240 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-02T17:26:42.246 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-02T17:26:42.246 DEBUG:teuthology.orchestra.run.smithi069:> test '!' -e /home/ubuntu/cephtest 2024-08-02T17:26:42.249 DEBUG:teuthology.orchestra.run.smithi077:> test '!' -e /home/ubuntu/cephtest 2024-08-02T17:26:42.251 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-08-02T17:26:42.267 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-02T17:26:42.274 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-02T17:26:42.274 DEBUG:teuthology.orchestra.run.smithi069:> test -z $(ls -A /var/lib/ceph) 2024-08-02T17:26:42.305 DEBUG:teuthology.orchestra.run.smithi077:> test -z $(ls -A /var/lib/ceph) 2024-08-02T17:26:42.307 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-08-02T17:26:42.411 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-02T17:26:42.483 INFO:teuthology.run_tasks:Running task kernel... 2024-08-02T17:26:42.497 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-02T17:26:42.497 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-08-02T17:26:42.497 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-08-02T17:26:42.498 DEBUG:teuthology.orchestra.run.smithi069:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-02T17:26:42.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T17:26:42.515 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-08-02T17:26:42.571 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-437.el9.x86_64 2024-08-02T17:26:42.571 INFO:teuthology.task.kernel:Running kernel on smithi069: 5.14.0-437.el9.x86_64 2024-08-02T17:26:42.572 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum install -y kernel 2024-08-02T17:26:45.708 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.2 MB 00:01 2024-08-02T17:26:48.844 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 20 MB 00:02 2024-08-02T17:26:54.184 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.6 MB 00:01 2024-08-02T17:26:55.867 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 18 kB 00:00 2024-08-02T17:26:57.029 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 22 MB/s | 22 MB 00:01 2024-08-02T17:27:02.494 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-08-02T17:27:04.467 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-02T17:27:04.467 INFO:teuthology.orchestra.run.smithi069.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-08-02T17:27:04.556 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-08-02T17:27:04.565 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-08-02T17:27:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: Package Architecture Version Repository Size 2024-08-02T17:27:04.565 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-08-02T17:27:04.565 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-08-02T17:27:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: kernel x86_64 5.14.0-480.el9 baseos 1.5 M 2024-08-02T17:27:04.565 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-08-02T17:27:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-core x86_64 5.14.0-480.el9 baseos 17 M 2024-08-02T17:27:04.565 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules x86_64 5.14.0-480.el9 baseos 36 M 2024-08-02T17:27:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-core x86_64 5.14.0-480.el9 baseos 30 M 2024-08-02T17:27:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:27:04.566 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-08-02T17:27:04.566 INFO:teuthology.orchestra.run.smithi069.stdout:================================================================================ 2024-08-02T17:27:04.566 INFO:teuthology.orchestra.run.smithi069.stdout:Install 4 Packages 2024-08-02T17:27:04.566 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:27:04.566 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 84 M 2024-08-02T17:27:04.567 INFO:teuthology.orchestra.run.smithi069.stdout:Installed size: 126 M 2024-08-02T17:27:04.567 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-08-02T17:27:05.112 INFO:teuthology.orchestra.run.smithi069.stdout:(1/4): kernel-5.14.0-480.el9.x86_64.rpm 3.5 MB/s | 1.5 MB 00:00 2024-08-02T17:27:07.570 INFO:teuthology.orchestra.run.smithi069.stdout:(2/4): kernel-core-5.14.0-480.el9.x86_64.rpm 5.9 MB/s | 17 MB 00:02 2024-08-02T17:27:09.812 INFO:teuthology.orchestra.run.smithi069.stdout:(3/4): kernel-modules-core-5.14.0-480.el9.x86_6 6.4 MB/s | 30 MB 00:04 2024-08-02T17:27:10.529 INFO:teuthology.orchestra.run.smithi069.stdout:(4/4): kernel-modules-5.14.0-480.el9.x86_64.rpm 6.1 MB/s | 36 MB 00:05 2024-08-02T17:27:10.529 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-08-02T17:27:10.529 INFO:teuthology.orchestra.run.smithi069.stdout:Total 14 MB/s | 84 MB 00:05 2024-08-02T17:27:11.264 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-08-02T17:27:11.612 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-08-02T17:27:11.612 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-08-02T17:27:12.615 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-08-02T17:27:12.616 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-08-02T17:27:15.088 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-08-02T17:27:15.760 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-modules-core-5.14.0-480.el9.x86_64 1/4 2024-08-02T17:27:15.862 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-08-02T17:27:18.519 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 2/4 2024-08-02T17:27:18.680 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-08-02T17:27:23.546 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-08-02T17:27:23.789 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : kernel-5.14.0-480.el9.x86_64 4/4 2024-08-02T17:27:28.674 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-08-02T17:27:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-core-5.14.0-480.el9.x86_64 4/4 2024-08-02T17:27:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-modules-5.14.0-480.el9.x86_64 4/4 2024-08-02T17:27:58.678 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: kernel-5.14.0-480.el9.x86_64 4/4 2024-08-02T17:27:58.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-5.14.0-480.el9.x86_64 1/4 2024-08-02T17:27:58.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-core-5.14.0-480.el9.x86_64 2/4 2024-08-02T17:27:58.679 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-modules-5.14.0-480.el9.x86_64 3/4 2024-08-02T17:27:59.501 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : kernel-modules-core-5.14.0-480.el9.x86_64 4/4 2024-08-02T17:27:59.501 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:27:59.501 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-08-02T17:27:59.501 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-5.14.0-480.el9.x86_64 2024-08-02T17:27:59.501 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-core-5.14.0-480.el9.x86_64 2024-08-02T17:27:59.501 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-5.14.0-480.el9.x86_64 2024-08-02T17:27:59.501 INFO:teuthology.orchestra.run.smithi069.stdout: kernel-modules-core-5.14.0-480.el9.x86_64 2024-08-02T17:27:59.501 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:27:59.502 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-08-02T17:27:59.876 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-08-02T17:27:59.966 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-02T17:27:59.967 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-02T17:27:59.967 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-480.el9.x86_64 2024-08-02T17:27:59.967 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-08-02T17:27:59.967 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-08-02T17:27:59.967 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-08-02T17:27:59.967 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi069.front.sepia.ceph.com, path=None, version=distro) 2024-08-02T17:27:59.968 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q kernel | sort -rV | head -n 1 2024-08-02T17:28:00.024 INFO:teuthology.orchestra.run.smithi069.stdout:kernel-5.14.0-480.el9.x86_64 2024-08-02T17:28:00.024 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-480.el9.x86_64 2024-08-02T17:28:00.024 DEBUG:teuthology.orchestra.run.smithi069:> sudo rpm -qi grub2-tools 2024-08-02T17:28:00.054 INFO:teuthology.orchestra.run.smithi069.stdout:Name : grub2-tools 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Epoch : 1 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Version : 2.06 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Release : 68.el9 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Architecture: x86_64 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Group : Unspecified 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Size : 8264254 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:License : GPLv3+ 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-02T17:28:00.055 INFO:teuthology.orchestra.run.smithi069.stdout:Packager : builder@centos.org 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:Vendor : CentOS 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:Summary : Support tools for GRUB. 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:Description : 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:hardware devices. 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:28:00.056 INFO:teuthology.orchestra.run.smithi069.stdout:This subpackage provides tools for support of all platforms. 2024-08-02T17:28:00.057 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-02T17:28:00.057 INFO:teuthology.task.kernel:Updating grub on smithi069 to boot 5.14.0-480.el9.x86_64 2024-08-02T17:28:00.058 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-02T17:28:00.710 INFO:teuthology.orchestra.run.smithi069.stderr:Generating grub configuration file ... 2024-08-02T17:28:02.066 INFO:teuthology.orchestra.run.smithi069.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-02T17:28:02.097 INFO:teuthology.orchestra.run.smithi069.stderr:done 2024-08-02T17:28:02.100 DEBUG:teuthology.orchestra.run.smithi069:> mktemp 2024-08-02T17:28:02.115 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/tmp.V8OTZYfQaX 2024-08-02T17:28:02.115 DEBUG:teuthology.orchestra.run.smithi069:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.V8OTZYfQaX 2024-08-02T17:28:02.181 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0666 /tmp/tmp.V8OTZYfQaX 2024-08-02T17:28:02.311 DEBUG:teuthology.orchestra.remote:smithi069:/tmp/tmp.V8OTZYfQaX is 6KB 2024-08-02T17:28:02.359 DEBUG:teuthology.orchestra.run.smithi069:> rm -fr /tmp/tmp.V8OTZYfQaX 2024-08-02T17:28:02.373 DEBUG:teuthology.orchestra.run.smithi069:> sudo /bin/ls /boot/loader/entries || true 2024-08-02T17:28:02.436 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-08-02T17:28:02.436 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-08-02T17:28:02.436 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-08-02T17:28:02.436 INFO:teuthology.orchestra.run.smithi069.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64.conf 2024-08-02T17:28:02.437 DEBUG:teuthology.orchestra.run.smithi069:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-480.el9.x86_64 2024-08-02T17:28:02.576 DEBUG:teuthology.orchestra.run.smithi069:> sudo shutdown -r now 2024-08-02T17:28:32.580 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-08-02T17:28:32.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-08-02T17:28:32.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-02T17:29:32.584 DEBUG:teuthology.orchestra.remote:timed out 2024-08-02T17:29:41.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-08-02T17:29:41.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-02T17:29:42.897 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-08-02T17:29:43.349 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-08-02T17:29:43.350 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-480.el9.x86_64"... 2024-08-02T17:29:43.350 DEBUG:teuthology.orchestra.run.smithi069:> uname -r 2024-08-02T17:29:43.365 INFO:teuthology.orchestra.run.smithi069.stdout:5.14.0-480.el9.x86_64 2024-08-02T17:29:43.365 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-480.el9.x86_64 vs 5.14.0-480.el9.x86_64 2024-08-02T17:29:43.365 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-02T17:29:43.365 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-02T17:29:44.366 DEBUG:teuthology.parallel:result is None 2024-08-02T17:29:44.367 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-02T17:29:44.376 INFO:teuthology.task.internal:Creating test directory... 2024-08-02T17:29:44.376 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-02T17:29:44.379 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-02T17:29:44.381 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-02T17:29:44.399 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-02T17:29:44.411 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-02T17:29:44.425 INFO:teuthology.task.internal:Creating archive directory... 2024-08-02T17:29:44.425 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-02T17:29:44.434 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-02T17:29:44.439 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-02T17:29:44.478 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-02T17:29:44.489 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-02T17:29:44.489 DEBUG:teuthology.orchestra.run.smithi077:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-02T17:29:44.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T17:29:44.527 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-02T17:29:44.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T17:29:44.544 DEBUG:teuthology.orchestra.run.smithi069:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-02T17:29:44.546 DEBUG:teuthology.orchestra.run.smithi077:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-02T17:29:44.570 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-02T17:29:44.600 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-02T17:29:44.618 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-02T17:29:44.638 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-02T17:29:44.650 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-02T17:29:44.664 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-02T17:29:44.680 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-02T17:29:44.681 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-02T17:29:44.688 INFO:teuthology.task.internal:Configuring sudo... 2024-08-02T17:29:44.688 DEBUG:teuthology.orchestra.run.smithi069:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-02T17:29:44.690 DEBUG:teuthology.orchestra.run.smithi077:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-02T17:29:44.693 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-02T17:29:44.732 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-02T17:29:44.745 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-02T17:29:44.745 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-02T17:29:44.755 DEBUG:teuthology.orchestra.run.smithi077:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-02T17:29:44.759 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-02T17:29:44.788 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-02T17:29:44.850 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-02T17:29:44.898 DEBUG:teuthology.orchestra.run.smithi069:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-02T17:29:44.977 DEBUG:teuthology.orchestra.run.smithi069:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-02T17:29:45.020 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:29:45.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-02T17:29:45.111 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-02T17:29:45.153 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-02T17:29:45.212 DEBUG:teuthology.orchestra.run.smithi077:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-02T17:29:45.291 DEBUG:teuthology.orchestra.run.smithi077:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-02T17:29:45.337 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:29:45.337 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-02T17:29:45.402 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-02T17:29:45.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-02T17:29:45.503 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-02T17:29:45.584 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-02T17:29:45.631 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:29:45.631 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-02T17:29:45.699 DEBUG:teuthology.orchestra.run.smithi069:> sudo service rsyslog restart 2024-08-02T17:29:45.701 DEBUG:teuthology.orchestra.run.smithi077:> sudo service rsyslog restart 2024-08-02T17:29:45.703 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-08-02T17:29:45.736 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-02T17:29:45.743 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-02T17:29:45.782 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-02T17:29:46.230 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-02T17:29:46.238 INFO:teuthology.task.internal:Starting timer... 2024-08-02T17:29:46.238 INFO:teuthology.run_tasks:Running task pcp... 2024-08-02T17:29:46.258 INFO:teuthology.run_tasks:Running task selinux... 2024-08-02T17:29:46.270 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-08-02T17:29:46.270 DEBUG:teuthology.orchestra.run.smithi069:> sudo service auditd rotate 2024-08-02T17:29:46.342 INFO:teuthology.orchestra.run.smithi069.stdout:Rotating logs: 2024-08-02T17:29:46.345 DEBUG:teuthology.orchestra.run.smithi077:> sudo service auditd rotate 2024-08-02T17:29:46.411 INFO:teuthology.orchestra.run.smithi077.stdout:Rotating logs: 2024-08-02T17:29:46.413 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-08-02T17:29:46.471 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-08-02T17:29:46.473 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-02T17:29:46.473 DEBUG:teuthology.orchestra.run.smithi069:> /usr/sbin/getenforce 2024-08-02T17:29:46.492 INFO:teuthology.orchestra.run.smithi069.stdout:Permissive 2024-08-02T17:29:46.493 DEBUG:teuthology.orchestra.run.smithi077:> /usr/sbin/getenforce 2024-08-02T17:29:46.518 INFO:teuthology.orchestra.run.smithi077.stdout:Permissive 2024-08-02T17:29:46.519 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-08-02T17:29:46.546 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-08-02T17:29:46.546 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi069.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi077.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi083.front.sepia.ceph.com': 'permissive'} 2024-08-02T17:29:46.547 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-02T17:29:46.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T17:29:46.572 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-02T17:29:46.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T17:29:46.599 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-02T17:29:46.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T17:29:46.627 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-02T17:29:46.627 DEBUG:teuthology.orchestra.run.smithi069:> sudo /usr/sbin/setenforce permissive 2024-08-02T17:29:46.675 DEBUG:teuthology.orchestra.run.smithi077:> sudo /usr/sbin/setenforce permissive 2024-08-02T17:29:46.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-08-02T17:29:46.728 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-02T17:29:46.738 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-08-02T17:29:46.741 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-08-02T17:29:46.846 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-02T17:29:46.858 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-02T17:29:46.859 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi069.front.sepia.ceph.com,smithi077.front.sepia.ceph.com,smithi083.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-02T17:34:28.311 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi069.front.sepia.ceph.com'), Remote(name='ubuntu@smithi077.front.sepia.ceph.com'), Remote(name='ubuntu@smithi083.front.sepia.ceph.com')] 2024-08-02T17:34:28.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-08-02T17:34:28.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi069.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-02T17:34:28.389 DEBUG:teuthology.orchestra.run.smithi069:> true 2024-08-02T17:34:28.474 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi069.front.sepia.ceph.com' 2024-08-02T17:34:28.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-08-02T17:34:28.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi077.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-02T17:34:28.549 DEBUG:teuthology.orchestra.run.smithi077:> true 2024-08-02T17:34:28.633 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi077.front.sepia.ceph.com' 2024-08-02T17:34:28.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-08-02T17:34:28.634 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-02T17:34:28.709 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-08-02T17:34:28.797 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-08-02T17:34:28.799 INFO:teuthology.run_tasks:Running task clock... 2024-08-02T17:34:28.809 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-02T17:34:28.809 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-02T17:34:28.810 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-02T17:34:28.813 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-02T17:34:28.813 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-02T17:34:28.815 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-02T17:34:28.815 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-02T17:34:28.840 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-02T17:34:28.841 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-02T17:34:28.854 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-02T17:34:28.854 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-02T17:34:28.866 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-02T17:34:28.878 INFO:teuthology.orchestra.run.smithi077.stderr:sudo: ntpd: command not found 2024-08-02T17:34:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:sudo: ntpd: command not found 2024-08-02T17:34:28.880 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-02T17:34:28.888 INFO:teuthology.orchestra.run.smithi077.stdout:506 Cannot talk to daemon 2024-08-02T17:34:28.890 INFO:teuthology.orchestra.run.smithi069.stdout:506 Cannot talk to daemon 2024-08-02T17:34:28.901 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-02T17:34:28.902 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-08-02T17:34:28.904 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-02T17:34:28.912 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-08-02T17:34:28.914 INFO:teuthology.orchestra.run.smithi077.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-02T17:34:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-02T17:34:28.925 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-02T17:34:28.938 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-02T17:34:28.966 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-08-02T17:34:28.969 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-02T17:34:28.970 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-08-02T17:34:28.970 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.970 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.970 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.970 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.978 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-08-02T17:34:28.981 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-02T17:34:28.981 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-08-02T17:34:28.981 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.981 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.982 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.982 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.990 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-08-02T17:34:28.992 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-02T17:34:28.993 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-08-02T17:34:28.993 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.993 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.993 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.993 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T17:34:28.994 INFO:teuthology.run_tasks:Running task install... 2024-08-02T17:34:29.002 DEBUG:teuthology.task.install:project ceph 2024-08-02T17:34:29.002 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea'}} 2024-08-02T17:34:29.002 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea'} 2024-08-02T17:34:29.002 INFO:teuthology.task.install:Using flavor: default 2024-08-02T17:34:29.008 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-02T17:34:29.008 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-02T17:34:29.008 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea', 'tag': None, 'wait_for_package': False} 2024-08-02T17:34:29.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:34:29.011 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea', 'tag': None, 'wait_for_package': False} 2024-08-02T17:34:29.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:34:29.012 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea', 'tag': None, 'wait_for_package': False} 2024-08-02T17:34:29.012 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:34:29.156 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/ 2024-08-02T17:34:29.156 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1260.g26c3fb8e 2024-08-02T17:34:29.164 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/ 2024-08-02T17:34:29.164 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1260.g26c3fb8e 2024-08-02T17:34:29.177 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/ 2024-08-02T17:34:29.177 INFO:teuthology.task.install.rpm:Package version is 19.1.0-1260.g26c3fb8e 2024-08-02T17:34:29.273 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-02T17:34:29.273 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:34:29.273 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-02T17:34:29.277 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-02T17:34:29.277 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:34:29.278 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-02T17:34:29.290 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-02T17:34:29.290 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:34:29.290 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-02T17:34:29.299 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-02T17:34:29.300 DEBUG:teuthology.orchestra.run.smithi069:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-02T17:34:29.302 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-02T17:34:29.306 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-02T17:34:29.315 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-02T17:34:29.315 DEBUG:teuthology.orchestra.run.smithi077:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-02T17:34:29.364 DEBUG:teuthology.orchestra.run.smithi069:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-02T17:34:29.368 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-02T17:34:29.379 DEBUG:teuthology.orchestra.run.smithi077:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-02T17:34:29.438 DEBUG:teuthology.orchestra.run.smithi069:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-02T17:34:29.441 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-02T17:34:29.452 DEBUG:teuthology.orchestra.run.smithi077:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-02T17:34:29.502 INFO:teuthology.orchestra.run.smithi069.stdout:check_obsoletes = 1 2024-08-02T17:34:29.503 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum clean all 2024-08-02T17:34:29.504 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-08-02T17:34:29.505 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-08-02T17:34:29.515 INFO:teuthology.orchestra.run.smithi077.stdout:check_obsoletes = 1 2024-08-02T17:34:29.516 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum clean all 2024-08-02T17:34:29.727 INFO:teuthology.orchestra.run.smithi083.stdout:45 files removed 2024-08-02T17:34:29.732 INFO:teuthology.orchestra.run.smithi069.stdout:45 files removed 2024-08-02T17:34:29.739 INFO:teuthology.orchestra.run.smithi077.stdout:45 files removed 2024-08-02T17:34:29.754 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-02T17:34:29.758 DEBUG:teuthology.orchestra.run.smithi069:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-02T17:34:29.766 DEBUG:teuthology.orchestra.run.smithi077:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-02T17:34:30.330 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 266 kB/s | 84 kB 00:00 2024-08-02T17:34:30.352 INFO:teuthology.orchestra.run.smithi069.stdout:ceph packages for x86_64 265 kB/s | 84 kB 00:00 2024-08-02T17:34:30.365 INFO:teuthology.orchestra.run.smithi077.stdout:ceph packages for x86_64 248 kB/s | 84 kB 00:00 2024-08-02T17:34:30.653 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 41 kB/s | 12 kB 00:00 2024-08-02T17:34:30.711 INFO:teuthology.orchestra.run.smithi069.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-08-02T17:34:30.715 INFO:teuthology.orchestra.run.smithi077.stdout:ceph noarch packages 38 kB/s | 12 kB 00:00 2024-08-02T17:34:30.979 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-08-02T17:34:31.024 INFO:teuthology.orchestra.run.smithi077.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-08-02T17:34:31.028 INFO:teuthology.orchestra.run.smithi069.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-08-02T17:34:31.729 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-08-02T17:34:31.746 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-08-02T17:34:31.924 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - BaseOS 9.5 MB/s | 8.2 MB 00:00 2024-08-02T17:34:33.984 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-08-02T17:34:34.650 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-08-02T17:34:37.196 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 20 MB 00:04 2024-08-02T17:34:38.435 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.6 MB 00:00 2024-08-02T17:34:40.236 INFO:teuthology.orchestra.run.smithi077.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 18 kB 00:00 2024-08-02T17:34:40.640 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 2.9 MB/s | 6.6 MB 00:02 2024-08-02T17:34:41.190 INFO:teuthology.orchestra.run.smithi077.stdout:Extra Packages for Enterprise Linux 27 MB/s | 22 MB 00:00 2024-08-02T17:34:41.422 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.6 MB 00:00 2024-08-02T17:34:42.599 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 24 kB/s | 18 kB 00:00 2024-08-02T17:34:43.006 INFO:teuthology.orchestra.run.smithi069.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 18 kB 00:00 2024-08-02T17:34:43.528 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-08-02T17:34:43.943 INFO:teuthology.orchestra.run.smithi069.stdout:Extra Packages for Enterprise Linux 27 MB/s | 22 MB 00:00 2024-08-02T17:34:46.571 INFO:teuthology.orchestra.run.smithi077.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-08-02T17:34:48.826 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-08-02T17:34:48.890 INFO:teuthology.orchestra.run.smithi077.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-02T17:34:48.890 INFO:teuthology.orchestra.run.smithi077.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-02T17:34:48.902 INFO:teuthology.orchestra.run.smithi077.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-02T17:34:48.902 INFO:teuthology.orchestra.run.smithi077.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-02T17:34:48.903 INFO:teuthology.orchestra.run.smithi077.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-02T17:34:48.904 INFO:teuthology.orchestra.run.smithi077.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-02T17:34:48.906 INFO:teuthology.orchestra.run.smithi077.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-02T17:34:48.908 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-02T17:34:48.909 INFO:teuthology.orchestra.run.smithi077.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-02T17:34:48.973 INFO:teuthology.orchestra.run.smithi077.stdout:Dependencies resolved. 2024-08-02T17:34:48.987 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-08-02T17:34:48.987 INFO:teuthology.orchestra.run.smithi077.stdout: Package Arch Version Repository Size 2024-08-02T17:34:48.987 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-08-02T17:34:48.987 INFO:teuthology.orchestra.run.smithi077.stdout:Installing: 2024-08-02T17:34:48.987 INFO:teuthology.orchestra.run.smithi077.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-02T17:34:48.987 INFO:teuthology.orchestra.run.smithi077.stdout: ceph x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 6.6 k 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 5.4 M 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 897 k 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 145 k 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 1.5 M 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 146 k 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 3.6 M 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 7.4 M 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 49 k 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 11 M 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 49 M 2024-08-02T17:34:48.988 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 274 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 765 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 224 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 27 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 18 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 32 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 751 k 2024-08-02T17:34:48.989 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 127 k 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 188 k 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 372 k 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 349 k 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 116 k 2024-08-02T17:34:48.990 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 85 k 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.1 M 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 170 k 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrading: 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: librados2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.4 M 2024-08-02T17:34:48.991 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.2 M 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout:Installing dependencies: 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 21 M 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 32 k 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 2.4 M 2024-08-02T17:34:48.992 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 249 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 4.7 M 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 17 M 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 16 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 25 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 164 k 2024-08-02T17:34:48.993 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 507 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 5.4 M 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-02T17:34:48.994 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 45 k 2024-08-02T17:34:48.995 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 136 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-02T17:34:48.996 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-02T17:34:48.997 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-02T17:34:48.998 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-02T17:34:48.999 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-02T17:34:49.000 INFO:teuthology.orchestra.run.smithi077.stdout:Installing weak dependencies: 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction Summary 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout:======================================================================================= 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout:Install 148 Packages 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout:Upgrade 12 Packages 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout:Total download size: 228 M 2024-08-02T17:34:49.001 INFO:teuthology.orchestra.run.smithi077.stdout:Downloading Packages: 2024-08-02T17:34:49.258 INFO:teuthology.orchestra.run.smithi069.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-08-02T17:34:49.522 INFO:teuthology.orchestra.run.smithi077.stdout:(1/160): ceph-19.1.0-1260.g26c3fb8e.el9.x86_64. 37 kB/s | 6.6 kB 00:00 2024-08-02T17:34:49.748 INFO:teuthology.orchestra.run.smithi077.stdout:(2/160): ceph-fuse-19.1.0-1260.g26c3fb8e.el9.x8 3.9 MB/s | 897 kB 00:00 2024-08-02T17:34:49.831 INFO:teuthology.orchestra.run.smithi077.stdout:(3/160): ceph-base-19.1.0-1260.g26c3fb8e.el9.x8 11 MB/s | 5.4 MB 00:00 2024-08-02T17:34:49.856 INFO:teuthology.orchestra.run.smithi077.stdout:(4/160): ceph-immutable-object-cache-19.1.0-126 1.3 MB/s | 145 kB 00:00 2024-08-02T17:34:49.973 INFO:teuthology.orchestra.run.smithi077.stdout:(5/160): ceph-mds-19.1.0-1260.g26c3fb8e.el9.x86 17 MB/s | 2.4 MB 00:00 2024-08-02T17:34:50.024 INFO:teuthology.orchestra.run.smithi077.stdout:(6/160): ceph-mgr-19.1.0-1260.g26c3fb8e.el9.x86 9.1 MB/s | 1.5 MB 00:00 2024-08-02T17:34:50.212 INFO:teuthology.orchestra.run.smithi077.stdout:(7/160): ceph-mon-19.1.0-1260.g26c3fb8e.el9.x86 20 MB/s | 4.7 MB 00:00 2024-08-02T17:34:50.774 INFO:teuthology.orchestra.run.smithi077.stdout:(8/160): ceph-radosgw-19.1.0-1260.g26c3fb8e.el9 19 MB/s | 11 MB 00:00 2024-08-02T17:34:50.983 INFO:teuthology.orchestra.run.smithi077.stdout:(9/160): ceph-common-19.1.0-1260.g26c3fb8e.el9. 13 MB/s | 21 MB 00:01 2024-08-02T17:34:51.008 INFO:teuthology.orchestra.run.smithi077.stdout:(10/160): ceph-selinux-19.1.0-1260.g26c3fb8e.el 108 kB/s | 25 kB 00:00 2024-08-02T17:34:51.058 INFO:teuthology.orchestra.run.smithi077.stdout:(11/160): cephfs-mirror-19.1.0-1260.g26c3fb8e.e 4.4 MB/s | 224 kB 00:00 2024-08-02T17:34:51.117 INFO:teuthology.orchestra.run.smithi077.stdout:(12/160): cephfs-shell-19.1.0-1260.g26c3fb8e.el 464 kB/s | 27 kB 00:00 2024-08-02T17:34:51.118 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-02T17:34:51.119 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-02T17:34:51.130 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-02T17:34:51.131 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-02T17:34:51.131 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-02T17:34:51.132 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-02T17:34:51.134 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-02T17:34:51.136 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-02T17:34:51.137 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-02T17:34:51.200 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-08-02T17:34:51.210 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 6.6 k 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 5.4 M 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 897 k 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 145 k 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 1.5 M 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 146 k 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 3.6 M 2024-08-02T17:34:51.211 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 7.4 M 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 49 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 11 M 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 49 M 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 274 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 765 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 224 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 27 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 18 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-02T17:34:51.212 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 32 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 751 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 127 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 188 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 372 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 349 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 116 k 2024-08-02T17:34:51.213 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 85 k 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.1 M 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 170 k 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.4 M 2024-08-02T17:34:51.214 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.2 M 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 21 M 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 32 k 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 2.4 M 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 249 k 2024-08-02T17:34:51.215 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 4.7 M 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 17 M 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 16 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 25 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 164 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-02T17:34:51.216 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 507 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 5.4 M 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-02T17:34:51.217 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-02T17:34:51.218 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 45 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 136 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-02T17:34:51.219 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-02T17:34:51.220 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-02T17:34:51.221 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-02T17:34:51.222 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-02T17:34:51.223 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-08-02T17:34:51.224 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-08-02T17:34:51.225 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 12 Packages 2024-08-02T17:34:51.225 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:34:51.225 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 228 M 2024-08-02T17:34:51.225 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-08-02T17:34:51.309 INFO:teuthology.orchestra.run.smithi077.stdout:(13/160): ceph-osd-19.1.0-1260.g26c3fb8e.el9.x8 13 MB/s | 17 MB 00:01 2024-08-02T17:34:51.334 INFO:teuthology.orchestra.run.smithi077.stdout:(14/160): libcephfs-devel-19.1.0-1260.g26c3fb8e 146 kB/s | 32 kB 00:00 2024-08-02T17:34:51.376 INFO:teuthology.orchestra.run.smithi077.stdout:(15/160): libcephfs2-19.1.0-1260.g26c3fb8e.el9. 11 MB/s | 751 kB 00:00 2024-08-02T17:34:51.402 INFO:teuthology.orchestra.run.smithi077.stdout:(16/160): libcephsqlite-19.1.0-1260.g26c3fb8e.e 2.4 MB/s | 164 kB 00:00 2024-08-02T17:34:51.427 INFO:teuthology.orchestra.run.smithi077.stdout:(17/160): librados-devel-19.1.0-1260.g26c3fb8e. 2.5 MB/s | 127 kB 00:00 2024-08-02T17:34:51.461 INFO:teuthology.orchestra.run.smithi077.stdout:(18/160): libradosstriper1-19.1.0-1260.g26c3fb8 8.4 MB/s | 507 kB 00:00 2024-08-02T17:34:51.520 INFO:teuthology.orchestra.run.smithi077.stdout:(19/160): python3-ceph-argparse-19.1.0-1260.g26 775 kB/s | 45 kB 00:00 2024-08-02T17:34:51.542 INFO:teuthology.orchestra.run.smithi069.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-02T17:34:51.543 INFO:teuthology.orchestra.run.smithi069.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-02T17:34:51.554 INFO:teuthology.orchestra.run.smithi069.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-08-02T17:34:51.555 INFO:teuthology.orchestra.run.smithi069.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-02T17:34:51.556 INFO:teuthology.orchestra.run.smithi069.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-02T17:34:51.557 INFO:teuthology.orchestra.run.smithi069.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-02T17:34:51.558 INFO:teuthology.orchestra.run.smithi069.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-02T17:34:51.561 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-02T17:34:51.562 INFO:teuthology.orchestra.run.smithi069.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-08-02T17:34:51.586 INFO:teuthology.orchestra.run.smithi077.stdout:(20/160): python3-ceph-common-19.1.0-1260.g26c3 2.0 MB/s | 136 kB 00:00 2024-08-02T17:34:51.624 INFO:teuthology.orchestra.run.smithi069.stdout:Dependencies resolved. 2024-08-02T17:34:51.654 INFO:teuthology.orchestra.run.smithi077.stdout:(21/160): python3-cephfs-19.1.0-1260.g26c3fb8e. 2.8 MB/s | 188 kB 00:00 2024-08-02T17:34:51.654 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-08-02T17:34:51.654 INFO:teuthology.orchestra.run.smithi069.stdout: Package Arch Version Repository Size 2024-08-02T17:34:51.654 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-08-02T17:34:51.655 INFO:teuthology.orchestra.run.smithi069.stdout:Installing: 2024-08-02T17:34:51.655 INFO:teuthology.orchestra.run.smithi069.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-02T17:34:51.655 INFO:teuthology.orchestra.run.smithi069.stdout: ceph x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 6.6 k 2024-08-02T17:34:51.655 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 5.4 M 2024-08-02T17:34:51.655 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 897 k 2024-08-02T17:34:51.655 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 145 k 2024-08-02T17:34:51.655 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 1.5 M 2024-08-02T17:34:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 146 k 2024-08-02T17:34:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 3.6 M 2024-08-02T17:34:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 7.4 M 2024-08-02T17:34:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 49 k 2024-08-02T17:34:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 11 M 2024-08-02T17:34:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 49 M 2024-08-02T17:34:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 274 k 2024-08-02T17:34:51.656 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 765 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 224 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 27 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 18 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 32 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 751 k 2024-08-02T17:34:51.657 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 127 k 2024-08-02T17:34:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-02T17:34:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-02T17:34:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-02T17:34:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-08-02T17:34:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-08-02T17:34:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-08-02T17:34:51.658 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 188 k 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 372 k 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 349 k 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 116 k 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 85 k 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.1 M 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 170 k 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel x86_64 6.4.0-3.el9 appstream 96 k 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrading: 2024-08-02T17:34:51.659 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager x86_64 1:1.48.2-2.el9 baseos 2.3 M 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm x86_64 1:1.48.2-2.el9 baseos 1.8 M 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team x86_64 1:1.48.2-2.el9 baseos 41 k 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui x86_64 1:1.48.2-2.el9 baseos 247 k 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: librados2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.4 M 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 3.2 M 2024-08-02T17:34:51.660 INFO:teuthology.orchestra.run.smithi069.stdout: openssl x86_64 1:3.2.2-2.el9 baseos 1.4 M 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs x86_64 1:3.2.2-2.el9 baseos 2.4 M 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs x86_64 6.4.0-3.el9 baseos 1.1 M 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout:Installing dependencies: 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 21 M 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 32 k 2024-08-02T17:34:51.661 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 2.4 M 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 249 k 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 4.7 M 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 17 M 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts noarch 2:19.1.0-1260.g26c3fb8e.el9 ceph-noarch 16 k 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 25 k 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-02T17:34:51.662 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 164 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 507 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2 x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 5.4 M 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-02T17:34:51.663 INFO:teuthology.orchestra.run.smithi069.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-02T17:34:51.664 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 45 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common x86_64 2:19.1.0-1260.g26c3fb8e.el9 ceph 136 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth noarch 1:2.32.0-1.el9 epel 216 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-02T17:34:51.665 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-02T17:34:51.666 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-02T17:34:51.667 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-02T17:34:51.668 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout:Installing weak dependencies: 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-02T17:34:51.669 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:34:51.670 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction Summary 2024-08-02T17:34:51.670 INFO:teuthology.orchestra.run.smithi069.stdout:======================================================================================= 2024-08-02T17:34:51.670 INFO:teuthology.orchestra.run.smithi069.stdout:Install 148 Packages 2024-08-02T17:34:51.670 INFO:teuthology.orchestra.run.smithi069.stdout:Upgrade 12 Packages 2024-08-02T17:34:51.670 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:34:51.670 INFO:teuthology.orchestra.run.smithi069.stdout:Total download size: 228 M 2024-08-02T17:34:51.670 INFO:teuthology.orchestra.run.smithi069.stdout:Downloading Packages: 2024-08-02T17:34:51.737 INFO:teuthology.orchestra.run.smithi077.stdout:(22/160): librgw2-19.1.0-1260.g26c3fb8e.el9.x86 17 MB/s | 5.4 MB 00:00 2024-08-02T17:34:51.762 INFO:teuthology.orchestra.run.smithi077.stdout:(23/160): python3-rados-19.1.0-1260.g26c3fb8e.e 3.4 MB/s | 372 kB 00:00 2024-08-02T17:34:51.788 INFO:teuthology.orchestra.run.smithi077.stdout:(24/160): python3-rbd-19.1.0-1260.g26c3fb8e.el9 6.8 MB/s | 349 kB 00:00 2024-08-02T17:34:51.814 INFO:teuthology.orchestra.run.smithi077.stdout:(25/160): python3-rgw-19.1.0-1260.g26c3fb8e.el9 2.3 MB/s | 116 kB 00:00 2024-08-02T17:34:51.838 INFO:teuthology.orchestra.run.smithi077.stdout:(26/160): rbd-fuse-19.1.0-1260.g26c3fb8e.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-02T17:34:51.889 INFO:teuthology.orchestra.run.smithi077.stdout:(27/160): rbd-nbd-19.1.0-1260.g26c3fb8e.el9.x86 3.3 MB/s | 170 kB 00:00 2024-08-02T17:34:51.947 INFO:teuthology.orchestra.run.smithi077.stdout:(28/160): ceph-grafana-dashboards-19.1.0-1260.g 547 kB/s | 32 kB 00:00 2024-08-02T17:34:52.014 INFO:teuthology.orchestra.run.smithi077.stdout:(29/160): rbd-mirror-19.1.0-1260.g26c3fb8e.el9. 15 MB/s | 3.1 MB 00:00 2024-08-02T17:34:52.040 INFO:teuthology.orchestra.run.smithi077.stdout:(30/160): ceph-mgr-cephadm-19.1.0-1260.g26c3fb8 1.6 MB/s | 146 kB 00:00 2024-08-02T17:34:52.186 INFO:teuthology.orchestra.run.smithi083.stdout:(1/160): ceph-19.1.0-1260.g26c3fb8e.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-08-02T17:34:52.265 INFO:teuthology.orchestra.run.smithi077.stdout:(31/160): ceph-mgr-dashboard-19.1.0-1260.g26c3f 14 MB/s | 3.6 MB 00:00 2024-08-02T17:34:52.267 INFO:teuthology.orchestra.run.smithi069.stdout:(1/160): ceph-19.1.0-1260.g26c3fb8e.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-08-02T17:34:52.340 INFO:teuthology.orchestra.run.smithi077.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1260.g26 3.3 MB/s | 249 kB 00:00 2024-08-02T17:34:52.416 INFO:teuthology.orchestra.run.smithi077.stdout:(33/160): ceph-mgr-rook-19.1.0-1260.g26c3fb8e.e 659 kB/s | 49 kB 00:00 2024-08-02T17:34:52.558 INFO:teuthology.orchestra.run.smithi077.stdout:(34/160): ceph-mgr-diskprediction-local-19.1.0- 14 MB/s | 7.4 MB 00:00 2024-08-02T17:34:52.583 INFO:teuthology.orchestra.run.smithi077.stdout:(35/160): ceph-prometheus-alerts-19.1.0-1260.g2 98 kB/s | 16 kB 00:00 2024-08-02T17:34:52.586 INFO:teuthology.orchestra.run.smithi083.stdout:(2/160): ceph-fuse-19.1.0-1260.g26c3fb8e.el9.x8 2.2 MB/s | 897 kB 00:00 2024-08-02T17:34:52.625 INFO:teuthology.orchestra.run.smithi077.stdout:(36/160): ceph-volume-19.1.0-1260.g26c3fb8e.el9 4.0 MB/s | 274 kB 00:00 2024-08-02T17:34:52.669 INFO:teuthology.orchestra.run.smithi083.stdout:(3/160): ceph-immutable-object-cache-19.1.0-126 1.7 MB/s | 145 kB 00:00 2024-08-02T17:34:52.675 INFO:teuthology.orchestra.run.smithi077.stdout:(37/160): cephadm-19.1.0-1260.g26c3fb8e.el9.noa 8.1 MB/s | 765 kB 00:00 2024-08-02T17:34:52.709 INFO:teuthology.orchestra.run.smithi077.stdout:(38/160): cephfs-top-19.1.0-1260.g26c3fb8e.el9. 218 kB/s | 18 kB 00:00 2024-08-02T17:34:52.778 INFO:teuthology.orchestra.run.smithi083.stdout:(4/160): ceph-base-19.1.0-1260.g26c3fb8e.el9.x8 6.6 MB/s | 5.4 MB 00:00 2024-08-02T17:34:52.784 INFO:teuthology.orchestra.run.smithi069.stdout:(2/160): ceph-fuse-19.1.0-1260.g26c3fb8e.el9.x8 1.7 MB/s | 897 kB 00:00 2024-08-02T17:34:52.909 INFO:teuthology.orchestra.run.smithi069.stdout:(3/160): ceph-immutable-object-cache-19.1.0-126 1.1 MB/s | 145 kB 00:00 2024-08-02T17:34:52.937 INFO:teuthology.orchestra.run.smithi083.stdout:(5/160): ceph-mgr-19.1.0-1260.g26c3fb8e.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-08-02T17:34:53.129 INFO:teuthology.orchestra.run.smithi083.stdout:(6/160): ceph-mds-19.1.0-1260.g26c3fb8e.el9.x86 5.2 MB/s | 2.4 MB 00:00 2024-08-02T17:34:53.204 INFO:teuthology.orchestra.run.smithi083.stdout:(7/160): ceph-mon-19.1.0-1260.g26c3fb8e.el9.x86 18 MB/s | 4.7 MB 00:00 2024-08-02T17:34:53.359 INFO:teuthology.orchestra.run.smithi077.stdout:(39/160): ceph-test-19.1.0-1260.g26c3fb8e.el9.x 20 MB/s | 49 MB 00:02 2024-08-02T17:34:53.368 INFO:teuthology.orchestra.run.smithi069.stdout:(4/160): ceph-mds-19.1.0-1260.g26c3fb8e.el9.x86 5.2 MB/s | 2.4 MB 00:00 2024-08-02T17:34:53.393 INFO:teuthology.orchestra.run.smithi077.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 111 kB/s | 80 kB 00:00 2024-08-02T17:34:53.485 INFO:teuthology.orchestra.run.smithi069.stdout:(5/160): ceph-base-19.1.0-1260.g26c3fb8e.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-08-02T17:34:53.710 INFO:teuthology.orchestra.run.smithi069.stdout:(6/160): ceph-mgr-19.1.0-1260.g26c3fb8e.el9.x86 4.4 MB/s | 1.5 MB 00:00 2024-08-02T17:34:53.721 INFO:teuthology.orchestra.run.smithi083.stdout:(8/160): ceph-radosgw-19.1.0-1260.g26c3fb8e.el9 20 MB/s | 11 MB 00:00 2024-08-02T17:34:53.779 INFO:teuthology.orchestra.run.smithi083.stdout:(9/160): ceph-selinux-19.1.0-1260.g26c3fb8e.el9 433 kB/s | 25 kB 00:00 2024-08-02T17:34:53.818 INFO:teuthology.orchestra.run.smithi077.stdout:(41/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 596 kB/s | 253 kB 00:00 2024-08-02T17:34:53.860 INFO:teuthology.orchestra.run.smithi077.stdout:(42/160): libquadmath-11.4.1-3.el9.x86_64.rpm 390 kB/s | 195 kB 00:00 2024-08-02T17:34:53.885 INFO:teuthology.orchestra.run.smithi077.stdout:(43/160): libgfortran-11.4.1-3.el9.x86_64.rpm 683 kB/s | 803 kB 00:01 2024-08-02T17:34:53.986 INFO:teuthology.orchestra.run.smithi077.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 851 kB/s | 106 kB 00:00 2024-08-02T17:34:54.011 INFO:teuthology.orchestra.run.smithi077.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-08-02T17:34:54.095 INFO:teuthology.orchestra.run.smithi077.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-08-02T17:34:54.120 INFO:teuthology.orchestra.run.smithi077.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-08-02T17:34:54.194 INFO:teuthology.orchestra.run.smithi069.stdout:(7/160): ceph-mon-19.1.0-1260.g26c3fb8e.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-08-02T17:34:54.262 INFO:teuthology.orchestra.run.smithi077.stdout:(48/160): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-08-02T17:34:54.354 INFO:teuthology.orchestra.run.smithi077.stdout:(49/160): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-08-02T17:34:54.404 INFO:teuthology.orchestra.run.smithi077.stdout:(50/160): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-08-02T17:34:54.430 INFO:teuthology.orchestra.run.smithi077.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 288 kB/s | 22 kB 00:00 2024-08-02T17:34:54.455 INFO:teuthology.orchestra.run.smithi077.stdout:(52/160): boost-program-options-1.75.0-8.el9.x8 555 kB/s | 107 kB 00:00 2024-08-02T17:34:54.480 INFO:teuthology.orchestra.run.smithi077.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-08-02T17:34:54.505 INFO:teuthology.orchestra.run.smithi077.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-08-02T17:34:54.531 INFO:teuthology.orchestra.run.smithi077.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-08-02T17:34:54.564 INFO:teuthology.orchestra.run.smithi077.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-08-02T17:34:54.589 INFO:teuthology.orchestra.run.smithi077.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-08-02T17:34:54.623 INFO:teuthology.orchestra.run.smithi077.stdout:(58/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-08-02T17:34:54.658 INFO:teuthology.orchestra.run.smithi077.stdout:(59/160): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-08-02T17:34:54.699 INFO:teuthology.orchestra.run.smithi077.stdout:(60/160): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-08-02T17:34:54.716 INFO:teuthology.orchestra.run.smithi077.stdout:(61/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-08-02T17:34:54.766 INFO:teuthology.orchestra.run.smithi077.stdout:(62/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.6 MB/s | 3.0 MB 00:00 2024-08-02T17:34:54.783 INFO:teuthology.orchestra.run.smithi077.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-08-02T17:34:54.808 INFO:teuthology.orchestra.run.smithi077.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-08-02T17:34:54.834 INFO:teuthology.orchestra.run.smithi077.stdout:(65/160): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-08-02T17:34:54.867 INFO:teuthology.orchestra.run.smithi077.stdout:(66/160): openblas-0.3.21-2.el9.x86_64.rpm 419 kB/s | 35 kB 00:00 2024-08-02T17:34:55.001 INFO:teuthology.orchestra.run.smithi077.stdout:(67/160): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-08-02T17:34:55.085 INFO:teuthology.orchestra.run.smithi077.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-08-02T17:34:55.152 INFO:teuthology.orchestra.run.smithi077.stdout:(69/160): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-08-02T17:34:55.177 INFO:teuthology.orchestra.run.smithi077.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-08-02T17:34:55.202 INFO:teuthology.orchestra.run.smithi077.stdout:(71/160): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-08-02T17:34:55.253 INFO:teuthology.orchestra.run.smithi077.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-08-02T17:34:55.278 INFO:teuthology.orchestra.run.smithi077.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-08-02T17:34:55.286 INFO:teuthology.orchestra.run.smithi069.stdout:(8/160): ceph-radosgw-19.1.0-1260.g26c3fb8e.el9 9.6 MB/s | 11 MB 00:01 2024-08-02T17:34:55.328 INFO:teuthology.orchestra.run.smithi077.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-02T17:34:55.369 INFO:teuthology.orchestra.run.smithi069.stdout:(9/160): ceph-selinux-19.1.0-1260.g26c3fb8e.el9 302 kB/s | 25 kB 00:00 2024-08-02T17:34:55.387 INFO:teuthology.orchestra.run.smithi077.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-08-02T17:34:55.462 INFO:teuthology.orchestra.run.smithi077.stdout:(76/160): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-08-02T17:34:55.487 INFO:teuthology.orchestra.run.smithi077.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-08-02T17:34:55.513 INFO:teuthology.orchestra.run.smithi077.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-08-02T17:34:55.588 INFO:teuthology.orchestra.run.smithi083.stdout:(10/160): ceph-osd-19.1.0-1260.g26c3fb8e.el9.x8 6.8 MB/s | 17 MB 00:02 2024-08-02T17:34:55.588 INFO:teuthology.orchestra.run.smithi077.stdout:(79/160): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-08-02T17:34:55.613 INFO:teuthology.orchestra.run.smithi077.stdout:(80/160): python3-jsonpointer-2.0-4.el9.noarch. 152 kB/s | 19 kB 00:00 2024-08-02T17:34:55.639 INFO:teuthology.orchestra.run.smithi077.stdout:(81/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-08-02T17:34:55.664 INFO:teuthology.orchestra.run.smithi077.stdout:(82/160): python3-markupsafe-1.1.1-12.el9.x86_6 459 kB/s | 35 kB 00:00 2024-08-02T17:34:55.680 INFO:teuthology.orchestra.run.smithi083.stdout:(11/160): cephfs-mirror-19.1.0-1260.g26c3fb8e.e 2.4 MB/s | 224 kB 00:00 2024-08-02T17:34:55.698 INFO:teuthology.orchestra.run.smithi077.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-08-02T17:34:55.723 INFO:teuthology.orchestra.run.smithi077.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-08-02T17:34:55.756 INFO:teuthology.orchestra.run.smithi077.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-08-02T17:34:55.780 INFO:teuthology.orchestra.run.smithi083.stdout:(12/160): cephfs-shell-19.1.0-1260.g26c3fb8e.el 270 kB/s | 27 kB 00:00 2024-08-02T17:34:55.782 INFO:teuthology.orchestra.run.smithi077.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-08-02T17:34:55.815 INFO:teuthology.orchestra.run.smithi077.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-08-02T17:34:55.841 INFO:teuthology.orchestra.run.smithi077.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-08-02T17:34:55.874 INFO:teuthology.orchestra.run.smithi077.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-08-02T17:34:55.880 INFO:teuthology.orchestra.run.smithi083.stdout:(13/160): libcephfs-devel-19.1.0-1260.g26c3fb8e 317 kB/s | 32 kB 00:00 2024-08-02T17:34:55.891 INFO:teuthology.orchestra.run.smithi077.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-02T17:34:55.959 INFO:teuthology.orchestra.run.smithi077.stdout:(91/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-08-02T17:34:55.983 INFO:teuthology.orchestra.run.smithi077.stdout:(92/160): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-08-02T17:34:56.017 INFO:teuthology.orchestra.run.smithi077.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-08-02T17:34:56.043 INFO:teuthology.orchestra.run.smithi077.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-08-02T17:34:56.056 INFO:teuthology.orchestra.run.smithi083.stdout:(14/160): libcephfs2-19.1.0-1260.g26c3fb8e.el9. 4.2 MB/s | 751 kB 00:00 2024-08-02T17:34:56.076 INFO:teuthology.orchestra.run.smithi077.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-08-02T17:34:56.102 INFO:teuthology.orchestra.run.smithi077.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 609 kB/s | 36 kB 00:00 2024-08-02T17:34:56.119 INFO:teuthology.orchestra.run.smithi069.stdout:(10/160): ceph-common-19.1.0-1260.g26c3fb8e.el9 5.2 MB/s | 21 MB 00:04 2024-08-02T17:34:56.131 INFO:teuthology.orchestra.run.smithi083.stdout:(15/160): libcephsqlite-19.1.0-1260.g26c3fb8e.e 2.1 MB/s | 164 kB 00:00 2024-08-02T17:34:56.166 INFO:teuthology.orchestra.run.smithi077.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 530 kB/s | 47 kB 00:00 2024-08-02T17:34:56.186 INFO:teuthology.orchestra.run.smithi069.stdout:(11/160): cephfs-mirror-19.1.0-1260.g26c3fb8e.e 3.3 MB/s | 224 kB 00:00 2024-08-02T17:34:56.202 INFO:teuthology.orchestra.run.smithi077.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-08-02T17:34:56.228 INFO:teuthology.orchestra.run.smithi077.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 941 kB/s | 58 kB 00:00 2024-08-02T17:34:56.232 INFO:teuthology.orchestra.run.smithi083.stdout:(16/160): librados-devel-19.1.0-1260.g26c3fb8e. 1.2 MB/s | 127 kB 00:00 2024-08-02T17:34:56.261 INFO:teuthology.orchestra.run.smithi077.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-08-02T17:34:56.286 INFO:teuthology.orchestra.run.smithi077.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-08-02T17:34:56.287 INFO:teuthology.orchestra.run.smithi069.stdout:(12/160): cephfs-shell-19.1.0-1260.g26c3fb8e.el 270 kB/s | 27 kB 00:00 2024-08-02T17:34:56.320 INFO:teuthology.orchestra.run.smithi077.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-08-02T17:34:56.348 INFO:teuthology.orchestra.run.smithi083.stdout:(17/160): libradosstriper1-19.1.0-1260.g26c3fb8 4.3 MB/s | 507 kB 00:00 2024-08-02T17:34:56.378 INFO:teuthology.orchestra.run.smithi069.stdout:(13/160): libcephfs-devel-19.1.0-1260.g26c3fb8e 346 kB/s | 32 kB 00:00 2024-08-02T17:34:56.546 INFO:teuthology.orchestra.run.smithi077.stdout:(103/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-08-02T17:34:56.554 INFO:teuthology.orchestra.run.smithi069.stdout:(14/160): libcephfs2-19.1.0-1260.g26c3fb8e.el9. 4.2 MB/s | 751 kB 00:00 2024-08-02T17:34:56.571 INFO:teuthology.orchestra.run.smithi077.stdout:(104/160): lua-devel-5.4.4-4.el9.x86_64.rpm 78 kB/s | 22 kB 00:00 2024-08-02T17:34:56.646 INFO:teuthology.orchestra.run.smithi069.stdout:(15/160): libcephsqlite-19.1.0-1260.g26c3fb8e.e 1.7 MB/s | 164 kB 00:00 2024-08-02T17:34:56.663 INFO:teuthology.orchestra.run.smithi077.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.6 MB/s | 548 kB 00:00 2024-08-02T17:34:56.688 INFO:teuthology.orchestra.run.smithi077.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-08-02T17:34:56.713 INFO:teuthology.orchestra.run.smithi077.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.1 MB/s | 309 kB 00:00 2024-08-02T17:34:56.730 INFO:teuthology.orchestra.run.smithi069.stdout:(16/160): librados-devel-19.1.0-1260.g26c3fb8e. 1.5 MB/s | 127 kB 00:00 2024-08-02T17:34:56.763 INFO:teuthology.orchestra.run.smithi077.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-08-02T17:34:56.782 INFO:teuthology.orchestra.run.smithi083.stdout:(18/160): ceph-test-19.1.0-1260.g26c3fb8e.el9.x 16 MB/s | 49 MB 00:03 2024-08-02T17:34:56.789 INFO:teuthology.orchestra.run.smithi077.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-02T17:34:56.814 INFO:teuthology.orchestra.run.smithi077.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-02T17:34:56.839 INFO:teuthology.orchestra.run.smithi077.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-08-02T17:34:56.890 INFO:teuthology.orchestra.run.smithi077.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-08-02T17:34:56.905 INFO:teuthology.orchestra.run.smithi069.stdout:(17/160): libradosstriper1-19.1.0-1260.g26c3fb8 2.8 MB/s | 507 kB 00:00 2024-08-02T17:34:56.949 INFO:teuthology.orchestra.run.smithi077.stdout:(113/160): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-08-02T17:34:56.974 INFO:teuthology.orchestra.run.smithi077.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-02T17:34:56.999 INFO:teuthology.orchestra.run.smithi077.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-02T17:34:57.025 INFO:teuthology.orchestra.run.smithi077.stdout:(116/160): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-08-02T17:34:57.050 INFO:teuthology.orchestra.run.smithi077.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-08-02T17:34:57.066 INFO:teuthology.orchestra.run.smithi083.stdout:(19/160): ceph-common-19.1.0-1260.g26c3fb8e.el9 4.2 MB/s | 21 MB 00:05 2024-08-02T17:34:57.091 INFO:teuthology.orchestra.run.smithi083.stdout:(20/160): python3-ceph-argparse-19.1.0-1260.g26 146 kB/s | 45 kB 00:00 2024-08-02T17:34:57.100 INFO:teuthology.orchestra.run.smithi077.stdout:(118/160): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-08-02T17:34:57.133 INFO:teuthology.orchestra.run.smithi083.stdout:(21/160): python3-ceph-common-19.1.0-1260.g26c3 2.0 MB/s | 136 kB 00:00 2024-08-02T17:34:57.133 INFO:teuthology.orchestra.run.smithi077.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-08-02T17:34:57.158 INFO:teuthology.orchestra.run.smithi083.stdout:(22/160): python3-cephfs-19.1.0-1260.g26c3fb8e. 2.7 MB/s | 188 kB 00:00 2024-08-02T17:34:57.159 INFO:teuthology.orchestra.run.smithi077.stdout:(120/160): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-08-02T17:34:57.172 INFO:teuthology.orchestra.run.smithi069.stdout:(18/160): ceph-osd-19.1.0-1260.g26c3fb8e.el9.x8 4.9 MB/s | 17 MB 00:03 2024-08-02T17:34:57.193 INFO:teuthology.orchestra.run.smithi077.stdout:(121/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.3 MB/s | 303 kB 00:00 2024-08-02T17:34:57.218 INFO:teuthology.orchestra.run.smithi077.stdout:(122/160): python3-google-auth-2.32.0-1.el9.noa 2.5 MB/s | 216 kB 00:00 2024-08-02T17:34:57.230 INFO:teuthology.orchestra.run.smithi069.stdout:(19/160): python3-ceph-argparse-19.1.0-1260.g26 776 kB/s | 45 kB 00:00 2024-08-02T17:34:57.233 INFO:teuthology.orchestra.run.smithi083.stdout:(23/160): librgw2-19.1.0-1260.g26c3fb8e.el9.x86 6.1 MB/s | 5.4 MB 00:00 2024-08-02T17:34:57.243 INFO:teuthology.orchestra.run.smithi077.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 421 kB/s | 11 kB 00:00 2024-08-02T17:34:57.259 INFO:teuthology.orchestra.run.smithi083.stdout:(24/160): python3-rados-19.1.0-1260.g26c3fb8e.e 2.9 MB/s | 372 kB 00:00 2024-08-02T17:34:57.269 INFO:teuthology.orchestra.run.smithi077.stdout:(124/160): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-08-02T17:34:57.284 INFO:teuthology.orchestra.run.smithi083.stdout:(25/160): python3-rbd-19.1.0-1260.g26c3fb8e.el9 2.7 MB/s | 349 kB 00:00 2024-08-02T17:34:57.294 INFO:teuthology.orchestra.run.smithi077.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-08-02T17:34:57.314 INFO:teuthology.orchestra.run.smithi069.stdout:(20/160): python3-ceph-common-19.1.0-1260.g26c3 1.6 MB/s | 136 kB 00:00 2024-08-02T17:34:57.318 INFO:teuthology.orchestra.run.smithi083.stdout:(26/160): python3-rgw-19.1.0-1260.g26c3fb8e.el9 1.4 MB/s | 116 kB 00:00 2024-08-02T17:34:57.319 INFO:teuthology.orchestra.run.smithi077.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 460 kB/s | 23 kB 00:00 2024-08-02T17:34:57.336 INFO:teuthology.orchestra.run.smithi077.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 463 kB/s | 19 kB 00:00 2024-08-02T17:34:57.343 INFO:teuthology.orchestra.run.smithi083.stdout:(27/160): rbd-fuse-19.1.0-1260.g26c3fb8e.el9.x8 1.0 MB/s | 85 kB 00:00 2024-08-02T17:34:57.353 INFO:teuthology.orchestra.run.smithi077.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 584 kB/s | 20 kB 00:00 2024-08-02T17:34:57.370 INFO:teuthology.orchestra.run.smithi077.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-08-02T17:34:57.395 INFO:teuthology.orchestra.run.smithi077.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-02T17:34:57.402 INFO:teuthology.orchestra.run.smithi083.stdout:(28/160): rbd-nbd-19.1.0-1260.g26c3fb8e.el9.x86 2.0 MB/s | 170 kB 00:00 2024-08-02T17:34:57.413 INFO:teuthology.orchestra.run.smithi077.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-08-02T17:34:57.427 INFO:teuthology.orchestra.run.smithi083.stdout:(29/160): ceph-grafana-dashboards-19.1.0-1260.g 381 kB/s | 32 kB 00:00 2024-08-02T17:34:57.438 INFO:teuthology.orchestra.run.smithi077.stdout:(132/160): python3-pecan-1.4.2-3.el9.noarch.rpm 11 MB/s | 272 kB 00:00 2024-08-02T17:34:57.469 INFO:teuthology.orchestra.run.smithi083.stdout:(30/160): ceph-mgr-cephadm-19.1.0-1260.g26c3fb8 2.1 MB/s | 146 kB 00:00 2024-08-02T17:34:57.471 INFO:teuthology.orchestra.run.smithi077.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-08-02T17:34:57.497 INFO:teuthology.orchestra.run.smithi077.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-08-02T17:34:57.522 INFO:teuthology.orchestra.run.smithi077.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-02T17:34:57.527 INFO:teuthology.orchestra.run.smithi083.stdout:(31/160): rbd-mirror-19.1.0-1260.g26c3fb8e.el9. 13 MB/s | 3.1 MB 00:00 2024-08-02T17:34:57.547 INFO:teuthology.orchestra.run.smithi077.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-08-02T17:34:57.573 INFO:teuthology.orchestra.run.smithi077.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-08-02T17:34:57.586 INFO:teuthology.orchestra.run.smithi083.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1260.g26 4.2 MB/s | 249 kB 00:00 2024-08-02T17:34:57.598 INFO:teuthology.orchestra.run.smithi077.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-02T17:34:57.624 INFO:teuthology.orchestra.run.smithi077.stdout:(139/160): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-02T17:34:57.637 INFO:teuthology.orchestra.run.smithi083.stdout:(33/160): ceph-mgr-rook-19.1.0-1260.g26c3fb8e.e 989 kB/s | 49 kB 00:00 2024-08-02T17:34:57.649 INFO:teuthology.orchestra.run.smithi077.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 709 kB/s | 36 kB 00:00 2024-08-02T17:34:57.674 INFO:teuthology.orchestra.run.smithi077.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-02T17:34:57.687 INFO:teuthology.orchestra.run.smithi083.stdout:(34/160): ceph-prometheus-alerts-19.1.0-1260.g2 329 kB/s | 16 kB 00:00 2024-08-02T17:34:57.699 INFO:teuthology.orchestra.run.smithi077.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-02T17:34:57.725 INFO:teuthology.orchestra.run.smithi077.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-02T17:34:57.746 INFO:teuthology.orchestra.run.smithi083.stdout:(35/160): ceph-volume-19.1.0-1260.g26c3fb8e.el9 4.6 MB/s | 274 kB 00:00 2024-08-02T17:34:57.750 INFO:teuthology.orchestra.run.smithi077.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-08-02T17:34:57.792 INFO:teuthology.orchestra.run.smithi077.stdout:(145/160): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-08-02T17:34:57.798 INFO:teuthology.orchestra.run.smithi069.stdout:(21/160): ceph-test-19.1.0-1260.g26c3fb8e.el9.x 20 MB/s | 49 MB 00:02 2024-08-02T17:34:57.813 INFO:teuthology.orchestra.run.smithi083.stdout:(36/160): cephadm-19.1.0-1260.g26c3fb8e.el9.noa 11 MB/s | 765 kB 00:00 2024-08-02T17:34:57.817 INFO:teuthology.orchestra.run.smithi077.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-08-02T17:34:57.823 INFO:teuthology.orchestra.run.smithi069.stdout:(22/160): python3-cephfs-19.1.0-1260.g26c3fb8e. 370 kB/s | 188 kB 00:00 2024-08-02T17:34:57.848 INFO:teuthology.orchestra.run.smithi069.stdout:(23/160): python3-rados-19.1.0-1260.g26c3fb8e.e 7.3 MB/s | 372 kB 00:00 2024-08-02T17:34:57.851 INFO:teuthology.orchestra.run.smithi077.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-08-02T17:34:57.871 INFO:teuthology.orchestra.run.smithi083.stdout:(37/160): ceph-mgr-dashboard-19.1.0-1260.g26c3f 8.0 MB/s | 3.6 MB 00:00 2024-08-02T17:34:57.888 INFO:teuthology.orchestra.run.smithi083.stdout:(38/160): cephfs-top-19.1.0-1260.g26c3fb8e.el9. 243 kB/s | 18 kB 00:00 2024-08-02T17:34:57.892 INFO:teuthology.orchestra.run.smithi077.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-02T17:34:57.915 INFO:teuthology.orchestra.run.smithi069.stdout:(24/160): librgw2-19.1.0-1260.g26c3fb8e.el9.x86 5.3 MB/s | 5.4 MB 00:01 2024-08-02T17:34:57.940 INFO:teuthology.orchestra.run.smithi069.stdout:(25/160): python3-rgw-19.1.0-1260.g26c3fb8e.el9 1.2 MB/s | 116 kB 00:00 2024-08-02T17:34:57.965 INFO:teuthology.orchestra.run.smithi069.stdout:(26/160): python3-rbd-19.1.0-1260.g26c3fb8e.el9 2.4 MB/s | 349 kB 00:00 2024-08-02T17:34:57.991 INFO:teuthology.orchestra.run.smithi069.stdout:(27/160): rbd-fuse-19.1.0-1260.g26c3fb8e.el9.x8 1.1 MB/s | 85 kB 00:00 2024-08-02T17:34:58.005 INFO:teuthology.orchestra.run.smithi083.stdout:(39/160): ceph-mgr-diskprediction-local-19.1.0- 14 MB/s | 7.4 MB 00:00 2024-08-02T17:34:58.050 INFO:teuthology.orchestra.run.smithi069.stdout:(28/160): rbd-nbd-19.1.0-1260.g26c3fb8e.el9.x86 2.0 MB/s | 170 kB 00:00 2024-08-02T17:34:58.075 INFO:teuthology.orchestra.run.smithi069.stdout:(29/160): ceph-grafana-dashboards-19.1.0-1260.g 381 kB/s | 32 kB 00:00 2024-08-02T17:34:58.133 INFO:teuthology.orchestra.run.smithi069.stdout:(30/160): rbd-mirror-19.1.0-1260.g26c3fb8e.el9. 16 MB/s | 3.1 MB 00:00 2024-08-02T17:34:58.158 INFO:teuthology.orchestra.run.smithi069.stdout:(31/160): ceph-mgr-cephadm-19.1.0-1260.g26c3fb8 1.3 MB/s | 146 kB 00:00 2024-08-02T17:34:58.193 INFO:teuthology.orchestra.run.smithi077.stdout:(149/160): librados2-19.1.0-1260.g26c3fb8e.el9. 9.1 MB/s | 3.4 MB 00:00 2024-08-02T17:34:58.234 INFO:teuthology.orchestra.run.smithi069.stdout:(32/160): ceph-mgr-modules-core-19.1.0-1260.g26 3.2 MB/s | 249 kB 00:00 2024-08-02T17:34:58.252 INFO:teuthology.orchestra.run.smithi077.stdout:(150/160): librbd1-19.1.0-1260.g26c3fb8e.el9.x8 7.9 MB/s | 3.2 MB 00:00 2024-08-02T17:34:58.301 INFO:teuthology.orchestra.run.smithi069.stdout:(33/160): ceph-mgr-rook-19.1.0-1260.g26c3fb8e.e 740 kB/s | 49 kB 00:00 2024-08-02T17:34:58.360 INFO:teuthology.orchestra.run.smithi069.stdout:(34/160): ceph-prometheus-alerts-19.1.0-1260.g2 282 kB/s | 16 kB 00:00 2024-08-02T17:34:58.435 INFO:teuthology.orchestra.run.smithi077.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 222 kB/s | 41 kB 00:00 2024-08-02T17:34:58.460 INFO:teuthology.orchestra.run.smithi069.stdout:(35/160): ceph-mgr-diskprediction-local-19.1.0- 23 MB/s | 7.4 MB 00:00 2024-08-02T17:34:58.485 INFO:teuthology.orchestra.run.smithi069.stdout:(36/160): ceph-volume-19.1.0-1260.g26c3fb8e.el9 2.1 MB/s | 274 kB 00:00 2024-08-02T17:34:58.519 INFO:teuthology.orchestra.run.smithi069.stdout:(37/160): cephadm-19.1.0-1260.g26c3fb8e.el9.noa 13 MB/s | 765 kB 00:00 2024-08-02T17:34:58.536 INFO:teuthology.orchestra.run.smithi069.stdout:(38/160): cephfs-top-19.1.0-1260.g26c3fb8e.el9. 364 kB/s | 18 kB 00:00 2024-08-02T17:34:58.581 INFO:teuthology.orchestra.run.smithi083.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 113 kB/s | 80 kB 00:00 2024-08-02T17:34:58.586 INFO:teuthology.orchestra.run.smithi077.stdout:(152/160): NetworkManager-1.48.2-2.el9.x86_64.r 3.3 MB/s | 2.3 MB 00:00 2024-08-02T17:34:58.594 INFO:teuthology.orchestra.run.smithi069.stdout:(39/160): ceph-mgr-dashboard-19.1.0-1260.g26c3f 6.9 MB/s | 3.6 MB 00:00 2024-08-02T17:34:58.711 INFO:teuthology.orchestra.run.smithi077.stdout:(153/160): elfutils-debuginfod-client-0.191-4.e 293 kB/s | 37 kB 00:00 2024-08-02T17:34:58.736 INFO:teuthology.orchestra.run.smithi077.stdout:(154/160): NetworkManager-tui-1.48.2-2.el9.x86_ 822 kB/s | 247 kB 00:00 2024-08-02T17:34:58.745 INFO:teuthology.orchestra.run.smithi069.stdout:(40/160): fuse-2.9.9-16.el9.x86_64.rpm 354 kB/s | 80 kB 00:00 2024-08-02T17:34:58.814 INFO:teuthology.orchestra.run.smithi083.stdout:(41/160): libquadmath-11.4.1-3.el9.x86_64.rpm 242 kB/s | 195 kB 00:00 2024-08-02T17:34:58.828 INFO:teuthology.orchestra.run.smithi077.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 1.7 MB/s | 207 kB 00:00 2024-08-02T17:34:58.853 INFO:teuthology.orchestra.run.smithi077.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 2.2 MB/s | 260 kB 00:00 2024-08-02T17:34:58.861 INFO:teuthology.orchestra.run.smithi069.stdout:(41/160): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-08-02T17:34:58.887 INFO:teuthology.orchestra.run.smithi069.stdout:(42/160): libquadmath-11.4.1-3.el9.x86_64.rpm 668 kB/s | 195 kB 00:00 2024-08-02T17:34:58.906 INFO:teuthology.orchestra.run.smithi083.stdout:(42/160): libgfortran-11.4.1-3.el9.x86_64.rpm 789 kB/s | 803 kB 00:01 2024-08-02T17:34:58.912 INFO:teuthology.orchestra.run.smithi069.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-08-02T17:34:58.945 INFO:teuthology.orchestra.run.smithi077.stdout:(157/160): NetworkManager-libnm-1.48.2-2.el9.x8 2.4 MB/s | 1.8 MB 00:00 2024-08-02T17:34:58.971 INFO:teuthology.orchestra.run.smithi069.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-08-02T17:34:58.987 INFO:teuthology.orchestra.run.smithi069.stdout:(45/160): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-08-02T17:34:59.030 INFO:teuthology.orchestra.run.smithi069.stdout:(46/160): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-08-02T17:34:59.062 INFO:teuthology.orchestra.run.smithi077.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-08-02T17:34:59.063 INFO:teuthology.orchestra.run.smithi069.stdout:(47/160): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-08-02T17:34:59.171 INFO:teuthology.orchestra.run.smithi077.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 4.8 MB/s | 1.1 MB 00:00 2024-08-02T17:34:59.238 INFO:teuthology.orchestra.run.smithi077.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 6.2 MB/s | 2.4 MB 00:00 2024-08-02T17:34:59.238 INFO:teuthology.orchestra.run.smithi069.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-08-02T17:34:59.243 INFO:teuthology.orchestra.run.smithi077.stdout:-------------------------------------------------------------------------------- 2024-08-02T17:34:59.243 INFO:teuthology.orchestra.run.smithi077.stdout:Total 22 MB/s | 228 MB 00:10 2024-08-02T17:34:59.272 INFO:teuthology.orchestra.run.smithi069.stdout:(49/160): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-08-02T17:34:59.297 INFO:teuthology.orchestra.run.smithi069.stdout:(50/160): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-08-02T17:34:59.331 INFO:teuthology.orchestra.run.smithi069.stdout:(51/160): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 943 kB 00:00 2024-08-02T17:34:59.356 INFO:teuthology.orchestra.run.smithi069.stdout:(52/160): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-08-02T17:34:59.381 INFO:teuthology.orchestra.run.smithi069.stdout:(53/160): elfutils-libelf-devel-0.191-4.el9.x86 199 kB/s | 22 kB 00:00 2024-08-02T17:34:59.390 INFO:teuthology.orchestra.run.smithi083.stdout:(43/160): python3-cffi-1.14.5-5.el9.x86_64.rpm 313 kB/s | 253 kB 00:00 2024-08-02T17:34:59.407 INFO:teuthology.orchestra.run.smithi069.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 433 kB/s | 33 kB 00:00 2024-08-02T17:34:59.432 INFO:teuthology.orchestra.run.smithi069.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-08-02T17:34:59.449 INFO:teuthology.orchestra.run.smithi069.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-08-02T17:34:59.482 INFO:teuthology.orchestra.run.smithi069.stdout:(57/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-08-02T17:34:59.533 INFO:teuthology.orchestra.run.smithi069.stdout:(58/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-08-02T17:34:59.549 INFO:teuthology.orchestra.run.smithi069.stdout:(59/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-08-02T17:34:59.575 INFO:teuthology.orchestra.run.smithi069.stdout:(60/160): librdkafka-1.6.1-102.el9.x86_64.rpm 7.0 MB/s | 662 kB 00:00 2024-08-02T17:34:59.590 INFO:teuthology.orchestra.run.smithi083.stdout:(44/160): python3-ply-3.11-14.el9.noarch.rpm 156 kB/s | 106 kB 00:00 2024-08-02T17:34:59.608 INFO:teuthology.orchestra.run.smithi069.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-08-02T17:34:59.625 INFO:teuthology.orchestra.run.smithi069.stdout:(62/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 336 kB/s | 25 kB 00:00 2024-08-02T17:34:59.642 INFO:teuthology.orchestra.run.smithi069.stdout:(63/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-08-02T17:34:59.676 INFO:teuthology.orchestra.run.smithi069.stdout:(64/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 292 kB 00:00 2024-08-02T17:34:59.693 INFO:teuthology.orchestra.run.smithi069.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 698 kB/s | 35 kB 00:00 2024-08-02T17:34:59.752 INFO:teuthology.orchestra.run.smithi069.stdout:(66/160): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-08-02T17:34:59.807 INFO:teuthology.orchestra.run.smithi083.stdout:(45/160): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-08-02T17:34:59.819 INFO:teuthology.orchestra.run.smithi069.stdout:(67/160): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2024-08-02T17:34:59.894 INFO:teuthology.orchestra.run.smithi069.stdout:(68/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-08-02T17:34:59.938 INFO:teuthology.orchestra.run.smithi069.stdout:(69/160): postgresql-contrib-13.14-1.el9.x86_64 7.5 MB/s | 911 kB 00:00 2024-08-02T17:34:59.961 INFO:teuthology.orchestra.run.smithi069.stdout:(70/160): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-08-02T17:35:00.003 INFO:teuthology.orchestra.run.smithi069.stdout:(71/160): postgresql-13.14-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-08-02T17:35:00.045 INFO:teuthology.orchestra.run.smithi069.stdout:(72/160): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-08-02T17:35:00.070 INFO:teuthology.orchestra.run.smithi069.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-08-02T17:35:00.104 INFO:teuthology.orchestra.run.smithi069.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-08-02T17:35:00.140 INFO:teuthology.orchestra.run.smithi083.stdout:(46/160): python3-pycparser-2.20-6.el9.noarch.r 180 kB/s | 135 kB 00:00 2024-08-02T17:35:00.179 INFO:teuthology.orchestra.run.smithi069.stdout:(75/160): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-08-02T17:35:00.205 INFO:teuthology.orchestra.run.smithi069.stdout:(76/160): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-08-02T17:35:00.230 INFO:teuthology.orchestra.run.smithi069.stdout:(77/160): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-02T17:35:00.247 INFO:teuthology.orchestra.run.smithi069.stdout:(78/160): python3-jsonpatch-1.21-16.el9.noarch. 611 kB/s | 26 kB 00:00 2024-08-02T17:35:00.264 INFO:teuthology.orchestra.run.smithi069.stdout:(79/160): python3-jsonpointer-2.0-4.el9.noarch. 567 kB/s | 19 kB 00:00 2024-08-02T17:35:00.299 INFO:teuthology.orchestra.run.smithi083.stdout:(47/160): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-08-02T17:35:00.347 INFO:teuthology.orchestra.run.smithi069.stdout:(80/160): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-08-02T17:35:00.373 INFO:teuthology.orchestra.run.smithi069.stdout:(81/160): python3-markupsafe-1.1.1-12.el9.x86_6 321 kB/s | 35 kB 00:00 2024-08-02T17:35:00.383 INFO:teuthology.orchestra.run.smithi083.stdout:(48/160): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-08-02T17:35:00.398 INFO:teuthology.orchestra.run.smithi069.stdout:(82/160): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-08-02T17:35:00.423 INFO:teuthology.orchestra.run.smithi069.stdout:(83/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-08-02T17:35:00.448 INFO:teuthology.orchestra.run.smithi069.stdout:(84/160): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-08-02T17:35:00.474 INFO:teuthology.orchestra.run.smithi069.stdout:(85/160): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-02T17:35:00.490 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction check 2024-08-02T17:35:00.499 INFO:teuthology.orchestra.run.smithi069.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-08-02T17:35:00.524 INFO:teuthology.orchestra.run.smithi069.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-02T17:35:00.558 INFO:teuthology.orchestra.run.smithi083.stdout:(49/160): boost-program-options-1.75.0-8.el9.x8 613 kB/s | 107 kB 00:00 2024-08-02T17:35:00.566 INFO:teuthology.orchestra.run.smithi069.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-08-02T17:35:00.591 INFO:teuthology.orchestra.run.smithi083.stdout:(50/160): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-08-02T17:35:00.592 INFO:teuthology.orchestra.run.smithi069.stdout:(89/160): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-08-02T17:35:00.617 INFO:teuthology.orchestra.run.smithi069.stdout:(90/160): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-02T17:35:00.617 INFO:teuthology.orchestra.run.smithi083.stdout:(51/160): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-08-02T17:35:00.642 INFO:teuthology.orchestra.run.smithi069.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-02T17:35:00.642 INFO:teuthology.orchestra.run.smithi083.stdout:(52/160): python3-urllib3-1.26.5-6.el9.noarch.r 266 kB/s | 215 kB 00:00 2024-08-02T17:35:00.668 INFO:teuthology.orchestra.run.smithi083.stdout:(53/160): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 313 kB 00:00 2024-08-02T17:35:00.679 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction check succeeded. 2024-08-02T17:35:00.679 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction test 2024-08-02T17:35:00.693 INFO:teuthology.orchestra.run.smithi083.stdout:(54/160): flexiblas-3.0.4-8.el9.x86_64.rpm 438 kB/s | 33 kB 00:00 2024-08-02T17:35:00.718 INFO:teuthology.orchestra.run.smithi083.stdout:(55/160): flexiblas-openblas-openmp-3.0.4-8.el9 361 kB/s | 18 kB 00:00 2024-08-02T17:35:00.734 INFO:teuthology.orchestra.run.smithi069.stdout:(92/160): python3-numpy-1.20.1-5.el9.x86_64.rpm 13 MB/s | 5.1 MB 00:00 2024-08-02T17:35:00.744 INFO:teuthology.orchestra.run.smithi083.stdout:(56/160): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-08-02T17:35:00.759 INFO:teuthology.orchestra.run.smithi069.stdout:(93/160): python3-toml-0.10.2-6.el9.noarch.rpm 355 kB/s | 42 kB 00:00 2024-08-02T17:35:00.785 INFO:teuthology.orchestra.run.smithi069.stdout:(94/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 880 kB/s | 44 kB 00:00 2024-08-02T17:35:00.794 INFO:teuthology.orchestra.run.smithi083.stdout:(57/160): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-08-02T17:35:00.818 INFO:teuthology.orchestra.run.smithi069.stdout:(95/160): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-08-02T17:35:00.844 INFO:teuthology.orchestra.run.smithi069.stdout:(96/160): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-08-02T17:35:00.869 INFO:teuthology.orchestra.run.smithi069.stdout:(97/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-08-02T17:35:00.894 INFO:teuthology.orchestra.run.smithi069.stdout:(98/160): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-08-02T17:35:00.902 INFO:teuthology.orchestra.run.smithi083.stdout:(58/160): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-08-02T17:35:00.920 INFO:teuthology.orchestra.run.smithi069.stdout:(99/160): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-02T17:35:00.928 INFO:teuthology.orchestra.run.smithi083.stdout:(59/160): libpmemobj-1.12.1-1.el9.x86_64.rpm 766 kB/s | 160 kB 00:00 2024-08-02T17:35:00.945 INFO:teuthology.orchestra.run.smithi069.stdout:(100/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-02T17:35:00.970 INFO:teuthology.orchestra.run.smithi069.stdout:(101/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-02T17:35:00.978 INFO:teuthology.orchestra.run.smithi083.stdout:(60/160): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-08-02T17:35:00.995 INFO:teuthology.orchestra.run.smithi069.stdout:(102/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-08-02T17:35:01.004 INFO:teuthology.orchestra.run.smithi083.stdout:(61/160): libtool-2.4.6-46.el9.x86_64.rpm 5.6 MB/s | 581 kB 00:00 2024-08-02T17:35:01.021 INFO:teuthology.orchestra.run.smithi069.stdout:(103/160): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-08-02T17:35:01.029 INFO:teuthology.orchestra.run.smithi083.stdout:(62/160): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-08-02T17:35:01.063 INFO:teuthology.orchestra.run.smithi083.stdout:(63/160): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-08-02T17:35:01.079 INFO:teuthology.orchestra.run.smithi069.stdout:(104/160): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-08-02T17:35:01.088 INFO:teuthology.orchestra.run.smithi083.stdout:(64/160): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-08-02T17:35:01.105 INFO:teuthology.orchestra.run.smithi083.stdout:(65/160): openblas-0.3.21-2.el9.x86_64.rpm 842 kB/s | 35 kB 00:00 2024-08-02T17:35:01.147 INFO:teuthology.orchestra.run.smithi069.stdout:(105/160): abseil-cpp-20211102.0-3.el9.x86_64.r 4.3 MB/s | 548 kB 00:00 2024-08-02T17:35:01.171 INFO:teuthology.orchestra.run.smithi069.stdout:(106/160): grpc-data-1.46.7-10.el9.noarch.rpm 866 kB/s | 19 kB 00:00 2024-08-02T17:35:01.213 INFO:teuthology.orchestra.run.smithi069.stdout:(107/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.3 MB/s | 309 kB 00:00 2024-08-02T17:35:01.239 INFO:teuthology.orchestra.run.smithi069.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-02T17:35:01.264 INFO:teuthology.orchestra.run.smithi069.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-02T17:35:01.289 INFO:teuthology.orchestra.run.smithi069.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-08-02T17:35:01.315 INFO:teuthology.orchestra.run.smithi069.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-08-02T17:35:01.373 INFO:teuthology.orchestra.run.smithi069.stdout:(112/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-08-02T17:35:01.449 INFO:teuthology.orchestra.run.smithi069.stdout:(113/160): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-08-02T17:35:01.474 INFO:teuthology.orchestra.run.smithi069.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-02T17:35:01.499 INFO:teuthology.orchestra.run.smithi069.stdout:(115/160): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-08-02T17:35:01.524 INFO:teuthology.orchestra.run.smithi069.stdout:(116/160): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-08-02T17:35:01.550 INFO:teuthology.orchestra.run.smithi069.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-08-02T17:35:01.563 INFO:teuthology.orchestra.run.smithi083.stdout:(66/160): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.2 MB/s | 3.0 MB 00:00 2024-08-02T17:35:01.639 INFO:teuthology.orchestra.run.smithi083.stdout:(67/160): openssl-devel-3.2.2-2.el9.x86_64.rpm 8.3 MB/s | 4.4 MB 00:00 2024-08-02T17:35:01.709 INFO:teuthology.orchestra.run.smithi069.stdout:(118/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-08-02T17:35:01.730 INFO:teuthology.orchestra.run.smithi083.stdout:(68/160): openblas-openmp-0.3.21-2.el9.x86_64.r 7.4 MB/s | 4.7 MB 00:00 2024-08-02T17:35:01.734 INFO:teuthology.orchestra.run.smithi069.stdout:(119/160): python3-cheroot-8.6.0-4.el9.noarch.r 821 kB/s | 172 kB 00:00 2024-08-02T17:35:01.781 INFO:teuthology.orchestra.run.smithi083.stdout:(69/160): postgresql-private-libs-13.14-1.el9.x 2.6 MB/s | 133 kB 00:00 2024-08-02T17:35:01.784 INFO:teuthology.orchestra.run.smithi069.stdout:(120/160): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-08-02T17:35:01.809 INFO:teuthology.orchestra.run.smithi069.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.2 MB/s | 216 kB 00:00 2024-08-02T17:35:01.815 INFO:teuthology.orchestra.run.smithi083.stdout:(70/160): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-08-02T17:35:01.843 INFO:teuthology.orchestra.run.smithi069.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-02T17:35:01.868 INFO:teuthology.orchestra.run.smithi069.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-08-02T17:35:01.894 INFO:teuthology.orchestra.run.smithi069.stdout:(124/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-08-02T17:35:01.939 INFO:teuthology.orchestra.run.smithi069.stdout:(125/160): python3-jaraco-classes-3.2.1-5.el9.n 252 kB/s | 18 kB 00:00 2024-08-02T17:35:01.961 INFO:teuthology.orchestra.run.smithi069.stdout:(126/160): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-08-02T17:35:01.965 INFO:teuthology.orchestra.run.smithi083.stdout:(71/160): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-08-02T17:35:01.986 INFO:teuthology.orchestra.run.smithi069.stdout:(127/160): python3-jaraco-functools-3.5.0-2.el9 412 kB/s | 19 kB 00:00 2024-08-02T17:35:02.011 INFO:teuthology.orchestra.run.smithi069.stdout:(128/160): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-08-02T17:35:02.036 INFO:teuthology.orchestra.run.smithi083.stdout:(72/160): postgresql-13.14-1.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-08-02T17:35:02.037 INFO:teuthology.orchestra.run.smithi069.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-02T17:35:02.057 INFO:teuthology.orchestra.run.smithi083.stdout:(73/160): python3-attrs-20.3.0-7.el9.noarch.rpm 954 kB/s | 87 kB 00:00 2024-08-02T17:35:02.062 INFO:teuthology.orchestra.run.smithi069.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-02T17:35:02.079 INFO:teuthology.orchestra.run.smithi069.stdout:(131/160): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-08-02T17:35:02.112 INFO:teuthology.orchestra.run.smithi069.stdout:(132/160): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-08-02T17:35:02.146 INFO:teuthology.orchestra.run.smithi069.stdout:(133/160): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-08-02T17:35:02.163 INFO:teuthology.orchestra.run.smithi069.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-08-02T17:35:02.180 INFO:teuthology.orchestra.run.smithi069.stdout:(135/160): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-08-02T17:35:02.205 INFO:teuthology.orchestra.run.smithi069.stdout:(136/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-08-02T17:35:02.231 INFO:teuthology.orchestra.run.smithi069.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-08-02T17:35:02.256 INFO:teuthology.orchestra.run.smithi069.stdout:(138/160): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-02T17:35:02.290 INFO:teuthology.orchestra.run.smithi069.stdout:(139/160): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-08-02T17:35:02.315 INFO:teuthology.orchestra.run.smithi069.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 612 kB/s | 36 kB 00:00 2024-08-02T17:35:02.324 INFO:teuthology.orchestra.run.smithi083.stdout:(74/160): python3-colorama-0.4.6-3.el9.noarch.r 229 kB/s | 61 kB 00:00 2024-08-02T17:35:02.357 INFO:teuthology.orchestra.run.smithi069.stdout:(141/160): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-08-02T17:35:02.382 INFO:teuthology.orchestra.run.smithi069.stdout:(142/160): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-08-02T17:35:02.407 INFO:teuthology.orchestra.run.smithi069.stdout:(143/160): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-02T17:35:02.433 INFO:teuthology.orchestra.run.smithi069.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-08-02T17:35:02.458 INFO:teuthology.orchestra.run.smithi069.stdout:(145/160): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-08-02T17:35:02.483 INFO:teuthology.orchestra.run.smithi069.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-02T17:35:02.492 INFO:teuthology.orchestra.run.smithi077.stdout:Transaction test succeeded. 2024-08-02T17:35:02.492 INFO:teuthology.orchestra.run.smithi077.stdout:Running transaction 2024-08-02T17:35:02.508 INFO:teuthology.orchestra.run.smithi069.stdout:(147/160): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-02T17:35:02.524 INFO:teuthology.orchestra.run.smithi083.stdout:(75/160): python3-devel-3.9.18-3.el9.x86_64.rpm 1.2 MB/s | 244 kB 00:00 2024-08-02T17:35:02.550 INFO:teuthology.orchestra.run.smithi069.stdout:(148/160): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-08-02T17:35:02.675 INFO:teuthology.orchestra.run.smithi083.stdout:(76/160): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-08-02T17:35:02.742 INFO:teuthology.orchestra.run.smithi069.stdout:(149/160): librbd1-19.1.0-1260.g26c3fb8e.el9.x8 14 MB/s | 3.2 MB 00:00 2024-08-02T17:35:02.766 INFO:teuthology.orchestra.run.smithi083.stdout:(77/160): python3-jsonpatch-1.21-16.el9.noarch. 280 kB/s | 26 kB 00:00 2024-08-02T17:35:02.843 INFO:teuthology.orchestra.run.smithi069.stdout:(150/160): librados2-19.1.0-1260.g26c3fb8e.el9. 9.5 MB/s | 3.4 MB 00:00 2024-08-02T17:35:02.858 INFO:teuthology.orchestra.run.smithi083.stdout:(78/160): python3-jsonpointer-2.0-4.el9.noarch. 209 kB/s | 19 kB 00:00 2024-08-02T17:35:02.942 INFO:teuthology.orchestra.run.smithi083.stdout:(79/160): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-08-02T17:35:02.976 INFO:teuthology.orchestra.run.smithi069.stdout:(151/160): NetworkManager-team-1.48.2-2.el9.x86 305 kB/s | 41 kB 00:00 2024-08-02T17:35:03.076 INFO:teuthology.orchestra.run.smithi083.stdout:(80/160): python3-markupsafe-1.1.1-12.el9.x86_6 260 kB/s | 35 kB 00:00 2024-08-02T17:35:03.110 INFO:teuthology.orchestra.run.smithi069.stdout:(152/160): NetworkManager-tui-1.48.2-2.el9.x86_ 1.8 MB/s | 247 kB 00:00 2024-08-02T17:35:03.152 INFO:teuthology.orchestra.run.smithi069.stdout:(153/160): NetworkManager-1.48.2-2.el9.x86_64.r 3.8 MB/s | 2.3 MB 00:00 2024-08-02T17:35:03.177 INFO:teuthology.orchestra.run.smithi069.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 548 kB/s | 37 kB 00:00 2024-08-02T17:35:03.219 INFO:teuthology.orchestra.run.smithi069.stdout:(155/160): NetworkManager-libnm-1.48.2-2.el9.x8 3.8 MB/s | 1.8 MB 00:00 2024-08-02T17:35:03.245 INFO:teuthology.orchestra.run.smithi069.stdout:(156/160): elfutils-libs-0.191-4.el9.x86_64.rpm 3.8 MB/s | 260 kB 00:00 2024-08-02T17:35:03.326 INFO:teuthology.orchestra.run.smithi083.stdout:(81/160): postgresql-server-13.14-1.el9.x86_64. 3.7 MB/s | 5.8 MB 00:01 2024-08-02T17:35:03.386 INFO:teuthology.orchestra.run.smithi069.stdout:(157/160): elfutils-libelf-0.191-4.el9.x86_64.r 884 kB/s | 207 kB 00:00 2024-08-02T17:35:03.503 INFO:teuthology.orchestra.run.smithi069.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 4.8 MB/s | 1.4 MB 00:00 2024-08-02T17:35:03.677 INFO:teuthology.orchestra.run.smithi083.stdout:(82/160): python3-numpy-f2py-1.20.1-5.el9.x86_6 716 kB/s | 250 kB 00:00 2024-08-02T17:35:03.712 INFO:teuthology.orchestra.run.smithi069.stdout:(159/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 5.1 MB/s | 2.4 MB 00:00 2024-08-02T17:35:03.810 INFO:teuthology.orchestra.run.smithi083.stdout:(83/160): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 222 kB 00:00 2024-08-02T17:35:03.885 INFO:teuthology.orchestra.run.smithi083.stdout:(84/160): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-08-02T17:35:03.986 INFO:teuthology.orchestra.run.smithi083.stdout:(85/160): python3-babel-2.9.1-2.el9.noarch.rpm 3.1 MB/s | 6.0 MB 00:01 2024-08-02T17:35:04.011 INFO:teuthology.orchestra.run.smithi083.stdout:(86/160): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-08-02T17:35:04.136 INFO:teuthology.orchestra.run.smithi083.stdout:(87/160): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-08-02T17:35:04.162 INFO:teuthology.orchestra.run.smithi083.stdout:(88/160): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-08-02T17:35:04.245 INFO:teuthology.orchestra.run.smithi083.stdout:(89/160): python3-pytz-2021.1-5.el9.noarch.rpm 611 kB/s | 51 kB 00:00 2024-08-02T17:35:04.271 INFO:teuthology.orchestra.run.smithi083.stdout:(90/160): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-08-02T17:35:04.321 INFO:teuthology.orchestra.run.smithi083.stdout:(91/160): python3-requests-oauthlib-1.3.0-12.el 714 kB/s | 54 kB 00:00 2024-08-02T17:35:04.396 INFO:teuthology.orchestra.run.smithi083.stdout:(92/160): python3-toml-0.10.2-6.el9.noarch.rpm 556 kB/s | 42 kB 00:00 2024-08-02T17:35:04.412 INFO:teuthology.orchestra.run.smithi069.stdout:(160/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 1.1 MB/s | 1.1 MB 00:01 2024-08-02T17:35:04.419 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-08-02T17:35:04.419 INFO:teuthology.orchestra.run.smithi069.stdout:Total 18 MB/s | 228 MB 00:12 2024-08-02T17:35:04.472 INFO:teuthology.orchestra.run.smithi083.stdout:(93/160): python3-wcwidth-0.2.5-8.el9.noarch.rp 591 kB/s | 44 kB 00:00 2024-08-02T17:35:04.597 INFO:teuthology.orchestra.run.smithi083.stdout:(94/160): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-08-02T17:35:04.672 INFO:teuthology.orchestra.run.smithi083.stdout:(95/160): qatlib-service-24.02.0-1.el9.x86_64.r 478 kB/s | 36 kB 00:00 2024-08-02T17:35:04.748 INFO:teuthology.orchestra.run.smithi083.stdout:(96/160): qatzip-libs-1.2.0-1.el9.x86_64.rpm 634 kB/s | 47 kB 00:00 2024-08-02T17:35:04.873 INFO:teuthology.orchestra.run.smithi083.stdout:(97/160): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-08-02T17:35:04.948 INFO:teuthology.orchestra.run.smithi083.stdout:(98/160): uuid-1.6.2-55.el9.x86_64.rpm 771 kB/s | 58 kB 00:00 2024-08-02T17:35:05.023 INFO:teuthology.orchestra.run.smithi083.stdout:(99/160): xfsprogs-devel-6.4.0-3.el9.x86_64.rpm 1.2 MB/s | 96 kB 00:00 2024-08-02T17:35:05.107 INFO:teuthology.orchestra.run.smithi083.stdout:(100/160): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-08-02T17:35:05.182 INFO:teuthology.orchestra.run.smithi083.stdout:(101/160): zlib-devel-1.2.11-41.el9.x86_64.rpm 599 kB/s | 45 kB 00:00 2024-08-02T17:35:05.366 INFO:teuthology.orchestra.run.smithi083.stdout:(102/160): lua-devel-5.4.4-4.el9.x86_64.rpm 122 kB/s | 22 kB 00:00 2024-08-02T17:35:05.683 INFO:teuthology.orchestra.run.smithi083.stdout:(103/160): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-08-02T17:35:05.684 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction check 2024-08-02T17:35:05.750 INFO:teuthology.orchestra.run.smithi083.stdout:(104/160): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-08-02T17:35:05.817 INFO:teuthology.orchestra.run.smithi083.stdout:(105/160): python3-numpy-1.20.1-5.el9.x86_64.rp 1.9 MB/s | 5.1 MB 00:02 2024-08-02T17:35:05.867 INFO:teuthology.orchestra.run.smithi083.stdout:(106/160): gperftools-libs-2.9.1-2.el9.x86_64.r 2.6 MB/s | 309 kB 00:00 2024-08-02T17:35:05.887 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction check succeeded. 2024-08-02T17:35:05.887 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction test 2024-08-02T17:35:05.895 INFO:teuthology.orchestra.run.smithi083.stdout:(107/160): grpc-data-1.46.7-10.el9.noarch.rpm 252 kB/s | 19 kB 00:00 2024-08-02T17:35:05.929 INFO:teuthology.orchestra.run.smithi083.stdout:(108/160): libarrow-doc-9.0.0-11.el9.noarch.rpm 779 kB/s | 26 kB 00:00 2024-08-02T17:35:05.954 INFO:teuthology.orchestra.run.smithi083.stdout:(109/160): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-02T17:35:05.987 INFO:teuthology.orchestra.run.smithi083.stdout:(110/160): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-08-02T17:35:06.021 INFO:teuthology.orchestra.run.smithi083.stdout:(111/160): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-08-02T17:35:06.096 INFO:teuthology.orchestra.run.smithi083.stdout:(112/160): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-08-02T17:35:06.130 INFO:teuthology.orchestra.run.smithi083.stdout:(113/160): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-08-02T17:35:06.155 INFO:teuthology.orchestra.run.smithi083.stdout:(114/160): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-02T17:35:06.180 INFO:teuthology.orchestra.run.smithi083.stdout:(115/160): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-02T17:35:06.206 INFO:teuthology.orchestra.run.smithi083.stdout:(116/160): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-08-02T17:35:06.231 INFO:teuthology.orchestra.run.smithi083.stdout:(117/160): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-08-02T17:35:06.256 INFO:teuthology.orchestra.run.smithi083.stdout:(118/160): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-02T17:35:06.282 INFO:teuthology.orchestra.run.smithi083.stdout:(119/160): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-08-02T17:35:06.307 INFO:teuthology.orchestra.run.smithi083.stdout:(120/160): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-08-02T17:35:06.332 INFO:teuthology.orchestra.run.smithi083.stdout:(121/160): python3-google-auth-2.32.0-1.el9.noa 4.2 MB/s | 216 kB 00:00 2024-08-02T17:35:06.366 INFO:teuthology.orchestra.run.smithi083.stdout:(122/160): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-02T17:35:06.391 INFO:teuthology.orchestra.run.smithi083.stdout:(123/160): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-08-02T17:35:06.417 INFO:teuthology.orchestra.run.smithi083.stdout:(124/160): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-08-02T17:35:06.442 INFO:teuthology.orchestra.run.smithi083.stdout:(125/160): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-08-02T17:35:06.467 INFO:teuthology.orchestra.run.smithi083.stdout:(126/160): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-08-02T17:35:06.493 INFO:teuthology.orchestra.run.smithi083.stdout:(127/160): python3-jaraco-text-3.2.0-6.el9.noar 784 kB/s | 20 kB 00:00 2024-08-02T17:35:06.543 INFO:teuthology.orchestra.run.smithi083.stdout:(128/160): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-08-02T17:35:06.568 INFO:teuthology.orchestra.run.smithi083.stdout:(129/160): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-02T17:35:06.585 INFO:teuthology.orchestra.run.smithi083.stdout:(130/160): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-08-02T17:35:06.619 INFO:teuthology.orchestra.run.smithi083.stdout:(131/160): python3-kubernetes-26.1.0-2.el9.noar 8.1 MB/s | 1.0 MB 00:00 2024-08-02T17:35:06.644 INFO:teuthology.orchestra.run.smithi083.stdout:(132/160): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-08-02T17:35:06.669 INFO:teuthology.orchestra.run.smithi083.stdout:(133/160): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-08-02T17:35:06.695 INFO:teuthology.orchestra.run.smithi083.stdout:(134/160): python3-portend-3.1.0-2.el9.noarch.r 325 kB/s | 16 kB 00:00 2024-08-02T17:35:06.720 INFO:teuthology.orchestra.run.smithi083.stdout:(135/160): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-08-02T17:35:06.745 INFO:teuthology.orchestra.run.smithi083.stdout:(136/160): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-08-02T17:35:06.771 INFO:teuthology.orchestra.run.smithi083.stdout:(137/160): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-08-02T17:35:06.796 INFO:teuthology.orchestra.run.smithi083.stdout:(138/160): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-08-02T17:35:06.821 INFO:teuthology.orchestra.run.smithi083.stdout:(139/160): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-08-02T17:35:06.838 INFO:teuthology.orchestra.run.smithi083.stdout:(140/160): python3-tempora-5.0.0-2.el9.noarch.r 858 kB/s | 36 kB 00:00 2024-08-02T17:35:06.863 INFO:teuthology.orchestra.run.smithi083.stdout:(141/160): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-08-02T17:35:06.897 INFO:teuthology.orchestra.run.smithi083.stdout:(142/160): python3-websocket-client-1.2.3-2.el9 2.6 MB/s | 90 kB 00:00 2024-08-02T17:35:06.923 INFO:teuthology.orchestra.run.smithi083.stdout:(143/160): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-08-02T17:35:06.939 INFO:teuthology.orchestra.run.smithi083.stdout:(144/160): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-08-02T17:35:06.965 INFO:teuthology.orchestra.run.smithi083.stdout:(145/160): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-08-02T17:35:06.990 INFO:teuthology.orchestra.run.smithi083.stdout:(146/160): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-02T17:35:07.041 INFO:teuthology.orchestra.run.smithi083.stdout:(147/160): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-08-02T17:35:07.299 INFO:teuthology.orchestra.run.smithi083.stdout:(148/160): librbd1-19.1.0-1260.g26c3fb8e.el9.x8 12 MB/s | 3.2 MB 00:00 2024-08-02T17:35:07.350 INFO:teuthology.orchestra.run.smithi083.stdout:(149/160): librados2-19.1.0-1260.g26c3fb8e.el9. 9.5 MB/s | 3.4 MB 00:00 2024-08-02T17:35:08.019 INFO:teuthology.orchestra.run.smithi069.stdout:Transaction test succeeded. 2024-08-02T17:35:08.020 INFO:teuthology.orchestra.run.smithi069.stdout:Running transaction 2024-08-02T17:35:08.316 INFO:teuthology.orchestra.run.smithi083.stdout:(150/160): NetworkManager-1.48.2-2.el9.x86_64.r 2.3 MB/s | 2.3 MB 00:01 2024-08-02T17:35:08.358 INFO:teuthology.orchestra.run.smithi083.stdout:(151/160): NetworkManager-libnm-1.48.2-2.el9.x8 1.8 MB/s | 1.8 MB 00:01 2024-08-02T17:35:08.896 INFO:teuthology.orchestra.run.smithi077.stdout: Preparing : 1/1 2024-08-02T17:35:08.917 INFO:teuthology.orchestra.run.smithi083.stdout:(152/160): NetworkManager-team-1.48.2-2.el9.x86 68 kB/s | 41 kB 00:00 2024-08-02T17:35:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-02T17:35:09.005 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-02T17:35:09.067 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-02T17:35:09.101 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-02T17:35:09.127 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-02T17:35:09.170 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-02T17:35:09.192 INFO:teuthology.orchestra.run.smithi083.stdout:(153/160): NetworkManager-tui-1.48.2-2.el9.x86_ 297 kB/s | 247 kB 00:00 2024-08-02T17:35:09.501 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-02T17:35:09.526 INFO:teuthology.orchestra.run.smithi083.stdout:(154/160): elfutils-debuginfod-client-0.191-4.e 60 kB/s | 37 kB 00:00 2024-08-02T17:35:09.612 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:35:09.732 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:35:09.763 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:35:09.849 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:35:09.875 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 10/172 2024-08-02T17:35:09.908 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-02T17:35:09.968 INFO:teuthology.orchestra.run.smithi083.stdout:(155/160): elfutils-libelf-0.191-4.el9.x86_64.r 267 kB/s | 207 kB 00:00 2024-08-02T17:35:10.108 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-02T17:35:10.159 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-02T17:35:10.187 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:35:10.291 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:35:10.313 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 15/172 2024-08-02T17:35:10.326 INFO:teuthology.orchestra.run.smithi083.stdout:(156/160): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.7 MB/s | 16 MB 00:06 2024-08-02T17:35:10.359 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 15/172 2024-08-02T17:35:10.381 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-02T17:35:10.385 INFO:teuthology.orchestra.run.smithi083.stdout:(157/160): elfutils-libs-0.191-4.el9.x86_64.rpm 303 kB/s | 260 kB 00:00 2024-08-02T17:35:10.404 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-02T17:35:10.589 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-02T17:35:10.682 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-02T17:35:10.865 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-02T17:35:11.047 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-02T17:35:11.107 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-02T17:35:11.118 INFO:teuthology.orchestra.run.smithi083.stdout:(158/160): openssl-3.2.2-2.el9.x86_64.rpm 1.2 MB/s | 1.4 MB 00:01 2024-08-02T17:35:11.192 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:11.275 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:11.319 INFO:teuthology.orchestra.run.smithi083.stdout:(159/160): xfsprogs-6.4.0-3.el9.x86_64.rpm 1.2 MB/s | 1.1 MB 00:00 2024-08-02T17:35:11.394 INFO:teuthology.orchestra.run.smithi083.stdout:(160/160): openssl-libs-3.2.2-2.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-08-02T17:35:11.401 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-08-02T17:35:11.401 INFO:teuthology.orchestra.run.smithi083.stdout:Total 11 MB/s | 228 MB 00:20 2024-08-02T17:35:11.488 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:11.600 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-02T17:35:11.702 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-02T17:35:11.730 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-02T17:35:11.765 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-02T17:35:11.831 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-02T17:35:11.875 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-02T17:35:11.906 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-02T17:35:11.951 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-02T17:35:11.983 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-02T17:35:12.011 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-02T17:35:12.079 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-02T17:35:12.132 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-02T17:35:12.163 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-02T17:35:12.198 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-02T17:35:12.273 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-02T17:35:12.309 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9. 36/172 2024-08-02T17:35:12.355 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el 37/172 2024-08-02T17:35:12.382 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_6 38/172 2024-08-02T17:35:12.450 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-02T17:35:12.496 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-02T17:35:12.569 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-02T17:35:12.651 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-08-02T17:35:12.701 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-02T17:35:12.756 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-02T17:35:12.840 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-08-02T17:35:12.840 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-08-02T17:35:13.268 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-02T17:35:13.351 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-02T17:35:13.381 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-02T17:35:13.417 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-02T17:35:13.467 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-02T17:35:13.505 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_6 49/172 2024-08-02T17:35:13.546 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-02T17:35:13.707 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-02T17:35:13.766 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-02T17:35:13.821 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-02T17:35:13.892 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-02T17:35:13.925 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-02T17:35:13.961 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-02T17:35:14.008 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-02T17:35:14.052 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-02T17:35:14.096 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-02T17:35:14.137 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-02T17:35:14.271 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-02T17:35:14.323 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-02T17:35:14.347 INFO:teuthology.orchestra.run.smithi069.stdout: Preparing : 1/1 2024-08-02T17:35:14.364 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-02T17:35:14.425 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-02T17:35:14.461 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-02T17:35:14.469 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-02T17:35:14.489 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-08-02T17:35:14.526 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-02T17:35:14.558 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-02T17:35:14.584 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-02T17:35:14.596 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-02T17:35:14.640 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-02T17:35:14.724 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-08-02T17:35:14.725 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-08-02T17:35:14.839 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-02T17:35:14.897 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-02T17:35:14.965 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-02T17:35:14.977 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-02T17:35:15.068 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:35:15.108 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-02T17:35:15.170 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-02T17:35:15.198 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-02T17:35:15.198 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:35:15.227 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:35:15.232 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-02T17:35:15.271 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-02T17:35:15.303 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-02T17:35:15.323 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:35:15.332 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-02T17:35:15.358 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 10/172 2024-08-02T17:35:15.367 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-02T17:35:15.400 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-02T17:35:15.405 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-02T17:35:15.442 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-02T17:35:15.477 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-02T17:35:15.563 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-02T17:35:15.588 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-02T17:35:15.607 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-02T17:35:15.638 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-02T17:35:15.646 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-02T17:35:15.674 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:35:15.740 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-02T17:35:15.771 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:35:15.792 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 15/172 2024-08-02T17:35:15.797 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-02T17:35:15.838 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 15/172 2024-08-02T17:35:15.860 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-02T17:35:15.886 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-02T17:35:16.079 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-02T17:35:16.198 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-02T17:35:16.441 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-02T17:35:16.514 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-02T17:35:16.579 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-02T17:35:16.584 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-02T17:35:16.619 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-02T17:35:16.628 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-02T17:35:16.717 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-02T17:35:16.742 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:16.766 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-02T17:35:16.810 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-02T17:35:16.813 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:16.897 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-02T17:35:16.955 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-02T17:35:17.015 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:17.130 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-02T17:35:17.133 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-02T17:35:17.161 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-02T17:35:17.222 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-02T17:35:17.238 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-02T17:35:17.260 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-02T17:35:17.295 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-02T17:35:17.360 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-02T17:35:17.404 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-02T17:35:17.436 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-02T17:35:17.480 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-02T17:35:17.521 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-02T17:35:17.549 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-02T17:35:17.618 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-02T17:35:17.670 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-02T17:35:17.710 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-02T17:35:17.744 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-02T17:35:17.820 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-02T17:35:17.856 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9. 36/172 2024-08-02T17:35:17.878 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-02T17:35:17.912 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el 37/172 2024-08-02T17:35:17.953 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_6 38/172 2024-08-02T17:35:17.984 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-02T17:35:18.024 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-02T17:35:18.067 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-02T17:35:18.142 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-02T17:35:18.274 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-02T17:35:18.328 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-02T17:35:18.843 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-02T17:35:18.915 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-02T17:35:18.953 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-02T17:35:18.990 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-02T17:35:19.038 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-02T17:35:19.076 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_6 49/172 2024-08-02T17:35:19.118 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-02T17:35:19.202 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-02T17:35:19.282 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-02T17:35:19.298 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-02T17:35:19.325 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-02T17:35:19.329 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-02T17:35:19.358 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-02T17:35:19.383 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-02T17:35:19.457 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-02T17:35:19.488 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-02T17:35:19.524 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-02T17:35:19.572 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-02T17:35:19.607 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-02T17:35:19.649 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-02T17:35:19.667 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-02T17:35:19.694 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-02T17:35:19.713 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 104/172 2024-08-02T17:35:19.807 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 104/172 2024-08-02T17:35:19.830 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-02T17:35:19.835 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 105/172 2024-08-02T17:35:19.870 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-02T17:35:19.871 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-02T17:35:19.912 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-02T17:35:20.010 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-02T17:35:20.054 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-08-02T17:35:20.095 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-02T17:35:20.127 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-02T17:35:20.152 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-02T17:35:20.233 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-02T17:35:20.345 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-02T17:35:20.361 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-02T17:35:20.374 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 108/172 2024-08-02T17:35:20.393 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-02T17:35:20.446 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 108/172 2024-08-02T17:35:20.476 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 109/172 2024-08-02T17:35:20.655 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-02T17:35:20.855 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-02T17:35:20.896 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-02T17:35:20.953 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-02T17:35:21.093 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-02T17:35:21.144 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-08-02T17:35:21.305 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-02T17:35:21.357 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.2.2-2.el9.x86_64 1/172 2024-08-02T17:35:21.395 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/172 2024-08-02T17:35:21.449 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/172 2024-08-02T17:35:21.465 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-02T17:35:21.491 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/172 2024-08-02T17:35:21.516 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/172 2024-08-02T17:35:21.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/172 2024-08-02T17:35:21.576 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-02T17:35:21.609 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-02T17:35:21.699 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-02T17:35:21.748 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-02T17:35:21.778 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-02T17:35:21.883 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-02T17:35:21.889 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/172 2024-08-02T17:35:21.937 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-02T17:35:22.001 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:35:22.122 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:35:22.144 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:35:22.239 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:35:22.264 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 10/172 2024-08-02T17:35:22.297 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/172 2024-08-02T17:35:22.509 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/172 2024-08-02T17:35:22.571 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/172 2024-08-02T17:35:22.599 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:35:22.648 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-02T17:35:22.703 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:35:22.717 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-02T17:35:22.723 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:22.725 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 15/172 2024-08-02T17:35:22.751 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-02T17:35:22.761 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:22.788 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 15/172 2024-08-02T17:35:22.809 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-02T17:35:22.829 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/172 2024-08-02T17:35:22.849 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-02T17:35:22.906 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-02T17:35:22.950 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-02T17:35:23.034 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-02T17:35:23.039 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-02T17:35:23.078 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-02T17:35:23.128 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/172 2024-08-02T17:35:23.246 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-02T17:35:23.276 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-02T17:35:23.299 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : xfsprogs-6.4.0-3.el9.x86_64 18/172 2024-08-02T17:35:23.302 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:23.354 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-02T17:35:23.468 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-02T17:35:23.529 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 19/172 2024-08-02T17:35:23.551 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 111/172 2024-08-02T17:35:23.634 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:23.721 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:23.727 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 111/172 2024-08-02T17:35:23.727 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-02T17:35:23.727 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-02T17:35:23.727 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:23.778 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 112/172 2024-08-02T17:35:23.926 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.48.2-2.el9.x86_64 20/172 2024-08-02T17:35:23.992 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-02T17:35:24.038 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 21/172 2024-08-02T17:35:24.115 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-02T17:35:24.141 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 22/172 2024-08-02T17:35:24.185 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 23/172 2024-08-02T17:35:24.220 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 24/172 2024-08-02T17:35:24.286 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/172 2024-08-02T17:35:24.330 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/172 2024-08-02T17:35:24.370 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 27/172 2024-08-02T17:35:24.413 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/172 2024-08-02T17:35:24.446 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 29/172 2024-08-02T17:35:24.474 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/172 2024-08-02T17:35:24.551 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/172 2024-08-02T17:35:24.621 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 32/172 2024-08-02T17:35:24.652 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 33/172 2024-08-02T17:35:24.687 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 34/172 2024-08-02T17:35:24.763 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 35/172 2024-08-02T17:35:24.798 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9. 36/172 2024-08-02T17:35:24.843 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el 37/172 2024-08-02T17:35:24.861 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_6 38/172 2024-08-02T17:35:24.932 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 39/172 2024-08-02T17:35:24.977 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/172 2024-08-02T17:35:25.052 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 41/172 2024-08-02T17:35:25.184 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 42/172 2024-08-02T17:35:25.237 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/172 2024-08-02T17:35:25.352 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-02T17:35:25.438 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-02T17:35:25.465 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-02T17:35:25.498 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-02T17:35:25.771 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/172 2024-08-02T17:35:25.808 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-02T17:35:25.845 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 104/172 2024-08-02T17:35:25.850 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 45/172 2024-08-02T17:35:25.895 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 46/172 2024-08-02T17:35:25.923 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 47/172 2024-08-02T17:35:25.939 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 104/172 2024-08-02T17:35:25.967 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 105/172 2024-08-02T17:35:25.972 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 48/172 2024-08-02T17:35:26.002 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-02T17:35:26.010 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_6 49/172 2024-08-02T17:35:26.051 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/172 2024-08-02T17:35:26.205 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 51/172 2024-08-02T17:35:26.280 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.2.2-2.el9.x86_64 52/172 2024-08-02T17:35:26.335 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/172 2024-08-02T17:35:26.407 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/172 2024-08-02T17:35:26.439 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 55/172 2024-08-02T17:35:26.475 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 56/172 2024-08-02T17:35:26.476 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-02T17:35:26.506 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 108/172 2024-08-02T17:35:26.538 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 57/172 2024-08-02T17:35:26.575 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 58/172 2024-08-02T17:35:26.587 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 108/172 2024-08-02T17:35:26.616 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 109/172 2024-08-02T17:35:26.618 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 59/172 2024-08-02T17:35:26.660 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/172 2024-08-02T17:35:26.797 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 61/172 2024-08-02T17:35:26.837 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 62/172 2024-08-02T17:35:26.878 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 63/172 2024-08-02T17:35:26.977 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 64/172 2024-08-02T17:35:27.011 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.32.0-1.el9.noarch 65/172 2024-08-02T17:35:27.052 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 66/172 2024-08-02T17:35:27.086 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 67/172 2024-08-02T17:35:27.110 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 68/172 2024-08-02T17:35:27.184 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 69/172 2024-08-02T17:35:27.323 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 70/172 2024-08-02T17:35:27.359 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 71/172 2024-08-02T17:35:27.387 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 72/172 2024-08-02T17:35:27.422 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 73/172 2024-08-02T17:35:27.461 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 74/172 2024-08-02T17:35:27.492 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 75/172 2024-08-02T17:35:27.522 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 76/172 2024-08-02T17:35:27.556 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 77/172 2024-08-02T17:35:27.590 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 78/172 2024-08-02T17:35:27.631 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 79/172 2024-08-02T17:35:27.666 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 80/172 2024-08-02T17:35:27.754 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 81/172 2024-08-02T17:35:27.796 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 82/172 2024-08-02T17:35:27.828 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 83/172 2024-08-02T17:35:27.931 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 84/172 2024-08-02T17:35:27.987 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-02T17:35:28.706 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 86/172 2024-08-02T17:35:28.751 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 87/172 2024-08-02T17:35:28.784 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 88/172 2024-08-02T17:35:28.884 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 89/172 2024-08-02T17:35:28.901 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:28.938 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:28.940 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 90/172 2024-08-02T17:35:28.985 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 91/172 2024-08-02T17:35:29.073 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 92/172 2024-08-02T17:35:29.104 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 93/172 2024-08-02T17:35:29.282 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/172 2024-08-02T17:35:29.341 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/172 2024-08-02T17:35:29.423 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/172 2024-08-02T17:35:29.500 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:29.541 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 111/172 2024-08-02T17:35:29.611 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 111/172 2024-08-02T17:35:29.612 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-02T17:35:29.612 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-02T17:35:29.612 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:29.642 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 112/172 2024-08-02T17:35:30.073 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/172 2024-08-02T17:35:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/172 2024-08-02T17:35:31.403 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/172 2024-08-02T17:35:31.481 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/172 2024-08-02T17:35:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 101/172 2024-08-02T17:35:31.542 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 102/172 2024-08-02T17:35:31.850 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 103/172 2024-08-02T17:35:31.889 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 104/172 2024-08-02T17:35:31.983 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 104/172 2024-08-02T17:35:32.010 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 105/172 2024-08-02T17:35:32.046 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 106/172 2024-08-02T17:35:32.518 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 107/172 2024-08-02T17:35:32.550 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 108/172 2024-08-02T17:35:32.622 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 108/172 2024-08-02T17:35:32.652 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 109/172 2024-08-02T17:35:35.001 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:35.038 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:35.275 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 112/172 2024-08-02T17:35:35.275 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /sys 2024-08-02T17:35:35.275 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /proc 2024-08-02T17:35:35.275 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /mnt 2024-08-02T17:35:35.275 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /var/tmp 2024-08-02T17:35:35.276 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /home 2024-08-02T17:35:35.276 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /root 2024-08-02T17:35:35.276 INFO:teuthology.orchestra.run.smithi077.stdout:skipping the directory /tmp 2024-08-02T17:35:35.276 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:35.449 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 113/172 2024-08-02T17:35:35.465 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 113/172 2024-08-02T17:35:35.465 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:35.465 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-02T17:35:35.465 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:35.578 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 110/172 2024-08-02T17:35:35.616 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 111/172 2024-08-02T17:35:35.707 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 111/172 2024-08-02T17:35:35.707 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-02T17:35:35.707 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-02T17:35:35.707 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:35.734 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 112/172 2024-08-02T17:35:36.970 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 114/172 2024-08-02T17:35:36.996 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 114/172 2024-08-02T17:35:36.996 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:36.996 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-02T17:35:36.996 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-02T17:35:36.997 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-02T17:35:36.997 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:37.229 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 115/172 2024-08-02T17:35:37.252 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 115/172 2024-08-02T17:35:37.252 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:37.252 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-02T17:35:37.252 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-02T17:35:37.252 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-02T17:35:37.252 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:37.688 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 116/172 2024-08-02T17:35:37.711 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 116/172 2024-08-02T17:35:37.711 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:37.711 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-02T17:35:37.711 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-02T17:35:37.712 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-02T17:35:37.712 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:37.827 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-02T17:35:37.884 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-02T17:35:37.938 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-02T17:35:38.123 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-02T17:35:38.172 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-02T17:35:38.249 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-02T17:35:38.337 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-02T17:35:38.476 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-02T17:35:38.522 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-02T17:35:38.554 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-02T17:35:38.582 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-02T17:35:39.025 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 128/172 2024-08-02T17:35:39.055 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 128/172 2024-08-02T17:35:39.077 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.e 129/172 2024-08-02T17:35:39.140 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e. 130/172 2024-08-02T17:35:39.169 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 131/172 2024-08-02T17:35:40.207 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 131/172 2024-08-02T17:35:40.246 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 132/172 2024-08-02T17:35:41.203 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 132/172 2024-08-02T17:35:41.230 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 133/172 2024-08-02T17:35:41.273 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 112/172 2024-08-02T17:35:41.274 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /sys 2024-08-02T17:35:41.274 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /proc 2024-08-02T17:35:41.274 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /mnt 2024-08-02T17:35:41.274 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /var/tmp 2024-08-02T17:35:41.274 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /home 2024-08-02T17:35:41.274 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /root 2024-08-02T17:35:41.274 INFO:teuthology.orchestra.run.smithi069.stdout:skipping the directory /tmp 2024-08-02T17:35:41.274 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:41.330 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 133/172 2024-08-02T17:35:41.443 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 113/172 2024-08-02T17:35:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 113/172 2024-08-02T17:35:41.459 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:41.459 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-02T17:35:41.459 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:41.492 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el 134/172 2024-08-02T17:35:41.532 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 135/172 2024-08-02T17:35:41.555 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 135/172 2024-08-02T17:35:41.555 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:41.555 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-02T17:35:41.555 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-02T17:35:41.555 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-02T17:35:41.556 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:41.603 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 136/172 2024-08-02T17:35:41.615 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 136/172 2024-08-02T17:35:41.717 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 137/172 2024-08-02T17:35:41.763 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 138/172 2024-08-02T17:35:42.671 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-02T17:35:42.702 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 140/172 2024-08-02T17:35:42.725 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 140/172 2024-08-02T17:35:42.725 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:42.725 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-02T17:35:42.725 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-02T17:35:42.726 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-02T17:35:42.726 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:42.770 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 141/172 2024-08-02T17:35:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 141/172 2024-08-02T17:35:42.791 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:42.791 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-02T17:35:42.791 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:42.836 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 142/172 2024-08-02T17:35:42.858 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 142/172 2024-08-02T17:35:42.858 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:42.859 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-02T17:35:42.859 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-02T17:35:42.859 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-02T17:35:42.859 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:42.937 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 114/172 2024-08-02T17:35:42.964 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 114/172 2024-08-02T17:35:42.964 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:42.965 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-02T17:35:42.965 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-02T17:35:42.965 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-02T17:35:42.965 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:43.162 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 143/172 2024-08-02T17:35:43.185 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 143/172 2024-08-02T17:35:43.185 INFO:teuthology.orchestra.run.smithi077.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:43.185 INFO:teuthology.orchestra.run.smithi077.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-02T17:35:43.185 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-02T17:35:43.185 INFO:teuthology.orchestra.run.smithi077.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-02T17:35:43.186 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:43.188 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 115/172 2024-08-02T17:35:43.211 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 115/172 2024-08-02T17:35:43.211 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:43.211 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-02T17:35:43.212 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-02T17:35:43.212 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-02T17:35:43.212 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:43.655 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 116/172 2024-08-02T17:35:43.678 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 116/172 2024-08-02T17:35:43.678 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:43.679 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-02T17:35:43.679 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-02T17:35:43.679 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-02T17:35:43.679 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:43.785 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-02T17:35:43.833 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-02T17:35:43.891 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-02T17:35:44.074 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-02T17:35:44.122 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-02T17:35:44.198 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-02T17:35:44.288 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-02T17:35:44.444 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-02T17:35:44.480 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-02T17:35:44.521 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-02T17:35:44.557 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-02T17:35:45.002 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 128/172 2024-08-02T17:35:45.048 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 128/172 2024-08-02T17:35:45.079 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.e 129/172 2024-08-02T17:35:45.142 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e. 130/172 2024-08-02T17:35:45.171 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 131/172 2024-08-02T17:35:46.207 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 131/172 2024-08-02T17:35:46.248 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 132/172 2024-08-02T17:35:47.203 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 132/172 2024-08-02T17:35:47.231 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 133/172 2024-08-02T17:35:47.334 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 133/172 2024-08-02T17:35:47.501 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el 134/172 2024-08-02T17:35:47.534 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 135/172 2024-08-02T17:35:47.544 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 112/172 2024-08-02T17:35:47.544 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-08-02T17:35:47.544 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-08-02T17:35:47.544 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-08-02T17:35:47.545 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-08-02T17:35:47.545 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-08-02T17:35:47.545 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-08-02T17:35:47.545 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-08-02T17:35:47.545 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:47.558 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 135/172 2024-08-02T17:35:47.558 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:47.558 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-02T17:35:47.558 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-02T17:35:47.558 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-02T17:35:47.558 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:47.609 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 136/172 2024-08-02T17:35:47.621 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 136/172 2024-08-02T17:35:47.716 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 144/172 2024-08-02T17:35:47.717 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 113/172 2024-08-02T17:35:47.718 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 137/172 2024-08-02T17:35:47.733 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 113/172 2024-08-02T17:35:47.733 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:47.733 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-02T17:35:47.733 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:47.757 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 145/172 2024-08-02T17:35:47.778 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 138/172 2024-08-02T17:35:47.806 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 146/172 2024-08-02T17:35:47.838 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-02T17:35:47.865 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-02T17:35:48.048 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 149/172 2024-08-02T17:35:48.086 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-02T17:35:48.119 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_ 151/172 2024-08-02T17:35:48.159 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-02T17:35:48.189 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-02T17:35:48.235 INFO:teuthology.orchestra.run.smithi077.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-02T17:35:48.688 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-02T17:35:48.729 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 140/172 2024-08-02T17:35:48.752 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 140/172 2024-08-02T17:35:48.752 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:48.752 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-02T17:35:48.752 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-02T17:35:48.752 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-02T17:35:48.752 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:48.796 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 141/172 2024-08-02T17:35:48.804 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:35:48.818 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 141/172 2024-08-02T17:35:48.818 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:48.818 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-02T17:35:48.819 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:48.853 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:35:48.863 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 142/172 2024-08-02T17:35:48.873 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:35:48.886 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 142/172 2024-08-02T17:35:48.886 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:48.886 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-02T17:35:48.886 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-02T17:35:48.887 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-02T17:35:48.887 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:49.197 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 143/172 2024-08-02T17:35:49.220 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 114/172 2024-08-02T17:35:49.220 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 143/172 2024-08-02T17:35:49.220 INFO:teuthology.orchestra.run.smithi069.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:49.220 INFO:teuthology.orchestra.run.smithi069.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-02T17:35:49.221 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-02T17:35:49.221 INFO:teuthology.orchestra.run.smithi069.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-02T17:35:49.221 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:35:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 114/172 2024-08-02T17:35:49.246 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:49.246 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-02T17:35:49.246 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-02T17:35:49.246 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-02T17:35:49.246 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:49.471 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 115/172 2024-08-02T17:35:49.493 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 115/172 2024-08-02T17:35:49.494 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:49.494 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-02T17:35:49.494 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-02T17:35:49.494 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-02T17:35:49.494 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:49.691 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 156/172 2024-08-02T17:35:49.792 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-02T17:35:49.834 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-02T17:35:49.931 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 116/172 2024-08-02T17:35:49.932 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-02T17:35:49.953 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 116/172 2024-08-02T17:35:49.953 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:49.953 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-02T17:35:49.953 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-02T17:35:49.953 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-02T17:35:49.953 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:49.963 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-02T17:35:49.988 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-02T17:35:50.014 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-02T17:35:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-02T17:35:50.040 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-02T17:35:50.060 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 117/172 2024-08-02T17:35:50.116 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 118/172 2024-08-02T17:35:50.124 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-02T17:35:50.149 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-02T17:35:50.149 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-02T17:35:50.172 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 119/172 2024-08-02T17:35:50.234 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-02T17:35:50.259 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-02T17:35:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:35:50.264 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:35:50.357 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 120/172 2024-08-02T17:35:50.422 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 121/172 2024-08-02T17:35:50.452 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:35:50.452 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-02T17:35:50.500 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 122/172 2024-08-02T17:35:50.537 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-02T17:35:50.562 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-02T17:35:50.588 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-02T17:35:50.588 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 123/172 2024-08-02T17:35:50.653 INFO:teuthology.orchestra.run.smithi077.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:35:50.735 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 124/172 2024-08-02T17:35:50.772 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 125/172 2024-08-02T17:35:50.804 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 126/172 2024-08-02T17:35:50.832 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-16.el9.x86_64 127/172 2024-08-02T17:35:51.242 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 128/172 2024-08-02T17:35:51.272 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 128/172 2024-08-02T17:35:51.304 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.e 129/172 2024-08-02T17:35:51.375 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e. 130/172 2024-08-02T17:35:51.404 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 131/172 2024-08-02T17:35:52.441 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 131/172 2024-08-02T17:35:52.481 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 132/172 2024-08-02T17:35:53.441 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 132/172 2024-08-02T17:35:53.473 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 133/172 2024-08-02T17:35:53.575 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 133/172 2024-08-02T17:35:53.744 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el 134/172 2024-08-02T17:35:53.759 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 144/172 2024-08-02T17:35:53.792 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 135/172 2024-08-02T17:35:53.800 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 145/172 2024-08-02T17:35:53.818 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 135/172 2024-08-02T17:35:53.818 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:53.818 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-02T17:35:53.818 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-02T17:35:53.818 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-02T17:35:53.819 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:53.841 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 146/172 2024-08-02T17:35:53.859 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 136/172 2024-08-02T17:35:53.870 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 136/172 2024-08-02T17:35:53.873 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-02T17:35:53.900 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-02T17:35:53.968 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 137/172 2024-08-02T17:35:54.015 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 138/172 2024-08-02T17:35:54.101 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 149/172 2024-08-02T17:35:54.146 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-02T17:35:54.179 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_ 151/172 2024-08-02T17:35:54.219 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-02T17:35:54.257 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-02T17:35:54.304 INFO:teuthology.orchestra.run.smithi069.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-02T17:35:54.890 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:35:54.926 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:35:54.926 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-46.el9.x86_64 139/172 2024-08-02T17:35:54.946 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:35:54.962 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 140/172 2024-08-02T17:35:54.986 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 140/172 2024-08-02T17:35:54.986 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:54.986 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-02T17:35:54.986 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-02T17:35:54.987 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-02T17:35:54.987 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:55.038 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 141/172 2024-08-02T17:35:55.060 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 141/172 2024-08-02T17:35:55.060 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:55.060 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-02T17:35:55.060 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:55.105 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 142/172 2024-08-02T17:35:55.128 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 142/172 2024-08-02T17:35:55.128 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:55.128 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-02T17:35:55.128 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-02T17:35:55.128 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-02T17:35:55.129 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:55.439 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 143/172 2024-08-02T17:35:55.462 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 143/172 2024-08-02T17:35:55.462 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-02T17:35:55.462 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-02T17:35:55.462 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-02T17:35:55.462 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-02T17:35:55.462 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:35:55.765 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 156/172 2024-08-02T17:35:55.856 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-02T17:35:55.897 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-02T17:35:55.979 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-02T17:35:56.018 INFO:teuthology.orchestra.run.smithi069.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-02T17:35:56.043 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-02T17:35:56.068 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-02T17:35:56.094 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-02T17:35:56.094 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 1/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 3/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 4/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 5/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 6/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 7/172 2024-08-02T17:35:56.172 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 10/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 11/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 12/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 13/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_ 15/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 16/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 17/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_6 18/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 19/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 20/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el 21/172 2024-08-02T17:35:56.173 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9. 22/172 2024-08-02T17:35:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_6 23/172 2024-08-02T17:35:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 24/172 2024-08-02T17:35:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 25/172 2024-08-02T17:35:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 26/172 2024-08-02T17:35:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 27/172 2024-08-02T17:35:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 28/172 2024-08-02T17:35:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 29/172 2024-08-02T17:35:56.174 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e. 30/172 2024-08-02T17:35:56.188 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 31/172 2024-08-02T17:35:56.188 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 32/172 2024-08-02T17:35:56.188 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 33/172 2024-08-02T17:35:56.188 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el 34/172 2024-08-02T17:35:56.188 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 35/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.e 36/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 37/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 38/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 39/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-02T17:35:56.189 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-02T17:35:56.190 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-02T17:35:56.190 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-02T17:35:56.190 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-02T17:35:56.190 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-02T17:35:56.190 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-02T17:35:56.190 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-02T17:35:56.190 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-02T17:35:56.190 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-02T17:35:56.191 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-02T17:35:56.192 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-02T17:35:56.193 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-02T17:35:56.194 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-02T17:35:56.195 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-02T17:35:56.196 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 149/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 151/172 2024-08-02T17:35:56.197 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-02T17:35:56.198 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-02T17:35:56.199 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-02T17:35:56.199 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-02T17:35:56.199 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-02T17:35:56.199 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-02T17:35:56.199 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-02T17:35:56.199 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-02T17:35:56.229 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-02T17:35:56.229 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-02T17:35:56.313 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-02T17:35:56.339 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-02T17:35:56.344 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:35:56.344 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:35:56.540 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:35:56.540 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-02T17:35:56.607 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-02T17:35:56.640 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-02T17:35:56.675 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-02T17:35:56.739 INFO:teuthology.orchestra.run.smithi069.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:35:59.153 INFO:teuthology.orchestra.run.smithi077.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout:Upgraded: 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.154 INFO:teuthology.orchestra.run.smithi077.stdout: librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout:Installed: 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: automake-1.16.2-8.el9.noarch 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: bison-3.7.4-5.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.155 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-immutable-object-cache-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.156 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: flex-2.6.4-9.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-02T17:35:59.157 INFO:teuthology.orchestra.run.smithi077.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-02T17:35:59.158 INFO:teuthology.orchestra.run.smithi077.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: lua-5.4.4-4.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-02T17:35:59.159 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-13.14-1.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-02T17:35:59.160 INFO:teuthology.orchestra.run.smithi077.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-02T17:35:59.161 INFO:teuthology.orchestra.run.smithi077.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-02T17:35:59.162 INFO:teuthology.orchestra.run.smithi077.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-ply-3.11-14.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-02T17:35:59.163 INFO:teuthology.orchestra.run.smithi077.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.164 INFO:teuthology.orchestra.run.smithi077.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-02T17:35:59.165 INFO:teuthology.orchestra.run.smithi077.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: re2-1:20211101-3.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-02T17:35:59.166 INFO:teuthology.orchestra.run.smithi077.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-02T17:35:59.167 INFO:teuthology.orchestra.run.smithi077.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-02T17:35:59.170 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:35:59.170 INFO:teuthology.orchestra.run.smithi077.stdout:Complete! 2024-08-02T17:35:59.569 DEBUG:teuthology.parallel:result is None 2024-08-02T17:35:59.976 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 144/172 2024-08-02T17:36:00.025 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 145/172 2024-08-02T17:36:00.066 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 146/172 2024-08-02T17:36:00.098 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.4.0-3.el9.x86_64 147/172 2024-08-02T17:36:00.126 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 148/172 2024-08-02T17:36:00.307 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 149/172 2024-08-02T17:36:00.347 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 150/172 2024-08-02T17:36:00.379 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_ 151/172 2024-08-02T17:36:00.428 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 152/172 2024-08-02T17:36:00.458 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-team-1:1.48.2-2.el9.x86_64 153/172 2024-08-02T17:36:00.504 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : NetworkManager-tui-1:1.48.2-2.el9.x86_64 154/172 2024-08-02T17:36:01.146 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:36:01.191 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:36:01.210 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 155/172 2024-08-02T17:36:02.041 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 156/172 2024-08-02T17:36:02.152 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 157/172 2024-08-02T17:36:02.202 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 158/172 2024-08-02T17:36:02.292 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 159/172 2024-08-02T17:36:02.330 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 160/172 2024-08-02T17:36:02.356 INFO:teuthology.orchestra.run.smithi069.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:36:02.356 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 1/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 3/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 4/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 5/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 6/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 7/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 10/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 11/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 12/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 13/172 2024-08-02T17:36:02.357 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_ 15/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 16/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 17/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_6 18/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 19/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 20/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el 21/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9. 22/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_6 23/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 24/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 25/172 2024-08-02T17:36:02.358 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 26/172 2024-08-02T17:36:02.359 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 161/172 2024-08-02T17:36:02.372 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 27/172 2024-08-02T17:36:02.372 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 28/172 2024-08-02T17:36:02.372 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 29/172 2024-08-02T17:36:02.372 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e. 30/172 2024-08-02T17:36:02.372 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 31/172 2024-08-02T17:36:02.372 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 32/172 2024-08-02T17:36:02.372 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 33/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el 34/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 35/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.e 36/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 37/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 38/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 39/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-02T17:36:02.373 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-02T17:36:02.374 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-02T17:36:02.375 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-02T17:36:02.376 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-02T17:36:02.377 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-02T17:36:02.378 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-02T17:36:02.379 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-02T17:36:02.380 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-02T17:36:02.381 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-02T17:36:02.381 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-02T17:36:02.381 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-02T17:36:02.381 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-02T17:36:02.381 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-02T17:36:02.382 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-02T17:36:02.382 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-02T17:36:02.382 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-02T17:36:02.382 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-02T17:36:02.382 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-02T17:36:02.382 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-02T17:36:02.383 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-02T17:36:02.383 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-02T17:36:02.383 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-02T17:36:02.383 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-02T17:36:02.383 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 149/172 2024-08-02T17:36:02.383 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-02T17:36:02.383 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 151/172 2024-08-02T17:36:02.383 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-02T17:36:02.384 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-02T17:36:02.385 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-02T17:36:02.385 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-02T17:36:02.385 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-02T17:36:02.385 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-02T17:36:02.385 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-02T17:36:02.385 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-02T17:36:02.407 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 163/172 2024-08-02T17:36:02.407 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-02T17:36:02.476 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 164/172 2024-08-02T17:36:02.509 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 165/172 2024-08-02T17:36:02.509 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-02T17:36:02.593 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 166/172 2024-08-02T17:36:02.628 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 167/172 2024-08-02T17:36:02.632 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:36:02.632 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:36:02.820 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 168/172 2024-08-02T17:36:02.821 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-02T17:36:02.905 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 169/172 2024-08-02T17:36:02.929 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-02T17:36:02.956 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 171/172 2024-08-02T17:36:03.027 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:36:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:36:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:36:05.258 INFO:teuthology.orchestra.run.smithi069.stdout:Upgraded: 2024-08-02T17:36:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-02T17:36:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-02T17:36:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-02T17:36:05.258 INFO:teuthology.orchestra.run.smithi069.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout:Installed: 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: automake-1.16.2-8.el9.noarch 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: bison-3.7.4-5.el9.x86_64 2024-08-02T17:36:05.259 INFO:teuthology.orchestra.run.smithi069.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-immutable-object-cache-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.260 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: flex-2.6.4-9.el9.x86_64 2024-08-02T17:36:05.261 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-02T17:36:05.262 INFO:teuthology.orchestra.run.smithi069.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: lua-5.4.4-4.el9.x86_64 2024-08-02T17:36:05.263 INFO:teuthology.orchestra.run.smithi069.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-13.14-1.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-02T17:36:05.264 INFO:teuthology.orchestra.run.smithi069.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-02T17:36:05.265 INFO:teuthology.orchestra.run.smithi069.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-02T17:36:05.266 INFO:teuthology.orchestra.run.smithi069.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-ply-3.11-14.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-02T17:36:05.267 INFO:teuthology.orchestra.run.smithi069.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-02T17:36:05.268 INFO:teuthology.orchestra.run.smithi069.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-02T17:36:05.269 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: re2-1:20211101-3.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-02T17:36:05.270 INFO:teuthology.orchestra.run.smithi069.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-02T17:36:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-02T17:36:05.271 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:36:05.271 INFO:teuthology.orchestra.run.smithi069.stdout:Complete! 2024-08-02T17:36:05.692 DEBUG:teuthology.parallel:result is None 2024-08-02T17:36:08.797 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:36:08.797 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 1/172 2024-08-02T17:36:08.797 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2/172 2024-08-02T17:36:08.797 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 3/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 4/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.1.0-1260.g26c3f 5/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 6/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 7/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 8/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 9/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 10/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 11/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 12/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 13/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 14/172 2024-08-02T17:36:08.798 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_ 15/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 16/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 17/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_6 18/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86 19/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 20/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el 21/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9. 22/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_6 23/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 24/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 25/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 26/172 2024-08-02T17:36:08.799 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 27/172 2024-08-02T17:36:08.812 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 28/172 2024-08-02T17:36:08.812 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 29/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e. 30/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noa 31/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.n 32/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c 33/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el 34/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 35/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.e 36/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 37/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 38/172 2024-08-02T17:36:08.813 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 39/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/172 2024-08-02T17:36:08.814 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/172 2024-08-02T17:36:08.815 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/172 2024-08-02T17:36:08.816 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/172 2024-08-02T17:36:08.817 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/172 2024-08-02T17:36:08.818 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.4.0-3.el9.x86_64 101/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/172 2024-08-02T17:36:08.819 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.32.0-1.el9.noarch 122/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/172 2024-08-02T17:36:08.820 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/172 2024-08-02T17:36:08.821 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 149/172 2024-08-02T17:36:08.822 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 151/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.48.2-2.el9.x86_64 153/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.48.2-2.el9.x86_64 155/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.48.2-2.el9.x86_64 157/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.48.2-2.el9.x86_64 159/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/172 2024-08-02T17:36:08.823 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/172 2024-08-02T17:36:08.824 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/172 2024-08-02T17:36:08.824 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/172 2024-08-02T17:36:08.824 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/172 2024-08-02T17:36:08.824 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.2-2.el9.x86_64 167/172 2024-08-02T17:36:08.824 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/172 2024-08-02T17:36:08.824 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.2-2.el9.x86_64 169/172 2024-08-02T17:36:08.824 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/172 2024-08-02T17:36:08.824 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-6.4.0-3.el9.x86_64 171/172 2024-08-02T17:36:11.427 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 172/172 2024-08-02T17:36:11.427 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:36:11.427 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-08-02T17:36:11.427 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-1:1.48.2-2.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-libnm-1:1.48.2-2.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-team-1:1.48.2-2.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: NetworkManager-tui-1:1.48.2-2.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.2.2-2.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.2.2-2.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-6.4.0-3.el9.x86_64 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-08-02T17:36:11.428 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.429 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.430 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.433 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.433 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.433 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.1.0-1260.g26c3fb8e.el9.noarch 2024-08-02T17:36:11.433 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-02T17:36:11.433 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-08-02T17:36:11.433 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.434 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-02T17:36:11.435 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-08-02T17:36:11.436 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-02T17:36:11.437 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.32.0-1.el9.noarch 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-02T17:36:11.438 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-02T17:36:11.439 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-02T17:36:11.440 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-02T17:36:11.441 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.1.0-1260.g26c3fb8e.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-02T17:36:11.442 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-02T17:36:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.4.0-3.el9.x86_64 2024-08-02T17:36:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-02T17:36:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-02T17:36:11.443 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:36:11.443 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-08-02T17:36:11.868 DEBUG:teuthology.parallel:result is None 2024-08-02T17:36:11.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:36:12.012 DEBUG:teuthology.orchestra.run.smithi069:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-02T17:36:12.038 INFO:teuthology.orchestra.run.smithi069.stdout:19.1.0-1260.g26c3fb8e.el9 2024-08-02T17:36:12.039 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1260.g26c3fb8e.el9 2024-08-02T17:36:12.039 INFO:teuthology.task.install:The correct ceph version 19.1.0-1260.g26c3fb8e is installed. 2024-08-02T17:36:12.041 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:36:12.184 DEBUG:teuthology.orchestra.run.smithi077:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-02T17:36:12.209 INFO:teuthology.orchestra.run.smithi077.stdout:19.1.0-1260.g26c3fb8e.el9 2024-08-02T17:36:12.210 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1260.g26c3fb8e.el9 2024-08-02T17:36:12.210 INFO:teuthology.task.install:The correct ceph version 19.1.0-1260.g26c3fb8e is installed. 2024-08-02T17:36:12.211 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:36:12.354 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-02T17:36:12.377 INFO:teuthology.orchestra.run.smithi083.stdout:19.1.0-1260.g26c3fb8e.el9 2024-08-02T17:36:12.377 INFO:teuthology.packaging:The installed version of ceph is 19.1.0-1260.g26c3fb8e.el9 2024-08-02T17:36:12.378 INFO:teuthology.task.install:The correct ceph version 19.1.0-1260.g26c3fb8e is installed. 2024-08-02T17:36:12.379 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-02T17:36:12.380 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:36:12.380 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-02T17:36:12.413 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:36:12.413 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-02T17:36:12.444 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:36:12.444 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-02T17:36:12.471 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-02T17:36:12.471 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:36:12.471 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/daemon-helper 2024-08-02T17:36:12.500 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-02T17:36:12.568 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:36:12.569 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/daemon-helper 2024-08-02T17:36:12.593 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-02T17:36:12.656 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:36:12.656 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-08-02T17:36:12.682 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-02T17:36:12.746 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-02T17:36:12.747 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:36:12.747 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-02T17:36:12.776 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-02T17:36:12.845 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:36:12.845 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-02T17:36:12.869 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-02T17:36:12.933 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:36:12.933 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-02T17:36:12.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-02T17:36:13.024 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-02T17:36:13.024 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:36:13.024 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/usr/bin/stdin-killer 2024-08-02T17:36:13.054 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-02T17:36:13.122 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:36:13.122 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/usr/bin/stdin-killer 2024-08-02T17:36:13.145 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-02T17:36:13.210 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:36:13.210 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-08-02T17:36:13.237 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-02T17:36:13.302 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-02T17:36:13.398 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-08-02T17:36:13.398 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:36:13.399 INFO:tasks.cephadm:Cluster fsid is b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:36:13.399 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-02T17:36:13.399 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.69', 'mon.b': '172.21.15.77', 'mon.c': '172.21.15.83'} 2024-08-02T17:36:13.399 INFO:tasks.cephadm:First mon is mon.a on smithi069 2024-08-02T17:36:13.399 INFO:tasks.cephadm:First mgr is x 2024-08-02T17:36:13.399 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-02T17:36:13.400 DEBUG:teuthology.orchestra.run.smithi069:> sudo hostname $(hostname -s) 2024-08-02T17:36:13.430 DEBUG:teuthology.orchestra.run.smithi077:> sudo hostname $(hostname -s) 2024-08-02T17:36:13.453 DEBUG:teuthology.orchestra.run.smithi083:> sudo hostname $(hostname -s) 2024-08-02T17:36:13.479 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-02T17:36:13.479 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:36:13.625 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/81786/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.0-1260-g26c3fb8e', 'node_name': '172.21.5.40+adami10', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.0-1260.g26c3fb8e'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-01 22:01:07.411737', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid-release', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-08-02T17:36:13.770 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid-release, sha1 26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:36:13.772 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/x86_64/flavors/default/cephadm 2024-08-02T17:36:13.772 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/x86_64/flavors/default/cephadm 2024-08-02T17:36:13.773 DEBUG:teuthology.orchestra.run.smithi069:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:14.329 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-r--r--. 1 ubuntu ubuntu 784604 Aug 2 17:36 /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:14.329 DEBUG:teuthology.orchestra.run.smithi077:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:14.890 INFO:teuthology.orchestra.run.smithi077.stdout:-rw-r--r--. 1 ubuntu ubuntu 784604 Aug 2 17:36 /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:14.890 DEBUG:teuthology.orchestra.run.smithi083:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid-release/26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:15.436 INFO:teuthology.orchestra.run.smithi083.stdout:-rw-r--r--. 1 ubuntu ubuntu 784604 Aug 2 17:36 /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:15.437 DEBUG:teuthology.orchestra.run.smithi069:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:15.456 DEBUG:teuthology.orchestra.run.smithi077:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:15.474 DEBUG:teuthology.orchestra.run.smithi083:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-02T17:36:15.508 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea on all hosts... 2024-08-02T17:36:15.508 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea pull 2024-08-02T17:36:15.511 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea pull 2024-08-02T17:36:15.517 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea pull 2024-08-02T17:36:16.330 INFO:teuthology.orchestra.run.smithi077.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea... 2024-08-02T17:36:16.401 INFO:teuthology.orchestra.run.smithi083.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea... 2024-08-02T17:36:16.557 INFO:teuthology.orchestra.run.smithi069.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea... 2024-08-02T17:36:54.808 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:36:54.808 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.1.0-1260-g26c3fb8e (26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea) squid (rc)", 2024-08-02T17:36:54.808 INFO:teuthology.orchestra.run.smithi069.stdout: "image_id": "564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208", 2024-08-02T17:36:54.808 INFO:teuthology.orchestra.run.smithi069.stdout: "repo_digests": [ 2024-08-02T17:36:54.809 INFO:teuthology.orchestra.run.smithi069.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6" 2024-08-02T17:36:54.809 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T17:36:54.809 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:36:55.348 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-08-02T17:36:55.348 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_version": "ceph version 19.1.0-1260-g26c3fb8e (26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea) squid (rc)", 2024-08-02T17:36:55.349 INFO:teuthology.orchestra.run.smithi077.stdout: "image_id": "564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208", 2024-08-02T17:36:55.349 INFO:teuthology.orchestra.run.smithi077.stdout: "repo_digests": [ 2024-08-02T17:36:55.349 INFO:teuthology.orchestra.run.smithi077.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6" 2024-08-02T17:36:55.349 INFO:teuthology.orchestra.run.smithi077.stdout: ] 2024-08-02T17:36:55.349 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-08-02T17:36:55.778 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-02T17:36:55.779 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_version": "ceph version 19.1.0-1260-g26c3fb8e (26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea) squid (rc)", 2024-08-02T17:36:55.779 INFO:teuthology.orchestra.run.smithi083.stdout: "image_id": "564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208", 2024-08-02T17:36:55.779 INFO:teuthology.orchestra.run.smithi083.stdout: "repo_digests": [ 2024-08-02T17:36:55.779 INFO:teuthology.orchestra.run.smithi083.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6" 2024-08-02T17:36:55.779 INFO:teuthology.orchestra.run.smithi083.stdout: ] 2024-08-02T17:36:55.779 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-02T17:36:55.804 DEBUG:teuthology.orchestra.run.smithi069:> sudo mkdir -p /etc/ceph 2024-08-02T17:36:55.838 DEBUG:teuthology.orchestra.run.smithi077:> sudo mkdir -p /etc/ceph 2024-08-02T17:36:55.864 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph 2024-08-02T17:36:55.890 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 777 /etc/ceph 2024-08-02T17:36:55.919 DEBUG:teuthology.orchestra.run.smithi077:> sudo chmod 777 /etc/ceph 2024-08-02T17:36:55.943 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /etc/ceph 2024-08-02T17:36:55.966 INFO:tasks.cephadm:Writing seed config... 2024-08-02T17:36:55.967 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-08-02T17:36:55.967 INFO:tasks.cephadm: override: [client] debug client = 20 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-08-02T17:36:55.968 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-02T17:36:55.969 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-02T17:36:55.970 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-02T17:36:55.971 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-02T17:36:55.971 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-02T17:36:55.971 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-02T17:36:55.971 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-08-02T17:36:55.971 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-02T17:36:55.971 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-08-02T17:36:55.972 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:36:55.972 DEBUG:teuthology.orchestra.run.smithi069:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-02T17:36:55.990 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b6b04634-50f5-11ef-bcca-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-08-02T17:36:55.991 DEBUG:teuthology.orchestra.run.smithi069:mon.a> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a.service 2024-08-02T17:36:56.033 DEBUG:teuthology.orchestra.run.smithi069:mgr.x> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.x.service 2024-08-02T17:36:56.076 INFO:tasks.cephadm:Bootstrapping... 2024-08-02T17:36:56.076 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea -v bootstrap --fsid b6b04634-50f5-11ef-bcca-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.69 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-02T17:36:56.285 INFO:teuthology.orchestra.run.smithi069.stdout:-------------------------------------------------------------------------------- 2024-08-02T17:36:56.285 INFO:teuthology.orchestra.run.smithi069.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea', '-v', 'bootstrap', '--fsid', 'b6b04634-50f5-11ef-bcca-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.69', '--skip-admin-label'] 2024-08-02T17:36:56.285 INFO:teuthology.orchestra.run.smithi069.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-02T17:36:56.286 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying podman|docker is present... 2024-08-02T17:36:56.311 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 5.1.2 2024-08-02T17:36:56.311 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying lvm2 is present... 2024-08-02T17:36:56.311 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying time synchronization is in place... 2024-08-02T17:36:56.318 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-02T17:36:56.318 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-02T17:36:56.325 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-02T17:36:56.325 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-08-02T17:36:56.331 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-08-02T17:36:56.337 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-08-02T17:36:56.338 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-08-02T17:36:56.338 INFO:teuthology.orchestra.run.smithi069.stdout:Repeating the final host check... 2024-08-02T17:36:56.362 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 5.1.2 2024-08-02T17:36:56.362 INFO:teuthology.orchestra.run.smithi069.stdout:podman (/bin/podman) version 5.1.2 is present 2024-08-02T17:36:56.362 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl is present 2024-08-02T17:36:56.362 INFO:teuthology.orchestra.run.smithi069.stdout:lvcreate is present 2024-08-02T17:36:56.369 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-02T17:36:56.369 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-02T17:36:56.375 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-02T17:36:56.375 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-08-02T17:36:56.381 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout enabled 2024-08-02T17:36:56.388 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout active 2024-08-02T17:36:56.388 INFO:teuthology.orchestra.run.smithi069.stdout:Unit chronyd.service is enabled and running 2024-08-02T17:36:56.388 INFO:teuthology.orchestra.run.smithi069.stdout:Host looks OK 2024-08-02T17:36:56.388 INFO:teuthology.orchestra.run.smithi069.stdout:Cluster fsid: b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:36:56.388 INFO:teuthology.orchestra.run.smithi069.stdout:Acquiring lock 139762291021952 on /run/cephadm/b6b04634-50f5-11ef-bcca-c7b262605968.lock 2024-08-02T17:36:56.388 INFO:teuthology.orchestra.run.smithi069.stdout:Lock 139762291021952 acquired on /run/cephadm/b6b04634-50f5-11ef-bcca-c7b262605968.lock 2024-08-02T17:36:56.389 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 3300 ... 2024-08-02T17:36:56.389 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying IP 172.21.15.69 port 6789 ... 2024-08-02T17:36:56.389 INFO:teuthology.orchestra.run.smithi069.stdout:Base mon IP(s) is [172.21.15.69:3300, 172.21.15.69:6789], mon addrv is [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-08-02T17:36:56.392 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.69 metric 100 2024-08-02T17:36:56.392 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.69 metric 100 2024-08-02T17:36:56.394 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-02T17:36:56.395 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-08-02T17:36:56.395 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1759sec hoplimit 64 pref medium 2024-08-02T17:36:56.397 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-02T17:36:56.397 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-02T17:36:56.397 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-02T17:36:56.397 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-08-02T17:36:56.397 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-08-02T17:36:56.397 INFO:teuthology.orchestra.run.smithi069.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-02T17:36:56.398 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-08-02T17:36:56.398 INFO:teuthology.orchestra.run.smithi069.stdout:Mon IP `172.21.15.69` is in CIDR network `172.21.0.0/20` 2024-08-02T17:36:56.398 INFO:teuthology.orchestra.run.smithi069.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-02T17:36:56.398 INFO:teuthology.orchestra.run.smithi069.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-02T17:36:56.399 INFO:teuthology.orchestra.run.smithi069.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea... 2024-08-02T17:36:56.868 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stdout 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 2024-08-02T17:36:56.868 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea... 2024-08-02T17:36:56.869 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Getting image source signatures 2024-08-02T17:36:56.869 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:251e09273f2ec31de181954d061e48f87f3421518b30a5bbf1722961df606297 2024-08-02T17:36:56.869 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-02T17:36:56.869 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Copying config sha256:564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 2024-08-02T17:36:56.869 INFO:teuthology.orchestra.run.smithi069.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-02T17:36:57.731 INFO:teuthology.orchestra.run.smithi069.stdout:ceph: stdout ceph version 19.1.0-1260-g26c3fb8e (26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea) squid (rc) 2024-08-02T17:36:57.731 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph version: ceph version 19.1.0-1260-g26c3fb8e (26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea) squid (rc) 2024-08-02T17:36:57.731 INFO:teuthology.orchestra.run.smithi069.stdout:Extracting ceph user uid/gid from container image... 2024-08-02T17:36:58.639 INFO:teuthology.orchestra.run.smithi069.stdout:stat: stdout 167 167 2024-08-02T17:36:58.640 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial keys... 2024-08-02T17:36:59.617 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQA6Ga1mc8kwNxAAMKKQMZTb1NyPN2gSTUkrQw== 2024-08-02T17:37:00.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQA7Ga1mK96INRAAL2q9egZPmqZ4q9Vv1ethTQ== 2024-08-02T17:37:01.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph-authtool: stdout AQA8Ga1mgBFqMRAAJE7nTCuHV0PyfOPX5Go3QA== 2024-08-02T17:37:01.345 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial monmap... 2024-08-02T17:37:02.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-02T17:37:02.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-02T17:37:02.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:02.242 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-02T17:37:02.242 INFO:teuthology.orchestra.run.smithi069.stdout:monmaptool for a [v2:172.21.15.69:3300,v1:172.21.15.69:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-02T17:37:02.243 INFO:teuthology.orchestra.run.smithi069.stdout:setting min_mon_release = quincy 2024-08-02T17:37:02.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: set fsid to b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:02.243 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-02T17:37:02.243 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:37:02.243 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mon... 2024-08-02T17:37:03.181 INFO:teuthology.orchestra.run.smithi069.stdout:create mon.a on 2024-08-02T17:37:03.382 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-02T17:37:03.539 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-02T17:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b6b04634-50f5-11ef-bcca-c7b262605968.target → /etc/systemd/system/ceph-b6b04634-50f5-11ef-bcca-c7b262605968.target. 2024-08-02T17:37:03.716 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b6b04634-50f5-11ef-bcca-c7b262605968.target → /etc/systemd/system/ceph-b6b04634-50f5-11ef-bcca-c7b262605968.target. 2024-08-02T17:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a 2024-08-02T17:37:03.982 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a.service: Unit ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a.service not loaded. 2024-08-02T17:37:04.161 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b6b04634-50f5-11ef-bcca-c7b262605968.target.wants/ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b6b04634-50f5-11ef-bcca-c7b262605968@.service. 2024-08-02T17:37:04.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 systemd[1]: Starting Ceph mon.a for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:37:04.499 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 podman[32360]: 2024-08-02 17:37:04.397424598 +0000 UTC m=+0.104636774 container create ee75e1189cde9d83ef185272c922c57fdc2320dc470c513a12b0975920ec7b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-08-02T17:37:04.499 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 podman[32360]: 2024-08-02 17:37:04.310232215 +0000 UTC m=+0.017444417 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:37:04.658 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-02T17:37:04.659 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-08-02T17:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-02T17:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-08-02T17:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-08-02T17:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-08-02T17:37:04.663 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon to start... 2024-08-02T17:37:04.664 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mon... 2024-08-02T17:37:04.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 podman[32360]: 2024-08-02 17:37:04.574542369 +0000 UTC m=+0.281754562 container init ee75e1189cde9d83ef185272c922c57fdc2320dc470c513a12b0975920ec7b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-02T17:37:04.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 podman[32360]: 2024-08-02 17:37:04.577811501 +0000 UTC m=+0.285023677 container start ee75e1189cde9d83ef185272c922c57fdc2320dc470c513a12b0975920ec7b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-02T17:37:04.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 ceph-mon[32379]: mkfs b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:04.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 ceph-mon[32379]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-02T17:37:04.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 bash[32360]: ee75e1189cde9d83ef185272c922c57fdc2320dc470c513a12b0975920ec7b63 2024-08-02T17:37:04.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:04 smithi069 systemd[1]: Started Ceph mon.a for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:37:05.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-02T17:37:05.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: monmap epoch 1 2024-08-02T17:37:05.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:05.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: last_changed 2024-08-02T17:37:01.657332+0000 2024-08-02T17:37:05.707 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:37:05.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: min_mon_release 19 (squid) 2024-08-02T17:37:05.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: election_strategy: 1 2024-08-02T17:37:05.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:37:05.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: fsmap 2024-08-02T17:37:05.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: osdmap e1: 0 total, 0 up, 0 in 2024-08-02T17:37:05.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: mgrmap e1: no daemons active 2024-08-02T17:37:05.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:05 smithi069 ceph-mon[32379]: from='client.? 172.21.15.69:0/385795902' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-08-02T17:37:05.709 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout cluster: 2024-08-02T17:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout id: b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-02T17:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout services: 2024-08-02T17:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.535973s) 2024-08-02T17:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-02T17:37:05.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout data: 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout pgs: 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.086+0000 7fab9e8d2640 1 Processor -- start 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.086+0000 7fab9e8d2640 1 -- start start 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.087+0000 7fab9e8d2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab98105c80 0x7fab98106080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.087+0000 7fab9e8d2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab9807bd40 con 0x7fab98105c80 2024-08-02T17:37:05.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.087+0000 7fab97fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab98105c80 0x7fab98106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.087+0000 7fab97fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab98105c80 0x7fab98106080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40572/0 (socket says 172.21.15.69:40572) 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.087+0000 7fab97fff640 1 -- 172.21.15.69:0/3258956948 learned_addr learned my addr 172.21.15.69:0/3258956948 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.087+0000 7fab97fff640 1 -- 172.21.15.69:0/3258956948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab981065c0 con 0x7fab98105c80 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.089+0000 7fab97fff640 1 --2- 172.21.15.69:0/3258956948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab98105c80 0x7fab98106080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab88009820 tx=0x7fab8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b0e7a1b5ae57f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.089+0000 7fab977fe640 1 -- 172.21.15.69:0/3258956948 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab88037070 con 0x7fab98105c80 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.089+0000 7fab977fe640 1 -- 172.21.15.69:0/3258956948 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab88032440 con 0x7fab98105c80 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.089+0000 7fab977fe640 1 -- 172.21.15.69:0/3258956948 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab88032740 con 0x7fab98105c80 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.090+0000 7fab9e8d2640 1 -- 172.21.15.69:0/3258956948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab98105c80 msgr2=0x7fab98106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.090+0000 7fab9e8d2640 1 --2- 172.21.15.69:0/3258956948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab98105c80 0x7fab98106080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fab88009820 tx=0x7fab8802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:05.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.090+0000 7fab9e8d2640 1 -- 172.21.15.69:0/3258956948 shutdown_connections 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.090+0000 7fab9e8d2640 1 --2- 172.21.15.69:0/3258956948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab98105c80 0x7fab98106080 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.090+0000 7fab9e8d2640 1 -- 172.21.15.69:0/3258956948 >> 172.21.15.69:0/3258956948 conn(0x7fab98101a90 msgr2=0x7fab98103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.090+0000 7fab9e8d2640 1 -- 172.21.15.69:0/3258956948 shutdown_connections 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.090+0000 7fab9e8d2640 1 -- 172.21.15.69:0/3258956948 wait complete. 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.090+0000 7fab9e8d2640 1 Processor -- start 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab9e8d2640 1 -- start start 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab9e8d2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9819ce80 0x7fab9819d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab9e8d2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab98106b60 con 0x7fab9819ce80 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab97fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9819ce80 0x7fab9819d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab97fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9819ce80 0x7fab9819d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40584/0 (socket says 172.21.15.69:40584) 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab97fff640 1 -- 172.21.15.69:0/385795902 learned_addr learned my addr 172.21.15.69:0/385795902 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:05.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab97fff640 1 -- 172.21.15.69:0/385795902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9819d7e0 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab97fff640 1 --2- 172.21.15.69:0/385795902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9819ce80 0x7fab9819d2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab88035040 tx=0x7fab880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab95ffb640 1 -- 172.21.15.69:0/385795902 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab88041070 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab95ffb640 1 -- 172.21.15.69:0/385795902 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab8803c7a0 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9819da70 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.091+0000 7fab95ffb640 1 -- 172.21.15.69:0/385795902 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab88037040 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.092+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab981a0550 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.092+0000 7fab95ffb640 1 -- 172.21.15.69:0/385795902 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab88044070 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.092+0000 7fab95ffb640 1 -- 172.21.15.69:0/385795902 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab8803d730 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.092+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab98106080 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.095+0000 7fab95ffb640 1 -- 172.21.15.69:0/385795902 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fab8803cad0 con 0x7fab9819ce80 2024-08-02T17:37:05.719 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.152+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fab981a0cf0 con 0x7fab9819ce80 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.153+0000 7fab95ffb640 1 -- 172.21.15.69:0/385795902 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fab8803ccb0 con 0x7fab9819ce80 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.153+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9819ce80 msgr2=0x7fab9819d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.153+0000 7fab9e8d2640 1 --2- 172.21.15.69:0/385795902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9819ce80 0x7fab9819d2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab88035040 tx=0x7fab880046c0 comp rx=0 tx=0).stop 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.154+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 shutdown_connections 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.154+0000 7fab9e8d2640 1 --2- 172.21.15.69:0/385795902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab9819ce80 0x7fab9819d2a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.154+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 >> 172.21.15.69:0/385795902 conn(0x7fab98101a90 msgr2=0x7fab98193320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.154+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 shutdown_connections 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:05.154+0000 7fab9e8d2640 1 -- 172.21.15.69:0/385795902 wait complete. 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:mon is available 2024-08-02T17:37:05.720 INFO:teuthology.orchestra.run.smithi069.stdout:Assimilating anything we can from ceph.conf... 2024-08-02T17:37:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:06 smithi069 ceph-mon[32379]: from='client.? 172.21.15.69:0/2711063845' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-02T17:37:06.704 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:06 smithi069 ceph-mon[32379]: from='client.? 172.21.15.69:0/2711063845' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-02T17:37:06.707 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:06.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-08-02T17:37:06.713 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-02T17:37:06.714 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.093+0000 7ff18e052640 1 Processor -- start 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.093+0000 7ff18e052640 1 -- start start 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.094+0000 7ff18e052640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18810a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.094+0000 7ff18e052640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff18807bd90 con 0x7ff188107e00 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.094+0000 7ff1877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18810a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.094+0000 7ff1877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18810a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40598/0 (socket says 172.21.15.69:40598) 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.094+0000 7ff1877fe640 1 -- 172.21.15.69:0/2898365422 learned_addr learned my addr 172.21.15.69:0/2898365422 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.094+0000 7ff1877fe640 1 -- 172.21.15.69:0/2898365422 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18810a730 con 0x7ff188107e00 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff1877fe640 1 --2- 172.21.15.69:0/2898365422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18810a1f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff178009820 tx=0x7ff17802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7561fdcf93364d21 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:06.715 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff186ffd640 1 -- 172.21.15.69:0/2898365422 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff178037070 con 0x7ff188107e00 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff186ffd640 1 -- 172.21.15.69:0/2898365422 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff178032440 con 0x7ff188107e00 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff186ffd640 1 -- 172.21.15.69:0/2898365422 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff178032740 con 0x7ff188107e00 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff18e052640 1 -- 172.21.15.69:0/2898365422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 msgr2=0x7ff18810a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff18e052640 1 --2- 172.21.15.69:0/2898365422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18810a1f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff178009820 tx=0x7ff17802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff18e052640 1 -- 172.21.15.69:0/2898365422 shutdown_connections 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff18e052640 1 --2- 172.21.15.69:0/2898365422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18810a1f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff18e052640 1 -- 172.21.15.69:0/2898365422 >> 172.21.15.69:0/2898365422 conn(0x7ff188101a00 msgr2=0x7ff188103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.095+0000 7ff18e052640 1 -- 172.21.15.69:0/2898365422 shutdown_connections 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.096+0000 7ff18e052640 1 -- 172.21.15.69:0/2898365422 wait complete. 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.096+0000 7ff18e052640 1 Processor -- start 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.096+0000 7ff18e052640 1 -- start start 2024-08-02T17:37:06.716 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.096+0000 7ff18e052640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.096+0000 7ff18e052640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff18810abb0 con 0x7ff188107e00 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40614/0 (socket says 172.21.15.69:40614) 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1877fe640 1 -- 172.21.15.69:0/2711063845 learned_addr learned my addr 172.21.15.69:0/2711063845 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1877fe640 1 -- 172.21.15.69:0/2711063845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18819b020 con 0x7ff188107e00 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1877fe640 1 --2- 172.21.15.69:0/2711063845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18819aae0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff1780090b0 tx=0x7ff1780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1857fa640 1 -- 172.21.15.69:0/2711063845 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff178041070 con 0x7ff188107e00 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1857fa640 1 -- 172.21.15.69:0/2711063845 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17803c7a0 con 0x7ff188107e00 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1857fa640 1 -- 172.21.15.69:0/2711063845 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff178037040 con 0x7ff188107e00 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18819b2b0 con 0x7ff188107e00 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18819dfa0 con 0x7ff188107e00 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1857fa640 1 -- 172.21.15.69:0/2711063845 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff178044070 con 0x7ff188107e00 2024-08-02T17:37:06.717 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.097+0000 7ff1857fa640 1 -- 172.21.15.69:0/2711063845 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff17803d3f0 con 0x7ff188107e00 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.098+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff188108260 con 0x7ff188107e00 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.100+0000 7ff1857fa640 1 -- 172.21.15.69:0/2711063845 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7ff178046450 con 0x7ff188107e00 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.156+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ff18802d720 con 0x7ff188107e00 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.159+0000 7ff1857fa640 1 -- 172.21.15.69:0/2711063845 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7ff178046630 con 0x7ff188107e00 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.160+0000 7ff1857fa640 1 -- 172.21.15.69:0/2711063845 <== mon.0 v2:172.21.15.69:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff178056070 con 0x7ff188107e00 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.160+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 msgr2=0x7ff18819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.160+0000 7ff18e052640 1 --2- 172.21.15.69:0/2711063845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18819aae0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff1780090b0 tx=0x7ff1780046c0 comp rx=0 tx=0).stop 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.160+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 shutdown_connections 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.160+0000 7ff18e052640 1 --2- 172.21.15.69:0/2711063845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff188107e00 0x7ff18819aae0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.160+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 >> 172.21.15.69:0/2711063845 conn(0x7ff188101a00 msgr2=0x7ff188102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.160+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 shutdown_connections 2024-08-02T17:37:06.718 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:06.161+0000 7ff18e052640 1 -- 172.21.15.69:0/2711063845 wait complete. 2024-08-02T17:37:06.719 INFO:teuthology.orchestra.run.smithi069.stdout:Generating new minimal ceph.conf... 2024-08-02T17:37:07.675 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.142+0000 7fec53a6d640 1 Processor -- start 2024-08-02T17:37:07.675 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.142+0000 7fec53a6d640 1 -- start start 2024-08-02T17:37:07.675 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.142+0000 7fec53a6d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c107c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:07.675 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.142+0000 7fec53a6d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c108240 con 0x7fec4c107870 2024-08-02T17:37:07.675 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.143+0000 7fec51809640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c107c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.143+0000 7fec51809640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c107c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40616/0 (socket says 172.21.15.69:40616) 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.143+0000 7fec51809640 1 -- 172.21.15.69:0/950649808 learned_addr learned my addr 172.21.15.69:0/950649808 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.143+0000 7fec51809640 1 -- 172.21.15.69:0/950649808 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c106480 con 0x7fec4c107870 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.143+0000 7fec51809640 1 --2- 172.21.15.69:0/950649808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c107c70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec48009820 tx=0x7fec4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a6114ceadede2e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec50807640 1 -- 172.21.15.69:0/950649808 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec48037070 con 0x7fec4c107870 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec50807640 1 -- 172.21.15.69:0/950649808 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fec48032900 con 0x7fec4c107870 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec50807640 1 -- 172.21.15.69:0/950649808 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec48030850 con 0x7fec4c107870 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec53a6d640 1 -- 172.21.15.69:0/950649808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 msgr2=0x7fec4c107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec53a6d640 1 --2- 172.21.15.69:0/950649808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c107c70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec48009820 tx=0x7fec4802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec53a6d640 1 -- 172.21.15.69:0/950649808 shutdown_connections 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec53a6d640 1 --2- 172.21.15.69:0/950649808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c107c70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:07.676 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec53a6d640 1 -- 172.21.15.69:0/950649808 >> 172.21.15.69:0/950649808 conn(0x7fec4c101200 msgr2=0x7fec4c103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec53a6d640 1 -- 172.21.15.69:0/950649808 shutdown_connections 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.144+0000 7fec53a6d640 1 -- 172.21.15.69:0/950649808 wait complete. 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.145+0000 7fec53a6d640 1 Processor -- start 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.145+0000 7fec53a6d640 1 -- start start 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.145+0000 7fec53a6d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.145+0000 7fec53a6d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c10a840 con 0x7fec4c107870 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.145+0000 7fec51809640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.145+0000 7fec51809640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40618/0 (socket says 172.21.15.69:40618) 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.145+0000 7fec51809640 1 -- 172.21.15.69:0/2844750785 learned_addr learned my addr 172.21.15.69:0/2844750785 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.145+0000 7fec51809640 1 -- 172.21.15.69:0/2844750785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c19a7a0 con 0x7fec4c107870 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.146+0000 7fec51809640 1 --2- 172.21.15.69:0/2844750785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c19a260 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fec48009940 tx=0x7fec48032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.146+0000 7fec42ffd640 1 -- 172.21.15.69:0/2844750785 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec48042070 con 0x7fec4c107870 2024-08-02T17:37:07.677 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.146+0000 7fec42ffd640 1 -- 172.21.15.69:0/2844750785 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fec480308c0 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.146+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c19aa30 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.146+0000 7fec42ffd640 1 -- 172.21.15.69:0/2844750785 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fec48037040 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.146+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c19d720 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.146+0000 7fec42ffd640 1 -- 172.21.15.69:0/2844750785 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fec48045070 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.146+0000 7fec42ffd640 1 -- 172.21.15.69:0/2844750785 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fec48047b70 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.147+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4c10a9c0 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.149+0000 7fec42ffd640 1 -- 172.21.15.69:0/2844750785 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fec480327b0 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.205+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fec4c107cf0 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.205+0000 7fec42ffd640 1 -- 172.21.15.69:0/2844750785 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fec4c107cf0 con 0x7fec4c107870 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.206+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 msgr2=0x7fec4c19a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:07.678 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.206+0000 7fec53a6d640 1 --2- 172.21.15.69:0/2844750785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c19a260 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fec48009940 tx=0x7fec48032ce0 comp rx=0 tx=0).stop 2024-08-02T17:37:07.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.206+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 shutdown_connections 2024-08-02T17:37:07.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.206+0000 7fec53a6d640 1 --2- 172.21.15.69:0/2844750785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c107870 0x7fec4c19a260 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:07.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.206+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 >> 172.21.15.69:0/2844750785 conn(0x7fec4c101200 msgr2=0x7fec4c101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:07.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.206+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 shutdown_connections 2024-08-02T17:37:07.679 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:07.206+0000 7fec53a6d640 1 -- 172.21.15.69:0/2844750785 wait complete. 2024-08-02T17:37:07.679 INFO:teuthology.orchestra.run.smithi069.stdout:Restarting the monitor... 2024-08-02T17:37:07.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:07 smithi069 ceph-mon[32379]: from='client.? 172.21.15.69:0/2844750785' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:37:07.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:07 smithi069 systemd[1]: Stopping Ceph mon.a for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:37:08.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:07 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a[32375]: 2024-08-02T17:37:07.945+0000 7f4146999640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T17:37:08.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:07 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a[32375]: 2024-08-02T17:37:07.945+0000 7f4146999640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-02T17:37:08.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:08 smithi069 podman[32621]: 2024-08-02 17:37:08.048641391 +0000 UTC m=+0.217912247 container died ee75e1189cde9d83ef185272c922c57fdc2320dc470c513a12b0975920ec7b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-02T17:37:08.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:08 smithi069 podman[32621]: 2024-08-02 17:37:08.38732253 +0000 UTC m=+0.556593388 container cleanup ee75e1189cde9d83ef185272c922c57fdc2320dc470c513a12b0975920ec7b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-02T17:37:08.597 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:08 smithi069 bash[32621]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a 2024-08-02T17:37:08.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:08 smithi069 podman[32635]: 2024-08-02 17:37:08.596602086 +0000 UTC m=+0.543682387 container remove ee75e1189cde9d83ef185272c922c57fdc2320dc470c513a12b0975920ec7b63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-02T17:37:08.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:08 smithi069 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-02T17:37:08.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:08 smithi069 systemd[1]: Stopped Ceph mon.a for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:37:08.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:08 smithi069 systemd[1]: Starting Ceph mon.a for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:37:09.300 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 podman[32727]: 2024-08-02 17:37:09.014183095 +0000 UTC m=+0.105755074 container create fe3a259ab4b38fbcaa4e7cded27afcd0823751885d63f84dfd20b811e99e1f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-02T17:37:09.300 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 podman[32727]: 2024-08-02 17:37:08.925406534 +0000 UTC m=+0.016978514 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:37:09.300 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 podman[32727]: 2024-08-02 17:37:09.224538937 +0000 UTC m=+0.316110926 container init fe3a259ab4b38fbcaa4e7cded27afcd0823751885d63f84dfd20b811e99e1f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2) 2024-08-02T17:37:09.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 podman[32727]: 2024-08-02 17:37:09.227538398 +0000 UTC m=+0.319110376 container start fe3a259ab4b38fbcaa4e7cded27afcd0823751885d63f84dfd20b811e99e1f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-02T17:37:09.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: set uid:gid to 167:167 (ceph:ceph) 2024-08-02T17:37:09.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: ceph version 19.1.0-1260-g26c3fb8e (26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea) squid (rc), process ceph-mon, pid 2 2024-08-02T17:37:09.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: pidfile_write: ignore empty --pid-file 2024-08-02T17:37:09.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: load: jerasure load: lrc 2024-08-02T17:37:09.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: RocksDB version: 7.9.2 2024-08-02T17:37:09.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Git sha 0 2024-08-02T17:37:09.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Compile date 2024-08-01 20:56:42 2024-08-02T17:37:09.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: DB SUMMARY 2024-08-02T17:37:09.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: DB Session ID: TJXMNC5VXA78292RJG1S 2024-08-02T17:37:09.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: CURRENT file: CURRENT 2024-08-02T17:37:09.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: IDENTITY file: IDENTITY 2024-08-02T17:37:09.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105288 ; 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.error_if_exists: 0 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.create_if_missing: 0 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.paranoid_checks: 1 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.env: 0x56378a3777a0 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.fs: PosixFileSystem 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.info_log: 0x56378c0b7500 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_file_opening_threads: 16 2024-08-02T17:37:09.303 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.statistics: (nil) 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.use_fsync: 0 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_log_file_size: 0 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.keep_log_file_num: 1000 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.recycle_log_file_num: 0 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.allow_fallocate: 1 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.allow_mmap_reads: 0 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.allow_mmap_writes: 0 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.use_direct_reads: 0 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-02T17:37:09.304 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.create_missing_column_families: 0 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.db_log_dir: 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.wal_dir: 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.advise_random_on_open: 1 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.db_write_buffer_size: 0 2024-08-02T17:37:09.305 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.write_buffer_manager: 0x56378c0c6dc0 2024-08-02T17:37:09.306 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-02T17:37:09.306 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-02T17:37:09.306 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-02T17:37:09.306 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.rate_limiter: (nil) 2024-08-02T17:37:09.306 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-02T17:37:09.307 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.wal_recovery_mode: 2 2024-08-02T17:37:09.307 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.enable_thread_tracking: 0 2024-08-02T17:37:09.307 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.enable_pipelined_write: 0 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.unordered_write: 0 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.row_cache: None 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.wal_filter: None 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.allow_ingest_behind: 0 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.two_write_queues: 0 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.manual_wal_flush: 0 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.wal_compression: 0 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.atomic_flush: 0 2024-08-02T17:37:09.308 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.log_readahead_size: 0 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.best_efforts_recovery: 0 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.allow_data_in_errors: 0 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.db_host_id: __hostname__ 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_background_jobs: 2 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_background_compactions: -1 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_subcompactions: 1 2024-08-02T17:37:09.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_total_wal_size: 0 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_open_files: -1 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bytes_per_sync: 0 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-02T17:37:09.310 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_readahead_size: 0 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_background_flushes: -1 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Compression algorithms supported: 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: kZSTD supported: 0 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: kXpressCompression supported: 0 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: kBZip2Compression supported: 0 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: kLZ4Compression supported: 1 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: kZlibCompression supported: 1 2024-08-02T17:37:09.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: kLZ4HCCompression supported: 1 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: kSnappyCompression supported: 1 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.merge_operator: 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_filter: None 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_filter_factory: None 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.sst_partitioner_factory: None 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-02T17:37:09.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56378c0b7120) 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks: 1 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: pin_top_level_index_and_filter: 1 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_type: 0 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_index_type: 0 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_shortening: 1 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: checksum: 4 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: no_block_cache: 0 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache: 0x56378c0ab090 2024-08-02T17:37:09.313 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_name: BinnedLRUCache 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_options: 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: capacity : 536870912 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_shard_bits : 4 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: strict_capacity_limit : 0 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: high_pri_pool_ratio: 0.000 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_cache_compressed: (nil) 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: persistent_cache: (nil) 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size: 4096 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_size_deviation: 10 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_restart_interval: 16 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: index_block_restart_interval: 1 2024-08-02T17:37:09.314 INFO:journalctl@ceph.mon.a.smithi069.stdout: metadata_block_size: 4096 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: partition_filters: 0 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: use_delta_encoding: 1 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: filter_policy: bloomfilter 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: whole_key_filtering: 1 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: verify_compression: 0 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: read_amp_bytes_per_bit: 0 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: format_version: 5 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_index_compression: 1 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: block_align: 0 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: max_auto_readahead_size: 262144 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: prepopulate_block_cache: 0 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: initial_auto_readahead_size: 8192 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout: num_file_reads_for_auto_readahead: 2 2024-08-02T17:37:09.315 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.write_buffer_size: 33554432 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_write_buffer_number: 2 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression: NoCompression 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression: Disabled 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.prefix_extractor: nullptr 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.num_levels: 7 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-02T17:37:09.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.level: 32767 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.strategy: 0 2024-08-02T17:37:09.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.enabled: false 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.target_file_size_base: 67108864 2024-08-02T17:37:09.318 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-02T17:37:09.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.arena_block_size: 1048576 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.disable_auto_compactions: 0 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-02T17:37:09.320 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.inplace_update_support: 0 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.bloom_locality: 0 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.max_successive_merges: 0 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-02T17:37:09.321 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.paranoid_file_checks: 0 2024-08-02T17:37:09.322 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.force_consistency_checks: 1 2024-08-02T17:37:09.322 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.report_bg_io_stats: 0 2024-08-02T17:37:09.322 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.ttl: 2592000 2024-08-02T17:37:09.322 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-02T17:37:09.322 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-02T17:37:09.322 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-02T17:37:09.322 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.enable_blob_files: false 2024-08-02T17:37:09.323 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.min_blob_size: 0 2024-08-02T17:37:09.323 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.blob_file_size: 268435456 2024-08-02T17:37:09.323 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-02T17:37:09.323 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-02T17:37:09.323 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-02T17:37:09.323 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-02T17:37:09.323 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-02T17:37:09.324 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.blob_file_starting_level: 0 2024-08-02T17:37:09.324 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-02T17:37:09.324 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-02T17:37:09.324 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-02T17:37:09.324 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a4fdf257-be5a-4714-b0eb-338c3d8989d9 2024-08-02T17:37:09.324 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722620229253630, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-02T17:37:09.324 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-02T17:37:09.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722620229254653, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100344, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98238, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14300, "raw_average_key_size": 49, "raw_value_size": 90503, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722620229, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a4fdf257-be5a-4714-b0eb-338c3d8989d9", "db_session_id": "TJXMNC5VXA78292RJG1S", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-02T17:37:09.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722620229254748, "job": 1, "event": "recovery_finished"} 2024-08-02T17:37:09.327 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-02T17:37:09.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-02T17:37:09.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56378c0d8e00 2024-08-02T17:37:09.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: DB pointer 0x56378c1d2000 2024-08-02T17:37:09.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-02T17:37:09.329 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-02T17:37:09.329 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** DB Stats ** 2024-08-02T17:37:09.329 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-02T17:37:09.329 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: L0 2/0 99.85 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: Sum 2/0 99.85 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-02T17:37:09.330 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** Compaction Stats [default] ** 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-02T17:37:09.331 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: Cumulative compaction: 0.00 GB write, 30.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: Interval compaction: 0.00 GB write, 30.88 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache BinnedLRUCache@0x56378c0ab090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: starting mon.a rank 0 at public addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] at bind addrs [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a@-1(???) e1 preinit fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a@-1(???).mds e1 new map 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a@-1(???).mds e1 print_map 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: e1 2024-08-02T17:37:09.332 INFO:journalctl@ceph.mon.a.smithi069.stdout: btime 2024-08-02T17:37:04:618450+0000 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout: legacy client fscid: -1 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout: 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout: No filesystems configured 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 bash[32727]: fe3a259ab4b38fbcaa4e7cded27afcd0823751885d63f84dfd20b811e99e1f00 2024-08-02T17:37:09.333 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 systemd[1]: Started Ceph mon.a for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:37:09.334 INFO:teuthology.orchestra.run.smithi069.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-02T17:37:09.586 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-02T17:37:09.586 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: monmap epoch 1 2024-08-02T17:37:09.586 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:09.586 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: last_changed 2024-08-02T17:37:01.657332+0000 2024-08-02T17:37:09.586 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:37:09.586 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: min_mon_release 19 (squid) 2024-08-02T17:37:09.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: election_strategy: 1 2024-08-02T17:37:09.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:37:09.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: fsmap 2024-08-02T17:37:09.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: osdmap e1: 0 total, 0 up, 0 in 2024-08-02T17:37:09.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:09 smithi069 ceph-mon[32746]: mgrmap e1: no daemons active 2024-08-02T17:37:10.493 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.665+0000 7f58eb0f1640 1 Processor -- start 2024-08-02T17:37:10.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.665+0000 7f58eb0f1640 1 -- start start 2024-08-02T17:37:10.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.665+0000 7f58eb0f1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e410a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:10.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.665+0000 7f58eb0f1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f58e407bd80 con 0x7f58e4107e70 2024-08-02T17:37:10.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.665+0000 7f58e8e8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e410a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:10.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.665+0000 7f58e8e8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e410a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40630/0 (socket says 172.21.15.69:40630) 2024-08-02T17:37:10.497 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.665+0000 7f58e8e8d640 1 -- 172.21.15.69:0/1737425375 learned_addr learned my addr 172.21.15.69:0/1737425375 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.666+0000 7f58e8e8d640 1 -- 172.21.15.69:0/1737425375 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e410a7a0 con 0x7f58e4107e70 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.667+0000 7f58e8e8d640 1 --2- 172.21.15.69:0/1737425375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e410a260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009820 tx=0x7f58cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296d63d5f3b60ba7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.667+0000 7f58e37fe640 1 -- 172.21.15.69:0/1737425375 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58cc037070 con 0x7f58e4107e70 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58e37fe640 1 -- 172.21.15.69:0/1737425375 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58cc032900 con 0x7f58e4107e70 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58e37fe640 1 -- 172.21.15.69:0/1737425375 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58cc030850 con 0x7f58e4107e70 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58eb0f1640 1 -- 172.21.15.69:0/1737425375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 msgr2=0x7f58e410a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58eb0f1640 1 --2- 172.21.15.69:0/1737425375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e410a260 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009820 tx=0x7f58cc02a270 comp rx=0 tx=0).stop 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58eb0f1640 1 -- 172.21.15.69:0/1737425375 shutdown_connections 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58eb0f1640 1 --2- 172.21.15.69:0/1737425375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e410a260 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58eb0f1640 1 -- 172.21.15.69:0/1737425375 >> 172.21.15.69:0/1737425375 conn(0x7f58e4101a90 msgr2=0x7f58e4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58eb0f1640 1 -- 172.21.15.69:0/1737425375 shutdown_connections 2024-08-02T17:37:10.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.668+0000 7f58eb0f1640 1 -- 172.21.15.69:0/1737425375 wait complete. 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58eb0f1640 1 Processor -- start 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58eb0f1640 1 -- start start 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58eb0f1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58eb0f1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f58e410b780 con 0x7f58e4107e70 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58e8e8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58e8e8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40636/0 (socket says 172.21.15.69:40636) 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58e8e8d640 1 -- 172.21.15.69:0/4198422638 learned_addr learned my addr 172.21.15.69:0/4198422638 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58e8e8d640 1 -- 172.21.15.69:0/4198422638 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e419b880 con 0x7f58e4107e70 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.669+0000 7f58e8e8d640 1 --2- 172.21.15.69:0/4198422638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e419b340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f58cc032ce0 tx=0x7f58cc032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.670+0000 7f58e1ffb640 1 -- 172.21.15.69:0/4198422638 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58cc042070 con 0x7f58e4107e70 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.670+0000 7f58e1ffb640 1 -- 172.21.15.69:0/4198422638 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f58cc0308c0 con 0x7f58e4107e70 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.670+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e419bb10 con 0x7f58e4107e70 2024-08-02T17:37:10.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.670+0000 7f58e1ffb640 1 -- 172.21.15.69:0/4198422638 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58cc037040 con 0x7f58e4107e70 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.670+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e419e800 con 0x7f58e4107e70 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.670+0000 7f58e1ffb640 1 -- 172.21.15.69:0/4198422638 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f58cc045070 con 0x7f58e4107e70 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.670+0000 7f58e1ffb640 1 -- 172.21.15.69:0/4198422638 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f58cc047b70 con 0x7f58e4107e70 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.671+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e41082d0 con 0x7f58e4107e70 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.673+0000 7f58e1ffb640 1 -- 172.21.15.69:0/4198422638 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f58cc03c720 con 0x7f58e4107e70 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.728+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f58e419edf0 con 0x7f58e4107e70 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.733+0000 7f58e1ffb640 1 -- 172.21.15.69:0/4198422638 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f58cc03c900 con 0x7f58e4107e70 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.734+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 msgr2=0x7f58e419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.734+0000 7f58eb0f1640 1 --2- 172.21.15.69:0/4198422638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e419b340 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f58cc032ce0 tx=0x7f58cc032d10 comp rx=0 tx=0).stop 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.734+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 shutdown_connections 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.734+0000 7f58eb0f1640 1 --2- 172.21.15.69:0/4198422638 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f58e4107e70 0x7f58e419b340 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:10.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.734+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 >> 172.21.15.69:0/4198422638 conn(0x7f58e4101a90 msgr2=0x7f58e4102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:10.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.734+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 shutdown_connections 2024-08-02T17:37:10.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:09.734+0000 7f58eb0f1640 1 -- 172.21.15.69:0/4198422638 wait complete. 2024-08-02T17:37:10.501 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-02T17:37:10.522 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-02T17:37:10.523 INFO:teuthology.orchestra.run.smithi069.stdout:Creating mgr... 2024-08-02T17:37:10.523 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-02T17:37:10.524 INFO:teuthology.orchestra.run.smithi069.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-02T17:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.x 2024-08-02T17:37:10.781 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.x.service: Unit ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.x.service not loaded. 2024-08-02T17:37:10.933 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b6b04634-50f5-11ef-bcca-c7b262605968.target.wants/ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b6b04634-50f5-11ef-bcca-c7b262605968@.service. 2024-08-02T17:37:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:10 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4198422638' entity='client.admin' 2024-08-02T17:37:11.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:10 smithi069 systemd[1]: Starting Ceph mgr.x for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:37:11.461 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 podman[32961]: 2024-08-02 17:37:11.166077024 +0000 UTC m=+0.111969865 container create a2cd66d57a9b523e66414b87e6cdc3da67f3ad7c2cbf76b1c5d4d71679d7d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-02T17:37:11.461 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 podman[32961]: 2024-08-02 17:37:11.071311624 +0000 UTC m=+0.017204474 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea 2024-08-02T17:37:11.461 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 podman[32961]: 2024-08-02 17:37:11.376475138 +0000 UTC m=+0.322367990 container init a2cd66d57a9b523e66414b87e6cdc3da67f3ad7c2cbf76b1c5d4d71679d7d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-02T17:37:11.461 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 podman[32961]: 2024-08-02 17:37:11.379236693 +0000 UTC m=+0.325129538 container start a2cd66d57a9b523e66414b87e6cdc3da67f3ad7c2cbf76b1c5d4d71679d7d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-08-02T17:37:11.461 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 bash[32961]: a2cd66d57a9b523e66414b87e6cdc3da67f3ad7c2cbf76b1c5d4d71679d7d2c1 2024-08-02T17:37:11.474 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-02T17:37:11.474 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-08-02T17:37:11.481 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-02T17:37:11.481 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-08-02T17:37:11.482 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-08-02T17:37:11.482 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to enable service . firewalld.service is not available 2024-08-02T17:37:11.490 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-02T17:37:11.490 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-08-02T17:37:11.498 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-02T17:37:11.498 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-08-02T17:37:11.498 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-08-02T17:37:11.498 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-02T17:37:11.498 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr to start... 2024-08-02T17:37:11.498 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr... 2024-08-02T17:37:11.731 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 systemd[1]: Started Ceph mgr.x for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:37:11.732 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:11.504+0000 7f0430c04040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-02T17:37:11.732 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:11.659+0000 7f0430c04040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:37:11.732 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:11.730+0000 7f0430c04040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-02T17:37:11.988 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:11 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3481079822' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-02T17:37:12.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:11.986+0000 7f0430c04040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-02T17:37:12.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:12 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:12.047+0000 7f0430c04040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-02T17:37:12.472 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:12.472 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-08-02T17:37:12.473 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-02T17:37:12.474 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:12.475 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "btime": "2024-08-02T17:37:04:618450+0000", 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:12.476 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-08-02T17:37:04.619106+0000", 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.859+0000 7f28a29f3640 1 Processor -- start 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.859+0000 7f28a29f3640 1 -- start start 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.859+0000 7f28a29f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c105fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.859+0000 7f28a29f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f289c07bdb0 con 0x7f289c105bd0 2024-08-02T17:37:12.477 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.859+0000 7f289bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c105fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.859+0000 7f289bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c105fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40670/0 (socket says 172.21.15.69:40670) 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.859+0000 7f289bfff640 1 -- 172.21.15.69:0/4033039475 learned_addr learned my addr 172.21.15.69:0/4033039475 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.860+0000 7f289bfff640 1 -- 172.21.15.69:0/4033039475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c106510 con 0x7f289c105bd0 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.860+0000 7f289bfff640 1 --2- 172.21.15.69:0/4033039475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c105fd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f288c009820 tx=0x7f288c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc482a357a78f5ff server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.860+0000 7f289b7fe640 1 -- 172.21.15.69:0/4033039475 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f288c037070 con 0x7f289c105bd0 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f289b7fe640 1 -- 172.21.15.69:0/4033039475 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f288c032900 con 0x7f289c105bd0 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f289b7fe640 1 -- 172.21.15.69:0/4033039475 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f288c030850 con 0x7f289c105bd0 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f28a29f3640 1 -- 172.21.15.69:0/4033039475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 msgr2=0x7f289c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f28a29f3640 1 --2- 172.21.15.69:0/4033039475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c105fd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f288c009820 tx=0x7f288c02a270 comp rx=0 tx=0).stop 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f28a29f3640 1 -- 172.21.15.69:0/4033039475 shutdown_connections 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f28a29f3640 1 --2- 172.21.15.69:0/4033039475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c105fd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:12.478 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f28a29f3640 1 -- 172.21.15.69:0/4033039475 >> 172.21.15.69:0/4033039475 conn(0x7f289c101a00 msgr2=0x7f289c103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f28a29f3640 1 -- 172.21.15.69:0/4033039475 shutdown_connections 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.861+0000 7f28a29f3640 1 -- 172.21.15.69:0/4033039475 wait complete. 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.862+0000 7f28a29f3640 1 Processor -- start 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.862+0000 7f28a29f3640 1 -- start start 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.862+0000 7f28a29f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.862+0000 7f28a29f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f289c107610 con 0x7f289c105bd0 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.862+0000 7f289bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.862+0000 7f289bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40674/0 (socket says 172.21.15.69:40674) 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.862+0000 7f289bfff640 1 -- 172.21.15.69:0/3481079822 learned_addr learned my addr 172.21.15.69:0/3481079822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f289bfff640 1 -- 172.21.15.69:0/3481079822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c19da10 con 0x7f289c105bd0 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f289bfff640 1 --2- 172.21.15.69:0/3481079822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c19d4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f288c0097f0 tx=0x7f288c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f2899ffb640 1 -- 172.21.15.69:0/3481079822 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f288c037040 con 0x7f289c105bd0 2024-08-02T17:37:12.479 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f2899ffb640 1 -- 172.21.15.69:0/3481079822 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f288c040070 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f2899ffb640 1 -- 172.21.15.69:0/3481079822 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f288c03a390 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c19dca0 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c1a0990 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f2899ffb640 1 -- 172.21.15.69:0/3481079822 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f288c03a630 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.863+0000 7f2899ffb640 1 -- 172.21.15.69:0/3481079822 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f288c0459a0 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.864+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c105fd0 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.866+0000 7f2899ffb640 1 -- 172.21.15.69:0/3481079822 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7f288c045460 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.926+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f289c1a0b60 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.927+0000 7f2899ffb640 1 -- 172.21.15.69:0/3481079822 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f288c03b430 con 0x7f289c105bd0 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.928+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 msgr2=0x7f289c19d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.928+0000 7f28a29f3640 1 --2- 172.21.15.69:0/3481079822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c19d4d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f288c0097f0 tx=0x7f288c030d80 comp rx=0 tx=0).stop 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.928+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 shutdown_connections 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.928+0000 7f28a29f3640 1 --2- 172.21.15.69:0/3481079822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f289c105bd0 0x7f289c19d4d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:12.480 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.928+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 >> 172.21.15.69:0/3481079822 conn(0x7f289c101a00 msgr2=0x7f289c1917f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:12.481 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.928+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 shutdown_connections 2024-08-02T17:37:12.481 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:11.928+0000 7f28a29f3640 1 -- 172.21.15.69:0/3481079822 wait complete. 2024-08-02T17:37:12.481 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (1/15)... 2024-08-02T17:37:13.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:12 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:12.995+0000 7f0430c04040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-02T17:37:13.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:13.109+0000 7f0430c04040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-02T17:37:13.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:13.165+0000 7f0430c04040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-02T17:37:13.644 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:13.374+0000 7f0430c04040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-02T17:37:13.645 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:13.434+0000 7f0430c04040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-02T17:37:13.645 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:13.500+0000 7f0430c04040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-02T17:37:13.645 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:13.643+0000 7f0430c04040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:37:14.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:14.052+0000 7f0430c04040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-02T17:37:14.690 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:14.369+0000 7f0430c04040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-02T17:37:14.691 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:14.434+0000 7f0430c04040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-02T17:37:14.691 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:14.588+0000 7f0430c04040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-02T17:37:14.965 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:14.911+0000 7f0430c04040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-02T17:37:15.317 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:15 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2410305871' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-02T17:37:15.317 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:14.977+0000 7f0430c04040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-02T17:37:15.317 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:15.165+0000 7f0430c04040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-02T17:37:15.318 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:15.316+0000 7f0430c04040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-02T17:37:15.518 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-08-02T17:37:15.519 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-02T17:37:15.520 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-02T17:37:15.521 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "btime": "2024-08-02T17:37:04:618450+0000", 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-02T17:37:15.522 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-08-02T17:37:04.619106+0000", 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.897+0000 7fd82e106640 1 Processor -- start 2024-08-02T17:37:15.523 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.897+0000 7fd82e106640 1 -- start start 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.897+0000 7fd82e106640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.897+0000 7fd82e106640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd828106bc0 con 0x7fd8281061f0 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.897+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.897+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57818/0 (socket says 172.21.15.69:57818) 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.897+0000 7fd8277fe640 1 -- 172.21.15.69:0/3413298368 learned_addr learned my addr 172.21.15.69:0/3413298368 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.897+0000 7fd8277fe640 1 -- 172.21.15.69:0/3413298368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8281073f0 con 0x7fd8281061f0 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.898+0000 7fd8277fe640 1 --2- 172.21.15.69:0/3413298368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281065f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd81800bac0 tx=0x7fd81802c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233f500f9645ba6f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.898+0000 7fd826ffd640 1 -- 172.21.15.69:0/3413298368 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8180326d0 con 0x7fd8281061f0 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.898+0000 7fd826ffd640 1 -- 172.21.15.69:0/3413298368 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd818038070 con 0x7fd8281061f0 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.898+0000 7fd826ffd640 1 -- 172.21.15.69:0/3413298368 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd818033d70 con 0x7fd8281061f0 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.899+0000 7fd82e106640 1 -- 172.21.15.69:0/3413298368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 msgr2=0x7fd8281065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.899+0000 7fd82e106640 1 --2- 172.21.15.69:0/3413298368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281065f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd81800bac0 tx=0x7fd81802c920 comp rx=0 tx=0).stop 2024-08-02T17:37:15.524 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.899+0000 7fd82e106640 1 -- 172.21.15.69:0/3413298368 shutdown_connections 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.899+0000 7fd82e106640 1 --2- 172.21.15.69:0/3413298368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281065f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.899+0000 7fd82e106640 1 -- 172.21.15.69:0/3413298368 >> 172.21.15.69:0/3413298368 conn(0x7fd828101a00 msgr2=0x7fd828103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.899+0000 7fd82e106640 1 -- 172.21.15.69:0/3413298368 shutdown_connections 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.899+0000 7fd82e106640 1 -- 172.21.15.69:0/3413298368 wait complete. 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd82e106640 1 Processor -- start 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd82e106640 1 -- start start 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd82e106640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd82e106640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd82807b0d0 con 0x7fd8281061f0 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57828/0 (socket says 172.21.15.69:57828) 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd8277fe640 1 -- 172.21.15.69:0/2410305871 learned_addr learned my addr 172.21.15.69:0/2410305871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:15.525 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd8277fe640 1 -- 172.21.15.69:0/2410305871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8281995f0 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.900+0000 7fd8277fe640 1 --2- 172.21.15.69:0/2410305871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281990b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd81800b2c0 tx=0x7fd818032910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.901+0000 7fd8257fa640 1 -- 172.21.15.69:0/2410305871 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd818042070 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.901+0000 7fd8257fa640 1 -- 172.21.15.69:0/2410305871 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd81803e900 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.901+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd828199880 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.901+0000 7fd8257fa640 1 -- 172.21.15.69:0/2410305871 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd81803c950 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.901+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82819c570 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.902+0000 7fd8257fa640 1 -- 172.21.15.69:0/2410305871 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd818045070 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.901+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8281065f0 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.902+0000 7fd8257fa640 1 -- 172.21.15.69:0/2410305871 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd818030050 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.904+0000 7fd8257fa640 1 -- 172.21.15.69:0/2410305871 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73482 (secure 0 0 0) 0x7fd81803eab0 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.964+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd82819c740 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.964+0000 7fd8257fa640 1 -- 172.21.15.69:0/2410305871 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fd81803ec90 con 0x7fd8281061f0 2024-08-02T17:37:15.526 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.965+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 msgr2=0x7fd8281990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:15.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.965+0000 7fd82e106640 1 --2- 172.21.15.69:0/2410305871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281990b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd81800b2c0 tx=0x7fd818032910 comp rx=0 tx=0).stop 2024-08-02T17:37:15.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.965+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 shutdown_connections 2024-08-02T17:37:15.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.965+0000 7fd82e106640 1 --2- 172.21.15.69:0/2410305871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd8281061f0 0x7fd8281990b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:15.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.965+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 >> 172.21.15.69:0/2410305871 conn(0x7fd828101a00 msgr2=0x7fd82818f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:15.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.965+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 shutdown_connections 2024-08-02T17:37:15.527 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:14.965+0000 7fd82e106640 1 -- 172.21.15.69:0/2410305871 wait complete. 2024-08-02T17:37:15.527 INFO:teuthology.orchestra.run.smithi069.stdout:mgr not available, waiting (2/15)... 2024-08-02T17:37:15.578 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:15.380+0000 7f0430c04040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-02T17:37:15.578 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:15.463+0000 7f0430c04040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-02T17:37:15.578 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:15.522+0000 7f0430c04040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-02T17:37:16.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:16.048+0000 7f0430c04040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-02T17:37:16.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:16.203+0000 7f0430c04040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-02T17:37:16.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: Activating manager daemon x 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: mgrmap e2: x(active, starting, since 0.00324226s) 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: Manager daemon x is now available 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' 2024-08-02T17:37:16.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:16 smithi069 ceph-mon[32746]: from='mgr.14100 172.21.15.69:0/450283186' entity='mgr.x' 2024-08-02T17:37:18.575 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:18 smithi069 ceph-mon[32746]: mgrmap e3: x(active, since 1.00713s) 2024-08-02T17:37:19.464 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:19 smithi069 ceph-mon[32746]: mgrmap e4: x(active, since 2s) 2024-08-02T17:37:21.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:20 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/32235569' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-08-02T17:37:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-08-02T17:37:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:37:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "health": { 2024-08-02T17:37:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-02T17:37:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-02T17:37:27.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 0 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "a" 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-02T17:37:27.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-02T17:37:27.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:27.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-02T17:37:27.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:27.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-02T17:37:27.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-02T17:37:27.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-02T17:37:27.228 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "btime": "2024-08-02T17:37:04:618450+0000", 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-02T17:37:27.229 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "restful" 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ], 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "modified": "2024-08-02T17:37:04.619106+0000", 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout }, 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.527+0000 7f1da60ef640 1 Processor -- start 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.527+0000 7f1da60ef640 1 -- start start 2024-08-02T17:37:27.230 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.527+0000 7f1da60ef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da0106020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.527+0000 7f1da60ef640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1da007bd80 con 0x7f1da0105c20 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.527+0000 7f1d9f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da0106020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.528+0000 7f1d9f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da0106020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57902/0 (socket says 172.21.15.69:57902) 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.528+0000 7f1d9f7fe640 1 -- 172.21.15.69:0/578848576 learned_addr learned my addr 172.21.15.69:0/578848576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.528+0000 7f1d9f7fe640 1 -- 172.21.15.69:0/578848576 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da0106560 con 0x7f1da0105c20 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.529+0000 7f1d9f7fe640 1 --2- 172.21.15.69:0/578848576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da0106020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009820 tx=0x7f1d9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafbc7e66415891e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.529+0000 7f1d9e7fc640 1 -- 172.21.15.69:0/578848576 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d90037070 con 0x7f1da0105c20 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.529+0000 7f1d9e7fc640 1 -- 172.21.15.69:0/578848576 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d90032900 con 0x7f1da0105c20 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.529+0000 7f1d9e7fc640 1 -- 172.21.15.69:0/578848576 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d90030850 con 0x7f1da0105c20 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 -- 172.21.15.69:0/578848576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 msgr2=0x7f1da0106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 --2- 172.21.15.69:0/578848576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da0106020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009820 tx=0x7f1d9002a270 comp rx=0 tx=0).stop 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 -- 172.21.15.69:0/578848576 shutdown_connections 2024-08-02T17:37:27.231 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 --2- 172.21.15.69:0/578848576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da0106020 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 -- 172.21.15.69:0/578848576 >> 172.21.15.69:0/578848576 conn(0x7f1da0101a50 msgr2=0x7f1da0103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 -- 172.21.15.69:0/578848576 shutdown_connections 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 -- 172.21.15.69:0/578848576 wait complete. 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 Processor -- start 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.530+0000 7f1da60ef640 1 -- start start 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1da60ef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1da60ef640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1da0107660 con 0x7f1da0105c20 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1d9f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1d9f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57914/0 (socket says 172.21.15.69:57914) 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1d9f7fe640 1 -- 172.21.15.69:0/32235569 learned_addr learned my addr 172.21.15.69:0/32235569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1d9f7fe640 1 -- 172.21.15.69:0/32235569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da019da10 con 0x7f1da0105c20 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1d9f7fe640 1 --2- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da019d4d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1d90032ce0 tx=0x7f1d90032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:27.232 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1d9cff9640 1 -- 172.21.15.69:0/32235569 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d90042070 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1d9cff9640 1 -- 172.21.15.69:0/32235569 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1d900308c0 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da019dca0 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1d9cff9640 1 -- 172.21.15.69:0/32235569 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d90037040 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.531+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da01a0990 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.532+0000 7f1d9cff9640 1 -- 172.21.15.69:0/32235569 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1d90045070 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.532+0000 7f1d9cff9640 1 --2- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f1d88038f70 0x7f1d8803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.532+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da0106020 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.533+0000 7f1d9cff9640 1 -- 172.21.15.69:0/32235569 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1d9006c030 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.533+0000 7f1d9effd640 1 --2- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f1d88038f70 0x7f1d8803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.533+0000 7f1d9effd640 1 --2- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f1d88038f70 0x7f1d8803b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d940098c0 tx=0x7f1d94006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.538+0000 7f1d9cff9640 1 -- 172.21.15.69:0/32235569 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1d90045360 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.773+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1da002d830 con 0x7f1da0105c20 2024-08-02T17:37:27.233 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.773+0000 7f1d9cff9640 1 -- 172.21.15.69:0/32235569 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f1d9003c740 con 0x7f1da0105c20 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.775+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f1d88038f70 msgr2=0x7f1d8803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.775+0000 7f1da60ef640 1 --2- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f1d88038f70 0x7f1d8803b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1d940098c0 tx=0x7f1d94006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.775+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 msgr2=0x7f1da019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.775+0000 7f1da60ef640 1 --2- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da019d4d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1d90032ce0 tx=0x7f1d90032d10 comp rx=0 tx=0).stop 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.776+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 shutdown_connections 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.776+0000 7f1da60ef640 1 --2- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f1d88038f70 0x7f1d8803b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.776+0000 7f1da60ef640 1 --2- 172.21.15.69:0/32235569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1da0105c20 0x7f1da019d4d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.776+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 >> 172.21.15.69:0/32235569 conn(0x7f1da0101a50 msgr2=0x7f1da0194170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.776+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 shutdown_connections 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:20.776+0000 7f1da60ef640 1 -- 172.21.15.69:0/32235569 wait complete. 2024-08-02T17:37:27.234 INFO:teuthology.orchestra.run.smithi069.stdout:mgr is available 2024-08-02T17:37:29.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:28 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4220507948' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-08-02T17:37:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:29.196 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [global] 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout fsid = b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.69:3300,v1:172.21.15.69:6789] 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout [osd] 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-02T17:37:29.197 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.448+0000 7f572ceab640 1 Processor -- start 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.448+0000 7f572ceab640 1 -- start start 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.448+0000 7f572ceab640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f57281077f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.448+0000 7f572ceab640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5728107dc0 con 0x7f57281073f0 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.448+0000 7f572659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f57281077f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.448+0000 7f572659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f57281077f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43342/0 (socket says 172.21.15.69:43342) 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.448+0000 7f572659c640 1 -- 172.21.15.69:0/1246775614 learned_addr learned my addr 172.21.15.69:0/1246775614 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.449+0000 7f572659c640 1 -- 172.21.15.69:0/1246775614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5728108550 con 0x7f57281073f0 2024-08-02T17:37:29.198 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.449+0000 7f572659c640 1 --2- 172.21.15.69:0/1246775614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f57281077f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5718009820 tx=0x7f571802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90061d3142b72416 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572559a640 1 -- 172.21.15.69:0/1246775614 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5718037070 con 0x7f57281073f0 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572559a640 1 -- 172.21.15.69:0/1246775614 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5718032900 con 0x7f57281073f0 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572559a640 1 -- 172.21.15.69:0/1246775614 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5718030850 con 0x7f57281073f0 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572ceab640 1 -- 172.21.15.69:0/1246775614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 msgr2=0x7f57281077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572ceab640 1 --2- 172.21.15.69:0/1246775614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f57281077f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5718009820 tx=0x7f571802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572ceab640 1 -- 172.21.15.69:0/1246775614 shutdown_connections 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572ceab640 1 --2- 172.21.15.69:0/1246775614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f57281077f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572ceab640 1 -- 172.21.15.69:0/1246775614 >> 172.21.15.69:0/1246775614 conn(0x7f5728102ba0 msgr2=0x7f5728104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572ceab640 1 -- 172.21.15.69:0/1246775614 shutdown_connections 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.450+0000 7f572ceab640 1 -- 172.21.15.69:0/1246775614 wait complete. 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.451+0000 7f572ceab640 1 Processor -- start 2024-08-02T17:37:29.199 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.451+0000 7f572ceab640 1 -- start start 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.451+0000 7f572ceab640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f572807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.451+0000 7f572ceab640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f572810b900 con 0x7f57281073f0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.451+0000 7f572659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f572807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.451+0000 7f572659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f572807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43348/0 (socket says 172.21.15.69:43348) 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.451+0000 7f572659c640 1 -- 172.21.15.69:0/4220507948 learned_addr learned my addr 172.21.15.69:0/4220507948 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.452+0000 7f572659c640 1 -- 172.21.15.69:0/4220507948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f572807e030 con 0x7f57281073f0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.452+0000 7f572659c640 1 --2- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f572807daf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5718009940 tx=0x7f5718032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.452+0000 7f57177fe640 1 -- 172.21.15.69:0/4220507948 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5718042070 con 0x7f57281073f0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.452+0000 7f57177fe640 1 -- 172.21.15.69:0/4220507948 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f57180308c0 con 0x7f57281073f0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.452+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f572807c220 con 0x7f57281073f0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.452+0000 7f57177fe640 1 -- 172.21.15.69:0/4220507948 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5718037040 con 0x7f57281073f0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.452+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f572807c760 con 0x7f57281073f0 2024-08-02T17:37:29.200 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.453+0000 7f57177fe640 1 -- 172.21.15.69:0/4220507948 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5718045070 con 0x7f57281073f0 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.453+0000 7f57177fe640 1 --2- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f5710038f20 0x7f571003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.453+0000 7f57177fe640 1 -- 172.21.15.69:0/4220507948 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f571806bfd0 con 0x7f57281073f0 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.453+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f572810ba80 con 0x7f57281073f0 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.453+0000 7f5725d9b640 1 --2- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f5710038f20 0x7f571003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.454+0000 7f5725d9b640 1 --2- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f5710038f20 0x7f571003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f571c0098c0 tx=0x7f571c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.458+0000 7f57177fe640 1 -- 172.21.15.69:0/4220507948 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f571803cd90 con 0x7f57281073f0 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.623+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5728107870 con 0x7f57281073f0 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.624+0000 7f57177fe640 1 -- 172.21.15.69:0/4220507948 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f5728107870 con 0x7f57281073f0 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f5710038f20 msgr2=0x7f571003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 --2- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f5710038f20 0x7f571003b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f571c0098c0 tx=0x7f571c006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 msgr2=0x7f572807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 --2- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f572807daf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5718009940 tx=0x7f5718032ce0 comp rx=0 tx=0).stop 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 shutdown_connections 2024-08-02T17:37:29.201 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 --2- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f5710038f20 0x7f571003b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 --2- 172.21.15.69:0/4220507948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f57281073f0 0x7f572807daf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 >> 172.21.15.69:0/4220507948 conn(0x7f5728102ba0 msgr2=0x7f5728103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 shutdown_connections 2024-08-02T17:37:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:28.626+0000 7f572ceab640 1 -- 172.21.15.69:0/4220507948 wait complete. 2024-08-02T17:37:29.202 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling cephadm module... 2024-08-02T17:37:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:29 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2221924445' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-08-02T17:37:30.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:29 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ignoring --setuser ceph since I am not root 2024-08-02T17:37:30.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:29 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ignoring --setgroup ceph since I am not root 2024-08-02T17:37:30.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:29 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:29.983+0000 7f6df7ae7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-02T17:37:30.408 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:30 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:30.129+0000 7f6df7ae7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:37:30.408 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:30 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:30.195+0000 7f6df7ae7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-02T17:37:30.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.573+0000 7fc98d99e640 1 Processor -- start 2024-08-02T17:37:30.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.573+0000 7fc98d99e640 1 -- start start 2024-08-02T17:37:30.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.573+0000 7fc98d99e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc98810a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:30.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.573+0000 7fc98d99e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc98807bd50 con 0x7fc988107e40 2024-08-02T17:37:30.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.574+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc98810a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:30.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.574+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc98810a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43354/0 (socket says 172.21.15.69:43354) 2024-08-02T17:37:30.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.574+0000 7fc986ffd640 1 -- 172.21.15.69:0/4094501266 learned_addr learned my addr 172.21.15.69:0/4094501266 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:30.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.574+0000 7fc986ffd640 1 -- 172.21.15.69:0/4094501266 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc98810a770 con 0x7fc988107e40 2024-08-02T17:37:30.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.574+0000 7fc986ffd640 1 --2- 172.21.15.69:0/4094501266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc98810a230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc978009e80 tx=0x7fc97802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf53af2b210e5d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:30.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc985ffb640 1 -- 172.21.15.69:0/4094501266 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc978036070 con 0x7fc988107e40 2024-08-02T17:37:30.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc985ffb640 1 -- 172.21.15.69:0/4094501266 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc9780318c0 con 0x7fc988107e40 2024-08-02T17:37:30.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc985ffb640 1 -- 172.21.15.69:0/4094501266 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc978007ab0 con 0x7fc988107e40 2024-08-02T17:37:30.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc98d99e640 1 -- 172.21.15.69:0/4094501266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 msgr2=0x7fc98810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:30.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc98d99e640 1 --2- 172.21.15.69:0/4094501266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc98810a230 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc978009e80 tx=0x7fc97802a7a0 comp rx=0 tx=0).stop 2024-08-02T17:37:30.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc98d99e640 1 -- 172.21.15.69:0/4094501266 shutdown_connections 2024-08-02T17:37:30.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc98d99e640 1 --2- 172.21.15.69:0/4094501266 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc98810a230 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:30.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc98d99e640 1 -- 172.21.15.69:0/4094501266 >> 172.21.15.69:0/4094501266 conn(0x7fc988101a00 msgr2=0x7fc988103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:30.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc98d99e640 1 -- 172.21.15.69:0/4094501266 shutdown_connections 2024-08-02T17:37:30.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.575+0000 7fc98d99e640 1 -- 172.21.15.69:0/4094501266 wait complete. 2024-08-02T17:37:30.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.576+0000 7fc98d99e640 1 Processor -- start 2024-08-02T17:37:30.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.576+0000 7fc98d99e640 1 -- start start 2024-08-02T17:37:30.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.576+0000 7fc98d99e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc9881076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:30.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.576+0000 7fc98d99e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc98810b6f0 con 0x7fc988107e40 2024-08-02T17:37:30.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.576+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc9881076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:30.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.576+0000 7fc986ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc9881076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43360/0 (socket says 172.21.15.69:43360) 2024-08-02T17:37:30.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.576+0000 7fc986ffd640 1 -- 172.21.15.69:0/2221924445 learned_addr learned my addr 172.21.15.69:0/2221924445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:30.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.577+0000 7fc986ffd640 1 -- 172.21.15.69:0/2221924445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc988107c00 con 0x7fc988107e40 2024-08-02T17:37:30.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.577+0000 7fc986ffd640 1 --2- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc9881076c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc978031ca0 tx=0x7fc978031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:30.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.577+0000 7fc98c99c640 1 -- 172.21.15.69:0/2221924445 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc978036050 con 0x7fc988107e40 2024-08-02T17:37:30.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.577+0000 7fc98c99c640 1 -- 172.21.15.69:0/2221924445 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc978042070 con 0x7fc988107e40 2024-08-02T17:37:30.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.577+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc988105df0 con 0x7fc988107e40 2024-08-02T17:37:30.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.577+0000 7fc98c99c640 1 -- 172.21.15.69:0/2221924445 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc97803d820 con 0x7fc988107e40 2024-08-02T17:37:30.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.577+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc988106330 con 0x7fc988107e40 2024-08-02T17:37:30.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.578+0000 7fc98c99c640 1 -- 172.21.15.69:0/2221924445 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc97803b610 con 0x7fc988107e40 2024-08-02T17:37:30.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.578+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9881082a0 con 0x7fc988107e40 2024-08-02T17:37:30.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.578+0000 7fc98c99c640 1 --2- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fc974038f70 0x7fc97403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:30.427 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.578+0000 7fc98c99c640 1 -- 172.21.15.69:0/2221924445 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc978007e00 con 0x7fc988107e40 2024-08-02T17:37:30.428 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.578+0000 7fc9867fc640 1 --2- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fc974038f70 0x7fc97403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:30.428 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.579+0000 7fc9867fc640 1 --2- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fc974038f70 0x7fc97403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc9700098c0 tx=0x7fc970006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:30.428 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.583+0000 7fc98c99c640 1 -- 172.21.15.69:0/2221924445 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc978042220 con 0x7fc988107e40 2024-08-02T17:37:30.428 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.791+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc988106c10 con 0x7fc988107e40 2024-08-02T17:37:30.428 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.842+0000 7fc98c99c640 1 -- 172.21.15.69:0/2221924445 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc97806cc20 con 0x7fc988107e40 2024-08-02T17:37:30.429 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.844+0000 7fc98c99c640 1 -- 172.21.15.69:0/2221924445 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc978047460 con 0x7fc988107e40 2024-08-02T17:37:30.429 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fc974038f70 msgr2=0x7fc97403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:30.429 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 --2- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fc974038f70 0x7fc97403b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc9700098c0 tx=0x7fc970006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:30.429 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 msgr2=0x7fc9881076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:30.429 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 --2- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc9881076c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc978031ca0 tx=0x7fc978031cd0 comp rx=0 tx=0).stop 2024-08-02T17:37:30.429 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 shutdown_connections 2024-08-02T17:37:30.430 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 --2- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fc974038f70 0x7fc97403b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:30.430 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 --2- 172.21.15.69:0/2221924445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc988107e40 0x7fc9881076c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:30.430 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 >> 172.21.15.69:0/2221924445 conn(0x7fc988101a00 msgr2=0x7fc9881025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:30.430 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 shutdown_connections 2024-08-02T17:37:30.430 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:29.845+0000 7fc98d99e640 1 -- 172.21.15.69:0/2221924445 wait complete. 2024-08-02T17:37:30.727 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:30 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:30.444+0000 7f6df7ae7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-02T17:37:30.727 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:30 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:30.505+0000 7f6df7ae7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-02T17:37:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:30 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2221924445' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-02T17:37:31.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:30 smithi069 ceph-mon[32746]: mgrmap e5: x(active, since 13s) 2024-08-02T17:37:31.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:31.454+0000 7f6df7ae7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-02T17:37:31.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:31.569+0000 7f6df7ae7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-02T17:37:31.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-08-02T17:37:31.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-02T17:37:31.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-02T17:37:31.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-02T17:37:31.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-02T17:37:31.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-08-02T17:37:31.640 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.803+0000 7f20ed738640 1 Processor -- start 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.803+0000 7f20ed738640 1 -- start start 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.803+0000 7f20ed738640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e810a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.803+0000 7f20ed738640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20e807bdc0 con 0x7f20e8107df0 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.803+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e810a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.803+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e810a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43386/0 (socket says 172.21.15.69:43386) 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.803+0000 7f20e6ffd640 1 -- 172.21.15.69:0/3754778945 learned_addr learned my addr 172.21.15.69:0/3754778945 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.803+0000 7f20e6ffd640 1 -- 172.21.15.69:0/3754778945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e810a720 con 0x7f20e8107df0 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.804+0000 7f20e6ffd640 1 --2- 172.21.15.69:0/3754778945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e810a1e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009820 tx=0x7f20d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79027252bb37495d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.804+0000 7f20e67fc640 1 -- 172.21.15.69:0/3754778945 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d8037070 con 0x7f20e8107df0 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.804+0000 7f20e67fc640 1 -- 172.21.15.69:0/3754778945 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20d8032900 con 0x7f20e8107df0 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.804+0000 7f20e67fc640 1 -- 172.21.15.69:0/3754778945 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d8030850 con 0x7f20e8107df0 2024-08-02T17:37:31.641 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.805+0000 7f20ed738640 1 -- 172.21.15.69:0/3754778945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 msgr2=0x7f20e810a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.805+0000 7f20ed738640 1 --2- 172.21.15.69:0/3754778945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e810a1e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009820 tx=0x7f20d802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.805+0000 7f20ed738640 1 -- 172.21.15.69:0/3754778945 shutdown_connections 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.805+0000 7f20ed738640 1 --2- 172.21.15.69:0/3754778945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e810a1e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.805+0000 7f20ed738640 1 -- 172.21.15.69:0/3754778945 >> 172.21.15.69:0/3754778945 conn(0x7f20e8101a50 msgr2=0x7f20e8103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.805+0000 7f20ed738640 1 -- 172.21.15.69:0/3754778945 shutdown_connections 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.805+0000 7f20ed738640 1 -- 172.21.15.69:0/3754778945 wait complete. 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20ed738640 1 Processor -- start 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20ed738640 1 -- start start 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20ed738640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e8107670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20ed738640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20e810b700 con 0x7f20e8107df0 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e8107670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20e6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e8107670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43394/0 (socket says 172.21.15.69:43394) 2024-08-02T17:37:31.642 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20e6ffd640 1 -- 172.21.15.69:0/373172783 learned_addr learned my addr 172.21.15.69:0/373172783 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20e6ffd640 1 -- 172.21.15.69:0/373172783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e8107bb0 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20e6ffd640 1 --2- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e8107670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097f0 tx=0x7f20d8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.806+0000 7f20e4ff9640 1 -- 172.21.15.69:0/373172783 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d8037040 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.807+0000 7f20e4ff9640 1 -- 172.21.15.69:0/373172783 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20d8040070 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.807+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e8105da0 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.807+0000 7f20e4ff9640 1 -- 172.21.15.69:0/373172783 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20d803a390 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.807+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e8106280 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.808+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e8108250 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.808+0000 7f20e4ff9640 1 -- 172.21.15.69:0/373172783 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f20d803a5e0 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.808+0000 7f20e4ff9640 1 --2- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f20d4038fc0 0x7f20d403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.808+0000 7f20dffff640 1 -- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f20d4038fc0 msgr2=0x7f20d403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3960560393 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.808+0000 7f20e4ff9640 1 -- 172.21.15.69:0/373172783 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f20d806c250 con 0x7f20e8107df0 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.808+0000 7f20dffff640 1 --2- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f20d4038fc0 0x7f20d403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:37:31.643 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:30.813+0000 7f20e4ff9640 1 -- 172.21.15.69:0/373172783 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f20d8043360 con 0x7f20e8107df0 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.009+0000 7f20dffff640 1 -- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f20d4038fc0 msgr2=0x7f20d403b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3960560393 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.009+0000 7f20dffff640 1 --2- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f20d4038fc0 0x7f20d403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.029+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f20e8106500 con 0x7f20e8107df0 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.029+0000 7f20e4ff9640 1 -- 172.21.15.69:0/373172783 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f20d803a9d0 con 0x7f20e8107df0 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f20d4038fc0 msgr2=0x7f20d403b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 --2- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f20d4038fc0 0x7f20d403b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 msgr2=0x7f20e8107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 --2- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e8107670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f20d80097f0 tx=0x7f20d8030d80 comp rx=0 tx=0).stop 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 shutdown_connections 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 --2- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7f20d4038fc0 0x7f20d403b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 --2- 172.21.15.69:0/373172783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f20e8107df0 0x7f20e8107670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 >> 172.21.15.69:0/373172783 conn(0x7f20e8101a50 msgr2=0x7f20e8102650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.031+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 shutdown_connections 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.032+0000 7f20ed738640 1 -- 172.21.15.69:0/373172783 wait complete. 2024-08-02T17:37:31.644 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-08-02T17:37:31.645 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 5... 2024-08-02T17:37:31.901 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:31 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/373172783' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-02T17:37:31.901 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:31.625+0000 7f6df7ae7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-02T17:37:31.901 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:31.847+0000 7f6df7ae7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-02T17:37:32.184 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:31.909+0000 7f6df7ae7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-02T17:37:32.185 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:31.978+0000 7f6df7ae7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-02T17:37:32.185 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:32 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:32.122+0000 7f6df7ae7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:37:32.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:32 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:32.531+0000 7f6df7ae7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-02T17:37:32.841 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:32 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:32.840+0000 7f6df7ae7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-02T17:37:33.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:32 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:32.905+0000 7f6df7ae7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-02T17:37:33.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:33.050+0000 7f6df7ae7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-02T17:37:33.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:33.353+0000 7f6df7ae7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-02T17:37:33.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:33.413+0000 7f6df7ae7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-02T17:37:33.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:33.593+0000 7f6df7ae7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-02T17:37:34.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:33.736+0000 7f6df7ae7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-02T17:37:34.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:33.798+0000 7f6df7ae7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-02T17:37:34.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:33.879+0000 7f6df7ae7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-02T17:37:34.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:33.937+0000 7f6df7ae7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-02T17:37:34.858 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:34.460+0000 7f6df7ae7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-02T17:37:34.858 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:34.615+0000 7f6df7ae7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-02T17:37:34.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: Active manager daemon x restarted 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: Activating manager daemon x 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: osdmap e2: 0 total, 0 up, 0 in 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: mgrmap e6: x(active, starting, since 0.00339151s) 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: Manager daemon x is now available 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:37:34.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:37:34.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-02T17:37:34.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:34 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-02T17:37:35.929 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:35 smithi069 ceph-mon[32746]: Found migration_current of "None". Setting to last migration. 2024-08-02T17:37:35.929 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:35 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:35.929 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:35 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:35.930 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:35 smithi069 ceph-mon[32746]: mgrmap e7: x(active, since 1.00521s) 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:36.114+0000 7f6d64805640 -1 log_channel(cephadm) log [ERR] : [02/Aug/2024:17:37:36] ENGINE Error in HTTPServer.serve 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: Traceback (most recent call last): 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._connections.run(self.expiration_interval) 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._run(expiration_interval) 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: new_conn = self._from_server_socket(self.server.socket) 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-02T17:37:36.205 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: s = self.context.wrap_socket( 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: return self.sslsocket_class._create( 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self.do_handshake() 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._sslobj.do_handshake() 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-02T17:37:36.206 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.977+0000 7fd3644d2640 1 Processor -- start 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.977+0000 7fd3644d2640 1 -- start start 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.977+0000 7fd3644d2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c07c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.977+0000 7fd3644d2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c07c7c0 con 0x7fd35c07dd80 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.978+0000 7fd36226e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c07c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.978+0000 7fd36226e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c07c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43398/0 (socket says 172.21.15.69:43398) 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.978+0000 7fd36226e640 1 -- 172.21.15.69:0/3202008181 learned_addr learned my addr 172.21.15.69:0/3202008181 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:36.214 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.978+0000 7fd36226e640 1 -- 172.21.15.69:0/3202008181 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c07c940 con 0x7fd35c07dd80 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.979+0000 7fd36226e640 1 --2- 172.21.15.69:0/3202008181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c07c280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009820 tx=0x7fd34c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3041c8101c64b96c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.979+0000 7fd36126c640 1 -- 172.21.15.69:0/3202008181 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd34c037070 con 0x7fd35c07dd80 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.979+0000 7fd36126c640 1 -- 172.21.15.69:0/3202008181 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd34c032900 con 0x7fd35c07dd80 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.979+0000 7fd36126c640 1 -- 172.21.15.69:0/3202008181 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd34c030850 con 0x7fd35c07dd80 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.979+0000 7fd3644d2640 1 -- 172.21.15.69:0/3202008181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 msgr2=0x7fd35c07c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.979+0000 7fd3644d2640 1 --2- 172.21.15.69:0/3202008181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c07c280 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009820 tx=0x7fd34c02a270 comp rx=0 tx=0).stop 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 -- 172.21.15.69:0/3202008181 shutdown_connections 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 --2- 172.21.15.69:0/3202008181 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c07c280 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 -- 172.21.15.69:0/3202008181 >> 172.21.15.69:0/3202008181 conn(0x7fd35c102ba0 msgr2=0x7fd35c104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 -- 172.21.15.69:0/3202008181 shutdown_connections 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 -- 172.21.15.69:0/3202008181 wait complete. 2024-08-02T17:37:36.215 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 Processor -- start 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 -- start start 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd3644d2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c10d9a0 con 0x7fd35c07dd80 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd36226e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd36226e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43408/0 (socket says 172.21.15.69:43408) 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.980+0000 7fd36226e640 1 -- 172.21.15.69:0/1074594835 learned_addr learned my addr 172.21.15.69:0/1074594835 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.981+0000 7fd36226e640 1 -- 172.21.15.69:0/1074594835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c19dad0 con 0x7fd35c07dd80 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.981+0000 7fd36226e640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c19d590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd34c031000 tx=0x7fd34c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.981+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd34c042070 con 0x7fd35c07dd80 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.981+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd34c032be0 con 0x7fd35c07dd80 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.981+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd34c037040 con 0x7fd35c07dd80 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.981+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c19dd60 con 0x7fd35c07dd80 2024-08-02T17:37:36.216 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.981+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c1a0a50 con 0x7fd35c07dd80 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.982+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd34c045070 con 0x7fd35c07dd80 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.982+0000 7fd3537fe640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 0x7fd34803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.982+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 --> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd34803bb90 con 0x7fd348038fc0 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.982+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd34c06bf30 con 0x7fd35c07dd80 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.982+0000 7fd361a6d640 1 -- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 msgr2=0x7fd34803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3960560393 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:31.982+0000 7fd361a6d640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 0x7fd34803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:32.183+0000 7fd361a6d640 1 -- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 msgr2=0x7fd34803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3960560393 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:32.183+0000 7fd361a6d640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 0x7fd34803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:32.584+0000 7fd361a6d640 1 -- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 msgr2=0x7fd34803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3960560393 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:32.584+0000 7fd361a6d640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 0x7fd34803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:33.385+0000 7fd361a6d640 1 -- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 msgr2=0x7fd34803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3960560393 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:33.385+0000 7fd361a6d640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 0x7fd34803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-02T17:37:36.217 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:34.618+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fd34c06a9f0 con 0x7fd35c07dd80 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:34.618+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 msgr2=0x7fd34803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:34.618+0000 7fd3537fe640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 0x7fd34803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.619+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fd34c047440 con 0x7fd35c07dd80 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.619+0000 7fd3537fe640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fd34803c8f0 0x7fd34803ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.619+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd34c03ee50 con 0x7fd34803c8f0 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.620+0000 7fd361a6d640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fd34803c8f0 0x7fd34803ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.620+0000 7fd361a6d640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fd34803c8f0 0x7fd34803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd358003980 tx=0x7fd3580092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.621+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mgr.14118 v2:172.21.15.69:6800/3376234785 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd34c03ee50 con 0x7fd34803c8f0 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.633+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd35c07c300 con 0x7fd34803c8f0 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.633+0000 7fd3537fe640 1 -- 172.21.15.69:0/1074594835 <== mgr.14118 v2:172.21.15.69:6800/3376234785 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd35c07c300 con 0x7fd34803c8f0 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fd34803c8f0 msgr2=0x7fd34803ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:36.218 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fd34803c8f0 0x7fd34803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd358003980 tx=0x7fd3580092b0 comp rx=0 tx=0).stop 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 msgr2=0x7fd35c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c19d590 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd34c031000 tx=0x7fd34c0326e0 comp rx=0 tx=0).stop 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 shutdown_connections 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fd34803c8f0 0x7fd34803ece0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:6800/3960560393,v1:172.21.15.69:6801/3960560393] conn(0x7fd348038fc0 0x7fd34803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 --2- 172.21.15.69:0/1074594835 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd35c07dd80 0x7fd35c19d590 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 >> 172.21.15.69:0/1074594835 conn(0x7fd35c102ba0 msgr2=0x7fd35c103780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 shutdown_connections 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:35.634+0000 7fd3644d2640 1 -- 172.21.15.69:0/1074594835 wait complete. 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 5 is available 2024-08-02T17:37:36.219 INFO:teuthology.orchestra.run.smithi069.stdout:Setting orchestrator backend to cephadm... 2024-08-02T17:37:36.810 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-mon[32746]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-02T17:37:36.810 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-mon[32746]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-02T17:37:36.810 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:36 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:37:37.367 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.616+0000 7f0ba2f3c640 1 Processor -- start 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.616+0000 7f0ba2f3c640 1 -- start start 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.616+0000 7f0ba2f3c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c1077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.616+0000 7f0ba2f3c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c107d20 con 0x7f0b9c1073e0 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.616+0000 7f0ba0cd8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c1077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.616+0000 7f0ba0cd8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c1077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51304/0 (socket says 172.21.15.69:51304) 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.616+0000 7f0ba0cd8640 1 -- 172.21.15.69:0/1359175637 learned_addr learned my addr 172.21.15.69:0/1359175637 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.616+0000 7f0ba0cd8640 1 -- 172.21.15.69:0/1359175637 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c107ea0 con 0x7f0b9c1073e0 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.617+0000 7f0ba0cd8640 1 --2- 172.21.15.69:0/1359175637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c1077e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009820 tx=0x7f0b9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a713f3b7017dd078 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.617+0000 7f0b9b7fe640 1 -- 172.21.15.69:0/1359175637 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b90037070 con 0x7f0b9c1073e0 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.617+0000 7f0b9b7fe640 1 -- 172.21.15.69:0/1359175637 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b90032900 con 0x7f0b9c1073e0 2024-08-02T17:37:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.617+0000 7f0b9b7fe640 1 -- 172.21.15.69:0/1359175637 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b90030850 con 0x7f0b9c1073e0 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.618+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/1359175637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 msgr2=0x7f0b9c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.618+0000 7f0ba2f3c640 1 --2- 172.21.15.69:0/1359175637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c1077e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009820 tx=0x7f0b9002a270 comp rx=0 tx=0).stop 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.618+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/1359175637 shutdown_connections 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.618+0000 7f0ba2f3c640 1 --2- 172.21.15.69:0/1359175637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c1077e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.618+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/1359175637 >> 172.21.15.69:0/1359175637 conn(0x7f0b9c102c50 msgr2=0x7f0b9c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.618+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/1359175637 shutdown_connections 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.618+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/1359175637 wait complete. 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.618+0000 7f0ba2f3c640 1 Processor -- start 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba2f3c640 1 -- start start 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba2f3c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba2f3c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c10b800 con 0x7f0b9c1073e0 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba0cd8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:37.369 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba0cd8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51318/0 (socket says 172.21.15.69:51318) 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba0cd8640 1 -- 172.21.15.69:0/2774882457 learned_addr learned my addr 172.21.15.69:0/2774882457 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba0cd8640 1 -- 172.21.15.69:0/2774882457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c19b830 con 0x7f0b9c1073e0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba0cd8640 1 --2- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c19b2f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0b900097f0 tx=0x7f0b90030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0b99ffb640 1 -- 172.21.15.69:0/2774882457 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b90037040 con 0x7f0b9c1073e0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9c19bac0 con 0x7f0b9c1073e0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0b99ffb640 1 -- 172.21.15.69:0/2774882457 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b90040070 con 0x7f0b9c1073e0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.619+0000 7f0b99ffb640 1 -- 172.21.15.69:0/2774882457 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b9003a390 con 0x7f0b9c1073e0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.620+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9c19e7b0 con 0x7f0b9c1073e0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.620+0000 7f0b99ffb640 1 -- 172.21.15.69:0/2774882457 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f0b9003a630 con 0x7f0b9c1073e0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.621+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b9c107860 con 0x7f0b9c1073e0 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.621+0000 7f0b99ffb640 1 --2- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f0b8c038e60 0x7f0b8c03b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:37.370 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.621+0000 7f0b99ffb640 1 -- 172.21.15.69:0/2774882457 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0b9006c2d0 con 0x7f0b9c1073e0 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.621+0000 7f0b9bfff640 1 --2- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f0b8c038e60 0x7f0b8c03b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.621+0000 7f0b9bfff640 1 --2- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f0b8c038e60 0x7f0b8c03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0b880098c0 tx=0x7f0b88006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.625+0000 7f0b99ffb640 1 -- 172.21.15.69:0/2774882457 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0b9004a200 con 0x7f0b9c1073e0 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.674+0000 7f0b99ffb640 1 -- 172.21.15.69:0/2774882457 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0b90045800 con 0x7f0b9c1073e0 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.806+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f0b9c103de0 con 0x7f0b8c038e60 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.812+0000 7f0b99ffb640 1 -- 172.21.15.69:0/2774882457 <== mgr.14118 v2:172.21.15.69:6800/3376234785 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0b9c103de0 con 0x7f0b8c038e60 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.814+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f0b8c038e60 msgr2=0x7f0b8c03b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.814+0000 7f0ba2f3c640 1 --2- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f0b8c038e60 0x7f0b8c03b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0b880098c0 tx=0x7f0b88006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.814+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 msgr2=0x7f0b9c19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.814+0000 7f0ba2f3c640 1 --2- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c19b2f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0b900097f0 tx=0x7f0b90030d80 comp rx=0 tx=0).stop 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.815+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 shutdown_connections 2024-08-02T17:37:37.371 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.815+0000 7f0ba2f3c640 1 --2- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f0b8c038e60 0x7f0b8c03b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:37.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.815+0000 7f0ba2f3c640 1 --2- 172.21.15.69:0/2774882457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b9c1073e0 0x7f0b9c19b2f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:37.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.815+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 >> 172.21.15.69:0/2774882457 conn(0x7f0b9c102c50 msgr2=0x7f0b9c1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:37.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.815+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 shutdown_connections 2024-08-02T17:37:37.372 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:36.815+0000 7f0ba2f3c640 1 -- 172.21.15.69:0/2774882457 wait complete. 2024-08-02T17:37:37.918 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:35] ENGINE Bus STARTING 2024-08-02T17:37:37.919 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:36] ENGINE Serving on https://172.21.15.69:7150 2024-08-02T17:37:37.919 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:36] ENGINE Error in HTTPServer.serve 2024-08-02T17:37:37.919 INFO:journalctl@ceph.mon.a.smithi069.stdout: Traceback (most recent call last): 2024-08-02T17:37:37.919 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-02T17:37:37.919 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._connections.run(self.expiration_interval) 2024-08-02T17:37:37.919 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-02T17:37:37.919 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._run(expiration_interval) 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: s = self.context.wrap_socket( 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: return self.sslsocket_class._create( 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: self.do_handshake() 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._sslobj.do_handshake() 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-02T17:37:37.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:36] ENGINE Serving on http://172.21.15.69:8765 2024-08-02T17:37:37.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:36] ENGINE Bus STARTED 2024-08-02T17:37:37.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: mgrmap e8: x(active, since 2s) 2024-08-02T17:37:37.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:37:37.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:37.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:37 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:37:38.498 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.738+0000 7fad7437f640 1 Processor -- start 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.739+0000 7fad7437f640 1 -- start start 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.739+0000 7fad7437f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c10a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.739+0000 7fad7437f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c07bd80 con 0x7fad6c107e30 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.739+0000 7fad7211b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c10a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.739+0000 7fad7211b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c10a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51326/0 (socket says 172.21.15.69:51326) 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.739+0000 7fad7211b640 1 -- 172.21.15.69:0/1639323414 learned_addr learned my addr 172.21.15.69:0/1639323414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.739+0000 7fad7211b640 1 -- 172.21.15.69:0/1639323414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c10a760 con 0x7fad6c107e30 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.740+0000 7fad7211b640 1 --2- 172.21.15.69:0/1639323414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c10a220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fad68009820 tx=0x7fad6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d585ca1f2f1df71e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.740+0000 7fad71119640 1 -- 172.21.15.69:0/1639323414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad68037070 con 0x7fad6c107e30 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.740+0000 7fad71119640 1 -- 172.21.15.69:0/1639323414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad68032900 con 0x7fad6c107e30 2024-08-02T17:37:38.499 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.740+0000 7fad71119640 1 -- 172.21.15.69:0/1639323414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad68030850 con 0x7fad6c107e30 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 -- 172.21.15.69:0/1639323414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 msgr2=0x7fad6c10a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 --2- 172.21.15.69:0/1639323414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c10a220 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fad68009820 tx=0x7fad6802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 -- 172.21.15.69:0/1639323414 shutdown_connections 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 --2- 172.21.15.69:0/1639323414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c10a220 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 -- 172.21.15.69:0/1639323414 >> 172.21.15.69:0/1639323414 conn(0x7fad6c101a50 msgr2=0x7fad6c103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 -- 172.21.15.69:0/1639323414 shutdown_connections 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 -- 172.21.15.69:0/1639323414 wait complete. 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 Processor -- start 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.741+0000 7fad7437f640 1 -- start start 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7437f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7437f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c10b740 con 0x7fad6c107e30 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7211b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7211b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51332/0 (socket says 172.21.15.69:51332) 2024-08-02T17:37:38.500 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7211b640 1 -- 172.21.15.69:0/4228606917 learned_addr learned my addr 172.21.15.69:0/4228606917 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7211b640 1 -- 172.21.15.69:0/4228606917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c19b840 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7211b640 1 --2- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c19b300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad680097f0 tx=0x7fad68030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad637fe640 1 -- 172.21.15.69:0/4228606917 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad68037040 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c19bad0 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad637fe640 1 -- 172.21.15.69:0/4228606917 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fad68040070 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad637fe640 1 -- 172.21.15.69:0/4228606917 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad6803a390 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.742+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c19e7c0 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.743+0000 7fad637fe640 1 -- 172.21.15.69:0/4228606917 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fad6803a630 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.743+0000 7fad637fe640 1 --2- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fad58038f70 0x7fad5803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.743+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c108290 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.743+0000 7fad637fe640 1 -- 172.21.15.69:0/4228606917 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad6806c610 con 0x7fad6c107e30 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.743+0000 7fad7191a640 1 --2- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fad58038f70 0x7fad5803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:38.501 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.744+0000 7fad7191a640 1 --2- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fad58038f70 0x7fad5803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0098c0 tx=0x7fad5c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:38.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.748+0000 7fad637fe640 1 -- 172.21.15.69:0/4228606917 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fad68043bf0 con 0x7fad6c107e30 2024-08-02T17:37:38.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.916+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fad6c102d00 con 0x7fad58038f70 2024-08-02T17:37:38.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.917+0000 7fad637fe640 1 -- 172.21.15.69:0/4228606917 <== mgr.14118 v2:172.21.15.69:6800/3376234785 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fad6c102d00 con 0x7fad58038f70 2024-08-02T17:37:38.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fad58038f70 msgr2=0x7fad5803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:38.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 --2- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fad58038f70 0x7fad5803b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0098c0 tx=0x7fad5c006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:38.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 msgr2=0x7fad6c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:38.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 --2- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c19b300 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad680097f0 tx=0x7fad68030d80 comp rx=0 tx=0).stop 2024-08-02T17:37:38.502 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 shutdown_connections 2024-08-02T17:37:38.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 --2- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fad58038f70 0x7fad5803b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:38.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 --2- 172.21.15.69:0/4228606917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fad6c107e30 0x7fad6c19b300 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:38.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 >> 172.21.15.69:0/4228606917 conn(0x7fad6c101a50 msgr2=0x7fad6c1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:38.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 shutdown_connections 2024-08-02T17:37:38.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:37.919+0000 7fad7437f640 1 -- 172.21.15.69:0/4228606917 wait complete. 2024-08-02T17:37:38.503 INFO:teuthology.orchestra.run.smithi069.stdout:Generating ssh key... 2024-08-02T17:37:39.048 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:38 smithi069 ceph-mon[32746]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:37:39.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: Generating public/private rsa key pair. 2024-08-02T17:37:39.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: Your identification has been saved in /tmp/tmpeg8q90g3/key 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: Your public key has been saved in /tmp/tmpeg8q90g3/key.pub 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: The key fingerprint is: 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: SHA256:B3ffKsnhFC/wJEQYOQv5XlDQcLTSdpuhc5lGYoo0KLw ceph-b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: The key's randomart image is: 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: +---[RSA 3072]----+ 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | .+@* | 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | . .o =+.. | 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | o . ooo+X B | 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | o . ooB.@ X . | 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | E ..S.+ % o .| 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | .. B + . | 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | = . | 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | . | 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: | | 2024-08-02T17:37:39.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: +----[SHA256]-----+ 2024-08-02T17:37:39.787 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.868+0000 7fdc91ba7640 1 Processor -- start 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.868+0000 7fdc91ba7640 1 -- start start 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.869+0000 7fdc91ba7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c1077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.869+0000 7fdc91ba7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c107d20 con 0x7fdc8c1073e0 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.869+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c1077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.869+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c1077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51344/0 (socket says 172.21.15.69:51344) 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.869+0000 7fdc8b7fe640 1 -- 172.21.15.69:0/2364304533 learned_addr learned my addr 172.21.15.69:0/2364304533 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.869+0000 7fdc8b7fe640 1 -- 172.21.15.69:0/2364304533 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c107ea0 con 0x7fdc8c1073e0 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc8b7fe640 1 --2- 172.21.15.69:0/2364304533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c1077e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009e80 tx=0x7fdc7c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4debf6e448574e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc8a7fc640 1 -- 172.21.15.69:0/2364304533 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c036070 con 0x7fdc8c1073e0 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc8a7fc640 1 -- 172.21.15.69:0/2364304533 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc7c0318c0 con 0x7fdc8c1073e0 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc8a7fc640 1 -- 172.21.15.69:0/2364304533 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c007ab0 con 0x7fdc8c1073e0 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc91ba7640 1 -- 172.21.15.69:0/2364304533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 msgr2=0x7fdc8c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:39.788 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc91ba7640 1 --2- 172.21.15.69:0/2364304533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c1077e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c009e80 tx=0x7fdc7c02a7a0 comp rx=0 tx=0).stop 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc91ba7640 1 -- 172.21.15.69:0/2364304533 shutdown_connections 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc91ba7640 1 --2- 172.21.15.69:0/2364304533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c1077e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.870+0000 7fdc91ba7640 1 -- 172.21.15.69:0/2364304533 >> 172.21.15.69:0/2364304533 conn(0x7fdc8c102c50 msgr2=0x7fdc8c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.871+0000 7fdc91ba7640 1 -- 172.21.15.69:0/2364304533 shutdown_connections 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.871+0000 7fdc91ba7640 1 -- 172.21.15.69:0/2364304533 wait complete. 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.871+0000 7fdc91ba7640 1 Processor -- start 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.871+0000 7fdc91ba7640 1 -- start start 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.871+0000 7fdc91ba7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.871+0000 7fdc91ba7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc8c10b800 con 0x7fdc8c1073e0 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51348/0 (socket says 172.21.15.69:51348) 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc8b7fe640 1 -- 172.21.15.69:0/4109063780 learned_addr learned my addr 172.21.15.69:0/4109063780 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc8b7fe640 1 -- 172.21.15.69:0/4109063780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc8c19b830 con 0x7fdc8c1073e0 2024-08-02T17:37:39.789 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc8b7fe640 1 --2- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c19b2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c031ca0 tx=0x7fdc7c031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc88ff9640 1 -- 172.21.15.69:0/4109063780 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c036050 con 0x7fdc8c1073e0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc8c19bac0 con 0x7fdc8c1073e0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc88ff9640 1 -- 172.21.15.69:0/4109063780 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdc7c042070 con 0x7fdc8c1073e0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc88ff9640 1 -- 172.21.15.69:0/4109063780 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc7c03d820 con 0x7fdc8c1073e0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.872+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8c19e7b0 con 0x7fdc8c1073e0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.873+0000 7fdc88ff9640 1 -- 172.21.15.69:0/4109063780 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdc7c03db20 con 0x7fdc8c1073e0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.873+0000 7fdc88ff9640 1 --2- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc78038f70 0x7fdc7803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.873+0000 7fdc88ff9640 1 -- 172.21.15.69:0/4109063780 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc7c007e00 con 0x7fdc8c1073e0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.873+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc8c107860 con 0x7fdc8c1073e0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.873+0000 7fdc8affd640 1 --2- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc78038f70 0x7fdc7803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.874+0000 7fdc8affd640 1 --2- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc78038f70 0x7fdc7803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdc740098c0 tx=0x7fdc74006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:39.790 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:38.878+0000 7fdc88ff9640 1 -- 172.21.15.69:0/4109063780 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc7c0371f0 con 0x7fdc8c1073e0 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.046+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fdc8c103de0 con 0x7fdc78038f70 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.280+0000 7fdc88ff9640 1 -- 172.21.15.69:0/4109063780 <== mgr.14118 v2:172.21.15.69:6800/3376234785 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdc8c103de0 con 0x7fdc78038f70 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.282+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc78038f70 msgr2=0x7fdc7803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.282+0000 7fdc91ba7640 1 --2- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc78038f70 0x7fdc7803b430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fdc740098c0 tx=0x7fdc74006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.282+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 msgr2=0x7fdc8c19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.282+0000 7fdc91ba7640 1 --2- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c19b2f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c031ca0 tx=0x7fdc7c031cd0 comp rx=0 tx=0).stop 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.282+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 shutdown_connections 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.282+0000 7fdc91ba7640 1 --2- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc78038f70 0x7fdc7803b430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.282+0000 7fdc91ba7640 1 --2- 172.21.15.69:0/4109063780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc8c1073e0 0x7fdc8c19b2f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.282+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 >> 172.21.15.69:0/4109063780 conn(0x7fdc8c102c50 msgr2=0x7fdc8c1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.283+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 shutdown_connections 2024-08-02T17:37:39.791 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:39.283+0000 7fdc91ba7640 1 -- 172.21.15.69:0/4109063780 wait complete. 2024-08-02T17:37:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:40 smithi069 ceph-mon[32746]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:37:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:40 smithi069 ceph-mon[32746]: Generating ssh key... 2024-08-02T17:37:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:40 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:40 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZrcJB3BsVUjzi6g8TzqPXKUKDfpsELg0+pY48Wjyl4zERDATf+hO6EKQfc6iu6bv3nf6Ph9yONejdgzqvY3aHWVxtFhA5FHGh7mfoWX1O+QcrQ25dQrQLnQX+FGgQzcreVANycnrPNpSEWjMoIbPtS8Qs7RHyfGaVaYpaYrSREocpXlnjsGQCDN3lycgSa6niG7blJz9C6JrIKeZ66kLeAfof0w1dH+nCKqQVoNNAOic4iWfl8n/hthKnJo1VL1AaYWWJiSYBAG2jeRzoOJpwXPZVrYZSK/W9jBmUQkAmg6OmsQxsW4bCHrrQcDBr6TRHGrqxyrwKrEq1HIJveC6gajvB/mqI7JAv8nWHpWD4TsSkT65WsXAenFpz+ad6n7pEVh6wiMPJTtyg4U3uR1OKWiXop+q21sTz/PSSX6Zd2b8CKq+yhAWFprqD7eE43ZBrxKLEkMlTiPJdACgR+o64kCAVyG+DJNCqPMIaSpjtpPTodEOamvPersR8LY4Vehs= ceph-b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.181+0000 7f2beddaf640 1 Processor -- start 2024-08-02T17:37:40.906 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.181+0000 7f2beddaf640 1 -- start start 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.181+0000 7f2beddaf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be81065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.181+0000 7f2beddaf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8106bc0 con 0x7f2be81061f0 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.182+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be81065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.182+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be81065f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51364/0 (socket says 172.21.15.69:51364) 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.182+0000 7f2be77fe640 1 -- 172.21.15.69:0/951923705 learned_addr learned my addr 172.21.15.69:0/951923705 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.182+0000 7f2be77fe640 1 -- 172.21.15.69:0/951923705 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be81073f0 con 0x7f2be81061f0 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.182+0000 7f2be77fe640 1 --2- 172.21.15.69:0/951923705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be81065f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009820 tx=0x7f2bd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56430ff4c08e5ae7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2be67fc640 1 -- 172.21.15.69:0/951923705 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bd8037070 con 0x7f2be81061f0 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2be67fc640 1 -- 172.21.15.69:0/951923705 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bd8032900 con 0x7f2be81061f0 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2be67fc640 1 -- 172.21.15.69:0/951923705 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bd8030850 con 0x7f2be81061f0 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2beddaf640 1 -- 172.21.15.69:0/951923705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 msgr2=0x7f2be81065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:40.907 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2beddaf640 1 --2- 172.21.15.69:0/951923705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be81065f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009820 tx=0x7f2bd802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2beddaf640 1 -- 172.21.15.69:0/951923705 shutdown_connections 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2beddaf640 1 --2- 172.21.15.69:0/951923705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be81065f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2beddaf640 1 -- 172.21.15.69:0/951923705 >> 172.21.15.69:0/951923705 conn(0x7f2be8101a00 msgr2=0x7f2be8103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2beddaf640 1 -- 172.21.15.69:0/951923705 shutdown_connections 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.183+0000 7f2beddaf640 1 -- 172.21.15.69:0/951923705 wait complete. 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.184+0000 7f2beddaf640 1 Processor -- start 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.184+0000 7f2beddaf640 1 -- start start 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.184+0000 7f2beddaf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.184+0000 7f2beddaf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2be810a7a0 con 0x7f2be81061f0 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.184+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.184+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51370/0 (socket says 172.21.15.69:51370) 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.184+0000 7f2be77fe640 1 -- 172.21.15.69:0/2765922067 learned_addr learned my addr 172.21.15.69:0/2765922067 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.185+0000 7f2be77fe640 1 -- 172.21.15.69:0/2765922067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be819a7d0 con 0x7f2be81061f0 2024-08-02T17:37:40.908 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.185+0000 7f2be77fe640 1 --2- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be819a290 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009940 tx=0x7f2bd8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.185+0000 7f2be4ff9640 1 -- 172.21.15.69:0/2765922067 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bd8042070 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.185+0000 7f2be4ff9640 1 -- 172.21.15.69:0/2765922067 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bd80308c0 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.185+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be819aa60 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.185+0000 7f2be4ff9640 1 -- 172.21.15.69:0/2765922067 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bd8037040 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.185+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be819d750 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.186+0000 7f2be4ff9640 1 -- 172.21.15.69:0/2765922067 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2bd8045070 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.186+0000 7f2be4ff9640 1 --2- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f2bd4038f70 0x7f2bd403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.186+0000 7f2be4ff9640 1 -- 172.21.15.69:0/2765922067 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2bd806c080 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.186+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be810a920 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.186+0000 7f2be6ffd640 1 --2- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f2bd4038f70 0x7f2bd403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.186+0000 7f2be6ffd640 1 --2- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f2bd4038f70 0x7f2bd403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00098c0 tx=0x7f2bd0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.191+0000 7f2be4ff9640 1 -- 172.21.15.69:0/2765922067 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2bd8045520 con 0x7f2be81061f0 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.357+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2be8102cc0 con 0x7f2bd4038f70 2024-08-02T17:37:40.909 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.358+0000 7f2be4ff9640 1 -- 172.21.15.69:0/2765922067 <== mgr.14118 v2:172.21.15.69:6800/3376234785 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2be8102cc0 con 0x7f2bd4038f70 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.359+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f2bd4038f70 msgr2=0x7f2bd403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.359+0000 7f2beddaf640 1 --2- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f2bd4038f70 0x7f2bd403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00098c0 tx=0x7f2bd0006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.359+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 msgr2=0x7f2be819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.359+0000 7f2beddaf640 1 --2- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be819a290 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009940 tx=0x7f2bd8032ce0 comp rx=0 tx=0).stop 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.359+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 shutdown_connections 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.359+0000 7f2beddaf640 1 --2- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f2bd4038f70 0x7f2bd403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.359+0000 7f2beddaf640 1 --2- 172.21.15.69:0/2765922067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2be81061f0 0x7f2be819a290 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.359+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 >> 172.21.15.69:0/2765922067 conn(0x7f2be8101a00 msgr2=0x7f2be81025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.360+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 shutdown_connections 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:40.360+0000 7f2beddaf640 1 -- 172.21.15.69:0/2765922067 wait complete. 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:Adding key to root@localhost authorized_keys... 2024-08-02T17:37:40.910 INFO:teuthology.orchestra.run.smithi069.stdout:Adding host smithi069... 2024-08-02T17:37:41.448 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:41 smithi069 ceph-mon[32746]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:37:42.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:42 smithi069 ceph-mon[32746]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:37:43.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:43 smithi069 ceph-mon[32746]: Deploying cephadm binary to smithi069 2024-08-02T17:37:45.101 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Added host 'smithi069' with addr '172.21.15.69' 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.266+0000 7f4e92383640 1 Processor -- start 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.266+0000 7f4e92383640 1 -- start start 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.266+0000 7f4e92383640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c10a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.266+0000 7f4e92383640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c07bdb0 con 0x7f4e8c107e10 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.267+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c10a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.267+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c10a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51376/0 (socket says 172.21.15.69:51376) 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.267+0000 7f4e8bfff640 1 -- 172.21.15.69:0/1240294043 learned_addr learned my addr 172.21.15.69:0/1240294043 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.267+0000 7f4e8bfff640 1 -- 172.21.15.69:0/1240294043 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c10a740 con 0x7f4e8c107e10 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e8bfff640 1 --2- 172.21.15.69:0/1240294043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c10a200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009820 tx=0x7f4e7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcecae5779a49ee5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e8affd640 1 -- 172.21.15.69:0/1240294043 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7c037070 con 0x7f4e8c107e10 2024-08-02T17:37:45.102 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e8affd640 1 -- 172.21.15.69:0/1240294043 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e7c032900 con 0x7f4e8c107e10 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e8affd640 1 -- 172.21.15.69:0/1240294043 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7c030850 con 0x7f4e8c107e10 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e92383640 1 -- 172.21.15.69:0/1240294043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 msgr2=0x7f4e8c10a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e92383640 1 --2- 172.21.15.69:0/1240294043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c10a200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009820 tx=0x7f4e7c02a270 comp rx=0 tx=0).stop 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e92383640 1 -- 172.21.15.69:0/1240294043 shutdown_connections 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e92383640 1 --2- 172.21.15.69:0/1240294043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c10a200 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e92383640 1 -- 172.21.15.69:0/1240294043 >> 172.21.15.69:0/1240294043 conn(0x7f4e8c101ab0 msgr2=0x7f4e8c103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e92383640 1 -- 172.21.15.69:0/1240294043 shutdown_connections 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.268+0000 7f4e92383640 1 -- 172.21.15.69:0/1240294043 wait complete. 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.269+0000 7f4e92383640 1 Processor -- start 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.269+0000 7f4e92383640 1 -- start start 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.269+0000 7f4e92383640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.269+0000 7f4e92383640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c10b720 con 0x7f4e8c107e10 2024-08-02T17:37:45.103 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51380/0 (socket says 172.21.15.69:51380) 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e8bfff640 1 -- 172.21.15.69:0/3307571177 learned_addr learned my addr 172.21.15.69:0/3307571177 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e8bfff640 1 -- 172.21.15.69:0/3307571177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c19b890 con 0x7f4e8c107e10 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e8bfff640 1 --2- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c19b350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c031000 tx=0x7f4e7c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e897fa640 1 -- 172.21.15.69:0/3307571177 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7c042070 con 0x7f4e8c107e10 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e897fa640 1 -- 172.21.15.69:0/3307571177 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e7c032be0 con 0x7f4e8c107e10 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c19bb20 con 0x7f4e8c107e10 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e897fa640 1 -- 172.21.15.69:0/3307571177 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7c037040 con 0x7f4e8c107e10 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.270+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c19e810 con 0x7f4e8c107e10 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.271+0000 7f4e897fa640 1 -- 172.21.15.69:0/3307571177 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4e7c045070 con 0x7f4e8c107e10 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.271+0000 7f4e897fa640 1 --2- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f4e74038f70 0x7f4e7403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.271+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8c108270 con 0x7f4e8c107e10 2024-08-02T17:37:45.104 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.271+0000 7f4e897fa640 1 -- 172.21.15.69:0/3307571177 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4e7c06c150 con 0x7f4e8c107e10 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.271+0000 7f4e8b7fe640 1 --2- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f4e74038f70 0x7f4e7403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.272+0000 7f4e8b7fe640 1 --2- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f4e74038f70 0x7f4e7403b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e800098c0 tx=0x7f4e80006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.276+0000 7f4e897fa640 1 -- 172.21.15.69:0/3307571177 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4e7c045dd0 con 0x7f4e8c107e10 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:41.447+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi069", "addr": "172.21.15.69", "target": ["mon-mgr", ""]}) -- 0x7f4e8c102d80 con 0x7f4e74038f70 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.572+0000 7f4e897fa640 1 -- 172.21.15.69:0/3307571177 <== mgr.14118 v2:172.21.15.69:6800/3376234785 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4e8c102d80 con 0x7f4e74038f70 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.575+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f4e74038f70 msgr2=0x7f4e7403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.575+0000 7f4e92383640 1 --2- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f4e74038f70 0x7f4e7403b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e800098c0 tx=0x7f4e80006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.575+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 msgr2=0x7f4e8c19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.575+0000 7f4e92383640 1 --2- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c19b350 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c031000 tx=0x7f4e7c0326e0 comp rx=0 tx=0).stop 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.575+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 shutdown_connections 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.575+0000 7f4e92383640 1 --2- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f4e74038f70 0x7f4e7403b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.575+0000 7f4e92383640 1 --2- 172.21.15.69:0/3307571177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4e8c107e10 0x7f4e8c19b350 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:45.105 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.575+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 >> 172.21.15.69:0/3307571177 conn(0x7f4e8c101ab0 msgr2=0x7f4e8c102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:45.106 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.576+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 shutdown_connections 2024-08-02T17:37:45.106 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:44.576+0000 7f4e92383640 1 -- 172.21.15.69:0/3307571177 wait complete. 2024-08-02T17:37:45.106 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mon service... 2024-08-02T17:37:45.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:45 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:45.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:45 smithi069 ceph-mon[32746]: Added host smithi069 2024-08-02T17:37:45.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:45 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:37:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-02T17:37:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.492+0000 7f796bda3640 1 Processor -- start 2024-08-02T17:37:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.492+0000 7f796bda3640 1 -- start start 2024-08-02T17:37:46.419 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.492+0000 7f796bda3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f79641077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.492+0000 7f796bda3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7964107d20 con 0x7f79641073e0 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.493+0000 7f7969b3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f79641077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.493+0000 7f7969b3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f79641077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32782/0 (socket says 172.21.15.69:32782) 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.493+0000 7f7969b3f640 1 -- 172.21.15.69:0/3257766957 learned_addr learned my addr 172.21.15.69:0/3257766957 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.493+0000 7f7969b3f640 1 -- 172.21.15.69:0/3257766957 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7964107ea0 con 0x7f79641073e0 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.493+0000 7f7969b3f640 1 --2- 172.21.15.69:0/3257766957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f79641077e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7960009820 tx=0x7f796002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1851290693448c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f7968b3d640 1 -- 172.21.15.69:0/3257766957 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7960037070 con 0x7f79641073e0 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f7968b3d640 1 -- 172.21.15.69:0/3257766957 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7960032900 con 0x7f79641073e0 2024-08-02T17:37:46.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f7968b3d640 1 -- 172.21.15.69:0/3257766957 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7960030850 con 0x7f79641073e0 2024-08-02T17:37:46.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f796bda3640 1 -- 172.21.15.69:0/3257766957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 msgr2=0x7f79641077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:46.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f796bda3640 1 --2- 172.21.15.69:0/3257766957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f79641077e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7960009820 tx=0x7f796002a270 comp rx=0 tx=0).stop 2024-08-02T17:37:46.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f796bda3640 1 -- 172.21.15.69:0/3257766957 shutdown_connections 2024-08-02T17:37:46.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f796bda3640 1 --2- 172.21.15.69:0/3257766957 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f79641077e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:46.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f796bda3640 1 -- 172.21.15.69:0/3257766957 >> 172.21.15.69:0/3257766957 conn(0x7f7964102c50 msgr2=0x7f7964105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:46.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f796bda3640 1 -- 172.21.15.69:0/3257766957 shutdown_connections 2024-08-02T17:37:46.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.494+0000 7f796bda3640 1 -- 172.21.15.69:0/3257766957 wait complete. 2024-08-02T17:37:46.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.495+0000 7f796bda3640 1 Processor -- start 2024-08-02T17:37:46.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.495+0000 7f796bda3640 1 -- start start 2024-08-02T17:37:46.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.495+0000 7f796bda3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f796419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:46.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.495+0000 7f796bda3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f796410b800 con 0x7f79641073e0 2024-08-02T17:37:46.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.495+0000 7f7969b3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f796419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:46.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.495+0000 7f7969b3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f796419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32796/0 (socket says 172.21.15.69:32796) 2024-08-02T17:37:46.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.495+0000 7f7969b3f640 1 -- 172.21.15.69:0/1913581242 learned_addr learned my addr 172.21.15.69:0/1913581242 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:46.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.496+0000 7f7969b3f640 1 -- 172.21.15.69:0/1913581242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796419b830 con 0x7f79641073e0 2024-08-02T17:37:46.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.496+0000 7f7969b3f640 1 --2- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f796419b2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7960032ce0 tx=0x7f7960032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:46.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.496+0000 7f795affd640 1 -- 172.21.15.69:0/1913581242 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7960042070 con 0x7f79641073e0 2024-08-02T17:37:46.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.496+0000 7f795affd640 1 -- 172.21.15.69:0/1913581242 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f79600308c0 con 0x7f79641073e0 2024-08-02T17:37:46.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.496+0000 7f795affd640 1 -- 172.21.15.69:0/1913581242 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7960037040 con 0x7f79641073e0 2024-08-02T17:37:46.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.496+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f796419bac0 con 0x7f79641073e0 2024-08-02T17:37:46.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.496+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796419e7b0 con 0x7f79641073e0 2024-08-02T17:37:46.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.497+0000 7f795affd640 1 -- 172.21.15.69:0/1913581242 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7960045070 con 0x7f79641073e0 2024-08-02T17:37:46.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.497+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7964107860 con 0x7f79641073e0 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.497+0000 7f795affd640 1 --2- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f7950038f70 0x7f795003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.497+0000 7f795affd640 1 -- 172.21.15.69:0/1913581242 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f796006c050 con 0x7f79641073e0 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.497+0000 7f796933e640 1 --2- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f7950038f70 0x7f795003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.498+0000 7f796933e640 1 --2- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f7950038f70 0x7f795003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f79540098c0 tx=0x7f7954006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.502+0000 7f795affd640 1 -- 172.21.15.69:0/1913581242 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f796004b170 con 0x7f79641073e0 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.704+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f7964103de0 con 0x7f7950038f70 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.708+0000 7f795affd640 1 -- 172.21.15.69:0/1913581242 <== mgr.14118 v2:172.21.15.69:6800/3376234785 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f7964103de0 con 0x7f7950038f70 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f7950038f70 msgr2=0x7f795003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 --2- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f7950038f70 0x7f795003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f79540098c0 tx=0x7f7954006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 msgr2=0x7f796419b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 --2- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f796419b2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7960032ce0 tx=0x7f7960032d10 comp rx=0 tx=0).stop 2024-08-02T17:37:46.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 shutdown_connections 2024-08-02T17:37:46.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 --2- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f7950038f70 0x7f795003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:46.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 --2- 172.21.15.69:0/1913581242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79641073e0 0x7f796419b2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:46.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 >> 172.21.15.69:0/1913581242 conn(0x7f7964102c50 msgr2=0x7f79641036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:46.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 shutdown_connections 2024-08-02T17:37:46.426 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:45.710+0000 7f796bda3640 1 -- 172.21.15.69:0/1913581242 wait complete. 2024-08-02T17:37:46.426 INFO:teuthology.orchestra.run.smithi069.stdout:Deploying unmanaged mgr service... 2024-08-02T17:37:47.021 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:46 smithi069 ceph-mon[32746]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:37:47.022 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:46 smithi069 ceph-mon[32746]: Saving service mon spec with placement count:5 2024-08-02T17:37:47.022 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:46 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:47.617 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.825+0000 7fe7e19e1640 1 Processor -- start 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.825+0000 7fe7e19e1640 1 -- start start 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.826+0000 7fe7e19e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc105fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.826+0000 7fe7e19e1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc07bdd0 con 0x7fe7dc105bb0 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.826+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc105fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.826+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc105fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32798/0 (socket says 172.21.15.69:32798) 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.826+0000 7fe7daffd640 1 -- 172.21.15.69:0/2870271398 learned_addr learned my addr 172.21.15.69:0/2870271398 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.826+0000 7fe7daffd640 1 -- 172.21.15.69:0/2870271398 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc1064f0 con 0x7fe7dc105bb0 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.827+0000 7fe7daffd640 1 --2- 172.21.15.69:0/2870271398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc105fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009820 tx=0x7fe7cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b442c41445bb3a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.827+0000 7fe7da7fc640 1 -- 172.21.15.69:0/2870271398 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7cc037070 con 0x7fe7dc105bb0 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.827+0000 7fe7da7fc640 1 -- 172.21.15.69:0/2870271398 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7cc032900 con 0x7fe7dc105bb0 2024-08-02T17:37:47.618 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.827+0000 7fe7da7fc640 1 -- 172.21.15.69:0/2870271398 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7cc030850 con 0x7fe7dc105bb0 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2870271398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 msgr2=0x7fe7dc105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 --2- 172.21.15.69:0/2870271398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc105fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009820 tx=0x7fe7cc02a270 comp rx=0 tx=0).stop 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2870271398 shutdown_connections 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 --2- 172.21.15.69:0/2870271398 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc105fb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2870271398 >> 172.21.15.69:0/2870271398 conn(0x7fe7dc101a00 msgr2=0x7fe7dc103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2870271398 shutdown_connections 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2870271398 wait complete. 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 Processor -- start 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.828+0000 7fe7e19e1640 1 -- start start 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7e19e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7e19e1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc1075f0 con 0x7fe7dc105bb0 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:47.619 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7daffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32808/0 (socket says 172.21.15.69:32808) 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7daffd640 1 -- 172.21.15.69:0/2495694038 learned_addr learned my addr 172.21.15.69:0/2495694038 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7daffd640 1 -- 172.21.15.69:0/2495694038 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc19da10 con 0x7fe7dc105bb0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7daffd640 1 --2- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc19d4d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc032ce0 tx=0x7fe7cc032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7e09df640 1 -- 172.21.15.69:0/2495694038 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7cc042070 con 0x7fe7dc105bb0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7e09df640 1 -- 172.21.15.69:0/2495694038 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe7cc0308c0 con 0x7fe7dc105bb0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc19dca0 con 0x7fe7dc105bb0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7e09df640 1 -- 172.21.15.69:0/2495694038 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7cc037040 con 0x7fe7dc105bb0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.829+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc1a0990 con 0x7fe7dc105bb0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.830+0000 7fe7e09df640 1 -- 172.21.15.69:0/2495694038 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe7cc045070 con 0x7fe7dc105bb0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.830+0000 7fe7e09df640 1 --2- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fe7c8038fc0 0x7fe7c803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.830+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7dc105fb0 con 0x7fe7dc105bb0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.830+0000 7fe7d3fff640 1 --2- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fe7c8038fc0 0x7fe7c803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:47.620 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.830+0000 7fe7e09df640 1 -- 172.21.15.69:0/2495694038 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe7cc06c180 con 0x7fe7dc105bb0 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.831+0000 7fe7d3fff640 1 --2- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fe7c8038fc0 0x7fe7c803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40098c0 tx=0x7fe7c4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:46.835+0000 7fe7e09df640 1 -- 172.21.15.69:0/2495694038 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe7cc045360 con 0x7fe7dc105bb0 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.020+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe7dc062360 con 0x7fe7c8038fc0 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.025+0000 7fe7e09df640 1 -- 172.21.15.69:0/2495694038 <== mgr.14118 v2:172.21.15.69:6800/3376234785 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe7dc062360 con 0x7fe7c8038fc0 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fe7c8038fc0 msgr2=0x7fe7c803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 --2- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fe7c8038fc0 0x7fe7c803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40098c0 tx=0x7fe7c4006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 msgr2=0x7fe7dc19d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 --2- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc19d4d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc032ce0 tx=0x7fe7cc032d10 comp rx=0 tx=0).stop 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 shutdown_connections 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 --2- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fe7c8038fc0 0x7fe7c803b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 --2- 172.21.15.69:0/2495694038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe7dc105bb0 0x7fe7dc19d4d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 >> 172.21.15.69:0/2495694038 conn(0x7fe7dc101a00 msgr2=0x7fe7dc1917d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:47.621 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 shutdown_connections 2024-08-02T17:37:47.622 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:47.027+0000 7fe7e19e1640 1 -- 172.21.15.69:0/2495694038 wait complete. 2024-08-02T17:37:48.248 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:48 smithi069 ceph-mon[32746]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:37:48.248 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:48 smithi069 ceph-mon[32746]: Saving service mgr spec with placement count:2 2024-08-02T17:37:48.248 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:48 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.061+0000 7f9ab872d640 1 Processor -- start 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.061+0000 7f9ab872d640 1 -- start start 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.061+0000 7f9ab872d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab0107c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.061+0000 7f9ab872d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01081d0 con 0x7f9ab0107800 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.061+0000 7f9ab64c9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab0107c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.061+0000 7f9ab64c9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab0107c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32822/0 (socket says 172.21.15.69:32822) 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.061+0000 7f9ab64c9640 1 -- 172.21.15.69:0/79229751 learned_addr learned my addr 172.21.15.69:0/79229751 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.062+0000 7f9ab64c9640 1 -- 172.21.15.69:0/79229751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab0108350 con 0x7f9ab0107800 2024-08-02T17:37:49.261 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.062+0000 7f9ab64c9640 1 --2- 172.21.15.69:0/79229751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab0107c00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009820 tx=0x7f9aac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dea4787e878c45 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.062+0000 7f9ab54c7640 1 -- 172.21.15.69:0/79229751 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac037070 con 0x7f9ab0107800 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab54c7640 1 -- 172.21.15.69:0/79229751 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9aac032900 con 0x7f9ab0107800 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab54c7640 1 -- 172.21.15.69:0/79229751 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac030850 con 0x7f9ab0107800 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab872d640 1 -- 172.21.15.69:0/79229751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 msgr2=0x7f9ab0107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab872d640 1 --2- 172.21.15.69:0/79229751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab0107c00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009820 tx=0x7f9aac02a270 comp rx=0 tx=0).stop 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab872d640 1 -- 172.21.15.69:0/79229751 shutdown_connections 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab872d640 1 --2- 172.21.15.69:0/79229751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab0107c00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab872d640 1 -- 172.21.15.69:0/79229751 >> 172.21.15.69:0/79229751 conn(0x7f9ab01011a0 msgr2=0x7f9ab01035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab872d640 1 -- 172.21.15.69:0/79229751 shutdown_connections 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.063+0000 7f9ab872d640 1 -- 172.21.15.69:0/79229751 wait complete. 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.064+0000 7f9ab872d640 1 Processor -- start 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.064+0000 7f9ab872d640 1 -- start start 2024-08-02T17:37:49.262 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.064+0000 7f9ab872d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.064+0000 7f9ab872d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab010a780 con 0x7f9ab0107800 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9ab64c9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9ab64c9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32832/0 (socket says 172.21.15.69:32832) 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9ab64c9640 1 -- 172.21.15.69:0/3814321672 learned_addr learned my addr 172.21.15.69:0/3814321672 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9ab64c9640 1 -- 172.21.15.69:0/3814321672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab019b8e0 con 0x7f9ab0107800 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9ab64c9640 1 --2- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab019b3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0097f0 tx=0x7f9aac030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9aa77fe640 1 -- 172.21.15.69:0/3814321672 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac037040 con 0x7f9ab0107800 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab019bb70 con 0x7f9ab0107800 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9aa77fe640 1 -- 172.21.15.69:0/3814321672 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9aac040070 con 0x7f9ab0107800 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9aa77fe640 1 -- 172.21.15.69:0/3814321672 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aac03a390 con 0x7f9ab0107800 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.065+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab019bf90 con 0x7f9ab0107800 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.066+0000 7f9aa77fe640 1 -- 172.21.15.69:0/3814321672 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9aac03a610 con 0x7f9ab0107800 2024-08-02T17:37:49.263 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.066+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab0107c80 con 0x7f9ab0107800 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.067+0000 7f9aa77fe640 1 --2- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f9a9c038f70 0x7f9a9c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.067+0000 7f9aa77fe640 1 -- 172.21.15.69:0/3814321672 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9aac06c480 con 0x7f9ab0107800 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.067+0000 7f9ab5cc8640 1 --2- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f9a9c038f70 0x7f9a9c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.067+0000 7f9ab5cc8640 1 --2- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f9a9c038f70 0x7f9a9c03b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00098c0 tx=0x7f9aa0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.075+0000 7f9aa77fe640 1 -- 172.21.15.69:0/3814321672 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9aac042e30 con 0x7f9ab0107800 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.246+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f9ab018fb50 con 0x7f9ab0107800 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.251+0000 7f9aa77fe640 1 -- 172.21.15.69:0/3814321672 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f9aac043760 con 0x7f9ab0107800 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f9a9c038f70 msgr2=0x7f9a9c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 --2- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f9a9c038f70 0x7f9a9c03b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00098c0 tx=0x7f9aa0006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 msgr2=0x7f9ab019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 --2- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab019b3a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0097f0 tx=0x7f9aac030d80 comp rx=0 tx=0).stop 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 shutdown_connections 2024-08-02T17:37:49.264 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 --2- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7f9a9c038f70 0x7f9a9c03b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:49.265 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 --2- 172.21.15.69:0/3814321672 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ab0107800 0x7f9ab019b3a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:49.265 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 >> 172.21.15.69:0/3814321672 conn(0x7f9ab01011a0 msgr2=0x7f9ab0101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:49.265 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 shutdown_connections 2024-08-02T17:37:49.265 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:48.254+0000 7f9ab872d640 1 -- 172.21.15.69:0/3814321672 wait complete. 2024-08-02T17:37:49.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:49 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3814321672' entity='client.admin' 2024-08-02T17:37:49.573 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:49 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.651+0000 7efe3779a640 1 Processor -- start 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.651+0000 7efe3779a640 1 -- start start 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.651+0000 7efe3779a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe30105a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.651+0000 7efe3779a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe30106060 con 0x7efe30105690 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.651+0000 7efe35536640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe30105a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.651+0000 7efe35536640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe30105a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32844/0 (socket says 172.21.15.69:32844) 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.651+0000 7efe35536640 1 -- 172.21.15.69:0/2688073087 learned_addr learned my addr 172.21.15.69:0/2688073087 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.651+0000 7efe35536640 1 -- 172.21.15.69:0/2688073087 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe30108670 con 0x7efe30105690 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.652+0000 7efe35536640 1 --2- 172.21.15.69:0/2688073087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe30105a90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009820 tx=0x7efe2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83174450ccdb512 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.652+0000 7efe27fff640 1 -- 172.21.15.69:0/2688073087 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c037070 con 0x7efe30105690 2024-08-02T17:37:50.384 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.652+0000 7efe27fff640 1 -- 172.21.15.69:0/2688073087 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe2c032920 con 0x7efe30105690 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.652+0000 7efe27fff640 1 -- 172.21.15.69:0/2688073087 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c030810 con 0x7efe30105690 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.653+0000 7efe3779a640 1 -- 172.21.15.69:0/2688073087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 msgr2=0x7efe30105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.653+0000 7efe3779a640 1 --2- 172.21.15.69:0/2688073087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe30105a90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009820 tx=0x7efe2c02a270 comp rx=0 tx=0).stop 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.653+0000 7efe3779a640 1 -- 172.21.15.69:0/2688073087 shutdown_connections 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.653+0000 7efe3779a640 1 --2- 172.21.15.69:0/2688073087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe30105a90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.653+0000 7efe3779a640 1 -- 172.21.15.69:0/2688073087 >> 172.21.15.69:0/2688073087 conn(0x7efe30101200 msgr2=0x7efe30103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.653+0000 7efe3779a640 1 -- 172.21.15.69:0/2688073087 shutdown_connections 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.653+0000 7efe3779a640 1 -- 172.21.15.69:0/2688073087 wait complete. 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.653+0000 7efe3779a640 1 Processor -- start 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe3779a640 1 -- start start 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe3779a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe3019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:50.385 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe3779a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efe3010b8a0 con 0x7efe30105690 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe35536640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe3019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe35536640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe3019b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32854/0 (socket says 172.21.15.69:32854) 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe35536640 1 -- 172.21.15.69:0/3548400525 learned_addr learned my addr 172.21.15.69:0/3548400525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe35536640 1 -- 172.21.15.69:0/3548400525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3019b8d0 con 0x7efe30105690 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe35536640 1 --2- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe3019b390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097f0 tx=0x7efe2c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe267fc640 1 -- 172.21.15.69:0/3548400525 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c037040 con 0x7efe30105690 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe267fc640 1 -- 172.21.15.69:0/3548400525 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe2c030cc0 con 0x7efe30105690 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe267fc640 1 -- 172.21.15.69:0/3548400525 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c03a640 con 0x7efe30105690 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3019bb60 con 0x7efe30105690 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.654+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe3019e850 con 0x7efe30105690 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.655+0000 7efe267fc640 1 -- 172.21.15.69:0/3548400525 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efe2c03b3f0 con 0x7efe30105690 2024-08-02T17:37:50.386 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.655+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe30105b10 con 0x7efe30105690 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.655+0000 7efe267fc640 1 --2- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7efe1c038fc0 0x7efe1c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.655+0000 7efe267fc640 1 -- 172.21.15.69:0/3548400525 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efe2c06c4b0 con 0x7efe30105690 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.656+0000 7efe34d35640 1 --2- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7efe1c038fc0 0x7efe1c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.656+0000 7efe34d35640 1 --2- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7efe1c038fc0 0x7efe1c03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efe200098c0 tx=0x7efe20006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.660+0000 7efe267fc640 1 -- 172.21.15.69:0/3548400525 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efe2c042e10 con 0x7efe30105690 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.853+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7efe3019ea20 con 0x7efe30105690 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.856+0000 7efe267fc640 1 -- 172.21.15.69:0/3548400525 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7efe2c03ba40 con 0x7efe30105690 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.858+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7efe1c038fc0 msgr2=0x7efe1c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.858+0000 7efe3779a640 1 --2- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7efe1c038fc0 0x7efe1c03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efe200098c0 tx=0x7efe20006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.858+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 msgr2=0x7efe3019b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.858+0000 7efe3779a640 1 --2- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe3019b390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097f0 tx=0x7efe2c0327b0 comp rx=0 tx=0).stop 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.858+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 shutdown_connections 2024-08-02T17:37:50.387 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.858+0000 7efe3779a640 1 --2- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7efe1c038fc0 0x7efe1c03b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:50.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.858+0000 7efe3779a640 1 --2- 172.21.15.69:0/3548400525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efe30105690 0x7efe3019b390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:50.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.858+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 >> 172.21.15.69:0/3548400525 conn(0x7efe30101200 msgr2=0x7efe30101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:50.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.859+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 shutdown_connections 2024-08-02T17:37:50.388 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:49.859+0000 7efe3779a640 1 -- 172.21.15.69:0/3548400525 wait complete. 2024-08-02T17:37:50.388 INFO:teuthology.orchestra.run.smithi069.stdout:Enabling the dashboard module... 2024-08-02T17:37:51.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:50 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:51.035 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:50 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3548400525' entity='client.admin' 2024-08-02T17:37:52.018 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:51 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3793454028' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-08-02T17:37:52.018 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:51 smithi069 ceph-mon[32746]: from='mgr.14118 172.21.15.69:0/3013146913' entity='mgr.x' 2024-08-02T17:37:52.019 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ignoring --setuser ceph since I am not root 2024-08-02T17:37:52.019 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ignoring --setgroup ceph since I am not root 2024-08-02T17:37:52.019 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:51.937+0000 7f7453d2d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-02T17:37:52.336 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:52 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:52.088+0000 7f7453d2d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:37:52.336 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:52 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:52.154+0000 7f7453d2d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-02T17:37:52.343 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.797+0000 7fc753e3f640 1 Processor -- start 2024-08-02T17:37:52.343 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.797+0000 7fc753e3f640 1 -- start start 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.797+0000 7fc753e3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c1077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.797+0000 7fc753e3f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c107d20 con 0x7fc74c1073e0 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.798+0000 7fc751bdb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c1077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.798+0000 7fc751bdb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c1077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32864/0 (socket says 172.21.15.69:32864) 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.798+0000 7fc751bdb640 1 -- 172.21.15.69:0/2361968023 learned_addr learned my addr 172.21.15.69:0/2361968023 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.798+0000 7fc751bdb640 1 -- 172.21.15.69:0/2361968023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c107ea0 con 0x7fc74c1073e0 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.798+0000 7fc751bdb640 1 --2- 172.21.15.69:0/2361968023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c1077e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc748009820 tx=0x7fc74802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6c419b1cdff572 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc750bd9640 1 -- 172.21.15.69:0/2361968023 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc748037070 con 0x7fc74c1073e0 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc750bd9640 1 -- 172.21.15.69:0/2361968023 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748032920 con 0x7fc74c1073e0 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc750bd9640 1 -- 172.21.15.69:0/2361968023 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc748030810 con 0x7fc74c1073e0 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc753e3f640 1 -- 172.21.15.69:0/2361968023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 msgr2=0x7fc74c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:52.344 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc753e3f640 1 --2- 172.21.15.69:0/2361968023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c1077e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc748009820 tx=0x7fc74802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc753e3f640 1 -- 172.21.15.69:0/2361968023 shutdown_connections 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc753e3f640 1 --2- 172.21.15.69:0/2361968023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c1077e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc753e3f640 1 -- 172.21.15.69:0/2361968023 >> 172.21.15.69:0/2361968023 conn(0x7fc74c102c50 msgr2=0x7fc74c105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc753e3f640 1 -- 172.21.15.69:0/2361968023 shutdown_connections 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.799+0000 7fc753e3f640 1 -- 172.21.15.69:0/2361968023 wait complete. 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc753e3f640 1 Processor -- start 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc753e3f640 1 -- start start 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc753e3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc753e3f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c10b800 con 0x7fc74c1073e0 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc751bdb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc751bdb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c19b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32870/0 (socket says 172.21.15.69:32870) 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc751bdb640 1 -- 172.21.15.69:0/3793454028 learned_addr learned my addr 172.21.15.69:0/3793454028 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc751bdb640 1 -- 172.21.15.69:0/3793454028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c19b840 con 0x7fc74c1073e0 2024-08-02T17:37:52.345 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.800+0000 7fc751bdb640 1 --2- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c19b300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc7480097f0 tx=0x7fc7480327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.801+0000 7fc742ffd640 1 -- 172.21.15.69:0/3793454028 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc748037040 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.801+0000 7fc742ffd640 1 -- 172.21.15.69:0/3793454028 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc748030cc0 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.801+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74c19bad0 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.801+0000 7fc742ffd640 1 -- 172.21.15.69:0/3793454028 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc74803a6b0 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.801+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc74c19e7c0 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.802+0000 7fc742ffd640 1 -- 172.21.15.69:0/3793454028 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc74803b3f0 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.802+0000 7fc742ffd640 1 --2- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fc738038f70 0x7fc73803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.802+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc74c107860 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.802+0000 7fc742ffd640 1 -- 172.21.15.69:0/3793454028 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc74806c560 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.802+0000 7fc7513da640 1 --2- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fc738038f70 0x7fc73803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.802+0000 7fc7513da640 1 --2- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fc738038f70 0x7fc73803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0098c0 tx=0x7fc73c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:50.807+0000 7fc742ffd640 1 -- 172.21.15.69:0/3793454028 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc7480371e0 con 0x7fc74c1073e0 2024-08-02T17:37:52.346 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.034+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc74c19eda0 con 0x7fc74c1073e0 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.772+0000 7fc742ffd640 1 -- 172.21.15.69:0/3793454028 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fc74806bef0 con 0x7fc74c1073e0 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.774+0000 7fc742ffd640 1 -- 172.21.15.69:0/3793454028 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fc74803baf0 con 0x7fc74c1073e0 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fc738038f70 msgr2=0x7fc73803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 --2- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fc738038f70 0x7fc73803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0098c0 tx=0x7fc73c006db0 comp rx=0 tx=0).stop 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 msgr2=0x7fc74c19b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 --2- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c19b300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc7480097f0 tx=0x7fc7480327b0 comp rx=0 tx=0).stop 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 shutdown_connections 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 --2- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fc738038f70 0x7fc73803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 --2- 172.21.15.69:0/3793454028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc74c1073e0 0x7fc74c19b300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 >> 172.21.15.69:0/3793454028 conn(0x7fc74c102c50 msgr2=0x7fc74c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 shutdown_connections 2024-08-02T17:37:52.347 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:51.775+0000 7fc753e3f640 1 -- 172.21.15.69:0/3793454028 wait complete. 2024-08-02T17:37:52.599 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:52 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:52.400+0000 7f7453d2d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-02T17:37:52.599 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:52 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:52.459+0000 7f7453d2d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-02T17:37:52.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:52 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3793454028' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-02T17:37:52.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:52 smithi069 ceph-mon[32746]: mgrmap e9: x(active, since 17s) 2024-08-02T17:37:53.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:53.424+0000 7f7453d2d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-02T17:37:53.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:53.539+0000 7f7453d2d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-02T17:37:53.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:53.597+0000 7f7453d2d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-02T17:37:53.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-08-02T17:37:53.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-02T17:37:53.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-02T17:37:53.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-02T17:37:53.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-02T17:37:53.659 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.766+0000 7fdc733dc640 1 Processor -- start 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.767+0000 7fdc733dc640 1 -- start start 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.767+0000 7fdc733dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c1077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.767+0000 7fdc733dc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c107da0 con 0x7fdc6c1073d0 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.767+0000 7fdc71178640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c1077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.767+0000 7fdc71178640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c1077d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32888/0 (socket says 172.21.15.69:32888) 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.767+0000 7fdc71178640 1 -- 172.21.15.69:0/557621264 learned_addr learned my addr 172.21.15.69:0/557621264 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.767+0000 7fdc71178640 1 -- 172.21.15.69:0/557621264 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c108530 con 0x7fdc6c1073d0 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.768+0000 7fdc71178640 1 --2- 172.21.15.69:0/557621264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c1077d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009820 tx=0x7fdc6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1357373a38793be7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:53.660 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.768+0000 7fdc63fff640 1 -- 172.21.15.69:0/557621264 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc68037070 con 0x7fdc6c1073d0 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.768+0000 7fdc63fff640 1 -- 172.21.15.69:0/557621264 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc68032920 con 0x7fdc6c1073d0 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.768+0000 7fdc63fff640 1 -- 172.21.15.69:0/557621264 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc68030810 con 0x7fdc6c1073d0 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 -- 172.21.15.69:0/557621264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 msgr2=0x7fdc6c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 --2- 172.21.15.69:0/557621264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c1077d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009820 tx=0x7fdc6802a270 comp rx=0 tx=0).stop 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 -- 172.21.15.69:0/557621264 shutdown_connections 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 --2- 172.21.15.69:0/557621264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c1077d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 -- 172.21.15.69:0/557621264 >> 172.21.15.69:0/557621264 conn(0x7fdc6c102b80 msgr2=0x7fdc6c104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 -- 172.21.15.69:0/557621264 shutdown_connections 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 -- 172.21.15.69:0/557621264 wait complete. 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 Processor -- start 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.769+0000 7fdc733dc640 1 -- start start 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc733dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc733dc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c10b8e0 con 0x7fdc6c1073d0 2024-08-02T17:37:53.661 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc71178640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc71178640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32896/0 (socket says 172.21.15.69:32896) 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc71178640 1 -- 172.21.15.69:0/2773407411 learned_addr learned my addr 172.21.15.69:0/2773407411 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc71178640 1 -- 172.21.15.69:0/2773407411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c19daa0 con 0x7fdc6c1073d0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc71178640 1 --2- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c19d560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc68030560 tx=0x7fdc68032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc627fc640 1 -- 172.21.15.69:0/2773407411 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc68037040 con 0x7fdc6c1073d0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc627fc640 1 -- 172.21.15.69:0/2773407411 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc68040070 con 0x7fdc6c1073d0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6c19dd30 con 0x7fdc6c1073d0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc627fc640 1 -- 172.21.15.69:0/2773407411 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc6803a740 con 0x7fdc6c1073d0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.770+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc6c1a0a20 con 0x7fdc6c1073d0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.771+0000 7fdc627fc640 1 -- 172.21.15.69:0/2773407411 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fdc6803a9b0 con 0x7fdc6c1073d0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.771+0000 7fdc627fc640 1 --2- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc58038fc0 0x7fdc5803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.771+0000 7fdc627fc640 1 -- 172.21.15.69:0/2773407411 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc6806c5f0 con 0x7fdc6c1073d0 2024-08-02T17:37:53.662 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.771+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc6c10ba60 con 0x7fdc6c1073d0 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.771+0000 7fdc70977640 1 -- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc58038fc0 msgr2=0x7fdc5803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3376234785 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.772+0000 7fdc70977640 1 --2- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc58038fc0 0x7fdc5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.776+0000 7fdc627fc640 1 -- 172.21.15.69:0/2773407411 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc68042e30 con 0x7fdc6c1073d0 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.973+0000 7fdc70977640 1 -- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc58038fc0 msgr2=0x7fdc5803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3376234785 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.973+0000 7fdc70977640 1 --2- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc58038fc0 0x7fdc5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.997+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fdc6c1a0bf0 con 0x7fdc6c1073d0 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:52.998+0000 7fdc627fc640 1 -- 172.21.15.69:0/2773407411 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fdc680328f0 con 0x7fdc6c1073d0 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.001+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc58038fc0 msgr2=0x7fdc5803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.001+0000 7fdc733dc640 1 --2- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc58038fc0 0x7fdc5803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.001+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 msgr2=0x7fdc6c19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.001+0000 7fdc733dc640 1 --2- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c19d560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdc68030560 tx=0x7fdc68032670 comp rx=0 tx=0).stop 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.001+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 shutdown_connections 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.001+0000 7fdc733dc640 1 --2- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fdc58038fc0 0x7fdc5803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:53.663 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.001+0000 7fdc733dc640 1 --2- 172.21.15.69:0/2773407411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1073d0 0x7fdc6c19d560 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:53.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.001+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 >> 172.21.15.69:0/2773407411 conn(0x7fdc6c102b80 msgr2=0x7fdc6c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:53.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.002+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 shutdown_connections 2024-08-02T17:37:53.664 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:53.002+0000 7fdc733dc640 1 -- 172.21.15.69:0/2773407411 wait complete. 2024-08-02T17:37:53.664 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for the mgr to restart... 2024-08-02T17:37:53.664 INFO:teuthology.orchestra.run.smithi069.stdout:Waiting for mgr epoch 9... 2024-08-02T17:37:53.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:53 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2773407411' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-08-02T17:37:53.949 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:53.815+0000 7f7453d2d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-02T17:37:53.949 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:53.875+0000 7f7453d2d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-02T17:37:53.949 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:53.947+0000 7f7453d2d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-02T17:37:54.279 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:54 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:54.098+0000 7f7453d2d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:37:54.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:54 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:54.511+0000 7f7453d2d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-02T17:37:55.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:54 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:54.819+0000 7f7453d2d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-02T17:37:55.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:54 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:54.886+0000 7f7453d2d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-02T17:37:55.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:55.032+0000 7f7453d2d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-02T17:37:55.552 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:55.341+0000 7f7453d2d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-02T17:37:55.552 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:55.405+0000 7f7453d2d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-02T17:37:55.822 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:55.597+0000 7f7453d2d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-02T17:37:55.823 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:55.754+0000 7f7453d2d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-02T17:37:55.823 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:55.821+0000 7f7453d2d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-02T17:37:56.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:55.907+0000 7f7453d2d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-02T17:37:56.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:55.970+0000 7f7453d2d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-02T17:37:56.833 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:56.527+0000 7f7453d2d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-02T17:37:56.833 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:56.682+0000 7f7453d2d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: Active manager daemon x restarted 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: Activating manager daemon x 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: osdmap e3: 0 total, 0 up, 0 in 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: mgrmap e10: x(active, starting, since 0.00339914s) 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: Manager daemon x is now available 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:37:56.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-02T17:37:56.835 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-02T17:37:58.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:57.840+0000 7f73b99a9640 -1 log_channel(cephadm) log [ERR] : [02/Aug/2024:17:37:57] ENGINE Error in HTTPServer.serve 2024-08-02T17:37:58.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: Traceback (most recent call last): 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._connections.run(self.expiration_interval) 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._run(expiration_interval) 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: new_conn = self._from_server_socket(self.server.socket) 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: s = self.context.wrap_socket( 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: return self.sslsocket_class._create( 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-02T17:37:58.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self.do_handshake() 2024-08-02T17:37:58.110 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-02T17:37:58.110 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._sslobj.do_handshake() 2024-08-02T17:37:58.110 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-02T17:37:58.110 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:37:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:37:58.419 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout { 2024-08-02T17:37:58.419 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout } 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.071+0000 7fa170dbe640 1 Processor -- start 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.071+0000 7fa170dbe640 1 -- start start 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.071+0000 7fa170dbe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c1066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.071+0000 7fa170dbe640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c106c80 con 0x7fa16c1062b0 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.072+0000 7fa16a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c1066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.072+0000 7fa16a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c1066b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32900/0 (socket says 172.21.15.69:32900) 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.072+0000 7fa16a59c640 1 -- 172.21.15.69:0/1087794431 learned_addr learned my addr 172.21.15.69:0/1087794431 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.072+0000 7fa16a59c640 1 -- 172.21.15.69:0/1087794431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c1074b0 con 0x7fa16c1062b0 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.073+0000 7fa16a59c640 1 --2- 172.21.15.69:0/1087794431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c1066b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009820 tx=0x7fa15c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587c49ebadc28395 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:58.420 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.073+0000 7fa16959a640 1 -- 172.21.15.69:0/1087794431 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa15c037070 con 0x7fa16c1062b0 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.073+0000 7fa16959a640 1 -- 172.21.15.69:0/1087794431 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa15c032920 con 0x7fa16c1062b0 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.073+0000 7fa16959a640 1 -- 172.21.15.69:0/1087794431 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa15c030810 con 0x7fa16c1062b0 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.073+0000 7fa170dbe640 1 -- 172.21.15.69:0/1087794431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 msgr2=0x7fa16c1066b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.073+0000 7fa170dbe640 1 --2- 172.21.15.69:0/1087794431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c1066b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009820 tx=0x7fa15c02a270 comp rx=0 tx=0).stop 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.074+0000 7fa170dbe640 1 -- 172.21.15.69:0/1087794431 shutdown_connections 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.074+0000 7fa170dbe640 1 --2- 172.21.15.69:0/1087794431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c1066b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.074+0000 7fa170dbe640 1 -- 172.21.15.69:0/1087794431 >> 172.21.15.69:0/1087794431 conn(0x7fa16c101b00 msgr2=0x7fa16c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.074+0000 7fa170dbe640 1 -- 172.21.15.69:0/1087794431 shutdown_connections 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.074+0000 7fa170dbe640 1 -- 172.21.15.69:0/1087794431 wait complete. 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.074+0000 7fa170dbe640 1 Processor -- start 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.074+0000 7fa170dbe640 1 -- start start 2024-08-02T17:37:58.421 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa170dbe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa170dbe640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c10a860 con 0x7fa16c1062b0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa16a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa16a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32906/0 (socket says 172.21.15.69:32906) 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa16a59c640 1 -- 172.21.15.69:0/4260792056 learned_addr learned my addr 172.21.15.69:0/4260792056 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa16a59c640 1 -- 172.21.15.69:0/4260792056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c19b8f0 con 0x7fa16c1062b0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa16a59c640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c19b3b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa15c030560 tx=0x7fa15c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa15c037040 con 0x7fa16c1062b0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16c19bb80 con 0x7fa16c1062b0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa15c040070 con 0x7fa16c1062b0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa15c03a740 con 0x7fa16c1062b0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.075+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa16c19e870 con 0x7fa16c1062b0 2024-08-02T17:37:58.422 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.076+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa15c03a9b0 con 0x7fa16c1062b0 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.077+0000 7fa1637fe640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 0x7fa15803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.077+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 --> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa15803bb90 con 0x7fa158038fc0 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.077+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa15c06c640 con 0x7fa16c1062b0 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.077+0000 7fa169d9b640 1 -- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 msgr2=0x7fa15803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3376234785 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.077+0000 7fa169d9b640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 0x7fa15803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.278+0000 7fa169d9b640 1 -- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 msgr2=0x7fa15803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3376234785 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.278+0000 7fa169d9b640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 0x7fa15803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.679+0000 7fa169d9b640 1 -- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 msgr2=0x7fa15803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3376234785 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:54.679+0000 7fa169d9b640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 0x7fa15803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:55.480+0000 7fa169d9b640 1 -- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 msgr2=0x7fa15803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/3376234785 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:55.480+0000 7fa169d9b640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 0x7fa15803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:56.686+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fa15c06b0f0 con 0x7fa16c1062b0 2024-08-02T17:37:58.423 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:56.686+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 msgr2=0x7fa15803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:56.686+0000 7fa1637fe640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 0x7fa15803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.688+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fa15c03aca0 con 0x7fa16c1062b0 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.688+0000 7fa1637fe640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa15803ca80 0x7fa15803ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.688+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa15803bb90 con 0x7fa15803ca80 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.688+0000 7fa169d9b640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa15803ca80 0x7fa15803ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.689+0000 7fa169d9b640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa15803ca80 0x7fa15803ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa154003980 tx=0x7fa154006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.689+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fa15803bb90 con 0x7fa15803ca80 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.696+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa16c106730 con 0x7fa15803ca80 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa1637fe640 1 -- 172.21.15.69:0/4260792056 <== mgr.14150 v2:172.21.15.69:6800/452236630 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa16c106730 con 0x7fa15803ca80 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa15803ca80 msgr2=0x7fa15803ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa15803ca80 0x7fa15803ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa154003980 tx=0x7fa154006e10 comp rx=0 tx=0).stop 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 msgr2=0x7fa16c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c19b3b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa15c030560 tx=0x7fa15c032670 comp rx=0 tx=0).stop 2024-08-02T17:37:58.424 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 shutdown_connections 2024-08-02T17:37:58.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa15803ca80 0x7fa15803ee70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:58.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:6800/3376234785,v1:172.21.15.69:6801/3376234785] conn(0x7fa158038fc0 0x7fa15803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:58.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 --2- 172.21.15.69:0/4260792056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa16c1062b0 0x7fa16c19b3b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:37:58.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 >> 172.21.15.69:0/4260792056 conn(0x7fa16c101b00 msgr2=0x7fa16c102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:37:58.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 shutdown_connections 2024-08-02T17:37:58.425 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:57.697+0000 7fa170dbe640 1 -- 172.21.15.69:0/4260792056 wait complete. 2024-08-02T17:37:58.425 INFO:teuthology.orchestra.run.smithi069.stdout:mgr epoch 9 is available 2024-08-02T17:37:58.425 INFO:teuthology.orchestra.run.smithi069.stdout:Generating a dashboard self-signed certificate... 2024-08-02T17:37:58.796 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:58 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:57] ENGINE Bus STARTING 2024-08-02T17:37:58.796 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:58 smithi069 ceph-mon[32746]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-02T17:37:58.796 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:58 smithi069 ceph-mon[32746]: mgrmap e11: x(active, since 1.00634s) 2024-08-02T17:37:58.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:58 smithi069 ceph-mon[32746]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-02T17:37:58.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:58 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:57] ENGINE Serving on http://172.21.15.69:8765 2024-08-02T17:37:58.797 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:58 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:59 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:57] ENGINE Serving on https://172.21.15.69:7150 2024-08-02T17:38:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:59 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:57] ENGINE Bus STARTED 2024-08-02T17:38:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:59 smithi069 ceph-mon[32746]: [02/Aug/2024:17:37:57] ENGINE Error in HTTPServer.serve 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: Traceback (most recent call last): 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._connections.run(self.expiration_interval) 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._run(expiration_interval) 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: s = self.context.wrap_socket( 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: return self.sslsocket_class._create( 2024-08-02T17:38:00.109 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-02T17:38:00.110 INFO:journalctl@ceph.mon.a.smithi069.stdout: self.do_handshake() 2024-08-02T17:38:00.110 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-02T17:38:00.110 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._sslobj.do_handshake() 2024-08-02T17:38:00.110 INFO:journalctl@ceph.mon.a.smithi069.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-02T17:38:00.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:00.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:59 smithi069 ceph-mon[32746]: mgrmap e12: x(active, since 2s) 2024-08-02T17:38:00.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:00.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:37:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:00.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-08-02T17:38:00.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.874+0000 7fc0da319640 1 Processor -- start 2024-08-02T17:38:00.503 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.874+0000 7fc0da319640 1 -- start start 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.874+0000 7fc0da319640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d4107770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.874+0000 7fc0da319640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4107cb0 con 0x7fc0d4107370 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.875+0000 7fc0d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d4107770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.875+0000 7fc0d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d4107770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46950/0 (socket says 172.21.15.69:46950) 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.875+0000 7fc0d3fff640 1 -- 172.21.15.69:0/3594954853 learned_addr learned my addr 172.21.15.69:0/3594954853 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.875+0000 7fc0d3fff640 1 -- 172.21.15.69:0/3594954853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d4107e30 con 0x7fc0d4107370 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.876+0000 7fc0d3fff640 1 --2- 172.21.15.69:0/3594954853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d4107770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009820 tx=0x7fc0c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801682288b1ea823 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.876+0000 7fc0d2ffd640 1 -- 172.21.15.69:0/3594954853 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0c4037070 con 0x7fc0d4107370 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.876+0000 7fc0d2ffd640 1 -- 172.21.15.69:0/3594954853 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c4032920 con 0x7fc0d4107370 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.876+0000 7fc0d2ffd640 1 -- 172.21.15.69:0/3594954853 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0c4030810 con 0x7fc0d4107370 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.876+0000 7fc0da319640 1 -- 172.21.15.69:0/3594954853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 msgr2=0x7fc0d4107770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.876+0000 7fc0da319640 1 --2- 172.21.15.69:0/3594954853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d4107770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009820 tx=0x7fc0c402a270 comp rx=0 tx=0).stop 2024-08-02T17:38:00.504 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.877+0000 7fc0da319640 1 -- 172.21.15.69:0/3594954853 shutdown_connections 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.877+0000 7fc0da319640 1 --2- 172.21.15.69:0/3594954853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d4107770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.877+0000 7fc0da319640 1 -- 172.21.15.69:0/3594954853 >> 172.21.15.69:0/3594954853 conn(0x7fc0d4102ba0 msgr2=0x7fc0d4104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.877+0000 7fc0da319640 1 -- 172.21.15.69:0/3594954853 shutdown_connections 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.877+0000 7fc0da319640 1 -- 172.21.15.69:0/3594954853 wait complete. 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.877+0000 7fc0da319640 1 Processor -- start 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.877+0000 7fc0da319640 1 -- start start 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0da319640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0da319640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d410b7e0 con 0x7fc0d4107370 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46952/0 (socket says 172.21.15.69:46952) 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0d3fff640 1 -- 172.21.15.69:0/1382085011 learned_addr learned my addr 172.21.15.69:0/1382085011 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0d3fff640 1 -- 172.21.15.69:0/1382085011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d419b860 con 0x7fc0d4107370 2024-08-02T17:38:00.505 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0d3fff640 1 --2- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d419b320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4030380 tx=0x7fc0c4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0d17fa640 1 -- 172.21.15.69:0/1382085011 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0c4037040 con 0x7fc0d4107370 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0d17fa640 1 -- 172.21.15.69:0/1382085011 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c4040070 con 0x7fc0d4107370 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0d17fa640 1 -- 172.21.15.69:0/1382085011 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0c403a7b0 con 0x7fc0d4107370 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d419baf0 con 0x7fc0d4107370 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.878+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d419e7e0 con 0x7fc0d4107370 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.879+0000 7fc0d17fa640 1 -- 172.21.15.69:0/1382085011 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fc0c403a9e0 con 0x7fc0d4107370 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.879+0000 7fc0d17fa640 1 --2- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc0c0038f00 0x7fc0c003b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.879+0000 7fc0d17fa640 1 -- 172.21.15.69:0/1382085011 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc0c406c560 con 0x7fc0d4107370 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.879+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d41077f0 con 0x7fc0d4107370 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.880+0000 7fc0d37fe640 1 --2- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc0c0038f00 0x7fc0c003b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:00.506 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.880+0000 7fc0d37fe640 1 --2- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc0c0038f00 0x7fc0c003b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0098c0 tx=0x7fc0bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:58.888+0000 7fc0d17fa640 1 -- 172.21.15.69:0/1382085011 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc0c4044c70 con 0x7fc0d4107370 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.074+0000 7fc0d17fa640 1 -- 172.21.15.69:0/1382085011 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fc0c403ad70 con 0x7fc0d4107370 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.080+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fc0d4103e20 con 0x7fc0c0038f00 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.149+0000 7fc0d17fa640 1 -- 172.21.15.69:0/1382085011 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fc0d4103e20 con 0x7fc0c0038f00 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc0c0038f00 msgr2=0x7fc0c003b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 --2- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc0c0038f00 0x7fc0c003b3c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0098c0 tx=0x7fc0bc006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 msgr2=0x7fc0d419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 --2- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d419b320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4030380 tx=0x7fc0c4032670 comp rx=0 tx=0).stop 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 shutdown_connections 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 --2- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc0c0038f00 0x7fc0c003b3c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 --2- 172.21.15.69:0/1382085011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc0d4107370 0x7fc0d419b320 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 >> 172.21.15.69:0/1382085011 conn(0x7fc0d4102ba0 msgr2=0x7fc0d4103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 shutdown_connections 2024-08-02T17:38:00.507 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:37:59.151+0000 7fc0da319640 1 -- 172.21.15.69:0/1382085011 wait complete. 2024-08-02T17:38:00.508 INFO:teuthology.orchestra.run.smithi069.stdout:Creating initial admin user... 2024-08-02T17:38:00.798 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:00 smithi069 ceph-mon[32746]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:01.961 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$R71r8IF26vS5aHOPsqgGx.CWNZRlgfBOAnhAtefXOC1LD5bbzuWpW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1722620281, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-02T17:38:01.961 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.881+0000 7f3a8d00f640 1 Processor -- start 2024-08-02T17:38:01.961 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.881+0000 7f3a8d00f640 1 -- start start 2024-08-02T17:38:01.961 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.881+0000 7f3a8d00f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a8810a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.881+0000 7f3a8d00f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8807be30 con 0x7f3a88107d60 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.881+0000 7f3a86d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a8810a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.881+0000 7f3a86d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a8810a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46960/0 (socket says 172.21.15.69:46960) 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.881+0000 7f3a86d9d640 1 -- 172.21.15.69:0/2037868253 learned_addr learned my addr 172.21.15.69:0/2037868253 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.881+0000 7f3a86d9d640 1 -- 172.21.15.69:0/2037868253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8810a690 con 0x7f3a88107d60 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.882+0000 7f3a86d9d640 1 --2- 172.21.15.69:0/2037868253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a8810a150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009820 tx=0x7f3a7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bdf3566a6ea1dd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.882+0000 7f3a85d9b640 1 -- 172.21.15.69:0/2037868253 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a78037070 con 0x7f3a88107d60 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.882+0000 7f3a85d9b640 1 -- 172.21.15.69:0/2037868253 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a78032920 con 0x7f3a88107d60 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.882+0000 7f3a85d9b640 1 -- 172.21.15.69:0/2037868253 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a78030810 con 0x7f3a88107d60 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.883+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2037868253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 msgr2=0x7f3a8810a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.883+0000 7f3a8d00f640 1 --2- 172.21.15.69:0/2037868253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a8810a150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009820 tx=0x7f3a7802a270 comp rx=0 tx=0).stop 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.883+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2037868253 shutdown_connections 2024-08-02T17:38:01.962 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.883+0000 7f3a8d00f640 1 --2- 172.21.15.69:0/2037868253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a8810a150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.883+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2037868253 >> 172.21.15.69:0/2037868253 conn(0x7f3a88101a00 msgr2=0x7f3a88103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.883+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2037868253 shutdown_connections 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.883+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2037868253 wait complete. 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.884+0000 7f3a8d00f640 1 Processor -- start 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.884+0000 7f3a8d00f640 1 -- start start 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.884+0000 7f3a8d00f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a881075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.884+0000 7f3a8d00f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8810b770 con 0x7f3a88107d60 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.884+0000 7f3a86d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a881075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.884+0000 7f3a86d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a881075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46968/0 (socket says 172.21.15.69:46968) 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.884+0000 7f3a86d9d640 1 -- 172.21.15.69:0/2836009817 learned_addr learned my addr 172.21.15.69:0/2836009817 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:01.963 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.885+0000 7f3a86d9d640 1 -- 172.21.15.69:0/2836009817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a88107b20 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.885+0000 7f3a86d9d640 1 --2- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a881075e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a780097f0 tx=0x7f3a780327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.885+0000 7f3a6ffff640 1 -- 172.21.15.69:0/2836009817 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a78037040 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.885+0000 7f3a6ffff640 1 -- 172.21.15.69:0/2836009817 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a78030cc0 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.885+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a88105d10 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.885+0000 7f3a6ffff640 1 -- 172.21.15.69:0/2836009817 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a7803a6b0 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.885+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a88106250 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.886+0000 7f3a6ffff640 1 -- 172.21.15.69:0/2836009817 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f3a7803b3f0 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.886+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a881081c0 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.886+0000 7f3a6ffff640 1 --2- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a70038fc0 0x7f3a7003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.886+0000 7f3a6ffff640 1 -- 172.21.15.69:0/2836009817 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a7806c620 con 0x7f3a88107d60 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.886+0000 7f3a8659c640 1 --2- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a70038fc0 0x7f3a7003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:01.964 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.887+0000 7f3a8659c640 1 --2- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a70038fc0 0x7f3a7003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0098c0 tx=0x7f3a7c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:00.891+0000 7f3a6ffff640 1 -- 172.21.15.69:0/2836009817 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3a7803a8a0 con 0x7f3a88107d60 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.070+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3a881069f0 con 0x7f3a70038fc0 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.290+0000 7f3a6ffff640 1 -- 172.21.15.69:0/2836009817 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3a881069f0 con 0x7f3a70038fc0 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a70038fc0 msgr2=0x7f3a7003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 --2- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a70038fc0 0x7f3a7003b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0098c0 tx=0x7f3a7c006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 msgr2=0x7f3a881075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 --2- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a881075e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a780097f0 tx=0x7f3a780327b0 comp rx=0 tx=0).stop 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 shutdown_connections 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 --2- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a70038fc0 0x7f3a7003b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 --2- 172.21.15.69:0/2836009817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3a88107d60 0x7f3a881075e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 >> 172.21.15.69:0/2836009817 conn(0x7f3a88101a00 msgr2=0x7f3a88102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 shutdown_connections 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:01.292+0000 7f3a8d00f640 1 -- 172.21.15.69:0/2836009817 wait complete. 2024-08-02T17:38:01.965 INFO:teuthology.orchestra.run.smithi069.stdout:Fetching dashboard port number... 2024-08-02T17:38:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:02 smithi069 ceph-mon[32746]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:02.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:02 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:03.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stdout 8443 2024-08-02T17:38:03.091 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.357+0000 7f7db5753640 1 Processor -- start 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.357+0000 7f7db5753640 1 -- start start 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.357+0000 7f7db5753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db010a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.357+0000 7f7db5753640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7db007bdd0 con 0x7f7db0107dc0 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.357+0000 7f7daeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db010a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.358+0000 7f7daeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db010a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46982/0 (socket says 172.21.15.69:46982) 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.358+0000 7f7daeffd640 1 -- 172.21.15.69:0/2398570408 learned_addr learned my addr 172.21.15.69:0/2398570408 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.358+0000 7f7daeffd640 1 -- 172.21.15.69:0/2398570408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db010a6f0 con 0x7f7db0107dc0 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.358+0000 7f7daeffd640 1 --2- 172.21.15.69:0/2398570408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db010a1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009e80 tx=0x7f7da002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d68becf79edb313 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.358+0000 7f7dadffb640 1 -- 172.21.15.69:0/2398570408 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7da0036070 con 0x7f7db0107dc0 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7dadffb640 1 -- 172.21.15.69:0/2398570408 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da00318e0 con 0x7f7db0107dc0 2024-08-02T17:38:03.092 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7dadffb640 1 -- 172.21.15.69:0/2398570408 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7da0007a00 con 0x7f7db0107dc0 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7db5753640 1 -- 172.21.15.69:0/2398570408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 msgr2=0x7f7db010a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7db5753640 1 --2- 172.21.15.69:0/2398570408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db010a1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7da0009e80 tx=0x7f7da002a7a0 comp rx=0 tx=0).stop 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7db5753640 1 -- 172.21.15.69:0/2398570408 shutdown_connections 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7db5753640 1 --2- 172.21.15.69:0/2398570408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db010a1b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7db5753640 1 -- 172.21.15.69:0/2398570408 >> 172.21.15.69:0/2398570408 conn(0x7f7db0101a00 msgr2=0x7f7db0103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7db5753640 1 -- 172.21.15.69:0/2398570408 shutdown_connections 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.359+0000 7f7db5753640 1 -- 172.21.15.69:0/2398570408 wait complete. 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.360+0000 7f7db5753640 1 Processor -- start 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.360+0000 7f7db5753640 1 -- start start 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.360+0000 7f7db5753640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db019b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.360+0000 7f7db5753640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7db010b770 con 0x7f7db0107dc0 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.360+0000 7f7daeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db019b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:03.093 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.360+0000 7f7daeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db019b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46992/0 (socket says 172.21.15.69:46992) 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.360+0000 7f7daeffd640 1 -- 172.21.15.69:0/1744049152 learned_addr learned my addr 172.21.15.69:0/1744049152 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.361+0000 7f7daeffd640 1 -- 172.21.15.69:0/1744049152 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db019b850 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.361+0000 7f7daeffd640 1 --2- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db019b310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7da0031cf0 tx=0x7f7da0031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.361+0000 7f7d97fff640 1 -- 172.21.15.69:0/1744049152 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7da0036050 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.361+0000 7f7d97fff640 1 -- 172.21.15.69:0/1744049152 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7da0042070 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.361+0000 7f7d97fff640 1 -- 172.21.15.69:0/1744049152 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7da003d9b0 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.361+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db019bae0 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.361+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db019e7d0 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.362+0000 7f7d97fff640 1 -- 172.21.15.69:0/1744049152 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f7da003dc60 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.362+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db0108220 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.362+0000 7f7d97fff640 1 --2- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7d98038fc0 0x7f7d9803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.362+0000 7f7d97fff640 1 -- 172.21.15.69:0/1744049152 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7da006d640 con 0x7f7db0107dc0 2024-08-02T17:38:03.094 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.362+0000 7f7dae7fc640 1 --2- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7d98038fc0 0x7f7d9803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.362+0000 7f7dae7fc640 1 --2- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7d98038fc0 0x7f7d9803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7da40098c0 tx=0x7f7da4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.367+0000 7f7d97fff640 1 -- 172.21.15.69:0/1744049152 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7da003bcf0 con 0x7f7db0107dc0 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.532+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f7db019edc0 con 0x7f7db0107dc0 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.533+0000 7f7d97fff640 1 -- 172.21.15.69:0/1744049152 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f7da0047980 con 0x7f7db0107dc0 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7d98038fc0 msgr2=0x7f7d9803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 --2- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7d98038fc0 0x7f7d9803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7da40098c0 tx=0x7f7da4006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 msgr2=0x7f7db019b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 --2- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db019b310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7da0031cf0 tx=0x7f7da0031d20 comp rx=0 tx=0).stop 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 shutdown_connections 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 --2- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7d98038fc0 0x7f7d9803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 --2- 172.21.15.69:0/1744049152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7db0107dc0 0x7f7db019b310 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 >> 172.21.15.69:0/1744049152 conn(0x7f7db0101a00 msgr2=0x7f7db01025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:03.095 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 shutdown_connections 2024-08-02T17:38:03.096 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:02.535+0000 7f7db5753640 1 -- 172.21.15.69:0/1744049152 wait complete. 2024-08-02T17:38:03.099 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-02T17:38:03.099 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout disabled 2024-08-02T17:38:03.106 INFO:teuthology.orchestra.run.smithi069.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-02T17:38:03.106 INFO:teuthology.orchestra.run.smithi069.stdout:systemctl: stdout inactive 2024-08-02T17:38:03.106 INFO:teuthology.orchestra.run.smithi069.stdout:firewalld.service is not enabled 2024-08-02T17:38:03.106 INFO:teuthology.orchestra.run.smithi069.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-02T17:38:03.108 INFO:teuthology.orchestra.run.smithi069.stdout:Ceph Dashboard is now available at: 2024-08-02T17:38:03.108 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:03.108 INFO:teuthology.orchestra.run.smithi069.stdout: URL: https://smithi069.front.sepia.ceph.com:8443/ 2024-08-02T17:38:03.108 INFO:teuthology.orchestra.run.smithi069.stdout: User: admin 2024-08-02T17:38:03.108 INFO:teuthology.orchestra.run.smithi069.stdout: Password: z6u9e70j9q 2024-08-02T17:38:03.108 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:03.109 INFO:teuthology.orchestra.run.smithi069.stdout:Saving cluster configuration to /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config directory 2024-08-02T17:38:03.341 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:03 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1744049152' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-08-02T17:38:04.349 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:04 smithi069 ceph-mon[32746]: mgrmap e13: x(active, since 6s) 2024-08-02T17:38:04.349 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:04 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3560333350' entity='client.admin' 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.546+0000 7fa8c5a3c640 1 Processor -- start 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.546+0000 7fa8c5a3c640 1 -- start start 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.546+0000 7fa8c5a3c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c010a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.546+0000 7fa8c5a3c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c007bd50 con 0x7fa8c0107e40 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.546+0000 7fa8beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c010a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.546+0000 7fa8beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c010a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46998/0 (socket says 172.21.15.69:46998) 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.546+0000 7fa8beffd640 1 -- 172.21.15.69:0/2303126782 learned_addr learned my addr 172.21.15.69:0/2303126782 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.547+0000 7fa8beffd640 1 -- 172.21.15.69:0/2303126782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c010a770 con 0x7fa8c0107e40 2024-08-02T17:38:04.357 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.547+0000 7fa8beffd640 1 --2- 172.21.15.69:0/2303126782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c010a230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009e80 tx=0x7fa8b802a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca7db6e0dd00d06 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8bdffb640 1 -- 172.21.15.69:0/2303126782 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8036070 con 0x7fa8c0107e40 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8bdffb640 1 -- 172.21.15.69:0/2303126782 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b80318e0 con 0x7fa8c0107e40 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8bdffb640 1 -- 172.21.15.69:0/2303126782 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8007a00 con 0x7fa8c0107e40 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/2303126782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 msgr2=0x7fa8c010a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8c5a3c640 1 --2- 172.21.15.69:0/2303126782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c010a230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009e80 tx=0x7fa8b802a7a0 comp rx=0 tx=0).stop 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/2303126782 shutdown_connections 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8c5a3c640 1 --2- 172.21.15.69:0/2303126782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c010a230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/2303126782 >> 172.21.15.69:0/2303126782 conn(0x7fa8c0101a00 msgr2=0x7fa8c0103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/2303126782 shutdown_connections 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.548+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/2303126782 wait complete. 2024-08-02T17:38:04.358 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.549+0000 7fa8c5a3c640 1 Processor -- start 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.549+0000 7fa8c5a3c640 1 -- start start 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.549+0000 7fa8c5a3c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.549+0000 7fa8c5a3c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c010b730 con 0x7fa8c0107e40 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.549+0000 7fa8beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.549+0000 7fa8beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47010/0 (socket says 172.21.15.69:47010) 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.549+0000 7fa8beffd640 1 -- 172.21.15.69:0/3560333350 learned_addr learned my addr 172.21.15.69:0/3560333350 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.549+0000 7fa8beffd640 1 -- 172.21.15.69:0/3560333350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c019b860 con 0x7fa8c0107e40 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.550+0000 7fa8beffd640 1 --2- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c019b320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8031cf0 tx=0x7fa8b8031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.550+0000 7fa8c4a3a640 1 -- 172.21.15.69:0/3560333350 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b8036050 con 0x7fa8c0107e40 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.550+0000 7fa8c4a3a640 1 -- 172.21.15.69:0/3560333350 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b8042070 con 0x7fa8c0107e40 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.550+0000 7fa8c4a3a640 1 -- 172.21.15.69:0/3560333350 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8b803d9b0 con 0x7fa8c0107e40 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.550+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c019baf0 con 0x7fa8c0107e40 2024-08-02T17:38:04.359 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.550+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c019e7e0 con 0x7fa8c0107e40 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.551+0000 7fa8c4a3a640 1 -- 172.21.15.69:0/3560333350 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa8b803db50 con 0x7fa8c0107e40 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.551+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c01082a0 con 0x7fa8c0107e40 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.551+0000 7fa8c4a3a640 1 --2- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa8ac0390c0 0x7fa8ac03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.551+0000 7fa8c4a3a640 1 -- 172.21.15.69:0/3560333350 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa8b806d620 con 0x7fa8c0107e40 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.551+0000 7fa8be7fc640 1 --2- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa8ac0390c0 0x7fa8ac03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.552+0000 7fa8be7fc640 1 --2- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa8ac0390c0 0x7fa8ac03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80098c0 tx=0x7fa8a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.556+0000 7fa8c4a3a640 1 -- 172.21.15.69:0/3560333350 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa8b8047980 con 0x7fa8c0107e40 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.808+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fa8c019edd0 con 0x7fa8c0107e40 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.811+0000 7fa8c4a3a640 1 -- 172.21.15.69:0/3560333350 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7fa8b8047460 con 0x7fa8c0107e40 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa8ac0390c0 msgr2=0x7fa8ac03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 --2- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa8ac0390c0 0x7fa8ac03b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa8a80098c0 tx=0x7fa8a8006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:04.360 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 msgr2=0x7fa8c019b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 --2- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c019b320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8031cf0 tx=0x7fa8b8031d20 comp rx=0 tx=0).stop 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 shutdown_connections 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 --2- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa8ac0390c0 0x7fa8ac03b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 --2- 172.21.15.69:0/3560333350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8c0107e40 0x7fa8c019b320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 >> 172.21.15.69:0/3560333350 conn(0x7fa8c0101a00 msgr2=0x7fa8c01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 shutdown_connections 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:/usr/bin/ceph: stderr 2024-08-02T17:38:03.813+0000 7fa8c5a3c640 1 -- 172.21.15.69:0/3560333350 wait complete. 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout:Or, if you are only running a single cluster on this host: 2024-08-02T17:38:04.361 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: ceph telemetry on 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout:For more information see: 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:04.362 INFO:teuthology.orchestra.run.smithi069.stdout:Bootstrap complete. 2024-08-02T17:38:04.395 INFO:tasks.cephadm:Fetching config... 2024-08-02T17:38:04.396 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:38:04.396 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-02T17:38:04.411 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-02T17:38:04.412 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:38:04.412 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-02T17:38:04.468 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-02T17:38:04.468 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:38:04.468 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-02T17:38:04.533 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-02T17:38:04.533 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:38:04.533 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-02T17:38:04.588 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-02T17:38:04.589 DEBUG:teuthology.orchestra.run.smithi069:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZrcJB3BsVUjzi6g8TzqPXKUKDfpsELg0+pY48Wjyl4zERDATf+hO6EKQfc6iu6bv3nf6Ph9yONejdgzqvY3aHWVxtFhA5FHGh7mfoWX1O+QcrQ25dQrQLnQX+FGgQzcreVANycnrPNpSEWjMoIbPtS8Qs7RHyfGaVaYpaYrSREocpXlnjsGQCDN3lycgSa6niG7blJz9C6JrIKeZ66kLeAfof0w1dH+nCKqQVoNNAOic4iWfl8n/hthKnJo1VL1AaYWWJiSYBAG2jeRzoOJpwXPZVrYZSK/W9jBmUQkAmg6OmsQxsW4bCHrrQcDBr6TRHGrqxyrwKrEq1HIJveC6gajvB/mqI7JAv8nWHpWD4TsSkT65WsXAenFpz+ad6n7pEVh6wiMPJTtyg4U3uR1OKWiXop+q21sTz/PSSX6Zd2b8CKq+yhAWFprqD7eE43ZBrxKLEkMlTiPJdACgR+o64kCAVyG+DJNCqPMIaSpjtpPTodEOamvPersR8LY4Vehs= ceph-b6b04634-50f5-11ef-bcca-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-02T17:38:04.668 INFO:teuthology.orchestra.run.smithi069.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZrcJB3BsVUjzi6g8TzqPXKUKDfpsELg0+pY48Wjyl4zERDATf+hO6EKQfc6iu6bv3nf6Ph9yONejdgzqvY3aHWVxtFhA5FHGh7mfoWX1O+QcrQ25dQrQLnQX+FGgQzcreVANycnrPNpSEWjMoIbPtS8Qs7RHyfGaVaYpaYrSREocpXlnjsGQCDN3lycgSa6niG7blJz9C6JrIKeZ66kLeAfof0w1dH+nCKqQVoNNAOic4iWfl8n/hthKnJo1VL1AaYWWJiSYBAG2jeRzoOJpwXPZVrYZSK/W9jBmUQkAmg6OmsQxsW4bCHrrQcDBr6TRHGrqxyrwKrEq1HIJveC6gajvB/mqI7JAv8nWHpWD4TsSkT65WsXAenFpz+ad6n7pEVh6wiMPJTtyg4U3uR1OKWiXop+q21sTz/PSSX6Zd2b8CKq+yhAWFprqD7eE43ZBrxKLEkMlTiPJdACgR+o64kCAVyG+DJNCqPMIaSpjtpPTodEOamvPersR8LY4Vehs= ceph-b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:04.680 DEBUG:teuthology.orchestra.run.smithi077:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZrcJB3BsVUjzi6g8TzqPXKUKDfpsELg0+pY48Wjyl4zERDATf+hO6EKQfc6iu6bv3nf6Ph9yONejdgzqvY3aHWVxtFhA5FHGh7mfoWX1O+QcrQ25dQrQLnQX+FGgQzcreVANycnrPNpSEWjMoIbPtS8Qs7RHyfGaVaYpaYrSREocpXlnjsGQCDN3lycgSa6niG7blJz9C6JrIKeZ66kLeAfof0w1dH+nCKqQVoNNAOic4iWfl8n/hthKnJo1VL1AaYWWJiSYBAG2jeRzoOJpwXPZVrYZSK/W9jBmUQkAmg6OmsQxsW4bCHrrQcDBr6TRHGrqxyrwKrEq1HIJveC6gajvB/mqI7JAv8nWHpWD4TsSkT65WsXAenFpz+ad6n7pEVh6wiMPJTtyg4U3uR1OKWiXop+q21sTz/PSSX6Zd2b8CKq+yhAWFprqD7eE43ZBrxKLEkMlTiPJdACgR+o64kCAVyG+DJNCqPMIaSpjtpPTodEOamvPersR8LY4Vehs= ceph-b6b04634-50f5-11ef-bcca-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-02T17:38:04.718 INFO:teuthology.orchestra.run.smithi077.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZrcJB3BsVUjzi6g8TzqPXKUKDfpsELg0+pY48Wjyl4zERDATf+hO6EKQfc6iu6bv3nf6Ph9yONejdgzqvY3aHWVxtFhA5FHGh7mfoWX1O+QcrQ25dQrQLnQX+FGgQzcreVANycnrPNpSEWjMoIbPtS8Qs7RHyfGaVaYpaYrSREocpXlnjsGQCDN3lycgSa6niG7blJz9C6JrIKeZ66kLeAfof0w1dH+nCKqQVoNNAOic4iWfl8n/hthKnJo1VL1AaYWWJiSYBAG2jeRzoOJpwXPZVrYZSK/W9jBmUQkAmg6OmsQxsW4bCHrrQcDBr6TRHGrqxyrwKrEq1HIJveC6gajvB/mqI7JAv8nWHpWD4TsSkT65WsXAenFpz+ad6n7pEVh6wiMPJTtyg4U3uR1OKWiXop+q21sTz/PSSX6Zd2b8CKq+yhAWFprqD7eE43ZBrxKLEkMlTiPJdACgR+o64kCAVyG+DJNCqPMIaSpjtpPTodEOamvPersR8LY4Vehs= ceph-b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:04.730 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZrcJB3BsVUjzi6g8TzqPXKUKDfpsELg0+pY48Wjyl4zERDATf+hO6EKQfc6iu6bv3nf6Ph9yONejdgzqvY3aHWVxtFhA5FHGh7mfoWX1O+QcrQ25dQrQLnQX+FGgQzcreVANycnrPNpSEWjMoIbPtS8Qs7RHyfGaVaYpaYrSREocpXlnjsGQCDN3lycgSa6niG7blJz9C6JrIKeZ66kLeAfof0w1dH+nCKqQVoNNAOic4iWfl8n/hthKnJo1VL1AaYWWJiSYBAG2jeRzoOJpwXPZVrYZSK/W9jBmUQkAmg6OmsQxsW4bCHrrQcDBr6TRHGrqxyrwKrEq1HIJveC6gajvB/mqI7JAv8nWHpWD4TsSkT65WsXAenFpz+ad6n7pEVh6wiMPJTtyg4U3uR1OKWiXop+q21sTz/PSSX6Zd2b8CKq+yhAWFprqD7eE43ZBrxKLEkMlTiPJdACgR+o64kCAVyG+DJNCqPMIaSpjtpPTodEOamvPersR8LY4Vehs= ceph-b6b04634-50f5-11ef-bcca-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-02T17:38:04.769 INFO:teuthology.orchestra.run.smithi083.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZrcJB3BsVUjzi6g8TzqPXKUKDfpsELg0+pY48Wjyl4zERDATf+hO6EKQfc6iu6bv3nf6Ph9yONejdgzqvY3aHWVxtFhA5FHGh7mfoWX1O+QcrQ25dQrQLnQX+FGgQzcreVANycnrPNpSEWjMoIbPtS8Qs7RHyfGaVaYpaYrSREocpXlnjsGQCDN3lycgSa6niG7blJz9C6JrIKeZ66kLeAfof0w1dH+nCKqQVoNNAOic4iWfl8n/hthKnJo1VL1AaYWWJiSYBAG2jeRzoOJpwXPZVrYZSK/W9jBmUQkAmg6OmsQxsW4bCHrrQcDBr6TRHGrqxyrwKrEq1HIJveC6gajvB/mqI7JAv8nWHpWD4TsSkT65WsXAenFpz+ad6n7pEVh6wiMPJTtyg4U3uR1OKWiXop+q21sTz/PSSX6Zd2b8CKq+yhAWFprqD7eE43ZBrxKLEkMlTiPJdACgR+o64kCAVyG+DJNCqPMIaSpjtpPTodEOamvPersR8LY4Vehs= ceph-b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:04.781 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-02T17:38:05.034 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:05.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.527+0000 7f29863d3640 1 -- 172.21.15.69:0/377849528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 msgr2=0x7f2980100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:05.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.527+0000 7f29863d3640 1 --2- 172.21.15.69:0/377849528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 0x7f2980100640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2970009880 tx=0x7f297002a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 -- 172.21.15.69:0/377849528 shutdown_connections 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 --2- 172.21.15.69:0/377849528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 0x7f2980100640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 -- 172.21.15.69:0/377849528 >> 172.21.15.69:0/377849528 conn(0x7f29800fb9d0 msgr2=0x7f29800fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 -- 172.21.15.69:0/377849528 shutdown_connections 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 -- 172.21.15.69:0/377849528 wait complete. 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 Processor -- start 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 -- start start 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 0x7f2980196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f29863d3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2980106350 con 0x7f2980100240 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f297ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 0x7f2980196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f297ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 0x7f2980196a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44222/0 (socket says 172.21.15.69:44222) 2024-08-02T17:38:05.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.528+0000 7f297ffff640 1 -- 172.21.15.69:0/3999429982 learned_addr learned my addr 172.21.15.69:0/3999429982 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.529+0000 7f297ffff640 1 -- 172.21.15.69:0/3999429982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2980196fd0 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.529+0000 7f297ffff640 1 --2- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 0x7f2980196a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2970004f60 tx=0x7f29700048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.529+0000 7f297dffb640 1 -- 172.21.15.69:0/3999429982 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f297003f070 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.529+0000 7f297dffb640 1 -- 172.21.15.69:0/3999429982 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2970004d40 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.529+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2980197260 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.529+0000 7f297dffb640 1 -- 172.21.15.69:0/3999429982 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2970035030 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.529+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2980199f50 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.530+0000 7f297dffb640 1 -- 172.21.15.69:0/3999429982 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2970042070 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.530+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2980106ad0 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.530+0000 7f297dffb640 1 --2- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f296c039060 0x7f296c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.530+0000 7f297dffb640 1 -- 172.21.15.69:0/3999429982 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f297006cd10 con 0x7f2980100240 2024-08-02T17:38:05.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.530+0000 7f2977fff640 1 --2- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f296c039060 0x7f296c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:05.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.531+0000 7f2977fff640 1 --2- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f296c039060 0x7f296c03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29680098c0 tx=0x7f2968006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:05.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.535+0000 7f297dffb640 1 -- 172.21.15.69:0/3999429982 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f297004a260 con 0x7f2980100240 2024-08-02T17:38:05.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.704+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f298019a3a0 con 0x7f2980100240 2024-08-02T17:38:05.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.706+0000 7f297dffb640 1 -- 172.21.15.69:0/3999429982 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f2970003e10 con 0x7f2980100240 2024-08-02T17:38:05.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.708+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f296c039060 msgr2=0x7f296c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:05.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.708+0000 7f29863d3640 1 --2- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f296c039060 0x7f296c03b520 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29680098c0 tx=0x7f2968006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:05.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.708+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 msgr2=0x7f2980196a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:05.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.708+0000 7f29863d3640 1 --2- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 0x7f2980196a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2970004f60 tx=0x7f29700048f0 comp rx=0 tx=0).stop 2024-08-02T17:38:05.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.709+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 shutdown_connections 2024-08-02T17:38:05.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.709+0000 7f29863d3640 1 --2- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f296c039060 0x7f296c03b520 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:05.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.709+0000 7f29863d3640 1 --2- 172.21.15.69:0/3999429982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2980100240 0x7f2980196a90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:05.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.709+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 >> 172.21.15.69:0/3999429982 conn(0x7f29800fb9d0 msgr2=0x7f29800fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:05.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.709+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 shutdown_connections 2024-08-02T17:38:05.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:05.709+0000 7f29863d3640 1 -- 172.21.15.69:0/3999429982 wait complete. 2024-08-02T17:38:06.287 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-02T17:38:06.287 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-02T17:38:06.753 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:06.980 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:06 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3999429982' entity='client.admin' 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.189+0000 7f3d2b657640 1 -- 172.21.15.69:0/413674900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 msgr2=0x7f3d24100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.189+0000 7f3d2b657640 1 --2- 172.21.15.69:0/413674900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 0x7f3d24100640 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009880 tx=0x7f3d2002a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 -- 172.21.15.69:0/413674900 shutdown_connections 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 --2- 172.21.15.69:0/413674900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 0x7f3d24100640 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 -- 172.21.15.69:0/413674900 >> 172.21.15.69:0/413674900 conn(0x7f3d240fb9b0 msgr2=0x7f3d240fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 -- 172.21.15.69:0/413674900 shutdown_connections 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 -- 172.21.15.69:0/413674900 wait complete. 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 Processor -- start 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 -- start start 2024-08-02T17:38:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 0x7f3d2406a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d2b657640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3d24106350 con 0x7f3d24100240 2024-08-02T17:38:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.190+0000 7f3d293f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 0x7f3d2406a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d293f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 0x7f3d2406a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44250/0 (socket says 172.21.15.69:44250) 2024-08-02T17:38:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d293f3640 1 -- 172.21.15.69:0/2835691040 learned_addr learned my addr 172.21.15.69:0/2835691040 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d293f3640 1 -- 172.21.15.69:0/2835691040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d24199840 con 0x7f3d24100240 2024-08-02T17:38:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d293f3640 1 --2- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 0x7f3d2406a680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d20004f60 tx=0x7f3d200048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d1a7fc640 1 -- 172.21.15.69:0/2835691040 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d2003f070 con 0x7f3d24100240 2024-08-02T17:38:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d1a7fc640 1 -- 172.21.15.69:0/2835691040 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d20004d40 con 0x7f3d24100240 2024-08-02T17:38:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d1a7fc640 1 -- 172.21.15.69:0/2835691040 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d20035030 con 0x7f3d24100240 2024-08-02T17:38:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d24199a10 con 0x7f3d24100240 2024-08-02T17:38:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.191+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d24199db0 con 0x7f3d24100240 2024-08-02T17:38:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.192+0000 7f3d1a7fc640 1 -- 172.21.15.69:0/2835691040 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3d20042070 con 0x7f3d24100240 2024-08-02T17:38:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.192+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d24106ad0 con 0x7f3d24100240 2024-08-02T17:38:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.192+0000 7f3d1a7fc640 1 --2- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3d100390c0 0x7f3d1003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.192+0000 7f3d1a7fc640 1 -- 172.21.15.69:0/2835691040 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3d2006ccc0 con 0x7f3d24100240 2024-08-02T17:38:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.192+0000 7f3d28bf2640 1 --2- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3d100390c0 0x7f3d1003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.193+0000 7f3d28bf2640 1 --2- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3d100390c0 0x7f3d1003b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d140098c0 tx=0x7f3d14006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:07.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.197+0000 7f3d1a7fc640 1 -- 172.21.15.69:0/2835691040 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3d2004a260 con 0x7f3d24100240 2024-08-02T17:38:07.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.367+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f3d2402d750 con 0x7f3d100390c0 2024-08-02T17:38:07.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.370+0000 7f3d1a7fc640 1 -- 172.21.15.69:0/2835691040 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d2402d750 con 0x7f3d100390c0 2024-08-02T17:38:07.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3d100390c0 msgr2=0x7f3d1003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 --2- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3d100390c0 0x7f3d1003b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3d140098c0 tx=0x7f3d14006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 msgr2=0x7f3d2406a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 --2- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 0x7f3d2406a680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d20004f60 tx=0x7f3d200048f0 comp rx=0 tx=0).stop 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 shutdown_connections 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 --2- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3d100390c0 0x7f3d1003b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 --2- 172.21.15.69:0/2835691040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3d24100240 0x7f3d2406a680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 >> 172.21.15.69:0/2835691040 conn(0x7f3d240fb9b0 msgr2=0x7f3d240fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 shutdown_connections 2024-08-02T17:38:07.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:07.373+0000 7f3d2b657640 1 -- 172.21.15.69:0/2835691040 wait complete. 2024-08-02T17:38:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi069", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:38:08.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:08.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:08.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:08.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:08.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:08.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:08.039 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:08.068 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi077 2024-08-02T17:38:08.068 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:38:08.068 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.conf 2024-08-02T17:38:08.086 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:38:08.086 DEBUG:teuthology.orchestra.run.smithi077:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:08.143 INFO:tasks.cephadm:Adding host smithi077 to orchestrator... 2024-08-02T17:38:08.143 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch host add smithi077 2024-08-02T17:38:08.412 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:08.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:08 smithi069 ceph-mon[32746]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:08.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:08 smithi069 ceph-mon[32746]: Updating smithi069:/etc/ceph/ceph.conf 2024-08-02T17:38:08.791 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:08 smithi069 ceph-mon[32746]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:08.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:08 smithi069 ceph-mon[32746]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:08.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:08.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:08.792 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.812+0000 7f0589706640 1 -- 172.21.15.69:0/3585450521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 msgr2=0x7f0584074730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.812+0000 7f0589706640 1 --2- 172.21.15.69:0/3585450521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 0x7f0584074730 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0574009c20 tx=0x7f057402ab20 comp rx=0 tx=0).stop 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.812+0000 7f0589706640 1 -- 172.21.15.69:0/3585450521 shutdown_connections 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.812+0000 7f0589706640 1 --2- 172.21.15.69:0/3585450521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 0x7f0584074730 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.812+0000 7f0589706640 1 -- 172.21.15.69:0/3585450521 >> 172.21.15.69:0/3585450521 conn(0x7f05840fcfe0 msgr2=0x7f05840ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.812+0000 7f0589706640 1 -- 172.21.15.69:0/3585450521 shutdown_connections 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.812+0000 7f0589706640 1 -- 172.21.15.69:0/3585450521 wait complete. 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.813+0000 7f0589706640 1 Processor -- start 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.813+0000 7f0589706640 1 -- start start 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.813+0000 7f0589706640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 0x7f0584199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:08.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.813+0000 7f0589706640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f058410dbf0 con 0x7f0584074350 2024-08-02T17:38:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.813+0000 7f0582ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 0x7f0584199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f0582ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 0x7f0584199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44274/0 (socket says 172.21.15.69:44274) 2024-08-02T17:38:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f0582ffd640 1 -- 172.21.15.69:0/194891049 learned_addr learned my addr 172.21.15.69:0/194891049 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f0582ffd640 1 -- 172.21.15.69:0/194891049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0584199660 con 0x7f0584074350 2024-08-02T17:38:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f0582ffd640 1 --2- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 0x7f0584199120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0574030040 tx=0x7f057402eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f056bfff640 1 -- 172.21.15.69:0/194891049 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0574047070 con 0x7f0584074350 2024-08-02T17:38:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f056bfff640 1 -- 172.21.15.69:0/194891049 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05740368e0 con 0x7f0584074350 2024-08-02T17:38:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05841998f0 con 0x7f0584074350 2024-08-02T17:38:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f056bfff640 1 -- 172.21.15.69:0/194891049 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0574042420 con 0x7f0584074350 2024-08-02T17:38:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.814+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0584199d50 con 0x7f0584074350 2024-08-02T17:38:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.815+0000 7f056bfff640 1 -- 172.21.15.69:0/194891049 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f057404a070 con 0x7f0584074350 2024-08-02T17:38:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.815+0000 7f056bfff640 1 --2- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f056c039060 0x7f056c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.815+0000 7f056bfff640 1 -- 172.21.15.69:0/194891049 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f057403d030 con 0x7f0584074350 2024-08-02T17:38:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.815+0000 7f05827fc640 1 --2- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f056c039060 0x7f056c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.815+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0584075a00 con 0x7f0584074350 2024-08-02T17:38:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.816+0000 7f05827fc640 1 --2- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f056c039060 0x7f056c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f05780098c0 tx=0x7f0578006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:08.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.820+0000 7f056bfff640 1 -- 172.21.15.69:0/194891049 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0574051070 con 0x7f0584074350 2024-08-02T17:38:08.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:08.988+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}) -- 0x7f058419a480 con 0x7f056c039060 2024-08-02T17:38:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:09 smithi069 ceph-mon[32746]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:38:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:10 smithi069 ceph-mon[32746]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi077", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:12.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:11 smithi069 ceph-mon[32746]: Deploying cephadm binary to smithi077 2024-08-02T17:38:12.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.142+0000 7f056bfff640 1 -- 172.21.15.69:0/194891049 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f058419a480 con 0x7f056c039060 2024-08-02T17:38:12.144 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi077' with addr '172.21.15.77' 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f056c039060 msgr2=0x7f056c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 --2- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f056c039060 0x7f056c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f05780098c0 tx=0x7f0578006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 msgr2=0x7f0584199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 --2- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 0x7f0584199120 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0574030040 tx=0x7f057402eb50 comp rx=0 tx=0).stop 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 shutdown_connections 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 --2- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f056c039060 0x7f056c03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 --2- 172.21.15.69:0/194891049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0584074350 0x7f0584199120 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 >> 172.21.15.69:0/194891049 conn(0x7f05840fcfe0 msgr2=0x7f0584107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 shutdown_connections 2024-08-02T17:38:12.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:12.144+0000 7f0589706640 1 -- 172.21.15.69:0/194891049 wait complete. 2024-08-02T17:38:12.766 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch host ls --format=json 2024-08-02T17:38:13.014 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:13.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:13.435 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:13 smithi069 ceph-mon[32746]: Added host smithi077 2024-08-02T17:38:13.436 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:13.436 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:13.436 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:13.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.433+0000 7fc6c496b640 1 -- 172.21.15.69:0/688539580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 msgr2=0x7fc6c0100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.433+0000 7fc6c496b640 1 --2- 172.21.15.69:0/688539580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 0x7fc6c0100820 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8009c20 tx=0x7fc6b802aa90 comp rx=0 tx=0).stop 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.434+0000 7fc6c496b640 1 -- 172.21.15.69:0/688539580 shutdown_connections 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.434+0000 7fc6c496b640 1 --2- 172.21.15.69:0/688539580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 0x7fc6c0100820 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.434+0000 7fc6c496b640 1 -- 172.21.15.69:0/688539580 >> 172.21.15.69:0/688539580 conn(0x7fc6c00fbec0 msgr2=0x7fc6c00fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.434+0000 7fc6c496b640 1 -- 172.21.15.69:0/688539580 shutdown_connections 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.434+0000 7fc6c496b640 1 -- 172.21.15.69:0/688539580 wait complete. 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.435+0000 7fc6c496b640 1 Processor -- start 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.435+0000 7fc6c496b640 1 -- start start 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.435+0000 7fc6c496b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 0x7fc6c0196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.435+0000 7fc6c496b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c010a890 con 0x7fc6c0100440 2024-08-02T17:38:13.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.435+0000 7fc6be59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 0x7fc6c0196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.435+0000 7fc6be59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 0x7fc6c0196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44292/0 (socket says 172.21.15.69:44292) 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.435+0000 7fc6be59c640 1 -- 172.21.15.69:0/884561939 learned_addr learned my addr 172.21.15.69:0/884561939 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.435+0000 7fc6be59c640 1 -- 172.21.15.69:0/884561939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c0197430 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.436+0000 7fc6be59c640 1 --2- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 0x7fc6c0196ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8005e30 tx=0x7fc6b802fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.436+0000 7fc6b77fe640 1 -- 172.21.15.69:0/884561939 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6b8047070 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.436+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c0197720 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.436+0000 7fc6b77fe640 1 -- 172.21.15.69:0/884561939 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6b8003df0 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.436+0000 7fc6b77fe640 1 -- 172.21.15.69:0/884561939 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc6b8033050 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.436+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c019b7c0 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.437+0000 7fc6b77fe640 1 -- 172.21.15.69:0/884561939 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc6b804a070 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.437+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c0101920 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.437+0000 7fc6b77fe640 1 --2- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc6ac039060 0x7fc6ac03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.437+0000 7fc6b77fe640 1 -- 172.21.15.69:0/884561939 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc6b803e030 con 0x7fc6c0100440 2024-08-02T17:38:13.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.437+0000 7fc6bdd9b640 1 --2- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc6ac039060 0x7fc6ac03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:13.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.438+0000 7fc6bdd9b640 1 --2- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc6ac039060 0x7fc6ac03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80098c0 tx=0x7fc6a8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:13.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.443+0000 7fc6b77fe640 1 -- 172.21.15.69:0/884561939 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc6b80373b0 con 0x7fc6c0100440 2024-08-02T17:38:13.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.616+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc6c0100900 con 0x7fc6ac039060 2024-08-02T17:38:13.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.617+0000 7fc6b77fe640 1 -- 172.21.15.69:0/884561939 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fc6c0100900 con 0x7fc6ac039060 2024-08-02T17:38:13.619 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:13.619 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}] 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc6ac039060 msgr2=0x7fc6ac03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 --2- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc6ac039060 0x7fc6ac03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc6a80098c0 tx=0x7fc6a8006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 msgr2=0x7fc6c0196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 --2- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 0x7fc6c0196ef0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8005e30 tx=0x7fc6b802fc10 comp rx=0 tx=0).stop 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 shutdown_connections 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 --2- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc6ac039060 0x7fc6ac03b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 --2- 172.21.15.69:0/884561939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc6c0100440 0x7fc6c0196ef0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 >> 172.21.15.69:0/884561939 conn(0x7fc6c00fbec0 msgr2=0x7fc6c0069c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 shutdown_connections 2024-08-02T17:38:13.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:13.619+0000 7fc6c496b640 1 -- 172.21.15.69:0/884561939 wait complete. 2024-08-02T17:38:14.245 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi083 2024-08-02T17:38:14.245 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:38:14.245 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.conf 2024-08-02T17:38:14.264 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:38:14.264 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:14.321 INFO:tasks.cephadm:Adding host smithi083 to orchestrator... 2024-08-02T17:38:14.322 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch host add smithi083 2024-08-02T17:38:14.571 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:14.794 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:14.795 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:14 smithi069 ceph-mon[32746]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:38:14.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.992+0000 7fe702709640 1 -- 172.21.15.69:0/3828232949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 msgr2=0x7fe6fc100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.992+0000 7fe702709640 1 --2- 172.21.15.69:0/3828232949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 0x7fe6fc100800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009880 tx=0x7fe6ec02a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.992+0000 7fe702709640 1 -- 172.21.15.69:0/3828232949 shutdown_connections 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.992+0000 7fe702709640 1 --2- 172.21.15.69:0/3828232949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 0x7fe6fc100800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.992+0000 7fe702709640 1 -- 172.21.15.69:0/3828232949 >> 172.21.15.69:0/3828232949 conn(0x7fe6fc0fbee0 msgr2=0x7fe6fc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.992+0000 7fe702709640 1 -- 172.21.15.69:0/3828232949 shutdown_connections 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.992+0000 7fe702709640 1 -- 172.21.15.69:0/3828232949 wait complete. 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.993+0000 7fe702709640 1 Processor -- start 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.993+0000 7fe702709640 1 -- start start 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.993+0000 7fe702709640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 0x7fe6fc197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.993+0000 7fe702709640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc10a870 con 0x7fe6fc100420 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.993+0000 7fe6fbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 0x7fe6fc197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.993+0000 7fe6fbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 0x7fe6fc197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46402/0 (socket says 172.21.15.69:46402) 2024-08-02T17:38:14.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.993+0000 7fe6fbfff640 1 -- 172.21.15.69:0/3864446952 learned_addr learned my addr 172.21.15.69:0/3864446952 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:14.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.993+0000 7fe6fbfff640 1 -- 172.21.15.69:0/3864446952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc198250 con 0x7fe6fc100420 2024-08-02T17:38:14.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.994+0000 7fe6fbfff640 1 --2- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 0x7fe6fc197d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009230 tx=0x7fe6ec003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:14.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.994+0000 7fe6f9ffb640 1 -- 172.21.15.69:0/3864446952 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6ec03f070 con 0x7fe6fc100420 2024-08-02T17:38:14.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.994+0000 7fe6f9ffb640 1 -- 172.21.15.69:0/3864446952 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ec03bc80 con 0x7fe6fc100420 2024-08-02T17:38:14.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.994+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc198540 con 0x7fe6fc100420 2024-08-02T17:38:14.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.994+0000 7fe6f9ffb640 1 -- 172.21.15.69:0/3864446952 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6ec035030 con 0x7fe6fc100420 2024-08-02T17:38:14.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.994+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc198a00 con 0x7fe6fc100420 2024-08-02T17:38:14.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.995+0000 7fe6f9ffb640 1 -- 172.21.15.69:0/3864446952 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe6ec042070 con 0x7fe6fc100420 2024-08-02T17:38:14.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.995+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6fc101900 con 0x7fe6fc100420 2024-08-02T17:38:14.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.995+0000 7fe6f9ffb640 1 --2- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6e80390c0 0x7fe6e803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:14.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.995+0000 7fe6f9ffb640 1 -- 172.21.15.69:0/3864446952 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe6ec06ca80 con 0x7fe6fc100420 2024-08-02T17:38:14.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.995+0000 7fe6f3fff640 1 --2- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6e80390c0 0x7fe6e803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:14.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:14.995+0000 7fe6f3fff640 1 --2- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6e80390c0 0x7fe6e803b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40098c0 tx=0x7fe6e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:15.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:15.000+0000 7fe6f9ffb640 1 -- 172.21.15.69:0/3864446952 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe6ec031d90 con 0x7fe6fc100420 2024-08-02T17:38:15.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:15.170+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}) -- 0x7fe6fc100880 con 0x7fe6e80390c0 2024-08-02T17:38:16.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:15 smithi069 ceph-mon[32746]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi083", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:17.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: Deploying cephadm binary to smithi083 2024-08-02T17:38:17.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:17.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:17.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:17.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:17.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi077", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:38:17.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:17.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:17.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: Updating smithi077:/etc/ceph/ceph.conf 2024-08-02T17:38:17.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:17.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:17.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:17 smithi069 ceph-mon[32746]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:18.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.358+0000 7fe6f9ffb640 1 -- 172.21.15.69:0/3864446952 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe6fc100880 con 0x7fe6e80390c0 2024-08-02T17:38:18.360 INFO:teuthology.orchestra.run.smithi069.stdout:Added host 'smithi083' with addr '172.21.15.83' 2024-08-02T17:38:18.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6e80390c0 msgr2=0x7fe6e803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 --2- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6e80390c0 0x7fe6e803b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40098c0 tx=0x7fe6e4006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 msgr2=0x7fe6fc197d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 --2- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 0x7fe6fc197d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec009230 tx=0x7fe6ec003d40 comp rx=0 tx=0).stop 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 shutdown_connections 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 --2- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6e80390c0 0x7fe6e803b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 --2- 172.21.15.69:0/3864446952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe6fc100420 0x7fe6fc197d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 >> 172.21.15.69:0/3864446952 conn(0x7fe6fc0fbee0 msgr2=0x7fe6fc0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 shutdown_connections 2024-08-02T17:38:18.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:18.360+0000 7fe702709640 1 -- 172.21.15.69:0/3864446952 wait complete. 2024-08-02T17:38:18.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:18 smithi069 ceph-mon[32746]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:38:18.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:18.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:18.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:18.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:18.631 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:18.958 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch host ls --format=json 2024-08-02T17:38:19.202 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/2012575552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 msgr2=0x7f1ca0105d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 --2- 172.21.15.69:0/2012575552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 0x7f1ca0105d40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009880 tx=0x7f1c9402a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/2012575552 shutdown_connections 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 --2- 172.21.15.69:0/2012575552 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 0x7f1ca0105d40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/2012575552 >> 172.21.15.69:0/2012575552 conn(0x7f1ca00fb620 msgr2=0x7f1ca00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/2012575552 shutdown_connections 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/2012575552 wait complete. 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 Processor -- start 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 -- start start 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 0x7f1ca0101530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:19.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.605+0000 7f1ca7c9c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca010a020 con 0x7f1ca0105960 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1ca5a38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 0x7f1ca0101530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1ca5a38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 0x7f1ca0101530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46420/0 (socket says 172.21.15.69:46420) 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1ca5a38640 1 -- 172.21.15.69:0/1931868153 learned_addr learned my addr 172.21.15.69:0/1931868153 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1ca5a38640 1 -- 172.21.15.69:0/1931868153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca0103420 con 0x7f1ca0105960 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1ca5a38640 1 --2- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 0x7f1ca0101530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009230 tx=0x7f1c94003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1c92ffd640 1 -- 172.21.15.69:0/1931868153 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c9403f070 con 0x7f1ca0105960 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1c92ffd640 1 -- 172.21.15.69:0/1931868153 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c9403bd40 con 0x7f1ca0105960 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca0101ad0 con 0x7f1ca0105960 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1c92ffd640 1 -- 172.21.15.69:0/1931868153 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c94035030 con 0x7f1ca0105960 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.606+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca0101f90 con 0x7f1ca0105960 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.607+0000 7f1c92ffd640 1 -- 172.21.15.69:0/1931868153 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1c94042070 con 0x7f1ca0105960 2024-08-02T17:38:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.607+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ca01010a0 con 0x7f1ca0105960 2024-08-02T17:38:19.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.607+0000 7f1c92ffd640 1 --2- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c8c039060 0x7f1c8c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:19.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.607+0000 7f1c92ffd640 1 -- 172.21.15.69:0/1931868153 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1c9406cee0 con 0x7f1ca0105960 2024-08-02T17:38:19.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.607+0000 7f1ca5237640 1 --2- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c8c039060 0x7f1c8c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:19.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.608+0000 7f1ca5237640 1 --2- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c8c039060 0x7f1c8c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0098c0 tx=0x7f1c9c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:19.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.612+0000 7f1c92ffd640 1 -- 172.21.15.69:0/1931868153 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1c9403a890 con 0x7f1ca0105960 2024-08-02T17:38:19.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.785+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ca0102a60 con 0x7f1c8c039060 2024-08-02T17:38:19.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.786+0000 7f1c92ffd640 1 -- 172.21.15.69:0/1931868153 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f1ca0102a60 con 0x7f1c8c039060 2024-08-02T17:38:19.788 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:19.788 INFO:teuthology.orchestra.run.smithi069.stdout:[{"addr": "172.21.15.69", "hostname": "smithi069", "labels": [], "status": ""}, {"addr": "172.21.15.77", "hostname": "smithi077", "labels": [], "status": ""}, {"addr": "172.21.15.83", "hostname": "smithi083", "labels": [], "status": ""}] 2024-08-02T17:38:19.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c8c039060 msgr2=0x7f1c8c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:19.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 --2- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c8c039060 0x7f1c8c03b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0098c0 tx=0x7f1c9c006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:19.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 msgr2=0x7f1ca0101530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:19.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 --2- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 0x7f1ca0101530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1c94009230 tx=0x7f1c94003d80 comp rx=0 tx=0).stop 2024-08-02T17:38:19.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 shutdown_connections 2024-08-02T17:38:19.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 --2- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c8c039060 0x7f1c8c03b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:19.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 --2- 172.21.15.69:0/1931868153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ca0105960 0x7f1ca0101530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:19.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 >> 172.21.15.69:0/1931868153 conn(0x7f1ca00fb620 msgr2=0x7f1ca01048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:19.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 shutdown_connections 2024-08-02T17:38:19.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:19.789+0000 7f1ca7c9c640 1 -- 172.21.15.69:0/1931868153 wait complete. 2024-08-02T17:38:19.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:19 smithi069 ceph-mon[32746]: Added host smithi083 2024-08-02T17:38:19.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:19 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:19.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:19 smithi069 ceph-mon[32746]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:19.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:19 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:20.315 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-02T17:38:20.316 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd crush tunables default 2024-08-02T17:38:20.561 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:20.982 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:20.982 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:20 smithi069 ceph-mon[32746]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:38:21.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.062+0000 7f947b2c6640 1 -- 172.21.15.69:0/1663013098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 msgr2=0x7f9474101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:21.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.062+0000 7f947b2c6640 1 --2- 172.21.15.69:0/1663013098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 0x7f9474101900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9464009880 tx=0x7f946402a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:21.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.062+0000 7f947b2c6640 1 -- 172.21.15.69:0/1663013098 shutdown_connections 2024-08-02T17:38:21.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.062+0000 7f947b2c6640 1 --2- 172.21.15.69:0/1663013098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 0x7f9474101900 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:21.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.062+0000 7f947b2c6640 1 -- 172.21.15.69:0/1663013098 >> 172.21.15.69:0/1663013098 conn(0x7f94740fcfc0 msgr2=0x7f94740ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:21.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.062+0000 7f947b2c6640 1 -- 172.21.15.69:0/1663013098 shutdown_connections 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.062+0000 7f947b2c6640 1 -- 172.21.15.69:0/1663013098 wait complete. 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.062+0000 7f947b2c6640 1 Processor -- start 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f947b2c6640 1 -- start start 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f947b2c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 0x7f9474074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f947b2c6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f947410b970 con 0x7f9474101520 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f9479062640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 0x7f9474074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f9479062640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 0x7f9474074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46440/0 (socket says 172.21.15.69:46440) 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f9479062640 1 -- 172.21.15.69:0/419228405 learned_addr learned my addr 172.21.15.69:0/419228405 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f9479062640 1 -- 172.21.15.69:0/419228405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9474076110 con 0x7f9474101520 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f9479062640 1 --2- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 0x7f9474074220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9464009230 tx=0x7f9464003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f946a7fc640 1 -- 172.21.15.69:0/419228405 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f946403f070 con 0x7f9474101520 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f946a7fc640 1 -- 172.21.15.69:0/419228405 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f946403bc80 con 0x7f9474101520 2024-08-02T17:38:21.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94740747c0 con 0x7f9474101520 2024-08-02T17:38:21.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f946a7fc640 1 -- 172.21.15.69:0/419228405 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9464035030 con 0x7f9474101520 2024-08-02T17:38:21.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.063+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9474074c80 con 0x7f9474101520 2024-08-02T17:38:21.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.064+0000 7f946a7fc640 1 -- 172.21.15.69:0/419228405 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9464042070 con 0x7f9474101520 2024-08-02T17:38:21.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.064+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9474102a00 con 0x7f9474101520 2024-08-02T17:38:21.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.065+0000 7f946a7fc640 1 --2- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f94600390c0 0x7f946003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:21.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.065+0000 7f946a7fc640 1 -- 172.21.15.69:0/419228405 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f946406c9f0 con 0x7f9474101520 2024-08-02T17:38:21.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.065+0000 7f9478861640 1 --2- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f94600390c0 0x7f946003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:21.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.065+0000 7f9478861640 1 --2- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f94600390c0 0x7f946003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f94700098c0 tx=0x7f9470006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:21.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.069+0000 7f946a7fc640 1 -- 172.21.15.69:0/419228405 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f94640433b0 con 0x7f9474101520 2024-08-02T17:38:21.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.238+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9474075750 con 0x7f9474101520 2024-08-02T17:38:21.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.703+0000 7f946a7fc640 1 -- 172.21.15.69:0/419228405 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f94640372d0 con 0x7f9474101520 2024-08-02T17:38:21.704 INFO:teuthology.orchestra.run.smithi069.stderr:adjusted tunables profile to default 2024-08-02T17:38:21.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f94600390c0 msgr2=0x7f946003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:21.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 --2- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f94600390c0 0x7f946003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f94700098c0 tx=0x7f9470006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:21.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 msgr2=0x7f9474074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:21.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 --2- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 0x7f9474074220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9464009230 tx=0x7f9464003d40 comp rx=0 tx=0).stop 2024-08-02T17:38:21.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 shutdown_connections 2024-08-02T17:38:21.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 --2- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f94600390c0 0x7f946003b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:21.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 --2- 172.21.15.69:0/419228405 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9474101520 0x7f9474074220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:21.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 >> 172.21.15.69:0/419228405 conn(0x7f94740fcfc0 msgr2=0x7f9474108f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:21.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 shutdown_connections 2024-08-02T17:38:21.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:21.706+0000 7f947b2c6640 1 -- 172.21.15.69:0/419228405 wait complete. 2024-08-02T17:38:22.002 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:21 smithi069 ceph-mon[32746]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:22.002 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:21 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/419228405' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-02T17:38:22.320 INFO:tasks.cephadm:Adding mon.a on smithi069 2024-08-02T17:38:22.320 INFO:tasks.cephadm:Adding mon.b on smithi077 2024-08-02T17:38:22.320 INFO:tasks.cephadm:Adding mon.c on smithi083 2024-08-02T17:38:22.321 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch apply mon '3;smithi069:172.21.15.69=a;smithi077:172.21.15.77=b;smithi083:172.21.15.83=c' 2024-08-02T17:38:22.643 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-02T17:38:22.691 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-02T17:38:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/419228405' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-02T17:38:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:23.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:23.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:38:23.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:23.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 -- 172.21.15.83:0/3672529161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 msgr2=0x7efdd4100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 --2- 172.21.15.83:0/3672529161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 0x7efdd4100670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009880 tx=0x7efdd002a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:23.146 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 -- 172.21.15.83:0/3672529161 shutdown_connections 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 --2- 172.21.15.83:0/3672529161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 0x7efdd4100670 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 -- 172.21.15.83:0/3672529161 >> 172.21.15.83:0/3672529161 conn(0x7efdd40fba00 msgr2=0x7efdd40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 -- 172.21.15.83:0/3672529161 shutdown_connections 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 -- 172.21.15.83:0/3672529161 wait complete. 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 Processor -- start 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.137+0000 7efddbfde640 1 -- start start 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.138+0000 7efddbfde640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 0x7efdd40fd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.138+0000 7efddbfde640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4106320 con 0x7efdd4100270 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.138+0000 7efdd9d7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 0x7efdd40fd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.138+0000 7efdd9d7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 0x7efdd40fd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.83:58888/0 (socket says 172.21.15.83:58888) 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.138+0000 7efdd9d7a640 1 -- 172.21.15.83:0/2984112237 learned_addr learned my addr 172.21.15.83:0/2984112237 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:38:23.147 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.138+0000 7efdd9d7a640 1 -- 172.21.15.83:0/2984112237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd40fde00 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.138+0000 7efdd9d7a640 1 --2- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 0x7efdd40fd8c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdd0004f60 tx=0x7efdd00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.139+0000 7efdcaffd640 1 -- 172.21.15.83:0/2984112237 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdd003f070 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.139+0000 7efdcaffd640 1 -- 172.21.15.83:0/2984112237 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd0004d40 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.139+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd40fbff0 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.139+0000 7efdcaffd640 1 -- 172.21.15.83:0/2984112237 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdd0035030 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.139+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd40fc530 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.140+0000 7efdcaffd640 1 -- 172.21.15.83:0/2984112237 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efdd0042070 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.140+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd4106aa0 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.140+0000 7efdcaffd640 1 --2- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7efdc0039060 0x7efdc003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.140+0000 7efdcaffd640 1 -- 172.21.15.83:0/2984112237 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efdd006cfb0 con 0x7efdd4100270 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.140+0000 7efdd9579640 1 --2- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7efdc0039060 0x7efdc003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.141+0000 7efdd9579640 1 --2- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7efdc0039060 0x7efdc003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efdc40098c0 tx=0x7efdc4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:23.148 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.145+0000 7efdcaffd640 1 -- 172.21.15.83:0/2984112237 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7efdd0042360 con 0x7efdd4100270 2024-08-02T17:38:23.333 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.331+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi077:172.21.15.77=b;smithi083:172.21.15.83=c", "target": ["mon-mgr", ""]}) -- 0x7efdd41900a0 con 0x7efdc0039060 2024-08-02T17:38:23.338 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.337+0000 7efdcaffd640 1 -- 172.21.15.83:0/2984112237 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efdd41900a0 con 0x7efdc0039060 2024-08-02T17:38:23.338 INFO:teuthology.orchestra.run.smithi083.stdout:Scheduled mon update... 2024-08-02T17:38:23.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7efdc0039060 msgr2=0x7efdc003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:23.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 --2- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7efdc0039060 0x7efdc003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efdc40098c0 tx=0x7efdc4006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:23.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 msgr2=0x7efdd40fd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:23.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 --2- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 0x7efdd40fd8c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdd0004f60 tx=0x7efdd00048f0 comp rx=0 tx=0).stop 2024-08-02T17:38:23.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 shutdown_connections 2024-08-02T17:38:23.340 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 --2- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7efdc0039060 0x7efdc003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:23.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 --2- 172.21.15.83:0/2984112237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efdd4100270 0x7efdd40fd8c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:23.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 >> 172.21.15.83:0/2984112237 conn(0x7efdd40fba00 msgr2=0x7efdd4069860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:23.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 shutdown_connections 2024-08-02T17:38:23.341 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:23.339+0000 7efddbfde640 1 -- 172.21.15.83:0/2984112237 wait complete. 2024-08-02T17:38:23.922 DEBUG:teuthology.orchestra.run.smithi077:mon.b> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.b.service 2024-08-02T17:38:23.925 DEBUG:teuthology.orchestra.run.smithi083:mon.c> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.c.service 2024-08-02T17:38:23.928 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-02T17:38:23.928 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph mon dump -f json 2024-08-02T17:38:24.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-02T17:38:24.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:24.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:24.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:24.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:24.898 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:24 smithi069 ceph-mon[32746]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:38:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:24 smithi069 ceph-mon[32746]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi077:172.21.15.77=b;smithi083:172.21.15.83=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:24 smithi069 ceph-mon[32746]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi077:172.21.15.77=b;smithi083:172.21.15.83=c;count:3 2024-08-02T17:38:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:24 smithi069 ceph-mon[32746]: Deploying daemon mon.c on smithi083 2024-08-02T17:38:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:25 smithi069 ceph-mon[32746]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:26.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.346+0000 7f5dc9723640 1 -- 172.21.15.83:0/1824138041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 msgr2=0x7f5dc41018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:26.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.346+0000 7f5dc9723640 1 --2- 172.21.15.83:0/1824138041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 0x7f5dc41018f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009880 tx=0x7f5db402a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:26.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.347+0000 7f5dc9723640 1 -- 172.21.15.83:0/1824138041 shutdown_connections 2024-08-02T17:38:26.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.347+0000 7f5dc9723640 1 --2- 172.21.15.83:0/1824138041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 0x7f5dc41018f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:26.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.347+0000 7f5dc9723640 1 -- 172.21.15.83:0/1824138041 >> 172.21.15.83:0/1824138041 conn(0x7f5dc40fcf90 msgr2=0x7f5dc40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:26.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.347+0000 7f5dc9723640 1 -- 172.21.15.83:0/1824138041 shutdown_connections 2024-08-02T17:38:26.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.347+0000 7f5dc9723640 1 -- 172.21.15.83:0/1824138041 wait complete. 2024-08-02T17:38:26.350 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.347+0000 7f5dc9723640 1 Processor -- start 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.348+0000 7f5dc9723640 1 -- start start 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.348+0000 7f5dc9723640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 0x7f5dc4196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.348+0000 7f5dc9723640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc410b960 con 0x7f5dc4101510 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.348+0000 7f5dc2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 0x7f5dc4196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.348+0000 7f5dc2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 0x7f5dc4196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.83:58906/0 (socket says 172.21.15.83:58906) 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.348+0000 7f5dc2ffd640 1 -- 172.21.15.83:0/70105926 learned_addr learned my addr 172.21.15.83:0/70105926 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.348+0000 7f5dc2ffd640 1 -- 172.21.15.83:0/70105926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc41973e0 con 0x7f5dc4101510 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.349+0000 7f5dc2ffd640 1 --2- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 0x7f5dc4196ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009230 tx=0x7f5db4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.349+0000 7f5dc0ff9640 1 -- 172.21.15.83:0/70105926 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db403f070 con 0x7f5dc4101510 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.349+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc41976d0 con 0x7f5dc4101510 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.349+0000 7f5dc0ff9640 1 -- 172.21.15.83:0/70105926 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db403bd40 con 0x7f5dc4101510 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.349+0000 7f5dc0ff9640 1 -- 172.21.15.83:0/70105926 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5db4035030 con 0x7f5dc4101510 2024-08-02T17:38:26.351 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.349+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc419b770 con 0x7f5dc4101510 2024-08-02T17:38:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.350+0000 7f5dc0ff9640 1 -- 172.21.15.83:0/70105926 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5db4042070 con 0x7f5dc4101510 2024-08-02T17:38:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.350+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc41029f0 con 0x7f5dc4101510 2024-08-02T17:38:26.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.350+0000 7f5dc0ff9640 1 --2- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5db0039060 0x7f5db003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.350+0000 7f5dc0ff9640 1 -- 172.21.15.83:0/70105926 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5db406cc70 con 0x7f5dc4101510 2024-08-02T17:38:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.350+0000 7f5dbbfff640 1 --2- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5db0039060 0x7f5db003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:26.353 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.351+0000 7f5dbbfff640 1 --2- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5db0039060 0x7f5db003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0098c0 tx=0x7f5dac006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:26.356 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.355+0000 7f5dc0ff9640 1 -- 172.21.15.83:0/70105926 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5db4036df0 con 0x7f5dc4101510 2024-08-02T17:38:26.599 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.598+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5dc4101970 con 0x7f5dc4101510 2024-08-02T17:38:26.600 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.599+0000 7f5dc0ff9640 1 -- 172.21.15.83:0/70105926 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f5db4030d60 con 0x7f5dc4101510 2024-08-02T17:38:26.600 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:38:26.601 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":1,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","modified":"2024-08-02T17:37:01.657332Z","created":"2024-08-02T17:37:01.657332Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-02T17:38:26.601 INFO:teuthology.orchestra.run.smithi083.stderr:dumped monmap epoch 1 2024-08-02T17:38:26.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5db0039060 msgr2=0x7f5db003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:26.602 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 --2- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5db0039060 0x7f5db003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0098c0 tx=0x7f5dac006db0 comp rx=0 tx=0).stop 2024-08-02T17:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 msgr2=0x7f5dc4196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 --2- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 0x7f5dc4196ea0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009230 tx=0x7f5db4003d80 comp rx=0 tx=0).stop 2024-08-02T17:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 shutdown_connections 2024-08-02T17:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 --2- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5db0039060 0x7f5db003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 --2- 172.21.15.83:0/70105926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dc4101510 0x7f5dc4196ea0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 >> 172.21.15.83:0/70105926 conn(0x7f5dc40fcf90 msgr2=0x7f5dc4108f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 shutdown_connections 2024-08-02T17:38:26.603 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:26.601+0000 7f5dc9723640 1 -- 172.21.15.83:0/70105926 wait complete. 2024-08-02T17:38:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:26 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/70105926' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:28.010 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:27 smithi083 podman[37010]: 2024-08-02 17:38:27.987845879 +0000 UTC m=+0.341423982 container start f8813bf2428c59769581a26e075798c35ea5e3f6fac6237f51109eded38d2781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-c, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-02T17:38:28.010 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: set uid:gid to 167:167 (ceph:ceph) 2024-08-02T17:38:28.010 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: ceph version 19.1.0-1260-g26c3fb8e (26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea) squid (rc), process ceph-mon, pid 2 2024-08-02T17:38:28.010 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: pidfile_write: ignore empty --pid-file 2024-08-02T17:38:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:27 smithi069 ceph-mon[32746]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: load: jerasure load: lrc 2024-08-02T17:38:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: RocksDB version: 7.9.2 2024-08-02T17:38:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Git sha 0 2024-08-02T17:38:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Compile date 2024-08-01 20:56:42 2024-08-02T17:38:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: DB SUMMARY 2024-08-02T17:38:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: DB Session ID: 0U6PJGMFH878J7XCX0FP 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: CURRENT file: CURRENT 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: IDENTITY file: IDENTITY 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.error_if_exists: 0 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.create_if_missing: 0 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.paranoid_checks: 1 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-02T17:38:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.env: 0x559b10b867a0 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.fs: PosixFileSystem 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.info_log: 0x559b11b19500 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_file_opening_threads: 16 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.statistics: (nil) 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.use_fsync: 0 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_log_file_size: 0 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.keep_log_file_num: 1000 2024-08-02T17:38:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.recycle_log_file_num: 0 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.allow_fallocate: 1 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.allow_mmap_reads: 0 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.allow_mmap_writes: 0 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.use_direct_reads: 0 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.create_missing_column_families: 0 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.db_log_dir: 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.wal_dir: 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-02T17:38:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.advise_random_on_open: 1 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.db_write_buffer_size: 0 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.write_buffer_manager: 0x559b11b28dc0 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.rate_limiter: (nil) 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.wal_recovery_mode: 2 2024-08-02T17:38:28.276 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.enable_thread_tracking: 0 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.enable_pipelined_write: 0 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.unordered_write: 0 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.row_cache: None 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.wal_filter: None 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.allow_ingest_behind: 0 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.two_write_queues: 0 2024-08-02T17:38:28.277 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.manual_wal_flush: 0 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.wal_compression: 0 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.atomic_flush: 0 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.log_readahead_size: 0 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.best_efforts_recovery: 0 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-02T17:38:28.278 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.allow_data_in_errors: 0 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.db_host_id: __hostname__ 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_background_jobs: 2 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_background_compactions: -1 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_subcompactions: 1 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_total_wal_size: 0 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-02T17:38:28.279 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-02T17:38:28.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-02T17:38:28.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_open_files: -1 2024-08-02T17:38:28.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bytes_per_sync: 0 2024-08-02T17:38:28.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-02T17:38:28.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-02T17:38:28.280 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_readahead_size: 0 2024-08-02T17:38:28.281 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_background_flushes: -1 2024-08-02T17:38:28.281 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Compression algorithms supported: 2024-08-02T17:38:28.281 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: kZSTD supported: 0 2024-08-02T17:38:28.281 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: kXpressCompression supported: 0 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: kBZip2Compression supported: 0 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: kLZ4Compression supported: 1 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: kZlibCompression supported: 1 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: kLZ4HCCompression supported: 1 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: kSnappyCompression supported: 1 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-08-02T17:38:28.282 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.merge_operator: 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_filter: None 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_filter_factory: None 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.sst_partitioner_factory: None 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559b11b19120) 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout: cache_index_and_filter_blocks: 1 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-02T17:38:28.283 INFO:journalctl@ceph.mon.c.smithi083.stdout: pin_top_level_index_and_filter: 1 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: index_type: 0 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: data_block_index_type: 0 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: index_shortening: 1 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: checksum: 4 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: no_block_cache: 0 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_cache: 0x559b11b0d090 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_cache_name: BinnedLRUCache 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_cache_options: 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: capacity : 536870912 2024-08-02T17:38:28.284 INFO:journalctl@ceph.mon.c.smithi083.stdout: num_shard_bits : 4 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: strict_capacity_limit : 0 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: high_pri_pool_ratio: 0.000 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_cache_compressed: (nil) 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: persistent_cache: (nil) 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_size: 4096 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_size_deviation: 10 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_restart_interval: 16 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: index_block_restart_interval: 1 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: metadata_block_size: 4096 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: partition_filters: 0 2024-08-02T17:38:28.285 INFO:journalctl@ceph.mon.c.smithi083.stdout: use_delta_encoding: 1 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: filter_policy: bloomfilter 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: whole_key_filtering: 1 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: verify_compression: 0 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: read_amp_bytes_per_bit: 0 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: format_version: 5 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: enable_index_compression: 1 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: block_align: 0 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: max_auto_readahead_size: 262144 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: prepopulate_block_cache: 0 2024-08-02T17:38:28.286 INFO:journalctl@ceph.mon.c.smithi083.stdout: initial_auto_readahead_size: 8192 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout: num_file_reads_for_auto_readahead: 2 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.write_buffer_size: 33554432 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_write_buffer_number: 2 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression: NoCompression 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression: Disabled 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.prefix_extractor: nullptr 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.num_levels: 7 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-02T17:38:28.287 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-02T17:38:28.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-02T17:38:28.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-02T17:38:28.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-02T17:38:28.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-02T17:38:28.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-02T17:38:28.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-02T17:38:28.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-02T17:38:28.288 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.level: 32767 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.strategy: 0 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-02T17:38:28.290 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.enabled: false 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.target_file_size_base: 67108864 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-02T17:38:28.291 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.arena_block_size: 1048576 2024-08-02T17:38:28.292 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.disable_auto_compactions: 0 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-02T17:38:28.293 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.inplace_update_support: 0 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.bloom_locality: 0 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.max_successive_merges: 0 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.paranoid_file_checks: 0 2024-08-02T17:38:28.294 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.force_consistency_checks: 1 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.report_bg_io_stats: 0 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.ttl: 2592000 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.enable_blob_files: false 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.min_blob_size: 0 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.blob_file_size: 268435456 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-02T17:38:28.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.blob_file_starting_level: 0 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b695a1c6-422e-4e78-b979-53a660ea9ed4 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722620308024845, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722620308025241, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1722620308, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b695a1c6-422e-4e78-b979-53a660ea9ed4", "db_session_id": "0U6PJGMFH878J7XCX0FP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: EVENT_LOG_v1 {"time_micros": 1722620308025333, "job": 1, "event": "recovery_finished"} 2024-08-02T17:38:28.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559b11b3ae00 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: DB pointer 0x559b11c34000 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout: ** DB Stats ** 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-02T17:38:28.297 INFO:journalctl@ceph.mon.c.smithi083.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: ** Compaction Stats [default] ** 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-02T17:38:28.298 INFO:journalctl@ceph.mon.c.smithi083.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: ** Compaction Stats [default] ** 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-02T17:38:28.299 INFO:journalctl@ceph.mon.c.smithi083.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: Cumulative compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: Interval compaction: 0.00 GB write, 0.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: Block cache BinnedLRUCache@0x559b11b0d090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: using public_addr v2:172.21.15.83:0/0 -> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: starting mon.c rank -1 at public addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] at bind addrs [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(???) e0 preinit fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:28.300 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).mds e1 new map 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).mds e1 print_map 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout: e1 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout: btime 2024-08-02T17:37:04:618450+0000 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout: legacy client fscid: -1 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout: 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout: No filesystems configured 2024-08-02T17:38:28.301 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.302 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: Added host smithi083 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/419228405' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/419228405' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:28.303 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi083", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.304 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi069:172.21.15.69=a;smithi077:172.21.15.77=b;smithi083:172.21.15.83=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: Saving service mon spec with placement smithi069:172.21.15.69=a;smithi077:172.21.15.77=b;smithi083:172.21.15.83=c;count:3 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: Deploying daemon mon.c on smithi083 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/70105926' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:28.305 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:28.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 ceph-mon[37031]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-02T17:38:28.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 bash[37010]: f8813bf2428c59769581a26e075798c35ea5e3f6fac6237f51109eded38d2781 2024-08-02T17:38:28.306 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:28 smithi083 systemd[1]: Started Ceph mon.c for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:38:28.535 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-02T17:38:28.536 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph mon dump -f json 2024-08-02T17:38:28.781 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:38:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: mon.a calling monitor election 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: mon.c calling monitor election 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: monmap epoch 2 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: last_changed 2024-08-02T17:38:28.049829+0000 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: min_mon_release 19 (squid) 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: election_strategy: 1 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.c 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: fsmap 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: mgrmap e13: x(active, since 36s) 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: overall HEALTH_OK 2024-08-02T17:38:33.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:33.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:33.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:33.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:33.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:33.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:38:33.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: mon.a calling monitor election 2024-08-02T17:38:33.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: mon.c calling monitor election 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-02T17:38:33.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: monmap epoch 2 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: last_changed 2024-08-02T17:38:28.049829+0000 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: min_mon_release 19 (squid) 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: election_strategy: 1 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.c 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: fsmap 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:33.612 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: mgrmap e13: x(active, since 36s) 2024-08-02T17:38:33.613 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: overall HEALTH_OK 2024-08-02T17:38:33.613 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:33.613 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:33.613 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:33.613 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:33.613 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:33.622 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.620+0000 7f78f7507640 1 -- 172.21.15.83:0/251998044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 msgr2=0x7f78dc0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:33.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.620+0000 7f78f7507640 1 --2- 172.21.15.83:0/251998044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 0x7f78dc0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f78f0101720 tx=0x7f78e002c490 comp rx=0 tx=0).stop 2024-08-02T17:38:33.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.620+0000 7f78f7507640 1 -- 172.21.15.83:0/251998044 shutdown_connections 2024-08-02T17:38:33.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.620+0000 7f78f7507640 1 --2- 172.21.15.83:0/251998044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 0x7f78dc0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:33.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.620+0000 7f78f7507640 1 -- 172.21.15.83:0/251998044 >> 172.21.15.83:0/251998044 conn(0x7f78f00fbf50 msgr2=0x7f78f00fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:33.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.620+0000 7f78f7507640 1 -- 172.21.15.83:0/251998044 shutdown_connections 2024-08-02T17:38:33.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.620+0000 7f78f7507640 1 -- 172.21.15.83:0/251998044 wait complete. 2024-08-02T17:38:33.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.621+0000 7f78f7507640 1 Processor -- start 2024-08-02T17:38:33.623 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.621+0000 7f78f7507640 1 -- start start 2024-08-02T17:38:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f7507640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 0x7f78f0197000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f7507640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 0x7f78f019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f7507640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f78f01083a0 con 0x7f78dc003530 2024-08-02T17:38:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f7507640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f78f0108220 con 0x7f78f0197540 2024-08-02T17:38:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f52a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 0x7f78f0197000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f4aa2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 0x7f78f019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f52a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 0x7f78f0197000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.83:54148/0 (socket says 172.21.15.83:54148) 2024-08-02T17:38:33.625 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f52a3640 1 -- 172.21.15.83:0/593583685 learned_addr learned my addr 172.21.15.83:0/593583685 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f4aa2640 1 -- 172.21.15.83:0/593583685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 msgr2=0x7f78f019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f4aa2640 1 -- 172.21.15.83:0/593583685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 msgr2=0x7f78f019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f4aa2640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 0x7f78f019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f4aa2640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 0x7f78f019b8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f52a3640 1 -- 172.21.15.83:0/593583685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 msgr2=0x7f78f019b8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f52a3640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 0x7f78f019b8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.622+0000 7f78f52a3640 1 -- 172.21.15.83:0/593583685 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f0197b60 con 0x7f78dc003530 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.623+0000 7f78f52a3640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 0x7f78f0197000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005f30 tx=0x7f78e0033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.623+0000 7f78e67fc640 1 -- 172.21.15.83:0/593583685 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f78e0038040 con 0x7f78dc003530 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.623+0000 7f78e67fc640 1 -- 172.21.15.83:0/593583685 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78e0032e00 con 0x7f78dc003530 2024-08-02T17:38:33.626 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.623+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f019be10 con 0x7f78dc003530 2024-08-02T17:38:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.623+0000 7f78e67fc640 1 -- 172.21.15.83:0/593583685 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f78e00049e0 con 0x7f78dc003530 2024-08-02T17:38:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.623+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f019c350 con 0x7f78dc003530 2024-08-02T17:38:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.624+0000 7f78e67fc640 1 -- 172.21.15.83:0/593583685 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f78e0043020 con 0x7f78dc003530 2024-08-02T17:38:33.627 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.624+0000 7f78e67fc640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f78d4038e50 0x7f78d403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.624+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f01019d0 con 0x7f78dc003530 2024-08-02T17:38:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.624+0000 7f78e67fc640 1 -- 172.21.15.83:0/593583685 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f78e006f570 con 0x7f78dc003530 2024-08-02T17:38:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.624+0000 7f78f4aa2640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f78d4038e50 0x7f78d403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:33.628 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.625+0000 7f78f4aa2640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f78d4038e50 0x7f78d403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f78ec006fd0 tx=0x7f78ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:33.630 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.629+0000 7f78e67fc640 1 -- 172.21.15.83:0/593583685 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f78e004a020 con 0x7f78dc003530 2024-08-02T17:38:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.863+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f78f019c630 con 0x7f78dc003530 2024-08-02T17:38:33.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.864+0000 7f78e67fc640 1 -- 172.21.15.83:0/593583685 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f78e004cbc0 con 0x7f78dc003530 2024-08-02T17:38:33.866 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:38:33.866 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","modified":"2024-08-02T17:38:28.049829Z","created":"2024-08-02T17:37:01.657332Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-02T17:38:33.868 INFO:teuthology.orchestra.run.smithi083.stderr:dumped monmap epoch 2 2024-08-02T17:38:33.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f78d4038e50 msgr2=0x7f78d403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:33.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f78d4038e50 0x7f78d403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f78ec006fd0 tx=0x7f78ec008040 comp rx=0 tx=0).stop 2024-08-02T17:38:33.868 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 msgr2=0x7f78f0197000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:33.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 0x7f78f0197000 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f78e0005f30 tx=0x7f78e0033680 comp rx=0 tx=0).stop 2024-08-02T17:38:33.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 shutdown_connections 2024-08-02T17:38:33.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f78d4038e50 0x7f78d403b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:33.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f78f0197540 0x7f78f019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:33.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 --2- 172.21.15.83:0/593583685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f78dc003530 0x7f78f0197000 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:33.869 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 >> 172.21.15.83:0/593583685 conn(0x7f78f00fbf50 msgr2=0x7f78f00fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:33.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 shutdown_connections 2024-08-02T17:38:33.870 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:33.866+0000 7f78f7507640 1 -- 172.21.15.83:0/593583685 wait complete. 2024-08-02T17:38:34.148 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:34 smithi083 ceph-mon[37031]: Deploying daemon mon.b on smithi077 2024-08-02T17:38:34.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:34 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/593583685' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:34.149 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:34.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:34 smithi069 ceph-mon[32746]: Deploying daemon mon.b on smithi077 2024-08-02T17:38:34.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:34 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/593583685' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:34.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:35.388 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-02T17:38:35.388 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph mon dump -f json 2024-08-02T17:38:35.414 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:35 smithi083 ceph-mon[37031]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:35.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:35 smithi069 ceph-mon[32746]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:35.643 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:38:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 -- 172.21.15.83:0/3418759445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 msgr2=0x7f96f41008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 --2- 172.21.15.83:0/3418759445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 0x7f96f41008f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009880 tx=0x7f96e402a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 -- 172.21.15.83:0/3418759445 shutdown_connections 2024-08-02T17:38:36.223 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 --2- 172.21.15.83:0/3418759445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 0x7f96f41008f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 -- 172.21.15.83:0/3418759445 >> 172.21.15.83:0/3418759445 conn(0x7f96f40fbfb0 msgr2=0x7f96f40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 -- 172.21.15.83:0/3418759445 shutdown_connections 2024-08-02T17:38:36.224 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 -- 172.21.15.83:0/3418759445 wait complete. 2024-08-02T17:38:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 Processor -- start 2024-08-02T17:38:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.222+0000 7f96f8f63640 1 -- start start 2024-08-02T17:38:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f8f63640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 0x7f96f4197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f8f63640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 0x7f96f419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f8f63640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f96f410a960 con 0x7f96f4100510 2024-08-02T17:38:36.225 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f8f63640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96f410a7e0 con 0x7f96f4197690 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 0x7f96f4197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96ebfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 0x7f96f419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 0x7f96f4197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.83:54174/0 (socket says 172.21.15.83:54174) 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f259c640 1 -- 172.21.15.83:0/2772635836 learned_addr learned my addr 172.21.15.83:0/2772635836 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96ebfff640 1 -- 172.21.15.83:0/2772635836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 msgr2=0x7f96f419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96ebfff640 1 -- 172.21.15.83:0/2772635836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 msgr2=0x7f96f419ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96ebfff640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 0x7f96f419ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96ebfff640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 0x7f96f419ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f259c640 1 -- 172.21.15.83:0/2772635836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 msgr2=0x7f96f419ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f259c640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 0x7f96f419ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f259c640 1 -- 172.21.15.83:0/2772635836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f4197cb0 con 0x7f96f4100510 2024-08-02T17:38:36.226 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.223+0000 7f96f259c640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 0x7f96f4197150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96e4004650 tx=0x7f96e4032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:36.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.224+0000 7f96eb7fe640 1 -- 172.21.15.83:0/2772635836 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f96e4035070 con 0x7f96f4100510 2024-08-02T17:38:36.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.224+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f419bf60 con 0x7f96f4100510 2024-08-02T17:38:36.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.224+0000 7f96eb7fe640 1 -- 172.21.15.83:0/2772635836 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e4003ce0 con 0x7f96f4100510 2024-08-02T17:38:36.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.224+0000 7f96eb7fe640 1 -- 172.21.15.83:0/2772635836 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f96e403bbd0 con 0x7f96f4100510 2024-08-02T17:38:36.227 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.224+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f419c4a0 con 0x7f96f4100510 2024-08-02T17:38:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.224+0000 7f96eb7fe640 1 -- 172.21.15.83:0/2772635836 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f96e4042030 con 0x7f96f4100510 2024-08-02T17:38:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.225+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f41019f0 con 0x7f96f4100510 2024-08-02T17:38:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.225+0000 7f96eb7fe640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96e0038e00 0x7f96e003b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.225+0000 7f96eb7fe640 1 -- 172.21.15.83:0/2772635836 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f96e4040070 con 0x7f96f4100510 2024-08-02T17:38:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.225+0000 7f96ebfff640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96e0038e00 0x7f96e003b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:36.228 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.225+0000 7f96ebfff640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96e0038e00 0x7f96e003b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006fd0 tx=0x7f96dc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:36.231 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.230+0000 7f96eb7fe640 1 -- 172.21.15.83:0/2772635836 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f96e4030e00 con 0x7f96f4100510 2024-08-02T17:38:36.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.460+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f96f419c750 con 0x7f96f4100510 2024-08-02T17:38:36.462 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.461+0000 7f96eb7fe640 1 -- 172.21.15.83:0/2772635836 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f96e403bd70 con 0x7f96f4100510 2024-08-02T17:38:36.463 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:38:36.463 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":2,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","modified":"2024-08-02T17:38:28.049829Z","created":"2024-08-02T17:37:01.657332Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-02T17:38:36.463 INFO:teuthology.orchestra.run.smithi083.stderr:dumped monmap epoch 2 2024-08-02T17:38:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96e0038e00 msgr2=0x7f96e003b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96e0038e00 0x7f96e003b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f96dc006fd0 tx=0x7f96dc008040 comp rx=0 tx=0).stop 2024-08-02T17:38:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 msgr2=0x7f96f4197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 0x7f96f4197150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f96e4004650 tx=0x7f96e4032c50 comp rx=0 tx=0).stop 2024-08-02T17:38:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 shutdown_connections 2024-08-02T17:38:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96e0038e00 0x7f96e003b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:36.465 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96f4197690 0x7f96f419ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:36.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 --2- 172.21.15.83:0/2772635836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96f4100510 0x7f96f4197150 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:36.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 >> 172.21.15.83:0/2772635836 conn(0x7f96f40fbfb0 msgr2=0x7f96f4069940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:36.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 shutdown_connections 2024-08-02T17:38:36.466 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:36.463+0000 7f96f8f63640 1 -- 172.21.15.83:0/2772635836 wait complete. 2024-08-02T17:38:36.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:36 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/2772635836' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:36.838 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:36 smithi077 systemd[1]: Started Ceph mon.b for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:38:36.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:36 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/2772635836' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:38.095 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-02T17:38:38.096 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph mon dump -f json 2024-08-02T17:38:38.352 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:38:41.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.810+0000 7f7e24e55640 1 -- 172.21.15.83:0/2658421024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 msgr2=0x7f7e080059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:41.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.810+0000 7f7e24e55640 1 --2- 172.21.15.83:0/2658421024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 0x7f7e080059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7e20102820 tx=0x7f7e1002bce0 comp rx=0 tx=0).stop 2024-08-02T17:38:41.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.810+0000 7f7e24e55640 1 -- 172.21.15.83:0/2658421024 shutdown_connections 2024-08-02T17:38:41.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.810+0000 7f7e24e55640 1 --2- 172.21.15.83:0/2658421024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 0x7f7e080059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:41.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.810+0000 7f7e24e55640 1 -- 172.21.15.83:0/2658421024 >> 172.21.15.83:0/2658421024 conn(0x7f7e200fd050 msgr2=0x7f7e200ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:41.812 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.810+0000 7f7e24e55640 1 -- 172.21.15.83:0/2658421024 shutdown_connections 2024-08-02T17:38:41.813 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.810+0000 7f7e24e55640 1 -- 172.21.15.83:0/2658421024 wait complete. 2024-08-02T17:38:41.814 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.811+0000 7f7e24e55640 1 Processor -- start 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e24e55640 1 -- start start 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e24e55640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 0x7f7e20074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e24e55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 0x7f7e20074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e24e55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 0x7f7e201a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e24e55640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e20109450 con 0x7f7e08003530 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e24e55640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201092d0 con 0x7f7e20075e60 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e24e55640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e201095d0 con 0x7f7e20074ca0 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 0x7f7e201a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 0x7f7e20074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:41.815 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 0x7f7e201a9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38822/0 (socket says 172.21.15.83:38822) 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1ed9d640 1 -- 172.21.15.83:0/76214898 learned_addr learned my addr 172.21.15.83:0/76214898 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 0x7f7e20074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.83:44706/0 (socket says 172.21.15.83:44706) 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1dd9b640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 0x7f7e20074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1ed9d640 1 -- 172.21.15.83:0/76214898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 msgr2=0x7f7e201a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1ed9d640 1 -- 172.21.15.83:0/76214898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 msgr2=0x7f7e201a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1ed9d640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 0x7f7e201a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1dd9b640 1 -- 172.21.15.83:0/76214898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 msgr2=0x7f7e20074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1dd9b640 1 -- 172.21.15.83:0/76214898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 msgr2=0x7f7e20074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1dd9b640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 0x7f7e20074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1dd9b640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 0x7f7e20074760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1ed9d640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 0x7f7e201a9a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1e59c640 1 -- 172.21.15.83:0/76214898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 msgr2=0x7f7e201a9a90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:38:41.816 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1e59c640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 0x7f7e201a9a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1e59c640 1 -- 172.21.15.83:0/76214898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 msgr2=0x7f7e20074760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1e59c640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 0x7f7e20074760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.812+0000 7f7e1e59c640 1 -- 172.21.15.83:0/76214898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e201aa160 con 0x7f7e08003530 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.813+0000 7f7e1e59c640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 0x7f7e20074220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e1002c2d0 tx=0x7f7e1002da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.813+0000 7f7e0f7fe640 1 -- 172.21.15.83:0/76214898 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e100035f0 con 0x7f7e08003530 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.813+0000 7f7e0f7fe640 1 -- 172.21.15.83:0/76214898 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e10003c70 con 0x7f7e08003530 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.813+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e201aa3f0 con 0x7f7e08003530 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.813+0000 7f7e0f7fe640 1 -- 172.21.15.83:0/76214898 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10037480 con 0x7f7e08003530 2024-08-02T17:38:41.817 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.813+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e201aa980 con 0x7f7e08003530 2024-08-02T17:38:41.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.814+0000 7f7e0f7fe640 1 -- 172.21.15.83:0/76214898 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7e10037620 con 0x7f7e08003530 2024-08-02T17:38:41.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.814+0000 7f7e0f7fe640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7e000394f0 0x7f7e0003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:41.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.814+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e20102ad0 con 0x7f7e08003530 2024-08-02T17:38:41.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.814+0000 7f7e0f7fe640 1 -- 172.21.15.83:0/76214898 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7e1006da00 con 0x7f7e08003530 2024-08-02T17:38:41.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.815+0000 7f7e1dd9b640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7e000394f0 0x7f7e0003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:41.818 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.815+0000 7f7e1dd9b640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7e000394f0 0x7f7e0003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7e14006fd0 tx=0x7f7e14008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:41.821 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:41.820+0000 7f7e0f7fe640 1 -- 172.21.15.83:0/76214898 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7e100403a0 con 0x7f7e08003530 2024-08-02T17:38:42.069 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.068+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7e20190c30 con 0x7f7e08003530 2024-08-02T17:38:42.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:42.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: mon.c calling monitor election 2024-08-02T17:38:42.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:38:42.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: mon.a calling monitor election 2024-08-02T17:38:42.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:42.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: monmap epoch 3 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: last_changed 2024-08-02T17:38:36.715577+0000 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:38:42.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: min_mon_release 19 (squid) 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: election_strategy: 1 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.c 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: fsmap 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: mgrmap e13: x(active, since 45s) 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: overall HEALTH_OK 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:42.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:42.073 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.069+0000 7f7e0f7fe640 1 -- 172.21.15.83:0/76214898 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f7e100472a0 con 0x7f7e08003530 2024-08-02T17:38:42.073 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:38:42.073 INFO:teuthology.orchestra.run.smithi083.stdout:{"epoch":3,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","modified":"2024-08-02T17:38:36.715577Z","created":"2024-08-02T17:37:01.657332Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:3300","nonce":0},{"type":"v1","addr":"172.21.15.69:6789","nonce":0}]},"addr":"172.21.15.69:6789/0","public_addr":"172.21.15.69:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:3300","nonce":0},{"type":"v1","addr":"172.21.15.83:6789","nonce":0}]},"addr":"172.21.15.83:6789/0","public_addr":"172.21.15.83:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:3300","nonce":0},{"type":"v1","addr":"172.21.15.77:6789","nonce":0}]},"addr":"172.21.15.77:6789/0","public_addr":"172.21.15.77:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-02T17:38:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:dumped monmap epoch 3 2024-08-02T17:38:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7e000394f0 msgr2=0x7f7e0003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7e000394f0 0x7f7e0003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7e14006fd0 tx=0x7f7e14008040 comp rx=0 tx=0).stop 2024-08-02T17:38:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 msgr2=0x7f7e20074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 0x7f7e20074220 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7e1002c2d0 tx=0x7f7e1002da80 comp rx=0 tx=0).stop 2024-08-02T17:38:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 shutdown_connections 2024-08-02T17:38:42.074 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7e000394f0 0x7f7e0003b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e20074ca0 0x7f7e201a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e20075e60 0x7f7e20074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 --2- 172.21.15.83:0/76214898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e08003530 0x7f7e20074220 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 >> 172.21.15.83:0/76214898 conn(0x7f7e200fd050 msgr2=0x7f7e200fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 shutdown_connections 2024-08-02T17:38:42.075 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:42.071+0000 7f7e24e55640 1 -- 172.21.15.83:0/76214898 wait complete. 2024-08-02T17:38:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: mon.c calling monitor election 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: mon.a calling monitor election 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-02T17:38:42.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: monmap epoch 3 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: last_changed 2024-08-02T17:38:36.715577+0000 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: min_mon_release 19 (squid) 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: election_strategy: 1 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.c 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: fsmap 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: mgrmap e13: x(active, since 45s) 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: overall HEALTH_OK 2024-08-02T17:38:42.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:42.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:42.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:42.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:42.622 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-02T17:38:42.623 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph config generate-minimal-conf 2024-08-02T17:38:42.888 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:42.955 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:42 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:42.955 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:42 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/76214898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:42.955 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:42 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:43.009 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:42 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:43.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:42 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/76214898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:43.010 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:42 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:43.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.323+0000 7fc796781640 1 -- 172.21.15.69:0/2614519546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 msgr2=0x7fc790100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.323+0000 7fc796781640 1 --2- 172.21.15.69:0/2614519546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 0x7fc790100800 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc780009880 tx=0x7fc78002a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.324+0000 7fc796781640 1 -- 172.21.15.69:0/2614519546 shutdown_connections 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.324+0000 7fc796781640 1 --2- 172.21.15.69:0/2614519546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 0x7fc790100800 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.324+0000 7fc796781640 1 -- 172.21.15.69:0/2614519546 >> 172.21.15.69:0/2614519546 conn(0x7fc7900fbee0 msgr2=0x7fc7900fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.324+0000 7fc796781640 1 -- 172.21.15.69:0/2614519546 shutdown_connections 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.324+0000 7fc796781640 1 -- 172.21.15.69:0/2614519546 wait complete. 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.324+0000 7fc796781640 1 Processor -- start 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.324+0000 7fc796781640 1 -- start start 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc796781640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 0x7fc790197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc796781640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7901976c0 0x7fc79019ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:43.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc796781640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc79019bf90 0x7fc79019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc796781640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc79010a820 con 0x7fc790100420 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc796781640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc79010a6a0 con 0x7fc7901976c0 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc796781640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc79010a9a0 con 0x7fc79019bf90 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 0x7fc790197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 0x7fc790197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:32988/0 (socket says 172.21.15.69:32988) 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 -- 172.21.15.69:0/3069465707 learned_addr learned my addr 172.21.15.69:0/3069465707 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78f7fe640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7901976c0 0x7fc79019ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 -- 172.21.15.69:0/3069465707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc79019bf90 msgr2=0x7fc79019c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc79019bf90 0x7fc79019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 -- 172.21.15.69:0/3069465707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7901976c0 msgr2=0x7fc79019ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7901976c0 0x7fc79019ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 -- 172.21.15.69:0/3069465707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7901a3d00 con 0x7fc790100420 2024-08-02T17:38:43.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78f7fe640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7901976c0 0x7fc79019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78ffff640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 0x7fc790197180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc780031550 tx=0x7fc7800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78d7fa640 1 -- 172.21.15.69:0/3069465707 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78003f030 con 0x7fc790100420 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78d7fa640 1 -- 172.21.15.69:0/3069465707 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc780003ce0 con 0x7fc790100420 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7901a3ed0 con 0x7fc790100420 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.325+0000 7fc78d7fa640 1 -- 172.21.15.69:0/3069465707 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc780035070 con 0x7fc790100420 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.326+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7901a4380 con 0x7fc790100420 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.326+0000 7fc78d7fa640 1 -- 172.21.15.69:0/3069465707 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc780042030 con 0x7fc790100420 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.326+0000 7fc78d7fa640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7780394f0 0x7fc77803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.327+0000 7fc78d7fa640 1 -- 172.21.15.69:0/3069465707 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc78006dc90 con 0x7fc790100420 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.327+0000 7fc78f7fe640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7780394f0 0x7fc77803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:43.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.327+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc790100880 con 0x7fc790100420 2024-08-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.327+0000 7fc78f7fe640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7780394f0 0x7fc77803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc784007920 tx=0x7fc784008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:43.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.332+0000 7fc78d7fa640 1 -- 172.21.15.69:0/3069465707 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc78004bd70 con 0x7fc790100420 2024-08-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.589+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc7901a4670 con 0x7fc790100420 2024-08-02T17:38:43.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.590+0000 7fc78d7fa640 1 -- 172.21.15.69:0/3069465707 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7fc78003ac00 con 0x7fc790100420 2024-08-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi069.stdout:# minimal ceph.conf for b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi069.stdout:[global] 2024-08-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi069.stdout: fsid = b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:43.592 INFO:teuthology.orchestra.run.smithi069.stdout: mon_host = [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] 2024-08-02T17:38:43.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7780394f0 msgr2=0x7fc77803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7780394f0 0x7fc77803b9b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc784007920 tx=0x7fc784008040 comp rx=0 tx=0).stop 2024-08-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 msgr2=0x7fc790197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 0x7fc790197180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc780031550 tx=0x7fc7800042f0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 shutdown_connections 2024-08-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7780394f0 0x7fc77803b9b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc79019bf90 0x7fc79019c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7901976c0 0x7fc79019ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 --2- 172.21.15.69:0/3069465707 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc790100420 0x7fc790197180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.593+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 >> 172.21.15.69:0/3069465707 conn(0x7fc7900fbee0 msgr2=0x7fc790108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.594+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 shutdown_connections 2024-08-02T17:38:43.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:43.594+0000 7fc796781640 1 -- 172.21.15.69:0/3069465707 wait complete. 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: mon.c calling monitor election 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: mon.a calling monitor election 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: monmap epoch 3 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: last_changed 2024-08-02T17:38:36.715577+0000 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: min_mon_release 19 (squid) 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: election_strategy: 1 2024-08-02T17:38:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.c 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: fsmap 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: mgrmap e13: x(active, since 45s) 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: overall HEALTH_OK 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:44.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/76214898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-08-02T17:38:44.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:44.286 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-02T17:38:44.286 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:38:44.286 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.conf 2024-08-02T17:38:44.314 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:38:44.314 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:44.379 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:38:44.379 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.conf 2024-08-02T17:38:44.406 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:38:44.406 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:44.473 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:38:44.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.conf 2024-08-02T17:38:44.500 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:38:44.500 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:38:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: mon.b calling monitor election 2024-08-02T17:38:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: Updating smithi069:/etc/ceph/ceph.conf 2024-08-02T17:38:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: Updating smithi077:/etc/ceph/ceph.conf 2024-08-02T17:38:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-02T17:38:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: mon.b calling monitor election 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: mon.c calling monitor election 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: mon.a calling monitor election 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: monmap epoch 3 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: last_changed 2024-08-02T17:38:36.715577+0000 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: min_mon_release 19 (squid) 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: election_strategy: 1 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:38:44.538 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.c 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: fsmap 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: mgrmap e13: x(active, since 47s) 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: overall HEALTH_OK 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.539 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:44.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:44.540 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:44.564 INFO:tasks.cephadm:Adding mgr.x on smithi069 2024-08-02T17:38:44.564 INFO:tasks.cephadm:Adding mgr.y on smithi077 2024-08-02T17:38:44.564 INFO:tasks.cephadm:Adding mgr.z on smithi083 2024-08-02T17:38:44.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch apply mgr '3;smithi069=x;smithi077=y;smithi083=z' 2024-08-02T17:38:44.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: mon.b calling monitor election 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: Updating smithi069:/etc/ceph/ceph.conf 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: Updating smithi077:/etc/ceph/ceph.conf 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: mon.b calling monitor election 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: mon.c calling monitor election 2024-08-02T17:38:44.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: mon.a calling monitor election 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: monmap epoch 3 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: last_changed 2024-08-02T17:38:36.715577+0000 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: min_mon_release 19 (squid) 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: election_strategy: 1 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.c 2024-08-02T17:38:44.773 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: fsmap 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: mgrmap e13: x(active, since 47s) 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: overall HEALTH_OK 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.774 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:44.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:38:44.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: mon.b calling monitor election 2024-08-02T17:38:44.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: Updating smithi069:/etc/ceph/ceph.conf 2024-08-02T17:38:44.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: Updating smithi077:/etc/ceph/ceph.conf 2024-08-02T17:38:44.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-02T17:38:44.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: mon.b calling monitor election 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: mon.c calling monitor election 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: mon.a calling monitor election 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: monmap epoch 3 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: last_changed 2024-08-02T17:38:36.715577+0000 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: created 2024-08-02T17:37:01.657332+0000 2024-08-02T17:38:44.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: min_mon_release 19 (squid) 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: election_strategy: 1 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: 0: [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] mon.a 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: 1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.c 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: 2: [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] mon.b 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: fsmap 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: osdmap e4: 0 total, 0 up, 0 in 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: mgrmap e13: x(active, since 47s) 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: overall HEALTH_OK 2024-08-02T17:38:44.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:44.852 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:44.852 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:44.852 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.213+0000 7f13a3b9c640 1 -- 172.21.15.83:0/2620982456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c101410 msgr2=0x7f139c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.213+0000 7f13a3b9c640 1 --2- 172.21.15.83:0/2620982456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c101410 0x7f139c101810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1398009880 tx=0x7f139802a5d0 comp rx=0 tx=0).stop 2024-08-02T17:38:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.213+0000 7f13a3b9c640 1 -- 172.21.15.83:0/2620982456 shutdown_connections 2024-08-02T17:38:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.213+0000 7f13a3b9c640 1 --2- 172.21.15.83:0/2620982456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c101410 0x7f139c101810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:45.217 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.213+0000 7f13a3b9c640 1 -- 172.21.15.83:0/2620982456 >> 172.21.15.83:0/2620982456 conn(0x7f139c0fcbc0 msgr2=0x7f139c0fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.213+0000 7f13a3b9c640 1 -- 172.21.15.83:0/2620982456 shutdown_connections 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.213+0000 7f13a3b9c640 1 -- 172.21.15.83:0/2620982456 wait complete. 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a3b9c640 1 Processor -- start 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a3b9c640 1 -- start start 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a3b9c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f139c101410 0x7f139c196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a3b9c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139c197240 0x7f139c19c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a3b9c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c1976c0 0x7f139c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a3b9c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f139c107520 con 0x7f139c1976c0 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a3b9c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f139c1073a0 con 0x7f139c197240 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a3b9c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f139c1076a0 con 0x7f139c101410 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a1938640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f139c101410 0x7f139c196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:45.218 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a1938640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f139c101410 0x7f139c196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:38830/0 (socket says 172.21.15.83:38830) 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a1938640 1 -- 172.21.15.83:0/4008479101 learned_addr learned my addr 172.21.15.83:0/4008479101 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.214+0000 7f13a2139640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c1976c0 0x7f139c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a1137640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139c197240 0x7f139c19c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a1938640 1 -- 172.21.15.83:0/4008479101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139c197240 msgr2=0x7f139c19c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a1938640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139c197240 0x7f139c19c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a1938640 1 -- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c1976c0 msgr2=0x7f139c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a1938640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c1976c0 0x7f139c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a1938640 1 -- 172.21.15.83:0/4008479101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139c19f0c0 con 0x7f139c101410 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a2139640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c1976c0 0x7f139c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a1137640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139c197240 0x7f139c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a1938640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f139c101410 0x7f139c196d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1398005ab0 tx=0x7f1398032d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f1392ffd640 1 -- 172.21.15.83:0/4008479101 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139803f030 con 0x7f139c101410 2024-08-02T17:38:45.219 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f1392ffd640 1 -- 172.21.15.83:0/4008479101 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1398003ce0 con 0x7f139c101410 2024-08-02T17:38:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f1392ffd640 1 -- 172.21.15.83:0/4008479101 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398035070 con 0x7f139c101410 2024-08-02T17:38:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139c19f350 con 0x7f139c101410 2024-08-02T17:38:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.215+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f139c19f8b0 con 0x7f139c101410 2024-08-02T17:38:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.216+0000 7f1392ffd640 1 -- 172.21.15.83:0/4008479101 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1398042030 con 0x7f139c101410 2024-08-02T17:38:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.216+0000 7f1392ffd640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13880394f0 0x7f138803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.216+0000 7f1392ffd640 1 -- 172.21.15.83:0/4008479101 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f139806cc00 con 0x7f139c101410 2024-08-02T17:38:45.220 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.216+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139c1902e0 con 0x7f139c101410 2024-08-02T17:38:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.216+0000 7f13a1137640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13880394f0 0x7f138803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:45.221 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.217+0000 7f13a1137640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13880394f0 0x7f138803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f138c007920 tx=0x7f138c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:45.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.221+0000 7f1392ffd640 1 -- 172.21.15.83:0/4008479101 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f139803bdc0 con 0x7f139c101410 2024-08-02T17:38:45.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.398+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi077=y;smithi083=z", "target": ["mon-mgr", ""]}) -- 0x7f139c062360 con 0x7f13880394f0 2024-08-02T17:38:45.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.403+0000 7f1392ffd640 1 -- 172.21.15.83:0/4008479101 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f139c062360 con 0x7f13880394f0 2024-08-02T17:38:45.404 INFO:teuthology.orchestra.run.smithi083.stdout:Scheduled mgr update... 2024-08-02T17:38:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.404+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13880394f0 msgr2=0x7f138803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.404+0000 7f13a3b9c640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13880394f0 0x7f138803b9b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f138c007920 tx=0x7f138c008040 comp rx=0 tx=0).stop 2024-08-02T17:38:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.404+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f139c101410 msgr2=0x7f139c196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.404+0000 7f13a3b9c640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f139c101410 0x7f139c196d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1398005ab0 tx=0x7f1398032d00 comp rx=0 tx=0).stop 2024-08-02T17:38:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.405+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 shutdown_connections 2024-08-02T17:38:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.405+0000 7f13a3b9c640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13880394f0 0x7f138803b9b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:45.406 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.405+0000 7f13a3b9c640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f139c1976c0 0x7f139c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:45.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.405+0000 7f13a3b9c640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f139c197240 0x7f139c19c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:45.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.405+0000 7f13a3b9c640 1 --2- 172.21.15.83:0/4008479101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f139c101410 0x7f139c196d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:45.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.405+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 >> 172.21.15.83:0/4008479101 conn(0x7f139c0fcbc0 msgr2=0x7f139c0fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:45.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.405+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 shutdown_connections 2024-08-02T17:38:45.407 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:38:45.405+0000 7f13a3b9c640 1 -- 172.21.15.83:0/4008479101 wait complete. 2024-08-02T17:38:45.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:45 smithi083 ceph-mon[37031]: Reconfiguring mon.a (unknown last config time)... 2024-08-02T17:38:45.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:45 smithi083 ceph-mon[37031]: Reconfiguring daemon mon.a on smithi069 2024-08-02T17:38:45.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:45 smithi083 ceph-mon[37031]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:45.491 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:45.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:45 smithi069 ceph-mon[32746]: Reconfiguring mon.a (unknown last config time)... 2024-08-02T17:38:45.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:45 smithi069 ceph-mon[32746]: Reconfiguring daemon mon.a on smithi069 2024-08-02T17:38:45.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:45 smithi069 ceph-mon[32746]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:45.834 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:45.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:45 smithi077 ceph-mon[37108]: Reconfiguring mon.a (unknown last config time)... 2024-08-02T17:38:45.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:45 smithi077 ceph-mon[37108]: Reconfiguring daemon mon.a on smithi069 2024-08-02T17:38:45.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:45 smithi077 ceph-mon[37108]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:45.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:38:45.976 DEBUG:teuthology.orchestra.run.smithi077:mgr.y> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.y.service 2024-08-02T17:38:45.978 DEBUG:teuthology.orchestra.run.smithi083:mgr.z> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.z.service 2024-08-02T17:38:45.981 INFO:tasks.cephadm:Deploying OSDs... 2024-08-02T17:38:45.981 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:38:45.981 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/scratch_devs of=/dev/stdout 2024-08-02T17:38:45.997 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-02T17:38:45.997 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_1 2024-08-02T17:38:46.052 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-02T17:38:46.052 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:46.052 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 673 Links: 1 2024-08-02T17:38:46.052 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:46.052 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:46.052 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-08-02 17:38:05.165346596 +0000 2024-08-02T17:38:46.053 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-08-02 17:33:31.688573709 +0000 2024-08-02T17:38:46.053 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-08-02 17:33:31.688573709 +0000 2024-08-02T17:38:46.053 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-08-02 17:33:31.688573709 +0000 2024-08-02T17:38:46.053 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-02T17:38:46.119 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-08-02T17:38:46.119 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-08-02T17:38:46.119 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000233062 s, 2.2 MB/s 2024-08-02T17:38:46.120 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-02T17:38:46.176 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_2 2024-08-02T17:38:46.231 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-02T17:38:46.232 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:46.232 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 695 Links: 1 2024-08-02T17:38:46.232 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:46.232 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:46.232 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-08-02 17:38:05.194345940 +0000 2024-08-02T17:38:46.232 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-08-02 17:33:32.023566189 +0000 2024-08-02T17:38:46.232 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-08-02 17:33:32.023566189 +0000 2024-08-02T17:38:46.232 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-08-02 17:33:32.023566189 +0000 2024-08-02T17:38:46.232 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-02T17:38:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-08-02T17:38:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-08-02T17:38:46.295 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000202239 s, 2.5 MB/s 2024-08-02T17:38:46.297 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-02T17:38:46.353 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_3 2024-08-02T17:38:46.408 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-02T17:38:46.408 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:46.409 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 712 Links: 1 2024-08-02T17:38:46.409 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:46.409 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:46.409 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-08-02 17:38:05.221345330 +0000 2024-08-02T17:38:46.409 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-08-02 17:33:32.368558444 +0000 2024-08-02T17:38:46.409 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-08-02 17:33:32.368558444 +0000 2024-08-02T17:38:46.409 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-08-02 17:33:32.368558444 +0000 2024-08-02T17:38:46.409 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-02T17:38:46.473 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-08-02T17:38:46.473 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-08-02T17:38:46.473 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000173668 s, 2.9 MB/s 2024-08-02T17:38:46.474 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-02T17:38:46.530 DEBUG:teuthology.orchestra.run.smithi069:> stat /dev/vg_nvme/lv_4 2024-08-02T17:38:46.586 INFO:teuthology.orchestra.run.smithi069.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-02T17:38:46.586 INFO:teuthology.orchestra.run.smithi069.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:46.586 INFO:teuthology.orchestra.run.smithi069.stdout:Device: 5h/5d Inode: 733 Links: 1 2024-08-02T17:38:46.586 INFO:teuthology.orchestra.run.smithi069.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:46.586 INFO:teuthology.orchestra.run.smithi069.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:46.586 INFO:teuthology.orchestra.run.smithi069.stdout:Access: 2024-08-02 17:38:05.247344742 +0000 2024-08-02T17:38:46.586 INFO:teuthology.orchestra.run.smithi069.stdout:Modify: 2024-08-02 17:33:32.697551059 +0000 2024-08-02T17:38:46.586 INFO:teuthology.orchestra.run.smithi069.stdout:Change: 2024-08-02 17:33:32.697551059 +0000 2024-08-02T17:38:46.587 INFO:teuthology.orchestra.run.smithi069.stdout: Birth: 2024-08-02 17:33:32.697551059 +0000 2024-08-02T17:38:46.587 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi077=y;smithi083=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: Saving service mgr spec with placement smithi069=x;smithi077=y;smithi083=z;count:3 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: Reconfiguring mgr.x (unknown last config time)... 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:46.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:46.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:46 smithi077 ceph-mon[37108]: Reconfiguring daemon mgr.x on smithi069 2024-08-02T17:38:46.649 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records in 2024-08-02T17:38:46.649 INFO:teuthology.orchestra.run.smithi069.stderr:1+0 records out 2024-08-02T17:38:46.649 INFO:teuthology.orchestra.run.smithi069.stderr:512 bytes copied, 0.000234128 s, 2.2 MB/s 2024-08-02T17:38:46.651 DEBUG:teuthology.orchestra.run.smithi069:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-02T17:38:46.707 DEBUG:teuthology.orchestra.run.smithi077:> set -ex 2024-08-02T17:38:46.707 DEBUG:teuthology.orchestra.run.smithi077:> dd if=/scratch_devs of=/dev/stdout 2024-08-02T17:38:46.726 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-02T17:38:46.726 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_1 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi077=y;smithi083=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: Saving service mgr spec with placement smithi069=x;smithi077=y;smithi083=z;count:3 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: Reconfiguring mgr.x (unknown last config time)... 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:46.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:46 smithi083 ceph-mon[37031]: Reconfiguring daemon mgr.x on smithi069 2024-08-02T17:38:46.784 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-02T17:38:46.784 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:46.784 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-08-02T17:38:46.784 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:46.784 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:46.786 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-08-02 17:38:36.146403121 +0000 2024-08-02T17:38:46.787 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-08-02 17:33:46.532292698 +0000 2024-08-02T17:38:46.787 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-08-02 17:33:46.532292698 +0000 2024-08-02T17:38:46.787 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-08-02 17:33:46.532292698 +0000 2024-08-02T17:38:46.787 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi069=x;smithi077=y;smithi083=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: Saving service mgr spec with placement smithi069=x;smithi077=y;smithi083=z;count:3 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: Reconfiguring mgr.x (unknown last config time)... 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:46.816 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:46 smithi069 ceph-mon[32746]: Reconfiguring daemon mgr.x on smithi069 2024-08-02T17:38:46.851 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-08-02T17:38:46.851 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-08-02T17:38:46.851 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000280515 s, 1.8 MB/s 2024-08-02T17:38:46.852 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-02T17:38:46.911 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_2 2024-08-02T17:38:46.966 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-02T17:38:46.966 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:46.966 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-02T17:38:46.966 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:46.966 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:46.967 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-08-02 17:38:36.147403099 +0000 2024-08-02T17:38:46.967 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-08-02 17:33:46.867285193 +0000 2024-08-02T17:38:46.967 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-08-02 17:33:46.867285193 +0000 2024-08-02T17:38:46.967 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-08-02 17:33:46.867285193 +0000 2024-08-02T17:38:46.967 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-02T17:38:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-08-02T17:38:47.028 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-08-02T17:38:47.029 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000201867 s, 2.5 MB/s 2024-08-02T17:38:47.030 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-02T17:38:47.085 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_3 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-08-02 17:38:36.147403099 +0000 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-08-02 17:33:47.196277821 +0000 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-08-02 17:33:47.196277821 +0000 2024-08-02T17:38:47.141 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-08-02 17:33:47.196277821 +0000 2024-08-02T17:38:47.141 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-02T17:38:47.203 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-08-02T17:38:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-08-02T17:38:47.204 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000208233 s, 2.5 MB/s 2024-08-02T17:38:47.205 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-02T17:38:47.260 DEBUG:teuthology.orchestra.run.smithi077:> stat /dev/vg_nvme/lv_4 2024-08-02T17:38:47.314 INFO:teuthology.orchestra.run.smithi077.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-02T17:38:47.315 INFO:teuthology.orchestra.run.smithi077.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:47.315 INFO:teuthology.orchestra.run.smithi077.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-02T17:38:47.315 INFO:teuthology.orchestra.run.smithi077.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:47.315 INFO:teuthology.orchestra.run.smithi077.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:47.315 INFO:teuthology.orchestra.run.smithi077.stdout:Access: 2024-08-02 17:38:36.148403077 +0000 2024-08-02T17:38:47.315 INFO:teuthology.orchestra.run.smithi077.stdout:Modify: 2024-08-02 17:33:47.515270674 +0000 2024-08-02T17:38:47.315 INFO:teuthology.orchestra.run.smithi077.stdout:Change: 2024-08-02 17:33:47.515270674 +0000 2024-08-02T17:38:47.315 INFO:teuthology.orchestra.run.smithi077.stdout: Birth: 2024-08-02 17:33:47.515270674 +0000 2024-08-02T17:38:47.315 DEBUG:teuthology.orchestra.run.smithi077:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-02T17:38:47.379 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records in 2024-08-02T17:38:47.379 INFO:teuthology.orchestra.run.smithi077.stderr:1+0 records out 2024-08-02T17:38:47.379 INFO:teuthology.orchestra.run.smithi077.stderr:512 bytes copied, 0.000189181 s, 2.7 MB/s 2024-08-02T17:38:47.380 DEBUG:teuthology.orchestra.run.smithi077:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-02T17:38:47.435 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-08-02T17:38:47.435 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-08-02T17:38:47.454 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-02T17:38:47.455 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-08-02T17:38:47.513 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-02T17:38:47.513 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:47.513 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-08-02T17:38:47.514 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:47.514 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:47.514 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-02 17:38:27.072490161 +0000 2024-08-02T17:38:47.514 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-02 17:33:46.781602816 +0000 2024-08-02T17:38:47.514 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-02 17:33:46.781602816 +0000 2024-08-02T17:38:47.514 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-02 17:33:46.781602816 +0000 2024-08-02T17:38:47.514 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-02T17:38:47.580 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-08-02T17:38:47.580 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-08-02T17:38:47.580 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000289482 s, 1.8 MB/s 2024-08-02T17:38:47.581 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-02T17:38:47.640 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-08-02T17:38:47.698 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-02T17:38:47.698 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:47.698 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-02T17:38:47.698 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:47.698 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:47.698 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-02 17:38:27.072490161 +0000 2024-08-02T17:38:47.698 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-02 17:33:47.120594810 +0000 2024-08-02T17:38:47.699 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-02 17:33:47.120594810 +0000 2024-08-02T17:38:47.699 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-02 17:33:47.120594810 +0000 2024-08-02T17:38:47.699 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-02T17:38:47.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-08-02T17:38:47.763 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-08-02T17:38:47.764 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000218354 s, 2.3 MB/s 2024-08-02T17:38:47.765 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-02T17:38:47.822 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-08-02T17:38:47.879 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-02T17:38:47.880 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:47.880 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-08-02T17:38:47.880 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:47.880 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:47.880 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-02 17:38:27.072490161 +0000 2024-08-02T17:38:47.880 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-02 17:33:47.459586803 +0000 2024-08-02T17:38:47.880 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-02 17:33:47.459586803 +0000 2024-08-02T17:38:47.880 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-02 17:33:47.459586803 +0000 2024-08-02T17:38:47.880 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-02T17:38:47.942 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:47 smithi083 ceph-mon[37031]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:47.942 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:47.942 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:47.942 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:47.942 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:47.942 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:47.942 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:47.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-08-02T17:38:47.944 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-08-02T17:38:47.944 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000178755 s, 2.9 MB/s 2024-08-02T17:38:47.945 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-02T17:38:48.003 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-08-02T17:38:48.046 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:47 smithi077 ceph-mon[37108]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:48.047 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:48.047 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:48.047 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:48.047 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:48.047 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:48.047 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:48.059 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-02T17:38:48.059 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-02T17:38:48.059 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-02T17:38:48.059 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-02T17:38:48.059 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-08-02T17:38:48.060 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-08-02 17:38:27.073490137 +0000 2024-08-02T17:38:48.060 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-08-02 17:33:47.786579080 +0000 2024-08-02T17:38:48.060 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-08-02 17:33:47.786579080 +0000 2024-08-02T17:38:48.060 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-08-02 17:33:47.786579080 +0000 2024-08-02T17:38:48.060 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-02T17:38:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:47 smithi069 ceph-mon[32746]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-08-02T17:38:48.123 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-08-02T17:38:48.124 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000212034 s, 2.4 MB/s 2024-08-02T17:38:48.125 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-02T17:38:48.180 INFO:tasks.cephadm:Deploying osd.0 on smithi069 with /dev/vg_nvme/lv_4... 2024-08-02T17:38:48.181 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-02T17:38:48.437 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:48.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:48 smithi069 ceph-mon[32746]: Reconfiguring mon.b (monmap changed)... 2024-08-02T17:38:48.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:48 smithi069 ceph-mon[32746]: Reconfiguring daemon mon.b on smithi077 2024-08-02T17:38:48.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:48.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:48.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:48.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:48.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:48 smithi083 ceph-mon[37031]: Reconfiguring mon.b (monmap changed)... 2024-08-02T17:38:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:48 smithi083 ceph-mon[37031]: Reconfiguring daemon mon.b on smithi077 2024-08-02T17:38:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:49.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:48 smithi077 ceph-mon[37108]: Reconfiguring mon.b (monmap changed)... 2024-08-02T17:38:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:48 smithi077 ceph-mon[37108]: Reconfiguring daemon mon.b on smithi077 2024-08-02T17:38:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-08-02T17:38:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-08-02T17:38:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: Reconfiguring mon.c (monmap changed)... 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: Reconfiguring daemon mon.c on smithi083 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-02T17:38:49.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:49.992 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:50.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: Reconfiguring mon.c (monmap changed)... 2024-08-02T17:38:50.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: Reconfiguring daemon mon.c on smithi083 2024-08-02T17:38:50.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:50.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:50.032 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:50.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:50.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:50.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:50.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:50.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:50.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-02T17:38:50.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:50.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: Reconfiguring mon.c (monmap changed)... 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: Reconfiguring daemon mon.c on smithi083 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-02T17:38:50.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:50.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:50.681 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:38:50.709 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_4 2024-08-02T17:38:50.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:50 smithi069 ceph-mon[32746]: Deploying daemon mgr.z on smithi083 2024-08-02T17:38:50.960 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:38:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:50 smithi083 ceph-mon[37031]: Deploying daemon mgr.z on smithi083 2024-08-02T17:38:51.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:50 smithi077 ceph-mon[37108]: Deploying daemon mgr.z on smithi083 2024-08-02T17:38:51.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 -- 172.21.15.69:0/3082695918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 msgr2=0x7f14f4107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:51.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/3082695918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 0x7f14f4107170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009900 tx=0x7f14e402a410 comp rx=0 tx=0).stop 2024-08-02T17:38:51.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 -- 172.21.15.69:0/3082695918 shutdown_connections 2024-08-02T17:38:51.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/3082695918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 0x7f14f4107170 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:51.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/3082695918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f41076b0 0x7f14f4109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/3082695918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14f4102450 0x7f14f4104840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 -- 172.21.15.69:0/3082695918 >> 172.21.15.69:0/3082695918 conn(0x7f14f40fbfd0 msgr2=0x7f14f40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:38:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 -- 172.21.15.69:0/3082695918 shutdown_connections 2024-08-02T17:38:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.370+0000 7f14fb3a2640 1 -- 172.21.15.69:0/3082695918 wait complete. 2024-08-02T17:38:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14fb3a2640 1 Processor -- start 2024-08-02T17:38:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14fb3a2640 1 -- start start 2024-08-02T17:38:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14fb3a2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14f4102450 0x7f14f4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:51.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14fb3a2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 0x7f14f41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14fb3a2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f41076b0 0x7f14f419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14fb3a2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f14f410d6a0 con 0x7f14f4104d80 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14fb3a2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f14f410d520 con 0x7f14f4102450 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14fb3a2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f14f410d820 con 0x7f14f41076b0 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14f893d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 0x7f14f41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14f893d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 0x7f14f41982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39420/0 (socket says 172.21.15.69:39420) 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14f893d640 1 -- 172.21.15.69:0/2802167991 learned_addr learned my addr 172.21.15.69:0/2802167991 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14f913e640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14f4102450 0x7f14f4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14f893d640 1 -- 172.21.15.69:0/2802167991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f41076b0 msgr2=0x7f14f419f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.371+0000 7f14f993f640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f41076b0 0x7f14f419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:51.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14f893d640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f41076b0 0x7f14f419f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14f893d640 1 -- 172.21.15.69:0/2802167991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14f4102450 msgr2=0x7f14f4197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14f893d640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14f4102450 0x7f14f4197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14f893d640 1 -- 172.21.15.69:0/2802167991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14f419f9e0 con 0x7f14f4104d80 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14f913e640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14f4102450 0x7f14f4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14f893d640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 0x7f14f41982f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f14e4031610 tx=0x7f14e40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e403f030 con 0x7f14f4104d80 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14e4003ce0 con 0x7f14f4104d80 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14f419fc70 con 0x7f14f4104d80 2024-08-02T17:38:51.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14e4035070 con 0x7f14f4104d80 2024-08-02T17:38:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.372+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14f41a01a0 con 0x7f14f4104d80 2024-08-02T17:38:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.373+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f14e4042030 con 0x7f14f4104d80 2024-08-02T17:38:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.373+0000 7f14ea7fc640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f14e00392c0 0x7f14e003b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:38:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.373+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f14e406cae0 con 0x7f14f4104d80 2024-08-02T17:38:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.373+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f40fdac0 con 0x7f14f4104d80 2024-08-02T17:38:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.373+0000 7f14f913e640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f14e00392c0 0x7f14e003b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:38:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.373+0000 7f14f913e640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f14e00392c0 0x7f14e003b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f14f0006fd0 tx=0x7f14f0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:38:51.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.378+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f14e403be10 con 0x7f14f4104d80 2024-08-02T17:38:51.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:51.553+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f14f4062360 con 0x7f14e00392c0 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:51.820+0000 7f80cefae040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:51.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:51.824 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-02T17:38:51.824 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:51.824 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:52.074 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:52.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:52.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-08-02T17:38:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:52.125 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:51 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:51.886+0000 7f80cefae040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-02T17:38:52.126 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:52.123+0000 7f80cefae040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-02T17:38:52.521 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:52.181+0000 7f80cefae040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-02T17:38:52.977 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:52 smithi069 ceph-mon[32746]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:52.977 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:52 smithi069 ceph-mon[32746]: Deploying daemon mgr.y on smithi077 2024-08-02T17:38:52.991 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:52 smithi077 ceph-mon[37108]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:52.991 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:52 smithi077 ceph-mon[37108]: Deploying daemon mgr.y on smithi077 2024-08-02T17:38:53.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:52 smithi083 ceph-mon[37031]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:38:53.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:52 smithi083 ceph-mon[37031]: Deploying daemon mgr.y on smithi077 2024-08-02T17:38:53.483 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:53.107+0000 7f80cefae040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-02T17:38:53.483 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:53.218+0000 7f80cefae040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-02T17:38:53.483 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:53.274+0000 7f80cefae040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-02T17:38:53.483 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:53.480+0000 7f80cefae040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-02T17:38:53.597 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:53 smithi077 systemd[1]: Starting Ceph mgr.y for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:38:53.738 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:53.541+0000 7f80cefae040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-02T17:38:53.738 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:53.606+0000 7f80cefae040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-02T17:38:53.903 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:53 smithi077 ceph-mon[37108]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:53.904 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:53 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/239169164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fad0068-7f9b-4e76-9c10-6338b2fa7619"}]: dispatch 2024-08-02T17:38:53.904 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:53 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/239169164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fad0068-7f9b-4e76-9c10-6338b2fa7619"}]': finished 2024-08-02T17:38:53.904 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:53 smithi077 ceph-mon[37108]: osdmap e5: 1 total, 0 up, 1 in 2024-08-02T17:38:53.904 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:53 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:38:53.904 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:53 smithi077 podman[38466]: 2024-08-02 17:38:53.650664767 +0000 UTC m=+0.113709601 container create 8a98cc974e1cfbe7a5c9249dd7c004bde53ed7586595c93c7c99022ba9d4441b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-02T17:38:53.904 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:53 smithi077 podman[38466]: 2024-08-02 17:38:53.553718635 +0000 UTC m=+0.016763484 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:38:53.904 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:53 smithi077 podman[38466]: 2024-08-02 17:38:53.903146331 +0000 UTC m=+0.366191169 container init 8a98cc974e1cfbe7a5c9249dd7c004bde53ed7586595c93c7c99022ba9d4441b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-08-02T17:38:53.990 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:53 smithi069 ceph-mon[32746]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:53.991 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:53 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/239169164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fad0068-7f9b-4e76-9c10-6338b2fa7619"}]: dispatch 2024-08-02T17:38:53.991 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:53 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/239169164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fad0068-7f9b-4e76-9c10-6338b2fa7619"}]': finished 2024-08-02T17:38:53.991 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:53 smithi069 ceph-mon[32746]: osdmap e5: 1 total, 0 up, 1 in 2024-08-02T17:38:53.991 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:53 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:38:54.021 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:53.749+0000 7f80cefae040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:38:54.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-mon[37031]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:54.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/239169164' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7fad0068-7f9b-4e76-9c10-6338b2fa7619"}]: dispatch 2024-08-02T17:38:54.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/239169164' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7fad0068-7f9b-4e76-9c10-6338b2fa7619"}]': finished 2024-08-02T17:38:54.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-mon[37031]: osdmap e5: 1 total, 0 up, 1 in 2024-08-02T17:38:54.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:53 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:38:54.184 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:53 smithi077 podman[38466]: 2024-08-02 17:38:53.905551415 +0000 UTC m=+0.368596252 container start 8a98cc974e1cfbe7a5c9249dd7c004bde53ed7586595c93c7c99022ba9d4441b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-02T17:38:54.184 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:53 smithi077 ceph-mgr[38485]: -- 172.21.15.77:0/167328418 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559808a650e0 con 0x559808a42800 2024-08-02T17:38:54.184 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:54 smithi077 bash[38466]: 8a98cc974e1cfbe7a5c9249dd7c004bde53ed7586595c93c7c99022ba9d4441b 2024-08-02T17:38:54.184 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:54 smithi077 systemd[1]: Started Ceph mgr.y for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:38:54.184 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:54.027+0000 7f29f5965040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-02T17:38:54.185 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:54.182+0000 7f29f5965040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:38:54.494 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:54.158+0000 7f80cefae040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-02T17:38:54.494 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:54.491+0000 7f80cefae040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-02T17:38:54.516 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:54.253+0000 7f29f5965040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-02T17:38:54.762 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:54.566+0000 7f80cefae040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-02T17:38:54.762 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:54.715+0000 7f80cefae040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-02T17:38:54.768 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:54.514+0000 7f29f5965040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-02T17:38:54.768 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:54.580+0000 7f29f5965040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-02T17:38:55.295 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:55.034+0000 7f80cefae040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-02T17:38:55.295 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:55.098+0000 7f80cefae040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-02T17:38:55.295 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:55.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1376840674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:38:55.296 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-mon[37031]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1376840674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:38:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-mon[37108]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:55 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1376840674' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:38:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:55 smithi069 ceph-mon[32746]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:55.592 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:55.292+0000 7f80cefae040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-02T17:38:55.593 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:55.444+0000 7f80cefae040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-02T17:38:55.593 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:55.508+0000 7f80cefae040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-02T17:38:55.593 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:55.590+0000 7f80cefae040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-02T17:38:55.847 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:55.554+0000 7f29f5965040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-02T17:38:55.847 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:55.667+0000 7f29f5965040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-02T17:38:55.848 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:55.724+0000 7f29f5965040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-02T17:38:56.021 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:55.649+0000 7f80cefae040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-02T17:38:56.203 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:55.932+0000 7f29f5965040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-02T17:38:56.203 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:55.991+0000 7f29f5965040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-02T17:38:56.203 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:56.057+0000 7f29f5965040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-02T17:38:56.203 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:56.200+0000 7f29f5965040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:38:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:56.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:56.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:56.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:56.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.521 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:56.176+0000 7f80cefae040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-02T17:38:56.521 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:38:56.332+0000 7f80cefae040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-02T17:38:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:56.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:56.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:56.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:56.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:56.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:38:56.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:56.932 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:56.614+0000 7f29f5965040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-02T17:38:56.932 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:56.929+0000 7f29f5965040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-02T17:38:57.237 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:56.993+0000 7f29f5965040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-02T17:38:57.237 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:57.142+0000 7f29f5965040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-02T17:38:57.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:57.243+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f14e40438e0 con 0x7f14f4104d80 2024-08-02T17:38:57.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-mon[37108]: Standby manager daemon z started 2024-08-02T17:38:57.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-02T17:38:57.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:38:57.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-02T17:38:57.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:38:57.515 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-mon[37108]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:57.516 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:57.516 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:57.454+0000 7f29f5965040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-02T17:38:57.516 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:57.513+0000 7f29f5965040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-02T17:38:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:57 smithi083 ceph-mon[37031]: Standby manager daemon z started 2024-08-02T17:38:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:57 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-02T17:38:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:57 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:38:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:57 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-02T17:38:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:57 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:38:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:57 smithi083 ceph-mon[37031]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:57.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:57 smithi069 ceph-mon[32746]: Standby manager daemon z started 2024-08-02T17:38:57.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:57 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-02T17:38:57.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:57 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:38:57.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:57 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-02T17:38:57.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:57 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.83:0/902361698' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:38:57.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:57 smithi069 ceph-mon[32746]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:57.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:38:57.842 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:57.694+0000 7f29f5965040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-02T17:38:57.843 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:57.840+0000 7f29f5965040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-02T17:38:58.097 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:57.905+0000 7f29f5965040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-02T17:38:58.098 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:57.987+0000 7f29f5965040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-02T17:38:58.098 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:58.047+0000 7f29f5965040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-02T17:38:58.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:58 smithi083 ceph-mon[37031]: mgrmap e14: x(active, since 60s), standbys: z 2024-08-02T17:38:58.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:58 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-02T17:38:58.556 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:58 smithi069 ceph-mon[32746]: mgrmap e14: x(active, since 60s), standbys: z 2024-08-02T17:38:58.556 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:58 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-02T17:38:58.580 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:58 smithi077 ceph-mon[37108]: mgrmap e14: x(active, since 60s), standbys: z 2024-08-02T17:38:58.580 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:58 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-02T17:38:58.848 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:58.578+0000 7f29f5965040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-02T17:38:58.848 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:38:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:38:58.735+0000 7f29f5965040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-02T17:38:59.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:38:59.253+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f14e403b460 con 0x7f14f4104d80 2024-08-02T17:38:59.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:59 smithi069 ceph-mon[32746]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:59.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:59 smithi069 ceph-mon[32746]: Standby manager daemon y started 2024-08-02T17:38:59.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:59 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-02T17:38:59.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:59 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:38:59.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:59 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-02T17:38:59.512 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:38:59 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:38:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:59 smithi083 ceph-mon[37031]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:59 smithi083 ceph-mon[37031]: Standby manager daemon y started 2024-08-02T17:38:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:59 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-02T17:38:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:59 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:38:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:59 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-02T17:38:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:38:59 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:38:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:59 smithi077 ceph-mon[37108]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:38:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:59 smithi077 ceph-mon[37108]: Standby manager daemon y started 2024-08-02T17:38:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:59 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-02T17:38:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:59 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:38:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:59 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-02T17:38:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:38:59 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.77:0/1372718740' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:39:00.395 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:00 smithi069 ceph-mon[32746]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-08-02T17:39:00.395 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-02T17:39:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:00 smithi083 ceph-mon[37031]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-08-02T17:39:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-02T17:39:00.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:00 smithi077 ceph-mon[37108]: mgrmap e15: x(active, since 62s), standbys: z, y 2024-08-02T17:39:00.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-02T17:39:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:01 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-02T17:39:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:01 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:01 smithi083 ceph-mon[37031]: Deploying daemon osd.0 on smithi069 2024-08-02T17:39:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:01 smithi083 ceph-mon[37031]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:01.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:01 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-02T17:39:01.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:01 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:01.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:01 smithi077 ceph-mon[37108]: Deploying daemon osd.0 on smithi069 2024-08-02T17:39:01.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:01 smithi077 ceph-mon[37108]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:01 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-08-02T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:01 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:01 smithi069 ceph-mon[32746]: Deploying daemon osd.0 on smithi069 2024-08-02T17:39:01.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:01 smithi069 ceph-mon[32746]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:03.937 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:03 smithi069 ceph-mon[32746]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:04.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:03 smithi083 ceph-mon[37031]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:04.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:03 smithi077 ceph-mon[37108]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:06.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:05 smithi083 ceph-mon[37031]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:06.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:06.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:06.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:06.056 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:05 smithi069 ceph-mon[32746]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:06.056 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:06.056 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:06.056 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:06.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:05 smithi077 ceph-mon[37108]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:06.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:06.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:06.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:07.964 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:07 smithi069 ceph-mon[32746]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:07.965 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:07.965 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:08.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:07 smithi083 ceph-mon[37031]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:08.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:08.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:08.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:07 smithi077 ceph-mon[37108]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:08.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:08.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:08.782 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 0 on host 'smithi069' 2024-08-02T17:39:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.780+0000 7f14ea7fc640 1 -- 172.21.15.69:0/2802167991 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f14f4062360 con 0x7f14e00392c0 2024-08-02T17:39:08.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.782+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f14e00392c0 msgr2=0x7f14e003b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:08.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.782+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f14e00392c0 0x7f14e003b780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f14f0006fd0 tx=0x7f14f0008040 comp rx=0 tx=0).stop 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 msgr2=0x7f14f41982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 0x7f14f41982f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f14e4031610 tx=0x7f14e40043b0 comp rx=0 tx=0).stop 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 shutdown_connections 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f14e00392c0 0x7f14e003b780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f14f4104d80 0x7f14f41982f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f14f41076b0 0x7f14f419f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 --2- 172.21.15.69:0/2802167991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f14f4102450 0x7f14f4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 >> 172.21.15.69:0/2802167991 conn(0x7f14f40fbfd0 msgr2=0x7f14f4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 shutdown_connections 2024-08-02T17:39:08.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:08.783+0000 7f14fb3a2640 1 -- 172.21.15.69:0/2802167991 wait complete. 2024-08-02T17:39:08.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:08.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:08.786 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:09.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:08 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:09.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:08 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:09.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:08 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:09.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:08 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:09.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:08 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:09.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:08 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:09.466 DEBUG:teuthology.orchestra.run.smithi069:osd.0> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.0.service 2024-08-02T17:39:09.469 INFO:tasks.cephadm:Deploying osd.1 on smithi077 with /dev/vg_nvme/lv_4... 2024-08-02T17:39:09.469 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-02T17:39:09.725 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T17:39:10.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:09 smithi083 ceph-mon[37031]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:10.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:09 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:10.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:09 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:10.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:09 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:10.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:09 smithi083 ceph-mon[37031]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-02T17:39:10.092 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:09 smithi077 ceph-mon[37108]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:10.093 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:09 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:10.093 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:09 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:10.093 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:09 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:10.093 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:09 smithi077 ceph-mon[37108]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-02T17:39:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:09 smithi069 ceph-mon[32746]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:09 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:09 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:09 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:09 smithi069 ceph-mon[32746]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-08-02T17:39:11.364 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:11 smithi077 ceph-mon[37108]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-02T17:39:11.364 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:11 smithi077 ceph-mon[37108]: osdmap e6: 1 total, 0 up, 1 in 2024-08-02T17:39:11.364 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:11 smithi077 ceph-mon[37108]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:39:11.364 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:11 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:11.364 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:11 smithi077 ceph-mon[37108]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:11.501 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:11 smithi069 ceph-mon[32746]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-02T17:39:11.501 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:11 smithi069 ceph-mon[32746]: osdmap e6: 1 total, 0 up, 1 in 2024-08-02T17:39:11.501 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:11 smithi069 ceph-mon[32746]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:39:11.501 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:11 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:11.501 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:11 smithi069 ceph-mon[32746]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:11.502 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0[44036]: 2024-08-02T17:39:11.120+0000 7fb74cd0f640 -1 osd.0 0 waiting for initial osdmap 2024-08-02T17:39:11.502 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0[44036]: 2024-08-02T17:39:11.125+0000 7fb748b37640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:39:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:11 smithi083 ceph-mon[37031]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-02T17:39:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:11 smithi083 ceph-mon[37031]: osdmap e6: 1 total, 0 up, 1 in 2024-08-02T17:39:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:11 smithi083 ceph-mon[37031]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:39:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:11 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:11 smithi083 ceph-mon[37031]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:12.048 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:39:12.073 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_4 2024-08-02T17:39:12.323 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T17:39:12.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:12 smithi077 ceph-mon[37108]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:39:12.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:12 smithi077 ceph-mon[37108]: osdmap e7: 1 total, 0 up, 1 in 2024-08-02T17:39:12.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:12 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:12.444 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:12 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:12 smithi083 ceph-mon[37031]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:39:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:12 smithi083 ceph-mon[37031]: osdmap e7: 1 total, 0 up, 1 in 2024-08-02T17:39:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:12 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:12 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:12 smithi069 ceph-mon[32746]: from='osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:39:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:12 smithi069 ceph-mon[32746]: osdmap e7: 1 total, 0 up, 1 in 2024-08-02T17:39:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:12 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:12 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 -- 172.21.15.77:0/2280727984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 msgr2=0x7ffab4100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 --2- 172.21.15.77:0/2280727984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 0x7ffab4100fa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009900 tx=0x7ffa9c02a410 comp rx=0 tx=0).stop 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 -- 172.21.15.77:0/2280727984 shutdown_connections 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 --2- 172.21.15.77:0/2280727984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab41014e0 0x7ffab4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 --2- 172.21.15.77:0/2280727984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 0x7ffab4100fa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 --2- 172.21.15.77:0/2280727984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab4100200 0x7ffab4100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 -- 172.21.15.77:0/2280727984 >> 172.21.15.77:0/2280727984 conn(0x7ffab40fbfb0 msgr2=0x7ffab40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 -- 172.21.15.77:0/2280727984 shutdown_connections 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 -- 172.21.15.77:0/2280727984 wait complete. 2024-08-02T17:39:12.754 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.752+0000 7ffab95cf640 1 Processor -- start 2024-08-02T17:39:12.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab95cf640 1 -- start start 2024-08-02T17:39:12.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab95cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4100200 0x7ffab4199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:12.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab95cf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 0x7ffab419a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:12.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab95cf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab41014e0 0x7ffab41a1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:12.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab95cf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410f770 con 0x7ffab41014e0 2024-08-02T17:39:12.756 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab95cf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410f5f0 con 0x7ffab4100b40 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab95cf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410f8f0 con 0x7ffab4100200 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 0x7ffab419a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 0x7ffab419a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:51864/0 (socket says 172.21.15.77:51864) 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab27fc640 1 -- 172.21.15.77:0/3984588823 learned_addr learned my addr 172.21.15.77:0/3984588823 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab37fe640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab41014e0 0x7ffab41a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab2ffd640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4100200 0x7ffab4199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab27fc640 1 -- 172.21.15.77:0/3984588823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4100200 msgr2=0x7ffab4199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab27fc640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4100200 0x7ffab4199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab27fc640 1 -- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab41014e0 msgr2=0x7ffab41a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab27fc640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab41014e0 0x7ffab41a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.753+0000 7ffab27fc640 1 -- 172.21.15.77:0/3984588823 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab41a1a90 con 0x7ffab4100b40 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.754+0000 7ffab27fc640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 0x7ffab419a3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009230 tx=0x7ffa9c004360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.754+0000 7ffaabfff640 1 -- 172.21.15.77:0/3984588823 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c03f030 con 0x7ffab4100b40 2024-08-02T17:39:12.757 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.754+0000 7ffaabfff640 1 -- 172.21.15.77:0/3984588823 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa9c003ce0 con 0x7ffab4100b40 2024-08-02T17:39:12.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.754+0000 7ffaabfff640 1 -- 172.21.15.77:0/3984588823 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c035070 con 0x7ffab4100b40 2024-08-02T17:39:12.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.754+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab41a1d20 con 0x7ffab4100b40 2024-08-02T17:39:12.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.754+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41a21b0 con 0x7ffab4100b40 2024-08-02T17:39:12.758 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.755+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffab410af50 con 0x7ffab4100b40 2024-08-02T17:39:12.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.756+0000 7ffaabfff640 1 -- 172.21.15.77:0/3984588823 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7ffa9c042030 con 0x7ffab4100b40 2024-08-02T17:39:12.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.756+0000 7ffaabfff640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ffaa00a2b20 0x7ffaa00a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:12.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.757+0000 7ffaabfff640 1 -- 172.21.15.77:0/3984588823 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7ffa9c0edbb0 con 0x7ffab4100b40 2024-08-02T17:39:12.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.757+0000 7ffab2ffd640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ffaa00a2b20 0x7ffaa00a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:12.759 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.757+0000 7ffab2ffd640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ffaa00a2b20 0x7ffaa00a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4005fd0 tx=0x7ffaa4009520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:39:12.762 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.761+0000 7ffaabfff640 1 -- 172.21.15.77:0/3984588823 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ffa9c0bc580 con 0x7ffab4100b40 2024-08-02T17:39:12.937 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:12.936+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ffab4062360 con 0x7ffaa00a2b20 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] boot 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: osdmap e8: 1 total, 1 up, 1 in 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi069 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.433 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi069 to 17087M 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:13.434 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:39:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:39:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] boot 2024-08-02T17:39:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: osdmap e8: 1 total, 1 up, 1 in 2024-08-02T17:39:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi069 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi069 to 17087M 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:13.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:13.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:13.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:39:13.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: osd.0 [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] boot 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: osdmap e8: 1 total, 1 up, 1 in 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi069 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi069 to 17087M 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:13.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:13.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-02T17:39:13.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:13.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:13.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:14.374 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:14 smithi077 ceph-mon[37108]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:14.374 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:14 smithi077 ceph-mon[37108]: osdmap e9: 1 total, 1 up, 1 in 2024-08-02T17:39:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:14 smithi083 ceph-mon[37031]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:14 smithi083 ceph-mon[37031]: osdmap e9: 1 total, 1 up, 1 in 2024-08-02T17:39:14.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:14 smithi069 ceph-mon[32746]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:14.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:14 smithi069 ceph-mon[32746]: osdmap e9: 1 total, 1 up, 1 in 2024-08-02T17:39:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:15 smithi083 ceph-mon[37031]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:15 smithi083 ceph-mon[37031]: from='client.? 172.21.15.77:0/3315858122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]: dispatch 2024-08-02T17:39:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:15 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]: dispatch 2024-08-02T17:39:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:15 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]': finished 2024-08-02T17:39:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:15 smithi083 ceph-mon[37031]: osdmap e10: 2 total, 1 up, 2 in 2024-08-02T17:39:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:15 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:15.553 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:15 smithi077 ceph-mon[37108]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:15.553 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:15 smithi077 ceph-mon[37108]: from='client.? 172.21.15.77:0/3315858122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]: dispatch 2024-08-02T17:39:15.554 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:15 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]: dispatch 2024-08-02T17:39:15.554 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:15 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]': finished 2024-08-02T17:39:15.554 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:15 smithi077 ceph-mon[37108]: osdmap e10: 2 total, 1 up, 2 in 2024-08-02T17:39:15.554 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:15 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:15.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:15 smithi069 ceph-mon[32746]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:15.774 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:15 smithi069 ceph-mon[32746]: from='client.? 172.21.15.77:0/3315858122' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]: dispatch 2024-08-02T17:39:15.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:15 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]: dispatch 2024-08-02T17:39:15.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:15 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb73073e-0fd6-4482-b800-03ef39b9c61e"}]': finished 2024-08-02T17:39:15.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:15 smithi069 ceph-mon[32746]: osdmap e10: 2 total, 1 up, 2 in 2024-08-02T17:39:15.775 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:15 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:16.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:16 smithi083 ceph-mon[37031]: from='client.? 172.21.15.77:0/2784369779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:16.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:16 smithi077 ceph-mon[37108]: from='client.? 172.21.15.77:0/2784369779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:16.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:16 smithi069 ceph-mon[32746]: from='client.? 172.21.15.77:0/2784369779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:17 smithi083 ceph-mon[37031]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:17.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:17 smithi077 ceph-mon[37108]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:17.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:17 smithi069 ceph-mon[32746]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:19.823 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:19 smithi077 ceph-mon[37108]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:20.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:19 smithi083 ceph-mon[37031]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:20.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:19 smithi069 ceph-mon[32746]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:21.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:21 smithi077 ceph-mon[37108]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:21.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:21 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-02T17:39:21.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:21 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:22.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:21 smithi083 ceph-mon[37031]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:22.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:21 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-02T17:39:22.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:21 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:22.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:21 smithi069 ceph-mon[32746]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:22.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:21 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-08-02T17:39:22.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:21 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:22 smithi083 ceph-mon[37031]: Deploying daemon osd.1 on smithi077 2024-08-02T17:39:23.064 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:22 smithi077 ceph-mon[37108]: Deploying daemon osd.1 on smithi077 2024-08-02T17:39:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:22 smithi069 ceph-mon[32746]: Deploying daemon osd.1 on smithi077 2024-08-02T17:39:24.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:23 smithi083 ceph-mon[37031]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:24.036 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:23 smithi077 ceph-mon[37108]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:24.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:23 smithi069 ceph-mon[32746]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:26.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:25 smithi083 ceph-mon[37031]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:26.034 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:25 smithi077 ceph-mon[37108]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:25 smithi069 ceph-mon[32746]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:27.019 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:27.019 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:27.019 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:27.979 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:27 smithi077 ceph-mon[37108]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:28.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:27 smithi083 ceph-mon[37031]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:27 smithi069 ceph-mon[32746]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:29.021 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.019+0000 7ffaabfff640 1 -- 172.21.15.77:0/3984588823 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffab4062360 con 0x7ffaa00a2b20 2024-08-02T17:39:29.021 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 1 on host 'smithi077' 2024-08-02T17:39:29.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ffaa00a2b20 msgr2=0x7ffaa00a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:29.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ffaa00a2b20 0x7ffaa00a4fe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4005fd0 tx=0x7ffaa4009520 comp rx=0 tx=0).stop 2024-08-02T17:39:29.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 msgr2=0x7ffab419a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:29.023 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 0x7ffab419a3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009230 tx=0x7ffa9c004360 comp rx=0 tx=0).stop 2024-08-02T17:39:29.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 shutdown_connections 2024-08-02T17:39:29.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ffaa00a2b20 0x7ffaa00a4fe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:29.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffab41014e0 0x7ffab41a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:29.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffab4100b40 0x7ffab419a3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:29.024 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 --2- 172.21.15.77:0/3984588823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffab4100200 0x7ffab4199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:29.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 >> 172.21.15.77:0/3984588823 conn(0x7ffab40fbfb0 msgr2=0x7ffab40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:39:29.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 shutdown_connections 2024-08-02T17:39:29.025 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:39:29.021+0000 7ffab95cf640 1 -- 172.21.15.77:0/3984588823 wait complete. 2024-08-02T17:39:29.623 DEBUG:teuthology.orchestra.run.smithi077:osd.1> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.1.service 2024-08-02T17:39:29.626 INFO:tasks.cephadm:Deploying osd.2 on smithi083 with /dev/vg_nvme/lv_4... 2024-08-02T17:39:29.626 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-02T17:39:29.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:29 smithi077 ceph-mon[37108]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:29.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:29.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:29.883 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:39:30.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:29 smithi083 ceph-mon[37031]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:30.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:30.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:29 smithi069 ceph-mon[32746]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:30.597 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:30 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1[41440]: 2024-08-02T17:39:30.274+0000 7f1cd2902640 -1 osd.1 0 log_to_monitors true 2024-08-02T17:39:31.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:31 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:31.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:31 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:31.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:31 smithi077 ceph-mon[37108]: from='osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-02T17:39:31.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:31 smithi077 ceph-mon[37108]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-02T17:39:31.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:31 smithi077 ceph-mon[37108]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:31 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:31 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:31 smithi083 ceph-mon[37031]: from='osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-02T17:39:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:31 smithi083 ceph-mon[37031]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-02T17:39:31.189 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:31 smithi083 ceph-mon[37031]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:31.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:31 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:31.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:31 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:31.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:31 smithi069 ceph-mon[32746]: from='osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-02T17:39:31.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:31 smithi069 ceph-mon[32746]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-08-02T17:39:31.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:31 smithi069 ceph-mon[32746]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:32.253 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:39:32.277 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_4 2024-08-02T17:39:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:32 smithi077 ceph-mon[37108]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-02T17:39:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:32 smithi077 ceph-mon[37108]: from='osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:39:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:32 smithi077 ceph-mon[37108]: osdmap e11: 2 total, 1 up, 2 in 2024-08-02T17:39:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:32 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:32 smithi077 ceph-mon[37108]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:39:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:32 smithi069 ceph-mon[32746]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-02T17:39:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:32 smithi069 ceph-mon[32746]: from='osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:39:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:32 smithi069 ceph-mon[32746]: osdmap e11: 2 total, 1 up, 2 in 2024-08-02T17:39:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:32 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:32 smithi069 ceph-mon[32746]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:39:32.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:32 smithi083 ceph-mon[37031]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-02T17:39:32.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:32 smithi083 ceph-mon[37031]: from='osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:39:32.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:32 smithi083 ceph-mon[37031]: osdmap e11: 2 total, 1 up, 2 in 2024-08-02T17:39:32.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:32 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:32.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:32 smithi083 ceph-mon[37031]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:39:32.532 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:39:32.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 -- 172.21.15.83:0/1958454998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7454100800 msgr2=0x7f7454100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:32.949 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 --2- 172.21.15.83:0/1958454998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7454100800 0x7f7454100c00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7448009900 tx=0x7f744802a410 comp rx=0 tx=0).stop 2024-08-02T17:39:32.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 -- 172.21.15.83:0/1958454998 shutdown_connections 2024-08-02T17:39:32.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 --2- 172.21.15.83:0/1958454998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7454101a00 0x7f7454101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:32.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 --2- 172.21.15.83:0/1958454998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74541023c0 0x7f7454108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:32.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 --2- 172.21.15.83:0/1958454998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7454100800 0x7f7454100c00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:32.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 -- 172.21.15.83:0/1958454998 >> 172.21.15.83:0/1958454998 conn(0x7f74540fbfb0 msgr2=0x7f74540fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:39:32.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 -- 172.21.15.83:0/1958454998 shutdown_connections 2024-08-02T17:39:32.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.938+0000 7f745a65a640 1 -- 172.21.15.83:0/1958454998 wait complete. 2024-08-02T17:39:32.950 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.939+0000 7f745a65a640 1 Processor -- start 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.939+0000 7f745a65a640 1 -- start start 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.939+0000 7f745a65a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7454100800 0x7f7454104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.939+0000 7f745a65a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7454101a00 0x7f7454102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.939+0000 7f745a65a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74541023c0 0x7f7454103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.939+0000 7f745a65a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f745410c490 con 0x7f7454100800 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.939+0000 7f745a65a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f745410c310 con 0x7f7454101a00 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.939+0000 7f745a65a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f745410c610 con 0x7f74541023c0 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74541023c0 0x7f7454103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:32.951 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74541023c0 0x7f7454103320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:39528/0 (socket says 172.21.15.83:39528) 2024-08-02T17:39:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 -- 172.21.15.83:0/2699499720 learned_addr learned my addr 172.21.15.83:0/2699499720 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:39:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f74537fe640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7454101a00 0x7f7454102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7453fff640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7454100800 0x7f7454104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 -- 172.21.15.83:0/2699499720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7454101a00 msgr2=0x7f7454102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7454101a00 0x7f7454102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 -- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7454100800 msgr2=0x7f7454104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7454100800 0x7f7454104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:32.952 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 -- 172.21.15.83:0/2699499720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7454103be0 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f7458bf7640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74541023c0 0x7f7454103320 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f744c00eaa0 tx=0x7f744c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f74517fa640 1 -- 172.21.15.83:0/2699499720 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00cdb0 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f74517fa640 1 -- 172.21.15.83:0/2699499720 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c005020 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74541a1e60 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f74517fa640 1 -- 172.21.15.83:0/2699499720 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c0116b0 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.940+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74541a2360 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.941+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74541a25f0 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.943+0000 7f74517fa640 1 -- 172.21.15.83:0/2699499720 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f744c005560 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.943+0000 7f74517fa640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f743c0a2b70 0x7f743c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.943+0000 7f74517fa640 1 -- 172.21.15.83:0/2699499720 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f744c0cdba0 con 0x7f74541023c0 2024-08-02T17:39:32.953 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.943+0000 7f7453fff640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f743c0a2b70 0x7f743c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:32.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.944+0000 7f7453fff640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f743c0a2b70 0x7f743c0a5030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74480096c0 tx=0x7f7448000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:39:32.954 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:32.947+0000 7f74517fa640 1 -- 172.21.15.83:0/2699499720 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f744c016030 con 0x7f74541023c0 2024-08-02T17:39:33.087 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:32 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1[41440]: 2024-08-02T17:39:32.799+0000 7f1cce885640 -1 osd.1 0 waiting for initial osdmap 2024-08-02T17:39:33.087 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:32 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1[41440]: 2024-08-02T17:39:32.803+0000 7f1cc66a5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:39:33.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:33.126+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f74541a27c0 con 0x7f743c0a2b70 2024-08-02T17:39:33.128 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:33 smithi083 ceph-mon[37031]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:39:33.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:33 smithi083 ceph-mon[37031]: osdmap e12: 2 total, 1 up, 2 in 2024-08-02T17:39:33.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:33.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:33.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:33 smithi083 ceph-mon[37031]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:33.129 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:33 smithi083 ceph-mon[37031]: from='osd.1 ' entity='osd.1' 2024-08-02T17:39:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:33 smithi077 ceph-mon[37108]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:39:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:33 smithi077 ceph-mon[37108]: osdmap e12: 2 total, 1 up, 2 in 2024-08-02T17:39:33.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:33.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:33.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:33 smithi077 ceph-mon[37108]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:33.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:33 smithi077 ceph-mon[37108]: from='osd.1 ' entity='osd.1' 2024-08-02T17:39:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:33 smithi069 ceph-mon[32746]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:39:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:33 smithi069 ceph-mon[32746]: osdmap e12: 2 total, 1 up, 2 in 2024-08-02T17:39:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:33 smithi069 ceph-mon[32746]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-02T17:39:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:33 smithi069 ceph-mon[32746]: from='osd.1 ' entity='osd.1' 2024-08-02T17:39:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:39:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:39:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi077 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi077 to 17088M 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:34.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:34.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] boot 2024-08-02T17:39:34.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: osdmap e13: 2 total, 2 up, 2 in 2024-08-02T17:39:34.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:34.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:39:34.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:39:34.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:34.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:34.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:34.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:34.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi077 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi077 to 17088M 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] boot 2024-08-02T17:39:34.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: osdmap e13: 2 total, 2 up, 2 in 2024-08-02T17:39:34.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:34.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:39:34.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi077 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:34.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi077 to 17088M 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: osd.1 [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] boot 2024-08-02T17:39:34.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: osdmap e13: 2 total, 2 up, 2 in 2024-08-02T17:39:34.361 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:39:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:35 smithi077 ceph-mon[37108]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:35.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:35 smithi069 ceph-mon[32746]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:35.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:35 smithi083 ceph-mon[37031]: pgmap v52: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:36 smithi083 ceph-mon[37031]: osdmap e14: 2 total, 2 up, 2 in 2024-08-02T17:39:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:36 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/247523827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]: dispatch 2024-08-02T17:39:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:36 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]: dispatch 2024-08-02T17:39:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:36 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]': finished 2024-08-02T17:39:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:36 smithi083 ceph-mon[37031]: osdmap e15: 3 total, 2 up, 3 in 2024-08-02T17:39:36.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:36 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:36.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:36 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/2424490460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:36.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:36 smithi077 ceph-mon[37108]: osdmap e14: 2 total, 2 up, 2 in 2024-08-02T17:39:36.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:36 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/247523827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]: dispatch 2024-08-02T17:39:36.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:36 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]: dispatch 2024-08-02T17:39:36.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:36 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]': finished 2024-08-02T17:39:36.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:36 smithi077 ceph-mon[37108]: osdmap e15: 3 total, 2 up, 3 in 2024-08-02T17:39:36.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:36 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:36.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:36 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/2424490460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:36.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:36 smithi069 ceph-mon[32746]: osdmap e14: 2 total, 2 up, 2 in 2024-08-02T17:39:36.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:36 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/247523827' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]: dispatch 2024-08-02T17:39:36.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:36 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]: dispatch 2024-08-02T17:39:36.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:36 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "941227f1-e5e4-4146-9e0d-4405d1f5271c"}]': finished 2024-08-02T17:39:36.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:36 smithi069 ceph-mon[32746]: osdmap e15: 3 total, 2 up, 3 in 2024-08-02T17:39:36.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:36 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:36.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:36 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/2424490460' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:37.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:37 smithi069 ceph-mon[32746]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:37.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:37 smithi083 ceph-mon[37031]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:37.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:37 smithi077 ceph-mon[37108]: pgmap v55: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:40.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:39 smithi083 ceph-mon[37031]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:40.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:39 smithi077 ceph-mon[37108]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:39 smithi069 ceph-mon[32746]: pgmap v56: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:42.009 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:41 smithi083 ceph-mon[37031]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:42.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:41 smithi077 ceph-mon[37108]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:41 smithi069 ceph-mon[32746]: pgmap v57: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:42 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-02T17:39:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:42 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:43.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:42 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-02T17:39:43.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:42 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:42 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-08-02T17:39:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:42 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:43.928 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:43 smithi083 ceph-mon[37031]: Deploying daemon osd.2 on smithi083 2024-08-02T17:39:43.929 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:43 smithi083 ceph-mon[37031]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:44.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:43 smithi077 ceph-mon[37108]: Deploying daemon osd.2 on smithi083 2024-08-02T17:39:44.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:43 smithi077 ceph-mon[37108]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:43 smithi069 ceph-mon[32746]: Deploying daemon osd.2 on smithi083 2024-08-02T17:39:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:43 smithi069 ceph-mon[32746]: pgmap v58: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:46.005 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:45 smithi083 ceph-mon[37031]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:46.040 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:45 smithi077 ceph-mon[37108]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:46.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:45 smithi069 ceph-mon[32746]: pgmap v59: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:47.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:47 smithi083 ceph-mon[37031]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:47.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:47.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:47.823 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:48.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:47 smithi077 ceph-mon[37108]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:48.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:48.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:48.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:47 smithi069 ceph-mon[32746]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:49.478 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 2 on host 'smithi083' 2024-08-02T17:39:49.478 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.475+0000 7f74517fa640 1 -- 172.21.15.83:0/2699499720 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f74541a27c0 con 0x7f743c0a2b70 2024-08-02T17:39:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f743c0a2b70 msgr2=0x7f743c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:49.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f743c0a2b70 0x7f743c0a5030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74480096c0 tx=0x7f7448000bc0 comp rx=0 tx=0).stop 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74541023c0 msgr2=0x7f7454103320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74541023c0 0x7f7454103320 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f744c00eaa0 tx=0x7f744c00ef60 comp rx=0 tx=0).stop 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 shutdown_connections 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f743c0a2b70 0x7f743c0a5030 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74541023c0 0x7f7454103320 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7454101a00 0x7f7454102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 --2- 172.21.15.83:0/2699499720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7454100800 0x7f7454104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 >> 172.21.15.83:0/2699499720 conn(0x7f74540fbfb0 msgr2=0x7f7454069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 shutdown_connections 2024-08-02T17:39:49.480 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:39:49.478+0000 7f745a65a640 1 -- 172.21.15.83:0/2699499720 wait complete. 2024-08-02T17:39:49.818 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:49 smithi083 ceph-mon[37031]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:49.818 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:49.818 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:50.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:49 smithi077 ceph-mon[37108]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:50.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:50.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:49 smithi069 ceph-mon[32746]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:50.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:50.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:50.472 DEBUG:teuthology.orchestra.run.smithi083:osd.2> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.2.service 2024-08-02T17:39:50.475 INFO:tasks.cephadm:Deploying osd.3 on smithi069 with /dev/vg_nvme/lv_3... 2024-08-02T17:39:50.475 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-02T17:39:50.740 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:39:51.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:50 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2[42021]: 2024-08-02T17:39:50.710+0000 7f33b4a82640 -1 osd.2 0 log_to_monitors true 2024-08-02T17:39:51.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:51.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:51.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:51 smithi069 ceph-mon[32746]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:51.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:51 smithi069 ceph-mon[32746]: from='osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-02T17:39:51.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:51 smithi069 ceph-mon[32746]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-02T17:39:51.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:51.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:51.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:51 smithi077 ceph-mon[37108]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:51.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:51 smithi077 ceph-mon[37108]: from='osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-02T17:39:51.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:51 smithi077 ceph-mon[37108]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-02T17:39:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:51 smithi083 ceph-mon[37031]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:51 smithi083 ceph-mon[37031]: from='osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-02T17:39:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:51 smithi083 ceph-mon[37031]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-08-02T17:39:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:52 smithi083 ceph-mon[37031]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-02T17:39:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:52 smithi083 ceph-mon[37031]: osdmap e16: 3 total, 2 up, 3 in 2024-08-02T17:39:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:52 smithi083 ceph-mon[37031]: from='osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:39:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:52 smithi083 ceph-mon[37031]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:39:52.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:52 smithi077 ceph-mon[37108]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-02T17:39:52.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:52 smithi077 ceph-mon[37108]: osdmap e16: 3 total, 2 up, 3 in 2024-08-02T17:39:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:52 smithi077 ceph-mon[37108]: from='osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:39:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:52 smithi077 ceph-mon[37108]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:39:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:52 smithi069 ceph-mon[32746]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-02T17:39:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:52 smithi069 ceph-mon[32746]: osdmap e16: 3 total, 2 up, 3 in 2024-08-02T17:39:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:52 smithi069 ceph-mon[32746]: from='osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:39:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:52 smithi069 ceph-mon[32746]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:39:53.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2[42021]: 2024-08-02T17:39:52.781+0000 7f33b0a05640 -1 osd.2 0 waiting for initial osdmap 2024-08-02T17:39:53.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2[42021]: 2024-08-02T17:39:52.785+0000 7f33a8825640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:39:53.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:53 smithi069 ceph-mon[32746]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:39:53.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:53 smithi069 ceph-mon[32746]: osdmap e17: 3 total, 2 up, 3 in 2024-08-02T17:39:53.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:53 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:53.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:53 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:53.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:53 smithi069 ceph-mon[32746]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:53.154 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:53 smithi069 ceph-mon[32746]: from='osd.2 ' entity='osd.2' 2024-08-02T17:39:53.156 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:39:53.182 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_3 2024-08-02T17:39:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:53 smithi077 ceph-mon[37108]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:39:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:53 smithi077 ceph-mon[37108]: osdmap e17: 3 total, 2 up, 3 in 2024-08-02T17:39:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:53 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:53 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:53 smithi077 ceph-mon[37108]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:53 smithi077 ceph-mon[37108]: from='osd.2 ' entity='osd.2' 2024-08-02T17:39:53.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:53 smithi083 ceph-mon[37031]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:39:53.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:53 smithi083 ceph-mon[37031]: osdmap e17: 3 total, 2 up, 3 in 2024-08-02T17:39:53.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:53 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:53.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:53 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:53.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:53 smithi083 ceph-mon[37031]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-08-02T17:39:53.409 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:53 smithi083 ceph-mon[37031]: from='osd.2 ' entity='osd.2' 2024-08-02T17:39:53.438 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:39:53.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 -- 172.21.15.69:0/356399592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0441034f0 msgr2=0x7fb044109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:53.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 --2- 172.21.15.69:0/356399592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0441034f0 0x7fb044109d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009ca0 tx=0x7fb03c02a8c0 comp rx=0 tx=0).stop 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 -- 172.21.15.69:0/356399592 shutdown_connections 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 --2- 172.21.15.69:0/356399592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0441034f0 0x7fb044109d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 --2- 172.21.15.69:0/356399592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb044102b30 0x7fb044102fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 --2- 172.21.15.69:0/356399592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb044101930 0x7fb044101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 -- 172.21.15.69:0/356399592 >> 172.21.15.69:0/356399592 conn(0x7fb0440fd0e0 msgr2=0x7fb0440ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 -- 172.21.15.69:0/356399592 shutdown_connections 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.941+0000 7fb049ef7640 1 -- 172.21.15.69:0/356399592 wait complete. 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb049ef7640 1 Processor -- start 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb049ef7640 1 -- start start 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb049ef7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb044101930 0x7fb044075cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb049ef7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb044102b30 0x7fb044074310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:53.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb049ef7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0441034f0 0x7fb044074850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb049ef7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb04410d590 con 0x7fb044102b30 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb049ef7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb04410d410 con 0x7fb0441034f0 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb049ef7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb04410d710 con 0x7fb044101930 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb044102b30 0x7fb044074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb042ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb044102b30 0x7fb044074310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36830/0 (socket says 172.21.15.69:36830) 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.942+0000 7fb042ffd640 1 -- 172.21.15.69:0/4219513053 learned_addr learned my addr 172.21.15.69:0/4219513053 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb0437fe640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb044101930 0x7fb044075cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb043fff640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0441034f0 0x7fb044074850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb042ffd640 1 -- 172.21.15.69:0/4219513053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb044101930 msgr2=0x7fb044075cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb042ffd640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb044101930 0x7fb044075cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb042ffd640 1 -- 172.21.15.69:0/4219513053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0441034f0 msgr2=0x7fb044074850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:39:53.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb042ffd640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0441034f0 0x7fb044074850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb042ffd640 1 -- 172.21.15.69:0/4219513053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb044075110 con 0x7fb044102b30 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb043fff640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0441034f0 0x7fb044074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb042ffd640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb044102b30 0x7fb044074310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb03800d950 tx=0x7fb03800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb040ff9640 1 -- 172.21.15.69:0/4219513053 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038017030 con 0x7fb044102b30 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0441a1ce0 con 0x7fb044102b30 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb040ff9640 1 -- 172.21.15.69:0/4219513053 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb038004020 con 0x7fb044102b30 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb040ff9640 1 -- 172.21.15.69:0/4219513053 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380047a0 con 0x7fb044102b30 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.943+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441a2190 con 0x7fb044102b30 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.944+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb044108c00 con 0x7fb044102b30 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.946+0000 7fb040ff9640 1 -- 172.21.15.69:0/4219513053 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fb0380041d0 con 0x7fb044102b30 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.947+0000 7fb040ff9640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb02c0a2af0 0x7fb02c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.947+0000 7fb0437fe640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb02c0a2af0 0x7fb02c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:39:53.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.947+0000 7fb040ff9640 1 -- 172.21.15.69:0/4219513053 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fb0380cd460 con 0x7fb044102b30 2024-08-02T17:39:53.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.947+0000 7fb0437fe640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb02c0a2af0 0x7fb02c0a4fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb0340058c0 tx=0x7fb034005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:39:53.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:53.951+0000 7fb040ff9640 1 -- 172.21.15.69:0/4219513053 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb03809b9f0 con 0x7fb044102b30 2024-08-02T17:39:54.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:54.134+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb044062360 con 0x7fb02c0a2af0 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi083 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi083 to 17088M 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:54.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:54.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:54.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:54.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] boot 2024-08-02T17:39:54.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: osdmap e18: 3 total, 3 up, 3 in 2024-08-02T17:39:54.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:54.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi083 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi083 to 17088M 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:54.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:54.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:54.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] boot 2024-08-02T17:39:54.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: osdmap e18: 3 total, 3 up, 3 in 2024-08-02T17:39:54.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:39:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi083 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi083 to 17088M 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:54.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:39:54.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:54.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: osd.2 [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] boot 2024-08-02T17:39:54.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: osdmap e18: 3 total, 3 up, 3 in 2024-08-02T17:39:54.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:39:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:55 smithi077 ceph-mon[37108]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:55 smithi077 ceph-mon[37108]: pgmap v67: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-02T17:39:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:55 smithi069 ceph-mon[32746]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:55 smithi069 ceph-mon[32746]: pgmap v67: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:55.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-02T17:39:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:55 smithi083 ceph-mon[37031]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:39:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:39:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:39:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:39:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:55 smithi083 ceph-mon[37031]: pgmap v67: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-08-02T17:39:56.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-02T17:39:56.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:56 smithi077 ceph-mon[37108]: osdmap e19: 3 total, 3 up, 3 in 2024-08-02T17:39:56.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-02T17:39:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-02T17:39:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:56 smithi069 ceph-mon[32746]: osdmap e19: 3 total, 3 up, 3 in 2024-08-02T17:39:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-02T17:39:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-02T17:39:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:56 smithi083 ceph-mon[37031]: osdmap e19: 3 total, 3 up, 3 in 2024-08-02T17:39:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-08-02T17:39:56.690 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:56 smithi069 sudo[48579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-02T17:39:56.690 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:56 smithi069 sudo[48579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:56.690 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:56 smithi069 sudo[48579]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:56.690 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:56 smithi069 sudo[48579]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:57.085 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:56 smithi069 sudo[48588]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-02T17:39:57.085 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:56 smithi069 sudo[48588]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:57.085 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:56 smithi069 sudo[48588]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:57.085 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 17:39:56 smithi069 sudo[48588]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:57.222 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:56 smithi077 sudo[45634]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-02T17:39:57.223 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:56 smithi077 sudo[45634]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:57.223 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:56 smithi077 sudo[45634]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:57.223 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:56 smithi077 sudo[45634]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:57.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-02T17:39:57.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:57 smithi077 ceph-mon[37108]: osdmap e20: 3 total, 3 up, 3 in 2024-08-02T17:39:57.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:57 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3643603101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9482789a-9684-4d80-90d1-4116f82bcce4"}]: dispatch 2024-08-02T17:39:57.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:57 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3643603101' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9482789a-9684-4d80-90d1-4116f82bcce4"}]': finished 2024-08-02T17:39:57.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:57 smithi077 ceph-mon[37108]: osdmap e21: 4 total, 3 up, 4 in 2024-08-02T17:39:57.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:39:57.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:57 smithi077 ceph-mon[37108]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:57.224 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-02T17:39:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:57 smithi069 ceph-mon[32746]: osdmap e20: 3 total, 3 up, 3 in 2024-08-02T17:39:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:57 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3643603101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9482789a-9684-4d80-90d1-4116f82bcce4"}]: dispatch 2024-08-02T17:39:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:57 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3643603101' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9482789a-9684-4d80-90d1-4116f82bcce4"}]': finished 2024-08-02T17:39:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:57 smithi069 ceph-mon[32746]: osdmap e21: 4 total, 3 up, 4 in 2024-08-02T17:39:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:39:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:57 smithi069 ceph-mon[32746]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:57.488 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-02T17:39:57.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:57 smithi083 ceph-mon[37031]: osdmap e20: 3 total, 3 up, 3 in 2024-08-02T17:39:57.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:57 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3643603101' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9482789a-9684-4d80-90d1-4116f82bcce4"}]: dispatch 2024-08-02T17:39:57.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:57 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3643603101' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9482789a-9684-4d80-90d1-4116f82bcce4"}]': finished 2024-08-02T17:39:57.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:57 smithi083 ceph-mon[37031]: osdmap e21: 4 total, 3 up, 4 in 2024-08-02T17:39:57.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:39:57.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:57 smithi083 ceph-mon[37031]: pgmap v71: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:57.489 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:57.597 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:57 smithi077 sudo[45637]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-02T17:39:57.598 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:57 smithi077 sudo[45637]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:57.598 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:57 smithi077 sudo[45637]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:57.598 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 17:39:57 smithi077 sudo[45637]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:57.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:39:57.726+0000 7fb040ff9640 1 -- 172.21.15.69:0/4219513053 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb038097960 con 0x7fb044102b30 2024-08-02T17:39:57.746 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:57 smithi083 sudo[45899]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-02T17:39:57.747 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:57 smithi083 sudo[45899]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:57.747 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:57 smithi083 sudo[45899]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:57.747 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:57 smithi083 sudo[45899]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:58.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:57 smithi083 sudo[45902]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-02T17:39:58.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:57 smithi083 sudo[45902]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:58.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:57 smithi083 sudo[45902]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:58.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 17:39:57 smithi083 sudo[45902]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:58.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 sudo[49031]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-02T17:39:58.275 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 sudo[49031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:58.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 sudo[49031]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:58.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 sudo[49031]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:58.536 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1592349944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:58.536 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:58.536 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 ceph-mon[37031]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-02T17:39:58.537 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:58.537 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 ceph-mon[37031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:58.537 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 sudo[45905]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-02T17:39:58.537 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 sudo[45905]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:58.537 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 sudo[45905]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:58.537 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 sudo[45905]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:58.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:58 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1592349944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:58.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:58 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:58.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:58 smithi077 ceph-mon[37108]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-02T17:39:58.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:58 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:58.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:58 smithi077 ceph-mon[37108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:58.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 sudo[49035]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-02T17:39:58.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 sudo[49035]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:58.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 sudo[49035]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:58.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 sudo[49035]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:58.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1592349944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:39:58.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:58.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 ceph-mon[32746]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-02T17:39:58.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:39:58.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:58 smithi069 ceph-mon[32746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:59.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 sudo[45908]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-02T17:39:59.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 sudo[45908]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:59.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 sudo[45908]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:59.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:58 smithi083 sudo[45908]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:59.273 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 sudo[45640]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-02T17:39:59.273 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 sudo[45640]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:59.274 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 sudo[45640]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:59.274 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 sudo[45640]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 sudo[45643]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-02T17:39:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 sudo[45643]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-02T17:39:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 sudo[45643]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-02T17:39:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 sudo[45643]: pam_unix(sudo:session): session closed for user root 2024-08-02T17:39:59.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:39:59.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:59.600 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.600 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.600 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.600 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:59.600 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:39:59.600 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:59.601 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.601 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.601 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.601 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:39:59 smithi077 ceph-mon[37108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:39:59.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:39:59.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:59.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:59.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:39:59.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:59.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.610 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.611 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:39:59 smithi069 ceph-mon[32746]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:39:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:39:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: pgmap v72: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:39:59.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:39:59 smithi083 ceph-mon[37031]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-02T17:40:00.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:00 smithi077 ceph-mon[37108]: overall HEALTH_OK 2024-08-02T17:40:00.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:00 smithi069 ceph-mon[32746]: overall HEALTH_OK 2024-08-02T17:40:00.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:00 smithi083 ceph-mon[37031]: overall HEALTH_OK 2024-08-02T17:40:01.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:01 smithi069 ceph-mon[32746]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:01.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:01 smithi077 ceph-mon[37108]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:01.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:01 smithi083 ceph-mon[37031]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:04.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:03 smithi083 ceph-mon[37031]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:04.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-02T17:40:04.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:04.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:03 smithi069 ceph-mon[32746]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:04.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-02T17:40:04.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:04.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:03 smithi077 ceph-mon[37108]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:04.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-08-02T17:40:04.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:04.844 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:04 smithi069 ceph-mon[32746]: Deploying daemon osd.3 on smithi069 2024-08-02T17:40:05.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:04 smithi083 ceph-mon[37031]: Deploying daemon osd.3 on smithi069 2024-08-02T17:40:05.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:04 smithi077 ceph-mon[37108]: Deploying daemon osd.3 on smithi069 2024-08-02T17:40:06.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:05 smithi083 ceph-mon[37031]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-08-02T17:40:06.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:05 smithi069 ceph-mon[32746]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-08-02T17:40:06.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:05 smithi077 ceph-mon[37108]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-08-02T17:40:07.946 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:07 smithi069 ceph-mon[32746]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-08-02T17:40:08.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:07 smithi077 ceph-mon[37108]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-08-02T17:40:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:07 smithi083 ceph-mon[37031]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-08-02T17:40:09.058 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:09.059 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:09.059 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:08 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:09.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:08 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:09.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:08 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:09.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:08 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:08 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:08 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:08 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:09.963 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:09 smithi069 ceph-mon[32746]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-08-02T17:40:10.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:09 smithi077 ceph-mon[37108]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-08-02T17:40:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:09 smithi083 ceph-mon[37031]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-08-02T17:40:11.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:11 smithi069 ceph-mon[32746]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-02T17:40:11.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:11 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:11.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:11 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:11.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:11 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:11.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:11 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:11.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:11 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:11.957 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 3 on host 'smithi069' 2024-08-02T17:40:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.955+0000 7fb040ff9640 1 -- 172.21.15.69:0/4219513053 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb044062360 con 0x7fb02c0a2af0 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb02c0a2af0 msgr2=0x7fb02c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb02c0a2af0 0x7fb02c0a4fb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb0340058c0 tx=0x7fb034005850 comp rx=0 tx=0).stop 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb044102b30 msgr2=0x7fb044074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb044102b30 0x7fb044074310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb03800d950 tx=0x7fb03800de10 comp rx=0 tx=0).stop 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 shutdown_connections 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb02c0a2af0 0x7fb02c0a4fb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb044102b30 0x7fb044074310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0441034f0 0x7fb044074850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 --2- 172.21.15.69:0/4219513053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb044101930 0x7fb044075cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 >> 172.21.15.69:0/4219513053 conn(0x7fb0440fd0e0 msgr2=0x7fb044105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:40:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 shutdown_connections 2024-08-02T17:40:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:11.957+0000 7fb049ef7640 1 -- 172.21.15.69:0/4219513053 wait complete. 2024-08-02T17:40:12.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:11 smithi077 ceph-mon[37108]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-02T17:40:12.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:11 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:11 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:11 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:12.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:11 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:12.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:11 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:11 smithi083 ceph-mon[37031]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-08-02T17:40:12.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:11 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:11 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:11 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:12.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:11 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:12.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:11 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.599 DEBUG:teuthology.orchestra.run.smithi069:osd.3> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.3.service 2024-08-02T17:40:12.602 INFO:tasks.cephadm:Deploying osd.4 on smithi077 with /dev/vg_nvme/lv_3... 2024-08-02T17:40:12.602 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-02T17:40:12.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:12 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:12.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:12 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:12 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:12 smithi069 ceph-mon[32746]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-02T17:40:12.863 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T17:40:12.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:12 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:12.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:12 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:12 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:12.985 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:12 smithi077 ceph-mon[37108]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-02T17:40:13.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:12 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:13.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:12 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:13.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:12 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:13.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:12 smithi083 ceph-mon[37031]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-08-02T17:40:14.095 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:13 smithi077 ceph-mon[37108]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-02T17:40:14.095 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:13 smithi077 ceph-mon[37108]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-02T17:40:14.095 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:13 smithi077 ceph-mon[37108]: osdmap e22: 4 total, 3 up, 4 in 2024-08-02T17:40:14.095 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:13 smithi077 ceph-mon[37108]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:40:14.096 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:14.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:13 smithi069 ceph-mon[32746]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-02T17:40:14.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:13 smithi069 ceph-mon[32746]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-02T17:40:14.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:13 smithi069 ceph-mon[32746]: osdmap e22: 4 total, 3 up, 4 in 2024-08-02T17:40:14.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:13 smithi069 ceph-mon[32746]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:40:14.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:14.109 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 17:40:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3[50600]: 2024-08-02T17:40:13.968+0000 7ff5f804e640 -1 osd.3 0 waiting for initial osdmap 2024-08-02T17:40:14.109 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 17:40:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3[50600]: 2024-08-02T17:40:13.973+0000 7ff5f3e76640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:40:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:13 smithi083 ceph-mon[37031]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-08-02T17:40:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:13 smithi083 ceph-mon[37031]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-02T17:40:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:13 smithi083 ceph-mon[37031]: osdmap e22: 4 total, 3 up, 4 in 2024-08-02T17:40:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:13 smithi083 ceph-mon[37031]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:40:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.165 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:14 smithi069 ceph-mon[32746]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:40:15.165 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:14 smithi069 ceph-mon[32746]: osdmap e23: 4 total, 3 up, 4 in 2024-08-02T17:40:15.166 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.166 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.166 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:14 smithi069 ceph-mon[32746]: osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] boot 2024-08-02T17:40:15.166 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:14 smithi069 ceph-mon[32746]: osdmap e24: 4 total, 4 up, 4 in 2024-08-02T17:40:15.166 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.166 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:14 smithi069 ceph-mon[32746]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:14 smithi083 ceph-mon[37031]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:40:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:14 smithi083 ceph-mon[37031]: osdmap e23: 4 total, 3 up, 4 in 2024-08-02T17:40:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:14 smithi083 ceph-mon[37031]: osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] boot 2024-08-02T17:40:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:14 smithi083 ceph-mon[37031]: osdmap e24: 4 total, 4 up, 4 in 2024-08-02T17:40:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:14 smithi083 ceph-mon[37031]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:15.307 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:14 smithi077 ceph-mon[37108]: from='osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:40:15.308 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:14 smithi077 ceph-mon[37108]: osdmap e23: 4 total, 3 up, 4 in 2024-08-02T17:40:15.308 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.308 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.308 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:14 smithi077 ceph-mon[37108]: osd.3 [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] boot 2024-08-02T17:40:15.308 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:14 smithi077 ceph-mon[37108]: osdmap e24: 4 total, 4 up, 4 in 2024-08-02T17:40:15.308 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:40:15.308 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:14 smithi077 ceph-mon[37108]: pgmap v83: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-08-02T17:40:15.310 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:40:15.335 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_3 2024-08-02T17:40:15.594 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 -- 172.21.15.77:0/4176814388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 msgr2=0x7f9e681070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/4176814388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 0x7f9e681070b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009900 tx=0x7f9e5802a410 comp rx=0 tx=0).stop 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 -- 172.21.15.77:0/4176814388 shutdown_connections 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/4176814388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e681075f0 0x7f9e68109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/4176814388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 0x7f9e681070b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/4176814388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102390 0x7f9e68104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 -- 172.21.15.77:0/4176814388 >> 172.21.15.77:0/4176814388 conn(0x7f9e680fbfb0 msgr2=0x7f9e680fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 -- 172.21.15.77:0/4176814388 shutdown_connections 2024-08-02T17:40:16.134 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.067+0000 7f9e6f54b640 1 -- 172.21.15.77:0/4176814388 wait complete. 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6f54b640 1 Processor -- start 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6f54b640 1 -- start start 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6f54b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102390 0x7f9e68199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6f54b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 0x7f9e6819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6f54b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e681075f0 0x7f9e681a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6f54b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e681014c0 con 0x7f9e68102390 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6f54b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9e68101340 con 0x7f9e68104cc0 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6f54b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e68101640 con 0x7f9e681075f0 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6cae6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 0x7f9e6819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6cae6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 0x7f9e6819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:49500/0 (socket says 172.21.15.77:49500) 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6cae6640 1 -- 172.21.15.77:0/2323431240 learned_addr learned my addr 172.21.15.77:0/2323431240 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-08-02T17:40:16.135 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6d2e7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102390 0x7f9e68199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6dae8640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e681075f0 0x7f9e681a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6cae6640 1 -- 172.21.15.77:0/2323431240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e681075f0 msgr2=0x7f9e681a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6cae6640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e681075f0 0x7f9e681a13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6cae6640 1 -- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102390 msgr2=0x7f9e68199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6cae6640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102390 0x7f9e68199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6cae6640 1 -- 172.21.15.77:0/2323431240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e681a1a60 con 0x7f9e68104cc0 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.068+0000 7f9e6dae8640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e681075f0 0x7f9e681a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.069+0000 7f9e6cae6640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 0x7f9e6819a370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e58031610 tx=0x7f9e580043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.069+0000 7f9e5e7fc640 1 -- 172.21.15.77:0/2323431240 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5803f030 con 0x7f9e68104cc0 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.069+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e681a1cf0 con 0x7f9e68104cc0 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.069+0000 7f9e5e7fc640 1 -- 172.21.15.77:0/2323431240 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e58003ce0 con 0x7f9e68104cc0 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.069+0000 7f9e5e7fc640 1 -- 172.21.15.77:0/2323431240 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e58035070 con 0x7f9e68104cc0 2024-08-02T17:40:16.136 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.069+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e681a21c0 con 0x7f9e68104cc0 2024-08-02T17:40:16.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.070+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e681a23f0 con 0x7f9e68104cc0 2024-08-02T17:40:16.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.071+0000 7f9e5e7fc640 1 -- 172.21.15.77:0/2323431240 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9e58042030 con 0x7f9e68104cc0 2024-08-02T17:40:16.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.072+0000 7f9e5e7fc640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e540a2bb0 0x7f9e540a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:16.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.072+0000 7f9e5e7fc640 1 -- 172.21.15.77:0/2323431240 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 3698+0+0 (secure 0 0 0) 0x7f9e580ee5d0 con 0x7f9e68104cc0 2024-08-02T17:40:16.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.073+0000 7f9e6d2e7640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e540a2bb0 0x7f9e540a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:16.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.073+0000 7f9e6d2e7640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e540a2bb0 0x7f9e540a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e640058c0 tx=0x7f9e6400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:40:16.137 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.076+0000 7f9e5e7fc640 1 -- 172.21.15.77:0/2323431240 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9e580bc730 con 0x7f9e68104cc0 2024-08-02T17:40:16.263 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:16.262+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f9e68109150 con 0x7f9e540a2bb0 2024-08-02T17:40:16.584 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:40:16.584 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:40:16.584 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: osdmap e25: 4 total, 4 up, 4 in 2024-08-02T17:40:16.584 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi069 2024-08-02T17:40:16.584 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.584 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi069 to 8543M 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:40:16.585 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:16 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:16.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:40:16.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:40:16.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: osdmap e25: 4 total, 4 up, 4 in 2024-08-02T17:40:16.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi069 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi069 to 8543M 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:40:16.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:16 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:16.627 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:40:16.627 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:40:16.627 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: osdmap e25: 4 total, 4 up, 4 in 2024-08-02T17:40:16.627 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi069 2024-08-02T17:40:16.627 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.627 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.627 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:16.627 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:16.628 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi069 to 8543M 2024-08-02T17:40:16.628 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.628 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:16.628 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:16.628 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:16.628 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:40:16.628 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:40:16.628 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:16 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:17.508 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:17 smithi077 ceph-mon[37108]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:40:17.508 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:17 smithi077 ceph-mon[37108]: osdmap e26: 4 total, 4 up, 4 in 2024-08-02T17:40:17.509 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:17 smithi077 ceph-mon[37108]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:17 smithi083 ceph-mon[37031]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:40:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:17 smithi083 ceph-mon[37031]: osdmap e26: 4 total, 4 up, 4 in 2024-08-02T17:40:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:17 smithi083 ceph-mon[37031]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:17.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:17 smithi069 ceph-mon[32746]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:40:17.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:17 smithi069 ceph-mon[32746]: osdmap e26: 4 total, 4 up, 4 in 2024-08-02T17:40:17.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:17 smithi069 ceph-mon[32746]: pgmap v86: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:18.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:18 smithi077 ceph-mon[37108]: osdmap e27: 4 total, 4 up, 4 in 2024-08-02T17:40:18.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:18 smithi069 ceph-mon[32746]: osdmap e27: 4 total, 4 up, 4 in 2024-08-02T17:40:18.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:18 smithi083 ceph-mon[37031]: osdmap e27: 4 total, 4 up, 4 in 2024-08-02T17:40:19.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:19 smithi077 ceph-mon[37108]: osdmap e28: 4 total, 4 up, 4 in 2024-08-02T17:40:19.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:19 smithi077 ceph-mon[37108]: from='client.? 172.21.15.77:0/554821948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]: dispatch 2024-08-02T17:40:19.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:19 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]: dispatch 2024-08-02T17:40:19.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:19 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]': finished 2024-08-02T17:40:19.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:19 smithi077 ceph-mon[37108]: osdmap e29: 5 total, 4 up, 5 in 2024-08-02T17:40:19.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:19 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:19.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:19 smithi077 ceph-mon[37108]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:19.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:19 smithi077 ceph-mon[37108]: from='client.? 172.21.15.77:0/2423093818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:40:19.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:19 smithi069 ceph-mon[32746]: osdmap e28: 4 total, 4 up, 4 in 2024-08-02T17:40:19.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:19 smithi069 ceph-mon[32746]: from='client.? 172.21.15.77:0/554821948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]: dispatch 2024-08-02T17:40:19.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:19 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]: dispatch 2024-08-02T17:40:19.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:19 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]': finished 2024-08-02T17:40:19.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:19 smithi069 ceph-mon[32746]: osdmap e29: 5 total, 4 up, 5 in 2024-08-02T17:40:19.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:19 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:19.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:19 smithi069 ceph-mon[32746]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:19.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:19 smithi069 ceph-mon[32746]: from='client.? 172.21.15.77:0/2423093818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:40:19.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:19 smithi083 ceph-mon[37031]: osdmap e28: 4 total, 4 up, 4 in 2024-08-02T17:40:19.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:19 smithi083 ceph-mon[37031]: from='client.? 172.21.15.77:0/554821948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]: dispatch 2024-08-02T17:40:19.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:19 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]: dispatch 2024-08-02T17:40:19.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:19 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e22a3571-a363-4b52-9b84-725cf9b5fe4c"}]': finished 2024-08-02T17:40:19.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:19 smithi083 ceph-mon[37031]: osdmap e29: 5 total, 4 up, 5 in 2024-08-02T17:40:19.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:19 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:19.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:19 smithi083 ceph-mon[37031]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:19.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:19 smithi083 ceph-mon[37031]: from='client.? 172.21.15.77:0/2423093818' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:40:22.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:21 smithi083 ceph-mon[37031]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-08-02T17:40:22.048 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:21 smithi077 ceph-mon[37108]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-08-02T17:40:22.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:21 smithi069 ceph-mon[32746]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-08-02T17:40:24.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:23 smithi083 ceph-mon[37031]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:40:24.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:23 smithi077 ceph-mon[37108]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:40:24.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:23 smithi069 ceph-mon[32746]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:40:26.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:25 smithi077 ceph-mon[37108]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-02T17:40:26.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:25 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-02T17:40:26.041 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:25 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:25 smithi069 ceph-mon[32746]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-02T17:40:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:25 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-02T17:40:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:25 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:25 smithi083 ceph-mon[37031]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-02T17:40:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:25 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-08-02T17:40:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:25 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:27.029 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:26 smithi077 ceph-mon[37108]: Deploying daemon osd.4 on smithi077 2024-08-02T17:40:27.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:26 smithi069 ceph-mon[32746]: Deploying daemon osd.4 on smithi077 2024-08-02T17:40:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:26 smithi083 ceph-mon[37031]: Deploying daemon osd.4 on smithi077 2024-08-02T17:40:28.067 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:27 smithi077 ceph-mon[37108]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-02T17:40:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:27 smithi069 ceph-mon[32746]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-02T17:40:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:27 smithi083 ceph-mon[37031]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-08-02T17:40:29.843 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:29 smithi077 ceph-mon[37108]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-08-02T17:40:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:29 smithi069 ceph-mon[32746]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-08-02T17:40:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:29 smithi083 ceph-mon[37031]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-08-02T17:40:31.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:31.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:31.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:31.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:31 smithi077 ceph-mon[37108]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-02T17:40:32.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:31 smithi069 ceph-mon[32746]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-02T17:40:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:31 smithi083 ceph-mon[37031]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-02T17:40:33.812 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 4 on host 'smithi077' 2024-08-02T17:40:33.813 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.810+0000 7f9e5e7fc640 1 -- 172.21.15.77:0/2323431240 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9e68109150 con 0x7f9e540a2bb0 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e540a2bb0 msgr2=0x7f9e540a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e540a2bb0 0x7f9e540a5070 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9e640058c0 tx=0x7f9e6400a2d0 comp rx=0 tx=0).stop 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 msgr2=0x7f9e6819a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 0x7f9e6819a370 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9e58031610 tx=0x7f9e580043b0 comp rx=0 tx=0).stop 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 shutdown_connections 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e540a2bb0 0x7f9e540a5070 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e681075f0 0x7f9e681a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e68104cc0 0x7f9e6819a370 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 --2- 172.21.15.77:0/2323431240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e68102390 0x7f9e68199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:33.815 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 >> 172.21.15.77:0/2323431240 conn(0x7f9e680fbfb0 msgr2=0x7f9e68107a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:40:33.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 shutdown_connections 2024-08-02T17:40:33.816 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:40:33.813+0000 7f9e6f54b640 1 -- 172.21.15.77:0/2323431240 wait complete. 2024-08-02T17:40:34.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:33 smithi077 ceph-mon[37108]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:34.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:34.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:34.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:33 smithi069 ceph-mon[32746]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:33 smithi083 ceph-mon[37031]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.468 DEBUG:teuthology.orchestra.run.smithi077:osd.4> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.4.service 2024-08-02T17:40:34.470 INFO:tasks.cephadm:Deploying osd.5 on smithi083 with /dev/vg_nvme/lv_3... 2024-08-02T17:40:34.470 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-02T17:40:34.729 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:40:34.928 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:34.928 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.928 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:34 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:34.928 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:34 smithi077 ceph-mon[37108]: from='osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-02T17:40:34.928 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:34 smithi077 ceph-mon[37108]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-02T17:40:35.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:35.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:35.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:34 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:35.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:34 smithi083 ceph-mon[37031]: from='osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-02T17:40:35.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:34 smithi083 ceph-mon[37031]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-02T17:40:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:34 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:34 smithi069 ceph-mon[32746]: from='osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-02T17:40:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:34 smithi069 ceph-mon[32746]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-08-02T17:40:35.873 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:35 smithi077 ceph-mon[37108]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:35.873 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:35 smithi077 ceph-mon[37108]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-02T17:40:35.873 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:35 smithi077 ceph-mon[37108]: from='osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:40:35.873 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:35 smithi077 ceph-mon[37108]: osdmap e30: 5 total, 4 up, 5 in 2024-08-02T17:40:35.873 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:35 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:35.873 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:35 smithi077 ceph-mon[37108]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:40:35.873 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 17:40:35 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4[48295]: 2024-08-02T17:40:35.823+0000 7f4e5aee9640 -1 osd.4 0 waiting for initial osdmap 2024-08-02T17:40:35.874 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 17:40:35 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4[48295]: 2024-08-02T17:40:35.830+0000 7f4e56d11640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:40:36.074 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:35 smithi083 ceph-mon[37031]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:36.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:35 smithi083 ceph-mon[37031]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-02T17:40:36.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:35 smithi083 ceph-mon[37031]: from='osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:40:36.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:35 smithi083 ceph-mon[37031]: osdmap e30: 5 total, 4 up, 5 in 2024-08-02T17:40:36.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:35 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:36.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:35 smithi083 ceph-mon[37031]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:40:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:35 smithi069 ceph-mon[32746]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:35 smithi069 ceph-mon[32746]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-02T17:40:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:35 smithi069 ceph-mon[32746]: from='osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:40:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:35 smithi069 ceph-mon[32746]: osdmap e30: 5 total, 4 up, 5 in 2024-08-02T17:40:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:35 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:36.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:35 smithi069 ceph-mon[32746]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:40:37.092 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:36 smithi077 ceph-mon[37108]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:40:37.092 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:36 smithi077 ceph-mon[37108]: osdmap e31: 5 total, 4 up, 5 in 2024-08-02T17:40:37.092 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:36 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.092 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:36 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.092 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:36 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:36 smithi069 ceph-mon[32746]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:40:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:36 smithi069 ceph-mon[32746]: osdmap e31: 5 total, 4 up, 5 in 2024-08-02T17:40:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:36 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:36 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:36 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.213 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:36 smithi083 ceph-mon[37031]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:40:37.214 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:36 smithi083 ceph-mon[37031]: osdmap e31: 5 total, 4 up, 5 in 2024-08-02T17:40:37.214 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:36 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.214 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:36 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.214 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:36 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.217 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:40:37.243 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_3 2024-08-02T17:40:37.495 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:40:37.871 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:40:37.871 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:40:37.871 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:37.871 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] boot 2024-08-02T17:40:37.871 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: osdmap e32: 5 total, 5 up, 5 in 2024-08-02T17:40:37.871 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:37.871 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:37.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:37.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:37.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:37.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:37.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:37.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:37.872 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:37.984 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 -- 172.21.15.83:0/1714846329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c102430 msgr2=0x7fc94c104820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 --2- 172.21.15.83:0/1714846329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c102430 0x7fc94c104820 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc93c009900 tx=0x7fc93c02a410 comp rx=0 tx=0).stop 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 -- 172.21.15.83:0/1714846329 shutdown_connections 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 --2- 172.21.15.83:0/1714846329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc94c107690 0x7fc94c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 --2- 172.21.15.83:0/1714846329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc94c104d60 0x7fc94c107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 --2- 172.21.15.83:0/1714846329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c102430 0x7fc94c104820 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 -- 172.21.15.83:0/1714846329 >> 172.21.15.83:0/1714846329 conn(0x7fc94c0fbfd0 msgr2=0x7fc94c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 -- 172.21.15.83:0/1714846329 shutdown_connections 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.947+0000 7fc9537bc640 1 -- 172.21.15.83:0/1714846329 wait complete. 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc9537bc640 1 Processor -- start 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc9537bc640 1 -- start start 2024-08-02T17:40:37.985 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc9537bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc94c102430 0x7fc94c197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc9537bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c104d60 0x7fc94c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc9537bc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc94c107690 0x7fc94c19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:37.986 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc9537bc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c10d670 con 0x7fc94c102430 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc9537bc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c10d4f0 con 0x7fc94c107690 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc9537bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc94c10d7f0 con 0x7fc94c104d60 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc950d57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c104d60 0x7fc94c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc950d57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c104d60 0x7fc94c198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:42074/0 (socket says 172.21.15.83:42074) 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc950d57640 1 -- 172.21.15.83:0/774254710 learned_addr learned my addr 172.21.15.83:0/774254710 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc951558640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc94c102430 0x7fc94c197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc951d59640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc94c107690 0x7fc94c19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc950d57640 1 -- 172.21.15.83:0/774254710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc94c107690 msgr2=0x7fc94c19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc950d57640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc94c107690 0x7fc94c19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:37.987 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc950d57640 1 -- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc94c102430 msgr2=0x7fc94c197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc950d57640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc94c102430 0x7fc94c197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.948+0000 7fc950d57640 1 -- 172.21.15.83:0/774254710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc94c19f930 con 0x7fc94c104d60 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.949+0000 7fc950d57640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c104d60 0x7fc94c198250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc94800d740 tx=0x7fc94800dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.949+0000 7fc9427fc640 1 -- 172.21.15.83:0/774254710 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948017070 con 0x7fc94c104d60 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.949+0000 7fc9427fc640 1 -- 172.21.15.83:0/774254710 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc94800de50 con 0x7fc94c104d60 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.949+0000 7fc9427fc640 1 -- 172.21.15.83:0/774254710 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948004e00 con 0x7fc94c104d60 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.949+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc94c19fc20 con 0x7fc94c104d60 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.949+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc94c1a0110 con 0x7fc94c104d60 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.950+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94c0fdac0 con 0x7fc94c104d60 2024-08-02T17:40:37.988 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.952+0000 7fc9427fc640 1 -- 172.21.15.83:0/774254710 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc948004fa0 con 0x7fc94c104d60 2024-08-02T17:40:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.952+0000 7fc9427fc640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9380a2b80 0x7fc9380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.952+0000 7fc9427fc640 1 -- 172.21.15.83:0/774254710 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fc9480cdd00 con 0x7fc94c104d60 2024-08-02T17:40:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.952+0000 7fc951558640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9380a2b80 0x7fc9380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.953+0000 7fc951558640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9380a2b80 0x7fc9380a5040 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc94c1991f0 tx=0x7fc93c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:40:37.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:37.956+0000 7fc9427fc640 1 -- 172.21.15.83:0/774254710 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc94809b680 con 0x7fc94c104d60 2024-08-02T17:40:38.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] boot 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: osdmap e32: 5 total, 5 up, 5 in 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:38.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:38.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:38.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:40:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:40:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-08-02T17:40:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: osd.4 [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] boot 2024-08-02T17:40:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: osdmap e32: 5 total, 5 up, 5 in 2024-08-02T17:40:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:40:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:38.132 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:38.131+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc94c062360 con 0x7fc9380a2b80 2024-08-02T17:40:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:38 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi077 2024-08-02T17:40:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:38 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi077 to 8544M 2024-08-02T17:40:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:38 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:40:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:38 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:40:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:38 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:38 smithi077 ceph-mon[37108]: osdmap e33: 5 total, 5 up, 5 in 2024-08-02T17:40:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:38 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi077 2024-08-02T17:40:39.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:38 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi077 to 8544M 2024-08-02T17:40:39.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:38 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:40:39.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:38 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:40:39.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:38 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:39.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:38 smithi069 ceph-mon[32746]: osdmap e33: 5 total, 5 up, 5 in 2024-08-02T17:40:39.198 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:38 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi077 2024-08-02T17:40:39.198 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:38 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi077 to 8544M 2024-08-02T17:40:39.198 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:38 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:40:39.198 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:38 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:40:39.198 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:38 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:39.199 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:38 smithi083 ceph-mon[37031]: osdmap e33: 5 total, 5 up, 5 in 2024-08-02T17:40:40.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:39 smithi083 ceph-mon[37031]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:40:40.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:39 smithi083 ceph-mon[37031]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:40.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:39 smithi077 ceph-mon[37108]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:40:40.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:39 smithi077 ceph-mon[37108]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:39 smithi069 ceph-mon[32746]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:40:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:39 smithi069 ceph-mon[32746]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:40.987 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:40 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/1571048566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]: dispatch 2024-08-02T17:40:40.987 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:40 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]: dispatch 2024-08-02T17:40:40.988 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:40 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]': finished 2024-08-02T17:40:40.988 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:40 smithi083 ceph-mon[37031]: osdmap e34: 6 total, 5 up, 6 in 2024-08-02T17:40:40.988 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:41.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:40 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/1571048566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]: dispatch 2024-08-02T17:40:41.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:40 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]: dispatch 2024-08-02T17:40:41.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:40 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]': finished 2024-08-02T17:40:41.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:40 smithi077 ceph-mon[37108]: osdmap e34: 6 total, 5 up, 6 in 2024-08-02T17:40:41.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:40 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/1571048566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]: dispatch 2024-08-02T17:40:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:40 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]: dispatch 2024-08-02T17:40:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:40 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3"}]': finished 2024-08-02T17:40:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:40 smithi069 ceph-mon[32746]: osdmap e34: 6 total, 5 up, 6 in 2024-08-02T17:40:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:42.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:41 smithi077 ceph-mon[37108]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:42.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:41 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/1724296079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:40:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:41 smithi069 ceph-mon[32746]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:41 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/1724296079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:40:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:41 smithi083 ceph-mon[37031]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:41 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/1724296079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:40:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:43 smithi069 ceph-mon[32746]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:43 smithi083 ceph-mon[37031]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:44.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:43 smithi077 ceph-mon[37108]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:46.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:45 smithi083 ceph-mon[37031]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:46.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:45 smithi069 ceph-mon[32746]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:45 smithi077 ceph-mon[37108]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:47 smithi069 ceph-mon[32746]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-02T17:40:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:48.116 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:47 smithi083 ceph-mon[37031]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:48.116 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-02T17:40:48.117 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:48.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:47 smithi077 ceph-mon[37108]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-08-02T17:40:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:49.004 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:48 smithi083 ceph-mon[37031]: Deploying daemon osd.5 on smithi083 2024-08-02T17:40:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:48 smithi077 ceph-mon[37108]: Deploying daemon osd.5 on smithi083 2024-08-02T17:40:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:48 smithi069 ceph-mon[32746]: Deploying daemon osd.5 on smithi083 2024-08-02T17:40:50.258 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:49 smithi083 ceph-mon[37031]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:50.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:49 smithi077 ceph-mon[37108]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:50.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:49 smithi069 ceph-mon[32746]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:52.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:51 smithi083 ceph-mon[37031]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:52.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:51 smithi077 ceph-mon[37108]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:51 smithi069 ceph-mon[32746]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:53.141 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:53.142 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:53.142 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:53.142 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:52 smithi083 ceph-mon[37031]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:52 smithi077 ceph-mon[37108]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:52 smithi069 ceph-mon[32746]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:55.343 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.341+0000 7fc9427fc640 1 -- 172.21.15.83:0/774254710 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc94c062360 con 0x7fc9380a2b80 2024-08-02T17:40:55.344 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 5 on host 'smithi083' 2024-08-02T17:40:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9380a2b80 msgr2=0x7fc9380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9380a2b80 0x7fc9380a5040 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc94c1991f0 tx=0x7fc93c000bc0 comp rx=0 tx=0).stop 2024-08-02T17:40:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c104d60 msgr2=0x7fc94c198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c104d60 0x7fc94c198250 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc94800d740 tx=0x7fc94800dc00 comp rx=0 tx=0).stop 2024-08-02T17:40:55.345 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 shutdown_connections 2024-08-02T17:40:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9380a2b80 0x7fc9380a5040 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc94c107690 0x7fc94c19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc94c104d60 0x7fc94c198250 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 --2- 172.21.15.83:0/774254710 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc94c102430 0x7fc94c197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 >> 172.21.15.83:0/774254710 conn(0x7fc94c0fbfd0 msgr2=0x7fc94c102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:40:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 shutdown_connections 2024-08-02T17:40:55.346 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:40:55.344+0000 7fc9537bc640 1 -- 172.21.15.83:0/774254710 wait complete. 2024-08-02T17:40:56.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:55 smithi083 ceph-mon[37031]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:56.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:56.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:56.061 DEBUG:teuthology.orchestra.run.smithi083:osd.5> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.5.service 2024-08-02T17:40:56.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:55 smithi069 ceph-mon[32746]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:56.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:56.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:56.064 INFO:tasks.cephadm:Deploying osd.6 on smithi069 with /dev/vg_nvme/lv_2... 2024-08-02T17:40:56.065 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-02T17:40:56.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:55 smithi077 ceph-mon[37108]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:56.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:56.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:56.329 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:40:56.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 17:40:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T17:40:56.418+0000 7facc8f9d640 -1 osd.5 0 log_to_monitors true 2024-08-02T17:40:57.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:56 smithi083 ceph-mon[37031]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-02T17:40:57.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:57.075 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:56 smithi077 ceph-mon[37108]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-02T17:40:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:56 smithi069 ceph-mon[32746]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-08-02T17:40:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:58.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:57 smithi077 ceph-mon[37108]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:57 smithi077 ceph-mon[37108]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-02T17:40:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:57 smithi077 ceph-mon[37108]: osdmap e35: 6 total, 5 up, 6 in 2024-08-02T17:40:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:57 smithi077 ceph-mon[37108]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:40:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:58.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:57 smithi083 ceph-mon[37031]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:58.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:57 smithi083 ceph-mon[37031]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-02T17:40:58.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:57 smithi083 ceph-mon[37031]: osdmap e35: 6 total, 5 up, 6 in 2024-08-02T17:40:58.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:57 smithi083 ceph-mon[37031]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:40:58.102 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:58.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:57 smithi069 ceph-mon[32746]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:58.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:57 smithi069 ceph-mon[32746]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-02T17:40:58.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:57 smithi069 ceph-mon[32746]: osdmap e35: 6 total, 5 up, 6 in 2024-08-02T17:40:58.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:57 smithi069 ceph-mon[32746]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:40:58.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:58.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T17:40:58.458+0000 7facc5734640 -1 osd.5 0 waiting for initial osdmap 2024-08-02T17:40:58.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T17:40:58.464+0000 7facbcd40640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:40:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:58 smithi069 ceph-mon[32746]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:40:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:58 smithi069 ceph-mon[32746]: osdmap e36: 6 total, 5 up, 6 in 2024-08-02T17:40:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:58 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:58 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:58 smithi069 ceph-mon[32746]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' 2024-08-02T17:40:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:58 smithi069 ceph-mon[32746]: osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] boot 2024-08-02T17:40:58.953 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:58 smithi069 ceph-mon[32746]: osdmap e37: 6 total, 6 up, 6 in 2024-08-02T17:40:58.954 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:58 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:58.956 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:40:58.983 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_2 2024-08-02T17:40:59.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:58 smithi077 ceph-mon[37108]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:40:59.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:58 smithi077 ceph-mon[37108]: osdmap e36: 6 total, 5 up, 6 in 2024-08-02T17:40:59.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:58 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:59.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:58 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:59.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:58 smithi077 ceph-mon[37108]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' 2024-08-02T17:40:59.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:58 smithi077 ceph-mon[37108]: osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] boot 2024-08-02T17:40:59.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:58 smithi077 ceph-mon[37108]: osdmap e37: 6 total, 6 up, 6 in 2024-08-02T17:40:59.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:58 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:59.239 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:40:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-mon[37031]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:40:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-mon[37031]: osdmap e36: 6 total, 5 up, 6 in 2024-08-02T17:40:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-mon[37031]: from='osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836]' entity='osd.5' 2024-08-02T17:40:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-mon[37031]: osd.5 [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] boot 2024-08-02T17:40:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-mon[37031]: osdmap e37: 6 total, 6 up, 6 in 2024-08-02T17:40:59.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:58 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:40:59.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.724+0000 7f4b6935e640 1 -- 172.21.15.69:0/144082754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b64104cc0 msgr2=0x7f4b641070b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.724+0000 7f4b6935e640 1 --2- 172.21.15.69:0/144082754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b64104cc0 0x7f4b641070b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009900 tx=0x7f4b5802a410 comp rx=0 tx=0).stop 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 -- 172.21.15.69:0/144082754 shutdown_connections 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 --2- 172.21.15.69:0/144082754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b64104cc0 0x7f4b641070b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 --2- 172.21.15.69:0/144082754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b641075f0 0x7f4b64109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 --2- 172.21.15.69:0/144082754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b64102390 0x7f4b64104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 -- 172.21.15.69:0/144082754 >> 172.21.15.69:0/144082754 conn(0x7f4b640fbfb0 msgr2=0x7f4b640fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 -- 172.21.15.69:0/144082754 shutdown_connections 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 -- 172.21.15.69:0/144082754 wait complete. 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 Processor -- start 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 -- start start 2024-08-02T17:40:59.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b64102390 0x7f4b64197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b64104cc0 0x7f4b641981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b641075f0 0x7f4b6419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.725+0000 7f4b6935e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6410d590 con 0x7f4b641075f0 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b6935e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6410d410 con 0x7f4b64104cc0 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b6935e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b6410d710 con 0x7f4b64102390 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b641075f0 0x7f4b6419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b641075f0 0x7f4b6419f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42594/0 (socket says 172.21.15.69:42594) 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 -- 172.21.15.69:0/2679430540 learned_addr learned my addr 172.21.15.69:0/2679430540 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b627fc640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b64104cc0 0x7f4b641981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:59.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b62ffd640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b64102390 0x7f4b64197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:59.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 -- 172.21.15.69:0/2679430540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b64102390 msgr2=0x7f4b64197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:59.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b64102390 0x7f4b64197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:59.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 -- 172.21.15.69:0/2679430540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b64104cc0 msgr2=0x7f4b641981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:40:59.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b64104cc0 0x7f4b641981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:40:59.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 -- 172.21.15.69:0/2679430540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b6419f880 con 0x7f4b641075f0 2024-08-02T17:40:59.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b62ffd640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b64102390 0x7f4b64197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:40:59.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b637fe640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b641075f0 0x7f4b6419f1e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00d750 tx=0x7f4b5c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b4bfff640 1 -- 172.21.15.69:0/2679430540 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c017070 con 0x7f4b641075f0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b6419fb70 con 0x7f4b641075f0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b4bfff640 1 -- 172.21.15.69:0/2679430540 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b5c005500 con 0x7f4b641075f0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b4bfff640 1 -- 172.21.15.69:0/2679430540 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c004d90 con 0x7f4b641075f0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.726+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b641a0100 con 0x7f4b641075f0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.727+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b640fdaa0 con 0x7f4b641075f0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.729+0000 7f4b4bfff640 1 -- 172.21.15.69:0/2679430540 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4b5c004f30 con 0x7f4b641075f0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.729+0000 7f4b4bfff640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4b4c0a2b80 0x7f4b4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.730+0000 7f4b62ffd640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4b4c0a2b80 0x7f4b4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.730+0000 7f4b4bfff640 1 -- 172.21.15.69:0/2679430540 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4562+0+0 (secure 0 0 0) 0x7f4b5c0cdf60 con 0x7f4b641075f0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.730+0000 7f4b62ffd640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4b4c0a2b80 0x7f4b4c0a5040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4b64199140 tx=0x7f4b54009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:40:59.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.735+0000 7f4b4bfff640 1 -- 172.21.15.69:0/2679430540 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4b5c09bcc0 con 0x7f4b641075f0 2024-08-02T17:40:59.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:40:59.918+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4b64062360 con 0x7f4b4c0a2b80 2024-08-02T17:40:59.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:40:59.920 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: osdmap e38: 6 total, 6 up, 6 in 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:40:59.921 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:40:59.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:40:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:41:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:41:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:41:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: osdmap e38: 6 total, 6 up, 6 in 2024-08-02T17:41:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:00.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:40:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: osdmap e38: 6 total, 6 up, 6 in 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:00.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:00.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:00.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:00.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:40:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:00.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:00 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi083 2024-08-02T17:41:00.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:00 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi083 to 8544M 2024-08-02T17:41:00.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:00.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:00.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:00.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:00 smithi069 ceph-mon[32746]: osdmap e39: 6 total, 6 up, 6 in 2024-08-02T17:41:01.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:00 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi083 2024-08-02T17:41:01.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:00 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi083 to 8544M 2024-08-02T17:41:01.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:01.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:01.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:01.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:00 smithi077 ceph-mon[37108]: osdmap e39: 6 total, 6 up, 6 in 2024-08-02T17:41:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:00 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi083 2024-08-02T17:41:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:00 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi083 to 8544M 2024-08-02T17:41:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:00 smithi083 ceph-mon[37031]: osdmap e39: 6 total, 6 up, 6 in 2024-08-02T17:41:01.982 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:01 smithi069 ceph-mon[32746]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:01.982 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:01 smithi069 ceph-mon[32746]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:01.982 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:01 smithi069 ceph-mon[32746]: osdmap e40: 6 total, 6 up, 6 in 2024-08-02T17:41:02.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:01 smithi077 ceph-mon[37108]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:02.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:01 smithi077 ceph-mon[37108]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:02.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:01 smithi077 ceph-mon[37108]: osdmap e40: 6 total, 6 up, 6 in 2024-08-02T17:41:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:01 smithi083 ceph-mon[37031]: from='client.14343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:01 smithi083 ceph-mon[37031]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:01 smithi083 ceph-mon[37031]: osdmap e40: 6 total, 6 up, 6 in 2024-08-02T17:41:03.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:03 smithi077 ceph-mon[37108]: osdmap e41: 6 total, 6 up, 6 in 2024-08-02T17:41:03.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:03 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2429357736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f58da4-cae1-4c9d-968f-4ab620039158"}]: dispatch 2024-08-02T17:41:03.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:03 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2429357736' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6f58da4-cae1-4c9d-968f-4ab620039158"}]': finished 2024-08-02T17:41:03.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:03 smithi077 ceph-mon[37108]: osdmap e42: 7 total, 6 up, 7 in 2024-08-02T17:41:03.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:03.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:03 smithi077 ceph-mon[37108]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:03.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:03 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2017301128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:03.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:03 smithi069 ceph-mon[32746]: osdmap e41: 6 total, 6 up, 6 in 2024-08-02T17:41:03.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:03 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2429357736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f58da4-cae1-4c9d-968f-4ab620039158"}]: dispatch 2024-08-02T17:41:03.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:03 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2429357736' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6f58da4-cae1-4c9d-968f-4ab620039158"}]': finished 2024-08-02T17:41:03.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:03 smithi069 ceph-mon[32746]: osdmap e42: 7 total, 6 up, 7 in 2024-08-02T17:41:03.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:03.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:03 smithi069 ceph-mon[32746]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:03.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:03 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2017301128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:03.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:03 smithi083 ceph-mon[37031]: osdmap e41: 6 total, 6 up, 6 in 2024-08-02T17:41:03.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:03 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2429357736' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e6f58da4-cae1-4c9d-968f-4ab620039158"}]: dispatch 2024-08-02T17:41:03.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:03 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2429357736' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e6f58da4-cae1-4c9d-968f-4ab620039158"}]': finished 2024-08-02T17:41:03.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:03 smithi083 ceph-mon[37031]: osdmap e42: 7 total, 6 up, 7 in 2024-08-02T17:41:03.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:03.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:03 smithi083 ceph-mon[37031]: pgmap v125: 1 pgs: 1 remapped+peering; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:03.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:03 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2017301128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:06.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:05 smithi077 ceph-mon[37108]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-08-02T17:41:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:05 smithi069 ceph-mon[32746]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-08-02T17:41:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:05 smithi083 ceph-mon[37031]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-08-02T17:41:08.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:07 smithi083 ceph-mon[37031]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:41:08.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:07 smithi077 ceph-mon[37108]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:41:08.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:07 smithi069 ceph-mon[32746]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:41:10.068 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:09 smithi069 ceph-mon[32746]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-02T17:41:10.069 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:09 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-02T17:41:10.069 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:09 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:10.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:09 smithi077 ceph-mon[37108]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-02T17:41:10.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:09 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-02T17:41:10.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:09 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:09 smithi083 ceph-mon[37031]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-08-02T17:41:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:09 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-08-02T17:41:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:09 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:10.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:10 smithi069 ceph-mon[32746]: Deploying daemon osd.6 on smithi069 2024-08-02T17:41:11.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:10 smithi077 ceph-mon[37108]: Deploying daemon osd.6 on smithi069 2024-08-02T17:41:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:10 smithi083 ceph-mon[37031]: Deploying daemon osd.6 on smithi069 2024-08-02T17:41:12.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:11 smithi077 ceph-mon[37108]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-02T17:41:12.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:11 smithi069 ceph-mon[32746]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-02T17:41:12.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:11 smithi083 ceph-mon[37031]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-08-02T17:41:14.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:13 smithi077 ceph-mon[37108]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-02T17:41:14.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:13 smithi069 ceph-mon[32746]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-02T17:41:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:13 smithi083 ceph-mon[37031]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-02T17:41:15.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:15.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:15.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:15.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:15.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:15.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:16.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:15 smithi077 ceph-mon[37108]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-02T17:41:16.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:15 smithi069 ceph-mon[32746]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-02T17:41:16.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:15 smithi083 ceph-mon[37031]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-02T17:41:17.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:17 smithi069 ceph-mon[32746]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:17.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:17.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:18.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:17 smithi077 ceph-mon[37108]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:18.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:17 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:18.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:17 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:18.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:17 smithi083 ceph-mon[37031]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:18.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:17 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:18.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:17 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:18.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.300+0000 7f4b4bfff640 1 -- 172.21.15.69:0/2679430540 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4b64062360 con 0x7f4b4c0a2b80 2024-08-02T17:41:18.302 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 6 on host 'smithi069' 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4b4c0a2b80 msgr2=0x7f4b4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4b4c0a2b80 0x7f4b4c0a5040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4b64199140 tx=0x7f4b54009600 comp rx=0 tx=0).stop 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b641075f0 msgr2=0x7f4b6419f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b641075f0 0x7f4b6419f1e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00d750 tx=0x7f4b5c00dc10 comp rx=0 tx=0).stop 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 shutdown_connections 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4b4c0a2b80 0x7f4b4c0a5040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b641075f0 0x7f4b6419f1e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b64104cc0 0x7f4b641981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:18.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 --2- 172.21.15.69:0/2679430540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b64102390 0x7f4b64197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:18.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 >> 172.21.15.69:0/2679430540 conn(0x7f4b640fbfb0 msgr2=0x7f4b64102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:41:18.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 shutdown_connections 2024-08-02T17:41:18.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:41:18.302+0000 7f4b6935e640 1 -- 172.21.15.69:0/2679430540 wait complete. 2024-08-02T17:41:18.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:18.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:18.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:18.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:18.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:18.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:18 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:18.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:18 smithi069 ceph-mon[32746]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-02T17:41:19.015 DEBUG:teuthology.orchestra.run.smithi069:osd.6> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.6.service 2024-08-02T17:41:19.018 INFO:tasks.cephadm:Deploying osd.7 on smithi077 with /dev/vg_nvme/lv_2... 2024-08-02T17:41:19.018 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-02T17:41:19.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:18 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:18 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:18 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:18 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:18 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:18 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:18 smithi077 ceph-mon[37108]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-02T17:41:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:18 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:18 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:18 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:18 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:18 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:18 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:18 smithi083 ceph-mon[37031]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-08-02T17:41:19.281 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T17:41:20.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:19 smithi077 ceph-mon[37108]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:20.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:19 smithi077 ceph-mon[37108]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-02T17:41:20.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:19 smithi077 ceph-mon[37108]: osdmap e43: 7 total, 6 up, 7 in 2024-08-02T17:41:20.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:19 smithi077 ceph-mon[37108]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:41:20.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:19 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:20.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:19 smithi069 ceph-mon[32746]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:20.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:19 smithi069 ceph-mon[32746]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-02T17:41:20.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:19 smithi069 ceph-mon[32746]: osdmap e43: 7 total, 6 up, 7 in 2024-08-02T17:41:20.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:19 smithi069 ceph-mon[32746]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:41:20.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:19 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:20.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:19 smithi083 ceph-mon[37031]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:20.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:19 smithi083 ceph-mon[37031]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-02T17:41:20.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:19 smithi083 ceph-mon[37031]: osdmap e43: 7 total, 6 up, 7 in 2024-08-02T17:41:20.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:19 smithi083 ceph-mon[37031]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:41:20.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:19 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:20.445 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 17:41:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T17:41:20.309+0000 7f919ac86640 -1 osd.6 0 waiting for initial osdmap 2024-08-02T17:41:20.445 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 17:41:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T17:41:20.316+0000 7f9196aae640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:41:21.523 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:21 smithi069 ceph-mon[32746]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:41:21.523 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:21 smithi069 ceph-mon[32746]: osdmap e44: 7 total, 6 up, 7 in 2024-08-02T17:41:21.523 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:21 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:21.523 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:21 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:21.524 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:21 smithi069 ceph-mon[32746]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:21.567 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:21 smithi077 ceph-mon[37108]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:41:21.567 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:21 smithi077 ceph-mon[37108]: osdmap e44: 7 total, 6 up, 7 in 2024-08-02T17:41:21.567 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:21 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:21.567 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:21 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:21.567 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:21 smithi077 ceph-mon[37108]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:21.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:21 smithi083 ceph-mon[37031]: from='osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:41:21.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:21 smithi083 ceph-mon[37031]: osdmap e44: 7 total, 6 up, 7 in 2024-08-02T17:41:21.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:21 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:21.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:21 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:21.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:21 smithi083 ceph-mon[37031]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-08-02T17:41:21.773 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:41:21.800 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_2 2024-08-02T17:41:22.058 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T17:41:22.481 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:41:22.481 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] boot 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: osdmap e45: 7 total, 7 up, 7 in 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi069 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi069 to 5695M 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.482 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:22.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:22.483 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 -- 172.21.15.77:0/2875874554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8102400 msgr2=0x7f13a8108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:22.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 --2- 172.21.15.77:0/2875874554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8102400 0x7f13a8108c90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009ca0 tx=0x7f13a002a8c0 comp rx=0 tx=0).stop 2024-08-02T17:41:22.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 -- 172.21.15.77:0/2875874554 shutdown_connections 2024-08-02T17:41:22.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 --2- 172.21.15.77:0/2875874554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8102400 0x7f13a8108c90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:22.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 --2- 172.21.15.77:0/2875874554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a8101a40 0x7f13a8101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:22.566 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 --2- 172.21.15.77:0/2875874554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a8100840 0x7f13a8100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 -- 172.21.15.77:0/2875874554 >> 172.21.15.77:0/2875874554 conn(0x7f13a80fbfd0 msgr2=0x7f13a80fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 -- 172.21.15.77:0/2875874554 shutdown_connections 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.562+0000 7f13ace5e640 1 -- 172.21.15.77:0/2875874554 wait complete. 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13ace5e640 1 Processor -- start 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13ace5e640 1 -- start start 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13ace5e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a8100840 0x7f13a806b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13ace5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8101a40 0x7f13a819cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13ace5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a8102400 0x7f13a819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13ace5e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13a810c5c0 con 0x7f13a8100840 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13ace5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13a810c440 con 0x7f13a8101a40 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13ace5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13a810c740 con 0x7f13a8102400 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f139ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8101a40 0x7f13a819cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:41:22.567 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f139ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8101a40 0x7f13a819cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:48540/0 (socket says 172.21.15.77:48540) 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f139ffff640 1 -- 172.21.15.77:0/3508102772 learned_addr learned my addr 172.21.15.77:0/3508102772 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13a659c640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a8100840 0x7f13a806b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f13a6d9d640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a8102400 0x7f13a819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f139ffff640 1 -- 172.21.15.77:0/3508102772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a8102400 msgr2=0x7f13a819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f139ffff640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a8102400 0x7f13a819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f139ffff640 1 -- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a8100840 msgr2=0x7f13a806b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f139ffff640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a8100840 0x7f13a806b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.563+0000 7f139ffff640 1 -- 172.21.15.77:0/3508102772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a819fa10 con 0x7f13a8101a40 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.564+0000 7f139ffff640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8101a40 0x7f13a819cae0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f139000b4a0 tx=0x7f139000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.564+0000 7f139f7fe640 1 -- 172.21.15.77:0/3508102772 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1390015020 con 0x7f13a8101a40 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.564+0000 7f139f7fe640 1 -- 172.21.15.77:0/3508102772 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1390011d10 con 0x7f13a8101a40 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.564+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a819fca0 con 0x7f13a8101a40 2024-08-02T17:41:22.568 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.564+0000 7f139f7fe640 1 -- 172.21.15.77:0/3508102772 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139000f2e0 con 0x7f13a8101a40 2024-08-02T17:41:22.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.564+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a81a0210 con 0x7f13a8101a40 2024-08-02T17:41:22.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.565+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a8107c60 con 0x7f13a8101a40 2024-08-02T17:41:22.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.566+0000 7f139f7fe640 1 -- 172.21.15.77:0/3508102772 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1390018020 con 0x7f13a8101a40 2024-08-02T17:41:22.569 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.566+0000 7f139f7fe640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13940a2b80 0x7f13940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:41:22.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.566+0000 7f139f7fe640 1 -- 172.21.15.77:0/3508102772 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4965+0+0 (secure 0 0 0) 0x7f13900cdc20 con 0x7f13a8101a40 2024-08-02T17:41:22.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.567+0000 7f13a659c640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13940a2b80 0x7f13940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:41:22.570 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.567+0000 7f13a659c640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13940a2b80 0x7f13940a5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f13a81018a0 tx=0x7f1398009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:41:22.572 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.571+0000 7f139f7fe640 1 -- 172.21.15.77:0/3508102772 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f139001f020 con 0x7f13a8101a40 2024-08-02T17:41:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:41:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:41:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] boot 2024-08-02T17:41:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: osdmap e45: 7 total, 7 up, 7 in 2024-08-02T17:41:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi069 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi069 to 5695M 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:22.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.747 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:22.746+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f13a8062360 con 0x7f13940a2b80 2024-08-02T17:41:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:41:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:41:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: osd.6 [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] boot 2024-08-02T17:41:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: osdmap e45: 7 total, 7 up, 7 in 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi069 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi069 to 5695M 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:22.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:23.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:23 smithi077 ceph-mon[37108]: osdmap e46: 7 total, 7 up, 7 in 2024-08-02T17:41:23.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:23 smithi077 ceph-mon[37108]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:23.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:23 smithi077 ceph-mon[37108]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:23.484 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:23 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:23.485 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:23 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:23.485 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:23 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:23.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:23 smithi069 ceph-mon[32746]: osdmap e46: 7 total, 7 up, 7 in 2024-08-02T17:41:23.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:23 smithi069 ceph-mon[32746]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:23.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:23 smithi069 ceph-mon[32746]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:23.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:23.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:23.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:23 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:23.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:23 smithi083 ceph-mon[37031]: osdmap e46: 7 total, 7 up, 7 in 2024-08-02T17:41:23.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:23 smithi083 ceph-mon[37031]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 209 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:23.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:23 smithi083 ceph-mon[37031]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:23.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:23 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:23.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:23 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:23.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:23 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:25 smithi077 ceph-mon[37108]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:25 smithi077 ceph-mon[37108]: from='client.? 172.21.15.77:0/959078770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]: dispatch 2024-08-02T17:41:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:25 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]: dispatch 2024-08-02T17:41:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:25 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]': finished 2024-08-02T17:41:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:25 smithi077 ceph-mon[37108]: osdmap e47: 8 total, 7 up, 8 in 2024-08-02T17:41:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:25 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:25 smithi077 ceph-mon[37108]: from='client.? 172.21.15.77:0/2156138277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:25 smithi069 ceph-mon[32746]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:25 smithi069 ceph-mon[32746]: from='client.? 172.21.15.77:0/959078770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]: dispatch 2024-08-02T17:41:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:25 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]: dispatch 2024-08-02T17:41:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:25 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]': finished 2024-08-02T17:41:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:25 smithi069 ceph-mon[32746]: osdmap e47: 8 total, 7 up, 8 in 2024-08-02T17:41:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:25 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:25 smithi069 ceph-mon[32746]: from='client.? 172.21.15.77:0/2156138277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:25 smithi083 ceph-mon[37031]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:25 smithi083 ceph-mon[37031]: from='client.? 172.21.15.77:0/959078770' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]: dispatch 2024-08-02T17:41:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:25 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]: dispatch 2024-08-02T17:41:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:25 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "392b17a2-c105-464b-beaa-4f531883e292"}]': finished 2024-08-02T17:41:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:25 smithi083 ceph-mon[37031]: osdmap e47: 8 total, 7 up, 8 in 2024-08-02T17:41:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:25 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:25 smithi083 ceph-mon[37031]: from='client.? 172.21.15.77:0/2156138277' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:28.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:27 smithi077 ceph-mon[37108]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:27 smithi069 ceph-mon[32746]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:27 smithi083 ceph-mon[37031]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:30.058 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:29 smithi077 ceph-mon[37108]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:29 smithi069 ceph-mon[32746]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:29 smithi083 ceph-mon[37031]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:31.986 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:31 smithi077 ceph-mon[37108]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:32.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:31 smithi069 ceph-mon[32746]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:31 smithi083 ceph-mon[37031]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:32 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-02T17:41:33.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:32 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:32 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-02T17:41:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:32 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:32 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-08-02T17:41:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:32 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:33.960 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:33 smithi077 ceph-mon[37108]: Deploying daemon osd.7 on smithi077 2024-08-02T17:41:33.960 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:33 smithi077 ceph-mon[37108]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:33 smithi069 ceph-mon[32746]: Deploying daemon osd.7 on smithi077 2024-08-02T17:41:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:33 smithi069 ceph-mon[32746]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:33 smithi083 ceph-mon[37031]: Deploying daemon osd.7 on smithi077 2024-08-02T17:41:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:33 smithi083 ceph-mon[37031]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:35.948 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:35 smithi077 ceph-mon[37108]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:36.073 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:35 smithi083 ceph-mon[37031]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:35 smithi069 ceph-mon[32746]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:38.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:37 smithi077 ceph-mon[37108]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:37 smithi069 ceph-mon[32746]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:37 smithi083 ceph-mon[37031]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:39.981 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:39 smithi077 ceph-mon[37108]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:39 smithi069 ceph-mon[32746]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:39 smithi083 ceph-mon[37031]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:40.663 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.661+0000 7f139f7fe640 1 -- 172.21.15.77:0/3508102772 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f13a8062360 con 0x7f13940a2b80 2024-08-02T17:41:40.663 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 7 on host 'smithi077' 2024-08-02T17:41:40.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13940a2b80 msgr2=0x7f13940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:40.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13940a2b80 0x7f13940a5040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f13a81018a0 tx=0x7f1398009110 comp rx=0 tx=0).stop 2024-08-02T17:41:40.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8101a40 msgr2=0x7f13a819cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:40.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8101a40 0x7f13a819cae0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f139000b4a0 tx=0x7f139000b8a0 comp rx=0 tx=0).stop 2024-08-02T17:41:40.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 shutdown_connections 2024-08-02T17:41:40.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13940a2b80 0x7f13940a5040 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:40.665 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a8102400 0x7f13a819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:40.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a8101a40 0x7f13a819cae0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:40.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 --2- 172.21.15.77:0/3508102772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a8100840 0x7f13a806b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:40.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 >> 172.21.15.77:0/3508102772 conn(0x7f13a80fbfd0 msgr2=0x7f13a8104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:41:40.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 shutdown_connections 2024-08-02T17:41:40.666 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:41:40.663+0000 7f13ace5e640 1 -- 172.21.15.77:0/3508102772 wait complete. 2024-08-02T17:41:41.212 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.213 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.213 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:41.213 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:41.213 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.213 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:41.213 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.213 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.213 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:40 smithi077 ceph-mon[37108]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:40 smithi083 ceph-mon[37031]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:41.276 DEBUG:teuthology.orchestra.run.smithi077:osd.7> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.7.service 2024-08-02T17:41:41.278 INFO:tasks.cephadm:Deploying osd.8 on smithi083 with /dev/vg_nvme/lv_2... 2024-08-02T17:41:41.279 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-02T17:41:41.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:41.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:41.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:41:41.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:41.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:40 smithi069 ceph-mon[32746]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:41.537 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:41:42.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:41 smithi077 ceph-mon[37108]: from='osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-02T17:41:42.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:41 smithi077 ceph-mon[37108]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-02T17:41:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:41 smithi083 ceph-mon[37031]: from='osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-02T17:41:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:41 smithi083 ceph-mon[37031]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-02T17:41:42.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:41 smithi069 ceph-mon[32746]: from='osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-02T17:41:42.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:41 smithi069 ceph-mon[32746]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-08-02T17:41:43.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:42 smithi077 ceph-mon[37108]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-02T17:41:43.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:42 smithi077 ceph-mon[37108]: from='osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:41:43.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:42 smithi077 ceph-mon[37108]: osdmap e48: 8 total, 7 up, 8 in 2024-08-02T17:41:43.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:42 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:43.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:42 smithi077 ceph-mon[37108]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:41:43.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:42 smithi077 ceph-mon[37108]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:43.190 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:42 smithi083 ceph-mon[37031]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-02T17:41:43.190 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:42 smithi083 ceph-mon[37031]: from='osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:41:43.190 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:42 smithi083 ceph-mon[37031]: osdmap e48: 8 total, 7 up, 8 in 2024-08-02T17:41:43.190 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:42 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:43.191 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:42 smithi083 ceph-mon[37031]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:41:43.191 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:42 smithi083 ceph-mon[37031]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:43.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:42 smithi069 ceph-mon[32746]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-02T17:41:43.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:42 smithi069 ceph-mon[32746]: from='osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:41:43.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:42 smithi069 ceph-mon[32746]: osdmap e48: 8 total, 7 up, 8 in 2024-08-02T17:41:43.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:42 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:43.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:42 smithi069 ceph-mon[32746]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:41:43.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:42 smithi069 ceph-mon[32746]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-08-02T17:41:43.848 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T17:41:43.692+0000 7f2d81c12640 -1 osd.7 0 waiting for initial osdmap 2024-08-02T17:41:43.848 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T17:41:43.704+0000 7f2d79a32640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:41:44.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:43 smithi083 ceph-mon[37031]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:41:44.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:43 smithi083 ceph-mon[37031]: osdmap e49: 8 total, 7 up, 8 in 2024-08-02T17:41:44.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:43 smithi083 ceph-mon[37031]: from='osd.7 ' entity='osd.7' 2024-08-02T17:41:44.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:43 smithi083 ceph-mon[37031]: osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] boot 2024-08-02T17:41:44.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:43 smithi083 ceph-mon[37031]: osdmap e50: 8 total, 8 up, 8 in 2024-08-02T17:41:44.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.064 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:41:44.091 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_2 2024-08-02T17:41:44.216 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-mon[37108]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:41:44.216 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-mon[37108]: osdmap e49: 8 total, 7 up, 8 in 2024-08-02T17:41:44.216 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.216 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.216 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-mon[37108]: from='osd.7 ' entity='osd.7' 2024-08-02T17:41:44.216 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-mon[37108]: osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] boot 2024-08-02T17:41:44.216 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-mon[37108]: osdmap e50: 8 total, 8 up, 8 in 2024-08-02T17:41:44.217 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.342 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:41:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:43 smithi069 ceph-mon[32746]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:41:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:43 smithi069 ceph-mon[32746]: osdmap e49: 8 total, 7 up, 8 in 2024-08-02T17:41:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:43 smithi069 ceph-mon[32746]: from='osd.7 ' entity='osd.7' 2024-08-02T17:41:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:43 smithi069 ceph-mon[32746]: osd.7 [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] boot 2024-08-02T17:41:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:43 smithi069 ceph-mon[32746]: osdmap e50: 8 total, 8 up, 8 in 2024-08-02T17:41:44.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:41:44.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2746102539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0102b30 msgr2=0x7fb1b0102fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:44.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2746102539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0102b30 0x7fb1b0102fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009900 tx=0x7fb1a402a410 comp rx=0 tx=0).stop 2024-08-02T17:41:44.840 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2746102539 shutdown_connections 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2746102539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1b01034f0 0x7fb1b0109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2746102539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0102b30 0x7fb1b0102fb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2746102539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1b0101930 0x7fb1b0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2746102539 >> 172.21.15.83:0/2746102539 conn(0x7fb1b00fd0c0 msgr2=0x7fb1b00ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2746102539 shutdown_connections 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.811+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2746102539 wait complete. 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b64ac640 1 Processor -- start 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b64ac640 1 -- start start 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b64ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0101930 0x7fb1b0197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b64ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1b0102b30 0x7fb1b01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b64ac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1b01034f0 0x7fb1b019f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:41:44.841 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b64ac640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b010d5b0 con 0x7fb1b01034f0 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b64ac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b010d430 con 0x7fb1b0102b30 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b64ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b010d730 con 0x7fb1b0101930 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0101930 0x7fb1b0197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0101930 0x7fb1b0197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:43436/0 (socket says 172.21.15.83:43436) 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 -- 172.21.15.83:0/2637242788 learned_addr learned my addr 172.21.15.83:0/2637242788 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1af7fe640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1b0102b30 0x7fb1b01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b4a49640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1b01034f0 0x7fb1b019f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 -- 172.21.15.83:0/2637242788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1b0102b30 msgr2=0x7fb1b01981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1b0102b30 0x7fb1b01981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 -- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1b01034f0 msgr2=0x7fb1b019f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1b01034f0 0x7fb1b019f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:41:44.842 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 -- 172.21.15.83:0/2637242788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b019f880 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1b4a49640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1b01034f0 0x7fb1b019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.812+0000 7fb1affff640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0101930 0x7fb1b0197c60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d950 tx=0x7fb1a000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.813+0000 7fb1ad7fa640 1 -- 172.21.15.83:0/2637242788 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0017070 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.813+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b019fb70 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.813+0000 7fb1ad7fa640 1 -- 172.21.15.83:0/2637242788 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1a0009c30 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.813+0000 7fb1ad7fa640 1 -- 172.21.15.83:0/2637242788 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000f600 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.813+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b01a00b0 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.814+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b0108d20 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.815+0000 7fb1ad7fa640 1 -- 172.21.15.83:0/2637242788 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb1a001a070 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.815+0000 7fb1ad7fa640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb1980a2b80 0x7fb1980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.815+0000 7fb1ad7fa640 1 -- 172.21.15.83:0/2637242788 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5426+0+0 (secure 0 0 0) 0x7fb1a00cdfe0 con 0x7fb1b0101930 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.816+0000 7fb1af7fe640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb1980a2b80 0x7fb1980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:41:44.843 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.816+0000 7fb1af7fe640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb1980a2b80 0x7fb1980a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40314e0 tx=0x7fb1a4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:41:44.844 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.819+0000 7fb1ad7fa640 1 -- 172.21.15.83:0/2637242788 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb1a0009de0 con 0x7fb1b0101930 2024-08-02T17:41:44.999 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:41:44.996+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb1b0062360 con 0x7fb1980a2b80 2024-08-02T17:41:44.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:41:44.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:41:44.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi077 2024-08-02T17:41:44.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:44.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:44.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:44.999 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:45.000 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:45.000 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi077 to 5696M 2024-08-02T17:41:45.000 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.000 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:45.000 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:45.000 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.000 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: osdmap e51: 8 total, 8 up, 8 in 2024-08-02T17:41:45.000 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:44 smithi083 ceph-mon[37031]: pgmap v155: 1 pgs: 1 unknown; 577 KiB data, 340 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi077 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi077 to 5696M 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:45.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:45.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: osdmap e51: 8 total, 8 up, 8 in 2024-08-02T17:41:45.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:44 smithi077 ceph-mon[37108]: pgmap v155: 1 pgs: 1 unknown; 577 KiB data, 340 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:41:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:41:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi077 2024-08-02T17:41:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi077 to 5696M 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: osdmap e51: 8 total, 8 up, 8 in 2024-08-02T17:41:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:44 smithi069 ceph-mon[32746]: pgmap v155: 1 pgs: 1 unknown; 577 KiB data, 340 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:46.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:45 smithi083 ceph-mon[37031]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:46.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:46.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:46.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:46.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:45 smithi083 ceph-mon[37031]: osdmap e52: 8 total, 8 up, 8 in 2024-08-02T17:41:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:45 smithi077 ceph-mon[37108]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:46.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:46.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:46.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:46.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:45 smithi077 ceph-mon[37108]: osdmap e52: 8 total, 8 up, 8 in 2024-08-02T17:41:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:45 smithi069 ceph-mon[32746]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:41:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:41:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:41:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:45 smithi069 ceph-mon[32746]: osdmap e52: 8 total, 8 up, 8 in 2024-08-02T17:41:47.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:47 smithi083 ceph-mon[37031]: osdmap e53: 8 total, 8 up, 8 in 2024-08-02T17:41:47.580 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:47 smithi083 ceph-mon[37031]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 340 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:47.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:47 smithi077 ceph-mon[37108]: osdmap e53: 8 total, 8 up, 8 in 2024-08-02T17:41:47.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:47 smithi077 ceph-mon[37108]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 340 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:47.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:47 smithi069 ceph-mon[32746]: osdmap e53: 8 total, 8 up, 8 in 2024-08-02T17:41:47.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:47 smithi069 ceph-mon[32746]: pgmap v158: 1 pgs: 1 unknown; 577 KiB data, 340 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:48.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:48 smithi077 ceph-mon[37108]: osdmap e54: 8 total, 8 up, 8 in 2024-08-02T17:41:48.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:48 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/4025634220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]: dispatch 2024-08-02T17:41:48.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:48 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]: dispatch 2024-08-02T17:41:48.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:48 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]': finished 2024-08-02T17:41:48.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:48 smithi077 ceph-mon[37108]: osdmap e55: 9 total, 8 up, 9 in 2024-08-02T17:41:48.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:41:48.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:48 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/2729340730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:48.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:48 smithi069 ceph-mon[32746]: osdmap e54: 8 total, 8 up, 8 in 2024-08-02T17:41:48.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:48 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/4025634220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]: dispatch 2024-08-02T17:41:48.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:48 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]: dispatch 2024-08-02T17:41:48.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:48 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]': finished 2024-08-02T17:41:48.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:48 smithi069 ceph-mon[32746]: osdmap e55: 9 total, 8 up, 9 in 2024-08-02T17:41:48.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:41:48.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:48 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/2729340730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:48.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:48 smithi083 ceph-mon[37031]: osdmap e54: 8 total, 8 up, 8 in 2024-08-02T17:41:48.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:48 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/4025634220' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]: dispatch 2024-08-02T17:41:48.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:48 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]: dispatch 2024-08-02T17:41:48.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:48 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "92a6e799-6b88-4d10-bc05-8a3b5c43c8be"}]': finished 2024-08-02T17:41:48.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:48 smithi083 ceph-mon[37031]: osdmap e55: 9 total, 8 up, 9 in 2024-08-02T17:41:48.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:41:48.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:48 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/2729340730' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:41:49.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:49 smithi077 ceph-mon[37108]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 341 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:49.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:49 smithi069 ceph-mon[32746]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 341 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:49.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:49 smithi083 ceph-mon[37031]: pgmap v161: 1 pgs: 1 unknown; 577 KiB data, 341 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:51.991 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:51 smithi083 ceph-mon[37031]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:52.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:51 smithi077 ceph-mon[37108]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:52.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:51 smithi069 ceph-mon[32746]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:54.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:53 smithi083 ceph-mon[37031]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:54.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:53 smithi077 ceph-mon[37108]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:54.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:53 smithi069 ceph-mon[32746]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:55.061 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-02T17:41:55.062 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:54 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:55.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-02T17:41:55.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:54 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:55.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-08-02T17:41:55.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:54 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:41:56.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:55 smithi083 ceph-mon[37031]: Deploying daemon osd.8 on smithi083 2024-08-02T17:41:56.051 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:55 smithi083 ceph-mon[37031]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:56.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:55 smithi077 ceph-mon[37108]: Deploying daemon osd.8 on smithi083 2024-08-02T17:41:56.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:55 smithi077 ceph-mon[37108]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:56.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:55 smithi069 ceph-mon[32746]: Deploying daemon osd.8 on smithi083 2024-08-02T17:41:56.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:55 smithi069 ceph-mon[32746]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:57.851 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:57 smithi083 ceph-mon[37031]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:58.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:57 smithi077 ceph-mon[37108]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:41:58.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:57 smithi069 ceph-mon[32746]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:00.071 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:59 smithi083 ceph-mon[37031]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:00.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:00.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:00.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:41:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:00.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:59 smithi077 ceph-mon[37108]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:41:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:59 smithi069 ceph-mon[32746]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:41:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:01.968 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:01 smithi083 ceph-mon[37031]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:02.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:01 smithi077 ceph-mon[37108]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:01 smithi069 ceph-mon[32746]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:02.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.887+0000 7fb1ad7fa640 1 -- 172.21.15.83:0/2637242788 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb1b0062360 con 0x7fb1980a2b80 2024-08-02T17:42:02.889 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 8 on host 'smithi083' 2024-08-02T17:42:02.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb1980a2b80 msgr2=0x7fb1980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb1980a2b80 0x7fb1980a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40314e0 tx=0x7fb1a4000bc0 comp rx=0 tx=0).stop 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0101930 msgr2=0x7fb1b0197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0101930 0x7fb1b0197c60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb1a000d950 tx=0x7fb1a000de10 comp rx=0 tx=0).stop 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 shutdown_connections 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb1980a2b80 0x7fb1980a5040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1b01034f0 0x7fb1b019f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1b0102b30 0x7fb1b01981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 --2- 172.21.15.83:0/2637242788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb1b0101930 0x7fb1b0197c60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 >> 172.21.15.83:0/2637242788 conn(0x7fb1b00fd0c0 msgr2=0x7fb1b0105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 shutdown_connections 2024-08-02T17:42:02.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:02.889+0000 7fb1b64ac640 1 -- 172.21.15.83:0/2637242788 wait complete. 2024-08-02T17:42:03.536 DEBUG:teuthology.orchestra.run.smithi083:osd.8> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.8.service 2024-08-02T17:42:03.539 INFO:tasks.cephadm:Deploying osd.9 on smithi069 with /dev/vg_nvme/lv_1... 2024-08-02T17:42:03.539 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-02T17:42:03.811 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:42:03.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:03 smithi069 ceph-mon[32746]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:03.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:03.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:03.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:03 smithi069 ceph-mon[32746]: from='osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-02T17:42:03.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:03 smithi069 ceph-mon[32746]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-02T17:42:03.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:03.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:04.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:03 smithi077 ceph-mon[37108]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:03 smithi077 ceph-mon[37108]: from='osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-02T17:42:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:03 smithi077 ceph-mon[37108]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-02T17:42:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:04.115 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:03 smithi083 ceph-mon[37031]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:04.115 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:04.116 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:04.116 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:03 smithi083 ceph-mon[37031]: from='osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-02T17:42:04.116 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:03 smithi083 ceph-mon[37031]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-08-02T17:42:04.116 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:04.116 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:05.160 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:04 smithi083 ceph-mon[37031]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-02T17:42:05.161 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:04 smithi083 ceph-mon[37031]: osdmap e56: 9 total, 8 up, 9 in 2024-08-02T17:42:05.161 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:04 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:05.161 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:04 smithi083 ceph-mon[37031]: from='osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:42:05.161 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:04 smithi083 ceph-mon[37031]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:42:05.161 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:04 smithi083 ceph-mon[37031]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:05.203 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:04 smithi069 ceph-mon[32746]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-02T17:42:05.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:04 smithi069 ceph-mon[32746]: osdmap e56: 9 total, 8 up, 9 in 2024-08-02T17:42:05.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:04 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:05.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:04 smithi069 ceph-mon[32746]: from='osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:42:05.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:04 smithi069 ceph-mon[32746]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:42:05.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:04 smithi069 ceph-mon[32746]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:05.272 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:04 smithi077 ceph-mon[37108]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-02T17:42:05.272 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:04 smithi077 ceph-mon[37108]: osdmap e56: 9 total, 8 up, 9 in 2024-08-02T17:42:05.272 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:04 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:05.272 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:04 smithi077 ceph-mon[37108]: from='osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:42:05.272 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:04 smithi077 ceph-mon[37108]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:42:05.272 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:04 smithi077 ceph-mon[37108]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-08-02T17:42:05.772 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 17:42:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T17:42:05.585+0000 7f8ce838d640 -1 osd.8 0 waiting for initial osdmap 2024-08-02T17:42:05.772 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 17:42:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T17:42:05.596+0000 7f8ce01ad640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:42:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:05 smithi069 ceph-mon[32746]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:42:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:05 smithi069 ceph-mon[32746]: osdmap e57: 9 total, 8 up, 9 in 2024-08-02T17:42:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:05 smithi069 ceph-mon[32746]: from='osd.8 ' entity='osd.8' 2024-08-02T17:42:06.162 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:05 smithi083 ceph-mon[37031]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:42:06.162 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:05 smithi083 ceph-mon[37031]: osdmap e57: 9 total, 8 up, 9 in 2024-08-02T17:42:06.163 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:06.163 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:06.163 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:05 smithi083 ceph-mon[37031]: from='osd.8 ' entity='osd.8' 2024-08-02T17:42:06.204 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:05 smithi077 ceph-mon[37108]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:42:06.204 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:05 smithi077 ceph-mon[37108]: osdmap e57: 9 total, 8 up, 9 in 2024-08-02T17:42:06.204 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:06.204 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:06.204 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:05 smithi077 ceph-mon[37108]: from='osd.8 ' entity='osd.8' 2024-08-02T17:42:06.608 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:42:06.635 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi069:vg_nvme/lv_1 2024-08-02T17:42:06.903 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] boot 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: osdmap e58: 9 total, 9 up, 9 in 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi083 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.160 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi083 to 5696M 2024-08-02T17:42:07.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:07.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:07.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:06 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:42:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:42:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] boot 2024-08-02T17:42:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: osdmap e58: 9 total, 9 up, 9 in 2024-08-02T17:42:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi083 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi083 to 5696M 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:07.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:06 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:42:07.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:42:07.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: osd.8 [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] boot 2024-08-02T17:42:07.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: osdmap e58: 9 total, 9 up, 9 in 2024-08-02T17:42:07.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:42:07.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:07.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi083 2024-08-02T17:42:07.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi083 to 5696M 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:07.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:06 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.360+0000 7f5c9f968640 1 -- 172.21.15.69:0/3644972390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 msgr2=0x7f5c981047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.360+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3644972390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 0x7f5c981047e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009900 tx=0x7f5c9402a410 comp rx=0 tx=0).stop 2024-08-02T17:42:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 -- 172.21.15.69:0/3644972390 shutdown_connections 2024-08-02T17:42:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3644972390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c98107650 0x7f5c98109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3644972390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c98104d20 0x7f5c98107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3644972390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 0x7f5c981047e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 -- 172.21.15.69:0/3644972390 >> 172.21.15.69:0/3644972390 conn(0x7f5c980fbfb0 msgr2=0x7f5c980fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:42:07.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 -- 172.21.15.69:0/3644972390 shutdown_connections 2024-08-02T17:42:07.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 -- 172.21.15.69:0/3644972390 wait complete. 2024-08-02T17:42:07.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 Processor -- start 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 -- start start 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 0x7f5c98197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c98104d20 0x7f5c981981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c98107650 0x7f5c9819f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.361+0000 7f5c9f968640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9810d590 con 0x7f5c981023f0 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9f968640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9810d410 con 0x7f5c98107650 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9f968640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9810d710 con 0x7f5c98104d20 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 0x7f5c98197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 0x7f5c98197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39016/0 (socket says 172.21.15.69:39016) 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 -- 172.21.15.69:0/3970081057 learned_addr learned my addr 172.21.15.69:0/3970081057 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:42:07.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9cf03640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c98104d20 0x7f5c981981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9df05640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c98107650 0x7f5c9819f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 -- 172.21.15.69:0/3970081057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c98104d20 msgr2=0x7f5c981981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c98104d20 0x7f5c981981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 -- 172.21.15.69:0/3970081057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c98107650 msgr2=0x7f5c9819f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c98107650 0x7f5c9819f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 -- 172.21.15.69:0/3970081057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9819f880 con 0x7f5c981023f0 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9cf03640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c98104d20 0x7f5c981981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9d704640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 0x7f5c98197c60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009230 tx=0x7f5c94031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c8e7fc640 1 -- 172.21.15.69:0/3970081057 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9403f030 con 0x7f5c981023f0 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c981001e0 con 0x7f5c981023f0 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c8e7fc640 1 -- 172.21.15.69:0/3970081057 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c94003ad0 con 0x7f5c981023f0 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c8e7fc640 1 -- 172.21.15.69:0/3970081057 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c94035070 con 0x7f5c981023f0 2024-08-02T17:42:07.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.362+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c98100710 con 0x7f5c981023f0 2024-08-02T17:42:07.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.363+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c980fdaa0 con 0x7f5c981023f0 2024-08-02T17:42:07.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.364+0000 7f5c8e7fc640 1 -- 172.21.15.69:0/3970081057 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5c94042030 con 0x7f5c981023f0 2024-08-02T17:42:07.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.365+0000 7f5c8e7fc640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5c840a2b80 0x7f5c840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:07.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.365+0000 7f5c9cf03640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5c840a2b80 0x7f5c840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:07.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.365+0000 7f5c8e7fc640 1 -- 172.21.15.69:0/3970081057 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5858+0+0 (secure 0 0 0) 0x7f5c940eeb90 con 0x7f5c981023f0 2024-08-02T17:42:07.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.366+0000 7f5c9cf03640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5c840a2b80 0x7f5c840a5040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5c9806a7a0 tx=0x7f5c88005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:42:07.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.370+0000 7f5c8e7fc640 1 -- 172.21.15.69:0/3970081057 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5c940bbe30 con 0x7f5c981023f0 2024-08-02T17:42:07.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:07.552+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5c98062360 con 0x7f5c840a2b80 2024-08-02T17:42:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: osdmap e59: 9 total, 9 up, 9 in 2024-08-02T17:42:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:08.259 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:08.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:08.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:08.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:08.260 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: osdmap e59: 9 total, 9 up, 9 in 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:08.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: osdmap e59: 9 total, 9 up, 9 in 2024-08-02T17:42:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: from='client.14397 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi069:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:09.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:08 smithi069 ceph-mon[32746]: osdmap e60: 9 total, 9 up, 9 in 2024-08-02T17:42:09.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:08 smithi069 ceph-mon[32746]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:08 smithi083 ceph-mon[37031]: osdmap e60: 9 total, 9 up, 9 in 2024-08-02T17:42:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:08 smithi083 ceph-mon[37031]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:08 smithi077 ceph-mon[37108]: osdmap e60: 9 total, 9 up, 9 in 2024-08-02T17:42:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:08 smithi077 ceph-mon[37108]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:09 smithi069 ceph-mon[32746]: osdmap e61: 9 total, 9 up, 9 in 2024-08-02T17:42:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:09 smithi083 ceph-mon[37031]: osdmap e61: 9 total, 9 up, 9 in 2024-08-02T17:42:10.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:09 smithi077 ceph-mon[37108]: osdmap e61: 9 total, 9 up, 9 in 2024-08-02T17:42:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:10 smithi069 ceph-mon[32746]: osdmap e62: 9 total, 9 up, 9 in 2024-08-02T17:42:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:10 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3112309522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ffe6b5e-7950-4db4-8dc5-6d0351b0a346"}]: dispatch 2024-08-02T17:42:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:10 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3112309522' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ffe6b5e-7950-4db4-8dc5-6d0351b0a346"}]': finished 2024-08-02T17:42:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:10 smithi069 ceph-mon[32746]: osdmap e63: 10 total, 9 up, 10 in 2024-08-02T17:42:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:10 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4002648160' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:10 smithi069 ceph-mon[32746]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-08-02T17:42:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:10 smithi083 ceph-mon[37031]: osdmap e62: 9 total, 9 up, 9 in 2024-08-02T17:42:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:10 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3112309522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ffe6b5e-7950-4db4-8dc5-6d0351b0a346"}]: dispatch 2024-08-02T17:42:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:10 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3112309522' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ffe6b5e-7950-4db4-8dc5-6d0351b0a346"}]': finished 2024-08-02T17:42:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:10 smithi083 ceph-mon[37031]: osdmap e63: 10 total, 9 up, 10 in 2024-08-02T17:42:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:11.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:10 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4002648160' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:11.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:10 smithi083 ceph-mon[37031]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-08-02T17:42:11.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:10 smithi077 ceph-mon[37108]: osdmap e62: 9 total, 9 up, 9 in 2024-08-02T17:42:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:10 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3112309522' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ffe6b5e-7950-4db4-8dc5-6d0351b0a346"}]: dispatch 2024-08-02T17:42:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:10 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3112309522' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ffe6b5e-7950-4db4-8dc5-6d0351b0a346"}]': finished 2024-08-02T17:42:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:10 smithi077 ceph-mon[37108]: osdmap e63: 10 total, 9 up, 10 in 2024-08-02T17:42:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:10 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4002648160' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:10 smithi077 ceph-mon[37108]: pgmap v180: 1 pgs: 1 peering; 577 KiB data, 676 MiB used, 804 GiB / 805 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-08-02T17:42:14.068 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:13 smithi069 ceph-mon[32746]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-02T17:42:14.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:13 smithi077 ceph-mon[37108]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-02T17:42:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:13 smithi083 ceph-mon[37031]: pgmap v181: 1 pgs: 1 peering; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-02T17:42:15.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:15 smithi069 ceph-mon[32746]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:42:16.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:15 smithi077 ceph-mon[37108]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:42:16.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:15 smithi083 ceph-mon[37031]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:42:18.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:17 smithi077 ceph-mon[37108]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-02T17:42:18.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:17 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-02T17:42:18.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:17 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:18.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:17 smithi069 ceph-mon[32746]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-02T17:42:18.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-02T17:42:18.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:17 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:18.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:17 smithi083 ceph-mon[37031]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-08-02T17:42:18.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:17 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-08-02T17:42:18.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:17 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:18.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:18 smithi069 ceph-mon[32746]: Deploying daemon osd.9 on smithi069 2024-08-02T17:42:19.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:18 smithi077 ceph-mon[37108]: Deploying daemon osd.9 on smithi069 2024-08-02T17:42:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:18 smithi083 ceph-mon[37031]: Deploying daemon osd.9 on smithi069 2024-08-02T17:42:20.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:19 smithi069 ceph-mon[32746]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:20.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:19 smithi077 ceph-mon[37108]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:20.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:19 smithi083 ceph-mon[37031]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:21.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:21 smithi069 ceph-mon[32746]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:22.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:21 smithi077 ceph-mon[37108]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:22.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:21 smithi083 ceph-mon[37031]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:23.061 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:23.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:23.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:22 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:23.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:23.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:22 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:23.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:23.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:23.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:22 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:24.020 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:23 smithi069 ceph-mon[32746]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:24.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:23 smithi077 ceph-mon[37108]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:24.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:23 smithi083 ceph-mon[37031]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:25.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.533+0000 7f5c8e7fc640 1 -- 172.21.15.69:0/3970081057 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5c98062360 con 0x7f5c840a2b80 2024-08-02T17:42:25.535 INFO:teuthology.orchestra.run.smithi069.stdout:Created osd(s) 9 on host 'smithi069' 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5c840a2b80 msgr2=0x7f5c840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5c840a2b80 0x7f5c840a5040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5c9806a7a0 tx=0x7f5c88005f90 comp rx=0 tx=0).stop 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 msgr2=0x7f5c98197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 0x7f5c98197c60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009230 tx=0x7f5c94031b60 comp rx=0 tx=0).stop 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 shutdown_connections 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5c840a2b80 0x7f5c840a5040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5c98107650 0x7f5c9819f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5c98104d20 0x7f5c981981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 --2- 172.21.15.69:0/3970081057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5c981023f0 0x7f5c98197c60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 >> 172.21.15.69:0/3970081057 conn(0x7f5c980fbfb0 msgr2=0x7f5c98102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:42:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 shutdown_connections 2024-08-02T17:42:25.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:42:25.535+0000 7f5c9f968640 1 -- 172.21.15.69:0/3970081057 wait complete. 2024-08-02T17:42:26.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:25 smithi077 ceph-mon[37108]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:26.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:25 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:25 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:25 smithi069 ceph-mon[32746]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:25 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:25 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:26.180 DEBUG:teuthology.orchestra.run.smithi069:osd.9> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.9.service 2024-08-02T17:42:26.183 INFO:tasks.cephadm:Deploying osd.10 on smithi077 with /dev/vg_nvme/lv_1... 2024-08-02T17:42:26.183 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-02T17:42:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:25 smithi083 ceph-mon[37031]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:25 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:25 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:26.442 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T17:42:26.817 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 17:42:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T17:42:26.535+0000 7f87a3bd5640 -1 osd.9 0 log_to_monitors true 2024-08-02T17:42:27.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:26 smithi077 ceph-mon[37108]: from='osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-02T17:42:27.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:26 smithi077 ceph-mon[37108]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-02T17:42:27.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:27.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:26 smithi069 ceph-mon[32746]: from='osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-02T17:42:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:26 smithi069 ceph-mon[32746]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-02T17:42:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:26 smithi083 ceph-mon[37031]: from='osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-02T17:42:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:26 smithi083 ceph-mon[37031]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-08-02T17:42:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:27.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:27 smithi077 ceph-mon[37108]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:27.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:27 smithi077 ceph-mon[37108]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-02T17:42:27.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:27 smithi077 ceph-mon[37108]: from='osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:42:27.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:27 smithi077 ceph-mon[37108]: osdmap e64: 10 total, 9 up, 10 in 2024-08-02T17:42:27.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:27 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:27.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:27 smithi077 ceph-mon[37108]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:42:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:27 smithi069 ceph-mon[32746]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:27 smithi069 ceph-mon[32746]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-02T17:42:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:27 smithi069 ceph-mon[32746]: from='osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:42:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:27 smithi069 ceph-mon[32746]: osdmap e64: 10 total, 9 up, 10 in 2024-08-02T17:42:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:27 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:28.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:27 smithi069 ceph-mon[32746]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:42:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:27 smithi083 ceph-mon[37031]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:27 smithi083 ceph-mon[37031]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-02T17:42:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:27 smithi083 ceph-mon[37031]: from='osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:42:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:27 smithi083 ceph-mon[37031]: osdmap e64: 10 total, 9 up, 10 in 2024-08-02T17:42:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:27 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:27 smithi083 ceph-mon[37031]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]: dispatch 2024-08-02T17:42:28.788 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 17:42:28 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T17:42:28.485+0000 7f879fb58640 -1 osd.9 0 waiting for initial osdmap 2024-08-02T17:42:28.788 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 17:42:28 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T17:42:28.497+0000 7f8797978640 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:42:28.991 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:28 smithi077 ceph-mon[37108]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:42:28.992 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:28 smithi077 ceph-mon[37108]: osdmap e65: 10 total, 9 up, 10 in 2024-08-02T17:42:28.992 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:28 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:28.992 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:28 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:28.992 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:28 smithi077 ceph-mon[37108]: from='osd.9 ' entity='osd.9' 2024-08-02T17:42:28.994 INFO:teuthology.orchestra.run.smithi077.stdout: 2024-08-02T17:42:29.023 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi077:vg_nvme/lv_1 2024-08-02T17:42:29.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:28 smithi069 ceph-mon[32746]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:42:29.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:28 smithi069 ceph-mon[32746]: osdmap e65: 10 total, 9 up, 10 in 2024-08-02T17:42:29.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:28 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:29.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:28 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:29.044 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:28 smithi069 ceph-mon[32746]: from='osd.9 ' entity='osd.9' 2024-08-02T17:42:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:28 smithi083 ceph-mon[37031]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi069", "root=default"]}]': finished 2024-08-02T17:42:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:28 smithi083 ceph-mon[37031]: osdmap e65: 10 total, 9 up, 10 in 2024-08-02T17:42:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:28 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:28 smithi083 ceph-mon[37031]: from='osd.9 ' entity='osd.9' 2024-08-02T17:42:29.273 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T17:42:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 -- 172.21.15.77:0/2289201204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec107630 msgr2=0x7f3cec109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/2289201204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec107630 0x7f3cec109a80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009ca0 tx=0x7f3ce402a8c0 comp rx=0 tx=0).stop 2024-08-02T17:42:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 -- 172.21.15.77:0/2289201204 shutdown_connections 2024-08-02T17:42:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/2289201204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cec104d00 0x7f3cec1070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/2289201204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec107630 0x7f3cec109a80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/2289201204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cec1023d0 0x7f3cec1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 -- 172.21.15.77:0/2289201204 >> 172.21.15.77:0/2289201204 conn(0x7f3cec0fbfd0 msgr2=0x7f3cec0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:42:29.737 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 -- 172.21.15.77:0/2289201204 shutdown_connections 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.726+0000 7f3cf2b85640 1 -- 172.21.15.77:0/2289201204 wait complete. 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 Processor -- start 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 -- start start 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec1023d0 0x7f3cec197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cec104d00 0x7f3cec1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cec107630 0x7f3cec19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d5d0 con 0x7f3cec104d00 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d450 con 0x7f3cec1023d0 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec10d750 con 0x7f3cec107630 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec1023d0 0x7f3cec197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec1023d0 0x7f3cec197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.77:54846/0 (socket says 172.21.15.77:54846) 2024-08-02T17:42:29.738 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 -- 172.21.15.77:0/3879359090 learned_addr learned my addr 172.21.15.77:0/3879359090 (peer_addr_for_me v2:172.21.15.77:0/0) 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cebfff640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cec104d00 0x7f3cec1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 -- 172.21.15.77:0/3879359090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cec107630 msgr2=0x7f3cec19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cec107630 0x7f3cec19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 -- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cec104d00 msgr2=0x7f3cec1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cec104d00 0x7f3cec1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 -- 172.21.15.77:0/3879359090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec19f7a0 con 0x7f3cec1023d0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf0921640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec1023d0 0x7f3cec197b70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000c920 tx=0x7f3ce000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3ce9ffb640 1 -- 172.21.15.77:0/3879359090 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0007c70 con 0x7f3cec1023d0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3ce9ffb640 1 -- 172.21.15.77:0/3879359090 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ce0007e10 con 0x7f3cec1023d0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.727+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec19fa90 con 0x7f3cec1023d0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.728+0000 7f3ce9ffb640 1 -- 172.21.15.77:0/3879359090 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce0018890 con 0x7f3cec1023d0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.728+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec19ffd0 con 0x7f3cec1023d0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.729+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cec0fdac0 con 0x7f3cec1023d0 2024-08-02T17:42:29.739 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.729+0000 7f3ce9ffb640 1 -- 172.21.15.77:0/3879359090 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3ce0018a30 con 0x7f3cec1023d0 2024-08-02T17:42:29.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.730+0000 7f3ce9ffb640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3cdc0a2bb0 0x7f3cdc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:29.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.730+0000 7f3ce9ffb640 1 -- 172.21.15.77:0/3879359090 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6261+0+0 (secure 0 0 0) 0x7f3ce00ce960 con 0x7f3cec1023d0 2024-08-02T17:42:29.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.730+0000 7f3cebfff640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3cdc0a2bb0 0x7f3cdc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:29.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.731+0000 7f3cebfff640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3cdc0a2bb0 0x7f3cdc0a5070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3cec199090 tx=0x7f3cd800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:42:29.740 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.734+0000 7f3ce9ffb640 1 -- 172.21.15.77:0/3879359090 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3ce0014030 con 0x7f3cec1023d0 2024-08-02T17:42:29.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:29 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:42:29.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:29 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:42:29.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:29 smithi069 ceph-mon[32746]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:29 smithi069 ceph-mon[32746]: osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] boot 2024-08-02T17:42:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:29 smithi069 ceph-mon[32746]: osdmap e66: 10 total, 10 up, 10 in 2024-08-02T17:42:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:29.912 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:29.911+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3cec062360 con 0x7f3cdc0a2bb0 2024-08-02T17:42:29.912 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:29 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:42:29.913 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:29 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:42:29.913 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:29 smithi077 ceph-mon[37108]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:29.913 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:29.913 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:29 smithi077 ceph-mon[37108]: osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] boot 2024-08-02T17:42:29.913 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:29 smithi077 ceph-mon[37108]: osdmap e66: 10 total, 10 up, 10 in 2024-08-02T17:42:29.913 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:29 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:42:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:29 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:42:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:29 smithi083 ceph-mon[37031]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-08-02T17:42:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:29 smithi083 ceph-mon[37031]: osd.9 [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] boot 2024-08-02T17:42:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:29 smithi083 ceph-mon[37031]: osdmap e66: 10 total, 10 up, 10 in 2024-08-02T17:42:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:42:30.995 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:30.996 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:30.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: osdmap e67: 10 total, 10 up, 10 in 2024-08-02T17:42:30.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:30.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:30.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:30.997 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:30 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: osdmap e67: 10 total, 10 up, 10 in 2024-08-02T17:42:31.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:31.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:31.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:31.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:30 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: osdmap e67: 10 total, 10 up, 10 in 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:31.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:30 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:32.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:31 smithi077 ceph-mon[37108]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:32.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:31 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi069 2024-08-02T17:42:32.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:31 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi069 to 4271M 2024-08-02T17:42:32.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:31 smithi077 ceph-mon[37108]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:32.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:31 smithi069 ceph-mon[32746]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:32.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:31 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi069 2024-08-02T17:42:32.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:31 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi069 to 4271M 2024-08-02T17:42:32.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:31 smithi069 ceph-mon[32746]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:31 smithi083 ceph-mon[37031]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi077:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:31 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi069 2024-08-02T17:42:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:31 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi069 to 4271M 2024-08-02T17:42:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:31 smithi083 ceph-mon[37031]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:32 smithi077 ceph-mon[37108]: from='client.? 172.21.15.77:0/3600087082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]: dispatch 2024-08-02T17:42:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:32 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]: dispatch 2024-08-02T17:42:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:32 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]': finished 2024-08-02T17:42:33.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:32 smithi077 ceph-mon[37108]: osdmap e68: 11 total, 10 up, 11 in 2024-08-02T17:42:33.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:32 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:32 smithi069 ceph-mon[32746]: from='client.? 172.21.15.77:0/3600087082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]: dispatch 2024-08-02T17:42:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:32 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]: dispatch 2024-08-02T17:42:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:32 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]': finished 2024-08-02T17:42:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:32 smithi069 ceph-mon[32746]: osdmap e68: 11 total, 10 up, 11 in 2024-08-02T17:42:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:32 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:32 smithi083 ceph-mon[37031]: from='client.? 172.21.15.77:0/3600087082' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]: dispatch 2024-08-02T17:42:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:32 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]: dispatch 2024-08-02T17:42:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:32 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88334ccf-216f-4d58-b9e1-6c173f4897ef"}]': finished 2024-08-02T17:42:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:32 smithi083 ceph-mon[37031]: osdmap e68: 11 total, 10 up, 11 in 2024-08-02T17:42:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:32 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:34.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:33 smithi077 ceph-mon[37108]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:34.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:33 smithi077 ceph-mon[37108]: from='client.? 172.21.15.77:0/3961664324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:33 smithi069 ceph-mon[32746]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:33 smithi069 ceph-mon[32746]: from='client.? 172.21.15.77:0/3961664324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:33 smithi083 ceph-mon[37031]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:33 smithi083 ceph-mon[37031]: from='client.? 172.21.15.77:0/3961664324' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:36.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:35 smithi077 ceph-mon[37108]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:35 smithi069 ceph-mon[32746]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:35 smithi083 ceph-mon[37031]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:38.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:37 smithi077 ceph-mon[37108]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:37 smithi069 ceph-mon[32746]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:37 smithi083 ceph-mon[37031]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:40.011 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:39 smithi077 ceph-mon[37108]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:40.011 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:39 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-02T17:42:40.011 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:39 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:39 smithi069 ceph-mon[32746]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:39 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-02T17:42:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:39 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:39 smithi083 ceph-mon[37031]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:39 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-08-02T17:42:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:39 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:41.024 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:40 smithi077 ceph-mon[37108]: Deploying daemon osd.10 on smithi077 2024-08-02T17:42:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:40 smithi069 ceph-mon[32746]: Deploying daemon osd.10 on smithi077 2024-08-02T17:42:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:40 smithi083 ceph-mon[37031]: Deploying daemon osd.10 on smithi077 2024-08-02T17:42:42.033 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:41 smithi077 ceph-mon[37108]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:41 smithi069 ceph-mon[32746]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:41 smithi083 ceph-mon[37031]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:43 smithi069 ceph-mon[32746]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:44.130 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:43 smithi077 ceph-mon[37108]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:43 smithi083 ceph-mon[37031]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:45.175 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:45.176 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:45.176 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:45.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:45.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:45.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:45.935 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:45 smithi077 ceph-mon[37108]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:46.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:45 smithi069 ceph-mon[32746]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:46.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:45 smithi083 ceph-mon[37031]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:47.967 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:47 smithi077 ceph-mon[37108]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:47.967 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:47.968 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:47 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:48.005 INFO:teuthology.orchestra.run.smithi077.stdout:Created osd(s) 10 on host 'smithi077' 2024-08-02T17:42:48.005 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.003+0000 7f3ce9ffb640 1 -- 172.21.15.77:0/3879359090 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3cec062360 con 0x7f3cdc0a2bb0 2024-08-02T17:42:48.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3cdc0a2bb0 msgr2=0x7f3cdc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:48.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3cdc0a2bb0 0x7f3cdc0a5070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3cec199090 tx=0x7f3cd800a2d0 comp rx=0 tx=0).stop 2024-08-02T17:42:48.007 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec1023d0 msgr2=0x7f3cec197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec1023d0 0x7f3cec197b70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000c920 tx=0x7f3ce000cde0 comp rx=0 tx=0).stop 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 shutdown_connections 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3cdc0a2bb0 0x7f3cdc0a5070 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3cec104d00 0x7f3cec1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cec107630 0x7f3cec19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 --2- 172.21.15.77:0/3879359090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3cec1023d0 0x7f3cec197b70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 >> 172.21.15.77:0/3879359090 conn(0x7f3cec0fbfd0 msgr2=0x7f3cec102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 shutdown_connections 2024-08-02T17:42:48.008 INFO:teuthology.orchestra.run.smithi077.stderr:2024-08-02T17:42:48.006+0000 7f3cf2b85640 1 -- 172.21.15.77:0/3879359090 wait complete. 2024-08-02T17:42:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:47 smithi069 ceph-mon[32746]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:47 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:47 smithi083 ceph-mon[37031]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:47 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:48.632 DEBUG:teuthology.orchestra.run.smithi077:osd.10> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.10.service 2024-08-02T17:42:48.635 INFO:tasks.cephadm:Deploying osd.11 on smithi083 with /dev/vg_nvme/lv_1... 2024-08-02T17:42:48.635 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-02T17:42:48.890 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:42:49.143 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:49.143 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:49.143 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:48 smithi077 ceph-mon[37108]: from='osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-02T17:42:49.143 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:48 smithi077 ceph-mon[37108]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-02T17:42:49.144 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:48 smithi077 ceph-mon[37108]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:49 smithi083 ceph-mon[37031]: from='osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-02T17:42:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:49 smithi083 ceph-mon[37031]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-02T17:42:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:49 smithi083 ceph-mon[37031]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:48 smithi069 ceph-mon[32746]: from='osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-02T17:42:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:48 smithi069 ceph-mon[32746]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-08-02T17:42:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:48 smithi069 ceph-mon[32746]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:50.222 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:50 smithi077 ceph-mon[37108]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-02T17:42:50.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:50 smithi077 ceph-mon[37108]: osdmap e69: 11 total, 10 up, 11 in 2024-08-02T17:42:50.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:50 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:50.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:50 smithi077 ceph-mon[37108]: from='osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:42:50.223 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:50 smithi077 ceph-mon[37108]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:42:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:50 smithi083 ceph-mon[37031]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-02T17:42:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:50 smithi083 ceph-mon[37031]: osdmap e69: 11 total, 10 up, 11 in 2024-08-02T17:42:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:50 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:50 smithi083 ceph-mon[37031]: from='osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:42:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:50 smithi083 ceph-mon[37031]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:42:50.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:50 smithi069 ceph-mon[32746]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-02T17:42:50.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:50 smithi069 ceph-mon[32746]: osdmap e69: 11 total, 10 up, 11 in 2024-08-02T17:42:50.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:50 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:50.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:50 smithi069 ceph-mon[32746]: from='osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:42:50.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:50 smithi069 ceph-mon[32746]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]: dispatch 2024-08-02T17:42:51.030 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 17:42:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T17:42:50.766+0000 7fda4463e640 -1 osd.10 0 waiting for initial osdmap 2024-08-02T17:42:51.030 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 17:42:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T17:42:50.779+0000 7fda3c45e640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:42:51.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:51 smithi083 ceph-mon[37031]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:42:51.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:51 smithi083 ceph-mon[37031]: osdmap e70: 11 total, 10 up, 11 in 2024-08-02T17:42:51.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:51.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:51.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:51 smithi083 ceph-mon[37031]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:51.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:51 smithi083 ceph-mon[37031]: from='osd.10 ' entity='osd.10' 2024-08-02T17:42:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:51 smithi077 ceph-mon[37108]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:42:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:51 smithi077 ceph-mon[37108]: osdmap e70: 11 total, 10 up, 11 in 2024-08-02T17:42:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:51.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:51.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:51 smithi077 ceph-mon[37108]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:51.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:51 smithi077 ceph-mon[37108]: from='osd.10 ' entity='osd.10' 2024-08-02T17:42:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:51 smithi069 ceph-mon[32746]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi077", "root=default"]}]': finished 2024-08-02T17:42:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:51 smithi069 ceph-mon[32746]: osdmap e70: 11 total, 10 up, 11 in 2024-08-02T17:42:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:51 smithi069 ceph-mon[32746]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-08-02T17:42:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:51 smithi069 ceph-mon[32746]: from='osd.10 ' entity='osd.10' 2024-08-02T17:42:51.447 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-08-02T17:42:51.474 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch daemon add osd smithi083:vg_nvme/lv_1 2024-08-02T17:42:51.737 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:42:52.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 -- 172.21.15.83:0/548413493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 msgr2=0x7fc584100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:52.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 --2- 172.21.15.83:0/548413493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 0x7fc584100c00 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc574009ca0 tx=0x7fc57402a8c0 comp rx=0 tx=0).stop 2024-08-02T17:42:52.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 -- 172.21.15.83:0/548413493 shutdown_connections 2024-08-02T17:42:52.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 --2- 172.21.15.83:0/548413493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc5841023c0 0x7fc584108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:52.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 --2- 172.21.15.83:0/548413493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc584101a00 0x7fc584101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:52.240 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 --2- 172.21.15.83:0/548413493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 0x7fc584100c00 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 -- 172.21.15.83:0/548413493 >> 172.21.15.83:0/548413493 conn(0x7fc5840fbfb0 msgr2=0x7fc5840fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 -- 172.21.15.83:0/548413493 shutdown_connections 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 -- 172.21.15.83:0/548413493 wait complete. 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.221+0000 7fc589a83640 1 Processor -- start 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc589a83640 1 -- start start 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc589a83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 0x7fc584104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc589a83640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc584101a00 0x7fc584102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc589a83640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5841023c0 0x7fc584103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc589a83640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc58410c560 con 0x7fc584101a00 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc589a83640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc58410c3e0 con 0x7fc5841023c0 2024-08-02T17:42:52.241 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc589a83640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc58410c6e0 con 0x7fc584100800 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 0x7fc584104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 0x7fc584104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:35402/0 (socket says 172.21.15.83:35402) 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 -- 172.21.15.83:0/3260557450 learned_addr learned my addr 172.21.15.83:0/3260557450 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc582ffd640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc584101a00 0x7fc584102de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc583fff640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5841023c0 0x7fc584103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 -- 172.21.15.83:0/3260557450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5841023c0 msgr2=0x7fc584103320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5841023c0 0x7fc584103320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 -- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc584101a00 msgr2=0x7fc584102de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc584101a00 0x7fc584102de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 -- 172.21.15.83:0/3260557450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc584103be0 con 0x7fc584100800 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc583fff640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5841023c0 0x7fc584103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:42:52.242 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.222+0000 7fc5837fe640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 0x7fc584104790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc574009430 tx=0x7fc57402f840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.223+0000 7fc580ff9640 1 -- 172.21.15.83:0/3260557450 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57402fe40 con 0x7fc584100800 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.223+0000 7fc580ff9640 1 -- 172.21.15.83:0/3260557450 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc574031df0 con 0x7fc584100800 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.223+0000 7fc580ff9640 1 -- 172.21.15.83:0/3260557450 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574046a00 con 0x7fc584100800 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.223+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5841a1f10 con 0x7fc584100800 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.223+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5841a23e0 con 0x7fc584100800 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.224+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5841a2610 con 0x7fc584100800 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.226+0000 7fc580ff9640 1 -- 172.21.15.83:0/3260557450 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc574030ca0 con 0x7fc584100800 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.226+0000 7fc580ff9640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc5700a2c00 0x7fc5700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.226+0000 7fc580ff9640 1 -- 172.21.15.83:0/3260557450 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6723+0+0 (secure 0 0 0) 0x7fc5740f0080 con 0x7fc584100800 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.226+0000 7fc582ffd640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc5700a2c00 0x7fc5700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.227+0000 7fc582ffd640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc5700a2c00 0x7fc5700a50c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc584104510 tx=0x7fc56c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:42:52.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.230+0000 7fc580ff9640 1 -- 172.21.15.83:0/3260557450 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc5740bd610 con 0x7fc584100800 2024-08-02T17:42:52.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] boot 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: osdmap e71: 11 total, 11 up, 11 in 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi077 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi077 to 4272M 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:52.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:42:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:42:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] boot 2024-08-02T17:42:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: osdmap e71: 11 total, 11 up, 11 in 2024-08-02T17:42:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi077 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi077 to 4272M 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:52.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:52.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:52.360 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.410 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:42:52.409+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc5841a27e0 con 0x7fc5700a2c00 2024-08-02T17:42:52.410 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:42:52.410 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:42:52.410 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: osd.10 [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] boot 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: osdmap e71: 11 total, 11 up, 11 in 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi077 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi077 to 4272M 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.411 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:52.412 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:52.412 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:52.412 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:42:52.412 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:52.412 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:42:52.412 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:42:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:53 smithi077 ceph-mon[37108]: osdmap e72: 11 total, 11 up, 11 in 2024-08-02T17:42:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:53 smithi077 ceph-mon[37108]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:53 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:53 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:53 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:53.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:53 smithi077 ceph-mon[37108]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:42:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:53 smithi069 ceph-mon[32746]: osdmap e72: 11 total, 11 up, 11 in 2024-08-02T17:42:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:53 smithi069 ceph-mon[32746]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:53 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:53.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:53 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:53.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:53 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:53.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:53 smithi069 ceph-mon[32746]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:42:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:53 smithi083 ceph-mon[37031]: osdmap e72: 11 total, 11 up, 11 in 2024-08-02T17:42:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:53 smithi083 ceph-mon[37031]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi083:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:42:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:53 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-08-02T17:42:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:53 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-08-02T17:42:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:53 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:42:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:53 smithi083 ceph-mon[37031]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:42:54.160 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:54 smithi083 ceph-mon[37031]: osdmap e73: 11 total, 11 up, 11 in 2024-08-02T17:42:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:54 smithi077 ceph-mon[37108]: osdmap e73: 11 total, 11 up, 11 in 2024-08-02T17:42:54.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:54 smithi069 ceph-mon[32746]: osdmap e73: 11 total, 11 up, 11 in 2024-08-02T17:42:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:55 smithi077 ceph-mon[37108]: osdmap e74: 11 total, 11 up, 11 in 2024-08-02T17:42:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:55 smithi077 ceph-mon[37108]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-02T17:42:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:55 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/3443233793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]: dispatch 2024-08-02T17:42:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:55 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]: dispatch 2024-08-02T17:42:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:55 smithi077 ceph-mon[37108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]': finished 2024-08-02T17:42:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:55 smithi077 ceph-mon[37108]: osdmap e75: 12 total, 11 up, 12 in 2024-08-02T17:42:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:42:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:55 smithi069 ceph-mon[32746]: osdmap e74: 11 total, 11 up, 11 in 2024-08-02T17:42:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:55 smithi069 ceph-mon[32746]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-02T17:42:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:55 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/3443233793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]: dispatch 2024-08-02T17:42:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:55 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]: dispatch 2024-08-02T17:42:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:55 smithi069 ceph-mon[32746]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]': finished 2024-08-02T17:42:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:55 smithi069 ceph-mon[32746]: osdmap e75: 12 total, 11 up, 12 in 2024-08-02T17:42:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:42:55.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:55 smithi083 ceph-mon[37031]: osdmap e74: 11 total, 11 up, 11 in 2024-08-02T17:42:55.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:55 smithi083 ceph-mon[37031]: pgmap v213: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-02T17:42:55.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:55 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/3443233793' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]: dispatch 2024-08-02T17:42:55.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:55 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]: dispatch 2024-08-02T17:42:55.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:55 smithi083 ceph-mon[37031]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e"}]': finished 2024-08-02T17:42:55.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:55 smithi083 ceph-mon[37031]: osdmap e75: 12 total, 11 up, 12 in 2024-08-02T17:42:55.429 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:42:56.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:56 smithi077 ceph-mon[37108]: from='client.? 172.21.15.83:0/2184166531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:56 smithi069 ceph-mon[32746]: from='client.? 172.21.15.83:0/2184166531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:56 smithi083 ceph-mon[37031]: from='client.? 172.21.15.83:0/2184166531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-08-02T17:42:57.062 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:42:56 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:42:56.706+0000 7f73ed3a4640 -1 calc_pg_upmaps abort due to max <= 0 2024-08-02T17:42:57.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:57 smithi077 ceph-mon[37108]: pgmap v215: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-08-02T17:42:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:57 smithi069 ceph-mon[32746]: pgmap v215: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-08-02T17:42:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:57 smithi083 ceph-mon[37031]: pgmap v215: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 340 MiB used, 983 GiB / 983 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-08-02T17:43:00.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:42:59 smithi077 ceph-mon[37108]: pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:43:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:42:59 smithi069 ceph-mon[32746]: pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:43:00.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:42:59 smithi083 ceph-mon[37031]: pgmap v216: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-02T17:43:02.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:01 smithi083 ceph-mon[37031]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-08-02T17:43:02.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:01 smithi077 ceph-mon[37108]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-08-02T17:43:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:01 smithi069 ceph-mon[32746]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-08-02T17:43:03.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:02 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-02T17:43:03.049 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:02 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:03.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:02 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-02T17:43:03.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:02 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:03.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:02 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-08-02T17:43:03.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:02 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:03.916 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:03 smithi083 ceph-mon[37031]: Deploying daemon osd.11 on smithi083 2024-08-02T17:43:03.917 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:03 smithi083 ceph-mon[37031]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:04.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:03 smithi077 ceph-mon[37108]: Deploying daemon osd.11 on smithi083 2024-08-02T17:43:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:03 smithi077 ceph-mon[37108]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:04.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:03 smithi069 ceph-mon[32746]: Deploying daemon osd.11 on smithi083 2024-08-02T17:43:04.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:03 smithi069 ceph-mon[32746]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:06.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:05 smithi077 ceph-mon[37108]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:05 smithi069 ceph-mon[32746]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:05 smithi083 ceph-mon[37031]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:07.972 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:07 smithi083 ceph-mon[37031]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:07.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:07.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:07.973 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:08.093 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:07 smithi077 ceph-mon[37108]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:08.094 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:08.094 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:08.094 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:08.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:07 smithi069 ceph-mon[32746]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:08.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:08.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:08.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:10.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:09 smithi083 ceph-mon[37031]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:10.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:09 smithi077 ceph-mon[37108]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:09 smithi069 ceph-mon[32746]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:10.796 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.793+0000 7fc580ff9640 1 -- 172.21.15.83:0/3260557450 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc5841a27e0 con 0x7fc5700a2c00 2024-08-02T17:43:10.796 INFO:teuthology.orchestra.run.smithi083.stdout:Created osd(s) 11 on host 'smithi083' 2024-08-02T17:43:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc5700a2c00 msgr2=0x7fc5700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc5700a2c00 0x7fc5700a50c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc584104510 tx=0x7fc56c009110 comp rx=0 tx=0).stop 2024-08-02T17:43:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 msgr2=0x7fc584104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 0x7fc584104790 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc574009430 tx=0x7fc57402f840 comp rx=0 tx=0).stop 2024-08-02T17:43:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 shutdown_connections 2024-08-02T17:43:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc5700a2c00 0x7fc5700a50c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc584101a00 0x7fc584102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc5841023c0 0x7fc584103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 --2- 172.21.15.83:0/3260557450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc584100800 0x7fc584104790 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:10.799 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.796+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 >> 172.21.15.83:0/3260557450 conn(0x7fc5840fbfb0 msgr2=0x7fc584069d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.797+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 shutdown_connections 2024-08-02T17:43:10.800 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:10.797+0000 7fc589a83640 1 -- 172.21.15.83:0/3260557450 wait complete. 2024-08-02T17:43:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:11.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:11.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:11.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:10 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:10 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:11.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:11.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:10 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:11.491 DEBUG:teuthology.orchestra.run.smithi083:osd.11> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.11.service 2024-08-02T17:43:11.493 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-02T17:43:11.494 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd stat -f json 2024-08-02T17:43:11.762 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:11.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:11 smithi083 ceph-mon[37031]: from='osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-02T17:43:11.998 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:11 smithi083 ceph-mon[37031]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-02T17:43:12.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:11 smithi069 ceph-mon[32746]: from='osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-02T17:43:12.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:11 smithi069 ceph-mon[32746]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-02T17:43:12.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 -- 172.21.15.69:0/4085659362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604100840 msgr2=0x7f8604100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:12.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 --2- 172.21.15.69:0/4085659362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604100840 0x7f8604100c40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009ca0 tx=0x7f85f402a8c0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 -- 172.21.15.69:0/4085659362 shutdown_connections 2024-08-02T17:43:12.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 --2- 172.21.15.69:0/4085659362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8604102400 0x7f8604108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 --2- 172.21.15.69:0/4085659362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8604101a40 0x7f8604101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 --2- 172.21.15.69:0/4085659362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604100840 0x7f8604100c40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 -- 172.21.15.69:0/4085659362 >> 172.21.15.69:0/4085659362 conn(0x7f86040fbfd0 msgr2=0x7f86040fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 -- 172.21.15.69:0/4085659362 shutdown_connections 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 -- 172.21.15.69:0/4085659362 wait complete. 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.202+0000 7f860a257640 1 Processor -- start 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f860a257640 1 -- start start 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f860a257640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8604100840 0x7f86041047d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f860a257640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604101a40 0x7f8604102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f860a257640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8604102400 0x7f8604103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:12.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f860a257640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f860410c4d0 con 0x7f8604101a40 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f860a257640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f860410c350 con 0x7f8604102400 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f860a257640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f860410c650 con 0x7f8604100840 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604101a40 0x7f8604102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604101a40 0x7f8604102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57786/0 (socket says 172.21.15.69:57786) 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 -- 172.21.15.69:0/3091230170 learned_addr learned my addr 172.21.15.69:0/3091230170 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f86037fe640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8604100840 0x7f86041047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 -- 172.21.15.69:0/3091230170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8604100840 msgr2=0x7f86041047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8604100840 0x7f86041047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8603fff640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8604102400 0x7f8604103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 -- 172.21.15.69:0/3091230170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8604102400 msgr2=0x7f8604103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8604102400 0x7f8604103360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 -- 172.21.15.69:0/3091230170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8604103c20 con 0x7f8604101a40 2024-08-02T17:43:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f86037fe640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8604100840 0x7f86041047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:43:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.203+0000 7f8602ffd640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604101a40 0x7f8604102e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c920 tx=0x7f85f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.204+0000 7f8600ff9640 1 -- 172.21.15.69:0/3091230170 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8007c70 con 0x7f8604101a40 2024-08-02T17:43:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.204+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86041a1d40 con 0x7f8604101a40 2024-08-02T17:43:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.204+0000 7f8600ff9640 1 -- 172.21.15.69:0/3091230170 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f8007e10 con 0x7f8604101a40 2024-08-02T17:43:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.204+0000 7f8600ff9640 1 -- 172.21.15.69:0/3091230170 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8018890 con 0x7f8604101a40 2024-08-02T17:43:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.204+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86041a2240 con 0x7f8604101a40 2024-08-02T17:43:12.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.205+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86041a24d0 con 0x7f8604101a40 2024-08-02T17:43:12.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.206+0000 7f8600ff9640 1 -- 172.21.15.69:0/3091230170 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f85f8018a70 con 0x7f8604101a40 2024-08-02T17:43:12.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.207+0000 7f8600ff9640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f85ec0a2c00 0x7f85ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:12.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.207+0000 7f86037fe640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f85ec0a2c00 0x7f85ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:12.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.207+0000 7f8600ff9640 1 -- 172.21.15.69:0/3091230170 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6831+0+0 (secure 0 0 0) 0x7f85f80cece0 con 0x7f8604101a40 2024-08-02T17:43:12.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.208+0000 7f86037fe640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f85ec0a2c00 0x7f85ec0a50c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006010 tx=0x7f85f4000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:12.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.211+0000 7f8600ff9640 1 -- 172.21.15.69:0/3091230170 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f85f8014030 con 0x7f8604101a40 2024-08-02T17:43:12.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:11 smithi077 ceph-mon[37108]: from='osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-02T17:43:12.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:11 smithi077 ceph-mon[37108]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-08-02T17:43:12.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.377+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f86041a2ba0 con 0x7f8604101a40 2024-08-02T17:43:12.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.378+0000 7f8600ff9640 1 -- 172.21.15.69:0/3091230170 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f85f800de20 con 0x7f8604101a40 2024-08-02T17:43:12.379 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f85ec0a2c00 msgr2=0x7f85ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f85ec0a2c00 0x7f85ec0a50c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006010 tx=0x7f85f4000f40 comp rx=0 tx=0).stop 2024-08-02T17:43:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604101a40 msgr2=0x7f8604102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:12.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604101a40 0x7f8604102e20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c920 tx=0x7f85f800cde0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 shutdown_connections 2024-08-02T17:43:12.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f85ec0a2c00 0x7f85ec0a50c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8604101a40 0x7f8604102e20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8604102400 0x7f8604103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.379+0000 7f860a257640 1 --2- 172.21.15.69:0/3091230170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8604100840 0x7f86041047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:12.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.380+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 >> 172.21.15.69:0/3091230170 conn(0x7f86040fbfd0 msgr2=0x7f86040fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:12.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.380+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 shutdown_connections 2024-08-02T17:43:12.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:12.380+0000 7f860a257640 1 -- 172.21.15.69:0/3091230170 wait complete. 2024-08-02T17:43:13.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:12 smithi083 ceph-mon[37031]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-02T17:43:13.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:12 smithi083 ceph-mon[37031]: osdmap e76: 12 total, 11 up, 12 in 2024-08-02T17:43:13.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:12 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:13.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:12 smithi083 ceph-mon[37031]: from='osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:43:13.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:12 smithi083 ceph-mon[37031]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:43:13.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:12 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3091230170' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-02T17:43:13.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:12 smithi083 ceph-mon[37031]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:13.166 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1722620571,"num_in_osds":12,"osd_in_since":1722620574,"num_remapped_pgs":0} 2024-08-02T17:43:13.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:12 smithi077 ceph-mon[37108]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-02T17:43:13.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:12 smithi077 ceph-mon[37108]: osdmap e76: 12 total, 11 up, 12 in 2024-08-02T17:43:13.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:12 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:13.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:12 smithi077 ceph-mon[37108]: from='osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:43:13.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:12 smithi077 ceph-mon[37108]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:43:13.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:12 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3091230170' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-02T17:43:13.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:12 smithi077 ceph-mon[37108]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:12 smithi069 ceph-mon[32746]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-02T17:43:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:12 smithi069 ceph-mon[32746]: osdmap e76: 12 total, 11 up, 12 in 2024-08-02T17:43:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:12 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:12 smithi069 ceph-mon[32746]: from='osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:43:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:12 smithi069 ceph-mon[32746]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]: dispatch 2024-08-02T17:43:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:12 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3091230170' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-02T17:43:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:12 smithi069 ceph-mon[32746]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:13.969 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 17:43:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T17:43:13.665+0000 7f8a175a7640 -1 osd.11 0 waiting for initial osdmap 2024-08-02T17:43:13.970 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 17:43:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T17:43:13.681+0000 7f8a0f3c7640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-02T17:43:14.167 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd stat -f json 2024-08-02T17:43:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:13 smithi083 ceph-mon[37031]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:43:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:13 smithi083 ceph-mon[37031]: osdmap e77: 12 total, 11 up, 12 in 2024-08-02T17:43:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:13 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:13 smithi083 ceph-mon[37031]: from='osd.11 ' entity='osd.11' 2024-08-02T17:43:14.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:13 smithi077 ceph-mon[37108]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:43:14.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:13 smithi077 ceph-mon[37108]: osdmap e77: 12 total, 11 up, 12 in 2024-08-02T17:43:14.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:14.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:13 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:14.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:13 smithi077 ceph-mon[37108]: from='osd.11 ' entity='osd.11' 2024-08-02T17:43:14.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:13 smithi069 ceph-mon[32746]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi083", "root=default"]}]': finished 2024-08-02T17:43:14.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:13 smithi069 ceph-mon[32746]: osdmap e77: 12 total, 11 up, 12 in 2024-08-02T17:43:14.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:14.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:13 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:14.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:13 smithi069 ceph-mon[32746]: from='osd.11 ' entity='osd.11' 2024-08-02T17:43:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.902+0000 7f6da2e9f640 1 -- 172.21.15.69:0/1625865463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c104d60 msgr2=0x7f6d9c107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.902+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/1625865463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c104d60 0x7f6d9c107150 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6d90009900 tx=0x7f6d9002a410 comp rx=0 tx=0).stop 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.902+0000 7f6da2e9f640 1 -- 172.21.15.69:0/1625865463 shutdown_connections 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.902+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/1625865463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c104d60 0x7f6d9c107150 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.902+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/1625865463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d9c107690 0x7f6d9c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.902+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/1625865463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9c102430 0x7f6d9c104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.902+0000 7f6da2e9f640 1 -- 172.21.15.69:0/1625865463 >> 172.21.15.69:0/1625865463 conn(0x7f6d9c0fbfd0 msgr2=0x7f6d9c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 -- 172.21.15.69:0/1625865463 shutdown_connections 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 -- 172.21.15.69:0/1625865463 wait complete. 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 Processor -- start 2024-08-02T17:43:14.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 -- start start 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d9c102430 0x7f6d9c101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9c104d60 0x7f6d9c1003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c107690 0x7f6d9c100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c10d5a0 con 0x7f6d9c107690 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c10d420 con 0x7f6d9c102430 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da2e9f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d9c10d720 con 0x7f6d9c104d60 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da143c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c107690 0x7f6d9c100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da143c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c107690 0x7f6d9c100920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35206/0 (socket says 172.21.15.69:35206) 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.903+0000 7f6da143c640 1 -- 172.21.15.69:0/2706444772 learned_addr learned my addr 172.21.15.69:0/2706444772 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6d9bfff640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9c104d60 0x7f6d9c1003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da0c3b640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d9c102430 0x7f6d9c101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da143c640 1 -- 172.21.15.69:0/2706444772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9c104d60 msgr2=0x7f6d9c1003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:14.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da143c640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9c104d60 0x7f6d9c1003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da143c640 1 -- 172.21.15.69:0/2706444772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d9c102430 msgr2=0x7f6d9c101d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da143c640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d9c102430 0x7f6d9c101d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da143c640 1 -- 172.21.15.69:0/2706444772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9c1011e0 con 0x7f6d9c107690 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da0c3b640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d9c102430 0x7f6d9c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6d9bfff640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9c104d60 0x7f6d9c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da143c640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c107690 0x7f6d9c100920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400cad0 tx=0x7f6d9400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6d99ffb640 1 -- 172.21.15.69:0/2706444772 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400ae20 con 0x7f6d9c107690 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6d99ffb640 1 -- 172.21.15.69:0/2706444772 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d94005020 con 0x7f6d9c107690 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6d99ffb640 1 -- 172.21.15.69:0/2706444772 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400f6f0 con 0x7f6d9c107690 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9c1a1da0 con 0x7f6d9c107690 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.904+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d9c1a2200 con 0x7f6d9c107690 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.905+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d9c0fdac0 con 0x7f6d9c107690 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.906+0000 7f6d99ffb640 1 -- 172.21.15.69:0/2706444772 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6d940051d0 con 0x7f6d9c107690 2024-08-02T17:43:14.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.907+0000 7f6d99ffb640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6d840a2b80 0x7f6d840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.907+0000 7f6da0c3b640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6d840a2b80 0x7f6d840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.907+0000 7f6d99ffb640 1 -- 172.21.15.69:0/2706444772 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f6d940cecc0 con 0x7f6d9c107690 2024-08-02T17:43:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.907+0000 7f6da0c3b640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6d840a2b80 0x7f6d840a5040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00baa0 tx=0x7f6d8c005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:14.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:14.911+0000 7f6d99ffb640 1 -- 172.21.15.69:0/2706444772 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6d94014030 con 0x7f6d9c107690 2024-08-02T17:43:15.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.081+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f6d9c02d7b0 con 0x7f6d9c107690 2024-08-02T17:43:15.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.081+0000 7f6d99ffb640 1 -- 172.21.15.69:0/2706444772 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f6d9409c0c0 con 0x7f6d9c107690 2024-08-02T17:43:15.083 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:15.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: purged_snaps scrub starts 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: purged_snaps scrub ok 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: Detected new or changed devices on smithi083 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi083 to 4272M 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.084 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] boot 2024-08-02T17:43:15.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: osdmap e78: 12 total, 12 up, 12 in 2024-08-02T17:43:15.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:15.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:14 smithi069 ceph-mon[32746]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:15.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6d840a2b80 msgr2=0x7f6d840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:15.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6d840a2b80 0x7f6d840a5040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d8c00baa0 tx=0x7f6d8c005d90 comp rx=0 tx=0).stop 2024-08-02T17:43:15.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c107690 msgr2=0x7f6d9c100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c107690 0x7f6d9c100920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400cad0 tx=0x7f6d9400cf90 comp rx=0 tx=0).stop 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 shutdown_connections 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6d840a2b80 0x7f6d840a5040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d9c107690 0x7f6d9c100920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d9c104d60 0x7f6d9c1003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.083+0000 7f6da2e9f640 1 --2- 172.21.15.69:0/2706444772 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d9c102430 0x7f6d9c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.084+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 >> 172.21.15.69:0/2706444772 conn(0x7f6d9c0fbfd0 msgr2=0x7f6d9c102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.084+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 shutdown_connections 2024-08-02T17:43:15.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:15.084+0000 7f6da2e9f640 1 -- 172.21.15.69:0/2706444772 wait complete. 2024-08-02T17:43:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: purged_snaps scrub starts 2024-08-02T17:43:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: purged_snaps scrub ok 2024-08-02T17:43:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: Detected new or changed devices on smithi083 2024-08-02T17:43:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi083 to 4272M 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] boot 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: osdmap e78: 12 total, 12 up, 12 in 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:15.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:14 smithi083 ceph-mon[37031]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: purged_snaps scrub starts 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: purged_snaps scrub ok 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: Detected new or changed devices on smithi083 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi083 to 4272M 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:15.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:15.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: osd.11 [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] boot 2024-08-02T17:43:15.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: osdmap e78: 12 total, 12 up, 12 in 2024-08-02T17:43:15.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:43:15.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:14 smithi077 ceph-mon[37108]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-08-02T17:43:15.852 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1722620594,"num_in_osds":12,"osd_in_since":1722620574,"num_remapped_pgs":0} 2024-08-02T17:43:15.853 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:43:16.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:15 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2706444772' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-02T17:43:16.108 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:16.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:15 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2706444772' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-02T17:43:16.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:15 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2706444772' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 -- 172.21.15.69:0/1632362077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 msgr2=0x7fa1a4116bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 --2- 172.21.15.69:0/1632362077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 0x7fa1a4116bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009ca0 tx=0x7fa19c02a950 comp rx=0 tx=0).stop 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 -- 172.21.15.69:0/1632362077 shutdown_connections 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 --2- 172.21.15.69:0/1632362077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 0x7fa1a4116bd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 --2- 172.21.15.69:0/1632362077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1a4109840 0x7fa1a41142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 --2- 172.21.15.69:0/1632362077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4108f20 0x7fa1a4109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 -- 172.21.15.69:0/1632362077 >> 172.21.15.69:0/1632362077 conn(0x7fa1a406de70 msgr2=0x7fa1a406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 -- 172.21.15.69:0/1632362077 shutdown_connections 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 -- 172.21.15.69:0/1632362077 wait complete. 2024-08-02T17:43:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.551+0000 7fa1a992a640 1 Processor -- start 2024-08-02T17:43:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a992a640 1 -- start start 2024-08-02T17:43:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a992a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4108f20 0x7fa1a41a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a992a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1a4109840 0x7fa1a41a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a992a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 0x7fa1a41a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a992a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a411a510 con 0x7fa1a41147e0 2024-08-02T17:43:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a992a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a411a390 con 0x7fa1a4109840 2024-08-02T17:43:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a992a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a411a690 con 0x7fa1a4108f20 2024-08-02T17:43:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 0x7fa1a41a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 0x7fa1a41a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35228/0 (socket says 172.21.15.69:35228) 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a37fe640 1 -- 172.21.15.69:0/3857865913 learned_addr learned my addr 172.21.15.69:0/3857865913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a2ffd640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4108f20 0x7fa1a41a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a27fc640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1a4109840 0x7fa1a41a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a37fe640 1 -- 172.21.15.69:0/3857865913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4108f20 msgr2=0x7fa1a41a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a37fe640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4108f20 0x7fa1a41a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a37fe640 1 -- 172.21.15.69:0/3857865913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1a4109840 msgr2=0x7fa1a41a6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a37fe640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1a4109840 0x7fa1a41a6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.552+0000 7fa1a37fe640 1 -- 172.21.15.69:0/3857865913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a41a0fe0 con 0x7fa1a41147e0 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.553+0000 7fa1a37fe640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 0x7fa1a41a0840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa19c005e30 tx=0x7fa19c02f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.553+0000 7fa1a8928640 1 -- 172.21.15.69:0/3857865913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c034040 con 0x7fa1a41147e0 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.553+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a41a1270 con 0x7fa1a41147e0 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.553+0000 7fa1a8928640 1 -- 172.21.15.69:0/3857865913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa19c030920 con 0x7fa1a41147e0 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.553+0000 7fa1a8928640 1 -- 172.21.15.69:0/3857865913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c004750 con 0x7fa1a41147e0 2024-08-02T17:43:16.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.553+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41ad520 con 0x7fa1a41147e0 2024-08-02T17:43:16.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.554+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a4073770 con 0x7fa1a41147e0 2024-08-02T17:43:16.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.555+0000 7fa1a8928640 1 -- 172.21.15.69:0/3857865913 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa19c037020 con 0x7fa1a41147e0 2024-08-02T17:43:16.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.555+0000 7fa1a8928640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa18c0a2b80 0x7fa18c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:16.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.556+0000 7fa1a2ffd640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa18c0a2b80 0x7fa18c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:16.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.556+0000 7fa1a8928640 1 -- 172.21.15.69:0/3857865913 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fa19c0efbb0 con 0x7fa1a41147e0 2024-08-02T17:43:16.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.556+0000 7fa1a2ffd640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa18c0a2b80 0x7fa18c0a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa194009890 tx=0x7fa194008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:16.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.560+0000 7fa1a8928640 1 -- 172.21.15.69:0/3857865913 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa19c0bcfb0 con 0x7fa1a41147e0 2024-08-02T17:43:16.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.727+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1a41ad6f0 con 0x7fa1a41147e0 2024-08-02T17:43:16.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.728+0000 7fa1a8928640 1 -- 172.21.15.69:0/3857865913 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18349 (secure 0 0 0) 0x7fa19c0bc8b0 con 0x7fa1a41147e0 2024-08-02T17:43:16.729 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:16.729 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":79,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:43:15.968025+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:43:16.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa18c0a2b80 msgr2=0x7fa18c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:16.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa18c0a2b80 0x7fa18c0a5040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa194009890 tx=0x7fa194008040 comp rx=0 tx=0).stop 2024-08-02T17:43:16.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 msgr2=0x7fa1a41a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:16.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 0x7fa1a41a0840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa19c005e30 tx=0x7fa19c02f630 comp rx=0 tx=0).stop 2024-08-02T17:43:16.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 shutdown_connections 2024-08-02T17:43:16.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa18c0a2b80 0x7fa18c0a5040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1a41147e0 0x7fa1a41a0840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1a4109840 0x7fa1a41a6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 --2- 172.21.15.69:0/3857865913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1a4108f20 0x7fa1a41a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:16.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 >> 172.21.15.69:0/3857865913 conn(0x7fa1a406de70 msgr2=0x7fa1a4070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:16.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 shutdown_connections 2024-08-02T17:43:16.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:16.730+0000 7fa1a992a640 1 -- 172.21.15.69:0/3857865913 wait complete. 2024-08-02T17:43:17.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:16 smithi083 ceph-mon[37031]: osdmap e79: 12 total, 12 up, 12 in 2024-08-02T17:43:17.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:16 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3857865913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:43:17.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:16 smithi083 ceph-mon[37031]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:17.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:16 smithi069 ceph-mon[32746]: osdmap e79: 12 total, 12 up, 12 in 2024-08-02T17:43:17.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:16 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3857865913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:43:17.328 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:16 smithi069 ceph-mon[32746]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:17.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:16 smithi077 ceph-mon[37108]: osdmap e79: 12 total, 12 up, 12 in 2024-08-02T17:43:17.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:16 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3857865913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:43:17.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:16 smithi077 ceph-mon[37108]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:17.355 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-02T17:39:54.984711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-02T17:43:17.356 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-02T17:43:17.603 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 -- 172.21.15.69:0/2575975169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a010bea0 msgr2=0x7f49a010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2575975169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a010bea0 0x7f49a010e340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4998009ca0 tx=0x7f499802a950 comp rx=0 tx=0).stop 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 -- 172.21.15.69:0/2575975169 shutdown_connections 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2575975169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a010bea0 0x7f49a010e340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2575975169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49a0103390 0x7f49a010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2575975169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a0102a70 0x7f49a0102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 -- 172.21.15.69:0/2575975169 >> 172.21.15.69:0/2575975169 conn(0x7f49a00fc4d0 msgr2=0x7f49a00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 -- 172.21.15.69:0/2575975169 shutdown_connections 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.029+0000 7f49a54fa640 1 -- 172.21.15.69:0/2575975169 wait complete. 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f49a54fa640 1 Processor -- start 2024-08-02T17:43:18.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f49a54fa640 1 -- start start 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f49a54fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a0102a70 0x7f49a01980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f49a54fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a0103390 0x7f49a0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f49a54fa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49a010bea0 0x7f49a019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f49a54fa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0111ca0 con 0x7f49a0102a70 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f49a54fa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0111b20 con 0x7f49a010bea0 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f49a54fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0111e20 con 0x7f49a0103390 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f499effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a0102a70 0x7f49a01980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f499effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a0102a70 0x7f49a01980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35240/0 (socket says 172.21.15.69:35240) 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.030+0000 7f499effd640 1 -- 172.21.15.69:0/2634944462 learned_addr learned my addr 172.21.15.69:0/2634944462 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f49967fc640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a0103390 0x7f49a0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499f7fe640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49a010bea0 0x7f49a019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:18.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499effd640 1 -- 172.21.15.69:0/2634944462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a0103390 msgr2=0x7f49a0198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499effd640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a0103390 0x7f49a0198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499effd640 1 -- 172.21.15.69:0/2634944462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49a010bea0 msgr2=0x7f49a019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499effd640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49a010bea0 0x7f49a019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499effd640 1 -- 172.21.15.69:0/2634944462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a019d070 con 0x7f49a0102a70 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499f7fe640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49a010bea0 0x7f49a019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499effd640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a0102a70 0x7f49a01980c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f499000d770 tx=0x7f499000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499cff9640 1 -- 172.21.15.69:0/2634944462 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990017070 con 0x7f49a0102a70 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499cff9640 1 -- 172.21.15.69:0/2634944462 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4990005500 con 0x7f49a0102a70 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a019d360 con 0x7f49a0102a70 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f499cff9640 1 -- 172.21.15.69:0/2634944462 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4990004e00 con 0x7f49a0102a70 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.031+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a01a4c40 con 0x7f49a0102a70 2024-08-02T17:43:18.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.032+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a006ada0 con 0x7f49a0102a70 2024-08-02T17:43:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.033+0000 7f499cff9640 1 -- 172.21.15.69:0/2634944462 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4990004fa0 con 0x7f49a0102a70 2024-08-02T17:43:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.034+0000 7f499cff9640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f498c0a2b80 0x7f498c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.034+0000 7f49967fc640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f498c0a2b80 0x7f498c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.034+0000 7f499cff9640 1 -- 172.21.15.69:0/2634944462 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f49900ce5d0 con 0x7f49a0102a70 2024-08-02T17:43:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.034+0000 7f49967fc640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f498c0a2b80 0x7f498c0a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49a010d1c0 tx=0x7f498800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:18.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.038+0000 7f499cff9640 1 -- 172.21.15.69:0/2634944462 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f499009b9d0 con 0x7f49a0102a70 2024-08-02T17:43:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.204+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49a01a4e10 con 0x7f49a0102a70 2024-08-02T17:43:18.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.205+0000 7f499cff9640 1 -- 172.21.15.69:0/2634944462 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f499009b2d0 con 0x7f49a0102a70 2024-08-02T17:43:18.207 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-08-02T17:43:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f498c0a2b80 msgr2=0x7f498c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f498c0a2b80 0x7f498c0a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49a010d1c0 tx=0x7f498800a250 comp rx=0 tx=0).stop 2024-08-02T17:43:18.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a0102a70 msgr2=0x7f49a01980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a0102a70 0x7f49a01980c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f499000d770 tx=0x7f499000dc30 comp rx=0 tx=0).stop 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 shutdown_connections 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f498c0a2b80 0x7f498c0a5040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f49a010bea0 0x7f49a019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f49a0103390 0x7f49a0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 --2- 172.21.15.69:0/2634944462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f49a0102a70 0x7f49a01980c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 >> 172.21.15.69:0/2634944462 conn(0x7f49a00fc4d0 msgr2=0x7f49a00fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 shutdown_connections 2024-08-02T17:43:18.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:18.207+0000 7f49a54fa640 1 -- 172.21.15.69:0/2634944462 wait complete. 2024-08-02T17:43:18.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:18 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2634944462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T17:43:18.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:18 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2634944462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T17:43:18.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:18 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2634944462' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T17:43:18.948 INFO:tasks.cephadm:Adding mds.a on smithi069 2024-08-02T17:43:18.948 INFO:tasks.cephadm:Adding mds.d on smithi069 2024-08-02T17:43:18.948 INFO:tasks.cephadm:Adding mds.g on smithi069 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.j on smithi069 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.b on smithi077 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.e on smithi077 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.h on smithi077 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.k on smithi077 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.c on smithi083 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.f on smithi083 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.i on smithi083 2024-08-02T17:43:18.949 INFO:tasks.cephadm:Adding mds.l on smithi083 2024-08-02T17:43:18.950 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph orch apply mds all '12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l' 2024-08-02T17:43:19.212 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T17:43:19.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:19 smithi083 ceph-mon[37031]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:19.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:19 smithi077 ceph-mon[37108]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:19.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:19 smithi069 ceph-mon[32746]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 -- 172.21.15.83:0/2057487515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304104d60 msgr2=0x7f6304107150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 --2- 172.21.15.83:0/2057487515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304104d60 0x7f6304107150 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009900 tx=0x7f62f402a410 comp rx=0 tx=0).stop 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 -- 172.21.15.83:0/2057487515 shutdown_connections 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 --2- 172.21.15.83:0/2057487515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6304107690 0x7f6304109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 --2- 172.21.15.83:0/2057487515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304104d60 0x7f6304107150 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 --2- 172.21.15.83:0/2057487515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6304102430 0x7f6304104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 -- 172.21.15.83:0/2057487515 >> 172.21.15.83:0/2057487515 conn(0x7f63040fbfb0 msgr2=0x7f63040fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 -- 172.21.15.83:0/2057487515 shutdown_connections 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.680+0000 7f630b00b640 1 -- 172.21.15.83:0/2057487515 wait complete. 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f630b00b640 1 Processor -- start 2024-08-02T17:43:19.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f630b00b640 1 -- start start 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f630b00b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304102430 0x7f6304101d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f630b00b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6304104d60 0x7f63041003e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f630b00b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6304107690 0x7f6304100920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f630b00b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f630410d650 con 0x7f6304107690 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f630b00b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f630410d4d0 con 0x7f6304104d60 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f630b00b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f630410d7d0 con 0x7f6304102430 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f6308da7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304102430 0x7f6304101d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f6308da7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304102430 0x7f6304101d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:48324/0 (socket says 172.21.15.83:48324) 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f6308da7640 1 -- 172.21.15.83:0/2994872126 learned_addr learned my addr 172.21.15.83:0/2994872126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f62fbfff640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6304104d60 0x7f63041003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f63095a8640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6304107690 0x7f6304100920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f6308da7640 1 -- 172.21.15.83:0/2994872126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6304104d60 msgr2=0x7f63041003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:19.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f6308da7640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6304104d60 0x7f63041003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f6308da7640 1 -- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6304107690 msgr2=0x7f6304100920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f6308da7640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6304107690 0x7f6304100920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.681+0000 7f6308da7640 1 -- 172.21.15.83:0/2994872126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63041011e0 con 0x7f6304102430 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.682+0000 7f6308da7640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304102430 0x7f6304101d90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f630000cce0 tx=0x7f630000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.682+0000 7f62f9ffb640 1 -- 172.21.15.83:0/2994872126 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6300004b20 con 0x7f6304102430 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.682+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63041a1e30 con 0x7f6304102430 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.682+0000 7f62f9ffb640 1 -- 172.21.15.83:0/2994872126 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6300004cc0 con 0x7f6304102430 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.682+0000 7f62f9ffb640 1 -- 172.21.15.83:0/2994872126 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630000f8c0 con 0x7f6304102430 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.682+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63041a2250 con 0x7f6304102430 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.683+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63040fdaa0 con 0x7f6304102430 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.684+0000 7f62f9ffb640 1 -- 172.21.15.83:0/2994872126 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f630000fa60 con 0x7f6304102430 2024-08-02T17:43:19.710 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.684+0000 7f62f9ffb640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f62ec0a2b80 0x7f62ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.685+0000 7f62f9ffb640 1 -- 172.21.15.83:0/2994872126 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f63000ced20 con 0x7f6304102430 2024-08-02T17:43:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.685+0000 7f62fbfff640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f62ec0a2b80 0x7f62ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.685+0000 7f62fbfff640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f62ec0a2b80 0x7f62ec0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62f402aa00 tx=0x7f62f4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:19.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.689+0000 7f62f9ffb640 1 -- 172.21.15.83:0/2994872126 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6300014030 con 0x7f6304102430 2024-08-02T17:43:19.865 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.863+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l", "target": ["mon-mgr", ""]}) -- 0x7f6304062460 con 0x7f62ec0a2b80 2024-08-02T17:43:19.873 INFO:teuthology.orchestra.run.smithi083.stdout:Scheduled mds.all update... 2024-08-02T17:43:19.873 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.872+0000 7f62f9ffb640 1 -- 172.21.15.83:0/2994872126 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6304062460 con 0x7f62ec0a2b80 2024-08-02T17:43:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f62ec0a2b80 msgr2=0x7f62ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f62ec0a2b80 0x7f62ec0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f62f402aa00 tx=0x7f62f4031400 comp rx=0 tx=0).stop 2024-08-02T17:43:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304102430 msgr2=0x7f6304101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304102430 0x7f6304101d90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f630000cce0 tx=0x7f630000a620 comp rx=0 tx=0).stop 2024-08-02T17:43:19.875 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 shutdown_connections 2024-08-02T17:43:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f62ec0a2b80 0x7f62ec0a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6304107690 0x7f6304100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6304104d60 0x7f63041003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 --2- 172.21.15.83:0/2994872126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6304102430 0x7f6304101d90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 >> 172.21.15.83:0/2994872126 conn(0x7f63040fbfb0 msgr2=0x7f6304102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 shutdown_connections 2024-08-02T17:43:19.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-08-02T17:43:19.874+0000 7f630b00b640 1 -- 172.21.15.83:0/2994872126 wait complete. 2024-08-02T17:43:20.683 DEBUG:teuthology.orchestra.run.smithi069:mds.a> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.a.service 2024-08-02T17:43:20.686 DEBUG:teuthology.orchestra.run.smithi069:mds.d> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.d.service 2024-08-02T17:43:20.729 DEBUG:teuthology.orchestra.run.smithi069:mds.g> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.g.service 2024-08-02T17:43:20.772 DEBUG:teuthology.orchestra.run.smithi069:mds.j> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.j.service 2024-08-02T17:43:20.815 DEBUG:teuthology.orchestra.run.smithi077:mds.b> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.b.service 2024-08-02T17:43:20.818 DEBUG:teuthology.orchestra.run.smithi077:mds.e> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.e.service 2024-08-02T17:43:20.861 DEBUG:teuthology.orchestra.run.smithi077:mds.h> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.h.service 2024-08-02T17:43:20.904 DEBUG:teuthology.orchestra.run.smithi077:mds.k> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.k.service 2024-08-02T17:43:20.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:43:20.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l;count:12 2024-08-02T17:43:20.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:20.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:20.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:20.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:20.943 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:20.944 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:20.944 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:20.944 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:20 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:20.947 DEBUG:teuthology.orchestra.run.smithi083:mds.c> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.c.service 2024-08-02T17:43:20.949 DEBUG:teuthology.orchestra.run.smithi083:mds.f> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.f.service 2024-08-02T17:43:20.992 DEBUG:teuthology.orchestra.run.smithi083:mds.i> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.i.service 2024-08-02T17:43:21.036 DEBUG:teuthology.orchestra.run.smithi083:mds.l> sudo journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.l.service 2024-08-02T17:43:21.080 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-02T17:43:21.081 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:43:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l;count:12 2024-08-02T17:43:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:21.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:21.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:21.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:21.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:21.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:21.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:21.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:20 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:21.104 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: Saving service mds.all spec with placement smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l;count:12 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:21.105 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:20 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:21.334 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 -- 172.21.15.69:0/1394671129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 msgr2=0x7fe768100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 --2- 172.21.15.69:0/1394671129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 0x7fe768100f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe758009900 tx=0x7fe75802a480 comp rx=0 tx=0).stop 2024-08-02T17:43:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 -- 172.21.15.69:0/1394671129 shutdown_connections 2024-08-02T17:43:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 --2- 172.21.15.69:0/1394671129 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe768102c20 0x7fe76810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 --2- 172.21.15.69:0/1394671129 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681022a0 0x7fe7681026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 --2- 172.21.15.69:0/1394671129 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 0x7fe768100f30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 -- 172.21.15.69:0/1394671129 >> 172.21.15.69:0/1394671129 conn(0x7fe7680fc500 msgr2=0x7fe7680fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:21.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 -- 172.21.15.69:0/1394671129 shutdown_connections 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 -- 172.21.15.69:0/1394671129 wait complete. 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.759+0000 7fe76d228640 1 Processor -- start 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76d228640 1 -- start start 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76d228640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 0x7fe768198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76d228640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681022a0 0x7fe7681987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76d228640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe768102c20 0x7fe76819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76d228640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe768110cd0 con 0x7fe768100b50 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76d228640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe768110b50 con 0x7fe768102c20 2024-08-02T17:43:21.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76d228640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe768110e50 con 0x7fe7681022a0 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe766d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 0x7fe768198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe766d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 0x7fe768198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35248/0 (socket says 172.21.15.69:35248) 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe766d9d640 1 -- 172.21.15.69:0/586145155 learned_addr learned my addr 172.21.15.69:0/586145155 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76659c640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681022a0 0x7fe7681987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76759e640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe768102c20 0x7fe76819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe766d9d640 1 -- 172.21.15.69:0/586145155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681022a0 msgr2=0x7fe7681987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe766d9d640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681022a0 0x7fe7681987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe766d9d640 1 -- 172.21.15.69:0/586145155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe768102c20 msgr2=0x7fe76819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe766d9d640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe768102c20 0x7fe76819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe766d9d640 1 -- 172.21.15.69:0/586145155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76819d210 con 0x7fe768100b50 2024-08-02T17:43:21.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.760+0000 7fe76759e640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe768102c20 0x7fe76819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.761+0000 7fe766d9d640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 0x7fe768198260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe758006010 tx=0x7fe758003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.761+0000 7fe74ffff640 1 -- 172.21.15.69:0/586145155 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75803f070 con 0x7fe768100b50 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.761+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76819d4a0 con 0x7fe768100b50 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.761+0000 7fe74ffff640 1 -- 172.21.15.69:0/586145155 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe758031e30 con 0x7fe768100b50 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.761+0000 7fe74ffff640 1 -- 172.21.15.69:0/586145155 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758035030 con 0x7fe768100b50 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.761+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681a4de0 con 0x7fe768100b50 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.762+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe768100f30 con 0x7fe768100b50 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.763+0000 7fe74ffff640 1 -- 172.21.15.69:0/586145155 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe758042070 con 0x7fe768100b50 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.763+0000 7fe74ffff640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe7500a2b80 0x7fe7500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.764+0000 7fe76659c640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe7500a2b80 0x7fe7500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:21.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.764+0000 7fe74ffff640 1 -- 172.21.15.69:0/586145155 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fe7580ef4b0 con 0x7fe768100b50 2024-08-02T17:43:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.764+0000 7fe76659c640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe7500a2b80 0x7fe7500a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7680ffd40 tx=0x7fe75c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:21.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.768+0000 7fe74ffff640 1 -- 172.21.15.69:0/586145155 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe7580bc8b0 con 0x7fe768100b50 2024-08-02T17:43:21.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.968+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe768109dd0 con 0x7fe768100b50 2024-08-02T17:43:21.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.968+0000 7fe74ffff640 1 -- 172.21.15.69:0/586145155 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fe7580bc1b0 con 0x7fe768100b50 2024-08-02T17:43:21.970 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:21.970 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":1,"btime":"2024-08-02T17:37:04:618450+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-02T17:43:21.970 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 1 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe7500a2b80 msgr2=0x7fe7500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe7500a2b80 0x7fe7500a5040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7680ffd40 tx=0x7fe75c009110 comp rx=0 tx=0).stop 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 msgr2=0x7fe768198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 0x7fe768198260 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe758006010 tx=0x7fe758003c80 comp rx=0 tx=0).stop 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 shutdown_connections 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe7500a2b80 0x7fe7500a5040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe768102c20 0x7fe76819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe7681022a0 0x7fe7681987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 --2- 172.21.15.69:0/586145155 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe768100b50 0x7fe768198260 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:21.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.970+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 >> 172.21.15.69:0/586145155 conn(0x7fe7680fc500 msgr2=0x7fe768069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.971+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 shutdown_connections 2024-08-02T17:43:21.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:21.971+0000 7fe76d228640 1 -- 172.21.15.69:0/586145155 wait complete. 2024-08-02T17:43:22.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:21 smithi083 ceph-mon[37031]: Deploying daemon mds.l on smithi083 2024-08-02T17:43:22.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:21 smithi083 ceph-mon[37031]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:22.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:21 smithi069 ceph-mon[32746]: Deploying daemon mds.l on smithi083 2024-08-02T17:43:22.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:21 smithi069 ceph-mon[32746]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:22.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:21 smithi077 ceph-mon[37108]: Deploying daemon mds.l on smithi083 2024-08-02T17:43:22.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:21 smithi077 ceph-mon[37108]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:22.771 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:22 smithi083 systemd[1]: Starting Ceph mds.l for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:43:23.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:22 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/586145155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:23.070 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:22 smithi083 ceph-mon[37031]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:23.071 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:22 smithi083 podman[66362]: 2024-08-02 17:43:22.799141166 +0000 UTC m=+0.114518098 container create abe436486d2a7b5b5784aa49a82c10a8902245d6ad55f9dce489f45f9b763d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-02T17:43:23.071 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:22 smithi083 podman[66362]: 2024-08-02 17:43:22.702206874 +0000 UTC m=+0.017583760 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:23.071 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:23 smithi083 podman[66362]: 2024-08-02 17:43:23.04445617 +0000 UTC m=+0.359833063 container init abe436486d2a7b5b5784aa49a82c10a8902245d6ad55f9dce489f45f9b763d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8) 2024-08-02T17:43:23.071 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:23 smithi083 podman[66362]: 2024-08-02 17:43:23.046920138 +0000 UTC m=+0.362297025 container start abe436486d2a7b5b5784aa49a82c10a8902245d6ad55f9dce489f45f9b763d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-08-02T17:43:23.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:22 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/586145155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:23.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:22 smithi077 ceph-mon[37108]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:23.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:22 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/586145155' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:23.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:22 smithi069 ceph-mon[32746]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:23.521 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:23 smithi083 ceph-mds[66381]: -- 172.21.15.83:0/2840265920 <== mon.1 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d3f7a29860 con 0x55d3f7a07400 2024-08-02T17:43:23.521 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l[66377]: starting mds.l at 2024-08-02T17:43:23.521 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:23 smithi083 bash[66362]: abe436486d2a7b5b5784aa49a82c10a8902245d6ad55f9dce489f45f9b763d49 2024-08-02T17:43:23.521 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 17:43:23 smithi083 systemd[1]: Started Ceph mds.l for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.83:6832/2696363098,v1:172.21.15.83:6833/2696363098] up:boot 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: fsmap 1 up:standby 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:24.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:24.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:24 smithi077 ceph-mon[37108]: Deploying daemon mds.k on smithi077 2024-08-02T17:43:24.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.83:6832/2696363098,v1:172.21.15.83:6833/2696363098] up:boot 2024-08-02T17:43:24.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: fsmap 1 up:standby 2024-08-02T17:43:24.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-02T17:43:24.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:24.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:24.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:24.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:24 smithi069 ceph-mon[32746]: Deploying daemon mds.k on smithi077 2024-08-02T17:43:24.477 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.83:6832/2696363098,v1:172.21.15.83:6833/2696363098] up:boot 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: fsmap 1 up:standby 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:24.478 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:24 smithi083 ceph-mon[37031]: Deploying daemon mds.k on smithi077 2024-08-02T17:43:24.674 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:24.933 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:25.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:25 smithi077 ceph-mon[37108]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:25.348 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 systemd[1]: Starting Ceph mds.k for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:43:25.348 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:25 smithi069 ceph-mon[32746]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:25.406 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:25 smithi083 ceph-mon[37031]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:25.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 -- 172.21.15.69:0/1987780967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a98102a80 msgr2=0x7f6a98102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:25.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 --2- 172.21.15.69:0/1987780967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a98102a80 0x7f6a98102e60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009ca0 tx=0x7f6a8802a950 comp rx=0 tx=0).stop 2024-08-02T17:43:25.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 -- 172.21.15.69:0/1987780967 shutdown_connections 2024-08-02T17:43:25.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 --2- 172.21.15.69:0/1987780967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a9810beb0 0x7f6a9810e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 --2- 172.21.15.69:0/1987780967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a981033a0 0x7f6a9810b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 --2- 172.21.15.69:0/1987780967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a98102a80 0x7f6a98102e60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 -- 172.21.15.69:0/1987780967 >> 172.21.15.69:0/1987780967 conn(0x7f6a980fc500 msgr2=0x7f6a980fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 -- 172.21.15.69:0/1987780967 shutdown_connections 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.424+0000 7f6a9d486640 1 -- 172.21.15.69:0/1987780967 wait complete. 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.425+0000 7f6a9d486640 1 Processor -- start 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.425+0000 7f6a9d486640 1 -- start start 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.425+0000 7f6a9d486640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a98102a80 0x7f6a981982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.425+0000 7f6a9d486640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a981033a0 0x7f6a98198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.425+0000 7f6a9d486640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9810beb0 0x7f6a9819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.425+0000 7f6a9d486640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a98111d70 con 0x7f6a981033a0 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.425+0000 7f6a9d486640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a98111bf0 con 0x7f6a98102a80 2024-08-02T17:43:25.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.425+0000 7f6a9d486640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a98111ef0 con 0x7f6a9810beb0 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a981033a0 0x7f6a98198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a981033a0 0x7f6a98198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53100/0 (socket says 172.21.15.69:53100) 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 -- 172.21.15.69:0/3675485200 learned_addr learned my addr 172.21.15.69:0/3675485200 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a977fe640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9810beb0 0x7f6a9819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a96ffd640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a98102a80 0x7f6a981982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 -- 172.21.15.69:0/3675485200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9810beb0 msgr2=0x7f6a9819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9810beb0 0x7f6a9819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 -- 172.21.15.69:0/3675485200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a98102a80 msgr2=0x7f6a981982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a98102a80 0x7f6a981982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 -- 172.21.15.69:0/3675485200 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9819d280 con 0x7f6a981033a0 2024-08-02T17:43:25.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a967fc640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a981033a0 0x7f6a98198810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00b3f0 tx=0x7f6a8c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a7ffff640 1 -- 172.21.15.69:0/3675485200 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c015020 con 0x7f6a981033a0 2024-08-02T17:43:25.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a7ffff640 1 -- 172.21.15.69:0/3675485200 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8c011d10 con 0x7f6a981033a0 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9819d570 con 0x7f6a981033a0 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a7ffff640 1 -- 172.21.15.69:0/3675485200 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c00f2e0 con 0x7f6a981033a0 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.426+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a981a4e50 con 0x7f6a981033a0 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.427+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a98102ee0 con 0x7f6a981033a0 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.428+0000 7f6a7ffff640 1 -- 172.21.15.69:0/3675485200 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6a8c018020 con 0x7f6a981033a0 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.429+0000 7f6a7ffff640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6a800a2c00 0x7f6a800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.429+0000 7f6a96ffd640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6a800a2c00 0x7f6a800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.429+0000 7f6a7ffff640 1 -- 172.21.15.69:0/3675485200 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f6a8c0ce2c0 con 0x7f6a981033a0 2024-08-02T17:43:25.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.430+0000 7f6a96ffd640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6a800a2c00 0x7f6a800a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a880093d0 tx=0x7f6a88009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:25.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.433+0000 7f6a7ffff640 1 -- 172.21.15.69:0/3675485200 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6a8c01f020 con 0x7f6a981033a0 2024-08-02T17:43:25.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.633+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a9802d5e0 con 0x7f6a981033a0 2024-08-02T17:43:25.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.633+0000 7f6a7ffff640 1 -- 172.21.15.69:0/3675485200 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f6a8c09b6c0 con 0x7f6a981033a0 2024-08-02T17:43:25.635 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:25.635 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":2,"btime":"2024-08-02T17:43:23:074798+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-02T17:43:25.636 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 2 2024-08-02T17:43:25.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6a800a2c00 msgr2=0x7f6a800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:25.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6a800a2c00 0x7f6a800a50c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a880093d0 tx=0x7f6a88009360 comp rx=0 tx=0).stop 2024-08-02T17:43:25.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a981033a0 msgr2=0x7f6a98198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:25.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a981033a0 0x7f6a98198810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00b3f0 tx=0x7f6a8c00b8b0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 shutdown_connections 2024-08-02T17:43:25.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f6a800a2c00 0x7f6a800a50c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a981033a0 0x7f6a98198810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a9810beb0 0x7f6a9819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 --2- 172.21.15.69:0/3675485200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a98102a80 0x7f6a981982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:25.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.635+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 >> 172.21.15.69:0/3675485200 conn(0x7f6a980fc500 msgr2=0x7f6a980fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:25.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.636+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 shutdown_connections 2024-08-02T17:43:25.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:25.636+0000 7f6a9d486640 1 -- 172.21.15.69:0/3675485200 wait complete. 2024-08-02T17:43:25.748 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 podman[65834]: 2024-08-02 17:43:25.365410608 +0000 UTC m=+0.017114193 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:25.748 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 podman[65834]: 2024-08-02 17:43:25.486773434 +0000 UTC m=+0.138477008 container create 9c0962489c3bb2b9b1c82805acdf8341d72d42993f57e2ab4297df4420372a70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-08-02T17:43:25.748 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 podman[65834]: 2024-08-02 17:43:25.747529909 +0000 UTC m=+0.399233483 container init 9c0962489c3bb2b9b1c82805acdf8341d72d42993f57e2ab4297df4420372a70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0) 2024-08-02T17:43:26.088 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 podman[65834]: 2024-08-02 17:43:25.750508361 +0000 UTC m=+0.402211941 container start 9c0962489c3bb2b9b1c82805acdf8341d72d42993f57e2ab4297df4420372a70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0) 2024-08-02T17:43:26.088 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 ceph-mds[65853]: -- 172.21.15.77:0/1438948502 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c61d4f860 con 0x561c61d2d000 2024-08-02T17:43:26.088 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k[65849]: starting mds.k at 2024-08-02T17:43:26.088 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 bash[65834]: 9c0962489c3bb2b9b1c82805acdf8341d72d42993f57e2ab4297df4420372a70 2024-08-02T17:43:26.088 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 17:43:25 smithi077 systemd[1]: Started Ceph mds.k for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:26.277 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:26 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3675485200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:26.277 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.277 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.277 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.277 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:26.277 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:26.277 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:26 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:26 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3675485200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:26.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:26.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:26 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:26 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3675485200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:26 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:27 smithi069 ceph-mon[32746]: Deploying daemon mds.j on smithi069 2024-08-02T17:43:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:27 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6832/2071376346,v1:172.21.15.77:6833/2071376346] up:boot 2024-08-02T17:43:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:27 smithi069 ceph-mon[32746]: fsmap 2 up:standby 2024-08-02T17:43:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:27 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T17:43:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:27 smithi069 ceph-mon[32746]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:27.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:27 smithi083 ceph-mon[37031]: Deploying daemon mds.j on smithi069 2024-08-02T17:43:27.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:27 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6832/2071376346,v1:172.21.15.77:6833/2071376346] up:boot 2024-08-02T17:43:27.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:27 smithi083 ceph-mon[37031]: fsmap 2 up:standby 2024-08-02T17:43:27.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:27 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T17:43:27.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:27 smithi083 ceph-mon[37031]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:27.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:27 smithi077 ceph-mon[37108]: Deploying daemon mds.j on smithi069 2024-08-02T17:43:27.689 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:27 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6832/2071376346,v1:172.21.15.77:6833/2071376346] up:boot 2024-08-02T17:43:27.689 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:27 smithi077 ceph-mon[37108]: fsmap 2 up:standby 2024-08-02T17:43:27.689 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:27 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T17:43:27.689 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:27 smithi077 ceph-mon[37108]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:28.092 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 systemd[1]: Starting Ceph mds.j for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:43:28.304 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:28.358 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 podman[69053]: 2024-08-02 17:43:28.166484022 +0000 UTC m=+0.018187413 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:28.358 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 podman[69053]: 2024-08-02 17:43:28.308554959 +0000 UTC m=+0.160258350 container create fec88f0b3439c29af92724fe38c2388885ebdb4ab9006e51f1aa3034525cde3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-02T17:43:28.690 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:28.817 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 podman[69053]: 2024-08-02 17:43:28.53909501 +0000 UTC m=+0.390798419 container init fec88f0b3439c29af92724fe38c2388885ebdb4ab9006e51f1aa3034525cde3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-02T17:43:28.817 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 podman[69053]: 2024-08-02 17:43:28.543276102 +0000 UTC m=+0.394979494 container start fec88f0b3439c29af92724fe38c2388885ebdb4ab9006e51f1aa3034525cde3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-02T17:43:28.817 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 ceph-mds[69112]: -- 172.21.15.69:0/2316217722 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa9d38f860 con 0x55aa9d36d400 2024-08-02T17:43:28.817 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j[69094]: starting mds.j at 2024-08-02T17:43:28.817 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 bash[69053]: fec88f0b3439c29af92724fe38c2388885ebdb4ab9006e51f1aa3034525cde3b 2024-08-02T17:43:28.818 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 17:43:28 smithi069 systemd[1]: Started Ceph mds.j for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 -- 172.21.15.69:0/2591264046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 msgr2=0x7fd3dc106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/2591264046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 0x7fd3dc106550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009930 tx=0x7fd3d002a420 comp rx=0 tx=0).stop 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 -- 172.21.15.69:0/2591264046 shutdown_connections 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/2591264046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 0x7fd3dc106550 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/2591264046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3dc106c60 0x7fd3dc10e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/2591264046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc100060 0x7fd3dc100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 -- 172.21.15.69:0/2591264046 >> 172.21.15.69:0/2591264046 conn(0x7fd3dc0fbc10 msgr2=0x7fd3dc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 -- 172.21.15.69:0/2591264046 shutdown_connections 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.215+0000 7fd3e29d0640 1 -- 172.21.15.69:0/2591264046 wait complete. 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3e29d0640 1 Processor -- start 2024-08-02T17:43:29.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3e29d0640 1 -- start start 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3e29d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3dc100060 0x7fd3dc198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3e29d0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 0x7fd3dc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3e29d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc106c60 0x7fd3dc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3e29d0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc111cf0 con 0x7fd3dc100a10 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3e29d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc111b70 con 0x7fd3dc100060 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3e29d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3dc111e70 con 0x7fd3dc106c60 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3db7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 0x7fd3dc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3db7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 0x7fd3dc198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53146/0 (socket says 172.21.15.69:53146) 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3db7fe640 1 -- 172.21.15.69:0/222091558 learned_addr learned my addr 172.21.15.69:0/222091558 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3dbfff640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3dc100060 0x7fd3dc198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3db7fe640 1 -- 172.21.15.69:0/222091558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc106c60 msgr2=0x7fd3dc19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:29.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3db7fe640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc106c60 0x7fd3dc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3db7fe640 1 -- 172.21.15.69:0/222091558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3dc100060 msgr2=0x7fd3dc198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.216+0000 7fd3db7fe640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3dc100060 0x7fd3dc198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.217+0000 7fd3db7fe640 1 -- 172.21.15.69:0/222091558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3dc19d2c0 con 0x7fd3dc100a10 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.217+0000 7fd3dbfff640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3dc100060 0x7fd3dc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.217+0000 7fd3db7fe640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 0x7fd3dc198850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009230 tx=0x7fd3d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.217+0000 7fd3d97fa640 1 -- 172.21.15.69:0/222091558 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d003f030 con 0x7fd3dc100a10 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.217+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3dc19d550 con 0x7fd3dc100a10 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.217+0000 7fd3d97fa640 1 -- 172.21.15.69:0/222091558 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3d0003ce0 con 0x7fd3dc100a10 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.217+0000 7fd3d97fa640 1 -- 172.21.15.69:0/222091558 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0035070 con 0x7fd3dc100a10 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.217+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3dc1a4e90 con 0x7fd3dc100a10 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.218+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3dc100440 con 0x7fd3dc100a10 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.219+0000 7fd3d97fa640 1 -- 172.21.15.69:0/222091558 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3d0042030 con 0x7fd3dc100a10 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.219+0000 7fd3d97fa640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd3c40a2b80 0x7fd3c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:29.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.220+0000 7fd3dbfff640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd3c40a2b80 0x7fd3c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:29.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.220+0000 7fd3d97fa640 1 -- 172.21.15.69:0/222091558 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fd3d00ef600 con 0x7fd3dc100a10 2024-08-02T17:43:29.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.220+0000 7fd3dbfff640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd3c40a2b80 0x7fd3c40a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009930 tx=0x7fd3cc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:29.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.223+0000 7fd3d97fa640 1 -- 172.21.15.69:0/222091558 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd3d00bca00 con 0x7fd3dc100a10 2024-08-02T17:43:29.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.438+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3dc109e30 con 0x7fd3dc100a10 2024-08-02T17:43:29.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.439+0000 7fd3d97fa640 1 -- 172.21.15.69:0/222091558 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7fd3d00bc300 con 0x7fd3dc100a10 2024-08-02T17:43:29.441 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:29.441 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":4,"btime":"2024-08-02T17:43:28:623334+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-02T17:43:29.441 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 4 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd3c40a2b80 msgr2=0x7fd3c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd3c40a2b80 0x7fd3c40a5040 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009930 tx=0x7fd3cc008040 comp rx=0 tx=0).stop 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 msgr2=0x7fd3dc198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 0x7fd3dc198850 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0009230 tx=0x7fd3d00042f0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 shutdown_connections 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd3c40a2b80 0x7fd3c40a5040 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd3dc100a10 0x7fd3dc198850 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd3dc106c60 0x7fd3dc19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 --2- 172.21.15.69:0/222091558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd3dc100060 0x7fd3dc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 >> 172.21.15.69:0/222091558 conn(0x7fd3dc0fbc10 msgr2=0x7fd3dc0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:29.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 shutdown_connections 2024-08-02T17:43:29.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:29.442+0000 7fd3e29d0640 1 -- 172.21.15.69:0/222091558 wait complete. 2024-08-02T17:43:29.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.69:6834/570537244,v1:172.21.15.69:6835/570537244] up:boot 2024-08-02T17:43:29.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: fsmap 3 up:standby 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: Deploying daemon mds.d on smithi069 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:29.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:29 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/222091558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.69:6834/570537244,v1:172.21.15.69:6835/570537244] up:boot 2024-08-02T17:43:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: fsmap 3 up:standby 2024-08-02T17:43:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-02T17:43:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:30.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:30.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:30.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: Deploying daemon mds.d on smithi069 2024-08-02T17:43:30.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:30.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:29 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/222091558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.69:6834/570537244,v1:172.21.15.69:6835/570537244] up:boot 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: fsmap 3 up:standby 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:30.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: Deploying daemon mds.d on smithi069 2024-08-02T17:43:30.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:30.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:29 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/222091558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:31.608 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:31 smithi069 systemd[1]: Starting Ceph mds.d for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:43:31.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:31 smithi069 ceph-mon[32746]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:31.943 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:31 smithi069 podman[69478]: 2024-08-02 17:43:31.565975089 +0000 UTC m=+0.017073556 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:31.943 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:31 smithi069 podman[69478]: 2024-08-02 17:43:31.682042773 +0000 UTC m=+0.133141299 container create 89156af5b8463ba83b91ca7115134f50166762162393ad6c133694bb17be52bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-02T17:43:31.943 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:31 smithi069 podman[69478]: 2024-08-02 17:43:31.909626645 +0000 UTC m=+0.360725102 container init 89156af5b8463ba83b91ca7115134f50166762162393ad6c133694bb17be52bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-02T17:43:31.943 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:31 smithi069 podman[69478]: 2024-08-02 17:43:31.912586995 +0000 UTC m=+0.363685455 container start 89156af5b8463ba83b91ca7115134f50166762162393ad6c133694bb17be52bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-02T17:43:32.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:31 smithi077 ceph-mon[37108]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:32.188 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:32.212 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:31 smithi069 ceph-mds[69497]: -- 172.21.15.69:0/4247680954 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f3d3091860 con 0x55f3d306f000 2024-08-02T17:43:32.213 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d[69493]: starting mds.d at 2024-08-02T17:43:32.213 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:32 smithi069 bash[69478]: 89156af5b8463ba83b91ca7115134f50166762162393ad6c133694bb17be52bd 2024-08-02T17:43:32.213 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 17:43:32 smithi069 systemd[1]: Started Ceph mds.d for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:31 smithi083 ceph-mon[37031]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:32.454 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:32.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 -- 172.21.15.69:0/3585430317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72441007b0 msgr2=0x7f7244100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 --2- 172.21.15.69:0/3585430317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72441007b0 0x7f7244100c10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f722c009930 tx=0x7f722c02a420 comp rx=0 tx=0).stop 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 -- 172.21.15.69:0/3585430317 shutdown_connections 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 --2- 172.21.15.69:0/3585430317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f72441007b0 0x7f7244100c10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 --2- 172.21.15.69:0/3585430317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7244101150 0x7f724410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 --2- 172.21.15.69:0/3585430317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72441067b0 0x7f7244106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 -- 172.21.15.69:0/3585430317 >> 172.21.15.69:0/3585430317 conn(0x7f72440fc4d0 msgr2=0x7f72440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 -- 172.21.15.69:0/3585430317 shutdown_connections 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 -- 172.21.15.69:0/3585430317 wait complete. 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 Processor -- start 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.960+0000 7f724afc9640 1 -- start start 2024-08-02T17:43:32.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f724afc9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72441007b0 0x7f7244197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f724afc9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7244101150 0x7f72441975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f724afc9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72441067b0 0x7f724419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f724afc9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7244110bb0 con 0x7f7244101150 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f724afc9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7244110a30 con 0x7f72441007b0 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f724afc9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7244110d30 con 0x7f72441067b0 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7244101150 0x7f72441975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7244101150 0x7f72441975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53174/0 (socket says 172.21.15.69:53174) 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 -- 172.21.15.69:0/4145544498 learned_addr learned my addr 172.21.15.69:0/4145544498 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7248d65640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72441007b0 0x7f7244197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 -- 172.21.15.69:0/4145544498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72441067b0 msgr2=0x7f724419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72441067b0 0x7f724419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 -- 172.21.15.69:0/4145544498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72441007b0 msgr2=0x7f7244197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:32.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72441007b0 0x7f7244197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 -- 172.21.15.69:0/4145544498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724419c010 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7243fff640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7244101150 0x7f72441975a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f722c009230 tx=0x7f722c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7241ffb640 1 -- 172.21.15.69:0/4145544498 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c03f030 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724419c2a0 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7241ffb640 1 -- 172.21.15.69:0/4145544498 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f722c003ad0 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f7241ffb640 1 -- 172.21.15.69:0/4145544498 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722c035070 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.961+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72441a3be0 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.963+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7244100c10 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.964+0000 7f7241ffb640 1 -- 172.21.15.69:0/4145544498 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f722c042030 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.964+0000 7f7241ffb640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f72300a2b80 0x7f72300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.964+0000 7f7248d65640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f72300a2b80 0x7f72300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.964+0000 7f7241ffb640 1 -- 172.21.15.69:0/4145544498 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f722c0ef490 con 0x7f7244101150 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.965+0000 7f7248d65640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f72300a2b80 0x7f72300a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f723400bb60 tx=0x7f723400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:32.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:32.968+0000 7f7241ffb640 1 -- 172.21.15.69:0/4145544498 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f722c0bc890 con 0x7f7244101150 2024-08-02T17:43:33.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.168+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72441983c0 con 0x7f7244101150 2024-08-02T17:43:33.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.168+0000 7f7241ffb640 1 -- 172.21.15.69:0/4145544498 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f72441983c0 con 0x7f7244101150 2024-08-02T17:43:33.170 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:33.170 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":5,"btime":"2024-08-02T17:43:32:814283+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-02T17:43:33.171 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 5 2024-08-02T17:43:33.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.170+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f72300a2b80 msgr2=0x7f72300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:33.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f72300a2b80 0x7f72300a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f723400bb60 tx=0x7f723400b390 comp rx=0 tx=0).stop 2024-08-02T17:43:33.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7244101150 msgr2=0x7f72441975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:33.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7244101150 0x7f72441975a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f722c009230 tx=0x7f722c004270 comp rx=0 tx=0).stop 2024-08-02T17:43:33.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 shutdown_connections 2024-08-02T17:43:33.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f72300a2b80 0x7f72300a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:33.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7244101150 0x7f72441975a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:33.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72441067b0 0x7f724419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:33.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 --2- 172.21.15.69:0/4145544498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f72441007b0 0x7f7244197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:33.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 >> 172.21.15.69:0/4145544498 conn(0x7f72440fc4d0 msgr2=0x7f72440fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:33.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 shutdown_connections 2024-08-02T17:43:33.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:33.171+0000 7f724afc9640 1 -- 172.21.15.69:0/4145544498 wait complete. 2024-08-02T17:43:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: Deploying daemon mds.e on smithi077 2024-08-02T17:43:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.69:6836/3783033503,v1:172.21.15.69:6837/3783033503] up:boot 2024-08-02T17:43:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: fsmap 4 up:standby 2024-08-02T17:43:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-02T17:43:33.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:33 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4145544498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: Deploying daemon mds.e on smithi077 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:33.543 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.69:6836/3783033503,v1:172.21.15.69:6837/3783033503] up:boot 2024-08-02T17:43:33.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: fsmap 4 up:standby 2024-08-02T17:43:33.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-02T17:43:33.544 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:33 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4145544498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:33.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: Deploying daemon mds.e on smithi077 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.69:6836/3783033503,v1:172.21.15.69:6837/3783033503] up:boot 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: fsmap 4 up:standby 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-02T17:43:33.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:33 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4145544498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:35.012 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 17:43:34 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e[66162]: starting mds.e at 2024-08-02T17:43:35.347 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 17:43:35 smithi077 bash[66147]: c94197eae4353ce0895c8e75f6e88a8e0b1be49bdf2c4393ed14a422b7f87457 2024-08-02T17:43:35.347 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 17:43:35 smithi077 systemd[1]: Started Ceph mds.e for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:35.880 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:36.068 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:35 smithi077 ceph-mon[37108]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:36.068 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:35 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.068 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:35 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.068 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:35 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.068 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:35 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:36.068 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:35 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:36.068 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:35 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:35 smithi069 ceph-mon[32746]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:35 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:35 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:35 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:35 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:35 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:36.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:35 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:36.145 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:35 smithi083 ceph-mon[37031]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:35 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:35 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:35 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:35 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:35 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:36.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:35 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:36.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.598+0000 7f29c630b640 1 -- 172.21.15.69:0/383831014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c01007b0 msgr2=0x7f29c0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:36.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 --2- 172.21.15.69:0/383831014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c01007b0 0x7f29c0100c10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009930 tx=0x7f29b402a420 comp rx=0 tx=0).stop 2024-08-02T17:43:36.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 -- 172.21.15.69:0/383831014 shutdown_connections 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 --2- 172.21.15.69:0/383831014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c01007b0 0x7f29c0100c10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 --2- 172.21.15.69:0/383831014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c0101150 0x7f29c010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 --2- 172.21.15.69:0/383831014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c01067b0 0x7f29c0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 -- 172.21.15.69:0/383831014 >> 172.21.15.69:0/383831014 conn(0x7f29c00fc4d0 msgr2=0x7f29c00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 -- 172.21.15.69:0/383831014 shutdown_connections 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 -- 172.21.15.69:0/383831014 wait complete. 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 Processor -- start 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.599+0000 7f29c630b640 1 -- start start 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c630b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c01007b0 0x7f29c01970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c630b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c0101150 0x7f29c0197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c630b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c01067b0 0x7f29c019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:36.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c630b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0110bb0 con 0x7f29c0101150 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c630b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0110a30 con 0x7f29c01007b0 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c630b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f29c0110d30 con 0x7f29c01067b0 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c0101150 0x7f29c0197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c0101150 0x7f29c0197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56966/0 (socket says 172.21.15.69:56966) 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 -- 172.21.15.69:0/3227990751 learned_addr learned my addr 172.21.15.69:0/3227990751 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bffff640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c01007b0 0x7f29c01970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c48a8640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c01067b0 0x7f29c019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 -- 172.21.15.69:0/3227990751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c01067b0 msgr2=0x7f29c019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c01067b0 0x7f29c019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 -- 172.21.15.69:0/3227990751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c01007b0 msgr2=0x7f29c01970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c01007b0 0x7f29c01970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 -- 172.21.15.69:0/3227990751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c019c090 con 0x7f29c0101150 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bffff640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c01007b0 0x7f29c01970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:43:36.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c48a8640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c01067b0 0x7f29c019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bf7fe640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c0101150 0x7f29c0197620 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009a50 tx=0x7f29b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bd7fa640 1 -- 172.21.15.69:0/3227990751 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b403f030 con 0x7f29c0101150 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bd7fa640 1 -- 172.21.15.69:0/3227990751 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b4003ce0 con 0x7f29c0101150 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c019c320 con 0x7f29c0101150 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29bd7fa640 1 -- 172.21.15.69:0/3227990751 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b4035070 con 0x7f29c0101150 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.600+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c01a3c60 con 0x7f29c0101150 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.602+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c0101ef0 con 0x7f29c0101150 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.602+0000 7f29bd7fa640 1 -- 172.21.15.69:0/3227990751 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f29b4042030 con 0x7f29c0101150 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.603+0000 7f29bd7fa640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f29a80a2b80 0x7f29a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.603+0000 7f29bffff640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f29a80a2b80 0x7f29a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.603+0000 7f29bd7fa640 1 -- 172.21.15.69:0/3227990751 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f29b40ef1f0 con 0x7f29c0101150 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.604+0000 7f29bffff640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f29a80a2b80 0x7f29a80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005d90 tx=0x7f29b0005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:36.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.607+0000 7f29bd7fa640 1 -- 172.21.15.69:0/3227990751 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f29b40bc5f0 con 0x7f29c0101150 2024-08-02T17:43:36.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.807+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29c0108d20 con 0x7f29c0101150 2024-08-02T17:43:36.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.807+0000 7f29bd7fa640 1 -- 172.21.15.69:0/3227990751 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4732 (secure 0 0 0) 0x7f29b40bbef0 con 0x7f29c0101150 2024-08-02T17:43:36.809 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:36.809 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":6,"btime":"2024-08-02T17:43:35:802182+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-02T17:43:36.810 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 6 2024-08-02T17:43:36.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f29a80a2b80 msgr2=0x7f29a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:36.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f29a80a2b80 0x7f29a80a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f29b0005d90 tx=0x7f29b0005d20 comp rx=0 tx=0).stop 2024-08-02T17:43:36.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c0101150 msgr2=0x7f29c0197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:36.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c0101150 0x7f29c0197620 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f29b4009a50 tx=0x7f29b4004270 comp rx=0 tx=0).stop 2024-08-02T17:43:36.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 shutdown_connections 2024-08-02T17:43:36.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f29a80a2b80 0x7f29a80a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f29c0101150 0x7f29c0197620 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f29c01067b0 0x7f29c019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 --2- 172.21.15.69:0/3227990751 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f29c01007b0 0x7f29c01970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:36.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 >> 172.21.15.69:0/3227990751 conn(0x7f29c00fc4d0 msgr2=0x7f29c00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:36.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 shutdown_connections 2024-08-02T17:43:36.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:36.810+0000 7f29c630b640 1 -- 172.21.15.69:0/3227990751 wait complete. 2024-08-02T17:43:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:36 smithi077 ceph-mon[37108]: Deploying daemon mds.f on smithi083 2024-08-02T17:43:37.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:36 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6834/3410285190,v1:172.21.15.77:6835/3410285190] up:boot 2024-08-02T17:43:37.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:36 smithi077 ceph-mon[37108]: fsmap 5 up:standby 2024-08-02T17:43:37.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:36 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T17:43:37.101 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:36 smithi083 ceph-mon[37031]: Deploying daemon mds.f on smithi083 2024-08-02T17:43:37.101 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:36 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6834/3410285190,v1:172.21.15.77:6835/3410285190] up:boot 2024-08-02T17:43:37.101 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:36 smithi083 ceph-mon[37031]: fsmap 5 up:standby 2024-08-02T17:43:37.101 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:36 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T17:43:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:36 smithi069 ceph-mon[32746]: Deploying daemon mds.f on smithi083 2024-08-02T17:43:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:36 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6834/3410285190,v1:172.21.15.77:6835/3410285190] up:boot 2024-08-02T17:43:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:36 smithi069 ceph-mon[32746]: fsmap 5 up:standby 2024-08-02T17:43:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:36 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T17:43:37.357 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 17:43:37 smithi083 podman[66679]: 2024-08-02 17:43:37.22790799 +0000 UTC m=+0.017395876 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:37.357 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 17:43:37 smithi083 podman[66679]: 2024-08-02 17:43:37.35595567 +0000 UTC m=+0.145443533 container create 51c056c7d810adbad40e2bddd54366451d60472557fe91932125dd85311d6bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS) 2024-08-02T17:43:37.611 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 17:43:37 smithi083 podman[66679]: 2024-08-02 17:43:37.584106958 +0000 UTC m=+0.373594836 container init 51c056c7d810adbad40e2bddd54366451d60472557fe91932125dd85311d6bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-08-02T17:43:37.611 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 17:43:37 smithi083 podman[66679]: 2024-08-02 17:43:37.587151296 +0000 UTC m=+0.376639162 container start 51c056c7d810adbad40e2bddd54366451d60472557fe91932125dd85311d6bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-02T17:43:38.021 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mds[66706]: -- 172.21.15.83:0/682368602 <== mon.1 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d1aacd860 con 0x563d1aaaa800 2024-08-02T17:43:38.021 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f[66697]: starting mds.f at 2024-08-02T17:43:38.021 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 17:43:37 smithi083 bash[66679]: 51c056c7d810adbad40e2bddd54366451d60472557fe91932125dd85311d6bf2 2024-08-02T17:43:38.021 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 17:43:37 smithi083 systemd[1]: Started Ceph mds.f for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:38.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mon[37031]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:38.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3227990751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:38.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:38.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:38.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:37 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:37 smithi077 ceph-mon[37108]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:37 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3227990751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:38.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:37 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:37 smithi069 ceph-mon[32746]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:37 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3227990751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:38.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:37 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:39.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:38 smithi077 ceph-mon[37108]: Deploying daemon mds.c on smithi083 2024-08-02T17:43:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:38 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.83:6834/3991686236,v1:172.21.15.83:6835/3991686236] up:boot 2024-08-02T17:43:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:38 smithi077 ceph-mon[37108]: fsmap 6 up:standby 2024-08-02T17:43:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:38 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T17:43:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:38 smithi069 ceph-mon[32746]: Deploying daemon mds.c on smithi083 2024-08-02T17:43:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:38 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.83:6834/3991686236,v1:172.21.15.83:6835/3991686236] up:boot 2024-08-02T17:43:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:38 smithi069 ceph-mon[32746]: fsmap 6 up:standby 2024-08-02T17:43:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:38 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T17:43:39.168 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:38 smithi083 ceph-mon[37031]: Deploying daemon mds.c on smithi083 2024-08-02T17:43:39.169 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:38 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.83:6834/3991686236,v1:172.21.15.83:6835/3991686236] up:boot 2024-08-02T17:43:39.169 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:38 smithi083 ceph-mon[37031]: fsmap 6 up:standby 2024-08-02T17:43:39.169 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:38 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T17:43:39.474 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:39.733 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:39.771 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:39 smithi083 systemd[1]: Starting Ceph mds.c for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:43:40.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:39 smithi077 ceph-mon[37108]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:39 smithi069 ceph-mon[32746]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:40.132 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:39 smithi083 ceph-mon[37031]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:40.132 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:39 smithi083 podman[66994]: 2024-08-02 17:43:39.763572406 +0000 UTC m=+0.017140828 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:40.132 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:39 smithi083 podman[66994]: 2024-08-02 17:43:39.878576542 +0000 UTC m=+0.132144996 container create 14dc08ae1bf9a07b51ef2542fcf3bc8c71c27a5fbfed81c51d590af975dcc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T17:43:40.132 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:40 smithi083 podman[66994]: 2024-08-02 17:43:40.131542259 +0000 UTC m=+0.385110678 container init 14dc08ae1bf9a07b51ef2542fcf3bc8c71c27a5fbfed81c51d590af975dcc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS) 2024-08-02T17:43:40.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1333086554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd05810bea0 msgr2=0x7fd05810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:40.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1333086554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd05810bea0 0x7fd05810e340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd050009ca0 tx=0x7fd05002a950 comp rx=0 tx=0).stop 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1333086554 shutdown_connections 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1333086554 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd05810bea0 0x7fd05810e340 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1333086554 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd058103390 0x7fd05810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1333086554 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd058102a70 0x7fd058102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1333086554 >> 172.21.15.69:0/1333086554 conn(0x7fd0580fc4d0 msgr2=0x7fd0580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1333086554 shutdown_connections 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.230+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1333086554 wait complete. 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05f4b7640 1 Processor -- start 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05f4b7640 1 -- start start 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05f4b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd058102a70 0x7fd0581980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05f4b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd058103390 0x7fd0581985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:40.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05f4b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd05810bea0 0x7fd05819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05f4b7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111c80 con 0x7fd058103390 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05f4b7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111b00 con 0x7fd05810bea0 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05f4b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111e00 con 0x7fd058102a70 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05ca52640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd058103390 0x7fd0581985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.231+0000 7fd05d253640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd058102a70 0x7fd0581980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05ca52640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd058103390 0x7fd0581985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56978/0 (socket says 172.21.15.69:56978) 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05ca52640 1 -- 172.21.15.69:0/1798255599 learned_addr learned my addr 172.21.15.69:0/1798255599 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05da54640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd05810bea0 0x7fd05819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05ca52640 1 -- 172.21.15.69:0/1798255599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd058102a70 msgr2=0x7fd0581980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05ca52640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd058102a70 0x7fd0581980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05ca52640 1 -- 172.21.15.69:0/1798255599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd05810bea0 msgr2=0x7fd05819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05ca52640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd05810bea0 0x7fd05819c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05ca52640 1 -- 172.21.15.69:0/1798255599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05819d050 con 0x7fd058103390 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05d253640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd058102a70 0x7fd0581980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05ca52640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd058103390 0x7fd0581985e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e9f0 tx=0x7fd04800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd04e7fc640 1 -- 172.21.15.69:0/1798255599 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04800cab0 con 0x7fd058103390 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd04e7fc640 1 -- 172.21.15.69:0/1798255599 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd048005500 con 0x7fd058103390 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05819d340 con 0x7fd058103390 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd04e7fc640 1 -- 172.21.15.69:0/1798255599 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048010430 con 0x7fd058103390 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.232+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0581a4c20 con 0x7fd058103390 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.233+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd058102e50 con 0x7fd058103390 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.234+0000 7fd04e7fc640 1 -- 172.21.15.69:0/1798255599 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd048004fb0 con 0x7fd058103390 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.235+0000 7fd04e7fc640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd0440a2c00 0x7fd0440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.235+0000 7fd05d253640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd0440a2c00 0x7fd0440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:40.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.235+0000 7fd04e7fc640 1 -- 172.21.15.69:0/1798255599 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fd0480cf050 con 0x7fd058103390 2024-08-02T17:43:40.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.235+0000 7fd05d253640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd0440a2c00 0x7fd0440a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd05400afb0 tx=0x7fd05400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:40.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.239+0000 7fd04e7fc640 1 -- 172.21.15.69:0/1798255599 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd04809c450 con 0x7fd058103390 2024-08-02T17:43:40.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.446+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0581a4f50 con 0x7fd058103390 2024-08-02T17:43:40.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.446+0000 7fd04e7fc640 1 -- 172.21.15.69:0/1798255599 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5557 (secure 0 0 0) 0x7fd04809bd50 con 0x7fd058103390 2024-08-02T17:43:40.448 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:40.448 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":7,"btime":"2024-08-02T17:43:37:813077+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-02T17:43:40.448 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 7 2024-08-02T17:43:40.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.448+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd0440a2c00 msgr2=0x7fd0440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:40.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.448+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd0440a2c00 0x7fd0440a50c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd05400afb0 tx=0x7fd05400a2d0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.448+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd058103390 msgr2=0x7fd0581985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:40.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.448+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd058103390 0x7fd0581985e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd04800e9f0 tx=0x7fd04800edf0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.449+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 shutdown_connections 2024-08-02T17:43:40.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.449+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd0440a2c00 0x7fd0440a50c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.449+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd058103390 0x7fd0581985e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.449+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd05810bea0 0x7fd05819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.449+0000 7fd05f4b7640 1 --2- 172.21.15.69:0/1798255599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd058102a70 0x7fd0581980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:40.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.449+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 >> 172.21.15.69:0/1798255599 conn(0x7fd0580fc4d0 msgr2=0x7fd0580fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:40.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.449+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 shutdown_connections 2024-08-02T17:43:40.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:40.449+0000 7fd05f4b7640 1 -- 172.21.15.69:0/1798255599 wait complete. 2024-08-02T17:43:40.521 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:40 smithi083 podman[66994]: 2024-08-02 17:43:40.134285317 +0000 UTC m=+0.387853733 container start 14dc08ae1bf9a07b51ef2542fcf3bc8c71c27a5fbfed81c51d590af975dcc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T17:43:40.521 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:40 smithi083 ceph-mds[67014]: -- 172.21.15.83:0/954406681 <== mon.1 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638791ed860 con 0x5638791ca800 2024-08-02T17:43:40.521 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:40 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c[67010]: starting mds.c at 2024-08-02T17:43:40.522 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:40 smithi083 bash[66994]: 14dc08ae1bf9a07b51ef2542fcf3bc8c71c27a5fbfed81c51d590af975dcc37c 2024-08-02T17:43:40.522 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 17:43:40 smithi083 systemd[1]: Started Ceph mds.c for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: Deploying daemon mds.h on smithi077 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1798255599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:41.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:41.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.83:6836/1744999893,v1:172.21.15.83:6837/1744999893] up:boot 2024-08-02T17:43:41.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: fsmap 7 up:standby 2024-08-02T17:43:41.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:41 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T17:43:41.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:41.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:41.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:41.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: Deploying daemon mds.h on smithi077 2024-08-02T17:43:41.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1798255599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:41.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:41.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.83:6836/1744999893,v1:172.21.15.83:6837/1744999893] up:boot 2024-08-02T17:43:41.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: fsmap 7 up:standby 2024-08-02T17:43:41.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:41 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T17:43:41.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:41.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:41.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:41.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:41.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: Deploying daemon mds.h on smithi077 2024-08-02T17:43:41.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1798255599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:41.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:41.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.83:6836/1744999893,v1:172.21.15.83:6837/1744999893] up:boot 2024-08-02T17:43:41.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: fsmap 7 up:standby 2024-08-02T17:43:41.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:41 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T17:43:42.598 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:42 smithi077 systemd[1]: Starting Ceph mds.h for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:43:42.893 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:42 smithi077 podman[66449]: 2024-08-02 17:43:42.507577371 +0000 UTC m=+0.017106853 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:42.893 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:42 smithi077 podman[66449]: 2024-08-02 17:43:42.622127709 +0000 UTC m=+0.131657173 container create 17ca429dd59d2e019838574f387b507ed3bba61bb468c2ebf0a6e171b9290c5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-02T17:43:42.893 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:42 smithi077 podman[66449]: 2024-08-02 17:43:42.891451373 +0000 UTC m=+0.400980852 container init 17ca429dd59d2e019838574f387b507ed3bba61bb468c2ebf0a6e171b9290c5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-02T17:43:43.147 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:43.348 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:42 smithi077 podman[66449]: 2024-08-02 17:43:42.894411746 +0000 UTC m=+0.403941218 container start 17ca429dd59d2e019838574f387b507ed3bba61bb468c2ebf0a6e171b9290c5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-02T17:43:43.348 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:42 smithi077 ceph-mds[66468]: -- 172.21.15.77:0/2467783093 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560098045860 con 0x560098023000 2024-08-02T17:43:43.348 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:42 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h[66464]: starting mds.h at 2024-08-02T17:43:43.348 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:43 smithi077 bash[66449]: 17ca429dd59d2e019838574f387b507ed3bba61bb468c2ebf0a6e171b9290c5f 2024-08-02T17:43:43.348 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 17:43:43 smithi077 systemd[1]: Started Ceph mds.h for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:43.407 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:43.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 -- 172.21.15.69:0/3138977074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc1007a0 msgr2=0x7fc1dc100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:43.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/3138977074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc1007a0 0x7fc1dc100c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009930 tx=0x7fc1cc02a420 comp rx=0 tx=0).stop 2024-08-02T17:43:43.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 -- 172.21.15.69:0/3138977074 shutdown_connections 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/3138977074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc1007a0 0x7fc1dc100c00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/3138977074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1dc101140 0x7fc1dc10d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/3138977074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1dc1067a0 0x7fc1dc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 -- 172.21.15.69:0/3138977074 >> 172.21.15.69:0/3138977074 conn(0x7fc1dc0fc4c0 msgr2=0x7fc1dc0fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 -- 172.21.15.69:0/3138977074 shutdown_connections 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.884+0000 7fc1e3f77640 1 -- 172.21.15.69:0/3138977074 wait complete. 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e3f77640 1 Processor -- start 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e3f77640 1 -- start start 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e3f77640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1dc1007a0 0x7fc1dc199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:43.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e3f77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc101140 0x7fc1dc199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e3f77640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1dc1067a0 0x7fc1dc1a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e3f77640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc110ba0 con 0x7fc1dc101140 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e3f77640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc110a20 con 0x7fc1dc1007a0 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e3f77640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc110d20 con 0x7fc1dc1067a0 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e1512640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc101140 0x7fc1dc199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e1512640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc101140 0x7fc1dc199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56996/0 (socket says 172.21.15.69:56996) 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e1d13640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1dc1007a0 0x7fc1dc199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e1512640 1 -- 172.21.15.69:0/2256863832 learned_addr learned my addr 172.21.15.69:0/2256863832 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:43.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.885+0000 7fc1e2514640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1dc1067a0 0x7fc1dc1a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e1d13640 1 -- 172.21.15.69:0/2256863832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1dc1067a0 msgr2=0x7fc1dc1a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e1d13640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1dc1067a0 0x7fc1dc1a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e1d13640 1 -- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc101140 msgr2=0x7fc1dc199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e1d13640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc101140 0x7fc1dc199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e1d13640 1 -- 172.21.15.69:0/2256863832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc1a45f0 con 0x7fc1dc1007a0 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e1512640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc101140 0x7fc1dc199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e1d13640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1dc1007a0 0x7fc1dc199030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8005fd0 tx=0x7fc1d800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1d2ffd640 1 -- 172.21.15.69:0/2256863832 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8009c30 con 0x7fc1dc1007a0 2024-08-02T17:43:43.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc196e20 con 0x7fc1dc1007a0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1d2ffd640 1 -- 172.21.15.69:0/2256863832 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1d8009dd0 con 0x7fc1dc1007a0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1d2ffd640 1 -- 172.21.15.69:0/2256863832 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8010460 con 0x7fc1dc1007a0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.886+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc197310 con 0x7fc1dc1007a0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.887+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc100c00 con 0x7fc1dc1007a0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.888+0000 7fc1d2ffd640 1 -- 172.21.15.69:0/2256863832 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc1d8003ef0 con 0x7fc1dc1007a0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.889+0000 7fc1d2ffd640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1c80a2b80 0x7fc1c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.889+0000 7fc1e1512640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1c80a2b80 0x7fc1c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.889+0000 7fc1d2ffd640 1 -- 172.21.15.69:0/2256863832 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc1d80ce5d0 con 0x7fc1dc1007a0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.889+0000 7fc1e1512640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1c80a2b80 0x7fc1c80a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc06b010 tx=0x7fc1cc009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:43.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:43.892+0000 7fc1d2ffd640 1 -- 172.21.15.69:0/2256863832 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc1d8017020 con 0x7fc1dc1007a0 2024-08-02T17:43:44.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:43 smithi069 ceph-mon[32746]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:44.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.070 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:44.071 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:44.071 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:43 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:44.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.095+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1dc108d30 con 0x7fc1dc1007a0 2024-08-02T17:43:44.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.096+0000 7fc1d2ffd640 1 -- 172.21.15.69:0/2256863832 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7207 (secure 0 0 0) 0x7fc1d809b380 con 0x7fc1dc1007a0 2024-08-02T17:43:44.097 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:44.097 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":9,"btime":"2024-08-02T17:43:43:803784+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-02T17:43:44.098 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 9 2024-08-02T17:43:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:43 smithi077 ceph-mon[37108]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:44.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:43 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:44.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1c80a2b80 msgr2=0x7fc1c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:44.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1c80a2b80 0x7fc1c80a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc06b010 tx=0x7fc1cc009560 comp rx=0 tx=0).stop 2024-08-02T17:43:44.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1dc1007a0 msgr2=0x7fc1dc199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1dc1007a0 0x7fc1dc199030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8005fd0 tx=0x7fc1d800dde0 comp rx=0 tx=0).stop 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 shutdown_connections 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1c80a2b80 0x7fc1c80a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1dc101140 0x7fc1dc199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1dc1067a0 0x7fc1dc1a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 --2- 172.21.15.69:0/2256863832 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1dc1007a0 0x7fc1dc199030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 >> 172.21.15.69:0/2256863832 conn(0x7fc1dc0fc4c0 msgr2=0x7fc1dc0fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 shutdown_connections 2024-08-02T17:43:44.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:44.098+0000 7fc1e3f77640 1 -- 172.21.15.69:0/2256863832 wait complete. 2024-08-02T17:43:44.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:43 smithi083 ceph-mon[37031]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:44.173 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:44.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:44.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:44.174 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:43 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:44.852 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:44 smithi083 ceph-mon[37031]: Deploying daemon mds.i on smithi083 2024-08-02T17:43:44.852 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:44 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] up:boot 2024-08-02T17:43:44.852 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:44 smithi083 ceph-mon[37031]: fsmap 8 up:standby 2024-08-02T17:43:44.852 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:44 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T17:43:44.852 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:44 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2256863832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:44 smithi077 ceph-mon[37108]: Deploying daemon mds.i on smithi083 2024-08-02T17:43:45.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:44 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] up:boot 2024-08-02T17:43:45.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:44 smithi077 ceph-mon[37108]: fsmap 8 up:standby 2024-08-02T17:43:45.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:44 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T17:43:45.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:44 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2256863832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:44 smithi069 ceph-mon[32746]: Deploying daemon mds.i on smithi083 2024-08-02T17:43:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:44 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] up:boot 2024-08-02T17:43:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:44 smithi069 ceph-mon[32746]: fsmap 8 up:standby 2024-08-02T17:43:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:44 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T17:43:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:44 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2256863832' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:45.245 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 systemd[1]: Starting Ceph mds.i for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:43:45.499 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 podman[67300]: 2024-08-02 17:43:45.147067814 +0000 UTC m=+0.017630669 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:45.499 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 podman[67300]: 2024-08-02 17:43:45.253766704 +0000 UTC m=+0.124329553 container create 6170b510fee657e08a2d13982abb9619ee77a700a284f7bd1fd170bdcf802d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8) 2024-08-02T17:43:45.499 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 podman[67300]: 2024-08-02 17:43:45.473181506 +0000 UTC m=+0.343744360 container init 6170b510fee657e08a2d13982abb9619ee77a700a284f7bd1fd170bdcf802d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-02T17:43:45.499 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 podman[67300]: 2024-08-02 17:43:45.475672585 +0000 UTC m=+0.346235433 container start 6170b510fee657e08a2d13982abb9619ee77a700a284f7bd1fd170bdcf802d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8) 2024-08-02T17:43:45.499 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-mds[67319]: -- 172.21.15.83:0/2416131546 <== mon.1 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5d02d7860 con 0x55f5d02b5000 2024-08-02T17:43:45.771 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i[67315]: starting mds.i at 2024-08-02T17:43:45.771 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 bash[67300]: 6170b510fee657e08a2d13982abb9619ee77a700a284f7bd1fd170bdcf802d78 2024-08-02T17:43:45.771 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 17:43:45 smithi083 systemd[1]: Started Ceph mds.i for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:46.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:45 smithi077 ceph-mon[37108]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:46.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:46.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:46.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:45 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:46.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:45 smithi069 ceph-mon[32746]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:46.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:46.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:46.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:45 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:46.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-mon[37031]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:46.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:46.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:46.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:46.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:45 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:46.861 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:47.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:46 smithi077 ceph-mon[37108]: Deploying daemon mds.g on smithi069 2024-08-02T17:43:47.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:46 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] up:boot 2024-08-02T17:43:47.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:46 smithi077 ceph-mon[37108]: fsmap 9 up:standby 2024-08-02T17:43:47.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:46 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T17:43:47.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:46 smithi069 ceph-mon[32746]: Deploying daemon mds.g on smithi069 2024-08-02T17:43:47.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:46 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] up:boot 2024-08-02T17:43:47.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:46 smithi069 ceph-mon[32746]: fsmap 9 up:standby 2024-08-02T17:43:47.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:46 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T17:43:47.197 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:46 smithi083 ceph-mon[37031]: Deploying daemon mds.g on smithi069 2024-08-02T17:43:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:46 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] up:boot 2024-08-02T17:43:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:46 smithi083 ceph-mon[37031]: fsmap 9 up:standby 2024-08-02T17:43:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:46 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T17:43:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 -- 172.21.15.69:0/3605531909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 msgr2=0x7f1db41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/3605531909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 0x7f1db41026e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009900 tx=0x7f1d9c02a480 comp rx=0 tx=0).stop 2024-08-02T17:43:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 -- 172.21.15.69:0/3605531909 shutdown_connections 2024-08-02T17:43:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/3605531909 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 0x7f1db41026e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/3605531909 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4102c20 0x7f1db410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/3605531909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db4100b50 0x7f1db4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 -- 172.21.15.69:0/3605531909 >> 172.21.15.69:0/3605531909 conn(0x7f1db40fc500 msgr2=0x7f1db40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 -- 172.21.15.69:0/3605531909 shutdown_connections 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.774+0000 7f1db8a4c640 1 -- 172.21.15.69:0/3605531909 wait complete. 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db8a4c640 1 Processor -- start 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db8a4c640 1 -- start start 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db8a4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db4100b50 0x7f1db4199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db8a4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 0x7f1db41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db8a4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4102c20 0x7f1db41a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db8a4c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4110d00 con 0x7f1db41022a0 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db8a4c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4110b80 con 0x7f1db4102c20 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db8a4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1db4110e80 con 0x7f1db4100b50 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1dabfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 0x7f1db41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1dabfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 0x7f1db41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34068/0 (socket says 172.21.15.69:34068) 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1dabfff640 1 -- 172.21.15.69:0/2686043105 learned_addr learned my addr 172.21.15.69:0/2686043105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:47.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.775+0000 7f1db259c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db4100b50 0x7f1db4199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dabfff640 1 -- 172.21.15.69:0/2686043105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db4100b50 msgr2=0x7f1db4199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dabfff640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db4100b50 0x7f1db4199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1db2d9d640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4102c20 0x7f1db41a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dabfff640 1 -- 172.21.15.69:0/2686043105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4102c20 msgr2=0x7f1db41a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dabfff640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4102c20 0x7f1db41a40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dabfff640 1 -- 172.21.15.69:0/2686043105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1db41a4630 con 0x7f1db41022a0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dabfff640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 0x7f1db41995b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c006010 tx=0x7f1d9c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dab7fe640 1 -- 172.21.15.69:0/2686043105 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c03f070 con 0x7f1db41022a0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1db41a4800 con 0x7f1db41022a0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dab7fe640 1 -- 172.21.15.69:0/2686043105 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d9c003e30 con 0x7f1db41022a0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1dab7fe640 1 -- 172.21.15.69:0/2686043105 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c035030 con 0x7f1db41022a0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.776+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1db41a4cb0 con 0x7f1db41022a0 2024-08-02T17:43:47.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.777+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1db4100f30 con 0x7f1db41022a0 2024-08-02T17:43:47.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.778+0000 7f1dab7fe640 1 -- 172.21.15.69:0/2686043105 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d9c042070 con 0x7f1db41022a0 2024-08-02T17:43:47.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.779+0000 7f1dab7fe640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1da00a2b80 0x7f1da00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:47.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.779+0000 7f1db259c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1da00a2b80 0x7f1da00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:47.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.779+0000 7f1dab7fe640 1 -- 172.21.15.69:0/2686043105 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f1d9c0ef660 con 0x7f1db41022a0 2024-08-02T17:43:47.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.779+0000 7f1db259c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1da00a2b80 0x7f1da00a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1da4005fd0 tx=0x7f1da4005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:47.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:47.783+0000 7f1dab7fe640 1 -- 172.21.15.69:0/2686043105 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1d9c0bca60 con 0x7f1db41022a0 2024-08-02T17:43:48.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:47 smithi069 ceph-mon[32746]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:48.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.004+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1db4108dc0 con 0x7f1db41022a0 2024-08-02T17:43:48.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.005+0000 7f1dab7fe640 1 -- 172.21.15.69:0/2686043105 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8034 (secure 0 0 0) 0x7f1d9c0bc360 con 0x7f1db41022a0 2024-08-02T17:43:48.008 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:48.008 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":10,"btime":"2024-08-02T17:43:45:821375+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-02T17:43:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 10 2024-08-02T17:43:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1da00a2b80 msgr2=0x7f1da00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1da00a2b80 0x7f1da00a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1da4005fd0 tx=0x7f1da4005e20 comp rx=0 tx=0).stop 2024-08-02T17:43:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 msgr2=0x7f1db41995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 0x7f1db41995b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c006010 tx=0x7f1d9c003c10 comp rx=0 tx=0).stop 2024-08-02T17:43:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 shutdown_connections 2024-08-02T17:43:48.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1da00a2b80 0x7f1da00a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:48.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1db41022a0 0x7f1db41995b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:48.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1db4102c20 0x7f1db41a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:48.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 --2- 172.21.15.69:0/2686043105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1db4100b50 0x7f1db4199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:48.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 >> 172.21.15.69:0/2686043105 conn(0x7f1db40fc500 msgr2=0x7f1db40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:48.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 shutdown_connections 2024-08-02T17:43:48.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:48.007+0000 7f1db8a4c640 1 -- 172.21.15.69:0/2686043105 wait complete. 2024-08-02T17:43:48.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:47 smithi077 ceph-mon[37108]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:47 smithi083 ceph-mon[37031]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:49.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:48 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2686043105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:48 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:48 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2686043105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:49.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:49.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:48 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:48 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2686043105' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:49.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:49.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:48 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:50.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:49 smithi077 ceph-mon[37108]: Deploying daemon mds.a on smithi069 2024-08-02T17:43:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:49 smithi077 ceph-mon[37108]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:49 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.69:6838/1157046912,v1:172.21.15.69:6839/1157046912] up:boot 2024-08-02T17:43:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:49 smithi077 ceph-mon[37108]: fsmap 10 up:standby 2024-08-02T17:43:50.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:49 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-02T17:43:50.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:49 smithi069 ceph-mon[32746]: Deploying daemon mds.a on smithi069 2024-08-02T17:43:50.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:49 smithi069 ceph-mon[32746]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:50.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:49 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.69:6838/1157046912,v1:172.21.15.69:6839/1157046912] up:boot 2024-08-02T17:43:50.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:49 smithi069 ceph-mon[32746]: fsmap 10 up:standby 2024-08-02T17:43:50.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:49 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-02T17:43:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:49 smithi083 ceph-mon[37031]: Deploying daemon mds.a on smithi069 2024-08-02T17:43:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:49 smithi083 ceph-mon[37031]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:49 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.69:6838/1157046912,v1:172.21.15.69:6839/1157046912] up:boot 2024-08-02T17:43:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:49 smithi083 ceph-mon[37031]: fsmap 10 up:standby 2024-08-02T17:43:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:49 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-02T17:43:51.108 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:50 smithi069 systemd[1]: Starting Ceph mds.a for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T17:43:51.225 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:51.470 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:51 smithi069 podman[70664]: 2024-08-02 17:43:51.116254442 +0000 UTC m=+0.016946005 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T17:43:51.471 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:51 smithi069 podman[70664]: 2024-08-02 17:43:51.214296132 +0000 UTC m=+0.114987695 container create 2d961c432555926381f9693337c9a3474933f3680dc06bdc1519a36399b4081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-08-02T17:43:51.471 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:51 smithi069 podman[70664]: 2024-08-02 17:43:51.442991059 +0000 UTC m=+0.343682632 container init 2d961c432555926381f9693337c9a3474933f3680dc06bdc1519a36399b4081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-08-02T17:43:51.471 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:51 smithi069 podman[70664]: 2024-08-02 17:43:51.446312754 +0000 UTC m=+0.347004315 container start 2d961c432555926381f9693337c9a3474933f3680dc06bdc1519a36399b4081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T17:43:51.627 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:51.749 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-mds[70712]: -- 172.21.15.69:0/1581754504 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5582d0417860 con 0x5582d03f5400 2024-08-02T17:43:51.749 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a[70697]: starting mds.a at 2024-08-02T17:43:51.749 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:51 smithi069 bash[70664]: 2d961c432555926381f9693337c9a3474933f3680dc06bdc1519a36399b4081f 2024-08-02T17:43:51.749 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 17:43:51 smithi069 systemd[1]: Started Ceph mds.a for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:52.006 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-mon[32746]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:52.006 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.006 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:52.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:52.007 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:51 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:52.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 -- 172.21.15.69:0/3442330716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 msgr2=0x7f171c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 --2- 172.21.15.69:0/3442330716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 0x7f171c1026e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1710009900 tx=0x7f171002a480 comp rx=0 tx=0).stop 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 -- 172.21.15.69:0/3442330716 shutdown_connections 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 --2- 172.21.15.69:0/3442330716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 0x7f171c1026e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 --2- 172.21.15.69:0/3442330716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f171c102c20 0x7f171c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 --2- 172.21.15.69:0/3442330716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171c100b50 0x7f171c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 -- 172.21.15.69:0/3442330716 >> 172.21.15.69:0/3442330716 conn(0x7f171c0fc500 msgr2=0x7f171c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 -- 172.21.15.69:0/3442330716 shutdown_connections 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 -- 172.21.15.69:0/3442330716 wait complete. 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.085+0000 7f172206e640 1 Processor -- start 2024-08-02T17:43:52.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f172206e640 1 -- start start 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f172206e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171c100b50 0x7f171c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f172206e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 0x7f171c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f172206e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f171c102c20 0x7f171c1a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f172206e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f171c110cb0 con 0x7f171c1022a0 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f172206e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f171c110b30 con 0x7f171c102c20 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f172206e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f171c110e30 con 0x7f171c100b50 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 0x7f171c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 0x7f171c199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34128/0 (socket says 172.21.15.69:34128) 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 -- 172.21.15.69:0/448794455 learned_addr learned my addr 172.21.15.69:0/448794455 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171b7fe640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171c100b50 0x7f171c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171bfff640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f171c102c20 0x7f171c1a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 -- 172.21.15.69:0/448794455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171c100b50 msgr2=0x7f171c199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:52.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171c100b50 0x7f171c199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 -- 172.21.15.69:0/448794455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f171c102c20 msgr2=0x7f171c1a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f171c102c20 0x7f171c1a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 -- 172.21.15.69:0/448794455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c1a45e0 con 0x7f171c1022a0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171b7fe640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171c100b50 0x7f171c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.086+0000 7f171affd640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 0x7f171c199560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1710006010 tx=0x7f1710003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.087+0000 7f1718ff9640 1 -- 172.21.15.69:0/448794455 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171003f070 con 0x7f171c1022a0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.087+0000 7f1718ff9640 1 -- 172.21.15.69:0/448794455 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1710031c30 con 0x7f171c1022a0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.087+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c1a47b0 con 0x7f171c1022a0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.087+0000 7f1718ff9640 1 -- 172.21.15.69:0/448794455 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1710035030 con 0x7f171c1022a0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.087+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c1a4c00 con 0x7f171c1022a0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.088+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171c100f30 con 0x7f171c1022a0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.089+0000 7f1718ff9640 1 -- 172.21.15.69:0/448794455 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1710042070 con 0x7f171c1022a0 2024-08-02T17:43:52.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.089+0000 7f1718ff9640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f17040a2b80 0x7f17040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:52.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.089+0000 7f171b7fe640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f17040a2b80 0x7f17040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:52.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.090+0000 7f1718ff9640 1 -- 172.21.15.69:0/448794455 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f17100ef8d0 con 0x7f171c1022a0 2024-08-02T17:43:52.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.090+0000 7f171b7fe640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f17040a2b80 0x7f17040a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f170c005fd0 tx=0x7f170c00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:52.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.093+0000 7f1718ff9640 1 -- 172.21.15.69:0/448794455 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1710031de0 con 0x7f171c1022a0 2024-08-02T17:43:52.195 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:51 smithi077 ceph-mon[37108]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:52.196 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.196 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.196 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.196 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:52.196 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:52.196 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:51 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:51 smithi083 ceph-mon[37031]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-08-02T17:43:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-02T17:43:52.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:51 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:52.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.295+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f171c1a4e60 con 0x7f171c1022a0 2024-08-02T17:43:52.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.296+0000 7f1718ff9640 1 -- 172.21.15.69:0/448794455 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9683 (secure 0 0 0) 0x7f17100bccd0 con 0x7f171c1022a0 2024-08-02T17:43:52.298 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:52.298 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":12,"btime":"2024-08-02T17:43:51:845679+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-02T17:43:52.298 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 12 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f17040a2b80 msgr2=0x7f17040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f17040a2b80 0x7f17040a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f170c005fd0 tx=0x7f170c00b5f0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 msgr2=0x7f171c199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 0x7f171c199560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1710006010 tx=0x7f1710003c10 comp rx=0 tx=0).stop 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 shutdown_connections 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f17040a2b80 0x7f17040a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f171c1022a0 0x7f171c199560 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f171c102c20 0x7f171c1a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 --2- 172.21.15.69:0/448794455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f171c100b50 0x7f171c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:52.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 >> 172.21.15.69:0/448794455 conn(0x7f171c0fc500 msgr2=0x7f171c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:52.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 shutdown_connections 2024-08-02T17:43:52.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:52.298+0000 7f172206e640 1 -- 172.21.15.69:0/448794455 wait complete. 2024-08-02T17:43:53.086 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:52 smithi069 ceph-mon[32746]: Deploying daemon mds.b on smithi077 2024-08-02T17:43:53.086 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:52 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.69:6840/251587008,v1:172.21.15.69:6841/251587008] up:boot 2024-08-02T17:43:53.086 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:52 smithi069 ceph-mon[32746]: fsmap 11 up:standby 2024-08-02T17:43:53.086 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:52 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-02T17:43:53.086 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:52 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/448794455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:53.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:52 smithi083 ceph-mon[37031]: Deploying daemon mds.b on smithi077 2024-08-02T17:43:53.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:52 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.69:6840/251587008,v1:172.21.15.69:6841/251587008] up:boot 2024-08-02T17:43:53.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:52 smithi083 ceph-mon[37031]: fsmap 11 up:standby 2024-08-02T17:43:53.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:52 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-02T17:43:53.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:52 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/448794455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:53.335 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:52 smithi077 ceph-mon[37108]: Deploying daemon mds.b on smithi077 2024-08-02T17:43:53.335 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:52 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.69:6840/251587008,v1:172.21.15.69:6841/251587008] up:boot 2024-08-02T17:43:53.335 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:52 smithi077 ceph-mon[37108]: fsmap 11 up:standby 2024-08-02T17:43:53.335 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:52 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-02T17:43:53.335 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:52 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/448794455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:54.016 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:53 smithi077 ceph-mon[37108]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:53 smithi083 ceph-mon[37031]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:54.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:53 smithi069 ceph-mon[32746]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:54.597 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 17:43:54 smithi077 systemd[1]: Started Ceph mds.b for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T17:43:55.118 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:43:55.710 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] up:boot 2024-08-02T17:43:55.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: fsmap 12 up:standby 2024-08-02T17:43:55.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:55 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T17:43:55.837 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.837 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.837 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.837 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.837 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.837 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:55.838 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:55.838 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] up:boot 2024-08-02T17:43:55.838 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: fsmap 12 up:standby 2024-08-02T17:43:55.838 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:55 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T17:43:55.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:55.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:55.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:55.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] up:boot 2024-08-02T17:43:55.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: fsmap 12 up:standby 2024-08-02T17:43:55.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:55 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T17:43:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 -- 172.21.15.69:0/272217601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4100b50 msgr2=0x7fa5d4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/272217601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4100b50 0x7fa5d4100f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009900 tx=0x7fa5d002a480 comp rx=0 tx=0).stop 2024-08-02T17:43:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 -- 172.21.15.69:0/272217601 shutdown_connections 2024-08-02T17:43:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/272217601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5d4102c20 0x7fa5d410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/272217601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d41022a0 0x7fa5d41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/272217601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d4100b50 0x7fa5d4100f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 -- 172.21.15.69:0/272217601 >> 172.21.15.69:0/272217601 conn(0x7fa5d40fc500 msgr2=0x7fa5d40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 -- 172.21.15.69:0/272217601 shutdown_connections 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.211+0000 7fa5dbf72640 1 -- 172.21.15.69:0/272217601 wait complete. 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5dbf72640 1 Processor -- start 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5dbf72640 1 -- start start 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5dbf72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5d4100b50 0x7fa5d4199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5dbf72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d41022a0 0x7fa5d41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5dbf72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d4102c20 0x7fa5d41a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5dbf72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d4110cb0 con 0x7fa5d41022a0 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5dbf72640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d4110b30 con 0x7fa5d4102c20 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5dbf72640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d4110e30 con 0x7fa5d4100b50 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5d950d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d41022a0 0x7fa5d41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:56.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5d950d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d41022a0 0x7fa5d41995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56556/0 (socket says 172.21.15.69:56556) 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5d950d640 1 -- 172.21.15.69:0/2862031758 learned_addr learned my addr 172.21.15.69:0/2862031758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5d9d0e640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5d4100b50 0x7fa5d4199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.212+0000 7fa5da50f640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d4102c20 0x7fa5d41a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5d950d640 1 -- 172.21.15.69:0/2862031758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5d4100b50 msgr2=0x7fa5d4199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5d950d640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5d4100b50 0x7fa5d4199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5d950d640 1 -- 172.21.15.69:0/2862031758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d4102c20 msgr2=0x7fa5d41a40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5d950d640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d4102c20 0x7fa5d41a40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5d950d640 1 -- 172.21.15.69:0/2862031758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d41a4620 con 0x7fa5d41022a0 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5d950d640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d41022a0 0x7fa5d41995a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400e9f0 tx=0x7fa5c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5caffd640 1 -- 172.21.15.69:0/2862031758 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400cd80 con 0x7fa5d41022a0 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5caffd640 1 -- 172.21.15.69:0/2862031758 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c4010910 con 0x7fa5d41022a0 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d41a47f0 con 0x7fa5d41022a0 2024-08-02T17:43:56.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5caffd640 1 -- 172.21.15.69:0/2862031758 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4018990 con 0x7fa5d41022a0 2024-08-02T17:43:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.213+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d41a4d30 con 0x7fa5d41022a0 2024-08-02T17:43:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.214+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d4100f30 con 0x7fa5d41022a0 2024-08-02T17:43:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.215+0000 7fa5caffd640 1 -- 172.21.15.69:0/2862031758 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5c4010ac0 con 0x7fa5d41022a0 2024-08-02T17:43:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.216+0000 7fa5caffd640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5c00a2c00 0x7fa5c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.216+0000 7fa5caffd640 1 -- 172.21.15.69:0/2862031758 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fa5c40cf230 con 0x7fa5d41022a0 2024-08-02T17:43:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.216+0000 7fa5d9d0e640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5c00a2c00 0x7fa5c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.217+0000 7fa5d9d0e640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5c00a2c00 0x7fa5c00a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009230 tx=0x7fa5d00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:56.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.219+0000 7fa5caffd640 1 -- 172.21.15.69:0/2862031758 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa5c409bfe0 con 0x7fa5d41022a0 2024-08-02T17:43:56.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.460+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5d4108dc0 con 0x7fa5d41022a0 2024-08-02T17:43:56.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.461+0000 7fa5caffd640 1 -- 172.21.15.69:0/2862031758 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10509 (secure 0 0 0) 0x7fa5c40a02e0 con 0x7fa5d41022a0 2024-08-02T17:43:56.463 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:56.463 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":13,"btime":"2024-08-02T17:43:54:859059+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24452,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-02T17:43:56.463 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 13 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5c00a2c00 msgr2=0x7fa5c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5c00a2c00 0x7fa5c00a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0009230 tx=0x7fa5d00058d0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d41022a0 msgr2=0x7fa5d41995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d41022a0 0x7fa5d41995a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400e9f0 tx=0x7fa5c400eeb0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 shutdown_connections 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5c00a2c00 0x7fa5c00a50c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5d41022a0 0x7fa5d41995a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5d4102c20 0x7fa5d41a40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 --2- 172.21.15.69:0/2862031758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5d4100b50 0x7fa5d4199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:56.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.463+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 >> 172.21.15.69:0/2862031758 conn(0x7fa5d40fc500 msgr2=0x7fa5d40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:56.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.464+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 shutdown_connections 2024-08-02T17:43:56.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:56.464+0000 7fa5dbf72640 1 -- 172.21.15.69:0/2862031758 wait complete. 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2862031758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:57.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2862031758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:57.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.398 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-02T17:43:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2862031758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:43:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:57.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:57.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:43:57.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:43:57.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:58.335 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:43:58.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:58 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:58.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:58 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:58.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:58 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 -- 172.21.15.69:0/669429584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88102c20 msgr2=0x7fcf8810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 --2- 172.21.15.69:0/669429584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88102c20 0x7fcf8810d770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009ca0 tx=0x7fcf8002a950 comp rx=0 tx=0).stop 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 -- 172.21.15.69:0/669429584 shutdown_connections 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 --2- 172.21.15.69:0/669429584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88102c20 0x7fcf8810d770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 --2- 172.21.15.69:0/669429584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf881022a0 0x7fcf881026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 --2- 172.21.15.69:0/669429584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf88100b50 0x7fcf88100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 -- 172.21.15.69:0/669429584 >> 172.21.15.69:0/669429584 conn(0x7fcf880fc500 msgr2=0x7fcf880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 -- 172.21.15.69:0/669429584 shutdown_connections 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.785+0000 7fcf8d443640 1 -- 172.21.15.69:0/669429584 wait complete. 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf8d443640 1 Processor -- start 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf8d443640 1 -- start start 2024-08-02T17:43:58.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf8d443640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88100b50 0x7fcf8810fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf8d443640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf881022a0 0x7fcf88110000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf8d443640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf88102c20 0x7fcf88110540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf8d443640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88111dd0 con 0x7fcf88100b50 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf8d443640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88111c50 con 0x7fcf881022a0 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf8d443640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf88111f50 con 0x7fcf88102c20 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf86ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88100b50 0x7fcf8810fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf86ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88100b50 0x7fcf8810fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56570/0 (socket says 172.21.15.69:56570) 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf86ffd640 1 -- 172.21.15.69:0/3826784307 learned_addr learned my addr 172.21.15.69:0/3826784307 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf877fe640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf88102c20 0x7fcf88110540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf867fc640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf881022a0 0x7fcf88110000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf86ffd640 1 -- 172.21.15.69:0/3826784307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf88102c20 msgr2=0x7fcf88110540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf86ffd640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf88102c20 0x7fcf88110540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:58.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf86ffd640 1 -- 172.21.15.69:0/3826784307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf881022a0 msgr2=0x7fcf88110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf86ffd640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf881022a0 0x7fcf88110000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.786+0000 7fcf86ffd640 1 -- 172.21.15.69:0/3826784307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf881a6610 con 0x7fcf88100b50 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.787+0000 7fcf86ffd640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88100b50 0x7fcf8810fac0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800d950 tx=0x7fcf7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.787+0000 7fcf6ffff640 1 -- 172.21.15.69:0/3826784307 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf78017070 con 0x7fcf88100b50 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.787+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf881a6900 con 0x7fcf88100b50 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.787+0000 7fcf6ffff640 1 -- 172.21.15.69:0/3826784307 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf78009c30 con 0x7fcf88100b50 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.787+0000 7fcf6ffff640 1 -- 172.21.15.69:0/3826784307 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800f6b0 con 0x7fcf88100b50 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.787+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf881a6e40 con 0x7fcf88100b50 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.788+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf88100f30 con 0x7fcf88100b50 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.789+0000 7fcf6ffff640 1 -- 172.21.15.69:0/3826784307 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcf780109d0 con 0x7fcf88100b50 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.789+0000 7fcf6ffff640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcf700a2b80 0x7fcf700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.790+0000 7fcf867fc640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcf700a2b80 0x7fcf700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:43:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.790+0000 7fcf6ffff640 1 -- 172.21.15.69:0/3826784307 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fcf780ce810 con 0x7fcf88100b50 2024-08-02T17:43:58.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.790+0000 7fcf867fc640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcf700a2b80 0x7fcf700a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf88111550 tx=0x7fcf7c00c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:43:58.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:58.794+0000 7fcf6ffff640 1 -- 172.21.15.69:0/3826784307 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcf78009de0 con 0x7fcf88100b50 2024-08-02T17:43:59.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.000+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcf881a7010 con 0x7fcf88100b50 2024-08-02T17:43:59.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.001+0000 7fcf6ffff640 1 -- 172.21.15.69:0/3826784307 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7fcf7809bc10 con 0x7fcf88100b50 2024-08-02T17:43:59.002 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:43:59.003 INFO:teuthology.orchestra.run.smithi069.stdout:[ 2024-08-02T17:43:59.003 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T17:43:59.003 INFO:teuthology.orchestra.run.smithi069.stdout: "poolnum": 1, 2024-08-02T17:43:59.003 INFO:teuthology.orchestra.run.smithi069.stdout: "poolname": ".mgr" 2024-08-02T17:43:59.003 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:43:59.003 INFO:teuthology.orchestra.run.smithi069.stdout:] 2024-08-02T17:43:59.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcf700a2b80 msgr2=0x7fcf700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:59.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcf700a2b80 0x7fcf700a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcf88111550 tx=0x7fcf7c00c4a0 comp rx=0 tx=0).stop 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88100b50 msgr2=0x7fcf8810fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88100b50 0x7fcf8810fac0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800d950 tx=0x7fcf7800de10 comp rx=0 tx=0).stop 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 shutdown_connections 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcf700a2b80 0x7fcf700a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf88102c20 0x7fcf88110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf881022a0 0x7fcf88110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 --2- 172.21.15.69:0/3826784307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf88100b50 0x7fcf8810fac0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 >> 172.21.15.69:0/3826784307 conn(0x7fcf880fc500 msgr2=0x7fcf881064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 shutdown_connections 2024-08-02T17:43:59.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:43:59.003+0000 7fcf8d443640 1 -- 172.21.15.69:0/3826784307 wait complete. 2024-08-02T17:43:59.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:59 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:59 smithi069 ceph-mon[32746]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:59.770 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:43:59 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3826784307' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-02T17:43:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:59 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:59 smithi083 ceph-mon[37031]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:59.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:43:59 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3826784307' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-02T17:43:59.798 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-02T17:43:59.798 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-02T17:43:59.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:59 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:43:59.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:59 smithi077 ceph-mon[37108]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:43:59.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:43:59 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3826784307' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-08-02T17:44:00.062 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:00.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 -- 172.21.15.69:0/3450945084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec4102c20 msgr2=0x7fcec410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 --2- 172.21.15.69:0/3450945084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec4102c20 0x7fcec410d770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc02a480 comp rx=0 tx=0).stop 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 -- 172.21.15.69:0/3450945084 shutdown_connections 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 --2- 172.21.15.69:0/3450945084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec4102c20 0x7fcec410d770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 --2- 172.21.15.69:0/3450945084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec41022a0 0x7fcec41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 --2- 172.21.15.69:0/3450945084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec4100b50 0x7fcec4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 -- 172.21.15.69:0/3450945084 >> 172.21.15.69:0/3450945084 conn(0x7fcec40fc500 msgr2=0x7fcec40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 -- 172.21.15.69:0/3450945084 shutdown_connections 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.572+0000 7fcec92fe640 1 -- 172.21.15.69:0/3450945084 wait complete. 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.573+0000 7fcec92fe640 1 Processor -- start 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.573+0000 7fcec92fe640 1 -- start start 2024-08-02T17:44:00.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.573+0000 7fcec92fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec4100b50 0x7fcec4197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.573+0000 7fcec92fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec41022a0 0x7fcec41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.573+0000 7fcec92fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec4102c20 0x7fcec419b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.573+0000 7fcec92fe640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4110cb0 con 0x7fcec41022a0 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.573+0000 7fcec92fe640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4110b30 con 0x7fcec4102c20 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.573+0000 7fcec92fe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4110e30 con 0x7fcec4100b50 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec41022a0 0x7fcec41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec41022a0 0x7fcec41975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56596/0 (socket says 172.21.15.69:56596) 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 -- 172.21.15.69:0/1062384846 learned_addr learned my addr 172.21.15.69:0/1062384846 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec37fe640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec4102c20 0x7fcec419b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec2ffd640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec4100b50 0x7fcec4197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 -- 172.21.15.69:0/1062384846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec4100b50 msgr2=0x7fcec4197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:00.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec4100b50 0x7fcec4197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 -- 172.21.15.69:0/1062384846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec4102c20 msgr2=0x7fcec419b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec4102c20 0x7fcec419b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 -- 172.21.15.69:0/1062384846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec419c030 con 0x7fcec41022a0 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec2ffd640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec4100b50 0x7fcec4197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec27fc640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec41022a0 0x7fcec41975c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c890 tx=0x7fceac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcebbfff640 1 -- 172.21.15.69:0/1062384846 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac004220 con 0x7fcec41022a0 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec419c320 con 0x7fcec41022a0 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcebbfff640 1 -- 172.21.15.69:0/1062384846 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceac0043c0 con 0x7fcec41022a0 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcebbfff640 1 -- 172.21.15.69:0/1062384846 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceac004a00 con 0x7fcec41022a0 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.574+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec41a3c00 con 0x7fcec41022a0 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.575+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec4100f30 con 0x7fcec41022a0 2024-08-02T17:44:00.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.576+0000 7fcebbfff640 1 -- 172.21.15.69:0/1062384846 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fceac0079b0 con 0x7fcec41022a0 2024-08-02T17:44:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.577+0000 7fcebbfff640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fceb00a2b80 0x7fceb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.577+0000 7fcec2ffd640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fceb00a2b80 0x7fceb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.577+0000 7fcebbfff640 1 -- 172.21.15.69:0/1062384846 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fceac0cec80 con 0x7fcec41022a0 2024-08-02T17:44:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.578+0000 7fcec2ffd640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fceb00a2b80 0x7fceb00a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005c40 tx=0x7fceb400a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:00.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.580+0000 7fcebbfff640 1 -- 172.21.15.69:0/1062384846 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fceac014030 con 0x7fcec41022a0 2024-08-02T17:44:00.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:00.746+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcec41983e0 con 0x7fcec41022a0 2024-08-02T17:44:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.032+0000 7fcebbfff640 1 -- 172.21.15.69:0/1062384846 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7fcec41983e0 con 0x7fcec41022a0 2024-08-02T17:44:01.034 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' created 2024-08-02T17:44:01.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:44:01.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:44:01.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.034+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fceb00a2b80 msgr2=0x7fceb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:01.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.034+0000 7fcec92fe640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fceb00a2b80 0x7fceb00a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005c40 tx=0x7fceb400a410 comp rx=0 tx=0).stop 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.034+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec41022a0 msgr2=0x7fcec41975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec41022a0 0x7fcec41975c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fceac00c890 tx=0x7fceac00cd50 comp rx=0 tx=0).stop 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 shutdown_connections 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fceb00a2b80 0x7fceb00a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcec41022a0 0x7fcec41975c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcec4102c20 0x7fcec419b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 --2- 172.21.15.69:0/1062384846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcec4100b50 0x7fcec4197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 >> 172.21.15.69:0/1062384846 conn(0x7fcec40fc500 msgr2=0x7fcec4105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 shutdown_connections 2024-08-02T17:44:01.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:01.035+0000 7fcec92fe640 1 -- 172.21.15.69:0/1062384846 wait complete. 2024-08-02T17:44:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:44:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:44:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:44:01.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:44:01.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:44:01.957 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-08-02T17:44:02.228 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:01 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1062384846' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-02T17:44:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:01 smithi083 ceph-mon[37031]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:01 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1062384846' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-02T17:44:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:01 smithi083 ceph-mon[37031]: osdmap e80: 12 total, 12 up, 12 in 2024-08-02T17:44:02.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:01 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1062384846' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-02T17:44:02.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:01 smithi077 ceph-mon[37108]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:02.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:01 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1062384846' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-02T17:44:02.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:01 smithi077 ceph-mon[37108]: osdmap e80: 12 total, 12 up, 12 in 2024-08-02T17:44:02.350 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:01 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1062384846' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-08-02T17:44:02.350 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:01 smithi069 ceph-mon[32746]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:02.350 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:01 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1062384846' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-02T17:44:02.350 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:01 smithi069 ceph-mon[32746]: osdmap e80: 12 total, 12 up, 12 in 2024-08-02T17:44:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 -- 172.21.15.69:0/3554835422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 msgr2=0x7f271c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:02.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 --2- 172.21.15.69:0/3554835422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 0x7f271c101d10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f270c009930 tx=0x7f270c02a420 comp rx=0 tx=0).stop 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 -- 172.21.15.69:0/3554835422 shutdown_connections 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 --2- 172.21.15.69:0/3554835422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 0x7f271c101d10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 --2- 172.21.15.69:0/3554835422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c102250 0x7f271c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 --2- 172.21.15.69:0/3554835422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f271c1078b0 0x7f271c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 -- 172.21.15.69:0/3554835422 >> 172.21.15.69:0/3554835422 conn(0x7f271c0fd5f0 msgr2=0x7f271c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 -- 172.21.15.69:0/3554835422 shutdown_connections 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.691+0000 7f27239f1640 1 -- 172.21.15.69:0/3554835422 wait complete. 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f27239f1640 1 Processor -- start 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f27239f1640 1 -- start start 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f27239f1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 0x7f271c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f27239f1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f271c102250 0x7f271c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:02.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f27239f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c1078b0 0x7f271c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f27239f1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f271c111cf0 con 0x7f271c1018b0 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f27239f1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f271c111b70 con 0x7f271c102250 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f27239f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f271c111e70 con 0x7f271c1078b0 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f272178d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 0x7f271c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f272178d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 0x7f271c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56610/0 (socket says 172.21.15.69:56610) 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f272178d640 1 -- 172.21.15.69:0/1840382698 learned_addr learned my addr 172.21.15.69:0/1840382698 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f272178d640 1 -- 172.21.15.69:0/1840382698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c1078b0 msgr2=0x7f271c19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.692+0000 7f2721f8e640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c1078b0 0x7f271c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f2720f8c640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f271c102250 0x7f271c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f272178d640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c1078b0 0x7f271c19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f272178d640 1 -- 172.21.15.69:0/1840382698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f271c102250 msgr2=0x7f271c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:02.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f272178d640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f271c102250 0x7f271c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f272178d640 1 -- 172.21.15.69:0/1840382698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c19d1e0 con 0x7f271c1018b0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f2721f8e640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c1078b0 0x7f271c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f2720f8c640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f271c102250 0x7f271c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f272178d640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 0x7f271c198230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f271800b4a0 tx=0x7f271800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f27127fc640 1 -- 172.21.15.69:0/1840382698 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718014070 con 0x7f271c1018b0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c19d4d0 con 0x7f271c1018b0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f27127fc640 1 -- 172.21.15.69:0/1840382698 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2718010af0 con 0x7f271c1018b0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f27127fc640 1 -- 172.21.15.69:0/1840382698 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2718017d10 con 0x7f271c1018b0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.693+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1a4db0 con 0x7f271c1018b0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.694+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c101d10 con 0x7f271c1018b0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.695+0000 7f27127fc640 1 -- 172.21.15.69:0/1840382698 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2718017070 con 0x7f271c1018b0 2024-08-02T17:44:02.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.696+0000 7f27127fc640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27080a2b80 0x7f27080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:02.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.696+0000 7f2720f8c640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27080a2b80 0x7f27080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:02.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.696+0000 7f27127fc640 1 -- 172.21.15.69:0/1840382698 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7f27180ce520 con 0x7f271c1018b0 2024-08-02T17:44:02.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.696+0000 7f2720f8c640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27080a2b80 0x7f27080a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f270c000c00 tx=0x7f270c032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:02.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.699+0000 7f27127fc640 1 -- 172.21.15.69:0/1840382698 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f271809b7b0 con 0x7f271c1018b0 2024-08-02T17:44:02.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:02.866+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f271c199590 con 0x7f271c1018b0 2024-08-02T17:44:03.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.106+0000 7f27127fc640 1 -- 172.21.15.69:0/1840382698 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) ==== 171+0+0 (secure 0 0 0) 0x7f271809b0b0 con 0x7f271c1018b0 2024-08-02T17:44:03.108 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' created 2024-08-02T17:44:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27080a2b80 msgr2=0x7f27080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27080a2b80 0x7f27080a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f270c000c00 tx=0x7f270c032c70 comp rx=0 tx=0).stop 2024-08-02T17:44:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 msgr2=0x7f271c198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 0x7f271c198230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f271800b4a0 tx=0x7f271800b960 comp rx=0 tx=0).stop 2024-08-02T17:44:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 shutdown_connections 2024-08-02T17:44:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27080a2b80 0x7f27080a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f271c1078b0 0x7f271c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:03.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f271c102250 0x7f271c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 --2- 172.21.15.69:0/1840382698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f271c1018b0 0x7f271c198230 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.108+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 >> 172.21.15.69:0/1840382698 conn(0x7f271c0fd5f0 msgr2=0x7f271c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.109+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 shutdown_connections 2024-08-02T17:44:03.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:03.109+0000 7f27239f1640 1 -- 172.21.15.69:0/1840382698 wait complete. 2024-08-02T17:44:03.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:03 smithi069 ceph-mon[32746]: osdmap e81: 12 total, 12 up, 12 in 2024-08-02T17:44:03.125 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:03 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1840382698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-02T17:44:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:03 smithi083 ceph-mon[37031]: osdmap e81: 12 total, 12 up, 12 in 2024-08-02T17:44:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:03 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1840382698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-02T17:44:03.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:03 smithi077 ceph-mon[37108]: osdmap e81: 12 total, 12 up, 12 in 2024-08-02T17:44:03.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:03 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1840382698' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-08-02T17:44:03.939 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-02T17:44:04.221 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:04.337 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:04 smithi069 ceph-mon[32746]: pgmap v254: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:04.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:04 smithi069 ceph-mon[32746]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-02T17:44:04.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:04 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1840382698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-02T17:44:04.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:04 smithi069 ceph-mon[32746]: osdmap e82: 12 total, 12 up, 12 in 2024-08-02T17:44:04.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:04 smithi083 ceph-mon[37031]: pgmap v254: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:04.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:04 smithi083 ceph-mon[37031]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-02T17:44:04.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:04 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1840382698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-02T17:44:04.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:04 smithi083 ceph-mon[37031]: osdmap e82: 12 total, 12 up, 12 in 2024-08-02T17:44:04.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:04 smithi077 ceph-mon[37108]: pgmap v254: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:04.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:04 smithi077 ceph-mon[37108]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-02T17:44:04.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:04 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1840382698' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-02T17:44:04.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:04 smithi077 ceph-mon[37108]: osdmap e82: 12 total, 12 up, 12 in 2024-08-02T17:44:04.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 -- 172.21.15.69:0/902751073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 msgr2=0x7f5a0c10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 --2- 172.21.15.69:0/902751073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 0x7f5a0c10e970 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a04009cd0 tx=0x7f5a0402a910 comp rx=0 tx=0).stop 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 -- 172.21.15.69:0/902751073 shutdown_connections 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 --2- 172.21.15.69:0/902751073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 0x7f5a0c10e970 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 --2- 172.21.15.69:0/902751073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c103410 0x7f5a0c1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 --2- 172.21.15.69:0/902751073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a0c102a60 0x7f5a0c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 -- 172.21.15.69:0/902751073 >> 172.21.15.69:0/902751073 conn(0x7f5a0c0fc500 msgr2=0x7f5a0c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 -- 172.21.15.69:0/902751073 shutdown_connections 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.681+0000 7f5a13882640 1 -- 172.21.15.69:0/902751073 wait complete. 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a13882640 1 Processor -- start 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a13882640 1 -- start start 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a13882640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a0c102a60 0x7f5a0c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a13882640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c103410 0x7f5a0c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:04.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a13882640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 0x7f5a0c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a13882640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c111e10 con 0x7f5a0c107fd0 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a13882640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c111c90 con 0x7f5a0c102a60 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a13882640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c111f90 con 0x7f5a0c103410 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 0x7f5a0c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 0x7f5a0c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56626/0 (socket says 172.21.15.69:56626) 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 -- 172.21.15.69:0/186486863 learned_addr learned my addr 172.21.15.69:0/186486863 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a1161e640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a0c102a60 0x7f5a0c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a10e1d640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c103410 0x7f5a0c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 -- 172.21.15.69:0/186486863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c103410 msgr2=0x7f5a0c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c103410 0x7f5a0c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 -- 172.21.15.69:0/186486863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a0c102a60 msgr2=0x7f5a0c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:04.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a0c102a60 0x7f5a0c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 -- 172.21.15.69:0/186486863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c19d300 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.682+0000 7f5a11e1f640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 0x7f5a0c19cc20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005cc0 tx=0x7f5a04003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.683+0000 7f5a027fc640 1 -- 172.21.15.69:0/186486863 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04031660 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.683+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c19d590 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.683+0000 7f5a027fc640 1 -- 172.21.15.69:0/186486863 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a04031800 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.683+0000 7f5a027fc640 1 -- 172.21.15.69:0/186486863 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0402fb50 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.683+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c1a4ed0 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.684+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0c102e40 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.685+0000 7f5a027fc640 1 -- 172.21.15.69:0/186486863 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5a0402fd10 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.686+0000 7f5a027fc640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f59f80a2b80 0x7f59f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.686+0000 7f5a1161e640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f59f80a2b80 0x7f59f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.686+0000 7f5a027fc640 1 -- 172.21.15.69:0/186486863 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f5a040f04b0 con 0x7f5a0c107fd0 2024-08-02T17:44:04.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.686+0000 7f5a1161e640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f59f80a2b80 0x7f59f80a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800bbe0 tx=0x7f5a0800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:04.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.690+0000 7f5a027fc640 1 -- 172.21.15.69:0/186486863 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5a040bd5c0 con 0x7f5a0c107fd0 2024-08-02T17:44:04.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:04.895+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5a0c109fe0 con 0x7f5a0c107fd0 2024-08-02T17:44:05.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.128+0000 7f5a027fc640 1 -- 172.21.15.69:0/186486863 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f5a040bcec0 con 0x7f5a0c107fd0 2024-08-02T17:44:05.130 INFO:teuthology.orchestra.run.smithi069.stderr:new fs with metadata pool 2 and data pool 3 2024-08-02T17:44:05.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f59f80a2b80 msgr2=0x7f59f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f59f80a2b80 0x7f59f80a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800bbe0 tx=0x7f5a0800b390 comp rx=0 tx=0).stop 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 msgr2=0x7f5a0c19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 0x7f5a0c19cc20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5a04005cc0 tx=0x7f5a04003f90 comp rx=0 tx=0).stop 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 shutdown_connections 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f59f80a2b80 0x7f59f80a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a0c107fd0 0x7f5a0c19cc20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a0c103410 0x7f5a0c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 --2- 172.21.15.69:0/186486863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a0c102a60 0x7f5a0c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 >> 172.21.15.69:0/186486863 conn(0x7f5a0c0fc500 msgr2=0x7f5a0c1060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 shutdown_connections 2024-08-02T17:44:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:05.130+0000 7f5a13882640 1 -- 172.21.15.69:0/186486863 wait complete. 2024-08-02T17:44:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:05 smithi069 ceph-mon[32746]: osdmap e83: 12 total, 12 up, 12 in 2024-08-02T17:44:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:05 smithi069 ceph-mon[32746]: pgmap v257: 97 pgs: 37 creating+peering, 44 unknown, 16 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:05 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/186486863' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-02T17:44:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:05 smithi069 ceph-mon[32746]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-02T17:44:05.499 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:05 smithi077 ceph-mon[37108]: osdmap e83: 12 total, 12 up, 12 in 2024-08-02T17:44:05.499 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:05 smithi077 ceph-mon[37108]: pgmap v257: 97 pgs: 37 creating+peering, 44 unknown, 16 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:05.499 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:05 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/186486863' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-02T17:44:05.499 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:05 smithi077 ceph-mon[37108]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-02T17:44:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:05 smithi083 ceph-mon[37031]: osdmap e83: 12 total, 12 up, 12 in 2024-08-02T17:44:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:05 smithi083 ceph-mon[37031]: pgmap v257: 97 pgs: 37 creating+peering, 44 unknown, 16 active+clean; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:05 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/186486863' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-08-02T17:44:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:05 smithi083 ceph-mon[37031]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-02T17:44:05.859 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-08-02T17:44:05.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-08-02T17:44:06.114 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:06.387 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:06 smithi069 ceph-mon[32746]: osdmap e84: 12 total, 12 up, 12 in 2024-08-02T17:44:06.387 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:06 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/186486863' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-02T17:44:06.387 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:06 smithi069 ceph-mon[32746]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-02T17:44:06.387 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:06 smithi069 ceph-mon[32746]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-02T17:44:06.422 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:06 smithi083 ceph-mon[37031]: osdmap e84: 12 total, 12 up, 12 in 2024-08-02T17:44:06.423 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:06 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/186486863' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-02T17:44:06.423 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:06 smithi083 ceph-mon[37031]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-02T17:44:06.423 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:06 smithi083 ceph-mon[37031]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-02T17:44:06.520 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:06 smithi077 ceph-mon[37108]: osdmap e84: 12 total, 12 up, 12 in 2024-08-02T17:44:06.520 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:06 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/186486863' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-02T17:44:06.520 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:06 smithi077 ceph-mon[37108]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-02T17:44:06.520 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:06 smithi077 ceph-mon[37108]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-02T17:44:06.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 -- 172.21.15.69:0/963396267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81001b0 msgr2=0x7f7ef8100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:06.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 --2- 172.21.15.69:0/963396267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81001b0 0x7f7ef8100590 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009900 tx=0x7f7ef402a410 comp rx=0 tx=0).stop 2024-08-02T17:44:06.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 -- 172.21.15.69:0/963396267 shutdown_connections 2024-08-02T17:44:06.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 --2- 172.21.15.69:0/963396267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef81052e0 0x7f7ef810ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:06.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 --2- 172.21.15.69:0/963396267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef8100b60 0x7f7ef8104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 --2- 172.21.15.69:0/963396267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81001b0 0x7f7ef8100590 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 -- 172.21.15.69:0/963396267 >> 172.21.15.69:0/963396267 conn(0x7f7ef8076f40 msgr2=0x7f7ef80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 -- 172.21.15.69:0/963396267 shutdown_connections 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.582+0000 7f7f007c3640 1 -- 172.21.15.69:0/963396267 wait complete. 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7f007c3640 1 Processor -- start 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7f007c3640 1 -- start start 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7f007c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef81001b0 0x7f7ef8198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7f007c3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef8100b60 0x7f7ef81986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7f007c3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81052e0 0x7f7ef819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7f007c3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8111cc0 con 0x7f7ef81052e0 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7f007c3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8111b40 con 0x7f7ef8100b60 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7f007c3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8111e40 con 0x7f7ef81001b0 2024-08-02T17:44:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efed60640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81052e0 0x7f7ef819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efe55f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef81001b0 0x7f7ef8198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efed60640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81052e0 0x7f7ef819ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41038/0 (socket says 172.21.15.69:41038) 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efed60640 1 -- 172.21.15.69:0/805551598 learned_addr learned my addr 172.21.15.69:0/805551598 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efdd5e640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef8100b60 0x7f7ef81986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efed60640 1 -- 172.21.15.69:0/805551598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef81001b0 msgr2=0x7f7ef8198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efed60640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef81001b0 0x7f7ef8198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efed60640 1 -- 172.21.15.69:0/805551598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef8100b60 msgr2=0x7f7ef81986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efed60640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef8100b60 0x7f7ef81986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.583+0000 7f7efed60640 1 -- 172.21.15.69:0/805551598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef819d130 con 0x7f7ef81052e0 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.584+0000 7f7efed60640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81052e0 0x7f7ef819ca50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000eaa0 tx=0x7f7ef000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.584+0000 7f7eef7fe640 1 -- 172.21.15.69:0/805551598 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000cdb0 con 0x7f7ef81052e0 2024-08-02T17:44:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.584+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef819d420 con 0x7f7ef81052e0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.584+0000 7f7eef7fe640 1 -- 172.21.15.69:0/805551598 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ef0005020 con 0x7f7ef81052e0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.584+0000 7f7eef7fe640 1 -- 172.21.15.69:0/805551598 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00116b0 con 0x7f7ef81052e0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.584+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef81a4d00 con 0x7f7ef81052e0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.585+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef8100590 con 0x7f7ef81052e0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.586+0000 7f7eef7fe640 1 -- 172.21.15.69:0/805551598 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7ef0005560 con 0x7f7ef81052e0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.586+0000 7f7eef7fe640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7ee40a2b80 0x7f7ee40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.587+0000 7f7efe55f640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7ee40a2b80 0x7f7ee40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.587+0000 7f7eef7fe640 1 -- 172.21.15.69:0/805551598 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f7ef00cf390 con 0x7f7ef81052e0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.587+0000 7f7efe55f640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7ee40a2b80 0x7f7ee40a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009230 tx=0x7f7ef40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:06.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.590+0000 7f7eef7fe640 1 -- 172.21.15.69:0/805551598 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7ef0016030 con 0x7f7ef81052e0 2024-08-02T17:44:06.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:06.765+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f7ef81994e0 con 0x7f7ef81052e0 2024-08-02T17:44:07.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.150+0000 7f7eef7fe640 1 -- 172.21.15.69:0/805551598 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f7ef81994e0 con 0x7f7ef81052e0 2024-08-02T17:44:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.152+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7ee40a2b80 msgr2=0x7f7ee40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.152+0000 7f7f007c3640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7ee40a2b80 0x7f7ee40a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009230 tx=0x7f7ef40058d0 comp rx=0 tx=0).stop 2024-08-02T17:44:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.152+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81052e0 msgr2=0x7f7ef819ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.152+0000 7f7f007c3640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81052e0 0x7f7ef819ca50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000eaa0 tx=0x7f7ef000ef60 comp rx=0 tx=0).stop 2024-08-02T17:44:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.153+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 shutdown_connections 2024-08-02T17:44:07.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.153+0000 7f7f007c3640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7ee40a2b80 0x7f7ee40a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.153+0000 7f7f007c3640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ef81052e0 0x7f7ef819ca50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.153+0000 7f7f007c3640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ef8100b60 0x7f7ef81986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.153+0000 7f7f007c3640 1 --2- 172.21.15.69:0/805551598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ef81001b0 0x7f7ef8198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.153+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 >> 172.21.15.69:0/805551598 conn(0x7f7ef8076f40 msgr2=0x7f7ef8103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.153+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 shutdown_connections 2024-08-02T17:44:07.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:07.153+0000 7f7f007c3640 1 -- 172.21.15.69:0/805551598 wait complete. 2024-08-02T17:44:07.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:07 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] up:active 2024-08-02T17:44:07.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:07 smithi069 ceph-mon[32746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:07.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:07 smithi069 ceph-mon[32746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:07.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:07 smithi069 ceph-mon[32746]: pgmap v259: 97 pgs: 34 creating+peering, 6 unknown, 57 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 216 B/s wr, 4 op/s 2024-08-02T17:44:07.460 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:07 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/805551598' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-02T17:44:07.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:07 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] up:active 2024-08-02T17:44:07.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:07 smithi083 ceph-mon[37031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:07.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:07 smithi083 ceph-mon[37031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:07.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:07 smithi083 ceph-mon[37031]: pgmap v259: 97 pgs: 34 creating+peering, 6 unknown, 57 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 216 B/s wr, 4 op/s 2024-08-02T17:44:07.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:07 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/805551598' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-02T17:44:07.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:07 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] up:active 2024-08-02T17:44:07.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:07 smithi077 ceph-mon[37108]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:07.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:07 smithi077 ceph-mon[37108]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:07.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:07 smithi077 ceph-mon[37108]: pgmap v259: 97 pgs: 34 creating+peering, 6 unknown, 57 active+clean; 577 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 216 B/s wr, 4 op/s 2024-08-02T17:44:07.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:07 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/805551598' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-08-02T17:44:07.917 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-08-02T17:44:08.170 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:08.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:08 smithi069 ceph-mon[32746]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-02T17:44:08.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:08 smithi069 ceph-mon[32746]: Cluster is now healthy 2024-08-02T17:44:08.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:08 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/805551598' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-02T17:44:08.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:08 smithi069 ceph-mon[32746]: osdmap e85: 12 total, 12 up, 12 in 2024-08-02T17:44:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:08 smithi083 ceph-mon[37031]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-02T17:44:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:08 smithi083 ceph-mon[37031]: Cluster is now healthy 2024-08-02T17:44:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:08 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/805551598' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-02T17:44:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:08 smithi083 ceph-mon[37031]: osdmap e85: 12 total, 12 up, 12 in 2024-08-02T17:44:08.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:08 smithi077 ceph-mon[37108]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-02T17:44:08.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:08 smithi077 ceph-mon[37108]: Cluster is now healthy 2024-08-02T17:44:08.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:08 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/805551598' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-02T17:44:08.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:08 smithi077 ceph-mon[37108]: osdmap e85: 12 total, 12 up, 12 in 2024-08-02T17:44:08.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 -- 172.21.15.69:0/3707772831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301067b0 msgr2=0x7f9830106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:08.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 --2- 172.21.15.69:0/3707772831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301067b0 0x7f9830106b90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9820009930 tx=0x7f982002a420 comp rx=0 tx=0).stop 2024-08-02T17:44:08.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 -- 172.21.15.69:0/3707772831 shutdown_connections 2024-08-02T17:44:08.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 --2- 172.21.15.69:0/3707772831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9830101150 0x7f983010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:08.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 --2- 172.21.15.69:0/3707772831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98301007b0 0x7f9830100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 --2- 172.21.15.69:0/3707772831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301067b0 0x7f9830106b90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 -- 172.21.15.69:0/3707772831 >> 172.21.15.69:0/3707772831 conn(0x7f98300fc4d0 msgr2=0x7f98300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 -- 172.21.15.69:0/3707772831 shutdown_connections 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.645+0000 7f9834813640 1 -- 172.21.15.69:0/3707772831 wait complete. 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f9834813640 1 Processor -- start 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f9834813640 1 -- start start 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f9834813640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301007b0 0x7f9830199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f9834813640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9830101150 0x7f9830199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f9834813640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98301067b0 0x7f98301a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f9834813640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9830110b70 con 0x7f98301007b0 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f9834813640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98301109f0 con 0x7f9830101150 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f9834813640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9830110cf0 con 0x7f98301067b0 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301007b0 0x7f9830199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:08.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301007b0 0x7f9830199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41054/0 (socket says 172.21.15.69:41054) 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982e59c640 1 -- 172.21.15.69:0/32794169 learned_addr learned my addr 172.21.15.69:0/32794169 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982dd9b640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9830101150 0x7f9830199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982e59c640 1 -- 172.21.15.69:0/32794169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98301067b0 msgr2=0x7f98301a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982ed9d640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98301067b0 0x7f98301a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982e59c640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98301067b0 0x7f98301a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982e59c640 1 -- 172.21.15.69:0/32794169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9830101150 msgr2=0x7f9830199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982e59c640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9830101150 0x7f9830199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982e59c640 1 -- 172.21.15.69:0/32794169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98301a45e0 con 0x7f98301007b0 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.646+0000 7f982dd9b640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9830101150 0x7f9830199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.647+0000 7f982e59c640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301007b0 0x7f9830199020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9820009a50 tx=0x7f9820004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.647+0000 7f98277fe640 1 -- 172.21.15.69:0/32794169 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982003f030 con 0x7f98301007b0 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.647+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98301a47b0 con 0x7f98301007b0 2024-08-02T17:44:08.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.647+0000 7f98277fe640 1 -- 172.21.15.69:0/32794169 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9820003ce0 con 0x7f98301007b0 2024-08-02T17:44:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.647+0000 7f98277fe640 1 -- 172.21.15.69:0/32794169 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820035070 con 0x7f98301007b0 2024-08-02T17:44:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.647+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98301a4c00 con 0x7f98301007b0 2024-08-02T17:44:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.648+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9830100c10 con 0x7f98301007b0 2024-08-02T17:44:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.649+0000 7f98277fe640 1 -- 172.21.15.69:0/32794169 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9820042030 con 0x7f98301007b0 2024-08-02T17:44:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.650+0000 7f98277fe640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f981c0a2b80 0x7f981c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.650+0000 7f982dd9b640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f981c0a2b80 0x7f981c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.650+0000 7f98277fe640 1 -- 172.21.15.69:0/32794169 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f98200ef740 con 0x7f98301007b0 2024-08-02T17:44:08.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.650+0000 7f982dd9b640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f981c0a2b80 0x7f981c0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f983006b010 tx=0x7f9818005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:08.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.654+0000 7f98277fe640 1 -- 172.21.15.69:0/32794169 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f98200bc610 con 0x7f98301007b0 2024-08-02T17:44:08.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:08.823+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9830108790 con 0x7f98301007b0 2024-08-02T17:44:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:09 smithi083 ceph-mon[37031]: pgmap v261: 97 pgs: 23 creating+peering, 74 active+clean; 578 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 725 B/s wr, 5 op/s 2024-08-02T17:44:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:09 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/32794169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-02T17:44:09.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:09 smithi077 ceph-mon[37108]: pgmap v261: 97 pgs: 23 creating+peering, 74 active+clean; 578 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 725 B/s wr, 5 op/s 2024-08-02T17:44:09.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:09 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/32794169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-02T17:44:09.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:09 smithi069 ceph-mon[32746]: pgmap v261: 97 pgs: 23 creating+peering, 74 active+clean; 578 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 725 B/s wr, 5 op/s 2024-08-02T17:44:09.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:09 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/32794169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-08-02T17:44:10.146 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 17:44:09 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b[66767]: 2024-08-02T17:44:09.867+0000 7fa055673640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-02T17:44:10.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.148+0000 7f98277fe640 1 -- 172.21.15.69:0/32794169 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f98200bbf10 con 0x7f98301007b0 2024-08-02T17:44:10.150 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data_ec' created 2024-08-02T17:44:10.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f981c0a2b80 msgr2=0x7f981c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:10.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f981c0a2b80 0x7f981c0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f983006b010 tx=0x7f9818005ec0 comp rx=0 tx=0).stop 2024-08-02T17:44:10.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301007b0 msgr2=0x7f9830199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301007b0 0x7f9830199020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9820009a50 tx=0x7f9820004270 comp rx=0 tx=0).stop 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 shutdown_connections 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f981c0a2b80 0x7f981c0a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98301067b0 0x7f98301a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9830101150 0x7f9830199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 --2- 172.21.15.69:0/32794169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98301007b0 0x7f9830199020 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 >> 172.21.15.69:0/32794169 conn(0x7f98300fc4d0 msgr2=0x7f98300fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 shutdown_connections 2024-08-02T17:44:10.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:10.152+0000 7f9834813640 1 -- 172.21.15.69:0/32794169 wait complete. 2024-08-02T17:44:10.479 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:10 smithi069 ceph-mon[32746]: osdmap e86: 12 total, 12 up, 12 in 2024-08-02T17:44:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:10 smithi083 ceph-mon[37031]: osdmap e86: 12 total, 12 up, 12 in 2024-08-02T17:44:10.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:10 smithi077 ceph-mon[37108]: osdmap e86: 12 total, 12 up, 12 in 2024-08-02T17:44:10.941 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-08-02T17:44:11.223 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:11.498 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:11 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/32794169' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-02T17:44:11.498 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:11 smithi069 ceph-mon[32746]: osdmap e87: 12 total, 12 up, 12 in 2024-08-02T17:44:11.499 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:11 smithi069 ceph-mon[32746]: pgmap v264: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-08-02T17:44:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:11 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/32794169' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-02T17:44:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:11 smithi083 ceph-mon[37031]: osdmap e87: 12 total, 12 up, 12 in 2024-08-02T17:44:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:11 smithi083 ceph-mon[37031]: pgmap v264: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-08-02T17:44:11.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:11 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/32794169' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-02T17:44:11.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:11 smithi077 ceph-mon[37108]: osdmap e87: 12 total, 12 up, 12 in 2024-08-02T17:44:11.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:11 smithi077 ceph-mon[37108]: pgmap v264: 129 pgs: 32 unknown, 97 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-08-02T17:44:11.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 -- 172.21.15.69:0/3994275095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c10bea0 msgr2=0x7ff55c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:11.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 --2- 172.21.15.69:0/3994275095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c10bea0 0x7ff55c10e340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff554009ca0 tx=0x7ff55402a950 comp rx=0 tx=0).stop 2024-08-02T17:44:11.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 -- 172.21.15.69:0/3994275095 shutdown_connections 2024-08-02T17:44:11.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 --2- 172.21.15.69:0/3994275095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c10bea0 0x7ff55c10e340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:11.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 --2- 172.21.15.69:0/3994275095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c103390 0x7ff55c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 --2- 172.21.15.69:0/3994275095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff55c102a70 0x7ff55c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 -- 172.21.15.69:0/3994275095 >> 172.21.15.69:0/3994275095 conn(0x7ff55c0fc4d0 msgr2=0x7ff55c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 -- 172.21.15.69:0/3994275095 shutdown_connections 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.725+0000 7ff561cfd640 1 -- 172.21.15.69:0/3994275095 wait complete. 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff561cfd640 1 Processor -- start 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff561cfd640 1 -- start start 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff561cfd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff55c102a70 0x7ff55c1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff561cfd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c103390 0x7ff55c100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff561cfd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c10bea0 0x7ff55c101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff561cfd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c111ca0 con 0x7ff55c103390 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff561cfd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c111b20 con 0x7ff55c102a70 2024-08-02T17:44:11.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff561cfd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c111e20 con 0x7ff55c10bea0 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff55affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c103390 0x7ff55c100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff55affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c103390 0x7ff55c100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41074/0 (socket says 172.21.15.69:41074) 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff55affd640 1 -- 172.21.15.69:0/55422444 learned_addr learned my addr 172.21.15.69:0/55422444 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.726+0000 7ff55b7fe640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff55c102a70 0x7ff55c1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff55bfff640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c10bea0 0x7ff55c101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff55affd640 1 -- 172.21.15.69:0/55422444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c10bea0 msgr2=0x7ff55c101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff55affd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c10bea0 0x7ff55c101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff55affd640 1 -- 172.21.15.69:0/55422444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff55c102a70 msgr2=0x7ff55c1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff55affd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff55c102a70 0x7ff55c1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff55affd640 1 -- 172.21.15.69:0/55422444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c1a6520 con 0x7ff55c103390 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff55b7fe640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff55c102a70 0x7ff55c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:11.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff55affd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c103390 0x7ff55c100c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c970 tx=0x7ff54400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff558ff9640 1 -- 172.21.15.69:0/55422444 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544007bb0 con 0x7ff55c103390 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff558ff9640 1 -- 172.21.15.69:0/55422444 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff544007d50 con 0x7ff55c103390 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c1a6810 con 0x7ff55c103390 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff558ff9640 1 -- 172.21.15.69:0/55422444 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544018820 con 0x7ff55c103390 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.727+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c1a6d50 con 0x7ff55c103390 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.728+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c102e50 con 0x7ff55c103390 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.729+0000 7ff558ff9640 1 -- 172.21.15.69:0/55422444 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff5440189c0 con 0x7ff55c103390 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.730+0000 7ff558ff9640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5480a2b80 0x7ff5480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.730+0000 7ff55b7fe640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5480a2b80 0x7ff5480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.730+0000 7ff558ff9640 1 -- 172.21.15.69:0/55422444 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7ff5440cf320 con 0x7ff55c103390 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.730+0000 7ff55b7fe640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5480a2b80 0x7ff5480a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0058c0 tx=0x7ff54c005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:11.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.735+0000 7ff558ff9640 1 -- 172.21.15.69:0/55422444 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff544014030 con 0x7ff55c103390 2024-08-02T17:44:11.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:11.902+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ff55c1a70c0 con 0x7ff55c103390 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.161+0000 7ff558ff9640 1 -- 172.21.15.69:0/55422444 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v89) ==== 169+0+0 (secure 0 0 0) 0x7ff54409c060 con 0x7ff55c103390 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:set pool 4 allow_ec_overwrites to true 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5480a2b80 msgr2=0x7ff5480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5480a2b80 0x7ff5480a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0058c0 tx=0x7ff54c005ec0 comp rx=0 tx=0).stop 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c103390 msgr2=0x7ff55c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c103390 0x7ff55c100c10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff54400c970 tx=0x7ff54400ce30 comp rx=0 tx=0).stop 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 shutdown_connections 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5480a2b80 0x7ff5480a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:12.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff55c103390 0x7ff55c100c10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff55c10bea0 0x7ff55c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 --2- 172.21.15.69:0/55422444 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff55c102a70 0x7ff55c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 >> 172.21.15.69:0/55422444 conn(0x7ff55c0fc4d0 msgr2=0x7ff55c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 shutdown_connections 2024-08-02T17:44:12.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:12.164+0000 7ff561cfd640 1 -- 172.21.15.69:0/55422444 wait complete. 2024-08-02T17:44:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:12 smithi083 ceph-mon[37031]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-02T17:44:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:12 smithi083 ceph-mon[37031]: osdmap e88: 12 total, 12 up, 12 in 2024-08-02T17:44:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:12 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/55422444' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-02T17:44:12.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:12 smithi077 ceph-mon[37108]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-02T17:44:12.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:12 smithi077 ceph-mon[37108]: osdmap e88: 12 total, 12 up, 12 in 2024-08-02T17:44:12.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:12 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/55422444' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-02T17:44:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:12 smithi069 ceph-mon[32746]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-02T17:44:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:12 smithi069 ceph-mon[32746]: osdmap e88: 12 total, 12 up, 12 in 2024-08-02T17:44:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:12 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/55422444' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-08-02T17:44:13.063 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-08-02T17:44:13.317 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:13 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/55422444' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-02T17:44:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:13 smithi069 ceph-mon[32746]: osdmap e89: 12 total, 12 up, 12 in 2024-08-02T17:44:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:13 smithi069 ceph-mon[32746]: pgmap v267: 129 pgs: 1 creating+peering, 28 unknown, 100 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-08-02T17:44:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:13 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/55422444' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-02T17:44:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:13 smithi083 ceph-mon[37031]: osdmap e89: 12 total, 12 up, 12 in 2024-08-02T17:44:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:13 smithi083 ceph-mon[37031]: pgmap v267: 129 pgs: 1 creating+peering, 28 unknown, 100 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-08-02T17:44:13.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:13 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/55422444' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-02T17:44:13.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:13 smithi077 ceph-mon[37108]: osdmap e89: 12 total, 12 up, 12 in 2024-08-02T17:44:13.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:13 smithi077 ceph-mon[37108]: pgmap v267: 129 pgs: 1 creating+peering, 28 unknown, 100 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-08-02T17:44:13.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 -- 172.21.15.69:0/3197499157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260103410 msgr2=0x7f92601078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:13.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 --2- 172.21.15.69:0/3197499157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260103410 0x7f92601078c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f925c009930 tx=0x7f925c02a420 comp rx=0 tx=0).stop 2024-08-02T17:44:13.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 -- 172.21.15.69:0/3197499157 shutdown_connections 2024-08-02T17:44:13.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 --2- 172.21.15.69:0/3197499157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260103410 0x7f92601078c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:13.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 --2- 172.21.15.69:0/3197499157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9260107fd0 0x7f926010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:13.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 --2- 172.21.15.69:0/3197499157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9260102a60 0x7f9260102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:13.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 -- 172.21.15.69:0/3197499157 >> 172.21.15.69:0/3197499157 conn(0x7f92600fc500 msgr2=0x7f92600fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:13.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 -- 172.21.15.69:0/3197499157 shutdown_connections 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.823+0000 7f9267273640 1 -- 172.21.15.69:0/3197499157 wait complete. 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9267273640 1 Processor -- start 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9267273640 1 -- start start 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9267273640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260102a60 0x7f92601006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9267273640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9260103410 0x7f9260100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9267273640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9260107fd0 0x7f9260101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9267273640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9260111df0 con 0x7f9260102a60 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9267273640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9260111c70 con 0x7f9260103410 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9267273640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9260111f70 con 0x7f9260107fd0 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f926500f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260102a60 0x7f92601006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f926500f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260102a60 0x7f92601006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41088/0 (socket says 172.21.15.69:41088) 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f926500f640 1 -- 172.21.15.69:0/3170794331 learned_addr learned my addr 172.21.15.69:0/3170794331 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:13.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9265810640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9260107fd0 0x7f9260101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f926500f640 1 -- 172.21.15.69:0/3170794331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9260107fd0 msgr2=0x7f9260101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f926500f640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9260107fd0 0x7f9260101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f926500f640 1 -- 172.21.15.69:0/3170794331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9260103410 msgr2=0x7f9260100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f926500f640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9260103410 0x7f9260100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f926500f640 1 -- 172.21.15.69:0/3170794331 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92601a6810 con 0x7f9260102a60 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.824+0000 7f9265810640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9260107fd0 0x7f9260101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.825+0000 7f926500f640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260102a60 0x7f92601006c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f925000d950 tx=0x7f925000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.825+0000 7f92567fc640 1 -- 172.21.15.69:0/3170794331 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9250017030 con 0x7f9260102a60 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.825+0000 7f92567fc640 1 -- 172.21.15.69:0/3170794331 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9250004020 con 0x7f9260102a60 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.825+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92601a6b00 con 0x7f9260102a60 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.825+0000 7f92567fc640 1 -- 172.21.15.69:0/3170794331 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92500047a0 con 0x7f9260102a60 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.825+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92601a7040 con 0x7f9260102a60 2024-08-02T17:44:13.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.826+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9260102e40 con 0x7f9260102a60 2024-08-02T17:44:13.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.827+0000 7f92567fc640 1 -- 172.21.15.69:0/3170794331 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f92500041d0 con 0x7f9260102a60 2024-08-02T17:44:13.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.828+0000 7f92567fc640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f924c0a2b80 0x7f924c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:13.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.828+0000 7f926480e640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f924c0a2b80 0x7f924c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:13.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.828+0000 7f92567fc640 1 -- 172.21.15.69:0/3170794331 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f92500cf370 con 0x7f9260102a60 2024-08-02T17:44:13.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.828+0000 7f926480e640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f924c0a2b80 0x7f924c0a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f925c02a940 tx=0x7f925c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:13.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:13.831+0000 7f92567fc640 1 -- 172.21.15.69:0/3170794331 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f925009c0b0 con 0x7f9260102a60 2024-08-02T17:44:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.037+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f9260101bf0 con 0x7f9260102a60 2024-08-02T17:44:14.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.176+0000 7f92567fc640 1 -- 172.21.15.69:0/3170794331 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f925009b9b0 con 0x7f9260102a60 2024-08-02T17:44:14.178 INFO:teuthology.orchestra.run.smithi069.stderr:added data pool 4 to fsmap 2024-08-02T17:44:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f924c0a2b80 msgr2=0x7f924c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f924c0a2b80 0x7f924c0a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f925c02a940 tx=0x7f925c000bc0 comp rx=0 tx=0).stop 2024-08-02T17:44:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260102a60 msgr2=0x7f92601006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260102a60 0x7f92601006c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f925000d950 tx=0x7f925000de10 comp rx=0 tx=0).stop 2024-08-02T17:44:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 shutdown_connections 2024-08-02T17:44:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f924c0a2b80 0x7f924c0a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9260107fd0 0x7f9260101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9260103410 0x7f9260100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 --2- 172.21.15.69:0/3170794331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9260102a60 0x7f92601006c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 >> 172.21.15.69:0/3170794331 conn(0x7f92600fc500 msgr2=0x7f9260105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 shutdown_connections 2024-08-02T17:44:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:14.178+0000 7f9267273640 1 -- 172.21.15.69:0/3170794331 wait complete. 2024-08-02T17:44:14.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:14 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3170794331' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-02T17:44:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:14 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3170794331' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-02T17:44:14.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:14 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3170794331' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-08-02T17:44:14.860 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:44:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:15.428 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:15 smithi069 ceph-mon[32746]: osdmap e90: 12 total, 12 up, 12 in 2024-08-02T17:44:15.429 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:15 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3170794331' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-02T17:44:15.429 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:15 smithi069 ceph-mon[32746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:15.429 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:15 smithi069 ceph-mon[32746]: pgmap v269: 129 pgs: 1 creating+peering, 9 unknown, 119 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:15 smithi083 ceph-mon[37031]: osdmap e90: 12 total, 12 up, 12 in 2024-08-02T17:44:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:15 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3170794331' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-02T17:44:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:15 smithi083 ceph-mon[37031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:15 smithi083 ceph-mon[37031]: pgmap v269: 129 pgs: 1 creating+peering, 9 unknown, 119 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:15.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:15 smithi077 ceph-mon[37108]: osdmap e90: 12 total, 12 up, 12 in 2024-08-02T17:44:15.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:15 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3170794331' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-02T17:44:15.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:15 smithi077 ceph-mon[37108]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:15.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:15 smithi077 ceph-mon[37108]: pgmap v269: 129 pgs: 1 creating+peering, 9 unknown, 119 active+clean; 579 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.596+0000 7f28e6342640 1 -- 172.21.15.69:0/3905278891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e01033f0 msgr2=0x7f28e010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.596+0000 7f28e6342640 1 --2- 172.21.15.69:0/3905278891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e01033f0 0x7f28e010b8e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009900 tx=0x7f28c802a480 comp rx=0 tx=0).stop 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 -- 172.21.15.69:0/3905278891 shutdown_connections 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 --2- 172.21.15.69:0/3905278891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e01033f0 0x7f28e010b8e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 --2- 172.21.15.69:0/3905278891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28e010beb0 0x7f28e010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 --2- 172.21.15.69:0/3905278891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e0102ad0 0x7f28e0102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 -- 172.21.15.69:0/3905278891 >> 172.21.15.69:0/3905278891 conn(0x7f28e00fc4d0 msgr2=0x7f28e00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 -- 172.21.15.69:0/3905278891 shutdown_connections 2024-08-02T17:44:15.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 -- 172.21.15.69:0/3905278891 wait complete. 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 Processor -- start 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 -- start start 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.597+0000 7f28e6342640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e0102ad0 0x7f28e0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28e6342640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e01033f0 0x7f28e0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28e6342640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28e010beb0 0x7f28e019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28e6342640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0111d50 con 0x7f28e0102ad0 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28e6342640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0111bd0 con 0x7f28e01033f0 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28e6342640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0111ed0 con 0x7f28e010beb0 2024-08-02T17:44:15.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e0102ad0 0x7f28e0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e0102ad0 0x7f28e0198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35428/0 (socket says 172.21.15.69:35428) 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 -- 172.21.15.69:0/872479224 learned_addr learned my addr 172.21.15.69:0/872479224 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28d7fff640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e01033f0 0x7f28e0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 -- 172.21.15.69:0/872479224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28e010beb0 msgr2=0x7f28e019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28e010beb0 0x7f28e019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 -- 172.21.15.69:0/872479224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e01033f0 msgr2=0x7f28e0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e01033f0 0x7f28e0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 -- 172.21.15.69:0/872479224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e019d2e0 con 0x7f28e0102ad0 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28d7fff640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e01033f0 0x7f28e0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28dffff640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e0102ad0 0x7f28e0198330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f28d000b4a0 tx=0x7f28d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28ddffb640 1 -- 172.21.15.69:0/872479224 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0014070 con 0x7f28e0102ad0 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28ddffb640 1 -- 172.21.15.69:0/872479224 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d0010af0 con 0x7f28e0102ad0 2024-08-02T17:44:15.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e019d5d0 con 0x7f28e0102ad0 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28ddffb640 1 -- 172.21.15.69:0/872479224 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0017d10 con 0x7f28e0102ad0 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.598+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e01a4eb0 con 0x7f28e0102ad0 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.599+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e0102f30 con 0x7f28e0102ad0 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.600+0000 7f28ddffb640 1 -- 172.21.15.69:0/872479224 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f28d0017070 con 0x7f28e0102ad0 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.601+0000 7f28ddffb640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f28cc0a2b80 0x7f28cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.601+0000 7f28d7fff640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f28cc0a2b80 0x7f28cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.601+0000 7f28ddffb640 1 -- 172.21.15.69:0/872479224 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f28d00ceaf0 con 0x7f28e0102ad0 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.601+0000 7f28d7fff640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f28cc0a2b80 0x7f28cc0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000c00 tx=0x7f28c8032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:15.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.604+0000 7f28ddffb640 1 -- 172.21.15.69:0/872479224 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f28d009b7b0 con 0x7f28e0102ad0 2024-08-02T17:44:15.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.803+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28e0106f20 con 0x7f28e0102ad0 2024-08-02T17:44:15.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.804+0000 7f28ddffb640 1 -- 172.21.15.69:0/872479224 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11827 (secure 0 0 0) 0x7f28d009b0b0 con 0x7f28e0102ad0 2024-08-02T17:44:15.806 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:15.806 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"btime":"2024-08-02T17:44:14:170198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:44:14.039103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24452,"qdb_cluster":[24452]},"id":1}]} 2024-08-02T17:44:15.806 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 17 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f28cc0a2b80 msgr2=0x7f28cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f28cc0a2b80 0x7f28cc0a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000c00 tx=0x7f28c8032d50 comp rx=0 tx=0).stop 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e0102ad0 msgr2=0x7f28e0198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e0102ad0 0x7f28e0198330 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f28d000b4a0 tx=0x7f28d000b960 comp rx=0 tx=0).stop 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 shutdown_connections 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f28cc0a2b80 0x7f28cc0a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28e010beb0 0x7f28e019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28e01033f0 0x7f28e0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 --2- 172.21.15.69:0/872479224 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28e0102ad0 0x7f28e0198330 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 >> 172.21.15.69:0/872479224 conn(0x7f28e00fc4d0 msgr2=0x7f28e00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 shutdown_connections 2024-08-02T17:44:15.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:15.806+0000 7f28e6342640 1 -- 172.21.15.69:0/872479224 wait complete. 2024-08-02T17:44:16.497 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:16 smithi069 ceph-mon[32746]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-02T17:44:16.497 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:16 smithi069 ceph-mon[32746]: Cluster is now healthy 2024-08-02T17:44:16.497 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:16 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/872479224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:44:16.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:16 smithi083 ceph-mon[37031]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-02T17:44:16.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:16 smithi083 ceph-mon[37031]: Cluster is now healthy 2024-08-02T17:44:16.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:16 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/872479224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:44:16.526 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:44:16.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:16 smithi077 ceph-mon[37108]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-02T17:44:16.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:16 smithi077 ceph-mon[37108]: Cluster is now healthy 2024-08-02T17:44:16.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:16 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/872479224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:44:16.783 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:17.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 -- 172.21.15.69:0/3614913752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27401022a0 msgr2=0x7f27401026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:17.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3614913752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27401022a0 0x7f27401026e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f273c009900 tx=0x7f273c02a480 comp rx=0 tx=0).stop 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 -- 172.21.15.69:0/3614913752 shutdown_connections 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3614913752 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27401022a0 0x7f27401026e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3614913752 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2740102c20 0x7f274010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3614913752 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2740100b50 0x7f2740100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 -- 172.21.15.69:0/3614913752 >> 172.21.15.69:0/3614913752 conn(0x7f27400fc500 msgr2=0x7f27400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 -- 172.21.15.69:0/3614913752 shutdown_connections 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 -- 172.21.15.69:0/3614913752 wait complete. 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 Processor -- start 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.237+0000 7f2747ea4640 1 -- start start 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2747ea4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2740100b50 0x7f27401981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:17.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2747ea4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27401022a0 0x7f2740198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2747ea4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740102c20 0x7f274019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2747ea4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2740111dd0 con 0x7f2740102c20 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2747ea4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2740111c50 con 0x7f27401022a0 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2747ea4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2740111f50 con 0x7f2740100b50 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2746441640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740102c20 0x7f274019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27401022a0 0x7f2740198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27401022a0 0x7f2740198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59100/0 (socket says 172.21.15.69:59100) 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 -- 172.21.15.69:0/3461794354 learned_addr learned my addr 172.21.15.69:0/3461794354 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2745c40640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2740100b50 0x7f27401981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 -- 172.21.15.69:0/3461794354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2740100b50 msgr2=0x7f27401981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2740100b50 0x7f27401981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 -- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740102c20 msgr2=0x7f274019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740102c20 0x7f274019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 -- 172.21.15.69:0/3461794354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274019d1a0 con 0x7f27401022a0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2746441640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740102c20 0x7f274019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f274543f640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27401022a0 0x7f2740198730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f273c006010 tx=0x7f273c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2732ffd640 1 -- 172.21.15.69:0/3461794354 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c03f070 con 0x7f27401022a0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274019d430 con 0x7f27401022a0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2732ffd640 1 -- 172.21.15.69:0/3461794354 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f273c031c30 con 0x7f27401022a0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.238+0000 7f2732ffd640 1 -- 172.21.15.69:0/3461794354 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c035030 con 0x7f27401022a0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.239+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27401a4d70 con 0x7f27401022a0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.239+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2740100f30 con 0x7f27401022a0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.241+0000 7f2732ffd640 1 -- 172.21.15.69:0/3461794354 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f273c042070 con 0x7f27401022a0 2024-08-02T17:44:17.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.241+0000 7f2732ffd640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f272c0a2b80 0x7f272c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:17.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.241+0000 7f2745c40640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f272c0a2b80 0x7f272c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:17.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.241+0000 7f2732ffd640 1 -- 172.21.15.69:0/3461794354 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f273c0eff80 con 0x7f27401022a0 2024-08-02T17:44:17.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.242+0000 7f2745c40640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f272c0a2b80 0x7f272c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2734005d00 tx=0x7f273400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:17.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.245+0000 7f2732ffd640 1 -- 172.21.15.69:0/3461794354 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f273c031de0 con 0x7f27401022a0 2024-08-02T17:44:17.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.413+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2740108dc0 con 0x7f27401022a0 2024-08-02T17:44:17.415 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:17 smithi069 ceph-mon[32746]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:17.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.414+0000 7f2732ffd640 1 -- 172.21.15.69:0/3461794354 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24021 (secure 0 0 0) 0x7f273c0bc710 con 0x7f27401022a0 2024-08-02T17:44:17.416 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:17.416 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:14.038953+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:44:17.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f272c0a2b80 msgr2=0x7f272c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:17.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f272c0a2b80 0x7f272c0a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2734005d00 tx=0x7f273400b410 comp rx=0 tx=0).stop 2024-08-02T17:44:17.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27401022a0 msgr2=0x7f2740198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:17.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27401022a0 0x7f2740198730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f273c006010 tx=0x7f273c003c10 comp rx=0 tx=0).stop 2024-08-02T17:44:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 shutdown_connections 2024-08-02T17:44:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f272c0a2b80 0x7f272c0a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2740102c20 0x7f274019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27401022a0 0x7f2740198730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 --2- 172.21.15.69:0/3461794354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2740100b50 0x7f27401981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 >> 172.21.15.69:0/3461794354 conn(0x7f27400fc500 msgr2=0x7f27400fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 shutdown_connections 2024-08-02T17:44:17.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:17.417+0000 7f2747ea4640 1 -- 172.21.15.69:0/3461794354 wait complete. 2024-08-02T17:44:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:17 smithi083 ceph-mon[37031]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:17.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:17 smithi077 ceph-mon[37108]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:18.068 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:44:18.322 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:18.443 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:18 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3461794354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:18.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:18 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3461794354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:18.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:18 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3461794354' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 -- 172.21.15.69:0/161219340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12201067b0 msgr2=0x7f1220106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 --2- 172.21.15.69:0/161219340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12201067b0 0x7f1220106b90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1210009930 tx=0x7f121002a420 comp rx=0 tx=0).stop 2024-08-02T17:44:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 -- 172.21.15.69:0/161219340 shutdown_connections 2024-08-02T17:44:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 --2- 172.21.15.69:0/161219340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1220101150 0x7f122010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 --2- 172.21.15.69:0/161219340 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12201007b0 0x7f1220100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 --2- 172.21.15.69:0/161219340 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12201067b0 0x7f1220106b90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 -- 172.21.15.69:0/161219340 >> 172.21.15.69:0/161219340 conn(0x7f12200fc4d0 msgr2=0x7f12200fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 -- 172.21.15.69:0/161219340 shutdown_connections 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 -- 172.21.15.69:0/161219340 wait complete. 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 Processor -- start 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.768+0000 7f1226610640 1 -- start start 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1226610640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12201007b0 0x7f1220199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1226610640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1220101150 0x7f1220199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1226610640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12201067b0 0x7f12201a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1226610640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1220110bd0 con 0x7f1220101150 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1226610640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1220110a50 con 0x7f12201007b0 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1226610640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1220110d50 con 0x7f12201067b0 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1220101150 0x7f1220199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1220101150 0x7f1220199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35466/0 (socket says 172.21.15.69:35466) 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 -- 172.21.15.69:0/2314889980 learned_addr learned my addr 172.21.15.69:0/2314889980 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f121ffff640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12201007b0 0x7f1220199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1224bad640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12201067b0 0x7f12201a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 -- 172.21.15.69:0/2314889980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12201067b0 msgr2=0x7f12201a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12201067b0 0x7f12201a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 -- 172.21.15.69:0/2314889980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12201007b0 msgr2=0x7f1220199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12201007b0 0x7f1220199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 -- 172.21.15.69:0/2314889980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12201a4600 con 0x7f1220101150 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1224bad640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12201067b0 0x7f12201a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1217fff640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1220101150 0x7f1220199580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f120800d950 tx=0x7f120800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f121dffb640 1 -- 172.21.15.69:0/2314889980 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208017070 con 0x7f1220101150 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f121dffb640 1 -- 172.21.15.69:0/2314889980 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1208009c30 con 0x7f1220101150 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12201a47d0 con 0x7f1220101150 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f121dffb640 1 -- 172.21.15.69:0/2314889980 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800f6b0 con 0x7f1220101150 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.769+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12201a4ce0 con 0x7f1220101150 2024-08-02T17:44:18.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.770+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1220101ef0 con 0x7f1220101150 2024-08-02T17:44:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.771+0000 7f121dffb640 1 -- 172.21.15.69:0/2314889980 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f12080109d0 con 0x7f1220101150 2024-08-02T17:44:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.772+0000 7f121dffb640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f120c0a2b80 0x7f120c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.772+0000 7f121ffff640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f120c0a2b80 0x7f120c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.772+0000 7f121dffb640 1 -- 172.21.15.69:0/2314889980 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f12080ceed0 con 0x7f1220101150 2024-08-02T17:44:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.772+0000 7f121ffff640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f120c0a2b80 0x7f120c0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1210009900 tx=0x7f12100058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:18.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.776+0000 7f121dffb640 1 -- 172.21.15.69:0/2314889980 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1208009de0 con 0x7f1220101150 2024-08-02T17:44:18.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.944+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1220108d20 con 0x7f1220101150 2024-08-02T17:44:18.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.945+0000 7f121dffb640 1 -- 172.21.15.69:0/2314889980 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24021 (secure 0 0 0) 0x7f120809bbf0 con 0x7f1220101150 2024-08-02T17:44:18.947 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:18.947 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:14.038953+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:44:18.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.947+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f120c0a2b80 msgr2=0x7f120c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:18.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.947+0000 7f1226610640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f120c0a2b80 0x7f120c0a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1210009900 tx=0x7f12100058d0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.947+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1220101150 msgr2=0x7f1220199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:18.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.947+0000 7f1226610640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1220101150 0x7f1220199580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f120800d950 tx=0x7f120800de10 comp rx=0 tx=0).stop 2024-08-02T17:44:18.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.948+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 shutdown_connections 2024-08-02T17:44:18.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.948+0000 7f1226610640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f120c0a2b80 0x7f120c0a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.948+0000 7f1226610640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1220101150 0x7f1220199580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.948+0000 7f1226610640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12201067b0 0x7f12201a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.948+0000 7f1226610640 1 --2- 172.21.15.69:0/2314889980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12201007b0 0x7f1220199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:18.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.948+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 >> 172.21.15.69:0/2314889980 conn(0x7f12200fc4d0 msgr2=0x7f12200fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:18.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.948+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 shutdown_connections 2024-08-02T17:44:18.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:18.948+0000 7f1226610640 1 -- 172.21.15.69:0/2314889980 wait complete. 2024-08-02T17:44:19.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:19 smithi069 ceph-mon[32746]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:19.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:19 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2314889980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:19.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:19 smithi083 ceph-mon[37031]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:19.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:19 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2314889980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:19.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:19 smithi077 ceph-mon[37108]: pgmap v271: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:19.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:19 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2314889980' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:19.678 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:44:19.930 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 -- 172.21.15.69:0/1310401329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0102c20 msgr2=0x7fe8c010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/1310401329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0102c20 0x7fe8c010d770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009ca0 tx=0x7fe8b802a950 comp rx=0 tx=0).stop 2024-08-02T17:44:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 -- 172.21.15.69:0/1310401329 shutdown_connections 2024-08-02T17:44:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/1310401329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0102c20 0x7fe8c010d770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/1310401329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c01022a0 0x7fe8c01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/1310401329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c0100b50 0x7fe8c0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 -- 172.21.15.69:0/1310401329 >> 172.21.15.69:0/1310401329 conn(0x7fe8c00fc500 msgr2=0x7fe8c00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 -- 172.21.15.69:0/1310401329 shutdown_connections 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 -- 172.21.15.69:0/1310401329 wait complete. 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 Processor -- start 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.559+0000 7fe8c82d4640 1 -- start start 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c82d4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0100b50 0x7fe8c0197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c82d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c01022a0 0x7fe8c0197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c82d4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c0102c20 0x7fe8c019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c82d4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0110cf0 con 0x7fe8c0100b50 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c82d4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0110b70 con 0x7fe8c0102c20 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c82d4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c0110e70 con 0x7fe8c01022a0 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0100b50 0x7fe8c0197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0100b50 0x7fe8c0197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35478/0 (socket says 172.21.15.69:35478) 2024-08-02T17:44:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 -- 172.21.15.69:0/294267056 learned_addr learned my addr 172.21.15.69:0/294267056 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6871640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c0102c20 0x7fe8c019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c586f640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c01022a0 0x7fe8c0197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 -- 172.21.15.69:0/294267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c01022a0 msgr2=0x7fe8c0197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c01022a0 0x7fe8c0197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 -- 172.21.15.69:0/294267056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c0102c20 msgr2=0x7fe8c019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c0102c20 0x7fe8c019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 -- 172.21.15.69:0/294267056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c019c0e0 con 0x7fe8c0100b50 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c586f640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c01022a0 0x7fe8c0197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6871640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c0102c20 0x7fe8c019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c6070640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0100b50 0x7fe8c0197130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00d990 tx=0x7fe8bc00de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8b37fe640 1 -- 172.21.15.69:0/294267056 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc01b070 con 0x7fe8c0100b50 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c019c3d0 con 0x7fe8c0100b50 2024-08-02T17:44:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8b37fe640 1 -- 172.21.15.69:0/294267056 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc00edf0 con 0x7fe8c0100b50 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8b37fe640 1 -- 172.21.15.69:0/294267056 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc014870 con 0x7fe8c0100b50 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.560+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c0069ca0 con 0x7fe8c0100b50 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.561+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c0100f30 con 0x7fe8c0100b50 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.563+0000 7fe8b37fe640 1 -- 172.21.15.69:0/294267056 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe8bc01e070 con 0x7fe8c0100b50 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.563+0000 7fe8b37fe640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.563+0000 7fe8c586f640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.563+0000 7fe8b37fe640 1 -- 172.21.15.69:0/294267056 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe8bc0d0ec0 con 0x7fe8c0100b50 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.564+0000 7fe8c586f640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00fe940 tx=0x7fe8b4005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.567+0000 7fe8b37fe640 1 -- 172.21.15.69:0/294267056 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe8bc09dbe0 con 0x7fe8c0100b50 2024-08-02T17:44:20.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.767+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8c0198490 con 0x7fe8c0100b50 2024-08-02T17:44:20.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.767+0000 7fe8b37fe640 1 -- 172.21.15.69:0/294267056 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11827 (secure 0 0 0) 0x7fe8bc09d4e0 con 0x7fe8c0100b50 2024-08-02T17:44:20.769 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:20.770 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"btime":"2024-08-02T17:44:14:170198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24403,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24413,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24427,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24431,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:44:14.039103+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24452},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24452,"qdb_cluster":[24452]},"id":1}]} 2024-08-02T17:44:20.770 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 17 2024-08-02T17:44:20.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8ac0a2b80 msgr2=0x7fe8ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:20.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe8c00fe940 tx=0x7fe8b4005d10 comp rx=0 tx=0).stop 2024-08-02T17:44:20.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0100b50 msgr2=0x7fe8c0197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:20.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0100b50 0x7fe8c0197130 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00d990 tx=0x7fe8bc00de50 comp rx=0 tx=0).stop 2024-08-02T17:44:20.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 shutdown_connections 2024-08-02T17:44:20.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8ac0a2b80 0x7fe8ac0a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8c0102c20 0x7fe8c019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8c01022a0 0x7fe8c0197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.769+0000 7fe8c82d4640 1 --2- 172.21.15.69:0/294267056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8c0100b50 0x7fe8c0197130 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:20.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.770+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 >> 172.21.15.69:0/294267056 conn(0x7fe8c00fc500 msgr2=0x7fe8c0106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:20.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.770+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 shutdown_connections 2024-08-02T17:44:20.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:20.770+0000 7fe8c82d4640 1 -- 172.21.15.69:0/294267056 wait complete. 2024-08-02T17:44:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:20 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/294267056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:44:21.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:20 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/294267056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:44:21.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:20 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/294267056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:44:21.448 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:44:21.698 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:22.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:21 smithi069 ceph-mon[32746]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:22.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:21 smithi077 ceph-mon[37108]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:22.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.162+0000 7fdc97569640 1 -- 172.21.15.69:0/3966460856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc90101150 msgr2=0x7fdc9010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.162+0000 7fdc97569640 1 --2- 172.21.15.69:0/3966460856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc90101150 0x7fdc9010d680 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdc88009cd0 tx=0x7fdc8802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.162+0000 7fdc97569640 1 -- 172.21.15.69:0/3966460856 shutdown_connections 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.162+0000 7fdc97569640 1 --2- 172.21.15.69:0/3966460856 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc90101150 0x7fdc9010d680 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.162+0000 7fdc97569640 1 --2- 172.21.15.69:0/3966460856 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc901007b0 0x7fdc90100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.162+0000 7fdc97569640 1 --2- 172.21.15.69:0/3966460856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc901067b0 0x7fdc90106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.162+0000 7fdc97569640 1 -- 172.21.15.69:0/3966460856 >> 172.21.15.69:0/3966460856 conn(0x7fdc900fc4d0 msgr2=0x7fdc900fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 -- 172.21.15.69:0/3966460856 shutdown_connections 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 -- 172.21.15.69:0/3966460856 wait complete. 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 Processor -- start 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 -- start start 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc901007b0 0x7fdc90198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:22.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90101150 0x7fdc901994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc901067b0 0x7fdc901a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90110bb0 con 0x7fdc901067b0 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90110a30 con 0x7fdc901007b0 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc97569640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc90110d30 con 0x7fdc90101150 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc95b06640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc901067b0 0x7fdc901a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.163+0000 7fdc95b06640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc901067b0 0x7fdc901a4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35494/0 (socket says 172.21.15.69:35494) 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc95b06640 1 -- 172.21.15.69:0/3876318217 learned_addr learned my addr 172.21.15.69:0/3876318217 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc94b04640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90101150 0x7fdc901994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc95305640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc901007b0 0x7fdc90198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc95b06640 1 -- 172.21.15.69:0/3876318217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90101150 msgr2=0x7fdc901994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc95b06640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90101150 0x7fdc901994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc95b06640 1 -- 172.21.15.69:0/3876318217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc901007b0 msgr2=0x7fdc90198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:22.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc95b06640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc901007b0 0x7fdc90198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc95b06640 1 -- 172.21.15.69:0/3876318217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc901a4540 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc95b06640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc901067b0 0x7fdc901a4000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdc88005cc0 tx=0x7fdc88032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc867fc640 1 -- 172.21.15.69:0/3876318217 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88004040 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc901a4710 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc901a4c40 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc867fc640 1 -- 172.21.15.69:0/3876318217 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc880046c0 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.164+0000 7fdc867fc640 1 -- 172.21.15.69:0/3876318217 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc88030e00 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.165+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc90101ef0 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.167+0000 7fdc867fc640 1 -- 172.21.15.69:0/3876318217 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdc880305a0 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.168+0000 7fdc867fc640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc7c0a2b80 0x7fdc7c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.168+0000 7fdc95305640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc7c0a2b80 0x7fdc7c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.168+0000 7fdc867fc640 1 -- 172.21.15.69:0/3876318217 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fdc8802e030 con 0x7fdc901067b0 2024-08-02T17:44:22.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.168+0000 7fdc95305640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc7c0a2b80 0x7fdc7c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c005fd0 tx=0x7fdc8c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:22.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.171+0000 7fdc867fc640 1 -- 172.21.15.69:0/3876318217 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdc8804b060 con 0x7fdc901067b0 2024-08-02T17:44:22.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:21 smithi083 ceph-mon[37031]: pgmap v272: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:22.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.338+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc90108d20 con 0x7fdc901067b0 2024-08-02T17:44:22.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.340+0000 7fdc867fc640 1 -- 172.21.15.69:0/3876318217 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24021 (secure 0 0 0) 0x7fdc880bd6e0 con 0x7fdc901067b0 2024-08-02T17:44:22.342 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:22.342 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:14.038953+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:44:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc7c0a2b80 msgr2=0x7fdc7c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc7c0a2b80 0x7fdc7c0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c005fd0 tx=0x7fdc8c005850 comp rx=0 tx=0).stop 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc901067b0 msgr2=0x7fdc901a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc901067b0 0x7fdc901a4000 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdc88005cc0 tx=0x7fdc88032f40 comp rx=0 tx=0).stop 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 shutdown_connections 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc7c0a2b80 0x7fdc7c0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc901067b0 0x7fdc901a4000 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc90101150 0x7fdc901994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 --2- 172.21.15.69:0/3876318217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc901007b0 0x7fdc90198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 >> 172.21.15.69:0/3876318217 conn(0x7fdc900fc4d0 msgr2=0x7fdc900fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 shutdown_connections 2024-08-02T17:44:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:22.342+0000 7fdc97569640 1 -- 172.21.15.69:0/3876318217 wait complete. 2024-08-02T17:44:23.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:22 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3876318217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:23.064 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-02T17:44:04.896710+0000', 'modified': '2024-08-02T17:44:14.039103+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24452}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.77:6839/1183977027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 1183977027}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 1183977027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24452, 'qdb_cluster': [24452]} 2024-08-02T17:44:23.064 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-02T17:44:23.064 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.b status 2024-08-02T17:44:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:22 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3876318217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:23.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:22 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3876318217' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:23.319 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:23.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.784+0000 7f061b300640 1 -- 172.21.15.69:0/3990017296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f061410bea0 msgr2=0x7f061410e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.784+0000 7f061b300640 1 --2- 172.21.15.69:0/3990017296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f061410bea0 0x7f061410e340 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f060c009ca0 tx=0x7f060c02a950 comp rx=0 tx=0).stop 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.784+0000 7f061b300640 1 -- 172.21.15.69:0/3990017296 shutdown_connections 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.784+0000 7f061b300640 1 --2- 172.21.15.69:0/3990017296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f061410bea0 0x7f061410e340 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.784+0000 7f061b300640 1 --2- 172.21.15.69:0/3990017296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614103390 0x7f061410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.784+0000 7f061b300640 1 --2- 172.21.15.69:0/3990017296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0614102a70 0x7f0614102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.784+0000 7f061b300640 1 -- 172.21.15.69:0/3990017296 >> 172.21.15.69:0/3990017296 conn(0x7f06140fc4d0 msgr2=0x7f06140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 -- 172.21.15.69:0/3990017296 shutdown_connections 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 -- 172.21.15.69:0/3990017296 wait complete. 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 Processor -- start 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 -- start start 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0614102a70 0x7f06141006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614103390 0x7f0614100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061410bea0 0x7f0614101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0614111ce0 con 0x7f0614102a70 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0614111b60 con 0x7f061410bea0 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061b300640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0614111e60 con 0x7f0614103390 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061909c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0614102a70 0x7f06141006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061909c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0614102a70 0x7f06141006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35518/0 (socket says 172.21.15.69:35518) 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.785+0000 7f061909c640 1 -- 172.21.15.69:0/1338927331 learned_addr learned my addr 172.21.15.69:0/1338927331 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061889b640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614103390 0x7f0614100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061989d640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061410bea0 0x7f0614101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061909c640 1 -- 172.21.15.69:0/1338927331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614103390 msgr2=0x7f0614100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061909c640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614103390 0x7f0614100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061909c640 1 -- 172.21.15.69:0/1338927331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061410bea0 msgr2=0x7f0614101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061909c640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061410bea0 0x7f0614101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061909c640 1 -- 172.21.15.69:0/1338927331 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06141a65c0 con 0x7f0614102a70 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061889b640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614103390 0x7f0614100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061909c640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0614102a70 0x7f06141006d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f061000c920 tx=0x7f061000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f060a7fc640 1 -- 172.21.15.69:0/1338927331 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610007c70 con 0x7f0614102a70 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061b300640 1 -- 172.21.15.69:0/1338927331 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06141a68b0 con 0x7f0614102a70 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f060a7fc640 1 -- 172.21.15.69:0/1338927331 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0610007e10 con 0x7f0614102a70 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f060a7fc640 1 -- 172.21.15.69:0/1338927331 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610018890 con 0x7f0614102a70 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.786+0000 7f061b300640 1 -- 172.21.15.69:0/1338927331 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06141a6e40 con 0x7f0614102a70 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.788+0000 7f060a7fc640 1 -- 172.21.15.69:0/1338927331 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0610018a70 con 0x7f0614102a70 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061d6fe740 1 Processor -- start 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061d6fe740 1 -- start start 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e16a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e23c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061d6fe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555aa5d7b710 con 0x555aa5e21810 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061d6fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aa5d13770 con 0x555aa5e16f50 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061d6fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555aa5d776d0 con 0x555aa5e16630 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f060a7fc640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f06000a2b80 0x7f06000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061889b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e23c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061889b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e23c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35534/0 (socket says 172.21.15.69:35534) 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061889b640 1 -- 172.21.15.69:0/4219895613 learned_addr learned my addr 172.21.15.69:0/4219895613 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.789+0000 7f061989d640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f06000a2b80 0x7f06000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061989d640 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e16a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f060a7fc640 1 -- 172.21.15.69:0/1338927331 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f06100cf480 con 0x7f0614102a70 2024-08-02T17:44:23.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061909c640 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061889b640 1 -- 172.21.15.69:0/4219895613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 msgr2=0x555aa5e16a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061889b640 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e16a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061889b640 1 -- 172.21.15.69:0/4219895613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 msgr2=0x555aa5e212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061889b640 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061889b640 1 -- 172.21.15.69:0/4219895613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa5d1a3e0 con 0x555aa5e21810 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061909c640 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061889b640 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e23c00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f06040077c0 tx=0x7f0604007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d910e053e1af436d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f061989d640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f06000a2b80 0x7f06000a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f060c030040 tx=0x7f060c005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f05ff7fe640 1 -- 172.21.15.69:0/4219895613 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060400d950 con 0x555aa5e21810 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f05ff7fe640 1 -- 172.21.15.69:0/4219895613 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0604018070 con 0x555aa5e21810 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.790+0000 7f05ff7fe640 1 -- 172.21.15.69:0/4219895613 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06040123e0 con 0x555aa5e21810 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 -- 172.21.15.69:0/4219895613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 msgr2=0x555aa5e23c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e23c00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f06040077c0 tx=0x7f0604007c80 comp rx=0 tx=0).stop 2024-08-02T17:44:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 -- 172.21.15.69:0/4219895613 shutdown_connections 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e23c00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 --2- 172.21.15.69:0/4219895613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e16a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 -- 172.21.15.69:0/4219895613 >> 172.21.15.69:0/4219895613 conn(0x555aa5e12bc0 msgr2=0x555aa5e12fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 -- 172.21.15.69:0/4219895613 shutdown_connections 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 -- 172.21.15.69:0/4219895613 wait complete. 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 -- start start 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.791+0000 7f061d6fe740 1 Processor -- start 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16630 0x555aa5e1e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16f50 0x555aa5e1eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 0x555aa5e1f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555aa5e243e0 con 0x555aa5e16630 2024-08-02T17:44:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aa5d36970 con 0x555aa5e21810 2024-08-02T17:44:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555aa5e24560 con 0x555aa5e16f50 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 10 client.0 _ll_register_callbacks cb 0x555aa5a8a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16630 0x555aa5e1e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16630 0x555aa5e1e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35540/0 (socket says 172.21.15.69:35540) 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 -- 172.21.15.69:0/1100661649 learned_addr learned my addr 172.21.15.69:0/1100661649 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061989d640 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 0x555aa5e1f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061889b640 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16f50 0x555aa5e1eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 -- 172.21.15.69:0/1100661649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16f50 msgr2=0x555aa5e1eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16f50 0x555aa5e1eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 -- 172.21.15.69:0/1100661649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 msgr2=0x555aa5e1f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 0x555aa5e1f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 -- 172.21.15.69:0/1100661649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa5aed2b0 con 0x555aa5e16630 2024-08-02T17:44:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061909c640 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16630 0x555aa5e1e640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0610007590 tx=0x7f061009b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894fc1c38137e8c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f05fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f05fcff9640 1 -- 172.21.15.69:0/1100661649 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061009baa0 con 0x555aa5e16630 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f05fcff9640 1 -- 172.21.15.69:0/1100661649 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f0610018d60 con 0x555aa5e16630 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 1 -- 172.21.15.69:0/1100661649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0604018070 con 0x555aa5e16630 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f05fcff9640 1 -- 172.21.15.69:0/1100661649 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f061009fe50 con 0x555aa5e16630 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 10 client.14664 fetch_fsmap learned FSMap version 17 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.792+0000 7f061d6fe740 1 -- 172.21.15.69:0/1100661649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x555aa5e2ff90 con 0x555aa5e16630 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f05fcff9640 1 -- 172.21.15.69:0/1100661649 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0604018070 con 0x555aa5e16630 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f05fcff9640 1 -- 172.21.15.69:0/1100661649 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f06100a4280 con 0x555aa5e16630 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f061d6fe740 10 client.14664 fetch_fsmap finished waiting for FSMap version 17 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f061d6fe740 10 client.14664 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f061d6fe740 20 client.14664 populate_metadata read hostname 'smithi069' 2024-08-02T17:44:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f061d6fe740 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa5e32510 0x555aa5e33980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f061d6fe740 4 client.14664 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f061d6fe740 1 -- 172.21.15.69:0/1100661649 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa5d88c80 con 0x555aa5e32510 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.793+0000 7f061889b640 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa5e32510 0x555aa5e33980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.794+0000 7f05fcff9640 10 client.14664 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.794+0000 7f061889b640 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa5e32510 0x555aa5e33980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d8606701e23f62 server_cookie=ea532cdc4c034e16 in_seq=0 out_seq=0 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.794+0000 7f05fcff9640 1 -- 172.21.15.69:0/1100661649 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555aa5d88c80 con 0x555aa5e32510 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.794+0000 7f05fcff9640 10 client.14664 handle_command_reply: tid=0 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.794+0000 7f061d6fe740 1 client.14664 shutdown 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 -- 172.21.15.69:0/1100661649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16630 msgr2=0x555aa5e1e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16630 0x555aa5e1e640 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0610007590 tx=0x7f061009b4c0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 -- 172.21.15.69:0/1100661649 shutdown_connections 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa5e32510 0x555aa5e33980 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 0x555aa5e1f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16f50 0x555aa5e1eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 --2- 172.21.15.69:0/1100661649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16630 0x555aa5e1e640 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 -- 172.21.15.69:0/1100661649 >> 172.21.15.69:0/1100661649 conn(0x555aa5e12bc0 msgr2=0x555aa5e188a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f05fcff9640 0 client.14664 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:44:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 -- 172.21.15.69:0/1100661649 shutdown_connections 2024-08-02T17:44:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 1 -- 172.21.15.69:0/1100661649 wait complete. 2024-08-02T17:44:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.795+0000 7f061d6fe740 20 client.14664 trim_cache size 0 max 16384 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061d6fe740 1 Processor -- start 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061d6fe740 1 -- start start 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e32be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e12fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061d6fe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555aa5e31cc0 con 0x555aa5e21810 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061d6fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aa5e32390 con 0x555aa5e16f50 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061d6fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555aa5e32a60 con 0x555aa5e16630 2024-08-02T17:44:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e12fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e12fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35546/0 (socket says 172.21.15.69:35546) 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 -- 172.21.15.69:0/2977979231 learned_addr learned my addr 172.21.15.69:0/2977979231 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061889b640 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e32be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 -- 172.21.15.69:0/2977979231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 msgr2=0x555aa5e20da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061909c640 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e20da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 -- 172.21.15.69:0/2977979231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 msgr2=0x555aa5e32be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e32be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 -- 172.21.15.69:0/2977979231 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa5e129f0 con 0x555aa5e21810 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061889b640 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e32be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.813+0000 7f061989d640 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e12fa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f060c035000 tx=0x7f060c03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8883d5a6ecacc51 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f05feffd640 1 -- 172.21.15.69:0/2977979231 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c034040 con 0x555aa5e21810 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f05feffd640 1 -- 172.21.15.69:0/2977979231 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c003c70 con 0x555aa5e21810 2024-08-02T17:44:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f05feffd640 1 -- 172.21.15.69:0/2977979231 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c04a860 con 0x555aa5e21810 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 -- 172.21.15.69:0/2977979231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 msgr2=0x555aa5e12fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e12fa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f060c035000 tx=0x7f060c03d040 comp rx=0 tx=0).stop 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 -- 172.21.15.69:0/2977979231 shutdown_connections 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e21810 0x555aa5e12fa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e16f50 0x555aa5e32be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 --2- 172.21.15.69:0/2977979231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e20da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 -- 172.21.15.69:0/2977979231 >> 172.21.15.69:0/2977979231 conn(0x555aa5e12bc0 msgr2=0x555aa5e1cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 -- 172.21.15.69:0/2977979231 shutdown_connections 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.814+0000 7f061d6fe740 1 -- 172.21.15.69:0/2977979231 wait complete. 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 1 -- start start 2024-08-02T17:44:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 1 Processor -- start 2024-08-02T17:44:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e22b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16f50 0x555aa5e230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 0x555aa5e23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555aa5e33800 con 0x555aa5e16f50 2024-08-02T17:44:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aa5e33130 con 0x555aa5e21810 2024-08-02T17:44:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555aa5e33ed0 con 0x555aa5e16630 2024-08-02T17:44:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061d6fe740 10 client.0 _ll_register_callbacks cb 0x7f061000a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:44:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061889b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16f50 0x555aa5e230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061889b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16f50 0x555aa5e230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35562/0 (socket says 172.21.15.69:35562) 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061889b640 1 -- 172.21.15.69:0/3809247431 learned_addr learned my addr 172.21.15.69:0/3809247431 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061889b640 1 -- 172.21.15.69:0/3809247431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 msgr2=0x555aa5e22b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061889b640 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e22b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061889b640 1 -- 172.21.15.69:0/3809247431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 msgr2=0x555aa5e23600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061889b640 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 0x555aa5e23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.815+0000 7f061889b640 1 -- 172.21.15.69:0/3809247431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa5e38420 con 0x555aa5e16f50 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061889b640 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16f50 0x555aa5e230c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f06040135c0 tx=0x7f0604013ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150abbe4934094c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f05fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f05fcff9640 1 -- 172.21.15.69:0/3809247431 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060402a070 con 0x555aa5e16f50 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f05fcff9640 1 -- 172.21.15.69:0/3809247431 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f060400db50 con 0x555aa5e16f50 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 1 -- 172.21.15.69:0/3809247431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f060c003c70 con 0x555aa5e16f50 2024-08-02T17:44:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f05fcff9640 1 -- 172.21.15.69:0/3809247431 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0604035050 con 0x555aa5e16f50 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 10 client.14670 fetch_fsmap learned FSMap version 17 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 1 -- 172.21.15.69:0/3809247431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x555aa5ea57f0 con 0x555aa5e16f50 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f05fcff9640 1 -- 172.21.15.69:0/3809247431 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c003c70 con 0x555aa5e16f50 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f05fcff9640 1 -- 172.21.15.69:0/3809247431 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f060401c3f0 con 0x555aa5e16f50 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 10 client.14670 fetch_fsmap finished waiting for FSMap version 17 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 10 client.14670 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 20 client.14670 populate_metadata read hostname 'smithi069' 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa5eaa0c0 0x555aa5ea8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 4 client.14670 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.816+0000 7f061d6fe740 1 -- 172.21.15.69:0/3809247431 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "status"}) -- 0x555aa5d83190 con 0x555aa5eaa0c0 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061909c640 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa5eaa0c0 0x555aa5ea8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f05fcff9640 10 client.14670 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061909c640 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa5eaa0c0 0x555aa5ea8dd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8867d080901184 server_cookie=d8e9ae2047aa65e2 in_seq=0 out_seq=0 2024-08-02T17:44:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f05fcff9640 1 -- 172.21.15.69:0/3809247431 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x555aa5d83190 con 0x555aa5eaa0c0 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f05fcff9640 10 client.14670 handle_command_reply: tid=0 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 client.14670 shutdown 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 -- 172.21.15.69:0/3809247431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16f50 msgr2=0x555aa5e230c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16f50 0x555aa5e230c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f06040135c0 tx=0x7f0604013ae0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 -- 172.21.15.69:0/3809247431 shutdown_connections 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa5e16f50 0x555aa5e230c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa5eaa0c0 0x555aa5ea8dd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa5e21810 0x555aa5e23600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f05fcff9640 0 client.14670 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 --2- 172.21.15.69:0/3809247431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa5e16630 0x555aa5e22b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.817+0000 7f061d6fe740 1 -- 172.21.15.69:0/3809247431 >> 172.21.15.69:0/3809247431 conn(0x555aa5e12bc0 msgr2=0x555aa5e2c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061d6fe740 1 -- 172.21.15.69:0/3809247431 shutdown_connections 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061d6fe740 1 -- 172.21.15.69:0/3809247431 wait complete. 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061d6fe740 20 client.14670 trim_cache size 0 max 16384 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24452, 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:44:23.825 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 18.687469935999999, 2024-08-02T17:44:23.826 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 17, 2024-08-02T17:44:23.826 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 90, 2024-08-02T17:44:23.826 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 90, 2024-08-02T17:44:23.826 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 29.502043256 2024-08-02T17:44:23.826 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 -- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f06000a2b80 msgr2=0x7f06000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f06000a2b80 0x7f06000a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f060c030040 tx=0x7f060c005ef0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 -- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0614102a70 msgr2=0x7f06141006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0614102a70 0x7f06141006d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f061000c920 tx=0x7f061000cde0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 -- 172.21.15.69:0/1338927331 shutdown_connections 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f06000a2b80 0x7f06000a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f061410bea0 0x7f0614101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0614103390 0x7f0614100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 --2- 172.21.15.69:0/1338927331 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0614102a70 0x7f06141006d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 -- 172.21.15.69:0/1338927331 >> 172.21.15.69:0/1338927331 conn(0x7f06140fc4d0 msgr2=0x7f06140fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 -- 172.21.15.69:0/1338927331 shutdown_connections 2024-08-02T17:44:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:23.818+0000 7f061b300640 1 -- 172.21.15.69:0/1338927331 wait complete. 2024-08-02T17:44:24.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:23 smithi077 ceph-mon[37108]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:24.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:23 smithi069 ceph-mon[32746]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:24.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:23 smithi083 ceph-mon[37031]: pgmap v273: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:24.596 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-02T17:44:24.597 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:24.597 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs ls 2024-08-02T17:44:24.848 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:25.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 -- 172.21.15.69:0/696240296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 msgr2=0x7fbf4010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 --2- 172.21.15.69:0/696240296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 0x7fbf4010e340 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009ca0 tx=0x7fbf3802a950 comp rx=0 tx=0).stop 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 -- 172.21.15.69:0/696240296 shutdown_connections 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 --2- 172.21.15.69:0/696240296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 0x7fbf4010e340 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 --2- 172.21.15.69:0/696240296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40103390 0x7fbf4010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 --2- 172.21.15.69:0/696240296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf40102a70 0x7fbf40102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 -- 172.21.15.69:0/696240296 >> 172.21.15.69:0/696240296 conn(0x7fbf400fc4d0 msgr2=0x7fbf400fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 -- 172.21.15.69:0/696240296 shutdown_connections 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.318+0000 7fbf46870640 1 -- 172.21.15.69:0/696240296 wait complete. 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf46870640 1 Processor -- start 2024-08-02T17:44:25.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf46870640 1 -- start start 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf46870640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40102a70 0x7fbf401006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf46870640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf40103390 0x7fbf40100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf46870640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 0x7fbf40101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf46870640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40111ca0 con 0x7fbf4010bea0 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf46870640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40111b20 con 0x7fbf40103390 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf46870640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40111e20 con 0x7fbf40102a70 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf44e0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 0x7fbf40101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf44e0d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 0x7fbf40101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34328/0 (socket says 172.21.15.69:34328) 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf44e0d640 1 -- 172.21.15.69:0/1300882883 learned_addr learned my addr 172.21.15.69:0/1300882883 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.319+0000 7fbf3f7fe640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf40103390 0x7fbf40100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf3ffff640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40102a70 0x7fbf401006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:25.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf44e0d640 1 -- 172.21.15.69:0/1300882883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40102a70 msgr2=0x7fbf401006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf44e0d640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40102a70 0x7fbf401006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf44e0d640 1 -- 172.21.15.69:0/1300882883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf40103390 msgr2=0x7fbf40100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf44e0d640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf40103390 0x7fbf40100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf44e0d640 1 -- 172.21.15.69:0/1300882883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf401a6520 con 0x7fbf4010bea0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf44e0d640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 0x7fbf40101150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbf3802ae70 tx=0x7fbf3802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf3d7fa640 1 -- 172.21.15.69:0/1300882883 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38034040 con 0x7fbf4010bea0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf3d7fa640 1 -- 172.21.15.69:0/1300882883 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf38030920 con 0x7fbf4010bea0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf401a67b0 con 0x7fbf4010bea0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf3d7fa640 1 -- 172.21.15.69:0/1300882883 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38045480 con 0x7fbf4010bea0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.320+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf401a6c90 con 0x7fbf4010bea0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.321+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf40102e50 con 0x7fbf4010bea0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.323+0000 7fbf3d7fa640 1 -- 172.21.15.69:0/1300882883 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbf38037020 con 0x7fbf4010bea0 2024-08-02T17:44:25.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.323+0000 7fbf3d7fa640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbf280a2b80 0x7fbf280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:25.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.324+0000 7fbf3ffff640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbf280a2b80 0x7fbf280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:25.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.324+0000 7fbf3d7fa640 1 -- 172.21.15.69:0/1300882883 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fbf380f01f0 con 0x7fbf4010bea0 2024-08-02T17:44:25.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.324+0000 7fbf3ffff640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbf280a2b80 0x7fbf280a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000bbe0 tx=0x7fbf3000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:25.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.327+0000 7fbf3d7fa640 1 -- 172.21.15.69:0/1300882883 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbf38080df0 con 0x7fbf4010bea0 2024-08-02T17:44:25.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.533+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbf401a6ef0 con 0x7fbf4010bea0 2024-08-02T17:44:25.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.533+0000 7fbf3d7fa640 1 -- 172.21.15.69:0/1300882883 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fbf380bcfb0 con 0x7fbf4010bea0 2024-08-02T17:44:25.535 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-02T17:44:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbf280a2b80 msgr2=0x7fbf280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbf280a2b80 0x7fbf280a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000bbe0 tx=0x7fbf3000b390 comp rx=0 tx=0).stop 2024-08-02T17:44:25.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 msgr2=0x7fbf40101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 0x7fbf40101150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbf3802ae70 tx=0x7fbf3802f5b0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 shutdown_connections 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbf280a2b80 0x7fbf280a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf4010bea0 0x7fbf40101150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf40103390 0x7fbf40100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 --2- 172.21.15.69:0/1300882883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf40102a70 0x7fbf401006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 >> 172.21.15.69:0/1300882883 conn(0x7fbf400fc4d0 msgr2=0x7fbf400fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 shutdown_connections 2024-08-02T17:44:25.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:25.535+0000 7fbf46870640 1 -- 172.21.15.69:0/1300882883 wait complete. 2024-08-02T17:44:26.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:25 smithi077 ceph-mon[37108]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:25 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1300882883' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-02T17:44:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:25 smithi069 ceph-mon[32746]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:25 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1300882883' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-02T17:44:26.228 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-08-02T17:44:26.228 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-02T17:44:26.228 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-08-02T17:44:26.229 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-08-02T17:44:26.229 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:26.229 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-08-02T17:44:26.229 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-02T17:44:26.229 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-02T17:44:26.229 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-08-02T17:44:26.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:26.229 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-08-02T17:44:26.245 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-02T17:44:26.245 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-02T17:44:26.245 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-08-02T17:44:26.245 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-08-02T17:44:26.245 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-08-02T17:44:26.245 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-08-02T17:44:26.245 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-02T17:44:26.246 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-08-02T17:44:26.247 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-02T17:44:26.247 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42316sec preferred_lft 42316sec 2024-08-02T17:44:26.247 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-08-02T17:44:26.247 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-08-02T17:44:26.247 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-02T17:44:26.248 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:44:26.248 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:44:26.248 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-08-02T17:44:26.248 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-08-02T17:44:26.248 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-08-02T17:44:26.248 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-02T17:44:26.248 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:44:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:25 smithi083 ceph-mon[37031]: pgmap v274: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:25 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1300882883' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-02T17:44:26.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:44:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:44:26.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:26.474 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-02T17:44:26.499 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-08-02T17:44:26.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:26.500 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-08-02T17:44:26.573 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-08-02T17:44:26.574 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-02T17:44:26.574 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-02T17:44:26.574 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-02T17:44:26.574 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-02T17:44:26.574 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:44:26.575 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:44:26.575 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-02T17:44:26.575 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-02T17:44:26.575 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-02T17:44:26.575 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:44:26.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:44:26.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:44:26.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:26.897 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-08-02T17:44:26.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:26.913 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-08-02T17:44:26.969 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:44:26.969 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:44:26.969 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-02T17:44:26.969 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-08-02T17:44:26.969 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:44:27.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:44:27.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:44:27.089 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-08-02T17:44:27.090 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:44:27.090 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:44:27.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-08-02T17:44:27.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-02T17:44:27.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-08-02T17:44:27.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-08-02T17:44:27.090 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-08-02T17:44:27.090 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:44:27.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:44:27.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:44:27.220 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:44:27.220 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:44:27.220 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-08-02T17:44:27.220 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-08-02T17:44:27.220 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:44:27.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:44:27.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:44:27.299 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-08-02T17:44:27.300 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-02T17:44:27.300 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:27.335 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-08-02T17:44:27.335 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-02T17:44:27.335 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:27.391 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-08-02T17:44:27.458 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-08-02T17:44:27.515 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-08-02T17:44:27.516 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/81a623d55dce676b45b5ca7ee164698d05a4a86e91c481db6e525bf3e9543867/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/4cff1f63e1cc5ca7a291337cdf387924698ddb088c0ec5c0cfea69c802f38f43/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/3a7e7c2ca5c66e4577e9957401a93315a5aa82b6771da2534c33bd456675acf2/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/c5fc831db816f6c2c71c035efb451e15ef1f4adc843de452993a602d6e58bb63/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/b64259de40e88880ffb156dc64bd2772ec04d136abc2c971788fb7e4b44bc5ed/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/9fa56a6f068e1da23284dd4780652605b1c530463b37eccc5cfc0c16b7d6c4d3/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/47adc0b3ddf67deac5bb2132eae9aca71130cb8d0d5e397fbfb07970995b7ce7/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/ba00bbc5a3996b64f736c25ef787e0b9b0a930feb8a10744a1698cb7d48c79ec/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5e420b9a05075761115aa777cf3e897fd0306b48cdd567ebbeadde6df70cbc59/merged 2024-08-02T17:44:27.517 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/8abd8e7f7ae007568d863875387f1a83376c00bdd0fb2304d084fbaceca2debc/merged 2024-08-02T17:44:27.518 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-08-02T17:44:27.518 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-02T17:44:27.518 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-02T17:44:27.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:27.519 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-08-02T17:44:27.570 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-02T17:44:27.570 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-08-02T17:44:27.612 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-08-02T17:44:27.637 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-08-02T17:44:27.692 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-08-02T17:44:27.693 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-08-02T17:44:27.694 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/81a623d55dce676b45b5ca7ee164698d05a4a86e91c481db6e525bf3e9543867/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/4cff1f63e1cc5ca7a291337cdf387924698ddb088c0ec5c0cfea69c802f38f43/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/3a7e7c2ca5c66e4577e9957401a93315a5aa82b6771da2534c33bd456675acf2/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/c5fc831db816f6c2c71c035efb451e15ef1f4adc843de452993a602d6e58bb63/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/b64259de40e88880ffb156dc64bd2772ec04d136abc2c971788fb7e4b44bc5ed/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/9fa56a6f068e1da23284dd4780652605b1c530463b37eccc5cfc0c16b7d6c4d3/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/47adc0b3ddf67deac5bb2132eae9aca71130cb8d0d5e397fbfb07970995b7ce7/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/ba00bbc5a3996b64f736c25ef787e0b9b0a930feb8a10744a1698cb7d48c79ec/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5e420b9a05075761115aa777cf3e897fd0306b48cdd567ebbeadde6df70cbc59/merged 2024-08-02T17:44:27.695 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/8abd8e7f7ae007568d863875387f1a83376c00bdd0fb2304d084fbaceca2debc/merged 2024-08-02T17:44:27.696 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-08-02T17:44:27.696 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-02T17:44:27.696 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-02T17:44:27.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:27.696 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-08-02T17:44:27.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:2024-08-02T17:44:27.746+0000 7f30bcc15480 -1 init, newargv = 0x559cfd9d29b0 newargc=15 2024-08-02T17:44:27.747 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[73977]: starting ceph client 2024-08-02T17:44:27.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[73977]: starting fuse 2024-08-02T17:44:28.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:27 smithi077 ceph-mon[37108]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:28.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:27 smithi069 ceph-mon[32746]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:27 smithi083 ceph-mon[37031]: pgmap v275: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:28.851 DEBUG:teuthology.orchestra.run.smithi069:> sudo modprobe fuse 2024-08-02T17:44:28.881 DEBUG:teuthology.orchestra.run.smithi069:> cat /proc/self/mounts | awk '{print $2}' 2024-08-02T17:44:28.938 INFO:teuthology.orchestra.run.smithi069.stdout:/proc 2024-08-02T17:44:28.938 INFO:teuthology.orchestra.run.smithi069.stdout:/sys 2024-08-02T17:44:28.938 INFO:teuthology.orchestra.run.smithi069.stdout:/dev 2024-08-02T17:44:28.938 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/security 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/shm 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/pts 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/run 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/cgroup 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/pstore 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/bpf 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/ 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/selinux 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/mqueue 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/dev/hugepages 2024-08-02T17:44:28.939 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/debug 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/tracing 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/fs/fuse/connections 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/sys/kernel/config 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-sysctl.service 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/nfs/rpc_pipefs 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/1000 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/ceph 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/81a623d55dce676b45b5ca7ee164698d05a4a86e91c481db6e525bf3e9543867/merged 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/4cff1f63e1cc5ca7a291337cdf387924698ddb088c0ec5c0cfea69c802f38f43/merged 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/run/user/0 2024-08-02T17:44:28.940 INFO:teuthology.orchestra.run.smithi069.stdout:/proc/sys/fs/binfmt_misc 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/3a7e7c2ca5c66e4577e9957401a93315a5aa82b6771da2534c33bd456675acf2/merged 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/c5fc831db816f6c2c71c035efb451e15ef1f4adc843de452993a602d6e58bb63/merged 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/b64259de40e88880ffb156dc64bd2772ec04d136abc2c971788fb7e4b44bc5ed/merged 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/9fa56a6f068e1da23284dd4780652605b1c530463b37eccc5cfc0c16b7d6c4d3/merged 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/47adc0b3ddf67deac5bb2132eae9aca71130cb8d0d5e397fbfb07970995b7ce7/merged 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/ba00bbc5a3996b64f736c25ef787e0b9b0a930feb8a10744a1698cb7d48c79ec/merged 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/5e420b9a05075761115aa777cf3e897fd0306b48cdd567ebbeadde6df70cbc59/merged 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/var/lib/containers/storage/overlay/8abd8e7f7ae007568d863875387f1a83376c00bdd0fb2304d084fbaceca2debc/merged 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-02T17:44:28.941 INFO:teuthology.orchestra.run.smithi069.stdout:/home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:28.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:28.942 DEBUG:teuthology.orchestra.run.smithi069:> ls /sys/fs/fuse/connections 2024-08-02T17:44:28.994 INFO:teuthology.orchestra.run.smithi069.stdout:119 2024-08-02T17:44:28.994 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-08-02T17:44:28.994 DEBUG:teuthology.orchestra.run.smithi069:> sudo stdin-killer -- python3 -c ' 2024-08-02T17:44:28.994 DEBUG:teuthology.orchestra.run.smithi069:> import glob 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> import re 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> import os 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> import subprocess 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> def _find_admin_socket(client_name): 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> files = glob.glob(asok_path) 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> # Given a non-glob path, it better be there 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> if "*" not in asok_path: 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> assert(len(files) == 1) 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> return files[0] 2024-08-02T17:44:28.995 DEBUG:teuthology.orchestra.run.smithi069:> 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> for f in files: 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> contents = proc_f.read() 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> if mountpoint in contents: 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> return f 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> print(_find_admin_socket("client.admin")) 2024-08-02T17:44:28.996 DEBUG:teuthology.orchestra.run.smithi069:> ' 2024-08-02T17:44:29.115 INFO:teuthology.orchestra.run.smithi069.stdout:/var/run/ceph/ceph-client.admin.73977.asok 2024-08-02T17:44:29.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:44:29.125 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.73977.asok 2024-08-02T17:44:29.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:29.125 DEBUG:teuthology.orchestra.run.smithi069:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.73977.asok status 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: "metadata": { 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_sha1": "26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea", 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_version": "ceph version 19.1.0-1260-g26c3fb8e (26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea) squid (rc)", 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: "entity_id": "admin", 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": "smithi069", 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: "pid": "73977", 2024-08-02T17:44:29.260 INFO:teuthology.orchestra.run.smithi069.stdout: "root": "/" 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_count": 0, 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "dentry_pinned_count": 0, 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 14682, 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "inst": { 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "name": { 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "client", 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "num": 14682 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-08-02T17:44:29.261 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 14368199 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": { 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "192.168.0.1:0", 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 14368199 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "inst_str": "client.14682 192.168.0.1:0/14368199", 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "addr_str": "192.168.0.1:0/14368199", 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "inode_count": 1, 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_epoch": 17, 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch": 90, 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_epoch_barrier": 0, 2024-08-02T17:44:29.262 INFO:teuthology.orchestra.run.smithi069.stdout: "blocklisted": false, 2024-08-02T17:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs" 2024-08-02T17:44:29.263 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:44:29.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:29.269 DEBUG:teuthology.orchestra.run.smithi069:> stat --file-system '--printf=%T 2024-08-02T17:44:29.269 DEBUG:teuthology.orchestra.run.smithi069:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:29.284 INFO:teuthology.orchestra.run.smithi069.stdout:fuseblk 2024-08-02T17:44:29.284 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:29.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:29.285 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:29.351 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-08-02T17:44:29.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-02T17:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout:# file: . 2024-08-02T17:44:29.458 INFO:teuthology.orchestra.run.smithi069.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-08-02T17:44:29.459 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:29.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:44:29.464 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:44:29.465 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-02T17:44:29.481 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd blocklist ls 2024-08-02T17:44:29.773 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:30.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:29 smithi077 ceph-mon[37108]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:29 smithi069 ceph-mon[32746]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:29 smithi083 ceph-mon[37031]: pgmap v276: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 -- 172.21.15.69:0/109482620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 msgr2=0x7f4dbc102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 --2- 172.21.15.69:0/109482620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 0x7f4dbc102d20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009ca0 tx=0x7f4db802a950 comp rx=0 tx=0).stop 2024-08-02T17:44:30.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 -- 172.21.15.69:0/109482620 shutdown_connections 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 --2- 172.21.15.69:0/109482620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc10bd20 0x7f4dbc10e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 --2- 172.21.15.69:0/109482620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4dbc103260 0x7f4dbc10b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 --2- 172.21.15.69:0/109482620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 0x7f4dbc102d20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 -- 172.21.15.69:0/109482620 >> 172.21.15.69:0/109482620 conn(0x7f4dbc0fc340 msgr2=0x7f4dbc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 -- 172.21.15.69:0/109482620 shutdown_connections 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 -- 172.21.15.69:0/109482620 wait complete. 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 Processor -- start 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.256+0000 7f4dc359d640 1 -- start start 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc359d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 0x7f4dbc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc359d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4dbc103260 0x7f4dbc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:30.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc359d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc10bd20 0x7f4dbc19c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc359d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc111af0 con 0x7f4dbc102940 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc359d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc111970 con 0x7f4dbc103260 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc359d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc111c70 con 0x7f4dbc10bd20 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 0x7f4dbc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 0x7f4dbc197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34354/0 (socket says 172.21.15.69:34354) 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 -- 172.21.15.69:0/3348804232 learned_addr learned my addr 172.21.15.69:0/3348804232 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc1d9a640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4dbc103260 0x7f4dbc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 -- 172.21.15.69:0/3348804232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc10bd20 msgr2=0x7f4dbc19c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc10bd20 0x7f4dbc19c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 -- 172.21.15.69:0/3348804232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4dbc103260 msgr2=0x7f4dbc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4dbc103260 0x7f4dbc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 -- 172.21.15.69:0/3348804232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc19ce70 con 0x7f4dbc102940 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc259b640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 0x7f4dbc197ec0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005cc0 tx=0x7f4db802f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4db37fe640 1 -- 172.21.15.69:0/3348804232 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8034040 con 0x7f4dbc102940 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4db37fe640 1 -- 172.21.15.69:0/3348804232 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4db8003e40 con 0x7f4dbc102940 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc19d100 con 0x7f4dbc102940 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.257+0000 7f4db37fe640 1 -- 172.21.15.69:0/3348804232 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8045480 con 0x7f4dbc102940 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.258+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc1a4a40 con 0x7f4dbc102940 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.259+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dbc06a500 con 0x7f4dbc102940 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.260+0000 7f4db37fe640 1 -- 172.21.15.69:0/3348804232 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4db8037020 con 0x7f4dbc102940 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.260+0000 7f4db37fe640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4da80a2b80 0x7f4da80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:30.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.260+0000 7f4dc1d9a640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4da80a2b80 0x7f4da80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.260+0000 7f4db37fe640 1 -- 172.21.15.69:0/3348804232 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f4db80f0560 con 0x7f4dbc102940 2024-08-02T17:44:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.261+0000 7f4dc1d9a640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4da80a2b80 0x7f4da80a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005e10 tx=0x7f4dac005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:30.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.264+0000 7f4db37fe640 1 -- 172.21.15.69:0/3348804232 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4db804d3a0 con 0x7f4dbc102940 2024-08-02T17:44:30.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.441+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4dbc102d20 con 0x7f4dbc102940 2024-08-02T17:44:30.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.442+0000 7f4db37fe640 1 -- 172.21.15.69:0/3348804232 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7f4dbc102d20 con 0x7f4dbc102940 2024-08-02T17:44:30.443 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2814849525 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:30.443 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2060958760 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/623492814 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3801376647 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3376234785 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2152855317 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3960560393 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3376234785 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3960560393 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4073447426 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:30.444 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-08-02T17:44:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4da80a2b80 msgr2=0x7f4da80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4da80a2b80 0x7f4da80a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005e10 tx=0x7f4dac005da0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 msgr2=0x7f4dbc197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 0x7f4dbc197ec0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4db8005cc0 tx=0x7f4db802f770 comp rx=0 tx=0).stop 2024-08-02T17:44:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 shutdown_connections 2024-08-02T17:44:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4da80a2b80 0x7f4da80a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4dbc10bd20 0x7f4dbc19c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4dbc103260 0x7f4dbc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 --2- 172.21.15.69:0/3348804232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4dbc102940 0x7f4dbc197ec0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:30.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 >> 172.21.15.69:0/3348804232 conn(0x7f4dbc0fc340 msgr2=0x7f4dbc0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:30.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 shutdown_connections 2024-08-02T17:44:30.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:30.444+0000 7f4dc359d640 1 -- 172.21.15.69:0/3348804232 wait complete. 2024-08-02T17:44:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:30 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3348804232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T17:44:31.158 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:44:31.159 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-02T17:44:31.176 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd blocklist ls 2024-08-02T17:44:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:30 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3348804232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T17:44:31.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:30 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3348804232' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T17:44:31.470 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:31.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/3089121842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81007b0 msgr2=0x7ff6f8100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:31.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/3089121842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81007b0 0x7ff6f8100c10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009930 tx=0x7ff6e002a420 comp rx=0 tx=0).stop 2024-08-02T17:44:31.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/3089121842 shutdown_connections 2024-08-02T17:44:31.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/3089121842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81007b0 0x7ff6f8100c10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/3089121842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f8101150 0x7ff6f810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/3089121842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f81067b0 0x7ff6f8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/3089121842 >> 172.21.15.69:0/3089121842 conn(0x7ff6f80fc4d0 msgr2=0x7ff6f80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/3089121842 shutdown_connections 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.920+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/3089121842 wait complete. 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fe9fd640 1 Processor -- start 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fe9fd640 1 -- start start 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fe9fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f81007b0 0x7ff6f8198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fe9fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f8101150 0x7ff6f8199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fe9fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81067b0 0x7ff6f81a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fe9fd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f8110b70 con 0x7ff6f81067b0 2024-08-02T17:44:31.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fe9fd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f81109f0 con 0x7ff6f81007b0 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fe9fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f8110cf0 con 0x7ff6f8101150 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fcf9a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81067b0 0x7ff6f81a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fcf9a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81067b0 0x7ff6f81a3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34364/0 (socket says 172.21.15.69:34364) 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6fcf9a640 1 -- 172.21.15.69:0/457229081 learned_addr learned my addr 172.21.15.69:0/457229081 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6f7fff640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f81007b0 0x7ff6f8198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.921+0000 7ff6f77fe640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f8101150 0x7ff6f8199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6fcf9a640 1 -- 172.21.15.69:0/457229081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f8101150 msgr2=0x7ff6f8199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6fcf9a640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f8101150 0x7ff6f8199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6fcf9a640 1 -- 172.21.15.69:0/457229081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f81007b0 msgr2=0x7ff6f8198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6fcf9a640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f81007b0 0x7ff6f8198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6fcf9a640 1 -- 172.21.15.69:0/457229081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f81a4500 con 0x7ff6f81067b0 2024-08-02T17:44:31.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6fcf9a640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81067b0 0x7ff6f81a3fc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000cce0 tx=0x7ff6f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6f57fa640 1 -- 172.21.15.69:0/457229081 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000f450 con 0x7ff6f81067b0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6f57fa640 1 -- 172.21.15.69:0/457229081 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f000fad0 con 0x7ff6f81067b0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6f57fa640 1 -- 172.21.15.69:0/457229081 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0018c10 con 0x7ff6f81067b0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f81a46d0 con 0x7ff6f81067b0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.922+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f81a4b90 con 0x7ff6f81067b0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.923+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f8100c10 con 0x7ff6f81067b0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.924+0000 7ff6f57fa640 1 -- 172.21.15.69:0/457229081 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff6f000f5f0 con 0x7ff6f81067b0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.925+0000 7ff6f57fa640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff6e40a2c00 0x7ff6e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.925+0000 7ff6f7fff640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff6e40a2c00 0x7ff6e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.925+0000 7ff6f57fa640 1 -- 172.21.15.69:0/457229081 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff6f00cfa10 con 0x7ff6f81067b0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.925+0000 7ff6f7fff640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff6e40a2c00 0x7ff6e40a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800cac0 tx=0x7ff6e800c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:31.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:31.928+0000 7ff6f57fa640 1 -- 172.21.15.69:0/457229081 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff6f0014030 con 0x7ff6f81067b0 2024-08-02T17:44:32.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.105+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6f81a4e50 con 0x7ff6f81067b0 2024-08-02T17:44:32.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:31 smithi069 ceph-mon[32746]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:32.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.105+0000 7ff6f57fa640 1 -- 172.21.15.69:0/457229081 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+591 (secure 0 0 0) 0x7ff6f009c840 con 0x7ff6f81067b0 2024-08-02T17:44:32.107 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2814849525 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2060958760 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/623492814 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3801376647 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3376234785 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2152855317 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3960560393 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3376234785 2024-08-03T17:37:56.685064+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3960560393 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4073447426 2024-08-03T17:37:34.617397+0000 2024-08-02T17:44:32.108 INFO:teuthology.orchestra.run.smithi069.stderr:listed 10 entries 2024-08-02T17:44:32.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff6e40a2c00 msgr2=0x7ff6e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:32.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff6e40a2c00 0x7ff6e40a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800cac0 tx=0x7ff6e800c4a0 comp rx=0 tx=0).stop 2024-08-02T17:44:32.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81067b0 msgr2=0x7ff6f81a3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:32.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81067b0 0x7ff6f81a3fc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000cce0 tx=0x7ff6f000a620 comp rx=0 tx=0).stop 2024-08-02T17:44:32.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 shutdown_connections 2024-08-02T17:44:32.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff6e40a2c00 0x7ff6e40a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:32.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff6f81067b0 0x7ff6f81a3fc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:32.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff6f8101150 0x7ff6f8199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:32.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 --2- 172.21.15.69:0/457229081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff6f81007b0 0x7ff6f8198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:32.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 >> 172.21.15.69:0/457229081 conn(0x7ff6f80fc4d0 msgr2=0x7ff6f80fcb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:32.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 shutdown_connections 2024-08-02T17:44:32.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:32.108+0000 7ff6fe9fd640 1 -- 172.21.15.69:0/457229081 wait complete. 2024-08-02T17:44:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:31 smithi083 ceph-mon[37031]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:32.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:31 smithi077 ceph-mon[37108]: pgmap v277: 129 pgs: 129 active+clean; 579 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:44:32.732 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi069.front.sepia.ceph.com... 2024-08-02T17:44:32.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:32.733 DEBUG:teuthology.orchestra.run.smithi069:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-08-02T17:44:32.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi069.stderr:ceph-fuse[73977]: fuse finished with error 0 and tester_r 0 2024-08-02T17:44:32.760 INFO:teuthology.orchestra.run:waiting for 300 2024-08-02T17:44:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:32 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/457229081' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T17:44:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:32 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/457229081' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T17:44:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:32 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/457229081' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T17:44:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:33 smithi069 ceph-mon[32746]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:33 smithi083 ceph-mon[37031]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:34.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:33 smithi077 ceph-mon[37108]: pgmap v278: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:36.149 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:35 smithi077 ceph-mon[37108]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:35 smithi083 ceph-mon[37031]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:36.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:35 smithi069 ceph-mon[32746]: pgmap v279: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:37 smithi069 ceph-mon[32746]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:37 smithi083 ceph-mon[37031]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:38.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:37 smithi077 ceph-mon[37108]: pgmap v280: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-08-02T17:44:38.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:44:38.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:44:38.862 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-08-02T17:44:38.891 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:44:39.195 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:39.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 -- 172.21.15.69:0/212346769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 msgr2=0x7fe1e0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/212346769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 0x7fe1e0100c40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0009900 tx=0x7fe1d002a470 comp rx=0 tx=0).stop 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 -- 172.21.15.69:0/212346769 shutdown_connections 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/212346769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e0105c70 0x7fe1e0106100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/212346769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0101180 0x7fe1e0105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/212346769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 0x7fe1e0100c40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 -- 172.21.15.69:0/212346769 >> 172.21.15.69:0/212346769 conn(0x7fe1e00fc4d0 msgr2=0x7fe1e00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 -- 172.21.15.69:0/212346769 shutdown_connections 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.628+0000 7fe1e80c1640 1 -- 172.21.15.69:0/212346769 wait complete. 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e80c1640 1 Processor -- start 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e80c1640 1 -- start start 2024-08-02T17:44:39.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e80c1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 0x7fe1e01a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e80c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e0101180 0x7fe1e01a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e80c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0105c70 0x7fe1e019e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e80c1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0118300 con 0x7fe1e0100860 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e80c1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0118180 con 0x7fe1e0105c70 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e80c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0118480 con 0x7fe1e0101180 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e5e5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 0x7fe1e01a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e5e5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 0x7fe1e01a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48764/0 (socket says 172.21.15.69:48764) 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e5e5d640 1 -- 172.21.15.69:0/2114090517 learned_addr learned my addr 172.21.15.69:0/2114090517 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e665e640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0105c70 0x7fe1e019e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.629+0000 7fe1e565c640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e0101180 0x7fe1e01a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1e5e5d640 1 -- 172.21.15.69:0/2114090517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e0101180 msgr2=0x7fe1e01a4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1e5e5d640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e0101180 0x7fe1e01a4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1e5e5d640 1 -- 172.21.15.69:0/2114090517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0105c70 msgr2=0x7fe1e019e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1e5e5d640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0105c70 0x7fe1e019e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1e5e5d640 1 -- 172.21.15.69:0/2114090517 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e019ed20 con 0x7fe1e0100860 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1e5e5d640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 0x7fe1e01a4430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005ab0 tx=0x7fe1d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1d6ffd640 1 -- 172.21.15.69:0/2114090517 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d003f030 con 0x7fe1e0100860 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e019efb0 con 0x7fe1e0100860 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1d6ffd640 1 -- 172.21.15.69:0/2114090517 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d0003ce0 con 0x7fe1e0100860 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1d6ffd640 1 -- 172.21.15.69:0/2114090517 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0035070 con 0x7fe1e0100860 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.630+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e01ab300 con 0x7fe1e0100860 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.631+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e0105650 con 0x7fe1e0100860 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.632+0000 7fe1d6ffd640 1 -- 172.21.15.69:0/2114090517 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe1d0042030 con 0x7fe1e0100860 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.633+0000 7fe1d6ffd640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1cc0a2b80 0x7fe1cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.633+0000 7fe1e565c640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1cc0a2b80 0x7fe1cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.633+0000 7fe1d6ffd640 1 -- 172.21.15.69:0/2114090517 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fe1d00efd30 con 0x7fe1e0100860 2024-08-02T17:44:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.633+0000 7fe1e565c640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1cc0a2b80 0x7fe1cc0a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00ffac0 tx=0x7fe1dc005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:39.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.636+0000 7fe1d6ffd640 1 -- 172.21.15.69:0/2114090517 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe1d00bc400 con 0x7fe1e0100860 2024-08-02T17:44:39.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.802+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1e01ab500 con 0x7fe1e0100860 2024-08-02T17:44:39.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.804+0000 7fe1d6ffd640 1 -- 172.21.15.69:0/2114090517 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24021 (secure 0 0 0) 0x7fe1d00c0700 con 0x7fe1e0100860 2024-08-02T17:44:39.806 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:39.806 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:14.038953+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1cc0a2b80 msgr2=0x7fe1cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1cc0a2b80 0x7fe1cc0a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00ffac0 tx=0x7fe1dc005f70 comp rx=0 tx=0).stop 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 msgr2=0x7fe1e01a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 0x7fe1e01a4430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005ab0 tx=0x7fe1d00042f0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 shutdown_connections 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1cc0a2b80 0x7fe1cc0a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe1e0105c70 0x7fe1e019e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1e0101180 0x7fe1e01a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 --2- 172.21.15.69:0/2114090517 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe1e0100860 0x7fe1e01a4430 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 >> 172.21.15.69:0/2114090517 conn(0x7fe1e00fc4d0 msgr2=0x7fe1e010f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 shutdown_connections 2024-08-02T17:44:39.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:39.806+0000 7fe1e80c1640 1 -- 172.21.15.69:0/2114090517 wait complete. 2024-08-02T17:44:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:39 smithi069 ceph-mon[32746]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:39 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2114090517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:39 smithi083 ceph-mon[37031]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:39 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2114090517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:40.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:39 smithi077 ceph-mon[37108]: pgmap v281: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:40.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:39 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2114090517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:40.543 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:44:40.798 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:41.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 -- 172.21.15.69:0/3078361069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 msgr2=0x7f02bc100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:41.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 --2- 172.21.15.69:0/3078361069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 0x7f02bc100c10 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009930 tx=0x7f02a402a420 comp rx=0 tx=0).stop 2024-08-02T17:44:41.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 -- 172.21.15.69:0/3078361069 shutdown_connections 2024-08-02T17:44:41.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 --2- 172.21.15.69:0/3078361069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 0x7f02bc100c10 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 --2- 172.21.15.69:0/3078361069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc101150 0x7f02bc10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 --2- 172.21.15.69:0/3078361069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02bc1067b0 0x7f02bc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 -- 172.21.15.69:0/3078361069 >> 172.21.15.69:0/3078361069 conn(0x7f02bc0fc4d0 msgr2=0x7f02bc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 -- 172.21.15.69:0/3078361069 shutdown_connections 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.266+0000 7f02c1579640 1 -- 172.21.15.69:0/3078361069 wait complete. 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02c1579640 1 Processor -- start 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02c1579640 1 -- start start 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02c1579640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 0x7f02bc197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02c1579640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02bc101150 0x7f02bc1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02c1579640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc1067b0 0x7f02bc19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02c1579640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc110bb0 con 0x7f02bc1007b0 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02c1579640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc110a30 con 0x7f02bc1067b0 2024-08-02T17:44:41.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02c1579640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02bc110d30 con 0x7f02bc101150 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02baffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 0x7f02bc197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02baffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 0x7f02bc197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48782/0 (socket says 172.21.15.69:48782) 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02baffd640 1 -- 172.21.15.69:0/231017596 learned_addr learned my addr 172.21.15.69:0/231017596 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02ba7fc640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02bc101150 0x7f02bc1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.267+0000 7f02bb7fe640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc1067b0 0x7f02bc19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02baffd640 1 -- 172.21.15.69:0/231017596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02bc101150 msgr2=0x7f02bc1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02baffd640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02bc101150 0x7f02bc1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02baffd640 1 -- 172.21.15.69:0/231017596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc1067b0 msgr2=0x7f02bc19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02baffd640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc1067b0 0x7f02bc19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02baffd640 1 -- 172.21.15.69:0/231017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02bc19c040 con 0x7f02bc1007b0 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02baffd640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 0x7f02bc197090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f02b400ea80 tx=0x7f02b400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02b3fff640 1 -- 172.21.15.69:0/231017596 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b400ce40 con 0x7f02bc1007b0 2024-08-02T17:44:41.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02bc19c330 con 0x7f02bc1007b0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02b3fff640 1 -- 172.21.15.69:0/231017596 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02b4005020 con 0x7f02bc1007b0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02bc1a3c10 con 0x7f02bc1007b0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.268+0000 7f02b3fff640 1 -- 172.21.15.69:0/231017596 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4011760 con 0x7f02bc1007b0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.269+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02bc101ef0 con 0x7f02bc1007b0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.270+0000 7f02b3fff640 1 -- 172.21.15.69:0/231017596 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f02b40051d0 con 0x7f02bc1007b0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.271+0000 7f02b3fff640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f02a80a2b80 0x7f02a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.271+0000 7f02ba7fc640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f02a80a2b80 0x7f02a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.271+0000 7f02b3fff640 1 -- 172.21.15.69:0/231017596 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f02b40cf960 con 0x7f02bc1007b0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.271+0000 7f02ba7fc640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f02a80a2b80 0x7f02a80a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009900 tx=0x7f02a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:41.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.274+0000 7f02b3fff640 1 -- 172.21.15.69:0/231017596 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f02b4016030 con 0x7f02bc1007b0 2024-08-02T17:44:41.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.442+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02bc100c10 con 0x7f02bc1007b0 2024-08-02T17:44:41.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.443+0000 7f02b3fff640 1 -- 172.21.15.69:0/231017596 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24021 (secure 0 0 0) 0x7f02b409c030 con 0x7f02bc1007b0 2024-08-02T17:44:41.445 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:41.445 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":90,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:14.038953+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f02a80a2b80 msgr2=0x7f02a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f02a80a2b80 0x7f02a80a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f02a4009900 tx=0x7f02a40058d0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 msgr2=0x7f02bc197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 0x7f02bc197090 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f02b400ea80 tx=0x7f02b400ef40 comp rx=0 tx=0).stop 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 shutdown_connections 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f02a80a2b80 0x7f02a80a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02bc1067b0 0x7f02bc19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02bc101150 0x7f02bc1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 --2- 172.21.15.69:0/231017596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02bc1007b0 0x7f02bc197090 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:41.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 >> 172.21.15.69:0/231017596 conn(0x7f02bc0fc4d0 msgr2=0x7f02bc0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:41.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 shutdown_connections 2024-08-02T17:44:41.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:41.445+0000 7f02c1579640 1 -- 172.21.15.69:0/231017596 wait complete. 2024-08-02T17:44:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:41 smithi069 ceph-mon[32746]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:41 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/231017596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:42.168 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-02T17:44:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:41 smithi083 ceph-mon[37031]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:41 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/231017596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:42.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:41 smithi077 ceph-mon[37108]: pgmap v282: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:42.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:41 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/231017596' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:44:42.418 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.886+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/603110608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8103410 msgr2=0x7f7bc81078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.886+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/603110608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8103410 0x7f7bc81078c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009930 tx=0x7f7bbc02a420 comp rx=0 tx=0).stop 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.886+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/603110608 shutdown_connections 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.886+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/603110608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8103410 0x7f7bc81078c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.886+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/603110608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc8107fd0 0x7f7bc810e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.886+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/603110608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc8102a60 0x7f7bc8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.886+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/603110608 >> 172.21.15.69:0/603110608 conn(0x7f7bc80fc500 msgr2=0x7f7bc80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.887+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/603110608 shutdown_connections 2024-08-02T17:44:42.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.887+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/603110608 wait complete. 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.887+0000 7f7bcf2aa640 1 Processor -- start 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.887+0000 7f7bcf2aa640 1 -- start start 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.887+0000 7f7bcf2aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc8102a60 0x7f7bc8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.887+0000 7f7bcf2aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc8103410 0x7f7bc81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcf2aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8107fd0 0x7f7bc819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcf2aa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8111e70 con 0x7f7bc8107fd0 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcf2aa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8111cf0 con 0x7f7bc8102a60 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcf2aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8111ff0 con 0x7f7bc8103410 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8107fd0 0x7f7bc819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8107fd0 0x7f7bc819cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48800/0 (socket says 172.21.15.69:48800) 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 -- 172.21.15.69:0/2470596945 learned_addr learned my addr 172.21.15.69:0/2470596945 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:42.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcc845640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc8103410 0x7f7bc81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd046640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc8102a60 0x7f7bc8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 -- 172.21.15.69:0/2470596945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc8103410 msgr2=0x7f7bc81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc8103410 0x7f7bc81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 -- 172.21.15.69:0/2470596945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc8102a60 msgr2=0x7f7bc8198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc8102a60 0x7f7bc8198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 -- 172.21.15.69:0/2470596945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc819d420 con 0x7f7bc8107fd0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcd847640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8107fd0 0x7f7bc819cd40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800b970 tx=0x7f7bb800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bc67fc640 1 -- 172.21.15.69:0/2470596945 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8019070 con 0x7f7bc8107fd0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc819d710 con 0x7f7bc8107fd0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bc67fc640 1 -- 172.21.15.69:0/2470596945 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bb8014440 con 0x7f7bc8107fd0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bc67fc640 1 -- 172.21.15.69:0/2470596945 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80075d0 con 0x7f7bc8107fd0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.888+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc81a4ff0 con 0x7f7bc8107fd0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.889+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc8102ec0 con 0x7f7bc8107fd0 2024-08-02T17:44:42.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.891+0000 7f7bc67fc640 1 -- 172.21.15.69:0/2470596945 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7bb801c070 con 0x7f7bc8107fd0 2024-08-02T17:44:42.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.891+0000 7f7bc67fc640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7bb00a2b80 0x7f7bb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:42.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.892+0000 7f7bcd046640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7bb00a2b80 0x7f7bb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:42.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.892+0000 7f7bc67fc640 1 -- 172.21.15.69:0/2470596945 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7bb80ce780 con 0x7f7bc8107fd0 2024-08-02T17:44:42.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.892+0000 7f7bcd046640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7bb00a2b80 0x7f7bb00a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009990 tx=0x7f7bc0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:42.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:42.896+0000 7f7bc67fc640 1 -- 172.21.15.69:0/2470596945 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7bb809ae50 con 0x7f7bc8107fd0 2024-08-02T17:44:43.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.099+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7bc8109fe0 con 0x7f7bc8107fd0 2024-08-02T17:44:43.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.870+0000 7f7bc67fc640 1 -- 172.21.15.69:0/2470596945 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f7bb809f150 con 0x7f7bc8107fd0 2024-08-02T17:44:43.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7bb00a2b80 msgr2=0x7f7bb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:43.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7bb00a2b80 0x7f7bb00a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0009990 tx=0x7f7bc0009110 comp rx=0 tx=0).stop 2024-08-02T17:44:43.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8107fd0 msgr2=0x7f7bc819cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:43.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8107fd0 0x7f7bc819cd40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7bb800b970 tx=0x7f7bb800bd70 comp rx=0 tx=0).stop 2024-08-02T17:44:43.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 shutdown_connections 2024-08-02T17:44:43.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7bb00a2b80 0x7f7bb00a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:43.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7bc8107fd0 0x7f7bc819cd40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:43.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7bc8103410 0x7f7bc81989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:43.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 --2- 172.21.15.69:0/2470596945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7bc8102a60 0x7f7bc8198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:43.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 >> 172.21.15.69:0/2470596945 conn(0x7f7bc80fc500 msgr2=0x7f7bc8105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:43.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 shutdown_connections 2024-08-02T17:44:43.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:43.872+0000 7f7bcf2aa640 1 -- 172.21.15.69:0/2470596945 wait complete. 2024-08-02T17:44:44.176 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:43 smithi069 ceph-mon[32746]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:44.176 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:43 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2470596945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-02T17:44:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:43 smithi083 ceph-mon[37031]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:43 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2470596945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-02T17:44:44.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:43 smithi077 ceph-mon[37108]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-02T17:44:44.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:43 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2470596945' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-08-02T17:44:44.487 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-08-02T17:44:44.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-08-02T17:44:44.740 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:45.129 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:44 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2470596945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-02T17:44:45.130 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:44 smithi069 ceph-mon[32746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:45.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.211+0000 7f7dc0bde640 1 -- 172.21.15.69:0/406683349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc103390 msgr2=0x7f7dbc10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:45.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.211+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/406683349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc103390 0x7f7dbc10b880 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009900 tx=0x7f7db002a480 comp rx=0 tx=0).stop 2024-08-02T17:44:45.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 -- 172.21.15.69:0/406683349 shutdown_connections 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/406683349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc103390 0x7f7dbc10b880 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/406683349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dbc10bea0 0x7f7dbc10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/406683349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dbc102a70 0x7f7dbc102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 -- 172.21.15.69:0/406683349 >> 172.21.15.69:0/406683349 conn(0x7f7dbc0fc4d0 msgr2=0x7f7dbc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 -- 172.21.15.69:0/406683349 shutdown_connections 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 -- 172.21.15.69:0/406683349 wait complete. 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 Processor -- start 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.212+0000 7f7dc0bde640 1 -- start start 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dc0bde640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dbc102a70 0x7f7dbc1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dc0bde640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dbc103390 0x7f7dbc100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dc0bde640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc10bea0 0x7f7dbc101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dc0bde640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc111ca0 con 0x7f7dbc10bea0 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dc0bde640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc111b20 con 0x7f7dbc102a70 2024-08-02T17:44:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dc0bde640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc111e20 con 0x7f7dbc103390 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc10bea0 0x7f7dbc101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc10bea0 0x7f7dbc101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49426/0 (socket says 172.21.15.69:49426) 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 -- 172.21.15.69:0/3095145641 learned_addr learned my addr 172.21.15.69:0/3095145641 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dba59c640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dbc102a70 0x7f7dbc1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7db9d9b640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dbc103390 0x7f7dbc100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 -- 172.21.15.69:0/3095145641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dbc103390 msgr2=0x7f7dbc100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dbc103390 0x7f7dbc100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 -- 172.21.15.69:0/3095145641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dbc102a70 msgr2=0x7f7dbc1006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dbc102a70 0x7f7dbc1006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 -- 172.21.15.69:0/3095145641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc1aab30 con 0x7f7dbc10bea0 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7db9d9b640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dbc103390 0x7f7dbc100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dbad9d640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc10bea0 0x7f7dbc101150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7db400eaa0 tx=0x7f7db400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dab7fe640 1 -- 172.21.15.69:0/3095145641 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db400cdb0 con 0x7f7dbc10bea0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc1aad60 con 0x7f7dbc10bea0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dab7fe640 1 -- 172.21.15.69:0/3095145641 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7db4005020 con 0x7f7dbc10bea0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dab7fe640 1 -- 172.21.15.69:0/3095145641 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db40116b0 con 0x7f7dbc10bea0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.213+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc1ab210 con 0x7f7dbc10bea0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.214+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc06ada0 con 0x7f7dbc10bea0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.216+0000 7f7dab7fe640 1 -- 172.21.15.69:0/3095145641 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7db4005560 con 0x7f7dbc10bea0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.216+0000 7f7dab7fe640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7da40a2b80 0x7f7da40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.216+0000 7f7dba59c640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7da40a2b80 0x7f7da40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.216+0000 7f7dab7fe640 1 -- 172.21.15.69:0/3095145641 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f7db40cf8c0 con 0x7f7dbc10bea0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.217+0000 7f7dba59c640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7da40a2b80 0x7f7da40a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7dac005fd0 tx=0x7f7dac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.219+0000 7f7dab7fe640 1 -- 172.21.15.69:0/3095145641 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7db4016030 con 0x7f7dbc10bea0 2024-08-02T17:44:45.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:44 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2470596945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-02T17:44:45.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:44 smithi083 ceph-mon[37031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:45.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:44 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2470596945' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-02T17:44:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:44 smithi077 ceph-mon[37108]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:45.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.421+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f7dbc1ab500 con 0x7f7dbc10bea0 2024-08-02T17:44:45.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.882+0000 7f7dab7fe640 1 -- 172.21.15.69:0/3095145641 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f7db409c5e0 con 0x7f7dbc10bea0 2024-08-02T17:44:45.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7da40a2b80 msgr2=0x7f7da40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:45.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7da40a2b80 0x7f7da40a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7dac005fd0 tx=0x7f7dac00b390 comp rx=0 tx=0).stop 2024-08-02T17:44:45.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc10bea0 msgr2=0x7f7dbc101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc10bea0 0x7f7dbc101150 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7db400eaa0 tx=0x7f7db400ef60 comp rx=0 tx=0).stop 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 shutdown_connections 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7da40a2b80 0x7f7da40a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7dbc10bea0 0x7f7dbc101150 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7dbc103390 0x7f7dbc100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 --2- 172.21.15.69:0/3095145641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7dbc102a70 0x7f7dbc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 >> 172.21.15.69:0/3095145641 conn(0x7f7dbc0fc4d0 msgr2=0x7f7dbc0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 shutdown_connections 2024-08-02T17:44:45.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:45.885+0000 7f7dc0bde640 1 -- 172.21.15.69:0/3095145641 wait complete. 2024-08-02T17:44:46.175 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:45 smithi069 ceph-mon[32746]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-08-02T17:44:46.175 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:45 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3095145641' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-02T17:44:46.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:45 smithi083 ceph-mon[37031]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-08-02T17:44:46.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:45 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3095145641' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-02T17:44:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:45 smithi077 ceph-mon[37108]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-08-02T17:44:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:45 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3095145641' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-08-02T17:44:46.546 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-02T17:44:46.821 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:47.173 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:46 smithi069 ceph-mon[32746]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-02T17:44:47.173 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:46 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3095145641' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-02T17:44:47.173 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:46 smithi069 ceph-mon[32746]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-02T17:44:47.173 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:46 smithi069 ceph-mon[32746]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:47.173 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:46 smithi069 ceph-mon[32746]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-08-02T17:44:47.173 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:46 smithi069 ceph-mon[32746]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-08-02T17:44:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:46 smithi083 ceph-mon[37031]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-02T17:44:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:46 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3095145641' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-02T17:44:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:46 smithi083 ceph-mon[37031]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-02T17:44:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:46 smithi083 ceph-mon[37031]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:46 smithi083 ceph-mon[37031]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-08-02T17:44:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:46 smithi083 ceph-mon[37031]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 -- 172.21.15.69:0/2913047064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0181007b0 msgr2=0x7ff018100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2913047064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0181007b0 0x7ff018100c10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff000009930 tx=0x7ff00002a420 comp rx=0 tx=0).stop 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 -- 172.21.15.69:0/2913047064 shutdown_connections 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2913047064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0181007b0 0x7ff018100c10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2913047064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff018101150 0x7ff01810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2913047064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0181067b0 0x7ff018106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 -- 172.21.15.69:0/2913047064 >> 172.21.15.69:0/2913047064 conn(0x7ff0180fc4d0 msgr2=0x7ff0180fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 -- 172.21.15.69:0/2913047064 shutdown_connections 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.295+0000 7ff01c8db640 1 -- 172.21.15.69:0/2913047064 wait complete. 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01c8db640 1 Processor -- start 2024-08-02T17:44:47.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01c8db640 1 -- start start 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01c8db640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0181007b0 0x7ff01819a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01c8db640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff018101150 0x7ff01819a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01c8db640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0181067b0 0x7ff01819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01c8db640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff018110bb0 con 0x7ff018101150 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01c8db640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff018110a30 con 0x7ff0181067b0 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01c8db640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff018110d30 con 0x7ff0181007b0 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff00ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff018101150 0x7ff01819a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff00ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff018101150 0x7ff01819a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49448/0 (socket says 172.21.15.69:49448) 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff00ffff640 1 -- 172.21.15.69:0/2257772047 learned_addr learned my addr 172.21.15.69:0/2257772047 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff01659c640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0181007b0 0x7ff01819a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff016d9d640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0181067b0 0x7ff01819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff00ffff640 1 -- 172.21.15.69:0/2257772047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0181007b0 msgr2=0x7ff01819a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:47.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff00ffff640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0181007b0 0x7ff01819a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff00ffff640 1 -- 172.21.15.69:0/2257772047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0181067b0 msgr2=0x7ff01819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff00ffff640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0181067b0 0x7ff01819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.296+0000 7ff00ffff640 1 -- 172.21.15.69:0/2257772047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01819f2d0 con 0x7ff018101150 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.297+0000 7ff00ffff640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff018101150 0x7ff01819a860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff000009230 tx=0x7ff000004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.297+0000 7ff00f7fe640 1 -- 172.21.15.69:0/2257772047 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00003f030 con 0x7ff018101150 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.297+0000 7ff00f7fe640 1 -- 172.21.15.69:0/2257772047 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff000003ad0 con 0x7ff018101150 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.297+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01819f560 con 0x7ff018101150 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.297+0000 7ff00f7fe640 1 -- 172.21.15.69:0/2257772047 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000035070 con 0x7ff018101150 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.297+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0181a6ea0 con 0x7ff018101150 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.298+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff018100c10 con 0x7ff018101150 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.299+0000 7ff00f7fe640 1 -- 172.21.15.69:0/2257772047 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff000042030 con 0x7ff018101150 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.299+0000 7ff00f7fe640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff0040a2b80 0x7ff0040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.300+0000 7ff01659c640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff0040a2b80 0x7ff0040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:47.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.300+0000 7ff00f7fe640 1 -- 172.21.15.69:0/2257772047 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7ff0000ef850 con 0x7ff018101150 2024-08-02T17:44:47.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.300+0000 7ff01659c640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff0040a2b80 0x7ff0040a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff008009b50 tx=0x7ff008005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:47.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.304+0000 7ff00f7fe640 1 -- 172.21.15.69:0/2257772047 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff0000bbf20 con 0x7ff018101150 2024-08-02T17:44:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:46 smithi077 ceph-mon[37108]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-02T17:44:47.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:46 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3095145641' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-02T17:44:47.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:46 smithi077 ceph-mon[37108]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-02T17:44:47.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:46 smithi077 ceph-mon[37108]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-02T17:44:47.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:46 smithi077 ceph-mon[37108]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-08-02T17:44:47.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:46 smithi077 ceph-mon[37108]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-08-02T17:44:47.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.506+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff0181a70a0 con 0x7ff018101150 2024-08-02T17:44:47.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.903+0000 7ff00f7fe640 1 -- 172.21.15.69:0/2257772047 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7ff0000c0220 con 0x7ff018101150 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff0040a2b80 msgr2=0x7ff0040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff0040a2b80 0x7ff0040a5040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff008009b50 tx=0x7ff008005f90 comp rx=0 tx=0).stop 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff018101150 msgr2=0x7ff01819a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff018101150 0x7ff01819a860 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff000009230 tx=0x7ff000004270 comp rx=0 tx=0).stop 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 shutdown_connections 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff0040a2b80 0x7ff0040a5040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff018101150 0x7ff01819a860 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0181067b0 0x7ff01819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 --2- 172.21.15.69:0/2257772047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0181007b0 0x7ff01819a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:47.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 >> 172.21.15.69:0/2257772047 conn(0x7ff0180fc4d0 msgr2=0x7ff01810c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:47.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 shutdown_connections 2024-08-02T17:44:47.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:47.905+0000 7ff01c8db640 1 -- 172.21.15.69:0/2257772047 wait complete. 2024-08-02T17:44:48.235 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-02T17:44:48.236 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] up:active 2024-08-02T17:44:48.236 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-02T17:44:48.236 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-02T17:44:48.236 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: Cluster is now healthy 2024-08-02T17:44:48.236 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-08-02T17:44:48.236 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-08-02T17:44:48.236 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-02T17:44:48.236 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:47 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2257772047' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-02T17:44:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-02T17:44:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] up:active 2024-08-02T17:44:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-02T17:44:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-02T17:44:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: Cluster is now healthy 2024-08-02T17:44:48.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-08-02T17:44:48.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-08-02T17:44:48.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-02T17:44:48.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:47 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2257772047' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-02T17:44:48.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-02T17:44:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] up:active 2024-08-02T17:44:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-02T17:44:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-02T17:44:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: Cluster is now healthy 2024-08-02T17:44:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-08-02T17:44:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-08-02T17:44:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-08-02T17:44:48.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:47 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2257772047' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-08-02T17:44:48.570 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-02T17:44:48.823 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:49.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:48 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2257772047' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-02T17:44:49.204 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:48 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] up:active 2024-08-02T17:44:49.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:48 smithi069 ceph-mon[32746]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-08-02T17:44:49.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:48 smithi069 ceph-mon[32746]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:44:49.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:48 smithi069 ceph-mon[32746]: pgmap v286: 129 pgs: 129 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-08-02T17:44:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:48 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2257772047' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-02T17:44:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:48 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] up:active 2024-08-02T17:44:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:48 smithi083 ceph-mon[37031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-08-02T17:44:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:48 smithi083 ceph-mon[37031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:44:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:48 smithi083 ceph-mon[37031]: pgmap v286: 129 pgs: 129 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-08-02T17:44:49.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- 172.21.15.69:0/2135798174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f01018b0 msgr2=0x7fb6f0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:49.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/2135798174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f01018b0 0x7fb6f0101d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009930 tx=0x7fb6dc02a420 comp rx=0 tx=0).stop 2024-08-02T17:44:49.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- 172.21.15.69:0/2135798174 shutdown_connections 2024-08-02T17:44:49.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/2135798174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f01018b0 0x7fb6f0101d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/2135798174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6f0102250 0x7fb6f010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/2135798174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6f01078b0 0x7fb6f0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- 172.21.15.69:0/2135798174 >> 172.21.15.69:0/2135798174 conn(0x7fb6f00fd5f0 msgr2=0x7fb6f00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- 172.21.15.69:0/2135798174 shutdown_connections 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- 172.21.15.69:0/2135798174 wait complete. 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 Processor -- start 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- start start 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6f01018b0 0x7fb6f0198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f0102250 0x7fb6f0198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6f01078b0 0x7fb6f019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0111cf0 con 0x7fb6f0102250 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0111b70 con 0x7fb6f01018b0 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.285+0000 7fb6f6c46640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0111e70 con 0x7fb6f01078b0 2024-08-02T17:44:49.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f0102250 0x7fb6f0198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f0102250 0x7fb6f0198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49466/0 (socket says 172.21.15.69:49466) 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 -- 172.21.15.69:0/3930666307 learned_addr learned my addr 172.21.15.69:0/3930666307 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6f51e3640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6f01078b0 0x7fb6f019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6f49e2640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6f01018b0 0x7fb6f0198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 -- 172.21.15.69:0/3930666307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6f01078b0 msgr2=0x7fb6f019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6f01078b0 0x7fb6f019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 -- 172.21.15.69:0/3930666307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6f01018b0 msgr2=0x7fb6f0198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6f01018b0 0x7fb6f0198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 -- 172.21.15.69:0/3930666307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f019d1e0 con 0x7fb6f0102250 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6effff640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f0102250 0x7fb6f0198770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009230 tx=0x7fb6dc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6edffb640 1 -- 172.21.15.69:0/3930666307 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc03f030 con 0x7fb6f0102250 2024-08-02T17:44:49.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f019d470 con 0x7fb6f0102250 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6edffb640 1 -- 172.21.15.69:0/3930666307 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6dc003ad0 con 0x7fb6f0102250 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6edffb640 1 -- 172.21.15.69:0/3930666307 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc035070 con 0x7fb6f0102250 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.286+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f01a4db0 con 0x7fb6f0102250 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.287+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f0102ff0 con 0x7fb6f0102250 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.288+0000 7fb6edffb640 1 -- 172.21.15.69:0/3930666307 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb6dc042030 con 0x7fb6f0102250 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.289+0000 7fb6edffb640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb6e00a2b80 0x7fb6e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.289+0000 7fb6edffb640 1 -- 172.21.15.69:0/3930666307 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fb6dc0ef820 con 0x7fb6f0102250 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.289+0000 7fb6f49e2640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb6e00a2b80 0x7fb6e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.290+0000 7fb6f49e2640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb6e00a2b80 0x7fb6e00a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009890 tx=0x7fb6e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:49.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.293+0000 7fb6edffb640 1 -- 172.21.15.69:0/3930666307 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb6dc0bc540 con 0x7fb6f0102250 2024-08-02T17:44:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:48 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2257772047' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-02T17:44:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:48 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] up:active 2024-08-02T17:44:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:48 smithi077 ceph-mon[37108]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-08-02T17:44:49.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:48 smithi077 ceph-mon[37108]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:44:49.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:48 smithi077 ceph-mon[37108]: pgmap v286: 129 pgs: 129 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 2 op/s 2024-08-02T17:44:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.494+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb6f0109e40 con 0x7fb6f0102250 2024-08-02T17:44:49.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.917+0000 7fb6edffb640 1 -- 172.21.15.69:0/3930666307 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fb6dc0bbe40 con 0x7fb6f0102250 2024-08-02T17:44:49.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.920+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb6e00a2b80 msgr2=0x7fb6e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:49.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.920+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb6e00a2b80 0x7fb6e00a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4009890 tx=0x7fb6e4008040 comp rx=0 tx=0).stop 2024-08-02T17:44:49.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.920+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f0102250 msgr2=0x7fb6f0198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:49.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.920+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f0102250 0x7fb6f0198770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc009230 tx=0x7fb6dc004270 comp rx=0 tx=0).stop 2024-08-02T17:44:49.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.921+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 shutdown_connections 2024-08-02T17:44:49.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.921+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb6e00a2b80 0x7fb6e00a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.921+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb6f0102250 0x7fb6f0198770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.921+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb6f01078b0 0x7fb6f019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.921+0000 7fb6f6c46640 1 --2- 172.21.15.69:0/3930666307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb6f01018b0 0x7fb6f0198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:49.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.921+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 >> 172.21.15.69:0/3930666307 conn(0x7fb6f00fd5f0 msgr2=0x7fb6f00fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:49.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.921+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 shutdown_connections 2024-08-02T17:44:49.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:49.921+0000 7fb6f6c46640 1 -- 172.21.15.69:0/3930666307 wait complete. 2024-08-02T17:44:50.242 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:49 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3930666307' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-02T17:44:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:49 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3930666307' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-02T17:44:50.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:49 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3930666307' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-08-02T17:44:50.348 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 17:44:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b[66767]: 2024-08-02T17:44:49.870+0000 7fa055673640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-02T17:44:50.348 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 17:44:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b[66767]: 2024-08-02T17:44:49.870+0000 7fa055673640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-08-02T17:44:50.555 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-08-02T17:44:50.555 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-02T17:44:50.809 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:51.178 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:50 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3930666307' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-02T17:44:51.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:50 smithi069 ceph-mon[32746]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:44:51.179 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:50 smithi069 ceph-mon[32746]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-02T17:44:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 -- 172.21.15.69:0/2990903447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781007b0 msgr2=0x7fa578100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 --2- 172.21.15.69:0/2990903447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781007b0 0x7fa578100c10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa564009930 tx=0x7fa56402a420 comp rx=0 tx=0).stop 2024-08-02T17:44:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 -- 172.21.15.69:0/2990903447 shutdown_connections 2024-08-02T17:44:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 --2- 172.21.15.69:0/2990903447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781007b0 0x7fa578100c10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 --2- 172.21.15.69:0/2990903447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa578101150 0x7fa57810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 --2- 172.21.15.69:0/2990903447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa5781067b0 0x7fa578106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 -- 172.21.15.69:0/2990903447 >> 172.21.15.69:0/2990903447 conn(0x7fa5780fc4d0 msgr2=0x7fa5780fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 -- 172.21.15.69:0/2990903447 shutdown_connections 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.260+0000 7fa57f216640 1 -- 172.21.15.69:0/2990903447 wait complete. 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57f216640 1 Processor -- start 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57f216640 1 -- start start 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57f216640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5781007b0 0x7fa57810e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57f216640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa578101150 0x7fa57810e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57f216640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781067b0 0x7fa57810eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57f216640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa578111cd0 con 0x7fa5781067b0 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57f216640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa578111b50 con 0x7fa578101150 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57f216640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa578111e50 con 0x7fa5781007b0 2024-08-02T17:44:51.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57d7b3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781067b0 0x7fa57810eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57d7b3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781067b0 0x7fa57810eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49484/0 (socket says 172.21.15.69:49484) 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57d7b3640 1 -- 172.21.15.69:0/1411769974 learned_addr learned my addr 172.21.15.69:0/1411769974 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa577fff640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa578101150 0x7fa57810e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57cfb2640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5781007b0 0x7fa57810e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57d7b3640 1 -- 172.21.15.69:0/1411769974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5781007b0 msgr2=0x7fa57810e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57d7b3640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5781007b0 0x7fa57810e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57d7b3640 1 -- 172.21.15.69:0/1411769974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa578101150 msgr2=0x7fa57810e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57d7b3640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa578101150 0x7fa57810e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57d7b3640 1 -- 172.21.15.69:0/1411769974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5781a6670 con 0x7fa5781067b0 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.261+0000 7fa57cfb2640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5781007b0 0x7fa57810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.262+0000 7fa57d7b3640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781067b0 0x7fa57810eb90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa57000cad0 tx=0x7fa57000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.262+0000 7fa575ffb640 1 -- 172.21.15.69:0/1411769974 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57000ae20 con 0x7fa5781067b0 2024-08-02T17:44:51.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.262+0000 7fa575ffb640 1 -- 172.21.15.69:0/1411769974 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa570005020 con 0x7fa5781067b0 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.262+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5781a6960 con 0x7fa5781067b0 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.262+0000 7fa575ffb640 1 -- 172.21.15.69:0/1411769974 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57000f6f0 con 0x7fa5781067b0 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.262+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5781a6e20 con 0x7fa5781067b0 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.263+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa578100c10 con 0x7fa5781067b0 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.264+0000 7fa575ffb640 1 -- 172.21.15.69:0/1411769974 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5700051d0 con 0x7fa5781067b0 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.265+0000 7fa575ffb640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5680a2b80 0x7fa5680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.265+0000 7fa57cfb2640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5680a2b80 0x7fa5680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.265+0000 7fa575ffb640 1 -- 172.21.15.69:0/1411769974 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa5700cf340 con 0x7fa5781067b0 2024-08-02T17:44:51.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.265+0000 7fa57cfb2640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5680a2b80 0x7fa5680a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009ab0 tx=0x7fa56c0094b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:51.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:50 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3930666307' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-02T17:44:51.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:50 smithi083 ceph-mon[37031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:44:51.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:50 smithi083 ceph-mon[37031]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-02T17:44:51.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.269+0000 7fa575ffb640 1 -- 172.21.15.69:0/1411769974 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa570014030 con 0x7fa5781067b0 2024-08-02T17:44:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:50 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3930666307' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-02T17:44:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:50 smithi077 ceph-mon[37108]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:44:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:50 smithi077 ceph-mon[37108]: pgmap v287: 129 pgs: 129 active+clean; 584 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-02T17:44:51.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.461+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa578108db0 con 0x7fa5680a2b80 2024-08-02T17:44:51.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.489+0000 7fa575ffb640 1 -- 172.21.15.69:0/1411769974 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa56c0096c0 con 0x7fa5680a2b80 2024-08-02T17:44:51.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5680a2b80 msgr2=0x7fa5680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:51.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5680a2b80 0x7fa5680a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009ab0 tx=0x7fa56c0094b0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781067b0 msgr2=0x7fa57810eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781067b0 0x7fa57810eb90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa57000cad0 tx=0x7fa57000cf90 comp rx=0 tx=0).stop 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 shutdown_connections 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa5680a2b80 0x7fa5680a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa5781067b0 0x7fa57810eb90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa578101150 0x7fa57810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 --2- 172.21.15.69:0/1411769974 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5781007b0 0x7fa57810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 >> 172.21.15.69:0/1411769974 conn(0x7fa5780fc4d0 msgr2=0x7fa5780fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 shutdown_connections 2024-08-02T17:44:51.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:51.491+0000 7fa57f216640 1 -- 172.21.15.69:0/1411769974 wait complete. 2024-08-02T17:44:51.798 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:44:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:44:51.466+0000 7f73bfa75640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:44:51.799 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:44:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:44:51.466+0000 7f73bfa75640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:44:51.799 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:44:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:44:51.466+0000 7f73bfa75640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:44:51.799 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:44:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:44:51.466+0000 7f73bfa75640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:44:51.799 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:44:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:44:51.466+0000 7f73bfa75640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:44:52.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:51 smithi069 ceph-mon[32746]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:52.235 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-02T17:44:52.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:51 smithi083 ceph-mon[37031]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:52.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:51 smithi077 ceph-mon[37108]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:52.488 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 -- 172.21.15.69:0/2223632921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801007b0 msgr2=0x7fa080100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 --2- 172.21.15.69:0/2223632921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801007b0 0x7fa080100c10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa074009930 tx=0x7fa07402a420 comp rx=0 tx=0).stop 2024-08-02T17:44:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 -- 172.21.15.69:0/2223632921 shutdown_connections 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 --2- 172.21.15.69:0/2223632921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801007b0 0x7fa080100c10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 --2- 172.21.15.69:0/2223632921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa080101150 0x7fa08010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 --2- 172.21.15.69:0/2223632921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa0801067b0 0x7fa080106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 -- 172.21.15.69:0/2223632921 >> 172.21.15.69:0/2223632921 conn(0x7fa0800fc4d0 msgr2=0x7fa0800fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 -- 172.21.15.69:0/2223632921 shutdown_connections 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.975+0000 7fa084b2f640 1 -- 172.21.15.69:0/2223632921 wait complete. 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa084b2f640 1 Processor -- start 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa084b2f640 1 -- start start 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa084b2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0801007b0 0x7fa08019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa084b2f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa080101150 0x7fa08019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa084b2f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801067b0 0x7fa08019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa084b2f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa080110bb0 con 0x7fa0801067b0 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa084b2f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa080110a30 con 0x7fa080101150 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa084b2f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa080110d30 con 0x7fa0801007b0 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801067b0 0x7fa08019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801067b0 0x7fa08019ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49510/0 (socket says 172.21.15.69:49510) 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 -- 172.21.15.69:0/963945071 learned_addr learned my addr 172.21.15.69:0/963945071 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07dd9b640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa080101150 0x7fa08019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07e59c640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0801007b0 0x7fa08019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 -- 172.21.15.69:0/963945071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0801007b0 msgr2=0x7fa08019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0801007b0 0x7fa08019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 -- 172.21.15.69:0/963945071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa080101150 msgr2=0x7fa08019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:52.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa080101150 0x7fa08019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:52.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 -- 172.21.15.69:0/963945071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08019f340 con 0x7fa0801067b0 2024-08-02T17:44:52.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.976+0000 7fa07ed9d640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801067b0 0x7fa08019ec60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa07800cad0 tx=0x7fa07800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:52.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.977+0000 7fa06f7fe640 1 -- 172.21.15.69:0/963945071 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07800ae20 con 0x7fa0801067b0 2024-08-02T17:44:52.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.977+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08019f630 con 0x7fa0801067b0 2024-08-02T17:44:52.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.977+0000 7fa06f7fe640 1 -- 172.21.15.69:0/963945071 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa078005020 con 0x7fa0801067b0 2024-08-02T17:44:52.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.977+0000 7fa06f7fe640 1 -- 172.21.15.69:0/963945071 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07800f6f0 con 0x7fa0801067b0 2024-08-02T17:44:52.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.977+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0801a6f10 con 0x7fa0801067b0 2024-08-02T17:44:52.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.978+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa080100c10 con 0x7fa0801067b0 2024-08-02T17:44:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.979+0000 7fa06f7fe640 1 -- 172.21.15.69:0/963945071 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa0780051d0 con 0x7fa0801067b0 2024-08-02T17:44:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.979+0000 7fa06f7fe640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa0680a2c10 0x7fa0680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.980+0000 7fa07e59c640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa0680a2c10 0x7fa0680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.980+0000 7fa06f7fe640 1 -- 172.21.15.69:0/963945071 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7fa0780cf820 con 0x7fa0801067b0 2024-08-02T17:44:52.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.980+0000 7fa07e59c640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa0680a2c10 0x7fa0680a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa070009ae0 tx=0x7fa070005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:52.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:52.984+0000 7fa06f7fe640 1 -- 172.21.15.69:0/963945071 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa078014030 con 0x7fa0801067b0 2024-08-02T17:44:53.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.173+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa08006ac10 con 0x7fa0680a2c10 2024-08-02T17:44:53.175 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:52 smithi069 ceph-mon[32746]: pgmap v288: 129 pgs: 129 active+clean; 590 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-08-02T17:44:53.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.195+0000 7fa06f7fe640 1 -- 172.21.15.69:0/963945071 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa08006ac10 con 0x7fa0680a2c10 2024-08-02T17:44:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa0680a2c10 msgr2=0x7fa0680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa0680a2c10 0x7fa0680a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa070009ae0 tx=0x7fa070005f90 comp rx=0 tx=0).stop 2024-08-02T17:44:53.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801067b0 msgr2=0x7fa08019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801067b0 0x7fa08019ec60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa07800cad0 tx=0x7fa07800cf90 comp rx=0 tx=0).stop 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 shutdown_connections 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fa0680a2c10 0x7fa0680a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa0801067b0 0x7fa08019ec60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa080101150 0x7fa08019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 --2- 172.21.15.69:0/963945071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0801007b0 0x7fa08019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 >> 172.21.15.69:0/963945071 conn(0x7fa0800fc4d0 msgr2=0x7fa08006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 shutdown_connections 2024-08-02T17:44:53.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:53.197+0000 7fa084b2f640 1 -- 172.21.15.69:0/963945071 wait complete. 2024-08-02T17:44:53.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:52 smithi083 ceph-mon[37031]: pgmap v288: 129 pgs: 129 active+clean; 590 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-08-02T17:44:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:52 smithi077 ceph-mon[37108]: pgmap v288: 129 pgs: 129 active+clean; 590 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-08-02T17:44:53.928 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-02T17:44:54.180 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:53 smithi083 ceph-mon[37031]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-02T17:44:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:53 smithi083 ceph-mon[37031]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:54.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:53 smithi069 ceph-mon[32746]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-02T17:44:54.302 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:53 smithi069 ceph-mon[32746]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:53 smithi077 ceph-mon[37108]: mgrmap e17: x(active, since 6m), standbys: z, y 2024-08-02T17:44:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:53 smithi077 ceph-mon[37108]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:54.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.628+0000 7f506b71b640 1 -- 172.21.15.69:0/273105109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 msgr2=0x7f506410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:54.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.628+0000 7f506b71b640 1 --2- 172.21.15.69:0/273105109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 0x7f506410b880 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5058009900 tx=0x7f505802a480 comp rx=0 tx=0).stop 2024-08-02T17:44:54.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 -- 172.21.15.69:0/273105109 shutdown_connections 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 --2- 172.21.15.69:0/273105109 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 0x7f506410b880 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 --2- 172.21.15.69:0/273105109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f506410bea0 0x7f506410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 --2- 172.21.15.69:0/273105109 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5064102a70 0x7f5064102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 -- 172.21.15.69:0/273105109 >> 172.21.15.69:0/273105109 conn(0x7f50640fc4d0 msgr2=0x7f50640fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 -- 172.21.15.69:0/273105109 shutdown_connections 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 -- 172.21.15.69:0/273105109 wait complete. 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 Processor -- start 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 -- start start 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5064102a70 0x7f5064198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 0x7f50641986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f506410bea0 0x7f506419ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:54.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5064111ca0 con 0x7f5064103390 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5064111b20 con 0x7f506410bea0 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.629+0000 7f506b71b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5064111e20 con 0x7f5064102a70 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 0x7f50641986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 0x7f50641986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49522/0 (socket says 172.21.15.69:49522) 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 -- 172.21.15.69:0/216956116 learned_addr learned my addr 172.21.15.69:0/216956116 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f50694b7640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5064102a70 0x7f5064198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5069cb8640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f506410bea0 0x7f506419ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 -- 172.21.15.69:0/216956116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5064102a70 msgr2=0x7f5064198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5064102a70 0x7f5064198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 -- 172.21.15.69:0/216956116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f506410bea0 msgr2=0x7f506419ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f506410bea0 0x7f506419ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 -- 172.21.15.69:0/216956116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506419d130 con 0x7f5064103390 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f5068cb6640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 0x7f50641986c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5058006010 tx=0x7f50580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:54.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f50627fc640 1 -- 172.21.15.69:0/216956116 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505803f030 con 0x7f5064103390 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f50627fc640 1 -- 172.21.15.69:0/216956116 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5058003ce0 con 0x7f5064103390 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f50627fc640 1 -- 172.21.15.69:0/216956116 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5058035070 con 0x7f5064103390 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506419d3c0 con 0x7f5064103390 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.630+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50641a4d00 con 0x7f5064103390 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.631+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5064102e50 con 0x7f5064103390 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.632+0000 7f50627fc640 1 -- 172.21.15.69:0/216956116 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5058042030 con 0x7f5064103390 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.633+0000 7f50627fc640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f504c0a2c90 0x7f504c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.633+0000 7f50627fc640 1 -- 172.21.15.69:0/216956116 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f50580efb80 con 0x7f5064103390 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.633+0000 7f50694b7640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f504c0a2c90 0x7f504c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.633+0000 7f50694b7640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f504c0a2c90 0x7f504c0a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5054005c40 tx=0x7f505400a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:54.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.636+0000 7f50627fc640 1 -- 172.21.15.69:0/216956116 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f50580bc800 con 0x7f5064103390 2024-08-02T17:44:54.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.829+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f50640fe2c0 con 0x7f504c0a2c90 2024-08-02T17:44:54.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.833+0000 7f50627fc640 1 -- 172.21.15.69:0/216956116 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f50640fe2c0 con 0x7f504c0a2c90 2024-08-02T17:44:54.834 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_0/9d228208-6b65-4a6a-ac45-ada56067b535 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f504c0a2c90 msgr2=0x7f504c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f504c0a2c90 0x7f504c0a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5054005c40 tx=0x7f505400a590 comp rx=0 tx=0).stop 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 msgr2=0x7f50641986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 0x7f50641986c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5058006010 tx=0x7f50580042f0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 shutdown_connections 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f504c0a2c90 0x7f504c0a5150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5064103390 0x7f50641986c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f506410bea0 0x7f506419ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 --2- 172.21.15.69:0/216956116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5064102a70 0x7f5064198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 >> 172.21.15.69:0/216956116 conn(0x7f50640fc4d0 msgr2=0x7f50640fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.835+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 shutdown_connections 2024-08-02T17:44:54.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:54.836+0000 7f506b71b640 1 -- 172.21.15.69:0/216956116 wait complete. 2024-08-02T17:44:55.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:54 smithi083 ceph-mon[37031]: pgmap v289: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-08-02T17:44:55.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:54 smithi083 ceph-mon[37031]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:55.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:54 smithi069 ceph-mon[32746]: pgmap v289: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-08-02T17:44:55.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:54 smithi069 ceph-mon[32746]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:54 smithi077 ceph-mon[37108]: pgmap v289: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-08-02T17:44:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:54 smithi077 ceph-mon[37108]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:55.611 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-02T17:44:55.865 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:56.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/450872631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 msgr2=0x7f2b58101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:56.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/450872631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 0x7f2b58101d20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c009930 tx=0x7f2b4c02a420 comp rx=0 tx=0).stop 2024-08-02T17:44:56.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/450872631 shutdown_connections 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/450872631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 0x7f2b58101d20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/450872631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b58102260 0x7f2b5810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/450872631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b581078c0 0x7f2b58107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/450872631 >> 172.21.15.69:0/450872631 conn(0x7f2b580fd5e0 msgr2=0x7f2b580ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/450872631 shutdown_connections 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.327+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/450872631 wait complete. 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b5ea5a640 1 Processor -- start 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b5ea5a640 1 -- start start 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b5ea5a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 0x7f2b58074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b5ea5a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b58102260 0x7f2b58074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b5ea5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b581078c0 0x7f2b58074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:56.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b5ea5a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58111cc0 con 0x7f2b581018c0 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b5ea5a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58111b40 con 0x7f2b58102260 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b5ea5a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b58111e40 con 0x7f2b581078c0 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.328+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 0x7f2b58074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b57fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 0x7f2b58074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38576/0 (socket says 172.21.15.69:38576) 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b57fff640 1 -- 172.21.15.69:0/1535352011 learned_addr learned my addr 172.21.15.69:0/1535352011 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b5cff7640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b581078c0 0x7f2b58074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b577fe640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b58102260 0x7f2b58074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b57fff640 1 -- 172.21.15.69:0/1535352011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b581078c0 msgr2=0x7f2b58074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b57fff640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b581078c0 0x7f2b58074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b57fff640 1 -- 172.21.15.69:0/1535352011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b58102260 msgr2=0x7f2b58074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b57fff640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b58102260 0x7f2b58074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b57fff640 1 -- 172.21.15.69:0/1535352011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b581a6610 con 0x7f2b581018c0 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b577fe640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b58102260 0x7f2b58074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:56.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b57fff640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 0x7f2b58074220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800c890 tx=0x7f2b4800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b557fa640 1 -- 172.21.15.69:0/1535352011 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b48004220 con 0x7f2b581018c0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b557fa640 1 -- 172.21.15.69:0/1535352011 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b480043c0 con 0x7f2b581018c0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b581a6900 con 0x7f2b581018c0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b557fa640 1 -- 172.21.15.69:0/1535352011 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b480049e0 con 0x7f2b581018c0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.329+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b581a6e10 con 0x7f2b581018c0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.330+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b58101d20 con 0x7f2b581018c0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.331+0000 7f2b557fa640 1 -- 172.21.15.69:0/1535352011 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2b480079b0 con 0x7f2b581018c0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.332+0000 7f2b557fa640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f2b400a2c90 0x7f2b400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.332+0000 7f2b577fe640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f2b400a2c90 0x7f2b400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.332+0000 7f2b557fa640 1 -- 172.21.15.69:0/1535352011 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8802+0+0 (secure 0 0 0) 0x7f2b480cf470 con 0x7f2b581018c0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.332+0000 7f2b577fe640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f2b400a2c90 0x7f2b400a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c000c00 tx=0x7f2b4c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:56.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.336+0000 7f2b557fa640 1 -- 172.21.15.69:0/1535352011 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2b48014030 con 0x7f2b581018c0 2024-08-02T17:44:56.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.528+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f2b58109ea0 con 0x7f2b400a2c90 2024-08-02T17:44:56.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.551+0000 7f2b557fa640 1 -- 172.21.15.69:0/1535352011 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2b4c031880 con 0x7f2b400a2c90 2024-08-02T17:44:56.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f2b400a2c90 msgr2=0x7f2b400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:56.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f2b400a2c90 0x7f2b400a5150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c000c00 tx=0x7f2b4c005a20 comp rx=0 tx=0).stop 2024-08-02T17:44:56.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 msgr2=0x7f2b58074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 0x7f2b58074220 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2b4800c890 tx=0x7f2b4800cd50 comp rx=0 tx=0).stop 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 shutdown_connections 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f2b400a2c90 0x7f2b400a5150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b581078c0 0x7f2b58074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b58102260 0x7f2b58074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 --2- 172.21.15.69:0/1535352011 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b581018c0 0x7f2b58074220 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 >> 172.21.15.69:0/1535352011 conn(0x7f2b580fd5e0 msgr2=0x7f2b580fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 shutdown_connections 2024-08-02T17:44:56.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:56.553+0000 7f2b5ea5a640 1 -- 172.21.15.69:0/1535352011 wait complete. 2024-08-02T17:44:56.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-08-02T17:44:56.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-08-02T17:44:56.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-02T17:44:56.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-08-02T17:44:56.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-08-02T17:44:56.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-08-02T17:44:56.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-08-02T17:44:56.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]: dispatch 2024-08-02T17:44:56.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-08-02T17:44:56.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-08-02T17:44:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-08-02T17:44:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-08-02T17:44:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-02T17:44:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-08-02T17:44:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-08-02T17:44:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-08-02T17:44:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-08-02T17:44:57.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]: dispatch 2024-08-02T17:44:57.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-08-02T17:44:57.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-08-02T17:44:57.220 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-02T17:44:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-08-02T17:44:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-08-02T17:44:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-08-02T17:44:57.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-08-02T17:44:57.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-08-02T17:44:57.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-08-02T17:44:57.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]: dispatch 2024-08-02T17:44:57.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]: dispatch 2024-08-02T17:44:57.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-08-02T17:44:57.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-08-02T17:44:57.474 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:57.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: pgmap v291: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-08-02T17:44:57.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:57 smithi069 ceph-mon[32746]: osdmap e91: 12 total, 12 up, 12 in 2024-08-02T17:44:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 -- 172.21.15.69:0/3970243556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1067b0 msgr2=0x7f32cc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 --2- 172.21.15.69:0/3970243556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1067b0 0x7f32cc106b90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009930 tx=0x7f32bc02a420 comp rx=0 tx=0).stop 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 -- 172.21.15.69:0/3970243556 shutdown_connections 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 --2- 172.21.15.69:0/3970243556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32cc101150 0x7f32cc10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 --2- 172.21.15.69:0/3970243556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32cc1007b0 0x7f32cc100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 --2- 172.21.15.69:0/3970243556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1067b0 0x7f32cc106b90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 -- 172.21.15.69:0/3970243556 >> 172.21.15.69:0/3970243556 conn(0x7f32cc0fc4d0 msgr2=0x7f32cc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 -- 172.21.15.69:0/3970243556 shutdown_connections 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.945+0000 7f32d27fd640 1 -- 172.21.15.69:0/3970243556 wait complete. 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32d27fd640 1 Processor -- start 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32d27fd640 1 -- start start 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32d27fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1007b0 0x7f32cc198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32d27fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32cc101150 0x7f32cc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32d27fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32cc1067b0 0x7f32cc19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32d27fd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc110bd0 con 0x7f32cc1007b0 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32d27fd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc110a50 con 0x7f32cc1067b0 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32d27fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc110d50 con 0x7f32cc101150 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32cbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1007b0 0x7f32cc198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32cbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1007b0 0x7f32cc198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38584/0 (socket says 172.21.15.69:38584) 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.946+0000 7f32cbfff640 1 -- 172.21.15.69:0/1697591229 learned_addr learned my addr 172.21.15.69:0/1697591229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32cb7fe640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32cc101150 0x7f32cc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32d0d9a640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32cc1067b0 0x7f32cc19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32cbfff640 1 -- 172.21.15.69:0/1697591229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32cc101150 msgr2=0x7f32cc198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32cbfff640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32cc101150 0x7f32cc198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32cbfff640 1 -- 172.21.15.69:0/1697591229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32cc1067b0 msgr2=0x7f32cc19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:57.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32cbfff640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32cc1067b0 0x7f32cc19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32cbfff640 1 -- 172.21.15.69:0/1697591229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc19d1e0 con 0x7f32cc1007b0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32cb7fe640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32cc101150 0x7f32cc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32d0d9a640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32cc1067b0 0x7f32cc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32cbfff640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1007b0 0x7f32cc198230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009a50 tx=0x7f32bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32c97fa640 1 -- 172.21.15.69:0/1697591229 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc03f030 con 0x7f32cc1007b0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc19d470 con 0x7f32cc1007b0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32c97fa640 1 -- 172.21.15.69:0/1697591229 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc003ce0 con 0x7f32cc1007b0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32c97fa640 1 -- 172.21.15.69:0/1697591229 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc035070 con 0x7f32cc1007b0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.947+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a4db0 con 0x7f32cc1007b0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.948+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc100c10 con 0x7f32cc1007b0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.949+0000 7f32c97fa640 1 -- 172.21.15.69:0/1697591229 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f32bc042030 con 0x7f32cc1007b0 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.950+0000 7f32c97fa640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32b40a2c10 0x7f32b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:57.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.950+0000 7f32c97fa640 1 -- 172.21.15.69:0/1697591229 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f32bc0efc00 con 0x7f32cc1007b0 2024-08-02T17:44:57.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.950+0000 7f32cb7fe640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32b40a2c10 0x7f32b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:57.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.950+0000 7f32cb7fe640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32b40a2c10 0x7f32b40a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f32cc104f10 tx=0x7f32c0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:57.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:57.954+0000 7f32c97fa640 1 -- 172.21.15.69:0/1697591229 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f32bc0bc640 con 0x7f32cc1007b0 2024-08-02T17:44:58.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-08-02T17:44:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: pgmap v291: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-08-02T17:44:58.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:57 smithi077 ceph-mon[37108]: osdmap e91: 12 total, 12 up, 12 in 2024-08-02T17:44:58.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.146+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f32cc062360 con 0x7f32b40a2c10 2024-08-02T17:44:58.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.152+0000 7f32c97fa640 1 -- 172.21.15.69:0/1697591229 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f32cc062360 con 0x7f32b40a2c10 2024-08-02T17:44:58.157 INFO:teuthology.orchestra.run.smithi069.stdout:/volumes/qa/sv_1/39fc5226-68dc-46b2-9a4d-e93437ee40a4 2024-08-02T17:44:58.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32b40a2c10 msgr2=0x7f32b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:58.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32b40a2c10 0x7f32b40a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f32cc104f10 tx=0x7f32c0008040 comp rx=0 tx=0).stop 2024-08-02T17:44:58.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1007b0 msgr2=0x7f32cc198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:58.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1007b0 0x7f32cc198230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009a50 tx=0x7f32bc004270 comp rx=0 tx=0).stop 2024-08-02T17:44:58.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 shutdown_connections 2024-08-02T17:44:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32b40a2c10 0x7f32b40a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32cc1067b0 0x7f32cc19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32cc101150 0x7f32cc198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 --2- 172.21.15.69:0/1697591229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32cc1007b0 0x7f32cc198230 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 >> 172.21.15.69:0/1697591229 conn(0x7f32cc0fc4d0 msgr2=0x7f32cc06a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 shutdown_connections 2024-08-02T17:44:58.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:58.155+0000 7f32d27fd640 1 -- 172.21.15.69:0/1697591229 wait complete. 2024-08-02T17:44:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:44:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-08-02T17:44:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-08-02T17:44:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-08-02T17:44:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-08-02T17:44:58.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-08-02T17:44:58.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-08-02T17:44:58.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [5, 8]}]': finished 2024-08-02T17:44:58.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 8]}]': finished 2024-08-02T17:44:58.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-08-02T17:44:58.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-08-02T17:44:58.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: pgmap v291: 129 pgs: 129 active+clean; 591 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-08-02T17:44:58.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:57 smithi083 ceph-mon[37031]: osdmap e91: 12 total, 12 up, 12 in 2024-08-02T17:44:58.817 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:44:59.069 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:44:59.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:58 smithi077 ceph-mon[37108]: osdmap e92: 12 total, 12 up, 12 in 2024-08-02T17:44:59.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:58 smithi069 ceph-mon[32746]: osdmap e92: 12 total, 12 up, 12 in 2024-08-02T17:44:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:58 smithi083 ceph-mon[37031]: osdmap e92: 12 total, 12 up, 12 in 2024-08-02T17:44:59.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/3862191681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 msgr2=0x7fb0a4100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/3862191681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 0x7fb0a4100c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb098009930 tx=0x7fb09802a420 comp rx=0 tx=0).stop 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/3862191681 shutdown_connections 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/3862191681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 0x7fb0a4100c10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/3862191681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0a4101150 0x7fb0a410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/3862191681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a41067b0 0x7fb0a4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/3862191681 >> 172.21.15.69:0/3862191681 conn(0x7fb0a40fc4d0 msgr2=0x7fb0a40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/3862191681 shutdown_connections 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.522+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/3862191681 wait complete. 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a8fe0640 1 Processor -- start 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a8fe0640 1 -- start start 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a8fe0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 0x7fb0a41970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:59.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a8fe0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0a4101150 0x7fb0a4197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a8fe0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a41067b0 0x7fb0a419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a8fe0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4110bb0 con 0x7fb0a41007b0 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a8fe0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4110a30 con 0x7fb0a4101150 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a8fe0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a4110d30 con 0x7fb0a41067b0 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 0x7fb0a41970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 0x7fb0a41970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38610/0 (socket says 172.21.15.69:38610) 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a259c640 1 -- 172.21.15.69:0/2554052653 learned_addr learned my addr 172.21.15.69:0/2554052653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a1d9b640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0a4101150 0x7fb0a4197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a2d9d640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a41067b0 0x7fb0a419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:59.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a259c640 1 -- 172.21.15.69:0/2554052653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a41067b0 msgr2=0x7fb0a419b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a259c640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a41067b0 0x7fb0a419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a259c640 1 -- 172.21.15.69:0/2554052653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0a4101150 msgr2=0x7fb0a4197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a259c640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0a4101150 0x7fb0a4197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a259c640 1 -- 172.21.15.69:0/2554052653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a419c080 con 0x7fb0a41007b0 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.523+0000 7fb0a1d9b640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0a4101150 0x7fb0a4197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.524+0000 7fb0a259c640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 0x7fb0a41970d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb09400c890 tx=0x7fb09400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.524+0000 7fb0937fe640 1 -- 172.21.15.69:0/2554052653 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094004220 con 0x7fb0a41007b0 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.524+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a419c370 con 0x7fb0a41007b0 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.524+0000 7fb0937fe640 1 -- 172.21.15.69:0/2554052653 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0940043c0 con 0x7fb0a41007b0 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.524+0000 7fb0937fe640 1 -- 172.21.15.69:0/2554052653 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0940049e0 con 0x7fb0a41007b0 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.524+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a41a3c50 con 0x7fb0a41007b0 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.525+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a4100c10 con 0x7fb0a41007b0 2024-08-02T17:44:59.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.526+0000 7fb0937fe640 1 -- 172.21.15.69:0/2554052653 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb0940079b0 con 0x7fb0a41007b0 2024-08-02T17:44:59.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.527+0000 7fb0937fe640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:44:59.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.527+0000 7fb0a1d9b640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:44:59.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.527+0000 7fb0937fe640 1 -- 172.21.15.69:0/2554052653 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fb0940cf640 con 0x7fb0a41007b0 2024-08-02T17:44:59.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.527+0000 7fb0a1d9b640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb098000c00 tx=0x7fb098032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:44:59.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.530+0000 7fb0937fe640 1 -- 172.21.15.69:0/2554052653 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb094014030 con 0x7fb0a41007b0 2024-08-02T17:44:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.731+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0a4198430 con 0x7fb0a41007b0 2024-08-02T17:44:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.732+0000 7fb0937fe640 1 -- 172.21.15.69:0/2554052653 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11900 (secure 0 0 0) 0x7fb09409c080 con 0x7fb0a41007b0 2024-08-02T17:44:59.733 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:44:59.733 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"btime":"2024-08-02T17:44:49:913571+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:44:49.496243+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24431,24452]},"id":1}]} 2024-08-02T17:44:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb08c0a2c10 msgr2=0x7fb08c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb098000c00 tx=0x7fb098032c70 comp rx=0 tx=0).stop 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 msgr2=0x7fb0a41970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 0x7fb0a41970d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb09400c890 tx=0x7fb09400cd50 comp rx=0 tx=0).stop 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 shutdown_connections 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fb08c0a2c10 0x7fb08c0a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0a41067b0 0x7fb0a419b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb0a4101150 0x7fb0a4197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 --2- 172.21.15.69:0/2554052653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb0a41007b0 0x7fb0a41970d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:44:59.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 >> 172.21.15.69:0/2554052653 conn(0x7fb0a40fc4d0 msgr2=0x7fb0a40fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:44:59.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 shutdown_connections 2024-08-02T17:44:59.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:44:59.734+0000 7fb0a8fe0640 1 -- 172.21.15.69:0/2554052653 wait complete. 2024-08-02T17:45:00.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:59 smithi077 ceph-mon[37108]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:45:00.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:59 smithi077 ceph-mon[37108]: pgmap v293: 129 pgs: 2 peering, 127 active+clean; 623 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-08-02T17:45:00.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:44:59 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2554052653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:59 smithi069 ceph-mon[32746]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:45:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:59 smithi069 ceph-mon[32746]: pgmap v293: 129 pgs: 2 peering, 127 active+clean; 623 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-08-02T17:45:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:44:59 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2554052653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:00.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:59 smithi083 ceph-mon[37031]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:45:00.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:59 smithi083 ceph-mon[37031]: pgmap v293: 129 pgs: 2 peering, 127 active+clean; 623 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 13 op/s 2024-08-02T17:45:00.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:44:59 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2554052653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:00.488 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:45:00.900 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:00 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:45:01.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:00 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:45:01.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:00 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:45:01.398 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:01.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 -- 172.21.15.69:0/2696696495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01007b0 msgr2=0x7fc1c0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:01.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/2696696495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01007b0 0x7fc1c0100c10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009930 tx=0x7fc1b402a420 comp rx=0 tx=0).stop 2024-08-02T17:45:01.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 -- 172.21.15.69:0/2696696495 shutdown_connections 2024-08-02T17:45:01.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/2696696495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01007b0 0x7fc1c0100c10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:01.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/2696696495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c0101150 0x7fc1c010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:01.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/2696696495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c01067b0 0x7fc1c0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 -- 172.21.15.69:0/2696696495 >> 172.21.15.69:0/2696696495 conn(0x7fc1c00fc4d0 msgr2=0x7fc1c00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 -- 172.21.15.69:0/2696696495 shutdown_connections 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.915+0000 7fc1c54eb640 1 -- 172.21.15.69:0/2696696495 wait complete. 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.916+0000 7fc1c54eb640 1 Processor -- start 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.916+0000 7fc1c54eb640 1 -- start start 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1c54eb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c01007b0 0x7fc1c0198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1c54eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0101150 0x7fc1c0199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1c54eb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01067b0 0x7fc1c01a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1c54eb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c0110bb0 con 0x7fc1c01067b0 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1c54eb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c0110a30 con 0x7fc1c0101150 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1c54eb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c0110d30 con 0x7fc1c01007b0 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01067b0 0x7fc1c01a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:01.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01067b0 0x7fc1c01a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38636/0 (socket says 172.21.15.69:38636) 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 -- 172.21.15.69:0/3650455591 learned_addr learned my addr 172.21.15.69:0/3650455591 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1be7fc640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0101150 0x7fc1c0199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1beffd640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c01007b0 0x7fc1c0198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 -- 172.21.15.69:0/3650455591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c01007b0 msgr2=0x7fc1c0198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c01007b0 0x7fc1c0198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 -- 172.21.15.69:0/3650455591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0101150 msgr2=0x7fc1c0199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0101150 0x7fc1c0199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 -- 172.21.15.69:0/3650455591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c01a44e0 con 0x7fc1c01067b0 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1be7fc640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0101150 0x7fc1c0199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1beffd640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c01007b0 0x7fc1c0198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1bf7fe640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01067b0 0x7fc1c01a3fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800eb10 tx=0x7fc1b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1a7fff640 1 -- 172.21.15.69:0/3650455591 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8004b20 con 0x7fc1c01067b0 2024-08-02T17:45:01.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1a7fff640 1 -- 172.21.15.69:0/3650455591 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1b8004cc0 con 0x7fc1c01067b0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1a7fff640 1 -- 172.21.15.69:0/3650455591 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8010760 con 0x7fc1c01067b0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c01a46b0 con 0x7fc1c01067b0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.917+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c01a4b70 con 0x7fc1c01067b0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.918+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c0100c10 con 0x7fc1c01067b0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.920+0000 7fc1a7fff640 1 -- 172.21.15.69:0/3650455591 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc1b8005000 con 0x7fc1c01067b0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.920+0000 7fc1a7fff640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1a80a2c10 0x7fc1a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.920+0000 7fc1beffd640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1a80a2c10 0x7fc1a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.920+0000 7fc1a7fff640 1 -- 172.21.15.69:0/3650455591 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc1b80d00a0 con 0x7fc1c01067b0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.921+0000 7fc1beffd640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1a80a2c10 0x7fc1a80a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0005e10 tx=0x7fc1b0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:01.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:01.923+0000 7fc1a7fff640 1 -- 172.21.15.69:0/3650455591 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc1b809cb40 con 0x7fc1c01067b0 2024-08-02T17:45:02.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:01 smithi077 ceph-mon[37108]: pgmap v294: 129 pgs: 2 peering, 127 active+clean; 623 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-08-02T17:45:02.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:01 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:01 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:01 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:01 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:01 smithi069 ceph-mon[32746]: pgmap v294: 129 pgs: 2 peering, 127 active+clean; 623 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-08-02T17:45:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:01 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:01 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:01 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:01 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.129+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1c01a4e30 con 0x7fc1c01067b0 2024-08-02T17:45:02.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.131+0000 7fc1a7fff640 1 -- 172.21.15.69:0/3650455591 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24491 (secure 0 0 0) 0x7fc1b809c440 con 0x7fc1c01067b0 2024-08-02T17:45:02.132 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:02.132 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:57.767460+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:45:02.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1a80a2c10 msgr2=0x7fc1a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:02.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1a80a2c10 0x7fc1a80a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0005e10 tx=0x7fc1b0005da0 comp rx=0 tx=0).stop 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01067b0 msgr2=0x7fc1c01a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01067b0 0x7fc1c01a3fa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800eb10 tx=0x7fc1b800c2d0 comp rx=0 tx=0).stop 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 shutdown_connections 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc1a80a2c10 0x7fc1a80a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc1c01067b0 0x7fc1c01a3fa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc1c0101150 0x7fc1c0199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 --2- 172.21.15.69:0/3650455591 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1c01007b0 0x7fc1c0198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 >> 172.21.15.69:0/3650455591 conn(0x7fc1c00fc4d0 msgr2=0x7fc1c00fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 shutdown_connections 2024-08-02T17:45:02.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:02.134+0000 7fc1c54eb640 1 -- 172.21.15.69:0/3650455591 wait complete. 2024-08-02T17:45:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:01 smithi083 ceph-mon[37031]: pgmap v294: 129 pgs: 2 peering, 127 active+clean; 623 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.4 KiB/s wr, 9 op/s; 14 B/s, 0 keys/s, 0 objects/s recovering 2024-08-02T17:45:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:01 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:01 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:01 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:01 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:02.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:02 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3650455591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:02.889 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-02T17:45:02.889 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph pg dump --format=json 2024-08-02T17:45:03.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:02 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3650455591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:03.164 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:03.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:02 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3650455591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:03.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 -- 172.21.15.69:0/3313886841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c10bea0 msgr2=0x7f5a7c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:03.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 --2- 172.21.15.69:0/3313886841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c10bea0 0x7f5a7c10e340 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009900 tx=0x7f5a7402a480 comp rx=0 tx=0).stop 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 -- 172.21.15.69:0/3313886841 shutdown_connections 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 --2- 172.21.15.69:0/3313886841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c10bea0 0x7f5a7c10e340 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 --2- 172.21.15.69:0/3313886841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a7c103390 0x7f5a7c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 --2- 172.21.15.69:0/3313886841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c102a70 0x7f5a7c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 -- 172.21.15.69:0/3313886841 >> 172.21.15.69:0/3313886841 conn(0x7f5a7c0fc4d0 msgr2=0x7f5a7c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 -- 172.21.15.69:0/3313886841 shutdown_connections 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 -- 172.21.15.69:0/3313886841 wait complete. 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.672+0000 7f5a836aa640 1 Processor -- start 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a836aa640 1 -- start start 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a836aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c102a70 0x7f5a7c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a836aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a7c103390 0x7f5a7c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a836aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c10bea0 0x7f5a7c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:03.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a836aa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c111ca0 con 0x7f5a7c102a70 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a836aa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c111b20 con 0x7f5a7c103390 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a836aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c111e20 con 0x7f5a7c10bea0 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c102a70 0x7f5a7c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c102a70 0x7f5a7c198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38660/0 (socket says 172.21.15.69:38660) 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 -- 172.21.15.69:0/2973409979 learned_addr learned my addr 172.21.15.69:0/2973409979 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81c47640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c10bea0 0x7f5a7c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a80c45640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a7c103390 0x7f5a7c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 -- 172.21.15.69:0/2973409979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c10bea0 msgr2=0x7f5a7c19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c10bea0 0x7f5a7c19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 -- 172.21.15.69:0/2973409979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a7c103390 msgr2=0x7f5a7c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a7c103390 0x7f5a7c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 -- 172.21.15.69:0/2973409979 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7c19d0f0 con 0x7f5a7c102a70 2024-08-02T17:45:03.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.673+0000 7f5a81446640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c102a70 0x7f5a7c198140 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00ea80 tx=0x7f5a6c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.674+0000 7f5a7a7fc640 1 -- 172.21.15.69:0/2973409979 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00ce40 con 0x7f5a7c102a70 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.674+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7c19d3e0 con 0x7f5a7c102a70 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.674+0000 7f5a7a7fc640 1 -- 172.21.15.69:0/2973409979 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a6c005020 con 0x7f5a7c102a70 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.674+0000 7f5a7a7fc640 1 -- 172.21.15.69:0/2973409979 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c011760 con 0x7f5a7c102a70 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.674+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7c1a4cc0 con 0x7f5a7c102a70 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.675+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a7c102e50 con 0x7f5a7c102a70 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.676+0000 7f5a7a7fc640 1 -- 172.21.15.69:0/2973409979 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5a6c0051d0 con 0x7f5a7c102a70 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.676+0000 7f5a7a7fc640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5a640a2c10 0x7f5a640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.676+0000 7f5a80c45640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5a640a2c10 0x7f5a640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.676+0000 7f5a7a7fc640 1 -- 172.21.15.69:0/2973409979 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5a6c0cfb10 con 0x7f5a7c102a70 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.677+0000 7f5a80c45640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5a640a2c10 0x7f5a640a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0fe9f0 tx=0x7f5a7000b360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:03.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.680+0000 7f5a7a7fc640 1 -- 172.21.15.69:0/2973409979 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5a6c016030 con 0x7f5a7c102a70 2024-08-02T17:45:03.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.847+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a7c0fe1b0 con 0x7f5a640a2c10 2024-08-02T17:45:03.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:03 smithi069 ceph-mon[32746]: pgmap v295: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.4 KiB/s wr, 9 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:03.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:03.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:03.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:45:03.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:45:03.849 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:03.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.856+0000 7f5a7a7fc640 1 -- 172.21.15.69:0/2973409979 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340430 (secure 0 0 0) 0x7f5a7c0fe1b0 con 0x7f5a640a2c10 2024-08-02T17:45:03.861 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:03.863 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.860+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5a640a2c10 msgr2=0x7f5a640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.860+0000 7f5a836aa640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5a640a2c10 0x7f5a640a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0fe9f0 tx=0x7f5a7000b360 comp rx=0 tx=0).stop 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.860+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c102a70 msgr2=0x7f5a7c198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.860+0000 7f5a836aa640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c102a70 0x7f5a7c198140 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00ea80 tx=0x7f5a6c00ef40 comp rx=0 tx=0).stop 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.861+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 shutdown_connections 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.861+0000 7f5a836aa640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5a640a2c10 0x7f5a640a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.861+0000 7f5a836aa640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a7c10bea0 0x7f5a7c19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.861+0000 7f5a836aa640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a7c103390 0x7f5a7c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.861+0000 7f5a836aa640 1 --2- 172.21.15.69:0/2973409979 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a7c102a70 0x7f5a7c198140 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.861+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 >> 172.21.15.69:0/2973409979 conn(0x7f5a7c0fc4d0 msgr2=0x7f5a7c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:03.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.861+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 shutdown_connections 2024-08-02T17:45:03.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:03.861+0000 7f5a836aa640 1 -- 172.21.15.69:0/2973409979 wait complete. 2024-08-02T17:45:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:03 smithi077 ceph-mon[37108]: pgmap v295: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.4 KiB/s wr, 9 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:04.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:45:04.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:45:04.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:04.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:03 smithi083 ceph-mon[37031]: pgmap v295: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 6.4 KiB/s wr, 9 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:04.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:04.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:04.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:45:04.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:45:04.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:45:04.657 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-08-02T17:45:02.770208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647399,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":243,"num_read_kb":237,"num_write":341,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392596,"kb_used_data":14244,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549228,"statfs":{"total":1151940427776,"available":1151538409472,"internally_reserved":0,"allocated":14585856,"data_stored":8063463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42084,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":54,"num_read_kb":72,"num_write":24,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":229,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001935"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.776089+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:44:57.776089+0000","last_peered":"2024-08-02T17:44:57.776089+0000","last_clean":"2024-08-02T17:44:57.776089+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:44:57.776089+0000","last_undegraded":"2024-08-02T17:44:57.776089+0000","last_fullsized":"2024-08-02T17:44:57.776089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778005+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:44:57.778005+0000","last_peered":"2024-08-02T17:44:57.778005+0000","last_clean":"2024-08-02T17:44:57.778005+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:44:57.778005+0000","last_undegraded":"2024-08-02T17:44:57.778005+0000","last_fullsized":"2024-08-02T17:44:57.778005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782153+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:44:57.782153+0000","last_peered":"2024-08-02T17:44:57.782153+0000","last_clean":"2024-08-02T17:44:57.782153+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:44:57.782153+0000","last_undegraded":"2024-08-02T17:44:57.782153+0000","last_fullsized":"2024-08-02T17:44:57.782153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778542+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:44:57.778542+0000","last_peered":"2024-08-02T17:44:57.778542+0000","last_clean":"2024-08-02T17:44:57.778542+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:44:57.778542+0000","last_undegraded":"2024-08-02T17:44:57.778542+0000","last_fullsized":"2024-08-02T17:44:57.778542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788699+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:44:57.788699+0000","last_peered":"2024-08-02T17:44:57.788699+0000","last_clean":"2024-08-02T17:44:57.788699+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:44:57.788699+0000","last_undegraded":"2024-08-02T17:44:57.788699+0000","last_fullsized":"2024-08-02T17:44:57.788699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777204+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:44:57.777204+0000","last_peered":"2024-08-02T17:44:57.777204+0000","last_clean":"2024-08-02T17:44:57.777204+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:44:57.777204+0000","last_undegraded":"2024-08-02T17:44:57.777204+0000","last_fullsized":"2024-08-02T17:44:57.777204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784045+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:44:57.784045+0000","last_peered":"2024-08-02T17:44:57.784045+0000","last_clean":"2024-08-02T17:44:57.784045+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:44:57.784045+0000","last_undegraded":"2024-08-02T17:44:57.784045+0000","last_fullsized":"2024-08-02T17:44:57.784045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058658+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:44:58.058658+0000","last_peered":"2024-08-02T17:44:58.058658+0000","last_clean":"2024-08-02T17:44:58.058658+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:44:58.058658+0000","last_undegraded":"2024-08-02T17:44:58.058658+0000","last_fullsized":"2024-08-02T17:44:58.058658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778197+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:44:57.778197+0000","last_peered":"2024-08-02T17:44:57.778197+0000","last_clean":"2024-08-02T17:44:57.778197+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:44:57.778197+0000","last_undegraded":"2024-08-02T17:44:57.778197+0000","last_fullsized":"2024-08-02T17:44:57.778197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783541+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:44:57.783541+0000","last_peered":"2024-08-02T17:44:57.783541+0000","last_clean":"2024-08-02T17:44:57.783541+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:44:57.783541+0000","last_undegraded":"2024-08-02T17:44:57.783541+0000","last_fullsized":"2024-08-02T17:44:57.783541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780343+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:44:57.780343+0000","last_peered":"2024-08-02T17:44:57.780343+0000","last_clean":"2024-08-02T17:44:57.780343+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:44:57.780343+0000","last_undegraded":"2024-08-02T17:44:57.780343+0000","last_fullsized":"2024-08-02T17:44:57.780343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777417+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:44:57.777417+0000","last_peered":"2024-08-02T17:44:57.777417+0000","last_clean":"2024-08-02T17:44:57.777417+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:44:57.777417+0000","last_undegraded":"2024-08-02T17:44:57.777417+0000","last_fullsized":"2024-08-02T17:44:57.777417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777532+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:44:57.777532+0000","last_peered":"2024-08-02T17:44:57.777532+0000","last_clean":"2024-08-02T17:44:57.777532+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:44:57.777532+0000","last_undegraded":"2024-08-02T17:44:57.777532+0000","last_fullsized":"2024-08-02T17:44:57.777532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783178+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:44:57.783178+0000","last_peered":"2024-08-02T17:44:57.783178+0000","last_clean":"2024-08-02T17:44:57.783178+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:44:57.783178+0000","last_undegraded":"2024-08-02T17:44:57.783178+0000","last_fullsized":"2024-08-02T17:44:57.783178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778786+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:44:57.778786+0000","last_peered":"2024-08-02T17:44:57.778786+0000","last_clean":"2024-08-02T17:44:57.778786+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:44:57.778786+0000","last_undegraded":"2024-08-02T17:44:57.778786+0000","last_fullsized":"2024-08-02T17:44:57.778786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778669+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:44:57.778669+0000","last_peered":"2024-08-02T17:44:57.778669+0000","last_clean":"2024-08-02T17:44:57.778669+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:44:57.778669+0000","last_undegraded":"2024-08-02T17:44:57.778669+0000","last_fullsized":"2024-08-02T17:44:57.778669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.793088+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:44:57.793088+0000","last_peered":"2024-08-02T17:44:57.793088+0000","last_clean":"2024-08-02T17:44:57.793088+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:44:57.793088+0000","last_undegraded":"2024-08-02T17:44:57.793088+0000","last_fullsized":"2024-08-02T17:44:57.793088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777673+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:44:57.777673+0000","last_peered":"2024-08-02T17:44:57.777673+0000","last_clean":"2024-08-02T17:44:57.777673+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:44:57.777673+0000","last_undegraded":"2024-08-02T17:44:57.777673+0000","last_fullsized":"2024-08-02T17:44:57.777673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778351+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:44:57.778351+0000","last_peered":"2024-08-02T17:44:57.778351+0000","last_clean":"2024-08-02T17:44:57.778351+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:44:57.778351+0000","last_undegraded":"2024-08-02T17:44:57.778351+0000","last_fullsized":"2024-08-02T17:44:57.778351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778275+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:44:57.778275+0000","last_peered":"2024-08-02T17:44:57.778275+0000","last_clean":"2024-08-02T17:44:57.778275+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:44:57.778275+0000","last_undegraded":"2024-08-02T17:44:57.778275+0000","last_fullsized":"2024-08-02T17:44:57.778275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775801+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:44:57.775801+0000","last_peered":"2024-08-02T17:44:57.775801+0000","last_clean":"2024-08-02T17:44:57.775801+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:44:57.775801+0000","last_undegraded":"2024-08-02T17:44:57.775801+0000","last_fullsized":"2024-08-02T17:44:57.775801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783178+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:44:57.783178+0000","last_peered":"2024-08-02T17:44:57.783178+0000","last_clean":"2024-08-02T17:44:57.783178+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:44:57.783178+0000","last_undegraded":"2024-08-02T17:44:57.783178+0000","last_fullsized":"2024-08-02T17:44:57.783178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058742+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:44:58.058742+0000","last_peered":"2024-08-02T17:44:58.058742+0000","last_clean":"2024-08-02T17:44:58.058742+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:44:58.058742+0000","last_undegraded":"2024-08-02T17:44:58.058742+0000","last_fullsized":"2024-08-02T17:44:58.058742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.789019+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:44:57.789019+0000","last_peered":"2024-08-02T17:44:57.789019+0000","last_clean":"2024-08-02T17:44:57.789019+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:44:57.789019+0000","last_undegraded":"2024-08-02T17:44:57.789019+0000","last_fullsized":"2024-08-02T17:44:57.789019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788382+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:44:57.788382+0000","last_peered":"2024-08-02T17:44:57.788382+0000","last_clean":"2024-08-02T17:44:57.788382+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:44:57.788382+0000","last_undegraded":"2024-08-02T17:44:57.788382+0000","last_fullsized":"2024-08-02T17:44:57.788382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779411+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:44:57.779411+0000","last_peered":"2024-08-02T17:44:57.779411+0000","last_clean":"2024-08-02T17:44:57.779411+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:44:57.779411+0000","last_undegraded":"2024-08-02T17:44:57.779411+0000","last_fullsized":"2024-08-02T17:44:57.779411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784040+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:44:57.784040+0000","last_peered":"2024-08-02T17:44:57.784040+0000","last_clean":"2024-08-02T17:44:57.784040+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:44:57.784040+0000","last_undegraded":"2024-08-02T17:44:57.784040+0000","last_fullsized":"2024-08-02T17:44:57.784040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778702+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:44:57.778702+0000","last_peered":"2024-08-02T17:44:57.778702+0000","last_clean":"2024-08-02T17:44:57.778702+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:44:57.778702+0000","last_undegraded":"2024-08-02T17:44:57.778702+0000","last_fullsized":"2024-08-02T17:44:57.778702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.790105+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:44:57.790105+0000","last_peered":"2024-08-02T17:44:57.790105+0000","last_clean":"2024-08-02T17:44:57.790105+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:44:57.790105+0000","last_undegraded":"2024-08-02T17:44:57.790105+0000","last_fullsized":"2024-08-02T17:44:57.790105+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777623+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:44:57.777623+0000","last_peered":"2024-08-02T17:44:57.777623+0000","last_clean":"2024-08-02T17:44:57.777623+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:44:57.777623+0000","last_undegraded":"2024-08-02T17:44:57.777623+0000","last_fullsized":"2024-08-02T17:44:57.777623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935126+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:44:57.935126+0000","last_peered":"2024-08-02T17:44:57.935126+0000","last_clean":"2024-08-02T17:44:57.935126+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:44:57.935126+0000","last_undegraded":"2024-08-02T17:44:57.935126+0000","last_fullsized":"2024-08-02T17:44:57.935126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784152+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:44:57.784152+0000","last_peered":"2024-08-02T17:44:57.784152+0000","last_clean":"2024-08-02T17:44:57.784152+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:44:57.784152+0000","last_undegraded":"2024-08-02T17:44:57.784152+0000","last_fullsized":"2024-08-02T17:44:57.784152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778014+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:44:57.778014+0000","last_peered":"2024-08-02T17:44:57.778014+0000","last_clean":"2024-08-02T17:44:57.778014+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:44:57.778014+0000","last_undegraded":"2024-08-02T17:44:57.778014+0000","last_fullsized":"2024-08-02T17:44:57.778014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779077+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:44:57.779077+0000","last_peered":"2024-08-02T17:44:57.779077+0000","last_clean":"2024-08-02T17:44:57.779077+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:44:57.779077+0000","last_undegraded":"2024-08-02T17:44:57.779077+0000","last_fullsized":"2024-08-02T17:44:57.779077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775849+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:44:57.775849+0000","last_peered":"2024-08-02T17:44:57.775849+0000","last_clean":"2024-08-02T17:44:57.775849+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:44:57.775849+0000","last_undegraded":"2024-08-02T17:44:57.775849+0000","last_fullsized":"2024-08-02T17:44:57.775849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784607+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:44:57.784607+0000","last_peered":"2024-08-02T17:44:57.784607+0000","last_clean":"2024-08-02T17:44:57.784607+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:44:57.784607+0000","last_undegraded":"2024-08-02T17:44:57.784607+0000","last_fullsized":"2024-08-02T17:44:57.784607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779247+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:44:57.779247+0000","last_peered":"2024-08-02T17:44:57.779247+0000","last_clean":"2024-08-02T17:44:57.779247+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:44:57.779247+0000","last_undegraded":"2024-08-02T17:44:57.779247+0000","last_fullsized":"2024-08-02T17:44:57.779247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777693+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:44:57.777693+0000","last_peered":"2024-08-02T17:44:57.777693+0000","last_clean":"2024-08-02T17:44:57.777693+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:44:57.777693+0000","last_undegraded":"2024-08-02T17:44:57.777693+0000","last_fullsized":"2024-08-02T17:44:57.777693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934899+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:44:57.934899+0000","last_peered":"2024-08-02T17:44:57.934899+0000","last_clean":"2024-08-02T17:44:57.934899+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:44:57.934899+0000","last_undegraded":"2024-08-02T17:44:57.934899+0000","last_fullsized":"2024-08-02T17:44:57.934899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779198+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:44:57.779198+0000","last_peered":"2024-08-02T17:44:57.779198+0000","last_clean":"2024-08-02T17:44:57.779198+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:44:57.779198+0000","last_undegraded":"2024-08-02T17:44:57.779198+0000","last_fullsized":"2024-08-02T17:44:57.779198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:59.937146+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:44:59.937146+0000","last_peered":"2024-08-02T17:44:59.937146+0000","last_clean":"2024-08-02T17:44:59.937146+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:44:59.937146+0000","last_undegraded":"2024-08-02T17:44:59.937146+0000","last_fullsized":"2024-08-02T17:44:59.937146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934625+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:44:57.934625+0000","last_peered":"2024-08-02T17:44:57.934625+0000","last_clean":"2024-08-02T17:44:57.934625+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:44:57.934625+0000","last_undegraded":"2024-08-02T17:44:57.934625+0000","last_fullsized":"2024-08-02T17:44:57.934625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934293+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:44:57.934293+0000","last_peered":"2024-08-02T17:44:57.934293+0000","last_clean":"2024-08-02T17:44:57.934293+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:44:57.934293+0000","last_undegraded":"2024-08-02T17:44:57.934293+0000","last_fullsized":"2024-08-02T17:44:57.934293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.062270+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:44:58.062270+0000","last_peered":"2024-08-02T17:44:58.062270+0000","last_clean":"2024-08-02T17:44:58.062270+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:44:58.062270+0000","last_undegraded":"2024-08-02T17:44:58.062270+0000","last_fullsized":"2024-08-02T17:44:58.062270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780849+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:44:57.780849+0000","last_peered":"2024-08-02T17:44:57.780849+0000","last_clean":"2024-08-02T17:44:57.780849+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:44:57.780849+0000","last_undegraded":"2024-08-02T17:44:57.780849+0000","last_fullsized":"2024-08-02T17:44:57.780849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778368+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:44:57.778368+0000","last_peered":"2024-08-02T17:44:57.778368+0000","last_clean":"2024-08-02T17:44:57.778368+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:44:57.778368+0000","last_undegraded":"2024-08-02T17:44:57.778368+0000","last_fullsized":"2024-08-02T17:44:57.778368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778623+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:44:57.778623+0000","last_peered":"2024-08-02T17:44:57.778623+0000","last_clean":"2024-08-02T17:44:57.778623+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:44:57.778623+0000","last_undegraded":"2024-08-02T17:44:57.778623+0000","last_fullsized":"2024-08-02T17:44:57.778623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778590+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:44:57.778590+0000","last_peered":"2024-08-02T17:44:57.778590+0000","last_clean":"2024-08-02T17:44:57.778590+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:44:57.778590+0000","last_undegraded":"2024-08-02T17:44:57.778590+0000","last_fullsized":"2024-08-02T17:44:57.778590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.793617+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:44:57.793617+0000","last_peered":"2024-08-02T17:44:57.793617+0000","last_clean":"2024-08-02T17:44:57.793617+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:44:57.793617+0000","last_undegraded":"2024-08-02T17:44:57.793617+0000","last_fullsized":"2024-08-02T17:44:57.793617+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778382+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:44:57.778382+0000","last_peered":"2024-08-02T17:44:57.778382+0000","last_clean":"2024-08-02T17:44:57.778382+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:44:57.778382+0000","last_undegraded":"2024-08-02T17:44:57.778382+0000","last_fullsized":"2024-08-02T17:44:57.778382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784666+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:44:57.784666+0000","last_peered":"2024-08-02T17:44:57.784666+0000","last_clean":"2024-08-02T17:44:57.784666+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:44:57.784666+0000","last_undegraded":"2024-08-02T17:44:57.784666+0000","last_fullsized":"2024-08-02T17:44:57.784666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058379+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:44:58.058379+0000","last_peered":"2024-08-02T17:44:58.058379+0000","last_clean":"2024-08-02T17:44:58.058379+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:44:58.058379+0000","last_undegraded":"2024-08-02T17:44:58.058379+0000","last_fullsized":"2024-08-02T17:44:58.058379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777756+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:44:57.777756+0000","last_peered":"2024-08-02T17:44:57.777756+0000","last_clean":"2024-08-02T17:44:57.777756+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:44:57.777756+0000","last_undegraded":"2024-08-02T17:44:57.777756+0000","last_fullsized":"2024-08-02T17:44:57.777756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780624+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:44:57.780624+0000","last_peered":"2024-08-02T17:44:57.780624+0000","last_clean":"2024-08-02T17:44:57.780624+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:44:57.780624+0000","last_undegraded":"2024-08-02T17:44:57.780624+0000","last_fullsized":"2024-08-02T17:44:57.780624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788602+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:44:57.788602+0000","last_peered":"2024-08-02T17:44:57.788602+0000","last_clean":"2024-08-02T17:44:57.788602+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:44:57.788602+0000","last_undegraded":"2024-08-02T17:44:57.788602+0000","last_fullsized":"2024-08-02T17:44:57.788602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778360+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:44:57.778360+0000","last_peered":"2024-08-02T17:44:57.778360+0000","last_clean":"2024-08-02T17:44:57.778360+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:44:57.778360+0000","last_undegraded":"2024-08-02T17:44:57.778360+0000","last_fullsized":"2024-08-02T17:44:57.778360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934751+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:44:57.934751+0000","last_peered":"2024-08-02T17:44:57.934751+0000","last_clean":"2024-08-02T17:44:57.934751+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:44:57.934751+0000","last_undegraded":"2024-08-02T17:44:57.934751+0000","last_fullsized":"2024-08-02T17:44:57.934751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778111+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:44:57.778111+0000","last_peered":"2024-08-02T17:44:57.778111+0000","last_clean":"2024-08-02T17:44:57.778111+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:44:57.778111+0000","last_undegraded":"2024-08-02T17:44:57.778111+0000","last_fullsized":"2024-08-02T17:44:57.778111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778966+0000","last_change":"2024-08-02T17:44:04.131281+0000","last_active":"2024-08-02T17:44:57.778966+0000","last_peered":"2024-08-02T17:44:57.778966+0000","last_clean":"2024-08-02T17:44:57.778966+0000","last_became_active":"2024-08-02T17:44:04.130938+0000","last_became_peered":"2024-08-02T17:44:04.130938+0000","last_unstale":"2024-08-02T17:44:57.778966+0000","last_undegraded":"2024-08-02T17:44:57.778966+0000","last_fullsized":"2024-08-02T17:44:57.778966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:19:53.292216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.785999+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:44:57.785999+0000","last_peered":"2024-08-02T17:44:57.785999+0000","last_clean":"2024-08-02T17:44:57.785999+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:44:57.785999+0000","last_undegraded":"2024-08-02T17:44:57.785999+0000","last_fullsized":"2024-08-02T17:44:57.785999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779421+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:44:57.779421+0000","last_peered":"2024-08-02T17:44:57.779421+0000","last_clean":"2024-08-02T17:44:57.779421+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:44:57.779421+0000","last_undegraded":"2024-08-02T17:44:57.779421+0000","last_fullsized":"2024-08-02T17:44:57.779421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779500+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:44:57.779500+0000","last_peered":"2024-08-02T17:44:57.779500+0000","last_clean":"2024-08-02T17:44:57.779500+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:44:57.779500+0000","last_undegraded":"2024-08-02T17:44:57.779500+0000","last_fullsized":"2024-08-02T17:44:57.779500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":74,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.800995+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:44:57.800995+0000","last_peered":"2024-08-02T17:44:57.800995+0000","last_clean":"2024-08-02T17:44:57.800995+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:44:57.800995+0000","last_undegraded":"2024-08-02T17:44:57.800995+0000","last_fullsized":"2024-08-02T17:44:57.800995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934674+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:44:57.934674+0000","last_peered":"2024-08-02T17:44:57.934674+0000","last_clean":"2024-08-02T17:44:57.934674+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:44:57.934674+0000","last_undegraded":"2024-08-02T17:44:57.934674+0000","last_fullsized":"2024-08-02T17:44:57.934674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777470+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:44:57.777470+0000","last_peered":"2024-08-02T17:44:57.777470+0000","last_clean":"2024-08-02T17:44:57.777470+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:44:57.777470+0000","last_undegraded":"2024-08-02T17:44:57.777470+0000","last_fullsized":"2024-08-02T17:44:57.777470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777985+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:44:57.777985+0000","last_peered":"2024-08-02T17:44:57.777985+0000","last_clean":"2024-08-02T17:44:57.777985+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:44:57.777985+0000","last_undegraded":"2024-08-02T17:44:57.777985+0000","last_fullsized":"2024-08-02T17:44:57.777985+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775973+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:44:57.775973+0000","last_peered":"2024-08-02T17:44:57.775973+0000","last_clean":"2024-08-02T17:44:57.775973+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:44:57.775973+0000","last_undegraded":"2024-08-02T17:44:57.775973+0000","last_fullsized":"2024-08-02T17:44:57.775973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777872+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:44:57.777872+0000","last_peered":"2024-08-02T17:44:57.777872+0000","last_clean":"2024-08-02T17:44:57.777872+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:44:57.777872+0000","last_undegraded":"2024-08-02T17:44:57.777872+0000","last_fullsized":"2024-08-02T17:44:57.777872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779850+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:44:57.779850+0000","last_peered":"2024-08-02T17:44:57.779850+0000","last_clean":"2024-08-02T17:44:57.779850+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:44:57.779850+0000","last_undegraded":"2024-08-02T17:44:57.779850+0000","last_fullsized":"2024-08-02T17:44:57.779850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935930+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:44:57.935930+0000","last_peered":"2024-08-02T17:44:57.935930+0000","last_clean":"2024-08-02T17:44:57.935930+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:44:57.935930+0000","last_undegraded":"2024-08-02T17:44:57.935930+0000","last_fullsized":"2024-08-02T17:44:57.935930+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934481+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:44:57.934481+0000","last_peered":"2024-08-02T17:44:57.934481+0000","last_clean":"2024-08-02T17:44:57.934481+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:44:57.934481+0000","last_undegraded":"2024-08-02T17:44:57.934481+0000","last_fullsized":"2024-08-02T17:44:57.934481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784275+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:44:57.784275+0000","last_peered":"2024-08-02T17:44:57.784275+0000","last_clean":"2024-08-02T17:44:57.784275+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:44:57.784275+0000","last_undegraded":"2024-08-02T17:44:57.784275+0000","last_fullsized":"2024-08-02T17:44:57.784275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780202+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:44:57.780202+0000","last_peered":"2024-08-02T17:44:57.780202+0000","last_clean":"2024-08-02T17:44:57.780202+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:44:57.780202+0000","last_undegraded":"2024-08-02T17:44:57.780202+0000","last_fullsized":"2024-08-02T17:44:57.780202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783877+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:44:57.783877+0000","last_peered":"2024-08-02T17:44:57.783877+0000","last_clean":"2024-08-02T17:44:57.783877+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:44:57.783877+0000","last_undegraded":"2024-08-02T17:44:57.783877+0000","last_fullsized":"2024-08-02T17:44:57.783877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'29","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:01.939524+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:45:01.939524+0000","last_peered":"2024-08-02T17:45:01.939524+0000","last_clean":"2024-08-02T17:45:01.939524+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:45:01.939524+0000","last_undegraded":"2024-08-02T17:45:01.939524+0000","last_fullsized":"2024-08-02T17:45:01.939524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52407,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":69,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.785527+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:44:57.785527+0000","last_peered":"2024-08-02T17:44:57.785527+0000","last_clean":"2024-08-02T17:44:57.785527+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:44:57.785527+0000","last_undegraded":"2024-08-02T17:44:57.785527+0000","last_fullsized":"2024-08-02T17:44:57.785527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058665+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:44:58.058665+0000","last_peered":"2024-08-02T17:44:58.058665+0000","last_clean":"2024-08-02T17:44:58.058665+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:44:58.058665+0000","last_undegraded":"2024-08-02T17:44:58.058665+0000","last_fullsized":"2024-08-02T17:44:58.058665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780514+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:44:57.780514+0000","last_peered":"2024-08-02T17:44:57.780514+0000","last_clean":"2024-08-02T17:44:57.780514+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:44:57.780514+0000","last_undegraded":"2024-08-02T17:44:57.780514+0000","last_fullsized":"2024-08-02T17:44:57.780514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778403+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:44:57.778403+0000","last_peered":"2024-08-02T17:44:57.778403+0000","last_clean":"2024-08-02T17:44:57.778403+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:44:57.778403+0000","last_undegraded":"2024-08-02T17:44:57.778403+0000","last_fullsized":"2024-08-02T17:44:57.778403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788772+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:44:57.788772+0000","last_peered":"2024-08-02T17:44:57.788772+0000","last_clean":"2024-08-02T17:44:57.788772+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:44:57.788772+0000","last_undegraded":"2024-08-02T17:44:57.788772+0000","last_fullsized":"2024-08-02T17:44:57.788772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778895+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:44:57.778895+0000","last_peered":"2024-08-02T17:44:57.778895+0000","last_clean":"2024-08-02T17:44:57.778895+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:44:57.778895+0000","last_undegraded":"2024-08-02T17:44:57.778895+0000","last_fullsized":"2024-08-02T17:44:57.778895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775322+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:44:57.775322+0000","last_peered":"2024-08-02T17:44:57.775322+0000","last_clean":"2024-08-02T17:44:57.775322+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:44:57.775322+0000","last_undegraded":"2024-08-02T17:44:57.775322+0000","last_fullsized":"2024-08-02T17:44:57.775322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058972+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:44:58.058972+0000","last_peered":"2024-08-02T17:44:58.058972+0000","last_clean":"2024-08-02T17:44:58.058972+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:44:58.058972+0000","last_undegraded":"2024-08-02T17:44:58.058972+0000","last_fullsized":"2024-08-02T17:44:58.058972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:00.937851+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:45:00.937851+0000","last_peered":"2024-08-02T17:45:00.937851+0000","last_clean":"2024-08-02T17:45:00.937851+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:45:00.937851+0000","last_undegraded":"2024-08-02T17:45:00.937851+0000","last_fullsized":"2024-08-02T17:45:00.937851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778076+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:44:57.778076+0000","last_peered":"2024-08-02T17:44:57.778076+0000","last_clean":"2024-08-02T17:44:57.778076+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:44:57.778076+0000","last_undegraded":"2024-08-02T17:44:57.778076+0000","last_fullsized":"2024-08-02T17:44:57.778076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788071+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:44:57.788071+0000","last_peered":"2024-08-02T17:44:57.788071+0000","last_clean":"2024-08-02T17:44:57.788071+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:44:57.788071+0000","last_undegraded":"2024-08-02T17:44:57.788071+0000","last_fullsized":"2024-08-02T17:44:57.788071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775451+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:44:57.775451+0000","last_peered":"2024-08-02T17:44:57.775451+0000","last_clean":"2024-08-02T17:44:57.775451+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:44:57.775451+0000","last_undegraded":"2024-08-02T17:44:57.775451+0000","last_fullsized":"2024-08-02T17:44:57.775451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779319+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:44:57.779319+0000","last_peered":"2024-08-02T17:44:57.779319+0000","last_clean":"2024-08-02T17:44:57.779319+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:44:57.779319+0000","last_undegraded":"2024-08-02T17:44:57.779319+0000","last_fullsized":"2024-08-02T17:44:57.779319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778983+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:44:57.778983+0000","last_peered":"2024-08-02T17:44:57.778983+0000","last_clean":"2024-08-02T17:44:57.778983+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:44:57.778983+0000","last_undegraded":"2024-08-02T17:44:57.778983+0000","last_fullsized":"2024-08-02T17:44:57.778983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777876+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:44:57.777876+0000","last_peered":"2024-08-02T17:44:57.777876+0000","last_clean":"2024-08-02T17:44:57.777876+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:44:57.777876+0000","last_undegraded":"2024-08-02T17:44:57.777876+0000","last_fullsized":"2024-08-02T17:44:57.777876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934332+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:44:57.934332+0000","last_peered":"2024-08-02T17:44:57.934332+0000","last_clean":"2024-08-02T17:44:57.934332+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:44:57.934332+0000","last_undegraded":"2024-08-02T17:44:57.934332+0000","last_fullsized":"2024-08-02T17:44:57.934332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778975+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:44:57.778975+0000","last_peered":"2024-08-02T17:44:57.778975+0000","last_clean":"2024-08-02T17:44:57.778975+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:44:57.778975+0000","last_undegraded":"2024-08-02T17:44:57.778975+0000","last_fullsized":"2024-08-02T17:44:57.778975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780193+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:44:57.780193+0000","last_peered":"2024-08-02T17:44:57.780193+0000","last_clean":"2024-08-02T17:44:57.780193+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:44:57.780193+0000","last_undegraded":"2024-08-02T17:44:57.780193+0000","last_fullsized":"2024-08-02T17:44:57.780193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784123+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:44:57.784123+0000","last_peered":"2024-08-02T17:44:57.784123+0000","last_clean":"2024-08-02T17:44:57.784123+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:44:57.784123+0000","last_undegraded":"2024-08-02T17:44:57.784123+0000","last_fullsized":"2024-08-02T17:44:57.784123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780351+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:44:57.780351+0000","last_peered":"2024-08-02T17:44:57.780351+0000","last_clean":"2024-08-02T17:44:57.780351+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:44:57.780351+0000","last_undegraded":"2024-08-02T17:44:57.780351+0000","last_fullsized":"2024-08-02T17:44:57.780351+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784723+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:44:57.784723+0000","last_peered":"2024-08-02T17:44:57.784723+0000","last_clean":"2024-08-02T17:44:57.784723+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:44:57.784723+0000","last_undegraded":"2024-08-02T17:44:57.784723+0000","last_fullsized":"2024-08-02T17:44:57.784723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784337+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:44:57.784337+0000","last_peered":"2024-08-02T17:44:57.784337+0000","last_clean":"2024-08-02T17:44:57.784337+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:44:57.784337+0000","last_undegraded":"2024-08-02T17:44:57.784337+0000","last_fullsized":"2024-08-02T17:44:57.784337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778261+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:44:57.778261+0000","last_peered":"2024-08-02T17:44:57.778261+0000","last_clean":"2024-08-02T17:44:57.778261+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:44:57.778261+0000","last_undegraded":"2024-08-02T17:44:57.778261+0000","last_fullsized":"2024-08-02T17:44:57.778261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784229+0000","last_change":"2024-08-02T17:44:04.127893+0000","last_active":"2024-08-02T17:44:57.784229+0000","last_peered":"2024-08-02T17:44:57.784229+0000","last_clean":"2024-08-02T17:44:57.784229+0000","last_became_active":"2024-08-02T17:44:04.127520+0000","last_became_peered":"2024-08-02T17:44:04.127520+0000","last_unstale":"2024-08-02T17:44:57.784229+0000","last_undegraded":"2024-08-02T17:44:57.784229+0000","last_fullsized":"2024-08-02T17:44:57.784229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:53.670606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782482+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:44:57.782482+0000","last_peered":"2024-08-02T17:44:57.782482+0000","last_clean":"2024-08-02T17:44:57.782482+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:44:57.782482+0000","last_undegraded":"2024-08-02T17:44:57.782482+0000","last_fullsized":"2024-08-02T17:44:57.782482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788689+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:44:57.788689+0000","last_peered":"2024-08-02T17:44:57.788689+0000","last_clean":"2024-08-02T17:44:57.788689+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:44:57.788689+0000","last_undegraded":"2024-08-02T17:44:57.788689+0000","last_fullsized":"2024-08-02T17:44:57.788689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935062+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:44:57.935062+0000","last_peered":"2024-08-02T17:44:57.935062+0000","last_clean":"2024-08-02T17:44:57.935062+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:44:57.935062+0000","last_undegraded":"2024-08-02T17:44:57.935062+0000","last_fullsized":"2024-08-02T17:44:57.935062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778967+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:44:57.778967+0000","last_peered":"2024-08-02T17:44:57.778967+0000","last_clean":"2024-08-02T17:44:57.778967+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:44:57.778967+0000","last_undegraded":"2024-08-02T17:44:57.778967+0000","last_fullsized":"2024-08-02T17:44:57.778967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782955+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:44:57.782955+0000","last_peered":"2024-08-02T17:44:57.782955+0000","last_clean":"2024-08-02T17:44:57.782955+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:44:57.782955+0000","last_undegraded":"2024-08-02T17:44:57.782955+0000","last_fullsized":"2024-08-02T17:44:57.782955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788682+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:44:57.788682+0000","last_peered":"2024-08-02T17:44:57.788682+0000","last_clean":"2024-08-02T17:44:57.788682+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:44:57.788682+0000","last_undegraded":"2024-08-02T17:44:57.788682+0000","last_fullsized":"2024-08-02T17:44:57.788682+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779156+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:44:57.779156+0000","last_peered":"2024-08-02T17:44:57.779156+0000","last_clean":"2024-08-02T17:44:57.779156+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:44:57.779156+0000","last_undegraded":"2024-08-02T17:44:57.779156+0000","last_fullsized":"2024-08-02T17:44:57.779156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778167+0000","last_change":"2024-08-02T17:44:04.124213+0000","last_active":"2024-08-02T17:44:57.778167+0000","last_peered":"2024-08-02T17:44:57.778167+0000","last_clean":"2024-08-02T17:44:57.778167+0000","last_became_active":"2024-08-02T17:44:04.123913+0000","last_became_peered":"2024-08-02T17:44:04.123913+0000","last_unstale":"2024-08-02T17:44:57.778167+0000","last_undegraded":"2024-08-02T17:44:57.778167+0000","last_fullsized":"2024-08-02T17:44:57.778167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:32:05.880054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058515+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:44:58.058515+0000","last_peered":"2024-08-02T17:44:58.058515+0000","last_clean":"2024-08-02T17:44:58.058515+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:44:58.058515+0000","last_undegraded":"2024-08-02T17:44:58.058515+0000","last_fullsized":"2024-08-02T17:44:58.058515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934630+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:44:57.934630+0000","last_peered":"2024-08-02T17:44:57.934630+0000","last_clean":"2024-08-02T17:44:57.934630+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:44:57.934630+0000","last_undegraded":"2024-08-02T17:44:57.934630+0000","last_fullsized":"2024-08-02T17:44:57.934630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778481+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:44:57.778481+0000","last_peered":"2024-08-02T17:44:57.778481+0000","last_clean":"2024-08-02T17:44:57.778481+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:44:57.778481+0000","last_undegraded":"2024-08-02T17:44:57.778481+0000","last_fullsized":"2024-08-02T17:44:57.778481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784986+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:44:57.784986+0000","last_peered":"2024-08-02T17:44:57.784986+0000","last_clean":"2024-08-02T17:44:57.784986+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:44:57.784986+0000","last_undegraded":"2024-08-02T17:44:57.784986+0000","last_fullsized":"2024-08-02T17:44:57.784986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.776049+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:44:57.776049+0000","last_peered":"2024-08-02T17:44:57.776049+0000","last_clean":"2024-08-02T17:44:57.776049+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:44:57.776049+0000","last_undegraded":"2024-08-02T17:44:57.776049+0000","last_fullsized":"2024-08-02T17:44:57.776049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779187+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:44:57.779187+0000","last_peered":"2024-08-02T17:44:57.779187+0000","last_clean":"2024-08-02T17:44:57.779187+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:44:57.779187+0000","last_undegraded":"2024-08-02T17:44:57.779187+0000","last_fullsized":"2024-08-02T17:44:57.779187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058678+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:44:58.058678+0000","last_peered":"2024-08-02T17:44:58.058678+0000","last_clean":"2024-08-02T17:44:58.058678+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:44:58.058678+0000","last_undegraded":"2024-08-02T17:44:58.058678+0000","last_fullsized":"2024-08-02T17:44:58.058678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788147+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:44:57.788147+0000","last_peered":"2024-08-02T17:44:57.788147+0000","last_clean":"2024-08-02T17:44:57.788147+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:44:57.788147+0000","last_undegraded":"2024-08-02T17:44:57.788147+0000","last_fullsized":"2024-08-02T17:44:57.788147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788368+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:44:57.788368+0000","last_peered":"2024-08-02T17:44:57.788368+0000","last_clean":"2024-08-02T17:44:57.788368+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:44:57.788368+0000","last_undegraded":"2024-08-02T17:44:57.788368+0000","last_fullsized":"2024-08-02T17:44:57.788368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934609+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:44:57.934609+0000","last_peered":"2024-08-02T17:44:57.934609+0000","last_clean":"2024-08-02T17:44:57.934609+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:44:57.934609+0000","last_undegraded":"2024-08-02T17:44:57.934609+0000","last_fullsized":"2024-08-02T17:44:57.934609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779201+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:44:57.779201+0000","last_peered":"2024-08-02T17:44:57.779201+0000","last_clean":"2024-08-02T17:44:57.779201+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:44:57.779201+0000","last_undegraded":"2024-08-02T17:44:57.779201+0000","last_fullsized":"2024-08-02T17:44:57.779201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778292+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:44:57.778292+0000","last_peered":"2024-08-02T17:44:57.778292+0000","last_clean":"2024-08-02T17:44:57.778292+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:44:57.778292+0000","last_undegraded":"2024-08-02T17:44:57.778292+0000","last_fullsized":"2024-08-02T17:44:57.778292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.786339+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:44:57.786339+0000","last_peered":"2024-08-02T17:44:57.786339+0000","last_clean":"2024-08-02T17:44:57.786339+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:44:57.786339+0000","last_undegraded":"2024-08-02T17:44:57.786339+0000","last_fullsized":"2024-08-02T17:44:57.786339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.059142+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:44:58.059142+0000","last_peered":"2024-08-02T17:44:58.059142+0000","last_clean":"2024-08-02T17:44:58.059142+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:44:58.059142+0000","last_undegraded":"2024-08-02T17:44:58.059142+0000","last_fullsized":"2024-08-02T17:44:58.059142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777697+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:44:57.777697+0000","last_peered":"2024-08-02T17:44:57.777697+0000","last_clean":"2024-08-02T17:44:57.777697+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:44:57.777697+0000","last_undegraded":"2024-08-02T17:44:57.777697+0000","last_fullsized":"2024-08-02T17:44:57.777697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778075+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:44:57.778075+0000","last_peered":"2024-08-02T17:44:57.778075+0000","last_clean":"2024-08-02T17:44:57.778075+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:44:57.778075+0000","last_undegraded":"2024-08-02T17:44:57.778075+0000","last_fullsized":"2024-08-02T17:44:57.778075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778851+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:44:57.778851+0000","last_peered":"2024-08-02T17:44:57.778851+0000","last_clean":"2024-08-02T17:44:57.778851+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:44:57.778851+0000","last_undegraded":"2024-08-02T17:44:57.778851+0000","last_fullsized":"2024-08-02T17:44:57.778851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783119+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:44:57.783119+0000","last_peered":"2024-08-02T17:44:57.783119+0000","last_clean":"2024-08-02T17:44:57.783119+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:44:57.783119+0000","last_undegraded":"2024-08-02T17:44:57.783119+0000","last_fullsized":"2024-08-02T17:44:57.783119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779098+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:44:57.779098+0000","last_peered":"2024-08-02T17:44:57.779098+0000","last_clean":"2024-08-02T17:44:57.779098+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:44:57.779098+0000","last_undegraded":"2024-08-02T17:44:57.779098+0000","last_fullsized":"2024-08-02T17:44:57.779098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788028+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:44:57.788028+0000","last_peered":"2024-08-02T17:44:57.788028+0000","last_clean":"2024-08-02T17:44:57.788028+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:44:57.788028+0000","last_undegraded":"2024-08-02T17:44:57.788028+0000","last_fullsized":"2024-08-02T17:44:57.788028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788302+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:44:57.788302+0000","last_peered":"2024-08-02T17:44:57.788302+0000","last_clean":"2024-08-02T17:44:57.788302+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:44:57.788302+0000","last_undegraded":"2024-08-02T17:44:57.788302+0000","last_fullsized":"2024-08-02T17:44:57.788302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779160+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:44:57.779160+0000","last_peered":"2024-08-02T17:44:57.779160+0000","last_clean":"2024-08-02T17:44:57.779160+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:44:57.779160+0000","last_undegraded":"2024-08-02T17:44:57.779160+0000","last_fullsized":"2024-08-02T17:44:57.779160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56797,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":107,"num_read_kb":118,"num_write":158,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":152255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449112,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":515190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678044,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37028,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708124,"statfs":{"total":95995035648,"available":95957118976,"internally_reserved":0,"allocated":1724416,"data_stored":1155255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841569,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":519522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103205,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":519442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364841,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1111644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790002,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712348,"statfs":{"total":95995035648,"available":95961444352,"internally_reserved":0,"allocated":1658880,"data_stored":1102746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953526,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":506294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":510796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1105920,"data_stored":555479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":553851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":502642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":44967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:45:04.666 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph pg dump --format=json 2024-08-02T17:45:04.923 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:05.045 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:04 smithi069 ceph-mon[32746]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:05.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:04 smithi077 ceph-mon[37108]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:05.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:04 smithi083 ceph-mon[37031]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 -- 172.21.15.69:0/3501099564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d0102250 msgr2=0x7fe8d010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 --2- 172.21.15.69:0/3501099564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d0102250 0x7fe8d010e780 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe8c8009cd0 tx=0x7fe8c802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 -- 172.21.15.69:0/3501099564 shutdown_connections 2024-08-02T17:45:05.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 --2- 172.21.15.69:0/3501099564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d0102250 0x7fe8d010e780 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 --2- 172.21.15.69:0/3501099564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d01018b0 0x7fe8d0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 --2- 172.21.15.69:0/3501099564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d01078b0 0x7fe8d0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 -- 172.21.15.69:0/3501099564 >> 172.21.15.69:0/3501099564 conn(0x7fe8d00fd5f0 msgr2=0x7fe8d00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 -- 172.21.15.69:0/3501099564 shutdown_connections 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.343+0000 7fe8d5825640 1 -- 172.21.15.69:0/3501099564 wait complete. 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8d5825640 1 Processor -- start 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8d5825640 1 -- start start 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8d5825640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d01018b0 0x7fe8d0074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8d5825640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0102250 0x7fe8d0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8d5825640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d01078b0 0x7fe8d0074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8d5825640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d0111cd0 con 0x7fe8d01018b0 2024-08-02T17:45:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8d5825640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d0111b50 con 0x7fe8d0102250 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8d5825640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d0111e50 con 0x7fe8d01078b0 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d01018b0 0x7fe8d0074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d01018b0 0x7fe8d0074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37816/0 (socket says 172.21.15.69:37816) 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.344+0000 7fe8ceffd640 1 -- 172.21.15.69:0/1630005636 learned_addr learned my addr 172.21.15.69:0/1630005636 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8ce7fc640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0102250 0x7fe8d0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8cf7fe640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d01078b0 0x7fe8d0074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8ceffd640 1 -- 172.21.15.69:0/1630005636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d01078b0 msgr2=0x7fe8d0074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8ceffd640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d01078b0 0x7fe8d0074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8ceffd640 1 -- 172.21.15.69:0/1630005636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0102250 msgr2=0x7fe8d0074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8ceffd640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0102250 0x7fe8d0074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8ceffd640 1 -- 172.21.15.69:0/1630005636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d01a6690 con 0x7fe8d01018b0 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8cf7fe640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d01078b0 0x7fe8d0074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:05.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8ceffd640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d01018b0 0x7fe8d0074220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b4a0 tx=0x7fe8b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8d4823640 1 -- 172.21.15.69:0/1630005636 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8014070 con 0x7fe8d01018b0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8d4823640 1 -- 172.21.15.69:0/1630005636 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b8010af0 con 0x7fe8d01018b0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d01a6980 con 0x7fe8d01018b0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8d4823640 1 -- 172.21.15.69:0/1630005636 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8017d10 con 0x7fe8d01018b0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.345+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d01a6e70 con 0x7fe8d01018b0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.346+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8d0101d10 con 0x7fe8d01018b0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.347+0000 7fe8d4823640 1 -- 172.21.15.69:0/1630005636 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe8b8017070 con 0x7fe8d01018b0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.348+0000 7fe8d4823640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8bc0a2c10 0x7fe8bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.348+0000 7fe8ce7fc640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8bc0a2c10 0x7fe8bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.348+0000 7fe8d4823640 1 -- 172.21.15.69:0/1630005636 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe8b80ce8a0 con 0x7fe8d01018b0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.349+0000 7fe8ce7fc640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8bc0a2c10 0x7fe8bc0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0075cb0 tx=0x7fe8c000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.352+0000 7fe8d4823640 1 -- 172.21.15.69:0/1630005636 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe8b809b450 con 0x7fe8d01018b0 2024-08-02T17:45:05.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.516+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8d0062360 con 0x7fe8bc0a2c10 2024-08-02T17:45:05.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.523+0000 7fe8d4823640 1 -- 172.21.15.69:0/1630005636 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340418 (secure 0 0 0) 0x7fe8d0062360 con 0x7fe8bc0a2c10 2024-08-02T17:45:05.529 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:05.530 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8bc0a2c10 msgr2=0x7fe8bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8bc0a2c10 0x7fe8bc0a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0075cb0 tx=0x7fe8c000a4c0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d01018b0 msgr2=0x7fe8d0074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d01018b0 0x7fe8d0074220 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800b4a0 tx=0x7fe8b800b960 comp rx=0 tx=0).stop 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 shutdown_connections 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8bc0a2c10 0x7fe8bc0a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe8d01078b0 0x7fe8d0074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe8d0102250 0x7fe8d0074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 --2- 172.21.15.69:0/1630005636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe8d01018b0 0x7fe8d0074220 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.526+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 >> 172.21.15.69:0/1630005636 conn(0x7fe8d00fd5f0 msgr2=0x7fe8d00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.527+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 shutdown_connections 2024-08-02T17:45:05.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:05.527+0000 7fe8d5825640 1 -- 172.21.15.69:0/1630005636 wait complete. 2024-08-02T17:45:06.073 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:05 smithi069 ceph-mon[32746]: pgmap v296: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.1 KiB/s wr, 9 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:06.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:05 smithi077 ceph-mon[37108]: pgmap v296: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.1 KiB/s wr, 9 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:06.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:05 smithi083 ceph-mon[37031]: pgmap v296: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 6.1 KiB/s wr, 9 op/s; 28 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:06.342 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-08-02T17:45:04.770699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647399,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":253,"num_read_kb":237,"num_write":341,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392656,"kb_used_data":14300,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549168,"statfs":{"total":1151940427776,"available":1151538348032,"internally_reserved":0,"allocated":14643200,"data_stored":8100407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42084,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":59,"num_read_kb":67,"num_write":20,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":229,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002677"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.776089+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:44:57.776089+0000","last_peered":"2024-08-02T17:44:57.776089+0000","last_clean":"2024-08-02T17:44:57.776089+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:44:57.776089+0000","last_undegraded":"2024-08-02T17:44:57.776089+0000","last_fullsized":"2024-08-02T17:44:57.776089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778005+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:44:57.778005+0000","last_peered":"2024-08-02T17:44:57.778005+0000","last_clean":"2024-08-02T17:44:57.778005+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:44:57.778005+0000","last_undegraded":"2024-08-02T17:44:57.778005+0000","last_fullsized":"2024-08-02T17:44:57.778005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782153+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:44:57.782153+0000","last_peered":"2024-08-02T17:44:57.782153+0000","last_clean":"2024-08-02T17:44:57.782153+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:44:57.782153+0000","last_undegraded":"2024-08-02T17:44:57.782153+0000","last_fullsized":"2024-08-02T17:44:57.782153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778542+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:44:57.778542+0000","last_peered":"2024-08-02T17:44:57.778542+0000","last_clean":"2024-08-02T17:44:57.778542+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:44:57.778542+0000","last_undegraded":"2024-08-02T17:44:57.778542+0000","last_fullsized":"2024-08-02T17:44:57.778542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788699+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:44:57.788699+0000","last_peered":"2024-08-02T17:44:57.788699+0000","last_clean":"2024-08-02T17:44:57.788699+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:44:57.788699+0000","last_undegraded":"2024-08-02T17:44:57.788699+0000","last_fullsized":"2024-08-02T17:44:57.788699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777204+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:44:57.777204+0000","last_peered":"2024-08-02T17:44:57.777204+0000","last_clean":"2024-08-02T17:44:57.777204+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:44:57.777204+0000","last_undegraded":"2024-08-02T17:44:57.777204+0000","last_fullsized":"2024-08-02T17:44:57.777204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784045+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:44:57.784045+0000","last_peered":"2024-08-02T17:44:57.784045+0000","last_clean":"2024-08-02T17:44:57.784045+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:44:57.784045+0000","last_undegraded":"2024-08-02T17:44:57.784045+0000","last_fullsized":"2024-08-02T17:44:57.784045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058658+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:44:58.058658+0000","last_peered":"2024-08-02T17:44:58.058658+0000","last_clean":"2024-08-02T17:44:58.058658+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:44:58.058658+0000","last_undegraded":"2024-08-02T17:44:58.058658+0000","last_fullsized":"2024-08-02T17:44:58.058658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778197+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:44:57.778197+0000","last_peered":"2024-08-02T17:44:57.778197+0000","last_clean":"2024-08-02T17:44:57.778197+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:44:57.778197+0000","last_undegraded":"2024-08-02T17:44:57.778197+0000","last_fullsized":"2024-08-02T17:44:57.778197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783541+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:44:57.783541+0000","last_peered":"2024-08-02T17:44:57.783541+0000","last_clean":"2024-08-02T17:44:57.783541+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:44:57.783541+0000","last_undegraded":"2024-08-02T17:44:57.783541+0000","last_fullsized":"2024-08-02T17:44:57.783541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780343+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:44:57.780343+0000","last_peered":"2024-08-02T17:44:57.780343+0000","last_clean":"2024-08-02T17:44:57.780343+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:44:57.780343+0000","last_undegraded":"2024-08-02T17:44:57.780343+0000","last_fullsized":"2024-08-02T17:44:57.780343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777417+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:44:57.777417+0000","last_peered":"2024-08-02T17:44:57.777417+0000","last_clean":"2024-08-02T17:44:57.777417+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:44:57.777417+0000","last_undegraded":"2024-08-02T17:44:57.777417+0000","last_fullsized":"2024-08-02T17:44:57.777417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777532+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:44:57.777532+0000","last_peered":"2024-08-02T17:44:57.777532+0000","last_clean":"2024-08-02T17:44:57.777532+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:44:57.777532+0000","last_undegraded":"2024-08-02T17:44:57.777532+0000","last_fullsized":"2024-08-02T17:44:57.777532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783178+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:44:57.783178+0000","last_peered":"2024-08-02T17:44:57.783178+0000","last_clean":"2024-08-02T17:44:57.783178+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:44:57.783178+0000","last_undegraded":"2024-08-02T17:44:57.783178+0000","last_fullsized":"2024-08-02T17:44:57.783178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778786+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:44:57.778786+0000","last_peered":"2024-08-02T17:44:57.778786+0000","last_clean":"2024-08-02T17:44:57.778786+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:44:57.778786+0000","last_undegraded":"2024-08-02T17:44:57.778786+0000","last_fullsized":"2024-08-02T17:44:57.778786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778669+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:44:57.778669+0000","last_peered":"2024-08-02T17:44:57.778669+0000","last_clean":"2024-08-02T17:44:57.778669+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:44:57.778669+0000","last_undegraded":"2024-08-02T17:44:57.778669+0000","last_fullsized":"2024-08-02T17:44:57.778669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.793088+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:44:57.793088+0000","last_peered":"2024-08-02T17:44:57.793088+0000","last_clean":"2024-08-02T17:44:57.793088+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:44:57.793088+0000","last_undegraded":"2024-08-02T17:44:57.793088+0000","last_fullsized":"2024-08-02T17:44:57.793088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777673+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:44:57.777673+0000","last_peered":"2024-08-02T17:44:57.777673+0000","last_clean":"2024-08-02T17:44:57.777673+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:44:57.777673+0000","last_undegraded":"2024-08-02T17:44:57.777673+0000","last_fullsized":"2024-08-02T17:44:57.777673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778351+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:44:57.778351+0000","last_peered":"2024-08-02T17:44:57.778351+0000","last_clean":"2024-08-02T17:44:57.778351+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:44:57.778351+0000","last_undegraded":"2024-08-02T17:44:57.778351+0000","last_fullsized":"2024-08-02T17:44:57.778351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778275+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:44:57.778275+0000","last_peered":"2024-08-02T17:44:57.778275+0000","last_clean":"2024-08-02T17:44:57.778275+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:44:57.778275+0000","last_undegraded":"2024-08-02T17:44:57.778275+0000","last_fullsized":"2024-08-02T17:44:57.778275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775801+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:44:57.775801+0000","last_peered":"2024-08-02T17:44:57.775801+0000","last_clean":"2024-08-02T17:44:57.775801+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:44:57.775801+0000","last_undegraded":"2024-08-02T17:44:57.775801+0000","last_fullsized":"2024-08-02T17:44:57.775801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783178+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:44:57.783178+0000","last_peered":"2024-08-02T17:44:57.783178+0000","last_clean":"2024-08-02T17:44:57.783178+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:44:57.783178+0000","last_undegraded":"2024-08-02T17:44:57.783178+0000","last_fullsized":"2024-08-02T17:44:57.783178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058742+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:44:58.058742+0000","last_peered":"2024-08-02T17:44:58.058742+0000","last_clean":"2024-08-02T17:44:58.058742+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:44:58.058742+0000","last_undegraded":"2024-08-02T17:44:58.058742+0000","last_fullsized":"2024-08-02T17:44:58.058742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.789019+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:44:57.789019+0000","last_peered":"2024-08-02T17:44:57.789019+0000","last_clean":"2024-08-02T17:44:57.789019+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:44:57.789019+0000","last_undegraded":"2024-08-02T17:44:57.789019+0000","last_fullsized":"2024-08-02T17:44:57.789019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788382+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:44:57.788382+0000","last_peered":"2024-08-02T17:44:57.788382+0000","last_clean":"2024-08-02T17:44:57.788382+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:44:57.788382+0000","last_undegraded":"2024-08-02T17:44:57.788382+0000","last_fullsized":"2024-08-02T17:44:57.788382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779411+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:44:57.779411+0000","last_peered":"2024-08-02T17:44:57.779411+0000","last_clean":"2024-08-02T17:44:57.779411+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:44:57.779411+0000","last_undegraded":"2024-08-02T17:44:57.779411+0000","last_fullsized":"2024-08-02T17:44:57.779411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784040+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:44:57.784040+0000","last_peered":"2024-08-02T17:44:57.784040+0000","last_clean":"2024-08-02T17:44:57.784040+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:44:57.784040+0000","last_undegraded":"2024-08-02T17:44:57.784040+0000","last_fullsized":"2024-08-02T17:44:57.784040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778702+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:44:57.778702+0000","last_peered":"2024-08-02T17:44:57.778702+0000","last_clean":"2024-08-02T17:44:57.778702+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:44:57.778702+0000","last_undegraded":"2024-08-02T17:44:57.778702+0000","last_fullsized":"2024-08-02T17:44:57.778702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.790105+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:44:57.790105+0000","last_peered":"2024-08-02T17:44:57.790105+0000","last_clean":"2024-08-02T17:44:57.790105+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:44:57.790105+0000","last_undegraded":"2024-08-02T17:44:57.790105+0000","last_fullsized":"2024-08-02T17:44:57.790105+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777623+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:44:57.777623+0000","last_peered":"2024-08-02T17:44:57.777623+0000","last_clean":"2024-08-02T17:44:57.777623+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:44:57.777623+0000","last_undegraded":"2024-08-02T17:44:57.777623+0000","last_fullsized":"2024-08-02T17:44:57.777623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935126+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:44:57.935126+0000","last_peered":"2024-08-02T17:44:57.935126+0000","last_clean":"2024-08-02T17:44:57.935126+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:44:57.935126+0000","last_undegraded":"2024-08-02T17:44:57.935126+0000","last_fullsized":"2024-08-02T17:44:57.935126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784152+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:44:57.784152+0000","last_peered":"2024-08-02T17:44:57.784152+0000","last_clean":"2024-08-02T17:44:57.784152+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:44:57.784152+0000","last_undegraded":"2024-08-02T17:44:57.784152+0000","last_fullsized":"2024-08-02T17:44:57.784152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778014+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:44:57.778014+0000","last_peered":"2024-08-02T17:44:57.778014+0000","last_clean":"2024-08-02T17:44:57.778014+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:44:57.778014+0000","last_undegraded":"2024-08-02T17:44:57.778014+0000","last_fullsized":"2024-08-02T17:44:57.778014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779077+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:44:57.779077+0000","last_peered":"2024-08-02T17:44:57.779077+0000","last_clean":"2024-08-02T17:44:57.779077+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:44:57.779077+0000","last_undegraded":"2024-08-02T17:44:57.779077+0000","last_fullsized":"2024-08-02T17:44:57.779077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775849+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:44:57.775849+0000","last_peered":"2024-08-02T17:44:57.775849+0000","last_clean":"2024-08-02T17:44:57.775849+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:44:57.775849+0000","last_undegraded":"2024-08-02T17:44:57.775849+0000","last_fullsized":"2024-08-02T17:44:57.775849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784607+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:44:57.784607+0000","last_peered":"2024-08-02T17:44:57.784607+0000","last_clean":"2024-08-02T17:44:57.784607+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:44:57.784607+0000","last_undegraded":"2024-08-02T17:44:57.784607+0000","last_fullsized":"2024-08-02T17:44:57.784607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779247+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:44:57.779247+0000","last_peered":"2024-08-02T17:44:57.779247+0000","last_clean":"2024-08-02T17:44:57.779247+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:44:57.779247+0000","last_undegraded":"2024-08-02T17:44:57.779247+0000","last_fullsized":"2024-08-02T17:44:57.779247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777693+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:44:57.777693+0000","last_peered":"2024-08-02T17:44:57.777693+0000","last_clean":"2024-08-02T17:44:57.777693+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:44:57.777693+0000","last_undegraded":"2024-08-02T17:44:57.777693+0000","last_fullsized":"2024-08-02T17:44:57.777693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934899+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:44:57.934899+0000","last_peered":"2024-08-02T17:44:57.934899+0000","last_clean":"2024-08-02T17:44:57.934899+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:44:57.934899+0000","last_undegraded":"2024-08-02T17:44:57.934899+0000","last_fullsized":"2024-08-02T17:44:57.934899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779198+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:44:57.779198+0000","last_peered":"2024-08-02T17:44:57.779198+0000","last_clean":"2024-08-02T17:44:57.779198+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:44:57.779198+0000","last_undegraded":"2024-08-02T17:44:57.779198+0000","last_fullsized":"2024-08-02T17:44:57.779198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:59.937146+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:44:59.937146+0000","last_peered":"2024-08-02T17:44:59.937146+0000","last_clean":"2024-08-02T17:44:59.937146+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:44:59.937146+0000","last_undegraded":"2024-08-02T17:44:59.937146+0000","last_fullsized":"2024-08-02T17:44:59.937146+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934625+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:44:57.934625+0000","last_peered":"2024-08-02T17:44:57.934625+0000","last_clean":"2024-08-02T17:44:57.934625+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:44:57.934625+0000","last_undegraded":"2024-08-02T17:44:57.934625+0000","last_fullsized":"2024-08-02T17:44:57.934625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934293+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:44:57.934293+0000","last_peered":"2024-08-02T17:44:57.934293+0000","last_clean":"2024-08-02T17:44:57.934293+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:44:57.934293+0000","last_undegraded":"2024-08-02T17:44:57.934293+0000","last_fullsized":"2024-08-02T17:44:57.934293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.062270+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:44:58.062270+0000","last_peered":"2024-08-02T17:44:58.062270+0000","last_clean":"2024-08-02T17:44:58.062270+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:44:58.062270+0000","last_undegraded":"2024-08-02T17:44:58.062270+0000","last_fullsized":"2024-08-02T17:44:58.062270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780849+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:44:57.780849+0000","last_peered":"2024-08-02T17:44:57.780849+0000","last_clean":"2024-08-02T17:44:57.780849+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:44:57.780849+0000","last_undegraded":"2024-08-02T17:44:57.780849+0000","last_fullsized":"2024-08-02T17:44:57.780849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778368+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:44:57.778368+0000","last_peered":"2024-08-02T17:44:57.778368+0000","last_clean":"2024-08-02T17:44:57.778368+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:44:57.778368+0000","last_undegraded":"2024-08-02T17:44:57.778368+0000","last_fullsized":"2024-08-02T17:44:57.778368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778623+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:44:57.778623+0000","last_peered":"2024-08-02T17:44:57.778623+0000","last_clean":"2024-08-02T17:44:57.778623+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:44:57.778623+0000","last_undegraded":"2024-08-02T17:44:57.778623+0000","last_fullsized":"2024-08-02T17:44:57.778623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778590+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:44:57.778590+0000","last_peered":"2024-08-02T17:44:57.778590+0000","last_clean":"2024-08-02T17:44:57.778590+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:44:57.778590+0000","last_undegraded":"2024-08-02T17:44:57.778590+0000","last_fullsized":"2024-08-02T17:44:57.778590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.793617+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:44:57.793617+0000","last_peered":"2024-08-02T17:44:57.793617+0000","last_clean":"2024-08-02T17:44:57.793617+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:44:57.793617+0000","last_undegraded":"2024-08-02T17:44:57.793617+0000","last_fullsized":"2024-08-02T17:44:57.793617+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778382+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:44:57.778382+0000","last_peered":"2024-08-02T17:44:57.778382+0000","last_clean":"2024-08-02T17:44:57.778382+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:44:57.778382+0000","last_undegraded":"2024-08-02T17:44:57.778382+0000","last_fullsized":"2024-08-02T17:44:57.778382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784666+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:44:57.784666+0000","last_peered":"2024-08-02T17:44:57.784666+0000","last_clean":"2024-08-02T17:44:57.784666+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:44:57.784666+0000","last_undegraded":"2024-08-02T17:44:57.784666+0000","last_fullsized":"2024-08-02T17:44:57.784666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058379+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:44:58.058379+0000","last_peered":"2024-08-02T17:44:58.058379+0000","last_clean":"2024-08-02T17:44:58.058379+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:44:58.058379+0000","last_undegraded":"2024-08-02T17:44:58.058379+0000","last_fullsized":"2024-08-02T17:44:58.058379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777756+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:44:57.777756+0000","last_peered":"2024-08-02T17:44:57.777756+0000","last_clean":"2024-08-02T17:44:57.777756+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:44:57.777756+0000","last_undegraded":"2024-08-02T17:44:57.777756+0000","last_fullsized":"2024-08-02T17:44:57.777756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780624+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:44:57.780624+0000","last_peered":"2024-08-02T17:44:57.780624+0000","last_clean":"2024-08-02T17:44:57.780624+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:44:57.780624+0000","last_undegraded":"2024-08-02T17:44:57.780624+0000","last_fullsized":"2024-08-02T17:44:57.780624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788602+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:44:57.788602+0000","last_peered":"2024-08-02T17:44:57.788602+0000","last_clean":"2024-08-02T17:44:57.788602+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:44:57.788602+0000","last_undegraded":"2024-08-02T17:44:57.788602+0000","last_fullsized":"2024-08-02T17:44:57.788602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778360+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:44:57.778360+0000","last_peered":"2024-08-02T17:44:57.778360+0000","last_clean":"2024-08-02T17:44:57.778360+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:44:57.778360+0000","last_undegraded":"2024-08-02T17:44:57.778360+0000","last_fullsized":"2024-08-02T17:44:57.778360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934751+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:44:57.934751+0000","last_peered":"2024-08-02T17:44:57.934751+0000","last_clean":"2024-08-02T17:44:57.934751+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:44:57.934751+0000","last_undegraded":"2024-08-02T17:44:57.934751+0000","last_fullsized":"2024-08-02T17:44:57.934751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778111+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:44:57.778111+0000","last_peered":"2024-08-02T17:44:57.778111+0000","last_clean":"2024-08-02T17:44:57.778111+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:44:57.778111+0000","last_undegraded":"2024-08-02T17:44:57.778111+0000","last_fullsized":"2024-08-02T17:44:57.778111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778966+0000","last_change":"2024-08-02T17:44:04.131281+0000","last_active":"2024-08-02T17:44:57.778966+0000","last_peered":"2024-08-02T17:44:57.778966+0000","last_clean":"2024-08-02T17:44:57.778966+0000","last_became_active":"2024-08-02T17:44:04.130938+0000","last_became_peered":"2024-08-02T17:44:04.130938+0000","last_unstale":"2024-08-02T17:44:57.778966+0000","last_undegraded":"2024-08-02T17:44:57.778966+0000","last_fullsized":"2024-08-02T17:44:57.778966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:19:53.292216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.785999+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:44:57.785999+0000","last_peered":"2024-08-02T17:44:57.785999+0000","last_clean":"2024-08-02T17:44:57.785999+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:44:57.785999+0000","last_undegraded":"2024-08-02T17:44:57.785999+0000","last_fullsized":"2024-08-02T17:44:57.785999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779421+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:44:57.779421+0000","last_peered":"2024-08-02T17:44:57.779421+0000","last_clean":"2024-08-02T17:44:57.779421+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:44:57.779421+0000","last_undegraded":"2024-08-02T17:44:57.779421+0000","last_fullsized":"2024-08-02T17:44:57.779421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779500+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:44:57.779500+0000","last_peered":"2024-08-02T17:44:57.779500+0000","last_clean":"2024-08-02T17:44:57.779500+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:44:57.779500+0000","last_undegraded":"2024-08-02T17:44:57.779500+0000","last_fullsized":"2024-08-02T17:44:57.779500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":74,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.800995+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:44:57.800995+0000","last_peered":"2024-08-02T17:44:57.800995+0000","last_clean":"2024-08-02T17:44:57.800995+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:44:57.800995+0000","last_undegraded":"2024-08-02T17:44:57.800995+0000","last_fullsized":"2024-08-02T17:44:57.800995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934674+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:44:57.934674+0000","last_peered":"2024-08-02T17:44:57.934674+0000","last_clean":"2024-08-02T17:44:57.934674+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:44:57.934674+0000","last_undegraded":"2024-08-02T17:44:57.934674+0000","last_fullsized":"2024-08-02T17:44:57.934674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777470+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:44:57.777470+0000","last_peered":"2024-08-02T17:44:57.777470+0000","last_clean":"2024-08-02T17:44:57.777470+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:44:57.777470+0000","last_undegraded":"2024-08-02T17:44:57.777470+0000","last_fullsized":"2024-08-02T17:44:57.777470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:00.938823+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:45:00.938823+0000","last_peered":"2024-08-02T17:45:00.938823+0000","last_clean":"2024-08-02T17:45:00.938823+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:45:00.938823+0000","last_undegraded":"2024-08-02T17:45:00.938823+0000","last_fullsized":"2024-08-02T17:45:00.938823+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775973+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:44:57.775973+0000","last_peered":"2024-08-02T17:44:57.775973+0000","last_clean":"2024-08-02T17:44:57.775973+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:44:57.775973+0000","last_undegraded":"2024-08-02T17:44:57.775973+0000","last_fullsized":"2024-08-02T17:44:57.775973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777872+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:44:57.777872+0000","last_peered":"2024-08-02T17:44:57.777872+0000","last_clean":"2024-08-02T17:44:57.777872+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:44:57.777872+0000","last_undegraded":"2024-08-02T17:44:57.777872+0000","last_fullsized":"2024-08-02T17:44:57.777872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779850+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:44:57.779850+0000","last_peered":"2024-08-02T17:44:57.779850+0000","last_clean":"2024-08-02T17:44:57.779850+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:44:57.779850+0000","last_undegraded":"2024-08-02T17:44:57.779850+0000","last_fullsized":"2024-08-02T17:44:57.779850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935930+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:44:57.935930+0000","last_peered":"2024-08-02T17:44:57.935930+0000","last_clean":"2024-08-02T17:44:57.935930+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:44:57.935930+0000","last_undegraded":"2024-08-02T17:44:57.935930+0000","last_fullsized":"2024-08-02T17:44:57.935930+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934481+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:44:57.934481+0000","last_peered":"2024-08-02T17:44:57.934481+0000","last_clean":"2024-08-02T17:44:57.934481+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:44:57.934481+0000","last_undegraded":"2024-08-02T17:44:57.934481+0000","last_fullsized":"2024-08-02T17:44:57.934481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784275+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:44:57.784275+0000","last_peered":"2024-08-02T17:44:57.784275+0000","last_clean":"2024-08-02T17:44:57.784275+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:44:57.784275+0000","last_undegraded":"2024-08-02T17:44:57.784275+0000","last_fullsized":"2024-08-02T17:44:57.784275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780202+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:44:57.780202+0000","last_peered":"2024-08-02T17:44:57.780202+0000","last_clean":"2024-08-02T17:44:57.780202+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:44:57.780202+0000","last_undegraded":"2024-08-02T17:44:57.780202+0000","last_fullsized":"2024-08-02T17:44:57.780202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783877+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:44:57.783877+0000","last_peered":"2024-08-02T17:44:57.783877+0000","last_clean":"2024-08-02T17:44:57.783877+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:44:57.783877+0000","last_undegraded":"2024-08-02T17:44:57.783877+0000","last_fullsized":"2024-08-02T17:44:57.783877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'29","reported_seq":100,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:01.939524+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:45:01.939524+0000","last_peered":"2024-08-02T17:45:01.939524+0000","last_clean":"2024-08-02T17:45:01.939524+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:45:01.939524+0000","last_undegraded":"2024-08-02T17:45:01.939524+0000","last_fullsized":"2024-08-02T17:45:01.939524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52407,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":37,"num_read_kb":69,"num_write":30,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.785527+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:44:57.785527+0000","last_peered":"2024-08-02T17:44:57.785527+0000","last_clean":"2024-08-02T17:44:57.785527+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:44:57.785527+0000","last_undegraded":"2024-08-02T17:44:57.785527+0000","last_fullsized":"2024-08-02T17:44:57.785527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058665+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:44:58.058665+0000","last_peered":"2024-08-02T17:44:58.058665+0000","last_clean":"2024-08-02T17:44:58.058665+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:44:58.058665+0000","last_undegraded":"2024-08-02T17:44:58.058665+0000","last_fullsized":"2024-08-02T17:44:58.058665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780514+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:44:57.780514+0000","last_peered":"2024-08-02T17:44:57.780514+0000","last_clean":"2024-08-02T17:44:57.780514+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:44:57.780514+0000","last_undegraded":"2024-08-02T17:44:57.780514+0000","last_fullsized":"2024-08-02T17:44:57.780514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778403+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:44:57.778403+0000","last_peered":"2024-08-02T17:44:57.778403+0000","last_clean":"2024-08-02T17:44:57.778403+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:44:57.778403+0000","last_undegraded":"2024-08-02T17:44:57.778403+0000","last_fullsized":"2024-08-02T17:44:57.778403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788772+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:44:57.788772+0000","last_peered":"2024-08-02T17:44:57.788772+0000","last_clean":"2024-08-02T17:44:57.788772+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:44:57.788772+0000","last_undegraded":"2024-08-02T17:44:57.788772+0000","last_fullsized":"2024-08-02T17:44:57.788772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778895+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:44:57.778895+0000","last_peered":"2024-08-02T17:44:57.778895+0000","last_clean":"2024-08-02T17:44:57.778895+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:44:57.778895+0000","last_undegraded":"2024-08-02T17:44:57.778895+0000","last_fullsized":"2024-08-02T17:44:57.778895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775322+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:44:57.775322+0000","last_peered":"2024-08-02T17:44:57.775322+0000","last_clean":"2024-08-02T17:44:57.775322+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:44:57.775322+0000","last_undegraded":"2024-08-02T17:44:57.775322+0000","last_fullsized":"2024-08-02T17:44:57.775322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058972+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:44:58.058972+0000","last_peered":"2024-08-02T17:44:58.058972+0000","last_clean":"2024-08-02T17:44:58.058972+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:44:58.058972+0000","last_undegraded":"2024-08-02T17:44:58.058972+0000","last_fullsized":"2024-08-02T17:44:58.058972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:00.937851+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:45:00.937851+0000","last_peered":"2024-08-02T17:45:00.937851+0000","last_clean":"2024-08-02T17:45:00.937851+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:45:00.937851+0000","last_undegraded":"2024-08-02T17:45:00.937851+0000","last_fullsized":"2024-08-02T17:45:00.937851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778076+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:44:57.778076+0000","last_peered":"2024-08-02T17:44:57.778076+0000","last_clean":"2024-08-02T17:44:57.778076+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:44:57.778076+0000","last_undegraded":"2024-08-02T17:44:57.778076+0000","last_fullsized":"2024-08-02T17:44:57.778076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788071+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:44:57.788071+0000","last_peered":"2024-08-02T17:44:57.788071+0000","last_clean":"2024-08-02T17:44:57.788071+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:44:57.788071+0000","last_undegraded":"2024-08-02T17:44:57.788071+0000","last_fullsized":"2024-08-02T17:44:57.788071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775451+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:44:57.775451+0000","last_peered":"2024-08-02T17:44:57.775451+0000","last_clean":"2024-08-02T17:44:57.775451+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:44:57.775451+0000","last_undegraded":"2024-08-02T17:44:57.775451+0000","last_fullsized":"2024-08-02T17:44:57.775451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779319+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:44:57.779319+0000","last_peered":"2024-08-02T17:44:57.779319+0000","last_clean":"2024-08-02T17:44:57.779319+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:44:57.779319+0000","last_undegraded":"2024-08-02T17:44:57.779319+0000","last_fullsized":"2024-08-02T17:44:57.779319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778983+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:44:57.778983+0000","last_peered":"2024-08-02T17:44:57.778983+0000","last_clean":"2024-08-02T17:44:57.778983+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:44:57.778983+0000","last_undegraded":"2024-08-02T17:44:57.778983+0000","last_fullsized":"2024-08-02T17:44:57.778983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777876+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:44:57.777876+0000","last_peered":"2024-08-02T17:44:57.777876+0000","last_clean":"2024-08-02T17:44:57.777876+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:44:57.777876+0000","last_undegraded":"2024-08-02T17:44:57.777876+0000","last_fullsized":"2024-08-02T17:44:57.777876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934332+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:44:57.934332+0000","last_peered":"2024-08-02T17:44:57.934332+0000","last_clean":"2024-08-02T17:44:57.934332+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:44:57.934332+0000","last_undegraded":"2024-08-02T17:44:57.934332+0000","last_fullsized":"2024-08-02T17:44:57.934332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778975+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:44:57.778975+0000","last_peered":"2024-08-02T17:44:57.778975+0000","last_clean":"2024-08-02T17:44:57.778975+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:44:57.778975+0000","last_undegraded":"2024-08-02T17:44:57.778975+0000","last_fullsized":"2024-08-02T17:44:57.778975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780193+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:44:57.780193+0000","last_peered":"2024-08-02T17:44:57.780193+0000","last_clean":"2024-08-02T17:44:57.780193+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:44:57.780193+0000","last_undegraded":"2024-08-02T17:44:57.780193+0000","last_fullsized":"2024-08-02T17:44:57.780193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784123+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:44:57.784123+0000","last_peered":"2024-08-02T17:44:57.784123+0000","last_clean":"2024-08-02T17:44:57.784123+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:44:57.784123+0000","last_undegraded":"2024-08-02T17:44:57.784123+0000","last_fullsized":"2024-08-02T17:44:57.784123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:01.939841+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:45:01.939841+0000","last_peered":"2024-08-02T17:45:01.939841+0000","last_clean":"2024-08-02T17:45:01.939841+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:45:01.939841+0000","last_undegraded":"2024-08-02T17:45:01.939841+0000","last_fullsized":"2024-08-02T17:45:01.939841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784723+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:44:57.784723+0000","last_peered":"2024-08-02T17:44:57.784723+0000","last_clean":"2024-08-02T17:44:57.784723+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:44:57.784723+0000","last_undegraded":"2024-08-02T17:44:57.784723+0000","last_fullsized":"2024-08-02T17:44:57.784723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784337+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:44:57.784337+0000","last_peered":"2024-08-02T17:44:57.784337+0000","last_clean":"2024-08-02T17:44:57.784337+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:44:57.784337+0000","last_undegraded":"2024-08-02T17:44:57.784337+0000","last_fullsized":"2024-08-02T17:44:57.784337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778261+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:44:57.778261+0000","last_peered":"2024-08-02T17:44:57.778261+0000","last_clean":"2024-08-02T17:44:57.778261+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:44:57.778261+0000","last_undegraded":"2024-08-02T17:44:57.778261+0000","last_fullsized":"2024-08-02T17:44:57.778261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784229+0000","last_change":"2024-08-02T17:44:04.127893+0000","last_active":"2024-08-02T17:44:57.784229+0000","last_peered":"2024-08-02T17:44:57.784229+0000","last_clean":"2024-08-02T17:44:57.784229+0000","last_became_active":"2024-08-02T17:44:04.127520+0000","last_became_peered":"2024-08-02T17:44:04.127520+0000","last_unstale":"2024-08-02T17:44:57.784229+0000","last_undegraded":"2024-08-02T17:44:57.784229+0000","last_fullsized":"2024-08-02T17:44:57.784229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:53.670606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782482+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:44:57.782482+0000","last_peered":"2024-08-02T17:44:57.782482+0000","last_clean":"2024-08-02T17:44:57.782482+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:44:57.782482+0000","last_undegraded":"2024-08-02T17:44:57.782482+0000","last_fullsized":"2024-08-02T17:44:57.782482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788689+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:44:57.788689+0000","last_peered":"2024-08-02T17:44:57.788689+0000","last_clean":"2024-08-02T17:44:57.788689+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:44:57.788689+0000","last_undegraded":"2024-08-02T17:44:57.788689+0000","last_fullsized":"2024-08-02T17:44:57.788689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935062+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:44:57.935062+0000","last_peered":"2024-08-02T17:44:57.935062+0000","last_clean":"2024-08-02T17:44:57.935062+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:44:57.935062+0000","last_undegraded":"2024-08-02T17:44:57.935062+0000","last_fullsized":"2024-08-02T17:44:57.935062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778967+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:44:57.778967+0000","last_peered":"2024-08-02T17:44:57.778967+0000","last_clean":"2024-08-02T17:44:57.778967+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:44:57.778967+0000","last_undegraded":"2024-08-02T17:44:57.778967+0000","last_fullsized":"2024-08-02T17:44:57.778967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782955+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:44:57.782955+0000","last_peered":"2024-08-02T17:44:57.782955+0000","last_clean":"2024-08-02T17:44:57.782955+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:44:57.782955+0000","last_undegraded":"2024-08-02T17:44:57.782955+0000","last_fullsized":"2024-08-02T17:44:57.782955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788682+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:44:57.788682+0000","last_peered":"2024-08-02T17:44:57.788682+0000","last_clean":"2024-08-02T17:44:57.788682+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:44:57.788682+0000","last_undegraded":"2024-08-02T17:44:57.788682+0000","last_fullsized":"2024-08-02T17:44:57.788682+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779156+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:44:57.779156+0000","last_peered":"2024-08-02T17:44:57.779156+0000","last_clean":"2024-08-02T17:44:57.779156+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:44:57.779156+0000","last_undegraded":"2024-08-02T17:44:57.779156+0000","last_fullsized":"2024-08-02T17:44:57.779156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778167+0000","last_change":"2024-08-02T17:44:04.124213+0000","last_active":"2024-08-02T17:44:57.778167+0000","last_peered":"2024-08-02T17:44:57.778167+0000","last_clean":"2024-08-02T17:44:57.778167+0000","last_became_active":"2024-08-02T17:44:04.123913+0000","last_became_peered":"2024-08-02T17:44:04.123913+0000","last_unstale":"2024-08-02T17:44:57.778167+0000","last_undegraded":"2024-08-02T17:44:57.778167+0000","last_fullsized":"2024-08-02T17:44:57.778167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:32:05.880054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058515+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:44:58.058515+0000","last_peered":"2024-08-02T17:44:58.058515+0000","last_clean":"2024-08-02T17:44:58.058515+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:44:58.058515+0000","last_undegraded":"2024-08-02T17:44:58.058515+0000","last_fullsized":"2024-08-02T17:44:58.058515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934630+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:44:57.934630+0000","last_peered":"2024-08-02T17:44:57.934630+0000","last_clean":"2024-08-02T17:44:57.934630+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:44:57.934630+0000","last_undegraded":"2024-08-02T17:44:57.934630+0000","last_fullsized":"2024-08-02T17:44:57.934630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778481+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:44:57.778481+0000","last_peered":"2024-08-02T17:44:57.778481+0000","last_clean":"2024-08-02T17:44:57.778481+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:44:57.778481+0000","last_undegraded":"2024-08-02T17:44:57.778481+0000","last_fullsized":"2024-08-02T17:44:57.778481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784986+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:44:57.784986+0000","last_peered":"2024-08-02T17:44:57.784986+0000","last_clean":"2024-08-02T17:44:57.784986+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:44:57.784986+0000","last_undegraded":"2024-08-02T17:44:57.784986+0000","last_fullsized":"2024-08-02T17:44:57.784986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.776049+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:44:57.776049+0000","last_peered":"2024-08-02T17:44:57.776049+0000","last_clean":"2024-08-02T17:44:57.776049+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:44:57.776049+0000","last_undegraded":"2024-08-02T17:44:57.776049+0000","last_fullsized":"2024-08-02T17:44:57.776049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779187+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:44:57.779187+0000","last_peered":"2024-08-02T17:44:57.779187+0000","last_clean":"2024-08-02T17:44:57.779187+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:44:57.779187+0000","last_undegraded":"2024-08-02T17:44:57.779187+0000","last_fullsized":"2024-08-02T17:44:57.779187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058678+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:44:58.058678+0000","last_peered":"2024-08-02T17:44:58.058678+0000","last_clean":"2024-08-02T17:44:58.058678+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:44:58.058678+0000","last_undegraded":"2024-08-02T17:44:58.058678+0000","last_fullsized":"2024-08-02T17:44:58.058678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788147+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:44:57.788147+0000","last_peered":"2024-08-02T17:44:57.788147+0000","last_clean":"2024-08-02T17:44:57.788147+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:44:57.788147+0000","last_undegraded":"2024-08-02T17:44:57.788147+0000","last_fullsized":"2024-08-02T17:44:57.788147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788368+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:44:57.788368+0000","last_peered":"2024-08-02T17:44:57.788368+0000","last_clean":"2024-08-02T17:44:57.788368+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:44:57.788368+0000","last_undegraded":"2024-08-02T17:44:57.788368+0000","last_fullsized":"2024-08-02T17:44:57.788368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934609+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:44:57.934609+0000","last_peered":"2024-08-02T17:44:57.934609+0000","last_clean":"2024-08-02T17:44:57.934609+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:44:57.934609+0000","last_undegraded":"2024-08-02T17:44:57.934609+0000","last_fullsized":"2024-08-02T17:44:57.934609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779201+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:44:57.779201+0000","last_peered":"2024-08-02T17:44:57.779201+0000","last_clean":"2024-08-02T17:44:57.779201+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:44:57.779201+0000","last_undegraded":"2024-08-02T17:44:57.779201+0000","last_fullsized":"2024-08-02T17:44:57.779201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778292+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:44:57.778292+0000","last_peered":"2024-08-02T17:44:57.778292+0000","last_clean":"2024-08-02T17:44:57.778292+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:44:57.778292+0000","last_undegraded":"2024-08-02T17:44:57.778292+0000","last_fullsized":"2024-08-02T17:44:57.778292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.786339+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:44:57.786339+0000","last_peered":"2024-08-02T17:44:57.786339+0000","last_clean":"2024-08-02T17:44:57.786339+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:44:57.786339+0000","last_undegraded":"2024-08-02T17:44:57.786339+0000","last_fullsized":"2024-08-02T17:44:57.786339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.059142+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:44:58.059142+0000","last_peered":"2024-08-02T17:44:58.059142+0000","last_clean":"2024-08-02T17:44:58.059142+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:44:58.059142+0000","last_undegraded":"2024-08-02T17:44:58.059142+0000","last_fullsized":"2024-08-02T17:44:58.059142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777697+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:44:57.777697+0000","last_peered":"2024-08-02T17:44:57.777697+0000","last_clean":"2024-08-02T17:44:57.777697+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:44:57.777697+0000","last_undegraded":"2024-08-02T17:44:57.777697+0000","last_fullsized":"2024-08-02T17:44:57.777697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778075+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:44:57.778075+0000","last_peered":"2024-08-02T17:44:57.778075+0000","last_clean":"2024-08-02T17:44:57.778075+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:44:57.778075+0000","last_undegraded":"2024-08-02T17:44:57.778075+0000","last_fullsized":"2024-08-02T17:44:57.778075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778851+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:44:57.778851+0000","last_peered":"2024-08-02T17:44:57.778851+0000","last_clean":"2024-08-02T17:44:57.778851+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:44:57.778851+0000","last_undegraded":"2024-08-02T17:44:57.778851+0000","last_fullsized":"2024-08-02T17:44:57.778851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783119+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:44:57.783119+0000","last_peered":"2024-08-02T17:44:57.783119+0000","last_clean":"2024-08-02T17:44:57.783119+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:44:57.783119+0000","last_undegraded":"2024-08-02T17:44:57.783119+0000","last_fullsized":"2024-08-02T17:44:57.783119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779098+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:44:57.779098+0000","last_peered":"2024-08-02T17:44:57.779098+0000","last_clean":"2024-08-02T17:44:57.779098+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:44:57.779098+0000","last_undegraded":"2024-08-02T17:44:57.779098+0000","last_fullsized":"2024-08-02T17:44:57.779098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788028+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:44:57.788028+0000","last_peered":"2024-08-02T17:44:57.788028+0000","last_clean":"2024-08-02T17:44:57.788028+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:44:57.788028+0000","last_undegraded":"2024-08-02T17:44:57.788028+0000","last_fullsized":"2024-08-02T17:44:57.788028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788302+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:44:57.788302+0000","last_peered":"2024-08-02T17:44:57.788302+0000","last_clean":"2024-08-02T17:44:57.788302+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:44:57.788302+0000","last_undegraded":"2024-08-02T17:44:57.788302+0000","last_fullsized":"2024-08-02T17:44:57.788302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779160+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:44:57.779160+0000","last_peered":"2024-08-02T17:44:57.779160+0000","last_clean":"2024-08-02T17:44:57.779160+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:44:57.779160+0000","last_undegraded":"2024-08-02T17:44:57.779160+0000","last_fullsized":"2024-08-02T17:44:57.779160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56797,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":117,"num_read_kb":118,"num_write":158,"num_write_kb":104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":170391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449112,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":515190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678044,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37028,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708124,"statfs":{"total":95995035648,"available":95957118976,"internally_reserved":0,"allocated":1724416,"data_stored":1155255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841569,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":519522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103205,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":519442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364842,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1111644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790003,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712348,"statfs":{"total":95995035648,"available":95961444352,"internally_reserved":0,"allocated":1658880,"data_stored":1102746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215163,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":510796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36432,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708720,"statfs":{"total":95995035648,"available":95957729280,"internally_reserved":0,"allocated":1114112,"data_stored":564547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":562919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":54035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:45:06.347 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-02T17:45:06.348 INFO:tasks.cephadm:Setting up client nodes... 2024-08-02T17:45:06.348 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-02T17:45:06.607 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:06.886 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:06 smithi069 ceph-mon[32746]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:07.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:06 smithi077 ceph-mon[37108]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:07.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 -- 172.21.15.69:0/543363284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 msgr2=0x7fc77c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:07.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 --2- 172.21.15.69:0/543363284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 0x7fc77c100c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009900 tx=0x7fc76c02a410 comp rx=0 tx=0).stop 2024-08-02T17:45:07.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 -- 172.21.15.69:0/543363284 shutdown_connections 2024-08-02T17:45:07.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 --2- 172.21.15.69:0/543363284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c1023c0 0x7fc77c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 --2- 172.21.15.69:0/543363284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc77c101a00 0x7fc77c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 --2- 172.21.15.69:0/543363284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 0x7fc77c100c00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 -- 172.21.15.69:0/543363284 >> 172.21.15.69:0/543363284 conn(0x7fc77c0fbfb0 msgr2=0x7fc77c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 -- 172.21.15.69:0/543363284 shutdown_connections 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 -- 172.21.15.69:0/543363284 wait complete. 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 Processor -- start 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.115+0000 7fc781c25640 1 -- start start 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc781c25640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 0x7fc77c06a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc781c25640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc77c101a00 0x7fc77c19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc781c25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c1023c0 0x7fc77c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc781c25640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c10c460 con 0x7fc77c100800 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc781c25640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c10c2e0 con 0x7fc77c101a00 2024-08-02T17:45:07.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc781c25640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c10c5e0 con 0x7fc77c1023c0 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 0x7fc77c06a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 0x7fc77c06a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37836/0 (socket says 172.21.15.69:37836) 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 -- 172.21.15.69:0/972599976 learned_addr learned my addr 172.21.15.69:0/972599976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc773fff640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc77c101a00 0x7fc77c19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77bfff640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c1023c0 0x7fc77c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 -- 172.21.15.69:0/972599976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c1023c0 msgr2=0x7fc77c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c1023c0 0x7fc77c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 -- 172.21.15.69:0/972599976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc77c101a00 msgr2=0x7fc77c19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc77c101a00 0x7fc77c19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 -- 172.21.15.69:0/972599976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c19f910 con 0x7fc77c100800 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc773fff640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc77c101a00 0x7fc77c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:45:07.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc77b7fe640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 0x7fc77c06a800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc76c031610 tx=0x7fc76c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:07.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc7797fa640 1 -- 172.21.15.69:0/972599976 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c03f030 con 0x7fc77c100800 2024-08-02T17:45:07.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c19fb40 con 0x7fc77c100800 2024-08-02T17:45:07.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.116+0000 7fc7797fa640 1 -- 172.21.15.69:0/972599976 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc76c003ad0 con 0x7fc77c100800 2024-08-02T17:45:07.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.117+0000 7fc7797fa640 1 -- 172.21.15.69:0/972599976 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c035070 con 0x7fc77c100800 2024-08-02T17:45:07.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.117+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c1a0020 con 0x7fc77c100800 2024-08-02T17:45:07.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.118+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77c107ca0 con 0x7fc77c100800 2024-08-02T17:45:07.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.119+0000 7fc7797fa640 1 -- 172.21.15.69:0/972599976 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc76c042030 con 0x7fc77c100800 2024-08-02T17:45:07.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.119+0000 7fc7797fa640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7680a2c10 0x7fc7680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:07.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.119+0000 7fc773fff640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7680a2c10 0x7fc7680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:07.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.119+0000 7fc7797fa640 1 -- 172.21.15.69:0/972599976 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fc76c0efbb0 con 0x7fc77c100800 2024-08-02T17:45:07.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.120+0000 7fc773fff640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7680a2c10 0x7fc7680a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc77c06ad80 tx=0x7fc764005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:07.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.123+0000 7fc7797fa640 1 -- 172.21.15.69:0/972599976 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc76c0bc000 con 0x7fc77c100800 2024-08-02T17:45:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:06 smithi083 ceph-mon[37031]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:07.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.356+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fc77c062b50 con 0x7fc77c100800 2024-08-02T17:45:07.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.358+0000 7fc7797fa640 1 -- 172.21.15.69:0/972599976 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) ==== 170+0+59 (secure 0 0 0) 0x7fc76c0c0300 con 0x7fc77c100800 2024-08-02T17:45:07.359 INFO:teuthology.orchestra.run.smithi069.stdout:[client.0] 2024-08-02T17:45:07.359 INFO:teuthology.orchestra.run.smithi069.stdout: key = AQAjG61m3/RUFRAA4G1FNY10IDhWr6ykAF+dQQ== 2024-08-02T17:45:07.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7680a2c10 msgr2=0x7fc7680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7680a2c10 0x7fc7680a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc77c06ad80 tx=0x7fc764005f90 comp rx=0 tx=0).stop 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 msgr2=0x7fc77c06a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 0x7fc77c06a800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc76c031610 tx=0x7fc76c004270 comp rx=0 tx=0).stop 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 shutdown_connections 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc7680a2c10 0x7fc7680a50d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc77c1023c0 0x7fc77c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc77c101a00 0x7fc77c19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 --2- 172.21.15.69:0/972599976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc77c100800 0x7fc77c06a800 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 >> 172.21.15.69:0/972599976 conn(0x7fc77c0fbfb0 msgr2=0x7fc77c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 shutdown_connections 2024-08-02T17:45:07.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:07.360+0000 7fc781c25640 1 -- 172.21.15.69:0/972599976 wait complete. 2024-08-02T17:45:08.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:07 smithi077 ceph-mon[37108]: pgmap v297: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s; 22 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:08.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:07 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/972599976' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-02T17:45:08.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:07 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/972599976' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-02T17:45:08.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:07 smithi069 ceph-mon[32746]: pgmap v297: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s; 22 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:08.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:07 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/972599976' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-02T17:45:08.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:07 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/972599976' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-02T17:45:08.173 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:45:08.173 DEBUG:teuthology.orchestra.run.smithi069:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-02T17:45:08.174 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-02T17:45:08.209 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-02T17:45:08.210 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-02T17:45:08.210 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph mgr dump --format=json 2024-08-02T17:45:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:07 smithi083 ceph-mon[37031]: pgmap v297: 129 pgs: 129 active+clean; 632 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s; 22 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:07 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/972599976' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-08-02T17:45:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:07 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/972599976' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-02T17:45:08.500 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 -- 172.21.15.69:0/918224006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18102b00 msgr2=0x7faa18102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/918224006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18102b00 0x7faa18102ee0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa14009930 tx=0x7faa1402a420 comp rx=0 tx=0).stop 2024-08-02T17:45:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 -- 172.21.15.69:0/918224006 shutdown_connections 2024-08-02T17:45:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/918224006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa18108030 0x7faa1810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/918224006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa181034b0 0x7faa181078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/918224006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18102b00 0x7faa18102ee0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 -- 172.21.15.69:0/918224006 >> 172.21.15.69:0/918224006 conn(0x7faa180fc500 msgr2=0x7faa180fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:09.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 -- 172.21.15.69:0/918224006 shutdown_connections 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.003+0000 7faa1f6bf640 1 -- 172.21.15.69:0/918224006 wait complete. 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1f6bf640 1 Processor -- start 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1f6bf640 1 -- start start 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1f6bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa18102b00 0x7faa181982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1f6bf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa181034b0 0x7faa18198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1f6bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18108030 0x7faa1819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1f6bf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111e00 con 0x7faa18108030 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1f6bf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111c80 con 0x7faa181034b0 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1f6bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111f80 con 0x7faa18102b00 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1dc5c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18108030 0x7faa1819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:09.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1dc5c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18108030 0x7faa1819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37858/0 (socket says 172.21.15.69:37858) 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1dc5c640 1 -- 172.21.15.69:0/740432836 learned_addr learned my addr 172.21.15.69:0/740432836 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1d45b640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa18102b00 0x7faa181982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1cc5a640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa181034b0 0x7faa18198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1dc5c640 1 -- 172.21.15.69:0/740432836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa18102b00 msgr2=0x7faa181982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1dc5c640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa18102b00 0x7faa181982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1dc5c640 1 -- 172.21.15.69:0/740432836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa181034b0 msgr2=0x7faa18198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1dc5c640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa181034b0 0x7faa18198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.004+0000 7faa1dc5c640 1 -- 172.21.15.69:0/740432836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1819d290 con 0x7faa18108030 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.005+0000 7faa1dc5c640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18108030 0x7faa1819cbb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faa1000eb10 tx=0x7faa1000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.005+0000 7faa0a7fc640 1 -- 172.21.15.69:0/740432836 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10004b20 con 0x7faa18108030 2024-08-02T17:45:09.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.005+0000 7faa0a7fc640 1 -- 172.21.15.69:0/740432836 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa10004cc0 con 0x7faa18108030 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.005+0000 7faa0a7fc640 1 -- 172.21.15.69:0/740432836 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10010760 con 0x7faa18108030 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.005+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1819d580 con 0x7faa18108030 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.005+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa181a4e60 con 0x7faa18108030 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.006+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa1806a4e0 con 0x7faa18108030 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.007+0000 7faa0a7fc640 1 -- 172.21.15.69:0/740432836 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faa10005020 con 0x7faa18108030 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.008+0000 7faa0a7fc640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faa040a2c10 0x7faa040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.008+0000 7faa1d45b640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faa040a2c10 0x7faa040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.008+0000 7faa0a7fc640 1 -- 172.21.15.69:0/740432836 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faa100cfa20 con 0x7faa18108030 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.008+0000 7faa1d45b640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faa040a2c10 0x7faa040a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faa14009900 tx=0x7faa140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:09.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.012+0000 7faa0a7fc640 1 -- 172.21.15.69:0/740432836 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7faa1009c4c0 con 0x7faa18108030 2024-08-02T17:45:09.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.223+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7faa1810a010 con 0x7faa18108030 2024-08-02T17:45:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.227+0000 7faa0a7fc640 1 -- 172.21.15.69:0/740432836 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7faa1009bdc0 con 0x7faa18108030 2024-08-02T17:45:09.232 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faa040a2c10 msgr2=0x7faa040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faa040a2c10 0x7faa040a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faa14009900 tx=0x7faa140058d0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18108030 msgr2=0x7faa1819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18108030 0x7faa1819cbb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faa1000eb10 tx=0x7faa1000c2d0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 shutdown_connections 2024-08-02T17:45:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faa040a2c10 0x7faa040a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faa18108030 0x7faa1819cbb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faa181034b0 0x7faa18198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 --2- 172.21.15.69:0/740432836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa18102b00 0x7faa181982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 >> 172.21.15.69:0/740432836 conn(0x7faa180fc500 msgr2=0x7faa18105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 shutdown_connections 2024-08-02T17:45:09.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:09.230+0000 7faa1f6bf640 1 -- 172.21.15.69:0/740432836 wait complete. 2024-08-02T17:45:09.963 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:09 smithi069 ceph-mon[32746]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s; 20 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:09.963 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:09 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/740432836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-02T17:45:09.990 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6800","nonce":452236630},{"type":"v1","addr":"172.21.15.69:6801","nonce":452236630}]},"active_addr":"172.21.15.69:6801/452236630","active_change":"2024-08-02T17:37:56.685178+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.69:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3271476957}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":2680424375}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":4036393791}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3594391539}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.69:0","nonce":3033851323}]}]} 2024-08-02T17:45:09.995 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-02T17:45:09.995 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-02T17:45:09.996 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:45:10.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:09 smithi077 ceph-mon[37108]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s; 20 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:10.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:09 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/740432836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-02T17:45:10.247 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:09 smithi083 ceph-mon[37031]: pgmap v298: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.2 KiB/s wr, 7 op/s; 20 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:09 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/740432836' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-08-02T17:45:10.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 -- 172.21.15.69:0/180081286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e01022a0 msgr2=0x7fd2e01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:10.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/180081286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e01022a0 0x7fd2e01026e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009900 tx=0x7fd2c802a480 comp rx=0 tx=0).stop 2024-08-02T17:45:10.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 -- 172.21.15.69:0/180081286 shutdown_connections 2024-08-02T17:45:10.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/180081286 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e01022a0 0x7fd2e01026e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/180081286 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e0102c20 0x7fd2e010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/180081286 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e0100b50 0x7fd2e0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 -- 172.21.15.69:0/180081286 >> 172.21.15.69:0/180081286 conn(0x7fd2e00fc500 msgr2=0x7fd2e00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:10.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 -- 172.21.15.69:0/180081286 shutdown_connections 2024-08-02T17:45:10.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.694+0000 7fd2e5faf640 1 -- 172.21.15.69:0/180081286 wait complete. 2024-08-02T17:45:10.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2e5faf640 1 Processor -- start 2024-08-02T17:45:10.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2e5faf640 1 -- start start 2024-08-02T17:45:10.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2e5faf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e0100b50 0x7fd2e010e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:10.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2e5faf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e01022a0 0x7fd2e010e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:10.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2e5faf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e0102c20 0x7fd2e010ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:10.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2e5faf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0111dd0 con 0x7fd2e0100b50 2024-08-02T17:45:10.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2e5faf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0111c50 con 0x7fd2e01022a0 2024-08-02T17:45:10.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2e5faf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e0111f50 con 0x7fd2e0102c20 2024-08-02T17:45:10.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e0100b50 0x7fd2e010e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:10.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e0100b50 0x7fd2e010e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37876/0 (socket says 172.21.15.69:37876) 2024-08-02T17:45:10.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 -- 172.21.15.69:0/3378577412 learned_addr learned my addr 172.21.15.69:0/3378577412 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:10.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2d7fff640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e01022a0 0x7fd2e010e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:10.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2dffff640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e0102c20 0x7fd2e010ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:10.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 -- 172.21.15.69:0/3378577412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e0102c20 msgr2=0x7fd2e010ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:10.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e0102c20 0x7fd2e010ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 -- 172.21.15.69:0/3378577412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e01022a0 msgr2=0x7fd2e010e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e01022a0 0x7fd2e010e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 -- 172.21.15.69:0/3378577412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e01a66d0 con 0x7fd2e0100b50 2024-08-02T17:45:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2dffff640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e0102c20 0x7fd2e010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:10.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.695+0000 7fd2df7fe640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e0100b50 0x7fd2e010e200 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000d950 tx=0x7fd2d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.696+0000 7fd2dd7fa640 1 -- 172.21.15.69:0/3378577412 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0017070 con 0x7fd2e0100b50 2024-08-02T17:45:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.696+0000 7fd2dd7fa640 1 -- 172.21.15.69:0/3378577412 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2d0009c30 con 0x7fd2e0100b50 2024-08-02T17:45:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.696+0000 7fd2dd7fa640 1 -- 172.21.15.69:0/3378577412 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d000f6b0 con 0x7fd2e0100b50 2024-08-02T17:45:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.696+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e01a69c0 con 0x7fd2e0100b50 2024-08-02T17:45:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.696+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e01a6ed0 con 0x7fd2e0100b50 2024-08-02T17:45:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.697+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e0100f30 con 0x7fd2e0100b50 2024-08-02T17:45:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.698+0000 7fd2dd7fa640 1 -- 172.21.15.69:0/3378577412 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd2d00109d0 con 0x7fd2e0100b50 2024-08-02T17:45:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.698+0000 7fd2dd7fa640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd2cc0a2c10 0x7fd2cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.698+0000 7fd2d7fff640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd2cc0a2c10 0x7fd2cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.698+0000 7fd2dd7fa640 1 -- 172.21.15.69:0/3378577412 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fd2d00cf180 con 0x7fd2e0100b50 2024-08-02T17:45:10.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.699+0000 7fd2d7fff640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd2cc0a2c10 0x7fd2cc0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009230 tx=0x7fd2c80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:10.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.703+0000 7fd2dd7fa640 1 -- 172.21.15.69:0/3378577412 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd2d0009de0 con 0x7fd2e0100b50 2024-08-02T17:45:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.871+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2e01a7100 con 0x7fd2e0100b50 2024-08-02T17:45:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.873+0000 7fd2dd7fa640 1 -- 172.21.15.69:0/3378577412 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24491 (secure 0 0 0) 0x7fd2d009bc20 con 0x7fd2e0100b50 2024-08-02T17:45:10.875 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:10.875 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:57.767460+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:45:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd2cc0a2c10 msgr2=0x7fd2cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd2cc0a2c10 0x7fd2cc0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009230 tx=0x7fd2c80058d0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e0100b50 msgr2=0x7fd2e010e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e0100b50 0x7fd2e010e200 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd2d000d950 tx=0x7fd2d000de10 comp rx=0 tx=0).stop 2024-08-02T17:45:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 shutdown_connections 2024-08-02T17:45:10.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fd2cc0a2c10 0x7fd2cc0a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd2e0102c20 0x7fd2e010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2e01022a0 0x7fd2e010e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 --2- 172.21.15.69:0/3378577412 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2e0100b50 0x7fd2e010e200 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:10.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 >> 172.21.15.69:0/3378577412 conn(0x7fd2e00fc500 msgr2=0x7fd2e00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:10.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 shutdown_connections 2024-08-02T17:45:10.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:10.875+0000 7fd2e5faf640 1 -- 172.21.15.69:0/3378577412 wait complete. 2024-08-02T17:45:11.641 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-02T17:45:11.641 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:45:11.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:11 smithi069 ceph-mon[32746]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 7 op/s; 19 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:11.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:11 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3378577412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:11.904 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:12.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:11 smithi077 ceph-mon[37108]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 7 op/s; 19 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:12.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:11 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3378577412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:12.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:11 smithi083 ceph-mon[37031]: pgmap v299: 129 pgs: 129 active+clean; 641 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.9 KiB/s wr, 7 op/s; 19 B/s, 0 keys/s, 1 objects/s recovering 2024-08-02T17:45:12.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:11 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3378577412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 -- 172.21.15.69:0/3225627265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30102ac0 msgr2=0x7fac30102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 --2- 172.21.15.69:0/3225627265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30102ac0 0x7fac30102ea0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fac24009930 tx=0x7fac2402a420 comp rx=0 tx=0).stop 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 -- 172.21.15.69:0/3225627265 shutdown_connections 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 --2- 172.21.15.69:0/3225627265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac30107fc0 0x7fac3010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 --2- 172.21.15.69:0/3225627265 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac30103470 0x7fac301078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 --2- 172.21.15.69:0/3225627265 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30102ac0 0x7fac30102ea0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 -- 172.21.15.69:0/3225627265 >> 172.21.15.69:0/3225627265 conn(0x7fac300fc500 msgr2=0x7fac300fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 -- 172.21.15.69:0/3225627265 shutdown_connections 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 -- 172.21.15.69:0/3225627265 wait complete. 2024-08-02T17:45:12.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 Processor -- start 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.385+0000 7fac36b7f640 1 -- start start 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac36b7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac30102ac0 0x7fac30198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac36b7f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac30103470 0x7fac30198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac36b7f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30107fc0 0x7fac3019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac36b7f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac30111dc0 con 0x7fac30107fc0 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac36b7f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fac30111c40 con 0x7fac30103470 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac36b7f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac30111f40 con 0x7fac30102ac0 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30107fc0 0x7fac3019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30107fc0 0x7fac3019cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37886/0 (socket says 172.21.15.69:37886) 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 -- 172.21.15.69:0/2878053688 learned_addr learned my addr 172.21.15.69:0/2878053688 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac2ffff640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac30103470 0x7fac30198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3491b640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac30102ac0 0x7fac30198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:12.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 -- 172.21.15.69:0/2878053688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac30102ac0 msgr2=0x7fac30198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac30102ac0 0x7fac30198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 -- 172.21.15.69:0/2878053688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac30103470 msgr2=0x7fac30198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac30103470 0x7fac30198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 -- 172.21.15.69:0/2878053688 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3019d2b0 con 0x7fac30107fc0 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac2ffff640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac30103470 0x7fac30198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac3511c640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30107fc0 0x7fac3019cbd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fac2800cad0 tx=0x7fac2800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac2dffb640 1 -- 172.21.15.69:0/2878053688 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2800ae20 con 0x7fac30107fc0 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3019d5a0 con 0x7fac30107fc0 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac2dffb640 1 -- 172.21.15.69:0/2878053688 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac28005020 con 0x7fac30107fc0 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.386+0000 7fac2dffb640 1 -- 172.21.15.69:0/2878053688 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2800f6b0 con 0x7fac30107fc0 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.387+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac301a4e80 con 0x7fac30107fc0 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.388+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3006a520 con 0x7fac30107fc0 2024-08-02T17:45:12.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.389+0000 7fac2dffb640 1 -- 172.21.15.69:0/2878053688 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac280051d0 con 0x7fac30107fc0 2024-08-02T17:45:12.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.389+0000 7fac2dffb640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac200a2c10 0x7fac200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:12.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.390+0000 7fac3491b640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac200a2c10 0x7fac200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:12.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.390+0000 7fac2dffb640 1 -- 172.21.15.69:0/2878053688 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fac280cf760 con 0x7fac30107fc0 2024-08-02T17:45:12.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.390+0000 7fac3491b640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac200a2c10 0x7fac200a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac24009900 tx=0x7fac240058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:12.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.393+0000 7fac2dffb640 1 -- 172.21.15.69:0/2878053688 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fac28014030 con 0x7fac30107fc0 2024-08-02T17:45:12.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.562+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac30109fd0 con 0x7fac30107fc0 2024-08-02T17:45:12.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.564+0000 7fac2dffb640 1 -- 172.21.15.69:0/2878053688 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24491 (secure 0 0 0) 0x7fac2809bbb0 con 0x7fac30107fc0 2024-08-02T17:45:12.566 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:12.566 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:57.767460+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:45:12.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac200a2c10 msgr2=0x7fac200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac200a2c10 0x7fac200a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fac24009900 tx=0x7fac240058d0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30107fc0 msgr2=0x7fac3019cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30107fc0 0x7fac3019cbd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fac2800cad0 tx=0x7fac2800cf90 comp rx=0 tx=0).stop 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 shutdown_connections 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac200a2c10 0x7fac200a50d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac30107fc0 0x7fac3019cbd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac30103470 0x7fac30198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 --2- 172.21.15.69:0/2878053688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac30102ac0 0x7fac30198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 >> 172.21.15.69:0/2878053688 conn(0x7fac300fc500 msgr2=0x7fac30105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 shutdown_connections 2024-08-02T17:45:12.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:12.567+0000 7fac36b7f640 1 -- 172.21.15.69:0/2878053688 wait complete. 2024-08-02T17:45:13.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:12 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2878053688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:13.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:12 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2878053688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:13.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:12 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2878053688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:13.484 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-02T17:45:13.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-02T17:45:13.485 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-02T17:45:13.486 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-02T17:45:13.486 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-02T17:45:13.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-02T17:45:13.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-02T17:45:13.487 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-02T17:45:13.488 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-02T17:45:13.488 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-02T17:45:13.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-02T17:45:13.489 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-02T17:45:14.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:13 smithi077 ceph-mon[37108]: pgmap v300: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-08-02T17:45:14.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:13 smithi069 ceph-mon[32746]: pgmap v300: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-08-02T17:45:14.268 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:13 smithi083 ceph-mon[37031]: pgmap v300: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.0 KiB/s wr, 8 op/s; 9 B/s, 0 objects/s recovering 2024-08-02T17:45:14.275 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.275 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.276 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.286 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.290 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.290 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.295 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.299 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.303 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.306 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:14.306 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:15.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 -- 172.21.15.69:0/4150685824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c103390 msgr2=0x7f531c10b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 --2- 172.21.15.69:0/4150685824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c103390 0x7f531c10b880 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f530c009900 tx=0x7f530c02a480 comp rx=0 tx=0).stop 2024-08-02T17:45:15.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 -- 172.21.15.69:0/4150685824 shutdown_connections 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 --2- 172.21.15.69:0/4150685824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c103390 0x7f531c10b880 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 --2- 172.21.15.69:0/4150685824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c10bea0 0x7f531c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 --2- 172.21.15.69:0/4150685824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f531c102a70 0x7f531c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 -- 172.21.15.69:0/4150685824 >> 172.21.15.69:0/4150685824 conn(0x7f531c0fc4d0 msgr2=0x7f531c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 -- 172.21.15.69:0/4150685824 shutdown_connections 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.219+0000 7f5321126640 1 -- 172.21.15.69:0/4150685824 wait complete. 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f5321126640 1 Processor -- start 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f5321126640 1 -- start start 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f5321126640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c102a70 0x7f531c1006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f5321126640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f531c103390 0x7f531c100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f5321126640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c10bea0 0x7f531c101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f5321126640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f531c111d60 con 0x7f531c102a70 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f5321126640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f531c111be0 con 0x7f531c10bea0 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f5321126640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f531c111ee0 con 0x7f531c103390 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f531ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c102a70 0x7f531c1006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f531ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c102a70 0x7f531c1006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34066/0 (socket says 172.21.15.69:34066) 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.220+0000 7f531ad9d640 1 -- 172.21.15.69:0/2805930300 learned_addr learned my addr 172.21.15.69:0/2805930300 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531b59e640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c10bea0 0x7f531c101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531a59c640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f531c103390 0x7f531c100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531ad9d640 1 -- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f531c103390 msgr2=0x7f531c100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531ad9d640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f531c103390 0x7f531c100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531ad9d640 1 -- 172.21.15.69:0/2805930300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c10bea0 msgr2=0x7f531c101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531ad9d640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c10bea0 0x7f531c101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531ad9d640 1 -- 172.21.15.69:0/2805930300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c1a67c0 con 0x7f531c102a70 2024-08-02T17:45:15.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531b59e640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f531c10bea0 0x7f531c101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:15.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531a59c640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f531c103390 0x7f531c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:15.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f531ad9d640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c102a70 0x7f531c1006d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f531000b4a0 tx=0x7f531000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310014070 con 0x7f531c102a70 2024-08-02T17:45:15.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c1980c0 con 0x7f531c102a70 2024-08-02T17:45:15.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5310010af0 con 0x7f531c102a70 2024-08-02T17:45:15.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310017d10 con 0x7f531c102a70 2024-08-02T17:45:15.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.221+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c198600 con 0x7f531c102a70 2024-08-02T17:45:15.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.222+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f531c062630 con 0x7f531c102a70 2024-08-02T17:45:15.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.224+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5310017070 con 0x7f531c102a70 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.224+0000 7f5303fff640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f53040a2c10 0x7f53040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.224+0000 7f531a59c640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f53040a2c10 0x7f53040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.224+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f53100cec70 con 0x7f531c102a70 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.225+0000 7f5303fff640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] conn(0x7f53040adde0 0x7f53040b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.225+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 --> [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53040b0950 con 0x7f53040adde0 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.225+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f53100cf0c0 con 0x7f531c102a70 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.225+0000 7f531b59e640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] conn(0x7f53040adde0 0x7f53040b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.225+0000 7f531a59c640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f53040a2c10 0x7f53040a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f531c102160 tx=0x7f530c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.225+0000 7f531b59e640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] conn(0x7f53040adde0 0x7f53040b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.226+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 <== osd.2 v2:172.21.15.83:6800/2938396779 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53040b0950 con 0x7f53040adde0 2024-08-02T17:45:15.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.240+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 --> [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f531c062ab0 con 0x7f53040adde0 2024-08-02T17:45:15.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.240+0000 7f5303fff640 1 -- 172.21.15.69:0/2805930300 <== osd.2 v2:172.21.15.83:6800/2938396779 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f531c062ab0 con 0x7f53040adde0 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] conn(0x7f53040adde0 msgr2=0x7f53040b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] conn(0x7f53040adde0 0x7f53040b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f53040a2c10 msgr2=0x7f53040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f53040a2c10 0x7f53040a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f531c102160 tx=0x7f530c005a20 comp rx=0 tx=0).stop 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c102a70 msgr2=0x7f531c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 --2- 172.21.15.69:0/2805930300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f531c102a70 0x7f531c1006d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f531000b4a0 tx=0x7f531000b960 comp rx=0 tx=0).stop 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f531b59e640 1 -- 172.21.15.69:0/2805930300 reap_dead start 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 shutdown_connections 2024-08-02T17:45:15.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 >> 172.21.15.69:0/2805930300 conn(0x7f531c0fc4d0 msgr2=0x7f531c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:15.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 shutdown_connections 2024-08-02T17:45:15.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.241+0000 7f5321126640 1 -- 172.21.15.69:0/2805930300 wait complete. 2024-08-02T17:45:15.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.496+0000 7f6805c08640 1 -- 172.21.15.69:0/3254178211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 msgr2=0x7f680010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.496+0000 7f6805c08640 1 --2- 172.21.15.69:0/3254178211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 0x7f680010d770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009ca0 tx=0x7f67f802a950 comp rx=0 tx=0).stop 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.496+0000 7f6805c08640 1 -- 172.21.15.69:0/3254178211 shutdown_connections 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.496+0000 7f6805c08640 1 --2- 172.21.15.69:0/3254178211 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 0x7f680010d770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.496+0000 7f6805c08640 1 --2- 172.21.15.69:0/3254178211 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68001022a0 0x7f68001026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.496+0000 7f6805c08640 1 --2- 172.21.15.69:0/3254178211 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6800100b50 0x7f6800100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.496+0000 7f6805c08640 1 -- 172.21.15.69:0/3254178211 >> 172.21.15.69:0/3254178211 conn(0x7f68000fc500 msgr2=0x7f68000fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.497+0000 7f6805c08640 1 -- 172.21.15.69:0/3254178211 shutdown_connections 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.497+0000 7f6805c08640 1 -- 172.21.15.69:0/3254178211 wait complete. 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.497+0000 7f6805c08640 1 Processor -- start 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.497+0000 7f6805c08640 1 -- start start 2024-08-02T17:45:15.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.497+0000 7f6805c08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800100b50 0x7f68001981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.497+0000 7f6805c08640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68001022a0 0x7f6800198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.497+0000 7f6805c08640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 0x7f680019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f6805c08640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6800110cb0 con 0x7f6800102c20 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f6805c08640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6800110b30 con 0x7f68001022a0 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f6805c08640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6800110e30 con 0x7f6800100b50 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 0x7f680019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 0x7f680019cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34072/0 (socket says 172.21.15.69:34072) 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67ff7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800100b50 0x7f68001981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 -- 172.21.15.69:0/2724366449 learned_addr learned my addr 172.21.15.69:0/2724366449 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67f7fff640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68001022a0 0x7f6800198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 -- 172.21.15.69:0/2724366449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800100b50 msgr2=0x7f68001981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6800100b50 0x7f68001981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 -- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68001022a0 msgr2=0x7f6800198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68001022a0 0x7f6800198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 -- 172.21.15.69:0/2724366449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680019d190 con 0x7f6800102c20 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67f7fff640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68001022a0 0x7f6800198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fffff640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 0x7f680019cab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f67f802ae70 tx=0x7f67f802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8034040 con 0x7f6800102c20 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f8030920 con 0x7f6800102c20 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680019d420 con 0x7f6800102c20 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f8045480 con 0x7f6800102c20 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.498+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68001a4d60 con 0x7f6800102c20 2024-08-02T17:45:15.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.499+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6800062630 con 0x7f6800102c20 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.501+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f67f8037020 con 0x7f6800102c20 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.502+0000 7f67fd7fa640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f67ec0a2c90 0x7f67ec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.502+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f67f80f06b0 con 0x7f6800102c20 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.502+0000 7f67ff7fe640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f67ec0a2c90 0x7f67ec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.502+0000 7f67fd7fa640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] conn(0x7f67ec0ade60 0x7f67ec0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.502+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 --> [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67f803ce50 con 0x7f67ec0ade60 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.502+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f67f80f0b00 con 0x7f6800102c20 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.503+0000 7f67ff7fe640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f67ec0a2c90 0x7f67ec0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67f000bbe0 tx=0x7f67f000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.503+0000 7f67f7fff640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] conn(0x7f67ec0ade60 0x7f67ec0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.503+0000 7f67f7fff640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] conn(0x7f67ec0ade60 0x7f67ec0b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.504+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 <== osd.4 v2:172.21.15.77:6808/64984823 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f67f803ce50 con 0x7f67ec0ade60 2024-08-02T17:45:15.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.517+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 --> [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6800062a30 con 0x7f67ec0ade60 2024-08-02T17:45:15.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.519+0000 7f67fd7fa640 1 -- 172.21.15.69:0/2724366449 <== osd.4 v2:172.21.15.77:6808/64984823 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6800062a30 con 0x7f67ec0ade60 2024-08-02T17:45:15.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.519+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] conn(0x7f67ec0ade60 msgr2=0x7f67ec0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.519+0000 7f6805c08640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] conn(0x7f67ec0ade60 0x7f67ec0b0300 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f67ec0a2c90 msgr2=0x7f67ec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f6805c08640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f67ec0a2c90 0x7f67ec0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67f000bbe0 tx=0x7f67f000b390 comp rx=0 tx=0).stop 2024-08-02T17:45:15.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 msgr2=0x7f680019cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f6805c08640 1 --2- 172.21.15.69:0/2724366449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6800102c20 0x7f680019cab0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f67f802ae70 tx=0x7f67f802f5d0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f67fffff640 1 -- 172.21.15.69:0/2724366449 reap_dead start 2024-08-02T17:45:15.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 shutdown_connections 2024-08-02T17:45:15.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 >> 172.21.15.69:0/2724366449 conn(0x7f68000fc500 msgr2=0x7f6800069e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:15.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 shutdown_connections 2024-08-02T17:45:15.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.520+0000 7f6805c08640 1 -- 172.21.15.69:0/2724366449 wait complete. 2024-08-02T17:45:15.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 -- 172.21.15.69:0/4196546878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc48102c20 msgr2=0x7fdc4810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 --2- 172.21.15.69:0/4196546878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc48102c20 0x7fdc4810d770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009900 tx=0x7fdc3802a480 comp rx=0 tx=0).stop 2024-08-02T17:45:15.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 -- 172.21.15.69:0/4196546878 shutdown_connections 2024-08-02T17:45:15.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 --2- 172.21.15.69:0/4196546878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc48102c20 0x7fdc4810d770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 --2- 172.21.15.69:0/4196546878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc481022a0 0x7fdc481026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 --2- 172.21.15.69:0/4196546878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc48100b50 0x7fdc48100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 -- 172.21.15.69:0/4196546878 >> 172.21.15.69:0/4196546878 conn(0x7fdc480fc500 msgr2=0x7fdc480fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 -- 172.21.15.69:0/4196546878 shutdown_connections 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.604+0000 7fdc4cf43640 1 -- 172.21.15.69:0/4196546878 wait complete. 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4cf43640 1 Processor -- start 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4cf43640 1 -- start start 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4cf43640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc48100b50 0x7fdc48198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4cf43640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc481022a0 0x7fdc481994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4cf43640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc48102c20 0x7fdc481a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4cf43640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48110cb0 con 0x7fdc481022a0 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4cf43640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48110b30 con 0x7fdc48100b50 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4cf43640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc48110e30 con 0x7fdc48102c20 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc45d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc481022a0 0x7fdc481994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc45d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc481022a0 0x7fdc481994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34092/0 (socket says 172.21.15.69:34092) 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc45d9b640 1 -- 172.21.15.69:0/3272058488 learned_addr learned my addr 172.21.15.69:0/3272058488 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc4659c640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc48100b50 0x7fdc48198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.605+0000 7fdc46d9d640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc48102c20 0x7fdc481a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc45d9b640 1 -- 172.21.15.69:0/3272058488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc48102c20 msgr2=0x7fdc481a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc45d9b640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc48102c20 0x7fdc481a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc45d9b640 1 -- 172.21.15.69:0/3272058488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc48100b50 msgr2=0x7fdc48198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc45d9b640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc48100b50 0x7fdc48198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc45d9b640 1 -- 172.21.15.69:0/3272058488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc481a4560 con 0x7fdc481022a0 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc4659c640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc48100b50 0x7fdc48198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc45d9b640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc481022a0 0x7fdc481994e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000c920 tx=0x7fdc3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30007c70 con 0x7fdc481022a0 2024-08-02T17:45:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc30007e10 con 0x7fdc481022a0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc481a4730 con 0x7fdc481022a0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30018890 con 0x7fdc481022a0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.606+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc481a4c40 con 0x7fdc481022a0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.607+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc48106f10 con 0x7fdc481022a0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.608+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdc30018a30 con 0x7fdc481022a0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.609+0000 7fdc3f7fe640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc340a2c10 0x7fdc340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.609+0000 7fdc4659c640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc340a2c10 0x7fdc340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.609+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdc300cfbe0 con 0x7fdc481022a0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.609+0000 7fdc3f7fe640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] conn(0x7fdc340adde0 0x7fdc340b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.609+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 --> [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc340b0950 con 0x7fdc340adde0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.609+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fdc300cff90 con 0x7fdc481022a0 2024-08-02T17:45:15.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.609+0000 7fdc46d9d640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] conn(0x7fdc340adde0 0x7fdc340b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.609+0000 7fdc4659c640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc340a2c10 0x7fdc340a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009fd0 tx=0x7fdc40009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.610+0000 7fdc46d9d640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] conn(0x7fdc340adde0 0x7fdc340b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.610+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 <== osd.9 v2:172.21.15.69:6826/2821083254 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc340b0950 con 0x7fdc340adde0 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.624+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 --> [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc48101ee0 con 0x7fdc340adde0 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.624+0000 7fdc3f7fe640 1 -- 172.21.15.69:0/3272058488 <== osd.9 v2:172.21.15.69:6826/2821083254 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdc48101ee0 con 0x7fdc340adde0 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] conn(0x7fdc340adde0 msgr2=0x7fdc340b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] conn(0x7fdc340adde0 0x7fdc340b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc340a2c10 msgr2=0x7fdc340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdc340a2c10 0x7fdc340a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009fd0 tx=0x7fdc40009680 comp rx=0 tx=0).stop 2024-08-02T17:45:15.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc481022a0 msgr2=0x7fdc481994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:15.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 --2- 172.21.15.69:0/3272058488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc481022a0 0x7fdc481994e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000c920 tx=0x7fdc3000cde0 comp rx=0 tx=0).stop 2024-08-02T17:45:15.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc46d9d640 1 -- 172.21.15.69:0/3272058488 reap_dead start 2024-08-02T17:45:15.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 shutdown_connections 2024-08-02T17:45:15.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 >> 172.21.15.69:0/3272058488 conn(0x7fdc480fc500 msgr2=0x7fdc481064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:15.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 shutdown_connections 2024-08-02T17:45:15.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:15.625+0000 7fdc4cf43640 1 -- 172.21.15.69:0/3272058488 wait complete. 2024-08-02T17:45:16.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:15 smithi077 ceph-mon[37108]: pgmap v301: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-02T17:45:16.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:15 smithi069 ceph-mon[32746]: pgmap v301: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-02T17:45:16.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:15 smithi083 ceph-mon[37031]: pgmap v301: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-02T17:45:17.949 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:17 smithi069 ceph-mon[32746]: pgmap v302: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-02T17:45:18.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 -- 172.21.15.69:0/3767728839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1007b0 msgr2=0x7fac6c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 --2- 172.21.15.69:0/3767728839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1007b0 0x7fac6c100c10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0099b0 tx=0x7fac5c02c650 comp rx=0 tx=0).stop 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 -- 172.21.15.69:0/3767728839 shutdown_connections 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 --2- 172.21.15.69:0/3767728839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1007b0 0x7fac6c100c10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 --2- 172.21.15.69:0/3767728839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6c101150 0x7fac6c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 --2- 172.21.15.69:0/3767728839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac6c1067b0 0x7fac6c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 -- 172.21.15.69:0/3767728839 >> 172.21.15.69:0/3767728839 conn(0x7fac6c0fc4d0 msgr2=0x7fac6c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 -- 172.21.15.69:0/3767728839 shutdown_connections 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 -- 172.21.15.69:0/3767728839 wait complete. 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 Processor -- start 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.948+0000 7fac747d9640 1 -- start start 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac747d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6c1007b0 0x7fac6c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac747d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac6c101150 0x7fac6c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac747d9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1067b0 0x7fac6c19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac747d9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c10faa0 con 0x7fac6c1067b0 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac747d9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c10f920 con 0x7fac6c101150 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac747d9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac6c10fc20 con 0x7fac6c1007b0 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72d76640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1067b0 0x7fac6c19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac71d74640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac6c101150 0x7fac6c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72d76640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1067b0 0x7fac6c19ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34124/0 (socket says 172.21.15.69:34124) 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72d76640 1 -- 172.21.15.69:0/529200784 learned_addr learned my addr 172.21.15.69:0/529200784 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72575640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6c1007b0 0x7fac6c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72d76640 1 -- 172.21.15.69:0/529200784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6c1007b0 msgr2=0x7fac6c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72d76640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac6c1007b0 0x7fac6c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72d76640 1 -- 172.21.15.69:0/529200784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac6c101150 msgr2=0x7fac6c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72d76640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac6c101150 0x7fac6c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac72d76640 1 -- 172.21.15.69:0/529200784 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac6c19f320 con 0x7fac6c1067b0 2024-08-02T17:45:18.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.949+0000 7fac71d74640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fac6c101150 0x7fac6c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:45:18.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.950+0000 7fac72d76640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1067b0 0x7fac6c19ec40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fac6400cce0 tx=0x7fac6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.950+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac6400f450 con 0x7fac6c1067b0 2024-08-02T17:45:18.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.950+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac6400fad0 con 0x7fac6c1067b0 2024-08-02T17:45:18.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.950+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac6c19f610 con 0x7fac6c1067b0 2024-08-02T17:45:18.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.950+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac64018c10 con 0x7fac6c1067b0 2024-08-02T17:45:18.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.950+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac6c1a6ef0 con 0x7fac6c1067b0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.950+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac6c0626b0 con 0x7fac6c1067b0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.952+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac6400f5f0 con 0x7fac6c1067b0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac637fe640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac580a2c90 0x7fac580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fac640cfd40 con 0x7fac6c1067b0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac72575640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac580a2c90 0x7fac580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac637fe640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] conn(0x7fac580ade60 0x7fac580b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 --> [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac640047f0 con 0x7fac580ade60 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fac640d0190 con 0x7fac6c1067b0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac71d74640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] conn(0x7fac580ade60 0x7fac580b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac72575640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac580a2c90 0x7fac580a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac6800bb60 tx=0x7fac6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.953+0000 7fac71d74640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] conn(0x7fac580ade60 0x7fac580b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.954+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 <== osd.6 v2:172.21.15.69:6818/2370248409 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac640047f0 con 0x7fac580ade60 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 --> [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fac6c062b30 con 0x7fac580ade60 2024-08-02T17:45:18.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac637fe640 1 -- 172.21.15.69:0/529200784 <== osd.6 v2:172.21.15.69:6818/2370248409 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fac6c062b30 con 0x7fac580ade60 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] conn(0x7fac580ade60 msgr2=0x7fac580b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] conn(0x7fac580ade60 0x7fac580b0300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac580a2c90 msgr2=0x7fac580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fac580a2c90 0x7fac580a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac6800bb60 tx=0x7fac6800b390 comp rx=0 tx=0).stop 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1067b0 msgr2=0x7fac6c19ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 --2- 172.21.15.69:0/529200784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fac6c1067b0 0x7fac6c19ec40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fac6400cce0 tx=0x7fac6400a620 comp rx=0 tx=0).stop 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac72d76640 1 -- 172.21.15.69:0/529200784 reap_dead start 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 shutdown_connections 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 >> 172.21.15.69:0/529200784 conn(0x7fac6c0fc4d0 msgr2=0x7fac6c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 shutdown_connections 2024-08-02T17:45:18.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.968+0000 7fac747d9640 1 -- 172.21.15.69:0/529200784 wait complete. 2024-08-02T17:45:18.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:17 smithi077 ceph-mon[37108]: pgmap v302: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-02T17:45:18.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:17 smithi083 ceph-mon[37031]: pgmap v302: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-02T17:45:18.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 -- 172.21.15.69:0/2225023978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e8102c20 msgr2=0x7f92e810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 --2- 172.21.15.69:0/2225023978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e8102c20 0x7f92e810d770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f92e0009ca0 tx=0x7f92e002a950 comp rx=0 tx=0).stop 2024-08-02T17:45:18.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 -- 172.21.15.69:0/2225023978 shutdown_connections 2024-08-02T17:45:18.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 --2- 172.21.15.69:0/2225023978 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e8102c20 0x7f92e810d770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 --2- 172.21.15.69:0/2225023978 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e81022a0 0x7f92e81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 --2- 172.21.15.69:0/2225023978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92e8100b50 0x7f92e8100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 -- 172.21.15.69:0/2225023978 >> 172.21.15.69:0/2225023978 conn(0x7f92e80fc500 msgr2=0x7f92e80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 -- 172.21.15.69:0/2225023978 shutdown_connections 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.383+0000 7f92ecee0640 1 -- 172.21.15.69:0/2225023978 wait complete. 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92ecee0640 1 Processor -- start 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92ecee0640 1 -- start start 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92ecee0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e8100b50 0x7f92e8198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92ecee0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e81022a0 0x7f92e8199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92ecee0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92e8102c20 0x7f92e81a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92ecee0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8110cd0 con 0x7f92e81022a0 2024-08-02T17:45:18.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92ecee0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8110b50 con 0x7f92e8100b50 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92ecee0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92e8110e50 con 0x7f92e8102c20 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e5d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e81022a0 0x7f92e8199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e5d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e81022a0 0x7f92e8199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34148/0 (socket says 172.21.15.69:34148) 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e5d9b640 1 -- 172.21.15.69:0/4173391002 learned_addr learned my addr 172.21.15.69:0/4173391002 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e6d9d640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92e8102c20 0x7f92e81a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e659c640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e8100b50 0x7f92e8198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e5d9b640 1 -- 172.21.15.69:0/4173391002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92e8102c20 msgr2=0x7f92e81a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e5d9b640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92e8102c20 0x7f92e81a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e5d9b640 1 -- 172.21.15.69:0/4173391002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e8100b50 msgr2=0x7f92e8198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e5d9b640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e8100b50 0x7f92e8198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e5d9b640 1 -- 172.21.15.69:0/4173391002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e81a4590 con 0x7f92e81022a0 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.384+0000 7f92e659c640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92e8100b50 0x7f92e8198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:18.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.385+0000 7f92e5d9b640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e81022a0 0x7f92e8199510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c920 tx=0x7f92dc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.385+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc007c70 con 0x7f92e81022a0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.385+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e81a4760 con 0x7f92e81022a0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.385+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92dc007e10 con 0x7f92e81022a0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.385+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc018890 con 0x7f92e81022a0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.385+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e81a4ca0 con 0x7f92e81022a0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.386+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f92e8106f90 con 0x7f92e81022a0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.387+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f92dc018a30 con 0x7f92e81022a0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.387+0000 7f92d77fe640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f92d00a2c10 0x7f92d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.388+0000 7f92e659c640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f92d00a2c10 0x7f92d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.388+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f92dc0cfac0 con 0x7f92e81022a0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.388+0000 7f92d77fe640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] conn(0x7f92d00adde0 0x7f92d00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.388+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 --> [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92d00b0950 con 0x7f92d00adde0 2024-08-02T17:45:18.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.388+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f92dc0cff10 con 0x7f92e81022a0 2024-08-02T17:45:18.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.388+0000 7f92e6d9d640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] conn(0x7f92d00adde0 0x7f92d00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.388+0000 7f92e659c640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f92d00a2c10 0x7f92d00a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009930 tx=0x7f92d8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.388+0000 7f92e6d9d640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] conn(0x7f92d00adde0 0x7f92d00b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.389+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 <== osd.3 v2:172.21.15.69:6810/980470189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f92d00b0950 con 0x7f92d00adde0 2024-08-02T17:45:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.402+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 --> [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f92e8101ee0 con 0x7f92d00adde0 2024-08-02T17:45:18.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92d77fe640 1 -- 172.21.15.69:0/4173391002 <== osd.3 v2:172.21.15.69:6810/980470189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f92e8101ee0 con 0x7f92d00adde0 2024-08-02T17:45:18.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] conn(0x7f92d00adde0 msgr2=0x7f92d00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] conn(0x7f92d00adde0 0x7f92d00b0280 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f92d00a2c10 msgr2=0x7f92d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f92d00a2c10 0x7f92d00a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009930 tx=0x7f92d8008040 comp rx=0 tx=0).stop 2024-08-02T17:45:18.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e81022a0 msgr2=0x7f92e8199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 --2- 172.21.15.69:0/4173391002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92e81022a0 0x7f92e8199510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c920 tx=0x7f92dc00cde0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92e6d9d640 1 -- 172.21.15.69:0/4173391002 reap_dead start 2024-08-02T17:45:18.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 shutdown_connections 2024-08-02T17:45:18.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 >> 172.21.15.69:0/4173391002 conn(0x7f92e80fc500 msgr2=0x7f92e8106530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:18.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 shutdown_connections 2024-08-02T17:45:18.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.403+0000 7f92ecee0640 1 -- 172.21.15.69:0/4173391002 wait complete. 2024-08-02T17:45:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- 172.21.15.69:0/3327639474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c102a70 msgr2=0x7f279c102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 --2- 172.21.15.69:0/3327639474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c102a70 0x7f279c102e50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f278c009900 tx=0x7f278c02a480 comp rx=0 tx=0).stop 2024-08-02T17:45:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- 172.21.15.69:0/3327639474 shutdown_connections 2024-08-02T17:45:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 --2- 172.21.15.69:0/3327639474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f279c10bea0 0x7f279c10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 --2- 172.21.15.69:0/3327639474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f279c103390 0x7f279c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 --2- 172.21.15.69:0/3327639474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c102a70 0x7f279c102e50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- 172.21.15.69:0/3327639474 >> 172.21.15.69:0/3327639474 conn(0x7f279c0fc4d0 msgr2=0x7f279c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:18.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- 172.21.15.69:0/3327639474 shutdown_connections 2024-08-02T17:45:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- 172.21.15.69:0/3327639474 wait complete. 2024-08-02T17:45:18.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 Processor -- start 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- start start 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f279c102a70 0x7f279c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f279c103390 0x7f279c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c10bea0 0x7f279c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f279c111ca0 con 0x7f279c10bea0 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f279c111b20 con 0x7f279c102a70 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.761+0000 7f27a26fd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f279c111e20 con 0x7f279c103390 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c10bea0 0x7f279c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c10bea0 0x7f279c19c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34168/0 (socket says 172.21.15.69:34168) 2024-08-02T17:45:18.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 -- 172.21.15.69:0/1741600092 learned_addr learned my addr 172.21.15.69:0/1741600092 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:18.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f279bfff640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f279c102a70 0x7f279c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f2793fff640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f279c103390 0x7f279c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 -- 172.21.15.69:0/1741600092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f279c103390 msgr2=0x7f279c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f279c103390 0x7f279c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 -- 172.21.15.69:0/1741600092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f279c102a70 msgr2=0x7f279c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f279c102a70 0x7f279c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 -- 172.21.15.69:0/1741600092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279c19d040 con 0x7f279c10bea0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a0c9a640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c10bea0 0x7f279c19c960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f279400eaa0 tx=0x7f279400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279400cdb0 con 0x7f279c10bea0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2794005020 con 0x7f279c10bea0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279c19d330 con 0x7f279c10bea0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27940116b0 con 0x7f279c10bea0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.762+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279c1a4c10 con 0x7f279c10bea0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.763+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f279c0fe5b0 con 0x7f279c10bea0 2024-08-02T17:45:18.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.764+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2794005560 con 0x7f279c10bea0 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.765+0000 7f2799ffb640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27880a2c10 0x7f27880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.765+0000 7f279bfff640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27880a2c10 0x7f27880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.765+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f27940cfa90 con 0x7f279c10bea0 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.765+0000 7f2799ffb640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] conn(0x7f27880adde0 0x7f27880b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.765+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 --> [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2794014ec0 con 0x7f27880adde0 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.765+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f27940cfee0 con 0x7f279c10bea0 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.766+0000 7f2793fff640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] conn(0x7f27880adde0 0x7f27880b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.766+0000 7f279bfff640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27880a2c10 0x7f27880a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f278c009230 tx=0x7f278c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.766+0000 7f2793fff640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] conn(0x7f27880adde0 0x7f27880b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:18.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.767+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 <== osd.5 v2:172.21.15.83:6808/3536789836 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2794014ec0 con 0x7f27880adde0 2024-08-02T17:45:18.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.781+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 --> [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f279c06ada0 con 0x7f27880adde0 2024-08-02T17:45:18.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.781+0000 7f2799ffb640 1 -- 172.21.15.69:0/1741600092 <== osd.5 v2:172.21.15.83:6808/3536789836 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f279c06ada0 con 0x7f27880adde0 2024-08-02T17:45:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 >> [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] conn(0x7f27880adde0 msgr2=0x7f27880b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] conn(0x7f27880adde0 0x7f27880b0280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27880a2c10 msgr2=0x7f27880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27880a2c10 0x7f27880a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f278c009230 tx=0x7f278c0058d0 comp rx=0 tx=0).stop 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c10bea0 msgr2=0x7f279c19c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 --2- 172.21.15.69:0/1741600092 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f279c10bea0 0x7f279c19c960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f279400eaa0 tx=0x7f279400ef60 comp rx=0 tx=0).stop 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a0c9a640 1 -- 172.21.15.69:0/1741600092 reap_dead start 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 shutdown_connections 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 >> 172.21.15.69:0/1741600092 conn(0x7f279c0fc4d0 msgr2=0x7f279c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 shutdown_connections 2024-08-02T17:45:18.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:18.782+0000 7f27a26fd640 1 -- 172.21.15.69:0/1741600092 wait complete. 2024-08-02T17:45:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 -- 172.21.15.69:0/3580618598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701078b0 msgr2=0x7faf70107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 --2- 172.21.15.69:0/3580618598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701078b0 0x7faf70107c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faf64009930 tx=0x7faf6402a420 comp rx=0 tx=0).stop 2024-08-02T17:45:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 -- 172.21.15.69:0/3580618598 shutdown_connections 2024-08-02T17:45:19.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 --2- 172.21.15.69:0/3580618598 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70102250 0x7faf7010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 --2- 172.21.15.69:0/3580618598 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf701018b0 0x7faf70101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 --2- 172.21.15.69:0/3580618598 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701078b0 0x7faf70107c90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 -- 172.21.15.69:0/3580618598 >> 172.21.15.69:0/3580618598 conn(0x7faf700fd5f0 msgr2=0x7faf700ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 -- 172.21.15.69:0/3580618598 shutdown_connections 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.399+0000 7faf76c3f640 1 -- 172.21.15.69:0/3580618598 wait complete. 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf76c3f640 1 Processor -- start 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf76c3f640 1 -- start start 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf76c3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701018b0 0x7faf70198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf76c3f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70102250 0x7faf70198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf76c3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf701078b0 0x7faf7019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf76c3f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faf70111cd0 con 0x7faf701018b0 2024-08-02T17:45:19.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf76c3f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf70111b50 con 0x7faf70102250 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf76c3f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf70111e50 con 0x7faf701078b0 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.400+0000 7faf749db640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701018b0 0x7faf70198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf749db640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701018b0 0x7faf70198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34182/0 (socket says 172.21.15.69:34182) 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf749db640 1 -- 172.21.15.69:0/2956863515 learned_addr learned my addr 172.21.15.69:0/2956863515 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf6ffff640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70102250 0x7faf70198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf751dc640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf701078b0 0x7faf7019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf749db640 1 -- 172.21.15.69:0/2956863515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf701078b0 msgr2=0x7faf7019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf749db640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf701078b0 0x7faf7019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf749db640 1 -- 172.21.15.69:0/2956863515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70102250 msgr2=0x7faf70198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf749db640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf70102250 0x7faf70198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf749db640 1 -- 172.21.15.69:0/2956863515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7019d1e0 con 0x7faf701018b0 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf751dc640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf701078b0 0x7faf7019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf749db640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701018b0 0x7faf70198230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf64009230 tx=0x7faf64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:19.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6403f030 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf64003ad0 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7019d470 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf64035070 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.401+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf701a4db0 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.402+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf700626b0 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.403+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faf64042030 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.404+0000 7faf6dffb640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faf600a2c90 0x7faf600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.404+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7faf640efde0 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.404+0000 7faf6ffff640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faf600a2c90 0x7faf600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.404+0000 7faf6dffb640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] conn(0x7faf600ade60 0x7faf600b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.404+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 --> [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf600b09d0 con 0x7faf600ade60 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.404+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7faf640f0230 con 0x7faf701018b0 2024-08-02T17:45:19.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.405+0000 7faf6ffff640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faf600a2c90 0x7faf600a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faf700ffbe0 tx=0x7faf5c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:19.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.405+0000 7faf751dc640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] conn(0x7faf600ade60 0x7faf600b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.405+0000 7faf751dc640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] conn(0x7faf600ade60 0x7faf600b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:19.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.406+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 <== osd.10 v2:172.21.15.77:6824/2744845863 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf600b09d0 con 0x7faf600ade60 2024-08-02T17:45:19.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.419+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 --> [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf70062ab0 con 0x7faf600ade60 2024-08-02T17:45:19.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.420+0000 7faf6dffb640 1 -- 172.21.15.69:0/2956863515 <== osd.10 v2:172.21.15.77:6824/2744845863 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faf70062ab0 con 0x7faf600ade60 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.420+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 >> [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] conn(0x7faf600ade60 msgr2=0x7faf600b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.420+0000 7faf76c3f640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] conn(0x7faf600ade60 0x7faf600b0300 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.420+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faf600a2c90 msgr2=0x7faf600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.420+0000 7faf76c3f640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7faf600a2c90 0x7faf600a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faf700ffbe0 tx=0x7faf5c009110 comp rx=0 tx=0).stop 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.420+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701018b0 msgr2=0x7faf70198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.420+0000 7faf76c3f640 1 --2- 172.21.15.69:0/2956863515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf701018b0 0x7faf70198230 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf64009230 tx=0x7faf64004270 comp rx=0 tx=0).stop 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.420+0000 7faf751dc640 1 -- 172.21.15.69:0/2956863515 reap_dead start 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.421+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 shutdown_connections 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.421+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 >> 172.21.15.69:0/2956863515 conn(0x7faf700fd5f0 msgr2=0x7faf700fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:19.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.421+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 shutdown_connections 2024-08-02T17:45:19.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.421+0000 7faf76c3f640 1 -- 172.21.15.69:0/2956863515 wait complete. 2024-08-02T17:45:19.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 -- 172.21.15.69:0/1335233865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c102250 msgr2=0x7f530c10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 --2- 172.21.15.69:0/1335233865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c102250 0x7f530c10e780 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5304009cd0 tx=0x7f530402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 -- 172.21.15.69:0/1335233865 shutdown_connections 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 --2- 172.21.15.69:0/1335233865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c102250 0x7f530c10e780 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 --2- 172.21.15.69:0/1335233865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f530c1018b0 0x7f530c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 --2- 172.21.15.69:0/1335233865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530c1078b0 0x7f530c107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 -- 172.21.15.69:0/1335233865 >> 172.21.15.69:0/1335233865 conn(0x7f530c0fd5b0 msgr2=0x7f530c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 -- 172.21.15.69:0/1335233865 shutdown_connections 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.616+0000 7f5313c60640 1 -- 172.21.15.69:0/1335233865 wait complete. 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f5313c60640 1 Processor -- start 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f5313c60640 1 -- start start 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f5313c60640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c1018b0 0x7f530c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f5313c60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530c102250 0x7f530c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f5313c60640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f530c1078b0 0x7f530c19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f5313c60640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f530c111cb0 con 0x7f530c1018b0 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f5313c60640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f530c111b30 con 0x7f530c1078b0 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f5313c60640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f530c111e30 con 0x7f530c102250 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f53119fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c1018b0 0x7f530c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f53119fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c1018b0 0x7f530c198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34194/0 (socket says 172.21.15.69:34194) 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.617+0000 7f53119fc640 1 -- 172.21.15.69:0/934079523 learned_addr learned my addr 172.21.15.69:0/934079523 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53111fb640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530c102250 0x7f530c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53121fd640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f530c1078b0 0x7f530c19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53119fc640 1 -- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530c102250 msgr2=0x7f530c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53119fc640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530c102250 0x7f530c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53119fc640 1 -- 172.21.15.69:0/934079523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f530c1078b0 msgr2=0x7f530c19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53119fc640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f530c1078b0 0x7f530c19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53119fc640 1 -- 172.21.15.69:0/934079523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530c19d0e0 con 0x7f530c1018b0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53121fd640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f530c1078b0 0x7f530c19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53111fb640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f530c102250 0x7f530c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f53119fc640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c1018b0 0x7f530c198130 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f530800d770 tx=0x7f530800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308017070 con 0x7f530c1018b0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530c19d3d0 con 0x7f530c1018b0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5308005500 con 0x7f530c1018b0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308004e00 con 0x7f530c1018b0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.618+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f530c1a4cb0 con 0x7f530c1018b0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.619+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f530c0626b0 con 0x7f530c1018b0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.620+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5308004fa0 con 0x7f530c1018b0 2024-08-02T17:45:19.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.621+0000 7f5302ffd640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52f80a2c10 0x7f52f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.621+0000 7f53111fb640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52f80a2c10 0x7f52f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.621+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f53080cf3c0 con 0x7f530c1018b0 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.621+0000 7f5302ffd640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] conn(0x7f52f80add60 0x7f52f80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.621+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 --> [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5308001d30 con 0x7f52f80add60 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.621+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f53080cf810 con 0x7f530c1018b0 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.621+0000 7f53111fb640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52f80a2c10 0x7f52f80a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f530c106f30 tx=0x7f52fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.621+0000 7f53121fd640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] conn(0x7f52f80add60 0x7f52f80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.622+0000 7f53121fd640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] conn(0x7f52f80add60 0x7f52f80b0200 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:19.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.623+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 <== osd.11 v2:172.21.15.83:6824/697474506 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5308001d30 con 0x7f52f80add60 2024-08-02T17:45:19.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.636+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 --> [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f530c062b30 con 0x7f52f80add60 2024-08-02T17:45:19.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.637+0000 7f5302ffd640 1 -- 172.21.15.69:0/934079523 <== osd.11 v2:172.21.15.83:6824/697474506 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f530c062b30 con 0x7f52f80add60 2024-08-02T17:45:19.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.637+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] conn(0x7f52f80add60 msgr2=0x7f52f80b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.637+0000 7f5313c60640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] conn(0x7f52f80add60 0x7f52f80b0200 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52f80a2c10 msgr2=0x7f52f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f5313c60640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52f80a2c10 0x7f52f80a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f530c106f30 tx=0x7f52fc005f90 comp rx=0 tx=0).stop 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c1018b0 msgr2=0x7f530c198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f5313c60640 1 --2- 172.21.15.69:0/934079523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f530c1018b0 0x7f530c198130 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f530800d770 tx=0x7f530800dc30 comp rx=0 tx=0).stop 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f53121fd640 1 -- 172.21.15.69:0/934079523 reap_dead start 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 shutdown_connections 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 >> 172.21.15.69:0/934079523 conn(0x7f530c0fd5b0 msgr2=0x7f530c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 shutdown_connections 2024-08-02T17:45:19.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.638+0000 7f5313c60640 1 -- 172.21.15.69:0/934079523 wait complete. 2024-08-02T17:45:20.018 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:19 smithi069 ceph-mon[32746]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-08-02T17:45:20.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:19 smithi077 ceph-mon[37108]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-08-02T17:45:20.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:19 smithi083 ceph-mon[37031]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-08-02T17:45:20.491 INFO:teuthology.orchestra.run.smithi069.stdout:77309411396 2024-08-02T17:45:20.492 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-02T17:45:20.927 INFO:teuthology.orchestra.run.smithi069.stdout:193273528370 2024-08-02T17:45:20.927 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-02T17:45:21.327 INFO:teuthology.orchestra.run.smithi069.stdout:103079215167 2024-08-02T17:45:21.328 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-02T17:45:21.853 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:21 smithi069 ceph-mon[32746]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-02T17:45:22.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 -- 172.21.15.69:0/3977376280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c10bea0 msgr2=0x7f0b2c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 --2- 172.21.15.69:0/3977376280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c10bea0 0x7f0b2c10e340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009ca0 tx=0x7f0b2402a950 comp rx=0 tx=0).stop 2024-08-02T17:45:22.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 -- 172.21.15.69:0/3977376280 shutdown_connections 2024-08-02T17:45:22.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 --2- 172.21.15.69:0/3977376280 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c10bea0 0x7f0b2c10e340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 --2- 172.21.15.69:0/3977376280 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b2c103390 0x7f0b2c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 --2- 172.21.15.69:0/3977376280 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c102a70 0x7f0b2c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 -- 172.21.15.69:0/3977376280 >> 172.21.15.69:0/3977376280 conn(0x7f0b2c0fc4d0 msgr2=0x7f0b2c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:22.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 -- 172.21.15.69:0/3977376280 shutdown_connections 2024-08-02T17:45:22.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.044+0000 7f0b33f3f640 1 -- 172.21.15.69:0/3977376280 wait complete. 2024-08-02T17:45:22.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b33f3f640 1 Processor -- start 2024-08-02T17:45:22.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b33f3f640 1 -- start start 2024-08-02T17:45:22.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b33f3f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c102a70 0x7f0b2c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b33f3f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b2c103390 0x7f0b2c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b33f3f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c10bea0 0x7f0b2c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b33f3f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c111c60 con 0x7f0b2c102a70 2024-08-02T17:45:22.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b33f3f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c111ae0 con 0x7f0b2c103390 2024-08-02T17:45:22.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b33f3f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2c111de0 con 0x7f0b2c10bea0 2024-08-02T17:45:22.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b31cdb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c102a70 0x7f0b2c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b31cdb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c102a70 0x7f0b2c198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34230/0 (socket says 172.21.15.69:34230) 2024-08-02T17:45:22.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b31cdb640 1 -- 172.21.15.69:0/2734748977 learned_addr learned my addr 172.21.15.69:0/2734748977 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:22.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b314da640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b2c103390 0x7f0b2c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b324dc640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c10bea0 0x7f0b2c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b31cdb640 1 -- 172.21.15.69:0/2734748977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c10bea0 msgr2=0x7f0b2c19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b31cdb640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2c10bea0 0x7f0b2c19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b31cdb640 1 -- 172.21.15.69:0/2734748977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b2c103390 msgr2=0x7f0b2c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b31cdb640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0b2c103390 0x7f0b2c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.045+0000 7f0b31cdb640 1 -- 172.21.15.69:0/2734748977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2c19d0f0 con 0x7f0b2c102a70 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.046+0000 7f0b31cdb640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c102a70 0x7f0b2c198140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00c890 tx=0x7f0b1c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.046+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c004220 con 0x7f0b2c102a70 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.046+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b1c0043c0 con 0x7f0b2c102a70 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.046+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2c19d3e0 con 0x7f0b2c102a70 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.046+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c0049e0 con 0x7f0b2c102a70 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.046+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b2c1a4cc0 con 0x7f0b2c102a70 2024-08-02T17:45:22.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.047+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b2c0626b0 con 0x7f0b2c102a70 2024-08-02T17:45:22.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.048+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0b1c0079b0 con 0x7f0b2c102a70 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.049+0000 7f0b22ffd640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0b180a2c10 0x7f0b180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.049+0000 7f0b314da640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0b180a2c10 0x7f0b180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.049+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0b1c0cf6c0 con 0x7f0b2c102a70 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.049+0000 7f0b22ffd640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] conn(0x7f0b180adde0 0x7f0b180b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.049+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 --> [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b180b0950 con 0x7f0b180adde0 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.049+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c0cfb10 con 0x7f0b2c102a70 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.049+0000 7f0b314da640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0b180a2c10 0x7f0b180a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0fea60 tx=0x7f0b2800b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.049+0000 7f0b324dc640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] conn(0x7f0b180adde0 0x7f0b180b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.050+0000 7f0b324dc640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] conn(0x7f0b180adde0 0x7f0b180b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:22.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.051+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 <== osd.8 v2:172.21.15.83:6816/1048275845 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b180b0950 con 0x7f0b180adde0 2024-08-02T17:45:22.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 --> [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b2c062ab0 con 0x7f0b180adde0 2024-08-02T17:45:22.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b22ffd640 1 -- 172.21.15.69:0/2734748977 <== osd.8 v2:172.21.15.83:6816/1048275845 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b2c062ab0 con 0x7f0b180adde0 2024-08-02T17:45:22.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 >> [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] conn(0x7f0b180adde0 msgr2=0x7f0b180b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b33f3f640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] conn(0x7f0b180adde0 0x7f0b180b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0b180a2c10 msgr2=0x7f0b180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b33f3f640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0b180a2c10 0x7f0b180a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0fea60 tx=0x7f0b2800b2e0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c102a70 msgr2=0x7f0b2c198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b33f3f640 1 --2- 172.21.15.69:0/2734748977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0b2c102a70 0x7f0b2c198140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00c890 tx=0x7f0b1c00cd50 comp rx=0 tx=0).stop 2024-08-02T17:45:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.065+0000 7f0b324dc640 1 -- 172.21.15.69:0/2734748977 reap_dead start 2024-08-02T17:45:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.066+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 shutdown_connections 2024-08-02T17:45:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.066+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 >> 172.21.15.69:0/2734748977 conn(0x7f0b2c0fc4d0 msgr2=0x7f0b2c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.066+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 shutdown_connections 2024-08-02T17:45:22.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.066+0000 7f0b33f3f640 1 -- 172.21.15.69:0/2734748977 wait complete. 2024-08-02T17:45:22.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:21 smithi077 ceph-mon[37108]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-02T17:45:22.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 -- 172.21.15.69:0/3134364386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 msgr2=0x7f2548107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 --2- 172.21.15.69:0/3134364386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 0x7f2548107c90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2544009930 tx=0x7f254402a420 comp rx=0 tx=0).stop 2024-08-02T17:45:22.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 -- 172.21.15.69:0/3134364386 shutdown_connections 2024-08-02T17:45:22.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 --2- 172.21.15.69:0/3134364386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548102250 0x7f254810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 --2- 172.21.15.69:0/3134364386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25481018b0 0x7f2548101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 --2- 172.21.15.69:0/3134364386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 0x7f2548107c90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 -- 172.21.15.69:0/3134364386 >> 172.21.15.69:0/3134364386 conn(0x7f25480fd5f0 msgr2=0x7f25480ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 -- 172.21.15.69:0/3134364386 shutdown_connections 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.229+0000 7f25501f7640 1 -- 172.21.15.69:0/3134364386 wait complete. 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f25501f7640 1 Processor -- start 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f25501f7640 1 -- start start 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f25501f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25481018b0 0x7f254819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f25501f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548102250 0x7f254819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f25501f7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 0x7f254819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f25501f7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2548111cb0 con 0x7f25481078b0 2024-08-02T17:45:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f25501f7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2548111b30 con 0x7f25481018b0 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f25501f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2548111e30 con 0x7f2548102250 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254e794640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 0x7f254819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254e794640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 0x7f254819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34250/0 (socket says 172.21.15.69:34250) 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254e794640 1 -- 172.21.15.69:0/3110649296 learned_addr learned my addr 172.21.15.69:0/3110649296 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254df93640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25481018b0 0x7f254819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254d792640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548102250 0x7f254819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254e794640 1 -- 172.21.15.69:0/3110649296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548102250 msgr2=0x7f254819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254e794640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548102250 0x7f254819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254e794640 1 -- 172.21.15.69:0/3110649296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25481018b0 msgr2=0x7f254819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254e794640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25481018b0 0x7f254819a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254e794640 1 -- 172.21.15.69:0/3110649296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254819f340 con 0x7f25481078b0 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.230+0000 7f254d792640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2548102250 0x7f254819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.231+0000 7f254e794640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 0x7f254819ec60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f254000cad0 tx=0x7f254000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.231+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254000ae20 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.231+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2540005020 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.231+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254819f630 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.231+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254000f6f0 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.231+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25481a6f10 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.232+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f254810de70 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.233+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f25400051d0 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f253effd640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f25340a2c90 0x7f25340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f254df93640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f25340a2c90 0x7f25340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f25400cf930 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f253effd640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] conn(0x7f25340ade60 0x7f25340b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 --> [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2540012ec0 con 0x7f25340ade60 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f25400cfd80 con 0x7f25481078b0 2024-08-02T17:45:22.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f254d792640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] conn(0x7f25340ade60 0x7f25340b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:22.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f254df93640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f25340a2c90 0x7f25340a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2544009900 tx=0x7f25440058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:22.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.234+0000 7f254d792640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] conn(0x7f25340ade60 0x7f25340b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:22.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.235+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 <== osd.1 v2:172.21.15.77:6800/1438822900 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2540012ec0 con 0x7f25340ade60 2024-08-02T17:45:22.250 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.249+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 --> [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2548102ff0 con 0x7f25340ade60 2024-08-02T17:45:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f253effd640 1 -- 172.21.15.69:0/3110649296 <== osd.1 v2:172.21.15.77:6800/1438822900 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2548102ff0 con 0x7f25340ade60 2024-08-02T17:45:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 >> [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] conn(0x7f25340ade60 msgr2=0x7f25340b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] conn(0x7f25340ade60 0x7f25340b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f25340a2c90 msgr2=0x7f25340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f25340a2c90 0x7f25340a5150 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2544009900 tx=0x7f25440058d0 comp rx=0 tx=0).stop 2024-08-02T17:45:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 msgr2=0x7f254819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 --2- 172.21.15.69:0/3110649296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25481078b0 0x7f254819ec60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f254000cad0 tx=0x7f254000cf90 comp rx=0 tx=0).stop 2024-08-02T17:45:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f254e794640 1 -- 172.21.15.69:0/3110649296 reap_dead start 2024-08-02T17:45:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 shutdown_connections 2024-08-02T17:45:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 >> 172.21.15.69:0/3110649296 conn(0x7f25480fd5f0 msgr2=0x7f254810d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 shutdown_connections 2024-08-02T17:45:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:22.250+0000 7f25501f7640 1 -- 172.21.15.69:0/3110649296 wait complete. 2024-08-02T17:45:22.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:21 smithi083 ceph-mon[37031]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-02T17:45:22.907 INFO:teuthology.orchestra.run.smithi069.stdout:249108103210 2024-08-02T17:45:22.907 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-02T17:45:23.547 INFO:teuthology.orchestra.run.smithi069.stdout:55834574921 2024-08-02T17:45:23.547 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 -- 172.21.15.69:0/2988304325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c100b50 msgr2=0x7f089c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 --2- 172.21.15.69:0/2988304325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c100b50 0x7f089c100f30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f088c009900 tx=0x7f088c02a480 comp rx=0 tx=0).stop 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 -- 172.21.15.69:0/2988304325 shutdown_connections 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 --2- 172.21.15.69:0/2988304325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f089c102c20 0x7f089c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 --2- 172.21.15.69:0/2988304325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c1022a0 0x7f089c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 --2- 172.21.15.69:0/2988304325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c100b50 0x7f089c100f30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 -- 172.21.15.69:0/2988304325 >> 172.21.15.69:0/2988304325 conn(0x7f089c0fc500 msgr2=0x7f089c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 -- 172.21.15.69:0/2988304325 shutdown_connections 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.802+0000 7f08a3446640 1 -- 172.21.15.69:0/2988304325 wait complete. 2024-08-02T17:45:23.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a3446640 1 Processor -- start 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a3446640 1 -- start start 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a3446640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f089c100b50 0x7f089c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a3446640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c1022a0 0x7f089c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a3446640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c102c20 0x7f089c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a3446640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f089c110c70 con 0x7f089c1022a0 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a3446640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f089c110af0 con 0x7f089c100b50 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a3446640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f089c110df0 con 0x7f089c102c20 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c1022a0 0x7f089c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c1022a0 0x7f089c198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34206/0 (socket says 172.21.15.69:34206) 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 -- 172.21.15.69:0/1968485475 learned_addr learned my addr 172.21.15.69:0/1968485475 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a11e2640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f089c100b50 0x7f089c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 -- 172.21.15.69:0/1968485475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c102c20 msgr2=0x7f089c19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:23.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f089c102c20 0x7f089c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:23.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 -- 172.21.15.69:0/1968485475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f089c100b50 msgr2=0x7f089c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:23.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f089c100b50 0x7f089c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:23.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 -- 172.21.15.69:0/1968485475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c19d200 con 0x7f089c1022a0 2024-08-02T17:45:23.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a11e2640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f089c100b50 0x7f089c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:23.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.803+0000 7f08a09e1640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c1022a0 0x7f089c198790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f089800c920 tx=0x7f089800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:23.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.804+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898007c70 con 0x7f089c1022a0 2024-08-02T17:45:23.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.804+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0898007e10 con 0x7f089c1022a0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.804+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c19d4f0 con 0x7f089c1022a0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.804+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898018890 con 0x7f089c1022a0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.804+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c1a4dd0 con 0x7f089c1022a0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.804+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f089c0626b0 con 0x7f089c1022a0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.806+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0898018a30 con 0x7f089c1022a0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.806+0000 7f08927fc640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f08880a2c90 0x7f08880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.806+0000 7f08a11e2640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f08880a2c90 0x7f08880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.806+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f08980cfac0 con 0x7f089c1022a0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.807+0000 7f08927fc640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] conn(0x7f08880ade60 0x7f08880b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.807+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 --> [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08880b09d0 con 0x7f08880ade60 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.807+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f08980cff10 con 0x7f089c1022a0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.807+0000 7f08a19e3640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] conn(0x7f08880ade60 0x7f08880b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:23.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.807+0000 7f08a11e2640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f08880a2c90 0x7f08880a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f088c000c00 tx=0x7f088c004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.807+0000 7f08a19e3640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] conn(0x7f08880ade60 0x7f08880b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.808+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 <== osd.0 v2:172.21.15.69:6802/3281611290 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08880b09d0 con 0x7f08880ade60 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.821+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 --> [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f089c062ab0 con 0x7f08880ade60 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.821+0000 7f08927fc640 1 -- 172.21.15.69:0/1968485475 <== osd.0 v2:172.21.15.69:6802/3281611290 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f089c062ab0 con 0x7f08880ade60 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] conn(0x7f08880ade60 msgr2=0x7f08880b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] conn(0x7f08880ade60 0x7f08880b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f08880a2c90 msgr2=0x7f08880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f08880a2c90 0x7f08880a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f088c000c00 tx=0x7f088c004100 comp rx=0 tx=0).stop 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c1022a0 msgr2=0x7f089c198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 --2- 172.21.15.69:0/1968485475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f089c1022a0 0x7f089c198790 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f089800c920 tx=0x7f089800cde0 comp rx=0 tx=0).stop 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a19e3640 1 -- 172.21.15.69:0/1968485475 reap_dead start 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 shutdown_connections 2024-08-02T17:45:23.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 >> 172.21.15.69:0/1968485475 conn(0x7f089c0fc500 msgr2=0x7f089c069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:23.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 shutdown_connections 2024-08-02T17:45:23.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:19.822+0000 7f08a3446640 1 -- 172.21.15.69:0/1968485475 wait complete. 2024-08-02T17:45:24.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:23 smithi077 ceph-mon[37108]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-02T17:45:24.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:23 smithi069 ceph-mon[32746]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-02T17:45:24.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:23 smithi083 ceph-mon[37031]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-02T17:45:24.306 INFO:teuthology.orchestra.run.smithi069.stdout:34359738445 2024-08-02T17:45:24.306 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-02T17:45:24.668 INFO:teuthology.orchestra.run.smithi069.stdout:283467841573 2024-08-02T17:45:24.669 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-02T17:45:25.323 INFO:teuthology.orchestra.run.smithi069.stdout:304942678048 2024-08-02T17:45:25.323 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-02T17:45:25.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.039+0000 7f03f7a0a640 1 -- 172.21.15.69:0/482126148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0103410 msgr2=0x7f03f01078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.039+0000 7f03f7a0a640 1 --2- 172.21.15.69:0/482126148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0103410 0x7f03f01078c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009930 tx=0x7f03e402a420 comp rx=0 tx=0).stop 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.039+0000 7f03f7a0a640 1 -- 172.21.15.69:0/482126148 shutdown_connections 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.039+0000 7f03f7a0a640 1 --2- 172.21.15.69:0/482126148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0103410 0x7f03f01078c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.039+0000 7f03f7a0a640 1 --2- 172.21.15.69:0/482126148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f0107fd0 0x7f03f010e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.039+0000 7f03f7a0a640 1 --2- 172.21.15.69:0/482126148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03f0102a60 0x7f03f0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.039+0000 7f03f7a0a640 1 -- 172.21.15.69:0/482126148 >> 172.21.15.69:0/482126148 conn(0x7f03f00fc500 msgr2=0x7f03f00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.040+0000 7f03f7a0a640 1 -- 172.21.15.69:0/482126148 shutdown_connections 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.040+0000 7f03f7a0a640 1 -- 172.21.15.69:0/482126148 wait complete. 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.040+0000 7f03f7a0a640 1 Processor -- start 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f7a0a640 1 -- start start 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f7a0a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03f0102a60 0x7f03f0198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:25.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f7a0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f0103410 0x7f03f01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f7a0a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0107fd0 0x7f03f019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f7a0a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0111dd0 con 0x7f03f0107fd0 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f7a0a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0111c50 con 0x7f03f0102a60 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f7a0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03f0111f50 con 0x7f03f0103410 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f5fa7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0107fd0 0x7f03f019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f5fa7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0107fd0 0x7f03f019cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34096/0 (socket says 172.21.15.69:34096) 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f5fa7640 1 -- 172.21.15.69:0/3774260950 learned_addr learned my addr 172.21.15.69:0/3774260950 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f57a6640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03f0102a60 0x7f03f0198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f4fa5640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f0103410 0x7f03f01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f5fa7640 1 -- 172.21.15.69:0/3774260950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f0103410 msgr2=0x7f03f01988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f5fa7640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f0103410 0x7f03f01988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:25.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f5fa7640 1 -- 172.21.15.69:0/3774260950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03f0102a60 msgr2=0x7f03f0198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f5fa7640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03f0102a60 0x7f03f0198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f5fa7640 1 -- 172.21.15.69:0/3774260950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f019d320 con 0x7f03f0107fd0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.041+0000 7f03f4fa5640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03f0103410 0x7f03f01988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.042+0000 7f03f5fa7640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0107fd0 0x7f03f019cc40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f03e800d750 tx=0x7f03e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.042+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8017070 con 0x7f03f0107fd0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.042+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f019d610 con 0x7f03f0107fd0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.042+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03e8005500 con 0x7f03f0107fd0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.042+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8004d90 con 0x7f03f0107fd0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.042+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f01a4ef0 con 0x7f03f0107fd0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.043+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03f0062630 con 0x7f03f0107fd0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.044+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f03e8004f30 con 0x7f03f0107fd0 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.045+0000 7f03ee7fc640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f03d80a2c10 0x7f03d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:25.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.045+0000 7f03f57a6640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f03d80a2c10 0x7f03d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.045+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f03e80cf260 con 0x7f03f0107fd0 2024-08-02T17:45:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.045+0000 7f03ee7fc640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] conn(0x7f03d80add60 0x7f03d80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.045+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 --> [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03e8001d30 con 0x7f03d80add60 2024-08-02T17:45:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.045+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f03e80cf6b0 con 0x7f03f0107fd0 2024-08-02T17:45:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.045+0000 7f03f57a6640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f03d80a2c10 0x7f03d80a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f03e0005fd0 tx=0x7f03e0009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.045+0000 7f03f4fa5640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] conn(0x7f03d80add60 0x7f03d80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.046+0000 7f03f4fa5640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] conn(0x7f03d80add60 0x7f03d80b0200 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:25.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.046+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 <== osd.7 v2:172.21.15.77:6816/2324182206 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03e8001d30 con 0x7f03d80add60 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.060+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 --> [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03f0062ab0 con 0x7f03d80add60 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03ee7fc640 1 -- 172.21.15.69:0/3774260950 <== osd.7 v2:172.21.15.77:6816/2324182206 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f03f0062ab0 con 0x7f03d80add60 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 >> [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] conn(0x7f03d80add60 msgr2=0x7f03d80b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] conn(0x7f03d80add60 0x7f03d80b0200 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f03d80a2c10 msgr2=0x7f03d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f03d80a2c10 0x7f03d80a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f03e0005fd0 tx=0x7f03e0009680 comp rx=0 tx=0).stop 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0107fd0 msgr2=0x7f03f019cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 --2- 172.21.15.69:0/3774260950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03f0107fd0 0x7f03f019cc40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f03e800d750 tx=0x7f03e800dc10 comp rx=0 tx=0).stop 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f5fa7640 1 -- 172.21.15.69:0/3774260950 reap_dead start 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 shutdown_connections 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 >> 172.21.15.69:0/3774260950 conn(0x7f03f00fc500 msgr2=0x7f03f01060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 shutdown_connections 2024-08-02T17:45:25.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:17.061+0000 7f03f7a0a640 1 -- 172.21.15.69:0/3774260950 wait complete. 2024-08-02T17:45:25.988 INFO:teuthology.orchestra.run.smithi069.stdout:214748364845 2024-08-02T17:45:25.989 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-02T17:45:26.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:25 smithi077 ceph-mon[37108]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:25 smithi069 ceph-mon[32746]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:26.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:25 smithi083 ceph-mon[37031]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:26.443 INFO:teuthology.orchestra.run.smithi069.stdout:335007449116 2024-08-02T17:45:26.443 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-02T17:45:26.883 INFO:teuthology.orchestra.run.smithi069.stdout:158913790006 2024-08-02T17:45:26.883 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-02T17:45:27.315 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:27.358 INFO:teuthology.orchestra.run.smithi069.stdout:137438953530 2024-08-02T17:45:27.358 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-02T17:45:27.441 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:27.442 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:27.446 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:27.446 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:27.447 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:28.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:27 smithi069 ceph-mon[32746]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:28.178 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:28.178 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:27 smithi083 ceph-mon[37031]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:28.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:27 smithi077 ceph-mon[37108]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:29.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:28 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1546615997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-02T17:45:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:28 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1546615997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-02T17:45:29.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:28 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1546615997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-08-02T17:45:29.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 -- 172.21.15.69:0/622185150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 msgr2=0x7fca04100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/622185150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 0x7fca04100f30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fca00009900 tx=0x7fca0002a480 comp rx=0 tx=0).stop 2024-08-02T17:45:29.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 -- 172.21.15.69:0/622185150 shutdown_connections 2024-08-02T17:45:29.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/622185150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca04102c20 0x7fca0410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/622185150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca041022a0 0x7fca041026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/622185150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 0x7fca04100f30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 -- 172.21.15.69:0/622185150 >> 172.21.15.69:0/622185150 conn(0x7fca040fc500 msgr2=0x7fca040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 -- 172.21.15.69:0/622185150 shutdown_connections 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 -- 172.21.15.69:0/622185150 wait complete. 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.362+0000 7fca0b3dc640 1 Processor -- start 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca0b3dc640 1 -- start start 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca0b3dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 0x7fca04198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca0b3dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca041022a0 0x7fca04198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca0b3dc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca04102c20 0x7fca0419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca0b3dc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca04110cb0 con 0x7fca04100b50 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca0b3dc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca04110b30 con 0x7fca04102c20 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca0b3dc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca04110e30 con 0x7fca041022a0 2024-08-02T17:45:29.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 0x7fca04198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 0x7fca04198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39460/0 (socket says 172.21.15.69:39460) 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 -- 172.21.15.69:0/471382190 learned_addr learned my addr 172.21.15.69:0/471382190 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca08977640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca041022a0 0x7fca04198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09979640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca04102c20 0x7fca0419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 -- 172.21.15.69:0/471382190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca041022a0 msgr2=0x7fca04198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca041022a0 0x7fca04198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 -- 172.21.15.69:0/471382190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca04102c20 msgr2=0x7fca0419cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca04102c20 0x7fca0419cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 -- 172.21.15.69:0/471382190 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0419d200 con 0x7fca04100b50 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.363+0000 7fca09178640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 0x7fca04198250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca00006010 tx=0x7fca00003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.364+0000 7fc9fa7fc640 1 -- 172.21.15.69:0/471382190 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0003f070 con 0x7fca04100b50 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.364+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0419d490 con 0x7fca04100b50 2024-08-02T17:45:29.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.364+0000 7fc9fa7fc640 1 -- 172.21.15.69:0/471382190 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca00031c30 con 0x7fca04100b50 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.364+0000 7fc9fa7fc640 1 -- 172.21.15.69:0/471382190 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca00035030 con 0x7fca04100b50 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.364+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca041a4dd0 con 0x7fca04100b50 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.365+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca04101ee0 con 0x7fca04100b50 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.366+0000 7fc9fa7fc640 1 -- 172.21.15.69:0/471382190 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca00042070 con 0x7fca04100b50 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.367+0000 7fc9fa7fc640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9f00a2c10 0x7fc9f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.367+0000 7fca08977640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9f00a2c10 0x7fc9f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.367+0000 7fc9fa7fc640 1 -- 172.21.15.69:0/471382190 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fca000efb30 con 0x7fca04100b50 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.367+0000 7fca08977640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9f00a2c10 0x7fc9f00a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca0406acf0 tx=0x7fc9f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:29.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.370+0000 7fc9fa7fc640 1 -- 172.21.15.69:0/471382190 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fca000bc570 con 0x7fca04100b50 2024-08-02T17:45:29.509 INFO:teuthology.orchestra.run.smithi069.stdout:77309411398 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 -- 172.21.15.69:0/785475055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f20100b50 msgr2=0x7f7f20100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 --2- 172.21.15.69:0/785475055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f20100b50 0x7f7f20100f30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009ca0 tx=0x7f7f1002a950 comp rx=0 tx=0).stop 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 -- 172.21.15.69:0/785475055 shutdown_connections 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 --2- 172.21.15.69:0/785475055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f20102c20 0x7f7f2010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 --2- 172.21.15.69:0/785475055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f201022a0 0x7f7f201026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 --2- 172.21.15.69:0/785475055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f20100b50 0x7f7f20100f30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 -- 172.21.15.69:0/785475055 >> 172.21.15.69:0/785475055 conn(0x7f7f200fc500 msgr2=0x7f7f200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 -- 172.21.15.69:0/785475055 shutdown_connections 2024-08-02T17:45:29.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.943+0000 7f7f25c64640 1 -- 172.21.15.69:0/785475055 wait complete. 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f25c64640 1 Processor -- start 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f25c64640 1 -- start start 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f25c64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f20100b50 0x7f7f201981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f25c64640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f201022a0 0x7f7f20198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f25c64640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f20102c20 0x7f7f2019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f25c64640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20110cb0 con 0x7f7f201022a0 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f25c64640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20110b30 con 0x7f7f20102c20 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f25c64640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f20110e30 con 0x7f7f20100b50 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f1effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f201022a0 0x7f7f20198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f1f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f20100b50 0x7f7f201981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f1effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f201022a0 0x7f7f20198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39434/0 (socket says 172.21.15.69:39434) 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f1effd640 1 -- 172.21.15.69:0/1546615997 learned_addr learned my addr 172.21.15.69:0/1546615997 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:29.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.944+0000 7f7f1ffff640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f20102c20 0x7f7f2019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1effd640 1 -- 172.21.15.69:0/1546615997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f20100b50 msgr2=0x7f7f201981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1effd640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f20100b50 0x7f7f201981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1effd640 1 -- 172.21.15.69:0/1546615997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f20102c20 msgr2=0x7f7f2019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1effd640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f20102c20 0x7f7f2019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1effd640 1 -- 172.21.15.69:0/1546615997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2019d170 con 0x7f7f201022a0 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1ffff640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f20102c20 0x7f7f2019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1effd640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f201022a0 0x7f7f20198700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800c920 tx=0x7f7f0800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1cff9640 1 -- 172.21.15.69:0/1546615997 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08007c70 con 0x7f7f201022a0 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1cff9640 1 -- 172.21.15.69:0/1546615997 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f08007e10 con 0x7f7f201022a0 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2019d460 con 0x7f7f201022a0 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f1cff9640 1 -- 172.21.15.69:0/1546615997 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08018890 con 0x7f7f201022a0 2024-08-02T17:45:29.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.945+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f201a4d40 con 0x7f7f201022a0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.946+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f20100f30 con 0x7f7f201022a0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.947+0000 7f7f1cff9640 1 -- 172.21.15.69:0/1546615997 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7f08018a70 con 0x7f7f201022a0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.948+0000 7f7f1cff9640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7f0c0a2c90 0x7f7f0c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.948+0000 7f7f1f7fe640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7f0c0a2c90 0x7f7f0c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.948+0000 7f7f1cff9640 1 -- 172.21.15.69:0/1546615997 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f7f080cf620 con 0x7f7f201022a0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.948+0000 7f7f1f7fe640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7f0c0a2c90 0x7f7f0c0a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7f10004ad0 tx=0x7f7f10004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:27.952+0000 7f7f1cff9640 1 -- 172.21.15.69:0/1546615997 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7f08014030 con 0x7f7f201022a0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.129+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7f201a4fa0 con 0x7f7f201022a0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.129+0000 7f7f1cff9640 1 -- 172.21.15.69:0/1546615997 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f0809c0c0 con 0x7f7f201022a0 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7f0c0a2c90 msgr2=0x7f7f0c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7f0c0a2c90 0x7f7f0c0a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7f10004ad0 tx=0x7f7f10004a60 comp rx=0 tx=0).stop 2024-08-02T17:45:29.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f201022a0 msgr2=0x7f7f20198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f201022a0 0x7f7f20198700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7f0800c920 tx=0x7f7f0800cde0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 shutdown_connections 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f7f0c0a2c90 0x7f7f0c0a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7f201022a0 0x7f7f20198700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7f20102c20 0x7f7f2019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 --2- 172.21.15.69:0/1546615997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f20100b50 0x7f7f201981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 >> 172.21.15.69:0/1546615997 conn(0x7f7f200fc500 msgr2=0x7f7f20069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 shutdown_connections 2024-08-02T17:45:29.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:28.131+0000 7f7f25c64640 1 -- 172.21.15.69:0/1546615997 wait complete. 2024-08-02T17:45:29.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.579+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fca041995e0 con 0x7fca04100b50 2024-08-02T17:45:29.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.579+0000 7fc9fa7fc640 1 -- 172.21.15.69:0/471382190 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fca041995e0 con 0x7fca04100b50 2024-08-02T17:45:29.581 INFO:teuthology.orchestra.run.smithi069.stdout:103079215169 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9f00a2c10 msgr2=0x7fc9f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9f00a2c10 0x7fc9f00a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca0406acf0 tx=0x7fc9f4005f90 comp rx=0 tx=0).stop 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 msgr2=0x7fca04198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 0x7fca04198250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fca00006010 tx=0x7fca00003c10 comp rx=0 tx=0).stop 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 shutdown_connections 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fc9f00a2c10 0x7fc9f00a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca04102c20 0x7fca0419cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca041022a0 0x7fca04198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 --2- 172.21.15.69:0/471382190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca04100b50 0x7fca04198250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.581+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 >> 172.21.15.69:0/471382190 conn(0x7fca040fc500 msgr2=0x7fca04105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.582+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 shutdown_connections 2024-08-02T17:45:29.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.582+0000 7fca0b3dc640 1 -- 172.21.15.69:0/471382190 wait complete. 2024-08-02T17:45:29.752 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:29.753 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:29 smithi069 ceph-mon[32746]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:29 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1223815445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-02T17:45:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:29 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/471382190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-02T17:45:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:29 smithi083 ceph-mon[37031]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:29 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1223815445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-02T17:45:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:29 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/471382190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-02T17:45:30.302 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411396 got 77309411398 for osd.2 2024-08-02T17:45:30.302 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:30.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:29 smithi077 ceph-mon[37108]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:29 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1223815445' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-08-02T17:45:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:29 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/471382190' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-08-02T17:45:30.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 -- 172.21.15.69:0/1940617159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710100a10 msgr2=0x7f9710106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:30.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 --2- 172.21.15.69:0/1940617159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710100a10 0x7f9710106550 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9704009930 tx=0x7f970402a420 comp rx=0 tx=0).stop 2024-08-02T17:45:30.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 -- 172.21.15.69:0/1940617159 shutdown_connections 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 --2- 172.21.15.69:0/1940617159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710100a10 0x7f9710106550 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 --2- 172.21.15.69:0/1940617159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9710106c60 0x7f971010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 --2- 172.21.15.69:0/1940617159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9710100060 0x7f9710100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 -- 172.21.15.69:0/1940617159 >> 172.21.15.69:0/1940617159 conn(0x7f97100fbc10 msgr2=0x7f97100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 -- 172.21.15.69:0/1940617159 shutdown_connections 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.931+0000 7f97162e9640 1 -- 172.21.15.69:0/1940617159 wait complete. 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f97162e9640 1 Processor -- start 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f97162e9640 1 -- start start 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f97162e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9710100060 0x7f9710100fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f97162e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9710100a10 0x7f9710101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f97162e9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710106c60 0x7f9710101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f97162e9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9710111cb0 con 0x7f9710106c60 2024-08-02T17:45:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f97162e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9710111b30 con 0x7f9710100a10 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f97162e9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9710111e30 con 0x7f9710100060 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f9714886640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710106c60 0x7f9710101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f970ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9710100060 0x7f9710100fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.932+0000 7f9714886640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710106c60 0x7f9710101a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39492/0 (socket says 172.21.15.69:39492) 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f9714886640 1 -- 172.21.15.69:0/2469118917 learned_addr learned my addr 172.21.15.69:0/2469118917 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f970f7fe640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9710100a10 0x7f9710101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f9714886640 1 -- 172.21.15.69:0/2469118917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9710100060 msgr2=0x7f9710100fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f9714886640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9710100060 0x7f9710100fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f9714886640 1 -- 172.21.15.69:0/2469118917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9710100a10 msgr2=0x7f9710101520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f9714886640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9710100a10 0x7f9710101520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f9714886640 1 -- 172.21.15.69:0/2469118917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97101a66f0 con 0x7f9710106c60 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f9714886640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710106c60 0x7f9710101a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f970800cad0 tx=0x7f970800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f970d7fa640 1 -- 172.21.15.69:0/2469118917 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800ae20 con 0x7f9710106c60 2024-08-02T17:45:30.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f970d7fa640 1 -- 172.21.15.69:0/2469118917 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9708005020 con 0x7f9710106c60 2024-08-02T17:45:30.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97101a69e0 con 0x7f9710106c60 2024-08-02T17:45:30.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f970d7fa640 1 -- 172.21.15.69:0/2469118917 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970800f6f0 con 0x7f9710106c60 2024-08-02T17:45:30.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.933+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97101a6ef0 con 0x7f9710106c60 2024-08-02T17:45:30.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.934+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9710100440 con 0x7f9710106c60 2024-08-02T17:45:30.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.935+0000 7f970d7fa640 1 -- 172.21.15.69:0/2469118917 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f97080051d0 con 0x7f9710106c60 2024-08-02T17:45:30.359 INFO:teuthology.orchestra.run.smithi069.stdout:335007449118 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.936+0000 7f970d7fa640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96f80a2c10 0x7f96f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.936+0000 7f970ffff640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96f80a2c10 0x7f96f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.937+0000 7f970d7fa640 1 -- 172.21.15.69:0/2469118917 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f97080cf810 con 0x7f9710106c60 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.937+0000 7f970ffff640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96f80a2c10 0x7f96f80a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9700005d00 tx=0x7f970000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.941+0000 7f970d7fa640 1 -- 172.21.15.69:0/2469118917 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9708014030 con 0x7f9710106c60 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.113+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f9710109e30 con 0x7f9710106c60 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.114+0000 7f970d7fa640 1 -- 172.21.15.69:0/2469118917 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f970809c2b0 con 0x7f9710106c60 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.115+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96f80a2c10 msgr2=0x7f96f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.115+0000 7f97162e9640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96f80a2c10 0x7f96f80a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9700005d00 tx=0x7f970000b410 comp rx=0 tx=0).stop 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710106c60 msgr2=0x7f9710101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710106c60 0x7f9710101a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f970800cad0 tx=0x7f970800cf90 comp rx=0 tx=0).stop 2024-08-02T17:45:30.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 shutdown_connections 2024-08-02T17:45:30.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f96f80a2c10 0x7f96f80a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9710106c60 0x7f9710101a60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9710100a10 0x7f9710101520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 --2- 172.21.15.69:0/2469118917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9710100060 0x7f9710100fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:30.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 >> 172.21.15.69:0/2469118917 conn(0x7f97100fbc10 msgr2=0x7f97100fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:30.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 shutdown_connections 2024-08-02T17:45:30.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:30.116+0000 7f97162e9640 1 -- 172.21.15.69:0/2469118917 wait complete. 2024-08-02T17:45:30.961 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:30 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2469118917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-02T17:45:30.994 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449116 got 335007449118 for osd.11 2024-08-02T17:45:30.994 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:30 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2469118917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-02T17:45:31.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.263+0000 7f0ec7377640 1 -- 172.21.15.69:0/1985164143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec01022a0 msgr2=0x7f0ec01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:31.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.263+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1985164143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec01022a0 0x7f0ec01026e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009900 tx=0x7f0eb402a480 comp rx=0 tx=0).stop 2024-08-02T17:45:31.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.263+0000 7f0ec7377640 1 -- 172.21.15.69:0/1985164143 shutdown_connections 2024-08-02T17:45:31.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.263+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1985164143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec01022a0 0x7f0ec01026e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.263+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1985164143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec0102c20 0x7f0ec010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.263+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1985164143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0100b50 0x7f0ec0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.263+0000 7f0ec7377640 1 -- 172.21.15.69:0/1985164143 >> 172.21.15.69:0/1985164143 conn(0x7f0ec00fc500 msgr2=0x7f0ec00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:31.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 -- 172.21.15.69:0/1985164143 shutdown_connections 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 -- 172.21.15.69:0/1985164143 wait complete. 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 Processor -- start 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 -- start start 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec0100b50 0x7f0ec00fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec01022a0 0x7f0ec00fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0102c20 0x7f0ec00fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0110cb0 con 0x7f0ec0100b50 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0110b30 con 0x7f0ec01022a0 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec7377640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0110e30 con 0x7f0ec0102c20 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.264+0000 7f0ec5113640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec0100b50 0x7f0ec00fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5113640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec0100b50 0x7f0ec00fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39454/0 (socket says 172.21.15.69:39454) 2024-08-02T17:45:31.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5113640 1 -- 172.21.15.69:0/1223815445 learned_addr learned my addr 172.21.15.69:0/1223815445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:31.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5914640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0102c20 0x7f0ec00fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:31.306 INFO:teuthology.orchestra.run.smithi069.stdout:249108103211 2024-08-02T17:45:31.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec4912640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec01022a0 0x7f0ec00fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:31.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5113640 1 -- 172.21.15.69:0/1223815445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0102c20 msgr2=0x7f0ec00fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5113640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0102c20 0x7f0ec00fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5113640 1 -- 172.21.15.69:0/1223815445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec01022a0 msgr2=0x7f0ec00fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5113640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec01022a0 0x7f0ec00fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5113640 1 -- 172.21.15.69:0/1223815445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec01a66a0 con 0x7f0ec0100b50 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec5113640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec0100b50 0x7f0ec00fc9b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000cce0 tx=0x7f0eb000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ebe7fc640 1 -- 172.21.15.69:0/1223815445 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000f650 con 0x7f0ec0100b50 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec01a6990 con 0x7f0ec0100b50 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ebe7fc640 1 -- 172.21.15.69:0/1223815445 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eb000fcd0 con 0x7f0ec0100b50 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ebe7fc640 1 -- 172.21.15.69:0/1223815445 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0018c70 con 0x7f0ec0100b50 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.265+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec01a6ea0 con 0x7f0ec0100b50 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.266+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ec0100f30 con 0x7f0ec0100b50 2024-08-02T17:45:31.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.267+0000 7f0ebe7fc640 1 -- 172.21.15.69:0/1223815445 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0eb000f7f0 con 0x7f0ec0100b50 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.268+0000 7f0ebe7fc640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0ea80a2c10 0x7f0ea80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.268+0000 7f0ec4912640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0ea80a2c10 0x7f0ea80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.268+0000 7f0ebe7fc640 1 -- 172.21.15.69:0/1223815445 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f0eb00cf930 con 0x7f0ec0100b50 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.269+0000 7f0ec4912640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0ea80a2c10 0x7f0ea80a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009230 tx=0x7f0eb40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.272+0000 7f0ebe7fc640 1 -- 172.21.15.69:0/1223815445 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0eb0014030 con 0x7f0ec0100b50 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.483+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f0ec0109dd0 con 0x7f0ec0100b50 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.484+0000 7f0ebe7fc640 1 -- 172.21.15.69:0/1223815445 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0eb009c3d0 con 0x7f0ec0100b50 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0ea80a2c10 msgr2=0x7f0ea80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0ea80a2c10 0x7f0ea80a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009230 tx=0x7f0eb40058d0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec0100b50 msgr2=0x7f0ec00fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec0100b50 0x7f0ec00fc9b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000cce0 tx=0x7f0eb000a620 comp rx=0 tx=0).stop 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 shutdown_connections 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f0ea80a2c10 0x7f0ea80a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ec0102c20 0x7f0ec00fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ec01022a0 0x7f0ec00fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 --2- 172.21.15.69:0/1223815445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ec0100b50 0x7f0ec00fc9b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:31.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 >> 172.21.15.69:0/1223815445 conn(0x7f0ec00fc500 msgr2=0x7f0ec0069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:31.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 shutdown_connections 2024-08-02T17:45:31.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:29.486+0000 7f0ec7377640 1 -- 172.21.15.69:0/1223815445 wait complete. 2024-08-02T17:45:31.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:30 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2469118917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-08-02T17:45:31.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103210 got 249108103211 for osd.8 2024-08-02T17:45:31.962 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:32.186 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:31 smithi069 ceph-mon[32746]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 -- 172.21.15.69:0/670418046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a01067b0 msgr2=0x7f86a0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 --2- 172.21.15.69:0/670418046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a01067b0 0x7f86a0106b90 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8698009930 tx=0x7f869802a420 comp rx=0 tx=0).stop 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 -- 172.21.15.69:0/670418046 shutdown_connections 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 --2- 172.21.15.69:0/670418046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a0101150 0x7f86a010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 --2- 172.21.15.69:0/670418046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01007b0 0x7f86a0100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 --2- 172.21.15.69:0/670418046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a01067b0 0x7f86a0106b90 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 -- 172.21.15.69:0/670418046 >> 172.21.15.69:0/670418046 conn(0x7f86a00fc4d0 msgr2=0x7f86a00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 -- 172.21.15.69:0/670418046 shutdown_connections 2024-08-02T17:45:32.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.184+0000 7f86a8031640 1 -- 172.21.15.69:0/670418046 wait complete. 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a8031640 1 Processor -- start 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a8031640 1 -- start start 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a8031640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01007b0 0x7f86a019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a8031640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101150 0x7f86a019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a8031640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01067b0 0x7f86a019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a8031640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0110bb0 con 0x7f86a0101150 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a8031640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0110a30 con 0x7f86a01067b0 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a8031640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0110d30 con 0x7f86a01007b0 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a55cc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101150 0x7f86a019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a55cc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101150 0x7f86a019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39536/0 (socket says 172.21.15.69:39536) 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a55cc640 1 -- 172.21.15.69:0/4189766823 learned_addr learned my addr 172.21.15.69:0/4189766823 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a65ce640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01067b0 0x7f86a019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:32.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a5dcd640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01007b0 0x7f86a019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a55cc640 1 -- 172.21.15.69:0/4189766823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01007b0 msgr2=0x7f86a019a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a55cc640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01007b0 0x7f86a019a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a55cc640 1 -- 172.21.15.69:0/4189766823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01067b0 msgr2=0x7f86a019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a55cc640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01067b0 0x7f86a019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.185+0000 7f86a55cc640 1 -- 172.21.15.69:0/4189766823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a019f3a0 con 0x7f86a0101150 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.186+0000 7f86a55cc640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101150 0x7f86a019a930 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f869400d770 tx=0x7f869400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.186+0000 7f869effd640 1 -- 172.21.15.69:0/4189766823 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694017070 con 0x7f86a0101150 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.186+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a019f690 con 0x7f86a0101150 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.186+0000 7f869effd640 1 -- 172.21.15.69:0/4189766823 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8694005500 con 0x7f86a0101150 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.186+0000 7f869effd640 1 -- 172.21.15.69:0/4189766823 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8694004e00 con 0x7f86a0101150 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.186+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a01a6f70 con 0x7f86a0101150 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.187+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a0100c10 con 0x7f86a0101150 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.188+0000 7f869effd640 1 -- 172.21.15.69:0/4189766823 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8694004fa0 con 0x7f86a0101150 2024-08-02T17:45:32.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.189+0000 7f869effd640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f86880a2c10 0x7f86880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:32.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.189+0000 7f86a5dcd640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f86880a2c10 0x7f86880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:32.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.189+0000 7f869effd640 1 -- 172.21.15.69:0/4189766823 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f86940cf390 con 0x7f86a0101150 2024-08-02T17:45:32.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.189+0000 7f86a5dcd640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f86880a2c10 0x7f86880a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8698009900 tx=0x7f86980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:32.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.192+0000 7f869effd640 1 -- 172.21.15.69:0/4189766823 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f869409b780 con 0x7f86a0101150 2024-08-02T17:45:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:31 smithi083 ceph-mon[37031]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-02T17:45:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:31 smithi077 ceph-mon[37108]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-02T17:45:32.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.368+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f86a01a7170 con 0x7f86a0101150 2024-08-02T17:45:32.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.368+0000 7f869effd640 1 -- 172.21.15.69:0/4189766823 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f869409fa80 con 0x7f86a0101150 2024-08-02T17:45:32.370 INFO:teuthology.orchestra.run.smithi069.stdout:214748364848 2024-08-02T17:45:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f86880a2c10 msgr2=0x7f86880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f86880a2c10 0x7f86880a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8698009900 tx=0x7f86980058d0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101150 msgr2=0x7f86a019a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101150 0x7f86a019a930 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f869400d770 tx=0x7f869400dc30 comp rx=0 tx=0).stop 2024-08-02T17:45:32.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 shutdown_connections 2024-08-02T17:45:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f86880a2c10 0x7f86880a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101150 0x7f86a019a930 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01067b0 0x7f86a019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 --2- 172.21.15.69:0/4189766823 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01007b0 0x7f86a019a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 >> 172.21.15.69:0/4189766823 conn(0x7f86a00fc4d0 msgr2=0x7f86a006a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 shutdown_connections 2024-08-02T17:45:32.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.370+0000 7f86a8031640 1 -- 172.21.15.69:0/4189766823 wait complete. 2024-08-02T17:45:32.833 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215167 got 103079215169 for osd.3 2024-08-02T17:45:32.833 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:32 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2585771387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-02T17:45:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:32 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4189766823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-02T17:45:33.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 -- 172.21.15.69:0/3665028665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 msgr2=0x7ff51410e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 --2- 172.21.15.69:0/3665028665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 0x7ff51410e340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009ca0 tx=0x7ff50c02a950 comp rx=0 tx=0).stop 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 -- 172.21.15.69:0/3665028665 shutdown_connections 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 --2- 172.21.15.69:0/3665028665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 0x7ff51410e340 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 --2- 172.21.15.69:0/3665028665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff514103390 0x7ff51410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 --2- 172.21.15.69:0/3665028665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff514102a70 0x7ff514102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 -- 172.21.15.69:0/3665028665 >> 172.21.15.69:0/3665028665 conn(0x7ff5140fc4d0 msgr2=0x7ff5140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 -- 172.21.15.69:0/3665028665 shutdown_connections 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.075+0000 7ff519bea640 1 -- 172.21.15.69:0/3665028665 wait complete. 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.076+0000 7ff519bea640 1 Processor -- start 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.076+0000 7ff519bea640 1 -- start start 2024-08-02T17:45:33.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.076+0000 7ff519bea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff514102a70 0x7ff5141006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.076+0000 7ff519bea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff514103390 0x7ff514100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.076+0000 7ff519bea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 0x7ff514101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.076+0000 7ff519bea640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111ce0 con 0x7ff51410bea0 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.076+0000 7ff519bea640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111b60 con 0x7ff514103390 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.076+0000 7ff519bea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff514111e60 con 0x7ff514102a70 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 0x7ff514101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 0x7ff514101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39518/0 (socket says 172.21.15.69:39518) 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 -- 172.21.15.69:0/2585771387 learned_addr learned my addr 172.21.15.69:0/2585771387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff5137fe640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff514102a70 0x7ff5141006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff50affd640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff514103390 0x7ff514100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 -- 172.21.15.69:0/2585771387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff514102a70 msgr2=0x7ff5141006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff514102a70 0x7ff5141006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 -- 172.21.15.69:0/2585771387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff514103390 msgr2=0x7ff514100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff514103390 0x7ff514100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 -- 172.21.15.69:0/2585771387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5141a66f0 con 0x7ff51410bea0 2024-08-02T17:45:33.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff513fff640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 0x7ff514101150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02ae70 tx=0x7ff50c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:33.135 INFO:teuthology.orchestra.run.smithi069.stdout:34359738447 2024-08-02T17:45:33.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff5117fa640 1 -- 172.21.15.69:0/2585771387 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c034040 con 0x7ff51410bea0 2024-08-02T17:45:33.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5141a6980 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff5117fa640 1 -- 172.21.15.69:0/2585771387 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff50c030920 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff5117fa640 1 -- 172.21.15.69:0/2585771387 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50c0047b0 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.077+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5141a6eb0 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.078+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff514102e50 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.079+0000 7ff5117fa640 1 -- 172.21.15.69:0/2585771387 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff50c037020 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.080+0000 7ff5117fa640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5000a2c10 0x7ff5000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.080+0000 7ff5137fe640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5000a2c10 0x7ff5000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.080+0000 7ff5117fa640 1 -- 172.21.15.69:0/2585771387 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff50c0f0300 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.080+0000 7ff5137fe640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5000a2c10 0x7ff5000a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff504009990 tx=0x7ff504009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.083+0000 7ff5117fa640 1 -- 172.21.15.69:0/2585771387 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff50c0bcf50 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.266+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff514106f40 con 0x7ff51410bea0 2024-08-02T17:45:33.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.267+0000 7ff5117fa640 1 -- 172.21.15.69:0/2585771387 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff50c0bc850 con 0x7ff51410bea0 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5000a2c10 msgr2=0x7ff5000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5000a2c10 0x7ff5000a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff504009990 tx=0x7ff504009110 comp rx=0 tx=0).stop 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 msgr2=0x7ff514101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 0x7ff514101150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff50c02ae70 tx=0x7ff50c02f5b0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 shutdown_connections 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff5000a2c10 0x7ff5000a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff51410bea0 0x7ff514101150 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff514103390 0x7ff514100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 --2- 172.21.15.69:0/2585771387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff514102a70 0x7ff5141006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 >> 172.21.15.69:0/2585771387 conn(0x7ff5140fc4d0 msgr2=0x7ff5140fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 shutdown_connections 2024-08-02T17:45:33.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:32.269+0000 7ff519bea640 1 -- 172.21.15.69:0/2585771387 wait complete. 2024-08-02T17:45:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:32 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2585771387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-02T17:45:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:32 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4189766823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-02T17:45:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:32 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2585771387' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-08-02T17:45:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:32 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4189766823' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-08-02T17:45:33.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 -- 172.21.15.69:0/3487863177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9041022a0 msgr2=0x7ff9041026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 --2- 172.21.15.69:0/3487863177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9041022a0 0x7ff9041026e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009900 tx=0x7ff8ec02a480 comp rx=0 tx=0).stop 2024-08-02T17:45:33.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 -- 172.21.15.69:0/3487863177 shutdown_connections 2024-08-02T17:45:33.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 --2- 172.21.15.69:0/3487863177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff9041022a0 0x7ff9041026e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 --2- 172.21.15.69:0/3487863177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff904102c20 0x7ff90410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 --2- 172.21.15.69:0/3487863177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff904100b50 0x7ff904100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 -- 172.21.15.69:0/3487863177 >> 172.21.15.69:0/3487863177 conn(0x7ff9040fc500 msgr2=0x7ff9040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 -- 172.21.15.69:0/3487863177 shutdown_connections 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.525+0000 7ff909a4c640 1 -- 172.21.15.69:0/3487863177 wait complete. 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff909a4c640 1 Processor -- start 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff909a4c640 1 -- start start 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff909a4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff904100b50 0x7ff904197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff909a4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9041022a0 0x7ff904197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff909a4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff904102c20 0x7ff90419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff909a4c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff904110cb0 con 0x7ff904102c20 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff909a4c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff904110b30 con 0x7ff9041022a0 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff909a4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff904110e30 con 0x7ff904100b50 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff9037fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff904102c20 0x7ff90419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:33.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff9037fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff904102c20 0x7ff90419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39544/0 (socket says 172.21.15.69:39544) 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.526+0000 7ff9037fe640 1 -- 172.21.15.69:0/323954026 learned_addr learned my addr 172.21.15.69:0/323954026 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff902ffd640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff904100b50 0x7ff904197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff8fbfff640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9041022a0 0x7ff904197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff9037fe640 1 -- 172.21.15.69:0/323954026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff904100b50 msgr2=0x7ff904197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff9037fe640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff904100b50 0x7ff904197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff9037fe640 1 -- 172.21.15.69:0/323954026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9041022a0 msgr2=0x7ff904197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff9037fe640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9041022a0 0x7ff904197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff9037fe640 1 -- 172.21.15.69:0/323954026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90419c0c0 con 0x7ff904102c20 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff8fbfff640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9041022a0 0x7ff904197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff9037fe640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff904102c20 0x7ff90419b9e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00eaa0 tx=0x7ff8fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff908a4a640 1 -- 172.21.15.69:0/323954026 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00cdb0 con 0x7ff904102c20 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff908a4a640 1 -- 172.21.15.69:0/323954026 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc005020 con 0x7ff904102c20 2024-08-02T17:45:33.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90419c3b0 con 0x7ff904102c20 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff908a4a640 1 -- 172.21.15.69:0/323954026 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc0116b0 con 0x7ff904102c20 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.527+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041a3c90 con 0x7ff904102c20 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.528+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff904100f30 con 0x7ff904102c20 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.529+0000 7ff908a4a640 1 -- 172.21.15.69:0/323954026 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff8fc005560 con 0x7ff904102c20 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.530+0000 7ff908a4a640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff8f00a2c10 0x7ff8f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.530+0000 7ff902ffd640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff8f00a2c10 0x7ff8f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.530+0000 7ff908a4a640 1 -- 172.21.15.69:0/323954026 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff8fc0cf810 con 0x7ff904102c20 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.530+0000 7ff902ffd640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff8f00a2c10 0x7ff8f00a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4005fd0 tx=0x7ff8f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:33.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.534+0000 7ff908a4a640 1 -- 172.21.15.69:0/323954026 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff8fc016030 con 0x7ff904102c20 2024-08-02T17:45:33.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.709+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff904108dc0 con 0x7ff904102c20 2024-08-02T17:45:33.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.709+0000 7ff908a4a640 1 -- 172.21.15.69:0/323954026 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff8fc09c2b0 con 0x7ff904102c20 2024-08-02T17:45:33.711 INFO:teuthology.orchestra.run.smithi069.stdout:193273528373 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff8f00a2c10 msgr2=0x7ff8f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff8f00a2c10 0x7ff8f00a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4005fd0 tx=0x7ff8f4005850 comp rx=0 tx=0).stop 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff904102c20 msgr2=0x7ff90419b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff904102c20 0x7ff90419b9e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00eaa0 tx=0x7ff8fc00ef60 comp rx=0 tx=0).stop 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 shutdown_connections 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff8f00a2c10 0x7ff8f00a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff904102c20 0x7ff90419b9e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff9041022a0 0x7ff904197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 --2- 172.21.15.69:0/323954026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff904100b50 0x7ff904197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 >> 172.21.15.69:0/323954026 conn(0x7ff9040fc500 msgr2=0x7ff9041065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 shutdown_connections 2024-08-02T17:45:33.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:33.711+0000 7ff909a4c640 1 -- 172.21.15.69:0/323954026 wait complete. 2024-08-02T17:45:33.987 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:33 smithi069 ceph-mon[32746]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-08-02T17:45:33.987 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:33 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/323954026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-02T17:45:34.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.176+0000 7fe1524e8640 1 -- 172.21.15.69:0/1911109230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 msgr2=0x7fe14c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:34.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.176+0000 7fe1524e8640 1 --2- 172.21.15.69:0/1911109230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 0x7fe14c100c10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe140009930 tx=0x7fe14002a420 comp rx=0 tx=0).stop 2024-08-02T17:45:34.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.176+0000 7fe1524e8640 1 -- 172.21.15.69:0/1911109230 shutdown_connections 2024-08-02T17:45:34.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.176+0000 7fe1524e8640 1 --2- 172.21.15.69:0/1911109230 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 0x7fe14c100c10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.176+0000 7fe1524e8640 1 --2- 172.21.15.69:0/1911109230 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe14c101150 0x7fe14c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.176+0000 7fe1524e8640 1 --2- 172.21.15.69:0/1911109230 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1067b0 0x7fe14c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.176+0000 7fe1524e8640 1 -- 172.21.15.69:0/1911109230 >> 172.21.15.69:0/1911109230 conn(0x7fe14c0fc4d0 msgr2=0x7fe14c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.176+0000 7fe1524e8640 1 -- 172.21.15.69:0/1911109230 shutdown_connections 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 -- 172.21.15.69:0/1911109230 wait complete. 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 Processor -- start 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 -- start start 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 0x7fe14c0fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe14c101150 0x7fe14c0fd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1067b0 0x7fe14c0fd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:34.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c110b70 con 0x7fe14c1007b0 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c1109f0 con 0x7fe14c101150 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.177+0000 7fe1524e8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c110cf0 con 0x7fe14c1067b0 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 0x7fe14c0fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 0x7fe14c0fcac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39568/0 (socket says 172.21.15.69:39568) 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 -- 172.21.15.69:0/2517695486 learned_addr learned my addr 172.21.15.69:0/2517695486 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe150a85640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1067b0 0x7fe14c0fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14b7fe640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe14c101150 0x7fe14c0fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 -- 172.21.15.69:0/2517695486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1067b0 msgr2=0x7fe14c0fd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1067b0 0x7fe14c0fd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 -- 172.21.15.69:0/2517695486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe14c101150 msgr2=0x7fe14c0fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:34.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe14c101150 0x7fe14c0fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 -- 172.21.15.69:0/2517695486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c1a6640 con 0x7fe14c1007b0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14b7fe640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe14c101150 0x7fe14c0fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe14bfff640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 0x7fe14c0fcac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00c920 tx=0x7fe13c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe1497fa640 1 -- 172.21.15.69:0/2517695486 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c007c70 con 0x7fe14c1007b0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c1a6930 con 0x7fe14c1007b0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe1497fa640 1 -- 172.21.15.69:0/2517695486 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13c007e10 con 0x7fe14c1007b0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe1497fa640 1 -- 172.21.15.69:0/2517695486 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c018890 con 0x7fe14c1007b0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.178+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c1a6e40 con 0x7fe14c1007b0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.179+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe14c100c10 con 0x7fe14c1007b0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.181+0000 7fe1497fa640 1 -- 172.21.15.69:0/2517695486 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe13c018a50 con 0x7fe14c1007b0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.181+0000 7fe1497fa640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1340a2c10 0x7fe1340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.181+0000 7fe14b7fe640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1340a2c10 0x7fe1340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:34.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.181+0000 7fe1497fa640 1 -- 172.21.15.69:0/2517695486 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe13c0cf8a0 con 0x7fe14c1007b0 2024-08-02T17:45:34.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.182+0000 7fe14b7fe640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1340a2c10 0x7fe1340a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe140000c00 tx=0x7fe140005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:34.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.185+0000 7fe1497fa640 1 -- 172.21.15.69:0/2517695486 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe13c014030 con 0x7fe14c1007b0 2024-08-02T17:45:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:33 smithi083 ceph-mon[37031]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-08-02T17:45:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:33 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/323954026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-02T17:45:34.354 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:33 smithi077 ceph-mon[37108]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-08-02T17:45:34.354 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:33 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/323954026' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-08-02T17:45:34.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.362+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe14c109d80 con 0x7fe14c1007b0 2024-08-02T17:45:34.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.363+0000 7fe1497fa640 1 -- 172.21.15.69:0/2517695486 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe13c09c3d0 con 0x7fe14c1007b0 2024-08-02T17:45:34.364 INFO:teuthology.orchestra.run.smithi069.stdout:283467841576 2024-08-02T17:45:34.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.364+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1340a2c10 msgr2=0x7fe1340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:34.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.364+0000 7fe1524e8640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1340a2c10 0x7fe1340a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe140000c00 tx=0x7fe140005a20 comp rx=0 tx=0).stop 2024-08-02T17:45:34.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.364+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 msgr2=0x7fe14c0fcac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:34.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.364+0000 7fe1524e8640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 0x7fe14c0fcac0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00c920 tx=0x7fe13c00cde0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.365+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 shutdown_connections 2024-08-02T17:45:34.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.365+0000 7fe1524e8640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe1340a2c10 0x7fe1340a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.365+0000 7fe1524e8640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe14c1067b0 0x7fe14c0fd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.365+0000 7fe1524e8640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe14c101150 0x7fe14c0fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.365+0000 7fe1524e8640 1 --2- 172.21.15.69:0/2517695486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe14c1007b0 0x7fe14c0fcac0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:34.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.365+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 >> 172.21.15.69:0/2517695486 conn(0x7fe14c0fc4d0 msgr2=0x7fe14c06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:34.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.365+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 shutdown_connections 2024-08-02T17:45:34.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:34.365+0000 7fe1524e8640 1 -- 172.21.15.69:0/2517695486 wait complete. 2024-08-02T17:45:34.471 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:35.063 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:34 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2517695486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-02T17:45:35.091 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738447 for osd.0 2024-08-02T17:45:35.091 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:35.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:34 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2517695486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-02T17:45:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:34 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2517695486' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-08-02T17:45:35.849 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528370 got 193273528373 for osd.6 2024-08-02T17:45:35.850 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:36.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:35 smithi069 ceph-mon[32746]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:36.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 -- 172.21.15.69:0/1017248949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2784101150 msgr2=0x7f278410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:36.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 --2- 172.21.15.69:0/1017248949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2784101150 0x7f278410d680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f277c009cd0 tx=0x7f277c02a8e0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 -- 172.21.15.69:0/1017248949 shutdown_connections 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 --2- 172.21.15.69:0/1017248949 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2784101150 0x7f278410d680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 --2- 172.21.15.69:0/1017248949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27841007b0 0x7f2784100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 --2- 172.21.15.69:0/1017248949 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27841067b0 0x7f2784106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 -- 172.21.15.69:0/1017248949 >> 172.21.15.69:0/1017248949 conn(0x7f27840fc4d0 msgr2=0x7f27840fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 -- 172.21.15.69:0/1017248949 shutdown_connections 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 -- 172.21.15.69:0/1017248949 wait complete. 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 Processor -- start 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.115+0000 7f278c2d7640 1 -- start start 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278c2d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27841007b0 0x7f2784199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278c2d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2784101150 0x7f2784199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:36.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278c2d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27841067b0 0x7f27841a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278c2d7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2784110bd0 con 0x7f27841007b0 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278c2d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2784110a50 con 0x7f2784101150 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278c2d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2784110d50 con 0x7f27841067b0 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27841007b0 0x7f2784199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27841007b0 0x7f2784199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54686/0 (socket says 172.21.15.69:54686) 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 -- 172.21.15.69:0/656379898 learned_addr learned my addr 172.21.15.69:0/656379898 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f2789872640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2784101150 0x7f2784199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a874640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27841067b0 0x7f27841a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 -- 172.21.15.69:0/656379898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27841067b0 msgr2=0x7f27841a40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27841067b0 0x7f27841a40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 -- 172.21.15.69:0/656379898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2784101150 msgr2=0x7f2784199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2784101150 0x7f2784199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 -- 172.21.15.69:0/656379898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27841a45e0 con 0x7f27841007b0 2024-08-02T17:45:36.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a874640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27841067b0 0x7f27841a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:36.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278a073640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27841007b0 0x7f2784199020 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f278000c890 tx=0x7f278000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:36.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f277b7fe640 1 -- 172.21.15.69:0/656379898 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004220 con 0x7f27841007b0 2024-08-02T17:45:36.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27841a47b0 con 0x7f27841007b0 2024-08-02T17:45:36.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f277b7fe640 1 -- 172.21.15.69:0/656379898 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27800043c0 con 0x7f27841007b0 2024-08-02T17:45:36.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f277b7fe640 1 -- 172.21.15.69:0/656379898 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004a00 con 0x7f27841007b0 2024-08-02T17:45:36.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.116+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27841a4cf0 con 0x7f27841007b0 2024-08-02T17:45:36.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.117+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2784100c10 con 0x7f27841007b0 2024-08-02T17:45:36.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.118+0000 7f277b7fe640 1 -- 172.21.15.69:0/656379898 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f27800079b0 con 0x7f27841007b0 2024-08-02T17:45:36.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.119+0000 7f277b7fe640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27700a2c10 0x7f27700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:36.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.119+0000 7f277b7fe640 1 -- 172.21.15.69:0/656379898 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f27800cf720 con 0x7f27841007b0 2024-08-02T17:45:36.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.119+0000 7f2789872640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27700a2c10 0x7f27700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:36.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.120+0000 7f2789872640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27700a2c10 0x7f27700a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f278406b010 tx=0x7f2774008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:36.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.122+0000 7f277b7fe640 1 -- 172.21.15.69:0/656379898 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2780014030 con 0x7f27841007b0 2024-08-02T17:45:36.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:35 smithi083 ceph-mon[37031]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:36.249 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:35 smithi077 ceph-mon[37108]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:36.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.305+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2784108d40 con 0x7f27841007b0 2024-08-02T17:45:36.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.306+0000 7f277b7fe640 1 -- 172.21.15.69:0/656379898 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f278009c160 con 0x7f27841007b0 2024-08-02T17:45:36.307 INFO:teuthology.orchestra.run.smithi069.stdout:55834574924 2024-08-02T17:45:36.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27700a2c10 msgr2=0x7f27700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:36.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27700a2c10 0x7f27700a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f278406b010 tx=0x7f2774008040 comp rx=0 tx=0).stop 2024-08-02T17:45:36.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27841007b0 msgr2=0x7f2784199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:36.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27841007b0 0x7f2784199020 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f278000c890 tx=0x7f278000cd50 comp rx=0 tx=0).stop 2024-08-02T17:45:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 shutdown_connections 2024-08-02T17:45:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f27700a2c10 0x7f27700a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27841067b0 0x7f27841a40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2784101150 0x7f2784199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 --2- 172.21.15.69:0/656379898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27841007b0 0x7f2784199020 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 >> 172.21.15.69:0/656379898 conn(0x7f27840fc4d0 msgr2=0x7f27840fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 shutdown_connections 2024-08-02T17:45:36.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.308+0000 7f278c2d7640 1 -- 172.21.15.69:0/656379898 wait complete. 2024-08-02T17:45:36.703 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364845 got 214748364848 for osd.7 2024-08-02T17:45:36.703 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:36.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.879+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/2557742245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 msgr2=0x7ff7a4102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:36.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.879+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/2557742245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 0x7ff7a4102e40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009930 tx=0x7ff7a002a420 comp rx=0 tx=0).stop 2024-08-02T17:45:36.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/2557742245 shutdown_connections 2024-08-02T17:45:36.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/2557742245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a4107fd0 0x7ff7a410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/2557742245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a4103410 0x7ff7a41078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/2557742245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 0x7ff7a4102e40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/2557742245 >> 172.21.15.69:0/2557742245 conn(0x7ff7a40fc500 msgr2=0x7ff7a40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:36.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/2557742245 shutdown_connections 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/2557742245 wait complete. 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 Processor -- start 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 -- start start 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 0x7ff7a4198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a4103410 0x7ff7a4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a4107fd0 0x7ff7a419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111dd0 con 0x7ff7a4102a60 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111c50 con 0x7ff7a4103410 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.880+0000 7ff7ab7ed640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111f50 con 0x7ff7a4107fd0 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 0x7ff7a4198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 0x7ff7a4198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54710/0 (socket says 172.21.15.69:54710) 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 -- 172.21.15.69:0/3942426419 learned_addr learned my addr 172.21.15.69:0/3942426419 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:36.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9d8a640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a4107fd0 0x7ff7a419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a8d88640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a4103410 0x7ff7a4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 -- 172.21.15.69:0/3942426419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a4107fd0 msgr2=0x7ff7a419cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a4107fd0 0x7ff7a419cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 -- 172.21.15.69:0/3942426419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a4103410 msgr2=0x7ff7a4198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a4103410 0x7ff7a4198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 -- 172.21.15.69:0/3942426419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a419d300 con 0x7ff7a4102a60 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7a9589640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 0x7ff7a4198350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009230 tx=0x7ff7a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff79a7fc640 1 -- 172.21.15.69:0/3942426419 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a003f030 con 0x7ff7a4102a60 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff79a7fc640 1 -- 172.21.15.69:0/3942426419 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a0004440 con 0x7ff7a4102a60 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff79a7fc640 1 -- 172.21.15.69:0/3942426419 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0035070 con 0x7ff7a4102a60 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a419d590 con 0x7ff7a4102a60 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.881+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a41a4ed0 con 0x7ff7a4102a60 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.882+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a406a580 con 0x7ff7a4102a60 2024-08-02T17:45:36.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.884+0000 7ff79a7fc640 1 -- 172.21.15.69:0/3942426419 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff7a0042030 con 0x7ff7a4102a60 2024-08-02T17:45:36.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.884+0000 7ff79a7fc640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff7900a2c90 0x7ff7900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:36.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.884+0000 7ff7a8d88640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff7900a2c90 0x7ff7900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:36.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.884+0000 7ff79a7fc640 1 -- 172.21.15.69:0/3942426419 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7ff7a00efc30 con 0x7ff7a4102a60 2024-08-02T17:45:36.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.885+0000 7ff7a8d88640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff7900a2c90 0x7ff7900a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40febc0 tx=0x7ff794005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:36.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:36.889+0000 7ff79a7fc640 1 -- 172.21.15.69:0/3942426419 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff7a00bc080 con 0x7ff7a4102a60 2024-08-02T17:45:37.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:36 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/656379898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-02T17:45:37.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.061+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff7a4102e40 con 0x7ff7a4102a60 2024-08-02T17:45:37.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.061+0000 7ff79a7fc640 1 -- 172.21.15.69:0/3942426419 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff7a4102e40 con 0x7ff7a4102a60 2024-08-02T17:45:37.063 INFO:teuthology.orchestra.run.smithi069.stdout:158913790010 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff7900a2c90 msgr2=0x7ff7900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff7900a2c90 0x7ff7900a5150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff7a40febc0 tx=0x7ff794005f90 comp rx=0 tx=0).stop 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 msgr2=0x7ff7a4198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 0x7ff7a4198350 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0009230 tx=0x7ff7a0004040 comp rx=0 tx=0).stop 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 shutdown_connections 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7ff7900a2c90 0x7ff7900a5150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7a4107fd0 0x7ff7a419cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7a4103410 0x7ff7a4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 --2- 172.21.15.69:0/3942426419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7a4102a60 0x7ff7a4198350 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 >> 172.21.15.69:0/3942426419 conn(0x7ff7a40fc500 msgr2=0x7ff7a4105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:37.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.063+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 shutdown_connections 2024-08-02T17:45:37.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.064+0000 7ff7ab7ed640 1 -- 172.21.15.69:0/3942426419 wait complete. 2024-08-02T17:45:37.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 -- 172.21.15.69:0/2684062668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c102c20 msgr2=0x7f562c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:37.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 --2- 172.21.15.69:0/2684062668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c102c20 0x7f562c10d770 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5624009ca0 tx=0x7f562402a950 comp rx=0 tx=0).stop 2024-08-02T17:45:37.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 -- 172.21.15.69:0/2684062668 shutdown_connections 2024-08-02T17:45:37.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 --2- 172.21.15.69:0/2684062668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c102c20 0x7f562c10d770 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 --2- 172.21.15.69:0/2684062668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f562c1022a0 0x7f562c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 --2- 172.21.15.69:0/2684062668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f562c100b50 0x7f562c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 -- 172.21.15.69:0/2684062668 >> 172.21.15.69:0/2684062668 conn(0x7f562c0fc500 msgr2=0x7f562c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 -- 172.21.15.69:0/2684062668 shutdown_connections 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.108+0000 7f56308c3640 1 -- 172.21.15.69:0/2684062668 wait complete. 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f56308c3640 1 Processor -- start 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f56308c3640 1 -- start start 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f56308c3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f562c100b50 0x7f562c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f56308c3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c1022a0 0x7f562c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f56308c3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f562c102c20 0x7f562c1a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f56308c3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f562c110cb0 con 0x7f562c1022a0 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f56308c3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f562c110b30 con 0x7f562c102c20 2024-08-02T17:45:37.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f56308c3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f562c110e30 con 0x7f562c100b50 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f5621d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c1022a0 0x7f562c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f5621d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c1022a0 0x7f562c1995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54718/0 (socket says 172.21.15.69:54718) 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f5621d9b640 1 -- 172.21.15.69:0/3820340204 learned_addr learned my addr 172.21.15.69:0/3820340204 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.109+0000 7f562a59c640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f562c100b50 0x7f562c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f562ad9d640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f562c102c20 0x7f562c1a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5621d9b640 1 -- 172.21.15.69:0/3820340204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f562c100b50 msgr2=0x7f562c1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5621d9b640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f562c100b50 0x7f562c1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5621d9b640 1 -- 172.21.15.69:0/3820340204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f562c102c20 msgr2=0x7f562c1a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5621d9b640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f562c102c20 0x7f562c1a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5621d9b640 1 -- 172.21.15.69:0/3820340204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c1a4660 con 0x7f562c1022a0 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f562ad9d640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f562c102c20 0x7f562c1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:37.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5621d9b640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c1022a0 0x7f562c1995e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f561400d770 tx=0x7f561400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5623fff640 1 -- 172.21.15.69:0/3820340204 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614017070 con 0x7f562c1022a0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5623fff640 1 -- 172.21.15.69:0/3820340204 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5614005500 con 0x7f562c1022a0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c1a4830 con 0x7f562c1022a0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f5623fff640 1 -- 172.21.15.69:0/3820340204 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614004e00 con 0x7f562c1022a0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.110+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c1a4cf0 con 0x7f562c1022a0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.111+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562c100f30 con 0x7f562c1022a0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.112+0000 7f5623fff640 1 -- 172.21.15.69:0/3820340204 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5614004fa0 con 0x7f562c1022a0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.113+0000 7f5623fff640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f56180a2c10 0x7f56180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.113+0000 7f562a59c640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f56180a2c10 0x7f56180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:37.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.113+0000 7f5623fff640 1 -- 172.21.15.69:0/3820340204 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f56140cef50 con 0x7f562c1022a0 2024-08-02T17:45:37.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.113+0000 7f562a59c640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f56180a2c10 0x7f56180a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f561c00bbe0 tx=0x7f561c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:37.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.117+0000 7f5623fff640 1 -- 172.21.15.69:0/3820340204 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f561409b9f0 con 0x7f562c1022a0 2024-08-02T17:45:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:36 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/656379898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-02T17:45:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.288+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f562c108dc0 con 0x7f562c1022a0 2024-08-02T17:45:37.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.288+0000 7f5623fff640 1 -- 172.21.15.69:0/3820340204 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f561409b2f0 con 0x7f562c1022a0 2024-08-02T17:45:37.290 INFO:teuthology.orchestra.run.smithi069.stdout:304942678051 2024-08-02T17:45:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f56180a2c10 msgr2=0x7f56180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f56180a2c10 0x7f56180a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f561c00bbe0 tx=0x7f561c00b390 comp rx=0 tx=0).stop 2024-08-02T17:45:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c1022a0 msgr2=0x7f562c1995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c1022a0 0x7f562c1995e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f561400d770 tx=0x7f561400dc30 comp rx=0 tx=0).stop 2024-08-02T17:45:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 shutdown_connections 2024-08-02T17:45:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f56180a2c10 0x7f56180a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f562c1022a0 0x7f562c1995e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f562c102c20 0x7f562c1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 --2- 172.21.15.69:0/3820340204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f562c100b50 0x7f562c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 >> 172.21.15.69:0/3820340204 conn(0x7f562c0fc500 msgr2=0x7f562c1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 shutdown_connections 2024-08-02T17:45:37.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:37.291+0000 7f56308c3640 1 -- 172.21.15.69:0/3820340204 wait complete. 2024-08-02T17:45:37.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:36 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/656379898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-08-02T17:45:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:37 smithi069 ceph-mon[32746]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:37 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3942426419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-02T17:45:38.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:37 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3820340204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-02T17:45:38.177 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574924 for osd.1 2024-08-02T17:45:38.177 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:37 smithi083 ceph-mon[37031]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:37 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3942426419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-02T17:45:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:37 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3820340204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-02T17:45:38.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:37 smithi077 ceph-mon[37108]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:37 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3942426419' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-08-02T17:45:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:37 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3820340204' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-08-02T17:45:38.611 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:38.635 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678048 got 304942678051 for osd.10 2024-08-02T17:45:38.635 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:39.011 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790006 got 158913790010 for osd.5 2024-08-02T17:45:39.011 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:39.483 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841573 got 283467841576 for osd.9 2024-08-02T17:45:39.483 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:39.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 -- 172.21.15.69:0/324098206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 msgr2=0x7f1ac810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:39.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 --2- 172.21.15.69:0/324098206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 0x7f1ac810e340 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009ca0 tx=0x7f1ac002a950 comp rx=0 tx=0).stop 2024-08-02T17:45:39.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 -- 172.21.15.69:0/324098206 shutdown_connections 2024-08-02T17:45:39.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 --2- 172.21.15.69:0/324098206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 0x7f1ac810e340 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:39.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 --2- 172.21.15.69:0/324098206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac8103390 0x7f1ac810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:39.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 --2- 172.21.15.69:0/324098206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8102a70 0x7f1ac8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:39.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 -- 172.21.15.69:0/324098206 >> 172.21.15.69:0/324098206 conn(0x7f1ac80fc4d0 msgr2=0x7f1ac80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 -- 172.21.15.69:0/324098206 shutdown_connections 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.941+0000 7f1ad0106640 1 -- 172.21.15.69:0/324098206 wait complete. 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ad0106640 1 Processor -- start 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ad0106640 1 -- start start 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ad0106640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8102a70 0x7f1ac81006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ad0106640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac8103390 0x7f1ac8100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ad0106640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 0x7f1ac8101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ad0106640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111c60 con 0x7f1ac810bea0 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ad0106640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111ae0 con 0x7f1ac8103390 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ad0106640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8111de0 con 0x7f1ac8102a70 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ace6a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 0x7f1ac8101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ace6a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 0x7f1ac8101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54734/0 (socket says 172.21.15.69:54734) 2024-08-02T17:45:39.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1ace6a3640 1 -- 172.21.15.69:0/1707948838 learned_addr learned my addr 172.21.15.69:0/1707948838 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.942+0000 7f1acdea2640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8102a70 0x7f1ac81006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1acd6a1640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac8103390 0x7f1ac8100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1ace6a3640 1 -- 172.21.15.69:0/1707948838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8102a70 msgr2=0x7f1ac81006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1ace6a3640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8102a70 0x7f1ac81006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1ace6a3640 1 -- 172.21.15.69:0/1707948838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac8103390 msgr2=0x7f1ac8100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1ace6a3640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac8103390 0x7f1ac8100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1ace6a3640 1 -- 172.21.15.69:0/1707948838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac81a65c0 con 0x7f1ac810bea0 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1acd6a1640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac8103390 0x7f1ac8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1ace6a3640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 0x7f1ac8101150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ac002ae70 tx=0x7f1ac002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1abeffd640 1 -- 172.21.15.69:0/1707948838 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0034040 con 0x7f1ac810bea0 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1abeffd640 1 -- 172.21.15.69:0/1707948838 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac0030920 con 0x7f1ac810bea0 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac81a6850 con 0x7f1ac810bea0 2024-08-02T17:45:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1abeffd640 1 -- 172.21.15.69:0/1707948838 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0045480 con 0x7f1ac810bea0 2024-08-02T17:45:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.943+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a6d30 con 0x7f1ac810bea0 2024-08-02T17:45:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.944+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac8102e50 con 0x7f1ac810bea0 2024-08-02T17:45:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.945+0000 7f1abeffd640 1 -- 172.21.15.69:0/1707948838 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1ac0037020 con 0x7f1ac810bea0 2024-08-02T17:45:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.946+0000 7f1abeffd640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ab40a2c10 0x7f1ab40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.946+0000 7f1abeffd640 1 -- 172.21.15.69:0/1707948838 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1ac00f03e0 con 0x7f1ac810bea0 2024-08-02T17:45:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.946+0000 7f1acdea2640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ab40a2c10 0x7f1ab40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.946+0000 7f1acdea2640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ab40a2c10 0x7f1ab40a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80058c0 tx=0x7f1ab800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:39.950+0000 7f1abeffd640 1 -- 172.21.15.69:0/1707948838 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1ac00bcf30 con 0x7f1ac810bea0 2024-08-02T17:45:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:39 smithi069 ceph-mon[32746]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:40.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.119+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1ac802d5e0 con 0x7f1ac810bea0 2024-08-02T17:45:40.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.119+0000 7f1abeffd640 1 -- 172.21.15.69:0/1707948838 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1ac00bc830 con 0x7f1ac810bea0 2024-08-02T17:45:40.121 INFO:teuthology.orchestra.run.smithi069.stdout:137438953535 2024-08-02T17:45:40.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ab40a2c10 msgr2=0x7f1ab40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:40.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ab40a2c10 0x7f1ab40a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80058c0 tx=0x7f1ab800a2d0 comp rx=0 tx=0).stop 2024-08-02T17:45:40.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 msgr2=0x7f1ac8101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:40.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 0x7f1ac8101150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1ac002ae70 tx=0x7f1ac002f5b0 comp rx=0 tx=0).stop 2024-08-02T17:45:40.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 shutdown_connections 2024-08-02T17:45:40.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ab40a2c10 0x7f1ab40a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:40.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac810bea0 0x7f1ac8101150 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:40.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac8103390 0x7f1ac8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:40.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 --2- 172.21.15.69:0/1707948838 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8102a70 0x7f1ac81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:40.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.121+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 >> 172.21.15.69:0/1707948838 conn(0x7f1ac80fc4d0 msgr2=0x7f1ac80fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:40.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.122+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 shutdown_connections 2024-08-02T17:45:40.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:40.122+0000 7f1ad0106640 1 -- 172.21.15.69:0/1707948838 wait complete. 2024-08-02T17:45:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:39 smithi083 ceph-mon[37031]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:40.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:39 smithi077 ceph-mon[37108]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:40.786 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953530 got 137438953535 for osd.4 2024-08-02T17:45:40.787 DEBUG:teuthology.parallel:result is None 2024-08-02T17:45:40.787 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-02T17:45:40.787 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph pg dump --format=json 2024-08-02T17:45:41.043 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:41.165 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:40 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1707948838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-02T17:45:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:40 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1707948838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-02T17:45:41.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:40 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1707948838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1594331126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 msgr2=0x7fcbc8100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1594331126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 0x7fcbc8100c10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009930 tx=0x7fcbb002a420 comp rx=0 tx=0).stop 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1594331126 shutdown_connections 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1594331126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 0x7fcbc8100c10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1594331126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc8101150 0x7fcbc810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1594331126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbc81067b0 0x7fcbc8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1594331126 >> 172.21.15.69:0/1594331126 conn(0x7fcbc80fc4d0 msgr2=0x7fcbc80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1594331126 shutdown_connections 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.538+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1594331126 wait complete. 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbcd2bb640 1 Processor -- start 2024-08-02T17:45:41.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbcd2bb640 1 -- start start 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbcd2bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 0x7fcbc8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbcd2bb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbc8101150 0x7fcbc8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbcd2bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc81067b0 0x7fcbc81a4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbcd2bb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8110bb0 con 0x7fcbc81007b0 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbcd2bb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8110a30 con 0x7fcbc8101150 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbcd2bb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc8110d30 con 0x7fcbc81067b0 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbc6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 0x7fcbc8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbc6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 0x7fcbc8198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54744/0 (socket says 172.21.15.69:54744) 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbc6ffd640 1 -- 172.21.15.69:0/1316768654 learned_addr learned my addr 172.21.15.69:0/1316768654 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbbe7fc640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbc8101150 0x7fcbc8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbc77fe640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc81067b0 0x7fcbc81a4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.539+0000 7fcbc6ffd640 1 -- 172.21.15.69:0/1316768654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc81067b0 msgr2=0x7fcbc81a4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:41.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc6ffd640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc81067b0 0x7fcbc81a4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc6ffd640 1 -- 172.21.15.69:0/1316768654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbc8101150 msgr2=0x7fcbc8199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc6ffd640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbc8101150 0x7fcbc8199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc6ffd640 1 -- 172.21.15.69:0/1316768654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc81a45a0 con 0x7fcbc81007b0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc77fe640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc81067b0 0x7fcbc81a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc6ffd640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 0x7fcbc8198fe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800d950 tx=0x7fcbb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc4ff9640 1 -- 172.21.15.69:0/1316768654 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8017070 con 0x7fcbc81007b0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc4ff9640 1 -- 172.21.15.69:0/1316768654 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbb8009c30 con 0x7fcbc81007b0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc81a4770 con 0x7fcbc81007b0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbc4ff9640 1 -- 172.21.15.69:0/1316768654 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb800f6b0 con 0x7fcbc81007b0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.540+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc81a4c30 con 0x7fcbc81007b0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.541+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc8100c10 con 0x7fcbc81007b0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.542+0000 7fcbc4ff9640 1 -- 172.21.15.69:0/1316768654 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcbb80109d0 con 0x7fcbc81007b0 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.543+0000 7fcbc4ff9640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:41.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.544+0000 7fcbbe7fc640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.544+0000 7fcbc4ff9640 1 -- 172.21.15.69:0/1316768654 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fcbb80cf0c0 con 0x7fcbc81007b0 2024-08-02T17:45:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.544+0000 7fcbbe7fc640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009900 tx=0x7fcbb00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:41.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.548+0000 7fcbc4ff9640 1 -- 172.21.15.69:0/1316768654 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcbb8009de0 con 0x7fcbc81007b0 2024-08-02T17:45:41.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.712+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcbc8062360 con 0x7fcbb40a2c10 2024-08-02T17:45:41.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.720+0000 7fcbc4ff9640 1 -- 172.21.15.69:0/1316768654 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340370 (secure 0 0 0) 0x7fcbc8062360 con 0x7fcbb40a2c10 2024-08-02T17:45:41.725 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:41.726 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcbb40a2c10 msgr2=0x7fcbb40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009900 tx=0x7fcbb00058d0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 msgr2=0x7fcbc8198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 0x7fcbc8198fe0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800d950 tx=0x7fcbb800de10 comp rx=0 tx=0).stop 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 shutdown_connections 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fcbb40a2c10 0x7fcbb40a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcbc81067b0 0x7fcbc81a4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcbc8101150 0x7fcbc8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 --2- 172.21.15.69:0/1316768654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcbc81007b0 0x7fcbc8198fe0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:41.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 >> 172.21.15.69:0/1316768654 conn(0x7fcbc80fc4d0 msgr2=0x7fcbc80fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:41.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 shutdown_connections 2024-08-02T17:45:41.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:41.723+0000 7fcbcd2bb640 1 -- 172.21.15.69:0/1316768654 wait complete. 2024-08-02T17:45:42.031 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:41 smithi069 ceph-mon[32746]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:41 smithi083 ceph-mon[37031]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:42.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:41 smithi077 ceph-mon[37108]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:42.403 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-08-02T17:45:40.780248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657917,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":465,"num_read_kb":353,"num_write":352,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392648,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549176,"statfs":{"total":1151940427776,"available":1151538356224,"internally_reserved":0,"allocated":14680064,"data_stored":8131961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003239"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.776089+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:44:57.776089+0000","last_peered":"2024-08-02T17:44:57.776089+0000","last_clean":"2024-08-02T17:44:57.776089+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:44:57.776089+0000","last_undegraded":"2024-08-02T17:44:57.776089+0000","last_fullsized":"2024-08-02T17:44:57.776089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778005+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:44:57.778005+0000","last_peered":"2024-08-02T17:44:57.778005+0000","last_clean":"2024-08-02T17:44:57.778005+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:44:57.778005+0000","last_undegraded":"2024-08-02T17:44:57.778005+0000","last_fullsized":"2024-08-02T17:44:57.778005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782153+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:44:57.782153+0000","last_peered":"2024-08-02T17:44:57.782153+0000","last_clean":"2024-08-02T17:44:57.782153+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:44:57.782153+0000","last_undegraded":"2024-08-02T17:44:57.782153+0000","last_fullsized":"2024-08-02T17:44:57.782153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778542+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:44:57.778542+0000","last_peered":"2024-08-02T17:44:57.778542+0000","last_clean":"2024-08-02T17:44:57.778542+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:44:57.778542+0000","last_undegraded":"2024-08-02T17:44:57.778542+0000","last_fullsized":"2024-08-02T17:44:57.778542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788699+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:44:57.788699+0000","last_peered":"2024-08-02T17:44:57.788699+0000","last_clean":"2024-08-02T17:44:57.788699+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:44:57.788699+0000","last_undegraded":"2024-08-02T17:44:57.788699+0000","last_fullsized":"2024-08-02T17:44:57.788699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777204+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:44:57.777204+0000","last_peered":"2024-08-02T17:44:57.777204+0000","last_clean":"2024-08-02T17:44:57.777204+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:44:57.777204+0000","last_undegraded":"2024-08-02T17:44:57.777204+0000","last_fullsized":"2024-08-02T17:44:57.777204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784045+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:44:57.784045+0000","last_peered":"2024-08-02T17:44:57.784045+0000","last_clean":"2024-08-02T17:44:57.784045+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:44:57.784045+0000","last_undegraded":"2024-08-02T17:44:57.784045+0000","last_fullsized":"2024-08-02T17:44:57.784045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058658+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:44:58.058658+0000","last_peered":"2024-08-02T17:44:58.058658+0000","last_clean":"2024-08-02T17:44:58.058658+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:44:58.058658+0000","last_undegraded":"2024-08-02T17:44:58.058658+0000","last_fullsized":"2024-08-02T17:44:58.058658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778197+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:44:57.778197+0000","last_peered":"2024-08-02T17:44:57.778197+0000","last_clean":"2024-08-02T17:44:57.778197+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:44:57.778197+0000","last_undegraded":"2024-08-02T17:44:57.778197+0000","last_fullsized":"2024-08-02T17:44:57.778197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783541+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:44:57.783541+0000","last_peered":"2024-08-02T17:44:57.783541+0000","last_clean":"2024-08-02T17:44:57.783541+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:44:57.783541+0000","last_undegraded":"2024-08-02T17:44:57.783541+0000","last_fullsized":"2024-08-02T17:44:57.783541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780343+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:44:57.780343+0000","last_peered":"2024-08-02T17:44:57.780343+0000","last_clean":"2024-08-02T17:44:57.780343+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:44:57.780343+0000","last_undegraded":"2024-08-02T17:44:57.780343+0000","last_fullsized":"2024-08-02T17:44:57.780343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777417+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:44:57.777417+0000","last_peered":"2024-08-02T17:44:57.777417+0000","last_clean":"2024-08-02T17:44:57.777417+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:44:57.777417+0000","last_undegraded":"2024-08-02T17:44:57.777417+0000","last_fullsized":"2024-08-02T17:44:57.777417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777532+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:44:57.777532+0000","last_peered":"2024-08-02T17:44:57.777532+0000","last_clean":"2024-08-02T17:44:57.777532+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:44:57.777532+0000","last_undegraded":"2024-08-02T17:44:57.777532+0000","last_fullsized":"2024-08-02T17:44:57.777532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783178+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:44:57.783178+0000","last_peered":"2024-08-02T17:44:57.783178+0000","last_clean":"2024-08-02T17:44:57.783178+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:44:57.783178+0000","last_undegraded":"2024-08-02T17:44:57.783178+0000","last_fullsized":"2024-08-02T17:44:57.783178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778786+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:44:57.778786+0000","last_peered":"2024-08-02T17:44:57.778786+0000","last_clean":"2024-08-02T17:44:57.778786+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:44:57.778786+0000","last_undegraded":"2024-08-02T17:44:57.778786+0000","last_fullsized":"2024-08-02T17:44:57.778786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778669+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:44:57.778669+0000","last_peered":"2024-08-02T17:44:57.778669+0000","last_clean":"2024-08-02T17:44:57.778669+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:44:57.778669+0000","last_undegraded":"2024-08-02T17:44:57.778669+0000","last_fullsized":"2024-08-02T17:44:57.778669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.793088+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:44:57.793088+0000","last_peered":"2024-08-02T17:44:57.793088+0000","last_clean":"2024-08-02T17:44:57.793088+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:44:57.793088+0000","last_undegraded":"2024-08-02T17:44:57.793088+0000","last_fullsized":"2024-08-02T17:44:57.793088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777673+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:44:57.777673+0000","last_peered":"2024-08-02T17:44:57.777673+0000","last_clean":"2024-08-02T17:44:57.777673+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:44:57.777673+0000","last_undegraded":"2024-08-02T17:44:57.777673+0000","last_fullsized":"2024-08-02T17:44:57.777673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778351+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:44:57.778351+0000","last_peered":"2024-08-02T17:44:57.778351+0000","last_clean":"2024-08-02T17:44:57.778351+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:44:57.778351+0000","last_undegraded":"2024-08-02T17:44:57.778351+0000","last_fullsized":"2024-08-02T17:44:57.778351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778275+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:44:57.778275+0000","last_peered":"2024-08-02T17:44:57.778275+0000","last_clean":"2024-08-02T17:44:57.778275+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:44:57.778275+0000","last_undegraded":"2024-08-02T17:44:57.778275+0000","last_fullsized":"2024-08-02T17:44:57.778275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775801+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:44:57.775801+0000","last_peered":"2024-08-02T17:44:57.775801+0000","last_clean":"2024-08-02T17:44:57.775801+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:44:57.775801+0000","last_undegraded":"2024-08-02T17:44:57.775801+0000","last_fullsized":"2024-08-02T17:44:57.775801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783178+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:44:57.783178+0000","last_peered":"2024-08-02T17:44:57.783178+0000","last_clean":"2024-08-02T17:44:57.783178+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:44:57.783178+0000","last_undegraded":"2024-08-02T17:44:57.783178+0000","last_fullsized":"2024-08-02T17:44:57.783178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058742+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:44:58.058742+0000","last_peered":"2024-08-02T17:44:58.058742+0000","last_clean":"2024-08-02T17:44:58.058742+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:44:58.058742+0000","last_undegraded":"2024-08-02T17:44:58.058742+0000","last_fullsized":"2024-08-02T17:44:58.058742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.789019+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:44:57.789019+0000","last_peered":"2024-08-02T17:44:57.789019+0000","last_clean":"2024-08-02T17:44:57.789019+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:44:57.789019+0000","last_undegraded":"2024-08-02T17:44:57.789019+0000","last_fullsized":"2024-08-02T17:44:57.789019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788382+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:44:57.788382+0000","last_peered":"2024-08-02T17:44:57.788382+0000","last_clean":"2024-08-02T17:44:57.788382+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:44:57.788382+0000","last_undegraded":"2024-08-02T17:44:57.788382+0000","last_fullsized":"2024-08-02T17:44:57.788382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779411+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:44:57.779411+0000","last_peered":"2024-08-02T17:44:57.779411+0000","last_clean":"2024-08-02T17:44:57.779411+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:44:57.779411+0000","last_undegraded":"2024-08-02T17:44:57.779411+0000","last_fullsized":"2024-08-02T17:44:57.779411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784040+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:44:57.784040+0000","last_peered":"2024-08-02T17:44:57.784040+0000","last_clean":"2024-08-02T17:44:57.784040+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:44:57.784040+0000","last_undegraded":"2024-08-02T17:44:57.784040+0000","last_fullsized":"2024-08-02T17:44:57.784040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778702+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:44:57.778702+0000","last_peered":"2024-08-02T17:44:57.778702+0000","last_clean":"2024-08-02T17:44:57.778702+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:44:57.778702+0000","last_undegraded":"2024-08-02T17:44:57.778702+0000","last_fullsized":"2024-08-02T17:44:57.778702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'12","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:03.143779+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:45:03.143779+0000","last_peered":"2024-08-02T17:45:03.143779+0000","last_clean":"2024-08-02T17:45:03.143779+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:45:03.143779+0000","last_undegraded":"2024-08-02T17:45:03.143779+0000","last_fullsized":"2024-08-02T17:45:03.143779+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777623+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:44:57.777623+0000","last_peered":"2024-08-02T17:44:57.777623+0000","last_clean":"2024-08-02T17:44:57.777623+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:44:57.777623+0000","last_undegraded":"2024-08-02T17:44:57.777623+0000","last_fullsized":"2024-08-02T17:44:57.777623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935126+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:44:57.935126+0000","last_peered":"2024-08-02T17:44:57.935126+0000","last_clean":"2024-08-02T17:44:57.935126+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:44:57.935126+0000","last_undegraded":"2024-08-02T17:44:57.935126+0000","last_fullsized":"2024-08-02T17:44:57.935126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784152+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:44:57.784152+0000","last_peered":"2024-08-02T17:44:57.784152+0000","last_clean":"2024-08-02T17:44:57.784152+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:44:57.784152+0000","last_undegraded":"2024-08-02T17:44:57.784152+0000","last_fullsized":"2024-08-02T17:44:57.784152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778014+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:44:57.778014+0000","last_peered":"2024-08-02T17:44:57.778014+0000","last_clean":"2024-08-02T17:44:57.778014+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:44:57.778014+0000","last_undegraded":"2024-08-02T17:44:57.778014+0000","last_fullsized":"2024-08-02T17:44:57.778014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779077+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:44:57.779077+0000","last_peered":"2024-08-02T17:44:57.779077+0000","last_clean":"2024-08-02T17:44:57.779077+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:44:57.779077+0000","last_undegraded":"2024-08-02T17:44:57.779077+0000","last_fullsized":"2024-08-02T17:44:57.779077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775849+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:44:57.775849+0000","last_peered":"2024-08-02T17:44:57.775849+0000","last_clean":"2024-08-02T17:44:57.775849+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:44:57.775849+0000","last_undegraded":"2024-08-02T17:44:57.775849+0000","last_fullsized":"2024-08-02T17:44:57.775849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784607+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:44:57.784607+0000","last_peered":"2024-08-02T17:44:57.784607+0000","last_clean":"2024-08-02T17:44:57.784607+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:44:57.784607+0000","last_undegraded":"2024-08-02T17:44:57.784607+0000","last_fullsized":"2024-08-02T17:44:57.784607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779247+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:44:57.779247+0000","last_peered":"2024-08-02T17:44:57.779247+0000","last_clean":"2024-08-02T17:44:57.779247+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:44:57.779247+0000","last_undegraded":"2024-08-02T17:44:57.779247+0000","last_fullsized":"2024-08-02T17:44:57.779247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777693+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:44:57.777693+0000","last_peered":"2024-08-02T17:44:57.777693+0000","last_clean":"2024-08-02T17:44:57.777693+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:44:57.777693+0000","last_undegraded":"2024-08-02T17:44:57.777693+0000","last_fullsized":"2024-08-02T17:44:57.777693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934899+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:44:57.934899+0000","last_peered":"2024-08-02T17:44:57.934899+0000","last_clean":"2024-08-02T17:44:57.934899+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:44:57.934899+0000","last_undegraded":"2024-08-02T17:44:57.934899+0000","last_fullsized":"2024-08-02T17:44:57.934899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779198+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:44:57.779198+0000","last_peered":"2024-08-02T17:44:57.779198+0000","last_clean":"2024-08-02T17:44:57.779198+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:44:57.779198+0000","last_undegraded":"2024-08-02T17:44:57.779198+0000","last_fullsized":"2024-08-02T17:44:57.779198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":86,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:34.985080+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:45:34.985080+0000","last_peered":"2024-08-02T17:45:34.985080+0000","last_clean":"2024-08-02T17:45:34.985080+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:45:34.985080+0000","last_undegraded":"2024-08-02T17:45:34.985080+0000","last_fullsized":"2024-08-02T17:45:34.985080+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934625+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:44:57.934625+0000","last_peered":"2024-08-02T17:44:57.934625+0000","last_clean":"2024-08-02T17:44:57.934625+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:44:57.934625+0000","last_undegraded":"2024-08-02T17:44:57.934625+0000","last_fullsized":"2024-08-02T17:44:57.934625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934293+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:44:57.934293+0000","last_peered":"2024-08-02T17:44:57.934293+0000","last_clean":"2024-08-02T17:44:57.934293+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:44:57.934293+0000","last_undegraded":"2024-08-02T17:44:57.934293+0000","last_fullsized":"2024-08-02T17:44:57.934293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.062270+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:44:58.062270+0000","last_peered":"2024-08-02T17:44:58.062270+0000","last_clean":"2024-08-02T17:44:58.062270+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:44:58.062270+0000","last_undegraded":"2024-08-02T17:44:58.062270+0000","last_fullsized":"2024-08-02T17:44:58.062270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780849+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:44:57.780849+0000","last_peered":"2024-08-02T17:44:57.780849+0000","last_clean":"2024-08-02T17:44:57.780849+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:44:57.780849+0000","last_undegraded":"2024-08-02T17:44:57.780849+0000","last_fullsized":"2024-08-02T17:44:57.780849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778368+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:44:57.778368+0000","last_peered":"2024-08-02T17:44:57.778368+0000","last_clean":"2024-08-02T17:44:57.778368+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:44:57.778368+0000","last_undegraded":"2024-08-02T17:44:57.778368+0000","last_fullsized":"2024-08-02T17:44:57.778368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778623+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:44:57.778623+0000","last_peered":"2024-08-02T17:44:57.778623+0000","last_clean":"2024-08-02T17:44:57.778623+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:44:57.778623+0000","last_undegraded":"2024-08-02T17:44:57.778623+0000","last_fullsized":"2024-08-02T17:44:57.778623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778590+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:44:57.778590+0000","last_peered":"2024-08-02T17:44:57.778590+0000","last_clean":"2024-08-02T17:44:57.778590+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:44:57.778590+0000","last_undegraded":"2024-08-02T17:44:57.778590+0000","last_fullsized":"2024-08-02T17:44:57.778590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.793617+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:44:57.793617+0000","last_peered":"2024-08-02T17:44:57.793617+0000","last_clean":"2024-08-02T17:44:57.793617+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:44:57.793617+0000","last_undegraded":"2024-08-02T17:44:57.793617+0000","last_fullsized":"2024-08-02T17:44:57.793617+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778382+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:44:57.778382+0000","last_peered":"2024-08-02T17:44:57.778382+0000","last_clean":"2024-08-02T17:44:57.778382+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:44:57.778382+0000","last_undegraded":"2024-08-02T17:44:57.778382+0000","last_fullsized":"2024-08-02T17:44:57.778382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784666+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:44:57.784666+0000","last_peered":"2024-08-02T17:44:57.784666+0000","last_clean":"2024-08-02T17:44:57.784666+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:44:57.784666+0000","last_undegraded":"2024-08-02T17:44:57.784666+0000","last_fullsized":"2024-08-02T17:44:57.784666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058379+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:44:58.058379+0000","last_peered":"2024-08-02T17:44:58.058379+0000","last_clean":"2024-08-02T17:44:58.058379+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:44:58.058379+0000","last_undegraded":"2024-08-02T17:44:58.058379+0000","last_fullsized":"2024-08-02T17:44:58.058379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777756+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:44:57.777756+0000","last_peered":"2024-08-02T17:44:57.777756+0000","last_clean":"2024-08-02T17:44:57.777756+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:44:57.777756+0000","last_undegraded":"2024-08-02T17:44:57.777756+0000","last_fullsized":"2024-08-02T17:44:57.777756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780624+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:44:57.780624+0000","last_peered":"2024-08-02T17:44:57.780624+0000","last_clean":"2024-08-02T17:44:57.780624+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:44:57.780624+0000","last_undegraded":"2024-08-02T17:44:57.780624+0000","last_fullsized":"2024-08-02T17:44:57.780624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788602+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:44:57.788602+0000","last_peered":"2024-08-02T17:44:57.788602+0000","last_clean":"2024-08-02T17:44:57.788602+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:44:57.788602+0000","last_undegraded":"2024-08-02T17:44:57.788602+0000","last_fullsized":"2024-08-02T17:44:57.788602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778360+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:44:57.778360+0000","last_peered":"2024-08-02T17:44:57.778360+0000","last_clean":"2024-08-02T17:44:57.778360+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:44:57.778360+0000","last_undegraded":"2024-08-02T17:44:57.778360+0000","last_fullsized":"2024-08-02T17:44:57.778360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934751+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:44:57.934751+0000","last_peered":"2024-08-02T17:44:57.934751+0000","last_clean":"2024-08-02T17:44:57.934751+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:44:57.934751+0000","last_undegraded":"2024-08-02T17:44:57.934751+0000","last_fullsized":"2024-08-02T17:44:57.934751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778111+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:44:57.778111+0000","last_peered":"2024-08-02T17:44:57.778111+0000","last_clean":"2024-08-02T17:44:57.778111+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:44:57.778111+0000","last_undegraded":"2024-08-02T17:44:57.778111+0000","last_fullsized":"2024-08-02T17:44:57.778111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778966+0000","last_change":"2024-08-02T17:44:04.131281+0000","last_active":"2024-08-02T17:44:57.778966+0000","last_peered":"2024-08-02T17:44:57.778966+0000","last_clean":"2024-08-02T17:44:57.778966+0000","last_became_active":"2024-08-02T17:44:04.130938+0000","last_became_peered":"2024-08-02T17:44:04.130938+0000","last_unstale":"2024-08-02T17:44:57.778966+0000","last_undegraded":"2024-08-02T17:44:57.778966+0000","last_fullsized":"2024-08-02T17:44:57.778966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:19:53.292216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.785999+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:44:57.785999+0000","last_peered":"2024-08-02T17:44:57.785999+0000","last_clean":"2024-08-02T17:44:57.785999+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:44:57.785999+0000","last_undegraded":"2024-08-02T17:44:57.785999+0000","last_fullsized":"2024-08-02T17:44:57.785999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779421+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:44:57.779421+0000","last_peered":"2024-08-02T17:44:57.779421+0000","last_clean":"2024-08-02T17:44:57.779421+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:44:57.779421+0000","last_undegraded":"2024-08-02T17:44:57.779421+0000","last_fullsized":"2024-08-02T17:44:57.779421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779500+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:44:57.779500+0000","last_peered":"2024-08-02T17:44:57.779500+0000","last_clean":"2024-08-02T17:44:57.779500+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:44:57.779500+0000","last_undegraded":"2024-08-02T17:44:57.779500+0000","last_fullsized":"2024-08-02T17:44:57.779500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":74,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.800995+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:44:57.800995+0000","last_peered":"2024-08-02T17:44:57.800995+0000","last_clean":"2024-08-02T17:44:57.800995+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:44:57.800995+0000","last_undegraded":"2024-08-02T17:44:57.800995+0000","last_fullsized":"2024-08-02T17:44:57.800995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934674+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:44:57.934674+0000","last_peered":"2024-08-02T17:44:57.934674+0000","last_clean":"2024-08-02T17:44:57.934674+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:44:57.934674+0000","last_undegraded":"2024-08-02T17:44:57.934674+0000","last_fullsized":"2024-08-02T17:44:57.934674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777470+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:44:57.777470+0000","last_peered":"2024-08-02T17:44:57.777470+0000","last_clean":"2024-08-02T17:44:57.777470+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:44:57.777470+0000","last_undegraded":"2024-08-02T17:44:57.777470+0000","last_fullsized":"2024-08-02T17:44:57.777470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":91,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:35.986984+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:45:35.986984+0000","last_peered":"2024-08-02T17:45:35.986984+0000","last_clean":"2024-08-02T17:45:35.986984+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:45:35.986984+0000","last_undegraded":"2024-08-02T17:45:35.986984+0000","last_fullsized":"2024-08-02T17:45:35.986984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775973+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:44:57.775973+0000","last_peered":"2024-08-02T17:44:57.775973+0000","last_clean":"2024-08-02T17:44:57.775973+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:44:57.775973+0000","last_undegraded":"2024-08-02T17:44:57.775973+0000","last_fullsized":"2024-08-02T17:44:57.775973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777872+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:44:57.777872+0000","last_peered":"2024-08-02T17:44:57.777872+0000","last_clean":"2024-08-02T17:44:57.777872+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:44:57.777872+0000","last_undegraded":"2024-08-02T17:44:57.777872+0000","last_fullsized":"2024-08-02T17:44:57.777872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779850+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:44:57.779850+0000","last_peered":"2024-08-02T17:44:57.779850+0000","last_clean":"2024-08-02T17:44:57.779850+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:44:57.779850+0000","last_undegraded":"2024-08-02T17:44:57.779850+0000","last_fullsized":"2024-08-02T17:44:57.779850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935930+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:44:57.935930+0000","last_peered":"2024-08-02T17:44:57.935930+0000","last_clean":"2024-08-02T17:44:57.935930+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:44:57.935930+0000","last_undegraded":"2024-08-02T17:44:57.935930+0000","last_fullsized":"2024-08-02T17:44:57.935930+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934481+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:44:57.934481+0000","last_peered":"2024-08-02T17:44:57.934481+0000","last_clean":"2024-08-02T17:44:57.934481+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:44:57.934481+0000","last_undegraded":"2024-08-02T17:44:57.934481+0000","last_fullsized":"2024-08-02T17:44:57.934481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784275+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:44:57.784275+0000","last_peered":"2024-08-02T17:44:57.784275+0000","last_clean":"2024-08-02T17:44:57.784275+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:44:57.784275+0000","last_undegraded":"2024-08-02T17:44:57.784275+0000","last_fullsized":"2024-08-02T17:44:57.784275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780202+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:44:57.780202+0000","last_peered":"2024-08-02T17:44:57.780202+0000","last_clean":"2024-08-02T17:44:57.780202+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:44:57.780202+0000","last_undegraded":"2024-08-02T17:44:57.780202+0000","last_fullsized":"2024-08-02T17:44:57.780202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783877+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:44:57.783877+0000","last_peered":"2024-08-02T17:44:57.783877+0000","last_clean":"2024-08-02T17:44:57.783877+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:44:57.783877+0000","last_undegraded":"2024-08-02T17:44:57.783877+0000","last_fullsized":"2024-08-02T17:44:57.783877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'32","reported_seq":175,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:36.987622+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:45:36.987622+0000","last_peered":"2024-08-02T17:45:36.987622+0000","last_clean":"2024-08-02T17:45:36.987622+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:45:36.987622+0000","last_undegraded":"2024-08-02T17:45:36.987622+0000","last_fullsized":"2024-08-02T17:45:36.987622+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62925,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":109,"num_read_kb":115,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.785527+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:44:57.785527+0000","last_peered":"2024-08-02T17:44:57.785527+0000","last_clean":"2024-08-02T17:44:57.785527+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:44:57.785527+0000","last_undegraded":"2024-08-02T17:44:57.785527+0000","last_fullsized":"2024-08-02T17:44:57.785527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058665+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:44:58.058665+0000","last_peered":"2024-08-02T17:44:58.058665+0000","last_clean":"2024-08-02T17:44:58.058665+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:44:58.058665+0000","last_undegraded":"2024-08-02T17:44:58.058665+0000","last_fullsized":"2024-08-02T17:44:58.058665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780514+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:44:57.780514+0000","last_peered":"2024-08-02T17:44:57.780514+0000","last_clean":"2024-08-02T17:44:57.780514+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:44:57.780514+0000","last_undegraded":"2024-08-02T17:44:57.780514+0000","last_fullsized":"2024-08-02T17:44:57.780514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778403+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:44:57.778403+0000","last_peered":"2024-08-02T17:44:57.778403+0000","last_clean":"2024-08-02T17:44:57.778403+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:44:57.778403+0000","last_undegraded":"2024-08-02T17:44:57.778403+0000","last_fullsized":"2024-08-02T17:44:57.778403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788772+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:44:57.788772+0000","last_peered":"2024-08-02T17:44:57.788772+0000","last_clean":"2024-08-02T17:44:57.788772+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:44:57.788772+0000","last_undegraded":"2024-08-02T17:44:57.788772+0000","last_fullsized":"2024-08-02T17:44:57.788772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778895+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:44:57.778895+0000","last_peered":"2024-08-02T17:44:57.778895+0000","last_clean":"2024-08-02T17:44:57.778895+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:44:57.778895+0000","last_undegraded":"2024-08-02T17:44:57.778895+0000","last_fullsized":"2024-08-02T17:44:57.778895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775322+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:44:57.775322+0000","last_peered":"2024-08-02T17:44:57.775322+0000","last_clean":"2024-08-02T17:44:57.775322+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:44:57.775322+0000","last_undegraded":"2024-08-02T17:44:57.775322+0000","last_fullsized":"2024-08-02T17:44:57.775322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058972+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:44:58.058972+0000","last_peered":"2024-08-02T17:44:58.058972+0000","last_clean":"2024-08-02T17:44:58.058972+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:44:58.058972+0000","last_undegraded":"2024-08-02T17:44:58.058972+0000","last_fullsized":"2024-08-02T17:44:58.058972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":87,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:35.986214+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:45:35.986214+0000","last_peered":"2024-08-02T17:45:35.986214+0000","last_clean":"2024-08-02T17:45:35.986214+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:45:35.986214+0000","last_undegraded":"2024-08-02T17:45:35.986214+0000","last_fullsized":"2024-08-02T17:45:35.986214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778076+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:44:57.778076+0000","last_peered":"2024-08-02T17:44:57.778076+0000","last_clean":"2024-08-02T17:44:57.778076+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:44:57.778076+0000","last_undegraded":"2024-08-02T17:44:57.778076+0000","last_fullsized":"2024-08-02T17:44:57.778076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788071+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:44:57.788071+0000","last_peered":"2024-08-02T17:44:57.788071+0000","last_clean":"2024-08-02T17:44:57.788071+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:44:57.788071+0000","last_undegraded":"2024-08-02T17:44:57.788071+0000","last_fullsized":"2024-08-02T17:44:57.788071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775451+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:44:57.775451+0000","last_peered":"2024-08-02T17:44:57.775451+0000","last_clean":"2024-08-02T17:44:57.775451+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:44:57.775451+0000","last_undegraded":"2024-08-02T17:44:57.775451+0000","last_fullsized":"2024-08-02T17:44:57.775451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779319+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:44:57.779319+0000","last_peered":"2024-08-02T17:44:57.779319+0000","last_clean":"2024-08-02T17:44:57.779319+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:44:57.779319+0000","last_undegraded":"2024-08-02T17:44:57.779319+0000","last_fullsized":"2024-08-02T17:44:57.779319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778983+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:44:57.778983+0000","last_peered":"2024-08-02T17:44:57.778983+0000","last_clean":"2024-08-02T17:44:57.778983+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:44:57.778983+0000","last_undegraded":"2024-08-02T17:44:57.778983+0000","last_fullsized":"2024-08-02T17:44:57.778983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777876+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:44:57.777876+0000","last_peered":"2024-08-02T17:44:57.777876+0000","last_clean":"2024-08-02T17:44:57.777876+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:44:57.777876+0000","last_undegraded":"2024-08-02T17:44:57.777876+0000","last_fullsized":"2024-08-02T17:44:57.777876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934332+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:44:57.934332+0000","last_peered":"2024-08-02T17:44:57.934332+0000","last_clean":"2024-08-02T17:44:57.934332+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:44:57.934332+0000","last_undegraded":"2024-08-02T17:44:57.934332+0000","last_fullsized":"2024-08-02T17:44:57.934332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778975+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:44:57.778975+0000","last_peered":"2024-08-02T17:44:57.778975+0000","last_clean":"2024-08-02T17:44:57.778975+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:44:57.778975+0000","last_undegraded":"2024-08-02T17:44:57.778975+0000","last_fullsized":"2024-08-02T17:44:57.778975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780193+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:44:57.780193+0000","last_peered":"2024-08-02T17:44:57.780193+0000","last_clean":"2024-08-02T17:44:57.780193+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:44:57.780193+0000","last_undegraded":"2024-08-02T17:44:57.780193+0000","last_fullsized":"2024-08-02T17:44:57.780193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784123+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:44:57.784123+0000","last_peered":"2024-08-02T17:44:57.784123+0000","last_clean":"2024-08-02T17:44:57.784123+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:44:57.784123+0000","last_undegraded":"2024-08-02T17:44:57.784123+0000","last_fullsized":"2024-08-02T17:44:57.784123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":93,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:36.988350+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:45:36.988350+0000","last_peered":"2024-08-02T17:45:36.988350+0000","last_clean":"2024-08-02T17:45:36.988350+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:45:36.988350+0000","last_undegraded":"2024-08-02T17:45:36.988350+0000","last_fullsized":"2024-08-02T17:45:36.988350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784723+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:44:57.784723+0000","last_peered":"2024-08-02T17:44:57.784723+0000","last_clean":"2024-08-02T17:44:57.784723+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:44:57.784723+0000","last_undegraded":"2024-08-02T17:44:57.784723+0000","last_fullsized":"2024-08-02T17:44:57.784723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784337+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:44:57.784337+0000","last_peered":"2024-08-02T17:44:57.784337+0000","last_clean":"2024-08-02T17:44:57.784337+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:44:57.784337+0000","last_undegraded":"2024-08-02T17:44:57.784337+0000","last_fullsized":"2024-08-02T17:44:57.784337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778261+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:44:57.778261+0000","last_peered":"2024-08-02T17:44:57.778261+0000","last_clean":"2024-08-02T17:44:57.778261+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:44:57.778261+0000","last_undegraded":"2024-08-02T17:44:57.778261+0000","last_fullsized":"2024-08-02T17:44:57.778261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784229+0000","last_change":"2024-08-02T17:44:04.127893+0000","last_active":"2024-08-02T17:44:57.784229+0000","last_peered":"2024-08-02T17:44:57.784229+0000","last_clean":"2024-08-02T17:44:57.784229+0000","last_became_active":"2024-08-02T17:44:04.127520+0000","last_became_peered":"2024-08-02T17:44:04.127520+0000","last_unstale":"2024-08-02T17:44:57.784229+0000","last_undegraded":"2024-08-02T17:44:57.784229+0000","last_fullsized":"2024-08-02T17:44:57.784229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:53.670606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782482+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:44:57.782482+0000","last_peered":"2024-08-02T17:44:57.782482+0000","last_clean":"2024-08-02T17:44:57.782482+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:44:57.782482+0000","last_undegraded":"2024-08-02T17:44:57.782482+0000","last_fullsized":"2024-08-02T17:44:57.782482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788689+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:44:57.788689+0000","last_peered":"2024-08-02T17:44:57.788689+0000","last_clean":"2024-08-02T17:44:57.788689+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:44:57.788689+0000","last_undegraded":"2024-08-02T17:44:57.788689+0000","last_fullsized":"2024-08-02T17:44:57.788689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935062+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:44:57.935062+0000","last_peered":"2024-08-02T17:44:57.935062+0000","last_clean":"2024-08-02T17:44:57.935062+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:44:57.935062+0000","last_undegraded":"2024-08-02T17:44:57.935062+0000","last_fullsized":"2024-08-02T17:44:57.935062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778967+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:44:57.778967+0000","last_peered":"2024-08-02T17:44:57.778967+0000","last_clean":"2024-08-02T17:44:57.778967+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:44:57.778967+0000","last_undegraded":"2024-08-02T17:44:57.778967+0000","last_fullsized":"2024-08-02T17:44:57.778967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782955+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:44:57.782955+0000","last_peered":"2024-08-02T17:44:57.782955+0000","last_clean":"2024-08-02T17:44:57.782955+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:44:57.782955+0000","last_undegraded":"2024-08-02T17:44:57.782955+0000","last_fullsized":"2024-08-02T17:44:57.782955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788682+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:44:57.788682+0000","last_peered":"2024-08-02T17:44:57.788682+0000","last_clean":"2024-08-02T17:44:57.788682+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:44:57.788682+0000","last_undegraded":"2024-08-02T17:44:57.788682+0000","last_fullsized":"2024-08-02T17:44:57.788682+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779156+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:44:57.779156+0000","last_peered":"2024-08-02T17:44:57.779156+0000","last_clean":"2024-08-02T17:44:57.779156+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:44:57.779156+0000","last_undegraded":"2024-08-02T17:44:57.779156+0000","last_fullsized":"2024-08-02T17:44:57.779156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778167+0000","last_change":"2024-08-02T17:44:04.124213+0000","last_active":"2024-08-02T17:44:57.778167+0000","last_peered":"2024-08-02T17:44:57.778167+0000","last_clean":"2024-08-02T17:44:57.778167+0000","last_became_active":"2024-08-02T17:44:04.123913+0000","last_became_peered":"2024-08-02T17:44:04.123913+0000","last_unstale":"2024-08-02T17:44:57.778167+0000","last_undegraded":"2024-08-02T17:44:57.778167+0000","last_fullsized":"2024-08-02T17:44:57.778167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:32:05.880054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058515+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:44:58.058515+0000","last_peered":"2024-08-02T17:44:58.058515+0000","last_clean":"2024-08-02T17:44:58.058515+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:44:58.058515+0000","last_undegraded":"2024-08-02T17:44:58.058515+0000","last_fullsized":"2024-08-02T17:44:58.058515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934630+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:44:57.934630+0000","last_peered":"2024-08-02T17:44:57.934630+0000","last_clean":"2024-08-02T17:44:57.934630+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:44:57.934630+0000","last_undegraded":"2024-08-02T17:44:57.934630+0000","last_fullsized":"2024-08-02T17:44:57.934630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778481+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:44:57.778481+0000","last_peered":"2024-08-02T17:44:57.778481+0000","last_clean":"2024-08-02T17:44:57.778481+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:44:57.778481+0000","last_undegraded":"2024-08-02T17:44:57.778481+0000","last_fullsized":"2024-08-02T17:44:57.778481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784986+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:44:57.784986+0000","last_peered":"2024-08-02T17:44:57.784986+0000","last_clean":"2024-08-02T17:44:57.784986+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:44:57.784986+0000","last_undegraded":"2024-08-02T17:44:57.784986+0000","last_fullsized":"2024-08-02T17:44:57.784986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.776049+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:44:57.776049+0000","last_peered":"2024-08-02T17:44:57.776049+0000","last_clean":"2024-08-02T17:44:57.776049+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:44:57.776049+0000","last_undegraded":"2024-08-02T17:44:57.776049+0000","last_fullsized":"2024-08-02T17:44:57.776049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779187+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:44:57.779187+0000","last_peered":"2024-08-02T17:44:57.779187+0000","last_clean":"2024-08-02T17:44:57.779187+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:44:57.779187+0000","last_undegraded":"2024-08-02T17:44:57.779187+0000","last_fullsized":"2024-08-02T17:44:57.779187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058678+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:44:58.058678+0000","last_peered":"2024-08-02T17:44:58.058678+0000","last_clean":"2024-08-02T17:44:58.058678+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:44:58.058678+0000","last_undegraded":"2024-08-02T17:44:58.058678+0000","last_fullsized":"2024-08-02T17:44:58.058678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788147+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:44:57.788147+0000","last_peered":"2024-08-02T17:44:57.788147+0000","last_clean":"2024-08-02T17:44:57.788147+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:44:57.788147+0000","last_undegraded":"2024-08-02T17:44:57.788147+0000","last_fullsized":"2024-08-02T17:44:57.788147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788368+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:44:57.788368+0000","last_peered":"2024-08-02T17:44:57.788368+0000","last_clean":"2024-08-02T17:44:57.788368+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:44:57.788368+0000","last_undegraded":"2024-08-02T17:44:57.788368+0000","last_fullsized":"2024-08-02T17:44:57.788368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934609+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:44:57.934609+0000","last_peered":"2024-08-02T17:44:57.934609+0000","last_clean":"2024-08-02T17:44:57.934609+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:44:57.934609+0000","last_undegraded":"2024-08-02T17:44:57.934609+0000","last_fullsized":"2024-08-02T17:44:57.934609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779201+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:44:57.779201+0000","last_peered":"2024-08-02T17:44:57.779201+0000","last_clean":"2024-08-02T17:44:57.779201+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:44:57.779201+0000","last_undegraded":"2024-08-02T17:44:57.779201+0000","last_fullsized":"2024-08-02T17:44:57.779201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778292+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:44:57.778292+0000","last_peered":"2024-08-02T17:44:57.778292+0000","last_clean":"2024-08-02T17:44:57.778292+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:44:57.778292+0000","last_undegraded":"2024-08-02T17:44:57.778292+0000","last_fullsized":"2024-08-02T17:44:57.778292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.786339+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:44:57.786339+0000","last_peered":"2024-08-02T17:44:57.786339+0000","last_clean":"2024-08-02T17:44:57.786339+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:44:57.786339+0000","last_undegraded":"2024-08-02T17:44:57.786339+0000","last_fullsized":"2024-08-02T17:44:57.786339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.059142+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:44:58.059142+0000","last_peered":"2024-08-02T17:44:58.059142+0000","last_clean":"2024-08-02T17:44:58.059142+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:44:58.059142+0000","last_undegraded":"2024-08-02T17:44:58.059142+0000","last_fullsized":"2024-08-02T17:44:58.059142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777697+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:44:57.777697+0000","last_peered":"2024-08-02T17:44:57.777697+0000","last_clean":"2024-08-02T17:44:57.777697+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:44:57.777697+0000","last_undegraded":"2024-08-02T17:44:57.777697+0000","last_fullsized":"2024-08-02T17:44:57.777697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778075+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:44:57.778075+0000","last_peered":"2024-08-02T17:44:57.778075+0000","last_clean":"2024-08-02T17:44:57.778075+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:44:57.778075+0000","last_undegraded":"2024-08-02T17:44:57.778075+0000","last_fullsized":"2024-08-02T17:44:57.778075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778851+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:44:57.778851+0000","last_peered":"2024-08-02T17:44:57.778851+0000","last_clean":"2024-08-02T17:44:57.778851+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:44:57.778851+0000","last_undegraded":"2024-08-02T17:44:57.778851+0000","last_fullsized":"2024-08-02T17:44:57.778851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783119+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:44:57.783119+0000","last_peered":"2024-08-02T17:44:57.783119+0000","last_clean":"2024-08-02T17:44:57.783119+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:44:57.783119+0000","last_undegraded":"2024-08-02T17:44:57.783119+0000","last_fullsized":"2024-08-02T17:44:57.783119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779098+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:44:57.779098+0000","last_peered":"2024-08-02T17:44:57.779098+0000","last_clean":"2024-08-02T17:44:57.779098+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:44:57.779098+0000","last_undegraded":"2024-08-02T17:44:57.779098+0000","last_fullsized":"2024-08-02T17:44:57.779098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788028+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:44:57.788028+0000","last_peered":"2024-08-02T17:44:57.788028+0000","last_clean":"2024-08-02T17:44:57.788028+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:44:57.788028+0000","last_undegraded":"2024-08-02T17:44:57.788028+0000","last_fullsized":"2024-08-02T17:44:57.788028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788302+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:44:57.788302+0000","last_peered":"2024-08-02T17:44:57.788302+0000","last_clean":"2024-08-02T17:44:57.788302+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:44:57.788302+0000","last_undegraded":"2024-08-02T17:44:57.788302+0000","last_fullsized":"2024-08-02T17:44:57.788302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779160+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:44:57.779160+0000","last_peered":"2024-08-02T17:44:57.779160+0000","last_clean":"2024-08-02T17:44:57.779160+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:44:57.779160+0000","last_undegraded":"2024-08-02T17:44:57.779160+0000","last_fullsized":"2024-08-02T17:44:57.779160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67315,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":329,"num_read_kb":234,"num_write":169,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":515190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678052,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1736704,"data_stored":1165773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":519522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103213,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":519442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364850,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1111644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528374,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712348,"statfs":{"total":95995035648,"available":95961444352,"internally_reserved":0,"allocated":1658880,"data_stored":1102746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215172,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":510796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":575065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":573437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:45:42.410 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph pg dump --format=json 2024-08-02T17:45:42.665 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:43.095 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:42 smithi069 ceph-mon[32746]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 -- 172.21.15.69:0/2697578267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1078b0 msgr2=0x7f1c4c107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2697578267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1078b0 0x7f1c4c107c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009930 tx=0x7f1c3c02a420 comp rx=0 tx=0).stop 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 -- 172.21.15.69:0/2697578267 shutdown_connections 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2697578267 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c102250 0x7f1c4c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2697578267 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c4c1018b0 0x7f1c4c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2697578267 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1078b0 0x7f1c4c107c90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 -- 172.21.15.69:0/2697578267 >> 172.21.15.69:0/2697578267 conn(0x7f1c4c0fd5f0 msgr2=0x7f1c4c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 -- 172.21.15.69:0/2697578267 shutdown_connections 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 -- 172.21.15.69:0/2697578267 wait complete. 2024-08-02T17:45:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.173+0000 7f1c53da4640 1 Processor -- start 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c53da4640 1 -- start start 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c53da4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1018b0 0x7f1c4c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c53da4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c102250 0x7f1c4c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c53da4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c4c1078b0 0x7f1c4c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c53da4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c111cd0 con 0x7f1c4c1018b0 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c53da4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c111b50 con 0x7f1c4c1078b0 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c53da4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4c111e50 con 0x7f1c4c102250 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1018b0 0x7f1c4c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1018b0 0x7f1c4c198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54768/0 (socket says 172.21.15.69:54768) 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 -- 172.21.15.69:0/2778021600 learned_addr learned my addr 172.21.15.69:0/2778021600 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c5133f640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c102250 0x7f1c4c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c52341640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c4c1078b0 0x7f1c4c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 -- 172.21.15.69:0/2778021600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c102250 msgr2=0x7f1c4c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c102250 0x7f1c4c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 -- 172.21.15.69:0/2778021600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c4c1078b0 msgr2=0x7f1c4c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c4c1078b0 0x7f1c4c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 -- 172.21.15.69:0/2778021600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c4c19d100 con 0x7f1c4c1018b0 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c52341640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c4c1078b0 0x7f1c4c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c5133f640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c102250 0x7f1c4c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.174+0000 7f1c51b40640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1018b0 0x7f1c4c198150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009a50 tx=0x7f1c3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.175+0000 7f1c42ffd640 1 -- 172.21.15.69:0/2778021600 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c03f030 con 0x7f1c4c1018b0 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.175+0000 7f1c42ffd640 1 -- 172.21.15.69:0/2778021600 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c3c003ce0 con 0x7f1c4c1018b0 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.175+0000 7f1c42ffd640 1 -- 172.21.15.69:0/2778021600 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c035070 con 0x7f1c4c1018b0 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.175+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c4c19d390 con 0x7f1c4c1018b0 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.175+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c4c1a4cd0 con 0x7f1c4c1018b0 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.175+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4c101d10 con 0x7f1c4c1018b0 2024-08-02T17:45:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.177+0000 7f1c42ffd640 1 -- 172.21.15.69:0/2778021600 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1c3c042030 con 0x7f1c4c1018b0 2024-08-02T17:45:43.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.178+0000 7f1c42ffd640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c380a2c90 0x7f1c380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:43.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.178+0000 7f1c5133f640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c380a2c90 0x7f1c380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:43.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.178+0000 7f1c42ffd640 1 -- 172.21.15.69:0/2778021600 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f1c3c0eff60 con 0x7f1c4c1018b0 2024-08-02T17:45:43.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.178+0000 7f1c5133f640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c380a2c90 0x7f1c380a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0ffbd0 tx=0x7f1c48005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:43.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.181+0000 7f1c42ffd640 1 -- 172.21.15.69:0/2778021600 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1c3c0bca00 con 0x7f1c4c1018b0 2024-08-02T17:45:43.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:42 smithi083 ceph-mon[37031]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:43.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.343+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c4c0fe470 con 0x7f1c380a2c90 2024-08-02T17:45:43.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:42 smithi077 ceph-mon[37108]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:43.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.351+0000 7f1c42ffd640 1 -- 172.21.15.69:0/2778021600 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340370 (secure 0 0 0) 0x7f1c4c0fe470 con 0x7f1c380a2c90 2024-08-02T17:45:43.356 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:43.357 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c380a2c90 msgr2=0x7f1c380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c380a2c90 0x7f1c380a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c0ffbd0 tx=0x7f1c48005d90 comp rx=0 tx=0).stop 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1018b0 msgr2=0x7f1c4c198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1018b0 0x7f1c4c198150 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009a50 tx=0x7f1c3c004270 comp rx=0 tx=0).stop 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 shutdown_connections 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1c380a2c90 0x7f1c380a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c4c1078b0 0x7f1c4c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c4c102250 0x7f1c4c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 --2- 172.21.15.69:0/2778021600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c4c1018b0 0x7f1c4c198150 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:43.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 >> 172.21.15.69:0/2778021600 conn(0x7f1c4c0fd5f0 msgr2=0x7f1c4c0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:43.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 shutdown_connections 2024-08-02T17:45:43.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:43.354+0000 7f1c53da4640 1 -- 172.21.15.69:0/2778021600 wait complete. 2024-08-02T17:45:44.089 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:43 smithi069 ceph-mon[32746]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:45:44.117 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-08-02T17:45:42.780799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657917,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":485,"num_read_kb":368,"num_write":352,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":283,"ondisk_log_size":283,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392648,"kb_used_data":14336,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549176,"statfs":{"total":1151940427776,"available":1151538356224,"internally_reserved":0,"allocated":14680064,"data_stored":8131961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003164"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.776089+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:44:57.776089+0000","last_peered":"2024-08-02T17:44:57.776089+0000","last_clean":"2024-08-02T17:44:57.776089+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:44:57.776089+0000","last_undegraded":"2024-08-02T17:44:57.776089+0000","last_fullsized":"2024-08-02T17:44:57.776089+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778005+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:44:57.778005+0000","last_peered":"2024-08-02T17:44:57.778005+0000","last_clean":"2024-08-02T17:44:57.778005+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:44:57.778005+0000","last_undegraded":"2024-08-02T17:44:57.778005+0000","last_fullsized":"2024-08-02T17:44:57.778005+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782153+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:44:57.782153+0000","last_peered":"2024-08-02T17:44:57.782153+0000","last_clean":"2024-08-02T17:44:57.782153+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:44:57.782153+0000","last_undegraded":"2024-08-02T17:44:57.782153+0000","last_fullsized":"2024-08-02T17:44:57.782153+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778542+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:44:57.778542+0000","last_peered":"2024-08-02T17:44:57.778542+0000","last_clean":"2024-08-02T17:44:57.778542+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:44:57.778542+0000","last_undegraded":"2024-08-02T17:44:57.778542+0000","last_fullsized":"2024-08-02T17:44:57.778542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788699+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:44:57.788699+0000","last_peered":"2024-08-02T17:44:57.788699+0000","last_clean":"2024-08-02T17:44:57.788699+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:44:57.788699+0000","last_undegraded":"2024-08-02T17:44:57.788699+0000","last_fullsized":"2024-08-02T17:44:57.788699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777204+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:44:57.777204+0000","last_peered":"2024-08-02T17:44:57.777204+0000","last_clean":"2024-08-02T17:44:57.777204+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:44:57.777204+0000","last_undegraded":"2024-08-02T17:44:57.777204+0000","last_fullsized":"2024-08-02T17:44:57.777204+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784045+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:44:57.784045+0000","last_peered":"2024-08-02T17:44:57.784045+0000","last_clean":"2024-08-02T17:44:57.784045+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:44:57.784045+0000","last_undegraded":"2024-08-02T17:44:57.784045+0000","last_fullsized":"2024-08-02T17:44:57.784045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058658+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:44:58.058658+0000","last_peered":"2024-08-02T17:44:58.058658+0000","last_clean":"2024-08-02T17:44:58.058658+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:44:58.058658+0000","last_undegraded":"2024-08-02T17:44:58.058658+0000","last_fullsized":"2024-08-02T17:44:58.058658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778197+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:44:57.778197+0000","last_peered":"2024-08-02T17:44:57.778197+0000","last_clean":"2024-08-02T17:44:57.778197+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:44:57.778197+0000","last_undegraded":"2024-08-02T17:44:57.778197+0000","last_fullsized":"2024-08-02T17:44:57.778197+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783541+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:44:57.783541+0000","last_peered":"2024-08-02T17:44:57.783541+0000","last_clean":"2024-08-02T17:44:57.783541+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:44:57.783541+0000","last_undegraded":"2024-08-02T17:44:57.783541+0000","last_fullsized":"2024-08-02T17:44:57.783541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780343+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:44:57.780343+0000","last_peered":"2024-08-02T17:44:57.780343+0000","last_clean":"2024-08-02T17:44:57.780343+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:44:57.780343+0000","last_undegraded":"2024-08-02T17:44:57.780343+0000","last_fullsized":"2024-08-02T17:44:57.780343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777417+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:44:57.777417+0000","last_peered":"2024-08-02T17:44:57.777417+0000","last_clean":"2024-08-02T17:44:57.777417+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:44:57.777417+0000","last_undegraded":"2024-08-02T17:44:57.777417+0000","last_fullsized":"2024-08-02T17:44:57.777417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777532+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:44:57.777532+0000","last_peered":"2024-08-02T17:44:57.777532+0000","last_clean":"2024-08-02T17:44:57.777532+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:44:57.777532+0000","last_undegraded":"2024-08-02T17:44:57.777532+0000","last_fullsized":"2024-08-02T17:44:57.777532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783178+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:44:57.783178+0000","last_peered":"2024-08-02T17:44:57.783178+0000","last_clean":"2024-08-02T17:44:57.783178+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:44:57.783178+0000","last_undegraded":"2024-08-02T17:44:57.783178+0000","last_fullsized":"2024-08-02T17:44:57.783178+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778786+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:44:57.778786+0000","last_peered":"2024-08-02T17:44:57.778786+0000","last_clean":"2024-08-02T17:44:57.778786+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:44:57.778786+0000","last_undegraded":"2024-08-02T17:44:57.778786+0000","last_fullsized":"2024-08-02T17:44:57.778786+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778669+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:44:57.778669+0000","last_peered":"2024-08-02T17:44:57.778669+0000","last_clean":"2024-08-02T17:44:57.778669+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:44:57.778669+0000","last_undegraded":"2024-08-02T17:44:57.778669+0000","last_fullsized":"2024-08-02T17:44:57.778669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.793088+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:44:57.793088+0000","last_peered":"2024-08-02T17:44:57.793088+0000","last_clean":"2024-08-02T17:44:57.793088+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:44:57.793088+0000","last_undegraded":"2024-08-02T17:44:57.793088+0000","last_fullsized":"2024-08-02T17:44:57.793088+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777673+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:44:57.777673+0000","last_peered":"2024-08-02T17:44:57.777673+0000","last_clean":"2024-08-02T17:44:57.777673+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:44:57.777673+0000","last_undegraded":"2024-08-02T17:44:57.777673+0000","last_fullsized":"2024-08-02T17:44:57.777673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778351+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:44:57.778351+0000","last_peered":"2024-08-02T17:44:57.778351+0000","last_clean":"2024-08-02T17:44:57.778351+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:44:57.778351+0000","last_undegraded":"2024-08-02T17:44:57.778351+0000","last_fullsized":"2024-08-02T17:44:57.778351+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778275+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:44:57.778275+0000","last_peered":"2024-08-02T17:44:57.778275+0000","last_clean":"2024-08-02T17:44:57.778275+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:44:57.778275+0000","last_undegraded":"2024-08-02T17:44:57.778275+0000","last_fullsized":"2024-08-02T17:44:57.778275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775801+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:44:57.775801+0000","last_peered":"2024-08-02T17:44:57.775801+0000","last_clean":"2024-08-02T17:44:57.775801+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:44:57.775801+0000","last_undegraded":"2024-08-02T17:44:57.775801+0000","last_fullsized":"2024-08-02T17:44:57.775801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783178+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:44:57.783178+0000","last_peered":"2024-08-02T17:44:57.783178+0000","last_clean":"2024-08-02T17:44:57.783178+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:44:57.783178+0000","last_undegraded":"2024-08-02T17:44:57.783178+0000","last_fullsized":"2024-08-02T17:44:57.783178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058742+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:44:58.058742+0000","last_peered":"2024-08-02T17:44:58.058742+0000","last_clean":"2024-08-02T17:44:58.058742+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:44:58.058742+0000","last_undegraded":"2024-08-02T17:44:58.058742+0000","last_fullsized":"2024-08-02T17:44:58.058742+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.789019+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:44:57.789019+0000","last_peered":"2024-08-02T17:44:57.789019+0000","last_clean":"2024-08-02T17:44:57.789019+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:44:57.789019+0000","last_undegraded":"2024-08-02T17:44:57.789019+0000","last_fullsized":"2024-08-02T17:44:57.789019+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788382+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:44:57.788382+0000","last_peered":"2024-08-02T17:44:57.788382+0000","last_clean":"2024-08-02T17:44:57.788382+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:44:57.788382+0000","last_undegraded":"2024-08-02T17:44:57.788382+0000","last_fullsized":"2024-08-02T17:44:57.788382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779411+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:44:57.779411+0000","last_peered":"2024-08-02T17:44:57.779411+0000","last_clean":"2024-08-02T17:44:57.779411+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:44:57.779411+0000","last_undegraded":"2024-08-02T17:44:57.779411+0000","last_fullsized":"2024-08-02T17:44:57.779411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784040+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:44:57.784040+0000","last_peered":"2024-08-02T17:44:57.784040+0000","last_clean":"2024-08-02T17:44:57.784040+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:44:57.784040+0000","last_undegraded":"2024-08-02T17:44:57.784040+0000","last_fullsized":"2024-08-02T17:44:57.784040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778702+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:44:57.778702+0000","last_peered":"2024-08-02T17:44:57.778702+0000","last_clean":"2024-08-02T17:44:57.778702+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:44:57.778702+0000","last_undegraded":"2024-08-02T17:44:57.778702+0000","last_fullsized":"2024-08-02T17:44:57.778702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'12","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:03.143779+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:45:03.143779+0000","last_peered":"2024-08-02T17:45:03.143779+0000","last_clean":"2024-08-02T17:45:03.143779+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:45:03.143779+0000","last_undegraded":"2024-08-02T17:45:03.143779+0000","last_fullsized":"2024-08-02T17:45:03.143779+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777623+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:44:57.777623+0000","last_peered":"2024-08-02T17:44:57.777623+0000","last_clean":"2024-08-02T17:44:57.777623+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:44:57.777623+0000","last_undegraded":"2024-08-02T17:44:57.777623+0000","last_fullsized":"2024-08-02T17:44:57.777623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935126+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:44:57.935126+0000","last_peered":"2024-08-02T17:44:57.935126+0000","last_clean":"2024-08-02T17:44:57.935126+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:44:57.935126+0000","last_undegraded":"2024-08-02T17:44:57.935126+0000","last_fullsized":"2024-08-02T17:44:57.935126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784152+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:44:57.784152+0000","last_peered":"2024-08-02T17:44:57.784152+0000","last_clean":"2024-08-02T17:44:57.784152+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:44:57.784152+0000","last_undegraded":"2024-08-02T17:44:57.784152+0000","last_fullsized":"2024-08-02T17:44:57.784152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778014+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:44:57.778014+0000","last_peered":"2024-08-02T17:44:57.778014+0000","last_clean":"2024-08-02T17:44:57.778014+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:44:57.778014+0000","last_undegraded":"2024-08-02T17:44:57.778014+0000","last_fullsized":"2024-08-02T17:44:57.778014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779077+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:44:57.779077+0000","last_peered":"2024-08-02T17:44:57.779077+0000","last_clean":"2024-08-02T17:44:57.779077+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:44:57.779077+0000","last_undegraded":"2024-08-02T17:44:57.779077+0000","last_fullsized":"2024-08-02T17:44:57.779077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775849+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:44:57.775849+0000","last_peered":"2024-08-02T17:44:57.775849+0000","last_clean":"2024-08-02T17:44:57.775849+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:44:57.775849+0000","last_undegraded":"2024-08-02T17:44:57.775849+0000","last_fullsized":"2024-08-02T17:44:57.775849+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784607+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:44:57.784607+0000","last_peered":"2024-08-02T17:44:57.784607+0000","last_clean":"2024-08-02T17:44:57.784607+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:44:57.784607+0000","last_undegraded":"2024-08-02T17:44:57.784607+0000","last_fullsized":"2024-08-02T17:44:57.784607+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779247+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:44:57.779247+0000","last_peered":"2024-08-02T17:44:57.779247+0000","last_clean":"2024-08-02T17:44:57.779247+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:44:57.779247+0000","last_undegraded":"2024-08-02T17:44:57.779247+0000","last_fullsized":"2024-08-02T17:44:57.779247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777693+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:44:57.777693+0000","last_peered":"2024-08-02T17:44:57.777693+0000","last_clean":"2024-08-02T17:44:57.777693+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:44:57.777693+0000","last_undegraded":"2024-08-02T17:44:57.777693+0000","last_fullsized":"2024-08-02T17:44:57.777693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934899+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:44:57.934899+0000","last_peered":"2024-08-02T17:44:57.934899+0000","last_clean":"2024-08-02T17:44:57.934899+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:44:57.934899+0000","last_undegraded":"2024-08-02T17:44:57.934899+0000","last_fullsized":"2024-08-02T17:44:57.934899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779198+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:44:57.779198+0000","last_peered":"2024-08-02T17:44:57.779198+0000","last_clean":"2024-08-02T17:44:57.779198+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:44:57.779198+0000","last_undegraded":"2024-08-02T17:44:57.779198+0000","last_fullsized":"2024-08-02T17:44:57.779198+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":91,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:39.992103+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:45:39.992103+0000","last_peered":"2024-08-02T17:45:39.992103+0000","last_clean":"2024-08-02T17:45:39.992103+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:45:39.992103+0000","last_undegraded":"2024-08-02T17:45:39.992103+0000","last_fullsized":"2024-08-02T17:45:39.992103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934625+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:44:57.934625+0000","last_peered":"2024-08-02T17:44:57.934625+0000","last_clean":"2024-08-02T17:44:57.934625+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:44:57.934625+0000","last_undegraded":"2024-08-02T17:44:57.934625+0000","last_fullsized":"2024-08-02T17:44:57.934625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934293+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:44:57.934293+0000","last_peered":"2024-08-02T17:44:57.934293+0000","last_clean":"2024-08-02T17:44:57.934293+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:44:57.934293+0000","last_undegraded":"2024-08-02T17:44:57.934293+0000","last_fullsized":"2024-08-02T17:44:57.934293+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":46,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.062270+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:44:58.062270+0000","last_peered":"2024-08-02T17:44:58.062270+0000","last_clean":"2024-08-02T17:44:58.062270+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:44:58.062270+0000","last_undegraded":"2024-08-02T17:44:58.062270+0000","last_fullsized":"2024-08-02T17:44:58.062270+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780849+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:44:57.780849+0000","last_peered":"2024-08-02T17:44:57.780849+0000","last_clean":"2024-08-02T17:44:57.780849+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:44:57.780849+0000","last_undegraded":"2024-08-02T17:44:57.780849+0000","last_fullsized":"2024-08-02T17:44:57.780849+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778368+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:44:57.778368+0000","last_peered":"2024-08-02T17:44:57.778368+0000","last_clean":"2024-08-02T17:44:57.778368+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:44:57.778368+0000","last_undegraded":"2024-08-02T17:44:57.778368+0000","last_fullsized":"2024-08-02T17:44:57.778368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778623+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:44:57.778623+0000","last_peered":"2024-08-02T17:44:57.778623+0000","last_clean":"2024-08-02T17:44:57.778623+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:44:57.778623+0000","last_undegraded":"2024-08-02T17:44:57.778623+0000","last_fullsized":"2024-08-02T17:44:57.778623+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778590+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:44:57.778590+0000","last_peered":"2024-08-02T17:44:57.778590+0000","last_clean":"2024-08-02T17:44:57.778590+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:44:57.778590+0000","last_undegraded":"2024-08-02T17:44:57.778590+0000","last_fullsized":"2024-08-02T17:44:57.778590+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.793617+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:44:57.793617+0000","last_peered":"2024-08-02T17:44:57.793617+0000","last_clean":"2024-08-02T17:44:57.793617+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:44:57.793617+0000","last_undegraded":"2024-08-02T17:44:57.793617+0000","last_fullsized":"2024-08-02T17:44:57.793617+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778382+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:44:57.778382+0000","last_peered":"2024-08-02T17:44:57.778382+0000","last_clean":"2024-08-02T17:44:57.778382+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:44:57.778382+0000","last_undegraded":"2024-08-02T17:44:57.778382+0000","last_fullsized":"2024-08-02T17:44:57.778382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784666+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:44:57.784666+0000","last_peered":"2024-08-02T17:44:57.784666+0000","last_clean":"2024-08-02T17:44:57.784666+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:44:57.784666+0000","last_undegraded":"2024-08-02T17:44:57.784666+0000","last_fullsized":"2024-08-02T17:44:57.784666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058379+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:44:58.058379+0000","last_peered":"2024-08-02T17:44:58.058379+0000","last_clean":"2024-08-02T17:44:58.058379+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:44:58.058379+0000","last_undegraded":"2024-08-02T17:44:58.058379+0000","last_fullsized":"2024-08-02T17:44:58.058379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777756+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:44:57.777756+0000","last_peered":"2024-08-02T17:44:57.777756+0000","last_clean":"2024-08-02T17:44:57.777756+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:44:57.777756+0000","last_undegraded":"2024-08-02T17:44:57.777756+0000","last_fullsized":"2024-08-02T17:44:57.777756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780624+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:44:57.780624+0000","last_peered":"2024-08-02T17:44:57.780624+0000","last_clean":"2024-08-02T17:44:57.780624+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:44:57.780624+0000","last_undegraded":"2024-08-02T17:44:57.780624+0000","last_fullsized":"2024-08-02T17:44:57.780624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788602+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:44:57.788602+0000","last_peered":"2024-08-02T17:44:57.788602+0000","last_clean":"2024-08-02T17:44:57.788602+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:44:57.788602+0000","last_undegraded":"2024-08-02T17:44:57.788602+0000","last_fullsized":"2024-08-02T17:44:57.788602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778360+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:44:57.778360+0000","last_peered":"2024-08-02T17:44:57.778360+0000","last_clean":"2024-08-02T17:44:57.778360+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:44:57.778360+0000","last_undegraded":"2024-08-02T17:44:57.778360+0000","last_fullsized":"2024-08-02T17:44:57.778360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934751+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:44:57.934751+0000","last_peered":"2024-08-02T17:44:57.934751+0000","last_clean":"2024-08-02T17:44:57.934751+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:44:57.934751+0000","last_undegraded":"2024-08-02T17:44:57.934751+0000","last_fullsized":"2024-08-02T17:44:57.934751+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778111+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:44:57.778111+0000","last_peered":"2024-08-02T17:44:57.778111+0000","last_clean":"2024-08-02T17:44:57.778111+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:44:57.778111+0000","last_undegraded":"2024-08-02T17:44:57.778111+0000","last_fullsized":"2024-08-02T17:44:57.778111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778966+0000","last_change":"2024-08-02T17:44:04.131281+0000","last_active":"2024-08-02T17:44:57.778966+0000","last_peered":"2024-08-02T17:44:57.778966+0000","last_clean":"2024-08-02T17:44:57.778966+0000","last_became_active":"2024-08-02T17:44:04.130938+0000","last_became_peered":"2024-08-02T17:44:04.130938+0000","last_unstale":"2024-08-02T17:44:57.778966+0000","last_undegraded":"2024-08-02T17:44:57.778966+0000","last_fullsized":"2024-08-02T17:44:57.778966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:19:53.292216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.785999+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:44:57.785999+0000","last_peered":"2024-08-02T17:44:57.785999+0000","last_clean":"2024-08-02T17:44:57.785999+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:44:57.785999+0000","last_undegraded":"2024-08-02T17:44:57.785999+0000","last_fullsized":"2024-08-02T17:44:57.785999+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779421+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:44:57.779421+0000","last_peered":"2024-08-02T17:44:57.779421+0000","last_clean":"2024-08-02T17:44:57.779421+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:44:57.779421+0000","last_undegraded":"2024-08-02T17:44:57.779421+0000","last_fullsized":"2024-08-02T17:44:57.779421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779500+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:44:57.779500+0000","last_peered":"2024-08-02T17:44:57.779500+0000","last_clean":"2024-08-02T17:44:57.779500+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:44:57.779500+0000","last_undegraded":"2024-08-02T17:44:57.779500+0000","last_fullsized":"2024-08-02T17:44:57.779500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":74,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.800995+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:44:57.800995+0000","last_peered":"2024-08-02T17:44:57.800995+0000","last_clean":"2024-08-02T17:44:57.800995+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:44:57.800995+0000","last_undegraded":"2024-08-02T17:44:57.800995+0000","last_fullsized":"2024-08-02T17:44:57.800995+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934674+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:44:57.934674+0000","last_peered":"2024-08-02T17:44:57.934674+0000","last_clean":"2024-08-02T17:44:57.934674+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:44:57.934674+0000","last_undegraded":"2024-08-02T17:44:57.934674+0000","last_fullsized":"2024-08-02T17:44:57.934674+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777470+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:44:57.777470+0000","last_peered":"2024-08-02T17:44:57.777470+0000","last_clean":"2024-08-02T17:44:57.777470+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:44:57.777470+0000","last_undegraded":"2024-08-02T17:44:57.777470+0000","last_fullsized":"2024-08-02T17:44:57.777470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":91,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:35.986984+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:45:35.986984+0000","last_peered":"2024-08-02T17:45:35.986984+0000","last_clean":"2024-08-02T17:45:35.986984+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:45:35.986984+0000","last_undegraded":"2024-08-02T17:45:35.986984+0000","last_fullsized":"2024-08-02T17:45:35.986984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775973+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:44:57.775973+0000","last_peered":"2024-08-02T17:44:57.775973+0000","last_clean":"2024-08-02T17:44:57.775973+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:44:57.775973+0000","last_undegraded":"2024-08-02T17:44:57.775973+0000","last_fullsized":"2024-08-02T17:44:57.775973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777872+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:44:57.777872+0000","last_peered":"2024-08-02T17:44:57.777872+0000","last_clean":"2024-08-02T17:44:57.777872+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:44:57.777872+0000","last_undegraded":"2024-08-02T17:44:57.777872+0000","last_fullsized":"2024-08-02T17:44:57.777872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779850+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:44:57.779850+0000","last_peered":"2024-08-02T17:44:57.779850+0000","last_clean":"2024-08-02T17:44:57.779850+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:44:57.779850+0000","last_undegraded":"2024-08-02T17:44:57.779850+0000","last_fullsized":"2024-08-02T17:44:57.779850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935930+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:44:57.935930+0000","last_peered":"2024-08-02T17:44:57.935930+0000","last_clean":"2024-08-02T17:44:57.935930+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:44:57.935930+0000","last_undegraded":"2024-08-02T17:44:57.935930+0000","last_fullsized":"2024-08-02T17:44:57.935930+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934481+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:44:57.934481+0000","last_peered":"2024-08-02T17:44:57.934481+0000","last_clean":"2024-08-02T17:44:57.934481+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:44:57.934481+0000","last_undegraded":"2024-08-02T17:44:57.934481+0000","last_fullsized":"2024-08-02T17:44:57.934481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784275+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:44:57.784275+0000","last_peered":"2024-08-02T17:44:57.784275+0000","last_clean":"2024-08-02T17:44:57.784275+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:44:57.784275+0000","last_undegraded":"2024-08-02T17:44:57.784275+0000","last_fullsized":"2024-08-02T17:44:57.784275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780202+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:44:57.780202+0000","last_peered":"2024-08-02T17:44:57.780202+0000","last_clean":"2024-08-02T17:44:57.780202+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:44:57.780202+0000","last_undegraded":"2024-08-02T17:44:57.780202+0000","last_fullsized":"2024-08-02T17:44:57.780202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783877+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:44:57.783877+0000","last_peered":"2024-08-02T17:44:57.783877+0000","last_clean":"2024-08-02T17:44:57.783877+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:44:57.783877+0000","last_undegraded":"2024-08-02T17:44:57.783877+0000","last_fullsized":"2024-08-02T17:44:57.783877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'32","reported_seq":185,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:41.994523+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:45:41.994523+0000","last_peered":"2024-08-02T17:45:41.994523+0000","last_clean":"2024-08-02T17:45:41.994523+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:45:41.994523+0000","last_undegraded":"2024-08-02T17:45:41.994523+0000","last_fullsized":"2024-08-02T17:45:41.994523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62925,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":119,"num_read_kb":120,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.785527+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:44:57.785527+0000","last_peered":"2024-08-02T17:44:57.785527+0000","last_clean":"2024-08-02T17:44:57.785527+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:44:57.785527+0000","last_undegraded":"2024-08-02T17:44:57.785527+0000","last_fullsized":"2024-08-02T17:44:57.785527+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058665+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:44:58.058665+0000","last_peered":"2024-08-02T17:44:58.058665+0000","last_clean":"2024-08-02T17:44:58.058665+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:44:58.058665+0000","last_undegraded":"2024-08-02T17:44:58.058665+0000","last_fullsized":"2024-08-02T17:44:58.058665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780514+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:44:57.780514+0000","last_peered":"2024-08-02T17:44:57.780514+0000","last_clean":"2024-08-02T17:44:57.780514+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:44:57.780514+0000","last_undegraded":"2024-08-02T17:44:57.780514+0000","last_fullsized":"2024-08-02T17:44:57.780514+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778403+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:44:57.778403+0000","last_peered":"2024-08-02T17:44:57.778403+0000","last_clean":"2024-08-02T17:44:57.778403+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:44:57.778403+0000","last_undegraded":"2024-08-02T17:44:57.778403+0000","last_fullsized":"2024-08-02T17:44:57.778403+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788772+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:44:57.788772+0000","last_peered":"2024-08-02T17:44:57.788772+0000","last_clean":"2024-08-02T17:44:57.788772+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:44:57.788772+0000","last_undegraded":"2024-08-02T17:44:57.788772+0000","last_fullsized":"2024-08-02T17:44:57.788772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778895+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:44:57.778895+0000","last_peered":"2024-08-02T17:44:57.778895+0000","last_clean":"2024-08-02T17:44:57.778895+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:44:57.778895+0000","last_undegraded":"2024-08-02T17:44:57.778895+0000","last_fullsized":"2024-08-02T17:44:57.778895+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775322+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:44:57.775322+0000","last_peered":"2024-08-02T17:44:57.775322+0000","last_clean":"2024-08-02T17:44:57.775322+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:44:57.775322+0000","last_undegraded":"2024-08-02T17:44:57.775322+0000","last_fullsized":"2024-08-02T17:44:57.775322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058972+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:44:58.058972+0000","last_peered":"2024-08-02T17:44:58.058972+0000","last_clean":"2024-08-02T17:44:58.058972+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:44:58.058972+0000","last_undegraded":"2024-08-02T17:44:58.058972+0000","last_fullsized":"2024-08-02T17:44:58.058972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":92,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:40.993190+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:45:40.993190+0000","last_peered":"2024-08-02T17:45:40.993190+0000","last_clean":"2024-08-02T17:45:40.993190+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:45:40.993190+0000","last_undegraded":"2024-08-02T17:45:40.993190+0000","last_fullsized":"2024-08-02T17:45:40.993190+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778076+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:44:57.778076+0000","last_peered":"2024-08-02T17:44:57.778076+0000","last_clean":"2024-08-02T17:44:57.778076+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:44:57.778076+0000","last_undegraded":"2024-08-02T17:44:57.778076+0000","last_fullsized":"2024-08-02T17:44:57.778076+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788071+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:44:57.788071+0000","last_peered":"2024-08-02T17:44:57.788071+0000","last_clean":"2024-08-02T17:44:57.788071+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:44:57.788071+0000","last_undegraded":"2024-08-02T17:44:57.788071+0000","last_fullsized":"2024-08-02T17:44:57.788071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.775451+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:44:57.775451+0000","last_peered":"2024-08-02T17:44:57.775451+0000","last_clean":"2024-08-02T17:44:57.775451+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:44:57.775451+0000","last_undegraded":"2024-08-02T17:44:57.775451+0000","last_fullsized":"2024-08-02T17:44:57.775451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779319+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:44:57.779319+0000","last_peered":"2024-08-02T17:44:57.779319+0000","last_clean":"2024-08-02T17:44:57.779319+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:44:57.779319+0000","last_undegraded":"2024-08-02T17:44:57.779319+0000","last_fullsized":"2024-08-02T17:44:57.779319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778983+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:44:57.778983+0000","last_peered":"2024-08-02T17:44:57.778983+0000","last_clean":"2024-08-02T17:44:57.778983+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:44:57.778983+0000","last_undegraded":"2024-08-02T17:44:57.778983+0000","last_fullsized":"2024-08-02T17:44:57.778983+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777876+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:44:57.777876+0000","last_peered":"2024-08-02T17:44:57.777876+0000","last_clean":"2024-08-02T17:44:57.777876+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:44:57.777876+0000","last_undegraded":"2024-08-02T17:44:57.777876+0000","last_fullsized":"2024-08-02T17:44:57.777876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934332+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:44:57.934332+0000","last_peered":"2024-08-02T17:44:57.934332+0000","last_clean":"2024-08-02T17:44:57.934332+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:44:57.934332+0000","last_undegraded":"2024-08-02T17:44:57.934332+0000","last_fullsized":"2024-08-02T17:44:57.934332+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778975+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:44:57.778975+0000","last_peered":"2024-08-02T17:44:57.778975+0000","last_clean":"2024-08-02T17:44:57.778975+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:44:57.778975+0000","last_undegraded":"2024-08-02T17:44:57.778975+0000","last_fullsized":"2024-08-02T17:44:57.778975+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.780193+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:44:57.780193+0000","last_peered":"2024-08-02T17:44:57.780193+0000","last_clean":"2024-08-02T17:44:57.780193+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:44:57.780193+0000","last_undegraded":"2024-08-02T17:44:57.780193+0000","last_fullsized":"2024-08-02T17:44:57.780193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784123+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:44:57.784123+0000","last_peered":"2024-08-02T17:44:57.784123+0000","last_clean":"2024-08-02T17:44:57.784123+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:44:57.784123+0000","last_undegraded":"2024-08-02T17:44:57.784123+0000","last_fullsized":"2024-08-02T17:44:57.784123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":93,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:45:36.988350+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:45:36.988350+0000","last_peered":"2024-08-02T17:45:36.988350+0000","last_clean":"2024-08-02T17:45:36.988350+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:45:36.988350+0000","last_undegraded":"2024-08-02T17:45:36.988350+0000","last_fullsized":"2024-08-02T17:45:36.988350+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784723+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:44:57.784723+0000","last_peered":"2024-08-02T17:44:57.784723+0000","last_clean":"2024-08-02T17:44:57.784723+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:44:57.784723+0000","last_undegraded":"2024-08-02T17:44:57.784723+0000","last_fullsized":"2024-08-02T17:44:57.784723+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784337+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:44:57.784337+0000","last_peered":"2024-08-02T17:44:57.784337+0000","last_clean":"2024-08-02T17:44:57.784337+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:44:57.784337+0000","last_undegraded":"2024-08-02T17:44:57.784337+0000","last_fullsized":"2024-08-02T17:44:57.784337+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778261+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:44:57.778261+0000","last_peered":"2024-08-02T17:44:57.778261+0000","last_clean":"2024-08-02T17:44:57.778261+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:44:57.778261+0000","last_undegraded":"2024-08-02T17:44:57.778261+0000","last_fullsized":"2024-08-02T17:44:57.778261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784229+0000","last_change":"2024-08-02T17:44:04.127893+0000","last_active":"2024-08-02T17:44:57.784229+0000","last_peered":"2024-08-02T17:44:57.784229+0000","last_clean":"2024-08-02T17:44:57.784229+0000","last_became_active":"2024-08-02T17:44:04.127520+0000","last_became_peered":"2024-08-02T17:44:04.127520+0000","last_unstale":"2024-08-02T17:44:57.784229+0000","last_undegraded":"2024-08-02T17:44:57.784229+0000","last_fullsized":"2024-08-02T17:44:57.784229+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:53.670606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782482+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:44:57.782482+0000","last_peered":"2024-08-02T17:44:57.782482+0000","last_clean":"2024-08-02T17:44:57.782482+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:44:57.782482+0000","last_undegraded":"2024-08-02T17:44:57.782482+0000","last_fullsized":"2024-08-02T17:44:57.782482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788689+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:44:57.788689+0000","last_peered":"2024-08-02T17:44:57.788689+0000","last_clean":"2024-08-02T17:44:57.788689+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:44:57.788689+0000","last_undegraded":"2024-08-02T17:44:57.788689+0000","last_fullsized":"2024-08-02T17:44:57.788689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.935062+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:44:57.935062+0000","last_peered":"2024-08-02T17:44:57.935062+0000","last_clean":"2024-08-02T17:44:57.935062+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:44:57.935062+0000","last_undegraded":"2024-08-02T17:44:57.935062+0000","last_fullsized":"2024-08-02T17:44:57.935062+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778967+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:44:57.778967+0000","last_peered":"2024-08-02T17:44:57.778967+0000","last_clean":"2024-08-02T17:44:57.778967+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:44:57.778967+0000","last_undegraded":"2024-08-02T17:44:57.778967+0000","last_fullsized":"2024-08-02T17:44:57.778967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.782955+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:44:57.782955+0000","last_peered":"2024-08-02T17:44:57.782955+0000","last_clean":"2024-08-02T17:44:57.782955+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:44:57.782955+0000","last_undegraded":"2024-08-02T17:44:57.782955+0000","last_fullsized":"2024-08-02T17:44:57.782955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788682+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:44:57.788682+0000","last_peered":"2024-08-02T17:44:57.788682+0000","last_clean":"2024-08-02T17:44:57.788682+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:44:57.788682+0000","last_undegraded":"2024-08-02T17:44:57.788682+0000","last_fullsized":"2024-08-02T17:44:57.788682+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779156+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:44:57.779156+0000","last_peered":"2024-08-02T17:44:57.779156+0000","last_clean":"2024-08-02T17:44:57.779156+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:44:57.779156+0000","last_undegraded":"2024-08-02T17:44:57.779156+0000","last_fullsized":"2024-08-02T17:44:57.779156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778167+0000","last_change":"2024-08-02T17:44:04.124213+0000","last_active":"2024-08-02T17:44:57.778167+0000","last_peered":"2024-08-02T17:44:57.778167+0000","last_clean":"2024-08-02T17:44:57.778167+0000","last_became_active":"2024-08-02T17:44:04.123913+0000","last_became_peered":"2024-08-02T17:44:04.123913+0000","last_unstale":"2024-08-02T17:44:57.778167+0000","last_undegraded":"2024-08-02T17:44:57.778167+0000","last_fullsized":"2024-08-02T17:44:57.778167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:32:05.880054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058515+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:44:58.058515+0000","last_peered":"2024-08-02T17:44:58.058515+0000","last_clean":"2024-08-02T17:44:58.058515+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:44:58.058515+0000","last_undegraded":"2024-08-02T17:44:58.058515+0000","last_fullsized":"2024-08-02T17:44:58.058515+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934630+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:44:57.934630+0000","last_peered":"2024-08-02T17:44:57.934630+0000","last_clean":"2024-08-02T17:44:57.934630+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:44:57.934630+0000","last_undegraded":"2024-08-02T17:44:57.934630+0000","last_fullsized":"2024-08-02T17:44:57.934630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778481+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:44:57.778481+0000","last_peered":"2024-08-02T17:44:57.778481+0000","last_clean":"2024-08-02T17:44:57.778481+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:44:57.778481+0000","last_undegraded":"2024-08-02T17:44:57.778481+0000","last_fullsized":"2024-08-02T17:44:57.778481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.784986+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:44:57.784986+0000","last_peered":"2024-08-02T17:44:57.784986+0000","last_clean":"2024-08-02T17:44:57.784986+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:44:57.784986+0000","last_undegraded":"2024-08-02T17:44:57.784986+0000","last_fullsized":"2024-08-02T17:44:57.784986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.776049+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:44:57.776049+0000","last_peered":"2024-08-02T17:44:57.776049+0000","last_clean":"2024-08-02T17:44:57.776049+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:44:57.776049+0000","last_undegraded":"2024-08-02T17:44:57.776049+0000","last_fullsized":"2024-08-02T17:44:57.776049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779187+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:44:57.779187+0000","last_peered":"2024-08-02T17:44:57.779187+0000","last_clean":"2024-08-02T17:44:57.779187+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:44:57.779187+0000","last_undegraded":"2024-08-02T17:44:57.779187+0000","last_fullsized":"2024-08-02T17:44:57.779187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.058678+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:44:58.058678+0000","last_peered":"2024-08-02T17:44:58.058678+0000","last_clean":"2024-08-02T17:44:58.058678+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:44:58.058678+0000","last_undegraded":"2024-08-02T17:44:58.058678+0000","last_fullsized":"2024-08-02T17:44:58.058678+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788147+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:44:57.788147+0000","last_peered":"2024-08-02T17:44:57.788147+0000","last_clean":"2024-08-02T17:44:57.788147+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:44:57.788147+0000","last_undegraded":"2024-08-02T17:44:57.788147+0000","last_fullsized":"2024-08-02T17:44:57.788147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788368+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:44:57.788368+0000","last_peered":"2024-08-02T17:44:57.788368+0000","last_clean":"2024-08-02T17:44:57.788368+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:44:57.788368+0000","last_undegraded":"2024-08-02T17:44:57.788368+0000","last_fullsized":"2024-08-02T17:44:57.788368+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.934609+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:44:57.934609+0000","last_peered":"2024-08-02T17:44:57.934609+0000","last_clean":"2024-08-02T17:44:57.934609+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:44:57.934609+0000","last_undegraded":"2024-08-02T17:44:57.934609+0000","last_fullsized":"2024-08-02T17:44:57.934609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779201+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:44:57.779201+0000","last_peered":"2024-08-02T17:44:57.779201+0000","last_clean":"2024-08-02T17:44:57.779201+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:44:57.779201+0000","last_undegraded":"2024-08-02T17:44:57.779201+0000","last_fullsized":"2024-08-02T17:44:57.779201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778292+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:44:57.778292+0000","last_peered":"2024-08-02T17:44:57.778292+0000","last_clean":"2024-08-02T17:44:57.778292+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:44:57.778292+0000","last_undegraded":"2024-08-02T17:44:57.778292+0000","last_fullsized":"2024-08-02T17:44:57.778292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.786339+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:44:57.786339+0000","last_peered":"2024-08-02T17:44:57.786339+0000","last_clean":"2024-08-02T17:44:57.786339+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:44:57.786339+0000","last_undegraded":"2024-08-02T17:44:57.786339+0000","last_fullsized":"2024-08-02T17:44:57.786339+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:58.059142+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:44:58.059142+0000","last_peered":"2024-08-02T17:44:58.059142+0000","last_clean":"2024-08-02T17:44:58.059142+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:44:58.059142+0000","last_undegraded":"2024-08-02T17:44:58.059142+0000","last_fullsized":"2024-08-02T17:44:58.059142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.777697+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:44:57.777697+0000","last_peered":"2024-08-02T17:44:57.777697+0000","last_clean":"2024-08-02T17:44:57.777697+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:44:57.777697+0000","last_undegraded":"2024-08-02T17:44:57.777697+0000","last_fullsized":"2024-08-02T17:44:57.777697+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778075+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:44:57.778075+0000","last_peered":"2024-08-02T17:44:57.778075+0000","last_clean":"2024-08-02T17:44:57.778075+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:44:57.778075+0000","last_undegraded":"2024-08-02T17:44:57.778075+0000","last_fullsized":"2024-08-02T17:44:57.778075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.778851+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:44:57.778851+0000","last_peered":"2024-08-02T17:44:57.778851+0000","last_clean":"2024-08-02T17:44:57.778851+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:44:57.778851+0000","last_undegraded":"2024-08-02T17:44:57.778851+0000","last_fullsized":"2024-08-02T17:44:57.778851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.783119+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:44:57.783119+0000","last_peered":"2024-08-02T17:44:57.783119+0000","last_clean":"2024-08-02T17:44:57.783119+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:44:57.783119+0000","last_undegraded":"2024-08-02T17:44:57.783119+0000","last_fullsized":"2024-08-02T17:44:57.783119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779098+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:44:57.779098+0000","last_peered":"2024-08-02T17:44:57.779098+0000","last_clean":"2024-08-02T17:44:57.779098+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:44:57.779098+0000","last_undegraded":"2024-08-02T17:44:57.779098+0000","last_fullsized":"2024-08-02T17:44:57.779098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788028+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:44:57.788028+0000","last_peered":"2024-08-02T17:44:57.788028+0000","last_clean":"2024-08-02T17:44:57.788028+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:44:57.788028+0000","last_undegraded":"2024-08-02T17:44:57.788028+0000","last_fullsized":"2024-08-02T17:44:57.788028+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.788302+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:44:57.788302+0000","last_peered":"2024-08-02T17:44:57.788302+0000","last_clean":"2024-08-02T17:44:57.788302+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:44:57.788302+0000","last_undegraded":"2024-08-02T17:44:57.788302+0000","last_fullsized":"2024-08-02T17:44:57.788302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-08-02T17:44:57.779160+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:44:57.779160+0000","last_peered":"2024-08-02T17:44:57.779160+0000","last_clean":"2024-08-02T17:44:57.779160+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:44:57.779160+0000","last_undegraded":"2024-08-02T17:44:57.779160+0000","last_fullsized":"2024-08-02T17:44:57.779160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67315,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":349,"num_read_kb":249,"num_write":169,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":201945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449121,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":515190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678053,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1696,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1736704,"data_stored":1165773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841578,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":519522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103214,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1064960,"data_stored":519442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364850,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36960,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708192,"statfs":{"total":95995035648,"available":95957188608,"internally_reserved":0,"allocated":1654784,"data_stored":1111644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528375,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":510602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712348,"statfs":{"total":95995035648,"available":95961444352,"internally_reserved":0,"allocated":1658880,"data_stored":1102746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":515754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215172,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":510796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1126400,"data_stored":575065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":573437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":511990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":64553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:45:44.123 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-02T17:45:44.123 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-02T17:45:44.123 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-02T17:45:44.124 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph health --format=json 2024-08-02T17:45:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:43 smithi083 ceph-mon[37031]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:45:44.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:43 smithi077 ceph-mon[37108]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:45:44.376 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:44.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 -- 172.21.15.69:0/2710338562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88100b50 msgr2=0x7fda88100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:44.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 --2- 172.21.15.69:0/2710338562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88100b50 0x7fda88100f30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fda78009900 tx=0x7fda7802a480 comp rx=0 tx=0).stop 2024-08-02T17:45:44.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 -- 172.21.15.69:0/2710338562 shutdown_connections 2024-08-02T17:45:44.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 --2- 172.21.15.69:0/2710338562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda88102c20 0x7fda8810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 --2- 172.21.15.69:0/2710338562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda881022a0 0x7fda881026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 --2- 172.21.15.69:0/2710338562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88100b50 0x7fda88100f30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 -- 172.21.15.69:0/2710338562 >> 172.21.15.69:0/2710338562 conn(0x7fda880fc500 msgr2=0x7fda880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 -- 172.21.15.69:0/2710338562 shutdown_connections 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 -- 172.21.15.69:0/2710338562 wait complete. 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.861+0000 7fda8f594640 1 Processor -- start 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8f594640 1 -- start start 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8f594640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda88100b50 0x7fda88199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8f594640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda881022a0 0x7fda881995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8f594640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88102c20 0x7fda881a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8f594640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fda88110cb0 con 0x7fda88102c20 2024-08-02T17:45:44.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8f594640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda88110b30 con 0x7fda88100b50 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8f594640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda88110e30 con 0x7fda881022a0 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88102c20 0x7fda881a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88102c20 0x7fda881a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52984/0 (socket says 172.21.15.69:52984) 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 -- 172.21.15.69:0/2283963132 learned_addr learned my addr 172.21.15.69:0/2283963132 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8d330640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda88100b50 0x7fda88199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8cb2f640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda881022a0 0x7fda881995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 -- 172.21.15.69:0/2283963132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda881022a0 msgr2=0x7fda881995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda881022a0 0x7fda881995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 -- 172.21.15.69:0/2283963132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda88100b50 msgr2=0x7fda88199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda88100b50 0x7fda88199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 -- 172.21.15.69:0/2283963132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda881a4620 con 0x7fda88102c20 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8d330640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda88100b50 0x7fda88199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda8db31640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88102c20 0x7fda881a40e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fda8000eaa0 tx=0x7fda8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:44.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.862+0000 7fda7e7fc640 1 -- 172.21.15.69:0/2283963132 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8000cdb0 con 0x7fda88102c20 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.863+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda881a47f0 con 0x7fda88102c20 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.863+0000 7fda7e7fc640 1 -- 172.21.15.69:0/2283963132 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda80005020 con 0x7fda88102c20 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.863+0000 7fda7e7fc640 1 -- 172.21.15.69:0/2283963132 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda800116d0 con 0x7fda88102c20 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.863+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda881a4d30 con 0x7fda88102c20 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.863+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda88100f30 con 0x7fda88102c20 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.865+0000 7fda7e7fc640 1 -- 172.21.15.69:0/2283963132 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fda800054d0 con 0x7fda88102c20 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.865+0000 7fda7e7fc640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fda740a2c10 0x7fda740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.865+0000 7fda8d330640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fda740a2c10 0x7fda740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.865+0000 7fda7e7fc640 1 -- 172.21.15.69:0/2283963132 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fda800cf7e0 con 0x7fda88102c20 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.866+0000 7fda8d330640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fda740a2c10 0x7fda740a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fda78005ab0 tx=0x7fda78005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:44.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:44.869+0000 7fda7e7fc640 1 -- 172.21.15.69:0/2283963132 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fda80016030 con 0x7fda88102c20 2024-08-02T17:45:45.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.073+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fda88108dc0 con 0x7fda88102c20 2024-08-02T17:45:45.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:44 smithi069 ceph-mon[32746]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:45.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.074+0000 7fda7e7fc640 1 -- 172.21.15.69:0/2283963132 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fda8009c280 con 0x7fda88102c20 2024-08-02T17:45:45.076 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:45.076 INFO:teuthology.orchestra.run.smithi069.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-02T17:45:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fda740a2c10 msgr2=0x7fda740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fda740a2c10 0x7fda740a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fda78005ab0 tx=0x7fda78005a20 comp rx=0 tx=0).stop 2024-08-02T17:45:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88102c20 msgr2=0x7fda881a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88102c20 0x7fda881a40e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fda8000eaa0 tx=0x7fda8000ef60 comp rx=0 tx=0).stop 2024-08-02T17:45:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 shutdown_connections 2024-08-02T17:45:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fda740a2c10 0x7fda740a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:45.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda88102c20 0x7fda881a40e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda881022a0 0x7fda881995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 --2- 172.21.15.69:0/2283963132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda88100b50 0x7fda88199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.076+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 >> 172.21.15.69:0/2283963132 conn(0x7fda880fc500 msgr2=0x7fda880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.077+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 shutdown_connections 2024-08-02T17:45:45.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:45.077+0000 7fda8f594640 1 -- 172.21.15.69:0/2283963132 wait complete. 2024-08-02T17:45:45.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:44 smithi083 ceph-mon[37031]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:45.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:44 smithi077 ceph-mon[37108]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:45:45.676 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-02T17:45:45.677 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:45:45.929 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:46.176 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:45 smithi069 ceph-mon[32746]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:46.177 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:45 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2283963132' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-02T17:45:46.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:45 smithi083 ceph-mon[37031]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:46.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:45 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2283963132' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-02T17:45:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:45 smithi077 ceph-mon[37108]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:45 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2283963132' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-08-02T17:45:46.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/598429063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb081022a0 msgr2=0x7fdb081026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/598429063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb081022a0 0x7fdb081026e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009900 tx=0x7fdaf802a480 comp rx=0 tx=0).stop 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/598429063 shutdown_connections 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/598429063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb081022a0 0x7fdb081026e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/598429063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb08102c20 0x7fdb0810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/598429063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb08100b50 0x7fdb08100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/598429063 >> 172.21.15.69:0/598429063 conn(0x7fdb080fc500 msgr2=0x7fdb080fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/598429063 shutdown_connections 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.396+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/598429063 wait complete. 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0fe0e640 1 Processor -- start 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0fe0e640 1 -- start start 2024-08-02T17:45:46.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0fe0e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08100b50 0x7fdb08198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0fe0e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb081022a0 0x7fdb081987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0fe0e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb08102c20 0x7fdb0819cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0fe0e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08110cd0 con 0x7fdb08100b50 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0fe0e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08110b50 con 0x7fdb08102c20 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0fe0e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08110e50 con 0x7fdb081022a0 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08100b50 0x7fdb08198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08100b50 0x7fdb08198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53012/0 (socket says 172.21.15.69:53012) 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 -- 172.21.15.69:0/3060926354 learned_addr learned my addr 172.21.15.69:0/3060926354 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 -- 172.21.15.69:0/3060926354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb081022a0 msgr2=0x7fdb081987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb081022a0 0x7fdb081987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 -- 172.21.15.69:0/3060926354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb08102c20 msgr2=0x7fdb0819cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb08102c20 0x7fdb0819cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 -- 172.21.15.69:0/3060926354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0819d240 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.397+0000 7fdb0dbaa640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08100b50 0x7fdb08198290 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c890 tx=0x7fdb0400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.398+0000 7fdafeffd640 1 -- 172.21.15.69:0/3060926354 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04004220 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.398+0000 7fdafeffd640 1 -- 172.21.15.69:0/3060926354 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb040043c0 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.398+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0819d530 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.398+0000 7fdafeffd640 1 -- 172.21.15.69:0/3060926354 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb040049e0 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.398+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb081a4e10 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.399+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb08100f30 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.400+0000 7fdafeffd640 1 -- 172.21.15.69:0/3060926354 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdb040079b0 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.400+0000 7fdafeffd640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdaf40a2c10 0x7fdaf40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.401+0000 7fdb0d3a9640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdaf40a2c10 0x7fdaf40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.401+0000 7fdafeffd640 1 -- 172.21.15.69:0/3060926354 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fdb040cf690 con 0x7fdb08100b50 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.401+0000 7fdb0d3a9640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdaf40a2c10 0x7fdaf40a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80096c0 tx=0x7fdaf8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:46.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.404+0000 7fdafeffd640 1 -- 172.21.15.69:0/3060926354 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdb04014030 con 0x7fdb08100b50 2024-08-02T17:45:46.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.603+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb081995f0 con 0x7fdb08100b50 2024-08-02T17:45:46.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.605+0000 7fdafeffd640 1 -- 172.21.15.69:0/3060926354 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11900 (secure 0 0 0) 0x7fdb0409c130 con 0x7fdb08100b50 2024-08-02T17:45:46.606 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:46.606 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"btime":"2024-08-02T17:44:49:913571+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:44:49.496243+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24431,24452]},"id":1}]} 2024-08-02T17:45:46.607 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-08-02T17:45:46.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdaf40a2c10 msgr2=0x7fdaf40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:46.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdaf40a2c10 0x7fdaf40a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80096c0 tx=0x7fdaf8000bc0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08100b50 msgr2=0x7fdb08198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:46.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08100b50 0x7fdb08198290 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c890 tx=0x7fdb0400cd50 comp rx=0 tx=0).stop 2024-08-02T17:45:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 shutdown_connections 2024-08-02T17:45:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fdaf40a2c10 0x7fdaf40a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdb08102c20 0x7fdb0819cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdb081022a0 0x7fdb081987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 --2- 172.21.15.69:0/3060926354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdb08100b50 0x7fdb08198290 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 >> 172.21.15.69:0/3060926354 conn(0x7fdb080fc500 msgr2=0x7fdb08069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 shutdown_connections 2024-08-02T17:45:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:46.607+0000 7fdb0fe0e640 1 -- 172.21.15.69:0/3060926354 wait complete. 2024-08-02T17:45:47.248 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:46 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3060926354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:47.249 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:46 smithi069 ceph-mon[32746]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:46 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3060926354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:46 smithi083 ceph-mon[37031]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:47.275 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:45:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:46 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3060926354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:46 smithi077 ceph-mon[37108]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:47.531 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/2333420990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54100b50 msgr2=0x7f9e54100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/2333420990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54100b50 0x7f9e54100f30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009900 tx=0x7f9e5002a480 comp rx=0 tx=0).stop 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/2333420990 shutdown_connections 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/2333420990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e54102c20 0x7f9e5410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/2333420990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e541022a0 0x7f9e541026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/2333420990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54100b50 0x7f9e54100f30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/2333420990 >> 172.21.15.69:0/2333420990 conn(0x7f9e540fc500 msgr2=0x7f9e540fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/2333420990 shutdown_connections 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.025+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/2333420990 wait complete. 2024-08-02T17:45:48.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.026+0000 7f9e5bdaf640 1 Processor -- start 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.026+0000 7f9e5bdaf640 1 -- start start 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.026+0000 7f9e5bdaf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e54100b50 0x7f9e541991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.026+0000 7f9e5bdaf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e541022a0 0x7f9e541996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.026+0000 7f9e5bdaf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54102c20 0x7f9e541a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.026+0000 7f9e5bdaf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54110cd0 con 0x7f9e54102c20 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.026+0000 7f9e5bdaf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54110b50 con 0x7f9e541022a0 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.026+0000 7f9e5bdaf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54110e50 con 0x7f9e54100b50 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54102c20 0x7f9e541a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54102c20 0x7f9e541a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53040/0 (socket says 172.21.15.69:53040) 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 -- 172.21.15.69:0/4124802476 learned_addr learned my addr 172.21.15.69:0/4124802476 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e59b4b640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e54100b50 0x7f9e541991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5934a640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e541022a0 0x7f9e541996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:48.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 -- 172.21.15.69:0/4124802476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e54100b50 msgr2=0x7f9e541991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e54100b50 0x7f9e541991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 -- 172.21.15.69:0/4124802476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e541022a0 msgr2=0x7f9e541996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e541022a0 0x7f9e541996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 -- 172.21.15.69:0/4124802476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e541a4760 con 0x7f9e54102c20 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e59b4b640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e54100b50 0x7f9e541991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5a34c640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54102c20 0x7f9e541a4220 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00cad0 tx=0x7f9e4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e4affd640 1 -- 172.21.15.69:0/4124802476 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00ae20 con 0x7f9e54102c20 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e4affd640 1 -- 172.21.15.69:0/4124802476 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4c005020 con 0x7f9e54102c20 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e541a4930 con 0x7f9e54102c20 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e4affd640 1 -- 172.21.15.69:0/4124802476 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00f6f0 con 0x7f9e54102c20 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.027+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e541a4df0 con 0x7f9e54102c20 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.028+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e54101ee0 con 0x7f9e54102c20 2024-08-02T17:45:48.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.030+0000 7f9e4affd640 1 -- 172.21.15.69:0/4124802476 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9e4c0051d0 con 0x7f9e54102c20 2024-08-02T17:45:48.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.030+0000 7f9e4affd640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e400a2c10 0x7f9e400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:48.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.030+0000 7f9e59b4b640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e400a2c10 0x7f9e400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:48.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.030+0000 7f9e4affd640 1 -- 172.21.15.69:0/4124802476 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f9e4c0cfa10 con 0x7f9e54102c20 2024-08-02T17:45:48.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.031+0000 7f9e59b4b640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e400a2c10 0x7f9e400a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e50005ab0 tx=0x7f9e50005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:48.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.037+0000 7f9e4affd640 1 -- 172.21.15.69:0/4124802476 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9e4c014030 con 0x7f9e54102c20 2024-08-02T17:45:48.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.251+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e54100f30 con 0x7f9e54102c20 2024-08-02T17:45:48.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.251+0000 7f9e4affd640 1 -- 172.21.15.69:0/4124802476 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11900 (secure 0 0 0) 0x7f9e54100f30 con 0x7f9e54102c20 2024-08-02T17:45:48.253 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:48.253 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"btime":"2024-08-02T17:44:49:913571+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:44:49.496243+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24431,24452]},"id":1}]} 2024-08-02T17:45:48.253 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-08-02T17:45:48.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.253+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e400a2c10 msgr2=0x7f9e400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:48.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.253+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e400a2c10 0x7f9e400a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e50005ab0 tx=0x7f9e50005a20 comp rx=0 tx=0).stop 2024-08-02T17:45:48.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.253+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54102c20 msgr2=0x7f9e541a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:48.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54102c20 0x7f9e541a4220 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00cad0 tx=0x7f9e4c00cf90 comp rx=0 tx=0).stop 2024-08-02T17:45:48.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 shutdown_connections 2024-08-02T17:45:48.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f9e400a2c10 0x7f9e400a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9e54102c20 0x7f9e541a4220 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9e541022a0 0x7f9e541996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 --2- 172.21.15.69:0/4124802476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9e54100b50 0x7f9e541991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 >> 172.21.15.69:0/4124802476 conn(0x7f9e540fc500 msgr2=0x7f9e540fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 shutdown_connections 2024-08-02T17:45:48.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:48.254+0000 7f9e5bdaf640 1 -- 172.21.15.69:0/4124802476 wait complete. 2024-08-02T17:45:48.529 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:48 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4124802476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:48.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:48 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4124802476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:48.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:48 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4124802476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:48.884 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:45:49.140 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:49.534 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:49 smithi069 ceph-mon[32746]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:49.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:49 smithi077 ceph-mon[37108]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:49.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 -- 172.21.15.69:0/3113899462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090103390 msgr2=0x7f709010b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:49.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 --2- 172.21.15.69:0/3113899462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090103390 0x7f709010b880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7084009900 tx=0x7f708402a480 comp rx=0 tx=0).stop 2024-08-02T17:45:49.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 -- 172.21.15.69:0/3113899462 shutdown_connections 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 --2- 172.21.15.69:0/3113899462 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090103390 0x7f709010b880 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 --2- 172.21.15.69:0/3113899462 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f709010bea0 0x7f709010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 --2- 172.21.15.69:0/3113899462 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7090102a70 0x7f7090102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 -- 172.21.15.69:0/3113899462 >> 172.21.15.69:0/3113899462 conn(0x7f70900fc4d0 msgr2=0x7f70900fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 -- 172.21.15.69:0/3113899462 shutdown_connections 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.612+0000 7f70966b9640 1 -- 172.21.15.69:0/3113899462 wait complete. 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f70966b9640 1 Processor -- start 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f70966b9640 1 -- start start 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f70966b9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090102a70 0x7f70901006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f70966b9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7090103390 0x7f7090100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f70966b9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f709010bea0 0x7f7090101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f70966b9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7090111ca0 con 0x7f7090102a70 2024-08-02T17:45:49.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f70966b9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7090111b20 con 0x7f709010bea0 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f70966b9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7090111e20 con 0x7f7090103390 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f708ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090102a70 0x7f70901006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f708ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090102a70 0x7f70901006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53050/0 (socket says 172.21.15.69:53050) 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f708ffff640 1 -- 172.21.15.69:0/4151882163 learned_addr learned my addr 172.21.15.69:0/4151882163 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.613+0000 7f7094c56640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f709010bea0 0x7f7090101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708f7fe640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7090103390 0x7f7090100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708ffff640 1 -- 172.21.15.69:0/4151882163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7090103390 msgr2=0x7f7090100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708ffff640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7090103390 0x7f7090100c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708ffff640 1 -- 172.21.15.69:0/4151882163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f709010bea0 msgr2=0x7f7090101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708ffff640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f709010bea0 0x7f7090101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708ffff640 1 -- 172.21.15.69:0/4151882163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70901a6640 con 0x7f7090102a70 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708f7fe640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7090103390 0x7f7090100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708ffff640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090102a70 0x7f70901006d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f708000d950 tx=0x7f708000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:49.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708d7fa640 1 -- 172.21.15.69:0/4151882163 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7080017070 con 0x7f7090102a70 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70901a6930 con 0x7f7090102a70 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708d7fa640 1 -- 172.21.15.69:0/4151882163 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7080009c30 con 0x7f7090102a70 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f708d7fa640 1 -- 172.21.15.69:0/4151882163 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708000f6b0 con 0x7f7090102a70 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.614+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70901a6ec0 con 0x7f7090102a70 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.615+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7090102e50 con 0x7f7090102a70 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.616+0000 7f708d7fa640 1 -- 172.21.15.69:0/4151882163 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f70800109d0 con 0x7f7090102a70 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.617+0000 7f708d7fa640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f70780a2c10 0x7f70780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.617+0000 7f708f7fe640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f70780a2c10 0x7f70780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.617+0000 7f708d7fa640 1 -- 172.21.15.69:0/4151882163 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f70800cf1a0 con 0x7f7090102a70 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.618+0000 7f708f7fe640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f70780a2c10 0x7f70780a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7090102160 tx=0x7f7084005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:49.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.621+0000 7f708d7fa640 1 -- 172.21.15.69:0/4151882163 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7080009de0 con 0x7f7090102a70 2024-08-02T17:45:49.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:49 smithi083 ceph-mon[37031]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-02T17:45:49.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.789+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7090106f10 con 0x7f7090102a70 2024-08-02T17:45:49.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.791+0000 7f708d7fa640 1 -- 172.21.15.69:0/4151882163 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24491 (secure 0 0 0) 0x7f708009bc40 con 0x7f7090102a70 2024-08-02T17:45:49.792 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:49.793 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":92,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:44:57.767460+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:45:49.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f70780a2c10 msgr2=0x7f70780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f70780a2c10 0x7f70780a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7090102160 tx=0x7f7084005a20 comp rx=0 tx=0).stop 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090102a70 msgr2=0x7f70901006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090102a70 0x7f70901006d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f708000d950 tx=0x7f708000de10 comp rx=0 tx=0).stop 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 shutdown_connections 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f70780a2c10 0x7f70780a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f709010bea0 0x7f7090101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7090103390 0x7f7090100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 --2- 172.21.15.69:0/4151882163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7090102a70 0x7f70901006d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:49.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 >> 172.21.15.69:0/4151882163 conn(0x7f70900fc4d0 msgr2=0x7f70900fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:49.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 shutdown_connections 2024-08-02T17:45:49.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:49.793+0000 7f70966b9640 1 -- 172.21.15.69:0/4151882163 wait complete. 2024-08-02T17:45:50.461 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:45:50.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:50 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4151882163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:50.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:50 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4151882163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:50.715 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:50.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:50 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4151882163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:45:51.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 -- 172.21.15.69:0/2632027037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b4102250 msgr2=0x7f84b410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:51.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 --2- 172.21.15.69:0/2632027037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b4102250 0x7f84b410e780 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009cd0 tx=0x7f84ac02a8e0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 -- 172.21.15.69:0/2632027037 shutdown_connections 2024-08-02T17:45:51.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 --2- 172.21.15.69:0/2632027037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b4102250 0x7f84b410e780 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 --2- 172.21.15.69:0/2632027037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84b41018b0 0x7f84b4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 --2- 172.21.15.69:0/2632027037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b41078b0 0x7f84b4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 -- 172.21.15.69:0/2632027037 >> 172.21.15.69:0/2632027037 conn(0x7f84b40fd5f0 msgr2=0x7f84b40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 -- 172.21.15.69:0/2632027037 shutdown_connections 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.154+0000 7f84ba026640 1 -- 172.21.15.69:0/2632027037 wait complete. 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84ba026640 1 Processor -- start 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84ba026640 1 -- start start 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84ba026640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b41018b0 0x7f84b4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84ba026640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84b4102250 0x7f84b4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84ba026640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b41078b0 0x7f84b419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84ba026640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4111cb0 con 0x7f84b41078b0 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84ba026640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4111b30 con 0x7f84b4102250 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84ba026640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4111e30 con 0x7f84b41018b0 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84b3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b41078b0 0x7f84b419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:51.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84b3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b41078b0 0x7f84b419c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53068/0 (socket says 172.21.15.69:53068) 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.155+0000 7f84b3fff640 1 -- 172.21.15.69:0/2210160945 learned_addr learned my addr 172.21.15.69:0/2210160945 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b37fe640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b41018b0 0x7f84b4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b2ffd640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84b4102250 0x7f84b4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b3fff640 1 -- 172.21.15.69:0/2210160945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b41018b0 msgr2=0x7f84b4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b3fff640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b41018b0 0x7f84b4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b3fff640 1 -- 172.21.15.69:0/2210160945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84b4102250 msgr2=0x7f84b4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b3fff640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84b4102250 0x7f84b4198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b3fff640 1 -- 172.21.15.69:0/2210160945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b419cfd0 con 0x7f84b41078b0 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b37fe640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b41018b0 0x7f84b4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b2ffd640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84b4102250 0x7f84b4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b3fff640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b41078b0 0x7f84b419c8f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f84ac006010 tx=0x7f84ac003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b0ff9640 1 -- 172.21.15.69:0/2210160945 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac004450 con 0x7f84b41078b0 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b0ff9640 1 -- 172.21.15.69:0/2210160945 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac003e10 con 0x7f84b41078b0 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b419d260 con 0x7f84b41078b0 2024-08-02T17:45:51.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84b0ff9640 1 -- 172.21.15.69:0/2210160945 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac0466e0 con 0x7f84b41078b0 2024-08-02T17:45:51.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.156+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a4ba0 con 0x7f84b41078b0 2024-08-02T17:45:51.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.157+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84b4101d10 con 0x7f84b41078b0 2024-08-02T17:45:51.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.158+0000 7f84b0ff9640 1 -- 172.21.15.69:0/2210160945 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f84ac0305a0 con 0x7f84b41078b0 2024-08-02T17:45:51.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.159+0000 7f84b0ff9640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f849c0a2c10 0x7f849c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:51.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.159+0000 7f84b37fe640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f849c0a2c10 0x7f849c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:51.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.159+0000 7f84b0ff9640 1 -- 172.21.15.69:0/2210160945 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f84ac02e030 con 0x7f84b41078b0 2024-08-02T17:45:51.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.159+0000 7f84b37fe640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f849c0a2c10 0x7f849c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005e10 tx=0x7f84a4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:51.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.163+0000 7f84b0ff9640 1 -- 172.21.15.69:0/2210160945 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f84ac04b060 con 0x7f84b41078b0 2024-08-02T17:45:51.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.362+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84b41a4e00 con 0x7f84b41078b0 2024-08-02T17:45:51.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.363+0000 7f84b0ff9640 1 -- 172.21.15.69:0/2210160945 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11900 (secure 0 0 0) 0x7f84ac039380 con 0x7f84b41078b0 2024-08-02T17:45:51.365 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:45:51.365 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"btime":"2024-08-02T17:44:49:913571+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:44:49.496243+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24431,24452]},"id":1}]} 2024-08-02T17:45:51.365 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-08-02T17:45:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f849c0a2c10 msgr2=0x7f849c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f849c0a2c10 0x7f849c0a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f84a4005e10 tx=0x7f84a4005da0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b41078b0 msgr2=0x7f84b419c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b41078b0 0x7f84b419c8f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f84ac006010 tx=0x7f84ac003c20 comp rx=0 tx=0).stop 2024-08-02T17:45:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 shutdown_connections 2024-08-02T17:45:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f849c0a2c10 0x7f849c0a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84b41078b0 0x7f84b419c8f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84b4102250 0x7f84b4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 --2- 172.21.15.69:0/2210160945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84b41018b0 0x7f84b4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.365+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 >> 172.21.15.69:0/2210160945 conn(0x7f84b40fd5f0 msgr2=0x7f84b40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.366+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 shutdown_connections 2024-08-02T17:45:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:51.366+0000 7f84ba026640 1 -- 172.21.15.69:0/2210160945 wait complete. 2024-08-02T17:45:51.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:51 smithi077 ceph-mon[37108]: pgmap v319: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:51.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:51 smithi069 ceph-mon[32746]: pgmap v319: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:51.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:51 smithi083 ceph-mon[37031]: pgmap v319: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:52.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-02T17:44:04.896710+0000', 'modified': '2024-08-02T17:44:49.496243+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24431, 'mds_2': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24403': {'gid': 24403, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/3991686236', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3991686236}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3991686236}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/1744999893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1744999893}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1744999893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6835/3410285190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 3410285190}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 3410285190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.83:6839/2418869426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 2418869426}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 2418869426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.77:6837/3778799888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 3778799888}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 3778799888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.77:6839/1183977027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 1183977027}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 1183977027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24427, 24431, 24452]} 2024-08-02T17:45:52.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-02T17:45:52.009 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.i status 2024-08-02T17:45:52.261 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:52.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:52 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2210160945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:52.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:52 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2210160945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:52.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 -- 172.21.15.69:0/2788624106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 msgr2=0x7f524410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 --2- 172.21.15.69:0/2788624106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 0x7f524410d770 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f523c009ca0 tx=0x7f523c02a950 comp rx=0 tx=0).stop 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 -- 172.21.15.69:0/2788624106 shutdown_connections 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 --2- 172.21.15.69:0/2788624106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 0x7f524410d770 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 --2- 172.21.15.69:0/2788624106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52441022a0 0x7f52441026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 --2- 172.21.15.69:0/2788624106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5244100b50 0x7f5244100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 -- 172.21.15.69:0/2788624106 >> 172.21.15.69:0/2788624106 conn(0x7f52440fc500 msgr2=0x7f52440fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 -- 172.21.15.69:0/2788624106 shutdown_connections 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 -- 172.21.15.69:0/2788624106 wait complete. 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 Processor -- start 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.713+0000 7f524b7cc640 1 -- start start 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f524b7cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5244100b50 0x7f5244198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f524b7cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52441022a0 0x7f5244199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f524b7cc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 0x7f52441a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f524b7cc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5244110cb0 con 0x7f5244102c20 2024-08-02T17:45:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f524b7cc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5244110b30 con 0x7f5244100b50 2024-08-02T17:45:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f524b7cc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5244110e30 con 0x7f52441022a0 2024-08-02T17:45:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 0x7f52441a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 0x7f52441a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53090/0 (socket says 172.21.15.69:53090) 2024-08-02T17:45:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 -- 172.21.15.69:0/3829624664 learned_addr learned my addr 172.21.15.69:0/3829624664 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249568640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5244100b50 0x7f5244198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5248d67640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52441022a0 0x7f5244199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 -- 172.21.15.69:0/3829624664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52441022a0 msgr2=0x7f5244199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52441022a0 0x7f5244199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 -- 172.21.15.69:0/3829624664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5244100b50 msgr2=0x7f5244198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5244100b50 0x7f5244198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 -- 172.21.15.69:0/3829624664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52441a4580 con 0x7f5244102c20 2024-08-02T17:45:52.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5248d67640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52441022a0 0x7f5244199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f5249d69640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 0x7f52441a4040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f523c006010 tx=0x7f523c02f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f523a7fc640 1 -- 172.21.15.69:0/3829624664 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c034040 con 0x7f5244102c20 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f523a7fc640 1 -- 172.21.15.69:0/3829624664 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523c003e00 con 0x7f5244102c20 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f524b7cc640 1 -- 172.21.15.69:0/3829624664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52441a4750 con 0x7f5244102c20 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.714+0000 7f523a7fc640 1 -- 172.21.15.69:0/3829624664 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c045480 con 0x7f5244102c20 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.715+0000 7f524b7cc640 1 -- 172.21.15.69:0/3829624664 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52441a4ba0 con 0x7f5244102c20 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.717+0000 7f523a7fc640 1 -- 172.21.15.69:0/3829624664 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f523c037020 con 0x7f5244102c20 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.717+0000 7f524dbca740 1 Processor -- start 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f524dbca740 1 -- start start 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f524dbca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f524dbca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 0x557827c352d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f524dbca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 0x557827c37c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f524dbca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557827b8f710 con 0x557827c2a630 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f524dbca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557827b27770 con 0x557827c2af50 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f524dbca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557827b8b6d0 con 0x557827c35810 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c2aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53102/0 (socket says 172.21.15.69:53102) 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 -- 172.21.15.69:0/1324365599 learned_addr learned my addr 172.21.15.69:0/1324365599 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5249568640 1 --2- 172.21.15.69:0/1324365599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 0x557827c352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f523a7fc640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52300a2c10 0x7f52300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 -- 172.21.15.69:0/1324365599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 msgr2=0x557827c37c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 --2- 172.21.15.69:0/1324365599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 0x557827c37c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 -- 172.21.15.69:0/1324365599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 msgr2=0x557827c352d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 --2- 172.21.15.69:0/1324365599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 0x557827c352d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 -- 172.21.15.69:0/1324365599 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557827b2e3e0 con 0x557827c2a630 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5249d69640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52300a2c10 0x7f52300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f523a7fc640 1 -- 172.21.15.69:0/3829624664 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f523c0f0500 con 0x7f5244102c20 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.718+0000 7f5248d67640 1 --2- 172.21.15.69:0/1324365599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c2aa10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5234005d90 tx=0x7f5234010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe12510ad0e3555 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f522f7fe640 1 -- 172.21.15.69:0/1324365599 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234016070 con 0x557827c2a630 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f522f7fe640 1 -- 172.21.15.69:0/1324365599 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52340047a0 con 0x557827c2a630 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f522f7fe640 1 -- 172.21.15.69:0/1324365599 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52340127e0 con 0x557827c2a630 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f524dbca740 1 -- 172.21.15.69:0/1324365599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 msgr2=0x557827c2aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f5249d69640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52300a2c10 0x7f52300a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f523c0bc940 tx=0x7f523c02a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f524dbca740 1 --2- 172.21.15.69:0/1324365599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c2aa10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5234005d90 tx=0x7f5234010740 comp rx=0 tx=0).stop 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f524dbca740 1 -- 172.21.15.69:0/1324365599 shutdown_connections 2024-08-02T17:45:52.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f524dbca740 1 --2- 172.21.15.69:0/1324365599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 0x557827c37c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f524dbca740 1 --2- 172.21.15.69:0/1324365599 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 0x557827c352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f524dbca740 1 --2- 172.21.15.69:0/1324365599 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c2aa10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.719+0000 7f524dbca740 1 -- 172.21.15.69:0/1324365599 >> 172.21.15.69:0/1324365599 conn(0x557827c26bc0 msgr2=0x557827c26fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 -- 172.21.15.69:0/1324365599 shutdown_connections 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 -- 172.21.15.69:0/1324365599 wait complete. 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 -- start start 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 Processor -- start 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2a630 0x557827c326e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2af50 0x557827c32c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c35810 0x557827c33160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557827c383e0 con 0x557827c2af50 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557827b4a970 con 0x557827c35810 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557827c38560 con 0x557827c2a630 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f524dbca740 10 client.0 _ll_register_callbacks cb 0x55782789e7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f5249568640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2af50 0x557827c32c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f5249568640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2af50 0x557827c32c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53118/0 (socket says 172.21.15.69:53118) 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.720+0000 7f5249568640 1 -- 172.21.15.69:0/2242405381 learned_addr learned my addr 172.21.15.69:0/2242405381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f5248d67640 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2a630 0x557827c326e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f5249568640 1 -- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2a630 msgr2=0x557827c326e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f5249568640 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2a630 0x557827c326e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f5249568640 1 -- 172.21.15.69:0/2242405381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c35810 msgr2=0x557827c33160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:52.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f5249568640 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c35810 0x557827c33160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f5249568640 1 -- 172.21.15.69:0/2242405381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578279012b0 con 0x557827c2af50 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f5248d67640 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2a630 0x557827c326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f5249568640 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2af50 0x557827c32c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5240012890 tx=0x7f5240012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec82bf0f459b9f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f522cff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f522cff9640 1 -- 172.21.15.69:0/2242405381 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524001f060 con 0x557827c2af50 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f524dbca740 1 -- 172.21.15.69:0/2242405381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52340047a0 con 0x557827c2af50 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f522cff9640 1 -- 172.21.15.69:0/2242405381 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5240010bc0 con 0x557827c2af50 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f522cff9640 1 -- 172.21.15.69:0/2242405381 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5240026050 con 0x557827c2af50 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f524dbca740 10 client.15012 fetch_fsmap learned FSMap version 25 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f524dbca740 1 -- 172.21.15.69:0/2242405381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557827c44090 con 0x557827c2af50 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.721+0000 7f522cff9640 1 -- 172.21.15.69:0/2242405381 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52340047a0 con 0x557827c2af50 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f522cff9640 1 -- 172.21.15.69:0/2242405381 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f524000d340 con 0x557827c2af50 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f524dbca740 10 client.15012 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:45:52.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f524dbca740 10 client.15012 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f524dbca740 20 client.15012 populate_metadata read hostname 'smithi069' 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f524dbca740 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557827c46690 0x557827c47b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f524dbca740 4 client.15012 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f524dbca740 1 -- 172.21.15.69:0/2242405381 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557827b9cc80 con 0x557827c46690 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f5248d67640 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557827c46690 0x557827c47b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f522cff9640 10 client.15012 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.722+0000 7f5248d67640 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557827c46690 0x557827c47b00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc5263e258f13975 server_cookie=f54191a8728121ea in_seq=0 out_seq=0 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f522cff9640 1 -- 172.21.15.69:0/2242405381 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557827b9cc80 con 0x557827c46690 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f522cff9640 10 client.15012 handle_command_reply: tid=0 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 client.15012 shutdown 2024-08-02T17:45:52.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 -- 172.21.15.69:0/2242405381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2af50 msgr2=0x557827c32c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2af50 0x557827c32c20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5240012890 tx=0x7f5240012d50 comp rx=0 tx=0).stop 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 -- 172.21.15.69:0/2242405381 shutdown_connections 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557827c46690 0x557827c47b00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2af50 0x557827c32c20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c35810 0x557827c33160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 --2- 172.21.15.69:0/2242405381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2a630 0x557827c326e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f524dbca740 1 -- 172.21.15.69:0/2242405381 >> 172.21.15.69:0/2242405381 conn(0x557827c26bc0 msgr2=0x557827c2c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.723+0000 7f522cff9640 0 client.15012 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.724+0000 7f524dbca740 1 -- 172.21.15.69:0/2242405381 shutdown_connections 2024-08-02T17:45:52.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.724+0000 7f524dbca740 1 -- 172.21.15.69:0/2242405381 wait complete. 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.724+0000 7f524dbca740 20 client.15012 trim_cache size 0 max 16384 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f524dbca740 1 Processor -- start 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f524dbca740 1 -- start start 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f524dbca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c46690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f524dbca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2af50 0x557827c2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f524dbca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c35810 0x557827c316c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f524dbca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557827c45e40 con 0x557827c2a630 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f524dbca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557827c46510 con 0x557827c35810 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f524dbca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557827c46be0 con 0x557827c2af50 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f5248d67640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c46690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f5248d67640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c46690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53134/0 (socket says 172.21.15.69:53134) 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.741+0000 7f5248d67640 1 -- 172.21.15.69:0/4061257939 learned_addr learned my addr 172.21.15.69:0/4061257939 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f5248d67640 1 -- 172.21.15.69:0/4061257939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2af50 msgr2=0x557827c2d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:52.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f5248d67640 1 --2- 172.21.15.69:0/4061257939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2af50 0x557827c2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f5248d67640 1 -- 172.21.15.69:0/4061257939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c35810 msgr2=0x557827c316c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f5248d67640 1 --2- 172.21.15.69:0/4061257939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c35810 0x557827c316c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f5248d67640 1 -- 172.21.15.69:0/4061257939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557827c269f0 con 0x557827c2a630 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f5248d67640 1 --2- 172.21.15.69:0/4061257939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c46690 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5234001eb0 tx=0x7f523400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31c0222fb333b9e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f522effd640 1 -- 172.21.15.69:0/4061257939 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234016070 con 0x557827c2a630 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f522effd640 1 -- 172.21.15.69:0/4061257939 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52340055e0 con 0x557827c2a630 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.742+0000 7f522effd640 1 -- 172.21.15.69:0/4061257939 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234013890 con 0x557827c2a630 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- 172.21.15.69:0/4061257939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 msgr2=0x557827c46690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 --2- 172.21.15.69:0/4061257939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c46690 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5234001eb0 tx=0x7f523400cf50 comp rx=0 tx=0).stop 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- 172.21.15.69:0/4061257939 shutdown_connections 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 --2- 172.21.15.69:0/4061257939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c35810 0x557827c316c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 --2- 172.21.15.69:0/4061257939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c2af50 0x557827c2d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 --2- 172.21.15.69:0/4061257939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c46690 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- 172.21.15.69:0/4061257939 >> 172.21.15.69:0/4061257939 conn(0x557827c26bc0 msgr2=0x557827c30fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- 172.21.15.69:0/4061257939 shutdown_connections 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- 172.21.15.69:0/4061257939 wait complete. 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- start start 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 Processor -- start 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c50060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 0x557827c505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 0x557827c494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557827c47980 con 0x557827c2a630 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557827c472b0 con 0x557827c2af50 2024-08-02T17:45:52.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557827c48050 con 0x557827c35810 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.743+0000 7f524dbca740 10 client.0 _ll_register_callbacks cb 0x7f524000a710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c50060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c50060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53138/0 (socket says 172.21.15.69:53138) 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 -- 172.21.15.69:0/2584653470 learned_addr learned my addr 172.21.15.69:0/2584653470 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5249568640 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 0x557827c505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 0x557827c494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 -- 172.21.15.69:0/2584653470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 msgr2=0x557827c494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 0x557827c494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 -- 172.21.15.69:0/2584653470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 msgr2=0x557827c505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 0x557827c505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 -- 172.21.15.69:0/2584653470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557827c22130 con 0x557827c2a630 2024-08-02T17:45:52.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f5248d67640 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c50060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5234005840 tx=0x7f5234005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66a9c0e28aefdf0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f522cff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f522cff9640 1 -- 172.21.15.69:0/2584653470 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234038060 con 0x557827c2a630 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f522cff9640 1 -- 172.21.15.69:0/2584653470 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7f5234004a70 con 0x557827c2a630 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f524dbca740 1 -- 172.21.15.69:0/2584653470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52340055e0 con 0x557827c2a630 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f522cff9640 1 -- 172.21.15.69:0/2584653470 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5234016030 con 0x557827c2a630 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f524dbca740 10 client.15018 fetch_fsmap learned FSMap version 25 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.744+0000 7f524dbca740 1 -- 172.21.15.69:0/2584653470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557827c4e580 con 0x557827c2a630 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f522cff9640 1 -- 172.21.15.69:0/2584653470 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52340055e0 con 0x557827c2a630 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f522cff9640 1 -- 172.21.15.69:0/2584653470 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f52340275b0 con 0x557827c2a630 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f524dbca740 10 client.15018 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f524dbca740 10 client.15018 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-02T17:45:52.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f524dbca740 20 client.15018 populate_metadata read hostname 'smithi069' 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24427, 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 65.844480637000004, 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-08-02T17:45:52.776 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 90, 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 127.245941355 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f524dbca740 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557827cb4bc0 0x557827cb38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f524dbca740 4 client.15018 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f524dbca740 1 -- 172.21.15.69:0/2584653470 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "status"}) -- 0x557827b97190 con 0x557827cb4bc0 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f5249568640 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557827cb4bc0 0x557827cb38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f522cff9640 10 client.15018 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.745+0000 7f5249568640 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557827cb4bc0 0x557827cb38d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=328b1a714abe7074 server_cookie=f2880af9b8f526e in_seq=0 out_seq=0 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f522cff9640 1 -- 172.21.15.69:0/2584653470 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557827b97190 con 0x557827cb4bc0 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f522cff9640 10 client.15018 handle_command_reply: tid=0 2024-08-02T17:45:52.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 client.15018 shutdown 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 -- 172.21.15.69:0/2584653470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 msgr2=0x557827c50060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c50060 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5234005840 tx=0x7f5234005fa0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 -- 172.21.15.69:0/2584653470 shutdown_connections 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557827cb4bc0 0x557827cb38d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557827c35810 0x557827c494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557827c2af50 0x557827c505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 --2- 172.21.15.69:0/2584653470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557827c2a630 0x557827c50060 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f522cff9640 0 client.15018 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 -- 172.21.15.69:0/2584653470 >> 172.21.15.69:0/2584653470 conn(0x557827c26bc0 msgr2=0x557827c317d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 -- 172.21.15.69:0/2584653470 shutdown_connections 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 1 -- 172.21.15.69:0/2584653470 wait complete. 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524dbca740 20 client.15018 trim_cache size 0 max 16384 2024-08-02T17:45:52.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524b7cc640 1 -- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52300a2c10 msgr2=0x7f52300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.746+0000 7f524b7cc640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52300a2c10 0x7f52300a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f523c0bc940 tx=0x7f523c02a950 comp rx=0 tx=0).stop 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 -- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 msgr2=0x7f52441a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 0x7f52441a4040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f523c006010 tx=0x7f523c02f6f0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 -- 172.21.15.69:0/3829624664 shutdown_connections 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f52300a2c10 0x7f52300a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5244102c20 0x7f52441a4040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52441022a0 0x7f5244199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 --2- 172.21.15.69:0/3829624664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5244100b50 0x7f5244198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 -- 172.21.15.69:0/3829624664 >> 172.21.15.69:0/3829624664 conn(0x7f52440fc500 msgr2=0x7f5244106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 -- 172.21.15.69:0/3829624664 shutdown_connections 2024-08-02T17:45:52.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:52.747+0000 7f524b7cc640 1 -- 172.21.15.69:0/3829624664 wait complete. 2024-08-02T17:45:52.780 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:52 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2210160945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:45:53.446 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:53 smithi069 ceph-mon[32746]: pgmap v320: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:45:53.474 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.h status 2024-08-02T17:45:53.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:53 smithi077 ceph-mon[37108]: pgmap v320: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:45:53.727 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:53.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:53 smithi083 ceph-mon[37031]: pgmap v320: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:45:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 -- 172.21.15.69:0/3603417663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd20103390 msgr2=0x7fbd2010b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3603417663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd20103390 0x7fbd2010b880 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbd14009900 tx=0x7fbd1402a480 comp rx=0 tx=0).stop 2024-08-02T17:45:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 -- 172.21.15.69:0/3603417663 shutdown_connections 2024-08-02T17:45:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3603417663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd20103390 0x7fbd2010b880 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3603417663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd2010bea0 0x7fbd2010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3603417663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd20102a70 0x7fbd20102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 -- 172.21.15.69:0/3603417663 >> 172.21.15.69:0/3603417663 conn(0x7fbd200fc4d0 msgr2=0x7fbd200fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 -- 172.21.15.69:0/3603417663 shutdown_connections 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.178+0000 7fbd2490b640 1 -- 172.21.15.69:0/3603417663 wait complete. 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.179+0000 7fbd2490b640 1 Processor -- start 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.179+0000 7fbd2490b640 1 -- start start 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd2490b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102a70 0x7fbd20198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd2490b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd20103390 0x7fbd20198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd2490b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd2010bea0 0x7fbd2019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd2490b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20111d00 con 0x7fbd2010bea0 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd2490b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20111b80 con 0x7fbd20103390 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd2490b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd20111e80 con 0x7fbd20102a70 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd2010bea0 0x7fbd2019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd2010bea0 0x7fbd2019cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53166/0 (socket says 172.21.15.69:53166) 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 -- 172.21.15.69:0/3647147588 learned_addr learned my addr 172.21.15.69:0/3647147588 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd20103390 0x7fbd20198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102a70 0x7fbd20198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 -- 172.21.15.69:0/3647147588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102a70 msgr2=0x7fbd20198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102a70 0x7fbd20198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 -- 172.21.15.69:0/3647147588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd20103390 msgr2=0x7fbd20198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd20103390 0x7fbd20198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 -- 172.21.15.69:0/3647147588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2019d1b0 con 0x7fbd2010bea0 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102a70 0x7fbd20198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd20103390 0x7fbd20198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd1ed9d640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd2010bea0 0x7fbd2019cad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800eaa0 tx=0x7fbd1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd0f7fe640 1 -- 172.21.15.69:0/3647147588 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1800cdb0 con 0x7fbd2010bea0 2024-08-02T17:45:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd0f7fe640 1 -- 172.21.15.69:0/3647147588 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd18005020 con 0x7fbd2010bea0 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd0f7fe640 1 -- 172.21.15.69:0/3647147588 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd180116b0 con 0x7fbd2010bea0 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd2490b640 1 -- 172.21.15.69:0/3647147588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2019d4a0 con 0x7fbd2010bea0 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.180+0000 7fbd2490b640 1 -- 172.21.15.69:0/3647147588 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd201a4d80 con 0x7fbd2010bea0 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd0f7fe640 1 -- 172.21.15.69:0/3647147588 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbd18005560 con 0x7fbd2010bea0 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd26d09740 1 Processor -- start 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd26d09740 1 -- start start 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5aada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5ab8810 0x55c5c5abac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd26d09740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5a12710 con 0x55c5c5aadf50 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd26d09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c5c59aa770 con 0x55c5c5ab8810 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd26d09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5a0e6d0 con 0x55c5c5aad630 2024-08-02T17:45:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.183+0000 7fbd0f7fe640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbd080a2c10 0x7fbd080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1ed9d640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbd080a2c10 0x7fbd080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53168/0 (socket says 172.21.15.69:53168) 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 -- 172.21.15.69:0/614703568 learned_addr learned my addr 172.21.15.69:0/614703568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd0f7fe640 1 -- 172.21.15.69:0/3647147588 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbd180cf810 con 0x7fbd2010bea0 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5aada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5ab8810 0x55c5c5abac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 -- 172.21.15.69:0/614703568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 msgr2=0x55c5c5aada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5aada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 -- 172.21.15.69:0/614703568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5ab8810 msgr2=0x55c5c5abac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5ab8810 0x55c5c5abac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 -- 172.21.15.69:0/614703568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c59b13e0 con 0x55c5c5aadf50 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5aada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1ed9d640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbd080a2c10 0x7fbd080a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd1809b8b0 tx=0x7fbd1806dbc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab82d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000aa10 tx=0x7fbd1000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08e13373ce97530 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd0cff9640 1 -- 172.21.15.69:0/614703568 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10015070 con 0x55c5c5aadf50 2024-08-02T17:45:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd0cff9640 1 -- 172.21.15.69:0/614703568 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1000cc60 con 0x55c5c5aadf50 2024-08-02T17:45:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.184+0000 7fbd0cff9640 1 -- 172.21.15.69:0/614703568 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd100186b0 con 0x55c5c5aadf50 2024-08-02T17:45:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 -- 172.21.15.69:0/614703568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aadf50 msgr2=0x55c5c5ab82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab82d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbd1000aa10 tx=0x7fbd1000aed0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 -- 172.21.15.69:0/614703568 shutdown_connections 2024-08-02T17:45:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab82d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5ab8810 0x55c5c5abac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 --2- 172.21.15.69:0/614703568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5aada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 -- 172.21.15.69:0/614703568 >> 172.21.15.69:0/614703568 conn(0x55c5c5aa9bc0 msgr2=0x55c5c5aa9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 -- 172.21.15.69:0/614703568 shutdown_connections 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 -- 172.21.15.69:0/614703568 wait complete. 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.185+0000 7fbd26d09740 1 -- start start 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd26d09740 1 Processor -- start 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ab56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5ab8810 0x55c5c5ab6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd26d09740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5abb3e0 con 0x55c5c5aad630 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd26d09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c5c59cd970 con 0x55c5c5aadf50 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd26d09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5abb560 con 0x55c5c5ab8810 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd26d09740 10 client.0 _ll_register_callbacks cb 0x55c5c57217b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ab56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ab56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53176/0 (socket says 172.21.15.69:53176) 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1712556360 learned_addr learned my addr 172.21.15.69:0/1712556360 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1712556360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5ab8810 msgr2=0x55c5c5ab6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5ab8810 0x55c5c5ab6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 msgr2=0x55c5c5ab5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1712556360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c57842b0 con 0x55c5c5aad630 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ab56a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbd1402a9a0 tx=0x7fbd14031b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2d452a2f7cd778 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.186+0000 7fbd05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1712556360 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14035040 con 0x55c5c5aad630 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1712556360 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbd140035f0 con 0x55c5c5aad630 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 1 -- 172.21.15.69:0/1712556360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd1000cc60 con 0x55c5c5aad630 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1712556360 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd14003810 con 0x55c5c5aad630 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 10 client.15030 fetch_fsmap learned FSMap version 25 2024-08-02T17:45:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 1 -- 172.21.15.69:0/1712556360 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c5c5ac6f90 con 0x55c5c5aad630 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1712556360 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1000cc60 con 0x55c5c5aad630 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1712556360 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbd1403a040 con 0x55c5c5aad630 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 10 client.15030 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 10 client.15030 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 20 client.15030 populate_metadata read hostname 'smithi069' 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55c5c5ac9550 0x55c5c5aca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 4 client.15030 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.187+0000 7fbd26d09740 1 -- 172.21.15.69:0/1712556360 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5c5a1fc80 con 0x55c5c5ac9550 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.188+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55c5c5ac9550 0x55c5c5aca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.188+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55c5c5ac9550 0x55c5c5aca9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1eed8f9c812a841 server_cookie=8ffe529043c62e84 in_seq=0 out_seq=0 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.188+0000 7fbd05ffb640 10 client.15030 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1712556360 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c5c5a1fc80 con 0x55c5c5ac9550 2024-08-02T17:45:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd05ffb640 10 client.15030 handle_command_reply: tid=0 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 client.15030 shutdown 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 -- 172.21.15.69:0/1712556360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 msgr2=0x55c5c5ab56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ab56a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbd1402a9a0 tx=0x7fbd14031b20 comp rx=0 tx=0).stop 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 -- 172.21.15.69:0/1712556360 shutdown_connections 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55c5c5ac9550 0x55c5c5aca9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5ab8810 0x55c5c5ab6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd05ffb640 0 client.15030 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1712556360 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ab56a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 -- 172.21.15.69:0/1712556360 >> 172.21.15.69:0/1712556360 conn(0x55c5c5aa9bc0 msgr2=0x55c5c5aa8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 -- 172.21.15.69:0/1712556360 shutdown_connections 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 1 -- 172.21.15.69:0/1712556360 wait complete. 2024-08-02T17:45:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.189+0000 7fbd26d09740 20 client.15030 trim_cache size 0 max 16384 2024-08-02T17:45:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.207+0000 7fbd26d09740 1 Processor -- start 2024-08-02T17:45:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.207+0000 7fbd26d09740 1 -- start start 2024-08-02T17:45:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5ab0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5ab8810 0x55c5c5ac3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd26d09740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5ac8d00 con 0x55c5c5ab8810 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd26d09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5ac93d0 con 0x55c5c5aadf50 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd26d09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5ac9aa0 con 0x55c5c5aad630 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5ab8810 0x55c5c5ac3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5ab8810 0x55c5c5ac3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53180/0 (socket says 172.21.15.69:53180) 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1853911410 learned_addr learned my addr 172.21.15.69:0/1853911410 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5ab0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1853911410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 msgr2=0x55c5c5ab0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5ab0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1853911410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 msgr2=0x55c5c5ab4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1853911410 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5c5aaa770 con 0x55c5c5ab8810 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5ab8810 0x55c5c5ac3ec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbd14000c00 tx=0x7fbd140056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791f2a6075b365e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd07fff640 1 -- 172.21.15.69:0/1853911410 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1405b070 con 0x55c5c5ab8810 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd07fff640 1 -- 172.21.15.69:0/1853911410 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd14042070 con 0x55c5c5ab8810 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.208+0000 7fbd07fff640 1 -- 172.21.15.69:0/1853911410 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14035040 con 0x55c5c5ab8810 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 -- 172.21.15.69:0/1853911410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5ab8810 msgr2=0x55c5c5ac3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5ab8810 0x55c5c5ac3ec0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbd14000c00 tx=0x7fbd140056d0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 -- 172.21.15.69:0/1853911410 shutdown_connections 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5ab8810 0x55c5c5ac3ec0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5ab4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1853911410 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5aad630 0x55c5c5ab0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 -- 172.21.15.69:0/1853911410 >> 172.21.15.69:0/1853911410 conn(0x55c5c5aa9bc0 msgr2=0x55c5c5ac37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 -- 172.21.15.69:0/1853911410 shutdown_connections 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.209+0000 7fbd26d09740 1 -- 172.21.15.69:0/1853911410 wait complete. 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 1 -- start start 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 1 Processor -- start 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ad31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5acc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5ab8810 0x55c5c5acca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5aca840 con 0x55c5c5aad630 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5aca170 con 0x55c5c5aadf50 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c5c5acaf10 con 0x55c5c5ab8810 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd26d09740 10 client.0 _ll_register_callbacks cb 0x7fbd2006b520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ad31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ad31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53188/0 (socket says 172.21.15.69:53188) 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1541990174 learned_addr learned my addr 172.21.15.69:0/1541990174 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5acc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1541990174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5ab8810 msgr2=0x55c5c5acca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5ab8810 0x55c5c5acca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 msgr2=0x55c5c5acc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5acc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1dd9b640 1 -- 172.21.15.69:0/1541990174 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5c5acd040 con 0x55c5c5aad630 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.210+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5acc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd1dd9b640 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ad31c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbd14059040 tx=0x7fbd14009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d748b91371bde774 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1541990174 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1405b070 con 0x55c5c5aad630 2024-08-02T17:45:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1541990174 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fbd14047880 con 0x55c5c5aad630 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 1 -- 172.21.15.69:0/1541990174 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd14042070 con 0x55c5c5aad630 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1541990174 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd14035040 con 0x55c5c5aad630 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 10 client.15036 fetch_fsmap learned FSMap version 25 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 1 -- 172.21.15.69:0/1541990174 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c5c5ad1ad0 con 0x55c5c5aad630 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1541990174 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd14042070 con 0x55c5c5aad630 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1541990174 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbd1403dc90 con 0x55c5c5aad630 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 10 client.15036 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 10 client.15036 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-02T17:45:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 20 client.15036 populate_metadata read hostname 'smithi069' 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55c5c5b38040 0x55c5c5b36d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 4 client.15036 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.211+0000 7fbd26d09740 1 -- 172.21.15.69:0/1541990174 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "status"}) -- 0x55c5c5a1a190 con 0x55c5c5b38040 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55c5c5b38040 0x55c5c5b36d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd1e59c640 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55c5c5b38040 0x55c5c5b36d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8df6d557d888109a server_cookie=22a8cd6ee5eb944a in_seq=0 out_seq=0 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd05ffb640 10 client.15036 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd05ffb640 1 -- 172.21.15.69:0/1541990174 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c5c5a1a190 con 0x55c5c5b38040 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd05ffb640 10 client.15036 handle_command_reply: tid=0 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 client.15036 shutdown 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 -- 172.21.15.69:0/1541990174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 msgr2=0x55c5c5ad31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ad31c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbd14059040 tx=0x7fbd14009520 comp rx=0 tx=0).stop 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 -- 172.21.15.69:0/1541990174 shutdown_connections 2024-08-02T17:45:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55c5c5b38040 0x55c5c5b36d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c5c5ab8810 0x55c5c5acca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c5c5aadf50 0x55c5c5acc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd05ffb640 0 client.15036 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:45:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 --2- 172.21.15.69:0/1541990174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c5c5aad630 0x55c5c5ad31c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.212+0000 7fbd26d09740 1 -- 172.21.15.69:0/1541990174 >> 172.21.15.69:0/1541990174 conn(0x55c5c5aa9bc0 msgr2=0x55c5c5ac3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd26d09740 1 -- 172.21.15.69:0/1541990174 shutdown_connections 2024-08-02T17:45:54.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd26d09740 1 -- 172.21.15.69:0/1541990174 wait complete. 2024-08-02T17:45:54.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd26d09740 20 client.15036 trim_cache size 0 max 16384 2024-08-02T17:45:54.227 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:45:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:45:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-08-02T17:45:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24431, 2024-08-02T17:45:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:45:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:45:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:45:54.228 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 68.324315425999998, 2024-08-02T17:45:54.228 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-08-02T17:45:54.228 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-08-02T17:45:54.228 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 90, 2024-08-02T17:45:54.228 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 131.293895091 2024-08-02T17:45:54.228 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:45:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 -- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbd080a2c10 msgr2=0x7fbd080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbd080a2c10 0x7fbd080a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd1809b8b0 tx=0x7fbd1806dbc0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 -- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd2010bea0 msgr2=0x7fbd2019cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd2010bea0 0x7fbd2019cad0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800eaa0 tx=0x7fbd1800ef60 comp rx=0 tx=0).stop 2024-08-02T17:45:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 -- 172.21.15.69:0/3647147588 shutdown_connections 2024-08-02T17:45:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fbd080a2c10 0x7fbd080a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd2010bea0 0x7fbd2019cad0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd20103390 0x7fbd20198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 --2- 172.21.15.69:0/3647147588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd20102a70 0x7fbd20198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:54.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 -- 172.21.15.69:0/3647147588 >> 172.21.15.69:0/3647147588 conn(0x7fbd200fc4d0 msgr2=0x7fbd200fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:54.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 -- 172.21.15.69:0/3647147588 shutdown_connections 2024-08-02T17:45:54.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:54.213+0000 7fbd2490b640 1 -- 172.21.15.69:0/3647147588 wait complete. 2024-08-02T17:45:54.899 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.b status 2024-08-02T17:45:55.154 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 -- 172.21.15.69:0/3560207054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 msgr2=0x7fe88410b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 --2- 172.21.15.69:0/3560207054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 0x7fe88410b880 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe878009900 tx=0x7fe87802a480 comp rx=0 tx=0).stop 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 -- 172.21.15.69:0/3560207054 shutdown_connections 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 --2- 172.21.15.69:0/3560207054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 0x7fe88410b880 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 --2- 172.21.15.69:0/3560207054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe88410bea0 0x7fe88410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 --2- 172.21.15.69:0/3560207054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe884102a70 0x7fe884102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 -- 172.21.15.69:0/3560207054 >> 172.21.15.69:0/3560207054 conn(0x7fe8840fc4d0 msgr2=0x7fe8840fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 -- 172.21.15.69:0/3560207054 shutdown_connections 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.644+0000 7fe889f1a640 1 -- 172.21.15.69:0/3560207054 wait complete. 2024-08-02T17:45:55.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe889f1a640 1 Processor -- start 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe889f1a640 1 -- start start 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe889f1a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe884102a70 0x7fe8841982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe889f1a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 0x7fe8841987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe889f1a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe88410bea0 0x7fe88419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe889f1a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe884111d60 con 0x7fe884103390 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe889f1a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe884111be0 con 0x7fe884102a70 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe889f1a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe884111ee0 con 0x7fe88410bea0 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 0x7fe8841987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 0x7fe8841987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45120/0 (socket says 172.21.15.69:45120) 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.645+0000 7fe882ffd640 1 -- 172.21.15.69:0/2257942036 learned_addr learned my addr 172.21.15.69:0/2257942036 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe883fff640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe88410bea0 0x7fe88419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe8837fe640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe884102a70 0x7fe8841982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe882ffd640 1 -- 172.21.15.69:0/2257942036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe88410bea0 msgr2=0x7fe88419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe882ffd640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe88410bea0 0x7fe88419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe882ffd640 1 -- 172.21.15.69:0/2257942036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe884102a70 msgr2=0x7fe8841982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe882ffd640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe884102a70 0x7fe8841982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe882ffd640 1 -- 172.21.15.69:0/2257942036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88419d260 con 0x7fe884103390 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe8837fe640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe884102a70 0x7fe8841982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe882ffd640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 0x7fe8841987f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe878005ab0 tx=0x7fe8780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe880ff9640 1 -- 172.21.15.69:0/2257942036 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87803f030 con 0x7fe884103390 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe889f1a640 1 -- 172.21.15.69:0/2257942036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88419d4f0 con 0x7fe884103390 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe880ff9640 1 -- 172.21.15.69:0/2257942036 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe878003ce0 con 0x7fe884103390 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe880ff9640 1 -- 172.21.15.69:0/2257942036 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878035070 con 0x7fe884103390 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.646+0000 7fe889f1a640 1 -- 172.21.15.69:0/2257942036 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8841a4e30 con 0x7fe884103390 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.648+0000 7fe880ff9640 1 -- 172.21.15.69:0/2257942036 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe878042030 con 0x7fe884103390 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe88c318740 1 Processor -- start 2024-08-02T17:45:55.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe88c318740 1 -- start start 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe88c318740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d52a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe88c318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 0x556462d5d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe88c318740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d5fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe88c318740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556462cb7710 con 0x556462d52630 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe88c318740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556462c4f770 con 0x556462d5d810 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe88c318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556462cb36d0 con 0x556462d52f50 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d52a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe880ff9640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe86c0a2c10 0x7fe86c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d52a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45124/0 (socket says 172.21.15.69:45124) 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 -- 172.21.15.69:0/696973073 learned_addr learned my addr 172.21.15.69:0/696973073 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe86c0a2c10 0x7fe86c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe880ff9640 1 -- 172.21.15.69:0/2257942036 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe8780ef5c0 con 0x7fe884103390 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe882ffd640 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d5fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 -- 172.21.15.69:0/696973073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 msgr2=0x556462d5d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 0x556462d5d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 -- 172.21.15.69:0/696973073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 msgr2=0x556462d5fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d5fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe883fff640 1 -- 172.21.15.69:0/696973073 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556462c563e0 con 0x556462d52630 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.649+0000 7fe882ffd640 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.650+0000 7fe883fff640 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d52a10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe87c015450 tx=0x7fe87c015850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8766feb4d855b83 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.650+0000 7fe869ffb640 1 -- 172.21.15.69:0/696973073 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c01c070 con 0x556462d52630 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.650+0000 7fe869ffb640 1 -- 172.21.15.69:0/696973073 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87c018960 con 0x556462d52630 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.650+0000 7fe869ffb640 1 -- 172.21.15.69:0/696973073 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0218c0 con 0x556462d52630 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.650+0000 7fe88c318740 1 -- 172.21.15.69:0/696973073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 msgr2=0x556462d52a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.650+0000 7fe883fff640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe86c0a2c10 0x7fe86c0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe87c020000 tx=0x7fe87c01c000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.650+0000 7fe88c318740 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d52a10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe87c015450 tx=0x7fe87c015850 comp rx=0 tx=0).stop 2024-08-02T17:45:55.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 -- 172.21.15.69:0/696973073 shutdown_connections 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d5fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 0x556462d5d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 --2- 172.21.15.69:0/696973073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d52a10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 -- 172.21.15.69:0/696973073 >> 172.21.15.69:0/696973073 conn(0x556462d4ebc0 msgr2=0x556462d4efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 -- 172.21.15.69:0/696973073 shutdown_connections 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 -- 172.21.15.69:0/696973073 wait complete. 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 -- start start 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 Processor -- start 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52630 0x556462d5a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52f50 0x556462d5ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d5d810 0x556462d5b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556462d603e0 con 0x556462d52f50 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556462c72970 con 0x556462d52630 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556462d60560 con 0x556462d5d810 2024-08-02T17:45:55.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.651+0000 7fe88c318740 10 client.0 _ll_register_callbacks cb 0x5564629c67b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52f50 0x556462d5ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52f50 0x556462d5ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45138/0 (socket says 172.21.15.69:45138) 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 -- 172.21.15.69:0/3976700519 learned_addr learned my addr 172.21.15.69:0/3976700519 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe883fff640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d5d810 0x556462d5b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52630 0x556462d5a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 -- 172.21.15.69:0/3976700519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d5d810 msgr2=0x556462d5b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d5d810 0x556462d5b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 -- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52630 msgr2=0x556462d5a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52630 0x556462d5a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 -- 172.21.15.69:0/3976700519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556462a292b0 con 0x556462d52f50 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe883fff640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d5d810 0x556462d5b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:55.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52630 0x556462d5a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe882ffd640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52f50 0x556462d5ac50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe8780bb8f0 tx=0x7fe8780bb2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2af31f32d367d4e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe862ffd640 1 -- 172.21.15.69:0/3976700519 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe878049040 con 0x556462d52f50 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe862ffd640 1 -- 172.21.15.69:0/3976700519 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe87803b460 con 0x556462d52f50 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe88c318740 1 -- 172.21.15.69:0/3976700519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe87c018960 con 0x556462d52f50 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe862ffd640 1 -- 172.21.15.69:0/3976700519 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8780bfd10 con 0x556462d52f50 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe88c318740 10 client.15048 fetch_fsmap learned FSMap version 25 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.652+0000 7fe88c318740 1 -- 172.21.15.69:0/3976700519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556462d6bf80 con 0x556462d52f50 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe862ffd640 1 -- 172.21.15.69:0/3976700519 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87c018960 con 0x556462d52f50 2024-08-02T17:45:55.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe862ffd640 1 -- 172.21.15.69:0/3976700519 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe8780b7fd0 con 0x556462d52f50 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe88c318740 10 client.15048 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe88c318740 10 client.15048 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe88c318740 20 client.15048 populate_metadata read hostname 'smithi069' 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe88c318740 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556462d6e610 0x556462d6fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe88c318740 4 client.15048 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe88c318740 1 -- 172.21.15.69:0/3976700519 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556462cc4c80 con 0x556462d6e610 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.653+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556462d6e610 0x556462d6fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556462d6e610 0x556462d6fa80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1e106e011bca9d server_cookie=3ca2dd98deae6c in_seq=0 out_seq=0 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe862ffd640 10 client.15048 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe862ffd640 1 -- 172.21.15.69:0/3976700519 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556462cc4c80 con 0x556462d6e610 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe862ffd640 10 client.15048 handle_command_reply: tid=0 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 client.15048 shutdown 2024-08-02T17:45:55.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 -- 172.21.15.69:0/3976700519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52f50 msgr2=0x556462d5ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52f50 0x556462d5ac50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe8780bb8f0 tx=0x7fe8780bb2c0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 -- 172.21.15.69:0/3976700519 shutdown_connections 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52f50 0x556462d5ac50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556462d6e610 0x556462d6fa80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d5d810 0x556462d5b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 --2- 172.21.15.69:0/3976700519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52630 0x556462d5a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe862ffd640 0 client.15048 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.654+0000 7fe88c318740 1 -- 172.21.15.69:0/3976700519 >> 172.21.15.69:0/3976700519 conn(0x556462d4ebc0 msgr2=0x556462d548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.655+0000 7fe88c318740 1 -- 172.21.15.69:0/3976700519 shutdown_connections 2024-08-02T17:45:55.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.655+0000 7fe88c318740 1 -- 172.21.15.69:0/3976700519 wait complete. 2024-08-02T17:45:55.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.655+0000 7fe88c318740 20 client.15048 trim_cache size 0 max 16384 2024-08-02T17:45:55.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe88c318740 1 Processor -- start 2024-08-02T17:45:55.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe88c318740 1 -- start start 2024-08-02T17:45:55.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe88c318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52630 0x556462d5afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe88c318740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52f50 0x556462d57970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe88c318740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d5d810 0x556462d57eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe88c318740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556462d6ddc0 con 0x556462d5d810 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe88c318740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556462d6e490 con 0x556462d52f50 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe88c318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556462d6eb60 con 0x556462d52630 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.672+0000 7fe883fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d5d810 0x556462d57eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe883fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d5d810 0x556462d57eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45150/0 (socket says 172.21.15.69:45150) 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe883fff640 1 -- 172.21.15.69:0/3852153799 learned_addr learned my addr 172.21.15.69:0/3852153799 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe882ffd640 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52f50 0x556462d57970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe8837fe640 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52630 0x556462d5afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe883fff640 1 -- 172.21.15.69:0/3852153799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52630 msgr2=0x556462d5afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe883fff640 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52630 0x556462d5afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe883fff640 1 -- 172.21.15.69:0/3852153799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52f50 msgr2=0x556462d57970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:55.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe883fff640 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52f50 0x556462d57970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe883fff640 1 -- 172.21.15.69:0/3852153799 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556462d4f850 con 0x556462d5d810 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe883fff640 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d5d810 0x556462d57eb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe87c015d50 tx=0x7fe87c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18639c4285c18d64 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe8697fa640 1 -- 172.21.15.69:0/3852153799 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c02e070 con 0x556462d5d810 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe8697fa640 1 -- 172.21.15.69:0/3852153799 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87c021940 con 0x556462d5d810 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.673+0000 7fe8697fa640 1 -- 172.21.15.69:0/3852153799 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c018920 con 0x556462d5d810 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 -- 172.21.15.69:0/3852153799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d5d810 msgr2=0x556462d57eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d5d810 0x556462d57eb0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe87c015d50 tx=0x7fe87c008040 comp rx=0 tx=0).stop 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 -- 172.21.15.69:0/3852153799 shutdown_connections 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d5d810 0x556462d57eb0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d52f50 0x556462d57970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 --2- 172.21.15.69:0/3852153799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52630 0x556462d5afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 -- 172.21.15.69:0/3852153799 >> 172.21.15.69:0/3852153799 conn(0x556462d4ebc0 msgr2=0x556462d58fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 -- 172.21.15.69:0/3852153799 shutdown_connections 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 -- 172.21.15.69:0/3852153799 wait complete. 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 -- start start 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.674+0000 7fe88c318740 1 Processor -- start 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe88c318740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d71c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe88c318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 0x556462d721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe88c318740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d72700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe88c318740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556462d6f900 con 0x556462d52630 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe88c318740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556462d6f230 con 0x556462d5d810 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe88c318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556462d6ffd0 con 0x556462d52f50 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d71c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe88c318740 10 client.0 _ll_register_callbacks cb 0x7fe8780b7c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:45:55.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d71c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45158/0 (socket says 172.21.15.69:45158) 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 -- 172.21.15.69:0/319930001 learned_addr learned my addr 172.21.15.69:0/319930001 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe883fff640 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d72700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe882ffd640 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 0x556462d721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 -- 172.21.15.69:0/319930001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 msgr2=0x556462d721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 0x556462d721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 -- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 msgr2=0x556462d72700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d72700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 -- 172.21.15.69:0/319930001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556462d765a0 con 0x556462d52630 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe883fff640 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d72700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe8837fe640 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d71c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe874009f70 tx=0x7fe874016310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6703a5fca4ffe266 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe862ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe862ffd640 1 -- 172.21.15.69:0/319930001 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874025060 con 0x556462d52630 2024-08-02T17:45:55.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe862ffd640 1 -- 172.21.15.69:0/319930001 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9092+0+0 (secure 0 0 0) 0x7fe874016c30 con 0x556462d52630 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.675+0000 7fe88c318740 1 -- 172.21.15.69:0/319930001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe87c021940 con 0x556462d52630 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe862ffd640 1 -- 172.21.15.69:0/319930001 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe87402c050 con 0x556462d52630 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe88c318740 10 client.15054 fetch_fsmap learned FSMap version 25 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe88c318740 1 -- 172.21.15.69:0/319930001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556462dd91f0 con 0x556462d52630 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe862ffd640 1 -- 172.21.15.69:0/319930001 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe87c021940 con 0x556462d52630 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe862ffd640 1 -- 172.21.15.69:0/319930001 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe87401f030 con 0x556462d52630 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe88c318740 10 client.15054 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe88c318740 10 client.15054 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe88c318740 20 client.15054 populate_metadata read hostname 'smithi069' 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe88c318740 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556462ddde00 0x556462ddcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe88c318740 4 client.15054 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe88c318740 1 -- 172.21.15.69:0/319930001 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "status"}) -- 0x556462cbf190 con 0x556462ddde00 2024-08-02T17:45:55.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.676+0000 7fe882ffd640 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556462ddde00 0x556462ddcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe882ffd640 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556462ddde00 0x556462ddcb10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9125fcbc67f0bd server_cookie=5bd5b4cc674c4d69 in_seq=0 out_seq=0 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe862ffd640 10 client.15054 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe862ffd640 1 -- 172.21.15.69:0/319930001 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556462cbf190 con 0x556462ddde00 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe862ffd640 10 client.15054 handle_command_reply: tid=0 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 client.15054 shutdown 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 -- 172.21.15.69:0/319930001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 msgr2=0x556462d71c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d71c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe874009f70 tx=0x7fe874016310 comp rx=0 tx=0).stop 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 -- 172.21.15.69:0/319930001 shutdown_connections 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556462ddde00 0x556462ddcb10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556462d5d810 0x556462d72700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556462d52f50 0x556462d721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 --2- 172.21.15.69:0/319930001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556462d52630 0x556462d71c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 -- 172.21.15.69:0/319930001 >> 172.21.15.69:0/319930001 conn(0x556462d4ebc0 msgr2=0x556462d687b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe862ffd640 0 client.15054 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 -- 172.21.15.69:0/319930001 shutdown_connections 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 1 -- 172.21.15.69:0/319930001 wait complete. 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.677+0000 7fe88c318740 20 client.15054 trim_cache size 0 max 16384 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24452, 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:45:55.687 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:45:55.688 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:45:55.688 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 110.547123427, 2024-08-02T17:45:55.688 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-08-02T17:45:55.688 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 92, 2024-08-02T17:45:55.688 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 90, 2024-08-02T17:45:55.688 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 121.36168865499999 2024-08-02T17:45:55.688 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 -- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe86c0a2c10 msgr2=0x7fe86c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe86c0a2c10 0x7fe86c0a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe87c020000 tx=0x7fe87c01c000 comp rx=0 tx=0).stop 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 -- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 msgr2=0x7fe8841987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 0x7fe8841987f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe878005ab0 tx=0x7fe8780042f0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 -- 172.21.15.69:0/2257942036 shutdown_connections 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe86c0a2c10 0x7fe86c0a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe884103390 0x7fe8841987f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe88410bea0 0x7fe88419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 --2- 172.21.15.69:0/2257942036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe884102a70 0x7fe8841982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:55.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 -- 172.21.15.69:0/2257942036 >> 172.21.15.69:0/2257942036 conn(0x7fe8840fc4d0 msgr2=0x7fe8840fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:55.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 -- 172.21.15.69:0/2257942036 shutdown_connections 2024-08-02T17:45:55.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:55.678+0000 7fe889f1a640 1 -- 172.21.15.69:0/2257942036 wait complete. 2024-08-02T17:45:56.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:55 smithi077 ceph-mon[37108]: pgmap v321: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:56.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:55 smithi069 ceph-mon[32746]: pgmap v321: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:56.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:55 smithi083 ceph-mon[37031]: pgmap v321: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:45:56.413 INFO:tasks.cephadm:Setup complete, yielding 2024-08-02T17:45:56.413 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-02T17:45:56.424 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:45:56.424 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- bash -c 'ceph orch status' 2024-08-02T17:45:56.676 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:56.939 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-08-02T17:45:56.940 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-08-02T17:45:56.940 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:56 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-08-02T17:45:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-08-02T17:45:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-08-02T17:45:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:56 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-08-02T17:45:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- 172.21.15.69:0/1227918361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4100840 msgr2=0x7f3ae4100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1227918361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4100840 0x7f3ae4100c40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009900 tx=0x7f3ae002a410 comp rx=0 tx=0).stop 2024-08-02T17:45:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- 172.21.15.69:0/1227918361 shutdown_connections 2024-08-02T17:45:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1227918361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ae4102400 0x7f3ae4108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1227918361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae4101a40 0x7f3ae4101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1227918361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4100840 0x7f3ae4100c40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- 172.21.15.69:0/1227918361 >> 172.21.15.69:0/1227918361 conn(0x7f3ae40fbfd0 msgr2=0x7f3ae40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- 172.21.15.69:0/1227918361 shutdown_connections 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- 172.21.15.69:0/1227918361 wait complete. 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 Processor -- start 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- start start 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae4100840 0x7f3ae410b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4101a40 0x7f3ae4109810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ae4102400 0x7f3ae4109d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:57.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae410d630 con 0x7f3ae4101a40 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae410d4b0 con 0x7f3ae4100840 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.153+0000 7f3aeb805640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae410d7b0 con 0x7f3ae4102400 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae8da0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4101a40 0x7f3ae4109810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae95a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae4100840 0x7f3ae410b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae95a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae4100840 0x7f3ae410b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49920/0 (socket says 172.21.15.69:49920) 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae8da0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4101a40 0x7f3ae4109810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45168/0 (socket says 172.21.15.69:45168) 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae95a1640 1 -- 172.21.15.69:0/1352863891 learned_addr learned my addr 172.21.15.69:0/1352863891 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae9da2640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ae4102400 0x7f3ae4109d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae8da0640 1 -- 172.21.15.69:0/1352863891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ae4102400 msgr2=0x7f3ae4109d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae8da0640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ae4102400 0x7f3ae4109d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae8da0640 1 -- 172.21.15.69:0/1352863891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae4100840 msgr2=0x7f3ae410b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae8da0640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae4100840 0x7f3ae410b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae8da0640 1 -- 172.21.15.69:0/1352863891 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae410a610 con 0x7f3ae4101a40 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ae8da0640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4101a40 0x7f3ae4109810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c970 tx=0x7f3ad400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ada7fc640 1 -- 172.21.15.69:0/1352863891 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007bb0 con 0x7f3ae4101a40 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae41a1e40 con 0x7f3ae4101a40 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ada7fc640 1 -- 172.21.15.69:0/1352863891 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ad4007d50 con 0x7f3ae4101a40 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3ada7fc640 1 -- 172.21.15.69:0/1352863891 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad400f450 con 0x7f3ae4101a40 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.154+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae41a2320 con 0x7f3ae4101a40 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.155+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae4107c90 con 0x7f3ae4101a40 2024-08-02T17:45:57.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.156+0000 7f3ada7fc640 1 -- 172.21.15.69:0/1352863891 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3ad4004ed0 con 0x7f3ae4101a40 2024-08-02T17:45:57.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.157+0000 7f3ada7fc640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3ad00a2c10 0x7f3ad00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:57.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.157+0000 7f3ae95a1640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3ad00a2c10 0x7f3ad00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:57.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.157+0000 7f3ada7fc640 1 -- 172.21.15.69:0/1352863891 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9179+0+0 (secure 0 0 0) 0x7f3ad40cf200 con 0x7f3ae4101a40 2024-08-02T17:45:57.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.158+0000 7f3ae95a1640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3ad00a2c10 0x7f3ad00a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0000c00 tx=0x7f3ae0005d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:57.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.161+0000 7f3ada7fc640 1 -- 172.21.15.69:0/1352863891 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3ad4014030 con 0x7f3ae4101a40 2024-08-02T17:45:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-08-02T17:45:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-08-02T17:45:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:56 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-08-02T17:45:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.345+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3ae4062360 con 0x7f3ad00a2c10 2024-08-02T17:45:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.346+0000 7f3ada7fc640 1 -- 172.21.15.69:0/1352863891 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3ae4062360 con 0x7f3ad00a2c10 2024-08-02T17:45:57.347 INFO:teuthology.orchestra.run.smithi069.stdout:Backend: cephadm 2024-08-02T17:45:57.347 INFO:teuthology.orchestra.run.smithi069.stdout:Available: Yes 2024-08-02T17:45:57.347 INFO:teuthology.orchestra.run.smithi069.stdout:Paused: No 2024-08-02T17:45:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3ad00a2c10 msgr2=0x7f3ad00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3ad00a2c10 0x7f3ad00a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0000c00 tx=0x7f3ae0005d50 comp rx=0 tx=0).stop 2024-08-02T17:45:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4101a40 msgr2=0x7f3ae4109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4101a40 0x7f3ae4109810 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c970 tx=0x7f3ad400ce30 comp rx=0 tx=0).stop 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 shutdown_connections 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3ad00a2c10 0x7f3ad00a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ae4101a40 0x7f3ae4109810 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ae4102400 0x7f3ae4109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 --2- 172.21.15.69:0/1352863891 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ae4100840 0x7f3ae410b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 >> 172.21.15.69:0/1352863891 conn(0x7f3ae40fbfd0 msgr2=0x7f3ae4104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 shutdown_connections 2024-08-02T17:45:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:57.349+0000 7f3aeb805640 1 -- 172.21.15.69:0/1352863891 wait complete. 2024-08-02T17:45:57.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:57 smithi069 ceph-mon[32746]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:57.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-08-02T17:45:57.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-08-02T17:45:57.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:57 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-08-02T17:45:57.962 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:57 smithi069 ceph-mon[32746]: osdmap e93: 12 total, 12 up, 12 in 2024-08-02T17:45:57.992 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-02T17:45:58.247 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:45:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:57 smithi083 ceph-mon[37031]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-08-02T17:45:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-08-02T17:45:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:57 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-08-02T17:45:58.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:57 smithi083 ceph-mon[37031]: osdmap e93: 12 total, 12 up, 12 in 2024-08-02T17:45:58.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:57 smithi077 ceph-mon[37108]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:45:58.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-08-02T17:45:58.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-08-02T17:45:58.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:57 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-08-02T17:45:58.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:57 smithi077 ceph-mon[37108]: osdmap e93: 12 total, 12 up, 12 in 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.733+0000 7f2131c06640 1 -- 172.21.15.69:0/1749287352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c075f50 msgr2=0x7f212c0743b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.733+0000 7f2131c06640 1 --2- 172.21.15.69:0/1749287352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c075f50 0x7f212c0743b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f211c009900 tx=0x7f211c02a410 comp rx=0 tx=0).stop 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.733+0000 7f2131c06640 1 -- 172.21.15.69:0/1749287352 shutdown_connections 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.733+0000 7f2131c06640 1 --2- 172.21.15.69:0/1749287352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c1097a0 0x7f212c10bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 --2- 172.21.15.69:0/1749287352 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f212c074980 0x7f212c074de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 --2- 172.21.15.69:0/1749287352 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c075f50 0x7f212c0743b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 -- 172.21.15.69:0/1749287352 >> 172.21.15.69:0/1749287352 conn(0x7f212c0fd0f0 msgr2=0x7f212c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 -- 172.21.15.69:0/1749287352 shutdown_connections 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 -- 172.21.15.69:0/1749287352 wait complete. 2024-08-02T17:45:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 Processor -- start 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 -- start start 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c074980 0x7f212c199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f212c075f50 0x7f212c19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c1097a0 0x7f212c1a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f212c10f800 con 0x7f212c1097a0 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f212c10f680 con 0x7f212c075f50 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.734+0000 7f2131c06640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f212c10f980 con 0x7f212c074980 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c1097a0 0x7f212c1a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c1097a0 0x7f212c1a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45186/0 (socket says 172.21.15.69:45186) 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f2122ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f212c075f50 0x7f212c19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 -- 172.21.15.69:0/1901761429 learned_addr learned my addr 172.21.15.69:0/1901761429 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:45:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212b7fe640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c074980 0x7f212c199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 -- 172.21.15.69:0/1901761429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c074980 msgr2=0x7f212c199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c074980 0x7f212c199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 -- 172.21.15.69:0/1901761429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f212c075f50 msgr2=0x7f212c19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f212c075f50 0x7f212c19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 -- 172.21.15.69:0/1901761429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f212c1a1a60 con 0x7f212c1097a0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f212bfff640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c1097a0 0x7f212c1a13f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f212400b970 tx=0x7f212400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f21297fa640 1 -- 172.21.15.69:0/1901761429 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124019070 con 0x7f212c1097a0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f21297fa640 1 -- 172.21.15.69:0/1901761429 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f212400cb40 con 0x7f212c1097a0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212c1a1d50 con 0x7f212c1097a0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f21297fa640 1 -- 172.21.15.69:0/1901761429 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124007650 con 0x7f212c1097a0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.735+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212c1a2260 con 0x7f212c1097a0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.736+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f212c108520 con 0x7f212c1097a0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.738+0000 7f21297fa640 1 -- 172.21.15.69:0/1901761429 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f212401c070 con 0x7f212c1097a0 2024-08-02T17:45:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.738+0000 7f21297fa640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f21180a2c40 0x7f21180a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:45:58.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.738+0000 7f212b7fe640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f21180a2c40 0x7f21180a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:45:58.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.738+0000 7f21297fa640 1 -- 172.21.15.69:0/1901761429 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f21240ce9f0 con 0x7f212c1097a0 2024-08-02T17:45:58.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.739+0000 7f212b7fe640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f21180a2c40 0x7f21180a5100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f211c005ab0 tx=0x7f211c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:45:58.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.742+0000 7f21297fa640 1 -- 172.21.15.69:0/1901761429 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f212409af00 con 0x7f212c1097a0 2024-08-02T17:45:58.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.920+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f212c062360 con 0x7f21180a2c40 2024-08-02T17:45:58.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:58 smithi069 ceph-mon[32746]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:45:58.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:58 smithi069 ceph-mon[32746]: osdmap e94: 12 total, 12 up, 12 in 2024-08-02T17:45:58.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.932+0000 7f21297fa640 1 -- 172.21.15.69:0/1901761429 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f212c062360 con 0x7f21180a2c40 2024-08-02T17:45:58.933 INFO:teuthology.orchestra.run.smithi069.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-02T17:45:58.933 INFO:teuthology.orchestra.run.smithi069.stdout:mds.a smithi069 running (2m) 55s ago 2m 16.6M - 19.1.0-1260-g26c3fb8e 564b849e9656 2d961c432555 2024-08-02T17:45:58.933 INFO:teuthology.orchestra.run.smithi069.stdout:mds.b smithi077 running (2m) 57s ago 2m 26.2M - 19.1.0-1260-g26c3fb8e 564b849e9656 9e1c5a70448f 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.c smithi083 running (2m) 57s ago 2m 17.8M - 19.1.0-1260-g26c3fb8e 564b849e9656 14dc08ae1bf9 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.d smithi069 running (2m) 55s ago 2m 15.9M - 19.1.0-1260-g26c3fb8e 564b849e9656 89156af5b846 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.e smithi077 running (2m) 57s ago 2m 17.8M - 19.1.0-1260-g26c3fb8e 564b849e9656 c94197eae435 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.f smithi083 running (2m) 57s ago 2m 17.0M - 19.1.0-1260-g26c3fb8e 564b849e9656 51c056c7d810 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.g smithi069 running (2m) 55s ago 2m 14.4M - 19.1.0-1260-g26c3fb8e 564b849e9656 228cfeb24eb7 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.h smithi077 running (2m) 57s ago 2m 17.1M - 19.1.0-1260-g26c3fb8e 564b849e9656 17ca429dd59d 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.i smithi083 running (2m) 57s ago 2m 16.1M - 19.1.0-1260-g26c3fb8e 564b849e9656 6170b510fee6 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.j smithi069 running (2m) 55s ago 2m 14.5M - 19.1.0-1260-g26c3fb8e 564b849e9656 fec88f0b3439 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.k smithi077 running (2m) 57s ago 2m 14.4M - 19.1.0-1260-g26c3fb8e 564b849e9656 9c0962489c3b 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mds.l smithi083 running (2m) 57s ago 2m 14.4M - 19.1.0-1260-g26c3fb8e 564b849e9656 abe436486d2a 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.x smithi069 *:9283,8765 running (8m) 55s ago 8m 508M - 19.1.0-1260-g26c3fb8e 564b849e9656 a2cd66d57a9b 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.y smithi077 *:8443,8765 running (7m) 57s ago 7m 441M - 19.1.0-1260-g26c3fb8e 564b849e9656 8a98cc974e1c 2024-08-02T17:45:58.934 INFO:teuthology.orchestra.run.smithi069.stdout:mgr.z smithi083 *:8443,8765 running (7m) 57s ago 7m 444M - 19.1.0-1260-g26c3fb8e 564b849e9656 b13768607c21 2024-08-02T17:45:58.935 INFO:teuthology.orchestra.run.smithi069.stdout:mon.a smithi069 running (8m) 55s ago 8m 50.5M 2048M 19.1.0-1260-g26c3fb8e 564b849e9656 fe3a259ab4b3 2024-08-02T17:45:58.935 INFO:teuthology.orchestra.run.smithi069.stdout:mon.b smithi077 running (7m) 57s ago 7m 40.7M 2048M 19.1.0-1260-g26c3fb8e 564b849e9656 9105eb6fade4 2024-08-02T17:45:58.935 INFO:teuthology.orchestra.run.smithi069.stdout:mon.c smithi083 running (7m) 57s ago 7m 41.0M 2048M 19.1.0-1260-g26c3fb8e 564b849e9656 f8813bf2428c 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 smithi069 running (6m) 55s ago 6m 45.0M 4271M 19.1.0-1260-g26c3fb8e 564b849e9656 a38ea498500b 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 smithi077 running (6m) 57s ago 6m 54.1M 4272M 19.1.0-1260-g26c3fb8e 564b849e9656 4045e158d4a5 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 smithi083 running (6m) 57s ago 6m 56.0M 4272M 19.1.0-1260-g26c3fb8e 564b849e9656 5190e289d3a6 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 smithi069 running (5m) 55s ago 5m 47.3M 4271M 19.1.0-1260-g26c3fb8e 564b849e9656 241748359deb 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 smithi077 running (5m) 57s ago 5m 46.5M 4272M 19.1.0-1260-g26c3fb8e 564b849e9656 409bfd5fd941 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 smithi083 running (5m) 57s ago 5m 52.0M 4272M 19.1.0-1260-g26c3fb8e 564b849e9656 a13a07b0854e 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 smithi069 running (4m) 55s ago 4m 42.1M 4271M 19.1.0-1260-g26c3fb8e 564b849e9656 c3738a94ab16 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 smithi077 running (4m) 57s ago 4m 52.8M 4272M 19.1.0-1260-g26c3fb8e 564b849e9656 eea0c5da7b9d 2024-08-02T17:45:58.938 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 smithi083 running (3m) 57s ago 3m 52.3M 4272M 19.1.0-1260-g26c3fb8e 564b849e9656 c71cd84fac41 2024-08-02T17:45:58.939 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 smithi069 running (3m) 55s ago 3m 52.9M 4271M 19.1.0-1260-g26c3fb8e 564b849e9656 b0e950af058f 2024-08-02T17:45:58.939 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 smithi077 running (3m) 57s ago 3m 52.0M 4272M 19.1.0-1260-g26c3fb8e 564b849e9656 fdcc1f61f18a 2024-08-02T17:45:58.939 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 smithi083 running (2m) 57s ago 2m 49.0M 4272M 19.1.0-1260-g26c3fb8e 564b849e9656 2ab0b2648abd 2024-08-02T17:45:58.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f21180a2c40 msgr2=0x7f21180a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:58.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f21180a2c40 0x7f21180a5100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f211c005ab0 tx=0x7f211c000bc0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c1097a0 msgr2=0x7f212c1a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:45:58.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c1097a0 0x7f212c1a13f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f212400b970 tx=0x7f212400be30 comp rx=0 tx=0).stop 2024-08-02T17:45:58.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 shutdown_connections 2024-08-02T17:45:58.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f21180a2c40 0x7f21180a5100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f212c1097a0 0x7f212c1a13f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f212c075f50 0x7f212c19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 --2- 172.21.15.69:0/1901761429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f212c074980 0x7f212c199e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:45:58.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 >> 172.21.15.69:0/1901761429 conn(0x7f212c0fd0f0 msgr2=0x7f212c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:45:58.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 shutdown_connections 2024-08-02T17:45:58.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:45:58.934+0000 7f2131c06640 1 -- 172.21.15.69:0/1901761429 wait complete. 2024-08-02T17:45:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:58 smithi083 ceph-mon[37031]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:45:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:58 smithi083 ceph-mon[37031]: osdmap e94: 12 total, 12 up, 12 in 2024-08-02T17:45:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:58 smithi077 ceph-mon[37108]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:45:59.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:58 smithi077 ceph-mon[37108]: osdmap e94: 12 total, 12 up, 12 in 2024-08-02T17:45:59.711 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-02T17:45:59.968 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:00.090 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:45:59 smithi069 ceph-mon[32746]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:00.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:45:59 smithi083 ceph-mon[37031]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:00.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:45:59 smithi077 ceph-mon[37108]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:00.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2207399813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08107690 msgr2=0x7f4c08109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:00.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2207399813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08107690 0x7f4c08109a80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009ca0 tx=0x7f4c0002a8c0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2207399813 shutdown_connections 2024-08-02T17:46:00.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2207399813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08107690 0x7f4c08109a80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2207399813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08104d60 0x7f4c08107150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2207399813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08102430 0x7f4c08104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2207399813 >> 172.21.15.69:0/2207399813 conn(0x7f4c080fbfd0 msgr2=0x7f4c080fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2207399813 shutdown_connections 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.478+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2207399813 wait complete. 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0eb72640 1 Processor -- start 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0eb72640 1 -- start start 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0eb72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08102430 0x7f4c08197cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0eb72640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08104d60 0x7f4c08198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0eb72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08107690 0x7f4c0819f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0eb72640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810d670 con 0x7f4c08102430 2024-08-02T17:46:00.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0eb72640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810d4f0 con 0x7f4c08104d60 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0eb72640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0810d7f0 con 0x7f4c08107690 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08102430 0x7f4c08197cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08102430 0x7f4c08197cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45204/0 (socket says 172.21.15.69:45204) 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 -- 172.21.15.69:0/2954698746 learned_addr learned my addr 172.21.15.69:0/2954698746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c07fff640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08104d60 0x7f4c08198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0d10f640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08107690 0x7f4c0819f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 -- 172.21.15.69:0/2954698746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08107690 msgr2=0x7f4c0819f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08107690 0x7f4c0819f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 -- 172.21.15.69:0/2954698746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08104d60 msgr2=0x7f4c08198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08104d60 0x7f4c08198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 -- 172.21.15.69:0/2954698746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0819f910 con 0x7f4c08102430 2024-08-02T17:46:00.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.479+0000 7f4c0c90e640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08102430 0x7f4c08197cf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00b4a0 tx=0x7f4bfc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.480+0000 7f4c05ffb640 1 -- 172.21.15.69:0/2954698746 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc014070 con 0x7f4c08102430 2024-08-02T17:46:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.480+0000 7f4c05ffb640 1 -- 172.21.15.69:0/2954698746 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bfc010af0 con 0x7f4c08102430 2024-08-02T17:46:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.480+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0819fc00 con 0x7f4c08102430 2024-08-02T17:46:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.480+0000 7f4c05ffb640 1 -- 172.21.15.69:0/2954698746 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc017d10 con 0x7f4c08102430 2024-08-02T17:46:00.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.480+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c081a0140 con 0x7f4c08102430 2024-08-02T17:46:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.481+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c080fdac0 con 0x7f4c08102430 2024-08-02T17:46:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.482+0000 7f4c05ffb640 1 -- 172.21.15.69:0/2954698746 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4bfc017070 con 0x7f4c08102430 2024-08-02T17:46:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.482+0000 7f4c05ffb640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4bf80a2c10 0x7f4bf80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.482+0000 7f4c07fff640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4bf80a2c10 0x7f4bf80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.482+0000 7f4c05ffb640 1 -- 172.21.15.69:0/2954698746 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f4bfc0ce8f0 con 0x7f4c08102430 2024-08-02T17:46:00.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.483+0000 7f4c07fff640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4bf80a2c10 0x7f4bf80a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4005fd0 tx=0x7f4bf4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:00.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.487+0000 7f4c05ffb640 1 -- 172.21.15.69:0/2954698746 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f4bfc09b450 con 0x7f4c08102430 2024-08-02T17:46:00.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.663+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4c08062360 con 0x7f4bf80a2c10 2024-08-02T17:46:00.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.669+0000 7f4c05ffb640 1 -- 172.21.15.69:0/2954698746 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4c08062360 con 0x7f4bf80a2c10 2024-08-02T17:46:00.670 INFO:teuthology.orchestra.run.smithi069.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-02T17:46:00.670 INFO:teuthology.orchestra.run.smithi069.stdout:mds.all 12/12 59s ago 2m smithi069=a;smithi069=d;smithi069=g;smithi069=j;smithi077=b;smithi077=e;smithi077=h;smithi077=k;smithi083=c;smithi083=f;smithi083=i;smithi083=l;count:12 2024-08-02T17:46:00.670 INFO:teuthology.orchestra.run.smithi069.stdout:mgr 3/3 59s ago 7m smithi069=x;smithi077=y;smithi083=z;count:3 2024-08-02T17:46:00.671 INFO:teuthology.orchestra.run.smithi069.stdout:mon 3/3 59s ago 7m smithi069:172.21.15.69=a;smithi077:172.21.15.77=b;smithi083:172.21.15.83=c;count:3 2024-08-02T17:46:00.671 INFO:teuthology.orchestra.run.smithi069.stdout:osd 12 59s ago - 2024-08-02T17:46:00.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4bf80a2c10 msgr2=0x7f4bf80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4bf80a2c10 0x7f4bf80a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4005fd0 tx=0x7f4bf4005850 comp rx=0 tx=0).stop 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08102430 msgr2=0x7f4c08197cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08102430 0x7f4c08197cf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00b4a0 tx=0x7f4bfc00b960 comp rx=0 tx=0).stop 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 shutdown_connections 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f4bf80a2c10 0x7f4bf80a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c08107690 0x7f4c0819f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c08104d60 0x7f4c08198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 --2- 172.21.15.69:0/2954698746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c08102430 0x7f4c08197cf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 >> 172.21.15.69:0/2954698746 conn(0x7f4c080fbfd0 msgr2=0x7f4c08102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 shutdown_connections 2024-08-02T17:46:00.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:00.671+0000 7f4c0eb72640 1 -- 172.21.15.69:0/2954698746 wait complete. 2024-08-02T17:46:00.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:00 smithi069 ceph-mon[32746]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:00 smithi083 ceph-mon[37031]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:00 smithi077 ceph-mon[37108]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:01.473 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-02T17:46:01.727 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:01 smithi069 ceph-mon[32746]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:02.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:01 smithi069 ceph-mon[32746]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-08-02T17:46:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:01 smithi083 ceph-mon[37031]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:01 smithi083 ceph-mon[37031]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-08-02T17:46:02.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 -- 172.21.15.69:0/901323667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e8101a00 msgr2=0x7f39e8101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:02.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 --2- 172.21.15.69:0/901323667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e8101a00 0x7f39e8101e80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009900 tx=0x7f39d002a410 comp rx=0 tx=0).stop 2024-08-02T17:46:02.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 -- 172.21.15.69:0/901323667 shutdown_connections 2024-08-02T17:46:02.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 --2- 172.21.15.69:0/901323667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e8101a00 0x7f39e8101e80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 --2- 172.21.15.69:0/901323667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e81023c0 0x7f39e8108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 --2- 172.21.15.69:0/901323667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e8100800 0x7f39e8100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 -- 172.21.15.69:0/901323667 >> 172.21.15.69:0/901323667 conn(0x7f39e80fbfb0 msgr2=0x7f39e80fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 -- 172.21.15.69:0/901323667 shutdown_connections 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 -- 172.21.15.69:0/901323667 wait complete. 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.253+0000 7f39eda78640 1 Processor -- start 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39eda78640 1 -- start start 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39eda78640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e8100800 0x7f39e806a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39eda78640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e8101a00 0x7f39e819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39eda78640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e81023c0 0x7f39e819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39eda78640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f39e810c480 con 0x7f39e81023c0 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39eda78640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39e810c300 con 0x7f39e8100800 2024-08-02T17:46:02.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39eda78640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39e810c600 con 0x7f39e8101a00 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e81023c0 0x7f39e819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e81023c0 0x7f39e819efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45224/0 (socket says 172.21.15.69:45224) 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e7fff640 1 -- 172.21.15.69:0/4257761401 learned_addr learned my addr 172.21.15.69:0/4257761401 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e77fe640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e8100800 0x7f39e806a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e7fff640 1 -- 172.21.15.69:0/4257761401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e8101a00 msgr2=0x7f39e819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e6ffd640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e8101a00 0x7f39e819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e7fff640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e8101a00 0x7f39e819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e7fff640 1 -- 172.21.15.69:0/4257761401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e8100800 msgr2=0x7f39e806a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e7fff640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e8100800 0x7f39e806a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.254+0000 7f39e7fff640 1 -- 172.21.15.69:0/4257761401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e819f8d0 con 0x7f39e81023c0 2024-08-02T17:46:02.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.255+0000 7f39e7fff640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e81023c0 0x7f39e819efc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f39e000eaa0 tx=0x7f39e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.255+0000 7f39e4ff9640 1 -- 172.21.15.69:0/4257761401 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e000cdb0 con 0x7f39e81023c0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.255+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e819fb60 con 0x7f39e81023c0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.255+0000 7f39e4ff9640 1 -- 172.21.15.69:0/4257761401 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e0005020 con 0x7f39e81023c0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.255+0000 7f39e4ff9640 1 -- 172.21.15.69:0/4257761401 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e00116b0 con 0x7f39e81023c0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.255+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e81a0050 con 0x7f39e81023c0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.256+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e8107c70 con 0x7f39e81023c0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.257+0000 7f39e4ff9640 1 -- 172.21.15.69:0/4257761401 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f39e0005560 con 0x7f39e81023c0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.257+0000 7f39e4ff9640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f39d40a2c10 0x7f39d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.258+0000 7f39e77fe640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f39d40a2c10 0x7f39d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.258+0000 7f39e4ff9640 1 -- 172.21.15.69:0/4257761401 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f39e00cf880 con 0x7f39e81023c0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.258+0000 7f39e77fe640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f39d40a2c10 0x7f39d40a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39d80058c0 tx=0x7f39d800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:02.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.262+0000 7f39e4ff9640 1 -- 172.21.15.69:0/4257761401 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f39e0016030 con 0x7f39e81023c0 2024-08-02T17:46:02.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:01 smithi077 ceph-mon[37108]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:02.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:01 smithi077 ceph-mon[37108]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-08-02T17:46:02.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.435+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f39e8062360 con 0x7f39d40a2c10 2024-08-02T17:46:02.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.437+0000 7f39e4ff9640 1 -- 172.21.15.69:0/4257761401 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f39e8062360 con 0x7f39d40a2c10 2024-08-02T17:46:02.438 INFO:teuthology.orchestra.run.smithi069.stdout:HOST ADDR LABELS STATUS 2024-08-02T17:46:02.438 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 172.21.15.69 2024-08-02T17:46:02.439 INFO:teuthology.orchestra.run.smithi069.stdout:smithi077 172.21.15.77 2024-08-02T17:46:02.439 INFO:teuthology.orchestra.run.smithi069.stdout:smithi083 172.21.15.83 2024-08-02T17:46:02.439 INFO:teuthology.orchestra.run.smithi069.stdout:3 hosts in cluster 2024-08-02T17:46:02.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f39d40a2c10 msgr2=0x7f39d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f39d40a2c10 0x7f39d40a50d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39d80058c0 tx=0x7f39d800a4c0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e81023c0 msgr2=0x7f39e819efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e81023c0 0x7f39e819efc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f39e000eaa0 tx=0x7f39e000ef60 comp rx=0 tx=0).stop 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 shutdown_connections 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f39d40a2c10 0x7f39d40a50d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e81023c0 0x7f39e819efc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e8101a00 0x7f39e819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 --2- 172.21.15.69:0/4257761401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e8100800 0x7f39e806a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 >> 172.21.15.69:0/4257761401 conn(0x7f39e80fbfb0 msgr2=0x7f39e8104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 shutdown_connections 2024-08-02T17:46:02.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:02.439+0000 7f39eda78640 1 -- 172.21.15.69:0/4257761401 wait complete. 2024-08-02T17:46:03.139 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-02T17:46:03.563 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:03.974 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:03 smithi069 ceph-mon[32746]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:03.975 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:03 smithi069 ceph-mon[32746]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:03.975 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:03 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:04.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 -- 172.21.15.69:0/4252924652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc100800 msgr2=0x7f93bc100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:04.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 --2- 172.21.15.69:0/4252924652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc100800 0x7f93bc100c00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009ca0 tx=0x7f93b402a8c0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 -- 172.21.15.69:0/4252924652 shutdown_connections 2024-08-02T17:46:04.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 --2- 172.21.15.69:0/4252924652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93bc1023c0 0x7f93bc108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 --2- 172.21.15.69:0/4252924652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93bc101a00 0x7f93bc101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 --2- 172.21.15.69:0/4252924652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc100800 0x7f93bc100c00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 -- 172.21.15.69:0/4252924652 >> 172.21.15.69:0/4252924652 conn(0x7f93bc0fbfb0 msgr2=0x7f93bc0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 -- 172.21.15.69:0/4252924652 shutdown_connections 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 -- 172.21.15.69:0/4252924652 wait complete. 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.051+0000 7f93c1462640 1 Processor -- start 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93c1462640 1 -- start start 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93c1462640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93bc100800 0x7f93bc197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93c1462640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93bc101a00 0x7f93bc1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93c1462640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc1023c0 0x7f93bc19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93c1462640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc10d570 con 0x7f93bc1023c0 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93c1462640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc10d3f0 con 0x7f93bc100800 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93c1462640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93bc10d6f0 con 0x7f93bc101a00 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc1023c0 0x7f93bc19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:04.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc1023c0 0x7f93bc19f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45240/0 (socket says 172.21.15.69:45240) 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 -- 172.21.15.69:0/1886340411 learned_addr learned my addr 172.21.15.69:0/1886340411 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93baffd640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93bc100800 0x7f93bc197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93ba7fc640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93bc101a00 0x7f93bc1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 -- 172.21.15.69:0/1886340411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93bc101a00 msgr2=0x7f93bc1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93bc101a00 0x7f93bc1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 -- 172.21.15.69:0/1886340411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93bc100800 msgr2=0x7f93bc197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93bc100800 0x7f93bc197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 -- 172.21.15.69:0/1886340411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93bc19f880 con 0x7f93bc1023c0 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.052+0000 7f93bb7fe640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc1023c0 0x7f93bc19f1e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e970 tx=0x7f93ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.053+0000 7f93b3fff640 1 -- 172.21.15.69:0/1886340411 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac00ccc0 con 0x7f93bc1023c0 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.053+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93bc19fb70 con 0x7f93bc1023c0 2024-08-02T17:46:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.053+0000 7f93b3fff640 1 -- 172.21.15.69:0/1886340411 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93ac00ce60 con 0x7f93bc1023c0 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.053+0000 7f93b3fff640 1 -- 172.21.15.69:0/1886340411 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ac010490 con 0x7f93bc1023c0 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.053+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93bc1a00b0 con 0x7f93bc1023c0 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.054+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93bc107bf0 con 0x7f93bc1023c0 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.055+0000 7f93b3fff640 1 -- 172.21.15.69:0/1886340411 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f93ac005020 con 0x7f93bc1023c0 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.055+0000 7f93b3fff640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f93a80a2c10 0x7f93a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.056+0000 7f93baffd640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f93a80a2c10 0x7f93a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.056+0000 7f93b3fff640 1 -- 172.21.15.69:0/1886340411 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f93ac0cf9f0 con 0x7f93bc1023c0 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.056+0000 7f93baffd640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f93a80a2c10 0x7f93a80a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93b4031ae0 tx=0x7f93b4000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:04.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.059+0000 7f93b3fff640 1 -- 172.21.15.69:0/1886340411 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f93ac09c440 con 0x7f93bc1023c0 2024-08-02T17:46:04.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.256+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 --> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f93bc062360 con 0x7f93a80a2c10 2024-08-02T17:46:04.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.258+0000 7f93b3fff640 1 -- 172.21.15.69:0/1886340411 <== mgr.14150 v2:172.21.15.69:6800/452236630 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f93bc062360 con 0x7f93a80a2c10 2024-08-02T17:46:04.259 INFO:teuthology.orchestra.run.smithi069.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-02T17:46:04.259 INFO:teuthology.orchestra.run.smithi069.stdout:smithi069 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y0400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-02T17:46:04.259 INFO:teuthology.orchestra.run.smithi069.stdout:smithi077 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZJ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-02T17:46:04.259 INFO:teuthology.orchestra.run.smithi069.stdout:smithi083 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040148400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-02T17:46:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f93a80a2c10 msgr2=0x7f93a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f93a80a2c10 0x7f93a80a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93b4031ae0 tx=0x7f93b4000f40 comp rx=0 tx=0).stop 2024-08-02T17:46:04.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc1023c0 msgr2=0x7f93bc19f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc1023c0 0x7f93bc19f1e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f93ac00e970 tx=0x7f93ac00ed70 comp rx=0 tx=0).stop 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 shutdown_connections 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f93a80a2c10 0x7f93a80a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93bc1023c0 0x7f93bc19f1e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93bc101a00 0x7f93bc1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 --2- 172.21.15.69:0/1886340411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93bc100800 0x7f93bc197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 >> 172.21.15.69:0/1886340411 conn(0x7f93bc0fbfb0 msgr2=0x7f93bc104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 shutdown_connections 2024-08-02T17:46:04.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:04.260+0000 7f93c1462640 1 -- 172.21.15.69:0/1886340411 wait complete. 2024-08-02T17:46:04.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:03 smithi083 ceph-mon[37031]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:04.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:03 smithi083 ceph-mon[37031]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:04.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:03 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:04.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:03 smithi077 ceph-mon[37108]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:04.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:03 smithi077 ceph-mon[37108]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:04.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:03 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:05.215 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-02T17:46:05.227 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:46:05.227 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-02T17:46:05.627 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:06.001 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:05 smithi069 ceph-mon[32746]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:06.001 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:05 smithi069 ceph-mon[32746]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-08-02T17:46:06.001 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.001 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.001 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.001 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:05 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 -- 172.21.15.69:0/3669718866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c100800 msgr2=0x7fe84c100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:06.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 --2- 172.21.15.69:0/3669718866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c100800 0x7fe84c100c00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009900 tx=0x7fe83c02a410 comp rx=0 tx=0).stop 2024-08-02T17:46:06.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 -- 172.21.15.69:0/3669718866 shutdown_connections 2024-08-02T17:46:06.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 --2- 172.21.15.69:0/3669718866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c1023c0 0x7fe84c108c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 --2- 172.21.15.69:0/3669718866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe84c101a00 0x7fe84c101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 --2- 172.21.15.69:0/3669718866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c100800 0x7fe84c100c00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 -- 172.21.15.69:0/3669718866 >> 172.21.15.69:0/3669718866 conn(0x7fe84c0fbfb0 msgr2=0x7fe84c0fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 -- 172.21.15.69:0/3669718866 shutdown_connections 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.080+0000 7fe85146a640 1 -- 172.21.15.69:0/3669718866 wait complete. 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.081+0000 7fe85146a640 1 Processor -- start 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.081+0000 7fe85146a640 1 -- start start 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.081+0000 7fe85146a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe84c100800 0x7fe84c199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.081+0000 7fe85146a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c101a00 0x7fe84c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.081+0000 7fe85146a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c1023c0 0x7fe84c1a13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.081+0000 7fe85146a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c10d570 con 0x7fe84c1023c0 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.081+0000 7fe85146a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c10d3f0 con 0x7fe84c100800 2024-08-02T17:46:06.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.081+0000 7fe85146a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe84c10d6f0 con 0x7fe84c101a00 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c1023c0 0x7fe84c1a13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c1023c0 0x7fe84c1a13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37108/0 (socket says 172.21.15.69:37108) 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 -- 172.21.15.69:0/3860023318 learned_addr learned my addr 172.21.15.69:0/3860023318 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84affd640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe84c100800 0x7fe84c199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84a7fc640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c101a00 0x7fe84c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 -- 172.21.15.69:0/3860023318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c101a00 msgr2=0x7fe84c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c101a00 0x7fe84c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 -- 172.21.15.69:0/3860023318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe84c100800 msgr2=0x7fe84c199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe84c100800 0x7fe84c199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 -- 172.21.15.69:0/3860023318 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84c1a1a30 con 0x7fe84c1023c0 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84affd640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe84c100800 0x7fe84c199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe84b7fe640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c1023c0 0x7fe84c1a13c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe84400cce0 tx=0x7fe84400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:06.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe833fff640 1 -- 172.21.15.69:0/3860023318 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe84400f450 con 0x7fe84c1023c0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe833fff640 1 -- 172.21.15.69:0/3860023318 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe84400fad0 con 0x7fe84c1023c0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84c1a1d20 con 0x7fe84c1023c0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe833fff640 1 -- 172.21.15.69:0/3860023318 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe844018c10 con 0x7fe84c1023c0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.082+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe84c1a21b0 con 0x7fe84c1023c0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.083+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe84c1a2440 con 0x7fe84c1023c0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.084+0000 7fe833fff640 1 -- 172.21.15.69:0/3860023318 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe84400f5f0 con 0x7fe84c1023c0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.085+0000 7fe833fff640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8340a2c40 0x7fe8340a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.085+0000 7fe833fff640 1 -- 172.21.15.69:0/3860023318 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe8440cfb90 con 0x7fe84c1023c0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.085+0000 7fe84affd640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8340a2c40 0x7fe8340a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:06.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.086+0000 7fe84affd640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8340a2c40 0x7fe8340a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe83c005ab0 tx=0x7fe83c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:06.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.089+0000 7fe833fff640 1 -- 172.21.15.69:0/3860023318 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe844014030 con 0x7fe84c1023c0 2024-08-02T17:46:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:05 smithi083 ceph-mon[37031]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:05 smithi083 ceph-mon[37031]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-08-02T17:46:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:05 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.316+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fe84c1a27a0 con 0x7fe84c1023c0 2024-08-02T17:46:06.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.317+0000 7fe833fff640 1 -- 172.21.15.69:0/3860023318 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3080 (secure 0 0 0) 0x7fe84409c5e0 con 0x7fe84c1023c0 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout:e25 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout:btime 2024-08-02T17:44:49:913571+0000 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout:legacy client fscid: 1 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem 'cephfs' (1) 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout:fs_name cephfs 2024-08-02T17:46:06.319 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 25 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-08-02T17:44:04.896710+0000 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-08-02T17:44:49.496243+0000 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:tableserver 0 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:root 0 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:session_timeout 300 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:session_autoclose 300 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:max_file_size 1099511627776 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:max_xattr_size 65536 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:required_client_features {} 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure 0 2024-08-02T17:46:06.320 INFO:teuthology.orchestra.run.smithi069.stdout:last_failure_osd_epoch 0 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:max_mds 3 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:in 0,1,2 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:up {0=24452,1=24431,2=24427} 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:failed 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:damaged 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:stopped 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:data_pools [3,4] 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:metadata_pool 2 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:inline_data disabled 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:balancer 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:bal_rank_mask -1 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:standby_count_wanted 0 2024-08-02T17:46:06.321 INFO:teuthology.orchestra.run.smithi069.stdout:qdb_cluster leader: 24452 members: 24427,24431,24452 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.b{0:24452} state up:active seq 4 addr [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.e{0:24413} state up:standby-replay seq 1 addr [v2:172.21.15.77:6834/3410285190,v1:172.21.15.77:6835/3410285190] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.h{1:24431} state up:active seq 17 addr [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.c{1:24409} state up:standby-replay seq 1 addr [v2:172.21.15.83:6836/1744999893,v1:172.21.15.83:6837/1744999893] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.i{2:24427} state up:active seq 17 addr [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.f{2:24403} state up:standby-replay seq 1 addr [v2:172.21.15.83:6834/3991686236,v1:172.21.15.83:6835/3991686236] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:Standby daemons: 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.j{-1:14487} state up:standby seq 1 addr [v2:172.21.15.69:6834/570537244,v1:172.21.15.69:6835/570537244] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.d{-1:14499} state up:standby seq 1 addr [v2:172.21.15.69:6836/3783033503,v1:172.21.15.69:6837/3783033503] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.322 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.g{-1:14553} state up:standby seq 1 addr [v2:172.21.15.69:6838/1157046912,v1:172.21.15.69:6839/1157046912] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.323 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.a{-1:14559} state up:standby seq 1 addr [v2:172.21.15.69:6840/251587008,v1:172.21.15.69:6841/251587008] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.323 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.l{-1:24382} state up:standby seq 1 addr [v2:172.21.15.83:6832/2696363098,v1:172.21.15.83:6833/2696363098] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.323 INFO:teuthology.orchestra.run.smithi069.stdout:[mds.k{-1:24389} state up:standby seq 1 addr [v2:172.21.15.77:6832/2071376346,v1:172.21.15.77:6833/2071376346] compat {c=[1],r=[1],i=[1fff]}] 2024-08-02T17:46:06.323 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-08-02T17:46:06.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8340a2c40 msgr2=0x7fe8340a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:06.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8340a2c40 0x7fe8340a5100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe83c005ab0 tx=0x7fe83c000bc0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c1023c0 msgr2=0x7fe84c1a13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c1023c0 0x7fe84c1a13c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe84400cce0 tx=0x7fe84400a620 comp rx=0 tx=0).stop 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 shutdown_connections 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe8340a2c40 0x7fe8340a5100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe84c1023c0 0x7fe84c1a13c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe84c101a00 0x7fe84c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 --2- 172.21.15.69:0/3860023318 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe84c100800 0x7fe84c199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 >> 172.21.15.69:0/3860023318 conn(0x7fe84c0fbfb0 msgr2=0x7fe84c109490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 shutdown_connections 2024-08-02T17:46:06.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:06.320+0000 7fe85146a640 1 -- 172.21.15.69:0/3860023318 wait complete. 2024-08-02T17:46:06.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:05 smithi077 ceph-mon[37108]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:06.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:05 smithi077 ceph-mon[37108]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-08-02T17:46:06.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:06.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:05 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:07.075 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:06 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3860023318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-02T17:46:07.103 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-02T17:46:07.248 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:06 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3860023318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-02T17:46:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:06 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3860023318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-08-02T17:46:07.381 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 -- 172.21.15.69:0/1302037399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 msgr2=0x7f640c100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 --2- 172.21.15.69:0/1302037399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 0x7f640c100c40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009900 tx=0x7f63fc02a410 comp rx=0 tx=0).stop 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 -- 172.21.15.69:0/1302037399 shutdown_connections 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 --2- 172.21.15.69:0/1302037399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f640c102400 0x7f640c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 --2- 172.21.15.69:0/1302037399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c101a40 0x7f640c101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 --2- 172.21.15.69:0/1302037399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 0x7f640c100c40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 -- 172.21.15.69:0/1302037399 >> 172.21.15.69:0/1302037399 conn(0x7f640c0fbfd0 msgr2=0x7f640c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 -- 172.21.15.69:0/1302037399 shutdown_connections 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.887+0000 7f6411990640 1 -- 172.21.15.69:0/1302037399 wait complete. 2024-08-02T17:46:07.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6411990640 1 Processor -- start 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6411990640 1 -- start start 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6411990640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 0x7f640c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6411990640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c101a40 0x7f640c1981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6411990640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f640c102400 0x7f640c19f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6411990640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10c520 con 0x7f640c100840 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6411990640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10c3a0 con 0x7f640c102400 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6411990640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f640c10c6a0 con 0x7f640c101a40 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f640affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 0x7f640c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f640affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 0x7f640c197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37136/0 (socket says 172.21.15.69:37136) 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f640affd640 1 -- 172.21.15.69:0/149721473 learned_addr learned my addr 172.21.15.69:0/149721473 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f6403fff640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c101a40 0x7f640c1981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:07.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.888+0000 7f640b7fe640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f640c102400 0x7f640c19f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f640affd640 1 -- 172.21.15.69:0/149721473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c101a40 msgr2=0x7f640c1981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f640affd640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c101a40 0x7f640c1981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f640affd640 1 -- 172.21.15.69:0/149721473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f640c102400 msgr2=0x7f640c19f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f640affd640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f640c102400 0x7f640c19f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f640affd640 1 -- 172.21.15.69:0/149721473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c19f880 con 0x7f640c100840 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f640b7fe640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f640c102400 0x7f640c19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f6403fff640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c101a40 0x7f640c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f640affd640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 0x7f640c197c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63fc031610 tx=0x7f63fc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f641098e640 1 -- 172.21.15.69:0/149721473 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc03f030 con 0x7f640c100840 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f641098e640 1 -- 172.21.15.69:0/149721473 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63fc003ce0 con 0x7f640c100840 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c19fb10 con 0x7f640c100840 2024-08-02T17:46:07.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f641098e640 1 -- 172.21.15.69:0/149721473 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc035070 con 0x7f640c100840 2024-08-02T17:46:07.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.889+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c19fff0 con 0x7f640c100840 2024-08-02T17:46:07.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.890+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640c1a0250 con 0x7f640c100840 2024-08-02T17:46:07.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.891+0000 7f641098e640 1 -- 172.21.15.69:0/149721473 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f63fc042030 con 0x7f640c100840 2024-08-02T17:46:07.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.892+0000 7f641098e640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f63f80a2c10 0x7f63f80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:07.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.892+0000 7f6403fff640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f63f80a2c10 0x7f63f80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:07.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.892+0000 7f641098e640 1 -- 172.21.15.69:0/149721473 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f63fc0efce0 con 0x7f640c100840 2024-08-02T17:46:07.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.892+0000 7f6403fff640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f63f80a2c10 0x7f63f80a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f640c199170 tx=0x7f63f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:07.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:07.896+0000 7f641098e640 1 -- 172.21.15.69:0/149721473 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f63fc0bc730 con 0x7f640c100840 2024-08-02T17:46:08.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.063+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f640c1a0420 con 0x7f640c100840 2024-08-02T17:46:08.065 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:07 smithi069 ceph-mon[32746]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-08-02T17:46:08.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:08.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:08.066 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:07 smithi069 ceph-mon[32746]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.064+0000 7f641098e640 1 -- 172.21.15.69:0/149721473 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5643 (secure 0 0 0) 0x7f63fc0bc030 con 0x7f640c100840 2024-08-02T17:46:08.066 INFO:teuthology.orchestra.run.smithi069.stdout:epoch 94 2024-08-02T17:46:08.067 INFO:teuthology.orchestra.run.smithi069.stdout:fsid b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T17:46:08.067 INFO:teuthology.orchestra.run.smithi069.stdout:created 2024-08-02T17:37:04.618643+0000 2024-08-02T17:46:08.067 INFO:teuthology.orchestra.run.smithi069.stdout:modified 2024-08-02T17:45:57.847197+0000 2024-08-02T17:46:08.067 INFO:teuthology.orchestra.run.smithi069.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-02T17:46:08.067 INFO:teuthology.orchestra.run.smithi069.stdout:crush_version 27 2024-08-02T17:46:08.067 INFO:teuthology.orchestra.run.smithi069.stdout:full_ratio 0.95 2024-08-02T17:46:08.067 INFO:teuthology.orchestra.run.smithi069.stdout:backfillfull_ratio 0.9 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:nearfull_ratio 0.85 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:require_min_compat_client luminous 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:min_compat_client luminous 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:require_osd_release squid 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:stretch_mode_enabled false 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:max_osd 12 2024-08-02T17:46:08.068 INFO:teuthology.orchestra.run.smithi069.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6802/3281611290,v1:172.21.15.69:6803/3281611290] [v2:172.21.15.69:6804/3281611290,v1:172.21.15.69:6805/3281611290] exists,up 7fad0068-7f9b-4e76-9c10-6338b2fa7619 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6800/1438822900,v1:172.21.15.77:6801/1438822900] [v2:172.21.15.77:6802/1438822900,v1:172.21.15.77:6803/1438822900] exists,up eb73073e-0fd6-4482-b800-03ef39b9c61e 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.2 up in weight 1 up_from 18 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6800/2938396779,v1:172.21.15.83:6801/2938396779] [v2:172.21.15.83:6802/2938396779,v1:172.21.15.83:6803/2938396779] exists,up 941227f1-e5e4-4146-9e0d-4405d1f5271c 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.3 up in weight 1 up_from 24 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6810/980470189,v1:172.21.15.69:6811/980470189] [v2:172.21.15.69:6812/980470189,v1:172.21.15.69:6813/980470189] exists,up 9482789a-9684-4d80-90d1-4116f82bcce4 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.4 up in weight 1 up_from 32 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6808/64984823,v1:172.21.15.77:6809/64984823] [v2:172.21.15.77:6810/64984823,v1:172.21.15.77:6811/64984823] exists,up e22a3571-a363-4b52-9b84-725cf9b5fe4c 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.5 up in weight 1 up_from 37 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6808/3536789836,v1:172.21.15.83:6809/3536789836] [v2:172.21.15.83:6810/3536789836,v1:172.21.15.83:6811/3536789836] exists,up c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.6 up in weight 1 up_from 45 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6818/2370248409,v1:172.21.15.69:6819/2370248409] [v2:172.21.15.69:6820/2370248409,v1:172.21.15.69:6821/2370248409] exists,up e6f58da4-cae1-4c9d-968f-4ab620039158 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.7 up in weight 1 up_from 50 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6816/2324182206,v1:172.21.15.77:6817/2324182206] [v2:172.21.15.77:6818/2324182206,v1:172.21.15.77:6819/2324182206] exists,up 392b17a2-c105-464b-beaa-4f531883e292 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.8 up in weight 1 up_from 58 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6816/1048275845,v1:172.21.15.83:6817/1048275845] [v2:172.21.15.83:6818/1048275845,v1:172.21.15.83:6819/1048275845] exists,up 92a6e799-6b88-4d10-bc05-8a3b5c43c8be 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.9 up in weight 1 up_from 66 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.69:6826/2821083254,v1:172.21.15.69:6827/2821083254] [v2:172.21.15.69:6828/2821083254,v1:172.21.15.69:6829/2821083254] exists,up 2ffe6b5e-7950-4db4-8dc5-6d0351b0a346 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.10 up in weight 1 up_from 71 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.77:6824/2744845863,v1:172.21.15.77:6825/2744845863] [v2:172.21.15.77:6826/2744845863,v1:172.21.15.77:6827/2744845863] exists,up 88334ccf-216f-4d58-b9e1-6c173f4897ef 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:osd.11 up in weight 1 up_from 78 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.83:6824/697474506,v1:172.21.15.83:6825/697474506] [v2:172.21.15.83:6826/697474506,v1:172.21.15.83:6827/697474506] exists,up 2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e 2024-08-02T17:46:08.069 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.0 [8,1] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.6 [8,11] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.a [2,11] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 2.10 [2,4] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.e [5,9] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.21 [4,6] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.27 [4,8] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.29 [3,9] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 3.36 [4,9] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.5 [5,8] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.e [9,8] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.12 [9,10] 2024-08-02T17:46:08.070 INFO:teuthology.orchestra.run.smithi069.stdout:pg_upmap_items 4.14 [9,8] 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2814849525 expires 2024-08-03T17:37:56.685064+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2060958760 expires 2024-08-03T17:37:56.685064+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/623492814 expires 2024-08-03T17:37:34.617397+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/3801376647 expires 2024-08-03T17:37:34.617397+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/3376234785 expires 2024-08-03T17:37:56.685064+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/2152855317 expires 2024-08-03T17:37:56.685064+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6801/3960560393 expires 2024-08-03T17:37:34.617397+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/3376234785 expires 2024-08-03T17:37:56.685064+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:6800/3960560393 expires 2024-08-03T17:37:34.617397+0000 2024-08-02T17:46:08.071 INFO:teuthology.orchestra.run.smithi069.stdout:blocklist 172.21.15.69:0/4073447426 expires 2024-08-03T17:37:34.617397+0000 2024-08-02T17:46:08.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f63f80a2c10 msgr2=0x7f63f80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:08.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f63f80a2c10 0x7f63f80a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f640c199170 tx=0x7f63f4008040 comp rx=0 tx=0).stop 2024-08-02T17:46:08.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 msgr2=0x7f640c197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:08.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 0x7f640c197c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f63fc031610 tx=0x7f63fc0043b0 comp rx=0 tx=0).stop 2024-08-02T17:46:08.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 shutdown_connections 2024-08-02T17:46:08.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f63f80a2c10 0x7f63f80a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:08.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f640c102400 0x7f640c19f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:08.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f640c101a40 0x7f640c1981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:08.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 --2- 172.21.15.69:0/149721473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f640c100840 0x7f640c197c60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:08.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.066+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 >> 172.21.15.69:0/149721473 conn(0x7f640c0fbfd0 msgr2=0x7f640c06a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:08.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.067+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 shutdown_connections 2024-08-02T17:46:08.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:08.067+0000 7f6411990640 1 -- 172.21.15.69:0/149721473 wait complete. 2024-08-02T17:46:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:07 smithi083 ceph-mon[37031]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-08-02T17:46:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:07 smithi083 ceph-mon[37031]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:07 smithi077 ceph-mon[37108]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-08-02T17:46:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:07 smithi077 ceph-mon[37108]: from='mgr.14150 172.21.15.69:0/4002179050' entity='mgr.x' 2024-08-02T17:46:08.811 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-02T17:46:08.822 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:46:09.080 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:09.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:08 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/149721473' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-02T17:46:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:08 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/149721473' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-02T17:46:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:08 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/149721473' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/310169135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0103430 msgr2=0x7f3ab010b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/310169135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0103430 0x7f3ab010b920 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3a98009ca0 tx=0x7f3a9802a950 comp rx=0 tx=0).stop 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/310169135 shutdown_connections 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/310169135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0103430 0x7f3ab010b920 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/310169135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab010beb0 0x7f3ab010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/310169135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab0102b10 0x7f3ab0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/310169135 >> 172.21.15.69:0/310169135 conn(0x7f3ab00fc4d0 msgr2=0x7f3ab00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/310169135 shutdown_connections 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/310169135 wait complete. 2024-08-02T17:46:09.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 Processor -- start 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.536+0000 7f3ab6d7d640 1 -- start start 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab6d7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102b10 0x7f3ab019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab6d7d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab0103430 0x7f3ab019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab6d7d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab010beb0 0x7f3ab019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab6d7d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0111ce0 con 0x7f3ab0102b10 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab6d7d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0111b60 con 0x7f3ab0103430 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab6d7d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab0111e60 con 0x7f3ab010beb0 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102b10 0x7f3ab019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102b10 0x7f3ab019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37148/0 (socket says 172.21.15.69:37148) 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 -- 172.21.15.69:0/1402302158 learned_addr learned my addr 172.21.15.69:0/1402302158 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab531a640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab010beb0 0x7f3ab019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:09.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3aaffff640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab0103430 0x7f3ab019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 -- 172.21.15.69:0/1402302158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab010beb0 msgr2=0x7f3ab019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab010beb0 0x7f3ab019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 -- 172.21.15.69:0/1402302158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab0103430 msgr2=0x7f3ab019a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab0103430 0x7f3ab019a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 -- 172.21.15.69:0/1402302158 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab019f350 con 0x7f3ab0102b10 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab4b19640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102b10 0x7f3ab019a3a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000b4a0 tx=0x7f3aa000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3aadffb640 1 -- 172.21.15.69:0/1402302158 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0014070 con 0x7f3ab0102b10 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab019f640 con 0x7f3ab0102b10 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3aadffb640 1 -- 172.21.15.69:0/1402302158 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aa0010af0 con 0x7f3ab0102b10 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.537+0000 7f3aadffb640 1 -- 172.21.15.69:0/1402302158 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0017d10 con 0x7f3ab0102b10 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.538+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab01a6f20 con 0x7f3ab0102b10 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.539+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab0102ef0 con 0x7f3ab0102b10 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.540+0000 7f3aadffb640 1 -- 172.21.15.69:0/1402302158 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3aa0017070 con 0x7f3ab0102b10 2024-08-02T17:46:09.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.540+0000 7f3aadffb640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a9c0a2c10 0x7f3a9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:09.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.540+0000 7f3aaffff640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a9c0a2c10 0x7f3a9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:09.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.540+0000 7f3aadffb640 1 -- 172.21.15.69:0/1402302158 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f3aa00ce790 con 0x7f3ab0102b10 2024-08-02T17:46:09.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.541+0000 7f3aaffff640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a9c0a2c10 0x7f3a9c0a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00fe910 tx=0x7f3a98009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:09.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.545+0000 7f3aadffb640 1 -- 172.21.15.69:0/1402302158 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3aa009aca0 con 0x7f3ab0102b10 2024-08-02T17:46:09.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.744+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ab01a7150 con 0x7f3ab0102b10 2024-08-02T17:46:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.744+0000 7f3aadffb640 1 -- 172.21.15.69:0/1402302158 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11900 (secure 0 0 0) 0x7f3aa009efa0 con 0x7f3ab0102b10 2024-08-02T17:46:09.746 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:09.746 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"btime":"2024-08-02T17:44:49:913571+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:44:49.496243+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24431,24452]},"id":1}]} 2024-08-02T17:46:09.746 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-08-02T17:46:09.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.746+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a9c0a2c10 msgr2=0x7f3a9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:09.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.746+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a9c0a2c10 0x7f3a9c0a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00fe910 tx=0x7f3a98009360 comp rx=0 tx=0).stop 2024-08-02T17:46:09.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.746+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102b10 msgr2=0x7f3ab019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:09.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.746+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102b10 0x7f3ab019a3a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000b4a0 tx=0x7f3aa000b960 comp rx=0 tx=0).stop 2024-08-02T17:46:09.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.747+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 shutdown_connections 2024-08-02T17:46:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.747+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f3a9c0a2c10 0x7f3a9c0a50d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.747+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ab010beb0 0x7f3ab019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.747+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ab0103430 0x7f3ab019a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.747+0000 7f3ab6d7d640 1 --2- 172.21.15.69:0/1402302158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ab0102b10 0x7f3ab019a3a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.747+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 >> 172.21.15.69:0/1402302158 conn(0x7f3ab00fc4d0 msgr2=0x7f3ab01006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.747+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 shutdown_connections 2024-08-02T17:46:09.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:09.747+0000 7f3ab6d7d640 1 -- 172.21.15.69:0/1402302158 wait complete. 2024-08-02T17:46:10.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:09 smithi069 ceph-mon[32746]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-02T17:46:10.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:09 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1402302158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:09 smithi083 ceph-mon[37031]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-02T17:46:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:09 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1402302158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:10.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:09 smithi077 ceph-mon[37108]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-02T17:46:10.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:09 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1402302158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:10.507 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:46:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2124926733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e1010bea0 msgr2=0x7f5e1010e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2124926733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e1010bea0 0x7f5e1010e340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009ca0 tx=0x7f5e0802a950 comp rx=0 tx=0).stop 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2124926733 shutdown_connections 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2124926733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e1010bea0 0x7f5e1010e340 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2124926733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e10103390 0x7f5e1010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2124926733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e10102a70 0x7f5e10102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2124926733 >> 172.21.15.69:0/2124926733 conn(0x7f5e100fc4d0 msgr2=0x7f5e100fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2124926733 shutdown_connections 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.220+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2124926733 wait complete. 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e15c4f640 1 Processor -- start 2024-08-02T17:46:11.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e15c4f640 1 -- start start 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e15c4f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e10102a70 0x7f5e10198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e15c4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e10103390 0x7f5e101986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e15c4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1010bea0 0x7f5e1019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e15c4f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10111c80 con 0x7f5e10102a70 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e15c4f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10111b00 con 0x7f5e10103390 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e15c4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10111e00 con 0x7f5e1010bea0 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e0f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e10102a70 0x7f5e10198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e0f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e10102a70 0x7f5e10198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37164/0 (socket says 172.21.15.69:37164) 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.221+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2615836308 learned_addr learned my addr 172.21.15.69:0/2615836308 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0ffff640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1010bea0 0x7f5e1019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0effd640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e10103390 0x7f5e101986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2615836308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1010bea0 msgr2=0x7f5e1019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:11.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0f7fe640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1010bea0 0x7f5e1019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2615836308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e10103390 msgr2=0x7f5e101986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0f7fe640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e10103390 0x7f5e101986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2615836308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1019d140 con 0x7f5e10102a70 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0f7fe640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e10102a70 0x7f5e10198190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000c890 tx=0x7f5e0000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0cff9640 1 -- 172.21.15.69:0/2615836308 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00004220 con 0x7f5e10102a70 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0cff9640 1 -- 172.21.15.69:0/2615836308 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e000043c0 con 0x7f5e10102a70 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e0cff9640 1 -- 172.21.15.69:0/2615836308 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e000049e0 con 0x7f5e10102a70 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1019d430 con 0x7f5e10102a70 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.222+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e101a4d10 con 0x7f5e10102a70 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.223+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e10102e50 con 0x7f5e10102a70 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.224+0000 7f5e0cff9640 1 -- 172.21.15.69:0/2615836308 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5e000079b0 con 0x7f5e10102a70 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.225+0000 7f5e0cff9640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5df80a2c10 0x7f5df80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.225+0000 7f5e0effd640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5df80a2c10 0x7f5df80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:11.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.225+0000 7f5e0cff9640 1 -- 172.21.15.69:0/2615836308 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f5e000cf610 con 0x7f5e10102a70 2024-08-02T17:46:11.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.225+0000 7f5e0effd640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5df80a2c10 0x7f5df80a50d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005d70 tx=0x7f5e04005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:11.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.228+0000 7f5e0cff9640 1 -- 172.21.15.69:0/2615836308 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5e00014030 con 0x7f5e10102a70 2024-08-02T17:46:11.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.427+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e101994f0 con 0x7f5e10102a70 2024-08-02T17:46:11.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.428+0000 7f5e0cff9640 1 -- 172.21.15.69:0/2615836308 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11900 (secure 0 0 0) 0x7f5e0009c060 con 0x7f5e10102a70 2024-08-02T17:46:11.430 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:11.430 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":25,"btime":"2024-08-02T17:44:49:913571+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:44:49.496243+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24427,24431,24452]},"id":1}]} 2024-08-02T17:46:11.430 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 25 2024-08-02T17:46:11.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5df80a2c10 msgr2=0x7f5df80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:11.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5df80a2c10 0x7f5df80a50d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5e04005d70 tx=0x7f5e04005d00 comp rx=0 tx=0).stop 2024-08-02T17:46:11.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e10102a70 msgr2=0x7f5e10198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:11.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e10102a70 0x7f5e10198190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5e0000c890 tx=0x7f5e0000cd50 comp rx=0 tx=0).stop 2024-08-02T17:46:11.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 shutdown_connections 2024-08-02T17:46:11.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f5df80a2c10 0x7f5df80a50d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1010bea0 0x7f5e1019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e10103390 0x7f5e101986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 --2- 172.21.15.69:0/2615836308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e10102a70 0x7f5e10198190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:11.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 >> 172.21.15.69:0/2615836308 conn(0x7f5e100fc4d0 msgr2=0x7f5e100fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:11.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 shutdown_connections 2024-08-02T17:46:11.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:11.431+0000 7f5e15c4f640 1 -- 172.21.15.69:0/2615836308 wait complete. 2024-08-02T17:46:12.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:11 smithi069 ceph-mon[32746]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:46:12.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:11 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2615836308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:12.192 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:46:12.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:11 smithi083 ceph-mon[37031]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:46:12.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:11 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2615836308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:12.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:11 smithi077 ceph-mon[37108]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:46:12.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:11 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2615836308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 -- 172.21.15.69:0/3501099945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100190 msgr2=0x7fe19c100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 --2- 172.21.15.69:0/3501099945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100190 0x7fe19c100570 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe19000aa10 tx=0x7fe19002b720 comp rx=0 tx=0).stop 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 -- 172.21.15.69:0/3501099945 shutdown_connections 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 --2- 172.21.15.69:0/3501099945 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c1052c0 0x7fe19c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 --2- 172.21.15.69:0/3501099945 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe19c100b40 0x7fe19c104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 --2- 172.21.15.69:0/3501099945 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100190 0x7fe19c100570 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 -- 172.21.15.69:0/3501099945 >> 172.21.15.69:0/3501099945 conn(0x7fe19c0fbc10 msgr2=0x7fe19c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 -- 172.21.15.69:0/3501099945 shutdown_connections 2024-08-02T17:46:12.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.952+0000 7fe1a323e640 1 -- 172.21.15.69:0/3501099945 wait complete. 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a323e640 1 Processor -- start 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a323e640 1 -- start start 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a323e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c100190 0x7fe19c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a323e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100b40 0x7fe19c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a323e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe19c1052c0 0x7fe19c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a323e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10fb40 con 0x7fe19c100b40 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a323e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10f9c0 con 0x7fe19c1052c0 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a323e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10fcc0 con 0x7fe19c100190 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100b40 0x7fe19c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100b40 0x7fe19c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37188/0 (socket says 172.21.15.69:37188) 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe19bfff640 1 -- 172.21.15.69:0/2005525400 learned_addr learned my addr 172.21.15.69:0/2005525400 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:12.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.953+0000 7fe1a17db640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe19c1052c0 0x7fe19c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe1a0fda640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c100190 0x7fe19c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe19bfff640 1 -- 172.21.15.69:0/2005525400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c100190 msgr2=0x7fe19c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe19bfff640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c100190 0x7fe19c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe19bfff640 1 -- 172.21.15.69:0/2005525400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe19c1052c0 msgr2=0x7fe19c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe19bfff640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe19c1052c0 0x7fe19c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe19bfff640 1 -- 172.21.15.69:0/2005525400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c19d390 con 0x7fe19c100b40 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe1a0fda640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c100190 0x7fe19c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe19bfff640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100b40 0x7fe19c198920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe18800b4a0 tx=0x7fe18800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe199ffb640 1 -- 172.21.15.69:0/2005525400 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188014070 con 0x7fe19c100b40 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe199ffb640 1 -- 172.21.15.69:0/2005525400 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe188010af0 con 0x7fe19c100b40 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c19d680 con 0x7fe19c100b40 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe199ffb640 1 -- 172.21.15.69:0/2005525400 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe188017d10 con 0x7fe19c100b40 2024-08-02T17:46:12.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.954+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1a4f60 con 0x7fe19c100b40 2024-08-02T17:46:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.955+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c1005f0 con 0x7fe19c100b40 2024-08-02T17:46:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.956+0000 7fe199ffb640 1 -- 172.21.15.69:0/2005525400 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe188017070 con 0x7fe19c100b40 2024-08-02T17:46:12.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.957+0000 7fe199ffb640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe18c0a2c10 0x7fe18c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.957+0000 7fe1a0fda640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe18c0a2c10 0x7fe18c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.957+0000 7fe199ffb640 1 -- 172.21.15.69:0/2005525400 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe1880ce8f0 con 0x7fe19c100b40 2024-08-02T17:46:12.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.957+0000 7fe1a0fda640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe18c0a2c10 0x7fe18c0a50d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe190000c00 tx=0x7fe190030e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:12.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:12.961+0000 7fe199ffb640 1 -- 172.21.15.69:0/2005525400 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe18809b450 con 0x7fe19c100b40 2024-08-02T17:46:13.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.127+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe19c1a5160 con 0x7fe19c100b40 2024-08-02T17:46:13.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.129+0000 7fe199ffb640 1 -- 172.21.15.69:0/2005525400 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24631 (secure 0 0 0) 0x7fe18809ad50 con 0x7fe19c100b40 2024-08-02T17:46:13.131 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:13.131 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":94,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:45:57.847197+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:46:13.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe18c0a2c10 msgr2=0x7fe18c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe18c0a2c10 0x7fe18c0a50d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe190000c00 tx=0x7fe190030e50 comp rx=0 tx=0).stop 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100b40 msgr2=0x7fe19c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100b40 0x7fe19c198920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe18800b4a0 tx=0x7fe18800b960 comp rx=0 tx=0).stop 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 shutdown_connections 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe18c0a2c10 0x7fe18c0a50d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe19c100b40 0x7fe19c198920 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe19c1052c0 0x7fe19c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 --2- 172.21.15.69:0/2005525400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe19c100190 0x7fe19c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 >> 172.21.15.69:0/2005525400 conn(0x7fe19c0fbc10 msgr2=0x7fe19c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 shutdown_connections 2024-08-02T17:46:13.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:13.131+0000 7fe1a323e640 1 -- 172.21.15.69:0/2005525400 wait complete. 2024-08-02T17:46:13.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-02T17:44:04.896710+0000', 'modified': '2024-08-02T17:44:49.496243+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24431, 'mds_2': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24403': {'gid': 24403, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/3991686236', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3991686236}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3991686236}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/1744999893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1744999893}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1744999893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6835/3410285190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 3410285190}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 3410285190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.83:6839/2418869426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 2418869426}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 2418869426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.77:6837/3778799888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 3778799888}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 3778799888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.77:6839/1183977027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 1183977027}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 1183977027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24427, 24431, 24452]} 2024-08-02T17:46:13.902 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-02T17:46:13.902 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.i status 2024-08-02T17:46:14.156 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:13 smithi083 ceph-mon[37031]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:13 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2005525400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:14.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:13 smithi069 ceph-mon[32746]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:14.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:13 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2005525400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:14.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:13 smithi077 ceph-mon[37108]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-02T17:46:14.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:13 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2005525400' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:14.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 -- 172.21.15.69:0/1937428641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100fc0 msgr2=0x7f1ee410d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/1937428641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100fc0 0x7f1ee410d4d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1edc009cd0 tx=0x7f1edc02a8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 -- 172.21.15.69:0/1937428641 shutdown_connections 2024-08-02T17:46:14.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/1937428641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100fc0 0x7f1ee410d4d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/1937428641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ee4100600 0x7f1ee4100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/1937428641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee4106690 0x7f1ee4106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 -- 172.21.15.69:0/1937428641 >> 172.21.15.69:0/1937428641 conn(0x7f1ee40fc340 msgr2=0x7f1ee40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:14.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 -- 172.21.15.69:0/1937428641 shutdown_connections 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 -- 172.21.15.69:0/1937428641 wait complete. 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.774+0000 7f1ee3fff640 1 Processor -- start 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee3fff640 1 -- start start 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100600 0x7f1ee4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee4100fc0 0x7f1ee4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ee4106690 0x7f1ee41a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee3fff640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4110a10 con 0x7f1ee4100600 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee3fff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4110890 con 0x7f1ee4100fc0 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee3fff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee4110b90 con 0x7f1ee4106690 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100600 0x7f1ee4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100600 0x7f1ee4198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33312/0 (socket says 172.21.15.69:33312) 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee2ffd640 1 -- 172.21.15.69:0/2627345550 learned_addr learned my addr 172.21.15.69:0/2627345550 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:14.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ee4106690 0x7f1ee41a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee4100fc0 0x7f1ee4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee2ffd640 1 -- 172.21.15.69:0/2627345550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ee4106690 msgr2=0x7f1ee41a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ee4106690 0x7f1ee41a3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee2ffd640 1 -- 172.21.15.69:0/2627345550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee4100fc0 msgr2=0x7f1ee4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee4100fc0 0x7f1ee4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee2ffd640 1 -- 172.21.15.69:0/2627345550 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee41a42a0 con 0x7f1ee4100600 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ee4106690 0x7f1ee41a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.775+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee4100fc0 0x7f1ee4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.776+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100600 0x7f1ee4198ce0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c920 tx=0x7f1ed400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.776+0000 7f1edbfff640 1 -- 172.21.15.69:0/2627345550 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4007c70 con 0x7f1ee4100600 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.776+0000 7f1edbfff640 1 -- 172.21.15.69:0/2627345550 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed4007e10 con 0x7f1ee4100600 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.776+0000 7f1ee3fff640 1 -- 172.21.15.69:0/2627345550 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee41a4470 con 0x7f1ee4100600 2024-08-02T17:46:14.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.776+0000 7f1edbfff640 1 -- 172.21.15.69:0/2627345550 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed4018890 con 0x7f1ee4100600 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.776+0000 7f1ee3fff640 1 -- 172.21.15.69:0/2627345550 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee41a4980 con 0x7f1ee4100600 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.778+0000 7f1edbfff640 1 -- 172.21.15.69:0/2627345550 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1ed4018a70 con 0x7f1ee4100600 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.778+0000 7f1eeb800740 1 Processor -- start 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.778+0000 7f1eeb800740 1 -- start start 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735579b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735584450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c735586d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1eeb800740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c7354de710 con 0x55c7355797b0 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1eeb800740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c735476770 con 0x55c73557a0d0 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1eeb800740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c7354da6d0 con 0x55c735584990 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735579b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735579b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33318/0 (socket says 172.21.15.69:33318) 2024-08-02T17:46:14.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 -- 172.21.15.69:0/2565785855 learned_addr learned my addr 172.21.15.69:0/2565785855 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735584450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1edbfff640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ed00a2c10 0x7f1ed00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c735586d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ed00a2c10 0x7f1ed00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 -- 172.21.15.69:0/2565785855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 msgr2=0x55c735586d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c735586d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 -- 172.21.15.69:0/2565785855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 msgr2=0x55c735584450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735584450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 -- 172.21.15.69:0/2565785855 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73547d3e0 con 0x55c7355797b0 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1edbfff640 1 -- 172.21.15.69:0/2627345550 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1ed40cf780 con 0x7f1ee4100600 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.779+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735579b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1edc031b50 tx=0x7f1edc031b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f4187d791aa437 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1ed97fa640 1 -- 172.21.15.69:0/2565785855 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc004240 con 0x55c7355797b0 2024-08-02T17:46:14.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1ed97fa640 1 -- 172.21.15.69:0/2565785855 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1edc036040 con 0x55c7355797b0 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1ed97fa640 1 -- 172.21.15.69:0/2565785855 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc030cc0 con 0x55c7355797b0 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 -- 172.21.15.69:0/2565785855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 msgr2=0x55c735579b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735579b90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1edc031b50 tx=0x7f1edc031b80 comp rx=0 tx=0).stop 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ed00a2c10 0x7f1ed00a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1edc04d000 tx=0x7f1edc04b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 -- 172.21.15.69:0/2565785855 shutdown_connections 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c735586d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735584450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 --2- 172.21.15.69:0/2565785855 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735579b90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 -- 172.21.15.69:0/2565785855 >> 172.21.15.69:0/2565785855 conn(0x55c735575d40 msgr2=0x55c735576120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 -- 172.21.15.69:0/2565785855 shutdown_connections 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.780+0000 7f1eeb800740 1 -- 172.21.15.69:0/2565785855 wait complete. 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 1 -- start start 2024-08-02T17:46:14.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 1 Processor -- start 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7355797b0 0x55c735581870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735581db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355822f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c735587560 con 0x55c73557a0d0 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c735499970 con 0x55c7355797b0 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c7355876e0 con 0x55c735584990 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1eeb800740 10 client.0 _ll_register_callbacks cb 0x55c7351ed7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735581db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735581db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33326/0 (socket says 172.21.15.69:33326) 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee27fc640 1 -- 172.21.15.69:0/824457294 learned_addr learned my addr 172.21.15.69:0/824457294 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7355797b0 0x55c735581870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355822f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee27fc640 1 -- 172.21.15.69:0/824457294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 msgr2=0x55c7355822f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355822f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee27fc640 1 -- 172.21.15.69:0/824457294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7355797b0 msgr2=0x55c735581870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7355797b0 0x55c735581870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.781+0000 7f1ee27fc640 1 -- 172.21.15.69:0/824457294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7352502b0 con 0x55c73557a0d0 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7355797b0 0x55c735581870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735581db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00f4a0 tx=0x7f1ecc00f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab49326cf7f91765 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:14.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eca7fc640 1 -- 172.21.15.69:0/824457294 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc013850 con 0x55c73557a0d0 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eca7fc640 1 -- 172.21.15.69:0/824457294 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1ecc0139f0 con 0x55c73557a0d0 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eeb800740 1 -- 172.21.15.69:0/824457294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1edc036040 con 0x55c73557a0d0 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eca7fc640 1 -- 172.21.15.69:0/824457294 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ecc024050 con 0x55c73557a0d0 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eeb800740 10 client.15126 fetch_fsmap learned FSMap version 25 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eeb800740 1 -- 172.21.15.69:0/824457294 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7355931c0 con 0x55c73557a0d0 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eca7fc640 1 -- 172.21.15.69:0/824457294 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1edc036040 con 0x55c73557a0d0 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eca7fc640 1 -- 172.21.15.69:0/824457294 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ecc0113e0 con 0x55c73557a0d0 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eeb800740 10 client.15126 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eeb800740 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eeb800740 20 client.15126 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.782+0000 7f1eeb800740 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c735595830 0x55c735596ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.783+0000 7f1eeb800740 4 client.15126 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.783+0000 7f1eeb800740 1 -- 172.21.15.69:0/824457294 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7354ebc80 con 0x55c735595830 2024-08-02T17:46:14.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.783+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c735595830 0x55c735596ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.783+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c735595830 0x55c735596ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76562467c9897fb1 server_cookie=190f761327e4777f in_seq=0 out_seq=0 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.783+0000 7f1eca7fc640 10 client.15126 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eca7fc640 1 -- 172.21.15.69:0/824457294 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c7354ebc80 con 0x55c735595830 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eca7fc640 10 client.15126 handle_command_reply: tid=0 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 client.15126 shutdown 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 -- 172.21.15.69:0/824457294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 msgr2=0x55c735581db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735581db0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00f4a0 tx=0x7f1ecc00f960 comp rx=0 tx=0).stop 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 -- 172.21.15.69:0/824457294 shutdown_connections 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c735595830 0x55c735596ca0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735581db0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355822f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 --2- 172.21.15.69:0/824457294 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c7355797b0 0x55c735581870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eca7fc640 0 client.15126 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:46:14.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 -- 172.21.15.69:0/824457294 >> 172.21.15.69:0/824457294 conn(0x55c735575d40 msgr2=0x55c73557ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:14.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 -- 172.21.15.69:0/824457294 shutdown_connections 2024-08-02T17:46:14.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 1 -- 172.21.15.69:0/824457294 wait complete. 2024-08-02T17:46:14.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.784+0000 7f1eeb800740 20 client.15126 trim_cache size 0 max 16384 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1eeb800740 1 Processor -- start 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1eeb800740 1 -- start start 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7355797b0 0x55c735595830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735582140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c735584990 0x55c73557ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1eeb800740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c735594fe0 con 0x55c73557a0d0 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1eeb800740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c7355956b0 con 0x55c735584990 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1eeb800740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c735595d80 con 0x55c7355797b0 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735582140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1ee27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735582140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33330/0 (socket says 172.21.15.69:33330) 2024-08-02T17:46:14.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1ee27fc640 1 -- 172.21.15.69:0/957876592 learned_addr learned my addr 172.21.15.69:0/957876592 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.803+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7355797b0 0x55c735595830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c735584990 0x55c73557ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ee27fc640 1 -- 172.21.15.69:0/957876592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7355797b0 msgr2=0x55c735595830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7355797b0 0x55c735595830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ee27fc640 1 -- 172.21.15.69:0/957876592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c735584990 msgr2=0x55c73557ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c735584990 0x55c73557ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ee27fc640 1 -- 172.21.15.69:0/957876592 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7355755f0 con 0x55c73557a0d0 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7355797b0 0x55c735595830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735582140 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc008010 tx=0x7f1ecc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564ad180bad6f167 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ed8ff9640 1 -- 172.21.15.69:0/957876592 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc011df0 con 0x55c73557a0d0 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ed8ff9640 1 -- 172.21.15.69:0/957876592 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ecc0113e0 con 0x55c73557a0d0 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.804+0000 7f1ed8ff9640 1 -- 172.21.15.69:0/957876592 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc006440 con 0x55c73557a0d0 2024-08-02T17:46:14.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 -- 172.21.15.69:0/957876592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 msgr2=0x55c735582140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735582140 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc008010 tx=0x7f1ecc00a620 comp rx=0 tx=0).stop 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 -- 172.21.15.69:0/957876592 shutdown_connections 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c73557a0d0 0x55c735582140 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c735584990 0x55c73557ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 --2- 172.21.15.69:0/957876592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c7355797b0 0x55c735595830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 -- 172.21.15.69:0/957876592 >> 172.21.15.69:0/957876592 conn(0x55c735575d40 msgr2=0x55c735576120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 -- 172.21.15.69:0/957876592 shutdown_connections 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 -- 172.21.15.69:0/957876592 wait complete. 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 -- start start 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.805+0000 7f1eeb800740 1 Processor -- start 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735598440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735598980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1eeb800740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355fdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1eeb800740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c735596b20 con 0x55c7355797b0 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1eeb800740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c735596450 con 0x55c73557a0d0 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1eeb800740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c7355971f0 con 0x55c735584990 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735598440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1eeb800740 10 client.0 _ll_register_callbacks cb 0x7f1ecc00a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735598440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33332/0 (socket says 172.21.15.69:33332) 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 -- 172.21.15.69:0/3028015829 learned_addr learned my addr 172.21.15.69:0/3028015829 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735598980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355fdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 -- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 msgr2=0x55c7355fdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355fdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 -- 172.21.15.69:0/3028015829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 msgr2=0x55c735598980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735598980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 -- 172.21.15.69:0/3028015829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7355fe4f0 con 0x55c7355797b0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735598980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee37fe640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355fdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1ee2ffd640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735598440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4007590 tx=0x7f1ed409b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b752719431aa27 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.806+0000 7f1eca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eca7fc640 1 -- 172.21.15.69:0/3028015829 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed409fe80 con 0x55c7355797b0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eca7fc640 1 -- 172.21.15.69:0/3028015829 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f1ed4018d60 con 0x55c7355797b0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 1 -- 172.21.15.69:0/3028015829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ecc0113e0 con 0x55c7355797b0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eca7fc640 1 -- 172.21.15.69:0/3028015829 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ed40ccfd0 con 0x55c7355797b0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 10 client.15132 fetch_fsmap learned FSMap version 25 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 1 -- 172.21.15.69:0/3028015829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7355ff830 con 0x55c7355797b0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eca7fc640 1 -- 172.21.15.69:0/3028015829 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ecc0113e0 con 0x55c7355797b0 2024-08-02T17:46:14.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eca7fc640 1 -- 172.21.15.69:0/3028015829 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ed40a4340 con 0x55c7355797b0 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 10 client.15132 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 20 client.15132 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c735604360 0x55c735603070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 4 client.15132 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1eeb800740 1 -- 172.21.15.69:0/3028015829 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "status"}) -- 0x55c7354e6190 con 0x55c735604360 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.807+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c735604360 0x55c735603070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1ee27fc640 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c735604360 0x55c735603070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50088ca98274be30 server_cookie=5cac157258b93041 in_seq=0 out_seq=0 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eca7fc640 10 client.15132 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eca7fc640 1 -- 172.21.15.69:0/3028015829 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c7354e6190 con 0x55c735604360 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eca7fc640 10 client.15132 handle_command_reply: tid=0 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 client.15132 shutdown 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 -- 172.21.15.69:0/3028015829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 msgr2=0x55c735598440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735598440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4007590 tx=0x7f1ed409b9d0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 -- 172.21.15.69:0/3028015829 shutdown_connections 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c735604360 0x55c735603070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c735584990 0x55c7355fdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c73557a0d0 0x55c735598980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 --2- 172.21.15.69:0/3028015829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c7355797b0 0x55c735598440 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eca7fc640 0 client.15132 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 -- 172.21.15.69:0/3028015829 >> 172.21.15.69:0/3028015829 conn(0x55c735575d40 msgr2=0x55c7355745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.808+0000 7f1eeb800740 1 -- 172.21.15.69:0/3028015829 shutdown_connections 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1eeb800740 1 -- 172.21.15.69:0/3028015829 wait complete. 2024-08-02T17:46:14.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1eeb800740 20 client.15132 trim_cache size 0 max 16384 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24427, 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 87.906850738000003, 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 94, 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 90, 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 149.30831415099999 2024-08-02T17:46:14.821 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:46:14.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 -- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ed00a2c10 msgr2=0x7f1ed00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ed00a2c10 0x7f1ed00a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1edc04d000 tx=0x7f1edc04b000 comp rx=0 tx=0).stop 2024-08-02T17:46:14.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 -- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100600 msgr2=0x7f1ee4198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:14.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100600 0x7f1ee4198ce0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400c920 tx=0x7f1ed400cde0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 -- 172.21.15.69:0/2627345550 shutdown_connections 2024-08-02T17:46:14.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f1ed00a2c10 0x7f1ed00a50d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ee4106690 0x7f1ee41a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ee4100fc0 0x7f1ee4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 --2- 172.21.15.69:0/2627345550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ee4100600 0x7f1ee4198ce0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:14.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.809+0000 7f1ee3fff640 1 -- 172.21.15.69:0/2627345550 >> 172.21.15.69:0/2627345550 conn(0x7f1ee40fc340 msgr2=0x7f1ee40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:14.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.810+0000 7f1ee3fff640 1 -- 172.21.15.69:0/2627345550 shutdown_connections 2024-08-02T17:46:14.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:14.810+0000 7f1ee3fff640 1 -- 172.21.15.69:0/2627345550 wait complete. 2024-08-02T17:46:15.590 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.h status 2024-08-02T17:46:15.843 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:16.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:15 smithi069 ceph-mon[32746]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:46:16.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:15 smithi083 ceph-mon[37031]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:46:16.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 -- 172.21.15.69:0/1218040400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 msgr2=0x7f8bfc1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1218040400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 0x7f8bfc1026e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009900 tx=0x7f8be402a480 comp rx=0 tx=0).stop 2024-08-02T17:46:16.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 -- 172.21.15.69:0/1218040400 shutdown_connections 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1218040400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 0x7f8bfc1026e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1218040400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc102c20 0x7f8bfc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1218040400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bfc100b50 0x7f8bfc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 -- 172.21.15.69:0/1218040400 >> 172.21.15.69:0/1218040400 conn(0x7f8bfc0fc500 msgr2=0x7f8bfc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 -- 172.21.15.69:0/1218040400 shutdown_connections 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 -- 172.21.15.69:0/1218040400 wait complete. 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 Processor -- start 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.287+0000 7f8c030d1640 1 -- start start 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c030d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc100b50 0x7f8bfc1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c030d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 0x7f8bfc198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c030d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bfc102c20 0x7f8bfc19cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c030d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc110cb0 con 0x7f8bfc1022a0 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c030d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc110b30 con 0x7f8bfc102c20 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c030d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc110e30 con 0x7f8bfc100b50 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 0x7f8bfc198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 0x7f8bfc198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33354/0 (socket says 172.21.15.69:33354) 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 -- 172.21.15.69:0/1335200504 learned_addr learned my addr 172.21.15.69:0/1335200504 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc100b50 0x7f8bfc1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bfc102c20 0x7f8bfc19cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 -- 172.21.15.69:0/1335200504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc100b50 msgr2=0x7f8bfc1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc100b50 0x7f8bfc1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 -- 172.21.15.69:0/1335200504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bfc102c20 msgr2=0x7f8bfc19cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bfc102c20 0x7f8bfc19cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 -- 172.21.15.69:0/1335200504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc19d190 con 0x7f8bfc1022a0 2024-08-02T17:46:16.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bfc102c20 0x7f8bfc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 0x7f8bfc198720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8be4006010 tx=0x7f8be4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bf9ffb640 1 -- 172.21.15.69:0/1335200504 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be403f070 con 0x7f8bfc1022a0 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8c030d1640 1 -- 172.21.15.69:0/1335200504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc19d420 con 0x7f8bfc1022a0 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bf9ffb640 1 -- 172.21.15.69:0/1335200504 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be4031c30 con 0x7f8bfc1022a0 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.288+0000 7f8bf9ffb640 1 -- 172.21.15.69:0/1335200504 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4035030 con 0x7f8bfc1022a0 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.289+0000 7f8c030d1640 1 -- 172.21.15.69:0/1335200504 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc1a4d60 con 0x7f8bfc1022a0 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.291+0000 7f8c054cf740 1 Processor -- start 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c054cf740 1 -- start start 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8bf9ffb640 1 -- 172.21.15.69:0/1335200504 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8be4042070 con 0x7f8bfc1022a0 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68630 0x562aaba68a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68f50 0x562aaba732d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba73810 0x562aaba75c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c054cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562aab9cd710 con 0x562aaba68630 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c054cf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562aab965770 con 0x562aaba73810 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c054cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562aab9c96d0 con 0x562aaba68f50 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68630 0x562aaba68a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68630 0x562aaba68a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33366/0 (socket says 172.21.15.69:33366) 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 -- 172.21.15.69:0/190734190 learned_addr learned my addr 172.21.15.69:0/190734190 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68f50 0x562aaba732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba73810 0x562aaba75c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 -- 172.21.15.69:0/190734190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68f50 msgr2=0x562aaba732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68f50 0x562aaba732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 -- 172.21.15.69:0/190734190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba73810 msgr2=0x562aaba75c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba73810 0x562aaba75c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 -- 172.21.15.69:0/190734190 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aab96c3e0 con 0x562aaba68630 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68f50 0x562aaba732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.292+0000 7f8c0166e640 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68630 0x562aaba68a10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009ea0 tx=0x7f8bf4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cccfd0738c17ae server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8bf9ffb640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f8be80a2c10 0x7f8be80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8bf2ffd640 1 -- 172.21.15.69:0/190734190 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4004d20 con 0x562aaba68630 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8bf2ffd640 1 -- 172.21.15.69:0/190734190 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4005500 con 0x562aaba68630 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8bf2ffd640 1 -- 172.21.15.69:0/190734190 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4011870 con 0x562aaba68630 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f8be80a2c10 0x7f8be80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8bf9ffb640 1 -- 172.21.15.69:0/1335200504 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8be40efea0 con 0x7f8bfc1022a0 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 -- 172.21.15.69:0/190734190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68630 msgr2=0x562aaba68a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f8be80a2c10 0x7f8be80a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x562aaba65a20 tx=0x7f8bec00b710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68630 0x562aaba68a10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009ea0 tx=0x7f8bf4009f90 comp rx=0 tx=0).stop 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 -- 172.21.15.69:0/190734190 shutdown_connections 2024-08-02T17:46:16.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba73810 0x562aaba75c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68f50 0x562aaba732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 --2- 172.21.15.69:0/190734190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68630 0x562aaba68a10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 -- 172.21.15.69:0/190734190 >> 172.21.15.69:0/190734190 conn(0x562aaba64bc0 msgr2=0x562aaba64fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 -- 172.21.15.69:0/190734190 shutdown_connections 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.293+0000 7f8c054cf740 1 -- 172.21.15.69:0/190734190 wait complete. 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 1 -- start start 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 1 Processor -- start 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba706f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 0x562aaba70c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba71170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562aaba763e0 con 0x562aaba73810 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562aab988970 con 0x562aaba68f50 2024-08-02T17:46:16.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562aaba76560 con 0x562aaba68630 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c054cf740 10 client.0 _ll_register_callbacks cb 0x562aab6dc7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c0166e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba71170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c0166e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba71170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33376/0 (socket says 172.21.15.69:33376) 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c0166e640 1 -- 172.21.15.69:0/1191696125 learned_addr learned my addr 172.21.15.69:0/1191696125 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.294+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 0x562aaba70c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c0166e640 1 -- 172.21.15.69:0/1191696125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 msgr2=0x562aaba706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c0166e640 1 -- 172.21.15.69:0/1191696125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 msgr2=0x562aaba70c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 0x562aaba70c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c0166e640 1 -- 172.21.15.69:0/1191696125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aab73f2b0 con 0x562aaba73810 2024-08-02T17:46:16.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba71170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4006010 tx=0x7f8bf40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448effcb7e3c603f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:16.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8bdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:16.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1191696125 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4029060 con 0x562aaba73810 2024-08-02T17:46:16.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1191696125 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8bf4004c50 con 0x562aaba73810 2024-08-02T17:46:16.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c054cf740 1 -- 172.21.15.69:0/1191696125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bf4005500 con 0x562aaba73810 2024-08-02T17:46:16.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1191696125 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bf4030050 con 0x562aaba73810 2024-08-02T17:46:16.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c054cf740 10 client.15144 fetch_fsmap learned FSMap version 25 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8c054cf740 1 -- 172.21.15.69:0/1191696125 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562aaba81fe0 con 0x562aaba73810 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.295+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1191696125 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4005500 con 0x562aaba73810 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1191696125 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8bf4019430 con 0x562aaba73810 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8c054cf740 10 client.15144 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8c054cf740 10 client.15144 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8c054cf740 20 client.15144 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562aaba84260 0x562aaba856d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8c054cf740 4 client.15144 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8c054cf740 1 -- 172.21.15.69:0/1191696125 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562aab9dac80 con 0x562aaba84260 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562aaba84260 0x562aaba856d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8bdbfff640 10 client.15144 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:46:16.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.296+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562aaba84260 0x562aaba856d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4fecdc2e4f24c986 server_cookie=5001e3b8181453da in_seq=0 out_seq=0 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1191696125 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562aab9dac80 con 0x562aaba84260 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8bdbfff640 10 client.15144 handle_command_reply: tid=0 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 client.15144 shutdown 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 -- 172.21.15.69:0/1191696125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 msgr2=0x562aaba71170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba71170 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4006010 tx=0x7f8bf40058e0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 -- 172.21.15.69:0/1191696125 shutdown_connections 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562aaba84260 0x562aaba856d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba71170 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 0x562aaba70c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1191696125 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8bdbfff640 0 client.15144 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:46:16.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 -- 172.21.15.69:0/1191696125 >> 172.21.15.69:0/1191696125 conn(0x562aaba64bc0 msgr2=0x562aaba63d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:16.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 -- 172.21.15.69:0/1191696125 shutdown_connections 2024-08-02T17:46:16.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 1 -- 172.21.15.69:0/1191696125 wait complete. 2024-08-02T17:46:16.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.297+0000 7f8c054cf740 20 client.15144 trim_cache size 0 max 16384 2024-08-02T17:46:16.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c054cf740 1 Processor -- start 2024-08-02T17:46:16.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c054cf740 1 -- start start 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68630 0x562aaba70fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68f50 0x562aaba72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba73810 0x562aaba8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c054cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562aaba83a10 con 0x562aaba68f50 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c054cf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562aaba840e0 con 0x562aaba68630 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c054cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562aaba847b0 con 0x562aaba73810 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68f50 0x562aaba72550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68f50 0x562aaba72550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33388/0 (socket says 172.21.15.69:33388) 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8bfbfff640 1 -- 172.21.15.69:0/1575377403 learned_addr learned my addr 172.21.15.69:0/1575377403 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba73810 0x562aaba8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68630 0x562aaba70fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8bfbfff640 1 -- 172.21.15.69:0/1575377403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba73810 msgr2=0x562aaba8dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.315+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba73810 0x562aaba8dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8bfbfff640 1 -- 172.21.15.69:0/1575377403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68630 msgr2=0x562aaba70fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68630 0x562aaba70fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8bfbfff640 1 -- 172.21.15.69:0/1575377403 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aaba60130 con 0x562aaba68f50 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba73810 0x562aaba8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68f50 0x562aaba72550 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8be40b8650 tx=0x7f8be403b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244baecd385416c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8bf27fc640 1 -- 172.21.15.69:0/1575377403 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be4049070 con 0x562aaba68f50 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8bf27fc640 1 -- 172.21.15.69:0/1575377403 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be4031de0 con 0x562aaba68f50 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8bf27fc640 1 -- 172.21.15.69:0/1575377403 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8be40b8860 con 0x562aaba68f50 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8c054cf740 1 -- 172.21.15.69:0/1575377403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68f50 msgr2=0x562aaba72550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68f50 0x562aaba72550 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8be40b8650 tx=0x7f8be403b870 comp rx=0 tx=0).stop 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8c054cf740 1 -- 172.21.15.69:0/1575377403 shutdown_connections 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.316+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba68f50 0x562aaba72550 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba73810 0x562aaba8dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1575377403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68630 0x562aaba70fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 -- 172.21.15.69:0/1575377403 >> 172.21.15.69:0/1575377403 conn(0x562aaba64bc0 msgr2=0x562aaba71e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 -- 172.21.15.69:0/1575377403 shutdown_connections 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 -- 172.21.15.69:0/1575377403 wait complete. 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 -- start start 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 Processor -- start 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 0x562aaba86b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba87060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562aaba85550 con 0x562aaba73810 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562aaba84e80 con 0x562aaba68f50 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562aaba85c20 con 0x562aaba68630 2024-08-02T17:46:16.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c054cf740 10 client.0 _ll_register_callbacks cb 0x7f8bf4009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c0166e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba87060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.317+0000 7f8c0166e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba87060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33402/0 (socket says 172.21.15.69:33402) 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c0166e640 1 -- 172.21.15.69:0/1744472076 learned_addr learned my addr 172.21.15.69:0/1744472076 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8bfbfff640 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 0x562aaba86b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c0166e640 1 -- 172.21.15.69:0/1744472076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 msgr2=0x562aaba865e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba865e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c0166e640 1 -- 172.21.15.69:0/1744472076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 msgr2=0x562aaba86b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 0x562aaba86b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c0166e640 1 -- 172.21.15.69:0/1744472076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aabaf1b40 con 0x562aaba73810 2024-08-02T17:46:16.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba865e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c0166e640 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba87060 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4005cc0 tx=0x7f8bf4009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d955ff95d4aba35 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8bdbfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1744472076 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4029040 con 0x562aaba73810 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1744472076 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f8bf4019430 con 0x562aaba73810 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c054cf740 1 -- 172.21.15.69:0/1744472076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8be4031de0 con 0x562aaba73810 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1744472076 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8bf4019c30 con 0x562aaba73810 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c054cf740 10 client.15150 fetch_fsmap learned FSMap version 25 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8c054cf740 1 -- 172.21.15.69:0/1744472076 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562aabaf2dc0 con 0x562aaba73810 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.318+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1744472076 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8be4031de0 con 0x562aaba73810 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1744472076 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8bf40114d0 con 0x562aaba73810 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8c054cf740 10 client.15150 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:46:16.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8c054cf740 10 client.15150 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8c054cf740 20 client.15150 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562aabaf78d0 0x562aabaf65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8c054cf740 4 client.15150 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8c054cf740 1 -- 172.21.15.69:0/1744472076 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "status"}) -- 0x562aab9d5190 con 0x562aabaf78d0 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562aabaf78d0 0x562aabaf65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8bdbfff640 10 client.15150 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.319+0000 7f8c00e6d640 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562aabaf78d0 0x562aabaf65e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36c710d033264942 server_cookie=1d053a32904b4b2a in_seq=0 out_seq=0 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8bdbfff640 1 -- 172.21.15.69:0/1744472076 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562aab9d5190 con 0x562aabaf78d0 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8bdbfff640 10 client.15150 handle_command_reply: tid=0 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 client.15150 shutdown 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 -- 172.21.15.69:0/1744472076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 msgr2=0x562aaba87060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba87060 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4005cc0 tx=0x7f8bf4009850 comp rx=0 tx=0).stop 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 -- 172.21.15.69:0/1744472076 shutdown_connections 2024-08-02T17:46:16.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562aabaf78d0 0x562aabaf65e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8bdbfff640 0 client.15150 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562aaba73810 0x562aaba87060 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562aaba68f50 0x562aaba86b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 --2- 172.21.15.69:0/1744472076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562aaba68630 0x562aaba865e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 -- 172.21.15.69:0/1744472076 >> 172.21.15.69:0/1744472076 conn(0x562aaba64bc0 msgr2=0x562aaba7efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 -- 172.21.15.69:0/1744472076 shutdown_connections 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 1 -- 172.21.15.69:0/1744472076 wait complete. 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.320+0000 7f8c054cf740 20 client.15150 trim_cache size 0 max 16384 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:46:16.332 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24431, 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 90.431765325000001, 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 94, 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 90, 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 153.40134460499999 2024-08-02T17:46:16.333 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:46:16.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 -- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f8be80a2c10 msgr2=0x7f8be80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f8be80a2c10 0x7f8be80a50d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x562aaba65a20 tx=0x7f8bec00b710 comp rx=0 tx=0).stop 2024-08-02T17:46:16.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 -- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 msgr2=0x7f8bfc198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:16.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 0x7f8bfc198720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8be4006010 tx=0x7f8be4003c10 comp rx=0 tx=0).stop 2024-08-02T17:46:16.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 -- 172.21.15.69:0/1335200504 shutdown_connections 2024-08-02T17:46:16.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f8be80a2c10 0x7f8be80a50d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bfc1022a0 0x7f8bfc198720 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bfc102c20 0x7f8bfc19cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 --2- 172.21.15.69:0/1335200504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bfc100b50 0x7f8bfc1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:16.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 -- 172.21.15.69:0/1335200504 >> 172.21.15.69:0/1335200504 conn(0x7f8bfc0fc500 msgr2=0x7f8bfc069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:16.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 -- 172.21.15.69:0/1335200504 shutdown_connections 2024-08-02T17:46:16.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:16.321+0000 7f8c030d1640 1 -- 172.21.15.69:0/1335200504 wait complete. 2024-08-02T17:46:16.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:15 smithi077 ceph-mon[37108]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-02T17:46:17.016 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.b status 2024-08-02T17:46:17.288 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 -- 172.21.15.69:0/860635480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c81067b0 msgr2=0x7f13c8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 --2- 172.21.15.69:0/860635480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c81067b0 0x7f13c8106b90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009930 tx=0x7f13bc02a420 comp rx=0 tx=0).stop 2024-08-02T17:46:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 -- 172.21.15.69:0/860635480 shutdown_connections 2024-08-02T17:46:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 --2- 172.21.15.69:0/860635480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c8101150 0x7f13c810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 --2- 172.21.15.69:0/860635480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13c81007b0 0x7f13c8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 --2- 172.21.15.69:0/860635480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c81067b0 0x7f13c8106b90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 -- 172.21.15.69:0/860635480 >> 172.21.15.69:0/860635480 conn(0x7f13c80fc4d0 msgr2=0x7f13c80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:17.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 -- 172.21.15.69:0/860635480 shutdown_connections 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.762+0000 7f13cef0a640 1 -- 172.21.15.69:0/860635480 wait complete. 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cef0a640 1 Processor -- start 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cef0a640 1 -- start start 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cef0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c81007b0 0x7f13c8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cef0a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8101150 0x7f13c81986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cef0a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13c81067b0 0x7f13c819ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cef0a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8111d10 con 0x7f13c8101150 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cef0a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8111b90 con 0x7f13c81067b0 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cef0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13c8111e90 con 0x7f13c81007b0 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8101150 0x7f13c81986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8101150 0x7f13c81986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33424/0 (socket says 172.21.15.69:33424) 2024-08-02T17:46:17.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13c7fff640 1 -- 172.21.15.69:0/2866907619 learned_addr learned my addr 172.21.15.69:0/2866907619 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13ccca6640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c81007b0 0x7f13c8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13cd4a7640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13c81067b0 0x7f13c819ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13c7fff640 1 -- 172.21.15.69:0/2866907619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c81007b0 msgr2=0x7f13c8198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c81007b0 0x7f13c8198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13c7fff640 1 -- 172.21.15.69:0/2866907619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13c81067b0 msgr2=0x7f13c819ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13c81067b0 0x7f13c819ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.763+0000 7f13c7fff640 1 -- 172.21.15.69:0/2866907619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c819d140 con 0x7f13c8101150 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.764+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8101150 0x7f13c81986d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f13b400d950 tx=0x7f13b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.764+0000 7f13c5ffb640 1 -- 172.21.15.69:0/2866907619 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4017070 con 0x7f13c8101150 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.764+0000 7f13c5ffb640 1 -- 172.21.15.69:0/2866907619 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b4009c30 con 0x7f13c8101150 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.764+0000 7f13cef0a640 1 -- 172.21.15.69:0/2866907619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c819d430 con 0x7f13c8101150 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.764+0000 7f13c5ffb640 1 -- 172.21.15.69:0/2866907619 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b400f6b0 con 0x7f13c8101150 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.764+0000 7f13cef0a640 1 -- 172.21.15.69:0/2866907619 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81a4d10 con 0x7f13c8101150 2024-08-02T17:46:17.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.766+0000 7f13c5ffb640 1 -- 172.21.15.69:0/2866907619 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f13b40109d0 con 0x7f13c8101150 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.766+0000 7f13d1308740 1 Processor -- start 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.766+0000 7f13d1308740 1 -- start start 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13c5ffb640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13b80a2c10 0x7f13b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13d1308740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13d1308740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 0x55db78c572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13d1308740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c59c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13d1308740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55db78bb1710 con 0x55db78c4c630 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13d1308740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55db78b49770 con 0x55db78c57810 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13d1308740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db78bad6d0 con 0x55db78c4cf50 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13c5ffb640 1 -- 172.21.15.69:0/2866907619 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f13b40cf270 con 0x7f13c8101150 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13cd4a7640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13b80a2c10 0x7f13b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c4ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33434/0 (socket says 172.21.15.69:33434) 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 -- 172.21.15.69:0/2228614757 learned_addr learned my addr 172.21.15.69:0/2228614757 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 -- 172.21.15.69:0/2228614757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 msgr2=0x55db78c572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 --2- 172.21.15.69:0/2228614757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 0x55db78c572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 -- 172.21.15.69:0/2228614757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 msgr2=0x55db78c59c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 --2- 172.21.15.69:0/2228614757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c59c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 -- 172.21.15.69:0/2228614757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db78b503e0 con 0x55db78c4c630 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13ccca6640 1 --2- 172.21.15.69:0/2228614757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c4ca10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009930 tx=0x7f13bc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d769043281847b8b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13cd4a7640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13b80a2c10 0x7f13b80a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13c00092f0 tx=0x7f13c0010740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13b2ffd640 1 -- 172.21.15.69:0/2228614757 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc035040 con 0x55db78c4c630 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13b2ffd640 1 -- 172.21.15.69:0/2228614757 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13bc005520 con 0x55db78c4c630 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.767+0000 7f13b2ffd640 1 -- 172.21.15.69:0/2228614757 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0035f0 con 0x55db78c4c630 2024-08-02T17:46:17.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 -- 172.21.15.69:0/2228614757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 msgr2=0x55db78c4ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 --2- 172.21.15.69:0/2228614757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c4ca10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009930 tx=0x7f13bc005c00 comp rx=0 tx=0).stop 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 -- 172.21.15.69:0/2228614757 shutdown_connections 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 --2- 172.21.15.69:0/2228614757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c59c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 --2- 172.21.15.69:0/2228614757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 0x55db78c572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 --2- 172.21.15.69:0/2228614757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c4ca10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 -- 172.21.15.69:0/2228614757 >> 172.21.15.69:0/2228614757 conn(0x55db78c48bc0 msgr2=0x55db78c48fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 -- 172.21.15.69:0/2228614757 shutdown_connections 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.768+0000 7f13d1308740 1 -- 172.21.15.69:0/2228614757 wait complete. 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 1 -- start start 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 1 Processor -- start 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c4c630 0x55db78c54770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c54cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c57810 0x55db78c551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55db78c5a3e0 con 0x55db78c4cf50 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55db78b6c970 con 0x55db78c4c630 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db78c5a560 con 0x55db78c57810 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13d1308740 10 client.0 _ll_register_callbacks cb 0x55db788c07b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c54cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c54cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33446/0 (socket says 172.21.15.69:33446) 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.769+0000 7f13c7fff640 1 -- 172.21.15.69:0/3464990976 learned_addr learned my addr 172.21.15.69:0/3464990976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13cd4a7640 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c57810 0x55db78c551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13ccca6640 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c4c630 0x55db78c54770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13c7fff640 1 -- 172.21.15.69:0/3464990976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c57810 msgr2=0x55db78c551f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13c7fff640 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c57810 0x55db78c551f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13c7fff640 1 -- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c4c630 msgr2=0x55db78c54770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13c7fff640 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c4c630 0x55db78c54770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13c7fff640 1 -- 172.21.15.69:0/3464990976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db789232b0 con 0x55db78c4cf50 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13ccca6640 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c4c630 0x55db78c54770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13c7fff640 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c54cb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f13b40a3ca0 tx=0x7f13b4009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b93c0e254721ae server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13a7fff640 1 -- 172.21.15.69:0/3464990976 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4020040 con 0x55db78c4cf50 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13a7fff640 1 -- 172.21.15.69:0/3464990976 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f13b409a9b0 con 0x55db78c4cf50 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 1 -- 172.21.15.69:0/3464990976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13bc005520 con 0x55db78c4cf50 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13a7fff640 1 -- 172.21.15.69:0/3464990976 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13b409b560 con 0x55db78c4cf50 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 10 client.15162 fetch_fsmap learned FSMap version 25 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 1 -- 172.21.15.69:0/3464990976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db78c660b0 con 0x55db78c4cf50 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13a7fff640 1 -- 172.21.15.69:0/3464990976 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13bc005520 con 0x55db78c4cf50 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13a7fff640 1 -- 172.21.15.69:0/3464990976 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f13b40973d0 con 0x55db78c4cf50 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 10 client.15162 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:46:17.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 10 client.15162 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 20 client.15162 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55db78c686b0 0x55db78c69b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 4 client.15162 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.770+0000 7f13d1308740 1 -- 172.21.15.69:0/3464990976 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db78bbec80 con 0x55db78c686b0 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.771+0000 7f13ccca6640 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55db78c686b0 0x55db78c69b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.771+0000 7f13ccca6640 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55db78c686b0 0x55db78c69b20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4a3b35f3c53316 server_cookie=8d3921de8edb0238 in_seq=0 out_seq=0 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.771+0000 7f13a7fff640 10 client.15162 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13a7fff640 1 -- 172.21.15.69:0/3464990976 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db78bbec80 con 0x55db78c686b0 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13a7fff640 10 client.15162 handle_command_reply: tid=0 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 client.15162 shutdown 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 -- 172.21.15.69:0/3464990976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 msgr2=0x55db78c54cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c54cb0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f13b40a3ca0 tx=0x7f13b4009de0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 -- 172.21.15.69:0/3464990976 shutdown_connections 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c54cb0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55db78c686b0 0x55db78c69b20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c57810 0x55db78c551f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 --2- 172.21.15.69:0/3464990976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c4c630 0x55db78c54770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13a7fff640 0 client.15162 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 -- 172.21.15.69:0/3464990976 >> 172.21.15.69:0/3464990976 conn(0x55db78c48bc0 msgr2=0x55db78c47d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 -- 172.21.15.69:0/3464990976 shutdown_connections 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 1 -- 172.21.15.69:0/3464990976 wait complete. 2024-08-02T17:46:17.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.772+0000 7f13d1308740 20 client.15162 trim_cache size 0 max 16384 2024-08-02T17:46:17.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.790+0000 7f13d1308740 1 Processor -- start 2024-08-02T17:46:17.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.790+0000 7f13d1308740 1 -- start start 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.790+0000 7f13d1308740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4c630 0x55db78c4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.790+0000 7f13d1308740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c55040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.790+0000 7f13d1308740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c519d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.790+0000 7f13d1308740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55db78c67e60 con 0x55db78c4cf50 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.790+0000 7f13d1308740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55db78c68530 con 0x55db78c57810 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.790+0000 7f13d1308740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db78c68c00 con 0x55db78c4c630 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c55040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c55040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33450/0 (socket says 172.21.15.69:33450) 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 -- 172.21.15.69:0/2721526263 learned_addr learned my addr 172.21.15.69:0/2721526263 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13ccca6640 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4c630 0x55db78c4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13cd4a7640 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 -- 172.21.15.69:0/2721526263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4c630 msgr2=0x55db78c4f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4c630 0x55db78c4f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 -- 172.21.15.69:0/2721526263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 msgr2=0x55db78c519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 -- 172.21.15.69:0/2721526263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db78c44130 con 0x55db78c4cf50 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13cd4a7640 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c55040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009f90 tx=0x7f13b400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506fe877844fb226 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13b27fc640 1 -- 172.21.15.69:0/2721526263 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4020070 con 0x55db78c4cf50 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13b27fc640 1 -- 172.21.15.69:0/2721526263 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b409f910 con 0x55db78c4cf50 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.791+0000 7f13b27fc640 1 -- 172.21.15.69:0/2721526263 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4097510 con 0x55db78c4cf50 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 -- 172.21.15.69:0/2721526263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 msgr2=0x55db78c55040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c55040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009f90 tx=0x7f13b400bf60 comp rx=0 tx=0).stop 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 -- 172.21.15.69:0/2721526263 shutdown_connections 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4cf50 0x55db78c55040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 --2- 172.21.15.69:0/2721526263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4c630 0x55db78c4f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 -- 172.21.15.69:0/2721526263 >> 172.21.15.69:0/2721526263 conn(0x55db78c48bc0 msgr2=0x55db78c4fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 -- 172.21.15.69:0/2721526263 shutdown_connections 2024-08-02T17:46:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 -- 172.21.15.69:0/2721526263 wait complete. 2024-08-02T17:46:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 -- start start 2024-08-02T17:46:17.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.792+0000 7f13d1308740 1 Processor -- start 2024-08-02T17:46:17.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13d1308740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13d1308740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 0x55db78c6ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13d1308740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c6c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13d1308740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55db78c699a0 con 0x55db78c4c630 2024-08-02T17:46:17.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13d1308740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55db78c692d0 con 0x55db78c57810 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13d1308740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55db78c6a070 con 0x55db78c4cf50 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13d1308740 10 client.0 _ll_register_callbacks cb 0x7f13b400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33460/0 (socket says 172.21.15.69:33460) 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 -- 172.21.15.69:0/2329746990 learned_addr learned my addr 172.21.15.69:0/2329746990 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 0x55db78c6ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13cd4a7640 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c6c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 -- 172.21.15.69:0/2329746990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 msgr2=0x55db78c6ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 0x55db78c6ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 -- 172.21.15.69:0/2329746990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 msgr2=0x55db78c6c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c6c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 -- 172.21.15.69:0/2329746990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db78c701e0 con 0x55db78c4c630 2024-08-02T17:46:17.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13ccca6640 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c726c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0095d0 tx=0x7f13bc004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2055e36ecc35ede8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.793+0000 7f13a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13a7fff640 1 -- 172.21.15.69:0/2329746990 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc035040 con 0x55db78c4c630 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13a7fff640 1 -- 172.21.15.69:0/2329746990 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f13bc004f10 con 0x55db78c4c630 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 1 -- 172.21.15.69:0/2329746990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b409f910 con 0x55db78c4c630 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13a7fff640 1 -- 172.21.15.69:0/2329746990 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13bc005110 con 0x55db78c4c630 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 10 client.15168 fetch_fsmap learned FSMap version 25 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 1 -- 172.21.15.69:0/2329746990 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55db78cd2fd0 con 0x55db78c4c630 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13a7fff640 1 -- 172.21.15.69:0/2329746990 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13b409f910 con 0x55db78c4c630 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13a7fff640 1 -- 172.21.15.69:0/2329746990 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f13bc0035f0 con 0x55db78c4c630 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 10 client.15168 fetch_fsmap finished waiting for FSMap version 25 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 10 client.15168 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 20 client.15168 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:17.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55db78cd76f0 0x55db78cd6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 4 client.15168 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13d1308740 1 -- 172.21.15.69:0/2329746990 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "status"}) -- 0x55db78bb9190 con 0x55db78cd76f0 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.794+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55db78cd76f0 0x55db78cd6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13a7fff640 10 client.15168 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13c7fff640 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55db78cd76f0 0x55db78cd6400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aaf8bf58b5d727d server_cookie=fb751057173753dd in_seq=0 out_seq=0 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13a7fff640 1 -- 172.21.15.69:0/2329746990 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55db78bb9190 con 0x55db78cd76f0 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13a7fff640 10 client.15168 handle_command_reply: tid=0 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 client.15168 shutdown 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 -- 172.21.15.69:0/2329746990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 msgr2=0x55db78c726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c726c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f13bc0095d0 tx=0x7f13bc004c00 comp rx=0 tx=0).stop 2024-08-02T17:46:17.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 -- 172.21.15.69:0/2329746990 shutdown_connections 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55db78cd76f0 0x55db78cd6400 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55db78c57810 0x55db78c6c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13a7fff640 0 client.15168 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55db78c4cf50 0x55db78c6ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 --2- 172.21.15.69:0/2329746990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55db78c4c630 0x55db78c726c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 -- 172.21.15.69:0/2329746990 >> 172.21.15.69:0/2329746990 conn(0x55db78c48bc0 msgr2=0x55db78c627b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 -- 172.21.15.69:0/2329746990 shutdown_connections 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.795+0000 7f13d1308740 1 -- 172.21.15.69:0/2329746990 wait complete. 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13d1308740 20 client.15168 trim_cache size 0 max 16384 2024-08-02T17:46:17.803 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24452, 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 132.66528463, 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 25, 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 94, 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 90, 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 143.47984939 2024-08-02T17:46:17.804 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:46:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 -- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13b80a2c10 msgr2=0x7f13b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13b80a2c10 0x7f13b80a50d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13c00092f0 tx=0x7f13c0010740 comp rx=0 tx=0).stop 2024-08-02T17:46:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 -- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8101150 msgr2=0x7f13c81986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:17.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8101150 0x7f13c81986d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f13b400d950 tx=0x7f13b400de10 comp rx=0 tx=0).stop 2024-08-02T17:46:17.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 -- 172.21.15.69:0/2866907619 shutdown_connections 2024-08-02T17:46:17.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f13b80a2c10 0x7f13b80a50d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13c8101150 0x7f13c81986d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13c81067b0 0x7f13c819ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 --2- 172.21.15.69:0/2866907619 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13c81007b0 0x7f13c8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:17.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 -- 172.21.15.69:0/2866907619 >> 172.21.15.69:0/2866907619 conn(0x7f13c80fc4d0 msgr2=0x7f13c80fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:17.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 -- 172.21.15.69:0/2866907619 shutdown_connections 2024-08-02T17:46:17.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:17.796+0000 7f13cef0a640 1 -- 172.21.15.69:0/2866907619 wait complete. 2024-08-02T17:46:18.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:17 smithi069 ceph-mon[32746]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:46:18.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:17 smithi083 ceph-mon[37031]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:46:18.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:17 smithi077 ceph-mon[37108]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-02T17:46:18.667 INFO:teuthology.run_tasks:Running task sequential... 2024-08-02T17:46:18.681 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-02T17:46:18.681 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-02T17:46:18.682 INFO:teuthology.task.print:Enabling mgr modules 2024-08-02T17:46:18.682 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-08-02T17:46:18.683 INFO:teuthology.task.exec:Executing custom commands... 2024-08-02T17:46:18.683 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:46:18.683 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-08-02T17:46:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- 172.21.15.69:0/3937905210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8100780 msgr2=0x7f32f8100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3937905210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8100780 0x7f32f8100b80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f32e8009900 tx=0x7f32e802a410 comp rx=0 tx=0).stop 2024-08-02T17:46:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- 172.21.15.69:0/3937905210 shutdown_connections 2024-08-02T17:46:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3937905210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32f8102330 0x7f32f8108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3937905210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32f8101970 0x7f32f8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:18.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3937905210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8100780 0x7f32f8100b80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- 172.21.15.69:0/3937905210 >> 172.21.15.69:0/3937905210 conn(0x7f32f80fbee0 msgr2=0x7f32f80fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- 172.21.15.69:0/3937905210 shutdown_connections 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- 172.21.15.69:0/3937905210 wait complete. 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 Processor -- start 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- start start 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32f8100780 0x7f32f8197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32f8101970 0x7f32f8198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8102330 0x7f32f819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810d550 con 0x7f32f8102330 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810d3d0 con 0x7f32f8100780 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.787+0000 7f32ff13d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f32f810d6d0 con 0x7f32f8101970 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8102330 0x7f32f819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:18.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8102330 0x7f32f819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33486/0 (socket says 172.21.15.69:33486) 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 -- 172.21.15.69:0/3525030316 learned_addr learned my addr 172.21.15.69:0/3525030316 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32f7fff640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32f8101970 0x7f32f8198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fced9640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32f8100780 0x7f32f8197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 -- 172.21.15.69:0/3525030316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32f8101970 msgr2=0x7f32f8198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32f8101970 0x7f32f8198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 -- 172.21.15.69:0/3525030316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32f8100780 msgr2=0x7f32f8197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32f8100780 0x7f32f8197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 -- 172.21.15.69:0/3525030316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f819f860 con 0x7f32f8102330 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32fd6da640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8102330 0x7f32f819f1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f32f000eb10 tx=0x7f32f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32f5ffb640 1 -- 172.21.15.69:0/3525030316 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0004b20 con 0x7f32f8102330 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32f5ffb640 1 -- 172.21.15.69:0/3525030316 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f0004cc0 con 0x7f32f8102330 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f819fb50 con 0x7f32f8102330 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32f5ffb640 1 -- 172.21.15.69:0/3525030316 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0010760 con 0x7f32f8102330 2024-08-02T17:46:18.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.788+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f81a00a0 con 0x7f32f8102330 2024-08-02T17:46:18.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.789+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f8107c10 con 0x7f32f8102330 2024-08-02T17:46:18.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.790+0000 7f32f5ffb640 1 -- 172.21.15.69:0/3525030316 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f32f0005020 con 0x7f32f8102330 2024-08-02T17:46:18.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.791+0000 7f32f5ffb640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32e00a2c10 0x7f32e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:18.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.791+0000 7f32fced9640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32e00a2c10 0x7f32e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:18.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.791+0000 7f32f5ffb640 1 -- 172.21.15.69:0/3525030316 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7f32f00cfcb0 con 0x7f32f8102330 2024-08-02T17:46:18.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.791+0000 7f32fced9640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32e00a2c10 0x7f32e00a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32f8199120 tx=0x7f32e80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:18.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:18.795+0000 7f32f5ffb640 1 -- 172.21.15.69:0/3525030316 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f32f009c700 con 0x7f32f8102330 2024-08-02T17:46:19.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.023+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f32f802d790 con 0x7f32f8102330 2024-08-02T17:46:19.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.899+0000 7f32f5ffb640 1 -- 172.21.15.69:0/3525030316 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) ==== 92+0+0 (secure 0 0 0) 0x7f32f009c000 con 0x7f32f8102330 2024-08-02T17:46:19.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32e00a2c10 msgr2=0x7f32e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:19.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32e00a2c10 0x7f32e00a50d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f32f8199120 tx=0x7f32e80315b0 comp rx=0 tx=0).stop 2024-08-02T17:46:19.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8102330 msgr2=0x7f32f819f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:19.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8102330 0x7f32f819f1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f32f000eb10 tx=0x7f32f000c2d0 comp rx=0 tx=0).stop 2024-08-02T17:46:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 shutdown_connections 2024-08-02T17:46:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7f32e00a2c10 0x7f32e00a50d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f32f8102330 0x7f32f819f1c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f32f8101970 0x7f32f8198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 --2- 172.21.15.69:0/3525030316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f32f8100780 0x7f32f8197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 >> 172.21.15.69:0/3525030316 conn(0x7f32f80fbee0 msgr2=0x7f32f8104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 shutdown_connections 2024-08-02T17:46:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:19.902+0000 7f32ff13d640 1 -- 172.21.15.69:0/3525030316 wait complete. 2024-08-02T17:46:19.921 INFO:teuthology.run_tasks:Running task kclient... 2024-08-02T17:46:19.936 INFO:tasks.kclient:Mounting kernel clients... 2024-08-02T17:46:19.936 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-02T17:46:19.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:19.936 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-08-02T17:46:19.953 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-08-02T17:46:19.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:19.954 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-02T17:46:20.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:20.020 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-08-02T17:46:20.125 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-08-02T17:46:20.125 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-02T17:46:20.126 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-08-02T17:46:20.126 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-08-02T17:46:20.126 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs ls 2024-08-02T17:46:20.204 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:19 smithi083 ceph-mon[37031]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-02T17:46:20.204 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:19 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3525030316' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-02T17:46:20.205 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:19 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: ignoring --setuser ceph since I am not root 2024-08-02T17:46:20.205 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:19 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: ignoring --setgroup ceph since I am not root 2024-08-02T17:46:20.205 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:19 smithi083 ceph-mgr[38848]: -- 172.21.15.83:0/3355213681 <== mon.1 v2:172.21.15.83:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590ff1d10e0 con 0x5590ff1af400 2024-08-02T17:46:20.205 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:20.053+0000 7f47a08ec040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-02T17:46:20.205 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:20.203+0000 7f47a08ec040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:46:20.207 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:19 smithi077 ceph-mon[37108]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-02T17:46:20.207 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:19 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3525030316' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-02T17:46:20.207 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: ignoring --setuser ceph since I am not root 2024-08-02T17:46:20.207 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: ignoring --setgroup ceph since I am not root 2024-08-02T17:46:20.207 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:19 smithi077 ceph-mgr[38485]: -- 172.21.15.77:0/4194244344 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560cc747d0e0 con 0x560cc745a800 2024-08-02T17:46:20.207 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:20.056+0000 7f6c436fe040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-02T17:46:20.208 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:20.205+0000 7f6c436fe040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:46:20.227 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:19 smithi069 ceph-mon[32746]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-08-02T17:46:20.227 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:19 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3525030316' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-08-02T17:46:20.227 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:19 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ignoring --setuser ceph since I am not root 2024-08-02T17:46:20.227 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:19 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ignoring --setgroup ceph since I am not root 2024-08-02T17:46:20.227 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:20.067+0000 7fbb71e31040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-08-02T17:46:20.228 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:20.225+0000 7fbb71e31040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:46:20.393 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:20.508 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:20.269+0000 7f47a08ec040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-02T17:46:20.513 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:20.272+0000 7f6c436fe040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-02T17:46:20.513 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:20.511+0000 7f6c436fe040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-02T17:46:20.518 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:20.295+0000 7fbb71e31040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-08-02T17:46:20.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:20.506+0000 7f47a08ec040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-02T17:46:20.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:20.566+0000 7f47a08ec040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-02T17:46:20.775 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:20.558+0000 7fbb71e31040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-08-02T17:46:20.775 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:20.619+0000 7fbb71e31040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-02T17:46:20.847 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:20.571+0000 7f6c436fe040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 -- 172.21.15.69:0/2446166258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c107fd0 msgr2=0x7fe64c10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2446166258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c107fd0 0x7fe64c10e970 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe644009cd0 tx=0x7fe64402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 -- 172.21.15.69:0/2446166258 shutdown_connections 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2446166258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c107fd0 0x7fe64c10e970 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2446166258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64c103410 0x7fe64c1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2446166258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c102a60 0x7fe64c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 -- 172.21.15.69:0/2446166258 >> 172.21.15.69:0/2446166258 conn(0x7fe64c0fc500 msgr2=0x7fe64c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 -- 172.21.15.69:0/2446166258 shutdown_connections 2024-08-02T17:46:20.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 -- 172.21.15.69:0/2446166258 wait complete. 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.856+0000 7fe6538bc640 1 Processor -- start 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe6538bc640 1 -- start start 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe6538bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c102a60 0x7fe64c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe6538bc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64c103410 0x7fe64c1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe6538bc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c107fd0 0x7fe64c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe6538bc640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c111dd0 con 0x7fe64c102a60 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe6538bc640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c111c50 con 0x7fe64c107fd0 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe6538bc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c111f50 con 0x7fe64c103410 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651658640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c102a60 0x7fe64c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651e59640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c107fd0 0x7fe64c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651658640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c102a60 0x7fe64c198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33530/0 (socket says 172.21.15.69:33530) 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651658640 1 -- 172.21.15.69:0/2640269994 learned_addr learned my addr 172.21.15.69:0/2640269994 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:20.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe650e57640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64c103410 0x7fe64c1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651e59640 1 -- 172.21.15.69:0/2640269994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64c103410 msgr2=0x7fe64c1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651e59640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64c103410 0x7fe64c1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651e59640 1 -- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c102a60 msgr2=0x7fe64c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651e59640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c102a60 0x7fe64c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651e59640 1 -- 172.21.15.69:0/2640269994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c19d240 con 0x7fe64c107fd0 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.857+0000 7fe651658640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c102a60 0x7fe64c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.858+0000 7fe651e59640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c107fd0 0x7fe64c19cb60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe64402ae00 tx=0x7fe644031df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.858+0000 7fe63e7fc640 1 -- 172.21.15.69:0/2640269994 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644003d80 con 0x7fe64c107fd0 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.858+0000 7fe63e7fc640 1 -- 172.21.15.69:0/2640269994 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe644004520 con 0x7fe64c107fd0 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.858+0000 7fe63e7fc640 1 -- 172.21.15.69:0/2640269994 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe644030d00 con 0x7fe64c107fd0 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.858+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c19d4d0 con 0x7fe64c107fd0 2024-08-02T17:46:20.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.858+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c1a4e10 con 0x7fe64c107fd0 2024-08-02T17:46:20.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.859+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c102e40 con 0x7fe64c107fd0 2024-08-02T17:46:20.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.860+0000 7fe63e7fc640 1 -- 172.21.15.69:0/2640269994 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 18) ==== 137901+0+0 (secure 0 0 0) 0x7fe6440046d0 con 0x7fe64c107fd0 2024-08-02T17:46:20.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.860+0000 7fe63e7fc640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6380a2ce0 0x7fe6380a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:20.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.860+0000 7fe651658640 1 -- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6380a2ce0 msgr2=0x7fe6380a51a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/452236630 2024-08-02T17:46:20.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.861+0000 7fe63e7fc640 1 -- 172.21.15.69:0/2640269994 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9179+0+0 (secure 0 0 0) 0x7fe64402e030 con 0x7fe64c107fd0 2024-08-02T17:46:20.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.861+0000 7fe651658640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6380a2ce0 0x7fe6380a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-08-02T17:46:20.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:20.864+0000 7fe63e7fc640 1 -- 172.21.15.69:0/2640269994 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe64404b060 con 0x7fe64c107fd0 2024-08-02T17:46:21.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.062+0000 7fe651658640 1 -- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6380a2ce0 msgr2=0x7fe6380a51a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.69:6800/452236630 2024-08-02T17:46:21.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.062+0000 7fe651658640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6380a2ce0 0x7fe6380a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-08-02T17:46:21.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:20 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3525030316' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-02T17:46:21.064 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:20 smithi069 ceph-mon[32746]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-02T17:46:21.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.077+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe64c1995f0 con 0x7fe64c107fd0 2024-08-02T17:46:21.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.077+0000 7fe63e7fc640 1 -- 172.21.15.69:0/2640269994 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7fe64c1995f0 con 0x7fe64c107fd0 2024-08-02T17:46:21.078 INFO:teuthology.orchestra.run.smithi069.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-02T17:46:21.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6380a2ce0 msgr2=0x7fe6380a51a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:46:21.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6380a2ce0 0x7fe6380a51a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:21.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c107fd0 msgr2=0x7fe64c19cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:21.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c107fd0 0x7fe64c19cb60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe64402ae00 tx=0x7fe644031df0 comp rx=0 tx=0).stop 2024-08-02T17:46:21.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 shutdown_connections 2024-08-02T17:46:21.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:6800/452236630,v1:172.21.15.69:6801/452236630] conn(0x7fe6380a2ce0 0x7fe6380a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:21.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe64c107fd0 0x7fe64c19cb60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:21.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe64c103410 0x7fe64c1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:21.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 --2- 172.21.15.69:0/2640269994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe64c102a60 0x7fe64c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:21.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 >> 172.21.15.69:0/2640269994 conn(0x7fe64c0fc500 msgr2=0x7fe64c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:21.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 shutdown_connections 2024-08-02T17:46:21.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:21.079+0000 7fe6538bc640 1 -- 172.21.15.69:0/2640269994 wait complete. 2024-08-02T17:46:21.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:20 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3525030316' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-02T17:46:21.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:20 smithi083 ceph-mon[37031]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-02T17:46:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:20 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3525030316' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-02T17:46:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:20 smithi077 ceph-mon[37108]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-02T17:46:21.769 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:21 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:21.595+0000 7fbb71e31040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-02T17:46:21.769 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:21 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:21.710+0000 7fbb71e31040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-02T17:46:21.769 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:21 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:21.767+0000 7fbb71e31040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-02T17:46:21.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:21.493+0000 7f47a08ec040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-02T17:46:21.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:21.608+0000 7f47a08ec040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-02T17:46:21.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:21.665+0000 7f47a08ec040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-02T17:46:21.848 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:21.502+0000 7f6c436fe040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-08-02T17:46:21.848 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:21.616+0000 7f6c436fe040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-08-02T17:46:21.849 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:21.674+0000 7f6c436fe040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi069.front.sepia.ceph.com 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-02T17:46:21.927 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-08-02T17:46:21.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:21.928 DEBUG:teuthology.orchestra.run.smithi069:> ip addr 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout: inet 127.0.0.1/8 scope host lo 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 ::1/128 scope host 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c2 brd ff:ff:ff:ff:ff:ff 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f0 2024-08-02T17:46:21.944 INFO:teuthology.orchestra.run.smithi069.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:d8:d9:c3 brd ff:ff:ff:ff:ff:ff 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout: altname enp7s0f1 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:36 brd ff:ff:ff:ff:ff:ff 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout: link/ether 0c:c4:7a:8f:d7:37 brd ff:ff:ff:ff:ff:ff 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout: inet 172.21.15.69/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft 42201sec preferred_lft 42201sec 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout: inet6 fe80::ec4:7aff:fe8f:d737/64 scope link 2024-08-02T17:46:21.945 INFO:teuthology.orchestra.run.smithi069.stdout: valid_lft forever preferred_lft forever 2024-08-02T17:46:21.945 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-02T17:46:21.946 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:46:21.946 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:46:21.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add name ceph-brx type bridge 2024-08-02T17:46:21.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr flush dev ceph-brx 2024-08-02T17:46:21.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set ceph-brx up 2024-08-02T17:46:21.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-02T17:46:21.946 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:46:22.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:46:22.042 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:21 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2640269994' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-02T17:46:22.043 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:22 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:22.001+0000 7fbb71e31040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-02T17:46:22.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:22.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:22.090 DEBUG:teuthology.orchestra.run.smithi069:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-02T17:46:22.115 INFO:teuthology.orchestra.run.smithi069.stdout:1 2024-08-02T17:46:22.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:22.116 DEBUG:teuthology.orchestra.run.smithi069:> route 2024-08-02T17:46:22.156 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:21 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2640269994' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-02T17:46:22.156 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:21.879+0000 7f47a08ec040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-02T17:46:22.157 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:21.941+0000 7f47a08ec040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-02T17:46:22.157 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:22.008+0000 7f47a08ec040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-02T17:46:22.157 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:22.155+0000 7f47a08ec040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:46:22.160 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:21 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2640269994' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-08-02T17:46:22.161 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:21.885+0000 7f6c436fe040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-08-02T17:46:22.161 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:21.947+0000 7f6c436fe040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-02T17:46:22.161 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:22.013+0000 7f6c436fe040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-02T17:46:22.161 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:22.159+0000 7f6c436fe040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:46:22.173 INFO:teuthology.orchestra.run.smithi069.stdout:Kernel IP routing table 2024-08-02T17:46:22.173 INFO:teuthology.orchestra.run.smithi069.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-02T17:46:22.173 INFO:teuthology.orchestra.run.smithi069.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-02T17:46:22.173 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-02T17:46:22.173 INFO:teuthology.orchestra.run.smithi069.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-02T17:46:22.173 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:46:22.174 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:46:22.174 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-02T17:46:22.174 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-02T17:46:22.174 DEBUG:teuthology.orchestra.run.smithi069:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-02T17:46:22.174 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:46:22.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:46:22.294 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:22 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:22.068+0000 7fbb71e31040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-08-02T17:46:22.294 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:22 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:22.140+0000 7fbb71e31040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-08-02T17:46:22.296 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:22.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:22.303 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-08-02T17:46:22.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:22.318 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list-id 2024-08-02T17:46:22.374 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:46:22.374 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:46:22.374 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns add ceph-ns--tmp-cephfs 2024-08-02T17:46:22.374 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-08-02T17:46:22.374 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:46:22.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:46:22.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:22.496 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-08-02T17:46:22.496 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:46:22.496 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:46:22.497 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-08-02T17:46:22.497 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-02T17:46:22.497 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-08-02T17:46:22.497 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-08-02T17:46:22.497 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-08-02T17:46:22.497 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:46:22.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:46:22.555 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:22 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:22.295+0000 7fbb71e31040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-08-02T17:46:22.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:22.615 DEBUG:teuthology.orchestra.run.smithi069:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-02T17:46:22.615 DEBUG:teuthology.orchestra.run.smithi069:> set -e 2024-08-02T17:46:22.615 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set brx.0 up 2024-08-02T17:46:22.615 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link set dev brx.0 master ceph-brx 2024-08-02T17:46:22.615 DEBUG:teuthology.orchestra.run.smithi069:> ') 2024-08-02T17:46:22.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:46:22.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:22.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-02T17:46:22.695 DEBUG:teuthology.orchestra.run.smithi069:> mkdir -p -v /tmp/cephfs 2024-08-02T17:46:22.723 INFO:teuthology.orchestra.run.smithi069.stdout:mkdir: created directory '/tmp/cephfs' 2024-08-02T17:46:22.723 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-02T17:46:22.724 DEBUG:teuthology.orchestra.run.smithi069:> chmod 0000 /tmp/cephfs 2024-08-02T17:46:22.779 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-02T17:46:22.779 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-02T17:46:22.779 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/39fc5226-68dc-46b2-9a4d-e93437ee40a4 2024-08-02T17:46:22.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:22.780 DEBUG:teuthology.orchestra.run.smithi069:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/39fc5226-68dc-46b2-9a4d-e93437ee40a4 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,context=system_u:object_r:postgresql_db_t:s0 2024-08-02T17:46:22.840 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:22 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:22.747+0000 7fbb71e31040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-02T17:46:22.847 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:22.570+0000 7f6c436fe040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-02T17:46:22.863 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0" 2024-08-02T17:46:22.863 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-08-02T17:46:22.863 INFO:teuthology.orchestra.run.smithi069.stdout:invalid new device string format 2024-08-02T17:46:22.882 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:22.568+0000 7f47a08ec040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-08-02T17:46:23.028 INFO:teuthology.orchestra.run.smithi069.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0" 2024-08-02T17:46:23.028 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-08-02T17:46:23.028 INFO:teuthology.orchestra.run.smithi069.stdout:invalid new device string format 2024-08-02T17:46:23.028 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: resolved to: "172.21.15.69:3300,172.21.15.77:3300,172.21.15.83:3300" 2024-08-02T17:46:23.028 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.69:3300,172.21.15.77:3300,172.21.15.83:3300:/volumes/qa/sv_1/39fc5226-68dc-46b2-9a4d-e93437ee40a4 2024-08-02T17:46:23.028 INFO:teuthology.orchestra.run.smithi069.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=b6b04634-50f5-11ef-bcca-c7b262605968" will pass to kernel 2024-08-02T17:46:23.030 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-02T17:46:23.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T17:46:23.030 DEBUG:teuthology.orchestra.run.smithi069:> sudo chmod 1777 /tmp/cephfs 2024-08-02T17:46:23.092 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:23 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:23.078+0000 7fbb71e31040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-02T17:46:23.271 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:22.881+0000 7f47a08ec040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-02T17:46:23.271 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:22.949+0000 7f47a08ec040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-02T17:46:23.271 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:23.100+0000 7f47a08ec040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-02T17:46:23.347 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:22.884+0000 7f6c436fe040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-08-02T17:46:23.348 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:22.952+0000 7f6c436fe040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-02T17:46:23.348 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:23.103+0000 7f6c436fe040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-02T17:46:23.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:23 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:23.146+0000 7fbb71e31040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-08-02T17:46:23.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:23 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:23.295+0000 7fbb71e31040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-08-02T17:46:23.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:23.410+0000 7f47a08ec040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-02T17:46:23.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:23.470+0000 7f47a08ec040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-02T17:46:23.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:23.652+0000 7f47a08ec040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-02T17:46:23.804 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:23.415+0000 7f6c436fe040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-02T17:46:23.804 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:23.476+0000 7f6c436fe040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-02T17:46:23.804 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:23.658+0000 7f6c436fe040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-02T17:46:23.805 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:23.803+0000 7f6c436fe040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-02T17:46:24.003 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:23 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:23.607+0000 7fbb71e31040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-08-02T17:46:24.003 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:23 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:23.667+0000 7fbb71e31040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-08-02T17:46:24.003 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:23 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:23.850+0000 7fbb71e31040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-08-02T17:46:24.097 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:23.866+0000 7f6c436fe040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-02T17:46:24.097 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:23.949+0000 7f6c436fe040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-02T17:46:24.097 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:24.008+0000 7f6c436fe040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-02T17:46:24.271 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:23.798+0000 7f47a08ec040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-02T17:46:24.271 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:23.861+0000 7f47a08ec040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-02T17:46:24.271 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:23.945+0000 7f47a08ec040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-02T17:46:24.271 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:24.004+0000 7f47a08ec040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-02T17:46:24.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:23 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:23.997+0000 7fbb71e31040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-08-02T17:46:24.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:24.061+0000 7fbb71e31040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-08-02T17:46:24.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:24.143+0000 7fbb71e31040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-08-02T17:46:24.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:24.202+0000 7fbb71e31040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-08-02T17:46:24.847 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:24.538+0000 7f6c436fe040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y[38481]: 2024-08-02T17:46:24.694+0000 7f6c436fe040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: Standby manager daemon z restarted 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: Standby manager daemon z started 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: Standby manager daemon y restarted 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: Standby manager daemon y started 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:46:24.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-02T17:46:24.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-02T17:46:24.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:46:24.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:24 smithi077 ceph-mon[37108]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:46:24.964 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:24.538+0000 7f47a08ec040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z[38844]: 2024-08-02T17:46:24.694+0000 7f47a08ec040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: Standby manager daemon z restarted 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: Standby manager daemon z started 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: Standby manager daemon y restarted 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: Standby manager daemon y started 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:46:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:46:25.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-02T17:46:25.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-02T17:46:25.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:46:25.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:24 smithi083 ceph-mon[37031]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:46:25.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-02T17:46:25.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:25.081 INFO:teuthology.orchestra.run.smithi069.stdout:client24908 2024-08-02T17:46:25.082 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T17:46:25.082 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-02T17:46:25.105 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-02T17:46:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: Standby manager daemon z restarted 2024-08-02T17:46:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: Standby manager daemon z started 2024-08-02T17:46:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: Standby manager daemon y restarted 2024-08-02T17:46:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: Standby manager daemon y started 2024-08-02T17:46:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-08-02T17:46:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-08-02T17:46:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:46:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-08-02T17:46:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-08-02T17:46:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-08-02T17:46:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.83:0/3050169515' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:46:25.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-mon[32746]: from='mgr.? 172.21.15.77:0/1863958265' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-08-02T17:46:25.109 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:24.738+0000 7fbb71e31040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-08-02T17:46:25.110 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:24 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:24.898+0000 7fbb71e31040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-08-02T17:46:25.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-02T17:46:25.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:25.194 INFO:teuthology.orchestra.run.smithi069.stdout:client24908 2024-08-02T17:46:25.194 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-02T17:46:25.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-02T17:46:25.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:25.264 INFO:teuthology.orchestra.run.smithi069.stdout:client24908 2024-08-02T17:46:25.264 DEBUG:teuthology.orchestra.run.smithi069:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b6b04634-50f5-11ef-bcca-c7b262605968.client24908/status') 2024-08-02T17:46:25.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-02T17:46:25.325 INFO:teuthology.orchestra.run.smithi069.stdout:instance: client.24908 (3)172.21.15.69:0/4072981680 2024-08-02T17:46:25.325 INFO:teuthology.orchestra.run.smithi069.stdout:blocklisted: false 2024-08-02T17:46:25.326 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records in 2024-08-02T17:46:25.326 INFO:teuthology.orchestra.run.smithi069.stderr:0+1 records out 2024-08-02T17:46:25.326 INFO:teuthology.orchestra.run.smithi069.stderr:71 bytes copied, 2.1449e-05 s, 3.3 MB/s 2024-08-02T17:46:25.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-02T17:46:25.333 INFO:teuthology.run_tasks:Running task exec... 2024-08-02T17:46:25.343 INFO:teuthology.task.exec:Executing custom commands... 2024-08-02T17:46:25.343 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:46:25.343 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-08-02T17:46:25.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 -- 172.21.15.69:0/652543716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74100770 msgr2=0x7f2d74100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 --2- 172.21.15.69:0/652543716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74100770 0x7f2d74100b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009900 tx=0x7f2d6402a410 comp rx=0 tx=0).stop 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 -- 172.21.15.69:0/652543716 shutdown_connections 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 --2- 172.21.15.69:0/652543716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d74102320 0x7f2d74108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 --2- 172.21.15.69:0/652543716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d74101960 0x7f2d74101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 --2- 172.21.15.69:0/652543716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74100770 0x7f2d74100b70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 -- 172.21.15.69:0/652543716 >> 172.21.15.69:0/652543716 conn(0x7f2d740fbef0 msgr2=0x7f2d740fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 -- 172.21.15.69:0/652543716 shutdown_connections 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 -- 172.21.15.69:0/652543716 wait complete. 2024-08-02T17:46:25.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.447+0000 7f2d7a095640 1 Processor -- start 2024-08-02T17:46:25.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d7a095640 1 -- start start 2024-08-02T17:46:25.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d7a095640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d74100770 0x7f2d7406a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d7a095640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d74101960 0x7f2d7419c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d7a095640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74102320 0x7f2d7419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d7a095640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7410c3d0 con 0x7f2d74102320 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d7a095640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7410c250 con 0x7f2d74100770 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d7a095640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d7410c550 con 0x7f2d74101960 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d73fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74102320 0x7f2d7419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d73fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74102320 0x7f2d7419ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36310/0 (socket says 172.21.15.69:36310) 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d73fff640 1 -- 172.21.15.69:0/945847208 learned_addr learned my addr 172.21.15.69:0/945847208 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d737fe640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d74100770 0x7f2d7406a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d72ffd640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d74101960 0x7f2d7419c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d73fff640 1 -- 172.21.15.69:0/945847208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d74101960 msgr2=0x7f2d7419c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:25.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d73fff640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d74101960 0x7f2d7419c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d73fff640 1 -- 172.21.15.69:0/945847208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d74100770 msgr2=0x7f2d7406a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d73fff640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d74100770 0x7f2d7406a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.448+0000 7f2d73fff640 1 -- 172.21.15.69:0/945847208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7419f890 con 0x7f2d74102320 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.449+0000 7f2d73fff640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74102320 0x7f2d7419ef80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009fd0 tx=0x7f2d6c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.449+0000 7f2d70ff9640 1 -- 172.21.15.69:0/945847208 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c017070 con 0x7f2d74102320 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.449+0000 7f2d70ff9640 1 -- 172.21.15.69:0/945847208 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d6c003e50 con 0x7f2d74102320 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.449+0000 7f2d70ff9640 1 -- 172.21.15.69:0/945847208 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c0106c0 con 0x7f2d74102320 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.449+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7419fb20 con 0x7f2d74102320 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.449+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d741a0020 con 0x7f2d74102320 2024-08-02T17:46:25.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.450+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d74107c50 con 0x7f2d74102320 2024-08-02T17:46:25.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.451+0000 7f2d70ff9640 1 -- 172.21.15.69:0/945847208 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f2d6c003970 con 0x7f2d74102320 2024-08-02T17:46:25.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.452+0000 7f2d70ff9640 1 -- 172.21.15.69:0/945847208 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2d6c0ced20 con 0x7f2d74102320 2024-08-02T17:46:25.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.457+0000 7f2d70ff9640 1 -- 172.21.15.69:0/945847208 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2d6c09bba0 con 0x7f2d74102320 2024-08-02T17:46:25.608 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.188+0000 7fbad98b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.188+0000 7fbad98b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.188+0000 7fbad98b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.188+0000 7fbad98b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.188+0000 7fbad98b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.189+0000 7fbadc8b7640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.189+0000 7fbadc8b7640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.190+0000 7fbadc8b7640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.190+0000 7fbadc8b7640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.609 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:25.190+0000 7fbadc8b7640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:25.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.689+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f2d74062b50 con 0x7f2d74102320 2024-08-02T17:46:25.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.917+0000 7f2d70ff9640 1 -- 172.21.15.69:0/945847208 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f2d6c0971f0 con 0x7f2d74102320 2024-08-02T17:46:25.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.918+0000 7f2d70ff9640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d5c0ab070 0x7f2d5c0ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:25.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.919+0000 7f2d737fe640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d5c0ab070 0x7f2d5c0ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:25.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.922+0000 7f2d70ff9640 1 -- 172.21.15.69:0/945847208 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f2d6c056da0 con 0x7f2d74102320 2024-08-02T17:46:25.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.923+0000 7f2d737fe640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d5c0ab070 0x7f2d5c0ad460 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005ab0 tx=0x7f2d64000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d5c0ab070 msgr2=0x7f2d5c0ad460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d5c0ab070 0x7f2d5c0ad460 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2d64005ab0 tx=0x7f2d64000bc0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74102320 msgr2=0x7f2d7419ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74102320 0x7f2d7419ef80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009fd0 tx=0x7f2d6c00b260 comp rx=0 tx=0).stop 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 shutdown_connections 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d74102320 0x7f2d7419ef80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d5c0ab070 0x7f2d5c0ad460 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d74101960 0x7f2d7419c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 --2- 172.21.15.69:0/945847208 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d74100770 0x7f2d7406a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:25.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 >> 172.21.15.69:0/945847208 conn(0x7f2d740fbef0 msgr2=0x7f2d74104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:25.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 shutdown_connections 2024-08-02T17:46:25.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:25.925+0000 7f2d7a095640 1 -- 172.21.15.69:0/945847208 wait complete. 2024-08-02T17:46:25.938 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-08-02T17:46:26.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-02T17:46:26.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: Active manager daemon x restarted 2024-08-02T17:46:26.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: Activating manager daemon x 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: osdmap e95: 12 total, 12 up, 12 in 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: mgrmap e20: x(active, starting, since 0.0145995s), standbys: z, y 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T17:46:26.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:46:26.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: Manager daemon x is now available 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-02T17:46:26.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:25 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/945847208' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-02T17:46:26.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 -- 172.21.15.69:0/3934108546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8102310 msgr2=0x7f5ea8104700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 --2- 172.21.15.69:0/3934108546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8102310 0x7f5ea8104700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009900 tx=0x7f5e9802a410 comp rx=0 tx=0).stop 2024-08-02T17:46:26.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 -- 172.21.15.69:0/3934108546 shutdown_connections 2024-08-02T17:46:26.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 --2- 172.21.15.69:0/3934108546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ea8107570 0x7f5ea81099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 --2- 172.21.15.69:0/3934108546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea8104c40 0x7f5ea8107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 --2- 172.21.15.69:0/3934108546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8102310 0x7f5ea8104700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 -- 172.21.15.69:0/3934108546 >> 172.21.15.69:0/3934108546 conn(0x7f5ea80fbef0 msgr2=0x7f5ea80fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 -- 172.21.15.69:0/3934108546 shutdown_connections 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.041+0000 7f5eade9e640 1 -- 172.21.15.69:0/3934108546 wait complete. 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.042+0000 7f5eade9e640 1 Processor -- start 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.042+0000 7f5eade9e640 1 -- start start 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.042+0000 7f5eade9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ea8102310 0x7f5ea8197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.042+0000 7f5eade9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea8104c40 0x7f5ea81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.042+0000 7f5eade9e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8107570 0x7f5ea819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.042+0000 7f5eade9e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810d550 con 0x7f5ea8107570 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.042+0000 7f5eade9e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810d3d0 con 0x7f5ea8104c40 2024-08-02T17:46:26.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.042+0000 7f5eade9e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea810d6d0 con 0x7f5ea8102310 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8107570 0x7f5ea819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8107570 0x7f5ea819f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36324/0 (socket says 172.21.15.69:36324) 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 -- 172.21.15.69:0/298296559 learned_addr learned my addr 172.21.15.69:0/298296559 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea6ffd640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea8104c40 0x7f5ea81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea77fe640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ea8102310 0x7f5ea8197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 -- 172.21.15.69:0/298296559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ea8102310 msgr2=0x7f5ea8197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ea8102310 0x7f5ea8197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 -- 172.21.15.69:0/298296559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea8104c40 msgr2=0x7f5ea81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea8104c40 0x7f5ea81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 -- 172.21.15.69:0/298296559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea819f870 con 0x7f5ea8107570 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea7fff640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8107570 0x7f5ea819f1d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000dac0 tx=0x7f5ea000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:26.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea4ff9640 1 -- 172.21.15.69:0/298296559 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea0017070 con 0x7f5ea8107570 2024-08-02T17:46:26.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea819fb60 con 0x7f5ea8107570 2024-08-02T17:46:26.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea4ff9640 1 -- 172.21.15.69:0/298296559 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ea0003e50 con 0x7f5ea8107570 2024-08-02T17:46:26.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5ea4ff9640 1 -- 172.21.15.69:0/298296559 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea00106c0 con 0x7f5ea8107570 2024-08-02T17:46:26.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.043+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea81a0060 con 0x7f5ea8107570 2024-08-02T17:46:26.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.044+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea80fd9e0 con 0x7f5ea8107570 2024-08-02T17:46:26.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.046+0000 7f5ea4ff9640 1 -- 172.21.15.69:0/298296559 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f5ea0003970 con 0x7f5ea8107570 2024-08-02T17:46:26.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.047+0000 7f5ea4ff9640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e900a2970 0x7f5e900a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.047+0000 7f5ea77fe640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e900a2970 0x7f5e900a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.047+0000 7f5ea4ff9640 1 -- 172.21.15.69:0/298296559 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5ea00ced90 con 0x7f5ea8107570 2024-08-02T17:46:26.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.047+0000 7f5ea77fe640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e900a2970 0x7f5e900a4e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8199130 tx=0x7f5e98000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:26.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.050+0000 7f5ea4ff9640 1 -- 172.21.15.69:0/298296559 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5ea009b4b0 con 0x7f5ea8107570 2024-08-02T17:46:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-02T17:46:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: Active manager daemon x restarted 2024-08-02T17:46:26.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: Activating manager daemon x 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: osdmap e95: 12 total, 12 up, 12 in 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: mgrmap e20: x(active, starting, since 0.0145995s), standbys: z, y 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-02T17:46:26.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:46:26.100 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: Manager daemon x is now available 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:26.101 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-02T17:46:26.102 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-02T17:46:26.102 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:25 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/945847208' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-02T17:46:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: Active manager daemon x restarted 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: Activating manager daemon x 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: osdmap e95: 12 total, 12 up, 12 in 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: mgrmap e20: x(active, starting, since 0.0145995s), standbys: z, y 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-08-02T17:46:26.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-08-02T17:46:26.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: Manager daemon x is now available 2024-08-02T17:46:26.111 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:26.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-08-02T17:46:26.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-08-02T17:46:26.112 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:25 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/945847208' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-08-02T17:46:26.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.266+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5ea80624c0 con 0x7f5e900a2970 2024-08-02T17:46:26.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.280+0000 7f5ea4ff9640 1 -- 172.21.15.69:0/298296559 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f5ea80624c0 con 0x7f5e900a2970 2024-08-02T17:46:26.282 INFO:teuthology.orchestra.run.smithi069.stdout:{} 2024-08-02T17:46:26.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.282+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e900a2970 msgr2=0x7f5e900a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.282+0000 7f5eade9e640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e900a2970 0x7f5e900a4e30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8199130 tx=0x7f5e98000bc0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8107570 msgr2=0x7f5ea819f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8107570 0x7f5ea819f1d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000dac0 tx=0x7f5ea000df80 comp rx=0 tx=0).stop 2024-08-02T17:46:26.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 shutdown_connections 2024-08-02T17:46:26.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ea8107570 0x7f5ea819f1d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e900a2970 0x7f5e900a4e30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ea8104c40 0x7f5ea81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 --2- 172.21.15.69:0/298296559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ea8102310 0x7f5ea8197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 >> 172.21.15.69:0/298296559 conn(0x7f5ea80fbef0 msgr2=0x7f5ea8102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:26.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 shutdown_connections 2024-08-02T17:46:26.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.283+0000 7f5eade9e640 1 -- 172.21.15.69:0/298296559 wait complete. 2024-08-02T17:46:26.295 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-02T17:46:26.306 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-02T17:46:26.317 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-02T17:46:26.408 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:26.271+0000 7fbad70ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:26.409 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:26.271+0000 7fbad70ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:26.409 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:26.271+0000 7fbad70ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:26.409 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:26.271+0000 7fbad70ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:26.409 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:26.271+0000 7fbad70ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 -- 172.21.15.69:0/2631118055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c100780 msgr2=0x7f858c100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 --2- 172.21.15.69:0/2631118055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c100780 0x7f858c100b80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f857c009900 tx=0x7f857c02a410 comp rx=0 tx=0).stop 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 -- 172.21.15.69:0/2631118055 shutdown_connections 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 --2- 172.21.15.69:0/2631118055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858c102330 0x7f858c108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 --2- 172.21.15.69:0/2631118055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f858c101970 0x7f858c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 --2- 172.21.15.69:0/2631118055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c100780 0x7f858c100b80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 -- 172.21.15.69:0/2631118055 >> 172.21.15.69:0/2631118055 conn(0x7f858c0fbf20 msgr2=0x7f858c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 -- 172.21.15.69:0/2631118055 shutdown_connections 2024-08-02T17:46:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.419+0000 7f859166d640 1 -- 172.21.15.69:0/2631118055 wait complete. 2024-08-02T17:46:26.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f859166d640 1 Processor -- start 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f859166d640 1 -- start start 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f859166d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858c100780 0x7f858c104710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f859166d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f858c101970 0x7f858c102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f859166d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c102330 0x7f858c103290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f859166d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f858c10c3e0 con 0x7f858c102330 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f859166d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f858c10c260 con 0x7f858c101970 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f859166d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f858c10c560 con 0x7f858c100780 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f858b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c102330 0x7f858c103290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f858b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c102330 0x7f858c103290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36368/0 (socket says 172.21.15.69:36368) 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f858b7fe640 1 -- 172.21.15.69:0/890407269 learned_addr learned my addr 172.21.15.69:0/890407269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:26.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f858a7fc640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f858c101970 0x7f858c102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.420+0000 7f858affd640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858c100780 0x7f858c104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f858b7fe640 1 -- 172.21.15.69:0/890407269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858c100780 msgr2=0x7f858c104710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f858b7fe640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858c100780 0x7f858c104710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f858b7fe640 1 -- 172.21.15.69:0/890407269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f858c101970 msgr2=0x7f858c102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f858b7fe640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f858c101970 0x7f858c102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f858b7fe640 1 -- 172.21.15.69:0/890407269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c103b20 con 0x7f858c102330 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f858affd640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858c100780 0x7f858c104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f858b7fe640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c102330 0x7f858c103290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f858400eb10 tx=0x7f858400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f8573fff640 1 -- 172.21.15.69:0/890407269 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584004b20 con 0x7f858c102330 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858c1a1cd0 con 0x7f858c102330 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f8573fff640 1 -- 172.21.15.69:0/890407269 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8584004cc0 con 0x7f858c102330 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f8573fff640 1 -- 172.21.15.69:0/890407269 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584010760 con 0x7f858c102330 2024-08-02T17:46:26.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.421+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858c1a21d0 con 0x7f858c102330 2024-08-02T17:46:26.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.422+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f858c1a2400 con 0x7f858c102330 2024-08-02T17:46:26.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.424+0000 7f8573fff640 1 -- 172.21.15.69:0/890407269 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f8584010900 con 0x7f858c102330 2024-08-02T17:46:26.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.425+0000 7f8573fff640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85740a2990 0x7f85740a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.425+0000 7f858affd640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85740a2990 0x7f85740a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.425+0000 7f8573fff640 1 -- 172.21.15.69:0/890407269 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f85840cf9c0 con 0x7f858c102330 2024-08-02T17:46:26.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.426+0000 7f858affd640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85740a2990 0x7f85740a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f857c006010 tx=0x7f857c032cf0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:26.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.428+0000 7f8573fff640 1 -- 172.21.15.69:0/890407269 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8584053df0 con 0x7f858c102330 2024-08-02T17:46:26.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.612+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f858c1a2790 con 0x7f858c102330 2024-08-02T17:46:26.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.614+0000 7f8573fff640 1 -- 172.21.15.69:0/890407269 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25069 (secure 0 0 0) 0x7f858409c6f0 con 0x7f858c102330 2024-08-02T17:46:26.615 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:26.616 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":95,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:46:24.900245+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3271476957":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3594391539":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3033851323":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/2680424375":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/452236630":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4036393791":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6800/452236630":"2024-08-03T17:46:24.900194+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:46:26.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85740a2990 msgr2=0x7f85740a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85740a2990 0x7f85740a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f857c006010 tx=0x7f857c032cf0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c102330 msgr2=0x7f858c103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c102330 0x7f858c103290 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f858400eb10 tx=0x7f858400c2d0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 shutdown_connections 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f858c102330 0x7f858c103290 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85740a2990 0x7f85740a4e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f858c101970 0x7f858c102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 --2- 172.21.15.69:0/890407269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f858c100780 0x7f858c104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 >> 172.21.15.69:0/890407269 conn(0x7f858c0fbf20 msgr2=0x7f858c069250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 shutdown_connections 2024-08-02T17:46:26.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.616+0000 7f859166d640 1 -- 172.21.15.69:0/890407269 wait complete. 2024-08-02T17:46:26.629 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-02T17:39:54.984711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-02T17:44:00.981682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-02T17:44:03.102100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-02T17:44:09.380545+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-02T17:46:26.629 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 -- 172.21.15.69:0/378211800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e8100730 msgr2=0x7f27e8100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 --2- 172.21.15.69:0/378211800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e8100730 0x7f27e8100b30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009900 tx=0x7f27d802a410 comp rx=0 tx=0).stop 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 -- 172.21.15.69:0/378211800 shutdown_connections 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 --2- 172.21.15.69:0/378211800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27e81022e0 0x7f27e8108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 --2- 172.21.15.69:0/378211800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27e8101920 0x7f27e8101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 --2- 172.21.15.69:0/378211800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e8100730 0x7f27e8100b30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 -- 172.21.15.69:0/378211800 >> 172.21.15.69:0/378211800 conn(0x7f27e80fbeb0 msgr2=0x7f27e80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 -- 172.21.15.69:0/378211800 shutdown_connections 2024-08-02T17:46:26.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.745+0000 7f27ee828640 1 -- 172.21.15.69:0/378211800 wait complete. 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.746+0000 7f27ee828640 1 Processor -- start 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.746+0000 7f27ee828640 1 -- start start 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.746+0000 7f27ee828640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27e8100730 0x7f27e8196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.746+0000 7f27ee828640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27e8101920 0x7f27e8196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.746+0000 7f27ee828640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e81022e0 0x7f27e819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.746+0000 7f27ee828640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f27e810c3b0 con 0x7f27e81022e0 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.746+0000 7f27ee828640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f27e810c230 con 0x7f27e8101920 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.746+0000 7f27ee828640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f27e810c530 con 0x7f27e8100730 2024-08-02T17:46:26.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e81022e0 0x7f27e819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e81022e0 0x7f27e819dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36390/0 (socket says 172.21.15.69:36390) 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 -- 172.21.15.69:0/518312673 learned_addr learned my addr 172.21.15.69:0/518312673 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27e7fff640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27e8100730 0x7f27e8196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27e77fe640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27e8101920 0x7f27e8196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 -- 172.21.15.69:0/518312673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27e8100730 msgr2=0x7f27e8196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27e8100730 0x7f27e8196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 -- 172.21.15.69:0/518312673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27e8101920 msgr2=0x7f27e8196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27e8101920 0x7f27e8196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 -- 172.21.15.69:0/518312673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e819e630 con 0x7f27e81022e0 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ecdc5640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e81022e0 0x7f27e819dfc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f27e000eb10 tx=0x7f27e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:26.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27e57fa640 1 -- 172.21.15.69:0/518312673 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0004b20 con 0x7f27e81022e0 2024-08-02T17:46:26.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e819e920 con 0x7f27e81022e0 2024-08-02T17:46:26.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27e57fa640 1 -- 172.21.15.69:0/518312673 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27e0004cc0 con 0x7f27e81022e0 2024-08-02T17:46:26.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27e57fa640 1 -- 172.21.15.69:0/518312673 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0010760 con 0x7f27e81022e0 2024-08-02T17:46:26.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.747+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e819ee70 con 0x7f27e81022e0 2024-08-02T17:46:26.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.748+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27e8107b40 con 0x7f27e81022e0 2024-08-02T17:46:26.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.750+0000 7f27e57fa640 1 -- 172.21.15.69:0/518312673 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f27e0005000 con 0x7f27e81022e0 2024-08-02T17:46:26.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.750+0000 7f27e57fa640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f27d00a2a10 0x7f27d00a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:26.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.750+0000 7f27e57fa640 1 -- 172.21.15.69:0/518312673 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f27e00cfb80 con 0x7f27e81022e0 2024-08-02T17:46:26.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.750+0000 7f27e7fff640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f27d00a2a10 0x7f27d00a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:26.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.751+0000 7f27e7fff640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f27d00a2a10 0x7f27d00a4ed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27d8005ab0 tx=0x7f27d8000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:26.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.757+0000 7f27e57fa640 1 -- 172.21.15.69:0/518312673 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f27e009c8b0 con 0x7f27e81022e0 2024-08-02T17:46:26.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.941+0000 7f27e57fa640 1 -- 172.21.15.69:0/518312673 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 22) ==== 138004+0+0 (secure 0 0 0) 0x7f27e0098820 con 0x7f27e81022e0 2024-08-02T17:46:26.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.990+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27e802d830 con 0x7f27e81022e0 2024-08-02T17:46:26.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.990+0000 7f27e57fa640 1 -- 172.21.15.69:0/518312673 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f27e0026690 con 0x7f27e81022e0 2024-08-02T17:46:26.992 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f27d00a2a10 msgr2=0x7f27d00a4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f27d00a2a10 0x7f27d00a4ed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27d8005ab0 tx=0x7f27d8000bc0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e81022e0 msgr2=0x7f27e819dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e81022e0 0x7f27e819dfc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f27e000eb10 tx=0x7f27e000c2d0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 shutdown_connections 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f27e81022e0 0x7f27e819dfc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f27d00a2a10 0x7f27d00a4ed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f27e8101920 0x7f27e8196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 --2- 172.21.15.69:0/518312673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f27e8100730 0x7f27e8196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:26.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 >> 172.21.15.69:0/518312673 conn(0x7f27e80fbeb0 msgr2=0x7f27e8104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:26.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 shutdown_connections 2024-08-02T17:46:26.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:26.992+0000 7f27ee828640 1 -- 172.21.15.69:0/518312673 wait complete. 2024-08-02T17:46:27.006 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-02T17:46:27.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-mon[32746]: mgrmap e21: x(active, since 1.02209s), standbys: z, y 2024-08-02T17:46:27.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/945847208' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-02T17:46:27.033 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-mon[32746]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:46:27.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-mon[32746]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:27.034 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/890407269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:27.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:26.781+0000 7fbac104b640 -1 log_channel(cephadm) log [ERR] : [02/Aug/2024:17:46:26] ENGINE Error in HTTPServer.serve 2024-08-02T17:46:27.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: Traceback (most recent call last): 2024-08-02T17:46:27.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-02T17:46:27.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._connections.run(self.expiration_interval) 2024-08-02T17:46:27.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-02T17:46:27.034 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._run(expiration_interval) 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: new_conn = self._from_server_socket(self.server.socket) 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: s = self.context.wrap_socket( 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: return self.sslsocket_class._create( 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self.do_handshake() 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: self._sslobj.do_handshake() 2024-08-02T17:46:27.035 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-02T17:46:27.036 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:46:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:46:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 -- 172.21.15.69:0/1621987731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360102320 msgr2=0x7f9360108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 --2- 172.21.15.69:0/1621987731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360102320 0x7f9360108bb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9358009ca0 tx=0x7f935802a8c0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 -- 172.21.15.69:0/1621987731 shutdown_connections 2024-08-02T17:46:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 --2- 172.21.15.69:0/1621987731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360102320 0x7f9360108bb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 --2- 172.21.15.69:0/1621987731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9360101960 0x7f9360101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 --2- 172.21.15.69:0/1621987731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9360100770 0x7f9360100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 -- 172.21.15.69:0/1621987731 >> 172.21.15.69:0/1621987731 conn(0x7f93600fbef0 msgr2=0x7f93600fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 -- 172.21.15.69:0/1621987731 shutdown_connections 2024-08-02T17:46:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.124+0000 7f93668b6640 1 -- 172.21.15.69:0/1621987731 wait complete. 2024-08-02T17:46:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f93668b6640 1 Processor -- start 2024-08-02T17:46:27.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f93668b6640 1 -- start start 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f93668b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9360100770 0x7f93601969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f93668b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360101960 0x7f9360196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f93668b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9360102320 0x7f936019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f93668b6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f936010c3f0 con 0x7f9360101960 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f93668b6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f936010c270 con 0x7f9360102320 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f93668b6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f936010c570 con 0x7f9360100770 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f935f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360101960 0x7f9360196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f935f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360101960 0x7f9360196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36418/0 (socket says 172.21.15.69:36418) 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f935ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9360100770 0x7f93601969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f935f7fe640 1 -- 172.21.15.69:0/563619000 learned_addr learned my addr 172.21.15.69:0/563619000 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f9364e53640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9360102320 0x7f936019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.125+0000 7f935ffff640 1 -- 172.21.15.69:0/563619000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9360102320 msgr2=0x7f936019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935ffff640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9360102320 0x7f936019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935ffff640 1 -- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360101960 msgr2=0x7f9360196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935ffff640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360101960 0x7f9360196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935ffff640 1 -- 172.21.15.69:0/563619000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936019e600 con 0x7f9360100770 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935f7fe640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360101960 0x7f9360196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935ffff640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9360100770 0x7f93601969e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f935000e9c0 tx=0x7f935000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935d7fa640 1 -- 172.21.15.69:0/563619000 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935000ccc0 con 0x7f9360100770 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936019e8f0 con 0x7f9360100770 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935d7fa640 1 -- 172.21.15.69:0/563619000 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93500035f0 con 0x7f9360100770 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f935d7fa640 1 -- 172.21.15.69:0/563619000 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935000ce60 con 0x7f9360100770 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.126+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936019ee40 con 0x7f9360100770 2024-08-02T17:46:27.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.127+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9360107ab0 con 0x7f9360100770 2024-08-02T17:46:27.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.128+0000 7f935d7fa640 1 -- 172.21.15.69:0/563619000 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 22) ==== 138004+0+0 (secure 0 0 0) 0x7f93500037a0 con 0x7f9360100770 2024-08-02T17:46:27.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.129+0000 7f935d7fa640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93480a2d90 0x7f93480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.129+0000 7f935f7fe640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93480a2d90 0x7f93480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.129+0000 7f935d7fa640 1 -- 172.21.15.69:0/563619000 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f93500cfda0 con 0x7f9360100770 2024-08-02T17:46:27.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.129+0000 7f935f7fe640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93480a2d90 0x7f93480a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9360197ef0 tx=0x7f9354005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:27.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.133+0000 7f935d7fa640 1 -- 172.21.15.69:0/563619000 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f935009c430 con 0x7f9360100770 2024-08-02T17:46:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:26 smithi083 ceph-mon[37031]: mgrmap e21: x(active, since 1.02209s), standbys: z, y 2024-08-02T17:46:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:26 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/945847208' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-02T17:46:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:26 smithi083 ceph-mon[37031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:46:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:26 smithi083 ceph-mon[37031]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:26 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/890407269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:27.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.318+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9360062b20 con 0x7f9360100770 2024-08-02T17:46:27.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.318+0000 7f935d7fa640 1 -- 172.21.15.69:0/563619000 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) ==== 104+0+11 (secure 0 0 0) 0x7f935009bd30 con 0x7f9360100770 2024-08-02T17:46:27.320 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-08-02T17:46:27.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93480a2d90 msgr2=0x7f93480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93480a2d90 0x7f93480a5250 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9360197ef0 tx=0x7f9354005f90 comp rx=0 tx=0).stop 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9360100770 msgr2=0x7f93601969e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9360100770 0x7f93601969e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f935000e9c0 tx=0x7f935000ee80 comp rx=0 tx=0).stop 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 shutdown_connections 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9360101960 0x7f9360196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93480a2d90 0x7f93480a5250 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9360102320 0x7f936019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 --2- 172.21.15.69:0/563619000 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9360100770 0x7f93601969e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 >> 172.21.15.69:0/563619000 conn(0x7f93600fbef0 msgr2=0x7f9360104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 shutdown_connections 2024-08-02T17:46:27.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.320+0000 7f93668b6640 1 -- 172.21.15.69:0/563619000 wait complete. 2024-08-02T17:46:27.333 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-02T17:46:27.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:26 smithi077 ceph-mon[37108]: mgrmap e21: x(active, since 1.02209s), standbys: z, y 2024-08-02T17:46:27.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:26 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/945847208' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-02T17:46:27.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:26 smithi077 ceph-mon[37108]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T17:46:27.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:26 smithi077 ceph-mon[37108]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:27.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:26 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/890407269' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 -- 172.21.15.69:0/2706972378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50381022e0 msgr2=0x7f50381046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 --2- 172.21.15.69:0/2706972378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50381022e0 0x7f50381046d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5028009900 tx=0x7f502802a410 comp rx=0 tx=0).stop 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 -- 172.21.15.69:0/2706972378 shutdown_connections 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 --2- 172.21.15.69:0/2706972378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5038107540 0x7f50381099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 --2- 172.21.15.69:0/2706972378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5038104c10 0x7f5038107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 --2- 172.21.15.69:0/2706972378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50381022e0 0x7f50381046d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 -- 172.21.15.69:0/2706972378 >> 172.21.15.69:0/2706972378 conn(0x7f50380fbec0 msgr2=0x7f50380fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 -- 172.21.15.69:0/2706972378 shutdown_connections 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.492+0000 7f503cc9e640 1 -- 172.21.15.69:0/2706972378 wait complete. 2024-08-02T17:46:27.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.493+0000 7f503cc9e640 1 Processor -- start 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.493+0000 7f503cc9e640 1 -- start start 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.493+0000 7f503cc9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50381022e0 0x7f5038197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.493+0000 7f503cc9e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038104c10 0x7f5038198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.493+0000 7f503cc9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5038107540 0x7f503819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.493+0000 7f503cc9e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f503810d520 con 0x7f5038104c10 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.493+0000 7f503cc9e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f503810d3a0 con 0x7f50381022e0 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.493+0000 7f503cc9e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f503810d6a0 con 0x7f5038107540 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038104c10 0x7f5038198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038104c10 0x7f5038198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36440/0 (socket says 172.21.15.69:36440) 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 -- 172.21.15.69:0/1541186041 learned_addr learned my addr 172.21.15.69:0/1541186041 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5036d9d640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5038107540 0x7f503819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f503659c640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50381022e0 0x7f5038197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 -- 172.21.15.69:0/1541186041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5038107540 msgr2=0x7f503819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5038107540 0x7f503819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 -- 172.21.15.69:0/1541186041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50381022e0 msgr2=0x7f5038197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50381022e0 0x7f5038197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 -- 172.21.15.69:0/1541186041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503819f900 con 0x7f5038104c10 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f5035d9b640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038104c10 0x7f5038198200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d770 tx=0x7f502c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f50277fe640 1 -- 172.21.15.69:0/1541186041 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c017070 con 0x7f5038104c10 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503819fbf0 con 0x7f5038104c10 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f50277fe640 1 -- 172.21.15.69:0/1541186041 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502c009c30 con 0x7f5038104c10 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f50277fe640 1 -- 172.21.15.69:0/1541186041 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c00f620 con 0x7f5038104c10 2024-08-02T17:46:27.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.494+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a0140 con 0x7f5038104c10 2024-08-02T17:46:27.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.495+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50380fd9b0 con 0x7f5038104c10 2024-08-02T17:46:27.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.497+0000 7f50277fe640 1 -- 172.21.15.69:0/1541186041 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 138004+0+0 (secure 0 0 0) 0x7f502c01a070 con 0x7f5038104c10 2024-08-02T17:46:27.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.498+0000 7f50277fe640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50200a2d90 0x7f50200a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.498+0000 7f503659c640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50200a2d90 0x7f50200a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.498+0000 7f50277fe640 1 -- 172.21.15.69:0/1541186041 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f502c0cf480 con 0x7f5038104c10 2024-08-02T17:46:27.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.498+0000 7f503659c640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50200a2d90 0x7f50200a5250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5028006010 tx=0x7f50280315d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:27.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.501+0000 7f50277fe640 1 -- 172.21.15.69:0/1541186041 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f502c009de0 con 0x7f5038104c10 2024-08-02T17:46:27.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.685+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f503802d830 con 0x7f5038104c10 2024-08-02T17:46:27.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.685+0000 7f50277fe640 1 -- 172.21.15.69:0/1541186041 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) ==== 100+0+11 (secure 0 0 0) 0x7f502c09bc10 con 0x7f5038104c10 2024-08-02T17:46:27.687 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50200a2d90 msgr2=0x7f50200a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50200a2d90 0x7f50200a5250 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5028006010 tx=0x7f50280315d0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038104c10 msgr2=0x7f5038198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038104c10 0x7f5038198200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d770 tx=0x7f502c00dc30 comp rx=0 tx=0).stop 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 shutdown_connections 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5038104c10 0x7f5038198200 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50200a2d90 0x7f50200a5250 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5038107540 0x7f503819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 --2- 172.21.15.69:0/1541186041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50381022e0 0x7f5038197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 >> 172.21.15.69:0/1541186041 conn(0x7f50380fbec0 msgr2=0x7f50381040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:27.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 shutdown_connections 2024-08-02T17:46:27.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.687+0000 7f503cc9e640 1 -- 172.21.15.69:0/1541186041 wait complete. 2024-08-02T17:46:27.700 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-02T17:46:27.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 -- 172.21.15.69:0/352812877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8074980 msgr2=0x7fc2a8074de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/352812877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8074980 0x7fc2a8074de0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009900 tx=0x7fc29c02a410 comp rx=0 tx=0).stop 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 -- 172.21.15.69:0/352812877 shutdown_connections 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/352812877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8074980 0x7fc2a8074de0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/352812877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2a81096a0 0x7fc2a810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/352812877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a8075f60 0x7fc2a80743b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 -- 172.21.15.69:0/352812877 >> 172.21.15.69:0/352812877 conn(0x7fc2a80fcfb0 msgr2=0x7fc2a80ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 -- 172.21.15.69:0/352812877 shutdown_connections 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 -- 172.21.15.69:0/352812877 wait complete. 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.811+0000 7fc2ada7f640 1 Processor -- start 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2ada7f640 1 -- start start 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2ada7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a8074980 0x7fc2a8199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2ada7f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8075f60 0x7fc2a819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2ada7f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2a81096a0 0x7fc2a81a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2ada7f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a810f6f0 con 0x7fc2a8075f60 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2ada7f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a810f570 con 0x7fc2a81096a0 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2ada7f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a810f870 con 0x7fc2a8074980 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8075f60 0x7fc2a819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8075f60 0x7fc2a819a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36452/0 (socket says 172.21.15.69:36452) 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 -- 172.21.15.69:0/1669179866 learned_addr learned my addr 172.21.15.69:0/1669179866 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a7fff640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2a81096a0 0x7fc2a81a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a77fe640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a8074980 0x7fc2a8199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 -- 172.21.15.69:0/1669179866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a8074980 msgr2=0x7fc2a8199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a8074980 0x7fc2a8199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 -- 172.21.15.69:0/1669179866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2a81096a0 msgr2=0x7fc2a81a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2a81096a0 0x7fc2a81a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 -- 172.21.15.69:0/1669179866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a81a1970 con 0x7fc2a8075f60 2024-08-02T17:46:27.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a7fff640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2a81096a0 0x7fc2a81a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:27.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.812+0000 7fc2a6ffd640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8075f60 0x7fc2a819a290 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc29c031440 tx=0x7fc29c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:27.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.813+0000 7fc2a4ff9640 1 -- 172.21.15.69:0/1669179866 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c03f030 con 0x7fc2a8075f60 2024-08-02T17:46:27.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.813+0000 7fc2a4ff9640 1 -- 172.21.15.69:0/1669179866 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc29c003ce0 con 0x7fc2a8075f60 2024-08-02T17:46:27.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.813+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a81a1c00 con 0x7fc2a8075f60 2024-08-02T17:46:27.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.813+0000 7fc2a4ff9640 1 -- 172.21.15.69:0/1669179866 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29c035070 con 0x7fc2a8075f60 2024-08-02T17:46:27.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.813+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a81a20f0 con 0x7fc2a8075f60 2024-08-02T17:46:27.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.814+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a8108420 con 0x7fc2a8075f60 2024-08-02T17:46:27.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.815+0000 7fc2a4ff9640 1 -- 172.21.15.69:0/1669179866 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 138004+0+0 (secure 0 0 0) 0x7fc29c042030 con 0x7fc2a8075f60 2024-08-02T17:46:27.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.816+0000 7fc2a4ff9640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2900a2d90 0x7fc2900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:27.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.816+0000 7fc2a4ff9640 1 -- 172.21.15.69:0/1669179866 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc29c0efe20 con 0x7fc2a8075f60 2024-08-02T17:46:27.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.816+0000 7fc2a77fe640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2900a2d90 0x7fc2900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:27.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.816+0000 7fc2a77fe640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2900a2d90 0x7fc2900a5250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc298005fd0 tx=0x7fc298009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:27.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:27.820+0000 7fc2a4ff9640 1 -- 172.21.15.69:0/1669179866 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc29c0bbec0 con 0x7fc2a8075f60 2024-08-02T17:46:28.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.009+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc2a802d830 con 0x7fc2a8075f60 2024-08-02T17:46:28.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.010+0000 7fc2a4ff9640 1 -- 172.21.15.69:0/1669179866 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) ==== 103+0+11 (secure 0 0 0) 0x7fc29c0c01c0 con 0x7fc2a8075f60 2024-08-02T17:46:28.012 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2900a2d90 msgr2=0x7fc2900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2900a2d90 0x7fc2900a5250 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc298005fd0 tx=0x7fc298009500 comp rx=0 tx=0).stop 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8075f60 msgr2=0x7fc2a819a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8075f60 0x7fc2a819a290 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc29c031440 tx=0x7fc29c032ce0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 shutdown_connections 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2a8075f60 0x7fc2a819a290 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2900a2d90 0x7fc2900a5250 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2a81096a0 0x7fc2a81a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 --2- 172.21.15.69:0/1669179866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2a8074980 0x7fc2a8199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 >> 172.21.15.69:0/1669179866 conn(0x7fc2a80fcfb0 msgr2=0x7fc2a81055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:28.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 shutdown_connections 2024-08-02T17:46:28.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.012+0000 7fc2ada7f640 1 -- 172.21.15.69:0/1669179866 wait complete. 2024-08-02T17:46:28.025 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: [02/Aug/2024:17:46:26] ENGINE Bus STARTING 2024-08-02T17:46:28.025 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: [02/Aug/2024:17:46:26] ENGINE Serving on https://172.21.15.69:7150 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: [02/Aug/2024:17:46:26] ENGINE Error in HTTPServer.serve 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: Traceback (most recent call last): 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._connections.run(self.expiration_interval) 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._run(expiration_interval) 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-02T17:46:28.026 INFO:journalctl@ceph.mon.a.smithi069.stdout: s = self.context.wrap_socket( 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout: return self.sslsocket_class._create( 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout: self.do_handshake() 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout: self._sslobj.do_handshake() 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: [02/Aug/2024:17:46:26] ENGINE Serving on http://172.21.15.69:8765 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: [02/Aug/2024:17:46:26] ENGINE Bus STARTED 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/518312673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.027 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/563619000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1541186041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.028 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:28.029 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:27 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:28.030 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-02T17:46:28.030 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:46:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: [02/Aug/2024:17:46:26] ENGINE Bus STARTING 2024-08-02T17:46:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: [02/Aug/2024:17:46:26] ENGINE Serving on https://172.21.15.69:7150 2024-08-02T17:46:28.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: [02/Aug/2024:17:46:26] ENGINE Error in HTTPServer.serve 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: Traceback (most recent call last): 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: self._connections.run(self.expiration_interval) 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: self._run(expiration_interval) 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: s = self.context.wrap_socket( 2024-08-02T17:46:28.272 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout: return self.sslsocket_class._create( 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout: self.do_handshake() 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout: self._sslobj.do_handshake() 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: [02/Aug/2024:17:46:26] ENGINE Serving on http://172.21.15.69:8765 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: [02/Aug/2024:17:46:26] ENGINE Bus STARTED 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/518312673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/563619000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1541186041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:28.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:27 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:28.325 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:28.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: [02/Aug/2024:17:46:26] ENGINE Bus STARTING 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: [02/Aug/2024:17:46:26] ENGINE Serving on https://172.21.15.69:7150 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: [02/Aug/2024:17:46:26] ENGINE Error in HTTPServer.serve 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: Traceback (most recent call last): 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: self._connections.run(self.expiration_interval) 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: self._run(expiration_interval) 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-02T17:46:28.348 INFO:journalctl@ceph.mon.b.smithi077.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: s = self.context.wrap_socket( 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: return self.sslsocket_class._create( 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: self.do_handshake() 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: self._sslobj.do_handshake() 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: [02/Aug/2024:17:46:26] ENGINE Serving on http://172.21.15.69:8765 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: [02/Aug/2024:17:46:26] ENGINE Bus STARTED 2024-08-02T17:46:28.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/518312673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/563619000' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1541186041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.351 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.352 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:28.352 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.352 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.352 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.352 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:46:28.352 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:28.352 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:27 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:28.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 -- 172.21.15.69:0/2613099887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1180101150 msgr2=0x7f118010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:28.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 --2- 172.21.15.69:0/2613099887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1180101150 0x7f118010d680 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1178009cd0 tx=0x7f117802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 -- 172.21.15.69:0/2613099887 shutdown_connections 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 --2- 172.21.15.69:0/2613099887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1180101150 0x7f118010d680 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 --2- 172.21.15.69:0/2613099887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11801007b0 0x7f1180100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 --2- 172.21.15.69:0/2613099887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11801067b0 0x7f1180106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 -- 172.21.15.69:0/2613099887 >> 172.21.15.69:0/2613099887 conn(0x7f11800fc4d0 msgr2=0x7f11800fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 -- 172.21.15.69:0/2613099887 shutdown_connections 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.857+0000 7f11856c2640 1 -- 172.21.15.69:0/2613099887 wait complete. 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f11856c2640 1 Processor -- start 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f11856c2640 1 -- start start 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f11856c2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11801007b0 0x7f1180198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f11856c2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1180101150 0x7f1180199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f11856c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11801067b0 0x7f11801a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f11856c2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1180110bb0 con 0x7f11801007b0 2024-08-02T17:46:28.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f11856c2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1180110a30 con 0x7f1180101150 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f11856c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1180110d30 con 0x7f11801067b0 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f117effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11801007b0 0x7f1180198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f117effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11801007b0 0x7f1180198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36472/0 (socket says 172.21.15.69:36472) 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f117effd640 1 -- 172.21.15.69:0/1000653107 learned_addr learned my addr 172.21.15.69:0/1000653107 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f117e7fc640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1180101150 0x7f1180199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.858+0000 7f117f7fe640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11801067b0 0x7f11801a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f117effd640 1 -- 172.21.15.69:0/1000653107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11801067b0 msgr2=0x7f11801a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f117effd640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11801067b0 0x7f11801a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f117effd640 1 -- 172.21.15.69:0/1000653107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1180101150 msgr2=0x7f1180199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f117effd640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1180101150 0x7f1180199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f117effd640 1 -- 172.21.15.69:0/1000653107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11801a4580 con 0x7f11801007b0 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f117effd640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11801007b0 0x7f1180198fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f117000e9f0 tx=0x7f117000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:28.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f1167fff640 1 -- 172.21.15.69:0/1000653107 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117000cd80 con 0x7f11801007b0 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11801a4750 con 0x7f11801007b0 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f1167fff640 1 -- 172.21.15.69:0/1000653107 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1170005500 con 0x7f11801007b0 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f1167fff640 1 -- 172.21.15.69:0/1000653107 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1170010690 con 0x7f11801007b0 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.859+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11801a4c10 con 0x7f11801007b0 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.860+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1180100c10 con 0x7f11801007b0 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.861+0000 7f1167fff640 1 -- 172.21.15.69:0/1000653107 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 22) ==== 138004+0+0 (secure 0 0 0) 0x7f11700046c0 con 0x7f11801007b0 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.862+0000 7f1167fff640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11680a2d80 0x7f11680a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.862+0000 7f117e7fc640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11680a2d80 0x7f11680a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:28.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.862+0000 7f1167fff640 1 -- 172.21.15.69:0/1000653107 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f11700cfd50 con 0x7f11801007b0 2024-08-02T17:46:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.862+0000 7f117e7fc640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11680a2d80 0x7f11680a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f118006b010 tx=0x7f1174005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:28.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:28.866+0000 7f1167fff640 1 -- 172.21.15.69:0/1000653107 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f11700d3030 con 0x7f11801007b0 2024-08-02T17:46:29.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.080+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1180108d20 con 0x7f11801007b0 2024-08-02T17:46:29.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-08-02T17:46:29.081 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi069 to 175.9M 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Unable to set osd_memory_target on smithi069 to 184535859: error parsing value: Value '184535859' is below minimum 939524096 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Updating smithi069:/etc/ceph/ceph.conf 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Updating smithi077:/etc/ceph/ceph.conf 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1669179866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.082 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:29.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:29.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:29.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:28 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.080+0000 7f1167fff640 1 -- 172.21.15.69:0/1000653107 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f117009c440 con 0x7f11801007b0 2024-08-02T17:46:29.084 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:29.084 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:46:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:46:29.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11680a2d80 msgr2=0x7f11680a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11680a2d80 0x7f11680a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f118006b010 tx=0x7f1174005f90 comp rx=0 tx=0).stop 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11801007b0 msgr2=0x7f1180198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11801007b0 0x7f1180198fc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f117000e9f0 tx=0x7f117000edf0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 shutdown_connections 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11680a2d80 0x7f11680a5240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11801067b0 0x7f11801a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1180101150 0x7f1180199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 --2- 172.21.15.69:0/1000653107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11801007b0 0x7f1180198fc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.082+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 >> 172.21.15.69:0/1000653107 conn(0x7f11800fc4d0 msgr2=0x7f11800fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.083+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 shutdown_connections 2024-08-02T17:46:29.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.083+0000 7f11856c2640 1 -- 172.21.15.69:0/1000653107 wait complete. 2024-08-02T17:46:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-08-02T17:46:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-08-02T17:46:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-08-02T17:46:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-08-02T17:46:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi069 to 175.9M 2024-08-02T17:46:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Unable to set osd_memory_target on smithi069 to 184535859: error parsing value: Value '184535859' is below minimum 939524096 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Updating smithi069:/etc/ceph/ceph.conf 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Updating smithi077:/etc/ceph/ceph.conf 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1669179866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:29.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:29.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:29.273 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:28 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi069 to 175.9M 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Unable to set osd_memory_target on smithi069 to 184535859: error parsing value: Value '184535859' is below minimum 939524096 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Updating smithi069:/etc/ceph/ceph.conf 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Updating smithi077:/etc/ceph/ceph.conf 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: Updating smithi083:/etc/ceph/ceph.conf 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1669179866' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:29.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:28 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:29.883 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-02T17:46:29.884 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-02T17:46:29.884 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-02T17:46:29.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 -- 172.21.15.69:0/3590784218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee60106bb0 msgr2=0x7fee6010e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 --2- 172.21.15.69:0/3590784218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee60106bb0 0x7fee6010e700 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee58009ca0 tx=0x7fee5802a8c0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 -- 172.21.15.69:0/3590784218 shutdown_connections 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 --2- 172.21.15.69:0/3590784218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee60106bb0 0x7fee6010e700 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 --2- 172.21.15.69:0/3590784218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee60100990 0x7fee60106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 --2- 172.21.15.69:0/3590784218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee600fffe0 0x7fee601003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 -- 172.21.15.69:0/3590784218 >> 172.21.15.69:0/3590784218 conn(0x7fee60076e20 msgr2=0x7fee600fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 -- 172.21.15.69:0/3590784218 shutdown_connections 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.988+0000 7fee67729640 1 -- 172.21.15.69:0/3590784218 wait complete. 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee67729640 1 Processor -- start 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee67729640 1 -- start start 2024-08-02T17:46:29.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee67729640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee600fffe0 0x7fee60198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee67729640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee60106bb0 0x7fee601986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee67729640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee6019ca20 0x7fee6019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee67729640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee60111c10 con 0x7fee6019ca20 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee67729640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fee60111a90 con 0x7fee60106bb0 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee67729640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee60111d90 con 0x7fee600fffe0 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee65cc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee6019ca20 0x7fee6019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee65cc6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee6019ca20 0x7fee6019ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36478/0 (socket says 172.21.15.69:36478) 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee65cc6640 1 -- 172.21.15.69:0/1423936046 learned_addr learned my addr 172.21.15.69:0/1423936046 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee64cc4640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee60106bb0 0x7fee601986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee654c5640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee600fffe0 0x7fee60198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:29.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee65cc6640 1 -- 172.21.15.69:0/1423936046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee600fffe0 msgr2=0x7fee60198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee65cc6640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee600fffe0 0x7fee60198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee65cc6640 1 -- 172.21.15.69:0/1423936046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee60106bb0 msgr2=0x7fee601986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee65cc6640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee60106bb0 0x7fee601986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee65cc6640 1 -- 172.21.15.69:0/1423936046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6019d510 con 0x7fee6019ca20 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.989+0000 7fee654c5640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee600fffe0 0x7fee60198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.990+0000 7fee65cc6640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee6019ca20 0x7fee6019ced0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee58031a20 tx=0x7fee58031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.990+0000 7fee527fc640 1 -- 172.21.15.69:0/1423936046 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58003c30 con 0x7fee6019ca20 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.990+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6019d7a0 con 0x7fee6019ca20 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.990+0000 7fee527fc640 1 -- 172.21.15.69:0/1423936046 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee58003dd0 con 0x7fee6019ca20 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.990+0000 7fee527fc640 1 -- 172.21.15.69:0/1423936046 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5802fcf0 con 0x7fee6019ca20 2024-08-02T17:46:29.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.990+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee601a4fc0 con 0x7fee6019ca20 2024-08-02T17:46:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.991+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee601003c0 con 0x7fee6019ca20 2024-08-02T17:46:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.992+0000 7fee527fc640 1 -- 172.21.15.69:0/1423936046 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fee58004050 con 0x7fee6019ca20 2024-08-02T17:46:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.993+0000 7fee527fc640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee4c0a2e20 0x7fee4c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:29.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.993+0000 7fee654c5640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee4c0a2e20 0x7fee4c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:29.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.993+0000 7fee527fc640 1 -- 172.21.15.69:0/1423936046 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fee580f0e00 con 0x7fee6019ca20 2024-08-02T17:46:29.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.993+0000 7fee654c5640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee4c0a2e20 0x7fee4c0a52e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee54005d00 tx=0x7fee54009550 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:29.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:29.997+0000 7fee527fc640 1 -- 172.21.15.69:0/1423936046 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fee580bd5b0 con 0x7fee6019ca20 2024-08-02T17:46:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:30.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1000653107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:30.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:29 smithi069 ceph-mon[32746]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-08-02T17:46:30.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.172+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee60100990 con 0x7fee4c0a2e20 2024-08-02T17:46:30.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.181+0000 7fee527fc640 1 -- 172.21.15.69:0/1423936046 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339514 (secure 0 0 0) 0x7fee60100990 con 0x7fee4c0a2e20 2024-08-02T17:46:30.186 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:30.188 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:46:30.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee4c0a2e20 msgr2=0x7fee4c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee4c0a2e20 0x7fee4c0a52e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee54005d00 tx=0x7fee54009550 comp rx=0 tx=0).stop 2024-08-02T17:46:30.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee6019ca20 msgr2=0x7fee6019ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee6019ca20 0x7fee6019ced0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee58031a20 tx=0x7fee58031a50 comp rx=0 tx=0).stop 2024-08-02T17:46:30.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 shutdown_connections 2024-08-02T17:46:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee6019ca20 0x7fee6019ced0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee4c0a2e20 0x7fee4c0a52e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee60106bb0 0x7fee601986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 --2- 172.21.15.69:0/1423936046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee600fffe0 0x7fee60198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 >> 172.21.15.69:0/1423936046 conn(0x7fee60076e20 msgr2=0x7fee60104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 shutdown_connections 2024-08-02T17:46:30.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.184+0000 7fee67729640 1 -- 172.21.15.69:0/1423936046 wait complete. 2024-08-02T17:46:30.197 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-08-02T17:46:28.936638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190144,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":849,"num_read_kb":585,"num_write":481,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398040,"kb_used_data":15444,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543784,"statfs":{"total":1151940427776,"available":1151532834816,"internally_reserved":0,"allocated":15814656,"data_stored":9023382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":391492921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.008758"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915530+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:46:24.915530+0000","last_peered":"2024-08-02T17:46:24.915530+0000","last_clean":"2024-08-02T17:46:24.915530+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:46:24.915530+0000","last_undegraded":"2024-08-02T17:46:24.915530+0000","last_fullsized":"2024-08-02T17:46:24.915530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954285+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:46:24.954285+0000","last_peered":"2024-08-02T17:46:24.954285+0000","last_clean":"2024-08-02T17:46:24.954285+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:46:24.954285+0000","last_undegraded":"2024-08-02T17:46:24.954285+0000","last_fullsized":"2024-08-02T17:46:24.954285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915172+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:46:24.915172+0000","last_peered":"2024-08-02T17:46:24.915172+0000","last_clean":"2024-08-02T17:46:24.915172+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:46:24.915172+0000","last_undegraded":"2024-08-02T17:46:24.915172+0000","last_fullsized":"2024-08-02T17:46:24.915172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908437+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:46:24.908437+0000","last_peered":"2024-08-02T17:46:24.908437+0000","last_clean":"2024-08-02T17:46:24.908437+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:46:24.908437+0000","last_undegraded":"2024-08-02T17:46:24.908437+0000","last_fullsized":"2024-08-02T17:46:24.908437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959113+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:46:24.959113+0000","last_peered":"2024-08-02T17:46:24.959113+0000","last_clean":"2024-08-02T17:46:24.959113+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:46:24.959113+0000","last_undegraded":"2024-08-02T17:46:24.959113+0000","last_fullsized":"2024-08-02T17:46:24.959113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906783+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:46:24.906783+0000","last_peered":"2024-08-02T17:46:24.906783+0000","last_clean":"2024-08-02T17:46:24.906783+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:46:24.906783+0000","last_undegraded":"2024-08-02T17:46:24.906783+0000","last_fullsized":"2024-08-02T17:46:24.906783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906846+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:46:24.906846+0000","last_peered":"2024-08-02T17:46:24.906846+0000","last_clean":"2024-08-02T17:46:24.906846+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:46:24.906846+0000","last_undegraded":"2024-08-02T17:46:24.906846+0000","last_fullsized":"2024-08-02T17:46:24.906846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962043+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:46:24.962043+0000","last_peered":"2024-08-02T17:46:24.962043+0000","last_clean":"2024-08-02T17:46:24.962043+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:46:24.962043+0000","last_undegraded":"2024-08-02T17:46:24.962043+0000","last_fullsized":"2024-08-02T17:46:24.962043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906895+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:46:24.906895+0000","last_peered":"2024-08-02T17:46:24.906895+0000","last_clean":"2024-08-02T17:46:24.906895+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:46:24.906895+0000","last_undegraded":"2024-08-02T17:46:24.906895+0000","last_fullsized":"2024-08-02T17:46:24.906895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906857+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:46:24.906857+0000","last_peered":"2024-08-02T17:46:24.906857+0000","last_clean":"2024-08-02T17:46:24.906857+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:46:24.906857+0000","last_undegraded":"2024-08-02T17:46:24.906857+0000","last_fullsized":"2024-08-02T17:46:24.906857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959833+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:46:24.959833+0000","last_peered":"2024-08-02T17:46:24.959833+0000","last_clean":"2024-08-02T17:46:24.959833+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:46:24.959833+0000","last_undegraded":"2024-08-02T17:46:24.959833+0000","last_fullsized":"2024-08-02T17:46:24.959833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906814+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:46:24.906814+0000","last_peered":"2024-08-02T17:46:24.906814+0000","last_clean":"2024-08-02T17:46:24.906814+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:46:24.906814+0000","last_undegraded":"2024-08-02T17:46:24.906814+0000","last_fullsized":"2024-08-02T17:46:24.906814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961844+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:46:24.961844+0000","last_peered":"2024-08-02T17:46:24.961844+0000","last_clean":"2024-08-02T17:46:24.961844+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:46:24.961844+0000","last_undegraded":"2024-08-02T17:46:24.961844+0000","last_fullsized":"2024-08-02T17:46:24.961844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906478+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:46:24.906478+0000","last_peered":"2024-08-02T17:46:24.906478+0000","last_clean":"2024-08-02T17:46:24.906478+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:46:24.906478+0000","last_undegraded":"2024-08-02T17:46:24.906478+0000","last_fullsized":"2024-08-02T17:46:24.906478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907202+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:46:24.907202+0000","last_peered":"2024-08-02T17:46:24.907202+0000","last_clean":"2024-08-02T17:46:24.907202+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:46:24.907202+0000","last_undegraded":"2024-08-02T17:46:24.907202+0000","last_fullsized":"2024-08-02T17:46:24.907202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959010+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:46:24.959010+0000","last_peered":"2024-08-02T17:46:24.959010+0000","last_clean":"2024-08-02T17:46:24.959010+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:46:24.959010+0000","last_undegraded":"2024-08-02T17:46:24.959010+0000","last_fullsized":"2024-08-02T17:46:24.959010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920005+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:46:24.920005+0000","last_peered":"2024-08-02T17:46:24.920005+0000","last_clean":"2024-08-02T17:46:24.920005+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:46:24.920005+0000","last_undegraded":"2024-08-02T17:46:24.920005+0000","last_fullsized":"2024-08-02T17:46:24.920005+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907441+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:46:24.907441+0000","last_peered":"2024-08-02T17:46:24.907441+0000","last_clean":"2024-08-02T17:46:24.907441+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:46:24.907441+0000","last_undegraded":"2024-08-02T17:46:24.907441+0000","last_fullsized":"2024-08-02T17:46:24.907441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956539+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:46:24.956539+0000","last_peered":"2024-08-02T17:46:24.956539+0000","last_clean":"2024-08-02T17:46:24.956539+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:46:24.956539+0000","last_undegraded":"2024-08-02T17:46:24.956539+0000","last_fullsized":"2024-08-02T17:46:24.956539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907900+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:46:24.907900+0000","last_peered":"2024-08-02T17:46:24.907900+0000","last_clean":"2024-08-02T17:46:24.907900+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:46:24.907900+0000","last_undegraded":"2024-08-02T17:46:24.907900+0000","last_fullsized":"2024-08-02T17:46:24.907900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915312+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:46:24.915312+0000","last_peered":"2024-08-02T17:46:24.915312+0000","last_clean":"2024-08-02T17:46:24.915312+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:46:24.915312+0000","last_undegraded":"2024-08-02T17:46:24.915312+0000","last_fullsized":"2024-08-02T17:46:24.915312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924149+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:46:24.924149+0000","last_peered":"2024-08-02T17:46:24.924149+0000","last_clean":"2024-08-02T17:46:24.924149+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:46:24.924149+0000","last_undegraded":"2024-08-02T17:46:24.924149+0000","last_fullsized":"2024-08-02T17:46:24.924149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915653+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:46:24.915653+0000","last_peered":"2024-08-02T17:46:24.915653+0000","last_clean":"2024-08-02T17:46:24.915653+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:46:24.915653+0000","last_undegraded":"2024-08-02T17:46:24.915653+0000","last_fullsized":"2024-08-02T17:46:24.915653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907240+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:46:24.907240+0000","last_peered":"2024-08-02T17:46:24.907240+0000","last_clean":"2024-08-02T17:46:24.907240+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:46:24.907240+0000","last_undegraded":"2024-08-02T17:46:24.907240+0000","last_fullsized":"2024-08-02T17:46:24.907240+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961716+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:46:24.961716+0000","last_peered":"2024-08-02T17:46:24.961716+0000","last_clean":"2024-08-02T17:46:24.961716+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:46:24.961716+0000","last_undegraded":"2024-08-02T17:46:24.961716+0000","last_fullsized":"2024-08-02T17:46:24.961716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907489+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:46:24.907489+0000","last_peered":"2024-08-02T17:46:24.907489+0000","last_clean":"2024-08-02T17:46:24.907489+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:46:24.907489+0000","last_undegraded":"2024-08-02T17:46:24.907489+0000","last_fullsized":"2024-08-02T17:46:24.907489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915365+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:46:24.915365+0000","last_peered":"2024-08-02T17:46:24.915365+0000","last_clean":"2024-08-02T17:46:24.915365+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:46:24.915365+0000","last_undegraded":"2024-08-02T17:46:24.915365+0000","last_fullsized":"2024-08-02T17:46:24.915365+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956310+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:46:24.956310+0000","last_peered":"2024-08-02T17:46:24.956310+0000","last_clean":"2024-08-02T17:46:24.956310+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:46:24.956310+0000","last_undegraded":"2024-08-02T17:46:24.956310+0000","last_fullsized":"2024-08-02T17:46:24.956310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'47","reported_seq":122,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.154876+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:46:25.154876+0000","last_peered":"2024-08-02T17:46:25.154876+0000","last_clean":"2024-08-02T17:46:25.154876+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:46:25.154876+0000","last_undegraded":"2024-08-02T17:46:25.154876+0000","last_fullsized":"2024-08-02T17:46:25.154876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907676+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:46:24.907676+0000","last_peered":"2024-08-02T17:46:24.907676+0000","last_clean":"2024-08-02T17:46:24.907676+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.907676+0000","last_undegraded":"2024-08-02T17:46:24.907676+0000","last_fullsized":"2024-08-02T17:46:24.907676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950528+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:46:24.950528+0000","last_peered":"2024-08-02T17:46:24.950528+0000","last_clean":"2024-08-02T17:46:24.950528+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:46:24.950528+0000","last_undegraded":"2024-08-02T17:46:24.950528+0000","last_fullsized":"2024-08-02T17:46:24.950528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906878+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:46:24.906878+0000","last_peered":"2024-08-02T17:46:24.906878+0000","last_clean":"2024-08-02T17:46:24.906878+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:46:24.906878+0000","last_undegraded":"2024-08-02T17:46:24.906878+0000","last_fullsized":"2024-08-02T17:46:24.906878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920332+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:46:24.920332+0000","last_peered":"2024-08-02T17:46:24.920332+0000","last_clean":"2024-08-02T17:46:24.920332+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:46:24.920332+0000","last_undegraded":"2024-08-02T17:46:24.920332+0000","last_fullsized":"2024-08-02T17:46:24.920332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953960+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:46:24.953960+0000","last_peered":"2024-08-02T17:46:24.953960+0000","last_clean":"2024-08-02T17:46:24.953960+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:46:24.953960+0000","last_undegraded":"2024-08-02T17:46:24.953960+0000","last_fullsized":"2024-08-02T17:46:24.953960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915185+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:46:24.915185+0000","last_peered":"2024-08-02T17:46:24.915185+0000","last_clean":"2024-08-02T17:46:24.915185+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:46:24.915185+0000","last_undegraded":"2024-08-02T17:46:24.915185+0000","last_fullsized":"2024-08-02T17:46:24.915185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906282+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:46:24.906282+0000","last_peered":"2024-08-02T17:46:24.906282+0000","last_clean":"2024-08-02T17:46:24.906282+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:46:24.906282+0000","last_undegraded":"2024-08-02T17:46:24.906282+0000","last_fullsized":"2024-08-02T17:46:24.906282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956518+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:46:24.956518+0000","last_peered":"2024-08-02T17:46:24.956518+0000","last_clean":"2024-08-02T17:46:24.956518+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:46:24.956518+0000","last_undegraded":"2024-08-02T17:46:24.956518+0000","last_fullsized":"2024-08-02T17:46:24.956518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.909639+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:46:24.909639+0000","last_peered":"2024-08-02T17:46:24.909639+0000","last_clean":"2024-08-02T17:46:24.909639+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:46:24.909639+0000","last_undegraded":"2024-08-02T17:46:24.909639+0000","last_fullsized":"2024-08-02T17:46:24.909639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.918812+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:46:24.918812+0000","last_peered":"2024-08-02T17:46:24.918812+0000","last_clean":"2024-08-02T17:46:24.918812+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:46:24.918812+0000","last_undegraded":"2024-08-02T17:46:24.918812+0000","last_fullsized":"2024-08-02T17:46:24.918812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952882+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:46:24.952882+0000","last_peered":"2024-08-02T17:46:24.952882+0000","last_clean":"2024-08-02T17:46:24.952882+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:46:24.952882+0000","last_undegraded":"2024-08-02T17:46:24.952882+0000","last_fullsized":"2024-08-02T17:46:24.952882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":143,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919340+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:46:24.919340+0000","last_peered":"2024-08-02T17:46:24.919340+0000","last_clean":"2024-08-02T17:46:24.919340+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:46:24.919340+0000","last_undegraded":"2024-08-02T17:46:24.919340+0000","last_fullsized":"2024-08-02T17:46:24.919340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919707+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:46:24.919707+0000","last_peered":"2024-08-02T17:46:24.919707+0000","last_clean":"2024-08-02T17:46:24.919707+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.919707+0000","last_undegraded":"2024-08-02T17:46:24.919707+0000","last_fullsized":"2024-08-02T17:46:24.919707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947422+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:46:24.947422+0000","last_peered":"2024-08-02T17:46:24.947422+0000","last_clean":"2024-08-02T17:46:24.947422+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:46:24.947422+0000","last_undegraded":"2024-08-02T17:46:24.947422+0000","last_fullsized":"2024-08-02T17:46:24.947422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908927+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:46:24.908927+0000","last_peered":"2024-08-02T17:46:24.908927+0000","last_clean":"2024-08-02T17:46:24.908927+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:46:24.908927+0000","last_undegraded":"2024-08-02T17:46:24.908927+0000","last_fullsized":"2024-08-02T17:46:24.908927+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915349+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:46:24.915349+0000","last_peered":"2024-08-02T17:46:24.915349+0000","last_clean":"2024-08-02T17:46:24.915349+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:46:24.915349+0000","last_undegraded":"2024-08-02T17:46:24.915349+0000","last_fullsized":"2024-08-02T17:46:24.915349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962129+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:46:24.962129+0000","last_peered":"2024-08-02T17:46:24.962129+0000","last_clean":"2024-08-02T17:46:24.962129+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:46:24.962129+0000","last_undegraded":"2024-08-02T17:46:24.962129+0000","last_fullsized":"2024-08-02T17:46:24.962129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907233+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:46:24.907233+0000","last_peered":"2024-08-02T17:46:24.907233+0000","last_clean":"2024-08-02T17:46:24.907233+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:46:24.907233+0000","last_undegraded":"2024-08-02T17:46:24.907233+0000","last_fullsized":"2024-08-02T17:46:24.907233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908841+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:46:24.908841+0000","last_peered":"2024-08-02T17:46:24.908841+0000","last_clean":"2024-08-02T17:46:24.908841+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:46:24.908841+0000","last_undegraded":"2024-08-02T17:46:24.908841+0000","last_fullsized":"2024-08-02T17:46:24.908841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910641+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:46:24.910641+0000","last_peered":"2024-08-02T17:46:24.910641+0000","last_clean":"2024-08-02T17:46:24.910641+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:46:24.910641+0000","last_undegraded":"2024-08-02T17:46:24.910641+0000","last_fullsized":"2024-08-02T17:46:24.910641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960428+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:46:24.960428+0000","last_peered":"2024-08-02T17:46:24.960428+0000","last_clean":"2024-08-02T17:46:24.960428+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:46:24.960428+0000","last_undegraded":"2024-08-02T17:46:24.960428+0000","last_fullsized":"2024-08-02T17:46:24.960428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.916318+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:46:24.916318+0000","last_peered":"2024-08-02T17:46:24.916318+0000","last_clean":"2024-08-02T17:46:24.916318+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:46:24.916318+0000","last_undegraded":"2024-08-02T17:46:24.916318+0000","last_fullsized":"2024-08-02T17:46:24.916318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915831+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:46:24.915831+0000","last_peered":"2024-08-02T17:46:24.915831+0000","last_clean":"2024-08-02T17:46:24.915831+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:46:24.915831+0000","last_undegraded":"2024-08-02T17:46:24.915831+0000","last_fullsized":"2024-08-02T17:46:24.915831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948829+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:46:24.948829+0000","last_peered":"2024-08-02T17:46:24.948829+0000","last_clean":"2024-08-02T17:46:24.948829+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:46:24.948829+0000","last_undegraded":"2024-08-02T17:46:24.948829+0000","last_fullsized":"2024-08-02T17:46:24.948829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":87,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.150408+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:46:25.150408+0000","last_peered":"2024-08-02T17:46:25.150408+0000","last_clean":"2024-08-02T17:46:25.150408+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:46:25.150408+0000","last_undegraded":"2024-08-02T17:46:25.150408+0000","last_fullsized":"2024-08-02T17:46:25.150408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924774+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:46:24.924774+0000","last_peered":"2024-08-02T17:46:24.924774+0000","last_clean":"2024-08-02T17:46:24.924774+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:46:24.924774+0000","last_undegraded":"2024-08-02T17:46:24.924774+0000","last_fullsized":"2024-08-02T17:46:24.924774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955439+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:46:24.955439+0000","last_peered":"2024-08-02T17:46:24.955439+0000","last_clean":"2024-08-02T17:46:24.955439+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:46:24.955439+0000","last_undegraded":"2024-08-02T17:46:24.955439+0000","last_fullsized":"2024-08-02T17:46:24.955439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919542+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:46:24.919542+0000","last_peered":"2024-08-02T17:46:24.919542+0000","last_clean":"2024-08-02T17:46:24.919542+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:46:24.919542+0000","last_undegraded":"2024-08-02T17:46:24.919542+0000","last_fullsized":"2024-08-02T17:46:24.919542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914656+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:46:24.914656+0000","last_peered":"2024-08-02T17:46:24.914656+0000","last_clean":"2024-08-02T17:46:24.914656+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:46:24.914656+0000","last_undegraded":"2024-08-02T17:46:24.914656+0000","last_fullsized":"2024-08-02T17:46:24.914656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950756+0000","last_change":"2024-08-02T17:45:57.979722+0000","last_active":"2024-08-02T17:46:24.950756+0000","last_peered":"2024-08-02T17:46:24.950756+0000","last_clean":"2024-08-02T17:46:24.950756+0000","last_became_active":"2024-08-02T17:45:57.979427+0000","last_became_peered":"2024-08-02T17:45:57.979427+0000","last_unstale":"2024-08-02T17:46:24.950756+0000","last_undegraded":"2024-08-02T17:46:24.950756+0000","last_fullsized":"2024-08-02T17:46:24.950756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:16:52.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.920115+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:46:25.920115+0000","last_peered":"2024-08-02T17:46:25.920115+0000","last_clean":"2024-08-02T17:46:25.920115+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:46:25.920115+0000","last_undegraded":"2024-08-02T17:46:25.920115+0000","last_fullsized":"2024-08-02T17:46:25.920115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914880+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:46:24.914880+0000","last_peered":"2024-08-02T17:46:24.914880+0000","last_clean":"2024-08-02T17:46:24.914880+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:46:24.914880+0000","last_undegraded":"2024-08-02T17:46:24.914880+0000","last_fullsized":"2024-08-02T17:46:24.914880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953834+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:46:24.953834+0000","last_peered":"2024-08-02T17:46:24.953834+0000","last_clean":"2024-08-02T17:46:24.953834+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:46:24.953834+0000","last_undegraded":"2024-08-02T17:46:24.953834+0000","last_fullsized":"2024-08-02T17:46:24.953834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906346+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:46:24.906346+0000","last_peered":"2024-08-02T17:46:24.906346+0000","last_clean":"2024-08-02T17:46:24.906346+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:46:24.906346+0000","last_undegraded":"2024-08-02T17:46:24.906346+0000","last_fullsized":"2024-08-02T17:46:24.906346+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919446+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:46:24.919446+0000","last_peered":"2024-08-02T17:46:24.919446+0000","last_clean":"2024-08-02T17:46:24.919446+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:46:24.919446+0000","last_undegraded":"2024-08-02T17:46:24.919446+0000","last_fullsized":"2024-08-02T17:46:24.919446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950106+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:46:24.950106+0000","last_peered":"2024-08-02T17:46:24.950106+0000","last_clean":"2024-08-02T17:46:24.950106+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:46:24.950106+0000","last_undegraded":"2024-08-02T17:46:24.950106+0000","last_fullsized":"2024-08-02T17:46:24.950106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":152,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.202288+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:46:25.202288+0000","last_peered":"2024-08-02T17:46:25.202288+0000","last_clean":"2024-08-02T17:46:25.202288+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:46:25.202288+0000","last_undegraded":"2024-08-02T17:46:25.202288+0000","last_fullsized":"2024-08-02T17:46:25.202288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2207,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":6,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915368+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:46:24.915368+0000","last_peered":"2024-08-02T17:46:24.915368+0000","last_clean":"2024-08-02T17:46:24.915368+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:46:24.915368+0000","last_undegraded":"2024-08-02T17:46:24.915368+0000","last_fullsized":"2024-08-02T17:46:24.915368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957938+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:46:24.957938+0000","last_peered":"2024-08-02T17:46:24.957938+0000","last_clean":"2024-08-02T17:46:24.957938+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:46:24.957938+0000","last_undegraded":"2024-08-02T17:46:24.957938+0000","last_fullsized":"2024-08-02T17:46:24.957938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915099+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:46:24.915099+0000","last_peered":"2024-08-02T17:46:24.915099+0000","last_clean":"2024-08-02T17:46:24.915099+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:46:24.915099+0000","last_undegraded":"2024-08-02T17:46:24.915099+0000","last_fullsized":"2024-08-02T17:46:24.915099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920404+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:46:24.920404+0000","last_peered":"2024-08-02T17:46:24.920404+0000","last_clean":"2024-08-02T17:46:24.920404+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:46:24.920404+0000","last_undegraded":"2024-08-02T17:46:24.920404+0000","last_fullsized":"2024-08-02T17:46:24.920404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958381+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:46:24.958381+0000","last_peered":"2024-08-02T17:46:24.958381+0000","last_clean":"2024-08-02T17:46:24.958381+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:46:24.958381+0000","last_undegraded":"2024-08-02T17:46:24.958381+0000","last_fullsized":"2024-08-02T17:46:24.958381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915872+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:46:24.915872+0000","last_peered":"2024-08-02T17:46:24.915872+0000","last_clean":"2024-08-02T17:46:24.915872+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:46:24.915872+0000","last_undegraded":"2024-08-02T17:46:24.915872+0000","last_fullsized":"2024-08-02T17:46:24.915872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915458+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:46:24.915458+0000","last_peered":"2024-08-02T17:46:24.915458+0000","last_clean":"2024-08-02T17:46:24.915458+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:46:24.915458+0000","last_undegraded":"2024-08-02T17:46:24.915458+0000","last_fullsized":"2024-08-02T17:46:24.915458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955734+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:46:24.955734+0000","last_peered":"2024-08-02T17:46:24.955734+0000","last_clean":"2024-08-02T17:46:24.955734+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:46:24.955734+0000","last_undegraded":"2024-08-02T17:46:24.955734+0000","last_fullsized":"2024-08-02T17:46:24.955734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'38","reported_seq":285,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.207389+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:46:25.207389+0000","last_peered":"2024-08-02T17:46:25.207389+0000","last_clean":"2024-08-02T17:46:25.207389+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:46:25.207389+0000","last_undegraded":"2024-08-02T17:46:25.207389+0000","last_fullsized":"2024-08-02T17:46:25.207389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68576,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":168,"num_write":39,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911360+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:46:24.911360+0000","last_peered":"2024-08-02T17:46:24.911360+0000","last_clean":"2024-08-02T17:46:24.911360+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:46:24.911360+0000","last_undegraded":"2024-08-02T17:46:24.911360+0000","last_fullsized":"2024-08-02T17:46:24.911360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959993+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:46:24.959993+0000","last_peered":"2024-08-02T17:46:24.959993+0000","last_clean":"2024-08-02T17:46:24.959993+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:46:24.959993+0000","last_undegraded":"2024-08-02T17:46:24.959993+0000","last_fullsized":"2024-08-02T17:46:24.959993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915691+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:46:24.915691+0000","last_peered":"2024-08-02T17:46:24.915691+0000","last_clean":"2024-08-02T17:46:24.915691+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:46:24.915691+0000","last_undegraded":"2024-08-02T17:46:24.915691+0000","last_fullsized":"2024-08-02T17:46:24.915691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907020+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:46:24.907020+0000","last_peered":"2024-08-02T17:46:24.907020+0000","last_clean":"2024-08-02T17:46:24.907020+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:46:24.907020+0000","last_undegraded":"2024-08-02T17:46:24.907020+0000","last_fullsized":"2024-08-02T17:46:24.907020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960499+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:46:24.960499+0000","last_peered":"2024-08-02T17:46:24.960499+0000","last_clean":"2024-08-02T17:46:24.960499+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:46:24.960499+0000","last_undegraded":"2024-08-02T17:46:24.960499+0000","last_fullsized":"2024-08-02T17:46:24.960499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910476+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:46:24.910476+0000","last_peered":"2024-08-02T17:46:24.910476+0000","last_clean":"2024-08-02T17:46:24.910476+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:46:24.910476+0000","last_undegraded":"2024-08-02T17:46:24.910476+0000","last_fullsized":"2024-08-02T17:46:24.910476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914823+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:46:24.914823+0000","last_peered":"2024-08-02T17:46:24.914823+0000","last_clean":"2024-08-02T17:46:24.914823+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:46:24.914823+0000","last_undegraded":"2024-08-02T17:46:24.914823+0000","last_fullsized":"2024-08-02T17:46:24.914823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962330+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:46:24.962330+0000","last_peered":"2024-08-02T17:46:24.962330+0000","last_clean":"2024-08-02T17:46:24.962330+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:46:24.962330+0000","last_undegraded":"2024-08-02T17:46:24.962330+0000","last_fullsized":"2024-08-02T17:46:24.962330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":143,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.055690+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:46:25.055690+0000","last_peered":"2024-08-02T17:46:25.055690+0000","last_clean":"2024-08-02T17:46:25.055690+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:46:25.055690+0000","last_undegraded":"2024-08-02T17:46:25.055690+0000","last_fullsized":"2024-08-02T17:46:25.055690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907780+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:46:24.907780+0000","last_peered":"2024-08-02T17:46:24.907780+0000","last_clean":"2024-08-02T17:46:24.907780+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:46:24.907780+0000","last_undegraded":"2024-08-02T17:46:24.907780+0000","last_fullsized":"2024-08-02T17:46:24.907780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.926268+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:46:24.926268+0000","last_peered":"2024-08-02T17:46:24.926268+0000","last_clean":"2024-08-02T17:46:24.926268+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:46:24.926268+0000","last_undegraded":"2024-08-02T17:46:24.926268+0000","last_fullsized":"2024-08-02T17:46:24.926268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956250+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:46:24.956250+0000","last_peered":"2024-08-02T17:46:24.956250+0000","last_clean":"2024-08-02T17:46:24.956250+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:46:24.956250+0000","last_undegraded":"2024-08-02T17:46:24.956250+0000","last_fullsized":"2024-08-02T17:46:24.956250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914914+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:46:24.914914+0000","last_peered":"2024-08-02T17:46:24.914914+0000","last_clean":"2024-08-02T17:46:24.914914+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:46:24.914914+0000","last_undegraded":"2024-08-02T17:46:24.914914+0000","last_fullsized":"2024-08-02T17:46:24.914914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920061+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:46:24.920061+0000","last_peered":"2024-08-02T17:46:24.920061+0000","last_clean":"2024-08-02T17:46:24.920061+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:46:24.920061+0000","last_undegraded":"2024-08-02T17:46:24.920061+0000","last_fullsized":"2024-08-02T17:46:24.920061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958695+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:46:24.958695+0000","last_peered":"2024-08-02T17:46:24.958695+0000","last_clean":"2024-08-02T17:46:24.958695+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:46:24.958695+0000","last_undegraded":"2024-08-02T17:46:24.958695+0000","last_fullsized":"2024-08-02T17:46:24.958695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919202+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:46:24.919202+0000","last_peered":"2024-08-02T17:46:24.919202+0000","last_clean":"2024-08-02T17:46:24.919202+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:46:24.919202+0000","last_undegraded":"2024-08-02T17:46:24.919202+0000","last_fullsized":"2024-08-02T17:46:24.919202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911667+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:46:24.911667+0000","last_peered":"2024-08-02T17:46:24.911667+0000","last_clean":"2024-08-02T17:46:24.911667+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:46:24.911667+0000","last_undegraded":"2024-08-02T17:46:24.911667+0000","last_fullsized":"2024-08-02T17:46:24.911667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950748+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:46:24.950748+0000","last_peered":"2024-08-02T17:46:24.950748+0000","last_clean":"2024-08-02T17:46:24.950748+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:46:24.950748+0000","last_undegraded":"2024-08-02T17:46:24.950748+0000","last_fullsized":"2024-08-02T17:46:24.950748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915973+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:46:24.915973+0000","last_peered":"2024-08-02T17:46:24.915973+0000","last_clean":"2024-08-02T17:46:24.915973+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:46:24.915973+0000","last_undegraded":"2024-08-02T17:46:24.915973+0000","last_fullsized":"2024-08-02T17:46:24.915973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":149,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.056494+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:46:25.056494+0000","last_peered":"2024-08-02T17:46:25.056494+0000","last_clean":"2024-08-02T17:46:25.056494+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:46:25.056494+0000","last_undegraded":"2024-08-02T17:46:25.056494+0000","last_fullsized":"2024-08-02T17:46:25.056494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.940918+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:46:24.940918+0000","last_peered":"2024-08-02T17:46:24.940918+0000","last_clean":"2024-08-02T17:46:24.940918+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:46:24.940918+0000","last_undegraded":"2024-08-02T17:46:24.940918+0000","last_fullsized":"2024-08-02T17:46:24.940918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915638+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:46:24.915638+0000","last_peered":"2024-08-02T17:46:24.915638+0000","last_clean":"2024-08-02T17:46:24.915638+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:46:24.915638+0000","last_undegraded":"2024-08-02T17:46:24.915638+0000","last_fullsized":"2024-08-02T17:46:24.915638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954294+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:46:24.954294+0000","last_peered":"2024-08-02T17:46:24.954294+0000","last_clean":"2024-08-02T17:46:24.954294+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:46:24.954294+0000","last_undegraded":"2024-08-02T17:46:24.954294+0000","last_fullsized":"2024-08-02T17:46:24.954294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953139+0000","last_change":"2024-08-02T17:45:58.024328+0000","last_active":"2024-08-02T17:46:24.953139+0000","last_peered":"2024-08-02T17:46:24.953139+0000","last_clean":"2024-08-02T17:46:24.953139+0000","last_became_active":"2024-08-02T17:45:58.023931+0000","last_became_peered":"2024-08-02T17:45:58.023931+0000","last_unstale":"2024-08-02T17:46:24.953139+0000","last_undegraded":"2024-08-02T17:46:24.953139+0000","last_fullsized":"2024-08-02T17:46:24.953139+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:52:10.635263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.946206+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:46:24.946206+0000","last_peered":"2024-08-02T17:46:24.946206+0000","last_clean":"2024-08-02T17:46:24.946206+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:46:24.946206+0000","last_undegraded":"2024-08-02T17:46:24.946206+0000","last_fullsized":"2024-08-02T17:46:24.946206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961330+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:46:24.961330+0000","last_peered":"2024-08-02T17:46:24.961330+0000","last_clean":"2024-08-02T17:46:24.961330+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:46:24.961330+0000","last_undegraded":"2024-08-02T17:46:24.961330+0000","last_fullsized":"2024-08-02T17:46:24.961330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948673+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:46:24.948673+0000","last_peered":"2024-08-02T17:46:24.948673+0000","last_clean":"2024-08-02T17:46:24.948673+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:46:24.948673+0000","last_undegraded":"2024-08-02T17:46:24.948673+0000","last_fullsized":"2024-08-02T17:46:24.948673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948711+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:46:24.948711+0000","last_peered":"2024-08-02T17:46:24.948711+0000","last_clean":"2024-08-02T17:46:24.948711+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:46:24.948711+0000","last_undegraded":"2024-08-02T17:46:24.948711+0000","last_fullsized":"2024-08-02T17:46:24.948711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962508+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:46:24.962508+0000","last_peered":"2024-08-02T17:46:24.962508+0000","last_clean":"2024-08-02T17:46:24.962508+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:46:24.962508+0000","last_undegraded":"2024-08-02T17:46:24.962508+0000","last_fullsized":"2024-08-02T17:46:24.962508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949000+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:46:24.949000+0000","last_peered":"2024-08-02T17:46:24.949000+0000","last_clean":"2024-08-02T17:46:24.949000+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:46:24.949000+0000","last_undegraded":"2024-08-02T17:46:24.949000+0000","last_fullsized":"2024-08-02T17:46:24.949000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949651+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:46:24.949651+0000","last_peered":"2024-08-02T17:46:24.949651+0000","last_clean":"2024-08-02T17:46:24.949651+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:46:24.949651+0000","last_undegraded":"2024-08-02T17:46:24.949651+0000","last_fullsized":"2024-08-02T17:46:24.949651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949256+0000","last_change":"2024-08-02T17:45:57.863502+0000","last_active":"2024-08-02T17:46:24.949256+0000","last_peered":"2024-08-02T17:46:24.949256+0000","last_clean":"2024-08-02T17:46:24.949256+0000","last_became_active":"2024-08-02T17:45:57.863230+0000","last_became_peered":"2024-08-02T17:45:57.863230+0000","last_unstale":"2024-08-02T17:46:24.949256+0000","last_undegraded":"2024-08-02T17:46:24.949256+0000","last_fullsized":"2024-08-02T17:46:24.949256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:26:50.915451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962526+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:46:24.962526+0000","last_peered":"2024-08-02T17:46:24.962526+0000","last_clean":"2024-08-02T17:46:24.962526+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:46:24.962526+0000","last_undegraded":"2024-08-02T17:46:24.962526+0000","last_fullsized":"2024-08-02T17:46:24.962526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958665+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:46:24.958665+0000","last_peered":"2024-08-02T17:46:24.958665+0000","last_clean":"2024-08-02T17:46:24.958665+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:46:24.958665+0000","last_undegraded":"2024-08-02T17:46:24.958665+0000","last_fullsized":"2024-08-02T17:46:24.958665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947237+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:46:24.947237+0000","last_peered":"2024-08-02T17:46:24.947237+0000","last_clean":"2024-08-02T17:46:24.947237+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:46:24.947237+0000","last_undegraded":"2024-08-02T17:46:24.947237+0000","last_fullsized":"2024-08-02T17:46:24.947237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.951444+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:46:24.951444+0000","last_peered":"2024-08-02T17:46:24.951444+0000","last_clean":"2024-08-02T17:46:24.951444+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:46:24.951444+0000","last_undegraded":"2024-08-02T17:46:24.951444+0000","last_fullsized":"2024-08-02T17:46:24.951444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962281+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:46:24.962281+0000","last_peered":"2024-08-02T17:46:24.962281+0000","last_clean":"2024-08-02T17:46:24.962281+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:46:24.962281+0000","last_undegraded":"2024-08-02T17:46:24.962281+0000","last_fullsized":"2024-08-02T17:46:24.962281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959483+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:46:24.959483+0000","last_peered":"2024-08-02T17:46:24.959483+0000","last_clean":"2024-08-02T17:46:24.959483+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:46:24.959483+0000","last_undegraded":"2024-08-02T17:46:24.959483+0000","last_fullsized":"2024-08-02T17:46:24.959483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962369+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:46:24.962369+0000","last_peered":"2024-08-02T17:46:24.962369+0000","last_clean":"2024-08-02T17:46:24.962369+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:46:24.962369+0000","last_undegraded":"2024-08-02T17:46:24.962369+0000","last_fullsized":"2024-08-02T17:46:24.962369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959980+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:46:24.959980+0000","last_peered":"2024-08-02T17:46:24.959980+0000","last_clean":"2024-08-02T17:46:24.959980+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:46:24.959980+0000","last_undegraded":"2024-08-02T17:46:24.959980+0000","last_fullsized":"2024-08-02T17:46:24.959980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956497+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:46:24.956497+0000","last_peered":"2024-08-02T17:46:24.956497+0000","last_clean":"2024-08-02T17:46:24.956497+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:46:24.956497+0000","last_undegraded":"2024-08-02T17:46:24.956497+0000","last_fullsized":"2024-08-02T17:46:24.956497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952580+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:46:24.952580+0000","last_peered":"2024-08-02T17:46:24.952580+0000","last_clean":"2024-08-02T17:46:24.952580+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:46:24.952580+0000","last_undegraded":"2024-08-02T17:46:24.952580+0000","last_fullsized":"2024-08-02T17:46:24.952580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952561+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:46:24.952561+0000","last_peered":"2024-08-02T17:46:24.952561+0000","last_clean":"2024-08-02T17:46:24.952561+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:46:24.952561+0000","last_undegraded":"2024-08-02T17:46:24.952561+0000","last_fullsized":"2024-08-02T17:46:24.952561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960139+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:46:24.960139+0000","last_peered":"2024-08-02T17:46:24.960139+0000","last_clean":"2024-08-02T17:46:24.960139+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:46:24.960139+0000","last_undegraded":"2024-08-02T17:46:24.960139+0000","last_fullsized":"2024-08-02T17:46:24.960139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950344+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:46:24.950344+0000","last_peered":"2024-08-02T17:46:24.950344+0000","last_clean":"2024-08-02T17:46:24.950344+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:46:24.950344+0000","last_undegraded":"2024-08-02T17:46:24.950344+0000","last_fullsized":"2024-08-02T17:46:24.950344+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962150+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:46:24.962150+0000","last_peered":"2024-08-02T17:46:24.962150+0000","last_clean":"2024-08-02T17:46:24.962150+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:46:24.962150+0000","last_undegraded":"2024-08-02T17:46:24.962150+0000","last_fullsized":"2024-08-02T17:46:24.962150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958098+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:46:24.958098+0000","last_peered":"2024-08-02T17:46:24.958098+0000","last_clean":"2024-08-02T17:46:24.958098+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:46:24.958098+0000","last_undegraded":"2024-08-02T17:46:24.958098+0000","last_fullsized":"2024-08-02T17:46:24.958098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949100+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:46:24.949100+0000","last_peered":"2024-08-02T17:46:24.949100+0000","last_clean":"2024-08-02T17:46:24.949100+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:46:24.949100+0000","last_undegraded":"2024-08-02T17:46:24.949100+0000","last_fullsized":"2024-08-02T17:46:24.949100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947146+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:46:24.947146+0000","last_peered":"2024-08-02T17:46:24.947146+0000","last_clean":"2024-08-02T17:46:24.947146+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:46:24.947146+0000","last_undegraded":"2024-08-02T17:46:24.947146+0000","last_fullsized":"2024-08-02T17:46:24.947146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949146+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:46:24.949146+0000","last_peered":"2024-08-02T17:46:24.949146+0000","last_clean":"2024-08-02T17:46:24.949146+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:46:24.949146+0000","last_undegraded":"2024-08-02T17:46:24.949146+0000","last_fullsized":"2024-08-02T17:46:24.949146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955968+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:46:24.955968+0000","last_peered":"2024-08-02T17:46:24.955968+0000","last_clean":"2024-08-02T17:46:24.955968+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:46:24.955968+0000","last_undegraded":"2024-08-02T17:46:24.955968+0000","last_fullsized":"2024-08-02T17:46:24.955968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953754+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:46:24.953754+0000","last_peered":"2024-08-02T17:46:24.953754+0000","last_clean":"2024-08-02T17:46:24.953754+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:46:24.953754+0000","last_undegraded":"2024-08-02T17:46:24.953754+0000","last_fullsized":"2024-08-02T17:46:24.953754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957053+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:46:24.957053+0000","last_peered":"2024-08-02T17:46:24.957053+0000","last_clean":"2024-08-02T17:46:24.957053+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:46:24.957053+0000","last_undegraded":"2024-08-02T17:46:24.957053+0000","last_fullsized":"2024-08-02T17:46:24.957053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959352+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:46:24.959352+0000","last_peered":"2024-08-02T17:46:24.959352+0000","last_clean":"2024-08-02T17:46:24.959352+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:46:24.959352+0000","last_undegraded":"2024-08-02T17:46:24.959352+0000","last_fullsized":"2024-08-02T17:46:24.959352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599542,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":713,"num_read_kb":466,"num_write":298,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1798626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449129,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":544040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678061,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37336,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707816,"statfs":{"total":95995035648,"available":95956803584,"internally_reserved":0,"allocated":1925120,"data_stored":1333103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1507328,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":548292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1703936,"data_stored":1142254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":539452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1712128,"data_stored":1132249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":545053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":539646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1179648,"data_stored":608927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":607938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":540840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":203373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:46:30.202 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-02T17:46:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:30.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1000653107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:30.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:29 smithi083 ceph-mon[37031]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-08-02T17:46:30.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 -- 172.21.15.69:0/1696234296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c100970 msgr2=0x7f846c1064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 --2- 172.21.15.69:0/1696234296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c100970 0x7f846c1064a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f845c009900 tx=0x7f845c02a410 comp rx=0 tx=0).stop 2024-08-02T17:46:30.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 -- 172.21.15.69:0/1696234296 shutdown_connections 2024-08-02T17:46:30.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 --2- 172.21.15.69:0/1696234296 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c100970 0x7f846c1064a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 --2- 172.21.15.69:0/1696234296 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f846c106c00 0x7f846c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 --2- 172.21.15.69:0/1696234296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f846c0fffc0 0x7f846c1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 -- 172.21.15.69:0/1696234296 >> 172.21.15.69:0/1696234296 conn(0x7f846c076e20 msgr2=0x7f846c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 -- 172.21.15.69:0/1696234296 shutdown_connections 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.336+0000 7f84738bf640 1 -- 172.21.15.69:0/1696234296 wait complete. 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f84738bf640 1 Processor -- start 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f84738bf640 1 -- start start 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f84738bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f846c0fffc0 0x7f846c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f84738bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c106c00 0x7f846c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f84738bf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f846c19ca90 0x7f846c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f84738bf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f846c111ca0 con 0x7f846c106c00 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f84738bf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f846c111b20 con 0x7f846c19ca90 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f84738bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f846c111e20 con 0x7f846c0fffc0 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8470e5a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c106c00 0x7f846c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:30.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8470e5a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c106c00 0x7f846c198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36488/0 (socket says 172.21.15.69:36488) 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8470e5a640 1 -- 172.21.15.69:0/1475519469 learned_addr learned my addr 172.21.15.69:0/1475519469 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8471e5c640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f846c19ca90 0x7f846c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f847165b640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f846c0fffc0 0x7f846c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8470e5a640 1 -- 172.21.15.69:0/1475519469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f846c0fffc0 msgr2=0x7f846c1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8470e5a640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f846c0fffc0 0x7f846c1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8470e5a640 1 -- 172.21.15.69:0/1475519469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f846c19ca90 msgr2=0x7f846c19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8470e5a640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f846c19ca90 0x7f846c19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8470e5a640 1 -- 172.21.15.69:0/1475519469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c19d580 con 0x7f846c106c00 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.337+0000 7f8471e5c640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f846c19ca90 0x7f846c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.338+0000 7f8470e5a640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c106c00 0x7f846c198710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f845c005940 tx=0x7f845c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.338+0000 7f84627fc640 1 -- 172.21.15.69:0/1475519469 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c03f030 con 0x7f846c106c00 2024-08-02T17:46:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.338+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c19d810 con 0x7f846c106c00 2024-08-02T17:46:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.338+0000 7f84627fc640 1 -- 172.21.15.69:0/1475519469 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f845c003ce0 con 0x7f846c106c00 2024-08-02T17:46:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.338+0000 7f84627fc640 1 -- 172.21.15.69:0/1475519469 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c035070 con 0x7f846c106c00 2024-08-02T17:46:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.338+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a4fd0 con 0x7f846c106c00 2024-08-02T17:46:30.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.339+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f846c1003a0 con 0x7f846c106c00 2024-08-02T17:46:30.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.340+0000 7f84627fc640 1 -- 172.21.15.69:0/1475519469 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f845c042030 con 0x7f846c106c00 2024-08-02T17:46:30.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.340+0000 7f84627fc640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84580a2e20 0x7f84580a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:30.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.341+0000 7f847165b640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84580a2e20 0x7f84580a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:30.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.341+0000 7f84627fc640 1 -- 172.21.15.69:0/1475519469 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f845c0f0280 con 0x7f846c106c00 2024-08-02T17:46:30.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.341+0000 7f847165b640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84580a2e20 0x7f84580a52e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8468005fd0 tx=0x7f8468005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:30.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.344+0000 7f84627fc640 1 -- 172.21.15.69:0/1475519469 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f845c0bc960 con 0x7f846c106c00 2024-08-02T17:46:30.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.conf 2024-08-02T17:46:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi083:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi077:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi069:/etc/ceph/ceph.client.admin.keyring 2024-08-02T17:46:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: Updating smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/config/ceph.client.admin.keyring 2024-08-02T17:46:30.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T17:46:30.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1000653107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:30.349 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:29 smithi077 ceph-mon[37108]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-08-02T17:46:30.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.518+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f846c100970 con 0x7f84580a2e20 2024-08-02T17:46:30.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.526+0000 7f84627fc640 1 -- 172.21.15.69:0/1475519469 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339514 (secure 0 0 0) 0x7f846c100970 con 0x7f84580a2e20 2024-08-02T17:46:30.531 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84580a2e20 msgr2=0x7f84580a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84580a2e20 0x7f84580a52e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8468005fd0 tx=0x7f8468005850 comp rx=0 tx=0).stop 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c106c00 msgr2=0x7f846c198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c106c00 0x7f846c198710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f845c005940 tx=0x7f845c0042f0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 shutdown_connections 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f846c106c00 0x7f846c198710 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84580a2e20 0x7f84580a52e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f846c19ca90 0x7f846c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 --2- 172.21.15.69:0/1475519469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f846c0fffc0 0x7f846c1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 >> 172.21.15.69:0/1475519469 conn(0x7f846c076e20 msgr2=0x7f846c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 shutdown_connections 2024-08-02T17:46:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.529+0000 7f84738bf640 1 -- 172.21.15.69:0/1475519469 wait complete. 2024-08-02T17:46:30.542 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-08-02T17:46:28.936638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190144,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":849,"num_read_kb":585,"num_write":481,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398040,"kb_used_data":15444,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543784,"statfs":{"total":1151940427776,"available":1151532834816,"internally_reserved":0,"allocated":15814656,"data_stored":9023382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":391492921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.008758"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915530+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:46:24.915530+0000","last_peered":"2024-08-02T17:46:24.915530+0000","last_clean":"2024-08-02T17:46:24.915530+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:46:24.915530+0000","last_undegraded":"2024-08-02T17:46:24.915530+0000","last_fullsized":"2024-08-02T17:46:24.915530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954285+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:46:24.954285+0000","last_peered":"2024-08-02T17:46:24.954285+0000","last_clean":"2024-08-02T17:46:24.954285+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:46:24.954285+0000","last_undegraded":"2024-08-02T17:46:24.954285+0000","last_fullsized":"2024-08-02T17:46:24.954285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915172+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:46:24.915172+0000","last_peered":"2024-08-02T17:46:24.915172+0000","last_clean":"2024-08-02T17:46:24.915172+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:46:24.915172+0000","last_undegraded":"2024-08-02T17:46:24.915172+0000","last_fullsized":"2024-08-02T17:46:24.915172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908437+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:46:24.908437+0000","last_peered":"2024-08-02T17:46:24.908437+0000","last_clean":"2024-08-02T17:46:24.908437+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:46:24.908437+0000","last_undegraded":"2024-08-02T17:46:24.908437+0000","last_fullsized":"2024-08-02T17:46:24.908437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959113+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:46:24.959113+0000","last_peered":"2024-08-02T17:46:24.959113+0000","last_clean":"2024-08-02T17:46:24.959113+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:46:24.959113+0000","last_undegraded":"2024-08-02T17:46:24.959113+0000","last_fullsized":"2024-08-02T17:46:24.959113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906783+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:46:24.906783+0000","last_peered":"2024-08-02T17:46:24.906783+0000","last_clean":"2024-08-02T17:46:24.906783+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:46:24.906783+0000","last_undegraded":"2024-08-02T17:46:24.906783+0000","last_fullsized":"2024-08-02T17:46:24.906783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906846+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:46:24.906846+0000","last_peered":"2024-08-02T17:46:24.906846+0000","last_clean":"2024-08-02T17:46:24.906846+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:46:24.906846+0000","last_undegraded":"2024-08-02T17:46:24.906846+0000","last_fullsized":"2024-08-02T17:46:24.906846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962043+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:46:24.962043+0000","last_peered":"2024-08-02T17:46:24.962043+0000","last_clean":"2024-08-02T17:46:24.962043+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:46:24.962043+0000","last_undegraded":"2024-08-02T17:46:24.962043+0000","last_fullsized":"2024-08-02T17:46:24.962043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906895+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:46:24.906895+0000","last_peered":"2024-08-02T17:46:24.906895+0000","last_clean":"2024-08-02T17:46:24.906895+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:46:24.906895+0000","last_undegraded":"2024-08-02T17:46:24.906895+0000","last_fullsized":"2024-08-02T17:46:24.906895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906857+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:46:24.906857+0000","last_peered":"2024-08-02T17:46:24.906857+0000","last_clean":"2024-08-02T17:46:24.906857+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:46:24.906857+0000","last_undegraded":"2024-08-02T17:46:24.906857+0000","last_fullsized":"2024-08-02T17:46:24.906857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959833+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:46:24.959833+0000","last_peered":"2024-08-02T17:46:24.959833+0000","last_clean":"2024-08-02T17:46:24.959833+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:46:24.959833+0000","last_undegraded":"2024-08-02T17:46:24.959833+0000","last_fullsized":"2024-08-02T17:46:24.959833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906814+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:46:24.906814+0000","last_peered":"2024-08-02T17:46:24.906814+0000","last_clean":"2024-08-02T17:46:24.906814+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:46:24.906814+0000","last_undegraded":"2024-08-02T17:46:24.906814+0000","last_fullsized":"2024-08-02T17:46:24.906814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961844+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:46:24.961844+0000","last_peered":"2024-08-02T17:46:24.961844+0000","last_clean":"2024-08-02T17:46:24.961844+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:46:24.961844+0000","last_undegraded":"2024-08-02T17:46:24.961844+0000","last_fullsized":"2024-08-02T17:46:24.961844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906478+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:46:24.906478+0000","last_peered":"2024-08-02T17:46:24.906478+0000","last_clean":"2024-08-02T17:46:24.906478+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:46:24.906478+0000","last_undegraded":"2024-08-02T17:46:24.906478+0000","last_fullsized":"2024-08-02T17:46:24.906478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907202+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:46:24.907202+0000","last_peered":"2024-08-02T17:46:24.907202+0000","last_clean":"2024-08-02T17:46:24.907202+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:46:24.907202+0000","last_undegraded":"2024-08-02T17:46:24.907202+0000","last_fullsized":"2024-08-02T17:46:24.907202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959010+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:46:24.959010+0000","last_peered":"2024-08-02T17:46:24.959010+0000","last_clean":"2024-08-02T17:46:24.959010+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:46:24.959010+0000","last_undegraded":"2024-08-02T17:46:24.959010+0000","last_fullsized":"2024-08-02T17:46:24.959010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920005+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:46:24.920005+0000","last_peered":"2024-08-02T17:46:24.920005+0000","last_clean":"2024-08-02T17:46:24.920005+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:46:24.920005+0000","last_undegraded":"2024-08-02T17:46:24.920005+0000","last_fullsized":"2024-08-02T17:46:24.920005+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907441+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:46:24.907441+0000","last_peered":"2024-08-02T17:46:24.907441+0000","last_clean":"2024-08-02T17:46:24.907441+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:46:24.907441+0000","last_undegraded":"2024-08-02T17:46:24.907441+0000","last_fullsized":"2024-08-02T17:46:24.907441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956539+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:46:24.956539+0000","last_peered":"2024-08-02T17:46:24.956539+0000","last_clean":"2024-08-02T17:46:24.956539+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:46:24.956539+0000","last_undegraded":"2024-08-02T17:46:24.956539+0000","last_fullsized":"2024-08-02T17:46:24.956539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907900+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:46:24.907900+0000","last_peered":"2024-08-02T17:46:24.907900+0000","last_clean":"2024-08-02T17:46:24.907900+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:46:24.907900+0000","last_undegraded":"2024-08-02T17:46:24.907900+0000","last_fullsized":"2024-08-02T17:46:24.907900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915312+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:46:24.915312+0000","last_peered":"2024-08-02T17:46:24.915312+0000","last_clean":"2024-08-02T17:46:24.915312+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:46:24.915312+0000","last_undegraded":"2024-08-02T17:46:24.915312+0000","last_fullsized":"2024-08-02T17:46:24.915312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924149+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:46:24.924149+0000","last_peered":"2024-08-02T17:46:24.924149+0000","last_clean":"2024-08-02T17:46:24.924149+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:46:24.924149+0000","last_undegraded":"2024-08-02T17:46:24.924149+0000","last_fullsized":"2024-08-02T17:46:24.924149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915653+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:46:24.915653+0000","last_peered":"2024-08-02T17:46:24.915653+0000","last_clean":"2024-08-02T17:46:24.915653+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:46:24.915653+0000","last_undegraded":"2024-08-02T17:46:24.915653+0000","last_fullsized":"2024-08-02T17:46:24.915653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907240+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:46:24.907240+0000","last_peered":"2024-08-02T17:46:24.907240+0000","last_clean":"2024-08-02T17:46:24.907240+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:46:24.907240+0000","last_undegraded":"2024-08-02T17:46:24.907240+0000","last_fullsized":"2024-08-02T17:46:24.907240+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961716+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:46:24.961716+0000","last_peered":"2024-08-02T17:46:24.961716+0000","last_clean":"2024-08-02T17:46:24.961716+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:46:24.961716+0000","last_undegraded":"2024-08-02T17:46:24.961716+0000","last_fullsized":"2024-08-02T17:46:24.961716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907489+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:46:24.907489+0000","last_peered":"2024-08-02T17:46:24.907489+0000","last_clean":"2024-08-02T17:46:24.907489+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:46:24.907489+0000","last_undegraded":"2024-08-02T17:46:24.907489+0000","last_fullsized":"2024-08-02T17:46:24.907489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915365+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:46:24.915365+0000","last_peered":"2024-08-02T17:46:24.915365+0000","last_clean":"2024-08-02T17:46:24.915365+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:46:24.915365+0000","last_undegraded":"2024-08-02T17:46:24.915365+0000","last_fullsized":"2024-08-02T17:46:24.915365+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956310+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:46:24.956310+0000","last_peered":"2024-08-02T17:46:24.956310+0000","last_clean":"2024-08-02T17:46:24.956310+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:46:24.956310+0000","last_undegraded":"2024-08-02T17:46:24.956310+0000","last_fullsized":"2024-08-02T17:46:24.956310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'47","reported_seq":122,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.154876+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:46:25.154876+0000","last_peered":"2024-08-02T17:46:25.154876+0000","last_clean":"2024-08-02T17:46:25.154876+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:46:25.154876+0000","last_undegraded":"2024-08-02T17:46:25.154876+0000","last_fullsized":"2024-08-02T17:46:25.154876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907676+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:46:24.907676+0000","last_peered":"2024-08-02T17:46:24.907676+0000","last_clean":"2024-08-02T17:46:24.907676+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.907676+0000","last_undegraded":"2024-08-02T17:46:24.907676+0000","last_fullsized":"2024-08-02T17:46:24.907676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950528+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:46:24.950528+0000","last_peered":"2024-08-02T17:46:24.950528+0000","last_clean":"2024-08-02T17:46:24.950528+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:46:24.950528+0000","last_undegraded":"2024-08-02T17:46:24.950528+0000","last_fullsized":"2024-08-02T17:46:24.950528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906878+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:46:24.906878+0000","last_peered":"2024-08-02T17:46:24.906878+0000","last_clean":"2024-08-02T17:46:24.906878+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:46:24.906878+0000","last_undegraded":"2024-08-02T17:46:24.906878+0000","last_fullsized":"2024-08-02T17:46:24.906878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920332+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:46:24.920332+0000","last_peered":"2024-08-02T17:46:24.920332+0000","last_clean":"2024-08-02T17:46:24.920332+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:46:24.920332+0000","last_undegraded":"2024-08-02T17:46:24.920332+0000","last_fullsized":"2024-08-02T17:46:24.920332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953960+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:46:24.953960+0000","last_peered":"2024-08-02T17:46:24.953960+0000","last_clean":"2024-08-02T17:46:24.953960+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:46:24.953960+0000","last_undegraded":"2024-08-02T17:46:24.953960+0000","last_fullsized":"2024-08-02T17:46:24.953960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915185+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:46:24.915185+0000","last_peered":"2024-08-02T17:46:24.915185+0000","last_clean":"2024-08-02T17:46:24.915185+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:46:24.915185+0000","last_undegraded":"2024-08-02T17:46:24.915185+0000","last_fullsized":"2024-08-02T17:46:24.915185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906282+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:46:24.906282+0000","last_peered":"2024-08-02T17:46:24.906282+0000","last_clean":"2024-08-02T17:46:24.906282+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:46:24.906282+0000","last_undegraded":"2024-08-02T17:46:24.906282+0000","last_fullsized":"2024-08-02T17:46:24.906282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956518+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:46:24.956518+0000","last_peered":"2024-08-02T17:46:24.956518+0000","last_clean":"2024-08-02T17:46:24.956518+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:46:24.956518+0000","last_undegraded":"2024-08-02T17:46:24.956518+0000","last_fullsized":"2024-08-02T17:46:24.956518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.909639+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:46:24.909639+0000","last_peered":"2024-08-02T17:46:24.909639+0000","last_clean":"2024-08-02T17:46:24.909639+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:46:24.909639+0000","last_undegraded":"2024-08-02T17:46:24.909639+0000","last_fullsized":"2024-08-02T17:46:24.909639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.918812+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:46:24.918812+0000","last_peered":"2024-08-02T17:46:24.918812+0000","last_clean":"2024-08-02T17:46:24.918812+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:46:24.918812+0000","last_undegraded":"2024-08-02T17:46:24.918812+0000","last_fullsized":"2024-08-02T17:46:24.918812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952882+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:46:24.952882+0000","last_peered":"2024-08-02T17:46:24.952882+0000","last_clean":"2024-08-02T17:46:24.952882+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:46:24.952882+0000","last_undegraded":"2024-08-02T17:46:24.952882+0000","last_fullsized":"2024-08-02T17:46:24.952882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":143,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919340+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:46:24.919340+0000","last_peered":"2024-08-02T17:46:24.919340+0000","last_clean":"2024-08-02T17:46:24.919340+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:46:24.919340+0000","last_undegraded":"2024-08-02T17:46:24.919340+0000","last_fullsized":"2024-08-02T17:46:24.919340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919707+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:46:24.919707+0000","last_peered":"2024-08-02T17:46:24.919707+0000","last_clean":"2024-08-02T17:46:24.919707+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.919707+0000","last_undegraded":"2024-08-02T17:46:24.919707+0000","last_fullsized":"2024-08-02T17:46:24.919707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947422+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:46:24.947422+0000","last_peered":"2024-08-02T17:46:24.947422+0000","last_clean":"2024-08-02T17:46:24.947422+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:46:24.947422+0000","last_undegraded":"2024-08-02T17:46:24.947422+0000","last_fullsized":"2024-08-02T17:46:24.947422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908927+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:46:24.908927+0000","last_peered":"2024-08-02T17:46:24.908927+0000","last_clean":"2024-08-02T17:46:24.908927+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:46:24.908927+0000","last_undegraded":"2024-08-02T17:46:24.908927+0000","last_fullsized":"2024-08-02T17:46:24.908927+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915349+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:46:24.915349+0000","last_peered":"2024-08-02T17:46:24.915349+0000","last_clean":"2024-08-02T17:46:24.915349+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:46:24.915349+0000","last_undegraded":"2024-08-02T17:46:24.915349+0000","last_fullsized":"2024-08-02T17:46:24.915349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962129+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:46:24.962129+0000","last_peered":"2024-08-02T17:46:24.962129+0000","last_clean":"2024-08-02T17:46:24.962129+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:46:24.962129+0000","last_undegraded":"2024-08-02T17:46:24.962129+0000","last_fullsized":"2024-08-02T17:46:24.962129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907233+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:46:24.907233+0000","last_peered":"2024-08-02T17:46:24.907233+0000","last_clean":"2024-08-02T17:46:24.907233+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:46:24.907233+0000","last_undegraded":"2024-08-02T17:46:24.907233+0000","last_fullsized":"2024-08-02T17:46:24.907233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908841+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:46:24.908841+0000","last_peered":"2024-08-02T17:46:24.908841+0000","last_clean":"2024-08-02T17:46:24.908841+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:46:24.908841+0000","last_undegraded":"2024-08-02T17:46:24.908841+0000","last_fullsized":"2024-08-02T17:46:24.908841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910641+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:46:24.910641+0000","last_peered":"2024-08-02T17:46:24.910641+0000","last_clean":"2024-08-02T17:46:24.910641+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:46:24.910641+0000","last_undegraded":"2024-08-02T17:46:24.910641+0000","last_fullsized":"2024-08-02T17:46:24.910641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960428+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:46:24.960428+0000","last_peered":"2024-08-02T17:46:24.960428+0000","last_clean":"2024-08-02T17:46:24.960428+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:46:24.960428+0000","last_undegraded":"2024-08-02T17:46:24.960428+0000","last_fullsized":"2024-08-02T17:46:24.960428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.916318+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:46:24.916318+0000","last_peered":"2024-08-02T17:46:24.916318+0000","last_clean":"2024-08-02T17:46:24.916318+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:46:24.916318+0000","last_undegraded":"2024-08-02T17:46:24.916318+0000","last_fullsized":"2024-08-02T17:46:24.916318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915831+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:46:24.915831+0000","last_peered":"2024-08-02T17:46:24.915831+0000","last_clean":"2024-08-02T17:46:24.915831+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:46:24.915831+0000","last_undegraded":"2024-08-02T17:46:24.915831+0000","last_fullsized":"2024-08-02T17:46:24.915831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948829+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:46:24.948829+0000","last_peered":"2024-08-02T17:46:24.948829+0000","last_clean":"2024-08-02T17:46:24.948829+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:46:24.948829+0000","last_undegraded":"2024-08-02T17:46:24.948829+0000","last_fullsized":"2024-08-02T17:46:24.948829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":87,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.150408+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:46:25.150408+0000","last_peered":"2024-08-02T17:46:25.150408+0000","last_clean":"2024-08-02T17:46:25.150408+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:46:25.150408+0000","last_undegraded":"2024-08-02T17:46:25.150408+0000","last_fullsized":"2024-08-02T17:46:25.150408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924774+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:46:24.924774+0000","last_peered":"2024-08-02T17:46:24.924774+0000","last_clean":"2024-08-02T17:46:24.924774+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:46:24.924774+0000","last_undegraded":"2024-08-02T17:46:24.924774+0000","last_fullsized":"2024-08-02T17:46:24.924774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955439+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:46:24.955439+0000","last_peered":"2024-08-02T17:46:24.955439+0000","last_clean":"2024-08-02T17:46:24.955439+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:46:24.955439+0000","last_undegraded":"2024-08-02T17:46:24.955439+0000","last_fullsized":"2024-08-02T17:46:24.955439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919542+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:46:24.919542+0000","last_peered":"2024-08-02T17:46:24.919542+0000","last_clean":"2024-08-02T17:46:24.919542+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:46:24.919542+0000","last_undegraded":"2024-08-02T17:46:24.919542+0000","last_fullsized":"2024-08-02T17:46:24.919542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914656+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:46:24.914656+0000","last_peered":"2024-08-02T17:46:24.914656+0000","last_clean":"2024-08-02T17:46:24.914656+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:46:24.914656+0000","last_undegraded":"2024-08-02T17:46:24.914656+0000","last_fullsized":"2024-08-02T17:46:24.914656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950756+0000","last_change":"2024-08-02T17:45:57.979722+0000","last_active":"2024-08-02T17:46:24.950756+0000","last_peered":"2024-08-02T17:46:24.950756+0000","last_clean":"2024-08-02T17:46:24.950756+0000","last_became_active":"2024-08-02T17:45:57.979427+0000","last_became_peered":"2024-08-02T17:45:57.979427+0000","last_unstale":"2024-08-02T17:46:24.950756+0000","last_undegraded":"2024-08-02T17:46:24.950756+0000","last_fullsized":"2024-08-02T17:46:24.950756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:16:52.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.920115+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:46:25.920115+0000","last_peered":"2024-08-02T17:46:25.920115+0000","last_clean":"2024-08-02T17:46:25.920115+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:46:25.920115+0000","last_undegraded":"2024-08-02T17:46:25.920115+0000","last_fullsized":"2024-08-02T17:46:25.920115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914880+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:46:24.914880+0000","last_peered":"2024-08-02T17:46:24.914880+0000","last_clean":"2024-08-02T17:46:24.914880+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:46:24.914880+0000","last_undegraded":"2024-08-02T17:46:24.914880+0000","last_fullsized":"2024-08-02T17:46:24.914880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953834+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:46:24.953834+0000","last_peered":"2024-08-02T17:46:24.953834+0000","last_clean":"2024-08-02T17:46:24.953834+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:46:24.953834+0000","last_undegraded":"2024-08-02T17:46:24.953834+0000","last_fullsized":"2024-08-02T17:46:24.953834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906346+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:46:24.906346+0000","last_peered":"2024-08-02T17:46:24.906346+0000","last_clean":"2024-08-02T17:46:24.906346+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:46:24.906346+0000","last_undegraded":"2024-08-02T17:46:24.906346+0000","last_fullsized":"2024-08-02T17:46:24.906346+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919446+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:46:24.919446+0000","last_peered":"2024-08-02T17:46:24.919446+0000","last_clean":"2024-08-02T17:46:24.919446+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:46:24.919446+0000","last_undegraded":"2024-08-02T17:46:24.919446+0000","last_fullsized":"2024-08-02T17:46:24.919446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950106+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:46:24.950106+0000","last_peered":"2024-08-02T17:46:24.950106+0000","last_clean":"2024-08-02T17:46:24.950106+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:46:24.950106+0000","last_undegraded":"2024-08-02T17:46:24.950106+0000","last_fullsized":"2024-08-02T17:46:24.950106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":152,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.202288+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:46:25.202288+0000","last_peered":"2024-08-02T17:46:25.202288+0000","last_clean":"2024-08-02T17:46:25.202288+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:46:25.202288+0000","last_undegraded":"2024-08-02T17:46:25.202288+0000","last_fullsized":"2024-08-02T17:46:25.202288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2207,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":6,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915368+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:46:24.915368+0000","last_peered":"2024-08-02T17:46:24.915368+0000","last_clean":"2024-08-02T17:46:24.915368+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:46:24.915368+0000","last_undegraded":"2024-08-02T17:46:24.915368+0000","last_fullsized":"2024-08-02T17:46:24.915368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957938+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:46:24.957938+0000","last_peered":"2024-08-02T17:46:24.957938+0000","last_clean":"2024-08-02T17:46:24.957938+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:46:24.957938+0000","last_undegraded":"2024-08-02T17:46:24.957938+0000","last_fullsized":"2024-08-02T17:46:24.957938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915099+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:46:24.915099+0000","last_peered":"2024-08-02T17:46:24.915099+0000","last_clean":"2024-08-02T17:46:24.915099+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:46:24.915099+0000","last_undegraded":"2024-08-02T17:46:24.915099+0000","last_fullsized":"2024-08-02T17:46:24.915099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920404+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:46:24.920404+0000","last_peered":"2024-08-02T17:46:24.920404+0000","last_clean":"2024-08-02T17:46:24.920404+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:46:24.920404+0000","last_undegraded":"2024-08-02T17:46:24.920404+0000","last_fullsized":"2024-08-02T17:46:24.920404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958381+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:46:24.958381+0000","last_peered":"2024-08-02T17:46:24.958381+0000","last_clean":"2024-08-02T17:46:24.958381+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:46:24.958381+0000","last_undegraded":"2024-08-02T17:46:24.958381+0000","last_fullsized":"2024-08-02T17:46:24.958381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915872+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:46:24.915872+0000","last_peered":"2024-08-02T17:46:24.915872+0000","last_clean":"2024-08-02T17:46:24.915872+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:46:24.915872+0000","last_undegraded":"2024-08-02T17:46:24.915872+0000","last_fullsized":"2024-08-02T17:46:24.915872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915458+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:46:24.915458+0000","last_peered":"2024-08-02T17:46:24.915458+0000","last_clean":"2024-08-02T17:46:24.915458+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:46:24.915458+0000","last_undegraded":"2024-08-02T17:46:24.915458+0000","last_fullsized":"2024-08-02T17:46:24.915458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955734+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:46:24.955734+0000","last_peered":"2024-08-02T17:46:24.955734+0000","last_clean":"2024-08-02T17:46:24.955734+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:46:24.955734+0000","last_undegraded":"2024-08-02T17:46:24.955734+0000","last_fullsized":"2024-08-02T17:46:24.955734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'38","reported_seq":285,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.207389+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:46:25.207389+0000","last_peered":"2024-08-02T17:46:25.207389+0000","last_clean":"2024-08-02T17:46:25.207389+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:46:25.207389+0000","last_undegraded":"2024-08-02T17:46:25.207389+0000","last_fullsized":"2024-08-02T17:46:25.207389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68576,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":168,"num_write":39,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911360+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:46:24.911360+0000","last_peered":"2024-08-02T17:46:24.911360+0000","last_clean":"2024-08-02T17:46:24.911360+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:46:24.911360+0000","last_undegraded":"2024-08-02T17:46:24.911360+0000","last_fullsized":"2024-08-02T17:46:24.911360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959993+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:46:24.959993+0000","last_peered":"2024-08-02T17:46:24.959993+0000","last_clean":"2024-08-02T17:46:24.959993+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:46:24.959993+0000","last_undegraded":"2024-08-02T17:46:24.959993+0000","last_fullsized":"2024-08-02T17:46:24.959993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915691+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:46:24.915691+0000","last_peered":"2024-08-02T17:46:24.915691+0000","last_clean":"2024-08-02T17:46:24.915691+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:46:24.915691+0000","last_undegraded":"2024-08-02T17:46:24.915691+0000","last_fullsized":"2024-08-02T17:46:24.915691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907020+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:46:24.907020+0000","last_peered":"2024-08-02T17:46:24.907020+0000","last_clean":"2024-08-02T17:46:24.907020+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:46:24.907020+0000","last_undegraded":"2024-08-02T17:46:24.907020+0000","last_fullsized":"2024-08-02T17:46:24.907020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960499+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:46:24.960499+0000","last_peered":"2024-08-02T17:46:24.960499+0000","last_clean":"2024-08-02T17:46:24.960499+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:46:24.960499+0000","last_undegraded":"2024-08-02T17:46:24.960499+0000","last_fullsized":"2024-08-02T17:46:24.960499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910476+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:46:24.910476+0000","last_peered":"2024-08-02T17:46:24.910476+0000","last_clean":"2024-08-02T17:46:24.910476+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:46:24.910476+0000","last_undegraded":"2024-08-02T17:46:24.910476+0000","last_fullsized":"2024-08-02T17:46:24.910476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914823+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:46:24.914823+0000","last_peered":"2024-08-02T17:46:24.914823+0000","last_clean":"2024-08-02T17:46:24.914823+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:46:24.914823+0000","last_undegraded":"2024-08-02T17:46:24.914823+0000","last_fullsized":"2024-08-02T17:46:24.914823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962330+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:46:24.962330+0000","last_peered":"2024-08-02T17:46:24.962330+0000","last_clean":"2024-08-02T17:46:24.962330+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:46:24.962330+0000","last_undegraded":"2024-08-02T17:46:24.962330+0000","last_fullsized":"2024-08-02T17:46:24.962330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":143,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.055690+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:46:25.055690+0000","last_peered":"2024-08-02T17:46:25.055690+0000","last_clean":"2024-08-02T17:46:25.055690+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:46:25.055690+0000","last_undegraded":"2024-08-02T17:46:25.055690+0000","last_fullsized":"2024-08-02T17:46:25.055690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907780+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:46:24.907780+0000","last_peered":"2024-08-02T17:46:24.907780+0000","last_clean":"2024-08-02T17:46:24.907780+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:46:24.907780+0000","last_undegraded":"2024-08-02T17:46:24.907780+0000","last_fullsized":"2024-08-02T17:46:24.907780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.926268+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:46:24.926268+0000","last_peered":"2024-08-02T17:46:24.926268+0000","last_clean":"2024-08-02T17:46:24.926268+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:46:24.926268+0000","last_undegraded":"2024-08-02T17:46:24.926268+0000","last_fullsized":"2024-08-02T17:46:24.926268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956250+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:46:24.956250+0000","last_peered":"2024-08-02T17:46:24.956250+0000","last_clean":"2024-08-02T17:46:24.956250+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:46:24.956250+0000","last_undegraded":"2024-08-02T17:46:24.956250+0000","last_fullsized":"2024-08-02T17:46:24.956250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914914+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:46:24.914914+0000","last_peered":"2024-08-02T17:46:24.914914+0000","last_clean":"2024-08-02T17:46:24.914914+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:46:24.914914+0000","last_undegraded":"2024-08-02T17:46:24.914914+0000","last_fullsized":"2024-08-02T17:46:24.914914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920061+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:46:24.920061+0000","last_peered":"2024-08-02T17:46:24.920061+0000","last_clean":"2024-08-02T17:46:24.920061+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:46:24.920061+0000","last_undegraded":"2024-08-02T17:46:24.920061+0000","last_fullsized":"2024-08-02T17:46:24.920061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958695+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:46:24.958695+0000","last_peered":"2024-08-02T17:46:24.958695+0000","last_clean":"2024-08-02T17:46:24.958695+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:46:24.958695+0000","last_undegraded":"2024-08-02T17:46:24.958695+0000","last_fullsized":"2024-08-02T17:46:24.958695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919202+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:46:24.919202+0000","last_peered":"2024-08-02T17:46:24.919202+0000","last_clean":"2024-08-02T17:46:24.919202+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:46:24.919202+0000","last_undegraded":"2024-08-02T17:46:24.919202+0000","last_fullsized":"2024-08-02T17:46:24.919202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911667+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:46:24.911667+0000","last_peered":"2024-08-02T17:46:24.911667+0000","last_clean":"2024-08-02T17:46:24.911667+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:46:24.911667+0000","last_undegraded":"2024-08-02T17:46:24.911667+0000","last_fullsized":"2024-08-02T17:46:24.911667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950748+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:46:24.950748+0000","last_peered":"2024-08-02T17:46:24.950748+0000","last_clean":"2024-08-02T17:46:24.950748+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:46:24.950748+0000","last_undegraded":"2024-08-02T17:46:24.950748+0000","last_fullsized":"2024-08-02T17:46:24.950748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915973+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:46:24.915973+0000","last_peered":"2024-08-02T17:46:24.915973+0000","last_clean":"2024-08-02T17:46:24.915973+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:46:24.915973+0000","last_undegraded":"2024-08-02T17:46:24.915973+0000","last_fullsized":"2024-08-02T17:46:24.915973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":149,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.056494+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:46:25.056494+0000","last_peered":"2024-08-02T17:46:25.056494+0000","last_clean":"2024-08-02T17:46:25.056494+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:46:25.056494+0000","last_undegraded":"2024-08-02T17:46:25.056494+0000","last_fullsized":"2024-08-02T17:46:25.056494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.940918+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:46:24.940918+0000","last_peered":"2024-08-02T17:46:24.940918+0000","last_clean":"2024-08-02T17:46:24.940918+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:46:24.940918+0000","last_undegraded":"2024-08-02T17:46:24.940918+0000","last_fullsized":"2024-08-02T17:46:24.940918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915638+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:46:24.915638+0000","last_peered":"2024-08-02T17:46:24.915638+0000","last_clean":"2024-08-02T17:46:24.915638+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:46:24.915638+0000","last_undegraded":"2024-08-02T17:46:24.915638+0000","last_fullsized":"2024-08-02T17:46:24.915638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954294+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:46:24.954294+0000","last_peered":"2024-08-02T17:46:24.954294+0000","last_clean":"2024-08-02T17:46:24.954294+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:46:24.954294+0000","last_undegraded":"2024-08-02T17:46:24.954294+0000","last_fullsized":"2024-08-02T17:46:24.954294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953139+0000","last_change":"2024-08-02T17:45:58.024328+0000","last_active":"2024-08-02T17:46:24.953139+0000","last_peered":"2024-08-02T17:46:24.953139+0000","last_clean":"2024-08-02T17:46:24.953139+0000","last_became_active":"2024-08-02T17:45:58.023931+0000","last_became_peered":"2024-08-02T17:45:58.023931+0000","last_unstale":"2024-08-02T17:46:24.953139+0000","last_undegraded":"2024-08-02T17:46:24.953139+0000","last_fullsized":"2024-08-02T17:46:24.953139+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:52:10.635263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.946206+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:46:24.946206+0000","last_peered":"2024-08-02T17:46:24.946206+0000","last_clean":"2024-08-02T17:46:24.946206+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:46:24.946206+0000","last_undegraded":"2024-08-02T17:46:24.946206+0000","last_fullsized":"2024-08-02T17:46:24.946206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961330+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:46:24.961330+0000","last_peered":"2024-08-02T17:46:24.961330+0000","last_clean":"2024-08-02T17:46:24.961330+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:46:24.961330+0000","last_undegraded":"2024-08-02T17:46:24.961330+0000","last_fullsized":"2024-08-02T17:46:24.961330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948673+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:46:24.948673+0000","last_peered":"2024-08-02T17:46:24.948673+0000","last_clean":"2024-08-02T17:46:24.948673+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:46:24.948673+0000","last_undegraded":"2024-08-02T17:46:24.948673+0000","last_fullsized":"2024-08-02T17:46:24.948673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948711+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:46:24.948711+0000","last_peered":"2024-08-02T17:46:24.948711+0000","last_clean":"2024-08-02T17:46:24.948711+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:46:24.948711+0000","last_undegraded":"2024-08-02T17:46:24.948711+0000","last_fullsized":"2024-08-02T17:46:24.948711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962508+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:46:24.962508+0000","last_peered":"2024-08-02T17:46:24.962508+0000","last_clean":"2024-08-02T17:46:24.962508+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:46:24.962508+0000","last_undegraded":"2024-08-02T17:46:24.962508+0000","last_fullsized":"2024-08-02T17:46:24.962508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949000+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:46:24.949000+0000","last_peered":"2024-08-02T17:46:24.949000+0000","last_clean":"2024-08-02T17:46:24.949000+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:46:24.949000+0000","last_undegraded":"2024-08-02T17:46:24.949000+0000","last_fullsized":"2024-08-02T17:46:24.949000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949651+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:46:24.949651+0000","last_peered":"2024-08-02T17:46:24.949651+0000","last_clean":"2024-08-02T17:46:24.949651+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:46:24.949651+0000","last_undegraded":"2024-08-02T17:46:24.949651+0000","last_fullsized":"2024-08-02T17:46:24.949651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949256+0000","last_change":"2024-08-02T17:45:57.863502+0000","last_active":"2024-08-02T17:46:24.949256+0000","last_peered":"2024-08-02T17:46:24.949256+0000","last_clean":"2024-08-02T17:46:24.949256+0000","last_became_active":"2024-08-02T17:45:57.863230+0000","last_became_peered":"2024-08-02T17:45:57.863230+0000","last_unstale":"2024-08-02T17:46:24.949256+0000","last_undegraded":"2024-08-02T17:46:24.949256+0000","last_fullsized":"2024-08-02T17:46:24.949256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:26:50.915451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962526+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:46:24.962526+0000","last_peered":"2024-08-02T17:46:24.962526+0000","last_clean":"2024-08-02T17:46:24.962526+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:46:24.962526+0000","last_undegraded":"2024-08-02T17:46:24.962526+0000","last_fullsized":"2024-08-02T17:46:24.962526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958665+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:46:24.958665+0000","last_peered":"2024-08-02T17:46:24.958665+0000","last_clean":"2024-08-02T17:46:24.958665+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:46:24.958665+0000","last_undegraded":"2024-08-02T17:46:24.958665+0000","last_fullsized":"2024-08-02T17:46:24.958665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947237+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:46:24.947237+0000","last_peered":"2024-08-02T17:46:24.947237+0000","last_clean":"2024-08-02T17:46:24.947237+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:46:24.947237+0000","last_undegraded":"2024-08-02T17:46:24.947237+0000","last_fullsized":"2024-08-02T17:46:24.947237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.951444+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:46:24.951444+0000","last_peered":"2024-08-02T17:46:24.951444+0000","last_clean":"2024-08-02T17:46:24.951444+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:46:24.951444+0000","last_undegraded":"2024-08-02T17:46:24.951444+0000","last_fullsized":"2024-08-02T17:46:24.951444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962281+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:46:24.962281+0000","last_peered":"2024-08-02T17:46:24.962281+0000","last_clean":"2024-08-02T17:46:24.962281+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:46:24.962281+0000","last_undegraded":"2024-08-02T17:46:24.962281+0000","last_fullsized":"2024-08-02T17:46:24.962281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959483+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:46:24.959483+0000","last_peered":"2024-08-02T17:46:24.959483+0000","last_clean":"2024-08-02T17:46:24.959483+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:46:24.959483+0000","last_undegraded":"2024-08-02T17:46:24.959483+0000","last_fullsized":"2024-08-02T17:46:24.959483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962369+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:46:24.962369+0000","last_peered":"2024-08-02T17:46:24.962369+0000","last_clean":"2024-08-02T17:46:24.962369+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:46:24.962369+0000","last_undegraded":"2024-08-02T17:46:24.962369+0000","last_fullsized":"2024-08-02T17:46:24.962369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959980+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:46:24.959980+0000","last_peered":"2024-08-02T17:46:24.959980+0000","last_clean":"2024-08-02T17:46:24.959980+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:46:24.959980+0000","last_undegraded":"2024-08-02T17:46:24.959980+0000","last_fullsized":"2024-08-02T17:46:24.959980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956497+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:46:24.956497+0000","last_peered":"2024-08-02T17:46:24.956497+0000","last_clean":"2024-08-02T17:46:24.956497+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:46:24.956497+0000","last_undegraded":"2024-08-02T17:46:24.956497+0000","last_fullsized":"2024-08-02T17:46:24.956497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952580+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:46:24.952580+0000","last_peered":"2024-08-02T17:46:24.952580+0000","last_clean":"2024-08-02T17:46:24.952580+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:46:24.952580+0000","last_undegraded":"2024-08-02T17:46:24.952580+0000","last_fullsized":"2024-08-02T17:46:24.952580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952561+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:46:24.952561+0000","last_peered":"2024-08-02T17:46:24.952561+0000","last_clean":"2024-08-02T17:46:24.952561+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:46:24.952561+0000","last_undegraded":"2024-08-02T17:46:24.952561+0000","last_fullsized":"2024-08-02T17:46:24.952561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960139+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:46:24.960139+0000","last_peered":"2024-08-02T17:46:24.960139+0000","last_clean":"2024-08-02T17:46:24.960139+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:46:24.960139+0000","last_undegraded":"2024-08-02T17:46:24.960139+0000","last_fullsized":"2024-08-02T17:46:24.960139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950344+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:46:24.950344+0000","last_peered":"2024-08-02T17:46:24.950344+0000","last_clean":"2024-08-02T17:46:24.950344+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:46:24.950344+0000","last_undegraded":"2024-08-02T17:46:24.950344+0000","last_fullsized":"2024-08-02T17:46:24.950344+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962150+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:46:24.962150+0000","last_peered":"2024-08-02T17:46:24.962150+0000","last_clean":"2024-08-02T17:46:24.962150+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:46:24.962150+0000","last_undegraded":"2024-08-02T17:46:24.962150+0000","last_fullsized":"2024-08-02T17:46:24.962150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958098+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:46:24.958098+0000","last_peered":"2024-08-02T17:46:24.958098+0000","last_clean":"2024-08-02T17:46:24.958098+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:46:24.958098+0000","last_undegraded":"2024-08-02T17:46:24.958098+0000","last_fullsized":"2024-08-02T17:46:24.958098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949100+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:46:24.949100+0000","last_peered":"2024-08-02T17:46:24.949100+0000","last_clean":"2024-08-02T17:46:24.949100+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:46:24.949100+0000","last_undegraded":"2024-08-02T17:46:24.949100+0000","last_fullsized":"2024-08-02T17:46:24.949100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947146+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:46:24.947146+0000","last_peered":"2024-08-02T17:46:24.947146+0000","last_clean":"2024-08-02T17:46:24.947146+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:46:24.947146+0000","last_undegraded":"2024-08-02T17:46:24.947146+0000","last_fullsized":"2024-08-02T17:46:24.947146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949146+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:46:24.949146+0000","last_peered":"2024-08-02T17:46:24.949146+0000","last_clean":"2024-08-02T17:46:24.949146+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:46:24.949146+0000","last_undegraded":"2024-08-02T17:46:24.949146+0000","last_fullsized":"2024-08-02T17:46:24.949146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955968+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:46:24.955968+0000","last_peered":"2024-08-02T17:46:24.955968+0000","last_clean":"2024-08-02T17:46:24.955968+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:46:24.955968+0000","last_undegraded":"2024-08-02T17:46:24.955968+0000","last_fullsized":"2024-08-02T17:46:24.955968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953754+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:46:24.953754+0000","last_peered":"2024-08-02T17:46:24.953754+0000","last_clean":"2024-08-02T17:46:24.953754+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:46:24.953754+0000","last_undegraded":"2024-08-02T17:46:24.953754+0000","last_fullsized":"2024-08-02T17:46:24.953754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957053+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:46:24.957053+0000","last_peered":"2024-08-02T17:46:24.957053+0000","last_clean":"2024-08-02T17:46:24.957053+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:46:24.957053+0000","last_undegraded":"2024-08-02T17:46:24.957053+0000","last_fullsized":"2024-08-02T17:46:24.957053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959352+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:46:24.959352+0000","last_peered":"2024-08-02T17:46:24.959352+0000","last_clean":"2024-08-02T17:46:24.959352+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:46:24.959352+0000","last_undegraded":"2024-08-02T17:46:24.959352+0000","last_fullsized":"2024-08-02T17:46:24.959352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599542,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":713,"num_read_kb":466,"num_write":298,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1798626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449129,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":544040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678061,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37336,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707816,"statfs":{"total":95995035648,"available":95956803584,"internally_reserved":0,"allocated":1925120,"data_stored":1333103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1507328,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":548292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1703936,"data_stored":1142254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":539452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1712128,"data_stored":1132249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":545053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":539646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1179648,"data_stored":608927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":607938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":540840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":203373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:46:30.547 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-02T17:46:30.547 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 -- 172.21.15.69:0/669944726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4100710 msgr2=0x7f9da4100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 --2- 172.21.15.69:0/669944726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4100710 0x7f9da4100b70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9d90009930 tx=0x7f9d9002a420 comp rx=0 tx=0).stop 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 -- 172.21.15.69:0/669944726 shutdown_connections 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 --2- 172.21.15.69:0/669944726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4100710 0x7f9da4100b70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 --2- 172.21.15.69:0/669944726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9da41010b0 0x7f9da410d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 --2- 172.21.15.69:0/669944726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da4106720 0x7f9da4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 -- 172.21.15.69:0/669944726 >> 172.21.15.69:0/669944726 conn(0x7f9da40fc3f0 msgr2=0x7f9da40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 -- 172.21.15.69:0/669944726 shutdown_connections 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.682+0000 7f9daac23640 1 -- 172.21.15.69:0/669944726 wait complete. 2024-08-02T17:46:30.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9daac23640 1 Processor -- start 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9daac23640 1 -- start start 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9daac23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da4100710 0x7f9da4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9daac23640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9da41010b0 0x7f9da41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9daac23640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4106720 0x7f9da419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9daac23640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4110b60 con 0x7f9da4106720 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9daac23640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9da41109e0 con 0x7f9da41010b0 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9daac23640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9da4110ce0 con 0x7f9da4100710 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4106720 0x7f9da419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4106720 0x7f9da419c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36494/0 (socket says 172.21.15.69:36494) 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 -- 172.21.15.69:0/2069355499 learned_addr learned my addr 172.21.15.69:0/2069355499 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da3fff640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9da41010b0 0x7f9da41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da89bf640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da4100710 0x7f9da4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:30.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 -- 172.21.15.69:0/2069355499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da4100710 msgr2=0x7f9da4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da4100710 0x7f9da4198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 -- 172.21.15.69:0/2069355499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9da41010b0 msgr2=0x7f9da41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9da41010b0 0x7f9da41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 -- 172.21.15.69:0/2069355499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da419d050 con 0x7f9da4106720 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da3fff640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9da41010b0 0x7f9da41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.683+0000 7f9da91c0640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4106720 0x7f9da419c920 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00cce0 tx=0x7f9d9c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.684+0000 7f9da1ffb640 1 -- 172.21.15.69:0/2069355499 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c00f450 con 0x7f9da4106720 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.684+0000 7f9da1ffb640 1 -- 172.21.15.69:0/2069355499 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9c00fad0 con 0x7f9da4106720 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.684+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da419d340 con 0x7f9da4106720 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.684+0000 7f9da1ffb640 1 -- 172.21.15.69:0/2069355499 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c018c10 con 0x7f9da4106720 2024-08-02T17:46:30.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.684+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da41a4bd0 con 0x7f9da4106720 2024-08-02T17:46:30.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.685+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da4100b70 con 0x7f9da4106720 2024-08-02T17:46:30.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.686+0000 7f9da1ffb640 1 -- 172.21.15.69:0/2069355499 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f9d9c00f5f0 con 0x7f9da4106720 2024-08-02T17:46:30.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.686+0000 7f9da1ffb640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d940a2e20 0x7f9d940a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:30.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.686+0000 7f9da89bf640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d940a2e20 0x7f9d940a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:30.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.687+0000 7f9da1ffb640 1 -- 172.21.15.69:0/2069355499 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f9d9c0d0230 con 0x7f9da4106720 2024-08-02T17:46:30.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.687+0000 7f9da89bf640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d940a2e20 0x7f9d940a52e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800b9c0 tx=0x7f9d98005c50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:30.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.690+0000 7f9da1ffb640 1 -- 172.21.15.69:0/2069355499 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9d9c014030 con 0x7f9da4106720 2024-08-02T17:46:30.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.862+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9da4109cf0 con 0x7f9d940a2e20 2024-08-02T17:46:30.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.870+0000 7f9da1ffb640 1 -- 172.21.15.69:0/2069355499 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339514 (secure 0 0 0) 0x7f9da4109cf0 con 0x7f9d940a2e20 2024-08-02T17:46:30.875 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:30.877 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:46:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.873+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d940a2e20 msgr2=0x7f9d940a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.873+0000 7f9daac23640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d940a2e20 0x7f9d940a52e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800b9c0 tx=0x7f9d98005c50 comp rx=0 tx=0).stop 2024-08-02T17:46:30.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.873+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4106720 msgr2=0x7f9da419c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.873+0000 7f9daac23640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4106720 0x7f9da419c920 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00cce0 tx=0x7f9d9c00a620 comp rx=0 tx=0).stop 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.874+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 shutdown_connections 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.874+0000 7f9daac23640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9da4106720 0x7f9da419c920 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.874+0000 7f9daac23640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d940a2e20 0x7f9d940a52e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.874+0000 7f9daac23640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9da41010b0 0x7f9da41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.874+0000 7f9daac23640 1 --2- 172.21.15.69:0/2069355499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da4100710 0x7f9da4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.874+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 >> 172.21.15.69:0/2069355499 conn(0x7f9da40fc3f0 msgr2=0x7f9da406a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.874+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 shutdown_connections 2024-08-02T17:46:30.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:30.874+0000 7f9daac23640 1 -- 172.21.15.69:0/2069355499 wait complete. 2024-08-02T17:46:30.886 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-08-02T17:46:28.936638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190144,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":849,"num_read_kb":585,"num_write":481,"num_write_kb":4384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398040,"kb_used_data":15444,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543784,"statfs":{"total":1151940427776,"available":1151532834816,"internally_reserved":0,"allocated":15814656,"data_stored":9023382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":391492921},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.008758"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915530+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:46:24.915530+0000","last_peered":"2024-08-02T17:46:24.915530+0000","last_clean":"2024-08-02T17:46:24.915530+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:46:24.915530+0000","last_undegraded":"2024-08-02T17:46:24.915530+0000","last_fullsized":"2024-08-02T17:46:24.915530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954285+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:46:24.954285+0000","last_peered":"2024-08-02T17:46:24.954285+0000","last_clean":"2024-08-02T17:46:24.954285+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:46:24.954285+0000","last_undegraded":"2024-08-02T17:46:24.954285+0000","last_fullsized":"2024-08-02T17:46:24.954285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915172+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:46:24.915172+0000","last_peered":"2024-08-02T17:46:24.915172+0000","last_clean":"2024-08-02T17:46:24.915172+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:46:24.915172+0000","last_undegraded":"2024-08-02T17:46:24.915172+0000","last_fullsized":"2024-08-02T17:46:24.915172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908437+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:46:24.908437+0000","last_peered":"2024-08-02T17:46:24.908437+0000","last_clean":"2024-08-02T17:46:24.908437+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:46:24.908437+0000","last_undegraded":"2024-08-02T17:46:24.908437+0000","last_fullsized":"2024-08-02T17:46:24.908437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959113+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:46:24.959113+0000","last_peered":"2024-08-02T17:46:24.959113+0000","last_clean":"2024-08-02T17:46:24.959113+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:46:24.959113+0000","last_undegraded":"2024-08-02T17:46:24.959113+0000","last_fullsized":"2024-08-02T17:46:24.959113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906783+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:46:24.906783+0000","last_peered":"2024-08-02T17:46:24.906783+0000","last_clean":"2024-08-02T17:46:24.906783+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:46:24.906783+0000","last_undegraded":"2024-08-02T17:46:24.906783+0000","last_fullsized":"2024-08-02T17:46:24.906783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906846+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:46:24.906846+0000","last_peered":"2024-08-02T17:46:24.906846+0000","last_clean":"2024-08-02T17:46:24.906846+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:46:24.906846+0000","last_undegraded":"2024-08-02T17:46:24.906846+0000","last_fullsized":"2024-08-02T17:46:24.906846+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962043+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:46:24.962043+0000","last_peered":"2024-08-02T17:46:24.962043+0000","last_clean":"2024-08-02T17:46:24.962043+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:46:24.962043+0000","last_undegraded":"2024-08-02T17:46:24.962043+0000","last_fullsized":"2024-08-02T17:46:24.962043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906895+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:46:24.906895+0000","last_peered":"2024-08-02T17:46:24.906895+0000","last_clean":"2024-08-02T17:46:24.906895+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:46:24.906895+0000","last_undegraded":"2024-08-02T17:46:24.906895+0000","last_fullsized":"2024-08-02T17:46:24.906895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906857+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:46:24.906857+0000","last_peered":"2024-08-02T17:46:24.906857+0000","last_clean":"2024-08-02T17:46:24.906857+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:46:24.906857+0000","last_undegraded":"2024-08-02T17:46:24.906857+0000","last_fullsized":"2024-08-02T17:46:24.906857+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959833+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:46:24.959833+0000","last_peered":"2024-08-02T17:46:24.959833+0000","last_clean":"2024-08-02T17:46:24.959833+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:46:24.959833+0000","last_undegraded":"2024-08-02T17:46:24.959833+0000","last_fullsized":"2024-08-02T17:46:24.959833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906814+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:46:24.906814+0000","last_peered":"2024-08-02T17:46:24.906814+0000","last_clean":"2024-08-02T17:46:24.906814+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:46:24.906814+0000","last_undegraded":"2024-08-02T17:46:24.906814+0000","last_fullsized":"2024-08-02T17:46:24.906814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961844+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:46:24.961844+0000","last_peered":"2024-08-02T17:46:24.961844+0000","last_clean":"2024-08-02T17:46:24.961844+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:46:24.961844+0000","last_undegraded":"2024-08-02T17:46:24.961844+0000","last_fullsized":"2024-08-02T17:46:24.961844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906478+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:46:24.906478+0000","last_peered":"2024-08-02T17:46:24.906478+0000","last_clean":"2024-08-02T17:46:24.906478+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:46:24.906478+0000","last_undegraded":"2024-08-02T17:46:24.906478+0000","last_fullsized":"2024-08-02T17:46:24.906478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907202+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:46:24.907202+0000","last_peered":"2024-08-02T17:46:24.907202+0000","last_clean":"2024-08-02T17:46:24.907202+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:46:24.907202+0000","last_undegraded":"2024-08-02T17:46:24.907202+0000","last_fullsized":"2024-08-02T17:46:24.907202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959010+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:46:24.959010+0000","last_peered":"2024-08-02T17:46:24.959010+0000","last_clean":"2024-08-02T17:46:24.959010+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:46:24.959010+0000","last_undegraded":"2024-08-02T17:46:24.959010+0000","last_fullsized":"2024-08-02T17:46:24.959010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920005+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:46:24.920005+0000","last_peered":"2024-08-02T17:46:24.920005+0000","last_clean":"2024-08-02T17:46:24.920005+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:46:24.920005+0000","last_undegraded":"2024-08-02T17:46:24.920005+0000","last_fullsized":"2024-08-02T17:46:24.920005+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907441+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:46:24.907441+0000","last_peered":"2024-08-02T17:46:24.907441+0000","last_clean":"2024-08-02T17:46:24.907441+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:46:24.907441+0000","last_undegraded":"2024-08-02T17:46:24.907441+0000","last_fullsized":"2024-08-02T17:46:24.907441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956539+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:46:24.956539+0000","last_peered":"2024-08-02T17:46:24.956539+0000","last_clean":"2024-08-02T17:46:24.956539+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:46:24.956539+0000","last_undegraded":"2024-08-02T17:46:24.956539+0000","last_fullsized":"2024-08-02T17:46:24.956539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907900+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:46:24.907900+0000","last_peered":"2024-08-02T17:46:24.907900+0000","last_clean":"2024-08-02T17:46:24.907900+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:46:24.907900+0000","last_undegraded":"2024-08-02T17:46:24.907900+0000","last_fullsized":"2024-08-02T17:46:24.907900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915312+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:46:24.915312+0000","last_peered":"2024-08-02T17:46:24.915312+0000","last_clean":"2024-08-02T17:46:24.915312+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:46:24.915312+0000","last_undegraded":"2024-08-02T17:46:24.915312+0000","last_fullsized":"2024-08-02T17:46:24.915312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924149+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:46:24.924149+0000","last_peered":"2024-08-02T17:46:24.924149+0000","last_clean":"2024-08-02T17:46:24.924149+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:46:24.924149+0000","last_undegraded":"2024-08-02T17:46:24.924149+0000","last_fullsized":"2024-08-02T17:46:24.924149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915653+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:46:24.915653+0000","last_peered":"2024-08-02T17:46:24.915653+0000","last_clean":"2024-08-02T17:46:24.915653+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:46:24.915653+0000","last_undegraded":"2024-08-02T17:46:24.915653+0000","last_fullsized":"2024-08-02T17:46:24.915653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907240+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:46:24.907240+0000","last_peered":"2024-08-02T17:46:24.907240+0000","last_clean":"2024-08-02T17:46:24.907240+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:46:24.907240+0000","last_undegraded":"2024-08-02T17:46:24.907240+0000","last_fullsized":"2024-08-02T17:46:24.907240+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961716+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:46:24.961716+0000","last_peered":"2024-08-02T17:46:24.961716+0000","last_clean":"2024-08-02T17:46:24.961716+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:46:24.961716+0000","last_undegraded":"2024-08-02T17:46:24.961716+0000","last_fullsized":"2024-08-02T17:46:24.961716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907489+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:46:24.907489+0000","last_peered":"2024-08-02T17:46:24.907489+0000","last_clean":"2024-08-02T17:46:24.907489+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:46:24.907489+0000","last_undegraded":"2024-08-02T17:46:24.907489+0000","last_fullsized":"2024-08-02T17:46:24.907489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915365+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:46:24.915365+0000","last_peered":"2024-08-02T17:46:24.915365+0000","last_clean":"2024-08-02T17:46:24.915365+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:46:24.915365+0000","last_undegraded":"2024-08-02T17:46:24.915365+0000","last_fullsized":"2024-08-02T17:46:24.915365+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956310+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:46:24.956310+0000","last_peered":"2024-08-02T17:46:24.956310+0000","last_clean":"2024-08-02T17:46:24.956310+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:46:24.956310+0000","last_undegraded":"2024-08-02T17:46:24.956310+0000","last_fullsized":"2024-08-02T17:46:24.956310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'47","reported_seq":122,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.154876+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:46:25.154876+0000","last_peered":"2024-08-02T17:46:25.154876+0000","last_clean":"2024-08-02T17:46:25.154876+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:46:25.154876+0000","last_undegraded":"2024-08-02T17:46:25.154876+0000","last_fullsized":"2024-08-02T17:46:25.154876+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907676+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:46:24.907676+0000","last_peered":"2024-08-02T17:46:24.907676+0000","last_clean":"2024-08-02T17:46:24.907676+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.907676+0000","last_undegraded":"2024-08-02T17:46:24.907676+0000","last_fullsized":"2024-08-02T17:46:24.907676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950528+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:46:24.950528+0000","last_peered":"2024-08-02T17:46:24.950528+0000","last_clean":"2024-08-02T17:46:24.950528+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:46:24.950528+0000","last_undegraded":"2024-08-02T17:46:24.950528+0000","last_fullsized":"2024-08-02T17:46:24.950528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906878+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:46:24.906878+0000","last_peered":"2024-08-02T17:46:24.906878+0000","last_clean":"2024-08-02T17:46:24.906878+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:46:24.906878+0000","last_undegraded":"2024-08-02T17:46:24.906878+0000","last_fullsized":"2024-08-02T17:46:24.906878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920332+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:46:24.920332+0000","last_peered":"2024-08-02T17:46:24.920332+0000","last_clean":"2024-08-02T17:46:24.920332+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:46:24.920332+0000","last_undegraded":"2024-08-02T17:46:24.920332+0000","last_fullsized":"2024-08-02T17:46:24.920332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953960+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:46:24.953960+0000","last_peered":"2024-08-02T17:46:24.953960+0000","last_clean":"2024-08-02T17:46:24.953960+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:46:24.953960+0000","last_undegraded":"2024-08-02T17:46:24.953960+0000","last_fullsized":"2024-08-02T17:46:24.953960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915185+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:46:24.915185+0000","last_peered":"2024-08-02T17:46:24.915185+0000","last_clean":"2024-08-02T17:46:24.915185+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:46:24.915185+0000","last_undegraded":"2024-08-02T17:46:24.915185+0000","last_fullsized":"2024-08-02T17:46:24.915185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906282+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:46:24.906282+0000","last_peered":"2024-08-02T17:46:24.906282+0000","last_clean":"2024-08-02T17:46:24.906282+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:46:24.906282+0000","last_undegraded":"2024-08-02T17:46:24.906282+0000","last_fullsized":"2024-08-02T17:46:24.906282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956518+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:46:24.956518+0000","last_peered":"2024-08-02T17:46:24.956518+0000","last_clean":"2024-08-02T17:46:24.956518+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:46:24.956518+0000","last_undegraded":"2024-08-02T17:46:24.956518+0000","last_fullsized":"2024-08-02T17:46:24.956518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.909639+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:46:24.909639+0000","last_peered":"2024-08-02T17:46:24.909639+0000","last_clean":"2024-08-02T17:46:24.909639+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:46:24.909639+0000","last_undegraded":"2024-08-02T17:46:24.909639+0000","last_fullsized":"2024-08-02T17:46:24.909639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.918812+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:46:24.918812+0000","last_peered":"2024-08-02T17:46:24.918812+0000","last_clean":"2024-08-02T17:46:24.918812+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:46:24.918812+0000","last_undegraded":"2024-08-02T17:46:24.918812+0000","last_fullsized":"2024-08-02T17:46:24.918812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952882+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:46:24.952882+0000","last_peered":"2024-08-02T17:46:24.952882+0000","last_clean":"2024-08-02T17:46:24.952882+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:46:24.952882+0000","last_undegraded":"2024-08-02T17:46:24.952882+0000","last_fullsized":"2024-08-02T17:46:24.952882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":143,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919340+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:46:24.919340+0000","last_peered":"2024-08-02T17:46:24.919340+0000","last_clean":"2024-08-02T17:46:24.919340+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:46:24.919340+0000","last_undegraded":"2024-08-02T17:46:24.919340+0000","last_fullsized":"2024-08-02T17:46:24.919340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919707+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:46:24.919707+0000","last_peered":"2024-08-02T17:46:24.919707+0000","last_clean":"2024-08-02T17:46:24.919707+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.919707+0000","last_undegraded":"2024-08-02T17:46:24.919707+0000","last_fullsized":"2024-08-02T17:46:24.919707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947422+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:46:24.947422+0000","last_peered":"2024-08-02T17:46:24.947422+0000","last_clean":"2024-08-02T17:46:24.947422+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:46:24.947422+0000","last_undegraded":"2024-08-02T17:46:24.947422+0000","last_fullsized":"2024-08-02T17:46:24.947422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908927+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:46:24.908927+0000","last_peered":"2024-08-02T17:46:24.908927+0000","last_clean":"2024-08-02T17:46:24.908927+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:46:24.908927+0000","last_undegraded":"2024-08-02T17:46:24.908927+0000","last_fullsized":"2024-08-02T17:46:24.908927+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915349+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:46:24.915349+0000","last_peered":"2024-08-02T17:46:24.915349+0000","last_clean":"2024-08-02T17:46:24.915349+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:46:24.915349+0000","last_undegraded":"2024-08-02T17:46:24.915349+0000","last_fullsized":"2024-08-02T17:46:24.915349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962129+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:46:24.962129+0000","last_peered":"2024-08-02T17:46:24.962129+0000","last_clean":"2024-08-02T17:46:24.962129+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:46:24.962129+0000","last_undegraded":"2024-08-02T17:46:24.962129+0000","last_fullsized":"2024-08-02T17:46:24.962129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907233+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:46:24.907233+0000","last_peered":"2024-08-02T17:46:24.907233+0000","last_clean":"2024-08-02T17:46:24.907233+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:46:24.907233+0000","last_undegraded":"2024-08-02T17:46:24.907233+0000","last_fullsized":"2024-08-02T17:46:24.907233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":368,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908841+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:46:24.908841+0000","last_peered":"2024-08-02T17:46:24.908841+0000","last_clean":"2024-08-02T17:46:24.908841+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:46:24.908841+0000","last_undegraded":"2024-08-02T17:46:24.908841+0000","last_fullsized":"2024-08-02T17:46:24.908841+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910641+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:46:24.910641+0000","last_peered":"2024-08-02T17:46:24.910641+0000","last_clean":"2024-08-02T17:46:24.910641+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:46:24.910641+0000","last_undegraded":"2024-08-02T17:46:24.910641+0000","last_fullsized":"2024-08-02T17:46:24.910641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960428+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:46:24.960428+0000","last_peered":"2024-08-02T17:46:24.960428+0000","last_clean":"2024-08-02T17:46:24.960428+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:46:24.960428+0000","last_undegraded":"2024-08-02T17:46:24.960428+0000","last_fullsized":"2024-08-02T17:46:24.960428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.916318+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:46:24.916318+0000","last_peered":"2024-08-02T17:46:24.916318+0000","last_clean":"2024-08-02T17:46:24.916318+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:46:24.916318+0000","last_undegraded":"2024-08-02T17:46:24.916318+0000","last_fullsized":"2024-08-02T17:46:24.916318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915831+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:46:24.915831+0000","last_peered":"2024-08-02T17:46:24.915831+0000","last_clean":"2024-08-02T17:46:24.915831+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:46:24.915831+0000","last_undegraded":"2024-08-02T17:46:24.915831+0000","last_fullsized":"2024-08-02T17:46:24.915831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948829+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:46:24.948829+0000","last_peered":"2024-08-02T17:46:24.948829+0000","last_clean":"2024-08-02T17:46:24.948829+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:46:24.948829+0000","last_undegraded":"2024-08-02T17:46:24.948829+0000","last_fullsized":"2024-08-02T17:46:24.948829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":87,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.150408+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:46:25.150408+0000","last_peered":"2024-08-02T17:46:25.150408+0000","last_clean":"2024-08-02T17:46:25.150408+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:46:25.150408+0000","last_undegraded":"2024-08-02T17:46:25.150408+0000","last_fullsized":"2024-08-02T17:46:25.150408+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924774+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:46:24.924774+0000","last_peered":"2024-08-02T17:46:24.924774+0000","last_clean":"2024-08-02T17:46:24.924774+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:46:24.924774+0000","last_undegraded":"2024-08-02T17:46:24.924774+0000","last_fullsized":"2024-08-02T17:46:24.924774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955439+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:46:24.955439+0000","last_peered":"2024-08-02T17:46:24.955439+0000","last_clean":"2024-08-02T17:46:24.955439+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:46:24.955439+0000","last_undegraded":"2024-08-02T17:46:24.955439+0000","last_fullsized":"2024-08-02T17:46:24.955439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919542+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:46:24.919542+0000","last_peered":"2024-08-02T17:46:24.919542+0000","last_clean":"2024-08-02T17:46:24.919542+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:46:24.919542+0000","last_undegraded":"2024-08-02T17:46:24.919542+0000","last_fullsized":"2024-08-02T17:46:24.919542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914656+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:46:24.914656+0000","last_peered":"2024-08-02T17:46:24.914656+0000","last_clean":"2024-08-02T17:46:24.914656+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:46:24.914656+0000","last_undegraded":"2024-08-02T17:46:24.914656+0000","last_fullsized":"2024-08-02T17:46:24.914656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950756+0000","last_change":"2024-08-02T17:45:57.979722+0000","last_active":"2024-08-02T17:46:24.950756+0000","last_peered":"2024-08-02T17:46:24.950756+0000","last_clean":"2024-08-02T17:46:24.950756+0000","last_became_active":"2024-08-02T17:45:57.979427+0000","last_became_peered":"2024-08-02T17:45:57.979427+0000","last_unstale":"2024-08-02T17:46:24.950756+0000","last_undegraded":"2024-08-02T17:46:24.950756+0000","last_fullsized":"2024-08-02T17:46:24.950756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:16:52.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.920115+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:46:25.920115+0000","last_peered":"2024-08-02T17:46:25.920115+0000","last_clean":"2024-08-02T17:46:25.920115+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:46:25.920115+0000","last_undegraded":"2024-08-02T17:46:25.920115+0000","last_fullsized":"2024-08-02T17:46:25.920115+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914880+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:46:24.914880+0000","last_peered":"2024-08-02T17:46:24.914880+0000","last_clean":"2024-08-02T17:46:24.914880+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:46:24.914880+0000","last_undegraded":"2024-08-02T17:46:24.914880+0000","last_fullsized":"2024-08-02T17:46:24.914880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953834+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:46:24.953834+0000","last_peered":"2024-08-02T17:46:24.953834+0000","last_clean":"2024-08-02T17:46:24.953834+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:46:24.953834+0000","last_undegraded":"2024-08-02T17:46:24.953834+0000","last_fullsized":"2024-08-02T17:46:24.953834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906346+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:46:24.906346+0000","last_peered":"2024-08-02T17:46:24.906346+0000","last_clean":"2024-08-02T17:46:24.906346+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:46:24.906346+0000","last_undegraded":"2024-08-02T17:46:24.906346+0000","last_fullsized":"2024-08-02T17:46:24.906346+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919446+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:46:24.919446+0000","last_peered":"2024-08-02T17:46:24.919446+0000","last_clean":"2024-08-02T17:46:24.919446+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:46:24.919446+0000","last_undegraded":"2024-08-02T17:46:24.919446+0000","last_fullsized":"2024-08-02T17:46:24.919446+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950106+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:46:24.950106+0000","last_peered":"2024-08-02T17:46:24.950106+0000","last_clean":"2024-08-02T17:46:24.950106+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:46:24.950106+0000","last_undegraded":"2024-08-02T17:46:24.950106+0000","last_fullsized":"2024-08-02T17:46:24.950106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'8","reported_seq":152,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.202288+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:46:25.202288+0000","last_peered":"2024-08-02T17:46:25.202288+0000","last_clean":"2024-08-02T17:46:25.202288+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:46:25.202288+0000","last_undegraded":"2024-08-02T17:46:25.202288+0000","last_fullsized":"2024-08-02T17:46:25.202288+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2207,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":6,"num_write":7,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915368+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:46:24.915368+0000","last_peered":"2024-08-02T17:46:24.915368+0000","last_clean":"2024-08-02T17:46:24.915368+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:46:24.915368+0000","last_undegraded":"2024-08-02T17:46:24.915368+0000","last_fullsized":"2024-08-02T17:46:24.915368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957938+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:46:24.957938+0000","last_peered":"2024-08-02T17:46:24.957938+0000","last_clean":"2024-08-02T17:46:24.957938+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:46:24.957938+0000","last_undegraded":"2024-08-02T17:46:24.957938+0000","last_fullsized":"2024-08-02T17:46:24.957938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915099+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:46:24.915099+0000","last_peered":"2024-08-02T17:46:24.915099+0000","last_clean":"2024-08-02T17:46:24.915099+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:46:24.915099+0000","last_undegraded":"2024-08-02T17:46:24.915099+0000","last_fullsized":"2024-08-02T17:46:24.915099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920404+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:46:24.920404+0000","last_peered":"2024-08-02T17:46:24.920404+0000","last_clean":"2024-08-02T17:46:24.920404+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:46:24.920404+0000","last_undegraded":"2024-08-02T17:46:24.920404+0000","last_fullsized":"2024-08-02T17:46:24.920404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958381+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:46:24.958381+0000","last_peered":"2024-08-02T17:46:24.958381+0000","last_clean":"2024-08-02T17:46:24.958381+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:46:24.958381+0000","last_undegraded":"2024-08-02T17:46:24.958381+0000","last_fullsized":"2024-08-02T17:46:24.958381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915872+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:46:24.915872+0000","last_peered":"2024-08-02T17:46:24.915872+0000","last_clean":"2024-08-02T17:46:24.915872+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:46:24.915872+0000","last_undegraded":"2024-08-02T17:46:24.915872+0000","last_fullsized":"2024-08-02T17:46:24.915872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915458+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:46:24.915458+0000","last_peered":"2024-08-02T17:46:24.915458+0000","last_clean":"2024-08-02T17:46:24.915458+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:46:24.915458+0000","last_undegraded":"2024-08-02T17:46:24.915458+0000","last_fullsized":"2024-08-02T17:46:24.915458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955734+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:46:24.955734+0000","last_peered":"2024-08-02T17:46:24.955734+0000","last_clean":"2024-08-02T17:46:24.955734+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:46:24.955734+0000","last_undegraded":"2024-08-02T17:46:24.955734+0000","last_fullsized":"2024-08-02T17:46:24.955734+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'38","reported_seq":285,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.207389+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:46:25.207389+0000","last_peered":"2024-08-02T17:46:25.207389+0000","last_clean":"2024-08-02T17:46:25.207389+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:46:25.207389+0000","last_undegraded":"2024-08-02T17:46:25.207389+0000","last_fullsized":"2024-08-02T17:46:25.207389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68576,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":168,"num_write":39,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911360+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:46:24.911360+0000","last_peered":"2024-08-02T17:46:24.911360+0000","last_clean":"2024-08-02T17:46:24.911360+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:46:24.911360+0000","last_undegraded":"2024-08-02T17:46:24.911360+0000","last_fullsized":"2024-08-02T17:46:24.911360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959993+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:46:24.959993+0000","last_peered":"2024-08-02T17:46:24.959993+0000","last_clean":"2024-08-02T17:46:24.959993+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:46:24.959993+0000","last_undegraded":"2024-08-02T17:46:24.959993+0000","last_fullsized":"2024-08-02T17:46:24.959993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915691+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:46:24.915691+0000","last_peered":"2024-08-02T17:46:24.915691+0000","last_clean":"2024-08-02T17:46:24.915691+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:46:24.915691+0000","last_undegraded":"2024-08-02T17:46:24.915691+0000","last_fullsized":"2024-08-02T17:46:24.915691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907020+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:46:24.907020+0000","last_peered":"2024-08-02T17:46:24.907020+0000","last_clean":"2024-08-02T17:46:24.907020+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:46:24.907020+0000","last_undegraded":"2024-08-02T17:46:24.907020+0000","last_fullsized":"2024-08-02T17:46:24.907020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960499+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:46:24.960499+0000","last_peered":"2024-08-02T17:46:24.960499+0000","last_clean":"2024-08-02T17:46:24.960499+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:46:24.960499+0000","last_undegraded":"2024-08-02T17:46:24.960499+0000","last_fullsized":"2024-08-02T17:46:24.960499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910476+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:46:24.910476+0000","last_peered":"2024-08-02T17:46:24.910476+0000","last_clean":"2024-08-02T17:46:24.910476+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:46:24.910476+0000","last_undegraded":"2024-08-02T17:46:24.910476+0000","last_fullsized":"2024-08-02T17:46:24.910476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914823+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:46:24.914823+0000","last_peered":"2024-08-02T17:46:24.914823+0000","last_clean":"2024-08-02T17:46:24.914823+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:46:24.914823+0000","last_undegraded":"2024-08-02T17:46:24.914823+0000","last_fullsized":"2024-08-02T17:46:24.914823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962330+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:46:24.962330+0000","last_peered":"2024-08-02T17:46:24.962330+0000","last_clean":"2024-08-02T17:46:24.962330+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:46:24.962330+0000","last_undegraded":"2024-08-02T17:46:24.962330+0000","last_fullsized":"2024-08-02T17:46:24.962330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":143,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.055690+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:46:25.055690+0000","last_peered":"2024-08-02T17:46:25.055690+0000","last_clean":"2024-08-02T17:46:25.055690+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:46:25.055690+0000","last_undegraded":"2024-08-02T17:46:25.055690+0000","last_fullsized":"2024-08-02T17:46:25.055690+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907780+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:46:24.907780+0000","last_peered":"2024-08-02T17:46:24.907780+0000","last_clean":"2024-08-02T17:46:24.907780+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:46:24.907780+0000","last_undegraded":"2024-08-02T17:46:24.907780+0000","last_fullsized":"2024-08-02T17:46:24.907780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.926268+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:46:24.926268+0000","last_peered":"2024-08-02T17:46:24.926268+0000","last_clean":"2024-08-02T17:46:24.926268+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:46:24.926268+0000","last_undegraded":"2024-08-02T17:46:24.926268+0000","last_fullsized":"2024-08-02T17:46:24.926268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956250+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:46:24.956250+0000","last_peered":"2024-08-02T17:46:24.956250+0000","last_clean":"2024-08-02T17:46:24.956250+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:46:24.956250+0000","last_undegraded":"2024-08-02T17:46:24.956250+0000","last_fullsized":"2024-08-02T17:46:24.956250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914914+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:46:24.914914+0000","last_peered":"2024-08-02T17:46:24.914914+0000","last_clean":"2024-08-02T17:46:24.914914+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:46:24.914914+0000","last_undegraded":"2024-08-02T17:46:24.914914+0000","last_fullsized":"2024-08-02T17:46:24.914914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920061+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:46:24.920061+0000","last_peered":"2024-08-02T17:46:24.920061+0000","last_clean":"2024-08-02T17:46:24.920061+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:46:24.920061+0000","last_undegraded":"2024-08-02T17:46:24.920061+0000","last_fullsized":"2024-08-02T17:46:24.920061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958695+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:46:24.958695+0000","last_peered":"2024-08-02T17:46:24.958695+0000","last_clean":"2024-08-02T17:46:24.958695+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:46:24.958695+0000","last_undegraded":"2024-08-02T17:46:24.958695+0000","last_fullsized":"2024-08-02T17:46:24.958695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919202+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:46:24.919202+0000","last_peered":"2024-08-02T17:46:24.919202+0000","last_clean":"2024-08-02T17:46:24.919202+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:46:24.919202+0000","last_undegraded":"2024-08-02T17:46:24.919202+0000","last_fullsized":"2024-08-02T17:46:24.919202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911667+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:46:24.911667+0000","last_peered":"2024-08-02T17:46:24.911667+0000","last_clean":"2024-08-02T17:46:24.911667+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:46:24.911667+0000","last_undegraded":"2024-08-02T17:46:24.911667+0000","last_fullsized":"2024-08-02T17:46:24.911667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950748+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:46:24.950748+0000","last_peered":"2024-08-02T17:46:24.950748+0000","last_clean":"2024-08-02T17:46:24.950748+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:46:24.950748+0000","last_undegraded":"2024-08-02T17:46:24.950748+0000","last_fullsized":"2024-08-02T17:46:24.950748+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915973+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:46:24.915973+0000","last_peered":"2024-08-02T17:46:24.915973+0000","last_clean":"2024-08-02T17:46:24.915973+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:46:24.915973+0000","last_undegraded":"2024-08-02T17:46:24.915973+0000","last_fullsized":"2024-08-02T17:46:24.915973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":149,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.056494+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:46:25.056494+0000","last_peered":"2024-08-02T17:46:25.056494+0000","last_clean":"2024-08-02T17:46:25.056494+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:46:25.056494+0000","last_undegraded":"2024-08-02T17:46:25.056494+0000","last_fullsized":"2024-08-02T17:46:25.056494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.940918+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:46:24.940918+0000","last_peered":"2024-08-02T17:46:24.940918+0000","last_clean":"2024-08-02T17:46:24.940918+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:46:24.940918+0000","last_undegraded":"2024-08-02T17:46:24.940918+0000","last_fullsized":"2024-08-02T17:46:24.940918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915638+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:46:24.915638+0000","last_peered":"2024-08-02T17:46:24.915638+0000","last_clean":"2024-08-02T17:46:24.915638+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:46:24.915638+0000","last_undegraded":"2024-08-02T17:46:24.915638+0000","last_fullsized":"2024-08-02T17:46:24.915638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954294+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:46:24.954294+0000","last_peered":"2024-08-02T17:46:24.954294+0000","last_clean":"2024-08-02T17:46:24.954294+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:46:24.954294+0000","last_undegraded":"2024-08-02T17:46:24.954294+0000","last_fullsized":"2024-08-02T17:46:24.954294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953139+0000","last_change":"2024-08-02T17:45:58.024328+0000","last_active":"2024-08-02T17:46:24.953139+0000","last_peered":"2024-08-02T17:46:24.953139+0000","last_clean":"2024-08-02T17:46:24.953139+0000","last_became_active":"2024-08-02T17:45:58.023931+0000","last_became_peered":"2024-08-02T17:45:58.023931+0000","last_unstale":"2024-08-02T17:46:24.953139+0000","last_undegraded":"2024-08-02T17:46:24.953139+0000","last_fullsized":"2024-08-02T17:46:24.953139+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:52:10.635263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.946206+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:46:24.946206+0000","last_peered":"2024-08-02T17:46:24.946206+0000","last_clean":"2024-08-02T17:46:24.946206+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:46:24.946206+0000","last_undegraded":"2024-08-02T17:46:24.946206+0000","last_fullsized":"2024-08-02T17:46:24.946206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961330+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:46:24.961330+0000","last_peered":"2024-08-02T17:46:24.961330+0000","last_clean":"2024-08-02T17:46:24.961330+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:46:24.961330+0000","last_undegraded":"2024-08-02T17:46:24.961330+0000","last_fullsized":"2024-08-02T17:46:24.961330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948673+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:46:24.948673+0000","last_peered":"2024-08-02T17:46:24.948673+0000","last_clean":"2024-08-02T17:46:24.948673+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:46:24.948673+0000","last_undegraded":"2024-08-02T17:46:24.948673+0000","last_fullsized":"2024-08-02T17:46:24.948673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948711+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:46:24.948711+0000","last_peered":"2024-08-02T17:46:24.948711+0000","last_clean":"2024-08-02T17:46:24.948711+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:46:24.948711+0000","last_undegraded":"2024-08-02T17:46:24.948711+0000","last_fullsized":"2024-08-02T17:46:24.948711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962508+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:46:24.962508+0000","last_peered":"2024-08-02T17:46:24.962508+0000","last_clean":"2024-08-02T17:46:24.962508+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:46:24.962508+0000","last_undegraded":"2024-08-02T17:46:24.962508+0000","last_fullsized":"2024-08-02T17:46:24.962508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949000+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:46:24.949000+0000","last_peered":"2024-08-02T17:46:24.949000+0000","last_clean":"2024-08-02T17:46:24.949000+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:46:24.949000+0000","last_undegraded":"2024-08-02T17:46:24.949000+0000","last_fullsized":"2024-08-02T17:46:24.949000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949651+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:46:24.949651+0000","last_peered":"2024-08-02T17:46:24.949651+0000","last_clean":"2024-08-02T17:46:24.949651+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:46:24.949651+0000","last_undegraded":"2024-08-02T17:46:24.949651+0000","last_fullsized":"2024-08-02T17:46:24.949651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949256+0000","last_change":"2024-08-02T17:45:57.863502+0000","last_active":"2024-08-02T17:46:24.949256+0000","last_peered":"2024-08-02T17:46:24.949256+0000","last_clean":"2024-08-02T17:46:24.949256+0000","last_became_active":"2024-08-02T17:45:57.863230+0000","last_became_peered":"2024-08-02T17:45:57.863230+0000","last_unstale":"2024-08-02T17:46:24.949256+0000","last_undegraded":"2024-08-02T17:46:24.949256+0000","last_fullsized":"2024-08-02T17:46:24.949256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:26:50.915451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962526+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:46:24.962526+0000","last_peered":"2024-08-02T17:46:24.962526+0000","last_clean":"2024-08-02T17:46:24.962526+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:46:24.962526+0000","last_undegraded":"2024-08-02T17:46:24.962526+0000","last_fullsized":"2024-08-02T17:46:24.962526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958665+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:46:24.958665+0000","last_peered":"2024-08-02T17:46:24.958665+0000","last_clean":"2024-08-02T17:46:24.958665+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:46:24.958665+0000","last_undegraded":"2024-08-02T17:46:24.958665+0000","last_fullsized":"2024-08-02T17:46:24.958665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947237+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:46:24.947237+0000","last_peered":"2024-08-02T17:46:24.947237+0000","last_clean":"2024-08-02T17:46:24.947237+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:46:24.947237+0000","last_undegraded":"2024-08-02T17:46:24.947237+0000","last_fullsized":"2024-08-02T17:46:24.947237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.951444+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:46:24.951444+0000","last_peered":"2024-08-02T17:46:24.951444+0000","last_clean":"2024-08-02T17:46:24.951444+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:46:24.951444+0000","last_undegraded":"2024-08-02T17:46:24.951444+0000","last_fullsized":"2024-08-02T17:46:24.951444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962281+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:46:24.962281+0000","last_peered":"2024-08-02T17:46:24.962281+0000","last_clean":"2024-08-02T17:46:24.962281+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:46:24.962281+0000","last_undegraded":"2024-08-02T17:46:24.962281+0000","last_fullsized":"2024-08-02T17:46:24.962281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959483+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:46:24.959483+0000","last_peered":"2024-08-02T17:46:24.959483+0000","last_clean":"2024-08-02T17:46:24.959483+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:46:24.959483+0000","last_undegraded":"2024-08-02T17:46:24.959483+0000","last_fullsized":"2024-08-02T17:46:24.959483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962369+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:46:24.962369+0000","last_peered":"2024-08-02T17:46:24.962369+0000","last_clean":"2024-08-02T17:46:24.962369+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:46:24.962369+0000","last_undegraded":"2024-08-02T17:46:24.962369+0000","last_fullsized":"2024-08-02T17:46:24.962369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959980+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:46:24.959980+0000","last_peered":"2024-08-02T17:46:24.959980+0000","last_clean":"2024-08-02T17:46:24.959980+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:46:24.959980+0000","last_undegraded":"2024-08-02T17:46:24.959980+0000","last_fullsized":"2024-08-02T17:46:24.959980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956497+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:46:24.956497+0000","last_peered":"2024-08-02T17:46:24.956497+0000","last_clean":"2024-08-02T17:46:24.956497+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:46:24.956497+0000","last_undegraded":"2024-08-02T17:46:24.956497+0000","last_fullsized":"2024-08-02T17:46:24.956497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952580+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:46:24.952580+0000","last_peered":"2024-08-02T17:46:24.952580+0000","last_clean":"2024-08-02T17:46:24.952580+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:46:24.952580+0000","last_undegraded":"2024-08-02T17:46:24.952580+0000","last_fullsized":"2024-08-02T17:46:24.952580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952561+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:46:24.952561+0000","last_peered":"2024-08-02T17:46:24.952561+0000","last_clean":"2024-08-02T17:46:24.952561+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:46:24.952561+0000","last_undegraded":"2024-08-02T17:46:24.952561+0000","last_fullsized":"2024-08-02T17:46:24.952561+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960139+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:46:24.960139+0000","last_peered":"2024-08-02T17:46:24.960139+0000","last_clean":"2024-08-02T17:46:24.960139+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:46:24.960139+0000","last_undegraded":"2024-08-02T17:46:24.960139+0000","last_fullsized":"2024-08-02T17:46:24.960139+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950344+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:46:24.950344+0000","last_peered":"2024-08-02T17:46:24.950344+0000","last_clean":"2024-08-02T17:46:24.950344+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:46:24.950344+0000","last_undegraded":"2024-08-02T17:46:24.950344+0000","last_fullsized":"2024-08-02T17:46:24.950344+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962150+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:46:24.962150+0000","last_peered":"2024-08-02T17:46:24.962150+0000","last_clean":"2024-08-02T17:46:24.962150+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:46:24.962150+0000","last_undegraded":"2024-08-02T17:46:24.962150+0000","last_fullsized":"2024-08-02T17:46:24.962150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958098+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:46:24.958098+0000","last_peered":"2024-08-02T17:46:24.958098+0000","last_clean":"2024-08-02T17:46:24.958098+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:46:24.958098+0000","last_undegraded":"2024-08-02T17:46:24.958098+0000","last_fullsized":"2024-08-02T17:46:24.958098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949100+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:46:24.949100+0000","last_peered":"2024-08-02T17:46:24.949100+0000","last_clean":"2024-08-02T17:46:24.949100+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:46:24.949100+0000","last_undegraded":"2024-08-02T17:46:24.949100+0000","last_fullsized":"2024-08-02T17:46:24.949100+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947146+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:46:24.947146+0000","last_peered":"2024-08-02T17:46:24.947146+0000","last_clean":"2024-08-02T17:46:24.947146+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:46:24.947146+0000","last_undegraded":"2024-08-02T17:46:24.947146+0000","last_fullsized":"2024-08-02T17:46:24.947146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949146+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:46:24.949146+0000","last_peered":"2024-08-02T17:46:24.949146+0000","last_clean":"2024-08-02T17:46:24.949146+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:46:24.949146+0000","last_undegraded":"2024-08-02T17:46:24.949146+0000","last_fullsized":"2024-08-02T17:46:24.949146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955968+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:46:24.955968+0000","last_peered":"2024-08-02T17:46:24.955968+0000","last_clean":"2024-08-02T17:46:24.955968+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:46:24.955968+0000","last_undegraded":"2024-08-02T17:46:24.955968+0000","last_fullsized":"2024-08-02T17:46:24.955968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953754+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:46:24.953754+0000","last_peered":"2024-08-02T17:46:24.953754+0000","last_clean":"2024-08-02T17:46:24.953754+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:46:24.953754+0000","last_undegraded":"2024-08-02T17:46:24.953754+0000","last_fullsized":"2024-08-02T17:46:24.953754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957053+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:46:24.957053+0000","last_peered":"2024-08-02T17:46:24.957053+0000","last_clean":"2024-08-02T17:46:24.957053+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:46:24.957053+0000","last_undegraded":"2024-08-02T17:46:24.957053+0000","last_fullsized":"2024-08-02T17:46:24.957053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959352+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:46:24.959352+0000","last_peered":"2024-08-02T17:46:24.959352+0000","last_clean":"2024-08-02T17:46:24.959352+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:46:24.959352+0000","last_undegraded":"2024-08-02T17:46:24.959352+0000","last_fullsized":"2024-08-02T17:46:24.959352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599542,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":713,"num_read_kb":466,"num_write":298,"num_write_kb":1444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1798626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449129,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1093632,"data_stored":544040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678061,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37336,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707816,"statfs":{"total":95995035648,"available":95956803584,"internally_reserved":0,"allocated":1925120,"data_stored":1333103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841587,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32652,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712500,"statfs":{"total":95995035648,"available":95961600000,"internally_reserved":0,"allocated":1507328,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103222,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":548292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1703936,"data_stored":1142254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":539452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37012,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708140,"statfs":{"total":95995035648,"available":95957135360,"internally_reserved":0,"allocated":1712128,"data_stored":1132249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":545053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215180,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1101824,"data_stored":539646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1179648,"data_stored":608927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":607938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":540840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":70735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":203373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:46:30.891 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:46:31.177 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:31.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.636+0000 7fa143cad640 1 -- 172.21.15.69:0/72577136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 msgr2=0x7fa13c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.636+0000 7fa143cad640 1 --2- 172.21.15.69:0/72577136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 0x7fa13c10e790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009930 tx=0x7fa12c02a420 comp rx=0 tx=0).stop 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 -- 172.21.15.69:0/72577136 shutdown_connections 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 --2- 172.21.15.69:0/72577136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 0x7fa13c10e790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 --2- 172.21.15.69:0/72577136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa13c1018c0 0x7fa13c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 --2- 172.21.15.69:0/72577136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa13c1078c0 0x7fa13c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 -- 172.21.15.69:0/72577136 >> 172.21.15.69:0/72577136 conn(0x7fa13c0fd620 msgr2=0x7fa13c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 -- 172.21.15.69:0/72577136 shutdown_connections 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 -- 172.21.15.69:0/72577136 wait complete. 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 Processor -- start 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.637+0000 7fa143cad640 1 -- start start 2024-08-02T17:46:31.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa143cad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa13c1018c0 0x7fa13c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa143cad640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 0x7fa13c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa143cad640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa13c1078c0 0x7fa13c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa143cad640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c111ce0 con 0x7fa13c102260 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa143cad640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c111b60 con 0x7fa13c1078c0 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa143cad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa13c111e60 con 0x7fa13c1018c0 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 0x7fa13c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 0x7fa13c074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36504/0 (socket says 172.21.15.69:36504) 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 -- 172.21.15.69:0/3072672973 learned_addr learned my addr 172.21.15.69:0/3072672973 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa14224a640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa13c1078c0 0x7fa13c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141a49640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa13c1018c0 0x7fa13c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 -- 172.21.15.69:0/3072672973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa13c1018c0 msgr2=0x7fa13c074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa13c1018c0 0x7fa13c074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 -- 172.21.15.69:0/3072672973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa13c1078c0 msgr2=0x7fa13c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:31.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa13c1078c0 0x7fa13c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 -- 172.21.15.69:0/3072672973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa13c1aabc0 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa141248640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 0x7fa13c074760 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b4a0 tx=0x7fa13000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa13affd640 1 -- 172.21.15.69:0/3072672973 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130014070 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa13c1aadf0 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa13affd640 1 -- 172.21.15.69:0/3072672973 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa130010af0 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa13affd640 1 -- 172.21.15.69:0/3072672973 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130017d10 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.638+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa13c1ab1f0 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.639+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa13c103000 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.640+0000 7fa13affd640 1 -- 172.21.15.69:0/3072672973 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fa130017070 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.641+0000 7fa13affd640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1240a2e10 0x7fa1240a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.641+0000 7fa141a49640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1240a2e10 0x7fa1240a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.641+0000 7fa13affd640 1 -- 172.21.15.69:0/3072672973 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa1300cedc0 con 0x7fa13c102260 2024-08-02T17:46:31.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.642+0000 7fa141a49640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1240a2e10 0x7fa1240a52d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa134005fd0 tx=0x7fa134009870 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:31.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.645+0000 7fa13affd640 1 -- 172.21.15.69:0/3072672973 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa13009b4c0 con 0x7fa13c102260 2024-08-02T17:46:31.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.844+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa13c101d20 con 0x7fa13c102260 2024-08-02T17:46:31.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.845+0000 7fa13affd640 1 -- 172.21.15.69:0/3072672973 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fa13009adc0 con 0x7fa13c102260 2024-08-02T17:46:31.846 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:31.847 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:46:31.847 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:46:31.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1240a2e10 msgr2=0x7fa1240a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:31.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1240a2e10 0x7fa1240a52d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa134005fd0 tx=0x7fa134009870 comp rx=0 tx=0).stop 2024-08-02T17:46:31.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 msgr2=0x7fa13c074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:31.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 0x7fa13c074760 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa13000b4a0 tx=0x7fa13000b960 comp rx=0 tx=0).stop 2024-08-02T17:46:31.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 shutdown_connections 2024-08-02T17:46:31.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa13c102260 0x7fa13c074760 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1240a2e10 0x7fa1240a52d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa13c1078c0 0x7fa13c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 --2- 172.21.15.69:0/3072672973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa13c1018c0 0x7fa13c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:31.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 >> 172.21.15.69:0/3072672973 conn(0x7fa13c0fd620 msgr2=0x7fa13c0fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:31.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 shutdown_connections 2024-08-02T17:46:31.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:31.847+0000 7fa143cad640 1 -- 172.21.15.69:0/3072672973 wait complete. 2024-08-02T17:46:32.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:31 smithi069 ceph-mon[32746]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:31 smithi069 ceph-mon[32746]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:31 smithi069 ceph-mon[32746]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:31 smithi069 ceph-mon[32746]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.4 KiB/s wr, 32 op/s 2024-08-02T17:46:32.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:31 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3072672973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:31 smithi083 ceph-mon[37031]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:31 smithi083 ceph-mon[37031]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:31 smithi083 ceph-mon[37031]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:31 smithi083 ceph-mon[37031]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.4 KiB/s wr, 32 op/s 2024-08-02T17:46:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:31 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3072672973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:32.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:31 smithi077 ceph-mon[37108]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:31 smithi077 ceph-mon[37108]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:31 smithi077 ceph-mon[37108]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:32.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:31 smithi077 ceph-mon[37108]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.4 KiB/s wr, 32 op/s 2024-08-02T17:46:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:31 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3072672973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:32.556 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:46:32.809 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 -- 172.21.15.69:0/1028961418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 msgr2=0x7fcc90100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/1028961418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 0x7fcc90100f30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcc84009900 tx=0x7fcc8402a480 comp rx=0 tx=0).stop 2024-08-02T17:46:33.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 -- 172.21.15.69:0/1028961418 shutdown_connections 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/1028961418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc90102c20 0x7fcc9010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/1028961418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc901022a0 0x7fcc901026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/1028961418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 0x7fcc90100f30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 -- 172.21.15.69:0/1028961418 >> 172.21.15.69:0/1028961418 conn(0x7fcc900fc500 msgr2=0x7fcc900fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 -- 172.21.15.69:0/1028961418 shutdown_connections 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.318+0000 7fcc96dbb640 1 -- 172.21.15.69:0/1028961418 wait complete. 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc96dbb640 1 Processor -- start 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc96dbb640 1 -- start start 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc96dbb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 0x7fcc90199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc96dbb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc901022a0 0x7fcc90199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc96dbb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc90102c20 0x7fcc901a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:33.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc96dbb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90110d20 con 0x7fcc90100b50 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc96dbb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90110ba0 con 0x7fcc901022a0 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc96dbb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90110ea0 con 0x7fcc90102c20 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc94b57640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 0x7fcc90199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc94b57640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 0x7fcc90199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36530/0 (socket says 172.21.15.69:36530) 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc94b57640 1 -- 172.21.15.69:0/3163815894 learned_addr learned my addr 172.21.15.69:0/3163815894 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc8ffff640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc901022a0 0x7fcc90199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc94b57640 1 -- 172.21.15.69:0/3163815894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc90102c20 msgr2=0x7fcc901a41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc94b57640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc90102c20 0x7fcc901a41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc94b57640 1 -- 172.21.15.69:0/3163815894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc901022a0 msgr2=0x7fcc90199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc94b57640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc901022a0 0x7fcc90199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc94b57640 1 -- 172.21.15.69:0/3163815894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc901a46f0 con 0x7fcc90100b50 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.319+0000 7fcc8ffff640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc901022a0 0x7fcc90199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.320+0000 7fcc94b57640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 0x7fcc90199130 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006010 tx=0x7fcc84003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:33.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.320+0000 7fcc8dffb640 1 -- 172.21.15.69:0/3163815894 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8403f070 con 0x7fcc90100b50 2024-08-02T17:46:33.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.320+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc901a48c0 con 0x7fcc90100b50 2024-08-02T17:46:33.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.320+0000 7fcc8dffb640 1 -- 172.21.15.69:0/3163815894 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc84003e30 con 0x7fcc90100b50 2024-08-02T17:46:33.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.320+0000 7fcc8dffb640 1 -- 172.21.15.69:0/3163815894 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc84035030 con 0x7fcc90100b50 2024-08-02T17:46:33.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.320+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc901a4d70 con 0x7fcc90100b50 2024-08-02T17:46:33.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.321+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc90100f30 con 0x7fcc90100b50 2024-08-02T17:46:33.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.322+0000 7fcc8dffb640 1 -- 172.21.15.69:0/3163815894 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fcc84042070 con 0x7fcc90100b50 2024-08-02T17:46:33.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.323+0000 7fcc8dffb640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc800a2e10 0x7fcc800a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:33.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.323+0000 7fcc8ffff640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc800a2e10 0x7fcc800a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:33.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.323+0000 7fcc8dffb640 1 -- 172.21.15.69:0/3163815894 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcc840f04b0 con 0x7fcc90100b50 2024-08-02T17:46:33.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.323+0000 7fcc8ffff640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc800a2e10 0x7fcc800a52d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006a7f0 tx=0x7fcc7c008040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.327+0000 7fcc8dffb640 1 -- 172.21.15.69:0/3163815894 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fcc840bcb90 con 0x7fcc90100b50 2024-08-02T17:46:33.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.493+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc90108dc0 con 0x7fcc90100b50 2024-08-02T17:46:33.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.494+0000 7fcc8dffb640 1 -- 172.21.15.69:0/3163815894 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25069 (secure 0 0 0) 0x7fcc840bc490 con 0x7fcc90100b50 2024-08-02T17:46:33.496 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:33.497 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":95,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:46:24.900245+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3271476957":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3594391539":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3033851323":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/2680424375":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/452236630":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4036393791":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6800/452236630":"2024-08-03T17:46:24.900194+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:46:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.496+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc800a2e10 msgr2=0x7fcc800a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.496+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc800a2e10 0x7fcc800a52d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcc9006a7f0 tx=0x7fcc7c008040 comp rx=0 tx=0).stop 2024-08-02T17:46:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.496+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 msgr2=0x7fcc90199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:33.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.496+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 0x7fcc90199130 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006010 tx=0x7fcc84003c10 comp rx=0 tx=0).stop 2024-08-02T17:46:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.497+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 shutdown_connections 2024-08-02T17:46:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.497+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc800a2e10 0x7fcc800a52d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.497+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc90102c20 0x7fcc901a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.497+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc901022a0 0x7fcc90199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.497+0000 7fcc96dbb640 1 --2- 172.21.15.69:0/3163815894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc90100b50 0x7fcc90199130 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.497+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 >> 172.21.15.69:0/3163815894 conn(0x7fcc900fc500 msgr2=0x7fcc900fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.497+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 shutdown_connections 2024-08-02T17:46:33.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:33.497+0000 7fcc96dbb640 1 -- 172.21.15.69:0/3163815894 wait complete. 2024-08-02T17:46:34.226 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:33 smithi069 ceph-mon[32746]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1022 B/s wr, 23 op/s 2024-08-02T17:46:34.227 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:33 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3163815894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:34.258 DEBUG:tasks.fwd_scrub:Yielding 2024-08-02T17:46:34.258 INFO:teuthology.run_tasks:Running task exec... 2024-08-02T17:46:34.267 INFO:teuthology.task.exec:Executing custom commands... 2024-08-02T17:46:34.267 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:46:34.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-08-02T17:46:34.268 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-02T17:46:34.268 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:46:34.268 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:46:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:33 smithi083 ceph-mon[37031]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1022 B/s wr, 23 op/s 2024-08-02T17:46:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:33 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3163815894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:34.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:33 smithi077 ceph-mon[37108]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1022 B/s wr, 23 op/s 2024-08-02T17:46:34.391 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:33 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3163815894' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:34.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 -- 172.21.15.69:0/1392218281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 msgr2=0x7f9cbc106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/1392218281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 0x7f9cbc106ac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009930 tx=0x7f9cac02a420 comp rx=0 tx=0).stop 2024-08-02T17:46:34.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 -- 172.21.15.69:0/1392218281 shutdown_connections 2024-08-02T17:46:34.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/1392218281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cbc101070 0x7f9cbc10d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/1392218281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cbc1006d0 0x7f9cbc100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/1392218281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 0x7f9cbc106ac0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 -- 172.21.15.69:0/1392218281 >> 172.21.15.69:0/1392218281 conn(0x7f9cbc0776d0 msgr2=0x7f9cbc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 -- 172.21.15.69:0/1392218281 shutdown_connections 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 -- 172.21.15.69:0/1392218281 wait complete. 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.376+0000 7f9cc20c4640 1 Processor -- start 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cc20c4640 1 -- start start 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cc20c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cbc1006d0 0x7f9cbc198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cc20c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cbc101070 0x7f9cbc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cc20c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 0x7f9cbc1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cc20c4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc110b30 con 0x7f9cbc1066e0 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cc20c4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc1109b0 con 0x7f9cbc101070 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cc20c4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc110cb0 con 0x7f9cbc1006d0 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 0x7f9cbc1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:34.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 0x7f9cbc1a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36550/0 (socket says 172.21.15.69:36550) 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 -- 172.21.15.69:0/4252817373 learned_addr learned my addr 172.21.15.69:0/4252817373 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cb3fff640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cbc101070 0x7f9cbc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbb7fe640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cbc1006d0 0x7f9cbc198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 -- 172.21.15.69:0/4252817373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cbc1006d0 msgr2=0x7f9cbc198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cbc1006d0 0x7f9cbc198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 -- 172.21.15.69:0/4252817373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cbc101070 msgr2=0x7f9cbc199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cbc101070 0x7f9cbc199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 -- 172.21.15.69:0/4252817373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc1a44e0 con 0x7f9cbc1066e0 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cb3fff640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cbc101070 0x7f9cbc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cbbfff640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 0x7f9cbc1a3fa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400cce0 tx=0x7f9cb400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.377+0000 7f9cb97fa640 1 -- 172.21.15.69:0/4252817373 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb400f450 con 0x7f9cbc1066e0 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.378+0000 7f9cb97fa640 1 -- 172.21.15.69:0/4252817373 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cb400fad0 con 0x7f9cbc1066e0 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.378+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc1a46b0 con 0x7f9cbc1066e0 2024-08-02T17:46:34.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.378+0000 7f9cb97fa640 1 -- 172.21.15.69:0/4252817373 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4018c10 con 0x7f9cbc1066e0 2024-08-02T17:46:34.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.378+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc1a4b80 con 0x7f9cbc1066e0 2024-08-02T17:46:34.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.379+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cbc101e10 con 0x7f9cbc1066e0 2024-08-02T17:46:34.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.380+0000 7f9cb97fa640 1 -- 172.21.15.69:0/4252817373 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f9cb400f5f0 con 0x7f9cbc1066e0 2024-08-02T17:46:34.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.380+0000 7f9cb97fa640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9ca80a2e10 0x7f9ca80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:34.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.381+0000 7f9cbb7fe640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9ca80a2e10 0x7f9ca80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:34.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.381+0000 7f9cb97fa640 1 -- 172.21.15.69:0/4252817373 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f9cb40d0200 con 0x7f9cbc1066e0 2024-08-02T17:46:34.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.381+0000 7f9cbb7fe640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9ca80a2e10 0x7f9ca80a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0096c0 tx=0x7f9cac005fd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:34.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.384+0000 7f9cb97fa640 1 -- 172.21.15.69:0/4252817373 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9cb4014030 con 0x7f9cbc1066e0 2024-08-02T17:46:34.542 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:34.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.589+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f9cbc100b30 con 0x7f9cbc1066e0 2024-08-02T17:46:34.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.593+0000 7f9cb97fa640 1 -- 172.21.15.69:0/4252817373 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f9cb409c8a0 con 0x7f9cbc1066e0 2024-08-02T17:46:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.595+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9ca80a2e10 msgr2=0x7f9ca80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.595+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9ca80a2e10 0x7f9ca80a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0096c0 tx=0x7f9cac005fd0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.595+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 msgr2=0x7f9cbc1a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.595+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 0x7f9cbc1a3fa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9cb400cce0 tx=0x7f9cb400a620 comp rx=0 tx=0).stop 2024-08-02T17:46:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.596+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 shutdown_connections 2024-08-02T17:46:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.596+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9cbc1066e0 0x7f9cbc1a3fa0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.596+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9ca80a2e10 0x7f9ca80a52d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.596+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9cbc101070 0x7f9cbc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.596+0000 7f9cc20c4640 1 --2- 172.21.15.69:0/4252817373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9cbc1006d0 0x7f9cbc198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.596+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 >> 172.21.15.69:0/4252817373 conn(0x7f9cbc0776d0 msgr2=0x7f9cbc103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.596+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 shutdown_connections 2024-08-02T17:46:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.596+0000 7f9cc20c4640 1 -- 172.21.15.69:0/4252817373 wait complete. 2024-08-02T17:46:34.609 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-08-02T17:46:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 -- 172.21.15.69:0/326250449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 msgr2=0x7f68d410d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 --2- 172.21.15.69:0/326250449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 0x7f68d410d5e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009cd0 tx=0x7f68cc02a8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 -- 172.21.15.69:0/326250449 shutdown_connections 2024-08-02T17:46:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 --2- 172.21.15.69:0/326250449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 0x7f68d410d5e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 --2- 172.21.15.69:0/326250449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d4100710 0x7f68d4100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 --2- 172.21.15.69:0/326250449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d4106720 0x7f68d4106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 -- 172.21.15.69:0/326250449 >> 172.21.15.69:0/326250449 conn(0x7f68d40fc3f0 msgr2=0x7f68d40fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 -- 172.21.15.69:0/326250449 shutdown_connections 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.712+0000 7f68dad7a640 1 -- 172.21.15.69:0/326250449 wait complete. 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68dad7a640 1 Processor -- start 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68dad7a640 1 -- start start 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68dad7a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d4100710 0x7f68d4198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68dad7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 0x7f68d4198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68dad7a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d4106720 0x7f68d419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68dad7a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4111c50 con 0x7f68d41010b0 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68dad7a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4111ad0 con 0x7f68d4106720 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68dad7a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68d4111dd0 con 0x7f68d4100710 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 0x7f68d4198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 0x7f68d4198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36570/0 (socket says 172.21.15.69:36570) 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d3fff640 1 -- 172.21.15.69:0/2264402850 learned_addr learned my addr 172.21.15.69:0/2264402850 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d8b16640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d4100710 0x7f68d4198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d3fff640 1 -- 172.21.15.69:0/2264402850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d4100710 msgr2=0x7f68d4198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d3fff640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d4100710 0x7f68d4198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d3fff640 1 -- 172.21.15.69:0/2264402850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d4106720 msgr2=0x7f68d419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d9317640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d4106720 0x7f68d419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d3fff640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d4106720 0x7f68d419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d3fff640 1 -- 172.21.15.69:0/2264402850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d419d130 con 0x7f68d41010b0 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d8b16640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d4100710 0x7f68d4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.713+0000 7f68d9317640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d4106720 0x7f68d419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.714+0000 7f68d3fff640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 0x7f68d4198680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f68c000d770 tx=0x7f68c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.714+0000 7f68d1ffb640 1 -- 172.21.15.69:0/2264402850 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0017070 con 0x7f68d41010b0 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.714+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d419d420 con 0x7f68d41010b0 2024-08-02T17:46:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.714+0000 7f68d1ffb640 1 -- 172.21.15.69:0/2264402850 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68c0005500 con 0x7f68d41010b0 2024-08-02T17:46:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.714+0000 7f68d1ffb640 1 -- 172.21.15.69:0/2264402850 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0004e00 con 0x7f68d41010b0 2024-08-02T17:46:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.714+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d41a4cb0 con 0x7f68d41010b0 2024-08-02T17:46:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.715+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68d4100b70 con 0x7f68d41010b0 2024-08-02T17:46:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.716+0000 7f68d1ffb640 1 -- 172.21.15.69:0/2264402850 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f68c0004fa0 con 0x7f68d41010b0 2024-08-02T17:46:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.716+0000 7f68d1ffb640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68c40a2e10 0x7f68c40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.717+0000 7f68d8b16640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68c40a2e10 0x7f68c40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.717+0000 7f68d1ffb640 1 -- 172.21.15.69:0/2264402850 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f68c00cf5f0 con 0x7f68d41010b0 2024-08-02T17:46:34.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.717+0000 7f68d8b16640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68c40a2e10 0x7f68c40a52d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f68c8005d90 tx=0x7f68c8005d20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:34.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.721+0000 7f68d1ffb640 1 -- 172.21.15.69:0/2264402850 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f68c009bc00 con 0x7f68d41010b0 2024-08-02T17:46:34.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.930+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f68d4108cd0 con 0x7f68d41010b0 2024-08-02T17:46:34.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.935+0000 7f68d1ffb640 1 -- 172.21.15.69:0/2264402850 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f68c009b500 con 0x7f68d41010b0 2024-08-02T17:46:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68c40a2e10 msgr2=0x7f68c40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68c40a2e10 0x7f68c40a52d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f68c8005d90 tx=0x7f68c8005d20 comp rx=0 tx=0).stop 2024-08-02T17:46:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 msgr2=0x7f68d4198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 0x7f68d4198680 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f68c000d770 tx=0x7f68c000dc30 comp rx=0 tx=0).stop 2024-08-02T17:46:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 shutdown_connections 2024-08-02T17:46:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68d41010b0 0x7f68d4198680 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68c40a2e10 0x7f68c40a52d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68d4106720 0x7f68d419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 --2- 172.21.15.69:0/2264402850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68d4100710 0x7f68d4198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 >> 172.21.15.69:0/2264402850 conn(0x7f68d40fc3f0 msgr2=0x7f68d40fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.937+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 shutdown_connections 2024-08-02T17:46:34.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:34.938+0000 7f68dad7a640 1 -- 172.21.15.69:0/2264402850 wait complete. 2024-08-02T17:46:34.951 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 -- 172.21.15.69:0/678661934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc40ffe30 msgr2=0x7fdcc4100290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 --2- 172.21.15.69:0/678661934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc40ffe30 0x7fdcc4100290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009900 tx=0x7fdcb802a410 comp rx=0 tx=0).stop 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 -- 172.21.15.69:0/678661934 shutdown_connections 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 --2- 172.21.15.69:0/678661934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc40ffe30 0x7fdcc4100290 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 --2- 172.21.15.69:0/678661934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc41007d0 0x7fdcc410e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 --2- 172.21.15.69:0/678661934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcc41068b0 0x7fdcc4106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 -- 172.21.15.69:0/678661934 >> 172.21.15.69:0/678661934 conn(0x7fdcc4076f60 msgr2=0x7fdcc40fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 -- 172.21.15.69:0/678661934 shutdown_connections 2024-08-02T17:46:35.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.024+0000 7fdcc987e640 1 -- 172.21.15.69:0/678661934 wait complete. 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc987e640 1 Processor -- start 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc987e640 1 -- start start 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc987e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40ffe30 0x7fdcc4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc987e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcc41007d0 0x7fdcc4198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc987e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc41068b0 0x7fdcc419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc987e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111d80 con 0x7fdcc41068b0 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc987e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111c00 con 0x7fdcc41007d0 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc987e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4111f00 con 0x7fdcc40ffe30 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc41068b0 0x7fdcc419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc41068b0 0x7fdcc419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38592/0 (socket says 172.21.15.69:38592) 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.025+0000 7fdcc37fe640 1 -- 172.21.15.69:0/3175495214 learned_addr learned my addr 172.21.15.69:0/3175495214 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:35.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcc41007d0 0x7fdcc4198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40ffe30 0x7fdcc4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc37fe640 1 -- 172.21.15.69:0/3175495214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40ffe30 msgr2=0x7fdcc4198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40ffe30 0x7fdcc4198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc37fe640 1 -- 172.21.15.69:0/3175495214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcc41007d0 msgr2=0x7fdcc4198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcc41007d0 0x7fdcc4198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc37fe640 1 -- 172.21.15.69:0/3175495214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc419d1d0 con 0x7fdcc41068b0 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc41068b0 0x7fdcc419caf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e970 tx=0x7fdcb400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc887c640 1 -- 172.21.15.69:0/3175495214 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb400ccc0 con 0x7fdcc41068b0 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc887c640 1 -- 172.21.15.69:0/3175495214 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4005500 con 0x7fdcc41068b0 2024-08-02T17:46:35.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc887c640 1 -- 172.21.15.69:0/3175495214 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4010690 con 0x7fdcc41068b0 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc987e640 1 -- 172.21.15.69:0/3175495214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc419d4c0 con 0x7fdcc41068b0 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.026+0000 7fdcc987e640 1 -- 172.21.15.69:0/3175495214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41a4da0 con 0x7fdcc41068b0 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdcc887c640 1 -- 172.21.15.69:0/3175495214 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fdcb40046c0 con 0x7fdcc41068b0 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdccbc7c740 1 Processor -- start 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdccbc7c740 1 -- start start 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5f100 0x55867fd5f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdcc887c640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdcac0a2e90 0x7fdcac0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd6e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 0x55867fd70be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55867fd22e40 con 0x55867fd5fa20 2024-08-02T17:46:35.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55867fcc47f0 con 0x55867fd5f100 2024-08-02T17:46:35.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.029+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55867fc6bfb0 con 0x55867fd6e7f0 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd6e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdcac0a2e90 0x7fdcac0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc887c640 1 -- 172.21.15.69:0/3175495214 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fdcb40cfde0 con 0x7fdcc41068b0 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd6e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38602/0 (socket says 172.21.15.69:38602) 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 -- 172.21.15.69:0/1607592748 learned_addr learned my addr 172.21.15.69:0/1607592748 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 0x55867fd70be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5f100 0x55867fd5f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 -- 172.21.15.69:0/1607592748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 msgr2=0x55867fd70be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 0x55867fd70be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 -- 172.21.15.69:0/1607592748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 msgr2=0x55867fd6e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd6e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 -- 172.21.15.69:0/1607592748 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55867fa366a0 con 0x55867fd5f100 2024-08-02T17:46:35.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdcac0a2e90 0x7fdcac0a5350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005fd0 tx=0x7fdcbc009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.030+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5f100 0x55867fd5f4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdcb802a930 tx=0x7fdcb802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c339c8eb8087ef51 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.031+0000 7fdca9ffb640 1 -- 172.21.15.69:0/1607592748 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8035040 con 0x55867fd5f100 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.031+0000 7fdca9ffb640 1 -- 172.21.15.69:0/1607592748 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb8041b60 con 0x55867fd5f100 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.031+0000 7fdca9ffb640 1 -- 172.21.15.69:0/1607592748 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80047a0 con 0x55867fd5f100 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.031+0000 7fdccbc7c740 1 -- 172.21.15.69:0/1607592748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5f100 msgr2=0x55867fd5f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.031+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5f100 0x55867fd5f4e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdcb802a930 tx=0x7fdcb802a410 comp rx=0 tx=0).stop 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 -- 172.21.15.69:0/1607592748 shutdown_connections 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd6e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 0x55867fd70be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/1607592748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5f100 0x55867fd5f4e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 -- 172.21.15.69:0/1607592748 >> 172.21.15.69:0/1607592748 conn(0x55867fd5b790 msgr2=0x55867fd5bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 -- 172.21.15.69:0/1607592748 shutdown_connections 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 -- 172.21.15.69:0/1607592748 wait complete. 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 -- start start 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 Processor -- start 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd67840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.032+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5fa20 0x55867fd67d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55867fc7fcb0 con 0x55867fd5f100 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55867fcc07b0 con 0x55867fd6e7f0 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55867fd62020 con 0x55867fd5fa20 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdccbc7c740 10 client.0 _ll_register_callbacks cb 0x55867f9d3ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd67840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd67840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38606/0 (socket says 172.21.15.69:38606) 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc2ffd640 1 -- 172.21.15.69:0/3720438862 learned_addr learned my addr 172.21.15.69:0/3720438862 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5fa20 0x55867fd67d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc37fe640 1 -- 172.21.15.69:0/3720438862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5fa20 msgr2=0x55867fd67d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5fa20 0x55867fd67d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc37fe640 1 -- 172.21.15.69:0/3720438862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 msgr2=0x55867fd67840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd67840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc37fe640 1 -- 172.21.15.69:0/3720438862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55867fc7ebc0 con 0x55867fd6e7f0 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd67840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd682c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40ccff0 tx=0x7fdcb409b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2db3adfa282cbe1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdca2ffd640 1 -- 172.21.15.69:0/3720438862 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40056b0 con 0x55867fd6e7f0 2024-08-02T17:46:35.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdca2ffd640 1 -- 172.21.15.69:0/3720438862 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fdcb40a44d0 con 0x55867fd6e7f0 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.033+0000 7fdccbc7c740 1 -- 172.21.15.69:0/3720438862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb8041b60 con 0x55867fd6e7f0 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdca2ffd640 1 -- 172.21.15.69:0/3720438862 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdcb409c3f0 con 0x55867fd6e7f0 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdccbc7c740 10 client.25037 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdccbc7c740 1 -- 172.21.15.69:0/3720438862 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55867fd78fb0 con 0x55867fd6e7f0 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdca2ffd640 1 -- 172.21.15.69:0/3720438862 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb8041b60 con 0x55867fd6e7f0 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdca2ffd640 1 -- 172.21.15.69:0/3720438862 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdcb40cd2d0 con 0x55867fd6e7f0 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdccbc7c740 10 client.25037 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdccbc7c740 10 client.25037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdccbc7c740 20 client.25037 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55867fd7a880 0x55867fd7cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdccbc7c740 4 client.25037 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdccbc7c740 1 -- 172.21.15.69:0/3720438862 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55867fcd1d60 con 0x55867fd7a880 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.034+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55867fd7a880 0x55867fd7cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.035+0000 7fdca2ffd640 10 client.25037 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.035+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55867fd7a880 0x55867fd7cc90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b403a419e756e3 server_cookie=fda0b8dc0f121b89 in_seq=0 out_seq=0 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.035+0000 7fdca2ffd640 1 -- 172.21.15.69:0/3720438862 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55867fcd1d60 con 0x55867fd7a880 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.035+0000 7fdca2ffd640 10 client.25037 handle_command_reply: tid=0 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.035+0000 7fdccbc7c740 1 client.25037 shutdown 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 -- 172.21.15.69:0/3720438862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 msgr2=0x55867fd682c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd682c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40ccff0 tx=0x7fdcb409b960 comp rx=0 tx=0).stop 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 -- 172.21.15.69:0/3720438862 shutdown_connections 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55867fd7a880 0x55867fd7cc90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd682c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5fa20 0x55867fd67d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/3720438862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd67840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdca2ffd640 0 client.25037 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:35.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 -- 172.21.15.69:0/3720438862 >> 172.21.15.69:0/3720438862 conn(0x55867fd5b790 msgr2=0x55867fd614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 -- 172.21.15.69:0/3720438862 shutdown_connections 2024-08-02T17:46:35.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 1 -- 172.21.15.69:0/3720438862 wait complete. 2024-08-02T17:46:35.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.036+0000 7fdccbc7c740 20 client.25037 trim_cache size 0 max 16384 2024-08-02T17:46:35.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdccbc7c740 1 Processor -- start 2024-08-02T17:46:35.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdccbc7c740 1 -- start start 2024-08-02T17:46:35.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5f100 0x55867fd6b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd68110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd61390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41af2c0 con 0x55867fd5fa20 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41acb90 con 0x55867fd6e7f0 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41aa9e0 con 0x55867fd5f100 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdcc27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd68110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdcc27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd68110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38616/0 (socket says 172.21.15.69:38616) 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdcc27fc640 1 -- 172.21.15.69:0/4259727495 learned_addr learned my addr 172.21.15.69:0/4259727495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5f100 0x55867fd6b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.053+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd61390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdcc27fc640 1 -- 172.21.15.69:0/4259727495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5f100 msgr2=0x55867fd6b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5f100 0x55867fd6b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdcc27fc640 1 -- 172.21.15.69:0/4259727495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 msgr2=0x55867fd61390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:35.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd61390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdcc27fc640 1 -- 172.21.15.69:0/4259727495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55867fd7d2e0 con 0x55867fd5fa20 2024-08-02T17:46:35.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd68110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8031470 tx=0x7fdcb8004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a7b05ee9bbf969 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdca97fa640 1 -- 172.21.15.69:0/4259727495 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8055070 con 0x55867fd5fa20 2024-08-02T17:46:35.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdca97fa640 1 -- 172.21.15.69:0/4259727495 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb8041b60 con 0x55867fd5fa20 2024-08-02T17:46:35.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.054+0000 7fdca97fa640 1 -- 172.21.15.69:0/4259727495 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8048040 con 0x55867fd5fa20 2024-08-02T17:46:35.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 -- 172.21.15.69:0/4259727495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 msgr2=0x55867fd68110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd68110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8031470 tx=0x7fdcb8004250 comp rx=0 tx=0).stop 2024-08-02T17:46:35.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 -- 172.21.15.69:0/4259727495 shutdown_connections 2024-08-02T17:46:35.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5fa20 0x55867fd68110 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd6e7f0 0x55867fd61390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/4259727495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd5f100 0x55867fd6b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 -- 172.21.15.69:0/4259727495 >> 172.21.15.69:0/4259727495 conn(0x55867fd5b790 msgr2=0x55867fd60c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 -- 172.21.15.69:0/4259727495 shutdown_connections 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 -- 172.21.15.69:0/4259727495 wait complete. 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 -- start start 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.055+0000 7fdccbc7c740 1 Processor -- start 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd767a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5fa20 0x55867fd76ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdccbc7c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 0x55867fd71b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ab050 con 0x55867fd5f100 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ac4c0 con 0x55867fd5fa20 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdccbc7c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41ab720 con 0x55867fd6e7f0 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdccbc7c740 10 client.0 _ll_register_callbacks cb 0x7fdcb409bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd767a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38618/0 (socket says 172.21.15.69:38618) 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc2ffd640 1 -- 172.21.15.69:0/529575982 learned_addr learned my addr 172.21.15.69:0/529575982 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5fa20 0x55867fd76ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc37fe640 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 0x55867fd71b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc2ffd640 1 -- 172.21.15.69:0/529575982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 msgr2=0x55867fd71b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 0x55867fd71b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc2ffd640 1 -- 172.21.15.69:0/529575982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5fa20 msgr2=0x55867fd76ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5fa20 0x55867fd76ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.056+0000 7fdcc2ffd640 1 -- 172.21.15.69:0/529575982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55867fd5c920 con 0x55867fd5f100 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.057+0000 7fdcc2ffd640 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd767a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55867fd7d4e0 tx=0x7fdcbc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cc97db8adbed28 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.057+0000 7fdca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.057+0000 7fdca2ffd640 1 -- 172.21.15.69:0/529575982 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc037060 con 0x55867fd5f100 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.057+0000 7fdca2ffd640 1 -- 172.21.15.69:0/529575982 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fdcbc02aa40 con 0x55867fd5f100 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.057+0000 7fdccbc7c740 1 -- 172.21.15.69:0/529575982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb8041b60 con 0x55867fd5f100 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.057+0000 7fdca2ffd640 1 -- 172.21.15.69:0/529575982 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdcbc02ac40 con 0x55867fd5f100 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.057+0000 7fdccbc7c740 10 client.15354 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.057+0000 7fdccbc7c740 1 -- 172.21.15.69:0/529575982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55867fde4fa0 con 0x55867fd5f100 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdca2ffd640 1 -- 172.21.15.69:0/529575982 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb8041b60 con 0x55867fd5f100 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdca2ffd640 1 -- 172.21.15.69:0/529575982 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdcbc01e470 con 0x55867fd5f100 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdccbc7c740 10 client.15354 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdccbc7c740 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdccbc7c740 20 client.15354 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55867fde56a0 0x55867fde7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdccbc7c740 4 client.15354 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:46:35.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdccbc7c740 1 -- 172.21.15.69:0/529575982 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55867fccc270 con 0x55867fde56a0 2024-08-02T17:46:35.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55867fde56a0 0x55867fde7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdca2ffd640 10 client.15354 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:35.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.058+0000 7fdcc27fc640 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55867fde56a0 0x55867fde7ac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f692958648fa58c server_cookie=222956200705eb97 in_seq=0 out_seq=0 2024-08-02T17:46:35.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.059+0000 7fdca2ffd640 1 -- 172.21.15.69:0/529575982 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55867fccc270 con 0x55867fde56a0 2024-08-02T17:46:35.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.059+0000 7fdca2ffd640 10 client.15354 handle_command_reply: tid=0 2024-08-02T17:46:35.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.059+0000 7fdccbc7c740 1 client.15354 shutdown 2024-08-02T17:46:35.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 -- 172.21.15.69:0/529575982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 msgr2=0x55867fd767a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd767a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55867fd7d4e0 tx=0x7fdcbc004ab0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 -- 172.21.15.69:0/529575982 shutdown_connections 2024-08-02T17:46:35.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55867fde56a0 0x55867fde7ac0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55867fd6e7f0 0x55867fd71b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55867fd5fa20 0x55867fd76ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdca2ffd640 0 client.15354 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 --2- 172.21.15.69:0/529575982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55867fd5f100 0x55867fd767a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 -- 172.21.15.69:0/529575982 >> 172.21.15.69:0/529575982 conn(0x55867fd5b790 msgr2=0x55867fd614a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 -- 172.21.15.69:0/529575982 shutdown_connections 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 1 -- 172.21.15.69:0/529575982 wait complete. 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.060+0000 7fdccbc7c740 20 client.15354 trim_cache size 0 max 16384 2024-08-02T17:46:35.066 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:46:35.067 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:46:35.067 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "c68721fd-42ec-46b7-a809-7b8f8e271ec5", 2024-08-02T17:46:35.067 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:46:35.067 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:46:35.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 -- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdcac0a2e90 msgr2=0x7fdcac0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdcac0a2e90 0x7fdcac0a5350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc005fd0 tx=0x7fdcbc009500 comp rx=0 tx=0).stop 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 -- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc41068b0 msgr2=0x7fdcc419caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc41068b0 0x7fdcc419caf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e970 tx=0x7fdcb400ee30 comp rx=0 tx=0).stop 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 -- 172.21.15.69:0/3175495214 shutdown_connections 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdcc41068b0 0x7fdcc419caf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdcac0a2e90 0x7fdcac0a5350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdcc41007d0 0x7fdcc4198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 --2- 172.21.15.69:0/3175495214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdcc40ffe30 0x7fdcc4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 -- 172.21.15.69:0/3175495214 >> 172.21.15.69:0/3175495214 conn(0x7fdcc4076f60 msgr2=0x7fdcc41048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 -- 172.21.15.69:0/3175495214 shutdown_connections 2024-08-02T17:46:35.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.061+0000 7fdcc987e640 1 -- 172.21.15.69:0/3175495214 wait complete. 2024-08-02T17:46:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 -- 172.21.15.69:0/241713761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c381032f0 msgr2=0x7f6c3810b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/241713761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c381032f0 0x7f6c3810b7d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c009900 tx=0x7f6c2c02a480 comp rx=0 tx=0).stop 2024-08-02T17:46:35.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 -- 172.21.15.69:0/241713761 shutdown_connections 2024-08-02T17:46:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/241713761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c381032f0 0x7f6c3810b7d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/241713761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c3810be20 0x7f6c3810e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/241713761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c381029d0 0x7f6c38102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 -- 172.21.15.69:0/241713761 >> 172.21.15.69:0/241713761 conn(0x7f6c380fc3e0 msgr2=0x7f6c380fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 -- 172.21.15.69:0/241713761 shutdown_connections 2024-08-02T17:46:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 -- 172.21.15.69:0/241713761 wait complete. 2024-08-02T17:46:35.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 Processor -- start 2024-08-02T17:46:35.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.070+0000 7f6c3d75e640 1 -- start start 2024-08-02T17:46:35.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c3d75e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c381029d0 0x7f6c38100620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c3d75e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c381032f0 0x7f6c38100b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c3d75e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c3810be20 0x7f6c381010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c3d75e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38111c50 con 0x7f6c3810be20 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c3d75e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38111ad0 con 0x7f6c381029d0 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c3d75e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c38111dd0 con 0x7f6c381032f0 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c3810be20 0x7f6c381010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c3810be20 0x7f6c381010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38644/0 (socket says 172.21.15.69:38644) 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 -- 172.21.15.69:0/1558528368 learned_addr learned my addr 172.21.15.69:0/1558528368 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c367fc640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c381032f0 0x7f6c38100b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c36ffd640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c381029d0 0x7f6c38100620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 -- 172.21.15.69:0/1558528368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c381032f0 msgr2=0x7f6c38100b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c381032f0 0x7f6c38100b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 -- 172.21.15.69:0/1558528368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c381029d0 msgr2=0x7f6c38100620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c381029d0 0x7f6c38100620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 -- 172.21.15.69:0/1558528368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c381a6680 con 0x7f6c3810be20 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c36ffd640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c381029d0 0x7f6c38100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.071+0000 7f6c377fe640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c3810be20 0x7f6c381010a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000eaa0 tx=0x7f6c3000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.072+0000 7f6c1ffff640 1 -- 172.21.15.69:0/1558528368 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000cdb0 con 0x7f6c3810be20 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.072+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c381a6970 con 0x7f6c3810be20 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.072+0000 7f6c1ffff640 1 -- 172.21.15.69:0/1558528368 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c30005020 con 0x7f6c3810be20 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.072+0000 7f6c1ffff640 1 -- 172.21.15.69:0/1558528368 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300116b0 con 0x7f6c3810be20 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.072+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c381a6e40 con 0x7f6c3810be20 2024-08-02T17:46:35.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.073+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c38102db0 con 0x7f6c3810be20 2024-08-02T17:46:35.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.075+0000 7f6c1ffff640 1 -- 172.21.15.69:0/1558528368 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f6c30005560 con 0x7f6c3810be20 2024-08-02T17:46:35.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.075+0000 7f6c1ffff640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c200a2e10 0x7f6c200a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.075+0000 7f6c1ffff640 1 -- 172.21.15.69:0/1558528368 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f6c300d0060 con 0x7f6c3810be20 2024-08-02T17:46:35.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.075+0000 7f6c36ffd640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c200a2e10 0x7f6c200a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.076+0000 7f6c36ffd640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c200a2e10 0x7f6c200a52d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009c20 tx=0x7f6c28005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.079+0000 7f6c1ffff640 1 -- 172.21.15.69:0/1558528368 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6c30016030 con 0x7f6c3810be20 2024-08-02T17:46:35.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.271+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f6c38101b50 con 0x7f6c3810be20 2024-08-02T17:46:35.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.275+0000 7f6c1ffff640 1 -- 172.21.15.69:0/1558528368 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7f6c38101b50 con 0x7f6c3810be20 2024-08-02T17:46:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c200a2e10 msgr2=0x7f6c200a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c200a2e10 0x7f6c200a52d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c28009c20 tx=0x7f6c28005f90 comp rx=0 tx=0).stop 2024-08-02T17:46:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c3810be20 msgr2=0x7f6c381010a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c3810be20 0x7f6c381010a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000eaa0 tx=0x7f6c3000ef60 comp rx=0 tx=0).stop 2024-08-02T17:46:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 shutdown_connections 2024-08-02T17:46:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c3810be20 0x7f6c381010a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c200a2e10 0x7f6c200a52d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c381032f0 0x7f6c38100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 --2- 172.21.15.69:0/1558528368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c381029d0 0x7f6c38100620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.277+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 >> 172.21.15.69:0/1558528368 conn(0x7f6c380fc3e0 msgr2=0x7f6c380fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.278+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 shutdown_connections 2024-08-02T17:46:35.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.278+0000 7f6c3d75e640 1 -- 172.21.15.69:0/1558528368 wait complete. 2024-08-02T17:46:35.293 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-08-02T17:46:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 -- 172.21.15.69:0/683394237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 msgr2=0x7f0294106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 --2- 172.21.15.69:0/683394237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 0x7f0294106ac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0290009930 tx=0x7f029002a420 comp rx=0 tx=0).stop 2024-08-02T17:46:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 -- 172.21.15.69:0/683394237 shutdown_connections 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 --2- 172.21.15.69:0/683394237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0294101070 0x7f029410d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 --2- 172.21.15.69:0/683394237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02941006d0 0x7f0294100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 --2- 172.21.15.69:0/683394237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 0x7f0294106ac0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 -- 172.21.15.69:0/683394237 >> 172.21.15.69:0/683394237 conn(0x7f02940776d0 msgr2=0x7f02940fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 -- 172.21.15.69:0/683394237 shutdown_connections 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 -- 172.21.15.69:0/683394237 wait complete. 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 Processor -- start 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.398+0000 7f029b643640 1 -- start start 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f029b643640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02941006d0 0x7f0294198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f029b643640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294101070 0x7f0294199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f029b643640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 0x7f02941a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f029b643640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0294110b20 con 0x7f02941066e0 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f029b643640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02941109a0 con 0x7f0294101070 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f029b643640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0294110ca0 con 0x7f02941006d0 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 0x7f02941a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 0x7f02941a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38666/0 (socket says 172.21.15.69:38666) 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 -- 172.21.15.69:0/265006479 learned_addr learned my addr 172.21.15.69:0/265006479 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f02993df640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02941006d0 0x7f0294198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0298bde640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294101070 0x7f0294199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 -- 172.21.15.69:0/265006479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02941006d0 msgr2=0x7f0294198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02941006d0 0x7f0294198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 -- 172.21.15.69:0/265006479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294101070 msgr2=0x7f0294199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294101070 0x7f0294199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 -- 172.21.15.69:0/265006479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02941a4590 con 0x7f02941066e0 2024-08-02T17:46:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f0299be0640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 0x7f02941a4050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f028c00eb10 tx=0x7f028c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f028a7fc640 1 -- 172.21.15.69:0/265006479 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c004b20 con 0x7f02941066e0 2024-08-02T17:46:35.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02941a4760 con 0x7f02941066e0 2024-08-02T17:46:35.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f028a7fc640 1 -- 172.21.15.69:0/265006479 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f028c004cc0 con 0x7f02941066e0 2024-08-02T17:46:35.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f028a7fc640 1 -- 172.21.15.69:0/265006479 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028c010760 con 0x7f02941066e0 2024-08-02T17:46:35.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.399+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02941a4cd0 con 0x7f02941066e0 2024-08-02T17:46:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.401+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0294100b30 con 0x7f02941066e0 2024-08-02T17:46:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.402+0000 7f028a7fc640 1 -- 172.21.15.69:0/265006479 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f028c005020 con 0x7f02941066e0 2024-08-02T17:46:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.402+0000 7f028a7fc640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02800a2e90 0x7f02800a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.402+0000 7f02993df640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02800a2e90 0x7f02800a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.402+0000 7f028a7fc640 1 -- 172.21.15.69:0/265006479 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f028c0d01b0 con 0x7f02941066e0 2024-08-02T17:46:35.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.403+0000 7f02993df640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02800a2e90 0x7f02800a5350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0290000c00 tx=0x7f0290005a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.409+0000 7f028a7fc640 1 -- 172.21.15.69:0/265006479 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f028c09c7c0 con 0x7f02941066e0 2024-08-02T17:46:35.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.611+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0294062360 con 0x7f02800a2e90 2024-08-02T17:46:35.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.679+0000 7f028a7fc640 1 -- 172.21.15.69:0/265006479 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0294062360 con 0x7f02800a2e90 2024-08-02T17:46:35.681 INFO:teuthology.orchestra.run.smithi069.stdout:Schedule set for path / 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02800a2e90 msgr2=0x7f02800a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02800a2e90 0x7f02800a5350 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0290000c00 tx=0x7f0290005a20 comp rx=0 tx=0).stop 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 msgr2=0x7f02941a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 0x7f02941a4050 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f028c00eb10 tx=0x7f028c00c2d0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 shutdown_connections 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02941066e0 0x7f02941a4050 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02800a2e90 0x7f02800a5350 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0294101070 0x7f0294199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 --2- 172.21.15.69:0/265006479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02941006d0 0x7f0294198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 >> 172.21.15.69:0/265006479 conn(0x7f02940776d0 msgr2=0x7f02941036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 shutdown_connections 2024-08-02T17:46:35.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.681+0000 7f029b643640 1 -- 172.21.15.69:0/265006479 wait complete. 2024-08-02T17:46:35.693 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-08-02T17:46:35.755 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:46:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 -- 172.21.15.69:0/480178269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401032d0 msgr2=0x7f564010b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 --2- 172.21.15.69:0/480178269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401032d0 0x7f564010b7b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5634009900 tx=0x7f563402a480 comp rx=0 tx=0).stop 2024-08-02T17:46:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 -- 172.21.15.69:0/480178269 shutdown_connections 2024-08-02T17:46:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 --2- 172.21.15.69:0/480178269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401032d0 0x7f564010b7b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 --2- 172.21.15.69:0/480178269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f564010bdd0 0x7f564010e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 --2- 172.21.15.69:0/480178269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56401029b0 0x7f5640102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 -- 172.21.15.69:0/480178269 >> 172.21.15.69:0/480178269 conn(0x7f56400776d0 msgr2=0x7f56400fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 -- 172.21.15.69:0/480178269 shutdown_connections 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 -- 172.21.15.69:0/480178269 wait complete. 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.801+0000 7f56455ab640 1 Processor -- start 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f56455ab640 1 -- start start 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f56455ab640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401029b0 0x7f56401980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f56455ab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56401032d0 0x7f5640198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f56455ab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f564010bdd0 0x7f564019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f56455ab640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5640111bd0 con 0x7f56401029b0 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f56455ab640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5640111a50 con 0x7f564010bdd0 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f56455ab640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5640111d50 con 0x7f56401032d0 2024-08-02T17:46:35.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401029b0 0x7f56401980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401029b0 0x7f56401980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38680/0 (socket says 172.21.15.69:38680) 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 -- 172.21.15.69:0/660824143 learned_addr learned my addr 172.21.15.69:0/660824143 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563f7fe640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f564010bdd0 0x7f564019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 -- 172.21.15.69:0/660824143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56401032d0 msgr2=0x7f5640198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56401032d0 0x7f5640198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 -- 172.21.15.69:0/660824143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f564010bdd0 msgr2=0x7f564019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f564010bdd0 0x7f564019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 -- 172.21.15.69:0/660824143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564019d0b0 con 0x7f56401029b0 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.802+0000 7f563effd640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401029b0 0x7f56401980c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f563800c970 tx=0x7f563800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.803+0000 7f5627fff640 1 -- 172.21.15.69:0/660824143 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638007bb0 con 0x7f56401029b0 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.803+0000 7f5627fff640 1 -- 172.21.15.69:0/660824143 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5638007d50 con 0x7f56401029b0 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.803+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564019d3a0 con 0x7f56401029b0 2024-08-02T17:46:35.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.803+0000 7f5627fff640 1 -- 172.21.15.69:0/660824143 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563800f450 con 0x7f56401029b0 2024-08-02T17:46:35.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.803+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56401a4c30 con 0x7f56401029b0 2024-08-02T17:46:35.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.804+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f564006a010 con 0x7f56401029b0 2024-08-02T17:46:35.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.805+0000 7f5627fff640 1 -- 172.21.15.69:0/660824143 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f5638004fe0 con 0x7f56401029b0 2024-08-02T17:46:35.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.805+0000 7f5627fff640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f56280a2e90 0x7f56280a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:35.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.806+0000 7f563e7fc640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f56280a2e90 0x7f56280a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:35.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.806+0000 7f5627fff640 1 -- 172.21.15.69:0/660824143 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f56380cfc60 con 0x7f56401029b0 2024-08-02T17:46:35.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.806+0000 7f563e7fc640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f56280a2e90 0x7f56280a5350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f563402a9a0 tx=0x7f5634000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:35.813+0000 7f5627fff640 1 -- 172.21.15.69:0/660824143 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5638014030 con 0x7f56401029b0 2024-08-02T17:46:35.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4252817373' entity='client.admin' 2024-08-02T17:46:35.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:35.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:35.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:35.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:35.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2264402850' entity='client.admin' 2024-08-02T17:46:35.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 795 B/s wr, 18 op/s 2024-08-02T17:46:35.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1558528368' entity='client.admin' 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:35.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:35 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:35.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4252817373' entity='client.admin' 2024-08-02T17:46:35.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:35.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:35.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:35.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2264402850' entity='client.admin' 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 795 B/s wr, 18 op/s 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1558528368' entity='client.admin' 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:35.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:35 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:36.021 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:36.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4252817373' entity='client.admin' 2024-08-02T17:46:36.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:36.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:36.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2264402850' entity='client.admin' 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 795 B/s wr, 18 op/s 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:46:36.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:46:36.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:46:36.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:46:36.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1558528368' entity='client.admin' 2024-08-02T17:46:36.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:46:36.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:46:36.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:46:36.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:35 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:46:36.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.031+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f5640062360 con 0x7f56280a2e90 2024-08-02T17:46:36.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.091+0000 7f5627fff640 1 -- 172.21.15.69:0/660824143 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f5640062360 con 0x7f56280a2e90 2024-08-02T17:46:36.093 INFO:teuthology.orchestra.run.smithi069.stdout:Retention added to path / 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f56280a2e90 msgr2=0x7f56280a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f56280a2e90 0x7f56280a5350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f563402a9a0 tx=0x7f5634000bc0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401029b0 msgr2=0x7f56401980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401029b0 0x7f56401980c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f563800c970 tx=0x7f563800ce30 comp rx=0 tx=0).stop 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 shutdown_connections 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f56280a2e90 0x7f56280a5350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f564010bdd0 0x7f564019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f56401032d0 0x7f5640198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 --2- 172.21.15.69:0/660824143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f56401029b0 0x7f56401980c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 >> 172.21.15.69:0/660824143 conn(0x7f56400776d0 msgr2=0x7f56400fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 shutdown_connections 2024-08-02T17:46:36.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.093+0000 7f56455ab640 1 -- 172.21.15.69:0/660824143 wait complete. 2024-08-02T17:46:36.105 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-02T17:46:36.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.209+0000 7f5416db2640 1 -- 172.21.15.69:0/2498789827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 msgr2=0x7f5410106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.209+0000 7f5416db2640 1 --2- 172.21.15.69:0/2498789827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 0x7f5410106ac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5400009cd0 tx=0x7f540002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.209+0000 7f5416db2640 1 -- 172.21.15.69:0/2498789827 shutdown_connections 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 --2- 172.21.15.69:0/2498789827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5410101070 0x7f541010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 --2- 172.21.15.69:0/2498789827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54101006d0 0x7f5410100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 --2- 172.21.15.69:0/2498789827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 0x7f5410106ac0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 -- 172.21.15.69:0/2498789827 >> 172.21.15.69:0/2498789827 conn(0x7f54100776d0 msgr2=0x7f54100fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 -- 172.21.15.69:0/2498789827 shutdown_connections 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 -- 172.21.15.69:0/2498789827 wait complete. 2024-08-02T17:46:36.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 Processor -- start 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 -- start start 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54101006d0 0x7f5410197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5410101070 0x7f5410197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 0x7f541019b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5410110b20 con 0x7f54101066e0 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54101109a0 con 0x7f54101006d0 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f5416db2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5410110ca0 con 0x7f5410101070 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f541534f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 0x7f541019b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f541534f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 0x7f541019b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38706/0 (socket says 172.21.15.69:38706) 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.210+0000 7f541534f640 1 -- 172.21.15.69:0/3853389701 learned_addr learned my addr 172.21.15.69:0/3853389701 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f540ffff640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5410101070 0x7f5410197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f5414b4e640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54101006d0 0x7f5410197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f541534f640 1 -- 172.21.15.69:0/3853389701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5410101070 msgr2=0x7f5410197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f541534f640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5410101070 0x7f5410197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f541534f640 1 -- 172.21.15.69:0/3853389701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54101006d0 msgr2=0x7f5410197050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f541534f640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54101006d0 0x7f5410197050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f541534f640 1 -- 172.21.15.69:0/3853389701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541019c040 con 0x7f54101066e0 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f541534f640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 0x7f541019b910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5408009860 tx=0x7f5408009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f540dffb640 1 -- 172.21.15.69:0/3853389701 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408005020 con 0x7f54101066e0 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541019c330 con 0x7f54101066e0 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f540dffb640 1 -- 172.21.15.69:0/3853389701 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54080047a0 con 0x7f54101066e0 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f540dffb640 1 -- 172.21.15.69:0/3853389701 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408007b20 con 0x7f54101066e0 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.211+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5410069740 con 0x7f54101066e0 2024-08-02T17:46:36.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.212+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5410100b30 con 0x7f54101066e0 2024-08-02T17:46:36.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.213+0000 7f540dffb640 1 -- 172.21.15.69:0/3853389701 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f5408007cc0 con 0x7f54101066e0 2024-08-02T17:46:36.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.214+0000 7f540dffb640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f53fc0a2e10 0x7f53fc0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.214+0000 7f5414b4e640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f53fc0a2e10 0x7f53fc0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.214+0000 7f540dffb640 1 -- 172.21.15.69:0/3853389701 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f54080d0e40 con 0x7f54101066e0 2024-08-02T17:46:36.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.215+0000 7f5414b4e640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f53fc0a2e10 0x7f53fc0a52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5400006010 tx=0x7f5400009360 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.218+0000 7f540dffb640 1 -- 172.21.15.69:0/3853389701 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f540809ce00 con 0x7f54101066e0 2024-08-02T17:46:36.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.426+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f541006a1d0 con 0x7f53fc0a2e10 2024-08-02T17:46:36.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.434+0000 7f540dffb640 1 -- 172.21.15.69:0/3853389701 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f541006a1d0 con 0x7f53fc0a2e10 2024-08-02T17:46:36.435 INFO:teuthology.orchestra.run.smithi069.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-02T00:00:00", "created": "2024-08-02T17:46:35", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f53fc0a2e10 msgr2=0x7f53fc0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f53fc0a2e10 0x7f53fc0a52d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5400006010 tx=0x7f5400009360 comp rx=0 tx=0).stop 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 msgr2=0x7f541019b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 0x7f541019b910 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5408009860 tx=0x7f5408009d20 comp rx=0 tx=0).stop 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 shutdown_connections 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54101066e0 0x7f541019b910 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f53fc0a2e10 0x7f53fc0a52d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5410101070 0x7f5410197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 --2- 172.21.15.69:0/3853389701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54101006d0 0x7f5410197050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 >> 172.21.15.69:0/3853389701 conn(0x7f54100776d0 msgr2=0x7f54101036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 shutdown_connections 2024-08-02T17:46:36.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.436+0000 7f5416db2640 1 -- 172.21.15.69:0/3853389701 wait complete. 2024-08-02T17:46:36.449 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-02T17:46:36.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 -- 172.21.15.69:0/3254849657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 msgr2=0x7f778c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 --2- 172.21.15.69:0/3254849657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 0x7f778c10d770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f777c009900 tx=0x7f777c02a480 comp rx=0 tx=0).stop 2024-08-02T17:46:36.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 -- 172.21.15.69:0/3254849657 shutdown_connections 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 --2- 172.21.15.69:0/3254849657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 0x7f778c10d770 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 --2- 172.21.15.69:0/3254849657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f778c1022a0 0x7f778c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 --2- 172.21.15.69:0/3254849657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f778c100b50 0x7f778c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 -- 172.21.15.69:0/3254849657 >> 172.21.15.69:0/3254849657 conn(0x7f778c0fc500 msgr2=0x7f778c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 -- 172.21.15.69:0/3254849657 shutdown_connections 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.516+0000 7f7790b2f640 1 -- 172.21.15.69:0/3254849657 wait complete. 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f7790b2f640 1 Processor -- start 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f7790b2f640 1 -- start start 2024-08-02T17:46:36.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f7790b2f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f778c100b50 0x7f778c197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f7790b2f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f778c1022a0 0x7f778c197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f7790b2f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 0x7f778c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f7790b2f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f778c110cb0 con 0x7f778c102c20 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f7790b2f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f778c110b30 con 0x7f778c1022a0 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f7790b2f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f778c110e30 con 0x7f778c100b50 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f778a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f778c100b50 0x7f778c197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f778ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 0x7f778c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f778a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f778c100b50 0x7f778c197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54756/0 (socket says 172.21.15.69:54756) 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f778ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 0x7f778c19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38724/0 (socket says 172.21.15.69:38724) 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.517+0000 7f778a59c640 1 -- 172.21.15.69:0/1088908264 learned_addr learned my addr 172.21.15.69:0/1088908264 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:36.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f7789d9b640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f778c1022a0 0x7f778c197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f778a59c640 1 -- 172.21.15.69:0/1088908264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f778c1022a0 msgr2=0x7f778c197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f778a59c640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f778c1022a0 0x7f778c197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f778a59c640 1 -- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 msgr2=0x7f778c19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f778a59c640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 0x7f778c19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f778a59c640 1 -- 172.21.15.69:0/1088908264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c19c0e0 con 0x7f778c100b50 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f778a59c640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f778c100b50 0x7f778c197130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f778400eab0 tx=0x7f778400ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f77837fe640 1 -- 172.21.15.69:0/1088908264 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778400ce40 con 0x7f778c100b50 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f7790b2f640 1 -- 172.21.15.69:0/1088908264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c19c3d0 con 0x7f778c100b50 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f77837fe640 1 -- 172.21.15.69:0/1088908264 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7784011950 con 0x7f778c100b50 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f77837fe640 1 -- 172.21.15.69:0/1088908264 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784010ae0 con 0x7f778c100b50 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.518+0000 7f7790b2f640 1 -- 172.21.15.69:0/1088908264 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c069ca0 con 0x7f778c100b50 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.520+0000 7f77837fe640 1 -- 172.21.15.69:0/1088908264 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f7784011b00 con 0x7f778c100b50 2024-08-02T17:46:36.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7792f2d740 1 Processor -- start 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7792f2d740 1 -- start start 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2b30 0x561a568eceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568ed3f0 0x561a568ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7792f2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561a56847980 con 0x561a568ed3f0 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7792f2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a567dfa70 con 0x561a568e2b30 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7792f2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561a56843940 con 0x561a568e2210 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f778ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7789d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568ed3f0 0x561a568ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f778ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568e25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54766/0 (socket says 172.21.15.69:54766) 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f7789d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568ed3f0 0x561a568ef7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38726/0 (socket says 172.21.15.69:38726) 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f778ad9d640 1 -- 172.21.15.69:0/2330796800 learned_addr learned my addr 172.21.15.69:0/2330796800 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:36.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f77837fe640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f77780a2e10 0x7f77780a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.521+0000 7f778a59c640 1 --2- 172.21.15.69:0/2330796800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2b30 0x561a568eceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f778ad9d640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f77780a2e10 0x7f77780a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7789d9b640 1 -- 172.21.15.69:0/2330796800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 msgr2=0x561a568e25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7789d9b640 1 --2- 172.21.15.69:0/2330796800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568e25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7789d9b640 1 -- 172.21.15.69:0/2330796800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2b30 msgr2=0x561a568eceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7789d9b640 1 --2- 172.21.15.69:0/2330796800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2b30 0x561a568eceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7789d9b640 1 -- 172.21.15.69:0/2330796800 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a568426e0 con 0x561a568ed3f0 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f77837fe640 1 -- 172.21.15.69:0/1088908264 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f77840d0210 con 0x7f778c100b50 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7789d9b640 1 --2- 172.21.15.69:0/2330796800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568ed3f0 0x561a568ef7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7774009b40 tx=0x7f77740113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e52051bfc8136c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f778ad9d640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f77780a2e10 0x7f77780a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f777c0096c0 tx=0x7f777c02a480 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7780ff9640 1 -- 172.21.15.69:0/2330796800 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774011e10 con 0x561a568ed3f0 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7780ff9640 1 -- 172.21.15.69:0/2330796800 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7774012930 con 0x561a568ed3f0 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.522+0000 7f7780ff9640 1 -- 172.21.15.69:0/2330796800 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777400fa90 con 0x561a568ed3f0 2024-08-02T17:46:36.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 -- 172.21.15.69:0/2330796800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568ed3f0 msgr2=0x561a568ef7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 --2- 172.21.15.69:0/2330796800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568ed3f0 0x561a568ef7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7774009b40 tx=0x7f77740113e0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 -- 172.21.15.69:0/2330796800 shutdown_connections 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 --2- 172.21.15.69:0/2330796800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568ed3f0 0x561a568ef7e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 --2- 172.21.15.69:0/2330796800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2b30 0x561a568eceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 --2- 172.21.15.69:0/2330796800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 -- 172.21.15.69:0/2330796800 >> 172.21.15.69:0/2330796800 conn(0x561a568de7a0 msgr2=0x561a568deb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 -- 172.21.15.69:0/2330796800 shutdown_connections 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 -- 172.21.15.69:0/2330796800 wait complete. 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.523+0000 7f7792f2d740 1 -- start start 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7792f2d740 1 Processor -- start 2024-08-02T17:46:36.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2210 0x561a568ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568ed3f0 0x561a568eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7792f2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561a568effc0 con 0x561a568e2b30 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7792f2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a56802aa0 con 0x561a568e2210 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7792f2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561a568f0140 con 0x561a568ed3f0 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7792f2d740 10 client.0 _ll_register_callbacks cb 0x561a567f1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568ea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38730/0 (socket says 172.21.15.69:38730) 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 -- 172.21.15.69:0/3272640062 learned_addr learned my addr 172.21.15.69:0/3272640062 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f778ad9d640 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568ed3f0 0x561a568eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f778a59c640 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2210 0x561a568ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 -- 172.21.15.69:0/3272640062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568ed3f0 msgr2=0x561a568eac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568ed3f0 0x561a568eac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 -- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2210 msgr2=0x561a568ea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2210 0x561a568ea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 -- 172.21.15.69:0/3272640062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a567e6650 con 0x561a568e2b30 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f778a59c640 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2210 0x561a568ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7789d9b640 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568ea750 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7774012680 tx=0x7f77740126b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f2407a370611a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.524+0000 7f7771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7771ffb640 1 -- 172.21.15.69:0/3272640062 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774028070 con 0x561a568e2b30 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7771ffb640 1 -- 172.21.15.69:0/3272640062 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f7774004020 con 0x561a568e2b30 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 1 -- 172.21.15.69:0/3272640062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7774012930 con 0x561a568e2b30 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7771ffb640 1 -- 172.21.15.69:0/3272640062 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7774033050 con 0x561a568e2b30 2024-08-02T17:46:36.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 10 client.15390 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 1 -- 172.21.15.69:0/3272640062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a565b93c0 con 0x561a568e2b30 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7771ffb640 1 -- 172.21.15.69:0/3272640062 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7774012930 con 0x561a568e2b30 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7771ffb640 1 -- 172.21.15.69:0/3272640062 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f777400f870 con 0x561a568e2b30 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 10 client.15390 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 10 client.15390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 20 client.15390 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561a568fccf0 0x561a568ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 4 client.15390 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:36.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.525+0000 7f7792f2d740 1 -- 172.21.15.69:0/3272640062 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a56854ef0 con 0x561a568fccf0 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.526+0000 7f778a59c640 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561a568fccf0 0x561a568ff0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.526+0000 7f778a59c640 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561a568fccf0 0x561a568ff0e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b55bbefc61074c server_cookie=864f33c85a66a51f in_seq=0 out_seq=0 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.526+0000 7f7771ffb640 10 client.15390 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7771ffb640 1 -- 172.21.15.69:0/3272640062 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561a56854ef0 con 0x561a568fccf0 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7771ffb640 10 client.15390 handle_command_reply: tid=0 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 client.15390 shutdown 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 -- 172.21.15.69:0/3272640062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 msgr2=0x561a568ea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568ea750 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7774012680 tx=0x7f77740126b0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 -- 172.21.15.69:0/3272640062 shutdown_connections 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568ea750 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561a568fccf0 0x561a568ff0e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568ed3f0 0x561a568eac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 --2- 172.21.15.69:0/3272640062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568e2210 0x561a568ea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 -- 172.21.15.69:0/3272640062 >> 172.21.15.69:0/3272640062 conn(0x561a568de7a0 msgr2=0x561a568e4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7771ffb640 0 client.15390 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 -- 172.21.15.69:0/3272640062 shutdown_connections 2024-08-02T17:46:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 1 -- 172.21.15.69:0/3272640062 wait complete. 2024-08-02T17:46:36.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.527+0000 7f7792f2d740 20 client.15390 trim_cache size 0 max 16384 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.544+0000 7f7792f2d740 1 Processor -- start 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.544+0000 7f7792f2d740 1 -- start start 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.544+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a569046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.544+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a56904bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.544+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.544+0000 7f7792f2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1b1690 con 0x561a568e2b30 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.544+0000 7f7792f2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1aef60 con 0x561a568ed3f0 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.544+0000 7f7792f2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1acdb0 con 0x561a568e2210 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f7789d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a56904bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a569046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f7789d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a56904bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38738/0 (socket says 172.21.15.69:38738) 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a569046b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54778/0 (socket says 172.21.15.69:54778) 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f7789d9b640 1 -- 172.21.15.69:0/829354677 learned_addr learned my addr 172.21.15.69:0/829354677 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778ad9d640 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778a59c640 1 -- 172.21.15.69:0/829354677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 msgr2=0x561a568e3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778a59c640 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778a59c640 1 -- 172.21.15.69:0/829354677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 msgr2=0x561a56904bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778a59c640 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a56904bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778a59c640 1 -- 172.21.15.69:0/829354677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a568e2960 con 0x561a568e2210 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f7789d9b640 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a56904bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f778a59c640 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a569046b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77840cd3c0 tx=0x7f77840a5db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f101a6be82d177ea server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f7773fff640 1 -- 172.21.15.69:0/829354677 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840a4d30 con 0x561a568e2210 2024-08-02T17:46:36.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f7773fff640 1 -- 172.21.15.69:0/829354677 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778409c0c0 con 0x561a568e2210 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.545+0000 7f7773fff640 1 -- 172.21.15.69:0/829354677 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778409c320 con 0x561a568e2210 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 -- 172.21.15.69:0/829354677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 msgr2=0x561a569046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a569046b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77840cd3c0 tx=0x7f77840a5db0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 -- 172.21.15.69:0/829354677 shutdown_connections 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a56904bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 --2- 172.21.15.69:0/829354677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a569046b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 -- 172.21.15.69:0/829354677 >> 172.21.15.69:0/829354677 conn(0x561a568de7a0 msgr2=0x561a568e9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 -- 172.21.15.69:0/829354677 shutdown_connections 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.546+0000 7f7792f2d740 1 -- 172.21.15.69:0/829354677 wait complete. 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 1 -- start start 2024-08-02T17:46:36.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 1 Processor -- start 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568f5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1ad420 con 0x561a568e2b30 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1ae890 con 0x561a568ed3f0 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1adaf0 con 0x561a568e2210 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7792f2d740 10 client.0 _ll_register_callbacks cb 0x7f778409bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7789d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7789d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568f58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38754/0 (socket says 172.21.15.69:38754) 2024-08-02T17:46:36.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7789d9b640 1 -- 172.21.15.69:0/1324748667 learned_addr learned my addr 172.21.15.69:0/1324748667 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f778a59c640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f778ad9d640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f778a59c640 1 -- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 msgr2=0x561a568e6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f778a59c640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f778a59c640 1 -- 172.21.15.69:0/1324748667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 msgr2=0x561a568f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f778a59c640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f778a59c640 1 -- 172.21.15.69:0/1324748667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a568f5f90 con 0x561a568e2210 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f7789d9b640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:36.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.547+0000 7f778ad9d640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f778a59c640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568f5390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x561a56847ad0 tx=0x7f77840094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eaa759ad77619fa server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7771ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7771ffb640 1 -- 172.21.15.69:0/1324748667 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840a4d30 con 0x561a568e2210 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7771ffb640 1 -- 172.21.15.69:0/1324748667 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f77840a0170 con 0x561a568e2210 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 1 -- 172.21.15.69:0/1324748667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778409c0c0 con 0x561a568e2210 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7771ffb640 1 -- 172.21.15.69:0/1324748667 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7784098730 con 0x561a568e2210 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 10 client.24976 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 1 -- 172.21.15.69:0/1324748667 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a56966c50 con 0x561a568e2210 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7771ffb640 1 -- 172.21.15.69:0/1324748667 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f778409c0c0 con 0x561a568e2210 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7771ffb640 1 -- 172.21.15.69:0/1324748667 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f778400f040 con 0x561a568e2210 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 10 client.24976 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:36.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 10 client.24976 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 20 client.24976 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561a56967350 0x561a56969770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 4 client.24976 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.548+0000 7f7792f2d740 1 -- 172.21.15.69:0/1324748667 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a5684f400 con 0x561a56967350 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7789d9b640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561a56967350 0x561a56969770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7771ffb640 10 client.24976 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7789d9b640 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561a56967350 0x561a56969770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd4e76dd1f9f91d server_cookie=53a3cd80ff83c6ae in_seq=0 out_seq=0 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7771ffb640 1 -- 172.21.15.69:0/1324748667 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561a5684f400 con 0x561a56967350 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7771ffb640 10 client.24976 handle_command_reply: tid=0 2024-08-02T17:46:36.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 client.24976 shutdown 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 -- 172.21.15.69:0/1324748667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 msgr2=0x561a568f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568f5390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x561a56847ad0 tx=0x7f77840094a0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 -- 172.21.15.69:0/1324748667 shutdown_connections 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a568e2b30 0x561a568f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561a56967350 0x561a56969770 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a568ed3f0 0x561a568e6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 --2- 172.21.15.69:0/1324748667 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a568e2210 0x561a568f5390 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7771ffb640 0 client.24976 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 -- 172.21.15.69:0/1324748667 >> 172.21.15.69:0/1324748667 conn(0x561a568de7a0 msgr2=0x561a568dd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 -- 172.21.15.69:0/1324748667 shutdown_connections 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.549+0000 7f7792f2d740 1 -- 172.21.15.69:0/1324748667 wait complete. 2024-08-02T17:46:36.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7792f2d740 20 client.24976 trim_cache size 0 max 16384 2024-08-02T17:46:36.563 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:46:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:46:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:46:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "c68721fd-42ec-46b7-a809-7b8f8e271ec5": { 2024-08-02T17:46:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:46:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "c68721fd-42ec-46b7-a809-7b8f8e271ec5", 2024-08-02T17:46:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:46:36.563 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:46:36.564 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:46:36.564 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:46:36.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 -- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f77780a2e10 msgr2=0x7f77780a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f77780a2e10 0x7f77780a52d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f777c0096c0 tx=0x7f777c02a480 comp rx=0 tx=0).stop 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 -- 172.21.15.69:0/1088908264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f778c100b50 msgr2=0x7f778c197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f778c100b50 0x7f778c197130 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f778400eab0 tx=0x7f778400ef70 comp rx=0 tx=0).stop 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 -- 172.21.15.69:0/1088908264 shutdown_connections 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f778c102c20 0x7f778c19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f77780a2e10 0x7f77780a52d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f778c1022a0 0x7f778c197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 --2- 172.21.15.69:0/1088908264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f778c100b50 0x7f778c197130 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 -- 172.21.15.69:0/1088908264 >> 172.21.15.69:0/1088908264 conn(0x7f778c0fc500 msgr2=0x7f778c1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 -- 172.21.15.69:0/1088908264 shutdown_connections 2024-08-02T17:46:36.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.550+0000 7f7790b2f640 1 -- 172.21.15.69:0/1088908264 wait complete. 2024-08-02T17:46:36.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 -- 172.21.15.69:0/1203533965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 msgr2=0x7fb228100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 --2- 172.21.15.69:0/1203533965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 0x7fb228100920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009900 tx=0x7fb21c02a410 comp rx=0 tx=0).stop 2024-08-02T17:46:36.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 -- 172.21.15.69:0/1203533965 shutdown_connections 2024-08-02T17:46:36.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 --2- 172.21.15.69:0/1203533965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb228105230 0x7fb22810cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 --2- 172.21.15.69:0/1203533965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb228100ef0 0x7fb228104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 --2- 172.21.15.69:0/1203533965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 0x7fb228100920 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 -- 172.21.15.69:0/1203533965 >> 172.21.15.69:0/1203533965 conn(0x7fb228075be0 msgr2=0x7fb228075ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 -- 172.21.15.69:0/1203533965 shutdown_connections 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.557+0000 7fb22e251640 1 -- 172.21.15.69:0/1203533965 wait complete. 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb22e251640 1 Processor -- start 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb22e251640 1 -- start start 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb22e251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 0x7fb228198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb22e251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb228105230 0x7fb2281986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb22e251640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22819ca30 0x7fb22819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb22e251640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb22810fa10 con 0x7fb228100540 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb22e251640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb22810f890 con 0x7fb228105230 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb22e251640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb22810fb90 con 0x7fb22819ca30 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb2277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 0x7fb228198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb2277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 0x7fb228198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38768/0 (socket says 172.21.15.69:38768) 2024-08-02T17:46:36.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb227fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22819ca30 0x7fb22819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.558+0000 7fb2277fe640 1 -- 172.21.15.69:0/4000459061 learned_addr learned my addr 172.21.15.69:0/4000459061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb226ffd640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb228105230 0x7fb2281986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb2277fe640 1 -- 172.21.15.69:0/4000459061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22819ca30 msgr2=0x7fb22819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb2277fe640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22819ca30 0x7fb22819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb2277fe640 1 -- 172.21.15.69:0/4000459061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb228105230 msgr2=0x7fb2281986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb2277fe640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb228105230 0x7fb2281986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb2277fe640 1 -- 172.21.15.69:0/4000459061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22819d520 con 0x7fb228100540 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb226ffd640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb228105230 0x7fb2281986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb227fff640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22819ca30 0x7fb22819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb2277fe640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 0x7fb228198170 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb21c006010 tx=0x7fb21c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb224ff9640 1 -- 172.21.15.69:0/4000459061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c03f030 con 0x7fb228100540 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22819d7b0 con 0x7fb228100540 2024-08-02T17:46:36.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb224ff9640 1 -- 172.21.15.69:0/4000459061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb21c004440 con 0x7fb228100540 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb224ff9640 1 -- 172.21.15.69:0/4000459061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c035070 con 0x7fb228100540 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.559+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2281a4f70 con 0x7fb228100540 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.560+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2281015b0 con 0x7fb228100540 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.562+0000 7fb224ff9640 1 -- 172.21.15.69:0/4000459061 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fb21c042030 con 0x7fb228100540 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.562+0000 7fb224ff9640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2100a2e10 0x7fb2100a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.562+0000 7fb226ffd640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2100a2e10 0x7fb2100a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.562+0000 7fb224ff9640 1 -- 172.21.15.69:0/4000459061 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb21c0f0020 con 0x7fb228100540 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.563+0000 7fb226ffd640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2100a2e10 0x7fb2100a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2280fff10 tx=0x7fb218005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:36.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.566+0000 7fb224ff9640 1 -- 172.21.15.69:0/4000459061 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb21c0bc700 con 0x7fb228100540 2024-08-02T17:46:36.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.774+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb228100ef0 con 0x7fb2100a2e10 2024-08-02T17:46:36.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.781+0000 7fb224ff9640 1 -- 172.21.15.69:0/4000459061 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb228100ef0 con 0x7fb2100a2e10 2024-08-02T17:46:36.782 INFO:teuthology.orchestra.run.smithi069.stdout:/ 1m 6m3h 2024-08-02T17:46:36.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2100a2e10 msgr2=0x7fb2100a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2100a2e10 0x7fb2100a52d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb2280fff10 tx=0x7fb218005f90 comp rx=0 tx=0).stop 2024-08-02T17:46:36.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 msgr2=0x7fb228198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:36.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 0x7fb228198170 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb21c006010 tx=0x7fb21c004040 comp rx=0 tx=0).stop 2024-08-02T17:46:36.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 shutdown_connections 2024-08-02T17:46:36.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2100a2e10 0x7fb2100a52d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb22819ca30 0x7fb22819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb228105230 0x7fb2281986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 --2- 172.21.15.69:0/4000459061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb228100540 0x7fb228198170 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:36.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 >> 172.21.15.69:0/4000459061 conn(0x7fb228075be0 msgr2=0x7fb2280fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:36.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 shutdown_connections 2024-08-02T17:46:36.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:36.783+0000 7fb22e251640 1 -- 172.21.15.69:0/4000459061 wait complete. 2024-08-02T17:46:36.796 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-08-02T17:46:36.822 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-08-02T17:46:36.832 INFO:teuthology.run_tasks:Running task background_exec... 2024-08-02T17:46:36.843 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:46:36.843 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-08-02T17:46:36.864 INFO:teuthology.run_tasks:Running task quiescer... 2024-08-02T17:46:36.880 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-02T17:46:36.880 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph pg dump --format=json 2024-08-02T17:46:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:36 smithi069 ceph-mon[32746]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:37.235 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:36 smithi083 ceph-mon[37031]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:37.287 INFO:tasks.cephfs.filesystem:scrub status for tag:c68721fd-42ec-46b7-a809-7b8f8e271ec5 - {'path': '/', 'tag': 'c68721fd-42ec-46b7-a809-7b8f8e271ec5', 'options': 'recursive,force'} 2024-08-02T17:46:37.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:36 smithi077 ceph-mon[37108]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:37.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 -- 172.21.15.69:0/2734539583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 msgr2=0x7fe54c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 --2- 172.21.15.69:0/2734539583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 0x7fe54c100c10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe540009930 tx=0x7fe54002a420 comp rx=0 tx=0).stop 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 -- 172.21.15.69:0/2734539583 shutdown_connections 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 --2- 172.21.15.69:0/2734539583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 0x7fe54c100c10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 --2- 172.21.15.69:0/2734539583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe54c101150 0x7fe54c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 --2- 172.21.15.69:0/2734539583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54c1067b0 0x7fe54c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 -- 172.21.15.69:0/2734539583 >> 172.21.15.69:0/2734539583 conn(0x7fe54c0fc4d0 msgr2=0x7fe54c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 -- 172.21.15.69:0/2734539583 shutdown_connections 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.838+0000 7fe551e07640 1 -- 172.21.15.69:0/2734539583 wait complete. 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe551e07640 1 Processor -- start 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe551e07640 1 -- start start 2024-08-02T17:46:37.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe551e07640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 0x7fe54c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe551e07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54c101150 0x7fe54c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe551e07640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe54c1067b0 0x7fe54c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe551e07640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c111d10 con 0x7fe54c1007b0 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe551e07640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c111b90 con 0x7fe54c1067b0 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe551e07640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c111e90 con 0x7fe54c101150 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 0x7fe54c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 0x7fe54c198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38778/0 (socket says 172.21.15.69:38778) 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54b7fe640 1 -- 172.21.15.69:0/3485246205 learned_addr learned my addr 172.21.15.69:0/3485246205 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54affd640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54c101150 0x7fe54c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54bfff640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe54c1067b0 0x7fe54c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54b7fe640 1 -- 172.21.15.69:0/3485246205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54c101150 msgr2=0x7fe54c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:37.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54b7fe640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54c101150 0x7fe54c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54b7fe640 1 -- 172.21.15.69:0/3485246205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe54c1067b0 msgr2=0x7fe54c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54b7fe640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe54c1067b0 0x7fe54c19ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.839+0000 7fe54b7fe640 1 -- 172.21.15.69:0/3485246205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c19d120 con 0x7fe54c1007b0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.840+0000 7fe54b7fe640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 0x7fe54c198170 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c920 tx=0x7fe53c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.840+0000 7fe548ff9640 1 -- 172.21.15.69:0/3485246205 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c007c70 con 0x7fe54c1007b0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.840+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c19d410 con 0x7fe54c1007b0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.840+0000 7fe548ff9640 1 -- 172.21.15.69:0/3485246205 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe53c007e10 con 0x7fe54c1007b0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.840+0000 7fe548ff9640 1 -- 172.21.15.69:0/3485246205 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c018890 con 0x7fe54c1007b0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.840+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1a4cf0 con 0x7fe54c1007b0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.841+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54c100c10 con 0x7fe54c1007b0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.842+0000 7fe548ff9640 1 -- 172.21.15.69:0/3485246205 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fe53c018a70 con 0x7fe54c1007b0 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.842+0000 7fe548ff9640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe5340a2e10 0x7fe5340a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:37.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.843+0000 7fe54affd640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe5340a2e10 0x7fe5340a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:37.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.843+0000 7fe548ff9640 1 -- 172.21.15.69:0/3485246205 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe53c0cfc40 con 0x7fe54c1007b0 2024-08-02T17:46:37.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.843+0000 7fe54affd640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe5340a2e10 0x7fe5340a52d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe540006010 tx=0x7fe540000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:37.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:37.847+0000 7fe548ff9640 1 -- 172.21.15.69:0/3485246205 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe53c014030 con 0x7fe54c1007b0 2024-08-02T17:46:38.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.013+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe54c062360 con 0x7fe5340a2e10 2024-08-02T17:46:38.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.022+0000 7fe548ff9640 1 -- 172.21.15.69:0/3485246205 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339403 (secure 0 0 0) 0x7fe54c062360 con 0x7fe5340a2e10 2024-08-02T17:46:38.026 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:38.028 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:46:38.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe5340a2e10 msgr2=0x7fe5340a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe5340a2e10 0x7fe5340a52d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe540006010 tx=0x7fe540000bc0 comp rx=0 tx=0).stop 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 msgr2=0x7fe54c198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 0x7fe54c198170 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c920 tx=0x7fe53c00cde0 comp rx=0 tx=0).stop 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 shutdown_connections 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe5340a2e10 0x7fe5340a52d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe54c1067b0 0x7fe54c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe54c101150 0x7fe54c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 --2- 172.21.15.69:0/3485246205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe54c1007b0 0x7fe54c198170 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 >> 172.21.15.69:0/3485246205 conn(0x7fe54c0fc4d0 msgr2=0x7fe54c0fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 shutdown_connections 2024-08-02T17:46:38.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:38.025+0000 7fe551e07640 1 -- 172.21.15.69:0/3485246205 wait complete. 2024-08-02T17:46:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:37 smithi083 ceph-mon[37031]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:37 smithi083 ceph-mon[37031]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:37 smithi083 ceph-mon[37031]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:37 smithi083 ceph-mon[37031]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-08-02T17:46:38.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:37 smithi069 ceph-mon[32746]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:37 smithi069 ceph-mon[32746]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:37 smithi069 ceph-mon[32746]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.339 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:37 smithi069 ceph-mon[32746]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-08-02T17:46:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:37 smithi077 ceph-mon[37108]: from='client.15372 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:37 smithi077 ceph-mon[37108]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:37 smithi077 ceph-mon[37108]: from='client.15402 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T17:46:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:37 smithi077 ceph-mon[37108]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-08-02T17:46:38.712 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-08-02T17:46:36.941923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391214,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1129,"num_read_kb":938,"num_write":573,"num_write_kb":5045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":932,"ondisk_log_size":932,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":403344,"kb_used_data":16676,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124538480,"statfs":{"total":1151940427776,"available":1151527403520,"internally_reserved":0,"allocated":17076224,"data_stored":10283650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201070,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":280,"num_read_kb":353,"num_write":92,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.014044"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915568+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:46:24.915568+0000","last_peered":"2024-08-02T17:46:24.915568+0000","last_clean":"2024-08-02T17:46:24.915568+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:46:24.915568+0000","last_undegraded":"2024-08-02T17:46:24.915568+0000","last_fullsized":"2024-08-02T17:46:24.915568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954285+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:46:24.954285+0000","last_peered":"2024-08-02T17:46:24.954285+0000","last_clean":"2024-08-02T17:46:24.954285+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:46:24.954285+0000","last_undegraded":"2024-08-02T17:46:24.954285+0000","last_fullsized":"2024-08-02T17:46:24.954285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915172+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:46:24.915172+0000","last_peered":"2024-08-02T17:46:24.915172+0000","last_clean":"2024-08-02T17:46:24.915172+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:46:24.915172+0000","last_undegraded":"2024-08-02T17:46:24.915172+0000","last_fullsized":"2024-08-02T17:46:24.915172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908437+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:46:24.908437+0000","last_peered":"2024-08-02T17:46:24.908437+0000","last_clean":"2024-08-02T17:46:24.908437+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:46:24.908437+0000","last_undegraded":"2024-08-02T17:46:24.908437+0000","last_fullsized":"2024-08-02T17:46:24.908437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959113+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:46:24.959113+0000","last_peered":"2024-08-02T17:46:24.959113+0000","last_clean":"2024-08-02T17:46:24.959113+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:46:24.959113+0000","last_undegraded":"2024-08-02T17:46:24.959113+0000","last_fullsized":"2024-08-02T17:46:24.959113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906783+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:46:24.906783+0000","last_peered":"2024-08-02T17:46:24.906783+0000","last_clean":"2024-08-02T17:46:24.906783+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:46:24.906783+0000","last_undegraded":"2024-08-02T17:46:24.906783+0000","last_fullsized":"2024-08-02T17:46:24.906783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906892+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:46:24.906892+0000","last_peered":"2024-08-02T17:46:24.906892+0000","last_clean":"2024-08-02T17:46:24.906892+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:46:24.906892+0000","last_undegraded":"2024-08-02T17:46:24.906892+0000","last_fullsized":"2024-08-02T17:46:24.906892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962181+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:46:24.962181+0000","last_peered":"2024-08-02T17:46:24.962181+0000","last_clean":"2024-08-02T17:46:24.962181+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:46:24.962181+0000","last_undegraded":"2024-08-02T17:46:24.962181+0000","last_fullsized":"2024-08-02T17:46:24.962181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906895+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:46:24.906895+0000","last_peered":"2024-08-02T17:46:24.906895+0000","last_clean":"2024-08-02T17:46:24.906895+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:46:24.906895+0000","last_undegraded":"2024-08-02T17:46:24.906895+0000","last_fullsized":"2024-08-02T17:46:24.906895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906901+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:46:24.906901+0000","last_peered":"2024-08-02T17:46:24.906901+0000","last_clean":"2024-08-02T17:46:24.906901+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:46:24.906901+0000","last_undegraded":"2024-08-02T17:46:24.906901+0000","last_fullsized":"2024-08-02T17:46:24.906901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959903+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:46:24.959903+0000","last_peered":"2024-08-02T17:46:24.959903+0000","last_clean":"2024-08-02T17:46:24.959903+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:46:24.959903+0000","last_undegraded":"2024-08-02T17:46:24.959903+0000","last_fullsized":"2024-08-02T17:46:24.959903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906814+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:46:24.906814+0000","last_peered":"2024-08-02T17:46:24.906814+0000","last_clean":"2024-08-02T17:46:24.906814+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:46:24.906814+0000","last_undegraded":"2024-08-02T17:46:24.906814+0000","last_fullsized":"2024-08-02T17:46:24.906814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961844+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:46:24.961844+0000","last_peered":"2024-08-02T17:46:24.961844+0000","last_clean":"2024-08-02T17:46:24.961844+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:46:24.961844+0000","last_undegraded":"2024-08-02T17:46:24.961844+0000","last_fullsized":"2024-08-02T17:46:24.961844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906478+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:46:24.906478+0000","last_peered":"2024-08-02T17:46:24.906478+0000","last_clean":"2024-08-02T17:46:24.906478+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:46:24.906478+0000","last_undegraded":"2024-08-02T17:46:24.906478+0000","last_fullsized":"2024-08-02T17:46:24.906478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907202+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:46:24.907202+0000","last_peered":"2024-08-02T17:46:24.907202+0000","last_clean":"2024-08-02T17:46:24.907202+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:46:24.907202+0000","last_undegraded":"2024-08-02T17:46:24.907202+0000","last_fullsized":"2024-08-02T17:46:24.907202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959010+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:46:24.959010+0000","last_peered":"2024-08-02T17:46:24.959010+0000","last_clean":"2024-08-02T17:46:24.959010+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:46:24.959010+0000","last_undegraded":"2024-08-02T17:46:24.959010+0000","last_fullsized":"2024-08-02T17:46:24.959010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920005+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:46:24.920005+0000","last_peered":"2024-08-02T17:46:24.920005+0000","last_clean":"2024-08-02T17:46:24.920005+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:46:24.920005+0000","last_undegraded":"2024-08-02T17:46:24.920005+0000","last_fullsized":"2024-08-02T17:46:24.920005+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907441+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:46:24.907441+0000","last_peered":"2024-08-02T17:46:24.907441+0000","last_clean":"2024-08-02T17:46:24.907441+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:46:24.907441+0000","last_undegraded":"2024-08-02T17:46:24.907441+0000","last_fullsized":"2024-08-02T17:46:24.907441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956539+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:46:24.956539+0000","last_peered":"2024-08-02T17:46:24.956539+0000","last_clean":"2024-08-02T17:46:24.956539+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:46:24.956539+0000","last_undegraded":"2024-08-02T17:46:24.956539+0000","last_fullsized":"2024-08-02T17:46:24.956539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907900+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:46:24.907900+0000","last_peered":"2024-08-02T17:46:24.907900+0000","last_clean":"2024-08-02T17:46:24.907900+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:46:24.907900+0000","last_undegraded":"2024-08-02T17:46:24.907900+0000","last_fullsized":"2024-08-02T17:46:24.907900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915338+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:46:24.915338+0000","last_peered":"2024-08-02T17:46:24.915338+0000","last_clean":"2024-08-02T17:46:24.915338+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:46:24.915338+0000","last_undegraded":"2024-08-02T17:46:24.915338+0000","last_fullsized":"2024-08-02T17:46:24.915338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924149+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:46:24.924149+0000","last_peered":"2024-08-02T17:46:24.924149+0000","last_clean":"2024-08-02T17:46:24.924149+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:46:24.924149+0000","last_undegraded":"2024-08-02T17:46:24.924149+0000","last_fullsized":"2024-08-02T17:46:24.924149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915653+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:46:24.915653+0000","last_peered":"2024-08-02T17:46:24.915653+0000","last_clean":"2024-08-02T17:46:24.915653+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:46:24.915653+0000","last_undegraded":"2024-08-02T17:46:24.915653+0000","last_fullsized":"2024-08-02T17:46:24.915653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907285+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:46:24.907285+0000","last_peered":"2024-08-02T17:46:24.907285+0000","last_clean":"2024-08-02T17:46:24.907285+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:46:24.907285+0000","last_undegraded":"2024-08-02T17:46:24.907285+0000","last_fullsized":"2024-08-02T17:46:24.907285+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961716+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:46:24.961716+0000","last_peered":"2024-08-02T17:46:24.961716+0000","last_clean":"2024-08-02T17:46:24.961716+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:46:24.961716+0000","last_undegraded":"2024-08-02T17:46:24.961716+0000","last_fullsized":"2024-08-02T17:46:24.961716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907489+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:46:24.907489+0000","last_peered":"2024-08-02T17:46:24.907489+0000","last_clean":"2024-08-02T17:46:24.907489+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:46:24.907489+0000","last_undegraded":"2024-08-02T17:46:24.907489+0000","last_fullsized":"2024-08-02T17:46:24.907489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915365+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:46:24.915365+0000","last_peered":"2024-08-02T17:46:24.915365+0000","last_clean":"2024-08-02T17:46:24.915365+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:46:24.915365+0000","last_undegraded":"2024-08-02T17:46:24.915365+0000","last_fullsized":"2024-08-02T17:46:24.915365+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956310+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:46:24.956310+0000","last_peered":"2024-08-02T17:46:24.956310+0000","last_clean":"2024-08-02T17:46:24.956310+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:46:24.956310+0000","last_undegraded":"2024-08-02T17:46:24.956310+0000","last_fullsized":"2024-08-02T17:46:24.956310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'73","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:35.678209+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:46:35.678209+0000","last_peered":"2024-08-02T17:46:35.678209+0000","last_clean":"2024-08-02T17:46:35.678209+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:46:35.678209+0000","last_undegraded":"2024-08-02T17:46:35.678209+0000","last_fullsized":"2024-08-02T17:46:35.678209+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":132,"num_write_kb":1159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907676+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:46:24.907676+0000","last_peered":"2024-08-02T17:46:24.907676+0000","last_clean":"2024-08-02T17:46:24.907676+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.907676+0000","last_undegraded":"2024-08-02T17:46:24.907676+0000","last_fullsized":"2024-08-02T17:46:24.907676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950528+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:46:24.950528+0000","last_peered":"2024-08-02T17:46:24.950528+0000","last_clean":"2024-08-02T17:46:24.950528+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:46:24.950528+0000","last_undegraded":"2024-08-02T17:46:24.950528+0000","last_fullsized":"2024-08-02T17:46:24.950528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906878+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:46:24.906878+0000","last_peered":"2024-08-02T17:46:24.906878+0000","last_clean":"2024-08-02T17:46:24.906878+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:46:24.906878+0000","last_undegraded":"2024-08-02T17:46:24.906878+0000","last_fullsized":"2024-08-02T17:46:24.906878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920332+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:46:24.920332+0000","last_peered":"2024-08-02T17:46:24.920332+0000","last_clean":"2024-08-02T17:46:24.920332+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:46:24.920332+0000","last_undegraded":"2024-08-02T17:46:24.920332+0000","last_fullsized":"2024-08-02T17:46:24.920332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953960+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:46:24.953960+0000","last_peered":"2024-08-02T17:46:24.953960+0000","last_clean":"2024-08-02T17:46:24.953960+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:46:24.953960+0000","last_undegraded":"2024-08-02T17:46:24.953960+0000","last_fullsized":"2024-08-02T17:46:24.953960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915185+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:46:24.915185+0000","last_peered":"2024-08-02T17:46:24.915185+0000","last_clean":"2024-08-02T17:46:24.915185+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:46:24.915185+0000","last_undegraded":"2024-08-02T17:46:24.915185+0000","last_fullsized":"2024-08-02T17:46:24.915185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906282+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:46:24.906282+0000","last_peered":"2024-08-02T17:46:24.906282+0000","last_clean":"2024-08-02T17:46:24.906282+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:46:24.906282+0000","last_undegraded":"2024-08-02T17:46:24.906282+0000","last_fullsized":"2024-08-02T17:46:24.906282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956518+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:46:24.956518+0000","last_peered":"2024-08-02T17:46:24.956518+0000","last_clean":"2024-08-02T17:46:24.956518+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:46:24.956518+0000","last_undegraded":"2024-08-02T17:46:24.956518+0000","last_fullsized":"2024-08-02T17:46:24.956518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.909639+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:46:24.909639+0000","last_peered":"2024-08-02T17:46:24.909639+0000","last_clean":"2024-08-02T17:46:24.909639+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:46:24.909639+0000","last_undegraded":"2024-08-02T17:46:24.909639+0000","last_fullsized":"2024-08-02T17:46:24.909639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.918812+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:46:24.918812+0000","last_peered":"2024-08-02T17:46:24.918812+0000","last_clean":"2024-08-02T17:46:24.918812+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:46:24.918812+0000","last_undegraded":"2024-08-02T17:46:24.918812+0000","last_fullsized":"2024-08-02T17:46:24.918812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952882+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:46:24.952882+0000","last_peered":"2024-08-02T17:46:24.952882+0000","last_clean":"2024-08-02T17:46:24.952882+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:46:24.952882+0000","last_undegraded":"2024-08-02T17:46:24.952882+0000","last_fullsized":"2024-08-02T17:46:24.952882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":153,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:34.068540+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:46:34.068540+0000","last_peered":"2024-08-02T17:46:34.068540+0000","last_clean":"2024-08-02T17:46:34.068540+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:46:34.068540+0000","last_undegraded":"2024-08-02T17:46:34.068540+0000","last_fullsized":"2024-08-02T17:46:34.068540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919707+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:46:24.919707+0000","last_peered":"2024-08-02T17:46:24.919707+0000","last_clean":"2024-08-02T17:46:24.919707+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.919707+0000","last_undegraded":"2024-08-02T17:46:24.919707+0000","last_fullsized":"2024-08-02T17:46:24.919707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947422+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:46:24.947422+0000","last_peered":"2024-08-02T17:46:24.947422+0000","last_clean":"2024-08-02T17:46:24.947422+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:46:24.947422+0000","last_undegraded":"2024-08-02T17:46:24.947422+0000","last_fullsized":"2024-08-02T17:46:24.947422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908927+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:46:24.908927+0000","last_peered":"2024-08-02T17:46:24.908927+0000","last_clean":"2024-08-02T17:46:24.908927+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:46:24.908927+0000","last_undegraded":"2024-08-02T17:46:24.908927+0000","last_fullsized":"2024-08-02T17:46:24.908927+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915349+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:46:24.915349+0000","last_peered":"2024-08-02T17:46:24.915349+0000","last_clean":"2024-08-02T17:46:24.915349+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:46:24.915349+0000","last_undegraded":"2024-08-02T17:46:24.915349+0000","last_fullsized":"2024-08-02T17:46:24.915349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962129+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:46:24.962129+0000","last_peered":"2024-08-02T17:46:24.962129+0000","last_clean":"2024-08-02T17:46:24.962129+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:46:24.962129+0000","last_undegraded":"2024-08-02T17:46:24.962129+0000","last_fullsized":"2024-08-02T17:46:24.962129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907233+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:46:24.907233+0000","last_peered":"2024-08-02T17:46:24.907233+0000","last_clean":"2024-08-02T17:46:24.907233+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:46:24.907233+0000","last_undegraded":"2024-08-02T17:46:24.907233+0000","last_fullsized":"2024-08-02T17:46:24.907233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":412,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:27.046748+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:46:27.046748+0000","last_peered":"2024-08-02T17:46:27.046748+0000","last_clean":"2024-08-02T17:46:27.046748+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:46:27.046748+0000","last_undegraded":"2024-08-02T17:46:27.046748+0000","last_fullsized":"2024-08-02T17:46:27.046748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910641+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:46:24.910641+0000","last_peered":"2024-08-02T17:46:24.910641+0000","last_clean":"2024-08-02T17:46:24.910641+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:46:24.910641+0000","last_undegraded":"2024-08-02T17:46:24.910641+0000","last_fullsized":"2024-08-02T17:46:24.910641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960428+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:46:24.960428+0000","last_peered":"2024-08-02T17:46:24.960428+0000","last_clean":"2024-08-02T17:46:24.960428+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:46:24.960428+0000","last_undegraded":"2024-08-02T17:46:24.960428+0000","last_fullsized":"2024-08-02T17:46:24.960428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.916318+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:46:24.916318+0000","last_peered":"2024-08-02T17:46:24.916318+0000","last_clean":"2024-08-02T17:46:24.916318+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:46:24.916318+0000","last_undegraded":"2024-08-02T17:46:24.916318+0000","last_fullsized":"2024-08-02T17:46:24.916318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915831+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:46:24.915831+0000","last_peered":"2024-08-02T17:46:24.915831+0000","last_clean":"2024-08-02T17:46:24.915831+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:46:24.915831+0000","last_undegraded":"2024-08-02T17:46:24.915831+0000","last_fullsized":"2024-08-02T17:46:24.915831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948829+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:46:24.948829+0000","last_peered":"2024-08-02T17:46:24.948829+0000","last_clean":"2024-08-02T17:46:24.948829+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:46:24.948829+0000","last_undegraded":"2024-08-02T17:46:24.948829+0000","last_fullsized":"2024-08-02T17:46:24.948829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'57","reported_seq":115,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:35.673493+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:46:35.673493+0000","last_peered":"2024-08-02T17:46:35.673493+0000","last_clean":"2024-08-02T17:46:35.673493+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:46:35.673493+0000","last_undegraded":"2024-08-02T17:46:35.673493+0000","last_fullsized":"2024-08-02T17:46:35.673493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924774+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:46:24.924774+0000","last_peered":"2024-08-02T17:46:24.924774+0000","last_clean":"2024-08-02T17:46:24.924774+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:46:24.924774+0000","last_undegraded":"2024-08-02T17:46:24.924774+0000","last_fullsized":"2024-08-02T17:46:24.924774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955439+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:46:24.955439+0000","last_peered":"2024-08-02T17:46:24.955439+0000","last_clean":"2024-08-02T17:46:24.955439+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:46:24.955439+0000","last_undegraded":"2024-08-02T17:46:24.955439+0000","last_fullsized":"2024-08-02T17:46:24.955439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919542+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:46:24.919542+0000","last_peered":"2024-08-02T17:46:24.919542+0000","last_clean":"2024-08-02T17:46:24.919542+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:46:24.919542+0000","last_undegraded":"2024-08-02T17:46:24.919542+0000","last_fullsized":"2024-08-02T17:46:24.919542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914656+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:46:24.914656+0000","last_peered":"2024-08-02T17:46:24.914656+0000","last_clean":"2024-08-02T17:46:24.914656+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:46:24.914656+0000","last_undegraded":"2024-08-02T17:46:24.914656+0000","last_fullsized":"2024-08-02T17:46:24.914656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950756+0000","last_change":"2024-08-02T17:45:57.979722+0000","last_active":"2024-08-02T17:46:24.950756+0000","last_peered":"2024-08-02T17:46:24.950756+0000","last_clean":"2024-08-02T17:46:24.950756+0000","last_became_active":"2024-08-02T17:45:57.979427+0000","last_became_peered":"2024-08-02T17:45:57.979427+0000","last_unstale":"2024-08-02T17:46:24.950756+0000","last_undegraded":"2024-08-02T17:46:24.950756+0000","last_fullsized":"2024-08-02T17:46:24.950756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:16:52.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.921341+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:46:25.921341+0000","last_peered":"2024-08-02T17:46:25.921341+0000","last_clean":"2024-08-02T17:46:25.921341+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:46:25.921341+0000","last_undegraded":"2024-08-02T17:46:25.921341+0000","last_fullsized":"2024-08-02T17:46:25.921341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914880+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:46:24.914880+0000","last_peered":"2024-08-02T17:46:24.914880+0000","last_clean":"2024-08-02T17:46:24.914880+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:46:24.914880+0000","last_undegraded":"2024-08-02T17:46:24.914880+0000","last_fullsized":"2024-08-02T17:46:24.914880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953834+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:46:24.953834+0000","last_peered":"2024-08-02T17:46:24.953834+0000","last_clean":"2024-08-02T17:46:24.953834+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:46:24.953834+0000","last_undegraded":"2024-08-02T17:46:24.953834+0000","last_fullsized":"2024-08-02T17:46:24.953834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906346+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:46:24.906346+0000","last_peered":"2024-08-02T17:46:24.906346+0000","last_clean":"2024-08-02T17:46:24.906346+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:46:24.906346+0000","last_undegraded":"2024-08-02T17:46:24.906346+0000","last_fullsized":"2024-08-02T17:46:24.906346+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919489+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:46:24.919489+0000","last_peered":"2024-08-02T17:46:24.919489+0000","last_clean":"2024-08-02T17:46:24.919489+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:46:24.919489+0000","last_undegraded":"2024-08-02T17:46:24.919489+0000","last_fullsized":"2024-08-02T17:46:24.919489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950106+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:46:24.950106+0000","last_peered":"2024-08-02T17:46:24.950106+0000","last_clean":"2024-08-02T17:46:24.950106+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:46:24.950106+0000","last_undegraded":"2024-08-02T17:46:24.950106+0000","last_fullsized":"2024-08-02T17:46:24.950106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'9","reported_seq":165,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:34.068926+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:46:34.068926+0000","last_peered":"2024-08-02T17:46:34.068926+0000","last_clean":"2024-08-02T17:46:34.068926+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:46:34.068926+0000","last_undegraded":"2024-08-02T17:46:34.068926+0000","last_fullsized":"2024-08-02T17:46:34.068926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2597,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":8,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915368+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:46:24.915368+0000","last_peered":"2024-08-02T17:46:24.915368+0000","last_clean":"2024-08-02T17:46:24.915368+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:46:24.915368+0000","last_undegraded":"2024-08-02T17:46:24.915368+0000","last_fullsized":"2024-08-02T17:46:24.915368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957938+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:46:24.957938+0000","last_peered":"2024-08-02T17:46:24.957938+0000","last_clean":"2024-08-02T17:46:24.957938+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:46:24.957938+0000","last_undegraded":"2024-08-02T17:46:24.957938+0000","last_fullsized":"2024-08-02T17:46:24.957938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915099+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:46:24.915099+0000","last_peered":"2024-08-02T17:46:24.915099+0000","last_clean":"2024-08-02T17:46:24.915099+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:46:24.915099+0000","last_undegraded":"2024-08-02T17:46:24.915099+0000","last_fullsized":"2024-08-02T17:46:24.915099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920448+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:46:24.920448+0000","last_peered":"2024-08-02T17:46:24.920448+0000","last_clean":"2024-08-02T17:46:24.920448+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:46:24.920448+0000","last_undegraded":"2024-08-02T17:46:24.920448+0000","last_fullsized":"2024-08-02T17:46:24.920448+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958381+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:46:24.958381+0000","last_peered":"2024-08-02T17:46:24.958381+0000","last_clean":"2024-08-02T17:46:24.958381+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:46:24.958381+0000","last_undegraded":"2024-08-02T17:46:24.958381+0000","last_fullsized":"2024-08-02T17:46:24.958381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915872+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:46:24.915872+0000","last_peered":"2024-08-02T17:46:24.915872+0000","last_clean":"2024-08-02T17:46:24.915872+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:46:24.915872+0000","last_undegraded":"2024-08-02T17:46:24.915872+0000","last_fullsized":"2024-08-02T17:46:24.915872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915506+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:46:24.915506+0000","last_peered":"2024-08-02T17:46:24.915506+0000","last_clean":"2024-08-02T17:46:24.915506+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:46:24.915506+0000","last_undegraded":"2024-08-02T17:46:24.915506+0000","last_fullsized":"2024-08-02T17:46:24.915506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955796+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:46:24.955796+0000","last_peered":"2024-08-02T17:46:24.955796+0000","last_clean":"2024-08-02T17:46:24.955796+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:46:24.955796+0000","last_undegraded":"2024-08-02T17:46:24.955796+0000","last_fullsized":"2024-08-02T17:46:24.955796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":313,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:35.069220+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:46:35.069220+0000","last_peered":"2024-08-02T17:46:35.069220+0000","last_clean":"2024-08-02T17:46:35.069220+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:46:35.069220+0000","last_undegraded":"2024-08-02T17:46:35.069220+0000","last_fullsized":"2024-08-02T17:46:35.069220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72624,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":231,"num_read_kb":186,"num_write":42,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911360+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:46:24.911360+0000","last_peered":"2024-08-02T17:46:24.911360+0000","last_clean":"2024-08-02T17:46:24.911360+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:46:24.911360+0000","last_undegraded":"2024-08-02T17:46:24.911360+0000","last_fullsized":"2024-08-02T17:46:24.911360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959993+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:46:24.959993+0000","last_peered":"2024-08-02T17:46:24.959993+0000","last_clean":"2024-08-02T17:46:24.959993+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:46:24.959993+0000","last_undegraded":"2024-08-02T17:46:24.959993+0000","last_fullsized":"2024-08-02T17:46:24.959993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915691+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:46:24.915691+0000","last_peered":"2024-08-02T17:46:24.915691+0000","last_clean":"2024-08-02T17:46:24.915691+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:46:24.915691+0000","last_undegraded":"2024-08-02T17:46:24.915691+0000","last_fullsized":"2024-08-02T17:46:24.915691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907068+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:46:24.907068+0000","last_peered":"2024-08-02T17:46:24.907068+0000","last_clean":"2024-08-02T17:46:24.907068+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:46:24.907068+0000","last_undegraded":"2024-08-02T17:46:24.907068+0000","last_fullsized":"2024-08-02T17:46:24.907068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960681+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:46:24.960681+0000","last_peered":"2024-08-02T17:46:24.960681+0000","last_clean":"2024-08-02T17:46:24.960681+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:46:24.960681+0000","last_undegraded":"2024-08-02T17:46:24.960681+0000","last_fullsized":"2024-08-02T17:46:24.960681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910512+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:46:24.910512+0000","last_peered":"2024-08-02T17:46:24.910512+0000","last_clean":"2024-08-02T17:46:24.910512+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:46:24.910512+0000","last_undegraded":"2024-08-02T17:46:24.910512+0000","last_fullsized":"2024-08-02T17:46:24.910512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914907+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:46:24.914907+0000","last_peered":"2024-08-02T17:46:24.914907+0000","last_clean":"2024-08-02T17:46:24.914907+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:46:24.914907+0000","last_undegraded":"2024-08-02T17:46:24.914907+0000","last_fullsized":"2024-08-02T17:46:24.914907+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962425+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:46:24.962425+0000","last_peered":"2024-08-02T17:46:24.962425+0000","last_clean":"2024-08-02T17:46:24.962425+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:46:24.962425+0000","last_undegraded":"2024-08-02T17:46:24.962425+0000","last_fullsized":"2024-08-02T17:46:24.962425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":153,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:34.068226+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:46:34.068226+0000","last_peered":"2024-08-02T17:46:34.068226+0000","last_clean":"2024-08-02T17:46:34.068226+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:46:34.068226+0000","last_undegraded":"2024-08-02T17:46:34.068226+0000","last_fullsized":"2024-08-02T17:46:34.068226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907833+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:46:24.907833+0000","last_peered":"2024-08-02T17:46:24.907833+0000","last_clean":"2024-08-02T17:46:24.907833+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:46:24.907833+0000","last_undegraded":"2024-08-02T17:46:24.907833+0000","last_fullsized":"2024-08-02T17:46:24.907833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.926268+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:46:24.926268+0000","last_peered":"2024-08-02T17:46:24.926268+0000","last_clean":"2024-08-02T17:46:24.926268+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:46:24.926268+0000","last_undegraded":"2024-08-02T17:46:24.926268+0000","last_fullsized":"2024-08-02T17:46:24.926268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956375+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:46:24.956375+0000","last_peered":"2024-08-02T17:46:24.956375+0000","last_clean":"2024-08-02T17:46:24.956375+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:46:24.956375+0000","last_undegraded":"2024-08-02T17:46:24.956375+0000","last_fullsized":"2024-08-02T17:46:24.956375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914965+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:46:24.914965+0000","last_peered":"2024-08-02T17:46:24.914965+0000","last_clean":"2024-08-02T17:46:24.914965+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:46:24.914965+0000","last_undegraded":"2024-08-02T17:46:24.914965+0000","last_fullsized":"2024-08-02T17:46:24.914965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920061+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:46:24.920061+0000","last_peered":"2024-08-02T17:46:24.920061+0000","last_clean":"2024-08-02T17:46:24.920061+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:46:24.920061+0000","last_undegraded":"2024-08-02T17:46:24.920061+0000","last_fullsized":"2024-08-02T17:46:24.920061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959427+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:46:24.959427+0000","last_peered":"2024-08-02T17:46:24.959427+0000","last_clean":"2024-08-02T17:46:24.959427+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:46:24.959427+0000","last_undegraded":"2024-08-02T17:46:24.959427+0000","last_fullsized":"2024-08-02T17:46:24.959427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919245+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:46:24.919245+0000","last_peered":"2024-08-02T17:46:24.919245+0000","last_clean":"2024-08-02T17:46:24.919245+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:46:24.919245+0000","last_undegraded":"2024-08-02T17:46:24.919245+0000","last_fullsized":"2024-08-02T17:46:24.919245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911708+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:46:24.911708+0000","last_peered":"2024-08-02T17:46:24.911708+0000","last_clean":"2024-08-02T17:46:24.911708+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:46:24.911708+0000","last_undegraded":"2024-08-02T17:46:24.911708+0000","last_fullsized":"2024-08-02T17:46:24.911708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950898+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:46:24.950898+0000","last_peered":"2024-08-02T17:46:24.950898+0000","last_clean":"2024-08-02T17:46:24.950898+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:46:24.950898+0000","last_undegraded":"2024-08-02T17:46:24.950898+0000","last_fullsized":"2024-08-02T17:46:24.950898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915973+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:46:24.915973+0000","last_peered":"2024-08-02T17:46:24.915973+0000","last_clean":"2024-08-02T17:46:24.915973+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:46:24.915973+0000","last_undegraded":"2024-08-02T17:46:24.915973+0000","last_fullsized":"2024-08-02T17:46:24.915973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":161,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:34.068708+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:46:34.068708+0000","last_peered":"2024-08-02T17:46:34.068708+0000","last_clean":"2024-08-02T17:46:34.068708+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:46:34.068708+0000","last_undegraded":"2024-08-02T17:46:34.068708+0000","last_fullsized":"2024-08-02T17:46:34.068708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.942301+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:46:24.942301+0000","last_peered":"2024-08-02T17:46:24.942301+0000","last_clean":"2024-08-02T17:46:24.942301+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:46:24.942301+0000","last_undegraded":"2024-08-02T17:46:24.942301+0000","last_fullsized":"2024-08-02T17:46:24.942301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915638+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:46:24.915638+0000","last_peered":"2024-08-02T17:46:24.915638+0000","last_clean":"2024-08-02T17:46:24.915638+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:46:24.915638+0000","last_undegraded":"2024-08-02T17:46:24.915638+0000","last_fullsized":"2024-08-02T17:46:24.915638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954294+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:46:24.954294+0000","last_peered":"2024-08-02T17:46:24.954294+0000","last_clean":"2024-08-02T17:46:24.954294+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:46:24.954294+0000","last_undegraded":"2024-08-02T17:46:24.954294+0000","last_fullsized":"2024-08-02T17:46:24.954294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953139+0000","last_change":"2024-08-02T17:45:58.024328+0000","last_active":"2024-08-02T17:46:24.953139+0000","last_peered":"2024-08-02T17:46:24.953139+0000","last_clean":"2024-08-02T17:46:24.953139+0000","last_became_active":"2024-08-02T17:45:58.023931+0000","last_became_peered":"2024-08-02T17:45:58.023931+0000","last_unstale":"2024-08-02T17:46:24.953139+0000","last_undegraded":"2024-08-02T17:46:24.953139+0000","last_fullsized":"2024-08-02T17:46:24.953139+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:52:10.635263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.946206+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:46:24.946206+0000","last_peered":"2024-08-02T17:46:24.946206+0000","last_clean":"2024-08-02T17:46:24.946206+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:46:24.946206+0000","last_undegraded":"2024-08-02T17:46:24.946206+0000","last_fullsized":"2024-08-02T17:46:24.946206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961330+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:46:24.961330+0000","last_peered":"2024-08-02T17:46:24.961330+0000","last_clean":"2024-08-02T17:46:24.961330+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:46:24.961330+0000","last_undegraded":"2024-08-02T17:46:24.961330+0000","last_fullsized":"2024-08-02T17:46:24.961330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948673+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:46:24.948673+0000","last_peered":"2024-08-02T17:46:24.948673+0000","last_clean":"2024-08-02T17:46:24.948673+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:46:24.948673+0000","last_undegraded":"2024-08-02T17:46:24.948673+0000","last_fullsized":"2024-08-02T17:46:24.948673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948711+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:46:24.948711+0000","last_peered":"2024-08-02T17:46:24.948711+0000","last_clean":"2024-08-02T17:46:24.948711+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:46:24.948711+0000","last_undegraded":"2024-08-02T17:46:24.948711+0000","last_fullsized":"2024-08-02T17:46:24.948711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962508+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:46:24.962508+0000","last_peered":"2024-08-02T17:46:24.962508+0000","last_clean":"2024-08-02T17:46:24.962508+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:46:24.962508+0000","last_undegraded":"2024-08-02T17:46:24.962508+0000","last_fullsized":"2024-08-02T17:46:24.962508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949000+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:46:24.949000+0000","last_peered":"2024-08-02T17:46:24.949000+0000","last_clean":"2024-08-02T17:46:24.949000+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:46:24.949000+0000","last_undegraded":"2024-08-02T17:46:24.949000+0000","last_fullsized":"2024-08-02T17:46:24.949000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949651+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:46:24.949651+0000","last_peered":"2024-08-02T17:46:24.949651+0000","last_clean":"2024-08-02T17:46:24.949651+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:46:24.949651+0000","last_undegraded":"2024-08-02T17:46:24.949651+0000","last_fullsized":"2024-08-02T17:46:24.949651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949256+0000","last_change":"2024-08-02T17:45:57.863502+0000","last_active":"2024-08-02T17:46:24.949256+0000","last_peered":"2024-08-02T17:46:24.949256+0000","last_clean":"2024-08-02T17:46:24.949256+0000","last_became_active":"2024-08-02T17:45:57.863230+0000","last_became_peered":"2024-08-02T17:45:57.863230+0000","last_unstale":"2024-08-02T17:46:24.949256+0000","last_undegraded":"2024-08-02T17:46:24.949256+0000","last_fullsized":"2024-08-02T17:46:24.949256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:26:50.915451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962526+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:46:24.962526+0000","last_peered":"2024-08-02T17:46:24.962526+0000","last_clean":"2024-08-02T17:46:24.962526+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:46:24.962526+0000","last_undegraded":"2024-08-02T17:46:24.962526+0000","last_fullsized":"2024-08-02T17:46:24.962526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958665+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:46:24.958665+0000","last_peered":"2024-08-02T17:46:24.958665+0000","last_clean":"2024-08-02T17:46:24.958665+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:46:24.958665+0000","last_undegraded":"2024-08-02T17:46:24.958665+0000","last_fullsized":"2024-08-02T17:46:24.958665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947237+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:46:24.947237+0000","last_peered":"2024-08-02T17:46:24.947237+0000","last_clean":"2024-08-02T17:46:24.947237+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:46:24.947237+0000","last_undegraded":"2024-08-02T17:46:24.947237+0000","last_fullsized":"2024-08-02T17:46:24.947237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.951444+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:46:24.951444+0000","last_peered":"2024-08-02T17:46:24.951444+0000","last_clean":"2024-08-02T17:46:24.951444+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:46:24.951444+0000","last_undegraded":"2024-08-02T17:46:24.951444+0000","last_fullsized":"2024-08-02T17:46:24.951444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962281+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:46:24.962281+0000","last_peered":"2024-08-02T17:46:24.962281+0000","last_clean":"2024-08-02T17:46:24.962281+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:46:24.962281+0000","last_undegraded":"2024-08-02T17:46:24.962281+0000","last_fullsized":"2024-08-02T17:46:24.962281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959483+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:46:24.959483+0000","last_peered":"2024-08-02T17:46:24.959483+0000","last_clean":"2024-08-02T17:46:24.959483+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:46:24.959483+0000","last_undegraded":"2024-08-02T17:46:24.959483+0000","last_fullsized":"2024-08-02T17:46:24.959483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962369+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:46:24.962369+0000","last_peered":"2024-08-02T17:46:24.962369+0000","last_clean":"2024-08-02T17:46:24.962369+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:46:24.962369+0000","last_undegraded":"2024-08-02T17:46:24.962369+0000","last_fullsized":"2024-08-02T17:46:24.962369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959980+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:46:24.959980+0000","last_peered":"2024-08-02T17:46:24.959980+0000","last_clean":"2024-08-02T17:46:24.959980+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:46:24.959980+0000","last_undegraded":"2024-08-02T17:46:24.959980+0000","last_fullsized":"2024-08-02T17:46:24.959980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956497+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:46:24.956497+0000","last_peered":"2024-08-02T17:46:24.956497+0000","last_clean":"2024-08-02T17:46:24.956497+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:46:24.956497+0000","last_undegraded":"2024-08-02T17:46:24.956497+0000","last_fullsized":"2024-08-02T17:46:24.956497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952727+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:46:24.952727+0000","last_peered":"2024-08-02T17:46:24.952727+0000","last_clean":"2024-08-02T17:46:24.952727+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:46:24.952727+0000","last_undegraded":"2024-08-02T17:46:24.952727+0000","last_fullsized":"2024-08-02T17:46:24.952727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952746+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:46:24.952746+0000","last_peered":"2024-08-02T17:46:24.952746+0000","last_clean":"2024-08-02T17:46:24.952746+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:46:24.952746+0000","last_undegraded":"2024-08-02T17:46:24.952746+0000","last_fullsized":"2024-08-02T17:46:24.952746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960232+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:46:24.960232+0000","last_peered":"2024-08-02T17:46:24.960232+0000","last_clean":"2024-08-02T17:46:24.960232+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:46:24.960232+0000","last_undegraded":"2024-08-02T17:46:24.960232+0000","last_fullsized":"2024-08-02T17:46:24.960232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956382+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:46:24.956382+0000","last_peered":"2024-08-02T17:46:24.956382+0000","last_clean":"2024-08-02T17:46:24.956382+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:46:24.956382+0000","last_undegraded":"2024-08-02T17:46:24.956382+0000","last_fullsized":"2024-08-02T17:46:24.956382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962266+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:46:24.962266+0000","last_peered":"2024-08-02T17:46:24.962266+0000","last_clean":"2024-08-02T17:46:24.962266+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:46:24.962266+0000","last_undegraded":"2024-08-02T17:46:24.962266+0000","last_fullsized":"2024-08-02T17:46:24.962266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958299+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:46:24.958299+0000","last_peered":"2024-08-02T17:46:24.958299+0000","last_clean":"2024-08-02T17:46:24.958299+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:46:24.958299+0000","last_undegraded":"2024-08-02T17:46:24.958299+0000","last_fullsized":"2024-08-02T17:46:24.958299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950405+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:46:24.950405+0000","last_peered":"2024-08-02T17:46:24.950405+0000","last_clean":"2024-08-02T17:46:24.950405+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:46:24.950405+0000","last_undegraded":"2024-08-02T17:46:24.950405+0000","last_fullsized":"2024-08-02T17:46:24.950405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956086+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:46:24.956086+0000","last_peered":"2024-08-02T17:46:24.956086+0000","last_clean":"2024-08-02T17:46:24.956086+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:46:24.956086+0000","last_undegraded":"2024-08-02T17:46:24.956086+0000","last_fullsized":"2024-08-02T17:46:24.956086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949321+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:46:24.949321+0000","last_peered":"2024-08-02T17:46:24.949321+0000","last_clean":"2024-08-02T17:46:24.949321+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:46:24.949321+0000","last_undegraded":"2024-08-02T17:46:24.949321+0000","last_fullsized":"2024-08-02T17:46:24.949321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955968+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:46:24.955968+0000","last_peered":"2024-08-02T17:46:24.955968+0000","last_clean":"2024-08-02T17:46:24.955968+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:46:24.955968+0000","last_undegraded":"2024-08-02T17:46:24.955968+0000","last_fullsized":"2024-08-02T17:46:24.955968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953754+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:46:24.953754+0000","last_peered":"2024-08-02T17:46:24.953754+0000","last_clean":"2024-08-02T17:46:24.953754+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:46:24.953754+0000","last_undegraded":"2024-08-02T17:46:24.953754+0000","last_fullsized":"2024-08-02T17:46:24.953754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957053+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:46:24.957053+0000","last_peered":"2024-08-02T17:46:24.957053+0000","last_clean":"2024-08-02T17:46:24.957053+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:46:24.957053+0000","last_undegraded":"2024-08-02T17:46:24.957053+0000","last_fullsized":"2024-08-02T17:46:24.957053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959352+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:46:24.959352+0000","last_peered":"2024-08-02T17:46:24.959352+0000","last_clean":"2024-08-02T17:46:24.959352+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:46:24.959352+0000","last_undegraded":"2024-08-02T17:46:24.959352+0000","last_fullsized":"2024-08-02T17:46:24.959352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800612,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":892,"num_read_kb":608,"num_write":370,"num_write_kb":2105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2401836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449131,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1486848,"data_stored":937256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678063,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37412,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707740,"statfs":{"total":95995035648,"available":95956725760,"internally_reserved":0,"allocated":2125824,"data_stored":1534173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841589,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1507328,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103224,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":548292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364861,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1142644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528385,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":539452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37196,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707956,"statfs":{"total":95995035648,"available":95956946944,"internally_reserved":0,"allocated":1843200,"data_stored":1263727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":546223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1064462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":613007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":611986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":540840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":72624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":74783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":404443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:46:38.718 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph pg dump --format=json 2024-08-02T17:46:38.972 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:39.267 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:39 smithi069 ceph-mon[32746]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:39.267 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:39 smithi069 ceph-mon[32746]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-08-02T17:46:39.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:39 smithi083 ceph-mon[37031]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:39.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:39 smithi083 ceph-mon[37031]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-08-02T17:46:39.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:39 smithi077 ceph-mon[37108]: from='client.15408 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:39.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:39 smithi077 ceph-mon[37108]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-08-02T17:46:39.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 -- 172.21.15.69:0/2097944529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58101150 msgr2=0x7f7e5810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:39.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 --2- 172.21.15.69:0/2097944529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58101150 0x7f7e5810d680 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7e50009930 tx=0x7f7e5002a420 comp rx=0 tx=0).stop 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 -- 172.21.15.69:0/2097944529 shutdown_connections 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 --2- 172.21.15.69:0/2097944529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e58101150 0x7f7e5810d680 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 --2- 172.21.15.69:0/2097944529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e581007b0 0x7f7e58100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 --2- 172.21.15.69:0/2097944529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e581067b0 0x7f7e58106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 -- 172.21.15.69:0/2097944529 >> 172.21.15.69:0/2097944529 conn(0x7f7e580fc4d0 msgr2=0x7f7e580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 -- 172.21.15.69:0/2097944529 shutdown_connections 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.446+0000 7f7e5def5640 1 -- 172.21.15.69:0/2097944529 wait complete. 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.447+0000 7f7e5def5640 1 Processor -- start 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.447+0000 7f7e5def5640 1 -- start start 2024-08-02T17:46:39.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.447+0000 7f7e5def5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e581007b0 0x7f7e58198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.447+0000 7f7e5def5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e58101150 0x7f7e58199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.447+0000 7f7e5def5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e581067b0 0x7f7e581a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.447+0000 7f7e5def5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7e58110bb0 con 0x7f7e581067b0 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.447+0000 7f7e5def5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7e58110a30 con 0x7f7e58101150 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.447+0000 7f7e5def5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e58110d30 con 0x7f7e581007b0 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e581067b0 0x7f7e581a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e581067b0 0x7f7e581a4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38796/0 (socket says 172.21.15.69:38796) 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 -- 172.21.15.69:0/962657656 learned_addr learned my addr 172.21.15.69:0/962657656 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e56ffd640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e58101150 0x7f7e58199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e577fe640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e581007b0 0x7f7e58198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 -- 172.21.15.69:0/962657656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e581007b0 msgr2=0x7f7e58198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:39.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e581007b0 0x7f7e58198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 -- 172.21.15.69:0/962657656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e58101150 msgr2=0x7f7e58199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e58101150 0x7f7e58199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 -- 172.21.15.69:0/962657656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e581a4580 con 0x7f7e581067b0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e57fff640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e581067b0 0x7f7e581a4040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7e50005ab0 tx=0x7f7e500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e54ff9640 1 -- 172.21.15.69:0/962657656 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5003f030 con 0x7f7e581067b0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e54ff9640 1 -- 172.21.15.69:0/962657656 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e50003ce0 con 0x7f7e581067b0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e581a4750 con 0x7f7e581067b0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e54ff9640 1 -- 172.21.15.69:0/962657656 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50035070 con 0x7f7e581067b0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.448+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e581a4c00 con 0x7f7e581067b0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.449+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e58100c10 con 0x7f7e581067b0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.450+0000 7f7e54ff9640 1 -- 172.21.15.69:0/962657656 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f7e50042030 con 0x7f7e581067b0 2024-08-02T17:46:39.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.451+0000 7f7e54ff9640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7e400a2e90 0x7f7e400a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:39.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.451+0000 7f7e577fe640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7e400a2e90 0x7f7e400a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:39.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.451+0000 7f7e54ff9640 1 -- 172.21.15.69:0/962657656 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f7e500efd90 con 0x7f7e581067b0 2024-08-02T17:46:39.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.452+0000 7f7e577fe640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7e400a2e90 0x7f7e400a5350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e48005fd0 tx=0x7f7e48009680 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:39.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.455+0000 7f7e54ff9640 1 -- 172.21.15.69:0/962657656 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7e500bbe20 con 0x7f7e581067b0 2024-08-02T17:46:39.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.618+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7e58062360 con 0x7f7e400a2e90 2024-08-02T17:46:39.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.626+0000 7f7e54ff9640 1 -- 172.21.15.69:0/962657656 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339404 (secure 0 0 0) 0x7f7e58062360 con 0x7f7e400a2e90 2024-08-02T17:46:39.627 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:39.632 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7e400a2e90 msgr2=0x7f7e400a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7e400a2e90 0x7f7e400a5350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e48005fd0 tx=0x7f7e48009680 comp rx=0 tx=0).stop 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e581067b0 msgr2=0x7f7e581a4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e581067b0 0x7f7e581a4040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7e50005ab0 tx=0x7f7e500042f0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 shutdown_connections 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7e581067b0 0x7f7e581a4040 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7e400a2e90 0x7f7e400a5350 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7e58101150 0x7f7e58199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 --2- 172.21.15.69:0/962657656 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e581007b0 0x7f7e58198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:39.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.629+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 >> 172.21.15.69:0/962657656 conn(0x7f7e580fc4d0 msgr2=0x7f7e580fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:39.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.630+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 shutdown_connections 2024-08-02T17:46:39.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:39.630+0000 7f7e5def5640 1 -- 172.21.15.69:0/962657656 wait complete. 2024-08-02T17:46:40.257 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:40 smithi069 ceph-mon[32746]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:40 smithi083 ceph-mon[37031]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:40.287 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-08-02T17:46:38.942486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391214,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1129,"num_read_kb":938,"num_write":573,"num_write_kb":5045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":932,"ondisk_log_size":932,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":403344,"kb_used_data":16676,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124538480,"statfs":{"total":1151940427776,"available":1151527403520,"internally_reserved":0,"allocated":17076224,"data_stored":10283650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201070,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":280,"num_read_kb":353,"num_write":92,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006292"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915568+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:46:24.915568+0000","last_peered":"2024-08-02T17:46:24.915568+0000","last_clean":"2024-08-02T17:46:24.915568+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:46:24.915568+0000","last_undegraded":"2024-08-02T17:46:24.915568+0000","last_fullsized":"2024-08-02T17:46:24.915568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954285+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:46:24.954285+0000","last_peered":"2024-08-02T17:46:24.954285+0000","last_clean":"2024-08-02T17:46:24.954285+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:46:24.954285+0000","last_undegraded":"2024-08-02T17:46:24.954285+0000","last_fullsized":"2024-08-02T17:46:24.954285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915172+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:46:24.915172+0000","last_peered":"2024-08-02T17:46:24.915172+0000","last_clean":"2024-08-02T17:46:24.915172+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:46:24.915172+0000","last_undegraded":"2024-08-02T17:46:24.915172+0000","last_fullsized":"2024-08-02T17:46:24.915172+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908437+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:46:24.908437+0000","last_peered":"2024-08-02T17:46:24.908437+0000","last_clean":"2024-08-02T17:46:24.908437+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:46:24.908437+0000","last_undegraded":"2024-08-02T17:46:24.908437+0000","last_fullsized":"2024-08-02T17:46:24.908437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959113+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:46:24.959113+0000","last_peered":"2024-08-02T17:46:24.959113+0000","last_clean":"2024-08-02T17:46:24.959113+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:46:24.959113+0000","last_undegraded":"2024-08-02T17:46:24.959113+0000","last_fullsized":"2024-08-02T17:46:24.959113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906783+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:46:24.906783+0000","last_peered":"2024-08-02T17:46:24.906783+0000","last_clean":"2024-08-02T17:46:24.906783+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:46:24.906783+0000","last_undegraded":"2024-08-02T17:46:24.906783+0000","last_fullsized":"2024-08-02T17:46:24.906783+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906892+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:46:24.906892+0000","last_peered":"2024-08-02T17:46:24.906892+0000","last_clean":"2024-08-02T17:46:24.906892+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:46:24.906892+0000","last_undegraded":"2024-08-02T17:46:24.906892+0000","last_fullsized":"2024-08-02T17:46:24.906892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962181+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:46:24.962181+0000","last_peered":"2024-08-02T17:46:24.962181+0000","last_clean":"2024-08-02T17:46:24.962181+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:46:24.962181+0000","last_undegraded":"2024-08-02T17:46:24.962181+0000","last_fullsized":"2024-08-02T17:46:24.962181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906895+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:46:24.906895+0000","last_peered":"2024-08-02T17:46:24.906895+0000","last_clean":"2024-08-02T17:46:24.906895+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:46:24.906895+0000","last_undegraded":"2024-08-02T17:46:24.906895+0000","last_fullsized":"2024-08-02T17:46:24.906895+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906901+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:46:24.906901+0000","last_peered":"2024-08-02T17:46:24.906901+0000","last_clean":"2024-08-02T17:46:24.906901+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:46:24.906901+0000","last_undegraded":"2024-08-02T17:46:24.906901+0000","last_fullsized":"2024-08-02T17:46:24.906901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959903+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:46:24.959903+0000","last_peered":"2024-08-02T17:46:24.959903+0000","last_clean":"2024-08-02T17:46:24.959903+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:46:24.959903+0000","last_undegraded":"2024-08-02T17:46:24.959903+0000","last_fullsized":"2024-08-02T17:46:24.959903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906814+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:46:24.906814+0000","last_peered":"2024-08-02T17:46:24.906814+0000","last_clean":"2024-08-02T17:46:24.906814+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:46:24.906814+0000","last_undegraded":"2024-08-02T17:46:24.906814+0000","last_fullsized":"2024-08-02T17:46:24.906814+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961844+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:46:24.961844+0000","last_peered":"2024-08-02T17:46:24.961844+0000","last_clean":"2024-08-02T17:46:24.961844+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:46:24.961844+0000","last_undegraded":"2024-08-02T17:46:24.961844+0000","last_fullsized":"2024-08-02T17:46:24.961844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906478+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:46:24.906478+0000","last_peered":"2024-08-02T17:46:24.906478+0000","last_clean":"2024-08-02T17:46:24.906478+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:46:24.906478+0000","last_undegraded":"2024-08-02T17:46:24.906478+0000","last_fullsized":"2024-08-02T17:46:24.906478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907202+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:46:24.907202+0000","last_peered":"2024-08-02T17:46:24.907202+0000","last_clean":"2024-08-02T17:46:24.907202+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:46:24.907202+0000","last_undegraded":"2024-08-02T17:46:24.907202+0000","last_fullsized":"2024-08-02T17:46:24.907202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959010+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:46:24.959010+0000","last_peered":"2024-08-02T17:46:24.959010+0000","last_clean":"2024-08-02T17:46:24.959010+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:46:24.959010+0000","last_undegraded":"2024-08-02T17:46:24.959010+0000","last_fullsized":"2024-08-02T17:46:24.959010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920005+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:46:24.920005+0000","last_peered":"2024-08-02T17:46:24.920005+0000","last_clean":"2024-08-02T17:46:24.920005+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:46:24.920005+0000","last_undegraded":"2024-08-02T17:46:24.920005+0000","last_fullsized":"2024-08-02T17:46:24.920005+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907441+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:46:24.907441+0000","last_peered":"2024-08-02T17:46:24.907441+0000","last_clean":"2024-08-02T17:46:24.907441+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:46:24.907441+0000","last_undegraded":"2024-08-02T17:46:24.907441+0000","last_fullsized":"2024-08-02T17:46:24.907441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956539+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:46:24.956539+0000","last_peered":"2024-08-02T17:46:24.956539+0000","last_clean":"2024-08-02T17:46:24.956539+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:46:24.956539+0000","last_undegraded":"2024-08-02T17:46:24.956539+0000","last_fullsized":"2024-08-02T17:46:24.956539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907900+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:46:24.907900+0000","last_peered":"2024-08-02T17:46:24.907900+0000","last_clean":"2024-08-02T17:46:24.907900+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:46:24.907900+0000","last_undegraded":"2024-08-02T17:46:24.907900+0000","last_fullsized":"2024-08-02T17:46:24.907900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915338+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:46:24.915338+0000","last_peered":"2024-08-02T17:46:24.915338+0000","last_clean":"2024-08-02T17:46:24.915338+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:46:24.915338+0000","last_undegraded":"2024-08-02T17:46:24.915338+0000","last_fullsized":"2024-08-02T17:46:24.915338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924149+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:46:24.924149+0000","last_peered":"2024-08-02T17:46:24.924149+0000","last_clean":"2024-08-02T17:46:24.924149+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:46:24.924149+0000","last_undegraded":"2024-08-02T17:46:24.924149+0000","last_fullsized":"2024-08-02T17:46:24.924149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915653+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:46:24.915653+0000","last_peered":"2024-08-02T17:46:24.915653+0000","last_clean":"2024-08-02T17:46:24.915653+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:46:24.915653+0000","last_undegraded":"2024-08-02T17:46:24.915653+0000","last_fullsized":"2024-08-02T17:46:24.915653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907285+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:46:24.907285+0000","last_peered":"2024-08-02T17:46:24.907285+0000","last_clean":"2024-08-02T17:46:24.907285+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:46:24.907285+0000","last_undegraded":"2024-08-02T17:46:24.907285+0000","last_fullsized":"2024-08-02T17:46:24.907285+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961716+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:46:24.961716+0000","last_peered":"2024-08-02T17:46:24.961716+0000","last_clean":"2024-08-02T17:46:24.961716+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:46:24.961716+0000","last_undegraded":"2024-08-02T17:46:24.961716+0000","last_fullsized":"2024-08-02T17:46:24.961716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907489+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:46:24.907489+0000","last_peered":"2024-08-02T17:46:24.907489+0000","last_clean":"2024-08-02T17:46:24.907489+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:46:24.907489+0000","last_undegraded":"2024-08-02T17:46:24.907489+0000","last_fullsized":"2024-08-02T17:46:24.907489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915365+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:46:24.915365+0000","last_peered":"2024-08-02T17:46:24.915365+0000","last_clean":"2024-08-02T17:46:24.915365+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:46:24.915365+0000","last_undegraded":"2024-08-02T17:46:24.915365+0000","last_fullsized":"2024-08-02T17:46:24.915365+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956310+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:46:24.956310+0000","last_peered":"2024-08-02T17:46:24.956310+0000","last_clean":"2024-08-02T17:46:24.956310+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:46:24.956310+0000","last_undegraded":"2024-08-02T17:46:24.956310+0000","last_fullsized":"2024-08-02T17:46:24.956310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'73","reported_seq":178,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:35.678209+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:46:35.678209+0000","last_peered":"2024-08-02T17:46:35.678209+0000","last_clean":"2024-08-02T17:46:35.678209+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:46:35.678209+0000","last_undegraded":"2024-08-02T17:46:35.678209+0000","last_fullsized":"2024-08-02T17:46:35.678209+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":182,"num_read_kb":160,"num_write":132,"num_write_kb":1159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907676+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:46:24.907676+0000","last_peered":"2024-08-02T17:46:24.907676+0000","last_clean":"2024-08-02T17:46:24.907676+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.907676+0000","last_undegraded":"2024-08-02T17:46:24.907676+0000","last_fullsized":"2024-08-02T17:46:24.907676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950528+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:46:24.950528+0000","last_peered":"2024-08-02T17:46:24.950528+0000","last_clean":"2024-08-02T17:46:24.950528+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:46:24.950528+0000","last_undegraded":"2024-08-02T17:46:24.950528+0000","last_fullsized":"2024-08-02T17:46:24.950528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906878+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:46:24.906878+0000","last_peered":"2024-08-02T17:46:24.906878+0000","last_clean":"2024-08-02T17:46:24.906878+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:46:24.906878+0000","last_undegraded":"2024-08-02T17:46:24.906878+0000","last_fullsized":"2024-08-02T17:46:24.906878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920332+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:46:24.920332+0000","last_peered":"2024-08-02T17:46:24.920332+0000","last_clean":"2024-08-02T17:46:24.920332+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:46:24.920332+0000","last_undegraded":"2024-08-02T17:46:24.920332+0000","last_fullsized":"2024-08-02T17:46:24.920332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953960+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:46:24.953960+0000","last_peered":"2024-08-02T17:46:24.953960+0000","last_clean":"2024-08-02T17:46:24.953960+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:46:24.953960+0000","last_undegraded":"2024-08-02T17:46:24.953960+0000","last_fullsized":"2024-08-02T17:46:24.953960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915185+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:46:24.915185+0000","last_peered":"2024-08-02T17:46:24.915185+0000","last_clean":"2024-08-02T17:46:24.915185+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:46:24.915185+0000","last_undegraded":"2024-08-02T17:46:24.915185+0000","last_fullsized":"2024-08-02T17:46:24.915185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906282+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:46:24.906282+0000","last_peered":"2024-08-02T17:46:24.906282+0000","last_clean":"2024-08-02T17:46:24.906282+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:46:24.906282+0000","last_undegraded":"2024-08-02T17:46:24.906282+0000","last_fullsized":"2024-08-02T17:46:24.906282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956518+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:46:24.956518+0000","last_peered":"2024-08-02T17:46:24.956518+0000","last_clean":"2024-08-02T17:46:24.956518+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:46:24.956518+0000","last_undegraded":"2024-08-02T17:46:24.956518+0000","last_fullsized":"2024-08-02T17:46:24.956518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.909639+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:46:24.909639+0000","last_peered":"2024-08-02T17:46:24.909639+0000","last_clean":"2024-08-02T17:46:24.909639+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:46:24.909639+0000","last_undegraded":"2024-08-02T17:46:24.909639+0000","last_fullsized":"2024-08-02T17:46:24.909639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.918812+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:46:24.918812+0000","last_peered":"2024-08-02T17:46:24.918812+0000","last_clean":"2024-08-02T17:46:24.918812+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:46:24.918812+0000","last_undegraded":"2024-08-02T17:46:24.918812+0000","last_fullsized":"2024-08-02T17:46:24.918812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952882+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:46:24.952882+0000","last_peered":"2024-08-02T17:46:24.952882+0000","last_clean":"2024-08-02T17:46:24.952882+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:46:24.952882+0000","last_undegraded":"2024-08-02T17:46:24.952882+0000","last_fullsized":"2024-08-02T17:46:24.952882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":153,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:34.068540+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:46:34.068540+0000","last_peered":"2024-08-02T17:46:34.068540+0000","last_clean":"2024-08-02T17:46:34.068540+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:46:34.068540+0000","last_undegraded":"2024-08-02T17:46:34.068540+0000","last_fullsized":"2024-08-02T17:46:34.068540+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919707+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:46:24.919707+0000","last_peered":"2024-08-02T17:46:24.919707+0000","last_clean":"2024-08-02T17:46:24.919707+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.919707+0000","last_undegraded":"2024-08-02T17:46:24.919707+0000","last_fullsized":"2024-08-02T17:46:24.919707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947422+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:46:24.947422+0000","last_peered":"2024-08-02T17:46:24.947422+0000","last_clean":"2024-08-02T17:46:24.947422+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:46:24.947422+0000","last_undegraded":"2024-08-02T17:46:24.947422+0000","last_fullsized":"2024-08-02T17:46:24.947422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908927+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:46:24.908927+0000","last_peered":"2024-08-02T17:46:24.908927+0000","last_clean":"2024-08-02T17:46:24.908927+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:46:24.908927+0000","last_undegraded":"2024-08-02T17:46:24.908927+0000","last_fullsized":"2024-08-02T17:46:24.908927+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915349+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:46:24.915349+0000","last_peered":"2024-08-02T17:46:24.915349+0000","last_clean":"2024-08-02T17:46:24.915349+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:46:24.915349+0000","last_undegraded":"2024-08-02T17:46:24.915349+0000","last_fullsized":"2024-08-02T17:46:24.915349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962129+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:46:24.962129+0000","last_peered":"2024-08-02T17:46:24.962129+0000","last_clean":"2024-08-02T17:46:24.962129+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:46:24.962129+0000","last_undegraded":"2024-08-02T17:46:24.962129+0000","last_fullsized":"2024-08-02T17:46:24.962129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907233+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:46:24.907233+0000","last_peered":"2024-08-02T17:46:24.907233+0000","last_clean":"2024-08-02T17:46:24.907233+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:46:24.907233+0000","last_undegraded":"2024-08-02T17:46:24.907233+0000","last_fullsized":"2024-08-02T17:46:24.907233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":412,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:27.046748+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:46:27.046748+0000","last_peered":"2024-08-02T17:46:27.046748+0000","last_clean":"2024-08-02T17:46:27.046748+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:46:27.046748+0000","last_undegraded":"2024-08-02T17:46:27.046748+0000","last_fullsized":"2024-08-02T17:46:27.046748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910641+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:46:24.910641+0000","last_peered":"2024-08-02T17:46:24.910641+0000","last_clean":"2024-08-02T17:46:24.910641+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:46:24.910641+0000","last_undegraded":"2024-08-02T17:46:24.910641+0000","last_fullsized":"2024-08-02T17:46:24.910641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960428+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:46:24.960428+0000","last_peered":"2024-08-02T17:46:24.960428+0000","last_clean":"2024-08-02T17:46:24.960428+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:46:24.960428+0000","last_undegraded":"2024-08-02T17:46:24.960428+0000","last_fullsized":"2024-08-02T17:46:24.960428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.916318+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:46:24.916318+0000","last_peered":"2024-08-02T17:46:24.916318+0000","last_clean":"2024-08-02T17:46:24.916318+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:46:24.916318+0000","last_undegraded":"2024-08-02T17:46:24.916318+0000","last_fullsized":"2024-08-02T17:46:24.916318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915831+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:46:24.915831+0000","last_peered":"2024-08-02T17:46:24.915831+0000","last_clean":"2024-08-02T17:46:24.915831+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:46:24.915831+0000","last_undegraded":"2024-08-02T17:46:24.915831+0000","last_fullsized":"2024-08-02T17:46:24.915831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948829+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:46:24.948829+0000","last_peered":"2024-08-02T17:46:24.948829+0000","last_clean":"2024-08-02T17:46:24.948829+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:46:24.948829+0000","last_undegraded":"2024-08-02T17:46:24.948829+0000","last_fullsized":"2024-08-02T17:46:24.948829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'57","reported_seq":115,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:35.673493+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:46:35.673493+0000","last_peered":"2024-08-02T17:46:35.673493+0000","last_clean":"2024-08-02T17:46:35.673493+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:46:35.673493+0000","last_undegraded":"2024-08-02T17:46:35.673493+0000","last_fullsized":"2024-08-02T17:46:35.673493+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":20,"num_write":68,"num_write_kb":807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924774+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:46:24.924774+0000","last_peered":"2024-08-02T17:46:24.924774+0000","last_clean":"2024-08-02T17:46:24.924774+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:46:24.924774+0000","last_undegraded":"2024-08-02T17:46:24.924774+0000","last_fullsized":"2024-08-02T17:46:24.924774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955439+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:46:24.955439+0000","last_peered":"2024-08-02T17:46:24.955439+0000","last_clean":"2024-08-02T17:46:24.955439+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:46:24.955439+0000","last_undegraded":"2024-08-02T17:46:24.955439+0000","last_fullsized":"2024-08-02T17:46:24.955439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919542+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:46:24.919542+0000","last_peered":"2024-08-02T17:46:24.919542+0000","last_clean":"2024-08-02T17:46:24.919542+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:46:24.919542+0000","last_undegraded":"2024-08-02T17:46:24.919542+0000","last_fullsized":"2024-08-02T17:46:24.919542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914656+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:46:24.914656+0000","last_peered":"2024-08-02T17:46:24.914656+0000","last_clean":"2024-08-02T17:46:24.914656+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:46:24.914656+0000","last_undegraded":"2024-08-02T17:46:24.914656+0000","last_fullsized":"2024-08-02T17:46:24.914656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950756+0000","last_change":"2024-08-02T17:45:57.979722+0000","last_active":"2024-08-02T17:46:24.950756+0000","last_peered":"2024-08-02T17:46:24.950756+0000","last_clean":"2024-08-02T17:46:24.950756+0000","last_became_active":"2024-08-02T17:45:57.979427+0000","last_became_peered":"2024-08-02T17:45:57.979427+0000","last_unstale":"2024-08-02T17:46:24.950756+0000","last_undegraded":"2024-08-02T17:46:24.950756+0000","last_fullsized":"2024-08-02T17:46:24.950756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:16:52.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.921341+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:46:25.921341+0000","last_peered":"2024-08-02T17:46:25.921341+0000","last_clean":"2024-08-02T17:46:25.921341+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:46:25.921341+0000","last_undegraded":"2024-08-02T17:46:25.921341+0000","last_fullsized":"2024-08-02T17:46:25.921341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914880+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:46:24.914880+0000","last_peered":"2024-08-02T17:46:24.914880+0000","last_clean":"2024-08-02T17:46:24.914880+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:46:24.914880+0000","last_undegraded":"2024-08-02T17:46:24.914880+0000","last_fullsized":"2024-08-02T17:46:24.914880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953834+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:46:24.953834+0000","last_peered":"2024-08-02T17:46:24.953834+0000","last_clean":"2024-08-02T17:46:24.953834+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:46:24.953834+0000","last_undegraded":"2024-08-02T17:46:24.953834+0000","last_fullsized":"2024-08-02T17:46:24.953834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906346+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:46:24.906346+0000","last_peered":"2024-08-02T17:46:24.906346+0000","last_clean":"2024-08-02T17:46:24.906346+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:46:24.906346+0000","last_undegraded":"2024-08-02T17:46:24.906346+0000","last_fullsized":"2024-08-02T17:46:24.906346+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919489+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:46:24.919489+0000","last_peered":"2024-08-02T17:46:24.919489+0000","last_clean":"2024-08-02T17:46:24.919489+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:46:24.919489+0000","last_undegraded":"2024-08-02T17:46:24.919489+0000","last_fullsized":"2024-08-02T17:46:24.919489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950106+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:46:24.950106+0000","last_peered":"2024-08-02T17:46:24.950106+0000","last_clean":"2024-08-02T17:46:24.950106+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:46:24.950106+0000","last_undegraded":"2024-08-02T17:46:24.950106+0000","last_fullsized":"2024-08-02T17:46:24.950106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'9","reported_seq":165,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:34.068926+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:46:34.068926+0000","last_peered":"2024-08-02T17:46:34.068926+0000","last_clean":"2024-08-02T17:46:34.068926+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:46:34.068926+0000","last_undegraded":"2024-08-02T17:46:34.068926+0000","last_fullsized":"2024-08-02T17:46:34.068926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2597,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":8,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915368+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:46:24.915368+0000","last_peered":"2024-08-02T17:46:24.915368+0000","last_clean":"2024-08-02T17:46:24.915368+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:46:24.915368+0000","last_undegraded":"2024-08-02T17:46:24.915368+0000","last_fullsized":"2024-08-02T17:46:24.915368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957938+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:46:24.957938+0000","last_peered":"2024-08-02T17:46:24.957938+0000","last_clean":"2024-08-02T17:46:24.957938+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:46:24.957938+0000","last_undegraded":"2024-08-02T17:46:24.957938+0000","last_fullsized":"2024-08-02T17:46:24.957938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915099+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:46:24.915099+0000","last_peered":"2024-08-02T17:46:24.915099+0000","last_clean":"2024-08-02T17:46:24.915099+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:46:24.915099+0000","last_undegraded":"2024-08-02T17:46:24.915099+0000","last_fullsized":"2024-08-02T17:46:24.915099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920448+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:46:24.920448+0000","last_peered":"2024-08-02T17:46:24.920448+0000","last_clean":"2024-08-02T17:46:24.920448+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:46:24.920448+0000","last_undegraded":"2024-08-02T17:46:24.920448+0000","last_fullsized":"2024-08-02T17:46:24.920448+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958381+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:46:24.958381+0000","last_peered":"2024-08-02T17:46:24.958381+0000","last_clean":"2024-08-02T17:46:24.958381+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:46:24.958381+0000","last_undegraded":"2024-08-02T17:46:24.958381+0000","last_fullsized":"2024-08-02T17:46:24.958381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915872+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:46:24.915872+0000","last_peered":"2024-08-02T17:46:24.915872+0000","last_clean":"2024-08-02T17:46:24.915872+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:46:24.915872+0000","last_undegraded":"2024-08-02T17:46:24.915872+0000","last_fullsized":"2024-08-02T17:46:24.915872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915506+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:46:24.915506+0000","last_peered":"2024-08-02T17:46:24.915506+0000","last_clean":"2024-08-02T17:46:24.915506+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:46:24.915506+0000","last_undegraded":"2024-08-02T17:46:24.915506+0000","last_fullsized":"2024-08-02T17:46:24.915506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955796+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:46:24.955796+0000","last_peered":"2024-08-02T17:46:24.955796+0000","last_clean":"2024-08-02T17:46:24.955796+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:46:24.955796+0000","last_undegraded":"2024-08-02T17:46:24.955796+0000","last_fullsized":"2024-08-02T17:46:24.955796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":313,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:35.069220+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:46:35.069220+0000","last_peered":"2024-08-02T17:46:35.069220+0000","last_clean":"2024-08-02T17:46:35.069220+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:46:35.069220+0000","last_undegraded":"2024-08-02T17:46:35.069220+0000","last_fullsized":"2024-08-02T17:46:35.069220+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72624,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":231,"num_read_kb":186,"num_write":42,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911360+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:46:24.911360+0000","last_peered":"2024-08-02T17:46:24.911360+0000","last_clean":"2024-08-02T17:46:24.911360+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:46:24.911360+0000","last_undegraded":"2024-08-02T17:46:24.911360+0000","last_fullsized":"2024-08-02T17:46:24.911360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959993+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:46:24.959993+0000","last_peered":"2024-08-02T17:46:24.959993+0000","last_clean":"2024-08-02T17:46:24.959993+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:46:24.959993+0000","last_undegraded":"2024-08-02T17:46:24.959993+0000","last_fullsized":"2024-08-02T17:46:24.959993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915691+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:46:24.915691+0000","last_peered":"2024-08-02T17:46:24.915691+0000","last_clean":"2024-08-02T17:46:24.915691+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:46:24.915691+0000","last_undegraded":"2024-08-02T17:46:24.915691+0000","last_fullsized":"2024-08-02T17:46:24.915691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907068+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:46:24.907068+0000","last_peered":"2024-08-02T17:46:24.907068+0000","last_clean":"2024-08-02T17:46:24.907068+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:46:24.907068+0000","last_undegraded":"2024-08-02T17:46:24.907068+0000","last_fullsized":"2024-08-02T17:46:24.907068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960681+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:46:24.960681+0000","last_peered":"2024-08-02T17:46:24.960681+0000","last_clean":"2024-08-02T17:46:24.960681+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:46:24.960681+0000","last_undegraded":"2024-08-02T17:46:24.960681+0000","last_fullsized":"2024-08-02T17:46:24.960681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910512+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:46:24.910512+0000","last_peered":"2024-08-02T17:46:24.910512+0000","last_clean":"2024-08-02T17:46:24.910512+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:46:24.910512+0000","last_undegraded":"2024-08-02T17:46:24.910512+0000","last_fullsized":"2024-08-02T17:46:24.910512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914907+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:46:24.914907+0000","last_peered":"2024-08-02T17:46:24.914907+0000","last_clean":"2024-08-02T17:46:24.914907+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:46:24.914907+0000","last_undegraded":"2024-08-02T17:46:24.914907+0000","last_fullsized":"2024-08-02T17:46:24.914907+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962425+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:46:24.962425+0000","last_peered":"2024-08-02T17:46:24.962425+0000","last_clean":"2024-08-02T17:46:24.962425+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:46:24.962425+0000","last_undegraded":"2024-08-02T17:46:24.962425+0000","last_fullsized":"2024-08-02T17:46:24.962425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":153,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:34.068226+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:46:34.068226+0000","last_peered":"2024-08-02T17:46:34.068226+0000","last_clean":"2024-08-02T17:46:34.068226+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:46:34.068226+0000","last_undegraded":"2024-08-02T17:46:34.068226+0000","last_fullsized":"2024-08-02T17:46:34.068226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907833+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:46:24.907833+0000","last_peered":"2024-08-02T17:46:24.907833+0000","last_clean":"2024-08-02T17:46:24.907833+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:46:24.907833+0000","last_undegraded":"2024-08-02T17:46:24.907833+0000","last_fullsized":"2024-08-02T17:46:24.907833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.926268+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:46:24.926268+0000","last_peered":"2024-08-02T17:46:24.926268+0000","last_clean":"2024-08-02T17:46:24.926268+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:46:24.926268+0000","last_undegraded":"2024-08-02T17:46:24.926268+0000","last_fullsized":"2024-08-02T17:46:24.926268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956375+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:46:24.956375+0000","last_peered":"2024-08-02T17:46:24.956375+0000","last_clean":"2024-08-02T17:46:24.956375+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:46:24.956375+0000","last_undegraded":"2024-08-02T17:46:24.956375+0000","last_fullsized":"2024-08-02T17:46:24.956375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914965+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:46:24.914965+0000","last_peered":"2024-08-02T17:46:24.914965+0000","last_clean":"2024-08-02T17:46:24.914965+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:46:24.914965+0000","last_undegraded":"2024-08-02T17:46:24.914965+0000","last_fullsized":"2024-08-02T17:46:24.914965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920061+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:46:24.920061+0000","last_peered":"2024-08-02T17:46:24.920061+0000","last_clean":"2024-08-02T17:46:24.920061+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:46:24.920061+0000","last_undegraded":"2024-08-02T17:46:24.920061+0000","last_fullsized":"2024-08-02T17:46:24.920061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959427+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:46:24.959427+0000","last_peered":"2024-08-02T17:46:24.959427+0000","last_clean":"2024-08-02T17:46:24.959427+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:46:24.959427+0000","last_undegraded":"2024-08-02T17:46:24.959427+0000","last_fullsized":"2024-08-02T17:46:24.959427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919245+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:46:24.919245+0000","last_peered":"2024-08-02T17:46:24.919245+0000","last_clean":"2024-08-02T17:46:24.919245+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:46:24.919245+0000","last_undegraded":"2024-08-02T17:46:24.919245+0000","last_fullsized":"2024-08-02T17:46:24.919245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911708+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:46:24.911708+0000","last_peered":"2024-08-02T17:46:24.911708+0000","last_clean":"2024-08-02T17:46:24.911708+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:46:24.911708+0000","last_undegraded":"2024-08-02T17:46:24.911708+0000","last_fullsized":"2024-08-02T17:46:24.911708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950898+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:46:24.950898+0000","last_peered":"2024-08-02T17:46:24.950898+0000","last_clean":"2024-08-02T17:46:24.950898+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:46:24.950898+0000","last_undegraded":"2024-08-02T17:46:24.950898+0000","last_fullsized":"2024-08-02T17:46:24.950898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915973+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:46:24.915973+0000","last_peered":"2024-08-02T17:46:24.915973+0000","last_clean":"2024-08-02T17:46:24.915973+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:46:24.915973+0000","last_undegraded":"2024-08-02T17:46:24.915973+0000","last_fullsized":"2024-08-02T17:46:24.915973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":161,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:34.068708+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:46:34.068708+0000","last_peered":"2024-08-02T17:46:34.068708+0000","last_clean":"2024-08-02T17:46:34.068708+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:46:34.068708+0000","last_undegraded":"2024-08-02T17:46:34.068708+0000","last_fullsized":"2024-08-02T17:46:34.068708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.942301+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:46:24.942301+0000","last_peered":"2024-08-02T17:46:24.942301+0000","last_clean":"2024-08-02T17:46:24.942301+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:46:24.942301+0000","last_undegraded":"2024-08-02T17:46:24.942301+0000","last_fullsized":"2024-08-02T17:46:24.942301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915638+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:46:24.915638+0000","last_peered":"2024-08-02T17:46:24.915638+0000","last_clean":"2024-08-02T17:46:24.915638+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:46:24.915638+0000","last_undegraded":"2024-08-02T17:46:24.915638+0000","last_fullsized":"2024-08-02T17:46:24.915638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954294+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:46:24.954294+0000","last_peered":"2024-08-02T17:46:24.954294+0000","last_clean":"2024-08-02T17:46:24.954294+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:46:24.954294+0000","last_undegraded":"2024-08-02T17:46:24.954294+0000","last_fullsized":"2024-08-02T17:46:24.954294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953139+0000","last_change":"2024-08-02T17:45:58.024328+0000","last_active":"2024-08-02T17:46:24.953139+0000","last_peered":"2024-08-02T17:46:24.953139+0000","last_clean":"2024-08-02T17:46:24.953139+0000","last_became_active":"2024-08-02T17:45:58.023931+0000","last_became_peered":"2024-08-02T17:45:58.023931+0000","last_unstale":"2024-08-02T17:46:24.953139+0000","last_undegraded":"2024-08-02T17:46:24.953139+0000","last_fullsized":"2024-08-02T17:46:24.953139+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:52:10.635263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.946206+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:46:24.946206+0000","last_peered":"2024-08-02T17:46:24.946206+0000","last_clean":"2024-08-02T17:46:24.946206+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:46:24.946206+0000","last_undegraded":"2024-08-02T17:46:24.946206+0000","last_fullsized":"2024-08-02T17:46:24.946206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961330+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:46:24.961330+0000","last_peered":"2024-08-02T17:46:24.961330+0000","last_clean":"2024-08-02T17:46:24.961330+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:46:24.961330+0000","last_undegraded":"2024-08-02T17:46:24.961330+0000","last_fullsized":"2024-08-02T17:46:24.961330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948673+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:46:24.948673+0000","last_peered":"2024-08-02T17:46:24.948673+0000","last_clean":"2024-08-02T17:46:24.948673+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:46:24.948673+0000","last_undegraded":"2024-08-02T17:46:24.948673+0000","last_fullsized":"2024-08-02T17:46:24.948673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948711+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:46:24.948711+0000","last_peered":"2024-08-02T17:46:24.948711+0000","last_clean":"2024-08-02T17:46:24.948711+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:46:24.948711+0000","last_undegraded":"2024-08-02T17:46:24.948711+0000","last_fullsized":"2024-08-02T17:46:24.948711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962508+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:46:24.962508+0000","last_peered":"2024-08-02T17:46:24.962508+0000","last_clean":"2024-08-02T17:46:24.962508+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:46:24.962508+0000","last_undegraded":"2024-08-02T17:46:24.962508+0000","last_fullsized":"2024-08-02T17:46:24.962508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949000+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:46:24.949000+0000","last_peered":"2024-08-02T17:46:24.949000+0000","last_clean":"2024-08-02T17:46:24.949000+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:46:24.949000+0000","last_undegraded":"2024-08-02T17:46:24.949000+0000","last_fullsized":"2024-08-02T17:46:24.949000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949651+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:46:24.949651+0000","last_peered":"2024-08-02T17:46:24.949651+0000","last_clean":"2024-08-02T17:46:24.949651+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:46:24.949651+0000","last_undegraded":"2024-08-02T17:46:24.949651+0000","last_fullsized":"2024-08-02T17:46:24.949651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949256+0000","last_change":"2024-08-02T17:45:57.863502+0000","last_active":"2024-08-02T17:46:24.949256+0000","last_peered":"2024-08-02T17:46:24.949256+0000","last_clean":"2024-08-02T17:46:24.949256+0000","last_became_active":"2024-08-02T17:45:57.863230+0000","last_became_peered":"2024-08-02T17:45:57.863230+0000","last_unstale":"2024-08-02T17:46:24.949256+0000","last_undegraded":"2024-08-02T17:46:24.949256+0000","last_fullsized":"2024-08-02T17:46:24.949256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:26:50.915451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962526+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:46:24.962526+0000","last_peered":"2024-08-02T17:46:24.962526+0000","last_clean":"2024-08-02T17:46:24.962526+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:46:24.962526+0000","last_undegraded":"2024-08-02T17:46:24.962526+0000","last_fullsized":"2024-08-02T17:46:24.962526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958665+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:46:24.958665+0000","last_peered":"2024-08-02T17:46:24.958665+0000","last_clean":"2024-08-02T17:46:24.958665+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:46:24.958665+0000","last_undegraded":"2024-08-02T17:46:24.958665+0000","last_fullsized":"2024-08-02T17:46:24.958665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947237+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:46:24.947237+0000","last_peered":"2024-08-02T17:46:24.947237+0000","last_clean":"2024-08-02T17:46:24.947237+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:46:24.947237+0000","last_undegraded":"2024-08-02T17:46:24.947237+0000","last_fullsized":"2024-08-02T17:46:24.947237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.951444+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:46:24.951444+0000","last_peered":"2024-08-02T17:46:24.951444+0000","last_clean":"2024-08-02T17:46:24.951444+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:46:24.951444+0000","last_undegraded":"2024-08-02T17:46:24.951444+0000","last_fullsized":"2024-08-02T17:46:24.951444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962281+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:46:24.962281+0000","last_peered":"2024-08-02T17:46:24.962281+0000","last_clean":"2024-08-02T17:46:24.962281+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:46:24.962281+0000","last_undegraded":"2024-08-02T17:46:24.962281+0000","last_fullsized":"2024-08-02T17:46:24.962281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959483+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:46:24.959483+0000","last_peered":"2024-08-02T17:46:24.959483+0000","last_clean":"2024-08-02T17:46:24.959483+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:46:24.959483+0000","last_undegraded":"2024-08-02T17:46:24.959483+0000","last_fullsized":"2024-08-02T17:46:24.959483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962369+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:46:24.962369+0000","last_peered":"2024-08-02T17:46:24.962369+0000","last_clean":"2024-08-02T17:46:24.962369+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:46:24.962369+0000","last_undegraded":"2024-08-02T17:46:24.962369+0000","last_fullsized":"2024-08-02T17:46:24.962369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959980+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:46:24.959980+0000","last_peered":"2024-08-02T17:46:24.959980+0000","last_clean":"2024-08-02T17:46:24.959980+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:46:24.959980+0000","last_undegraded":"2024-08-02T17:46:24.959980+0000","last_fullsized":"2024-08-02T17:46:24.959980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956497+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:46:24.956497+0000","last_peered":"2024-08-02T17:46:24.956497+0000","last_clean":"2024-08-02T17:46:24.956497+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:46:24.956497+0000","last_undegraded":"2024-08-02T17:46:24.956497+0000","last_fullsized":"2024-08-02T17:46:24.956497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952727+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:46:24.952727+0000","last_peered":"2024-08-02T17:46:24.952727+0000","last_clean":"2024-08-02T17:46:24.952727+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:46:24.952727+0000","last_undegraded":"2024-08-02T17:46:24.952727+0000","last_fullsized":"2024-08-02T17:46:24.952727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952746+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:46:24.952746+0000","last_peered":"2024-08-02T17:46:24.952746+0000","last_clean":"2024-08-02T17:46:24.952746+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:46:24.952746+0000","last_undegraded":"2024-08-02T17:46:24.952746+0000","last_fullsized":"2024-08-02T17:46:24.952746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960232+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:46:24.960232+0000","last_peered":"2024-08-02T17:46:24.960232+0000","last_clean":"2024-08-02T17:46:24.960232+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:46:24.960232+0000","last_undegraded":"2024-08-02T17:46:24.960232+0000","last_fullsized":"2024-08-02T17:46:24.960232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956382+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:46:24.956382+0000","last_peered":"2024-08-02T17:46:24.956382+0000","last_clean":"2024-08-02T17:46:24.956382+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:46:24.956382+0000","last_undegraded":"2024-08-02T17:46:24.956382+0000","last_fullsized":"2024-08-02T17:46:24.956382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962266+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:46:24.962266+0000","last_peered":"2024-08-02T17:46:24.962266+0000","last_clean":"2024-08-02T17:46:24.962266+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:46:24.962266+0000","last_undegraded":"2024-08-02T17:46:24.962266+0000","last_fullsized":"2024-08-02T17:46:24.962266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958299+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:46:24.958299+0000","last_peered":"2024-08-02T17:46:24.958299+0000","last_clean":"2024-08-02T17:46:24.958299+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:46:24.958299+0000","last_undegraded":"2024-08-02T17:46:24.958299+0000","last_fullsized":"2024-08-02T17:46:24.958299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950405+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:46:24.950405+0000","last_peered":"2024-08-02T17:46:24.950405+0000","last_clean":"2024-08-02T17:46:24.950405+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:46:24.950405+0000","last_undegraded":"2024-08-02T17:46:24.950405+0000","last_fullsized":"2024-08-02T17:46:24.950405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956086+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:46:24.956086+0000","last_peered":"2024-08-02T17:46:24.956086+0000","last_clean":"2024-08-02T17:46:24.956086+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:46:24.956086+0000","last_undegraded":"2024-08-02T17:46:24.956086+0000","last_fullsized":"2024-08-02T17:46:24.956086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949321+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:46:24.949321+0000","last_peered":"2024-08-02T17:46:24.949321+0000","last_clean":"2024-08-02T17:46:24.949321+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:46:24.949321+0000","last_undegraded":"2024-08-02T17:46:24.949321+0000","last_fullsized":"2024-08-02T17:46:24.949321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955968+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:46:24.955968+0000","last_peered":"2024-08-02T17:46:24.955968+0000","last_clean":"2024-08-02T17:46:24.955968+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:46:24.955968+0000","last_undegraded":"2024-08-02T17:46:24.955968+0000","last_fullsized":"2024-08-02T17:46:24.955968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953754+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:46:24.953754+0000","last_peered":"2024-08-02T17:46:24.953754+0000","last_clean":"2024-08-02T17:46:24.953754+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:46:24.953754+0000","last_undegraded":"2024-08-02T17:46:24.953754+0000","last_fullsized":"2024-08-02T17:46:24.953754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957053+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:46:24.957053+0000","last_peered":"2024-08-02T17:46:24.957053+0000","last_clean":"2024-08-02T17:46:24.957053+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:46:24.957053+0000","last_undegraded":"2024-08-02T17:46:24.957053+0000","last_fullsized":"2024-08-02T17:46:24.957053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959352+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:46:24.959352+0000","last_peered":"2024-08-02T17:46:24.959352+0000","last_clean":"2024-08-02T17:46:24.959352+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:46:24.959352+0000","last_undegraded":"2024-08-02T17:46:24.959352+0000","last_fullsized":"2024-08-02T17:46:24.959352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800612,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":892,"num_read_kb":608,"num_write":370,"num_write_kb":2105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2401836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":282,"ondisk_log_size":282,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449131,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1486848,"data_stored":937256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678063,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37412,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707740,"statfs":{"total":95995035648,"available":95956725760,"internally_reserved":0,"allocated":2125824,"data_stored":1534173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841589,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1507328,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103224,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":548292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364861,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1142644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528385,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":539452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790022,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37196,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707956,"statfs":{"total":95995035648,"available":95956946944,"internally_reserved":0,"allocated":1843200,"data_stored":1263727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":546223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215182,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1064462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":613007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":611986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":540840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":72624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":74783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":404443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:46:40.292 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-02T17:46:40.292 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph pg dump --format=json 2024-08-02T17:46:40.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:40 smithi077 ceph-mon[37108]: from='client.15414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:40.547 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 -- 172.21.15.69:0/281698007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738103470 msgr2=0x7fe7381078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/281698007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738103470 0x7fe7381078b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009930 tx=0x7fe72c02a420 comp rx=0 tx=0).stop 2024-08-02T17:46:41.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 -- 172.21.15.69:0/281698007 shutdown_connections 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/281698007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738103470 0x7fe7381078b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/281698007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe738107fc0 0x7fe73810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/281698007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe738102ac0 0x7fe738102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 -- 172.21.15.69:0/281698007 >> 172.21.15.69:0/281698007 conn(0x7fe7380fc500 msgr2=0x7fe7380fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 -- 172.21.15.69:0/281698007 shutdown_connections 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.022+0000 7fe73f5f0640 1 -- 172.21.15.69:0/281698007 wait complete. 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73f5f0640 1 Processor -- start 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73f5f0640 1 -- start start 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73f5f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe738102ac0 0x7fe7381982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73f5f0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe738103470 0x7fe738198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73f5f0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738107fc0 0x7fe73819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:41.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73f5f0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe738111d90 con 0x7fe738107fc0 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73f5f0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe738111c10 con 0x7fe738103470 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73f5f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe738111f10 con 0x7fe738102ac0 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73db8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738107fc0 0x7fe73819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73db8d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738107fc0 0x7fe73819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38816/0 (socket says 172.21.15.69:38816) 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73db8d640 1 -- 172.21.15.69:0/3833001700 learned_addr learned my addr 172.21.15.69:0/3833001700 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73cb8b640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe738103470 0x7fe738198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.023+0000 7fe73d38c640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe738102ac0 0x7fe7381982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.024+0000 7fe73db8d640 1 -- 172.21.15.69:0/3833001700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe738102ac0 msgr2=0x7fe7381982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.024+0000 7fe73db8d640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe738102ac0 0x7fe7381982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.024+0000 7fe73db8d640 1 -- 172.21.15.69:0/3833001700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe738103470 msgr2=0x7fe738198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:41.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.024+0000 7fe73db8d640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe738103470 0x7fe738198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.024+0000 7fe73db8d640 1 -- 172.21.15.69:0/3833001700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73819d290 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.024+0000 7fe73db8d640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738107fc0 0x7fe73819cbb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe72800e970 tx=0x7fe72800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.025+0000 7fe7367fc640 1 -- 172.21.15.69:0/3833001700 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72800ccc0 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.025+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73819d580 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.025+0000 7fe7367fc640 1 -- 172.21.15.69:0/3833001700 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72800ce60 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.025+0000 7fe7367fc640 1 -- 172.21.15.69:0/3833001700 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe728010490 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.025+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7381a4e60 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.026+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe738102ea0 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.027+0000 7fe7367fc640 1 -- 172.21.15.69:0/3833001700 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fe728005020 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.028+0000 7fe7367fc640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7200a2e10 0x7fe7200a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.028+0000 7fe7367fc640 1 -- 172.21.15.69:0/3833001700 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe7280cfee0 con 0x7fe738107fc0 2024-08-02T17:46:41.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.028+0000 7fe73d38c640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7200a2e10 0x7fe7200a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:41.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.028+0000 7fe73d38c640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7200a2e10 0x7fe7200a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe730005fd0 tx=0x7fe730009760 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:41.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.032+0000 7fe7367fc640 1 -- 172.21.15.69:0/3833001700 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe72809c4f0 con 0x7fe738107fc0 2024-08-02T17:46:41.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.194+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe738062360 con 0x7fe7200a2e10 2024-08-02T17:46:41.195 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:41 smithi069 ceph-mon[32746]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-08-02T17:46:41.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.202+0000 7fe7367fc640 1 -- 172.21.15.69:0/3833001700 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339410 (secure 0 0 0) 0x7fe738062360 con 0x7fe7200a2e10 2024-08-02T17:46:41.206 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:41.208 INFO:teuthology.orchestra.run.smithi069.stderr:dumped all 2024-08-02T17:46:41.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7200a2e10 msgr2=0x7fe7200a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:41.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7200a2e10 0x7fe7200a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe730005fd0 tx=0x7fe730009760 comp rx=0 tx=0).stop 2024-08-02T17:46:41.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738107fc0 msgr2=0x7fe73819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:41.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738107fc0 0x7fe73819cbb0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe72800e970 tx=0x7fe72800ed70 comp rx=0 tx=0).stop 2024-08-02T17:46:41.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 shutdown_connections 2024-08-02T17:46:41.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe738107fc0 0x7fe73819cbb0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7200a2e10 0x7fe7200a52d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe738103470 0x7fe738198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 --2- 172.21.15.69:0/3833001700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe738102ac0 0x7fe7381982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:41.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 >> 172.21.15.69:0/3833001700 conn(0x7fe7380fc500 msgr2=0x7fe738105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:41.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 shutdown_connections 2024-08-02T17:46:41.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:41.206+0000 7fe73f5f0640 1 -- 172.21.15.69:0/3833001700 wait complete. 2024-08-02T17:46:41.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:41 smithi077 ceph-mon[37108]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-08-02T17:46:41.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:41 smithi083 ceph-mon[37031]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-08-02T17:46:41.852 INFO:teuthology.orchestra.run.smithi069.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-08-02T17:46:40.942859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391214,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1302,"num_read_kb":1083,"num_write":635,"num_write_kb":5307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":1180965,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":968,"ondisk_log_size":968,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":404028,"kb_used_data":17060,"kb_used_omap":18,"kb_used_meta":386477,"kb_avail":1124537796,"statfs":{"total":1151940427776,"available":1151526703104,"internally_reserved":0,"allocated":17469440,"data_stored":10678052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":395752761},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201070,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":453,"num_read_kb":498,"num_write":154,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006220"},"pg_stats":[{"pgid":"2.1f","version":"90'9","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915568+0000","last_change":"2024-08-02T17:44:02.044458+0000","last_active":"2024-08-02T17:46:24.915568+0000","last_peered":"2024-08-02T17:46:24.915568+0000","last_clean":"2024-08-02T17:46:24.915568+0000","last_became_active":"2024-08-02T17:44:02.044129+0000","last_became_peered":"2024-08-02T17:44:02.044129+0000","last_unstale":"2024-08-02T17:46:24.915568+0000","last_undegraded":"2024-08-02T17:46:24.915568+0000","last_fullsized":"2024-08-02T17:46:24.915568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:42.228369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954285+0000","last_change":"2024-08-02T17:44:04.120951+0000","last_active":"2024-08-02T17:46:24.954285+0000","last_peered":"2024-08-02T17:46:24.954285+0000","last_clean":"2024-08-02T17:46:24.954285+0000","last_became_active":"2024-08-02T17:44:04.120634+0000","last_became_peered":"2024-08-02T17:44:04.120634+0000","last_unstale":"2024-08-02T17:46:24.954285+0000","last_undegraded":"2024-08-02T17:46:24.954285+0000","last_fullsized":"2024-08-02T17:46:24.954285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:54:45.337477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915227+0000","last_change":"2024-08-02T17:44:11.177942+0000","last_active":"2024-08-02T17:46:24.915227+0000","last_peered":"2024-08-02T17:46:24.915227+0000","last_clean":"2024-08-02T17:46:24.915227+0000","last_became_active":"2024-08-02T17:44:11.177464+0000","last_became_peered":"2024-08-02T17:44:11.177464+0000","last_unstale":"2024-08-02T17:46:24.915227+0000","last_undegraded":"2024-08-02T17:46:24.915227+0000","last_fullsized":"2024-08-02T17:46:24.915227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:18:37.204660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908437+0000","last_change":"2024-08-02T17:44:02.202791+0000","last_active":"2024-08-02T17:46:24.908437+0000","last_peered":"2024-08-02T17:46:24.908437+0000","last_clean":"2024-08-02T17:46:24.908437+0000","last_became_active":"2024-08-02T17:44:02.202314+0000","last_became_peered":"2024-08-02T17:44:02.202314+0000","last_unstale":"2024-08-02T17:46:24.908437+0000","last_undegraded":"2024-08-02T17:46:24.908437+0000","last_fullsized":"2024-08-02T17:46:24.908437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:01:00.221465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959113+0000","last_change":"2024-08-02T17:44:04.128600+0000","last_active":"2024-08-02T17:46:24.959113+0000","last_peered":"2024-08-02T17:46:24.959113+0000","last_clean":"2024-08-02T17:46:24.959113+0000","last_became_active":"2024-08-02T17:44:04.128139+0000","last_became_peered":"2024-08-02T17:44:04.128139+0000","last_unstale":"2024-08-02T17:46:24.959113+0000","last_undegraded":"2024-08-02T17:46:24.959113+0000","last_fullsized":"2024-08-02T17:46:24.959113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:23:35.419998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906832+0000","last_change":"2024-08-02T17:44:11.174293+0000","last_active":"2024-08-02T17:46:24.906832+0000","last_peered":"2024-08-02T17:46:24.906832+0000","last_clean":"2024-08-02T17:46:24.906832+0000","last_became_active":"2024-08-02T17:44:11.173895+0000","last_became_peered":"2024-08-02T17:44:11.173895+0000","last_unstale":"2024-08-02T17:46:24.906832+0000","last_undegraded":"2024-08-02T17:46:24.906832+0000","last_fullsized":"2024-08-02T17:46:24.906832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:34:44.172874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906892+0000","last_change":"2024-08-02T17:44:02.047652+0000","last_active":"2024-08-02T17:46:24.906892+0000","last_peered":"2024-08-02T17:46:24.906892+0000","last_clean":"2024-08-02T17:46:24.906892+0000","last_became_active":"2024-08-02T17:44:02.047338+0000","last_became_peered":"2024-08-02T17:44:02.047338+0000","last_unstale":"2024-08-02T17:46:24.906892+0000","last_undegraded":"2024-08-02T17:46:24.906892+0000","last_fullsized":"2024-08-02T17:46:24.906892+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:41.529534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962181+0000","last_change":"2024-08-02T17:44:04.131768+0000","last_active":"2024-08-02T17:46:24.962181+0000","last_peered":"2024-08-02T17:46:24.962181+0000","last_clean":"2024-08-02T17:46:24.962181+0000","last_became_active":"2024-08-02T17:44:04.131298+0000","last_became_peered":"2024-08-02T17:44:04.131298+0000","last_unstale":"2024-08-02T17:46:24.962181+0000","last_undegraded":"2024-08-02T17:46:24.962181+0000","last_fullsized":"2024-08-02T17:46:24.962181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:20:40.193746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906953+0000","last_change":"2024-08-02T17:44:11.175076+0000","last_active":"2024-08-02T17:46:24.906953+0000","last_peered":"2024-08-02T17:46:24.906953+0000","last_clean":"2024-08-02T17:46:24.906953+0000","last_became_active":"2024-08-02T17:44:11.174345+0000","last_became_peered":"2024-08-02T17:44:11.174345+0000","last_unstale":"2024-08-02T17:46:24.906953+0000","last_undegraded":"2024-08-02T17:46:24.906953+0000","last_fullsized":"2024-08-02T17:46:24.906953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:29:35.461588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906901+0000","last_change":"2024-08-02T17:44:02.046359+0000","last_active":"2024-08-02T17:46:24.906901+0000","last_peered":"2024-08-02T17:46:24.906901+0000","last_clean":"2024-08-02T17:46:24.906901+0000","last_became_active":"2024-08-02T17:44:02.045947+0000","last_became_peered":"2024-08-02T17:44:02.045947+0000","last_unstale":"2024-08-02T17:46:24.906901+0000","last_undegraded":"2024-08-02T17:46:24.906901+0000","last_fullsized":"2024-08-02T17:46:24.906901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:58:23.665291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959903+0000","last_change":"2024-08-02T17:44:04.128165+0000","last_active":"2024-08-02T17:46:24.959903+0000","last_peered":"2024-08-02T17:46:24.959903+0000","last_clean":"2024-08-02T17:46:24.959903+0000","last_became_active":"2024-08-02T17:44:04.127762+0000","last_became_peered":"2024-08-02T17:44:04.127762+0000","last_unstale":"2024-08-02T17:46:24.959903+0000","last_undegraded":"2024-08-02T17:46:24.959903+0000","last_fullsized":"2024-08-02T17:46:24.959903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:05:37.158561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906864+0000","last_change":"2024-08-02T17:44:11.174937+0000","last_active":"2024-08-02T17:46:24.906864+0000","last_peered":"2024-08-02T17:46:24.906864+0000","last_clean":"2024-08-02T17:46:24.906864+0000","last_became_active":"2024-08-02T17:44:11.174263+0000","last_became_peered":"2024-08-02T17:44:11.174263+0000","last_unstale":"2024-08-02T17:46:24.906864+0000","last_undegraded":"2024-08-02T17:46:24.906864+0000","last_fullsized":"2024-08-02T17:46:24.906864+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:47:43.763003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961844+0000","last_change":"2024-08-02T17:44:04.137345+0000","last_active":"2024-08-02T17:46:24.961844+0000","last_peered":"2024-08-02T17:46:24.961844+0000","last_clean":"2024-08-02T17:46:24.961844+0000","last_became_active":"2024-08-02T17:44:04.137046+0000","last_became_peered":"2024-08-02T17:44:04.137046+0000","last_unstale":"2024-08-02T17:46:24.961844+0000","last_undegraded":"2024-08-02T17:46:24.961844+0000","last_fullsized":"2024-08-02T17:46:24.961844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:14:01.906836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906478+0000","last_change":"2024-08-02T17:44:02.049238+0000","last_active":"2024-08-02T17:46:24.906478+0000","last_peered":"2024-08-02T17:46:24.906478+0000","last_clean":"2024-08-02T17:46:24.906478+0000","last_became_active":"2024-08-02T17:44:02.048887+0000","last_became_peered":"2024-08-02T17:44:02.048887+0000","last_unstale":"2024-08-02T17:46:24.906478+0000","last_undegraded":"2024-08-02T17:46:24.906478+0000","last_fullsized":"2024-08-02T17:46:24.906478+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:44:59.317087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907202+0000","last_change":"2024-08-02T17:44:11.252652+0000","last_active":"2024-08-02T17:46:24.907202+0000","last_peered":"2024-08-02T17:46:24.907202+0000","last_clean":"2024-08-02T17:46:24.907202+0000","last_became_active":"2024-08-02T17:44:11.252196+0000","last_became_peered":"2024-08-02T17:44:11.252196+0000","last_unstale":"2024-08-02T17:46:24.907202+0000","last_undegraded":"2024-08-02T17:46:24.907202+0000","last_fullsized":"2024-08-02T17:46:24.907202+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:06:29.626752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959010+0000","last_change":"2024-08-02T17:44:04.122641+0000","last_active":"2024-08-02T17:46:24.959010+0000","last_peered":"2024-08-02T17:46:24.959010+0000","last_clean":"2024-08-02T17:46:24.959010+0000","last_became_active":"2024-08-02T17:44:04.122323+0000","last_became_peered":"2024-08-02T17:44:04.122323+0000","last_unstale":"2024-08-02T17:46:24.959010+0000","last_undegraded":"2024-08-02T17:46:24.959010+0000","last_fullsized":"2024-08-02T17:46:24.959010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:48:10.556108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920005+0000","last_change":"2024-08-02T17:44:57.792635+0000","last_active":"2024-08-02T17:46:24.920005+0000","last_peered":"2024-08-02T17:46:24.920005+0000","last_clean":"2024-08-02T17:46:24.920005+0000","last_became_active":"2024-08-02T17:44:57.787078+0000","last_became_peered":"2024-08-02T17:44:57.787078+0000","last_unstale":"2024-08-02T17:46:24.920005+0000","last_undegraded":"2024-08-02T17:46:24.920005+0000","last_fullsized":"2024-08-02T17:46:24.920005+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:41:07.616056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907441+0000","last_change":"2024-08-02T17:44:11.173542+0000","last_active":"2024-08-02T17:46:24.907441+0000","last_peered":"2024-08-02T17:46:24.907441+0000","last_clean":"2024-08-02T17:46:24.907441+0000","last_became_active":"2024-08-02T17:44:11.173128+0000","last_became_peered":"2024-08-02T17:44:11.173128+0000","last_unstale":"2024-08-02T17:46:24.907441+0000","last_undegraded":"2024-08-02T17:46:24.907441+0000","last_fullsized":"2024-08-02T17:46:24.907441+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:16.964448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956539+0000","last_change":"2024-08-02T17:44:04.126976+0000","last_active":"2024-08-02T17:46:24.956539+0000","last_peered":"2024-08-02T17:46:24.956539+0000","last_clean":"2024-08-02T17:46:24.956539+0000","last_became_active":"2024-08-02T17:44:04.126518+0000","last_became_peered":"2024-08-02T17:44:04.126518+0000","last_unstale":"2024-08-02T17:46:24.956539+0000","last_undegraded":"2024-08-02T17:46:24.956539+0000","last_fullsized":"2024-08-02T17:46:24.956539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:05:30.822189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907900+0000","last_change":"2024-08-02T17:44:02.044752+0000","last_active":"2024-08-02T17:46:24.907900+0000","last_peered":"2024-08-02T17:46:24.907900+0000","last_clean":"2024-08-02T17:46:24.907900+0000","last_became_active":"2024-08-02T17:44:02.044386+0000","last_became_peered":"2024-08-02T17:44:02.044386+0000","last_unstale":"2024-08-02T17:46:24.907900+0000","last_undegraded":"2024-08-02T17:46:24.907900+0000","last_fullsized":"2024-08-02T17:46:24.907900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:42:52.363481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915338+0000","last_change":"2024-08-02T17:44:11.172500+0000","last_active":"2024-08-02T17:46:24.915338+0000","last_peered":"2024-08-02T17:46:24.915338+0000","last_clean":"2024-08-02T17:46:24.915338+0000","last_became_active":"2024-08-02T17:44:11.172152+0000","last_became_peered":"2024-08-02T17:44:11.172152+0000","last_unstale":"2024-08-02T17:46:24.915338+0000","last_undegraded":"2024-08-02T17:46:24.915338+0000","last_fullsized":"2024-08-02T17:46:24.915338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:51.124827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"95'2","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924149+0000","last_change":"2024-08-02T17:44:04.132504+0000","last_active":"2024-08-02T17:46:24.924149+0000","last_peered":"2024-08-02T17:46:24.924149+0000","last_clean":"2024-08-02T17:46:24.924149+0000","last_became_active":"2024-08-02T17:44:04.132178+0000","last_became_peered":"2024-08-02T17:44:04.132178+0000","last_unstale":"2024-08-02T17:46:24.924149+0000","last_undegraded":"2024-08-02T17:46:24.924149+0000","last_fullsized":"2024-08-02T17:46:24.924149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:46:15.300276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915653+0000","last_change":"2024-08-02T17:44:02.202700+0000","last_active":"2024-08-02T17:46:24.915653+0000","last_peered":"2024-08-02T17:46:24.915653+0000","last_clean":"2024-08-02T17:46:24.915653+0000","last_became_active":"2024-08-02T17:44:02.202368+0000","last_became_peered":"2024-08-02T17:44:02.202368+0000","last_unstale":"2024-08-02T17:46:24.915653+0000","last_undegraded":"2024-08-02T17:46:24.915653+0000","last_fullsized":"2024-08-02T17:46:24.915653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:50.649649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907285+0000","last_change":"2024-08-02T17:44:57.789019+0000","last_active":"2024-08-02T17:46:24.907285+0000","last_peered":"2024-08-02T17:46:24.907285+0000","last_clean":"2024-08-02T17:46:24.907285+0000","last_became_active":"2024-08-02T17:44:57.788557+0000","last_became_peered":"2024-08-02T17:44:57.788557+0000","last_unstale":"2024-08-02T17:46:24.907285+0000","last_undegraded":"2024-08-02T17:46:24.907285+0000","last_fullsized":"2024-08-02T17:46:24.907285+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:58:56.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,8],"acting":[4,5,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961716+0000","last_change":"2024-08-02T17:44:04.133033+0000","last_active":"2024-08-02T17:46:24.961716+0000","last_peered":"2024-08-02T17:46:24.961716+0000","last_clean":"2024-08-02T17:46:24.961716+0000","last_became_active":"2024-08-02T17:44:04.132747+0000","last_became_peered":"2024-08-02T17:44:04.132747+0000","last_unstale":"2024-08-02T17:46:24.961716+0000","last_undegraded":"2024-08-02T17:46:24.961716+0000","last_fullsized":"2024-08-02T17:46:24.961716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:32:52.812120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907489+0000","last_change":"2024-08-02T17:44:02.047528+0000","last_active":"2024-08-02T17:46:24.907489+0000","last_peered":"2024-08-02T17:46:24.907489+0000","last_clean":"2024-08-02T17:46:24.907489+0000","last_became_active":"2024-08-02T17:44:02.045985+0000","last_became_peered":"2024-08-02T17:44:02.045985+0000","last_unstale":"2024-08-02T17:46:24.907489+0000","last_undegraded":"2024-08-02T17:46:24.907489+0000","last_fullsized":"2024-08-02T17:46:24.907489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:35:53.368744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915365+0000","last_change":"2024-08-02T17:44:11.172873+0000","last_active":"2024-08-02T17:46:24.915365+0000","last_peered":"2024-08-02T17:46:24.915365+0000","last_clean":"2024-08-02T17:46:24.915365+0000","last_became_active":"2024-08-02T17:44:11.172355+0000","last_became_peered":"2024-08-02T17:44:11.172355+0000","last_unstale":"2024-08-02T17:46:24.915365+0000","last_undegraded":"2024-08-02T17:46:24.915365+0000","last_fullsized":"2024-08-02T17:46:24.915365+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:53:10.174645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"95'5","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956310+0000","last_change":"2024-08-02T17:44:04.129113+0000","last_active":"2024-08-02T17:46:24.956310+0000","last_peered":"2024-08-02T17:46:24.956310+0000","last_clean":"2024-08-02T17:46:24.956310+0000","last_became_active":"2024-08-02T17:44:04.128827+0000","last_became_peered":"2024-08-02T17:44:04.128827+0000","last_unstale":"2024-08-02T17:46:24.956310+0000","last_undegraded":"2024-08-02T17:46:24.956310+0000","last_fullsized":"2024-08-02T17:46:24.956310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:54:23.180296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'101","reported_seq":245,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:36.780078+0000","last_change":"2024-08-02T17:44:57.789646+0000","last_active":"2024-08-02T17:46:36.780078+0000","last_peered":"2024-08-02T17:46:36.780078+0000","last_clean":"2024-08-02T17:46:36.780078+0000","last_became_active":"2024-08-02T17:44:57.782913+0000","last_became_peered":"2024-08-02T17:44:57.782913+0000","last_unstale":"2024-08-02T17:46:36.780078+0000","last_undegraded":"2024-08-02T17:46:36.780078+0000","last_fullsized":"2024-08-02T17:46:36.780078+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:11:31.806650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":186,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907676+0000","last_change":"2024-08-02T17:44:11.252787+0000","last_active":"2024-08-02T17:46:24.907676+0000","last_peered":"2024-08-02T17:46:24.907676+0000","last_clean":"2024-08-02T17:46:24.907676+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.907676+0000","last_undegraded":"2024-08-02T17:46:24.907676+0000","last_fullsized":"2024-08-02T17:46:24.907676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:44:32.405910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950528+0000","last_change":"2024-08-02T17:44:04.122105+0000","last_active":"2024-08-02T17:46:24.950528+0000","last_peered":"2024-08-02T17:46:24.950528+0000","last_clean":"2024-08-02T17:46:24.950528+0000","last_became_active":"2024-08-02T17:44:04.121825+0000","last_became_peered":"2024-08-02T17:44:04.121825+0000","last_unstale":"2024-08-02T17:46:24.950528+0000","last_undegraded":"2024-08-02T17:46:24.950528+0000","last_fullsized":"2024-08-02T17:46:24.950528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:41:23.647917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906878+0000","last_change":"2024-08-02T17:44:02.202543+0000","last_active":"2024-08-02T17:46:24.906878+0000","last_peered":"2024-08-02T17:46:24.906878+0000","last_clean":"2024-08-02T17:46:24.906878+0000","last_became_active":"2024-08-02T17:44:02.202129+0000","last_became_peered":"2024-08-02T17:44:02.202129+0000","last_unstale":"2024-08-02T17:46:24.906878+0000","last_undegraded":"2024-08-02T17:46:24.906878+0000","last_fullsized":"2024-08-02T17:46:24.906878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:47:19.412861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920332+0000","last_change":"2024-08-02T17:44:11.170424+0000","last_active":"2024-08-02T17:46:24.920332+0000","last_peered":"2024-08-02T17:46:24.920332+0000","last_clean":"2024-08-02T17:46:24.920332+0000","last_became_active":"2024-08-02T17:44:11.170002+0000","last_became_peered":"2024-08-02T17:44:11.170002+0000","last_unstale":"2024-08-02T17:46:24.920332+0000","last_undegraded":"2024-08-02T17:46:24.920332+0000","last_fullsized":"2024-08-02T17:46:24.920332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:56:30.370760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953960+0000","last_change":"2024-08-02T17:44:04.123088+0000","last_active":"2024-08-02T17:46:24.953960+0000","last_peered":"2024-08-02T17:46:24.953960+0000","last_clean":"2024-08-02T17:46:24.953960+0000","last_became_active":"2024-08-02T17:44:04.122705+0000","last_became_peered":"2024-08-02T17:44:04.122705+0000","last_unstale":"2024-08-02T17:46:24.953960+0000","last_undegraded":"2024-08-02T17:46:24.953960+0000","last_fullsized":"2024-08-02T17:46:24.953960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:56:42.300855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915185+0000","last_change":"2024-08-02T17:44:02.204446+0000","last_active":"2024-08-02T17:46:24.915185+0000","last_peered":"2024-08-02T17:46:24.915185+0000","last_clean":"2024-08-02T17:46:24.915185+0000","last_became_active":"2024-08-02T17:44:02.203887+0000","last_became_peered":"2024-08-02T17:44:02.203887+0000","last_unstale":"2024-08-02T17:46:24.915185+0000","last_undegraded":"2024-08-02T17:46:24.915185+0000","last_fullsized":"2024-08-02T17:46:24.915185+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:27:44.874389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906282+0000","last_change":"2024-08-02T17:44:11.252734+0000","last_active":"2024-08-02T17:46:24.906282+0000","last_peered":"2024-08-02T17:46:24.906282+0000","last_clean":"2024-08-02T17:46:24.906282+0000","last_became_active":"2024-08-02T17:44:11.252354+0000","last_became_peered":"2024-08-02T17:44:11.252354+0000","last_unstale":"2024-08-02T17:46:24.906282+0000","last_undegraded":"2024-08-02T17:46:24.906282+0000","last_fullsized":"2024-08-02T17:46:24.906282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:25:58.827766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956518+0000","last_change":"2024-08-02T17:44:04.127749+0000","last_active":"2024-08-02T17:46:24.956518+0000","last_peered":"2024-08-02T17:46:24.956518+0000","last_clean":"2024-08-02T17:46:24.956518+0000","last_became_active":"2024-08-02T17:44:04.127408+0000","last_became_peered":"2024-08-02T17:44:04.127408+0000","last_unstale":"2024-08-02T17:46:24.956518+0000","last_undegraded":"2024-08-02T17:46:24.956518+0000","last_fullsized":"2024-08-02T17:46:24.956518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:16:22.039473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.909639+0000","last_change":"2024-08-02T17:44:02.046881+0000","last_active":"2024-08-02T17:46:24.909639+0000","last_peered":"2024-08-02T17:46:24.909639+0000","last_clean":"2024-08-02T17:46:24.909639+0000","last_became_active":"2024-08-02T17:44:02.046563+0000","last_became_peered":"2024-08-02T17:44:02.046563+0000","last_unstale":"2024-08-02T17:46:24.909639+0000","last_undegraded":"2024-08-02T17:46:24.909639+0000","last_fullsized":"2024-08-02T17:46:24.909639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:28:25.144316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.918812+0000","last_change":"2024-08-02T17:44:11.171416+0000","last_active":"2024-08-02T17:46:24.918812+0000","last_peered":"2024-08-02T17:46:24.918812+0000","last_clean":"2024-08-02T17:46:24.918812+0000","last_became_active":"2024-08-02T17:44:11.171073+0000","last_became_peered":"2024-08-02T17:44:11.171073+0000","last_unstale":"2024-08-02T17:46:24.918812+0000","last_undegraded":"2024-08-02T17:46:24.918812+0000","last_fullsized":"2024-08-02T17:46:24.918812+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:27:06.143421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952882+0000","last_change":"2024-08-02T17:44:04.122942+0000","last_active":"2024-08-02T17:46:24.952882+0000","last_peered":"2024-08-02T17:46:24.952882+0000","last_clean":"2024-08-02T17:46:24.952882+0000","last_became_active":"2024-08-02T17:44:04.122331+0000","last_became_peered":"2024-08-02T17:44:04.122331+0000","last_unstale":"2024-08-02T17:46:24.952882+0000","last_undegraded":"2024-08-02T17:46:24.952882+0000","last_fullsized":"2024-08-02T17:46:24.952882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:16:07.296522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":158,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:39.074557+0000","last_change":"2024-08-02T17:44:02.202706+0000","last_active":"2024-08-02T17:46:39.074557+0000","last_peered":"2024-08-02T17:46:39.074557+0000","last_clean":"2024-08-02T17:46:39.074557+0000","last_became_active":"2024-08-02T17:44:02.202403+0000","last_became_peered":"2024-08-02T17:44:02.202403+0000","last_unstale":"2024-08-02T17:46:39.074557+0000","last_undegraded":"2024-08-02T17:46:39.074557+0000","last_fullsized":"2024-08-02T17:46:39.074557+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:48:03.725225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919707+0000","last_change":"2024-08-02T17:44:11.252875+0000","last_active":"2024-08-02T17:46:24.919707+0000","last_peered":"2024-08-02T17:46:24.919707+0000","last_clean":"2024-08-02T17:46:24.919707+0000","last_became_active":"2024-08-02T17:44:11.252414+0000","last_became_peered":"2024-08-02T17:44:11.252414+0000","last_unstale":"2024-08-02T17:46:24.919707+0000","last_undegraded":"2024-08-02T17:46:24.919707+0000","last_fullsized":"2024-08-02T17:46:24.919707+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:28:26.332908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947422+0000","last_change":"2024-08-02T17:44:04.127137+0000","last_active":"2024-08-02T17:46:24.947422+0000","last_peered":"2024-08-02T17:46:24.947422+0000","last_clean":"2024-08-02T17:46:24.947422+0000","last_became_active":"2024-08-02T17:44:04.126401+0000","last_became_peered":"2024-08-02T17:44:04.126401+0000","last_unstale":"2024-08-02T17:46:24.947422+0000","last_undegraded":"2024-08-02T17:46:24.947422+0000","last_fullsized":"2024-08-02T17:46:24.947422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:01.865591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.908927+0000","last_change":"2024-08-02T17:44:58.062270+0000","last_active":"2024-08-02T17:46:24.908927+0000","last_peered":"2024-08-02T17:46:24.908927+0000","last_clean":"2024-08-02T17:46:24.908927+0000","last_became_active":"2024-08-02T17:44:58.059431+0000","last_became_peered":"2024-08-02T17:44:58.059431+0000","last_unstale":"2024-08-02T17:46:24.908927+0000","last_undegraded":"2024-08-02T17:46:24.908927+0000","last_fullsized":"2024-08-02T17:46:24.908927+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:25.395019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915349+0000","last_change":"2024-08-02T17:44:11.253868+0000","last_active":"2024-08-02T17:46:24.915349+0000","last_peered":"2024-08-02T17:46:24.915349+0000","last_clean":"2024-08-02T17:46:24.915349+0000","last_became_active":"2024-08-02T17:44:11.253524+0000","last_became_peered":"2024-08-02T17:44:11.253524+0000","last_unstale":"2024-08-02T17:46:24.915349+0000","last_undegraded":"2024-08-02T17:46:24.915349+0000","last_fullsized":"2024-08-02T17:46:24.915349+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:17:09.988224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962129+0000","last_change":"2024-08-02T17:44:04.119196+0000","last_active":"2024-08-02T17:46:24.962129+0000","last_peered":"2024-08-02T17:46:24.962129+0000","last_clean":"2024-08-02T17:46:24.962129+0000","last_became_active":"2024-08-02T17:44:04.118899+0000","last_became_peered":"2024-08-02T17:44:04.118899+0000","last_unstale":"2024-08-02T17:46:24.962129+0000","last_undegraded":"2024-08-02T17:46:24.962129+0000","last_fullsized":"2024-08-02T17:46:24.962129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:28:21.190097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907233+0000","last_change":"2024-08-02T17:44:02.048244+0000","last_active":"2024-08-02T17:46:24.907233+0000","last_peered":"2024-08-02T17:46:24.907233+0000","last_clean":"2024-08-02T17:46:24.907233+0000","last_became_active":"2024-08-02T17:44:02.047905+0000","last_became_peered":"2024-08-02T17:44:02.047905+0000","last_unstale":"2024-08-02T17:46:24.907233+0000","last_undegraded":"2024-08-02T17:46:24.907233+0000","last_fullsized":"2024-08-02T17:46:24.907233+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:50:31.094388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":412,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:27.046748+0000","last_change":"2024-08-02T17:42:54.859090+0000","last_active":"2024-08-02T17:46:27.046748+0000","last_peered":"2024-08-02T17:46:27.046748+0000","last_clean":"2024-08-02T17:46:27.046748+0000","last_became_active":"2024-08-02T17:42:54.858630+0000","last_became_peered":"2024-08-02T17:42:54.858630+0000","last_unstale":"2024-08-02T17:46:27.046748+0000","last_undegraded":"2024-08-02T17:46:27.046748+0000","last_fullsized":"2024-08-02T17:46:27.046748+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:39:55.073226+0000","last_clean_scrub_stamp":"2024-08-02T17:39:55.073226+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:12:23.588686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910641+0000","last_change":"2024-08-02T17:44:57.793617+0000","last_active":"2024-08-02T17:46:24.910641+0000","last_peered":"2024-08-02T17:46:24.910641+0000","last_clean":"2024-08-02T17:46:24.910641+0000","last_became_active":"2024-08-02T17:44:57.786715+0000","last_became_peered":"2024-08-02T17:44:57.786715+0000","last_unstale":"2024-08-02T17:46:24.910641+0000","last_undegraded":"2024-08-02T17:46:24.910641+0000","last_fullsized":"2024-08-02T17:46:24.910641+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:07:37.908565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,11,9],"acting":[7,8,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960428+0000","last_change":"2024-08-02T17:44:04.130227+0000","last_active":"2024-08-02T17:46:24.960428+0000","last_peered":"2024-08-02T17:46:24.960428+0000","last_clean":"2024-08-02T17:46:24.960428+0000","last_became_active":"2024-08-02T17:44:04.129893+0000","last_became_peered":"2024-08-02T17:44:04.129893+0000","last_unstale":"2024-08-02T17:46:24.960428+0000","last_undegraded":"2024-08-02T17:46:24.960428+0000","last_fullsized":"2024-08-02T17:46:24.960428+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:43:39.181725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.916318+0000","last_change":"2024-08-02T17:44:02.203321+0000","last_active":"2024-08-02T17:46:24.916318+0000","last_peered":"2024-08-02T17:46:24.916318+0000","last_clean":"2024-08-02T17:46:24.916318+0000","last_became_active":"2024-08-02T17:44:02.202976+0000","last_became_peered":"2024-08-02T17:44:02.202976+0000","last_unstale":"2024-08-02T17:46:24.916318+0000","last_undegraded":"2024-08-02T17:46:24.916318+0000","last_fullsized":"2024-08-02T17:46:24.916318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:58:19.832130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915831+0000","last_change":"2024-08-02T17:44:11.170831+0000","last_active":"2024-08-02T17:46:24.915831+0000","last_peered":"2024-08-02T17:46:24.915831+0000","last_clean":"2024-08-02T17:46:24.915831+0000","last_became_active":"2024-08-02T17:44:11.170428+0000","last_became_peered":"2024-08-02T17:44:11.170428+0000","last_unstale":"2024-08-02T17:46:24.915831+0000","last_undegraded":"2024-08-02T17:46:24.915831+0000","last_fullsized":"2024-08-02T17:46:24.915831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:11:07.273271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948829+0000","last_change":"2024-08-02T17:44:04.133999+0000","last_active":"2024-08-02T17:46:24.948829+0000","last_peered":"2024-08-02T17:46:24.948829+0000","last_clean":"2024-08-02T17:46:24.948829+0000","last_became_active":"2024-08-02T17:44:04.133690+0000","last_became_peered":"2024-08-02T17:44:04.133690+0000","last_unstale":"2024-08-02T17:46:24.948829+0000","last_undegraded":"2024-08-02T17:46:24.948829+0000","last_fullsized":"2024-08-02T17:46:24.948829+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:36:29.901676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'65","reported_seq":136,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:36.431887+0000","last_change":"2024-08-02T17:44:02.048058+0000","last_active":"2024-08-02T17:46:36.431887+0000","last_peered":"2024-08-02T17:46:36.431887+0000","last_clean":"2024-08-02T17:46:36.431887+0000","last_became_active":"2024-08-02T17:44:02.047664+0000","last_became_peered":"2024-08-02T17:44:02.047664+0000","last_unstale":"2024-08-02T17:46:36.431887+0000","last_undegraded":"2024-08-02T17:46:36.431887+0000","last_fullsized":"2024-08-02T17:46:36.431887+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:13:28.256969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.924774+0000","last_change":"2024-08-02T17:44:11.167502+0000","last_active":"2024-08-02T17:46:24.924774+0000","last_peered":"2024-08-02T17:46:24.924774+0000","last_clean":"2024-08-02T17:46:24.924774+0000","last_became_active":"2024-08-02T17:44:11.167108+0000","last_became_peered":"2024-08-02T17:44:11.167108+0000","last_unstale":"2024-08-02T17:46:24.924774+0000","last_undegraded":"2024-08-02T17:46:24.924774+0000","last_fullsized":"2024-08-02T17:46:24.924774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:53.508149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955439+0000","last_change":"2024-08-02T17:44:04.136627+0000","last_active":"2024-08-02T17:46:24.955439+0000","last_peered":"2024-08-02T17:46:24.955439+0000","last_clean":"2024-08-02T17:46:24.955439+0000","last_became_active":"2024-08-02T17:44:04.136334+0000","last_became_peered":"2024-08-02T17:44:04.136334+0000","last_unstale":"2024-08-02T17:46:24.955439+0000","last_undegraded":"2024-08-02T17:46:24.955439+0000","last_fullsized":"2024-08-02T17:46:24.955439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:59:05.963648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919542+0000","last_change":"2024-08-02T17:44:02.049858+0000","last_active":"2024-08-02T17:46:24.919542+0000","last_peered":"2024-08-02T17:46:24.919542+0000","last_clean":"2024-08-02T17:46:24.919542+0000","last_became_active":"2024-08-02T17:44:02.049467+0000","last_became_peered":"2024-08-02T17:44:02.049467+0000","last_unstale":"2024-08-02T17:46:24.919542+0000","last_undegraded":"2024-08-02T17:46:24.919542+0000","last_fullsized":"2024-08-02T17:46:24.919542+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:50:09.612204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914656+0000","last_change":"2024-08-02T17:44:11.174486+0000","last_active":"2024-08-02T17:46:24.914656+0000","last_peered":"2024-08-02T17:46:24.914656+0000","last_clean":"2024-08-02T17:46:24.914656+0000","last_became_active":"2024-08-02T17:44:11.174103+0000","last_became_peered":"2024-08-02T17:44:11.174103+0000","last_unstale":"2024-08-02T17:46:24.914656+0000","last_undegraded":"2024-08-02T17:46:24.914656+0000","last_fullsized":"2024-08-02T17:46:24.914656+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:30:39.209727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"95'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950756+0000","last_change":"2024-08-02T17:45:57.979722+0000","last_active":"2024-08-02T17:46:24.950756+0000","last_peered":"2024-08-02T17:46:24.950756+0000","last_clean":"2024-08-02T17:46:24.950756+0000","last_became_active":"2024-08-02T17:45:57.979427+0000","last_became_peered":"2024-08-02T17:45:57.979427+0000","last_unstale":"2024-08-02T17:46:24.950756+0000","last_undegraded":"2024-08-02T17:46:24.950756+0000","last_fullsized":"2024-08-02T17:46:24.950756+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:16:52.457182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:25.921341+0000","last_change":"2024-08-02T17:44:02.046183+0000","last_active":"2024-08-02T17:46:25.921341+0000","last_peered":"2024-08-02T17:46:25.921341+0000","last_clean":"2024-08-02T17:46:25.921341+0000","last_became_active":"2024-08-02T17:44:02.045668+0000","last_became_peered":"2024-08-02T17:44:02.045668+0000","last_unstale":"2024-08-02T17:46:25.921341+0000","last_undegraded":"2024-08-02T17:46:25.921341+0000","last_fullsized":"2024-08-02T17:46:25.921341+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:43.199977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914880+0000","last_change":"2024-08-02T17:44:11.176223+0000","last_active":"2024-08-02T17:46:24.914880+0000","last_peered":"2024-08-02T17:46:24.914880+0000","last_clean":"2024-08-02T17:46:24.914880+0000","last_became_active":"2024-08-02T17:44:11.174878+0000","last_became_peered":"2024-08-02T17:44:11.174878+0000","last_unstale":"2024-08-02T17:46:24.914880+0000","last_undegraded":"2024-08-02T17:46:24.914880+0000","last_fullsized":"2024-08-02T17:46:24.914880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:04:19.338020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953834+0000","last_change":"2024-08-02T17:44:04.131715+0000","last_active":"2024-08-02T17:46:24.953834+0000","last_peered":"2024-08-02T17:46:24.953834+0000","last_clean":"2024-08-02T17:46:24.953834+0000","last_became_active":"2024-08-02T17:44:04.128665+0000","last_became_peered":"2024-08-02T17:44:04.128665+0000","last_unstale":"2024-08-02T17:46:24.953834+0000","last_undegraded":"2024-08-02T17:46:24.953834+0000","last_fullsized":"2024-08-02T17:46:24.953834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:41:03.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":81,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.906346+0000","last_change":"2024-08-02T17:44:57.800995+0000","last_active":"2024-08-02T17:46:24.906346+0000","last_peered":"2024-08-02T17:46:24.906346+0000","last_clean":"2024-08-02T17:46:24.906346+0000","last_became_active":"2024-08-02T17:44:57.789594+0000","last_became_peered":"2024-08-02T17:44:57.789594+0000","last_unstale":"2024-08-02T17:46:24.906346+0000","last_undegraded":"2024-08-02T17:46:24.906346+0000","last_fullsized":"2024-08-02T17:46:24.906346+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:37:58.835984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919489+0000","last_change":"2024-08-02T17:44:11.174091+0000","last_active":"2024-08-02T17:46:24.919489+0000","last_peered":"2024-08-02T17:46:24.919489+0000","last_clean":"2024-08-02T17:46:24.919489+0000","last_became_active":"2024-08-02T17:44:11.173713+0000","last_became_peered":"2024-08-02T17:44:11.173713+0000","last_unstale":"2024-08-02T17:46:24.919489+0000","last_undegraded":"2024-08-02T17:46:24.919489+0000","last_fullsized":"2024-08-02T17:46:24.919489+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:54:01.523803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950106+0000","last_change":"2024-08-02T17:44:04.128921+0000","last_active":"2024-08-02T17:46:24.950106+0000","last_peered":"2024-08-02T17:46:24.950106+0000","last_clean":"2024-08-02T17:46:24.950106+0000","last_became_active":"2024-08-02T17:44:04.128588+0000","last_became_peered":"2024-08-02T17:44:04.128588+0000","last_unstale":"2024-08-02T17:46:24.950106+0000","last_undegraded":"2024-08-02T17:46:24.950106+0000","last_fullsized":"2024-08-02T17:46:24.950106+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:12:07.379999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'9","reported_seq":170,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:39.074894+0000","last_change":"2024-08-02T17:44:02.044684+0000","last_active":"2024-08-02T17:46:39.074894+0000","last_peered":"2024-08-02T17:46:39.074894+0000","last_clean":"2024-08-02T17:46:39.074894+0000","last_became_active":"2024-08-02T17:44:02.044341+0000","last_became_peered":"2024-08-02T17:44:02.044341+0000","last_unstale":"2024-08-02T17:46:39.074894+0000","last_undegraded":"2024-08-02T17:46:39.074894+0000","last_fullsized":"2024-08-02T17:46:39.074894+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:15:21.214237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2597,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":8,"num_write":8,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915391+0000","last_change":"2024-08-02T17:44:11.164769+0000","last_active":"2024-08-02T17:46:24.915391+0000","last_peered":"2024-08-02T17:46:24.915391+0000","last_clean":"2024-08-02T17:46:24.915391+0000","last_became_active":"2024-08-02T17:44:11.164382+0000","last_became_peered":"2024-08-02T17:44:11.164382+0000","last_unstale":"2024-08-02T17:46:24.915391+0000","last_undegraded":"2024-08-02T17:46:24.915391+0000","last_fullsized":"2024-08-02T17:46:24.915391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:07:08.002015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957938+0000","last_change":"2024-08-02T17:44:04.131601+0000","last_active":"2024-08-02T17:46:24.957938+0000","last_peered":"2024-08-02T17:46:24.957938+0000","last_clean":"2024-08-02T17:46:24.957938+0000","last_became_active":"2024-08-02T17:44:04.131262+0000","last_became_peered":"2024-08-02T17:44:04.131262+0000","last_unstale":"2024-08-02T17:46:24.957938+0000","last_undegraded":"2024-08-02T17:46:24.957938+0000","last_fullsized":"2024-08-02T17:46:24.957938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:06:10.238258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915099+0000","last_change":"2024-08-02T17:44:02.046193+0000","last_active":"2024-08-02T17:46:24.915099+0000","last_peered":"2024-08-02T17:46:24.915099+0000","last_clean":"2024-08-02T17:46:24.915099+0000","last_became_active":"2024-08-02T17:44:02.045787+0000","last_became_peered":"2024-08-02T17:44:02.045787+0000","last_unstale":"2024-08-02T17:46:24.915099+0000","last_undegraded":"2024-08-02T17:46:24.915099+0000","last_fullsized":"2024-08-02T17:46:24.915099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:52.704652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920448+0000","last_change":"2024-08-02T17:44:57.935930+0000","last_active":"2024-08-02T17:46:24.920448+0000","last_peered":"2024-08-02T17:46:24.920448+0000","last_clean":"2024-08-02T17:46:24.920448+0000","last_became_active":"2024-08-02T17:44:57.935530+0000","last_became_peered":"2024-08-02T17:44:57.935530+0000","last_unstale":"2024-08-02T17:46:24.920448+0000","last_undegraded":"2024-08-02T17:46:24.920448+0000","last_fullsized":"2024-08-02T17:46:24.920448+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:03:44.630967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958381+0000","last_change":"2024-08-02T17:44:04.127293+0000","last_active":"2024-08-02T17:46:24.958381+0000","last_peered":"2024-08-02T17:46:24.958381+0000","last_clean":"2024-08-02T17:46:24.958381+0000","last_became_active":"2024-08-02T17:44:04.126781+0000","last_became_peered":"2024-08-02T17:44:04.126781+0000","last_unstale":"2024-08-02T17:46:24.958381+0000","last_undegraded":"2024-08-02T17:46:24.958381+0000","last_fullsized":"2024-08-02T17:46:24.958381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:03:27.359676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915872+0000","last_change":"2024-08-02T17:44:02.048805+0000","last_active":"2024-08-02T17:46:24.915872+0000","last_peered":"2024-08-02T17:46:24.915872+0000","last_clean":"2024-08-02T17:46:24.915872+0000","last_became_active":"2024-08-02T17:44:02.048449+0000","last_became_peered":"2024-08-02T17:44:02.048449+0000","last_unstale":"2024-08-02T17:46:24.915872+0000","last_undegraded":"2024-08-02T17:46:24.915872+0000","last_fullsized":"2024-08-02T17:46:24.915872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:47:43.326735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915506+0000","last_change":"2024-08-02T17:44:11.175180+0000","last_active":"2024-08-02T17:46:24.915506+0000","last_peered":"2024-08-02T17:46:24.915506+0000","last_clean":"2024-08-02T17:46:24.915506+0000","last_became_active":"2024-08-02T17:44:11.174830+0000","last_became_peered":"2024-08-02T17:44:11.174830+0000","last_unstale":"2024-08-02T17:46:24.915506+0000","last_undegraded":"2024-08-02T17:46:24.915506+0000","last_fullsized":"2024-08-02T17:46:24.915506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:53:46.024394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"95'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955796+0000","last_change":"2024-08-02T17:44:04.132364+0000","last_active":"2024-08-02T17:46:24.955796+0000","last_peered":"2024-08-02T17:46:24.955796+0000","last_clean":"2024-08-02T17:46:24.955796+0000","last_became_active":"2024-08-02T17:44:04.132058+0000","last_became_peered":"2024-08-02T17:44:04.132058+0000","last_unstale":"2024-08-02T17:46:24.955796+0000","last_undegraded":"2024-08-02T17:46:24.955796+0000","last_fullsized":"2024-08-02T17:46:24.955796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T02:50:08.996363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":323,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:40.075174+0000","last_change":"2024-08-02T17:44:02.050190+0000","last_active":"2024-08-02T17:46:40.075174+0000","last_peered":"2024-08-02T17:46:40.075174+0000","last_clean":"2024-08-02T17:46:40.075174+0000","last_became_active":"2024-08-02T17:44:02.049523+0000","last_became_peered":"2024-08-02T17:44:02.049523+0000","last_unstale":"2024-08-02T17:46:40.075174+0000","last_undegraded":"2024-08-02T17:46:40.075174+0000","last_fullsized":"2024-08-02T17:46:40.075174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:43:58.937610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72624,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":191,"num_write":42,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911360+0000","last_change":"2024-08-02T17:44:57.785527+0000","last_active":"2024-08-02T17:46:24.911360+0000","last_peered":"2024-08-02T17:46:24.911360+0000","last_clean":"2024-08-02T17:46:24.911360+0000","last_became_active":"2024-08-02T17:44:57.785223+0000","last_became_peered":"2024-08-02T17:44:57.785223+0000","last_unstale":"2024-08-02T17:46:24.911360+0000","last_undegraded":"2024-08-02T17:46:24.911360+0000","last_fullsized":"2024-08-02T17:46:24.911360+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:55:04.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959993+0000","last_change":"2024-08-02T17:44:04.138295+0000","last_active":"2024-08-02T17:46:24.959993+0000","last_peered":"2024-08-02T17:46:24.959993+0000","last_clean":"2024-08-02T17:46:24.959993+0000","last_became_active":"2024-08-02T17:44:04.137915+0000","last_became_peered":"2024-08-02T17:44:04.137915+0000","last_unstale":"2024-08-02T17:46:24.959993+0000","last_undegraded":"2024-08-02T17:46:24.959993+0000","last_fullsized":"2024-08-02T17:46:24.959993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:38:38.500117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915691+0000","last_change":"2024-08-02T17:44:02.204609+0000","last_active":"2024-08-02T17:46:24.915691+0000","last_peered":"2024-08-02T17:46:24.915691+0000","last_clean":"2024-08-02T17:46:24.915691+0000","last_became_active":"2024-08-02T17:44:02.204233+0000","last_became_peered":"2024-08-02T17:44:02.204233+0000","last_unstale":"2024-08-02T17:46:24.915691+0000","last_undegraded":"2024-08-02T17:46:24.915691+0000","last_fullsized":"2024-08-02T17:46:24.915691+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:39:28.574522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907068+0000","last_change":"2024-08-02T17:44:11.175546+0000","last_active":"2024-08-02T17:46:24.907068+0000","last_peered":"2024-08-02T17:46:24.907068+0000","last_clean":"2024-08-02T17:46:24.907068+0000","last_became_active":"2024-08-02T17:44:11.175004+0000","last_became_peered":"2024-08-02T17:44:11.175004+0000","last_unstale":"2024-08-02T17:46:24.907068+0000","last_undegraded":"2024-08-02T17:46:24.907068+0000","last_fullsized":"2024-08-02T17:46:24.907068+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:06:48.258046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960681+0000","last_change":"2024-08-02T17:44:04.129710+0000","last_active":"2024-08-02T17:46:24.960681+0000","last_peered":"2024-08-02T17:46:24.960681+0000","last_clean":"2024-08-02T17:46:24.960681+0000","last_became_active":"2024-08-02T17:44:04.129393+0000","last_became_peered":"2024-08-02T17:44:04.129393+0000","last_unstale":"2024-08-02T17:46:24.960681+0000","last_undegraded":"2024-08-02T17:46:24.960681+0000","last_fullsized":"2024-08-02T17:46:24.960681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:14:35.635053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.910512+0000","last_change":"2024-08-02T17:44:02.049622+0000","last_active":"2024-08-02T17:46:24.910512+0000","last_peered":"2024-08-02T17:46:24.910512+0000","last_clean":"2024-08-02T17:46:24.910512+0000","last_became_active":"2024-08-02T17:44:02.049223+0000","last_became_peered":"2024-08-02T17:44:02.049223+0000","last_unstale":"2024-08-02T17:46:24.910512+0000","last_undegraded":"2024-08-02T17:46:24.910512+0000","last_fullsized":"2024-08-02T17:46:24.910512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:06:12.619613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914907+0000","last_change":"2024-08-02T17:44:11.174156+0000","last_active":"2024-08-02T17:46:24.914907+0000","last_peered":"2024-08-02T17:46:24.914907+0000","last_clean":"2024-08-02T17:46:24.914907+0000","last_became_active":"2024-08-02T17:44:11.173803+0000","last_became_peered":"2024-08-02T17:44:11.173803+0000","last_unstale":"2024-08-02T17:46:24.914907+0000","last_undegraded":"2024-08-02T17:46:24.914907+0000","last_fullsized":"2024-08-02T17:46:24.914907+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:24:49.824059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962425+0000","last_change":"2024-08-02T17:44:04.135956+0000","last_active":"2024-08-02T17:46:24.962425+0000","last_peered":"2024-08-02T17:46:24.962425+0000","last_clean":"2024-08-02T17:46:24.962425+0000","last_became_active":"2024-08-02T17:44:04.133043+0000","last_became_peered":"2024-08-02T17:44:04.133043+0000","last_unstale":"2024-08-02T17:46:24.962425+0000","last_undegraded":"2024-08-02T17:46:24.962425+0000","last_fullsized":"2024-08-02T17:46:24.962425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:30:04.326593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'4","reported_seq":158,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:39.074330+0000","last_change":"2024-08-02T17:44:02.051112+0000","last_active":"2024-08-02T17:46:39.074330+0000","last_peered":"2024-08-02T17:46:39.074330+0000","last_clean":"2024-08-02T17:46:39.074330+0000","last_became_active":"2024-08-02T17:44:02.050650+0000","last_became_peered":"2024-08-02T17:44:02.050650+0000","last_unstale":"2024-08-02T17:46:39.074330+0000","last_undegraded":"2024-08-02T17:46:39.074330+0000","last_fullsized":"2024-08-02T17:46:39.074330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:25:56.855336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.907833+0000","last_change":"2024-08-02T17:44:11.178284+0000","last_active":"2024-08-02T17:46:24.907833+0000","last_peered":"2024-08-02T17:46:24.907833+0000","last_clean":"2024-08-02T17:46:24.907833+0000","last_became_active":"2024-08-02T17:44:11.177907+0000","last_became_peered":"2024-08-02T17:44:11.177907+0000","last_unstale":"2024-08-02T17:46:24.907833+0000","last_undegraded":"2024-08-02T17:46:24.907833+0000","last_fullsized":"2024-08-02T17:46:24.907833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:19:00.789841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.926268+0000","last_change":"2024-08-02T17:44:11.168358+0000","last_active":"2024-08-02T17:46:24.926268+0000","last_peered":"2024-08-02T17:46:24.926268+0000","last_clean":"2024-08-02T17:46:24.926268+0000","last_became_active":"2024-08-02T17:44:11.167986+0000","last_became_peered":"2024-08-02T17:44:11.167986+0000","last_unstale":"2024-08-02T17:46:24.926268+0000","last_undegraded":"2024-08-02T17:46:24.926268+0000","last_fullsized":"2024-08-02T17:46:24.926268+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:16:20.179716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956375+0000","last_change":"2024-08-02T17:44:04.127808+0000","last_active":"2024-08-02T17:46:24.956375+0000","last_peered":"2024-08-02T17:46:24.956375+0000","last_clean":"2024-08-02T17:46:24.956375+0000","last_became_active":"2024-08-02T17:44:04.127528+0000","last_became_peered":"2024-08-02T17:44:04.127528+0000","last_unstale":"2024-08-02T17:46:24.956375+0000","last_undegraded":"2024-08-02T17:46:24.956375+0000","last_fullsized":"2024-08-02T17:46:24.956375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:41:21.742018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.914965+0000","last_change":"2024-08-02T17:44:02.048405+0000","last_active":"2024-08-02T17:46:24.914965+0000","last_peered":"2024-08-02T17:46:24.914965+0000","last_clean":"2024-08-02T17:46:24.914965+0000","last_became_active":"2024-08-02T17:44:02.047904+0000","last_became_peered":"2024-08-02T17:44:02.047904+0000","last_unstale":"2024-08-02T17:46:24.914965+0000","last_undegraded":"2024-08-02T17:46:24.914965+0000","last_fullsized":"2024-08-02T17:46:24.914965+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:59:06.822108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.920061+0000","last_change":"2024-08-02T17:44:11.171916+0000","last_active":"2024-08-02T17:46:24.920061+0000","last_peered":"2024-08-02T17:46:24.920061+0000","last_clean":"2024-08-02T17:46:24.920061+0000","last_became_active":"2024-08-02T17:44:11.171536+0000","last_became_peered":"2024-08-02T17:44:11.171536+0000","last_unstale":"2024-08-02T17:46:24.920061+0000","last_undegraded":"2024-08-02T17:46:24.920061+0000","last_fullsized":"2024-08-02T17:46:24.920061+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:08:34.041790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959427+0000","last_change":"2024-08-02T17:44:04.125636+0000","last_active":"2024-08-02T17:46:24.959427+0000","last_peered":"2024-08-02T17:46:24.959427+0000","last_clean":"2024-08-02T17:46:24.959427+0000","last_became_active":"2024-08-02T17:44:04.125334+0000","last_became_peered":"2024-08-02T17:44:04.125334+0000","last_unstale":"2024-08-02T17:46:24.959427+0000","last_undegraded":"2024-08-02T17:46:24.959427+0000","last_fullsized":"2024-08-02T17:46:24.959427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:37:30.014922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.919245+0000","last_change":"2024-08-02T17:44:02.043583+0000","last_active":"2024-08-02T17:46:24.919245+0000","last_peered":"2024-08-02T17:46:24.919245+0000","last_clean":"2024-08-02T17:46:24.919245+0000","last_became_active":"2024-08-02T17:44:02.043319+0000","last_became_peered":"2024-08-02T17:44:02.043319+0000","last_unstale":"2024-08-02T17:46:24.919245+0000","last_undegraded":"2024-08-02T17:46:24.919245+0000","last_fullsized":"2024-08-02T17:46:24.919245+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:58:37.973525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.911708+0000","last_change":"2024-08-02T17:44:02.047235+0000","last_active":"2024-08-02T17:46:24.911708+0000","last_peered":"2024-08-02T17:46:24.911708+0000","last_clean":"2024-08-02T17:46:24.911708+0000","last_became_active":"2024-08-02T17:44:02.046826+0000","last_became_peered":"2024-08-02T17:44:02.046826+0000","last_unstale":"2024-08-02T17:46:24.911708+0000","last_undegraded":"2024-08-02T17:46:24.911708+0000","last_fullsized":"2024-08-02T17:46:24.911708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:13:23.692315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950898+0000","last_change":"2024-08-02T17:44:04.129794+0000","last_active":"2024-08-02T17:46:24.950898+0000","last_peered":"2024-08-02T17:46:24.950898+0000","last_clean":"2024-08-02T17:46:24.950898+0000","last_became_active":"2024-08-02T17:44:04.129099+0000","last_became_peered":"2024-08-02T17:44:04.129099+0000","last_unstale":"2024-08-02T17:46:24.950898+0000","last_undegraded":"2024-08-02T17:46:24.950898+0000","last_fullsized":"2024-08-02T17:46:24.950898+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:33:14.083434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915973+0000","last_change":"2024-08-02T17:44:11.177610+0000","last_active":"2024-08-02T17:46:24.915973+0000","last_peered":"2024-08-02T17:46:24.915973+0000","last_clean":"2024-08-02T17:46:24.915973+0000","last_became_active":"2024-08-02T17:44:11.176374+0000","last_became_peered":"2024-08-02T17:44:11.176374+0000","last_unstale":"2024-08-02T17:46:24.915973+0000","last_undegraded":"2024-08-02T17:46:24.915973+0000","last_fullsized":"2024-08-02T17:46:24.915973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:55:17.110056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'8","reported_seq":166,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:39.074848+0000","last_change":"2024-08-02T17:44:02.049579+0000","last_active":"2024-08-02T17:46:39.074848+0000","last_peered":"2024-08-02T17:46:39.074848+0000","last_clean":"2024-08-02T17:46:39.074848+0000","last_became_active":"2024-08-02T17:44:02.048794+0000","last_became_peered":"2024-08-02T17:44:02.048794+0000","last_unstale":"2024-08-02T17:46:39.074848+0000","last_undegraded":"2024-08-02T17:46:39.074848+0000","last_fullsized":"2024-08-02T17:46:39.074848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:01.031860+0000","last_clean_scrub_stamp":"2024-08-02T17:44:01.031860+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:13:21.750089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1543,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.942301+0000","last_change":"2024-08-02T17:44:04.129043+0000","last_active":"2024-08-02T17:46:24.942301+0000","last_peered":"2024-08-02T17:46:24.942301+0000","last_clean":"2024-08-02T17:46:24.942301+0000","last_became_active":"2024-08-02T17:44:04.125692+0000","last_became_peered":"2024-08-02T17:44:04.125692+0000","last_unstale":"2024-08-02T17:46:24.942301+0000","last_undegraded":"2024-08-02T17:46:24.942301+0000","last_fullsized":"2024-08-02T17:46:24.942301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:00:07.109944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.915638+0000","last_change":"2024-08-02T17:44:11.178355+0000","last_active":"2024-08-02T17:46:24.915638+0000","last_peered":"2024-08-02T17:46:24.915638+0000","last_clean":"2024-08-02T17:46:24.915638+0000","last_became_active":"2024-08-02T17:44:11.177838+0000","last_became_peered":"2024-08-02T17:44:11.177838+0000","last_unstale":"2024-08-02T17:46:24.915638+0000","last_undegraded":"2024-08-02T17:46:24.915638+0000","last_fullsized":"2024-08-02T17:46:24.915638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:10.144249+0000","last_clean_scrub_stamp":"2024-08-02T17:44:10.144249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:30:34.544552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.954294+0000","last_change":"2024-08-02T17:44:04.127326+0000","last_active":"2024-08-02T17:46:24.954294+0000","last_peered":"2024-08-02T17:46:24.954294+0000","last_clean":"2024-08-02T17:46:24.954294+0000","last_became_active":"2024-08-02T17:44:04.126861+0000","last_became_peered":"2024-08-02T17:44:04.126861+0000","last_unstale":"2024-08-02T17:46:24.954294+0000","last_undegraded":"2024-08-02T17:46:24.954294+0000","last_fullsized":"2024-08-02T17:46:24.954294+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:26:15.880283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"95'12","reported_seq":53,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953139+0000","last_change":"2024-08-02T17:45:58.024328+0000","last_active":"2024-08-02T17:46:24.953139+0000","last_peered":"2024-08-02T17:46:24.953139+0000","last_clean":"2024-08-02T17:46:24.953139+0000","last_became_active":"2024-08-02T17:45:58.023931+0000","last_became_peered":"2024-08-02T17:45:58.023931+0000","last_unstale":"2024-08-02T17:46:24.953139+0000","last_undegraded":"2024-08-02T17:46:24.953139+0000","last_fullsized":"2024-08-02T17:46:24.953139+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:52:10.635263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.946206+0000","last_change":"2024-08-02T17:44:04.133776+0000","last_active":"2024-08-02T17:46:24.946206+0000","last_peered":"2024-08-02T17:46:24.946206+0000","last_clean":"2024-08-02T17:46:24.946206+0000","last_became_active":"2024-08-02T17:44:04.133391+0000","last_became_peered":"2024-08-02T17:44:04.133391+0000","last_unstale":"2024-08-02T17:46:24.946206+0000","last_undegraded":"2024-08-02T17:46:24.946206+0000","last_fullsized":"2024-08-02T17:46:24.946206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:16:07.344993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.961330+0000","last_change":"2024-08-02T17:44:04.129677+0000","last_active":"2024-08-02T17:46:24.961330+0000","last_peered":"2024-08-02T17:46:24.961330+0000","last_clean":"2024-08-02T17:46:24.961330+0000","last_became_active":"2024-08-02T17:44:04.129409+0000","last_became_peered":"2024-08-02T17:44:04.129409+0000","last_unstale":"2024-08-02T17:46:24.961330+0000","last_undegraded":"2024-08-02T17:46:24.961330+0000","last_fullsized":"2024-08-02T17:46:24.961330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T19:23:33.367924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948673+0000","last_change":"2024-08-02T17:44:04.132228+0000","last_active":"2024-08-02T17:46:24.948673+0000","last_peered":"2024-08-02T17:46:24.948673+0000","last_clean":"2024-08-02T17:46:24.948673+0000","last_became_active":"2024-08-02T17:44:04.131577+0000","last_became_peered":"2024-08-02T17:44:04.131577+0000","last_unstale":"2024-08-02T17:46:24.948673+0000","last_undegraded":"2024-08-02T17:46:24.948673+0000","last_fullsized":"2024-08-02T17:46:24.948673+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:39:55.827022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.948711+0000","last_change":"2024-08-02T17:44:04.127323+0000","last_active":"2024-08-02T17:46:24.948711+0000","last_peered":"2024-08-02T17:46:24.948711+0000","last_clean":"2024-08-02T17:46:24.948711+0000","last_became_active":"2024-08-02T17:44:04.126598+0000","last_became_peered":"2024-08-02T17:44:04.126598+0000","last_unstale":"2024-08-02T17:46:24.948711+0000","last_undegraded":"2024-08-02T17:46:24.948711+0000","last_fullsized":"2024-08-02T17:46:24.948711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:42:11.371231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"95'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962508+0000","last_change":"2024-08-02T17:44:04.124485+0000","last_active":"2024-08-02T17:46:24.962508+0000","last_peered":"2024-08-02T17:46:24.962508+0000","last_clean":"2024-08-02T17:46:24.962508+0000","last_became_active":"2024-08-02T17:44:04.124119+0000","last_became_peered":"2024-08-02T17:44:04.124119+0000","last_unstale":"2024-08-02T17:46:24.962508+0000","last_undegraded":"2024-08-02T17:46:24.962508+0000","last_fullsized":"2024-08-02T17:46:24.962508+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:04:26.038126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"95'2","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949000+0000","last_change":"2024-08-02T17:44:57.788682+0000","last_active":"2024-08-02T17:46:24.949000+0000","last_peered":"2024-08-02T17:46:24.949000+0000","last_clean":"2024-08-02T17:46:24.949000+0000","last_became_active":"2024-08-02T17:44:57.788351+0000","last_became_peered":"2024-08-02T17:44:57.788351+0000","last_unstale":"2024-08-02T17:46:24.949000+0000","last_undegraded":"2024-08-02T17:46:24.949000+0000","last_fullsized":"2024-08-02T17:46:24.949000+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:29:31.557064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949651+0000","last_change":"2024-08-02T17:44:04.122624+0000","last_active":"2024-08-02T17:46:24.949651+0000","last_peered":"2024-08-02T17:46:24.949651+0000","last_clean":"2024-08-02T17:46:24.949651+0000","last_became_active":"2024-08-02T17:44:04.121030+0000","last_became_peered":"2024-08-02T17:44:04.121030+0000","last_unstale":"2024-08-02T17:46:24.949651+0000","last_undegraded":"2024-08-02T17:46:24.949651+0000","last_fullsized":"2024-08-02T17:46:24.949651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:28:47.743418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"95'8","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949256+0000","last_change":"2024-08-02T17:45:57.863502+0000","last_active":"2024-08-02T17:46:24.949256+0000","last_peered":"2024-08-02T17:46:24.949256+0000","last_clean":"2024-08-02T17:46:24.949256+0000","last_became_active":"2024-08-02T17:45:57.863230+0000","last_became_peered":"2024-08-02T17:45:57.863230+0000","last_unstale":"2024-08-02T17:46:24.949256+0000","last_undegraded":"2024-08-02T17:46:24.949256+0000","last_fullsized":"2024-08-02T17:46:24.949256+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:26:50.915451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962526+0000","last_change":"2024-08-02T17:44:04.131190+0000","last_active":"2024-08-02T17:46:24.962526+0000","last_peered":"2024-08-02T17:46:24.962526+0000","last_clean":"2024-08-02T17:46:24.962526+0000","last_became_active":"2024-08-02T17:44:04.130866+0000","last_became_peered":"2024-08-02T17:44:04.130866+0000","last_unstale":"2024-08-02T17:46:24.962526+0000","last_undegraded":"2024-08-02T17:46:24.962526+0000","last_fullsized":"2024-08-02T17:46:24.962526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:21:41.537262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958665+0000","last_change":"2024-08-02T17:44:04.126233+0000","last_active":"2024-08-02T17:46:24.958665+0000","last_peered":"2024-08-02T17:46:24.958665+0000","last_clean":"2024-08-02T17:46:24.958665+0000","last_became_active":"2024-08-02T17:44:04.125932+0000","last_became_peered":"2024-08-02T17:44:04.125932+0000","last_unstale":"2024-08-02T17:46:24.958665+0000","last_undegraded":"2024-08-02T17:46:24.958665+0000","last_fullsized":"2024-08-02T17:46:24.958665+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:07:50.710784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.947237+0000","last_change":"2024-08-02T17:44:04.133196+0000","last_active":"2024-08-02T17:46:24.947237+0000","last_peered":"2024-08-02T17:46:24.947237+0000","last_clean":"2024-08-02T17:46:24.947237+0000","last_became_active":"2024-08-02T17:44:04.132895+0000","last_became_peered":"2024-08-02T17:44:04.132895+0000","last_unstale":"2024-08-02T17:46:24.947237+0000","last_undegraded":"2024-08-02T17:46:24.947237+0000","last_fullsized":"2024-08-02T17:46:24.947237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:00:08.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.951444+0000","last_change":"2024-08-02T17:44:04.123403+0000","last_active":"2024-08-02T17:46:24.951444+0000","last_peered":"2024-08-02T17:46:24.951444+0000","last_clean":"2024-08-02T17:46:24.951444+0000","last_became_active":"2024-08-02T17:44:04.123061+0000","last_became_peered":"2024-08-02T17:44:04.123061+0000","last_unstale":"2024-08-02T17:46:24.951444+0000","last_undegraded":"2024-08-02T17:46:24.951444+0000","last_fullsized":"2024-08-02T17:46:24.951444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:57:10.839925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962281+0000","last_change":"2024-08-02T17:44:04.138184+0000","last_active":"2024-08-02T17:46:24.962281+0000","last_peered":"2024-08-02T17:46:24.962281+0000","last_clean":"2024-08-02T17:46:24.962281+0000","last_became_active":"2024-08-02T17:44:04.137895+0000","last_became_peered":"2024-08-02T17:44:04.137895+0000","last_unstale":"2024-08-02T17:46:24.962281+0000","last_undegraded":"2024-08-02T17:46:24.962281+0000","last_fullsized":"2024-08-02T17:46:24.962281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:30:13.891446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959483+0000","last_change":"2024-08-02T17:44:04.128294+0000","last_active":"2024-08-02T17:46:24.959483+0000","last_peered":"2024-08-02T17:46:24.959483+0000","last_clean":"2024-08-02T17:46:24.959483+0000","last_became_active":"2024-08-02T17:44:04.126721+0000","last_became_peered":"2024-08-02T17:44:04.126721+0000","last_unstale":"2024-08-02T17:46:24.959483+0000","last_undegraded":"2024-08-02T17:46:24.959483+0000","last_fullsized":"2024-08-02T17:46:24.959483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:01:08.235946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"95'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962369+0000","last_change":"2024-08-02T17:44:04.130359+0000","last_active":"2024-08-02T17:46:24.962369+0000","last_peered":"2024-08-02T17:46:24.962369+0000","last_clean":"2024-08-02T17:46:24.962369+0000","last_became_active":"2024-08-02T17:44:04.128540+0000","last_became_peered":"2024-08-02T17:44:04.128540+0000","last_unstale":"2024-08-02T17:46:24.962369+0000","last_undegraded":"2024-08-02T17:46:24.962369+0000","last_fullsized":"2024-08-02T17:46:24.962369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:37:27.639041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959980+0000","last_change":"2024-08-02T17:44:04.130453+0000","last_active":"2024-08-02T17:46:24.959980+0000","last_peered":"2024-08-02T17:46:24.959980+0000","last_clean":"2024-08-02T17:46:24.959980+0000","last_became_active":"2024-08-02T17:44:04.130149+0000","last_became_peered":"2024-08-02T17:44:04.130149+0000","last_unstale":"2024-08-02T17:46:24.959980+0000","last_undegraded":"2024-08-02T17:46:24.959980+0000","last_fullsized":"2024-08-02T17:46:24.959980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T22:44:57.287700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956497+0000","last_change":"2024-08-02T17:44:04.131313+0000","last_active":"2024-08-02T17:46:24.956497+0000","last_peered":"2024-08-02T17:46:24.956497+0000","last_clean":"2024-08-02T17:46:24.956497+0000","last_became_active":"2024-08-02T17:44:04.131037+0000","last_became_peered":"2024-08-02T17:44:04.131037+0000","last_unstale":"2024-08-02T17:46:24.956497+0000","last_undegraded":"2024-08-02T17:46:24.956497+0000","last_fullsized":"2024-08-02T17:46:24.956497+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:16:07.386369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"95'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952727+0000","last_change":"2024-08-02T17:44:04.124271+0000","last_active":"2024-08-02T17:46:24.952727+0000","last_peered":"2024-08-02T17:46:24.952727+0000","last_clean":"2024-08-02T17:46:24.952727+0000","last_became_active":"2024-08-02T17:44:04.123114+0000","last_became_peered":"2024-08-02T17:44:04.123114+0000","last_unstale":"2024-08-02T17:46:24.952727+0000","last_undegraded":"2024-08-02T17:46:24.952727+0000","last_fullsized":"2024-08-02T17:46:24.952727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T20:45:37.800439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"95'7","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.952746+0000","last_change":"2024-08-02T17:44:04.131987+0000","last_active":"2024-08-02T17:46:24.952746+0000","last_peered":"2024-08-02T17:46:24.952746+0000","last_clean":"2024-08-02T17:46:24.952746+0000","last_became_active":"2024-08-02T17:44:04.131633+0000","last_became_peered":"2024-08-02T17:44:04.131633+0000","last_unstale":"2024-08-02T17:46:24.952746+0000","last_undegraded":"2024-08-02T17:46:24.952746+0000","last_fullsized":"2024-08-02T17:46:24.952746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:39:29.605120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"95'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.960232+0000","last_change":"2024-08-02T17:44:04.134790+0000","last_active":"2024-08-02T17:46:24.960232+0000","last_peered":"2024-08-02T17:46:24.960232+0000","last_clean":"2024-08-02T17:46:24.960232+0000","last_became_active":"2024-08-02T17:44:04.134432+0000","last_became_peered":"2024-08-02T17:44:04.134432+0000","last_unstale":"2024-08-02T17:46:24.960232+0000","last_undegraded":"2024-08-02T17:46:24.960232+0000","last_fullsized":"2024-08-02T17:46:24.960232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T00:29:12.959666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"95'16","reported_seq":58,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956382+0000","last_change":"2024-08-02T17:44:57.786339+0000","last_active":"2024-08-02T17:46:24.956382+0000","last_peered":"2024-08-02T17:46:24.956382+0000","last_clean":"2024-08-02T17:46:24.956382+0000","last_became_active":"2024-08-02T17:44:57.785961+0000","last_became_peered":"2024-08-02T17:44:57.785961+0000","last_unstale":"2024-08-02T17:46:24.956382+0000","last_undegraded":"2024-08-02T17:46:24.956382+0000","last_fullsized":"2024-08-02T17:46:24.956382+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:29:13.221686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"95'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.962266+0000","last_change":"2024-08-02T17:44:04.120504+0000","last_active":"2024-08-02T17:46:24.962266+0000","last_peered":"2024-08-02T17:46:24.962266+0000","last_clean":"2024-08-02T17:46:24.962266+0000","last_became_active":"2024-08-02T17:44:04.120172+0000","last_became_peered":"2024-08-02T17:44:04.120172+0000","last_unstale":"2024-08-02T17:46:24.962266+0000","last_undegraded":"2024-08-02T17:46:24.962266+0000","last_fullsized":"2024-08-02T17:46:24.962266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T03:15:11.133519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"95'9","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.958299+0000","last_change":"2024-08-02T17:44:04.120260+0000","last_active":"2024-08-02T17:46:24.958299+0000","last_peered":"2024-08-02T17:46:24.958299+0000","last_clean":"2024-08-02T17:46:24.958299+0000","last_became_active":"2024-08-02T17:44:04.119932+0000","last_became_peered":"2024-08-02T17:44:04.119932+0000","last_unstale":"2024-08-02T17:46:24.958299+0000","last_undegraded":"2024-08-02T17:46:24.958299+0000","last_fullsized":"2024-08-02T17:46:24.958299+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T05:24:17.930590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"95'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.950405+0000","last_change":"2024-08-02T17:44:04.130779+0000","last_active":"2024-08-02T17:46:24.950405+0000","last_peered":"2024-08-02T17:46:24.950405+0000","last_clean":"2024-08-02T17:46:24.950405+0000","last_became_active":"2024-08-02T17:44:04.130294+0000","last_became_peered":"2024-08-02T17:44:04.130294+0000","last_unstale":"2024-08-02T17:46:24.950405+0000","last_undegraded":"2024-08-02T17:46:24.950405+0000","last_fullsized":"2024-08-02T17:46:24.950405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:36:13.570771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"95'6","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.956086+0000","last_change":"2024-08-02T17:44:04.128114+0000","last_active":"2024-08-02T17:46:24.956086+0000","last_peered":"2024-08-02T17:46:24.956086+0000","last_clean":"2024-08-02T17:46:24.956086+0000","last_became_active":"2024-08-02T17:44:04.127582+0000","last_became_peered":"2024-08-02T17:44:04.127582+0000","last_unstale":"2024-08-02T17:46:24.956086+0000","last_undegraded":"2024-08-02T17:46:24.956086+0000","last_fullsized":"2024-08-02T17:46:24.956086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T21:36:53.921253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"95'4","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.949321+0000","last_change":"2024-08-02T17:44:04.133426+0000","last_active":"2024-08-02T17:46:24.949321+0000","last_peered":"2024-08-02T17:46:24.949321+0000","last_clean":"2024-08-02T17:46:24.949321+0000","last_became_active":"2024-08-02T17:44:04.133097+0000","last_became_peered":"2024-08-02T17:44:04.133097+0000","last_unstale":"2024-08-02T17:46:24.949321+0000","last_undegraded":"2024-08-02T17:46:24.949321+0000","last_fullsized":"2024-08-02T17:46:24.949321+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T23:03:31.211995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"95'8","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.955968+0000","last_change":"2024-08-02T17:44:04.134043+0000","last_active":"2024-08-02T17:46:24.955968+0000","last_peered":"2024-08-02T17:46:24.955968+0000","last_clean":"2024-08-02T17:46:24.955968+0000","last_became_active":"2024-08-02T17:44:04.133692+0000","last_became_peered":"2024-08-02T17:44:04.133692+0000","last_unstale":"2024-08-02T17:46:24.955968+0000","last_undegraded":"2024-08-02T17:46:24.955968+0000","last_fullsized":"2024-08-02T17:46:24.955968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T01:38:56.064298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"95'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.953754+0000","last_change":"2024-08-02T17:44:04.127805+0000","last_active":"2024-08-02T17:46:24.953754+0000","last_peered":"2024-08-02T17:46:24.953754+0000","last_clean":"2024-08-02T17:46:24.953754+0000","last_became_active":"2024-08-02T17:44:04.127103+0000","last_became_peered":"2024-08-02T17:44:04.127103+0000","last_unstale":"2024-08-02T17:46:24.953754+0000","last_undegraded":"2024-08-02T17:46:24.953754+0000","last_fullsized":"2024-08-02T17:46:24.953754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T18:21:08.935023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"95'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.957053+0000","last_change":"2024-08-02T17:44:04.133184+0000","last_active":"2024-08-02T17:46:24.957053+0000","last_peered":"2024-08-02T17:46:24.957053+0000","last_clean":"2024-08-02T17:46:24.957053+0000","last_became_active":"2024-08-02T17:44:04.132265+0000","last_became_peered":"2024-08-02T17:44:04.132265+0000","last_unstale":"2024-08-02T17:46:24.957053+0000","last_undegraded":"2024-08-02T17:46:24.957053+0000","last_fullsized":"2024-08-02T17:46:24.957053+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-03T17:48:03.880913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"95'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-08-02T17:46:24.959352+0000","last_change":"2024-08-02T17:44:04.127985+0000","last_active":"2024-08-02T17:46:24.959352+0000","last_peered":"2024-08-02T17:46:24.959352+0000","last_clean":"2024-08-02T17:46:24.959352+0000","last_became_active":"2024-08-02T17:44:04.126253+0000","last_became_peered":"2024-08-02T17:44:04.126253+0000","last_unstale":"2024-08-02T17:46:24.959352+0000","last_undegraded":"2024-08-02T17:46:24.959352+0000","last_fullsized":"2024-08-02T17:46:24.959352+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-02T17:44:03.102517+0000","last_clean_scrub_stamp":"2024-08-02T17:44:03.102517+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-04T04:56:16.982878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":800612,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1065,"num_read_kb":753,"num_write":432,"num_write_kb":2367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":112,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2401836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449132,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1486848,"data_stored":937256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678064,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37532,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707620,"statfs":{"total":95995035648,"available":95956602880,"internally_reserved":0,"allocated":2125824,"data_stored":1534173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841590,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32648,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712504,"statfs":{"total":95995035648,"available":95961604096,"internally_reserved":0,"allocated":1507328,"data_stored":941588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103225,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36428,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708724,"statfs":{"total":95995035648,"available":95957733376,"internally_reserved":0,"allocated":1114112,"data_stored":548292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364862,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37000,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708152,"statfs":{"total":95995035648,"available":95957147648,"internally_reserved":0,"allocated":1703936,"data_stored":1142644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528386,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1089536,"data_stored":539452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790023,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37448,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707704,"statfs":{"total":95995035648,"available":95956688896,"internally_reserved":0,"allocated":2039808,"data_stored":1460359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1110016,"data_stored":546223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215183,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28920,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716232,"statfs":{"total":95995035648,"available":95965421568,"internally_reserved":0,"allocated":1826816,"data_stored":1261094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1183744,"data_stored":614145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":611986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":540840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":72624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":74783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":330629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":404443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-02T17:46:41.857 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:46:42.110 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:42.232 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:42 smithi069 ceph-mon[32746]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:42.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:42 smithi077 ceph-mon[37108]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:42.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:42 smithi083 ceph-mon[37031]: from='client.15420 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-02T17:46:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 -- 172.21.15.69:0/929933937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c84102250 msgr2=0x7f1c8410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 --2- 172.21.15.69:0/929933937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c84102250 0x7f1c8410e780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c009cd0 tx=0x7f1c7c02a8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 -- 172.21.15.69:0/929933937 shutdown_connections 2024-08-02T17:46:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 --2- 172.21.15.69:0/929933937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c84102250 0x7f1c8410e780 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 --2- 172.21.15.69:0/929933937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c841018b0 0x7f1c84101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 --2- 172.21.15.69:0/929933937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c841078b0 0x7f1c84107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 -- 172.21.15.69:0/929933937 >> 172.21.15.69:0/929933937 conn(0x7f1c840fd5f0 msgr2=0x7f1c840ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:42.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 -- 172.21.15.69:0/929933937 shutdown_connections 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 -- 172.21.15.69:0/929933937 wait complete. 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.597+0000 7f1c8984e640 1 Processor -- start 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c8984e640 1 -- start start 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c8984e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c841018b0 0x7f1c84198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c8984e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84102250 0x7f1c84198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c8984e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c841078b0 0x7f1c8419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c8984e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84111cb0 con 0x7f1c841018b0 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c8984e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84111b30 con 0x7f1c84102250 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c8984e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84111e30 con 0x7f1c841078b0 2024-08-02T17:46:42.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c841018b0 0x7f1c84198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c841018b0 0x7f1c84198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38838/0 (socket says 172.21.15.69:38838) 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 -- 172.21.15.69:0/663473925 learned_addr learned my addr 172.21.15.69:0/663473925 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c827fc640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84102250 0x7f1c84198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c837fe640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c841078b0 0x7f1c8419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 -- 172.21.15.69:0/663473925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c841078b0 msgr2=0x7f1c8419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c841078b0 0x7f1c8419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 -- 172.21.15.69:0/663473925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84102250 msgr2=0x7f1c84198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84102250 0x7f1c84198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 -- 172.21.15.69:0/663473925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8419d100 con 0x7f1c841018b0 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.598+0000 7f1c82ffd640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c841018b0 0x7f1c84198150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400e9f0 tx=0x7f1c7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.599+0000 7f1c8884c640 1 -- 172.21.15.69:0/663473925 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7400cd80 con 0x7f1c841018b0 2024-08-02T17:46:42.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.599+0000 7f1c8884c640 1 -- 172.21.15.69:0/663473925 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c74005500 con 0x7f1c841018b0 2024-08-02T17:46:42.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.599+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c8419d3f0 con 0x7f1c841018b0 2024-08-02T17:46:42.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.599+0000 7f1c8884c640 1 -- 172.21.15.69:0/663473925 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c74010690 con 0x7f1c841018b0 2024-08-02T17:46:42.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.599+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c841a4cd0 con 0x7f1c841018b0 2024-08-02T17:46:42.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.600+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c84101d10 con 0x7f1c841018b0 2024-08-02T17:46:42.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.601+0000 7f1c8884c640 1 -- 172.21.15.69:0/663473925 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f1c740046c0 con 0x7f1c841018b0 2024-08-02T17:46:42.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.601+0000 7f1c8884c640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1c6c0a2e90 0x7f1c6c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:42.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.602+0000 7f1c827fc640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1c6c0a2e90 0x7f1c6c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:42.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.602+0000 7f1c8884c640 1 -- 172.21.15.69:0/663473925 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f1c740cfe10 con 0x7f1c841018b0 2024-08-02T17:46:42.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.602+0000 7f1c827fc640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1c6c0a2e90 0x7f1c6c0a5350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c84106e60 tx=0x7f1c78005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:42.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.606+0000 7f1c8884c640 1 -- 172.21.15.69:0/663473925 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f1c7409c420 con 0x7f1c841018b0 2024-08-02T17:46:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.804+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c841994b0 con 0x7f1c841018b0 2024-08-02T17:46:42.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.805+0000 7f1c8884c640 1 -- 172.21.15.69:0/663473925 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f1c7409bd20 con 0x7f1c841018b0 2024-08-02T17:46:42.807 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:42.807 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:46:42.807 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:46:42.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1c6c0a2e90 msgr2=0x7f1c6c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1c6c0a2e90 0x7f1c6c0a5350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c84106e60 tx=0x7f1c78005f90 comp rx=0 tx=0).stop 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c841018b0 msgr2=0x7f1c84198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c841018b0 0x7f1c84198150 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400e9f0 tx=0x7f1c7400edf0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 shutdown_connections 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1c6c0a2e90 0x7f1c6c0a5350 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1c841078b0 0x7f1c8419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1c84102250 0x7f1c84198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 --2- 172.21.15.69:0/663473925 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1c841018b0 0x7f1c84198150 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 >> 172.21.15.69:0/663473925 conn(0x7f1c840fd5f0 msgr2=0x7f1c840fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 shutdown_connections 2024-08-02T17:46:42.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:42.807+0000 7f1c8984e640 1 -- 172.21.15.69:0/663473925 wait complete. 2024-08-02T17:46:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:43 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/663473925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:43 smithi069 ceph-mon[32746]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-02T17:46:43.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:43 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/663473925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:43.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:43 smithi077 ceph-mon[37108]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-02T17:46:43.458 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:46:43.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:43 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/663473925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:43.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:43 smithi083 ceph-mon[37031]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-02T17:46:43.710 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:44.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 -- 172.21.15.69:0/454187239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 msgr2=0x7f8438106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:44.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 --2- 172.21.15.69:0/454187239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 0x7f8438106b90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8428009930 tx=0x7f842802a420 comp rx=0 tx=0).stop 2024-08-02T17:46:44.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 -- 172.21.15.69:0/454187239 shutdown_connections 2024-08-02T17:46:44.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 --2- 172.21.15.69:0/454187239 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8438101150 0x7f843810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 --2- 172.21.15.69:0/454187239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f84381007b0 0x7f8438100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 --2- 172.21.15.69:0/454187239 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 0x7f8438106b90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 -- 172.21.15.69:0/454187239 >> 172.21.15.69:0/454187239 conn(0x7f84380fc4d0 msgr2=0x7f84380fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 -- 172.21.15.69:0/454187239 shutdown_connections 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 -- 172.21.15.69:0/454187239 wait complete. 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.164+0000 7f843eeb9640 1 Processor -- start 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843eeb9640 1 -- start start 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843eeb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84381007b0 0x7f84381980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843eeb9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101150 0x7f8438198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843eeb9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 0x7f843819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843eeb9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8438111cd0 con 0x7f84381067b0 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843eeb9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8438111b50 con 0x7f84381007b0 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843eeb9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8438111e50 con 0x7f8438101150 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 0x7f843819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:44.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 0x7f843819c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38846/0 (socket says 172.21.15.69:38846) 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843cc55640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84381007b0 0x7f84381980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 -- 172.21.15.69:0/1181667743 learned_addr learned my addr 172.21.15.69:0/1181667743 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f8437fff640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101150 0x7f8438198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 -- 172.21.15.69:0/1181667743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101150 msgr2=0x7f8438198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101150 0x7f8438198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 -- 172.21.15.69:0/1181667743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84381007b0 msgr2=0x7f84381980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84381007b0 0x7f84381980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 -- 172.21.15.69:0/1181667743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843819d0a0 con 0x7f84381067b0 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f8437fff640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101150 0x7f8438198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843d456640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 0x7f843819c9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f843000eaa0 tx=0x7f843000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f8435ffb640 1 -- 172.21.15.69:0/1181667743 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843000cdb0 con 0x7f84381067b0 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843819d390 con 0x7f84381067b0 2024-08-02T17:46:44.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f8435ffb640 1 -- 172.21.15.69:0/1181667743 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8430005020 con 0x7f84381067b0 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.165+0000 7f8435ffb640 1 -- 172.21.15.69:0/1181667743 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84300116b0 con 0x7f84381067b0 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.166+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381a4c70 con 0x7f84381067b0 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.167+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8438100c10 con 0x7f84381067b0 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.168+0000 7f8435ffb640 1 -- 172.21.15.69:0/1181667743 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f8430005560 con 0x7f84381067b0 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.169+0000 7f8435ffb640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84240a2e10 0x7f84240a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.169+0000 7f843cc55640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84240a2e10 0x7f84240a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.169+0000 7f8435ffb640 1 -- 172.21.15.69:0/1181667743 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f84300cfcb0 con 0x7f84381067b0 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.169+0000 7f843cc55640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84240a2e10 0x7f84240a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8428009900 tx=0x7f84280058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:44.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.173+0000 7f8435ffb640 1 -- 172.21.15.69:0/1181667743 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8430016030 con 0x7f84381067b0 2024-08-02T17:46:44.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.373+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84381a4f10 con 0x7f84381067b0 2024-08-02T17:46:44.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.373+0000 7f8435ffb640 1 -- 172.21.15.69:0/1181667743 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f843009c2c0 con 0x7f84381067b0 2024-08-02T17:46:44.375 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:44.375 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:46:44.376 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:46:44.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.375+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84240a2e10 msgr2=0x7f84240a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:44.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.375+0000 7f843eeb9640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84240a2e10 0x7f84240a52d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8428009900 tx=0x7f84280058d0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.375+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 msgr2=0x7f843819c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:44.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.375+0000 7f843eeb9640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 0x7f843819c9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f843000eaa0 tx=0x7f843000ef60 comp rx=0 tx=0).stop 2024-08-02T17:46:44.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.376+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 shutdown_connections 2024-08-02T17:46:44.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.376+0000 7f843eeb9640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f84381067b0 0x7f843819c9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.376+0000 7f843eeb9640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f84240a2e10 0x7f84240a52d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.376+0000 7f843eeb9640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8438101150 0x7f8438198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.376+0000 7f843eeb9640 1 --2- 172.21.15.69:0/1181667743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f84381007b0 0x7f84381980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:44.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.376+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 >> 172.21.15.69:0/1181667743 conn(0x7f84380fc4d0 msgr2=0x7f84380fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:44.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.376+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 shutdown_connections 2024-08-02T17:46:44.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:44.376+0000 7f843eeb9640 1 -- 172.21.15.69:0/1181667743 wait complete. 2024-08-02T17:46:44.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:44 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1181667743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:44.825 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:44 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1181667743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:44.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:44 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1181667743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:45.142 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T17:46:45.392 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:45.767 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:45 smithi069 ceph-mon[32746]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-02T17:46:45.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:45 smithi083 ceph-mon[37031]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-02T17:46:45.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:45 smithi077 ceph-mon[37108]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-08-02T17:46:45.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 -- 172.21.15.69:0/912971041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 msgr2=0x7fb164107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:45.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 --2- 172.21.15.69:0/912971041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 0x7fb164107c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb154009930 tx=0x7fb15402a420 comp rx=0 tx=0).stop 2024-08-02T17:46:45.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 -- 172.21.15.69:0/912971041 shutdown_connections 2024-08-02T17:46:45.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 --2- 172.21.15.69:0/912971041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb164102250 0x7fb16410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:45.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 --2- 172.21.15.69:0/912971041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1641018b0 0x7fb164101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:45.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 --2- 172.21.15.69:0/912971041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 0x7fb164107c90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:45.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 -- 172.21.15.69:0/912971041 >> 172.21.15.69:0/912971041 conn(0x7fb1640fd5f0 msgr2=0x7fb1640ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:45.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 -- 172.21.15.69:0/912971041 shutdown_connections 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 -- 172.21.15.69:0/912971041 wait complete. 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 Processor -- start 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.845+0000 7fb16af75640 1 -- start start 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb16af75640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1641018b0 0x7fb164198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb16af75640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb164102250 0x7fb164198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb16af75640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 0x7fb16419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb16af75640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb164111cd0 con 0x7fb1641078b0 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb16af75640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb164111b50 con 0x7fb1641018b0 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb16af75640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb164111e50 con 0x7fb164102250 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 0x7fb16419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 0x7fb16419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49536/0 (socket says 172.21.15.69:49536) 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 -- 172.21.15.69:0/120061434 learned_addr learned my addr 172.21.15.69:0/120061434 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb168d11640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1641018b0 0x7fb164198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:45.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 -- 172.21.15.69:0/120061434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb164102250 msgr2=0x7fb164198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb164102250 0x7fb164198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 -- 172.21.15.69:0/120061434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1641018b0 msgr2=0x7fb164198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1641018b0 0x7fb164198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 -- 172.21.15.69:0/120061434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16419d1c0 con 0x7fb1641078b0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb168d11640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1641018b0 0x7fb164198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb169512640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 0x7fb16419cae0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b970 tx=0x7fb15c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb161ffb640 1 -- 172.21.15.69:0/120061434 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c019070 con 0x7fb1641078b0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16419d4b0 con 0x7fb1641078b0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb161ffb640 1 -- 172.21.15.69:0/120061434 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c00cb40 con 0x7fb1641078b0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.846+0000 7fb161ffb640 1 -- 172.21.15.69:0/120061434 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c007610 con 0x7fb1641078b0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.847+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1641a4d90 con 0x7fb1641078b0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.848+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb164101d10 con 0x7fb1641078b0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.849+0000 7fb161ffb640 1 -- 172.21.15.69:0/120061434 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fb15c01c070 con 0x7fb1641078b0 2024-08-02T17:46:45.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.849+0000 7fb161ffb640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1500a2e10 0x7fb1500a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.849+0000 7fb161ffb640 1 -- 172.21.15.69:0/120061434 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb15c0ceff0 con 0x7fb1641078b0 2024-08-02T17:46:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.849+0000 7fb168d11640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1500a2e10 0x7fb1500a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.850+0000 7fb168d11640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1500a2e10 0x7fb1500a52d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb154000c00 tx=0x7fb154005a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:45.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:45.853+0000 7fb161ffb640 1 -- 172.21.15.69:0/120061434 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fb15c09b710 con 0x7fb1641078b0 2024-08-02T17:46:46.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.022+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb164109890 con 0x7fb1641078b0 2024-08-02T17:46:46.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.024+0000 7fb161ffb640 1 -- 172.21.15.69:0/120061434 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25069 (secure 0 0 0) 0x7fb15c09b010 con 0x7fb1641078b0 2024-08-02T17:46:46.025 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:46.026 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":95,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T17:46:24.900245+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3271476957":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3594391539":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3033851323":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/2680424375":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/452236630":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4036393791":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6800/452236630":"2024-08-03T17:46:24.900194+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T17:46:46.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1500a2e10 msgr2=0x7fb1500a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:46.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1500a2e10 0x7fb1500a52d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb154000c00 tx=0x7fb154005a20 comp rx=0 tx=0).stop 2024-08-02T17:46:46.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 msgr2=0x7fb16419cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 0x7fb16419cae0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b970 tx=0x7fb15c00be30 comp rx=0 tx=0).stop 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 shutdown_connections 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb1641078b0 0x7fb16419cae0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1500a2e10 0x7fb1500a52d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb164102250 0x7fb164198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 --2- 172.21.15.69:0/120061434 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb1641018b0 0x7fb164198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 >> 172.21.15.69:0/120061434 conn(0x7fb1640fd5f0 msgr2=0x7fb1640fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 shutdown_connections 2024-08-02T17:46:46.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:46.027+0000 7fb16af75640 1 -- 172.21.15.69:0/120061434 wait complete. 2024-08-02T17:46:46.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:46 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/120061434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:46.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-08-02T17:46:46.733 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:46 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/120061434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:46.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:46 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/120061434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T17:46:46.930 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:47.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:47 smithi069 ceph-mon[32746]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-08-02T17:46:47.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:47 smithi083 ceph-mon[37031]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-08-02T17:46:47.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:47 smithi077 ceph-mon[37108]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-08-02T17:46:48.064 INFO:teuthology.orchestra.run.smithi069.stdout:{"run_dir":"/var/run/ceph"} 2024-08-02T17:46:48.064 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-08-02T17:46:48.065 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 4941 2024-08-02T17:46:48.065 DEBUG:tasks.quiescer:Yielding 2024-08-02T17:46:48.065 INFO:teuthology.run_tasks:Running task exec... 2024-08-02T17:46:48.075 INFO:teuthology.task.exec:Executing custom commands... 2024-08-02T17:46:48.075 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T17:46:48.075 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-08-02T17:46:48.076 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:46:48.112 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:48.127 INFO:teuthology.orchestra.run.smithi069.stdout:Filesystem Size Used Avail Use% Mounted on 2024-08-02T17:46:48.128 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:3300,172.21.15.77:3300,172.21.15.83:3300:/volumes/qa/sv_1/39fc5226-68dc-46b2-9a4d-e93437ee40a4 1.1T 396M 1.1T 1% /tmp/cephfs 2024-08-02T17:46:48.130 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-08-02T17:46:48.222 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-08-02T17:46:48.261 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-08-02T17:46:48.351 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3894290881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88107fd0 msgr2=0x7f8c8810e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3894290881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88107fd0 0x7f8c8810e970 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009cd0 tx=0x7f8c8002a910 comp rx=0 tx=0).stop 2024-08-02T17:46:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3894290881 shutdown_connections 2024-08-02T17:46:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3894290881 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88107fd0 0x7f8c8810e970 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3894290881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88103410 0x7f8c881078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3894290881 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c88102a60 0x7f8c88102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:48.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3894290881 >> 172.21.15.69:0/3894290881 conn(0x7f8c880fc500 msgr2=0x7f8c880fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3894290881 shutdown_connections 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.807+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3894290881 wait complete. 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.808+0000 7f8c8dd03640 1 Processor -- start 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.808+0000 7f8c8dd03640 1 -- start start 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.808+0000 7f8c8dd03640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c88102a60 0x7f8c88198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.808+0000 7f8c8dd03640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88103410 0x7f8c88198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.808+0000 7f8c8dd03640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88107fd0 0x7f8c8819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.808+0000 7f8c8dd03640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88111e10 con 0x7f8c88103410 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.808+0000 7f8c8dd03640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88111c90 con 0x7f8c88102a60 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.808+0000 7f8c8dd03640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88111f90 con 0x7f8c88107fd0 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88103410 0x7f8c88198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88103410 0x7f8c88198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49554/0 (socket says 172.21.15.69:49554) 2024-08-02T17:46:48.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 -- 172.21.15.69:0/3419586041 learned_addr learned my addr 172.21.15.69:0/3419586041 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c87fff640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88107fd0 0x7f8c8819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c877fe640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c88102a60 0x7f8c88198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 -- 172.21.15.69:0/3419586041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88107fd0 msgr2=0x7f8c8819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88107fd0 0x7f8c8819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 -- 172.21.15.69:0/3419586041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c88102a60 msgr2=0x7f8c88198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c88102a60 0x7f8c88198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 -- 172.21.15.69:0/3419586041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8819d300 con 0x7f8c88103410 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c86ffd640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88103410 0x7f8c88198890 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c006fd0 tx=0x7f8c7c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c84ff9640 1 -- 172.21.15.69:0/3419586041 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c011830 con 0x7f8c88103410 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8819d5f0 con 0x7f8c88103410 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.809+0000 7f8c84ff9640 1 -- 172.21.15.69:0/3419586041 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c7c016070 con 0x7f8c88103410 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.810+0000 7f8c84ff9640 1 -- 172.21.15.69:0/3419586041 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c00f2e0 con 0x7f8c88103410 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.810+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c881a4ed0 con 0x7f8c88103410 2024-08-02T17:46:48.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.811+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c88102e40 con 0x7f8c88103410 2024-08-02T17:46:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.812+0000 7f8c84ff9640 1 -- 172.21.15.69:0/3419586041 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f8c7c0119d0 con 0x7f8c88103410 2024-08-02T17:46:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.813+0000 7f8c84ff9640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c700a2e10 0x7f8c700a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.813+0000 7f8c877fe640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c700a2e10 0x7f8c700a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.813+0000 7f8c84ff9640 1 -- 172.21.15.69:0/3419586041 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8c7c0cf630 con 0x7f8c88103410 2024-08-02T17:46:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.813+0000 7f8c877fe640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c700a2e10 0x7f8c700a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800bbe0 tx=0x7f8c7800b390 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:48.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:48.819+0000 7f8c84ff9640 1 -- 172.21.15.69:0/3419586041 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8c7c09bd50 con 0x7f8c88103410 2024-08-02T17:46:49.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.033+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c88109fe0 con 0x7f8c88103410 2024-08-02T17:46:49.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.033+0000 7f8c84ff9640 1 -- 172.21.15.69:0/3419586041 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f8c7c09b650 con 0x7f8c88103410 2024-08-02T17:46:49.035 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:49.035 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:46:49.036 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c700a2e10 msgr2=0x7f8c700a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c700a2e10 0x7f8c700a52d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800bbe0 tx=0x7f8c7800b390 comp rx=0 tx=0).stop 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88103410 msgr2=0x7f8c88198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88103410 0x7f8c88198890 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c006fd0 tx=0x7f8c7c00b8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 shutdown_connections 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c88103410 0x7f8c88198890 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c700a2e10 0x7f8c700a52d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c88107fd0 0x7f8c8819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 --2- 172.21.15.69:0/3419586041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c88102a60 0x7f8c88198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:49.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 >> 172.21.15.69:0/3419586041 conn(0x7f8c880fc500 msgr2=0x7f8c881060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:49.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 shutdown_connections 2024-08-02T17:46:49.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:49.036+0000 7f8c8dd03640 1 -- 172.21.15.69:0/3419586041 wait complete. 2024-08-02T17:46:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:49 smithi083 ceph-mon[37031]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-02T17:46:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:49 smithi077 ceph-mon[37108]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-02T17:46:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:49 smithi069 ceph-mon[32746]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-08-02T17:46:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 -- 172.21.15.69:0/981850987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec30101070 msgr2=0x7fec3010d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 --2- 172.21.15.69:0/981850987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec30101070 0x7fec3010d5a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fec28009cd0 tx=0x7fec2802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 -- 172.21.15.69:0/981850987 shutdown_connections 2024-08-02T17:46:50.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 --2- 172.21.15.69:0/981850987 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec30101070 0x7fec3010d5a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 --2- 172.21.15.69:0/981850987 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec301006d0 0x7fec30100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 --2- 172.21.15.69:0/981850987 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec301066e0 0x7fec30106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 -- 172.21.15.69:0/981850987 >> 172.21.15.69:0/981850987 conn(0x7fec300776d0 msgr2=0x7fec300fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 -- 172.21.15.69:0/981850987 shutdown_connections 2024-08-02T17:46:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 -- 172.21.15.69:0/981850987 wait complete. 2024-08-02T17:46:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 Processor -- start 2024-08-02T17:46:50.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:49.999+0000 7fec37287640 1 -- start start 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec37287640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301006d0 0x7fec30198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec37287640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec30101070 0x7fec301985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec37287640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec301066e0 0x7fec3019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec37287640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec30110b20 con 0x7fec301006d0 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec37287640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec301109a0 con 0x7fec301066e0 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec37287640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec30110ca0 con 0x7fec30101070 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301006d0 0x7fec30198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301006d0 0x7fec30198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49568/0 (socket says 172.21.15.69:49568) 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 -- 172.21.15.69:0/1291968632 learned_addr learned my addr 172.21.15.69:0/1291968632 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec34822640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec30101070 0x7fec301985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35824640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec301066e0 0x7fec3019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 -- 172.21.15.69:0/1291968632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec30101070 msgr2=0x7fec301985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec30101070 0x7fec301985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 -- 172.21.15.69:0/1291968632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec301066e0 msgr2=0x7fec3019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec301066e0 0x7fec3019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 -- 172.21.15.69:0/1291968632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3019d060 con 0x7fec301006d0 2024-08-02T17:46:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec35023640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301006d0 0x7fec30198070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00e9f0 tx=0x7fec2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec267fc640 1 -- 172.21.15.69:0/1291968632 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00ccc0 con 0x7fec301006d0 2024-08-02T17:46:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec267fc640 1 -- 172.21.15.69:0/1291968632 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec2c00ce60 con 0x7fec301006d0 2024-08-02T17:46:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec37287640 1 -- 172.21.15.69:0/1291968632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3019d350 con 0x7fec301006d0 2024-08-02T17:46:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.000+0000 7fec267fc640 1 -- 172.21.15.69:0/1291968632 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c010690 con 0x7fec301006d0 2024-08-02T17:46:50.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.001+0000 7fec37287640 1 -- 172.21.15.69:0/1291968632 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec301a4be0 con 0x7fec301006d0 2024-08-02T17:46:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec267fc640 1 -- 172.21.15.69:0/1291968632 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fec2c0046c0 con 0x7fec301006d0 2024-08-02T17:46:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec39fd9740 1 Processor -- start 2024-08-02T17:46:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec39fd9740 1 -- start start 2024-08-02T17:46:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01e6d0 0x563c3f01eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01eff0 0x563c3f02d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f030170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec39fd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563c3ef829b0 con 0x563c3f02dd80 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec39fd9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c3ef2a800 con 0x563c3f01eff0 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.003+0000 7fec39fd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c3ef7e970 con 0x563c3f01e6d0 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f030170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec267fc640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec1c0a2e10 0x7fec1c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f030170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49570/0 (socket says 172.21.15.69:49570) 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 -- 172.21.15.69:0/3517261542 learned_addr learned my addr 172.21.15.69:0/3517261542 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec35824640 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01e6d0 0x563c3f01eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec35023640 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01eff0 0x563c3f02d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec35824640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec1c0a2e10 0x7fec1c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec267fc640 1 -- 172.21.15.69:0/1291968632 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fec2c0cfec0 con 0x7fec301006d0 2024-08-02T17:46:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 -- 172.21.15.69:0/3517261542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01e6d0 msgr2=0x563c3f01eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01e6d0 0x563c3f01eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 -- 172.21.15.69:0/3517261542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01eff0 msgr2=0x563c3f02d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01eff0 0x563c3f02d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 -- 172.21.15.69:0/3517261542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3ec96210 con 0x563c3f02dd80 2024-08-02T17:46:50.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec34822640 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f030170 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fec2000aca0 tx=0x7fec2000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58422d576090fc7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec1b7fe640 1 -- 172.21.15.69:0/3517261542 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2001d070 con 0x563c3f02dd80 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec35824640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec1c0a2e10 0x7fec1c0a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec28005cc0 tx=0x7fec2802a8e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec1b7fe640 1 -- 172.21.15.69:0/3517261542 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec20018b20 con 0x563c3f02dd80 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.004+0000 7fec1b7fe640 1 -- 172.21.15.69:0/3517261542 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20010bb0 con 0x563c3f02dd80 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 -- 172.21.15.69:0/3517261542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 msgr2=0x563c3f030170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f030170 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fec2000aca0 tx=0x7fec2000f790 comp rx=0 tx=0).stop 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 -- 172.21.15.69:0/3517261542 shutdown_connections 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f030170 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01eff0 0x563c3f02d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3517261542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01e6d0 0x563c3f01eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 -- 172.21.15.69:0/3517261542 >> 172.21.15.69:0/3517261542 conn(0x563c3f01ad20 msgr2=0x563c3f01b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 -- 172.21.15.69:0/3517261542 shutdown_connections 2024-08-02T17:46:50.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 -- 172.21.15.69:0/3517261542 wait complete. 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 -- start start 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.005+0000 7fec39fd9740 1 Processor -- start 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f02ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f02b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f02b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec39fd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563c3f030950 con 0x563c3f01eff0 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec39fd9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563c3ef3e500 con 0x563c3f01e6d0 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec39fd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563c3f030ad0 con 0x563c3f02dd80 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec39fd9740 10 client.0 _ll_register_callbacks cb 0x563c3ec95a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f02b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f02b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49578/0 (socket says 172.21.15.69:49578) 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 -- 172.21.15.69:0/3976862468 learned_addr learned my addr 172.21.15.69:0/3976862468 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:50.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec35824640 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f02b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec35023640 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f02ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 -- 172.21.15.69:0/3976862468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 msgr2=0x563c3f02b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f02b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 -- 172.21.15.69:0/3976862468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 msgr2=0x563c3f02ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f02ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 -- 172.21.15.69:0/3976862468 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c3ed62620 con 0x563c3f01eff0 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec34822640 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f02b1b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fec2001b040 tx=0x7fec2000a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4ceb32ea986a9b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.006+0000 7fec18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec18ff9640 1 -- 172.21.15.69:0/3976862468 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2001d050 con 0x563c3f01eff0 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec18ff9640 1 -- 172.21.15.69:0/3976862468 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fec200108f0 con 0x563c3f01eff0 2024-08-02T17:46:50.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 1 -- 172.21.15.69:0/3976862468 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec20018b20 con 0x563c3f01eff0 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec18ff9640 1 -- 172.21.15.69:0/3976862468 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec20032050 con 0x563c3f01eff0 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 10 client.15456 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 1 -- 172.21.15.69:0/3976862468 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c3f03c5f0 con 0x563c3f01eff0 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec18ff9640 1 -- 172.21.15.69:0/3976862468 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec20018b20 con 0x563c3f01eff0 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec18ff9640 1 -- 172.21.15.69:0/3976862468 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec20004750 con 0x563c3f01eff0 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 10 client.15456 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 20 client.15456 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563c3f03da00 0x563c3f03fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 4 client.15456 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.007+0000 7fec39fd9740 1 -- 172.21.15.69:0/3976862468 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c3ef900e0 con 0x563c3f03da00 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.008+0000 7fec35023640 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563c3f03da00 0x563c3f03fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.008+0000 7fec18ff9640 10 client.15456 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.008+0000 7fec35023640 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563c3f03da00 0x563c3f03fdf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8933da4131e7e1b server_cookie=bbc51749a1e66730 in_seq=0 out_seq=0 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec18ff9640 1 -- 172.21.15.69:0/3976862468 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563c3ef900e0 con 0x563c3f03da00 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec18ff9640 10 client.15456 handle_command_reply: tid=0 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 client.15456 shutdown 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 -- 172.21.15.69:0/3976862468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 msgr2=0x563c3f02b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f02b1b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fec2001b040 tx=0x7fec2000a9f0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 -- 172.21.15.69:0/3976862468 shutdown_connections 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f02b1b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563c3f03da00 0x563c3f03fdf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f02b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 --2- 172.21.15.69:0/3976862468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f02ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec18ff9640 0 client.15456 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:50.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 -- 172.21.15.69:0/3976862468 >> 172.21.15.69:0/3976862468 conn(0x563c3f01ad20 msgr2=0x563c3f0209e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 -- 172.21.15.69:0/3976862468 shutdown_connections 2024-08-02T17:46:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 1 -- 172.21.15.69:0/3976862468 wait complete. 2024-08-02T17:46:50.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.009+0000 7fec39fd9740 20 client.15456 trim_cache size 0 max 16384 2024-08-02T17:46:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec39fd9740 1 Processor -- start 2024-08-02T17:46:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec39fd9740 1 -- start start 2024-08-02T17:46:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f045420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01eff0 0x563c3f045960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f027ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.033 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec39fd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec301af3e0 con 0x563c3f02dd80 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec39fd9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec301accb0 con 0x563c3f01e6d0 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec39fd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec301aab00 con 0x563c3f01eff0 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f027ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f027ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49586/0 (socket says 172.21.15.69:49586) 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 -- 172.21.15.69:0/2559068334 learned_addr learned my addr 172.21.15.69:0/2559068334 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35023640 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f045420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec34822640 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01eff0 0x563c3f045960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 -- 172.21.15.69:0/2559068334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01eff0 msgr2=0x563c3f045960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01eff0 0x563c3f045960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 -- 172.21.15.69:0/2559068334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 msgr2=0x563c3f045420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f045420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 -- 172.21.15.69:0/2559068334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c3f01ee20 con 0x563c3f02dd80 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.025+0000 7fec35824640 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f027ed0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fec28035000 tx=0x7fec28009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4a7c1a550d451c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec1affd640 1 -- 172.21.15.69:0/2559068334 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28004040 con 0x563c3f02dd80 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec1affd640 1 -- 172.21.15.69:0/2559068334 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec28036040 con 0x563c3f02dd80 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec1affd640 1 -- 172.21.15.69:0/2559068334 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28030d50 con 0x563c3f02dd80 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 -- 172.21.15.69:0/2559068334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 msgr2=0x563c3f027ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f027ed0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fec28035000 tx=0x7fec28009380 comp rx=0 tx=0).stop 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 -- 172.21.15.69:0/2559068334 shutdown_connections 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f02dd80 0x563c3f027ed0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f01eff0 0x563c3f045960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 --2- 172.21.15.69:0/2559068334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f045420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 -- 172.21.15.69:0/2559068334 >> 172.21.15.69:0/2559068334 conn(0x563c3f01ad20 msgr2=0x563c3f02cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 -- 172.21.15.69:0/2559068334 shutdown_connections 2024-08-02T17:46:50.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.026+0000 7fec39fd9740 1 -- 172.21.15.69:0/2559068334 wait complete. 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 1 -- start start 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 1 Processor -- start 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f027e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f0283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f0288e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec301ab170 con 0x563c3f01eff0 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec301ac5e0 con 0x563c3f01e6d0 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec301ab840 con 0x563c3f02dd80 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec39fd9740 10 client.0 _ll_register_callbacks cb 0x7fec2001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec34822640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f0283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec34822640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f0283a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49598/0 (socket says 172.21.15.69:49598) 2024-08-02T17:46:50.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec34822640 1 -- 172.21.15.69:0/1020320351 learned_addr learned my addr 172.21.15.69:0/1020320351 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.027+0000 7fec35824640 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f0288e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec35023640 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f027e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec34822640 1 -- 172.21.15.69:0/1020320351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 msgr2=0x563c3f0288e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec34822640 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f0288e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec34822640 1 -- 172.21.15.69:0/1020320351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 msgr2=0x563c3f027e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec34822640 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f027e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec34822640 1 -- 172.21.15.69:0/1020320351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c3f0a7640 con 0x563c3f01eff0 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec35824640 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f0288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec34822640 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f0283a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec20005f10 tx=0x7fec20018440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cb712af5ba177f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec18ff9640 1 -- 172.21.15.69:0/1020320351 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2001d070 con 0x563c3f01eff0 2024-08-02T17:46:50.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec18ff9640 1 -- 172.21.15.69:0/1020320351 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fec20004860 con 0x563c3f01eff0 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec39fd9740 1 -- 172.21.15.69:0/1020320351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec28036040 con 0x563c3f01eff0 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec18ff9640 1 -- 172.21.15.69:0/1020320351 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec20044050 con 0x563c3f01eff0 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec39fd9740 10 client.15462 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec39fd9740 1 -- 172.21.15.69:0/1020320351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563c3f0a8910 con 0x563c3f01eff0 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec18ff9640 1 -- 172.21.15.69:0/1020320351 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec28036040 con 0x563c3f01eff0 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec18ff9640 1 -- 172.21.15.69:0/1020320351 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec20005400 con 0x563c3f01eff0 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec39fd9740 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec39fd9740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.028+0000 7fec39fd9740 20 client.15462 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.029+0000 7fec39fd9740 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563c3f0a8fd0 0x563c3f0ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.029+0000 7fec39fd9740 4 client.15462 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.029+0000 7fec39fd9740 1 -- 172.21.15.69:0/1020320351 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c3ef8a430 con 0x563c3f0a8fd0 2024-08-02T17:46:50.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.029+0000 7fec35023640 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563c3f0a8fd0 0x563c3f0ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.029+0000 7fec18ff9640 10 client.15462 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:50.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.029+0000 7fec35023640 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563c3f0a8fd0 0x563c3f0ab3f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6623d17d5a8f3625 server_cookie=8c5c6beb5df31018 in_seq=0 out_seq=0 2024-08-02T17:46:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.042+0000 7fec18ff9640 1 -- 172.21.15.69:0/1020320351 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c3ef8a430 con 0x563c3f0a8fd0 2024-08-02T17:46:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.042+0000 7fec18ff9640 10 client.15462 handle_command_reply: tid=0 2024-08-02T17:46:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.042+0000 7fec39fd9740 1 client.15462 shutdown 2024-08-02T17:46:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 -- 172.21.15.69:0/1020320351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 msgr2=0x563c3f0283a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f0283a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fec20005f10 tx=0x7fec20018440 comp rx=0 tx=0).stop 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 -- 172.21.15.69:0/1020320351 shutdown_connections 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563c3f01eff0 0x563c3f0283a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563c3f0a8fd0 0x563c3f0ab3f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563c3f02dd80 0x563c3f0288e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 --2- 172.21.15.69:0/1020320351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563c3f01e6d0 0x563c3f027e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec18ff9640 0 client.15462 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 -- 172.21.15.69:0/1020320351 >> 172.21.15.69:0/1020320351 conn(0x563c3f01ad20 msgr2=0x563c3f02ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 -- 172.21.15.69:0/1020320351 shutdown_connections 2024-08-02T17:46:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 1 -- 172.21.15.69:0/1020320351 wait complete. 2024-08-02T17:46:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec39fd9740 20 client.15462 trim_cache size 0 max 16384 2024-08-02T17:46:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:46:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:46:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:46:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:46:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 -- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec1c0a2e10 msgr2=0x7fec1c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec1c0a2e10 0x7fec1c0a52d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fec28005cc0 tx=0x7fec2802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 -- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301006d0 msgr2=0x7fec30198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301006d0 0x7fec30198070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fec2c00e9f0 tx=0x7fec2c00edf0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 -- 172.21.15.69:0/1291968632 shutdown_connections 2024-08-02T17:46:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec1c0a2e10 0x7fec1c0a52d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec301066e0 0x7fec3019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec30101070 0x7fec301985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 --2- 172.21.15.69:0/1291968632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec301006d0 0x7fec30198070 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.043+0000 7fec37287640 1 -- 172.21.15.69:0/1291968632 >> 172.21.15.69:0/1291968632 conn(0x7fec300776d0 msgr2=0x7fec30069880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.044+0000 7fec37287640 1 -- 172.21.15.69:0/1291968632 shutdown_connections 2024-08-02T17:46:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:46:50.044+0000 7fec37287640 1 -- 172.21.15.69:0/1291968632 wait complete. 2024-08-02T17:46:50.231 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-08-02T17:46:50.268 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-08-02T17:46:50.270 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-08-02T17:46:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:50 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3419586041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:50 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:46:50.331 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:50 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3419586041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:50.331 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:50 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:46:50.347 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-02T17:46:50.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:50 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3419586041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:46:50.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:50 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:46:50.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-02T17:44:04.896710+0000', 'modified': '2024-08-02T17:46:25.691259+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24452, 'mds_1': 24431, 'mds_2': 24427}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24403': {'gid': 24403, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6835/3991686236', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6834', 'nonce': 3991686236}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3991686236}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24409': {'gid': 24409, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.83:6837/1744999893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6836', 'nonce': 1744999893}, {'type': 'v1', 'addr': '172.21.15.83:6837', 'nonce': 1744999893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24413': {'gid': 24413, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.77:6835/3410285190', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6834', 'nonce': 3410285190}, {'type': 'v1', 'addr': '172.21.15.77:6835', 'nonce': 3410285190}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24427': {'gid': 24427, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.83:6839/2418869426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6838', 'nonce': 2418869426}, {'type': 'v1', 'addr': '172.21.15.83:6839', 'nonce': 2418869426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24431': {'gid': 24431, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.77:6837/3778799888', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6836', 'nonce': 3778799888}, {'type': 'v1', 'addr': '172.21.15.77:6837', 'nonce': 3778799888}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24452': {'gid': 24452, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.77:6839/1183977027', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.77:6838', 'nonce': 1183977027}, {'type': 'v1', 'addr': '172.21.15.77:6839', 'nonce': 1183977027}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24452, 'qdb_cluster': [24452, 24431, 24427]} 2024-08-02T17:46:50.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-02T17:46:50.382 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.i status 2024-08-02T17:46:50.455 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2024-08-02T17:46:50.455 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:50.455 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T17:46:50.455 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 .. 2024-08-02T17:46:50.455 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 data 2024-08-02T17:46:50.455 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:50.455 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2024-08-02T17:46:50.455 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:50.456 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:50.456 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 .. 2024-08-02T17:46:50.457 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-08-02T17:46:50.541 INFO:teuthology.orchestra.run.smithi069.stdout:[Service] 2024-08-02T17:46:50.541 INFO:teuthology.orchestra.run.smithi069.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-08-02T17:46:50.541 INFO:teuthology.orchestra.run.smithi069.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-08-02T17:46:50.543 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-08-02T17:46:50.641 INFO:teuthology.orchestra.run.smithi069.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-08-02T17:46:50.689 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:51.091 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:51 smithi069 ceph-mon[32746]: pgmap v16: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-08-02T17:46:51.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/116740098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0100b50 msgr2=0x7fa1f0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/116740098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0100b50 0x7fa1f0100f30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009890 tx=0x7fa1e002a410 comp rx=0 tx=0).stop 2024-08-02T17:46:51.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/116740098 shutdown_connections 2024-08-02T17:46:51.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/116740098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f0102c20 0x7fa1f010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/116740098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f01022a0 0x7fa1f01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/116740098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f0100b50 0x7fa1f0100f30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/116740098 >> 172.21.15.69:0/116740098 conn(0x7fa1f00fc500 msgr2=0x7fa1f00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/116740098 shutdown_connections 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/116740098 wait complete. 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.175+0000 7fa1f4c2a640 1 Processor -- start 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1f4c2a640 1 -- start start 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1f4c2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f0100b50 0x7fa1f01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1f4c2a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f01022a0 0x7fa1f0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1f4c2a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f0102c20 0x7fa1f01a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1f4c2a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0110cb0 con 0x7fa1f0100b50 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1f4c2a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0110b30 con 0x7fa1f01022a0 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1f4c2a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f0110e30 con 0x7fa1f0102c20 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f01022a0 0x7fa1f0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1eed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f0102c20 0x7fa1f01a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f01022a0 0x7fa1f0199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49988/0 (socket says 172.21.15.69:49988) 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1e7fff640 1 -- 172.21.15.69:0/3879035217 learned_addr learned my addr 172.21.15.69:0/3879035217 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f0100b50 0x7fa1f01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1e7fff640 1 -- 172.21.15.69:0/3879035217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f0102c20 msgr2=0x7fa1f01a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f0102c20 0x7fa1f01a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1e7fff640 1 -- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f0100b50 msgr2=0x7fa1f01990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f0100b50 0x7fa1f01990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.176+0000 7fa1e7fff640 1 -- 172.21.15.69:0/3879035217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f01a46b0 con 0x7fa1f01022a0 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.177+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f01022a0 0x7fa1f0199630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c970 tx=0x7fa1d800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.177+0000 7fa1e77fe640 1 -- 172.21.15.69:0/3879035217 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8007bb0 con 0x7fa1f01022a0 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.177+0000 7fa1e77fe640 1 -- 172.21.15.69:0/3879035217 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d8007d50 con 0x7fa1f01022a0 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.177+0000 7fa1e77fe640 1 -- 172.21.15.69:0/3879035217 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800f450 con 0x7fa1f01022a0 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.177+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/3879035217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f01a4880 con 0x7fa1f01022a0 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.177+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/3879035217 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f01a4d90 con 0x7fa1f01022a0 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.180+0000 7fa1e77fe640 1 -- 172.21.15.69:0/3879035217 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7fa1d8005020 con 0x7fa1f01022a0 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.180+0000 7fa1f7028740 1 Processor -- start 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.180+0000 7fa1f7028740 1 -- start start 2024-08-02T17:46:51.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1e77fe640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1dc0a2e90 0x7fa1dc0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f7d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1f7028740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ee80ee2710 con 0x55ee80f7de10 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1f7028740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ee80e7a770 con 0x55ee80f8cc20 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1f7028740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee80ede6d0 con 0x55ee80f7d4f0 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1dc0a2e90 0x7fa1dc0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f7d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1e77fe640 1 -- 172.21.15.69:0/3879035217 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa1d80cf770 con 0x7fa1f01022a0 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f7d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38116/0 (socket says 172.21.15.69:38116) 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 -- 172.21.15.69:0/3193747576 learned_addr learned my addr 172.21.15.69:0/3193747576 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:51.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 -- 172.21.15.69:0/3193747576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 msgr2=0x55ee80f8f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 -- 172.21.15.69:0/3193747576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 msgr2=0x55ee80f8c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 -- 172.21.15.69:0/3193747576 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee80e813e0 con 0x55ee80f7d4f0 2024-08-02T17:46:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1dc0a2e90 0x7fa1dc0a5350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8005fd0 tx=0x7fa1e8009940 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:51.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.181+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f7d8d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009590 tx=0x7fa1e0030ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37dca3a10d836c23 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.182+0000 7fa1e4ff9640 1 -- 172.21.15.69:0/3193747576 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00303a0 con 0x55ee80f7d4f0 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.182+0000 7fa1e4ff9640 1 -- 172.21.15.69:0/3193747576 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0005040 con 0x55ee80f7d4f0 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.182+0000 7fa1e4ff9640 1 -- 172.21.15.69:0/3193747576 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0004b60 con 0x55ee80f7d4f0 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 -- 172.21.15.69:0/3193747576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 msgr2=0x55ee80f7d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f7d8d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009590 tx=0x7fa1e0030ab0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 -- 172.21.15.69:0/3193747576 shutdown_connections 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 --2- 172.21.15.69:0/3193747576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f7d8d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 -- 172.21.15.69:0/3193747576 >> 172.21.15.69:0/3193747576 conn(0x55ee80f79bc0 msgr2=0x55ee80f79fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 -- 172.21.15.69:0/3193747576 shutdown_connections 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 -- 172.21.15.69:0/3193747576 wait complete. 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 -- start start 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.183+0000 7fa1f7028740 1 Processor -- start 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f89b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1f7028740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ee80f8f7e0 con 0x55ee80f7de10 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1f7028740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ee80e9d970 con 0x55ee80f8cc20 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1f7028740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee80f8f960 con 0x55ee80f7d4f0 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1f7028740 10 client.0 _ll_register_callbacks cb 0x55ee80bf17b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1eed9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49630/0 (socket says 172.21.15.69:49630) 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1e7fff640 1 -- 172.21.15.69:0/4045535176 learned_addr learned my addr 172.21.15.69:0/4045535176 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1eed9d640 1 -- 172.21.15.69:0/4045535176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 msgr2=0x55ee80f89b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f89b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1eed9d640 1 -- 172.21.15.69:0/4045535176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 msgr2=0x55ee80f8a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1eed9d640 1 -- 172.21.15.69:0/4045535176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee80c542b0 con 0x55ee80f8cc20 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800de40 tx=0x7fa1e8004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9f7ca900e64f01 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.184+0000 7fa1d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/4045535176 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8004530 con 0x55ee80f8cc20 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 1 -- 172.21.15.69:0/4045535176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1e0005040 con 0x55ee80f8cc20 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/4045535176 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa1e8017050 con 0x55ee80f8cc20 2024-08-02T17:46:51.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/4045535176 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1e80046d0 con 0x55ee80f8cc20 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 10 client.25151 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 1 -- 172.21.15.69:0/4045535176 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee80f9b4f0 con 0x55ee80f8cc20 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/4045535176 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1e0005040 con 0x55ee80f8cc20 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/4045535176 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1e800b040 con 0x55ee80f8cc20 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 10 client.25151 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 10 client.25151 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 20 client.25151 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55ee80f9dae0 0x55ee80f9ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 4 client.25151 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.185+0000 7fa1f7028740 1 -- 172.21.15.69:0/4045535176 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee80eefc80 con 0x55ee80f9dae0 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.186+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55ee80f9dae0 0x55ee80f9ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.186+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55ee80f9dae0 0x55ee80f9ef50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61bf0c2b047531f0 server_cookie=529e5399668d1237 in_seq=0 out_seq=0 2024-08-02T17:46:51.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.186+0000 7fa1d5ffb640 10 client.25151 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/4045535176 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ee80eefc80 con 0x55ee80f9dae0 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1d5ffb640 10 client.25151 handle_command_reply: tid=0 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 client.25151 shutdown 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 -- 172.21.15.69:0/4045535176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 msgr2=0x55ee80f8a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a600 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800de40 tx=0x7fa1e8004ab0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 -- 172.21.15.69:0/4045535176 shutdown_connections 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55ee80f9dae0 0x55ee80f9ef50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a600 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 --2- 172.21.15.69:0/4045535176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f89b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1d5ffb640 0 client.25151 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 -- 172.21.15.69:0/4045535176 >> 172.21.15.69:0/4045535176 conn(0x55ee80f79bc0 msgr2=0x55ee80f7f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:51.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 -- 172.21.15.69:0/4045535176 shutdown_connections 2024-08-02T17:46:51.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 1 -- 172.21.15.69:0/4045535176 wait complete. 2024-08-02T17:46:51.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.187+0000 7fa1f7028740 20 client.25151 trim_cache size 0 max 16384 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1f7028740 1 Processor -- start 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1f7028740 1 -- start start 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f98300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1f7028740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ee80f9d290 con 0x55ee80f7de10 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1f7028740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ee80f9d960 con 0x55ee80f8cc20 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1f7028740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee80f9e030 con 0x55ee80f7d4f0 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49642/0 (socket says 172.21.15.69:49642) 2024-08-02T17:46:51.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 -- 172.21.15.69:0/2245315679 learned_addr learned my addr 172.21.15.69:0/2245315679 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f98300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 -- 172.21.15.69:0/2245315679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 msgr2=0x55ee80f98300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f98300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 -- 172.21.15.69:0/2245315679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 msgr2=0x55ee80f8a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 -- 172.21.15.69:0/2245315679 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee80f88b30 con 0x55ee80f7de10 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f98300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.205+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8097b90 tx=0x7fa1d8097390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dd8c213ec0ffbb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1d7fff640 1 -- 172.21.15.69:0/2245315679 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d801d040 con 0x55ee80f7de10 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1d7fff640 1 -- 172.21.15.69:0/2245315679 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d809b730 con 0x55ee80f7de10 2024-08-02T17:46:51.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1d7fff640 1 -- 172.21.15.69:0/2245315679 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8005310 con 0x55ee80f7de10 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 -- 172.21.15.69:0/2245315679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 msgr2=0x55ee80f8a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8097b90 tx=0x7fa1d8097390 comp rx=0 tx=0).stop 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 -- 172.21.15.69:0/2245315679 shutdown_connections 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7de10 0x55ee80f8a0d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f8cc20 0x55ee80f8a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 --2- 172.21.15.69:0/2245315679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f7d4f0 0x55ee80f98300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 -- 172.21.15.69:0/2245315679 >> 172.21.15.69:0/2245315679 conn(0x55ee80f79bc0 msgr2=0x55ee80f97bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 -- 172.21.15.69:0/2245315679 shutdown_connections 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.206+0000 7fa1f7028740 1 -- 172.21.15.69:0/2245315679 wait complete. 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 1 -- start start 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 1 Processor -- start 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7d4f0 0x55ee80fa70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f7de10 0x55ee80fa7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f8cc20 0x55ee80fa7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ee80f9edd0 con 0x55ee80f7d4f0 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ee80f9e700 con 0x55ee80f7de10 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ee80f9f4a0 con 0x55ee80f8cc20 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1f7028740 10 client.0 _ll_register_callbacks cb 0x7fa1e800e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7d4f0 0x55ee80fa70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f7de10 0x55ee80fa7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1ee59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7d4f0 0x55ee80fa70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49646/0 (socket says 172.21.15.69:49646) 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1e7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f7de10 0x55ee80fa7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50016/0 (socket says 172.21.15.69:50016) 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1ee59c640 1 -- 172.21.15.69:0/1655962329 learned_addr learned my addr 172.21.15.69:0/1655962329 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1eed9d640 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f8cc20 0x55ee80fa7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1ee59c640 1 -- 172.21.15.69:0/1655962329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f8cc20 msgr2=0x55ee80fa7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f8cc20 0x55ee80fa7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1ee59c640 1 -- 172.21.15.69:0/1655962329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f7de10 msgr2=0x55ee80fa7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f7de10 0x55ee80fa7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1ee59c640 1 -- 172.21.15.69:0/1655962329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee80fa2220 con 0x55ee80f7d4f0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.207+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f7de10 0x55ee80fa7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1ee59c640 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7d4f0 0x55ee80fa70c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00047d0 tx=0x7fa1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d31b5bd8080caf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/1655962329 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0055060 con 0x55ee80f7d4f0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/1655962329 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa1e0004450 con 0x55ee80f7d4f0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 1 -- 172.21.15.69:0/1655962329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d809b730 con 0x55ee80f7d4f0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/1655962329 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1e005c050 con 0x55ee80f7d4f0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 10 client.15480 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 1 -- 172.21.15.69:0/1655962329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ee81008690 con 0x55ee80f7d4f0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/1655962329 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1d809b730 con 0x55ee80f7d4f0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/1655962329 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1e004f070 con 0x55ee80f7d4f0 2024-08-02T17:46:51.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 10 client.15480 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 10 client.15480 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-02T17:46:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 20 client.15480 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55ee8100ce50 0x55ee8100bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 4 client.15480 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:46:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.208+0000 7fa1f7028740 1 -- 172.21.15.69:0/1655962329 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "status"}) -- 0x55ee80eea190 con 0x55ee8100ce50 2024-08-02T17:46:51.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55ee8100ce50 0x55ee8100bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1d5ffb640 10 client.15480 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:46:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1e7fff640 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55ee8100ce50 0x55ee8100bb60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ba74b891a039044 server_cookie=7af7256d8399a087 in_seq=0 out_seq=0 2024-08-02T17:46:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1d5ffb640 1 -- 172.21.15.69:0/1655962329 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ee80eea190 con 0x55ee8100ce50 2024-08-02T17:46:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1d5ffb640 10 client.15480 handle_command_reply: tid=0 2024-08-02T17:46:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 client.15480 shutdown 2024-08-02T17:46:51.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 -- 172.21.15.69:0/1655962329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7d4f0 msgr2=0x55ee80fa70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7d4f0 0x55ee80fa70c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00047d0 tx=0x7fa1e00046c0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 -- 172.21.15.69:0/1655962329 shutdown_connections 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55ee8100ce50 0x55ee8100bb60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ee80f8cc20 0x55ee80fa7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ee80f7de10 0x55ee80fa7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 --2- 172.21.15.69:0/1655962329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ee80f7d4f0 0x55ee80fa70c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 -- 172.21.15.69:0/1655962329 >> 172.21.15.69:0/1655962329 conn(0x55ee80f79bc0 msgr2=0x55ee80f98410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1d5ffb640 0 client.15480 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 -- 172.21.15.69:0/1655962329 shutdown_connections 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 1 -- 172.21.15.69:0/1655962329 wait complete. 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.209+0000 7fa1f7028740 20 client.15480 trim_cache size 0 max 16384 2024-08-02T17:46:51.217 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 2, 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24427, 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 124.307875131, 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 26, 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 95, 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 95, 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 185.70933461000001 2024-08-02T17:46:51.218 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:46:51.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1dc0a2e90 msgr2=0x7fa1dc0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1dc0a2e90 0x7fa1dc0a5350 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8005fd0 tx=0x7fa1e8009940 comp rx=0 tx=0).stop 2024-08-02T17:46:51.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/3879035217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f01022a0 msgr2=0x7fa1f0199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:51.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f01022a0 0x7fa1f0199630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c970 tx=0x7fa1d800ce30 comp rx=0 tx=0).stop 2024-08-02T17:46:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/3879035217 shutdown_connections 2024-08-02T17:46:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1dc0a2e90 0x7fa1dc0a5350 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1f0102c20 0x7fa1f01a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1f01022a0 0x7fa1f0199630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 --2- 172.21.15.69:0/3879035217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1f0100b50 0x7fa1f01990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/3879035217 >> 172.21.15.69:0/3879035217 conn(0x7fa1f00fc500 msgr2=0x7fa1f00fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/3879035217 shutdown_connections 2024-08-02T17:46:51.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:51.210+0000 7fa1f4c2a640 1 -- 172.21.15.69:0/3879035217 wait complete. 2024-08-02T17:46:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:51 smithi077 ceph-mon[37108]: pgmap v16: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-08-02T17:46:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:51 smithi083 ceph-mon[37031]: pgmap v16: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 31 op/s 2024-08-02T17:46:51.927 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.h status 2024-08-02T17:46:52.192 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 -- 172.21.15.69:0/103988816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 msgr2=0x7f0118107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 --2- 172.21.15.69:0/103988816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 0x7f0118107c90 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0108009930 tx=0x7f010802a420 comp rx=0 tx=0).stop 2024-08-02T17:46:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 -- 172.21.15.69:0/103988816 shutdown_connections 2024-08-02T17:46:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 --2- 172.21.15.69:0/103988816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0118102250 0x7f011810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 --2- 172.21.15.69:0/103988816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f01181018b0 0x7f0118101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 --2- 172.21.15.69:0/103988816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 0x7f0118107c90 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 -- 172.21.15.69:0/103988816 >> 172.21.15.69:0/103988816 conn(0x7f01180fd5f0 msgr2=0x7f01180ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:52.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 -- 172.21.15.69:0/103988816 shutdown_connections 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.657+0000 7f011d71f640 1 -- 172.21.15.69:0/103988816 wait complete. 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f011d71f640 1 Processor -- start 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f011d71f640 1 -- start start 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f011d71f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01181018b0 0x7f01181981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f011d71f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0118102250 0x7f01181986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f011d71f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 0x7f011819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f011d71f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0118111cb0 con 0x7f01181078b0 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f011d71f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0118111b30 con 0x7f0118102250 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f011d71f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0118111e30 con 0x7f01181018b0 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f01177fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 0x7f011819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f01177fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 0x7f011819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49678/0 (socket says 172.21.15.69:49678) 2024-08-02T17:46:52.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f01177fe640 1 -- 172.21.15.69:0/917356153 learned_addr learned my addr 172.21.15.69:0/917356153 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f0116ffd640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01181018b0 0x7f01181981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.658+0000 7f010e7fc640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0118102250 0x7f01181986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f01177fe640 1 -- 172.21.15.69:0/917356153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01181018b0 msgr2=0x7f01181981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f01177fe640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01181018b0 0x7f01181981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f01177fe640 1 -- 172.21.15.69:0/917356153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0118102250 msgr2=0x7f01181986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f01177fe640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0118102250 0x7f01181986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f01177fe640 1 -- 172.21.15.69:0/917356153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011819d160 con 0x7f01181078b0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f01177fe640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 0x7f011819ca80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f011000eaa0 tx=0x7f011000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f0114ff9640 1 -- 172.21.15.69:0/917356153 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011000cdb0 con 0x7f01181078b0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f011d71f640 1 -- 172.21.15.69:0/917356153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011819d450 con 0x7f01181078b0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f0114ff9640 1 -- 172.21.15.69:0/917356153 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0110005020 con 0x7f01181078b0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f0114ff9640 1 -- 172.21.15.69:0/917356153 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01100116d0 con 0x7f01181078b0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.659+0000 7f011d71f640 1 -- 172.21.15.69:0/917356153 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01181a4d30 con 0x7f01181078b0 2024-08-02T17:46:52.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.661+0000 7f0114ff9640 1 -- 172.21.15.69:0/917356153 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f01100054d0 con 0x7f01181078b0 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f011fb1d740 1 Processor -- start 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f011fb1d740 1 -- start start 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b42a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b4d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b4d810 0x561cf0b4fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f011fb1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561cf0aa7710 con 0x561cf0b42630 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f011fb1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561cf0a3f770 con 0x561cf0b4d810 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f011fb1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561cf0aa36d0 con 0x561cf0b42f50 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f010e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b42a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f010e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b42a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49684/0 (socket says 172.21.15.69:49684) 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f010e7fc640 1 -- 172.21.15.69:0/3249245016 learned_addr learned my addr 172.21.15.69:0/3249245016 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f0114ff9640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f01040a2e10 0x7f01040a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.662+0000 7f0116ffd640 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b4d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010e7fc640 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b4d810 0x561cf0b4fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f01177fe640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f01040a2e10 0x7f01040a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010e7fc640 1 -- 172.21.15.69:0/3249245016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 msgr2=0x561cf0b4d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010e7fc640 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b4d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010e7fc640 1 -- 172.21.15.69:0/3249245016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b4d810 msgr2=0x561cf0b4fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010e7fc640 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b4d810 0x561cf0b4fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010e7fc640 1 -- 172.21.15.69:0/3249245016 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf0a463e0 con 0x561cf0b42630 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f0116ffd640 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b4d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f0114ff9640 1 -- 172.21.15.69:0/917356153 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f01100cfd20 con 0x7f01181078b0 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010e7fc640 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b42a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f01000058c0 tx=0x7f010000b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9087330cf8eaea server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010d7fa640 1 -- 172.21.15.69:0/3249245016 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010000f3f0 con 0x561cf0b42630 2024-08-02T17:46:52.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010d7fa640 1 -- 172.21.15.69:0/3249245016 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010000fa70 con 0x561cf0b42630 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.663+0000 7f010d7fa640 1 -- 172.21.15.69:0/3249245016 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100012860 con 0x561cf0b42630 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 -- 172.21.15.69:0/3249245016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 msgr2=0x561cf0b42a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f01177fe640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f01040a2e10 0x7f01040a52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01100ccf30 tx=0x7f01100c6760 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b42a10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f01000058c0 tx=0x7f010000b6b0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 -- 172.21.15.69:0/3249245016 shutdown_connections 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b4d810 0x561cf0b4fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b4d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 --2- 172.21.15.69:0/3249245016 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b42a10 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 -- 172.21.15.69:0/3249245016 >> 172.21.15.69:0/3249245016 conn(0x561cf0b3ebc0 msgr2=0x561cf0b3efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 -- 172.21.15.69:0/3249245016 shutdown_connections 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 -- 172.21.15.69:0/3249245016 wait complete. 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 -- start start 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.664+0000 7f011fb1d740 1 Processor -- start 2024-08-02T17:46:52.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42630 0x561cf0b4a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b4abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b4b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f011fb1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b503e0 con 0x561cf0b4d810 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f011fb1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561cf0a62970 con 0x561cf0b42f50 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f011fb1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b50560 con 0x561cf0b42630 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f011fb1d740 10 client.0 _ll_register_callbacks cb 0x561cf07b67b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b4b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b4b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49696/0 (socket says 172.21.15.69:49696) 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 -- 172.21.15.69:0/2492033047 learned_addr learned my addr 172.21.15.69:0/2492033047 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f0116ffd640 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b4abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42630 0x561cf0b4a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 -- 172.21.15.69:0/2492033047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42630 msgr2=0x561cf0b4a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42630 0x561cf0b4a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 -- 172.21.15.69:0/2492033047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 msgr2=0x561cf0b4abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b4abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 -- 172.21.15.69:0/2492033047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf08192b0 con 0x561cf0b4d810 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f010e7fc640 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b4b110 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0100005fd0 tx=0x7f0100012500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e9468f828a8973 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f00fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f00fe7fc640 1 -- 172.21.15.69:0/2492033047 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100021060 con 0x561cf0b4d810 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f00fe7fc640 1 -- 172.21.15.69:0/2492033047 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0100004030 con 0x561cf0b4d810 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.665+0000 7f011fb1d740 1 -- 172.21.15.69:0/2492033047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010000fa70 con 0x561cf0b4d810 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f00fe7fc640 1 -- 172.21.15.69:0/2492033047 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0100004230 con 0x561cf0b4d810 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f011fb1d740 10 client.15492 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f011fb1d740 1 -- 172.21.15.69:0/2492033047 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561cf0b5bf90 con 0x561cf0b4d810 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f00fe7fc640 1 -- 172.21.15.69:0/2492033047 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010000fa70 con 0x561cf0b4d810 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f00fe7fc640 1 -- 172.21.15.69:0/2492033047 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f01000052f0 con 0x561cf0b4d810 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f011fb1d740 10 client.15492 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:52.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f011fb1d740 10 client.15492 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f011fb1d740 20 client.15492 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f011fb1d740 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561cf0b5e3c0 0x561cf0b5f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f011fb1d740 4 client.15492 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f011fb1d740 1 -- 172.21.15.69:0/2492033047 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cf0ab4c80 con 0x561cf0b5e3c0 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.666+0000 7f0116ffd640 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561cf0b5e3c0 0x561cf0b5f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.667+0000 7f0116ffd640 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561cf0b5e3c0 0x561cf0b5f830 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a95bb3c0e1892bc server_cookie=99525625f3e4c1cc in_seq=0 out_seq=0 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.667+0000 7f00fe7fc640 10 client.15492 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f00fe7fc640 1 -- 172.21.15.69:0/2492033047 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561cf0ab4c80 con 0x561cf0b5e3c0 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f00fe7fc640 10 client.15492 handle_command_reply: tid=0 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 client.15492 shutdown 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 -- 172.21.15.69:0/2492033047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 msgr2=0x561cf0b4b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b4b110 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0100005fd0 tx=0x7f0100012500 comp rx=0 tx=0).stop 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 -- 172.21.15.69:0/2492033047 shutdown_connections 2024-08-02T17:46:52.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561cf0b5e3c0 0x561cf0b5f830 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b4b110 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b4abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f00fe7fc640 0 client.15492 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:46:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 --2- 172.21.15.69:0/2492033047 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42630 0x561cf0b4a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 -- 172.21.15.69:0/2492033047 >> 172.21.15.69:0/2492033047 conn(0x561cf0b3ebc0 msgr2=0x561cf0b448a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 -- 172.21.15.69:0/2492033047 shutdown_connections 2024-08-02T17:46:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 1 -- 172.21.15.69:0/2492033047 wait complete. 2024-08-02T17:46:52.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.668+0000 7f011fb1d740 20 client.15492 trim_cache size 0 max 16384 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.685+0000 7f011fb1d740 1 Processor -- start 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.685+0000 7f011fb1d740 1 -- start start 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42630 0x561cf0b45090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b49f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b47910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f011fb1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b5db70 con 0x561cf0b4d810 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f011fb1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b5e240 con 0x561cf0b42630 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f011fb1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b5e910 con 0x561cf0b42f50 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b47910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b47910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49706/0 (socket says 172.21.15.69:49706) 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 -- 172.21.15.69:0/284089357 learned_addr learned my addr 172.21.15.69:0/284089357 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:52.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f0116ffd640 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b49f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42630 0x561cf0b45090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 -- 172.21.15.69:0/284089357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 msgr2=0x561cf0b49f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b49f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 -- 172.21.15.69:0/284089357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42630 msgr2=0x561cf0b45090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42630 0x561cf0b45090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 -- 172.21.15.69:0/284089357 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf0b42d80 con 0x561cf0b4d810 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.686+0000 7f010e7fc640 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b47910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0100005c70 tx=0x7f0100005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6ed3d9ee73306d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f010cff9640 1 -- 172.21.15.69:0/284089357 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100021070 con 0x561cf0b4d810 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f010cff9640 1 -- 172.21.15.69:0/284089357 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010000fa70 con 0x561cf0b4d810 2024-08-02T17:46:52.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f010cff9640 1 -- 172.21.15.69:0/284089357 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010000f810 con 0x561cf0b4d810 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 -- 172.21.15.69:0/284089357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 msgr2=0x561cf0b47910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b47910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0100005c70 tx=0x7f0100005970 comp rx=0 tx=0).stop 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 -- 172.21.15.69:0/284089357 shutdown_connections 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b4d810 0x561cf0b47910 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b42f50 0x561cf0b49f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 --2- 172.21.15.69:0/284089357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42630 0x561cf0b45090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 -- 172.21.15.69:0/284089357 >> 172.21.15.69:0/284089357 conn(0x561cf0b3ebc0 msgr2=0x561cf0b4c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 -- 172.21.15.69:0/284089357 shutdown_connections 2024-08-02T17:46:52.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.687+0000 7f011fb1d740 1 -- 172.21.15.69:0/284089357 wait complete. 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 1 -- start start 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 1 Processor -- start 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b47320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b4d810 0x561cf0b47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b5f6b0 con 0x561cf0b42630 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b5efe0 con 0x561cf0b42f50 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561cf0b5fd80 con 0x561cf0b4d810 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f011fb1d740 10 client.0 _ll_register_callbacks cb 0x7f010000bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b680d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49712/0 (socket says 172.21.15.69:49712) 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 -- 172.21.15.69:0/1324899001 learned_addr learned my addr 172.21.15.69:0/1324899001 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b4d810 0x561cf0b47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f0116ffd640 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b47320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 -- 172.21.15.69:0/1324899001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b4d810 msgr2=0x561cf0b47860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b4d810 0x561cf0b47860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 -- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 msgr2=0x561cf0b47320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b47320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f010e7fc640 1 -- 172.21.15.69:0/1324899001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf0b47f50 con 0x561cf0b42630 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.688+0000 7f0116ffd640 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b47320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f010e7fc640 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b680d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0100011f50 tx=0x7f010000f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72640d63c3c7c87d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f00fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f00fe7fc640 1 -- 172.21.15.69:0/1324899001 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100021040 con 0x561cf0b42630 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f00fe7fc640 1 -- 172.21.15.69:0/1324899001 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0100012be0 con 0x561cf0b42630 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 1 -- 172.21.15.69:0/1324899001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f010000fa70 con 0x561cf0b42630 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f00fe7fc640 1 -- 172.21.15.69:0/1324899001 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0100012de0 con 0x561cf0b42630 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 10 client.15498 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:52.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 1 -- 172.21.15.69:0/1324899001 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561cf0bc8ad0 con 0x561cf0b42630 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f00fe7fc640 1 -- 172.21.15.69:0/1324899001 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010000fa70 con 0x561cf0b42630 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f00fe7fc640 1 -- 172.21.15.69:0/1324899001 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f010001c5e0 con 0x561cf0b42630 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 10 client.15498 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 10 client.15498 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 20 client.15498 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561cf0bcd110 0x561cf0bcbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 4 client.15498 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.689+0000 7f011fb1d740 1 -- 172.21.15.69:0/1324899001 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "status"}) -- 0x561cf0aaf190 con 0x561cf0bcd110 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f0116ffd640 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561cf0bcd110 0x561cf0bcbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f0116ffd640 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561cf0bcd110 0x561cf0bcbe20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e798104a424e5315 server_cookie=e4332e88279ec786 in_seq=0 out_seq=0 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f00fe7fc640 10 client.15498 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f00fe7fc640 1 -- 172.21.15.69:0/1324899001 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x561cf0aaf190 con 0x561cf0bcd110 2024-08-02T17:46:52.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f00fe7fc640 10 client.15498 handle_command_reply: tid=0 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f011fb1d740 1 client.15498 shutdown 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f011fb1d740 1 -- 172.21.15.69:0/1324899001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 msgr2=0x561cf0b680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f011fb1d740 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b680d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0100011f50 tx=0x7f010000f3f0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f011fb1d740 1 -- 172.21.15.69:0/1324899001 shutdown_connections 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f011fb1d740 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561cf0bcd110 0x561cf0bcbe20 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f011fb1d740 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561cf0b4d810 0x561cf0b47860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f00fe7fc640 0 client.15498 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.690+0000 7f011fb1d740 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561cf0b42f50 0x561cf0b47320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011fb1d740 1 --2- 172.21.15.69:0/1324899001 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561cf0b42630 0x561cf0b680d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011fb1d740 1 -- 172.21.15.69:0/1324899001 >> 172.21.15.69:0/1324899001 conn(0x561cf0b3ebc0 msgr2=0x561cf0b4bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011fb1d740 1 -- 172.21.15.69:0/1324899001 shutdown_connections 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011fb1d740 1 -- 172.21.15.69:0/1324899001 wait complete. 2024-08-02T17:46:52.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011fb1d740 20 client.15498 trim_cache size 0 max 16384 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 1, 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24431, 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 126.802374517, 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 26, 2024-08-02T17:46:52.699 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 95, 2024-08-02T17:46:52.700 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 95, 2024-08-02T17:46:52.700 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 189.771953999 2024-08-02T17:46:52.700 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 -- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f01040a2e10 msgr2=0x7f01040a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f01040a2e10 0x7f01040a52d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01100ccf30 tx=0x7f01100c6760 comp rx=0 tx=0).stop 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 -- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 msgr2=0x7f011819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 0x7f011819ca80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f011000eaa0 tx=0x7f011000ef60 comp rx=0 tx=0).stop 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 -- 172.21.15.69:0/917356153 shutdown_connections 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f01181078b0 0x7f011819ca80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f01040a2e10 0x7f01040a52d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0118102250 0x7f01181986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 --2- 172.21.15.69:0/917356153 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01181018b0 0x7f01181981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 -- 172.21.15.69:0/917356153 >> 172.21.15.69:0/917356153 conn(0x7f01180fd5f0 msgr2=0x7f01180fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:52.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 -- 172.21.15.69:0/917356153 shutdown_connections 2024-08-02T17:46:52.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:52.691+0000 7f011d71f640 1 -- 172.21.15.69:0/917356153 wait complete. 2024-08-02T17:46:53.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:53 smithi083 ceph-mon[37031]: pgmap v17: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-08-02T17:46:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:53 smithi077 ceph-mon[37108]: pgmap v17: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-08-02T17:46:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:53 smithi069 ceph-mon[32746]: pgmap v17: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-08-02T17:46:53.427 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.b status 2024-08-02T17:46:53.747 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:46:54.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 -- 172.21.15.69:0/133118437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c102c20 msgr2=0x7f380c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 --2- 172.21.15.69:0/133118437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c102c20 0x7f380c10d770 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3804009ca0 tx=0x7f380402a950 comp rx=0 tx=0).stop 2024-08-02T17:46:54.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 -- 172.21.15.69:0/133118437 shutdown_connections 2024-08-02T17:46:54.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 --2- 172.21.15.69:0/133118437 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c102c20 0x7f380c10d770 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 --2- 172.21.15.69:0/133118437 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f380c1022a0 0x7f380c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 --2- 172.21.15.69:0/133118437 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f380c100b50 0x7f380c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 -- 172.21.15.69:0/133118437 >> 172.21.15.69:0/133118437 conn(0x7f380c0fc500 msgr2=0x7f380c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:54.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 -- 172.21.15.69:0/133118437 shutdown_connections 2024-08-02T17:46:54.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.184+0000 7f3810ff0640 1 -- 172.21.15.69:0/133118437 wait complete. 2024-08-02T17:46:54.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f3810ff0640 1 Processor -- start 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f3810ff0640 1 -- start start 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f3810ff0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c100b50 0x7f380c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f3810ff0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f380c1022a0 0x7f380c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f3810ff0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f380c102c20 0x7f380c1a4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f3810ff0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f380c110d20 con 0x7f380c100b50 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f3810ff0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f380c110ba0 con 0x7f380c1022a0 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f3810ff0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f380c110ea0 con 0x7f380c102c20 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.185+0000 7f380a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c100b50 0x7f380c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c100b50 0x7f380c199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49726/0 (socket says 172.21.15.69:49726) 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380a59c640 1 -- 172.21.15.69:0/4082387302 learned_addr learned my addr 172.21.15.69:0/4082387302 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f3809d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f380c1022a0 0x7f380c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380ad9d640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f380c102c20 0x7f380c1a4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380a59c640 1 -- 172.21.15.69:0/4082387302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f380c102c20 msgr2=0x7f380c1a4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380a59c640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f380c102c20 0x7f380c1a4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380a59c640 1 -- 172.21.15.69:0/4082387302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f380c1022a0 msgr2=0x7f380c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380a59c640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f380c1022a0 0x7f380c1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380a59c640 1 -- 172.21.15.69:0/4082387302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c1a4650 con 0x7f380c100b50 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f380a59c640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c100b50 0x7f380c199090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00e9f0 tx=0x7f37fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f37fb7fe640 1 -- 172.21.15.69:0/4082387302 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00ccc0 con 0x7f380c100b50 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f3810ff0640 1 -- 172.21.15.69:0/4082387302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c1a4820 con 0x7f380c100b50 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f37fb7fe640 1 -- 172.21.15.69:0/4082387302 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37fc010910 con 0x7f380c100b50 2024-08-02T17:46:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f37fb7fe640 1 -- 172.21.15.69:0/4082387302 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0188a0 con 0x7f380c100b50 2024-08-02T17:46:54.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.186+0000 7f3810ff0640 1 -- 172.21.15.69:0/4082387302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c1a4d60 con 0x7f380c100b50 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.189+0000 7f37fb7fe640 1 -- 172.21.15.69:0/4082387302 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 23) ==== 138060+0+0 (secure 0 0 0) 0x7f37fc010ac0 con 0x7f380c100b50 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.189+0000 7f37fb7fe640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37f40a2e10 0x7f37f40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.190+0000 7f37fb7fe640 1 -- 172.21.15.69:0/4082387302 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f37fc0cfef0 con 0x7f380c100b50 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.190+0000 7f3809d9b640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37f40a2e10 0x7f37f40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.190+0000 7f38133ee740 1 Processor -- start 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.190+0000 7f3809d9b640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37f40a2e10 0x7f37f40a52d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f380c06a7f0 tx=0x7f3800009110 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.190+0000 7f38133ee740 1 -- start start 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f38133ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f38133ee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974add2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f38133ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f38133ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558974a37710 con 0x558974ad2630 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f38133ee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589749cf770 con 0x558974ad2f50 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f38133ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558974a336d0 con 0x558974add810 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f380a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f3809d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974add2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f380a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49740/0 (socket says 172.21.15.69:49740) 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f380a59c640 1 -- 172.21.15.69:0/4140092394 learned_addr learned my addr 172.21.15.69:0/4140092394 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f380ad9d640 1 --2- 172.21.15.69:0/4140092394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f3809d9b640 1 -- 172.21.15.69:0/4140092394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 msgr2=0x558974adfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f3809d9b640 1 --2- 172.21.15.69:0/4140092394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f3809d9b640 1 -- 172.21.15.69:0/4140092394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 msgr2=0x558974ad2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f3809d9b640 1 --2- 172.21.15.69:0/4140092394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f3809d9b640 1 -- 172.21.15.69:0/4140092394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589749d63e0 con 0x558974ad2f50 2024-08-02T17:46:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.191+0000 7f3809d9b640 1 --2- 172.21.15.69:0/4140092394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974add2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f380000e5a0 tx=0x7f3800011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5650b886420e2a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f37f8ff9640 1 -- 172.21.15.69:0/4140092394 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380001b040 con 0x558974ad2f50 2024-08-02T17:46:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f37f8ff9640 1 -- 172.21.15.69:0/4140092394 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3800024070 con 0x558974ad2f50 2024-08-02T17:46:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f37f8ff9640 1 -- 172.21.15.69:0/4140092394 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380001f3f0 con 0x558974ad2f50 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f38133ee740 1 -- 172.21.15.69:0/4140092394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 msgr2=0x558974add2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f38133ee740 1 --2- 172.21.15.69:0/4140092394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974add2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f380000e5a0 tx=0x7f3800011000 comp rx=0 tx=0).stop 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f38133ee740 1 -- 172.21.15.69:0/4140092394 shutdown_connections 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f38133ee740 1 --2- 172.21.15.69:0/4140092394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f38133ee740 1 --2- 172.21.15.69:0/4140092394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974add2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f38133ee740 1 --2- 172.21.15.69:0/4140092394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.192+0000 7f38133ee740 1 -- 172.21.15.69:0/4140092394 >> 172.21.15.69:0/4140092394 conn(0x558974acebc0 msgr2=0x558974acefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 -- 172.21.15.69:0/4140092394 shutdown_connections 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 -- 172.21.15.69:0/4140092394 wait complete. 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 -- start start 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 Processor -- start 2024-08-02T17:46:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ada660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974adaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558974ae03e0 con 0x558974ad2630 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5589749f2970 con 0x558974ad2f50 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558974ae0560 con 0x558974add810 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f38133ee740 10 client.0 _ll_register_callbacks cb 0x5589747467b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f380a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ada660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f380a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ada660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49748/0 (socket says 172.21.15.69:49748) 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f380a59c640 1 -- 172.21.15.69:0/539974210 learned_addr learned my addr 172.21.15.69:0/539974210 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.193+0000 7f3809d9b640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974adaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f380ad9d640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f380a59c640 1 -- 172.21.15.69:0/539974210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 msgr2=0x558974adb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f380a59c640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f380a59c640 1 -- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 msgr2=0x558974adaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f380a59c640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974adaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f380a59c640 1 -- 172.21.15.69:0/539974210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589747a92b0 con 0x558974ad2630 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f380ad9d640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f3809d9b640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974adaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f380a59c640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ada660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f37fc010f50 tx=0x7f37fc09c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc0b3cbebb5bc1d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f37f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f37f1ffb640 1 -- 172.21.15.69:0/539974210 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0a4a70 con 0x558974ad2630 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f37f1ffb640 1 -- 172.21.15.69:0/539974210 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f37fc0cd5f0 con 0x558974ad2630 2024-08-02T17:46:54.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f38133ee740 1 -- 172.21.15.69:0/539974210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3800024070 con 0x558974ad2630 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f37f1ffb640 1 -- 172.21.15.69:0/539974210 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37fc09b250 con 0x558974ad2630 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f38133ee740 10 client.15510 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f38133ee740 1 -- 172.21.15.69:0/539974210 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558974aebf70 con 0x558974ad2630 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.194+0000 7f37f1ffb640 1 -- 172.21.15.69:0/539974210 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3800024070 con 0x558974ad2630 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f37f1ffb640 1 -- 172.21.15.69:0/539974210 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37fc09acc0 con 0x558974ad2630 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f38133ee740 10 client.15510 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f38133ee740 10 client.15510 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f38133ee740 20 client.15510 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f38133ee740 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558974aee4f0 0x558974aef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f38133ee740 4 client.15510 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f38133ee740 1 -- 172.21.15.69:0/539974210 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558974a44c80 con 0x558974aee4f0 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f3809d9b640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558974aee4f0 0x558974aef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f3809d9b640 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558974aee4f0 0x558974aef960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6e330559f7e82a server_cookie=44c3cc3512197374 in_seq=0 out_seq=0 2024-08-02T17:46:54.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.195+0000 7f37f1ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f37f1ffb640 1 -- 172.21.15.69:0/539974210 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558974a44c80 con 0x558974aee4f0 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f37f1ffb640 10 client.15510 handle_command_reply: tid=0 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 client.15510 shutdown 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 -- 172.21.15.69:0/539974210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 msgr2=0x558974ada660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ada660 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f37fc010f50 tx=0x7f37fc09c500 comp rx=0 tx=0).stop 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 -- 172.21.15.69:0/539974210 shutdown_connections 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558974aee4f0 0x558974aef960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974add810 0x558974adb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2f50 0x558974adaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f37f1ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 --2- 172.21.15.69:0/539974210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ada660 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.196+0000 7f38133ee740 1 -- 172.21.15.69:0/539974210 >> 172.21.15.69:0/539974210 conn(0x558974acebc0 msgr2=0x558974acdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.197+0000 7f38133ee740 1 -- 172.21.15.69:0/539974210 shutdown_connections 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.197+0000 7f38133ee740 1 -- 172.21.15.69:0/539974210 wait complete. 2024-08-02T17:46:54.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.197+0000 7f38133ee740 20 client.15510 trim_cache size 0 max 16384 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.214+0000 7f38133ee740 1 Processor -- start 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.214+0000 7f38133ee740 1 -- start start 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.214+0000 7f38133ee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2630 0x558974aef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.214+0000 7f38133ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 0x558974ad9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.214+0000 7f38133ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974add810 0x558974ae8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.214+0000 7f38133ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558974aedca0 con 0x558974add810 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.214+0000 7f38133ee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558974aee370 con 0x558974ad2630 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.214+0000 7f38133ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558974aeea40 con 0x558974ad2f50 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2630 0x558974aef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974add810 0x558974ae8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2630 0x558974aef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50114/0 (socket says 172.21.15.69:50114) 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380a59c640 1 -- 172.21.15.69:0/2461084827 learned_addr learned my addr 172.21.15.69:0/2461084827 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f3809d9b640 1 --2- 172.21.15.69:0/2461084827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 0x558974ad9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380ad9d640 1 -- 172.21.15.69:0/2461084827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 msgr2=0x558974ad9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380ad9d640 1 --2- 172.21.15.69:0/2461084827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 0x558974ad9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380ad9d640 1 -- 172.21.15.69:0/2461084827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2630 msgr2=0x558974aef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380ad9d640 1 --2- 172.21.15.69:0/2461084827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2630 0x558974aef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380ad9d640 1 -- 172.21.15.69:0/2461084827 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558974ad2d80 con 0x558974add810 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f380ad9d640 1 --2- 172.21.15.69:0/2461084827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974add810 0x558974ae8ec0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f380402af40 tx=0x7f380402f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea58cb952f9092 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f37f3fff640 1 -- 172.21.15.69:0/2461084827 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380404a070 con 0x558974add810 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f37f3fff640 1 -- 172.21.15.69:0/2461084827 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380402fe10 con 0x558974add810 2024-08-02T17:46:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.215+0000 7f37f3fff640 1 -- 172.21.15.69:0/2461084827 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3804034040 con 0x558974add810 2024-08-02T17:46:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 -- 172.21.15.69:0/2461084827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974add810 msgr2=0x558974ae8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 --2- 172.21.15.69:0/2461084827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974add810 0x558974ae8ec0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f380402af40 tx=0x7f380402f670 comp rx=0 tx=0).stop 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 -- 172.21.15.69:0/2461084827 shutdown_connections 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 --2- 172.21.15.69:0/2461084827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974add810 0x558974ae8ec0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 --2- 172.21.15.69:0/2461084827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 0x558974ad9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 --2- 172.21.15.69:0/2461084827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974ad2630 0x558974aef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 -- 172.21.15.69:0/2461084827 >> 172.21.15.69:0/2461084827 conn(0x558974acebc0 msgr2=0x558974ae87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 -- 172.21.15.69:0/2461084827 shutdown_connections 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 -- 172.21.15.69:0/2461084827 wait complete. 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 -- start start 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.216+0000 7f38133ee740 1 Processor -- start 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f38133ee740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f38133ee740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 0x558974ad7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f38133ee740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974add810 0x558974af6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f38133ee740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558974aef7e0 con 0x558974ad2630 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f38133ee740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558974aef110 con 0x558974add810 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f38133ee740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558974aefeb0 con 0x558974ad2f50 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f38133ee740 10 client.0 _ll_register_callbacks cb 0x7f37fc09ba10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49766/0 (socket says 172.21.15.69:49766) 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 -- 172.21.15.69:0/1213236910 learned_addr learned my addr 172.21.15.69:0/1213236910 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380ad9d640 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974add810 0x558974af6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f3809d9b640 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 0x558974ad7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 -- 172.21.15.69:0/1213236910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 msgr2=0x558974ad7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 0x558974ad7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 -- 172.21.15.69:0/1213236910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974add810 msgr2=0x558974af6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974add810 0x558974af6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 -- 172.21.15.69:0/1213236910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558974af7160 con 0x558974ad2630 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f380a59c640 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad7050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f37fc09a0b0 tx=0x7f37fc0cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d917333f1ff1be07 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:46:54.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f37f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f37f1ffb640 1 -- 172.21.15.69:0/1213236910 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0a4a70 con 0x558974ad2630 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f37f1ffb640 1 -- 172.21.15.69:0/1213236910 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f37fc010430 con 0x558974ad2630 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.217+0000 7f38133ee740 1 -- 172.21.15.69:0/1213236910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f380402fe10 con 0x558974ad2630 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f37f1ffb640 1 -- 172.21.15.69:0/1213236910 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37fc085120 con 0x558974ad2630 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f38133ee740 10 client.15516 fetch_fsmap learned FSMap version 26 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f38133ee740 1 -- 172.21.15.69:0/1213236910 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558974af8460 con 0x558974ad2630 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f37f1ffb640 1 -- 172.21.15.69:0/1213236910 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f380402fe10 con 0x558974ad2630 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f37f1ffb640 1 -- 172.21.15.69:0/1213236910 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37fc018a40 con 0x558974ad2630 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f38133ee740 10 client.15516 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f38133ee740 10 client.15516 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f38133ee740 20 client.15516 populate_metadata read hostname 'smithi069' 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f38133ee740 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558974b5cfe0 0x558974b5bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f38133ee740 4 client.15516 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f38133ee740 1 -- 172.21.15.69:0/1213236910 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "status"}) -- 0x558974a3f190 con 0x558974b5cfe0 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.218+0000 7f3809d9b640 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558974b5cfe0 0x558974b5bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:46:54.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f3809d9b640 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558974b5cfe0 0x558974b5bcf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1d9cf23de03589 server_cookie=26739491426fcfa7 in_seq=0 out_seq=0 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f37f1ffb640 10 client.15516 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f37f1ffb640 1 -- 172.21.15.69:0/1213236910 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558974a3f190 con 0x558974b5cfe0 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f37f1ffb640 10 client.15516 handle_command_reply: tid=0 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 client.15516 shutdown 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 -- 172.21.15.69:0/1213236910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 msgr2=0x558974ad7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad7050 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f37fc09a0b0 tx=0x7f37fc0cd040 comp rx=0 tx=0).stop 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 -- 172.21.15.69:0/1213236910 shutdown_connections 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558974b5cfe0 0x558974b5bcf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558974add810 0x558974af6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558974ad2f50 0x558974ad7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 --2- 172.21.15.69:0/1213236910 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558974ad2630 0x558974ad7050 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f37f1ffb640 0 client.15516 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 -- 172.21.15.69:0/1213236910 >> 172.21.15.69:0/1213236910 conn(0x558974acebc0 msgr2=0x558974acdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 -- 172.21.15.69:0/1213236910 shutdown_connections 2024-08-02T17:46:54.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.219+0000 7f38133ee740 1 -- 172.21.15.69:0/1213236910 wait complete. 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f38133ee740 20 client.15516 trim_cache size 0 max 16384 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "cluster_fsid": "b6b04634-50f5-11ef-bcca-c7b262605968", 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "whoami": 0, 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "id": 24452, 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "want_state": "up:active", 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "state": "up:active", 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "fs_name": "cephfs", 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "rank_uptime": 169.08939989000001, 2024-08-02T17:46:54.226 INFO:teuthology.orchestra.run.smithi069.stdout: "mdsmap_epoch": 26, 2024-08-02T17:46:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch": 95, 2024-08-02T17:46:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_epoch_barrier": 95, 2024-08-02T17:46:54.227 INFO:teuthology.orchestra.run.smithi069.stdout: "uptime": 179.903963132 2024-08-02T17:46:54.227 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:46:54.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 -- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37f40a2e10 msgr2=0x7f37f40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37f40a2e10 0x7f37f40a52d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f380c06a7f0 tx=0x7f3800009110 comp rx=0 tx=0).stop 2024-08-02T17:46:54.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 -- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c100b50 msgr2=0x7f380c199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c100b50 0x7f380c199090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00e9f0 tx=0x7f37fc00edf0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 -- 172.21.15.69:0/4082387302 shutdown_connections 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37f40a2e10 0x7f37f40a52d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f380c102c20 0x7f380c1a4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f380c1022a0 0x7f380c1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 --2- 172.21.15.69:0/4082387302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f380c100b50 0x7f380c199090 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 -- 172.21.15.69:0/4082387302 >> 172.21.15.69:0/4082387302 conn(0x7f380c0fc500 msgr2=0x7f380c106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 -- 172.21.15.69:0/4082387302 shutdown_connections 2024-08-02T17:46:54.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:46:54.220+0000 7f3810ff0640 1 -- 172.21.15.69:0/4082387302 wait complete. 2024-08-02T17:46:54.955 INFO:teuthology.orchestra.run.smithi069.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-08-02T17:46:54.958 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-02T17:46:54.988 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-08-02T17:46:55.002 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2024-08-02T17:46:55.002 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.002 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T17:46:55.002 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 .. 2024-08-02T17:46:55.002 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 data 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:total 35 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 . 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 .. 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 base 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Aug 2 17:46 global 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 log 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_commit_ts 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_dynshmem 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Aug 2 17:46 pg_hba.conf 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 2 17:46 pg_ident.conf 2024-08-02T17:46:55.003 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 pg_logical 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 pg_multixact 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_notify 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_replslot 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_serial 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_snapshots 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_stat 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_stat_tmp 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 pg_subtrans 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_tblspc 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_twophase 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 PG_VERSION 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 pg_wal 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 pg_xact 2024-08-02T17:46:55.004 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 2 17:46 postgresql.auto.conf 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Aug 2 17:46 postgresql.conf 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base: 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 . 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 1 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 13493 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 13494 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/1: 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-08-02T17:46:55.005 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 . 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 .. 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 112 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 113 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 1247 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1247_fsm 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1247_vm 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 2 17:46 1249 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1249_fsm 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1249_vm 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 2 17:46 1255 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1255_fsm 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1255_vm 2024-08-02T17:46:55.006 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 2 17:46 1259 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1259_fsm 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1259_vm 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 2 17:46 13344 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13344_fsm 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13344_vm 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13346 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13348 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13349_fsm 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349_vm 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13351 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13353 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354 2024-08-02T17:46:55.007 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13354_fsm 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354_vm 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13356 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13358 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13359_fsm 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359_vm 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13361 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13363 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1417 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1418 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 174 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 175 2024-08-02T17:46:55.008 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2187 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2224 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2328 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2336 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2337 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2579 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2600 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2600_fsm 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2600_vm 2024-08-02T17:46:55.009 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2601_fsm 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601_vm 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2602 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2602_fsm 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2602_vm 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2603 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2603_fsm 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2603_vm 2024-08-02T17:46:55.010 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2604 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2605 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2605_fsm 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2605_vm 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2606_fsm 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606_vm 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2607 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2607_fsm 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2607_vm 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 2 17:46 2608 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2608_fsm 2024-08-02T17:46:55.011 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2608_vm 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 2 17:46 2609 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2609_fsm 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2609_vm 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2610 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2610_fsm 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2610_vm 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2611 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2612_fsm 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612_vm 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2613 2024-08-02T17:46:55.012 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2615_fsm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615_vm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2616 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2616_fsm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2616_vm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 2 17:46 2617 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2617_fsm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2617_vm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 2 17:46 2618 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2618_fsm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2618_vm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 2 17:46 2619 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2619_fsm 2024-08-02T17:46:55.013 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2619_vm 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2620 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2650 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2651 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2652 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2653 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2654 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2655 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2656 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2657 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 2 17:46 2658 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 2659 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2660 2024-08-02T17:46:55.014 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2661 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2662 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2663 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2664 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2665 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2666 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2667 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2668 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2669 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2670 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 2 17:46 2673 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 2 17:46 2674 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 2 17:46 2675 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2678 2024-08-02T17:46:55.015 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2679 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2680 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2681 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2682 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2683 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2684 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2685 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2686 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2687 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2688 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2689 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 2 17:46 2690 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 2 17:46 2691 2024-08-02T17:46:55.016 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2692 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2693 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2696 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2699 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2701 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2702 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2703 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2704 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2753 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2753_fsm 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2753_vm 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2754 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2755 2024-08-02T17:46:55.017 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2756 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2757 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2830 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2831 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2832 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2833 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2834 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2835 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2836 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2837 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 2 17:46 2838 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2838_fsm 2024-08-02T17:46:55.018 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2838_vm 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2839 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840_fsm 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2840_vm 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2841 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2995 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2996 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3079_fsm 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079_vm 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3080 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3081 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3085 2024-08-02T17:46:55.019 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3118 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3119 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 2 17:46 3164 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3256 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3257 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3258 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3350 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3351 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3379 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3380 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3381 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3394 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3394_fsm 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3394_vm 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3395 2024-08-02T17:46:55.020 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3429 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3430 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3431 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3433 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3439 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3440 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3455 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 2 17:46 3456 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3456_fsm 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3456_vm 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3466 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3467 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3468 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3501 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3502 2024-08-02T17:46:55.021 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3503 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3534 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3541_fsm 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541_vm 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3542 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3574 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3575 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3576 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3596 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3597 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3598 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3599 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600 2024-08-02T17:46:55.022 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3600_fsm 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600_vm 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3601_fsm 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601_vm 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3602_fsm 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602_vm 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603_fsm 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3603_vm 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3604 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3605 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3606 2024-08-02T17:46:55.023 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3607 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3608 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 3609 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3712 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3764_fsm 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764_vm 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3766 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3767 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3997 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4143 2024-08-02T17:46:55.024 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4144 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4145 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4146 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4147 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4148 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4149 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4150 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4151 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4152 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4153 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4154 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4155 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4156 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4157 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4158 2024-08-02T17:46:55.025 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4159 2024-08-02T17:46:55.026 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4160 2024-08-02T17:46:55.026 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4161 2024-08-02T17:46:55.026 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4162 2024-08-02T17:46:55.026 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4163 2024-08-02T17:46:55.026 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4164 2024-08-02T17:46:55.026 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4165 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4166 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4167 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4168 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4169 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4170 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4171 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4172 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4173 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4174 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 5002 2024-08-02T17:46:55.029 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 548 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 549 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6102 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6104 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6106 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6110 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6111 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6112 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6113 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6117 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 826 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 827 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 828 2024-08-02T17:46:55.030 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 2 17:46 pg_filenode.map 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 PG_VERSION 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 . 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 .. 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 112 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 113 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 1247 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1247_fsm 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1247_vm 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 2 17:46 1249 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1249_fsm 2024-08-02T17:46:55.031 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1249_vm 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 2 17:46 1255 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1255_fsm 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1255_vm 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 2 17:46 1259 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1259_fsm 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1259_vm 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 2 17:46 13344 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13344_fsm 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13344_vm 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13346 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13348 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13349_fsm 2024-08-02T17:46:55.032 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349_vm 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13351 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13353 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13354_fsm 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354_vm 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13356 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13358 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13359_fsm 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359_vm 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13361 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13363 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1417 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1418 2024-08-02T17:46:55.033 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 174 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 175 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2187 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2224 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2328 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2336 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2337 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2579 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2600 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2600_fsm 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2600_vm 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2601_fsm 2024-08-02T17:46:55.034 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601_vm 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2602 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2602_fsm 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2602_vm 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2603 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2603_fsm 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2603_vm 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2604 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2605 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2605_fsm 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2605_vm 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2606_fsm 2024-08-02T17:46:55.035 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606_vm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2607 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2607_fsm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2607_vm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 2 17:46 2608 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2608_fsm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2608_vm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 2 17:46 2609 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2609_fsm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2609_vm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2610 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2610_fsm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2610_vm 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2611 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612 2024-08-02T17:46:55.036 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2612_fsm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612_vm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2613 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2615_fsm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615_vm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2616 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2616_fsm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2616_vm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 2 17:46 2617 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2617_fsm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2617_vm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 2 17:46 2618 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2618_fsm 2024-08-02T17:46:55.037 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2618_vm 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 2 17:46 2619 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2619_fsm 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2619_vm 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2620 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2650 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2651 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2652 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2653 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2654 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2655 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2656 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2657 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 2 17:46 2658 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 2659 2024-08-02T17:46:55.038 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2660 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2661 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2662 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2663 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2664 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2665 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2666 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2667 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2668 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2669 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2670 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 2 17:46 2673 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 2 17:46 2674 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 2 17:46 2675 2024-08-02T17:46:55.039 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2678 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2679 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2680 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2681 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2682 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2683 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2684 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2685 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2686 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2687 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2688 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2689 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 2 17:46 2690 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 2 17:46 2691 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2692 2024-08-02T17:46:55.040 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2693 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2696 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2699 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2701 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2702 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2703 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2704 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2753 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2753_fsm 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2753_vm 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2754 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2755 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2756 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2757 2024-08-02T17:46:55.041 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2830 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2831 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2832 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2833 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2834 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2835 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2836 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2837 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 2 17:46 2838 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2838_fsm 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2838_vm 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2839 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840_fsm 2024-08-02T17:46:55.042 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2840_vm 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2841 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2995 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2996 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3079_fsm 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079_vm 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3080 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3081 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3085 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3118 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3119 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 2 17:46 3164 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3256 2024-08-02T17:46:55.043 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3257 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3258 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3350 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3351 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3379 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3380 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3381 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3394 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3394_fsm 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3394_vm 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3395 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3429 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3430 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3431 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3433 2024-08-02T17:46:55.044 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3439 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3440 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3455 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 2 17:46 3456 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3456_fsm 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3456_vm 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3466 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3467 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3468 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3501 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3502 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3503 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3534 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541 2024-08-02T17:46:55.045 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3541_fsm 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541_vm 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3542 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3574 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3575 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3576 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3596 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3597 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3598 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3599 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3600_fsm 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600_vm 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601 2024-08-02T17:46:55.046 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3601_fsm 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601_vm 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3602_fsm 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602_vm 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603_fsm 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3603_vm 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3604 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3605 2024-08-02T17:46:55.047 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3606 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3607 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3608 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 3609 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3712 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3764_fsm 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764_vm 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3766 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3767 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3997 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4143 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4144 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4145 2024-08-02T17:46:55.048 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4146 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4147 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4148 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4149 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4150 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4151 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4152 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4153 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4154 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4155 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4156 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4157 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4158 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4159 2024-08-02T17:46:55.049 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4160 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4161 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4162 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4163 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4164 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4165 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4166 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4167 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4168 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4169 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4170 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4171 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4172 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4173 2024-08-02T17:46:55.050 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4174 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 5002 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 548 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 549 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6102 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6104 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6106 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6110 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6111 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6112 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6113 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6117 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 826 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 827 2024-08-02T17:46:55.051 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 828 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 2 17:46 pg_filenode.map 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 PG_VERSION 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 . 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 .. 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 112 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 113 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 1247 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1247_fsm 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1247_vm 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 2 17:46 1249 2024-08-02T17:46:55.052 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1249_fsm 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1249_vm 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 2 17:46 1255 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1255_fsm 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1255_vm 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 2 17:46 1259 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1259_fsm 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1259_vm 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 2 17:46 13344 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13344_fsm 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13344_vm 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13346 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13348 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349 2024-08-02T17:46:55.053 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13349_fsm 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349_vm 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13351 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13353 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13354_fsm 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354_vm 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13356 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13358 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13359_fsm 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359_vm 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13361 2024-08-02T17:46:55.054 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13363 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1417 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1418 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 174 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 175 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2187 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2224 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2328 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2336 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2337 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2579 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2600 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2600_fsm 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2600_vm 2024-08-02T17:46:55.055 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2601_fsm 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601_vm 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2602 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2602_fsm 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2602_vm 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2603 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2603_fsm 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2603_vm 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2604 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2605 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2605_fsm 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2605_vm 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606 2024-08-02T17:46:55.056 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2606_fsm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606_vm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2607 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2607_fsm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2607_vm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 2 17:46 2608 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2608_fsm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2608_vm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 2 17:46 2609 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2609_fsm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2609_vm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2610 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2610_fsm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2610_vm 2024-08-02T17:46:55.057 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2611 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2612_fsm 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612_vm 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2613 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2615_fsm 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615_vm 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2616 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2616_fsm 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2616_vm 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 2 17:46 2617 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2617_fsm 2024-08-02T17:46:55.058 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2617_vm 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 2 17:46 2618 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2618_fsm 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2618_vm 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 2 17:46 2619 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2619_fsm 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2619_vm 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2620 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2650 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2651 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2652 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2653 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2654 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2655 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2656 2024-08-02T17:46:55.059 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2657 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 2 17:46 2658 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 2659 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2660 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2661 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2662 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2663 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2664 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2665 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2666 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2667 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2668 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2669 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2670 2024-08-02T17:46:55.060 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 2 17:46 2673 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 2 17:46 2674 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 2 17:46 2675 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2678 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2679 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2680 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2681 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2682 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2683 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2684 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2685 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2686 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2687 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2688 2024-08-02T17:46:55.061 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2689 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 2 17:46 2690 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 2 17:46 2691 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2692 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2693 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2696 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2699 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2701 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2702 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2703 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2704 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2753 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2753_fsm 2024-08-02T17:46:55.062 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2753_vm 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2754 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2755 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2756 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2757 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2830 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2831 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2832 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2833 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2834 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2835 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2836 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2837 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 2 17:46 2838 2024-08-02T17:46:55.063 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2838_fsm 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2838_vm 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2839 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840_fsm 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2840_vm 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2841 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2995 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2996 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3079_fsm 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079_vm 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3080 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3081 2024-08-02T17:46:55.064 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3085 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3118 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3119 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 2 17:46 3164 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3256 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3257 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3258 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3350 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3351 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3379 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3380 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3381 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3394 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3394_fsm 2024-08-02T17:46:55.065 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3394_vm 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3395 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3429 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3430 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3431 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3433 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3439 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3440 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3455 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 2 17:46 3456 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3456_fsm 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3456_vm 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3466 2024-08-02T17:46:55.066 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3467 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3468 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3501 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3502 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3503 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3534 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3541_fsm 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541_vm 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3542 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3574 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3575 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3576 2024-08-02T17:46:55.067 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3596 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3597 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3598 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3599 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3600_fsm 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600_vm 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3601_fsm 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601_vm 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3602_fsm 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602_vm 2024-08-02T17:46:55.068 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603_fsm 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3603_vm 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3604 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3605 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3606 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3607 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3608 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 3609 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3712 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3764_fsm 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764_vm 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3766 2024-08-02T17:46:55.069 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3767 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3997 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4143 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4144 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4145 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4146 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4147 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4148 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4149 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4150 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4151 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4152 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4153 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4154 2024-08-02T17:46:55.070 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4155 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4156 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4157 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4158 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4159 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4160 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4161 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4162 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4163 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4164 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4165 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4166 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4167 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4168 2024-08-02T17:46:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4169 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4170 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4171 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4172 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4173 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4174 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 5002 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 548 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 549 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6102 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6104 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6106 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6110 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6111 2024-08-02T17:46:55.072 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6112 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6113 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6117 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 826 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 827 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 828 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 2 17:46 pg_filenode.map 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 PG_VERSION 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/global: 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:total 537 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Aug 2 17:46 . 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1213 2024-08-02T17:46:55.073 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1213_fsm 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1213_vm 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1214 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1214_fsm 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1214_vm 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 1232 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 1233 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1260 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1260_fsm 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1260_vm 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1261 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1261_fsm 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1261_vm 2024-08-02T17:46:55.074 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1262 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1262_fsm 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1262_vm 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2396 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2396_fsm 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2396_vm 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2397 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2671 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2672 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2676 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2677 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2694 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2695 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2697 2024-08-02T17:46:55.075 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2698 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2846 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2847 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2964 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2965 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2966 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2967 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3592 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3593 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4060 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4061 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4175 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4176 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4177 2024-08-02T17:46:55.076 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4178 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4181 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4182 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4183 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4184 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4185 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4186 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6000 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6001 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6002 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6100 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6114 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6115 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 pg_control 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 2 17:46 pg_filenode.map 2024-08-02T17:46:55.077 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/log: 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.078 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:total 1 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 . 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 mappings 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 2 17:46 replorigin_checkpoint 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 snapshots 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 .. 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.079 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 .. 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 . 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 members 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 offsets 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-08-02T17:46:55.080 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 .. 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 0000 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 .. 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 0000 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.081 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.082 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T17:46:55.083 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 0000 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-08-02T17:46:55.084 INFO:teuthology.orchestra.run.smithi069.stdout:total 16384 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 . 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:46 000000010000000000000001 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 archive_status 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 .. 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T17:46:55.085 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Aug 2 17:46 .. 2024-08-02T17:46:55.086 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 0000 2024-08-02T17:46:55.086 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-08-02T17:46:55.223 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:55 smithi069 ceph-mon[32746]: pgmap v18: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-08-02T17:46:55.226 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-08-02T17:46:55.271 INFO:teuthology.orchestra.run.smithi069.stderr:dropping old tables... 2024-08-02T17:46:55.271 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-08-02T17:46:55.271 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-08-02T17:46:55.271 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-08-02T17:46:55.271 INFO:teuthology.orchestra.run.smithi069.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-08-02T17:46:55.271 INFO:teuthology.orchestra.run.smithi069.stderr:creating tables... 2024-08-02T17:46:55.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:55 smithi083 ceph-mon[37031]: pgmap v18: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-08-02T17:46:55.282 INFO:teuthology.orchestra.run.smithi069.stderr:generating data (client-side)... 2024-08-02T17:46:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:55 smithi077 ceph-mon[37108]: pgmap v18: 129 pgs: 129 active+clean; 1.4 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-08-02T17:46:55.369 INFO:teuthology.orchestra.run.smithi069.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.07 s, remaining 2.20 s) 2024-08-02T17:46:55.521 INFO:teuthology.orchestra.run.smithi069.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.22 s, remaining 3.35 s) 2024-08-02T17:46:55.607 INFO:teuthology.orchestra.run.smithi069.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.31 s, remaining 2.99 s) 2024-08-02T17:46:55.752 INFO:teuthology.orchestra.run.smithi069.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.45 s, remaining 3.18 s) 2024-08-02T17:46:55.902 INFO:teuthology.orchestra.run.smithi069.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.60 s, remaining 3.26 s) 2024-08-02T17:46:55.984 INFO:teuthology.orchestra.run.smithi069.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.69 s, remaining 2.97 s) 2024-08-02T17:46:56.118 INFO:teuthology.orchestra.run.smithi069.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.82 s, remaining 2.93 s) 2024-08-02T17:46:56.278 INFO:teuthology.orchestra.run.smithi069.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.98 s, remaining 2.94 s) 2024-08-02T17:46:56.370 INFO:teuthology.orchestra.run.smithi069.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.07 s, remaining 2.74 s) 2024-08-02T17:46:56.499 INFO:teuthology.orchestra.run.smithi069.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.20 s, remaining 2.64 s) 2024-08-02T17:46:56.663 INFO:teuthology.orchestra.run.smithi069.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.37 s, remaining 2.61 s) 2024-08-02T17:46:56.750 INFO:teuthology.orchestra.run.smithi069.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.45 s, remaining 2.42 s) 2024-08-02T17:46:56.876 INFO:teuthology.orchestra.run.smithi069.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.58 s, remaining 2.31 s) 2024-08-02T17:46:57.053 INFO:teuthology.orchestra.run.smithi069.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.76 s, remaining 2.26 s) 2024-08-02T17:46:57.112 INFO:teuthology.orchestra.run.smithi069.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.81 s, remaining 2.06 s) 2024-08-02T17:46:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:57 smithi083 ceph-mon[37031]: pgmap v19: 129 pgs: 129 active+clean; 93 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 12 MiB/s wr, 123 op/s 2024-08-02T17:46:57.273 INFO:teuthology.orchestra.run.smithi069.stderr:1600000 of 3200000 tuples (50%) done (elapsed 1.97 s, remaining 1.97 s) 2024-08-02T17:46:57.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:57 smithi077 ceph-mon[37108]: pgmap v19: 129 pgs: 129 active+clean; 93 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 12 MiB/s wr, 123 op/s 2024-08-02T17:46:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:57 smithi069 ceph-mon[32746]: pgmap v19: 129 pgs: 129 active+clean; 93 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 12 MiB/s wr, 123 op/s 2024-08-02T17:46:57.414 INFO:teuthology.orchestra.run.smithi069.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.12 s, remaining 1.87 s) 2024-08-02T17:46:57.510 INFO:teuthology.orchestra.run.smithi069.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.21 s, remaining 1.72 s) 2024-08-02T17:46:57.660 INFO:teuthology.orchestra.run.smithi069.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.36 s, remaining 1.62 s) 2024-08-02T17:46:57.760 INFO:teuthology.orchestra.run.smithi069.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.46 s, remaining 1.48 s) 2024-08-02T17:46:57.905 INFO:teuthology.orchestra.run.smithi069.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.61 s, remaining 1.37 s) 2024-08-02T17:46:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.78 s, remaining 1.26 s) 2024-08-02T17:46:58.164 INFO:teuthology.orchestra.run.smithi069.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.87 s, remaining 1.12 s) 2024-08-02T17:46:58.303 INFO:teuthology.orchestra.run.smithi069.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.00 s, remaining 1.00 s) 2024-08-02T17:46:58.471 INFO:teuthology.orchestra.run.smithi069.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.17 s, remaining 0.89 s) 2024-08-02T17:46:58.562 INFO:teuthology.orchestra.run.smithi069.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.26 s, remaining 0.75 s) 2024-08-02T17:46:58.700 INFO:teuthology.orchestra.run.smithi069.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.40 s, remaining 0.63 s) 2024-08-02T17:46:58.808 INFO:teuthology.orchestra.run.smithi069.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.51 s, remaining 0.50 s) 2024-08-02T17:46:58.963 INFO:teuthology.orchestra.run.smithi069.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.67 s, remaining 0.38 s) 2024-08-02T17:46:59.092 INFO:teuthology.orchestra.run.smithi069.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.79 s, remaining 0.25 s) 2024-08-02T17:46:59.191 INFO:teuthology.orchestra.run.smithi069.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.89 s, remaining 0.13 s) 2024-08-02T17:46:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:46:59 smithi083 ceph-mon[37031]: pgmap v20: 129 pgs: 129 active+clean; 93 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 12 MiB/s wr, 121 op/s 2024-08-02T17:46:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.01 s, remaining 0.00 s) 2024-08-02T17:46:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:46:59 smithi077 ceph-mon[37108]: pgmap v20: 129 pgs: 129 active+clean; 93 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 12 MiB/s wr, 121 op/s 2024-08-02T17:46:59.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:46:59 smithi069 ceph-mon[32746]: pgmap v20: 129 pgs: 129 active+clean; 93 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 12 MiB/s wr, 121 op/s 2024-08-02T17:46:59.358 INFO:teuthology.orchestra.run.smithi069.stderr:vacuuming... 2024-08-02T17:46:59.942 INFO:teuthology.orchestra.run.smithi069.stderr:creating primary keys... 2024-08-02T17:47:00.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:47:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:47:00.100+0000 7fbab16ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:47:00.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:47:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:47:00.101+0000 7fbab16ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:47:00.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:47:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:47:00.101+0000 7fbab16ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:47:00.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:47:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:47:00.101+0000 7fbab16ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:47:00.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 17:47:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T17:47:00.101+0000 7fbab16ac640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-02T17:47:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:01 smithi083 ceph-mon[37031]: pgmap v21: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 60 MiB/s wr, 207 op/s 2024-08-02T17:47:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:01 smithi083 ceph-mon[37031]: mgrmap e24: x(active, since 36s), standbys: z, y 2024-08-02T17:47:02.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:01 smithi077 ceph-mon[37108]: pgmap v21: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 60 MiB/s wr, 207 op/s 2024-08-02T17:47:02.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:01 smithi077 ceph-mon[37108]: mgrmap e24: x(active, since 36s), standbys: z, y 2024-08-02T17:47:02.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:01 smithi069 ceph-mon[32746]: pgmap v21: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 60 MiB/s wr, 207 op/s 2024-08-02T17:47:02.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:01 smithi069 ceph-mon[32746]: mgrmap e24: x(active, since 36s), standbys: z, y 2024-08-02T17:47:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 -- 172.21.15.69:0/738970015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2228106730 msgr2=0x7f2228106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 --2- 172.21.15.69:0/738970015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2228106730 0x7f2228106b10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f221c00d960 tx=0x7f221c02c2b0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 -- 172.21.15.69:0/738970015 shutdown_connections 2024-08-02T17:47:03.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 --2- 172.21.15.69:0/738970015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2228100720 0x7f2228100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 --2- 172.21.15.69:0/738970015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22281010c0 0x7f222810d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 --2- 172.21.15.69:0/738970015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2228106730 0x7f2228106b10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 -- 172.21.15.69:0/738970015 >> 172.21.15.69:0/738970015 conn(0x7f22280fc420 msgr2=0x7f22280fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 -- 172.21.15.69:0/738970015 shutdown_connections 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 -- 172.21.15.69:0/738970015 wait complete. 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.153+0000 7f222ed92640 1 Processor -- start 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222ed92640 1 -- start start 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222ed92640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2228100720 0x7f222819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222ed92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22281010c0 0x7f222819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222ed92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2228106730 0x7f222819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222ed92640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f222810fa70 con 0x7f2228100720 2024-08-02T17:47:03.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222ed92640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f222810f8f0 con 0x7f22281010c0 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222ed92640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f222810fbf0 con 0x7f2228106730 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222cb2e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2228100720 0x7f222819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222cb2e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2228100720 0x7f222819a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53818/0 (socket says 172.21.15.69:53818) 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222cb2e640 1 -- 172.21.15.69:0/974258254 learned_addr learned my addr 172.21.15.69:0/974258254 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222d32f640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2228106730 0x7f222819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f2227fff640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22281010c0 0x7f222819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222cb2e640 1 -- 172.21.15.69:0/974258254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2228106730 msgr2=0x7f222819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222cb2e640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2228106730 0x7f222819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222cb2e640 1 -- 172.21.15.69:0/974258254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22281010c0 msgr2=0x7f222819a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222cb2e640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22281010c0 0x7f222819a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.154+0000 7f222cb2e640 1 -- 172.21.15.69:0/974258254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222819f2c0 con 0x7f2228100720 2024-08-02T17:47:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.155+0000 7f222cb2e640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2228100720 0x7f222819a2d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f221c033000 tx=0x7f221c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.155+0000 7f2225ffb640 1 -- 172.21.15.69:0/974258254 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c042070 con 0x7f2228100720 2024-08-02T17:47:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.155+0000 7f222ed92640 1 -- 172.21.15.69:0/974258254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f222819f550 con 0x7f2228100720 2024-08-02T17:47:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.155+0000 7f2225ffb640 1 -- 172.21.15.69:0/974258254 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f221c0054d0 con 0x7f2228100720 2024-08-02T17:47:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.155+0000 7f222ed92640 1 -- 172.21.15.69:0/974258254 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22281a6e40 con 0x7f2228100720 2024-08-02T17:47:03.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.155+0000 7f2225ffb640 1 -- 172.21.15.69:0/974258254 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221c039070 con 0x7f2228100720 2024-08-02T17:47:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.158+0000 7f2225ffb640 1 -- 172.21.15.69:0/974258254 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f221c045070 con 0x7f2228100720 2024-08-02T17:47:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.158+0000 7f2231bc1740 1 Processor -- start 2024-08-02T17:47:03.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f2231bc1740 1 -- start start 2024-08-02T17:47:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f39b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f3a0d0 0x55ce34f44430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 0x55ce34f46d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f2231bc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ce34e9d9b0 con 0x55ce34f3a0d0 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f2231bc1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ce34e45800 con 0x55ce34f44970 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f2231bc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ce34e99970 con 0x55ce34f397b0 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222d32f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f39b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222d32f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f39b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:58320/0 (socket says 172.21.15.69:58320) 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222d32f640 1 -- 172.21.15.69:0/3753341937 learned_addr learned my addr 172.21.15.69:0/3753341937 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f2227fff640 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 0x55ce34f46d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222cb2e640 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f3a0d0 0x55ce34f44430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222d32f640 1 -- 172.21.15.69:0/3753341937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 msgr2=0x55ce34f46d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222d32f640 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 0x55ce34f46d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222d32f640 1 -- 172.21.15.69:0/3753341937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f3a0d0 msgr2=0x55ce34f44430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222d32f640 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f3a0d0 0x55ce34f44430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222d32f640 1 -- 172.21.15.69:0/3753341937 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce34bb1210 con 0x55ce34f397b0 2024-08-02T17:47:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.159+0000 7f222cb2e640 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f3a0d0 0x55ce34f44430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.160+0000 7f2225ffb640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22180a2ea0 0x7f22180a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.160+0000 7f222d32f640 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f39b90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2220009f50 tx=0x7f22200118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3953f1be86f95c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.160+0000 7f222cb2e640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22180a2ea0 0x7f22180a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.160+0000 7f2212ffd640 1 -- 172.21.15.69:0/3753341937 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220004b20 con 0x55ce34f397b0 2024-08-02T17:47:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.160+0000 7f2212ffd640 1 -- 172.21.15.69:0/3753341937 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2220004cc0 con 0x55ce34f397b0 2024-08-02T17:47:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.160+0000 7f2212ffd640 1 -- 172.21.15.69:0/3753341937 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220005020 con 0x55ce34f397b0 2024-08-02T17:47:03.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.160+0000 7f2225ffb640 1 -- 172.21.15.69:0/974258254 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f221c0ef1f0 con 0x7f2228100720 2024-08-02T17:47:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 -- 172.21.15.69:0/3753341937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 msgr2=0x55ce34f39b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f222cb2e640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22180a2ea0 0x7f22180a5360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f221c0ec4b0 tx=0x7f221c0f1040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f39b90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2220009f50 tx=0x7f22200118d0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 -- 172.21.15.69:0/3753341937 shutdown_connections 2024-08-02T17:47:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f3a0d0 0x55ce34f44430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 0x55ce34f46d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 --2- 172.21.15.69:0/3753341937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f39b90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 -- 172.21.15.69:0/3753341937 >> 172.21.15.69:0/3753341937 conn(0x55ce34f35d20 msgr2=0x55ce34f36100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 -- 172.21.15.69:0/3753341937 shutdown_connections 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 -- 172.21.15.69:0/3753341937 wait complete. 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 -- start start 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.161+0000 7f2231bc1740 1 Processor -- start 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f397b0 0x55ce34f41830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f3a0d0 0x55ce34f41d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 0x55ce34f422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f2231bc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ce34f4fd50 con 0x55ce34f397b0 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f2231bc1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ce34e59500 con 0x55ce34f44970 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f2231bc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ce34f4fed0 con 0x55ce34f3a0d0 2024-08-02T17:47:03.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f2231bc1740 10 client.0 _ll_register_callbacks cb 0x55ce34bb0a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f222d32f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f397b0 0x55ce34f41830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f222d32f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f397b0 0x55ce34f41830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53844/0 (socket says 172.21.15.69:53844) 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f222d32f640 1 -- 172.21.15.69:0/2225953991 learned_addr learned my addr 172.21.15.69:0/2225953991 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f222d32f640 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 0x55ce34f422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.162+0000 7f2227fff640 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f3a0d0 0x55ce34f41d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f222d32f640 1 -- 172.21.15.69:0/2225953991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f3a0d0 msgr2=0x55ce34f41d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f222d32f640 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f3a0d0 0x55ce34f41d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f222d32f640 1 -- 172.21.15.69:0/2225953991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 msgr2=0x55ce34f422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f222d32f640 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 0x55ce34f422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f222d32f640 1 -- 172.21.15.69:0/2225953991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce34c7d620 con 0x55ce34f397b0 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f222d32f640 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f397b0 0x55ce34f41830 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2220012fd0 tx=0x7f22200044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a8bd5c19b09161 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f2207fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f2207fff640 1 -- 172.21.15.69:0/2225953991 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222002c020 con 0x55ce34f397b0 2024-08-02T17:47:03.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f2207fff640 1 -- 172.21.15.69:0/2225953991 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2220016040 con 0x55ce34f397b0 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f2231bc1740 1 -- 172.21.15.69:0/2225953991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2220004cc0 con 0x55ce34f397b0 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f2207fff640 1 -- 172.21.15.69:0/2225953991 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2220003c40 con 0x55ce34f397b0 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f2231bc1740 10 client.15531 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.163+0000 7f2231bc1740 1 -- 172.21.15.69:0/2225953991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce34f5b900 con 0x55ce34f397b0 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2207fff640 1 -- 172.21.15.69:0/2225953991 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2220004cc0 con 0x55ce34f397b0 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2207fff640 1 -- 172.21.15.69:0/2225953991 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f22200274d0 con 0x55ce34f397b0 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2231bc1740 10 client.15531 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2231bc1740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2231bc1740 20 client.15531 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2231bc1740 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce34f5cd10 0x55ce34f5f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2231bc1740 4 client.15531 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2231bc1740 1 -- 172.21.15.69:0/2225953991 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce34eab0e0 con 0x55ce34f5cd10 2024-08-02T17:47:03.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.164+0000 7f2227fff640 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce34f5cd10 0x55ce34f5f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.165+0000 7f2207fff640 10 client.15531 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:03.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.165+0000 7f2227fff640 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce34f5cd10 0x55ce34f5f100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc0bc76e02785196 server_cookie=c0c055772357ef9a in_seq=0 out_seq=0 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.165+0000 7f2207fff640 1 -- 172.21.15.69:0/2225953991 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ce34eab0e0 con 0x55ce34f5cd10 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.165+0000 7f2207fff640 10 client.15531 handle_command_reply: tid=0 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.165+0000 7f2231bc1740 1 client.15531 shutdown 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 -- 172.21.15.69:0/2225953991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f397b0 msgr2=0x55ce34f41830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f397b0 0x55ce34f41830 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2220012fd0 tx=0x7f22200044d0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 -- 172.21.15.69:0/2225953991 shutdown_connections 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce34f5cd10 0x55ce34f5f100 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f44970 0x55ce34f422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f3a0d0 0x55ce34f41d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 --2- 172.21.15.69:0/2225953991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f397b0 0x55ce34f41830 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 -- 172.21.15.69:0/2225953991 >> 172.21.15.69:0/2225953991 conn(0x55ce34f35d20 msgr2=0x55ce34f3ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2207fff640 0 client.15531 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 -- 172.21.15.69:0/2225953991 shutdown_connections 2024-08-02T17:47:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 1 -- 172.21.15.69:0/2225953991 wait complete. 2024-08-02T17:47:03.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.166+0000 7f2231bc1740 20 client.15531 trim_cache size 0 max 16384 2024-08-02T17:47:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2231bc1740 1 Processor -- start 2024-08-02T17:47:03.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2231bc1740 1 -- start start 2024-08-02T17:47:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f64d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 0x55ce34f34660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2231bc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22281af510 con 0x55ce34f44970 2024-08-02T17:47:03.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2231bc1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22281acde0 con 0x55ce34f3a0d0 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2231bc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22281aac50 con 0x55ce34f397b0 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f222d32f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 0x55ce34f34660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2227fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f64d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2227fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f64d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44036/0 (socket says 172.21.15.69:44036) 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.192+0000 7f2227fff640 1 -- 172.21.15.69:0/3159356335 learned_addr learned my addr 172.21.15.69:0/3159356335 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.193+0000 7f222d32f640 1 --2- 172.21.15.69:0/3159356335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.193+0000 7f2227fff640 1 -- 172.21.15.69:0/3159356335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 msgr2=0x55ce34f647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.193+0000 7f2227fff640 1 --2- 172.21.15.69:0/3159356335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.193+0000 7f2227fff640 1 -- 172.21.15.69:0/3159356335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 msgr2=0x55ce34f34660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.193+0000 7f2227fff640 1 --2- 172.21.15.69:0/3159356335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 0x55ce34f34660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.193+0000 7f2227fff640 1 -- 172.21.15.69:0/3159356335 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce34f2fe80 con 0x55ce34f3a0d0 2024-08-02T17:47:03.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.193+0000 7f2227fff640 1 --2- 172.21.15.69:0/3159356335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f64d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55ce34f3c350 tx=0x7f2214009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de6b40c2ad27f601 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.194+0000 7f22127fc640 1 -- 172.21.15.69:0/3159356335 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214003e40 con 0x55ce34f3a0d0 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.194+0000 7f22127fc640 1 -- 172.21.15.69:0/3159356335 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2214005500 con 0x55ce34f3a0d0 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.194+0000 7f22127fc640 1 -- 172.21.15.69:0/3159356335 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22140266c0 con 0x55ce34f3a0d0 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 -- 172.21.15.69:0/3159356335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 msgr2=0x55ce34f64d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 --2- 172.21.15.69:0/3159356335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f64d30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55ce34f3c350 tx=0x7f2214009260 comp rx=0 tx=0).stop 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 -- 172.21.15.69:0/3159356335 shutdown_connections 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 --2- 172.21.15.69:0/3159356335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 0x55ce34f34660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 --2- 172.21.15.69:0/3159356335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f64d30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 --2- 172.21.15.69:0/3159356335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 -- 172.21.15.69:0/3159356335 >> 172.21.15.69:0/3159356335 conn(0x55ce34f35d20 msgr2=0x55ce34f40900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 -- 172.21.15.69:0/3159356335 shutdown_connections 2024-08-02T17:47:03.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 -- 172.21.15.69:0/3159356335 wait complete. 2024-08-02T17:47:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 -- start start 2024-08-02T17:47:03.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.195+0000 7f2231bc1740 1 Processor -- start 2024-08-02T17:47:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f55900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2231bc1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 0x55ce34f3e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2231bc1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22281ab2a0 con 0x55ce34f44970 2024-08-02T17:47:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2231bc1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f22281ac710 con 0x55ce34f3a0d0 2024-08-02T17:47:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2231bc1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22281ab970 con 0x55ce34f397b0 2024-08-02T17:47:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2231bc1740 10 client.0 _ll_register_callbacks cb 0x7f22140097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2227fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f55900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f222d32f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 0x55ce34f3e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2227fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f55900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44044/0 (socket says 172.21.15.69:44044) 2024-08-02T17:47:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f2227fff640 1 -- 172.21.15.69:0/610154 learned_addr learned my addr 172.21.15.69:0/610154 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.196+0000 7f222d32f640 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2227fff640 1 -- 172.21.15.69:0/610154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 msgr2=0x55ce34f553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2227fff640 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2227fff640 1 -- 172.21.15.69:0/610154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 msgr2=0x55ce34f3e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2227fff640 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 0x55ce34f3e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2227fff640 1 -- 172.21.15.69:0/610154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce34f3e8d0 con 0x55ce34f3a0d0 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2227fff640 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f55900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2214005ec0 tx=0x7f2214004350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed975b2f216d2e6e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2207fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2207fff640 1 -- 172.21.15.69:0/610154 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214004930 con 0x55ce34f3a0d0 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2207fff640 1 -- 172.21.15.69:0/610154 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2214004ad0 con 0x55ce34f3a0d0 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.197+0000 7f2231bc1740 1 -- 172.21.15.69:0/610154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2214005500 con 0x55ce34f3a0d0 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.198+0000 7f2207fff640 1 -- 172.21.15.69:0/610154 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2214038050 con 0x55ce34f3a0d0 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.198+0000 7f2231bc1740 10 client.25208 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.198+0000 7f2231bc1740 1 -- 172.21.15.69:0/610154 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce34fc7310 con 0x55ce34f3a0d0 2024-08-02T17:47:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.198+0000 7f2207fff640 1 -- 172.21.15.69:0/610154 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22140052e0 con 0x55ce34f3a0d0 2024-08-02T17:47:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.198+0000 7f2207fff640 1 -- 172.21.15.69:0/610154 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2214026b90 con 0x55ce34f3a0d0 2024-08-02T17:47:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f2231bc1740 10 client.25208 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f2231bc1740 10 client.25208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f2231bc1740 20 client.25208 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f2231bc1740 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce34fc7930 0x55ce34fc9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f2231bc1740 4 client.25208 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:47:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f2231bc1740 1 -- 172.21.15.69:0/610154 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2214005500 con 0x55ce34fc7930 2024-08-02T17:47:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f222d32f640 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce34fc7930 0x55ce34fc9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f222d32f640 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce34fc7930 0x55ce34fc9d50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd23e011a80b1c78 server_cookie=49c09d7ae244c8ce in_seq=0 out_seq=0 2024-08-02T17:47:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.199+0000 7f2207fff640 10 client.25208 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2207fff640 1 -- 172.21.15.69:0/610154 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f2214005500 con 0x55ce34fc7930 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2207fff640 10 client.25208 handle_command_reply: tid=0 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 client.25208 shutdown 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 -- 172.21.15.69:0/610154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 msgr2=0x55ce34f55900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f55900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2214005ec0 tx=0x7f2214004350 comp rx=0 tx=0).stop 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 -- 172.21.15.69:0/610154 shutdown_connections 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce34f44970 0x55ce34f3e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce34fc7930 0x55ce34fc9d50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce34f3a0d0 0x55ce34f55900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2207fff640 0 client.25208 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 --2- 172.21.15.69:0/610154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce34f397b0 0x55ce34f553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 -- 172.21.15.69:0/610154 >> 172.21.15.69:0/610154 conn(0x55ce34f35d20 msgr2=0x55ce34f34660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 -- 172.21.15.69:0/610154 shutdown_connections 2024-08-02T17:47:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 1 -- 172.21.15.69:0/610154 wait complete. 2024-08-02T17:47:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.551+0000 7f2231bc1740 20 client.25208 trim_cache size 0 max 16384 2024-08-02T17:47:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:47:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:47:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:47:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:47:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 -- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22180a2ea0 msgr2=0x7f22180a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22180a2ea0 0x7f22180a5360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f221c0ec4b0 tx=0x7f221c0f1040 comp rx=0 tx=0).stop 2024-08-02T17:47:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 -- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2228100720 msgr2=0x7f222819a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2228100720 0x7f222819a2d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f221c033000 tx=0x7f221c032a10 comp rx=0 tx=0).stop 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 -- 172.21.15.69:0/974258254 shutdown_connections 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22180a2ea0 0x7f22180a5360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2228106730 0x7f222819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f22281010c0 0x7f222819a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 --2- 172.21.15.69:0/974258254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2228100720 0x7f222819a2d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 -- 172.21.15.69:0/974258254 >> 172.21.15.69:0/974258254 conn(0x7f22280fc420 msgr2=0x7f22280fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 -- 172.21.15.69:0/974258254 shutdown_connections 2024-08-02T17:47:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:03.552+0000 7f222ed92640 1 -- 172.21.15.69:0/974258254 wait complete. 2024-08-02T17:47:04.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:03 smithi083 ceph-mon[37031]: pgmap v22: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 59 MiB/s wr, 198 op/s 2024-08-02T17:47:04.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:03 smithi077 ceph-mon[37108]: pgmap v22: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 59 MiB/s wr, 198 op/s 2024-08-02T17:47:04.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:03 smithi069 ceph-mon[32746]: pgmap v22: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 59 MiB/s wr, 198 op/s 2024-08-02T17:47:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:05 smithi077 ceph-mon[37108]: pgmap v23: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 59 MiB/s wr, 198 op/s 2024-08-02T17:47:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:05 smithi069 ceph-mon[32746]: pgmap v23: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 59 MiB/s wr, 198 op/s 2024-08-02T17:47:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:05 smithi083 ceph-mon[37031]: pgmap v23: 129 pgs: 129 active+clean; 383 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 59 MiB/s wr, 198 op/s 2024-08-02T17:47:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:07 smithi083 ceph-mon[37031]: pgmap v24: 129 pgs: 129 active+clean; 392 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 61 MiB/s wr, 743 op/s 2024-08-02T17:47:07.289 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:47:07.316 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:07 smithi069 ceph-mon[32746]: pgmap v24: 129 pgs: 129 active+clean; 392 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 61 MiB/s wr, 743 op/s 2024-08-02T17:47:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:07 smithi077 ceph-mon[37108]: pgmap v24: 129 pgs: 129 active+clean; 392 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 61 MiB/s wr, 743 op/s 2024-08-02T17:47:07.631 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:08.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 -- 172.21.15.69:0/3714622939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530109b60 msgr2=0x7f3530109f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 --2- 172.21.15.69:0/3714622939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530109b60 0x7f3530109f40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f352c009930 tx=0x7f352c02a420 comp rx=0 tx=0).stop 2024-08-02T17:47:08.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 -- 172.21.15.69:0/3714622939 shutdown_connections 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 --2- 172.21.15.69:0/3714622939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3530104470 0x7f35301109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 --2- 172.21.15.69:0/3714622939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3530103ad0 0x7f3530103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 --2- 172.21.15.69:0/3714622939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530109b60 0x7f3530109f40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 -- 172.21.15.69:0/3714622939 >> 172.21.15.69:0/3714622939 conn(0x7f35300ff750 msgr2=0x7f3530101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 -- 172.21.15.69:0/3714622939 shutdown_connections 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.518+0000 7f35384bf640 1 -- 172.21.15.69:0/3714622939 wait complete. 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.519+0000 7f35384bf640 1 Processor -- start 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.519+0000 7f35384bf640 1 -- start start 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.519+0000 7f35384bf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3530103ad0 0x7f353019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.519+0000 7f35384bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3530104470 0x7f353019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f35384bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530109b60 0x7f353019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f35384bf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3530113ee0 con 0x7f3530109b60 2024-08-02T17:47:08.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f35384bf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3530113d60 con 0x7f3530103ad0 2024-08-02T17:47:08.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f35384bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3530114060 con 0x7f3530104470 2024-08-02T17:47:08.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f353625b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3530103ad0 0x7f353019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f353625b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3530103ad0 0x7f353019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45328/0 (socket says 172.21.15.69:45328) 2024-08-02T17:47:08.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f353625b640 1 -- 172.21.15.69:0/1216487435 learned_addr learned my addr 172.21.15.69:0/1216487435 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:08.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530109b60 0x7f353019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f3535a5a640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3530104470 0x7f353019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f3535a5a640 1 -- 172.21.15.69:0/1216487435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3530103ad0 msgr2=0x7f353019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f3535a5a640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3530103ad0 0x7f353019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f3535a5a640 1 -- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530109b60 msgr2=0x7f353019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f3535a5a640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530109b60 0x7f353019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f3535a5a640 1 -- 172.21.15.69:0/1216487435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353019f3f0 con 0x7f3530104470 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.520+0000 7f353625b640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3530103ad0 0x7f353019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.521+0000 7f3535a5a640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3530104470 0x7f353019a980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f352000e9f0 tx=0x7f352000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.521+0000 7f35277fe640 1 -- 172.21.15.69:0/1216487435 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352000cd80 con 0x7f3530104470 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.521+0000 7f35277fe640 1 -- 172.21.15.69:0/1216487435 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35200035f0 con 0x7f3530104470 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.521+0000 7f35384bf640 1 -- 172.21.15.69:0/1216487435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353019f6e0 con 0x7f3530104470 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.521+0000 7f35277fe640 1 -- 172.21.15.69:0/1216487435 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3520010690 con 0x7f3530104470 2024-08-02T17:47:08.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.521+0000 7f35384bf640 1 -- 172.21.15.69:0/1216487435 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35301a6fc0 con 0x7f3530104470 2024-08-02T17:47:08.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.524+0000 7f35277fe640 1 -- 172.21.15.69:0/1216487435 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f3520003b30 con 0x7f3530104470 2024-08-02T17:47:08.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.524+0000 7f353a8bd740 1 Processor -- start 2024-08-02T17:47:08.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f353a8bd740 1 -- start start 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b67e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b72c00 0x55b962b74ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f353a8bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b962ac4980 con 0x55b962b67a20 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f353a8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b962a5ca70 con 0x55b962b72c00 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f353a8bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b962ac0940 con 0x55b962b68340 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f3536a5c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b67e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f353625b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f3536a5c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b67e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52240/0 (socket says 172.21.15.69:52240) 2024-08-02T17:47:08.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f3536a5c640 1 -- 172.21.15.69:0/1175735487 learned_addr learned my addr 172.21.15.69:0/1175735487 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.525+0000 7f3536a5c640 1 -- 172.21.15.69:0/1175735487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 msgr2=0x55b962b726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1175735487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f3536a5c640 1 -- 172.21.15.69:0/1175735487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b72c00 msgr2=0x55b962b74ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1175735487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b72c00 0x55b962b74ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f3536a5c640 1 -- 172.21.15.69:0/1175735487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b962abf6e0 con 0x55b962b67a20 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f353625b640 1 --2- 172.21.15.69:0/1175735487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1175735487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b67e00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3528009f20 tx=0x7f3528010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a660ab758fb5c46 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f35277fe640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f351c0a2ff0 0x7f351c0a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f353625b640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f351c0a2ff0 0x7f351c0a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f3524ff9640 1 -- 172.21.15.69:0/1175735487 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528011700 con 0x55b962b67a20 2024-08-02T17:47:08.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f35277fe640 1 -- 172.21.15.69:0/1216487435 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f35200cfe30 con 0x7f3530104470 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.526+0000 7f3524ff9640 1 -- 172.21.15.69:0/1175735487 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528011d80 con 0x55b962b67a20 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f3524ff9640 1 -- 172.21.15.69:0/1175735487 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352801a380 con 0x55b962b67a20 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 -- 172.21.15.69:0/1175735487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 msgr2=0x55b962b67e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353625b640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f351c0a2ff0 0x7f351c0a54b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55b962b64e10 tx=0x7f352c02a920 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1175735487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b67e00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3528009f20 tx=0x7f3528010860 comp rx=0 tx=0).stop 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 -- 172.21.15.69:0/1175735487 shutdown_connections 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1175735487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b72c00 0x55b962b74ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1175735487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1175735487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b67e00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 -- 172.21.15.69:0/1175735487 >> 172.21.15.69:0/1175735487 conn(0x55b962b63fb0 msgr2=0x55b962b64390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 -- 172.21.15.69:0/1175735487 shutdown_connections 2024-08-02T17:47:08.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.527+0000 7f353a8bd740 1 -- 172.21.15.69:0/1175735487 wait complete. 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 1 -- start start 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 1 Processor -- start 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b68340 0x55b962b6ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b70520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b962b75870 con 0x55b962b68340 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b962a7faa0 con 0x55b962b67a20 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b962b759f0 con 0x55b962b72c00 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.528+0000 7f353a8bd740 10 client.0 _ll_register_callbacks cb 0x55b962a6ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b6faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45334/0 (socket says 172.21.15.69:45334) 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 -- 172.21.15.69:0/1841520158 learned_addr learned my addr 172.21.15.69:0/1841520158 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:08.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b70520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f3535a5a640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b68340 0x55b962b6ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 -- 172.21.15.69:0/1841520158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 msgr2=0x55b962b70520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b70520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 -- 172.21.15.69:0/1841520158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b68340 msgr2=0x55b962b6ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b68340 0x55b962b6ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 -- 172.21.15.69:0/1841520158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b962a63650 con 0x55b962b67a20 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f3535a5a640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b68340 0x55b962b6ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b70520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f353625b640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b6faa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f352c0096c0 tx=0x7f352c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93522cdc22b7ee40 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.529+0000 7f3519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.530+0000 7f3519ffb640 1 -- 172.21.15.69:0/1841520158 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c035040 con 0x55b962b67a20 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.530+0000 7f353a8bd740 1 -- 172.21.15.69:0/1841520158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528011d80 con 0x55b962b67a20 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.530+0000 7f3519ffb640 1 -- 172.21.15.69:0/1841520158 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f352c0317b0 con 0x55b962b67a20 2024-08-02T17:47:08.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.530+0000 7f3519ffb640 1 -- 172.21.15.69:0/1841520158 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f352c05a050 con 0x55b962b67a20 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.530+0000 7f353a8bd740 10 client.25217 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.530+0000 7f353a8bd740 1 -- 172.21.15.69:0/1841520158 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9628363c0 con 0x55b962b67a20 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.530+0000 7f3519ffb640 1 -- 172.21.15.69:0/1841520158 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c04c070 con 0x55b962b67a20 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f3519ffb640 1 -- 172.21.15.69:0/1841520158 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f352c044030 con 0x55b962b67a20 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f353a8bd740 10 client.25217 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f353a8bd740 10 client.25217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f353a8bd740 20 client.25217 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b962b82780 0x55b962b84b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f353a8bd740 4 client.25217 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f353a8bd740 1 -- 172.21.15.69:0/1841520158 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3528011d80 con 0x55b962b82780 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f3535a5a640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b962b82780 0x55b962b84b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.531+0000 7f3519ffb640 10 client.25217 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.532+0000 7f3535a5a640 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b962b82780 0x55b962b84b90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89174d3e8d45a7bb server_cookie=5a2ac60973b57e55 in_seq=0 out_seq=0 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f3519ffb640 1 -- 172.21.15.69:0/1841520158 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f3528011d80 con 0x55b962b82780 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f3519ffb640 10 client.25217 handle_command_reply: tid=0 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 client.25217 shutdown 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 -- 172.21.15.69:0/1841520158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 msgr2=0x55b962b6faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b6faa0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f352c0096c0 tx=0x7f352c009260 comp rx=0 tx=0).stop 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 -- 172.21.15.69:0/1841520158 shutdown_connections 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b68340 0x55b962b6ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b962b82780 0x55b962b84b90 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b70520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1841520158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b6faa0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 -- 172.21.15.69:0/1841520158 >> 172.21.15.69:0/1841520158 conn(0x55b962b63fb0 msgr2=0x55b962b5a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f3519ffb640 0 client.25217 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.533+0000 7f353a8bd740 1 -- 172.21.15.69:0/1841520158 shutdown_connections 2024-08-02T17:47:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.534+0000 7f353a8bd740 1 -- 172.21.15.69:0/1841520158 wait complete. 2024-08-02T17:47:08.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.534+0000 7f353a8bd740 20 client.25217 trim_cache size 0 max 16384 2024-08-02T17:47:08.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.558+0000 7f353a8bd740 1 Processor -- start 2024-08-02T17:47:08.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f353a8bd740 1 -- start start 2024-08-02T17:47:08.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b64390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b68340 0x55b962b6ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b8a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f353a8bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35301af4c0 con 0x55b962b67a20 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f353a8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35301acd90 con 0x55b962b68340 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f353a8bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35301aabe0 con 0x55b962b72c00 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3536a5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b8a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f353625b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b64390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3535a5a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b68340 0x55b962b6ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3536a5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b8a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33866/0 (socket says 172.21.15.69:33866) 2024-08-02T17:47:08.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3536a5c640 1 -- 172.21.15.69:0/1628190486 learned_addr learned my addr 172.21.15.69:0/1628190486 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3536a5c640 1 -- 172.21.15.69:0/1628190486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b68340 msgr2=0x55b962b6ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1628190486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b68340 0x55b962b6ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3536a5c640 1 -- 172.21.15.69:0/1628190486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 msgr2=0x55b962b64390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1628190486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b64390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.559+0000 7f3536a5c640 1 -- 172.21.15.69:0/1628190486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b962b68170 con 0x55b962b72c00 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.560+0000 7f3536a5c640 1 --2- 172.21.15.69:0/1628190486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b8a460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f352801cab0 tx=0x7f352801cae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33434cd1c1c46296 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.560+0000 7f351bfff640 1 -- 172.21.15.69:0/1628190486 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352801c790 con 0x55b962b72c00 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.560+0000 7f351bfff640 1 -- 172.21.15.69:0/1628190486 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3528003970 con 0x55b962b72c00 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.560+0000 7f351bfff640 1 -- 172.21.15.69:0/1628190486 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004440 con 0x55b962b72c00 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.561+0000 7f353a8bd740 1 -- 172.21.15.69:0/1628190486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 msgr2=0x55b962b8a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.561+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1628190486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b8a460 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f352801cab0 tx=0x7f352801cae0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.561+0000 7f353a8bd740 1 -- 172.21.15.69:0/1628190486 shutdown_connections 2024-08-02T17:47:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.561+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1628190486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b72c00 0x55b962b8a460 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.561+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1628190486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b68340 0x55b962b6ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.561+0000 7f353a8bd740 1 --2- 172.21.15.69:0/1628190486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b67a20 0x55b962b64390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.561+0000 7f353a8bd740 1 -- 172.21.15.69:0/1628190486 >> 172.21.15.69:0/1628190486 conn(0x55b962b63fb0 msgr2=0x55b962b6e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:08.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.562+0000 7f353a8bd740 1 -- 172.21.15.69:0/1628190486 shutdown_connections 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.562+0000 7f353a8bd740 1 -- 172.21.15.69:0/1628190486 wait complete. 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.562+0000 7f353a8bd740 1 -- start start 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.562+0000 7f353a8bd740 1 Processor -- start 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b75c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b76190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353a8bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b72c00 0x55b962b766d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353a8bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35301ab250 con 0x55b962b72c00 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353a8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35301ac6c0 con 0x55b962b67a20 2024-08-02T17:47:08.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353a8bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35301ab920 con 0x55b962b68340 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353625b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b75c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353625b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b75c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45344/0 (socket says 172.21.15.69:45344) 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353625b640 1 -- 172.21.15.69:0/457123071 learned_addr learned my addr 172.21.15.69:0/457123071 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353a8bd740 10 client.0 _ll_register_callbacks cb 0x7f352c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f3535a5a640 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b76190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f3536a5c640 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b72c00 0x55b962b766d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353625b640 1 -- 172.21.15.69:0/457123071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 msgr2=0x55b962b76190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353625b640 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b76190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353625b640 1 -- 172.21.15.69:0/457123071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b72c00 msgr2=0x55b962b766d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353625b640 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b72c00 0x55b962b766d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.563+0000 7f353625b640 1 -- 172.21.15.69:0/457123071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b962beb4a0 con 0x55b962b67a20 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.564+0000 7f353625b640 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b75c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f352c0059d0 tx=0x7f352c02aaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f8618433b6cc3e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.564+0000 7f3519ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:47:08.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.564+0000 7f3519ffb640 1 -- 172.21.15.69:0/457123071 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c0037f0 con 0x55b962b67a20 2024-08-02T17:47:08.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.564+0000 7f3519ffb640 1 -- 172.21.15.69:0/457123071 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f352c003990 con 0x55b962b67a20 2024-08-02T17:47:08.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.564+0000 7f353a8bd740 1 -- 172.21.15.69:0/457123071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528003970 con 0x55b962b67a20 2024-08-02T17:47:08.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f3519ffb640 1 -- 172.21.15.69:0/457123071 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f352c0315b0 con 0x55b962b67a20 2024-08-02T17:47:08.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f353a8bd740 10 client.25223 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:08.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f353a8bd740 1 -- 172.21.15.69:0/457123071 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b962bec770 con 0x55b962b67a20 2024-08-02T17:47:08.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f3519ffb640 1 -- 172.21.15.69:0/457123071 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c04c040 con 0x55b962b67a20 2024-08-02T17:47:08.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f3519ffb640 1 -- 172.21.15.69:0/457123071 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f352c044040 con 0x55b962b67a20 2024-08-02T17:47:08.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f353a8bd740 10 client.25223 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:08.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f353a8bd740 10 client.25223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:08.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f353a8bd740 20 client.25223 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:08.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f353a8bd740 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b962becec0 0x55b962bef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:08.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f353a8bd740 4 client.25223 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:47:08.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f353a8bd740 1 -- 172.21.15.69:0/457123071 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3528003970 con 0x55b962becec0 2024-08-02T17:47:08.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.565+0000 7f3535a5a640 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b962becec0 0x55b962bef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.566+0000 7f3519ffb640 10 client.25223 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.566+0000 7f3535a5a640 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b962becec0 0x55b962bef2e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f42e320c39ab18 server_cookie=91ab912d66e4a09e in_seq=0 out_seq=0 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.566+0000 7f3519ffb640 1 -- 172.21.15.69:0/457123071 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f3528003970 con 0x55b962becec0 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.566+0000 7f3519ffb640 10 client.25223 handle_command_reply: tid=0 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.566+0000 7f353a8bd740 1 client.25223 shutdown 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.566+0000 7f353a8bd740 1 -- 172.21.15.69:0/457123071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 msgr2=0x55b962b75c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.566+0000 7f353a8bd740 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b75c50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f352c0059d0 tx=0x7f352c02aaf0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 1 -- 172.21.15.69:0/457123071 shutdown_connections 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b962b72c00 0x55b962b766d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b962becec0 0x55b962bef2e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b962b68340 0x55b962b76190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f3519ffb640 0 client.25223 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 1 --2- 172.21.15.69:0/457123071 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b962b67a20 0x55b962b75c50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 1 -- 172.21.15.69:0/457123071 >> 172.21.15.69:0/457123071 conn(0x55b962b63fb0 msgr2=0x55b962b6eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 1 -- 172.21.15.69:0/457123071 shutdown_connections 2024-08-02T17:47:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 1 -- 172.21.15.69:0/457123071 wait complete. 2024-08-02T17:47:08.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.567+0000 7f353a8bd740 20 client.25223 trim_cache size 0 max 16384 2024-08-02T17:47:08.572 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:47:08.572 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:47:08.572 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:47:08.572 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:47:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 -- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f351c0a2ff0 msgr2=0x7f351c0a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f351c0a2ff0 0x7f351c0a54b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55b962b64e10 tx=0x7f352c02a920 comp rx=0 tx=0).stop 2024-08-02T17:47:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 -- 172.21.15.69:0/1216487435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3530104470 msgr2=0x7f353019a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3530104470 0x7f353019a980 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f352000e9f0 tx=0x7f352000eeb0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 -- 172.21.15.69:0/1216487435 shutdown_connections 2024-08-02T17:47:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3530109b60 0x7f353019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f351c0a2ff0 0x7f351c0a54b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3530104470 0x7f353019a980 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 --2- 172.21.15.69:0/1216487435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3530103ad0 0x7f353019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:08.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.568+0000 7f35384bf640 1 -- 172.21.15.69:0/1216487435 >> 172.21.15.69:0/1216487435 conn(0x7f35300ff750 msgr2=0x7f3530100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:08.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.569+0000 7f35384bf640 1 -- 172.21.15.69:0/1216487435 shutdown_connections 2024-08-02T17:47:08.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:08.569+0000 7f35384bf640 1 -- 172.21.15.69:0/1216487435 wait complete. 2024-08-02T17:47:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:09 smithi077 ceph-mon[37108]: pgmap v25: 129 pgs: 129 active+clean; 392 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 49 MiB/s wr, 631 op/s 2024-08-02T17:47:09.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:09 smithi069 ceph-mon[32746]: pgmap v25: 129 pgs: 129 active+clean; 392 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 49 MiB/s wr, 631 op/s 2024-08-02T17:47:09.439 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:47:09.439 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:47:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:09 smithi083 ceph-mon[37031]: pgmap v25: 129 pgs: 129 active+clean; 392 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 49 MiB/s wr, 631 op/s 2024-08-02T17:47:09.783 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:09.894 INFO:teuthology.orchestra.run.smithi069.stderr:done in 14.62 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.08 s, vacuum 0.58 s, primary keys 9.95 s). 2024-08-02T17:47:09.897 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-08-02T17:47:09.952 INFO:teuthology.orchestra.run.smithi069.stderr:starting vacuum...end. 2024-08-02T17:47:09.989 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-02T17:47:09.989 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.95 2024-08-02T17:47:09.989 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:47:10.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 -- 172.21.15.69:0/4246003720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febbc101150 msgr2=0x7febbc10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:10.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 --2- 172.21.15.69:0/4246003720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febbc101150 0x7febbc10d680 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7febb4009cd0 tx=0x7febb402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 -- 172.21.15.69:0/4246003720 shutdown_connections 2024-08-02T17:47:10.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 --2- 172.21.15.69:0/4246003720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febbc101150 0x7febbc10d680 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 --2- 172.21.15.69:0/4246003720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc1007b0 0x7febbc100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 --2- 172.21.15.69:0/4246003720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc1067b0 0x7febbc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 -- 172.21.15.69:0/4246003720 >> 172.21.15.69:0/4246003720 conn(0x7febbc0fc4d0 msgr2=0x7febbc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:10.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 -- 172.21.15.69:0/4246003720 shutdown_connections 2024-08-02T17:47:10.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.353+0000 7febc1ad9640 1 -- 172.21.15.69:0/4246003720 wait complete. 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.354+0000 7febc1ad9640 1 Processor -- start 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.354+0000 7febc1ad9640 1 -- start start 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.355+0000 7febc1ad9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc1007b0 0x7febbc1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.355+0000 7febc1ad9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc101150 0x7febbc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.355+0000 7febbb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc1007b0 0x7febbc1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.355+0000 7febc1ad9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febbc1067b0 0x7febbc19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.355+0000 7febc1ad9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7febbc110bd0 con 0x7febbc1067b0 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.355+0000 7febc1ad9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7febbc110a50 con 0x7febbc101150 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.355+0000 7febc1ad9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7febbc110d50 con 0x7febbc1007b0 2024-08-02T17:47:10.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febbb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc1007b0 0x7febbc1970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33886/0 (socket says 172.21.15.69:33886) 2024-08-02T17:47:10.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febbb7fe640 1 -- 172.21.15.69:0/3870247642 learned_addr learned my addr 172.21.15.69:0/3870247642 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:10.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febb2ffd640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc101150 0x7febbc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:10.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febbbfff640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febbc1067b0 0x7febbc19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febb2ffd640 1 -- 172.21.15.69:0/3870247642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc1007b0 msgr2=0x7febbc1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febb2ffd640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc1007b0 0x7febbc1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febb2ffd640 1 -- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febbc1067b0 msgr2=0x7febbc19b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febb2ffd640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febbc1067b0 0x7febbc19b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febb2ffd640 1 -- 172.21.15.69:0/3870247642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc19c060 con 0x7febbc101150 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febb2ffd640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc101150 0x7febbc1975f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feba400b3f0 tx=0x7feba400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.356+0000 7febb97fa640 1 -- 172.21.15.69:0/3870247642 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba401d070 con 0x7febbc101150 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.357+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc19c350 con 0x7febbc101150 2024-08-02T17:47:10.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.357+0000 7febb97fa640 1 -- 172.21.15.69:0/3870247642 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba4018440 con 0x7febbc101150 2024-08-02T17:47:10.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.357+0000 7febb97fa640 1 -- 172.21.15.69:0/3870247642 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba40165a0 con 0x7febbc101150 2024-08-02T17:47:10.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.357+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc06a4c0 con 0x7febbc101150 2024-08-02T17:47:10.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.358+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc100c10 con 0x7febbc101150 2024-08-02T17:47:10.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.359+0000 7febb97fa640 1 -- 172.21.15.69:0/3870247642 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7feba4020070 con 0x7febbc101150 2024-08-02T17:47:10.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.360+0000 7febb97fa640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feba80a2f20 0x7feba80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:10.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.360+0000 7febb97fa640 1 -- 172.21.15.69:0/3870247642 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7feba40d2b00 con 0x7febbc101150 2024-08-02T17:47:10.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.360+0000 7febbb7fe640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feba80a2f20 0x7feba80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:10.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.360+0000 7febbb7fe640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feba80a2f20 0x7feba80a53e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febac009c90 tx=0x7febac005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:10.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.364+0000 7febb97fa640 1 -- 172.21.15.69:0/3870247642 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feba409ebe0 con 0x7febbc101150 2024-08-02T17:47:10.451 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:10.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.687+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febbc198410 con 0x7febbc101150 2024-08-02T17:47:10.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.688+0000 7febb97fa640 1 -- 172.21.15.69:0/3870247642 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7febbc198410 con 0x7febbc101150 2024-08-02T17:47:10.690 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:47:10.690 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:47:10.690 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:47:10.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feba80a2f20 msgr2=0x7feba80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feba80a2f20 0x7feba80a53e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7febac009c90 tx=0x7febac005f90 comp rx=0 tx=0).stop 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc101150 msgr2=0x7febbc1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc101150 0x7febbc1975f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feba400b3f0 tx=0x7feba400b8b0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 shutdown_connections 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7febbc1067b0 0x7febbc19b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feba80a2f20 0x7feba80a53e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7febbc101150 0x7febbc1975f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 --2- 172.21.15.69:0/3870247642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febbc1007b0 0x7febbc1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 >> 172.21.15.69:0/3870247642 conn(0x7febbc0fc4d0 msgr2=0x7febbc0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 shutdown_connections 2024-08-02T17:47:10.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:10.690+0000 7febc1ad9640 1 -- 172.21.15.69:0/3870247642 wait complete. 2024-08-02T17:47:10.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:10 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3870247642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:10 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3870247642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:11.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:10 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3870247642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:12.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:11 smithi083 ceph-mon[37031]: pgmap v26: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 MiB/s wr, 840 op/s 2024-08-02T17:47:12.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:11 smithi077 ceph-mon[37108]: pgmap v26: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 MiB/s wr, 840 op/s 2024-08-02T17:47:12.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:11 smithi069 ceph-mon[32746]: pgmap v26: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 66 MiB/s wr, 840 op/s 2024-08-02T17:47:12.841 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:47:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 -- 172.21.15.69:0/2178600521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c1022a0 msgr2=0x7fef3c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 --2- 172.21.15.69:0/2178600521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c1022a0 0x7fef3c1026e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009900 tx=0x7fef2c02a480 comp rx=0 tx=0).stop 2024-08-02T17:47:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 -- 172.21.15.69:0/2178600521 shutdown_connections 2024-08-02T17:47:13.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 --2- 172.21.15.69:0/2178600521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c1022a0 0x7fef3c1026e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 --2- 172.21.15.69:0/2178600521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef3c102c20 0x7fef3c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 --2- 172.21.15.69:0/2178600521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3c100b50 0x7fef3c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 -- 172.21.15.69:0/2178600521 >> 172.21.15.69:0/2178600521 conn(0x7fef3c0fc500 msgr2=0x7fef3c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 -- 172.21.15.69:0/2178600521 shutdown_connections 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 -- 172.21.15.69:0/2178600521 wait complete. 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.002+0000 7fef43edf640 1 Processor -- start 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef43edf640 1 -- start start 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef43edf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c100b50 0x7fef3c195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef43edf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef3c1022a0 0x7fef3c1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef43edf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3c102c20 0x7fef3c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef43edf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c10fbb0 con 0x7fef3c100b50 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef43edf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c10fa30 con 0x7fef3c1022a0 2024-08-02T17:47:13.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef43edf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef3c10fd30 con 0x7fef3c102c20 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef41c7b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c100b50 0x7fef3c195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef41c7b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c100b50 0x7fef3c195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52296/0 (socket says 172.21.15.69:52296) 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef41c7b640 1 -- 172.21.15.69:0/2251462733 learned_addr learned my addr 172.21.15.69:0/2251462733 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef4247c640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3c102c20 0x7fef3c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef4147a640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef3c1022a0 0x7fef3c1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef41c7b640 1 -- 172.21.15.69:0/2251462733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3c102c20 msgr2=0x7fef3c19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef41c7b640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3c102c20 0x7fef3c19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef41c7b640 1 -- 172.21.15.69:0/2251462733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef3c1022a0 msgr2=0x7fef3c1964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef41c7b640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef3c1022a0 0x7fef3c1964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.003+0000 7fef41c7b640 1 -- 172.21.15.69:0/2251462733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3c19af60 con 0x7fef3c100b50 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.004+0000 7fef41c7b640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c100b50 0x7fef3c195fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fef3800e9f0 tx=0x7fef3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.005+0000 7fef32ffd640 1 -- 172.21.15.69:0/2251462733 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3800cd80 con 0x7fef3c100b50 2024-08-02T17:47:13.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.005+0000 7fef32ffd640 1 -- 172.21.15.69:0/2251462733 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef38010910 con 0x7fef3c100b50 2024-08-02T17:47:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.005+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3c19b250 con 0x7fef3c100b50 2024-08-02T17:47:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.005+0000 7fef32ffd640 1 -- 172.21.15.69:0/2251462733 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38018990 con 0x7fef3c100b50 2024-08-02T17:47:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.005+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef3c1a2b30 con 0x7fef3c100b50 2024-08-02T17:47:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.006+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef3c100f30 con 0x7fef3c100b50 2024-08-02T17:47:13.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.008+0000 7fef32ffd640 1 -- 172.21.15.69:0/2251462733 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7fef38010ac0 con 0x7fef3c100b50 2024-08-02T17:47:13.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.009+0000 7fef32ffd640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef280a7330 0x7fef280a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:13.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.009+0000 7fef32ffd640 1 -- 172.21.15.69:0/2251462733 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fef380d0180 con 0x7fef3c100b50 2024-08-02T17:47:13.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.010+0000 7fef4147a640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef280a7330 0x7fef280a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:13.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.010+0000 7fef4147a640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef280a7330 0x7fef280a97f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009230 tx=0x7fef2c0058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:13.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.014+0000 7fef32ffd640 1 -- 172.21.15.69:0/2251462733 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fef3809c150 con 0x7fef3c100b50 2024-08-02T17:47:13.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:13 smithi069 ceph-mon[32746]: pgmap v27: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 754 op/s 2024-08-02T17:47:13.182 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:13.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.310+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef3c1a2d00 con 0x7fef3c100b50 2024-08-02T17:47:13.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.311+0000 7fef32ffd640 1 -- 172.21.15.69:0/2251462733 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fef380a0450 con 0x7fef3c100b50 2024-08-02T17:47:13.312 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:47:13.313 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:47:13.313 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:47:13.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef280a7330 msgr2=0x7fef280a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:13.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef280a7330 0x7fef280a97f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009230 tx=0x7fef2c0058d0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c100b50 msgr2=0x7fef3c195fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:13.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c100b50 0x7fef3c195fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fef3800e9f0 tx=0x7fef3800eeb0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 shutdown_connections 2024-08-02T17:47:13.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef280a7330 0x7fef280a97f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef3c102c20 0x7fef3c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef3c1022a0 0x7fef3c1964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 --2- 172.21.15.69:0/2251462733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef3c100b50 0x7fef3c195fb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:13.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 >> 172.21.15.69:0/2251462733 conn(0x7fef3c0fc500 msgr2=0x7fef3c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:13.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 shutdown_connections 2024-08-02T17:47:13.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:13.313+0000 7fef43edf640 1 -- 172.21.15.69:0/2251462733 wait complete. 2024-08-02T17:47:13.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:13 smithi077 ceph-mon[37108]: pgmap v27: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 754 op/s 2024-08-02T17:47:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:13 smithi083 ceph-mon[37031]: pgmap v27: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 754 op/s 2024-08-02T17:47:14.211 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 141.95 --await-for 45.0' 2024-08-02T17:47:14.211 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 141.95 --await-for 45.0 2024-08-02T17:47:14.313 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:14 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2251462733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:14.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:14 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2251462733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 -- 172.21.15.69:0/4022786353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c1068b0 msgr2=0x7f489c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 --2- 172.21.15.69:0/4022786353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c1068b0 0x7f489c106c90 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f488c009900 tx=0x7f488c02a410 comp rx=0 tx=0).stop 2024-08-02T17:47:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 -- 172.21.15.69:0/4022786353 shutdown_connections 2024-08-02T17:47:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 --2- 172.21.15.69:0/4022786353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f489c1007d0 0x7f489c10e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 --2- 172.21.15.69:0/4022786353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489c0ffe30 0x7f489c100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 --2- 172.21.15.69:0/4022786353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c1068b0 0x7f489c106c90 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 -- 172.21.15.69:0/4022786353 >> 172.21.15.69:0/4022786353 conn(0x7f489c076f60 msgr2=0x7f489c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 -- 172.21.15.69:0/4022786353 shutdown_connections 2024-08-02T17:47:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.417+0000 7f48a1422640 1 -- 172.21.15.69:0/4022786353 wait complete. 2024-08-02T17:47:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.418+0000 7f48a1422640 1 Processor -- start 2024-08-02T17:47:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.418+0000 7f48a1422640 1 -- start start 2024-08-02T17:47:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.418+0000 7f48a1422640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c0ffe30 0x7f489c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.418+0000 7f48a1422640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f489c1007d0 0x7f489c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.418+0000 7f48a1422640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489c1068b0 0x7f489c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.418+0000 7f48a1422640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f489c111dd0 con 0x7f489c0ffe30 2024-08-02T17:47:14.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f48a1422640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f489c111c50 con 0x7f489c1068b0 2024-08-02T17:47:14.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c0ffe30 0x7f489c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f48a1422640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f489c111f50 con 0x7f489c1007d0 2024-08-02T17:47:14.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c0ffe30 0x7f489c198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52322/0 (socket says 172.21.15.69:52322) 2024-08-02T17:47:14.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 -- 172.21.15.69:0/2391291136 learned_addr learned my addr 172.21.15.69:0/2391291136 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489b7fe640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489c1068b0 0x7f489c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489a7fc640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f489c1007d0 0x7f489c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 -- 172.21.15.69:0/2391291136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f489c1007d0 msgr2=0x7f489c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f489c1007d0 0x7f489c198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 -- 172.21.15.69:0/2391291136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489c1068b0 msgr2=0x7f489c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489c1068b0 0x7f489c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 -- 172.21.15.69:0/2391291136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c19d1c0 con 0x7f489c0ffe30 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.419+0000 7f489affd640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c0ffe30 0x7f489c198210 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f488c004770 tx=0x7f488c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.420+0000 7f4883fff640 1 -- 172.21.15.69:0/2391291136 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c03f070 con 0x7f489c0ffe30 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.420+0000 7f4883fff640 1 -- 172.21.15.69:0/2391291136 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c031ca0 con 0x7f489c0ffe30 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.420+0000 7f48a1422640 1 -- 172.21.15.69:0/2391291136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c19d450 con 0x7f489c0ffe30 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.420+0000 7f4883fff640 1 -- 172.21.15.69:0/2391291136 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c035030 con 0x7f489c0ffe30 2024-08-02T17:47:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.420+0000 7f48a1422640 1 -- 172.21.15.69:0/2391291136 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c100f40 con 0x7f489c0ffe30 2024-08-02T17:47:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.423+0000 7f4883fff640 1 -- 172.21.15.69:0/2391291136 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f488c042070 con 0x7f489c0ffe30 2024-08-02T17:47:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.423+0000 7f4883fff640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48840a2f20 0x7f48840a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.423+0000 7f4883fff640 1 -- 172.21.15.69:0/2391291136 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f488c0f00f0 con 0x7f489c0ffe30 2024-08-02T17:47:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.423+0000 7f489a7fc640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48840a2f20 0x7f48840a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f48a3820740 1 Processor -- start 2024-08-02T17:47:14.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f48a3820740 1 -- start start 2024-08-02T17:47:14.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f48a3820740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f48a3820740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c91c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f48a3820740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21c945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f48a3820740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558d21beca40 con 0x558d21c86ff0 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f48a3820740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d21b849d0 con 0x558d21c87910 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f48a3820740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d21be8a00 con 0x558d21c921d0 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f489a7fc640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48840a2f20 0x7f48840a53e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f489c0ff4b0 tx=0x7f489000f820 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f489affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f489affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c873d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52330/0 (socket says 172.21.15.69:52330) 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f489affd640 1 -- 172.21.15.69:0/1873901491 learned_addr learned my addr 172.21.15.69:0/1873901491 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.424+0000 7f489b7fe640 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21c945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f489a7fc640 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c91c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f489affd640 1 -- 172.21.15.69:0/1873901491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 msgr2=0x558d21c945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f489affd640 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21c945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f489affd640 1 -- 172.21.15.69:0/1873901491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 msgr2=0x558d21c91c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f489affd640 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c91c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f489affd640 1 -- 172.21.15.69:0/1873901491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d21b8b650 con 0x558d21c86ff0 2024-08-02T17:47:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f489affd640 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c873d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f488c0bc050 tx=0x7f488c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc2863ee4efdb0c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f48817fa640 1 -- 172.21.15.69:0/1873901491 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c049050 con 0x558d21c86ff0 2024-08-02T17:47:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f48817fa640 1 -- 172.21.15.69:0/1873901491 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c0c03c0 con 0x558d21c86ff0 2024-08-02T17:47:14.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.425+0000 7f48817fa640 1 -- 172.21.15.69:0/1873901491 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0bc320 con 0x558d21c86ff0 2024-08-02T17:47:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 -- 172.21.15.69:0/1873901491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 msgr2=0x558d21c873d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c873d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f488c0bc050 tx=0x7f488c02a410 comp rx=0 tx=0).stop 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 -- 172.21.15.69:0/1873901491 shutdown_connections 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21c945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c91c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 --2- 172.21.15.69:0/1873901491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c873d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 -- 172.21.15.69:0/1873901491 >> 172.21.15.69:0/1873901491 conn(0x558d21c83580 msgr2=0x558d21c83960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 -- 172.21.15.69:0/1873901491 shutdown_connections 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.426+0000 7f48a3820740 1 -- 172.21.15.69:0/1873901491 wait complete. 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 1 -- start start 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 1 Processor -- start 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c8f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c8f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c921d0 0x558d21c8fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558d21c94b60 con 0x558d21c921d0 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d21ba7b20 con 0x558d21c87910 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d21c94ce0 con 0x558d21c86ff0 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f48a3820740 10 client.0 _ll_register_callbacks cb 0x558d21ba5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.427+0000 7f489affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c8f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c921d0 0x558d21c8fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c921d0 0x558d21c8fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52332/0 (socket says 172.21.15.69:52332) 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489b7fe640 1 -- 172.21.15.69:0/719785463 learned_addr learned my addr 172.21.15.69:0/719785463 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489a7fc640 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c8f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489a7fc640 1 -- 172.21.15.69:0/719785463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 msgr2=0x558d21c8f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489a7fc640 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c8f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489a7fc640 1 -- 172.21.15.69:0/719785463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c921d0 msgr2=0x558d21c8fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489a7fc640 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c921d0 0x558d21c8fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489a7fc640 1 -- 172.21.15.69:0/719785463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d2195e3c0 con 0x558d21c87910 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489b7fe640 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c921d0 0x558d21c8fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f489a7fc640 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c8f590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f489000fbd0 tx=0x7f4890004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f2e5e157d8cf6d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.428+0000 7f487eff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.429+0000 7f487eff5640 1 -- 172.21.15.69:0/719785463 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48900273f0 con 0x558d21c87910 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.429+0000 7f487eff5640 1 -- 172.21.15.69:0/719785463 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4890027590 con 0x558d21c87910 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.429+0000 7f48a3820740 1 -- 172.21.15.69:0/719785463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c0c03c0 con 0x558d21c87910 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.429+0000 7f487eff5640 1 -- 172.21.15.69:0/719785463 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4890035050 con 0x558d21c87910 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.429+0000 7f48a3820740 10 client.25247 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.429+0000 7f48a3820740 1 -- 172.21.15.69:0/719785463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d21ca0680 con 0x558d21c87910 2024-08-02T17:47:14.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.430+0000 7f487eff5640 1 -- 172.21.15.69:0/719785463 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f489002c070 con 0x558d21c87910 2024-08-02T17:47:14.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.430+0000 7f487eff5640 1 -- 172.21.15.69:0/719785463 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f48900306b0 con 0x558d21c87910 2024-08-02T17:47:14.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.430+0000 7f48a3820740 10 client.25247 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:14.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.430+0000 7f48a3820740 10 client.25247 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:47:14.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.430+0000 7f48a3820740 20 client.25247 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.430+0000 7f48a3820740 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558d21ca1b40 0x558d21ca3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.430+0000 7f48a3820740 4 client.25247 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.430+0000 7f48a3820740 1 -- 172.21.15.69:0/719785463 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f488c0c03c0 con 0x558d21ca1b40 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.431+0000 7f489affd640 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558d21ca1b40 0x558d21ca3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.431+0000 7f489affd640 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558d21ca1b40 0x558d21ca3fc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76e56e9345e93357 server_cookie=9529f83d982b9ed3 in_seq=0 out_seq=0 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.431+0000 7f487eff5640 10 client.25247 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.432+0000 7f487eff5640 1 -- 172.21.15.69:0/719785463 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f488c0c03c0 con 0x558d21ca1b40 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f487eff5640 10 client.25247 handle_command_reply: tid=0 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 client.25247 shutdown 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 -- 172.21.15.69:0/719785463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 msgr2=0x558d21c8f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c8f590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f489000fbd0 tx=0x7f4890004730 comp rx=0 tx=0).stop 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 -- 172.21.15.69:0/719785463 shutdown_connections 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558d21ca1b40 0x558d21ca3fc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c921d0 0x558d21c8fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f487eff5640 0 client.25247 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21c8f590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 --2- 172.21.15.69:0/719785463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c8f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 -- 172.21.15.69:0/719785463 >> 172.21.15.69:0/719785463 conn(0x558d21c83580 msgr2=0x558d21c892b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 -- 172.21.15.69:0/719785463 shutdown_connections 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.433+0000 7f48a3820740 1 -- 172.21.15.69:0/719785463 wait complete. 2024-08-02T17:47:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.434+0000 7f48a3820740 20 client.25247 trim_cache size 0 max 16384 2024-08-02T17:47:14.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.452+0000 7f48a3820740 1 Processor -- start 2024-08-02T17:47:14.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f48a3820740 1 -- start start 2024-08-02T17:47:14.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f48a3820740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c89a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f48a3820740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21ca9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f48a3820740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21ca9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f48a3820740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1af470 con 0x558d21c86ff0 2024-08-02T17:47:14.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f48a3820740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1acd40 con 0x558d21c87910 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f48a3820740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1aab90 con 0x558d21c921d0 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f489affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c89a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f489affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c89a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52336/0 (socket says 172.21.15.69:52336) 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f489affd640 1 -- 172.21.15.69:0/890347166 learned_addr learned my addr 172.21.15.69:0/890347166 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f489b7fe640 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21ca9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.453+0000 7f489a7fc640 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21ca9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f489affd640 1 -- 172.21.15.69:0/890347166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 msgr2=0x558d21ca9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f489affd640 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21ca9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f489affd640 1 -- 172.21.15.69:0/890347166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 msgr2=0x558d21ca9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f489affd640 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21ca9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f489affd640 1 -- 172.21.15.69:0/890347166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d21c87740 con 0x558d21c86ff0 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f489a7fc640 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21ca9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:14.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f489b7fe640 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21ca9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:14.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f489affd640 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c89a50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f488c0b8a90 tx=0x7f488c03b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf6af559b0a1d69 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:14.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f4880ff9640 1 -- 172.21.15.69:0/890347166 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0c9070 con 0x558d21c86ff0 2024-08-02T17:47:14.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f4880ff9640 1 -- 172.21.15.69:0/890347166 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c0c03c0 con 0x558d21c86ff0 2024-08-02T17:47:14.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.454+0000 7f4880ff9640 1 -- 172.21.15.69:0/890347166 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c049040 con 0x558d21c86ff0 2024-08-02T17:47:14.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 -- 172.21.15.69:0/890347166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 msgr2=0x558d21c89a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c89a50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f488c0b8a90 tx=0x7f488c03b890 comp rx=0 tx=0).stop 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 -- 172.21.15.69:0/890347166 shutdown_connections 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c921d0 0x558d21ca9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c87910 0x558d21ca9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 --2- 172.21.15.69:0/890347166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c86ff0 0x558d21c89a50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 -- 172.21.15.69:0/890347166 >> 172.21.15.69:0/890347166 conn(0x558d21c83580 msgr2=0x558d21c8d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 -- 172.21.15.69:0/890347166 shutdown_connections 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.455+0000 7f48a3820740 1 -- 172.21.15.69:0/890347166 wait complete. 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 1 -- start start 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 1 Processor -- start 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c9a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c87910 0x558d21c9a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c921d0 0x558d21c8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1ab200 con 0x558d21c87910 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1ac670 con 0x558d21c921d0 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f489c1ab8d0 con 0x558d21c86ff0 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f48a3820740 10 client.0 _ll_register_callbacks cb 0x7f489001b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f489affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c9a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f489affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c9a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33948/0 (socket says 172.21.15.69:33948) 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f489affd640 1 -- 172.21.15.69:0/1092709941 learned_addr learned my addr 172.21.15.69:0/1092709941 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.456+0000 7f489a7fc640 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c87910 0x558d21c9a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f489b7fe640 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c921d0 0x558d21c8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f489affd640 1 -- 172.21.15.69:0/1092709941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c921d0 msgr2=0x558d21c8b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f489affd640 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c921d0 0x558d21c8b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f489affd640 1 -- 172.21.15.69:0/1092709941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c87910 msgr2=0x558d21c9a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f489affd640 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c87910 0x558d21c9a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f489affd640 1 -- 172.21.15.69:0/1092709941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d21c8bfe0 con 0x558d21c86ff0 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f489b7fe640 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c921d0 0x558d21c8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:14.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f489affd640 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c9a030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f488c0c4ca0 tx=0x7f488c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f8a3bb09a829f3c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:14.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f487eff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:47:14.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f487eff5640 1 -- 172.21.15.69:0/1092709941 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c0c9070 con 0x558d21c86ff0 2024-08-02T17:47:14.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.457+0000 7f487eff5640 1 -- 172.21.15.69:0/1092709941 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f488c03b4f0 con 0x558d21c86ff0 2024-08-02T17:47:14.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 1 -- 172.21.15.69:0/1092709941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c0c03c0 con 0x558d21c86ff0 2024-08-02T17:47:14.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f487eff5640 1 -- 172.21.15.69:0/1092709941 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f488c049050 con 0x558d21c86ff0 2024-08-02T17:47:14.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 10 client.25141 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:14.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 1 -- 172.21.15.69:0/1092709941 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d21d0c260 con 0x558d21c86ff0 2024-08-02T17:47:14.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f487eff5640 1 -- 172.21.15.69:0/1092709941 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f488c0c03c0 con 0x558d21c86ff0 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f487eff5640 1 -- 172.21.15.69:0/1092709941 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f488c0bc400 con 0x558d21c86ff0 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 10 client.25141 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 10 client.25141 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 20 client.25141 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558d21ca1b40 0x558d21d0e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 4 client.25141 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f48a3820740 1 -- 172.21.15.69:0/1092709941 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d21bf9fb0 con 0x558d21ca1b40 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.458+0000 7f489a7fc640 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558d21ca1b40 0x558d21d0e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f487eff5640 10 client.25141 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f489a7fc640 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558d21ca1b40 0x558d21d0e610 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50b531508c9dcc27 server_cookie=6a8ba0b5bf9b0a7f in_seq=0 out_seq=0 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f487eff5640 1 -- 172.21.15.69:0/1092709941 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d21bf9fb0 con 0x558d21ca1b40 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f487eff5640 10 client.25141 handle_command_reply: tid=0 2024-08-02T17:47:14.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 client.25141 shutdown 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 -- 172.21.15.69:0/1092709941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 msgr2=0x558d21c9a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c9a030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f488c0c4ca0 tx=0x7f488c009680 comp rx=0 tx=0).stop 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 -- 172.21.15.69:0/1092709941 shutdown_connections 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558d21ca1b40 0x558d21d0e610 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f487eff5640 0 client.25141 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d21c87910 0x558d21c9a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d21c921d0 0x558d21c8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 --2- 172.21.15.69:0/1092709941 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d21c86ff0 0x558d21c9a030 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.459+0000 7f48a3820740 1 -- 172.21.15.69:0/1092709941 >> 172.21.15.69:0/1092709941 conn(0x558d21c83580 msgr2=0x558d21c81d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a3820740 1 -- 172.21.15.69:0/1092709941 shutdown_connections 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a3820740 1 -- 172.21.15.69:0/1092709941 wait complete. 2024-08-02T17:47:14.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a3820740 20 client.25141 trim_cache size 0 max 16384 2024-08-02T17:47:14.466 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:47:14.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 -- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48840a2f20 msgr2=0x7f48840a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48840a2f20 0x7f48840a53e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f489c0ff4b0 tx=0x7f489000f820 comp rx=0 tx=0).stop 2024-08-02T17:47:14.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 -- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c0ffe30 msgr2=0x7f489c198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:14.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c0ffe30 0x7f489c198210 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f488c004770 tx=0x7f488c003c10 comp rx=0 tx=0).stop 2024-08-02T17:47:14.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 -- 172.21.15.69:0/2391291136 shutdown_connections 2024-08-02T17:47:14.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48840a2f20 0x7f48840a53e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f489c1068b0 0x7f489c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f489c1007d0 0x7f489c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.460+0000 7f48a1422640 1 --2- 172.21.15.69:0/2391291136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f489c0ffe30 0x7f489c198210 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:14.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.461+0000 7f48a1422640 1 -- 172.21.15.69:0/2391291136 >> 172.21.15.69:0/2391291136 conn(0x7f489c076f60 msgr2=0x7f489c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:14.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.461+0000 7f48a1422640 1 -- 172.21.15.69:0/2391291136 shutdown_connections 2024-08-02T17:47:14.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:14.461+0000 7f48a1422640 1 -- 172.21.15.69:0/2391291136 wait complete. 2024-08-02T17:47:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:14 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2251462733' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:14.728 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:14.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 5.0 s, 1872.6 tps, lat 46.044 ms stddev 134.766 2024-08-02T17:47:15.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:15 smithi069 ceph-mon[32746]: pgmap v28: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 754 op/s 2024-08-02T17:47:15.175 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:47:15.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:15 smithi077 ceph-mon[37108]: pgmap v28: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 754 op/s 2024-08-02T17:47:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:15 smithi083 ceph-mon[37031]: pgmap v28: 129 pgs: 129 active+clean; 548 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 18 MiB/s wr, 754 op/s 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 -- 172.21.15.69:0/3541171330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c103470 msgr2=0x7f4c8c1078b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/3541171330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c103470 0x7f4c8c1078b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009930 tx=0x7f4c7402a420 comp rx=0 tx=0).stop 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 -- 172.21.15.69:0/3541171330 shutdown_connections 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/3541171330 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c103470 0x7f4c8c1078b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/3541171330 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c8c107fc0 0x7f4c8c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/3541171330 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c8c102ac0 0x7f4c8c102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 -- 172.21.15.69:0/3541171330 >> 172.21.15.69:0/3541171330 conn(0x7f4c8c0fc500 msgr2=0x7f4c8c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 -- 172.21.15.69:0/3541171330 shutdown_connections 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.664+0000 7f4c91e4f640 1 -- 172.21.15.69:0/3541171330 wait complete. 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c91e4f640 1 Processor -- start 2024-08-02T17:47:15.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c91e4f640 1 -- start start 2024-08-02T17:47:15.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c91e4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c8c102ac0 0x7f4c8c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c91e4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c8c103470 0x7f4c8c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c91e4f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c107fc0 0x7f4c8c19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c91e4f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c111e10 con 0x7f4c8c107fc0 2024-08-02T17:47:15.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c91e4f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c111c90 con 0x7f4c8c103470 2024-08-02T17:47:15.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c91e4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c111f90 con 0x7f4c8c102ac0 2024-08-02T17:47:15.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c107fc0 0x7f4c8c19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c83fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c8c103470 0x7f4c8c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c107fc0 0x7f4c8c19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49724/0 (socket says 172.21.15.69:49724) 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c83fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c8c103470 0x7f4c8c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55290/0 (socket says 172.21.15.69:55290) 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c8bfff640 1 -- 172.21.15.69:0/946623225 learned_addr learned my addr 172.21.15.69:0/946623225 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c8c102ac0 0x7f4c8c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c83fff640 1 -- 172.21.15.69:0/946623225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c8c102ac0 msgr2=0x7f4c8c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c83fff640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c8c102ac0 0x7f4c8c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c83fff640 1 -- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c107fc0 msgr2=0x7f4c8c19cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c83fff640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c107fc0 0x7f4c8c19cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c83fff640 1 -- 172.21.15.69:0/946623225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c19d300 con 0x7f4c8c103470 2024-08-02T17:47:15.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.665+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c8c102ac0 0x7f4c8c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.666+0000 7f4c83fff640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c8c103470 0x7f4c8c198890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009230 tx=0x7f4c74003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.666+0000 7f4c897fa640 1 -- 172.21.15.69:0/946623225 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7403f070 con 0x7f4c8c103470 2024-08-02T17:47:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.666+0000 7f4c897fa640 1 -- 172.21.15.69:0/946623225 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c74031c90 con 0x7f4c8c103470 2024-08-02T17:47:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.666+0000 7f4c91e4f640 1 -- 172.21.15.69:0/946623225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c19d590 con 0x7f4c8c103470 2024-08-02T17:47:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.666+0000 7f4c91e4f640 1 -- 172.21.15.69:0/946623225 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a4ed0 con 0x7f4c8c103470 2024-08-02T17:47:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.667+0000 7f4c897fa640 1 -- 172.21.15.69:0/946623225 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74035030 con 0x7f4c8c103470 2024-08-02T17:47:15.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.669+0000 7f4c897fa640 1 -- 172.21.15.69:0/946623225 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f4c74042070 con 0x7f4c8c103470 2024-08-02T17:47:15.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.669+0000 7f4c9424d740 1 Processor -- start 2024-08-02T17:47:15.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c9424d740 1 -- start start 2024-08-02T17:47:15.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e72c0 0x557c336e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e7be0 0x557c336f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336f24a0 0x557c336f4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c9424d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c3364c810 con 0x557c336e7be0 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c9424d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557c335f3c70 con 0x557c336f24a0 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c9424d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557c336487d0 con 0x557c336e72c0 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c83fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336f24a0 0x557c336f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c83fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336f24a0 0x557c336f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55302/0 (socket says 172.21.15.69:55302) 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c83fff640 1 -- 172.21.15.69:0/165971824 learned_addr learned my addr 172.21.15.69:0/165971824 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c8bfff640 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e72c0 0x557c336e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e7be0 0x557c336f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c83fff640 1 -- 172.21.15.69:0/165971824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e72c0 msgr2=0x557c336e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c83fff640 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e72c0 0x557c336e76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c83fff640 1 -- 172.21.15.69:0/165971824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e7be0 msgr2=0x557c336f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.670+0000 7f4c83fff640 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e7be0 0x557c336f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c83fff640 1 -- 172.21.15.69:0/165971824 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c333beba0 con 0x557c336f24a0 2024-08-02T17:47:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c897fa640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4c780a2f20 0x7f4c780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e7be0 0x557c336f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c897fa640 1 -- 172.21.15.69:0/946623225 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4c740eff70 con 0x7f4c8c103470 2024-08-02T17:47:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c83fff640 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336f24a0 0x557c336f4890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c740ed060 tx=0x7f4c740f8270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d599bc1b6d654f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c8bfff640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4c780a2f20 0x7f4c780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c81ffb640 1 -- 172.21.15.69:0/165971824 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74049070 con 0x557c336f24a0 2024-08-02T17:47:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c81ffb640 1 -- 172.21.15.69:0/165971824 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c74031e40 con 0x557c336f24a0 2024-08-02T17:47:15.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.671+0000 7f4c81ffb640 1 -- 172.21.15.69:0/165971824 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c740b8130 con 0x557c336f24a0 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 -- 172.21.15.69:0/165971824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336f24a0 msgr2=0x557c336f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c8bfff640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4c780a2f20 0x7f4c780a53e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c84005c40 tx=0x7f4c84009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336f24a0 0x557c336f4890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4c740ed060 tx=0x7f4c740f8270 comp rx=0 tx=0).stop 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 -- 172.21.15.69:0/165971824 shutdown_connections 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e7be0 0x557c336f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336f24a0 0x557c336f4890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 --2- 172.21.15.69:0/165971824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e72c0 0x557c336e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 -- 172.21.15.69:0/165971824 >> 172.21.15.69:0/165971824 conn(0x557c336e3850 msgr2=0x557c336e3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 -- 172.21.15.69:0/165971824 shutdown_connections 2024-08-02T17:47:15.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.672+0000 7f4c9424d740 1 -- 172.21.15.69:0/165971824 wait complete. 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 1 -- start start 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 1 Processor -- start 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 0x557c336ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c336efe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557c336f4e30 con 0x557c336f24a0 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557c336079a0 con 0x557c336e72c0 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557c336f4fb0 con 0x557c336e7be0 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c9424d740 10 client.0 _ll_register_callbacks cb 0x557c335f5c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c336efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c336efe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49744/0 (socket says 172.21.15.69:49744) 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c8bfff640 1 -- 172.21.15.69:0/3936480110 learned_addr learned my addr 172.21.15.69:0/3936480110 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:15.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.673+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c83fff640 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 0x557c336ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c8b7fe640 1 -- 172.21.15.69:0/3936480110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 msgr2=0x557c336ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 0x557c336ef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c8b7fe640 1 -- 172.21.15.69:0/3936480110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 msgr2=0x557c336efe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c336efe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c8b7fe640 1 -- 172.21.15.69:0/3936480110 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c33613220 con 0x557c336e72c0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336ef3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c005fd0 tx=0x7f4c7c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1325f31df81795 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c6affd640 1 -- 172.21.15.69:0/3936480110 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c005220 con 0x557c336e72c0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c6affd640 1 -- 172.21.15.69:0/3936480110 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4c7c0053c0 con 0x557c336e72c0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c9424d740 1 -- 172.21.15.69:0/3936480110 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c33659f40 con 0x557c336e72c0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.674+0000 7f4c6affd640 1 -- 172.21.15.69:0/3936480110 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c7c0055c0 con 0x557c336e72c0 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c9424d740 10 client.25265 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:15.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c9424d740 1 -- 172.21.15.69:0/3936480110 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c337092b0 con 0x557c336e72c0 2024-08-02T17:47:15.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c6affd640 1 -- 172.21.15.69:0/3936480110 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557c33659f40 con 0x557c336e72c0 2024-08-02T17:47:15.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c6affd640 1 -- 172.21.15.69:0/3936480110 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4c7c01ba40 con 0x557c336e72c0 2024-08-02T17:47:15.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c9424d740 10 client.25265 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:15.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c9424d740 10 client.25265 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:47:15.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c9424d740 20 client.25265 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:15.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557c3370a7c0 0x557c3370cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c9424d740 4 client.25265 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:15.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c9424d740 1 -- 172.21.15.69:0/3936480110 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c33654290 con 0x557c3370a7c0 2024-08-02T17:47:15.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.675+0000 7f4c83fff640 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557c3370a7c0 0x557c3370cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.676+0000 7f4c6affd640 10 client.25265 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:15.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.676+0000 7f4c83fff640 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557c3370a7c0 0x557c3370cbd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f7dff1d33f84965 server_cookie=1813b36d0a82fedb in_seq=0 out_seq=0 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c6affd640 1 -- 172.21.15.69:0/3936480110 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557c33654290 con 0x557c3370a7c0 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c6affd640 10 client.25265 handle_command_reply: tid=0 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 client.25265 shutdown 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 -- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 msgr2=0x557c336ef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336ef3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c005fd0 tx=0x7f4c7c0117b0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 -- 172.21.15.69:0/3936480110 shutdown_connections 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c336efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557c3370a7c0 0x557c3370cbd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 0x557c336ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c6affd640 0 client.25265 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:15.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3936480110 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336ef3b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 -- 172.21.15.69:0/3936480110 >> 172.21.15.69:0/3936480110 conn(0x557c336e3850 msgr2=0x557c336e2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:15.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 -- 172.21.15.69:0/3936480110 shutdown_connections 2024-08-02T17:47:15.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.677+0000 7f4c9424d740 1 -- 172.21.15.69:0/3936480110 wait complete. 2024-08-02T17:47:15.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.678+0000 7f4c9424d740 20 client.25265 trim_cache size 0 max 16384 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c9424d740 1 Processor -- start 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c9424d740 1 -- start start 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336e2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 0x557c336efc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c337124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c9424d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c012800 con 0x557c336f24a0 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c9424d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c0100d0 con 0x557c336e72c0 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c9424d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c00df20 con 0x557c336e7be0 2024-08-02T17:47:15.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c337124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.701+0000 7f4c8bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c337124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49758/0 (socket says 172.21.15.69:49758) 2024-08-02T17:47:15.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c8bfff640 1 -- 172.21.15.69:0/3262524371 learned_addr learned my addr 172.21.15.69:0/3262524371 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:15.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336e2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c83fff640 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 0x557c336efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c8bfff640 1 -- 172.21.15.69:0/3262524371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 msgr2=0x557c336efc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c8bfff640 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 0x557c336efc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c8bfff640 1 -- 172.21.15.69:0/3262524371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 msgr2=0x557c336e2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c8bfff640 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336e2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c8bfff640 1 -- 172.21.15.69:0/3262524371 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c336e7040 con 0x557c336f24a0 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c8bfff640 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c337124e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400f900 tx=0x7f4c84016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15ea4a0b39c7b05 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c817fa640 1 -- 172.21.15.69:0/3262524371 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84003970 con 0x557c336f24a0 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c817fa640 1 -- 172.21.15.69:0/3262524371 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84004020 con 0x557c336f24a0 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.702+0000 7f4c817fa640 1 -- 172.21.15.69:0/3262524371 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84028600 con 0x557c336f24a0 2024-08-02T17:47:15.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.703+0000 7f4c9424d740 1 -- 172.21.15.69:0/3262524371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 msgr2=0x557c337124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.703+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c337124e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400f900 tx=0x7f4c84016000 comp rx=0 tx=0).stop 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.703+0000 7f4c9424d740 1 -- 172.21.15.69:0/3262524371 shutdown_connections 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.703+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336f24a0 0x557c337124e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.703+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336e7be0 0x557c336efc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.703+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3262524371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e72c0 0x557c336e2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.703+0000 7f4c9424d740 1 -- 172.21.15.69:0/3262524371 >> 172.21.15.69:0/3262524371 conn(0x557c336e3850 msgr2=0x557c336f1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.704+0000 7f4c9424d740 1 -- 172.21.15.69:0/3262524371 shutdown_connections 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.704+0000 7f4c9424d740 1 -- 172.21.15.69:0/3262524371 wait complete. 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.704+0000 7f4c9424d740 1 -- start start 2024-08-02T17:47:15.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.704+0000 7f4c9424d740 1 Processor -- start 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.705+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e72c0 0x557c33702d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.705+0000 7f4c9424d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e7be0 0x557c33703250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.705+0000 7f4c8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e72c0 0x557c33702d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.705+0000 7f4c8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e72c0 0x557c33702d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49764/0 (socket says 172.21.15.69:49764) 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.705+0000 7f4c8b7fe640 1 -- 172.21.15.69:0/3164876668 learned_addr learned my addr 172.21.15.69:0/3164876668 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.705+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336f24a0 0x557c336ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.705+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c00e590 con 0x557c336e72c0 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.705+0000 7f4c83fff640 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e7be0 0x557c33703250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c00fa00 con 0x557c336e7be0 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6c00ec60 con 0x557c336f24a0 2024-08-02T17:47:15.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c8bfff640 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336f24a0 0x557c336ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c9424d740 10 client.0 _ll_register_callbacks cb 0x7f4c7c00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c83fff640 1 -- 172.21.15.69:0/3164876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336f24a0 msgr2=0x557c336ff1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c83fff640 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336f24a0 0x557c336ff1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c83fff640 1 -- 172.21.15.69:0/3164876668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e72c0 msgr2=0x557c33702d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c83fff640 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e72c0 0x557c33702d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c83fff640 1 -- 172.21.15.69:0/3164876668 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c336ff9f0 con 0x557c336e7be0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c83fff640 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e7be0 0x557c33703250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x557c336e4bd0 tx=0x7f4c74003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f13e7bbeb57faf0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.706+0000 7f4c6affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.707+0000 7f4c6affd640 1 -- 172.21.15.69:0/3164876668 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74049040 con 0x557c336e7be0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.707+0000 7f4c6affd640 1 -- 172.21.15.69:0/3164876668 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4c740f3b50 con 0x557c336e7be0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.707+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c84004020 con 0x557c336e7be0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.707+0000 7f4c6affd640 1 -- 172.21.15.69:0/3164876668 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c740bb7f0 con 0x557c336e7be0 2024-08-02T17:47:15.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.707+0000 7f4c9424d740 10 client.25271 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.707+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c337760c0 con 0x557c336e7be0 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.707+0000 7f4c6affd640 1 -- 172.21.15.69:0/3164876668 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c84004020 con 0x557c336e7be0 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.707+0000 7f4c6affd640 1 -- 172.21.15.69:0/3164876668 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4c740b8500 con 0x557c336e7be0 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c9424d740 10 client.25271 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c9424d740 10 client.25271 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c9424d740 20 client.25271 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557c33776720 0x557c33778b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c9424d740 4 client.25271 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.95, "await_for": 45.0}] 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.95, "await_for": 45.0}) -- 0x557c335d0f40 con 0x557c33776720 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557c33776720 0x557c33778b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c6affd640 10 client.25271 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:15.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:15.708+0000 7f4c8b7fe640 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557c33776720 0x557c33778b40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a912732cd047cafa server_cookie=4a0db579ebd535d3 in_seq=0 out_seq=0 2024-08-02T17:47:15.743 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 -- 172.21.15.69:0/1535130889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 msgr2=0x7f8bac106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/1535130889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 0x7f8bac106ac0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009930 tx=0x7f8b9c02a420 comp rx=0 tx=0).stop 2024-08-02T17:47:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 -- 172.21.15.69:0/1535130889 shutdown_connections 2024-08-02T17:47:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/1535130889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bac101070 0x7f8bac10d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/1535130889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac1006d0 0x7f8bac100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/1535130889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 0x7f8bac106ac0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 -- 172.21.15.69:0/1535130889 >> 172.21.15.69:0/1535130889 conn(0x7f8bac0776d0 msgr2=0x7f8bac0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 -- 172.21.15.69:0/1535130889 shutdown_connections 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 -- 172.21.15.69:0/1535130889 wait complete. 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.647+0000 7f8bb09ba640 1 Processor -- start 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8bb09ba640 1 -- start start 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8bb09ba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bac1006d0 0x7f8bac198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8bb09ba640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac101070 0x7f8bac199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8bb09ba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 0x7f8bac1a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8bb09ba640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac110ae0 con 0x7f8bac1066e0 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8bb09ba640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac110960 con 0x7f8bac1006d0 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8bb09ba640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac110c60 con 0x7f8bac101070 2024-08-02T17:47:16.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 0x7f8bac1a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baa59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bac1006d0 0x7f8bac198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8ba9d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac101070 0x7f8bac199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 0x7f8bac1a3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49822/0 (socket says 172.21.15.69:49822) 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 -- 172.21.15.69:0/3040661356 learned_addr learned my addr 172.21.15.69:0/3040661356 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baa59c640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bac1006d0 0x7f8bac198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55386/0 (socket says 172.21.15.69:55386) 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 -- 172.21.15.69:0/3040661356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac101070 msgr2=0x7f8bac199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac101070 0x7f8bac199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 -- 172.21.15.69:0/3040661356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bac1006d0 msgr2=0x7f8bac198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bac1006d0 0x7f8bac198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 -- 172.21.15.69:0/3040661356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac1a4490 con 0x7f8bac1066e0 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8baad9d640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 0x7f8bac1a3f50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400eb10 tx=0x7f8ba400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.648+0000 7f8b9b7fe640 1 -- 172.21.15.69:0/3040661356 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400ce20 con 0x7f8bac1066e0 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.649+0000 7f8bb09ba640 1 -- 172.21.15.69:0/3040661356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bac1a4660 con 0x7f8bac1066e0 2024-08-02T17:47:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.649+0000 7f8b9b7fe640 1 -- 172.21.15.69:0/3040661356 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba4005020 con 0x7f8bac1066e0 2024-08-02T17:47:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.649+0000 7f8b9b7fe640 1 -- 172.21.15.69:0/3040661356 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4010690 con 0x7f8bac1066e0 2024-08-02T17:47:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.649+0000 7f8bb09ba640 1 -- 172.21.15.69:0/3040661356 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bac1a4bb0 con 0x7f8bac1066e0 2024-08-02T17:47:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8b9b7fe640 1 -- 172.21.15.69:0/3040661356 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f8ba4005540 con 0x7f8bac1066e0 2024-08-02T17:47:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8bb37ea740 1 Processor -- start 2024-08-02T17:47:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8bb37ea740 1 -- start start 2024-08-02T17:47:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb17b0 0x555aa0bb1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0bbc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 0x555aa0bbed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555aa0b159b0 con 0x555aa0bb17b0 2024-08-02T17:47:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aa0abd800 con 0x555aa0bbc970 2024-08-02T17:47:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555aa0b11970 con 0x555aa0bb20d0 2024-08-02T17:47:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8ba9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb17b0 0x555aa0bb1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8ba9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb17b0 0x555aa0bb1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49828/0 (socket says 172.21.15.69:49828) 2024-08-02T17:47:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8baa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0bbc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.651+0000 7f8ba9d9b640 1 -- 172.21.15.69:0/2412719049 learned_addr learned my addr 172.21.15.69:0/2412719049 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 0x555aa0bbed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8b9b7fe640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8b940a2ea0 0x7f8b940a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8ba9d9b640 1 -- 172.21.15.69:0/2412719049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 msgr2=0x555aa0bbc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0bbc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8baad9d640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8b940a2ea0 0x7f8b940a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8ba9d9b640 1 -- 172.21.15.69:0/2412719049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 msgr2=0x555aa0bbed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 0x555aa0bbed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8ba9d9b640 1 -- 172.21.15.69:0/2412719049 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa0829210 con 0x555aa0bb17b0 2024-08-02T17:47:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8baa59c640 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0bbc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8b9b7fe640 1 -- 172.21.15.69:0/3040661356 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8ba40d0010 con 0x7f8bac1066e0 2024-08-02T17:47:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb17b0 0x555aa0bb1b90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0013610 tx=0x7f8ba000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516ad0f531cf1fee server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8b98ff9640 1 -- 172.21.15.69:0/2412719049 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000d710 con 0x555aa0bb17b0 2024-08-02T17:47:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8b98ff9640 1 -- 172.21.15.69:0/2412719049 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba000dd90 con 0x555aa0bb17b0 2024-08-02T17:47:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8b98ff9640 1 -- 172.21.15.69:0/2412719049 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000fca0 con 0x555aa0bb17b0 2024-08-02T17:47:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.652+0000 7f8baad9d640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8b940a2ea0 0x7f8b940a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40cd1c0 tx=0x7f8ba40054b0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2412719049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb17b0 msgr2=0x555aa0bb1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb17b0 0x555aa0bb1b90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0013610 tx=0x7f8ba000a710 comp rx=0 tx=0).stop 2024-08-02T17:47:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2412719049 shutdown_connections 2024-08-02T17:47:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 0x555aa0bbed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0bbc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/2412719049 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb17b0 0x555aa0bb1b90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2412719049 >> 172.21.15.69:0/2412719049 conn(0x555aa0badd20 msgr2=0x555aa0bae100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2412719049 shutdown_connections 2024-08-02T17:47:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2412719049 wait complete. 2024-08-02T17:47:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 -- start start 2024-08-02T17:47:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.653+0000 7f8bb37ea740 1 Processor -- start 2024-08-02T17:47:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0bb9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bb9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bbc970 0x555aa0bba2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555aa0bbf540 con 0x555aa0bb20d0 2024-08-02T17:47:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555aa0ad1500 con 0x555aa0bb17b0 2024-08-02T17:47:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555aa0bbf6c0 con 0x555aa0bbc970 2024-08-02T17:47:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8bb37ea740 10 client.0 _ll_register_callbacks cb 0x555aa0828a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8baa59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bb9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8ba9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0bb9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8ba9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0bb9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55398/0 (socket says 172.21.15.69:55398) 2024-08-02T17:47:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8ba9d9b640 1 -- 172.21.15.69:0/1781736996 learned_addr learned my addr 172.21.15.69:0/1781736996 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bbc970 0x555aa0bba2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8baa59c640 1 -- 172.21.15.69:0/1781736996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bbc970 msgr2=0x555aa0bba2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8baa59c640 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bbc970 0x555aa0bba2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8baa59c640 1 -- 172.21.15.69:0/1781736996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 msgr2=0x555aa0bb9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8baa59c640 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0bb9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8baa59c640 1 -- 172.21.15.69:0/1781736996 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa08f5620 con 0x555aa0bb20d0 2024-08-02T17:47:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8baa59c640 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bb9d80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c005940 tx=0x7f8b9c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87709e3bc7c58e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8b91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8b91ffb640 1 -- 172.21.15.69:0/1781736996 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c035040 con 0x555aa0bb20d0 2024-08-02T17:47:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8b91ffb640 1 -- 172.21.15.69:0/1781736996 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8b9c004240 con 0x555aa0bb20d0 2024-08-02T17:47:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.654+0000 7f8bb37ea740 1 -- 172.21.15.69:0/1781736996 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ba000dd90 con 0x555aa0bb20d0 2024-08-02T17:47:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8b91ffb640 1 -- 172.21.15.69:0/1781736996 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b9c004440 con 0x555aa0bb20d0 2024-08-02T17:47:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8bb37ea740 10 client.15633 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8bb37ea740 1 -- 172.21.15.69:0/1781736996 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555aa0bcb160 con 0x555aa0bb20d0 2024-08-02T17:47:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8b91ffb640 1 -- 172.21.15.69:0/1781736996 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba000dd90 con 0x555aa0bb20d0 2024-08-02T17:47:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8b91ffb640 1 -- 172.21.15.69:0/1781736996 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8b9c004bf0 con 0x555aa0bb20d0 2024-08-02T17:47:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8bb37ea740 10 client.15633 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8bb37ea740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8bb37ea740 20 client.15633 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa0bcc620 0x555aa0bcea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8bb37ea740 4 client.15633 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8bb37ea740 1 -- 172.21.15.69:0/1781736996 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aa0b230e0 con 0x555aa0bcc620 2024-08-02T17:47:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa0bcc620 0x555aa0bcea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8b91ffb640 10 client.15633 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.655+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa0bcc620 0x555aa0bcea10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ef1fe0d46f6c6fc server_cookie=64a6235d9c20c103 in_seq=0 out_seq=0 2024-08-02T17:47:16.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8b91ffb640 1 -- 172.21.15.69:0/1781736996 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555aa0b230e0 con 0x555aa0bcc620 2024-08-02T17:47:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8b91ffb640 10 client.15633 handle_command_reply: tid=0 2024-08-02T17:47:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 client.15633 shutdown 2024-08-02T17:47:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 -- 172.21.15.69:0/1781736996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 msgr2=0x555aa0bb9d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bb9d80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c005940 tx=0x7f8b9c031920 comp rx=0 tx=0).stop 2024-08-02T17:47:16.687 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 -- 172.21.15.69:0/1781736996 shutdown_connections 2024-08-02T17:47:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bb9d80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa0bcc620 0x555aa0bcea10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bbc970 0x555aa0bba2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8b91ffb640 0 client.15633 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/1781736996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0bb9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 -- 172.21.15.69:0/1781736996 >> 172.21.15.69:0/1781736996 conn(0x555aa0badd20 msgr2=0x555aa0bb3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 -- 172.21.15.69:0/1781736996 shutdown_connections 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 1 -- 172.21.15.69:0/1781736996 wait complete. 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.658+0000 7f8bb37ea740 20 client.15633 trim_cache size 0 max 16384 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.673+0000 7f8bb37ea740 1 Processor -- start 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.673+0000 7f8bb37ea740 1 -- start start 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb17b0 0x555aa0bc4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bc5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 0x555aa0bd42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac1b1570 con 0x555aa0bb20d0 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac1aee40 con 0x555aa0bbc970 2024-08-02T17:47:16.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac1acc90 con 0x555aa0bb17b0 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bc5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bc5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49850/0 (socket says 172.21.15.69:49850) 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 -- 172.21.15.69:0/2777850830 learned_addr learned my addr 172.21.15.69:0/2777850830 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb17b0 0x555aa0bc4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 0x555aa0bd42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 -- 172.21.15.69:0/2777850830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb17b0 msgr2=0x555aa0bc4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb17b0 0x555aa0bc4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 -- 172.21.15.69:0/2777850830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 msgr2=0x555aa0bd42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 0x555aa0bd42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 -- 172.21.15.69:0/2777850830 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aa0bcf040 con 0x555aa0bb20d0 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8baa59c640 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bc5270 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c031550 tx=0x7f8b9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad011e2f14c7853c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8b93fff640 1 -- 172.21.15.69:0/2777850830 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c000e60 con 0x555aa0bb20d0 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8b93fff640 1 -- 172.21.15.69:0/2777850830 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9c004140 con 0x555aa0bb20d0 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.674+0000 7f8b93fff640 1 -- 172.21.15.69:0/2777850830 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c035030 con 0x555aa0bb20d0 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2777850830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 msgr2=0x555aa0bc5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bc5270 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c031550 tx=0x7f8b9c000bc0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2777850830 shutdown_connections 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bb20d0 0x555aa0bc5270 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bbc970 0x555aa0bd42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/2777850830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb17b0 0x555aa0bc4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2777850830 >> 172.21.15.69:0/2777850830 conn(0x555aa0badd20 msgr2=0x555aa0bbb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2777850830 shutdown_connections 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 -- 172.21.15.69:0/2777850830 wait complete. 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 -- start start 2024-08-02T17:47:16.691 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.675+0000 7f8bb37ea740 1 Processor -- start 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0c34580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0c34ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8bb37ea740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bbc970 0x555aa0c332a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac1ad300 con 0x555aa0bbc970 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac1ae770 con 0x555aa0bb17b0 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8bb37ea740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac1ad9d0 con 0x555aa0bb20d0 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8bb37ea740 10 client.0 _ll_register_callbacks cb 0x7f8b9c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8ba9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0c34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8baa59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0c34ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8ba9d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0c34580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55416/0 (socket says 172.21.15.69:55416) 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8ba9d9b640 1 -- 172.21.15.69:0/3985380157 learned_addr learned my addr 172.21.15.69:0/3985380157 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bbc970 0x555aa0c332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8baa59c640 1 -- 172.21.15.69:0/3985380157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 msgr2=0x555aa0c34580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8baa59c640 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0c34580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8baa59c640 1 -- 172.21.15.69:0/3985380157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bbc970 msgr2=0x555aa0c332a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8baa59c640 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bbc970 0x555aa0c332a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8baa59c640 1 -- 172.21.15.69:0/3985380157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aa0bae9b0 con 0x555aa0bb20d0 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0c34580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8baa59c640 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0c34ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c036040 tx=0x7f8b9c009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4baa8832a2fe8a4f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8b91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8b91ffb640 1 -- 172.21.15.69:0/3985380157 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c000e40 con 0x555aa0bb20d0 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8bb37ea740 1 -- 172.21.15.69:0/3985380157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b9c004140 con 0x555aa0bb20d0 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.676+0000 7f8b91ffb640 1 -- 172.21.15.69:0/3985380157 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8b9c003e20 con 0x555aa0bb20d0 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8b91ffb640 1 -- 172.21.15.69:0/3985380157 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8b9c004770 con 0x555aa0bb20d0 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8bb37ea740 10 client.25192 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8bb37ea740 1 -- 172.21.15.69:0/3985380157 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555aa0badb50 con 0x555aa0bb20d0 2024-08-02T17:47:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8b91ffb640 1 -- 172.21.15.69:0/3985380157 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b9c004140 con 0x555aa0bb20d0 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8b91ffb640 1 -- 172.21.15.69:0/3985380157 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8b9c0053d0 con 0x555aa0bb20d0 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8bb37ea740 10 client.25192 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8bb37ea740 10 client.25192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8bb37ea740 20 client.25192 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa0c3c020 0x555aa0c3e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8bb37ea740 4 client.25192 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8bb37ea740 1 -- 172.21.15.69:0/3985380157 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555aa0b1d430 con 0x555aa0c3c020 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.677+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa0c3c020 0x555aa0c3e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.678+0000 7f8b91ffb640 10 client.25192 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.678+0000 7f8ba9d9b640 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa0c3c020 0x555aa0c3e410 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d69fb259c281f58 server_cookie=6458458e2d1d32a in_seq=0 out_seq=0 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8b91ffb640 1 -- 172.21.15.69:0/3985380157 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555aa0b1d430 con 0x555aa0c3c020 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8b91ffb640 10 client.25192 handle_command_reply: tid=0 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 client.25192 shutdown 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 -- 172.21.15.69:0/3985380157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 msgr2=0x555aa0c34ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0c34ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c036040 tx=0x7f8b9c009300 comp rx=0 tx=0).stop 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 -- 172.21.15.69:0/3985380157 shutdown_connections 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555aa0bbc970 0x555aa0c332a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555aa0c3c020 0x555aa0c3e410 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555aa0bb20d0 0x555aa0c34ac0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8b91ffb640 0 client.25192 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 --2- 172.21.15.69:0/3985380157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555aa0bb17b0 0x555aa0c34580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 -- 172.21.15.69:0/3985380157 >> 172.21.15.69:0/3985380157 conn(0x555aa0badd20 msgr2=0x555aa0bb97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.890+0000 7f8bb37ea740 1 -- 172.21.15.69:0/3985380157 shutdown_connections 2024-08-02T17:47:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb37ea740 1 -- 172.21.15.69:0/3985380157 wait complete. 2024-08-02T17:47:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb37ea740 20 client.25192 trim_cache size 0 max 16384 2024-08-02T17:47:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:47:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:47:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:47:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:47:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 -- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8b940a2ea0 msgr2=0x7f8b940a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8b940a2ea0 0x7f8b940a5360 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40cd1c0 tx=0x7f8ba40054b0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 -- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 msgr2=0x7f8bac1a3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 0x7f8bac1a3f50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400eb10 tx=0x7f8ba400ef10 comp rx=0 tx=0).stop 2024-08-02T17:47:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 -- 172.21.15.69:0/3040661356 shutdown_connections 2024-08-02T17:47:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bac1066e0 0x7f8bac1a3f50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8b940a2ea0 0x7f8b940a5360 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bac101070 0x7f8bac199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 --2- 172.21.15.69:0/3040661356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bac1006d0 0x7f8bac198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 -- 172.21.15.69:0/3040661356 >> 172.21.15.69:0/3040661356 conn(0x7f8bac0776d0 msgr2=0x7f8bac103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 -- 172.21.15.69:0/3040661356 shutdown_connections 2024-08-02T17:47:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:16.891+0000 7f8bb09ba640 1 -- 172.21.15.69:0/3040661356 wait complete. 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/963900922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc1007c0 msgr2=0x7f5fdc100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/963900922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc1007c0 0x7f5fdc100c20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009930 tx=0x7f5fc402a420 comp rx=0 tx=0).stop 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/963900922 shutdown_connections 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/963900922 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc1007c0 0x7f5fdc100c20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/963900922 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fdc101160 0x7f5fdc10d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/963900922 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fdc1067c0 0x7f5fdc106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/963900922 >> 172.21.15.69:0/963900922 conn(0x7f5fdc0fc500 msgr2=0x7f5fdc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/963900922 shutdown_connections 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.239+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/963900922 wait complete. 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fe1d8c640 1 Processor -- start 2024-08-02T17:47:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fe1d8c640 1 -- start start 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fe1d8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fdc1007c0 0x7f5fdc198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fe1d8c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc101160 0x7f5fdc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fe1d8c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fdc1067c0 0x7f5fdc19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fe1d8c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc110bc0 con 0x7f5fdc101160 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fe1d8c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc110a40 con 0x7f5fdc1067c0 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fe1d8c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc110d40 con 0x7f5fdc1007c0 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc101160 0x7f5fdc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc101160 0x7f5fdc1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49776/0 (socket says 172.21.15.69:49776) 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fd3fff640 1 -- 172.21.15.69:0/998759492 learned_addr learned my addr 172.21.15.69:0/998759492 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fdc1007c0 0x7f5fdc198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fdc1067c0 0x7f5fdc19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fd3fff640 1 -- 172.21.15.69:0/998759492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fdc1007c0 msgr2=0x7f5fdc198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fdc1007c0 0x7f5fdc198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fd3fff640 1 -- 172.21.15.69:0/998759492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fdc1067c0 msgr2=0x7f5fdc19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fdc1067c0 0x7f5fdc19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.240+0000 7f5fd3fff640 1 -- 172.21.15.69:0/998759492 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc19d110 con 0x7f5fdc101160 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.241+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc101160 0x7f5fdc1986a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009230 tx=0x7f5fc4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.241+0000 7f5fd97fa640 1 -- 172.21.15.69:0/998759492 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc403f030 con 0x7f5fdc101160 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.241+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/998759492 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc19d3a0 con 0x7f5fdc101160 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.241+0000 7f5fd97fa640 1 -- 172.21.15.69:0/998759492 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fc4004440 con 0x7f5fdc101160 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.241+0000 7f5fd97fa640 1 -- 172.21.15.69:0/998759492 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4035070 con 0x7f5fdc101160 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.241+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/998759492 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1a4ce0 con 0x7f5fdc101160 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.243+0000 7f5fd97fa640 1 -- 172.21.15.69:0/998759492 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f5fc4042030 con 0x7f5fdc101160 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.243+0000 7f5fe418a740 1 Processor -- start 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.243+0000 7f5fe418a740 1 -- start start 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a67f0 0x55e7293b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293b55e0 0x55e7293b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fe418a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e72930ba40 con 0x55e7293a5ed0 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fe418a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e7292a39d0 con 0x55e7293a67f0 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fe418a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e729307a00 con 0x55e7293b55e0 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a67f0 0x55e7293b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293a62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49780/0 (socket says 172.21.15.69:49780) 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdbfff640 1 -- 172.21.15.69:0/2026787962 learned_addr learned my addr 172.21.15.69:0/2026787962 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293b55e0 0x55e7293b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/2026787962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293b55e0 msgr2=0x55e7293b79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293b55e0 0x55e7293b79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/2026787962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 msgr2=0x55e7293a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/2026787962 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7292aa650 con 0x55e7293a67f0 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fd97fa640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fc80a2ea0 0x7f5fc80a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fc80a2ea0 0x7f5fc80a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a67f0 0x55e7293b50a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009c20 tx=0x7f5fcc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eee7fe8dc679282 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fd97fa640 1 -- 172.21.15.69:0/998759492 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5fc40efdd0 con 0x7f5fdc101160 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.244+0000 7f5fd1ffb640 1 -- 172.21.15.69:0/2026787962 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc004050 con 0x55e7293a67f0 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fd1ffb640 1 -- 172.21.15.69:0/2026787962 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fcc005500 con 0x55e7293a67f0 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fd1ffb640 1 -- 172.21.15.69:0/2026787962 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc004e00 con 0x55e7293a67f0 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 -- 172.21.15.69:0/2026787962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a67f0 msgr2=0x55e7293b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fc80a2ea0 0x7f5fc80a5360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4011850 tx=0x7f5fd4009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a67f0 0x55e7293b50a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009c20 tx=0x7f5fcc011810 comp rx=0 tx=0).stop 2024-08-02T17:47:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 -- 172.21.15.69:0/2026787962 shutdown_connections 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293b55e0 0x55e7293b79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a67f0 0x55e7293b50a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 --2- 172.21.15.69:0/2026787962 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 -- 172.21.15.69:0/2026787962 >> 172.21.15.69:0/2026787962 conn(0x55e7293a2580 msgr2=0x55e7293a2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 -- 172.21.15.69:0/2026787962 shutdown_connections 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.245+0000 7f5fe418a740 1 -- 172.21.15.69:0/2026787962 wait complete. 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 1 -- start start 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 1 Processor -- start 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e7293b81b0 con 0x55e7293b55e0 2024-08-02T17:47:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e7292c6b20 con 0x55e7293a5ed0 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7293b8330 con 0x55e7293a67f0 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fe418a740 10 client.0 _ll_register_callbacks cb 0x55e7292c4bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fd3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55348/0 (socket says 172.21.15.69:55348) 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/1008351494 learned_addr learned my addr 172.21.15.69:0/1008351494 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.246+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49782/0 (socket says 172.21.15.69:49782) 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/1008351494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 msgr2=0x55e7293b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/1008351494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 msgr2=0x55e7293b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/1008351494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72907d3c0 con 0x55e7293a5ed0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b24f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc01b040 tx=0x7f5fcc004550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01a29356aa95ddb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1008351494 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc029070 con 0x55e7293a5ed0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fe418a740 1 -- 172.21.15.69:0/1008351494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fcc005500 con 0x55e7293a5ed0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1008351494 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5fcc004d80 con 0x55e7293a5ed0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1008351494 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc035050 con 0x55e7293a5ed0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fe418a740 10 client.25283 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fe418a740 1 -- 172.21.15.69:0/1008351494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7293c3e20 con 0x55e7293a5ed0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1008351494 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fcc005500 con 0x55e7293a5ed0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.247+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1008351494 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5fcc023430 con 0x55e7293a5ed0 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fe418a740 10 client.25283 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:16.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fe418a740 10 client.25283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fe418a740 20 client.25283 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e7293c51d0 0x55e7293c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fe418a740 4 client.25283 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fe418a740 1 -- 172.21.15.69:0/1008351494 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e729318fb0 con 0x55e7293c51d0 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e7293c51d0 0x55e7293c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fbaffd640 10 client.25283 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.248+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e7293c51d0 0x55e7293c75c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc849d77daecbb40 server_cookie=a0ba25eeb73ab47f in_seq=0 out_seq=0 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1008351494 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e729318fb0 con 0x55e7293c51d0 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fbaffd640 10 client.25283 handle_command_reply: tid=0 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 client.25283 shutdown 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 -- 172.21.15.69:0/1008351494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 msgr2=0x55e7293b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b24f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc01b040 tx=0x7f5fcc004550 comp rx=0 tx=0).stop 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 -- 172.21.15.69:0/1008351494 shutdown_connections 2024-08-02T17:47:16.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e7293c51d0 0x55e7293c75c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1008351494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b24f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fbaffd640 0 client.25283 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 -- 172.21.15.69:0/1008351494 >> 172.21.15.69:0/1008351494 conn(0x55e7293a2580 msgr2=0x55e7293a82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 -- 172.21.15.69:0/1008351494 shutdown_connections 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 1 -- 172.21.15.69:0/1008351494 wait complete. 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.249+0000 7f5fe418a740 20 client.25283 trim_cache size 0 max 16384 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fe418a740 1 Processor -- start 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fe418a740 1 -- start start 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293b55e0 0x55e7293cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fe418a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1af480 con 0x55e7293a5ed0 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fe418a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1acd50 con 0x55e7293b55e0 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fe418a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1aabc0 con 0x55e7293a67f0 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49798/0 (socket says 172.21.15.69:49798) 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/4195452678 learned_addr learned my addr 172.21.15.69:0/4195452678 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293b55e0 0x55e7293cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/4195452678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 msgr2=0x55e7293ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/4195452678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293b55e0 msgr2=0x55e7293cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293b55e0 0x55e7293cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdb7fe640 1 -- 172.21.15.69:0/4195452678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7293b7d70 con 0x55e7293a5ed0 2024-08-02T17:47:16.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.262+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293b55e0 0x55e7293cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:16.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293b1ca0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009560 tx=0x7f5fcc0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b3430bd720ecc4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fd17fa640 1 -- 172.21.15.69:0/4195452678 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc029070 con 0x55e7293a5ed0 2024-08-02T17:47:16.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fd17fa640 1 -- 172.21.15.69:0/4195452678 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fcc005b90 con 0x55e7293a5ed0 2024-08-02T17:47:16.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fd17fa640 1 -- 172.21.15.69:0/4195452678 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc02e440 con 0x55e7293a5ed0 2024-08-02T17:47:16.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 -- 172.21.15.69:0/4195452678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 msgr2=0x55e7293b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293b1ca0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009560 tx=0x7f5fcc0045c0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 -- 172.21.15.69:0/4195452678 shutdown_connections 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293b55e0 0x55e7293cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 --2- 172.21.15.69:0/4195452678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293a5ed0 0x55e7293b1ca0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 -- 172.21.15.69:0/4195452678 >> 172.21.15.69:0/4195452678 conn(0x55e7293a2580 msgr2=0x55e7293a0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 -- 172.21.15.69:0/4195452678 shutdown_connections 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.263+0000 7f5fe418a740 1 -- 172.21.15.69:0/4195452678 wait complete. 2024-08-02T17:47:16.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 1 -- start start 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 1 Processor -- start 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1ab210 con 0x55e7293b55e0 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1ac680 con 0x55e7293a5ed0 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc1ab8e0 con 0x55e7293a67f0 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fe418a740 10 client.0 _ll_register_callbacks cb 0x7f5fd4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49814/0 (socket says 172.21.15.69:49814) 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdbfff640 1 -- 172.21.15.69:0/1007948113 learned_addr learned my addr 172.21.15.69:0/1007948113 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fd3fff640 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdbfff640 1 -- 172.21.15.69:0/1007948113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 msgr2=0x55e7293b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdbfff640 1 -- 172.21.15.69:0/1007948113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 msgr2=0x55e7293b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.264+0000 7f5fdbfff640 1 -- 172.21.15.69:0/1007948113 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e72942dd50 con 0x55e7293b55e0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fdbfff640 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b9000 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4011700 tx=0x7f5fd4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c0d1a443047c5c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1007948113 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4028430 con 0x55e7293b55e0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1007948113 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5fd40285d0 con 0x55e7293b55e0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 1 -- 172.21.15.69:0/1007948113 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fcc005b90 con 0x55e7293b55e0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1007948113 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5fd4036050 con 0x55e7293b55e0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 10 client.15621 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 1 -- 172.21.15.69:0/1007948113 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e72942f090 con 0x55e7293b55e0 2024-08-02T17:47:16.946 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1007948113 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fcc005b90 con 0x55e7293b55e0 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1007948113 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5fd4031a60 con 0x55e7293b55e0 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 10 client.15621 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 10 client.15621 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 20 client.15621 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e72942f890 0x55e729431cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 4 client.15621 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.265+0000 7f5fe418a740 1 -- 172.21.15.69:0/1007948113 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e7293134c0 con 0x55e72942f890 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e72942f890 0x55e729431cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fbaffd640 10 client.15621 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fdb7fe640 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e72942f890 0x55e729431cb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b79348fd6ef2f0c server_cookie=d914d82b6aa82e3c in_seq=0 out_seq=0 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fbaffd640 1 -- 172.21.15.69:0/1007948113 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e7293134c0 con 0x55e72942f890 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fbaffd640 10 client.15621 handle_command_reply: tid=0 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 client.15621 shutdown 2024-08-02T17:47:16.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 -- 172.21.15.69:0/1007948113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 msgr2=0x55e7293b9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b9000 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4011700 tx=0x7f5fd4004ab0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 -- 172.21.15.69:0/1007948113 shutdown_connections 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e72942f890 0x55e729431cb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7293b55e0 0x55e7293b9000 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7293a67f0 0x55e7293b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 --2- 172.21.15.69:0/1007948113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7293a5ed0 0x55e7293b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 -- 172.21.15.69:0/1007948113 >> 172.21.15.69:0/1007948113 conn(0x55e7293a2580 msgr2=0x55e7293c0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fbaffd640 0 client.15621 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 -- 172.21.15.69:0/1007948113 shutdown_connections 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 1 -- 172.21.15.69:0/1007948113 wait complete. 2024-08-02T17:47:16.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.266+0000 7f5fe418a740 20 client.15621 trim_cache size 0 max 16384 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fc80a2ea0 msgr2=0x7f5fc80a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fc80a2ea0 0x7f5fc80a5360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4011850 tx=0x7f5fd4009500 comp rx=0 tx=0).stop 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc101160 msgr2=0x7f5fdc1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc101160 0x7f5fdc1986a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009230 tx=0x7f5fc4004040 comp rx=0 tx=0).stop 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/998759492 shutdown_connections 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fdc101160 0x7f5fdc1986a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fc80a2ea0 0x7f5fc80a5360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fdc1067c0 0x7f5fdc19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 --2- 172.21.15.69:0/998759492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fdc1007c0 0x7f5fdc198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/998759492 >> 172.21.15.69:0/998759492 conn(0x7f5fdc0fc500 msgr2=0x7f5fdc06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/998759492 shutdown_connections 2024-08-02T17:47:16.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:16.267+0000 7f5fe1d8c640 1 -- 172.21.15.69:0/998759492 wait complete. 2024-08-02T17:47:17.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.013+0000 7f4c6affd640 1 -- 172.21.15.69:0/3164876668 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x557c335d0f40 con 0x557c33776720 2024-08-02T17:47:17.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.013+0000 7f4c6affd640 10 client.25271 handle_command_reply: tid=0 2024-08-02T17:47:17.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.013+0000 7f4c9424d740 1 client.25271 shutdown 2024-08-02T17:47:17.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e7be0 msgr2=0x557c33703250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e7be0 0x557c33703250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x557c336e4bd0 tx=0x7f4c74003f90 comp rx=0 tx=0).stop 2024-08-02T17:47:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 shutdown_connections 2024-08-02T17:47:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557c33776720 0x557c33778b40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557c336f24a0 0x557c336ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557c336e7be0 0x557c33703250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c6affd640 0 client.25271 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:17.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 --2- 172.21.15.69:0/3164876668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557c336e72c0 0x557c33702d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:17.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 >> 172.21.15.69:0/3164876668 conn(0x557c336e3850 msgr2=0x557c336e8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:17.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 shutdown_connections 2024-08-02T17:47:17.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 1 -- 172.21.15.69:0/3164876668 wait complete. 2024-08-02T17:47:17.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c9424d740 20 client.25271 trim_cache size 0 max 16384 2024-08-02T17:47:17.018 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:47:17.018 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:47:17.018 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 2, 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "14b377d1": { 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 2, 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 141.9, 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:47:17.019 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:17.020 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c91e4f640 1 -- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4c780a2f20 msgr2=0x7f4c780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4c780a2f20 0x7f4c780a53e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c84005c40 tx=0x7f4c84009500 comp rx=0 tx=0).stop 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c91e4f640 1 -- 172.21.15.69:0/946623225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c8c103470 msgr2=0x7f4c8c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.014+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c8c103470 0x7f4c8c198890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009230 tx=0x7f4c74003b90 comp rx=0 tx=0).stop 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.015+0000 7f4c91e4f640 1 -- 172.21.15.69:0/946623225 shutdown_connections 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.015+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4c8c107fc0 0x7f4c8c19cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.015+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4c780a2f20 0x7f4c780a53e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.015+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4c8c103470 0x7f4c8c198890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.015+0000 7f4c91e4f640 1 --2- 172.21.15.69:0/946623225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4c8c102ac0 0x7f4c8c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.015+0000 7f4c91e4f640 1 -- 172.21.15.69:0/946623225 >> 172.21.15.69:0/946623225 conn(0x7f4c8c0fc500 msgr2=0x7f4c8c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:17.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.015+0000 7f4c91e4f640 1 -- 172.21.15.69:0/946623225 shutdown_connections 2024-08-02T17:47:17.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:17.015+0000 7f4c91e4f640 1 -- 172.21.15.69:0/946623225 wait complete. 2024-08-02T17:47:17.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:17 smithi077 ceph-mon[37108]: pgmap v29: 129 pgs: 129 active+clean; 555 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 910 op/s 2024-08-02T17:47:17.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:17 smithi069 ceph-mon[32746]: pgmap v29: 129 pgs: 129 active+clean; 555 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 910 op/s 2024-08-02T17:47:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:17 smithi083 ceph-mon[37031]: pgmap v29: 129 pgs: 129 active+clean; 555 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 19 MiB/s wr, 910 op/s 2024-08-02T17:47:18.218 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:47:18.751 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '14b377d1', quiesce took 8.8 seconds. Will release after: 13.149999999999999 2024-08-02T17:47:18.814 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:19.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 -- 172.21.15.69:0/1475448513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c1078b0 msgr2=0x7f476c107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 --2- 172.21.15.69:0/1475448513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c1078b0 0x7f476c107c90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f475c009930 tx=0x7f475c02a420 comp rx=0 tx=0).stop 2024-08-02T17:47:19.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 -- 172.21.15.69:0/1475448513 shutdown_connections 2024-08-02T17:47:19.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 --2- 172.21.15.69:0/1475448513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f476c102250 0x7f476c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 --2- 172.21.15.69:0/1475448513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f476c1018b0 0x7f476c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 --2- 172.21.15.69:0/1475448513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c1078b0 0x7f476c107c90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 -- 172.21.15.69:0/1475448513 >> 172.21.15.69:0/1475448513 conn(0x7f476c0fd5f0 msgr2=0x7f476c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 -- 172.21.15.69:0/1475448513 shutdown_connections 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.251+0000 7f4772635640 1 -- 172.21.15.69:0/1475448513 wait complete. 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f4772635640 1 Processor -- start 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f4772635640 1 -- start start 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f4772635640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f476c1018b0 0x7f476c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f4772635640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c102250 0x7f476c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f4772635640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f476c1078b0 0x7f476c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f4772635640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f476c111d50 con 0x7f476c102250 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f4772635640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f476c111bd0 con 0x7f476c1018b0 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f4772635640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f476c111ed0 con 0x7f476c1078b0 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.252+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c102250 0x7f476c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c102250 0x7f476c074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49884/0 (socket says 172.21.15.69:49884) 2024-08-02T17:47:19.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476b7fe640 1 -- 172.21.15.69:0/39298141 learned_addr learned my addr 172.21.15.69:0/39298141 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476bfff640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f476c1018b0 0x7f476c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f4770bd2640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f476c1078b0 0x7f476c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476b7fe640 1 -- 172.21.15.69:0/39298141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f476c1078b0 msgr2=0x7f476c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476b7fe640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f476c1078b0 0x7f476c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476b7fe640 1 -- 172.21.15.69:0/39298141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f476c1018b0 msgr2=0x7f476c074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476b7fe640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f476c1018b0 0x7f476c074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476b7fe640 1 -- 172.21.15.69:0/39298141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c1a6830 con 0x7f476c102250 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f476b7fe640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c102250 0x7f476c074760 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f476000b4a0 tx=0x7f476000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f47697fa640 1 -- 172.21.15.69:0/39298141 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760014070 con 0x7f476c102250 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f4772635640 1 -- 172.21.15.69:0/39298141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c1a6b20 con 0x7f476c102250 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f47697fa640 1 -- 172.21.15.69:0/39298141 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760010af0 con 0x7f476c102250 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f47697fa640 1 -- 172.21.15.69:0/39298141 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760017d10 con 0x7f476c102250 2024-08-02T17:47:19.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.253+0000 7f4772635640 1 -- 172.21.15.69:0/39298141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c1a7060 con 0x7f476c102250 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.255+0000 7f47697fa640 1 -- 172.21.15.69:0/39298141 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f4760017070 con 0x7f476c102250 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4774a33740 1 Processor -- start 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4774a33740 1 -- start start 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f47697fa640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f47540a2ea0 0x7f47540a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4774a33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 0x564d465873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4774a33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d46591c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4774a33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 0x564d465945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4774a33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564d464eca40 con 0x564d46587910 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4774a33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564d464849d0 con 0x564d465921d0 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4774a33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564d464e8a00 con 0x564d46586ff0 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f4770bd2640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f47540a2ea0 0x7f47540a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f47697fa640 1 -- 172.21.15.69:0/39298141 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f47600ced00 con 0x7f476c102250 2024-08-02T17:47:19.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d46591c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d46591c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49900/0 (socket says 172.21.15.69:49900) 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f476b7fe640 1 -- 172.21.15.69:0/3676364857 learned_addr learned my addr 172.21.15.69:0/3676364857 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.256+0000 7f476bfff640 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 0x564d465873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f4770bd2640 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 0x564d465945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f476b7fe640 1 -- 172.21.15.69:0/3676364857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 msgr2=0x564d465873d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f476b7fe640 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 0x564d465873d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f476b7fe640 1 -- 172.21.15.69:0/3676364857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 msgr2=0x564d465945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f476b7fe640 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 0x564d465945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f476b7fe640 1 -- 172.21.15.69:0/3676364857 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4648b650 con 0x564d46587910 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f4770bd2640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f47540a2ea0 0x7f47540a5360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4764009900 tx=0x7f4764009650 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f476b7fe640 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d46591c90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f476009ad70 tx=0x7f476009a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0f80813f3fe805 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f47527fc640 1 -- 172.21.15.69:0/3676364857 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476001e070 con 0x564d46587910 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f47527fc640 1 -- 172.21.15.69:0/3676364857 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47600a39a0 con 0x564d46587910 2024-08-02T17:47:19.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.257+0000 7f47527fc640 1 -- 172.21.15.69:0/3676364857 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476009a230 con 0x564d46587910 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 -- 172.21.15.69:0/3676364857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 msgr2=0x564d46591c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d46591c90 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f476009ad70 tx=0x7f476009a800 comp rx=0 tx=0).stop 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 -- 172.21.15.69:0/3676364857 shutdown_connections 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d46591c90 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 0x564d465945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 --2- 172.21.15.69:0/3676364857 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 0x564d465873d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 -- 172.21.15.69:0/3676364857 >> 172.21.15.69:0/3676364857 conn(0x564d46583580 msgr2=0x564d46583960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 -- 172.21.15.69:0/3676364857 shutdown_connections 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 -- 172.21.15.69:0/3676364857 wait complete. 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 -- start start 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 Processor -- start 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.258+0000 7f4774a33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 0x564d4658f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d4658f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 0x564d4658fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564d46594da0 con 0x564d46587910 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564d464a7b20 con 0x564d465921d0 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564d46594f20 con 0x564d46586ff0 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 10 client.0 _ll_register_callbacks cb 0x564d464a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d4658f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d4658f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49916/0 (socket says 172.21.15.69:49916) 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 -- 172.21.15.69:0/2175758045 learned_addr learned my addr 172.21.15.69:0/2175758045 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4770bd2640 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 0x564d4658fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476bfff640 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 0x564d4658f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 -- 172.21.15.69:0/2175758045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 msgr2=0x564d4658f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 0x564d4658f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 -- 172.21.15.69:0/2175758045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 msgr2=0x564d4658fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 0x564d4658fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 -- 172.21.15.69:0/2175758045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4625e3c0 con 0x564d46587910 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f476b7fe640 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d4658f5d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f47600971f0 tx=0x7f47600cbf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a09109daa221138 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f474b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f474b7fe640 1 -- 172.21.15.69:0/2175758045 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476001e070 con 0x564d46587910 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f474b7fe640 1 -- 172.21.15.69:0/2175758045 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f476009a430 con 0x564d46587910 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 1 -- 172.21.15.69:0/2175758045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47600a39a0 con 0x564d46587910 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f474b7fe640 1 -- 172.21.15.69:0/2175758045 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4760010610 con 0x564d46587910 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 10 client.15651 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.259+0000 7f4774a33740 1 -- 172.21.15.69:0/2175758045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d465a09b0 con 0x564d46587910 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f474b7fe640 1 -- 172.21.15.69:0/2175758045 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47600a39a0 con 0x564d46587910 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f474b7fe640 1 -- 172.21.15.69:0/2175758045 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f47600b1020 con 0x564d46587910 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f4774a33740 10 client.15651 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:19.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f4774a33740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f4774a33740 20 client.15651 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f4774a33740 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564d465a1d00 0x564d465a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f4774a33740 4 client.15651 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f4774a33740 1 -- 172.21.15.69:0/2175758045 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d464f9fb0 con 0x564d465a1d00 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f476bfff640 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564d465a1d00 0x564d465a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f474b7fe640 10 client.15651 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.260+0000 7f476bfff640 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564d465a1d00 0x564d465a40f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a9b10f22a896b4 server_cookie=6f26b1cf6e2e4ebd in_seq=0 out_seq=0 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f474b7fe640 1 -- 172.21.15.69:0/2175758045 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564d464f9fb0 con 0x564d465a1d00 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f474b7fe640 10 client.15651 handle_command_reply: tid=0 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 client.15651 shutdown 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 -- 172.21.15.69:0/2175758045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 msgr2=0x564d4658f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d4658f5d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f47600971f0 tx=0x7f47600cbf50 comp rx=0 tx=0).stop 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 -- 172.21.15.69:0/2175758045 shutdown_connections 2024-08-02T17:47:19.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d46587910 0x564d4658f5d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564d465a1d00 0x564d465a40f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d465921d0 0x564d4658fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 --2- 172.21.15.69:0/2175758045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46586ff0 0x564d4658f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f474b7fe640 0 client.15651 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:19.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 -- 172.21.15.69:0/2175758045 >> 172.21.15.69:0/2175758045 conn(0x564d46583580 msgr2=0x564d465892b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:19.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 -- 172.21.15.69:0/2175758045 shutdown_connections 2024-08-02T17:47:19.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.261+0000 7f4774a33740 1 -- 172.21.15.69:0/2175758045 wait complete. 2024-08-02T17:47:19.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.262+0000 7f4774a33740 20 client.15651 trim_cache size 0 max 16384 2024-08-02T17:47:19.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.274+0000 7f4774a33740 1 Processor -- start 2024-08-02T17:47:19.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.274+0000 7f4774a33740 1 -- start start 2024-08-02T17:47:19.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:19 smithi069 ceph-mon[32746]: pgmap v30: 129 pgs: 129 active+clean; 555 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 365 op/s 2024-08-02T17:47:19.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.274+0000 7f4774a33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d46582860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.274+0000 7f4774a33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.274+0000 7f4774a33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.274+0000 7f4774a33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1b3940 con 0x564d465921d0 2024-08-02T17:47:19.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.274+0000 7f4774a33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1b1210 con 0x564d46586ff0 2024-08-02T17:47:19.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.274+0000 7f4774a33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1af060 con 0x564d46587910 2024-08-02T17:47:19.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f4770bd2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f4770bd2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49928/0 (socket says 172.21.15.69:49928) 2024-08-02T17:47:19.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f4770bd2640 1 -- 172.21.15.69:0/2312901463 learned_addr learned my addr 172.21.15.69:0/2312901463 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:19.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f476b7fe640 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f476bfff640 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d46582860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f476b7fe640 1 -- 172.21.15.69:0/2312901463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 msgr2=0x564d46582860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:19.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f476b7fe640 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d46582860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f476b7fe640 1 -- 172.21.15.69:0/2312901463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 msgr2=0x564d4658f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:19.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f476b7fe640 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f476b7fe640 1 -- 172.21.15.69:0/2312901463 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d46594900 con 0x564d46587910 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f4770bd2640 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.275+0000 7f476b7fe640 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658e310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f476009ad70 tx=0x7f476000bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec1c07ec9857292 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4751ffb640 1 -- 172.21.15.69:0/2312901463 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476001e050 con 0x564d46587910 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4751ffb640 1 -- 172.21.15.69:0/2312901463 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760099ca0 con 0x564d46587910 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4751ffb640 1 -- 172.21.15.69:0/2312901463 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476009a230 con 0x564d46587910 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 -- 172.21.15.69:0/2312901463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 msgr2=0x564d4658e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658e310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f476009ad70 tx=0x7f476000bf90 comp rx=0 tx=0).stop 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 -- 172.21.15.69:0/2312901463 shutdown_connections 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658e310 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 --2- 172.21.15.69:0/2312901463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d46582860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 -- 172.21.15.69:0/2312901463 >> 172.21.15.69:0/2312901463 conn(0x564d46583580 msgr2=0x564d46591000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:19.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 -- 172.21.15.69:0/2312901463 shutdown_connections 2024-08-02T17:47:19.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.276+0000 7f4774a33740 1 -- 172.21.15.69:0/2312901463 wait complete. 2024-08-02T17:47:19.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 1 -- start start 2024-08-02T17:47:19.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 1 Processor -- start 2024-08-02T17:47:19.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d4659a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1af6d0 con 0x564d465921d0 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1b0b40 con 0x564d46586ff0 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f476c1afda0 con 0x564d46587910 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4774a33740 10 client.0 _ll_register_callbacks cb 0x7f475c005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4770bd2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f476bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d4659a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f476b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:44858/0 (socket says 172.21.15.69:44858) 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f476b7fe640 1 -- 172.21.15.69:0/1860239391 learned_addr learned my addr 172.21.15.69:0/1860239391 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.277+0000 7f4770bd2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49944/0 (socket says 172.21.15.69:49944) 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f476b7fe640 1 -- 172.21.15.69:0/1860239391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 msgr2=0x564d4659a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:19.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f476b7fe640 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d4659a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f476b7fe640 1 -- 172.21.15.69:0/1860239391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 msgr2=0x564d4658c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f476b7fe640 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f476b7fe640 1 -- 172.21.15.69:0/1860239391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4658c690 con 0x564d46587910 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f476bfff640 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d4659a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f4770bd2640 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f476b7fe640 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658bb60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f476000b470 tx=0x7f4760009330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6248d8f7b04b5b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f474b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f474b7fe640 1 -- 172.21.15.69:0/1860239391 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476001e050 con 0x564d46587910 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f474b7fe640 1 -- 172.21.15.69:0/1860239391 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f47600107b0 con 0x564d46587910 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f4774a33740 1 -- 172.21.15.69:0/1860239391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4760099ca0 con 0x564d46587910 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f474b7fe640 1 -- 172.21.15.69:0/1860239391 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f47600a39a0 con 0x564d46587910 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f4774a33740 10 client.25207 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.278+0000 7f4774a33740 1 -- 172.21.15.69:0/1860239391 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d4658c690 con 0x564d46587910 2024-08-02T17:47:19.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f474b7fe640 1 -- 172.21.15.69:0/1860239391 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760099ca0 con 0x564d46587910 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f474b7fe640 1 -- 172.21.15.69:0/1860239391 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f47600b1070 con 0x564d46587910 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f4774a33740 10 client.25207 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f4774a33740 10 client.25207 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f4774a33740 20 client.25207 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f4774a33740 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564d4660c7c0 0x564d4660ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f4774a33740 4 client.25207 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f4774a33740 1 -- 172.21.15.69:0/1860239391 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d464f44c0 con 0x564d4660c7c0 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f476bfff640 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564d4660c7c0 0x564d4660ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f476bfff640 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564d4660c7c0 0x564d4660ebb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3d24779d580d76 server_cookie=672d2542ed1fc072 in_seq=0 out_seq=0 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.279+0000 7f474b7fe640 10 client.25207 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f474b7fe640 1 -- 172.21.15.69:0/1860239391 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d464f44c0 con 0x564d4660c7c0 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f474b7fe640 10 client.25207 handle_command_reply: tid=0 2024-08-02T17:47:19.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 client.25207 shutdown 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 -- 172.21.15.69:0/1860239391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 msgr2=0x564d4658bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658bb60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f476000b470 tx=0x7f4760009330 comp rx=0 tx=0).stop 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 -- 172.21.15.69:0/1860239391 shutdown_connections 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564d465921d0 0x564d4658c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564d4660c7c0 0x564d4660ebb0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564d46587910 0x564d4658bb60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 --2- 172.21.15.69:0/1860239391 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564d46586ff0 0x564d4659a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f474b7fe640 0 client.25207 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 -- 172.21.15.69:0/1860239391 >> 172.21.15.69:0/1860239391 conn(0x564d46583580 msgr2=0x564d46590640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 -- 172.21.15.69:0/1860239391 shutdown_connections 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 1 -- 172.21.15.69:0/1860239391 wait complete. 2024-08-02T17:47:19.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.280+0000 7f4774a33740 20 client.25207 trim_cache size 0 max 16384 2024-08-02T17:47:19.287 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:47:19.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 -- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f47540a2ea0 msgr2=0x7f47540a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f47540a2ea0 0x7f47540a5360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4764009900 tx=0x7f4764009650 comp rx=0 tx=0).stop 2024-08-02T17:47:19.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 -- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c102250 msgr2=0x7f476c074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:19.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c102250 0x7f476c074760 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f476000b4a0 tx=0x7f476000b960 comp rx=0 tx=0).stop 2024-08-02T17:47:19.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 -- 172.21.15.69:0/39298141 shutdown_connections 2024-08-02T17:47:19.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f476c102250 0x7f476c074760 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f47540a2ea0 0x7f47540a5360 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f476c1078b0 0x7f476c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 --2- 172.21.15.69:0/39298141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f476c1018b0 0x7f476c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:19.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 -- 172.21.15.69:0/39298141 >> 172.21.15.69:0/39298141 conn(0x7f476c0fd5f0 msgr2=0x7f476c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:19.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 -- 172.21.15.69:0/39298141 shutdown_connections 2024-08-02T17:47:19.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:19.281+0000 7f4772635640 1 -- 172.21.15.69:0/39298141 wait complete. 2024-08-02T17:47:19.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:19 smithi077 ceph-mon[37108]: pgmap v30: 129 pgs: 129 active+clean; 555 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 365 op/s 2024-08-02T17:47:19.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:19 smithi083 ceph-mon[37031]: pgmap v30: 129 pgs: 129 active+clean; 555 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 18 MiB/s wr, 365 op/s 2024-08-02T17:47:19.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 10.0 s, 32.6 tps, lat 600.890 ms stddev 397.053 2024-08-02T17:47:21.074 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:47:21.074 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:47:21.332 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:21 smithi077 ceph-mon[37108]: pgmap v31: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 19 MiB/s wr, 462 op/s 2024-08-02T17:47:21.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:21 smithi069 ceph-mon[32746]: pgmap v31: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 19 MiB/s wr, 462 op/s 2024-08-02T17:47:21.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:21 smithi083 ceph-mon[37031]: pgmap v31: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 19 MiB/s wr, 462 op/s 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 -- 172.21.15.69:0/1653565423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301078c0 msgr2=0x7fd130107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 --2- 172.21.15.69:0/1653565423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301078c0 0x7fd130107ca0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd120009cd0 tx=0x7fd12002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 -- 172.21.15.69:0/1653565423 shutdown_connections 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 --2- 172.21.15.69:0/1653565423 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd130102260 0x7fd13010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 --2- 172.21.15.69:0/1653565423 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd1301018c0 0x7fd130101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 --2- 172.21.15.69:0/1653565423 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301078c0 0x7fd130107ca0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 -- 172.21.15.69:0/1653565423 >> 172.21.15.69:0/1653565423 conn(0x7fd1300fd620 msgr2=0x7fd1300ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 -- 172.21.15.69:0/1653565423 shutdown_connections 2024-08-02T17:47:21.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.784+0000 7fd137f09640 1 -- 172.21.15.69:0/1653565423 wait complete. 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd137f09640 1 Processor -- start 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd137f09640 1 -- start start 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd137f09640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301018c0 0x7fd130074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd137f09640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd130102260 0x7fd130074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd137f09640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1301078c0 0x7fd130074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd137f09640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd130111cc0 con 0x7fd1301018c0 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd137f09640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd130111b40 con 0x7fd130102260 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd137f09640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd130111e40 con 0x7fd1301078c0 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd135ca5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301018c0 0x7fd130074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd135ca5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301018c0 0x7fd130074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49970/0 (socket says 172.21.15.69:49970) 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd135ca5640 1 -- 172.21.15.69:0/4090575951 learned_addr learned my addr 172.21.15.69:0/4090575951 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd1364a6640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1301078c0 0x7fd130074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd135ca5640 1 -- 172.21.15.69:0/4090575951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1301078c0 msgr2=0x7fd130074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd135ca5640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1301078c0 0x7fd130074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd135ca5640 1 -- 172.21.15.69:0/4090575951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd130102260 msgr2=0x7fd130074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.785+0000 7fd1354a4640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd130102260 0x7fd130074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd135ca5640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd130102260 0x7fd130074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd135ca5640 1 -- 172.21.15.69:0/4090575951 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1301a6660 con 0x7fd1301018c0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd1364a6640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1301078c0 0x7fd130074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd135ca5640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301018c0 0x7fd130074220 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd12002ae00 tx=0x7fd120003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd12effd640 1 -- 172.21.15.69:0/4090575951 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120004040 con 0x7fd1301018c0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd137f09640 1 -- 172.21.15.69:0/4090575951 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1301a68f0 con 0x7fd1301018c0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd12effd640 1 -- 172.21.15.69:0/4090575951 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1200046c0 con 0x7fd1301018c0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd12effd640 1 -- 172.21.15.69:0/4090575951 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120046480 con 0x7fd1301018c0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.786+0000 7fd137f09640 1 -- 172.21.15.69:0/4090575951 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1301a6e30 con 0x7fd1301018c0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.788+0000 7fd12effd640 1 -- 172.21.15.69:0/4090575951 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7fd1200305a0 con 0x7fd1301018c0 2024-08-02T17:47:21.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.788+0000 7fd13a307740 1 Processor -- start 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd13a307740 1 -- start start 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd12effd640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1180a2ea0 0x7fd1180a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd13a307740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98db85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd13a307740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8b20 0x555e98dc2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd13a307740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd13a307740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555e98d7be40 con 0x555e98dc33e0 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd13a307740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e98d1d7f0 con 0x555e98db8200 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd13a307740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e98cc4fb0 con 0x555e98db8b20 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd1364a6640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1180a2ea0 0x7fd1180a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd135ca5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98db85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd12effd640 1 -- 172.21.15.69:0/4090575951 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd12002e030 con 0x7fd1301018c0 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd135ca5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98db85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55480/0 (socket says 172.21.15.69:55480) 2024-08-02T17:47:21.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd1364a6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd135ca5640 1 -- 172.21.15.69:0/2663486739 learned_addr learned my addr 172.21.15.69:0/2663486739 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.789+0000 7fd1354a4640 1 --2- 172.21.15.69:0/2663486739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8b20 0x555e98dc2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd135ca5640 1 -- 172.21.15.69:0/2663486739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8b20 msgr2=0x555e98dc2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2663486739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8b20 0x555e98dc2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd135ca5640 1 -- 172.21.15.69:0/2663486739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 msgr2=0x555e98dc57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2663486739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd135ca5640 1 -- 172.21.15.69:0/2663486739 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e98a8f6a0 con 0x555e98db8200 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd1364a6640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1180a2ea0 0x7fd1180a5360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1280058c0 tx=0x7fd12800b9c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2663486739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98db85e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd1200bd220 tx=0x7fd120049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1264f91cfce7fb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd117fff640 1 -- 172.21.15.69:0/2663486739 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1200bcdd0 con 0x555e98db8200 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd117fff640 1 -- 172.21.15.69:0/2663486739 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd120036040 con 0x555e98db8200 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.790+0000 7fd117fff640 1 -- 172.21.15.69:0/2663486739 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1200bcbf0 con 0x555e98db8200 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 -- 172.21.15.69:0/2663486739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 msgr2=0x555e98db85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 --2- 172.21.15.69:0/2663486739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98db85e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd1200bd220 tx=0x7fd120049000 comp rx=0 tx=0).stop 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 -- 172.21.15.69:0/2663486739 shutdown_connections 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 --2- 172.21.15.69:0/2663486739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 --2- 172.21.15.69:0/2663486739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8b20 0x555e98dc2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 --2- 172.21.15.69:0/2663486739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98db85e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 -- 172.21.15.69:0/2663486739 >> 172.21.15.69:0/2663486739 conn(0x555e98db4790 msgr2=0x555e98db4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 -- 172.21.15.69:0/2663486739 shutdown_connections 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 -- 172.21.15.69:0/2663486739 wait complete. 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 -- start start 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.791+0000 7fd13a307740 1 Processor -- start 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd13a307740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8200 0x555e98dc0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd13a307740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8b20 0x555e98dc0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd13a307740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd13a307740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555e98cd8cb0 con 0x555e98dc33e0 2024-08-02T17:47:21.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd13a307740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e98d197b0 con 0x555e98db8b20 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd13a307740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e98dc5d70 con 0x555e98db8200 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd13a307740 10 client.0 _ll_register_callbacks cb 0x555e98a2cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49990/0 (socket says 172.21.15.69:49990) 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd135ca5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8200 0x555e98dc0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 -- 172.21.15.69:0/2684813145 learned_addr learned my addr 172.21.15.69:0/2684813145 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1354a4640 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8b20 0x555e98dc0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 -- 172.21.15.69:0/2684813145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8200 msgr2=0x555e98dc0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8200 0x555e98dc0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 -- 172.21.15.69:0/2684813145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8b20 msgr2=0x555e98dc0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8b20 0x555e98dc0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 -- 172.21.15.69:0/2684813145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e98cd7bc0 con 0x555e98dc33e0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1364a6640 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc0dd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd128010940 tx=0x7fd128005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509380bd45858887 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.792+0000 7fd1157fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd1157fa640 1 -- 172.21.15.69:0/2684813145 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd128011ae0 con 0x555e98dc33e0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd1157fa640 1 -- 172.21.15.69:0/2684813145 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd128011c80 con 0x555e98dc33e0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 1 -- 172.21.15.69:0/2684813145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd120036040 con 0x555e98dc33e0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd1157fa640 1 -- 172.21.15.69:0/2684813145 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd128033050 con 0x555e98dc33e0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 10 client.15669 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 1 -- 172.21.15.69:0/2684813145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e98dd1c70 con 0x555e98dc33e0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd1157fa640 1 -- 172.21.15.69:0/2684813145 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd120036040 con 0x555e98dc33e0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd1157fa640 1 -- 172.21.15.69:0/2684813145 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd12800f040 con 0x555e98dc33e0 2024-08-02T17:47:21.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 10 client.15669 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 10 client.15669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 20 client.15669 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e98dd31d0 0x555e98dd5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 4 client.15669 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd13a307740 1 -- 172.21.15.69:0/2684813145 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e98d2ad60 con 0x555e98dd31d0 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.793+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e98dd31d0 0x555e98dd5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.794+0000 7fd1157fa640 10 client.15669 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.794+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e98dd31d0 0x555e98dd5620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0f7cbc154ff761 server_cookie=ccf3b3b6736bb8f9 in_seq=0 out_seq=0 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.794+0000 7fd1157fa640 1 -- 172.21.15.69:0/2684813145 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555e98d2ad60 con 0x555e98dd31d0 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.794+0000 7fd1157fa640 10 client.15669 handle_command_reply: tid=0 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.794+0000 7fd13a307740 1 client.15669 shutdown 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 -- 172.21.15.69:0/2684813145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 msgr2=0x555e98dc0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc0dd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd128010940 tx=0x7fd128005850 comp rx=0 tx=0).stop 2024-08-02T17:47:21.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 -- 172.21.15.69:0/2684813145 shutdown_connections 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98dc33e0 0x555e98dc0dd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e98dd31d0 0x555e98dd5620 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8b20 0x555e98dc0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 --2- 172.21.15.69:0/2684813145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98db8200 0x555e98dc0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 -- 172.21.15.69:0/2684813145 >> 172.21.15.69:0/2684813145 conn(0x555e98db4790 msgr2=0x555e98dba4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd1157fa640 0 client.15669 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 -- 172.21.15.69:0/2684813145 shutdown_connections 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 1 -- 172.21.15.69:0/2684813145 wait complete. 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.795+0000 7fd13a307740 20 client.15669 trim_cache size 0 max 16384 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd13a307740 1 Processor -- start 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd13a307740 1 -- start start 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd13a307740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98ddaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd13a307740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98ddb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd13a307740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 0x555e98dc1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd13a307740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b3810 con 0x555e98db8b20 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd13a307740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b10e0 con 0x555e98db8200 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd13a307740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301aef30 con 0x555e98dc33e0 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd1354a4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98ddb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd1354a4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98ddb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49998/0 (socket says 172.21.15.69:49998) 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd1354a4640 1 -- 172.21.15.69:0/3802734806 learned_addr learned my addr 172.21.15.69:0/3802734806 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd135ca5640 1 --2- 172.21.15.69:0/3802734806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98ddaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd1354a4640 1 -- 172.21.15.69:0/3802734806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 msgr2=0x555e98dc1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:21.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd1354a4640 1 --2- 172.21.15.69:0/3802734806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 0x555e98dc1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd1354a4640 1 -- 172.21.15.69:0/3802734806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 msgr2=0x555e98ddaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.812+0000 7fd1354a4640 1 --2- 172.21.15.69:0/3802734806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98ddaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd1354a4640 1 -- 172.21.15.69:0/3802734806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e98db8030 con 0x555e98db8b20 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd1354a4640 1 --2- 172.21.15.69:0/3802734806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98ddb470 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd1240058c0 tx=0x7fd12400df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a29a1b4ddeff61 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd1177fe640 1 -- 172.21.15.69:0/3802734806 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124010e40 con 0x555e98db8b20 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd1177fe640 1 -- 172.21.15.69:0/3802734806 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd124005500 con 0x555e98db8b20 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd1177fe640 1 -- 172.21.15.69:0/3802734806 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124019670 con 0x555e98db8b20 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd13a307740 1 -- 172.21.15.69:0/3802734806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 msgr2=0x555e98ddb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd13a307740 1 --2- 172.21.15.69:0/3802734806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98ddb470 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd1240058c0 tx=0x7fd12400df80 comp rx=0 tx=0).stop 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd13a307740 1 -- 172.21.15.69:0/3802734806 shutdown_connections 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd13a307740 1 --2- 172.21.15.69:0/3802734806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98ddb470 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd13a307740 1 --2- 172.21.15.69:0/3802734806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 0x555e98dc1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd13a307740 1 --2- 172.21.15.69:0/3802734806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98ddaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.813+0000 7fd13a307740 1 -- 172.21.15.69:0/3802734806 >> 172.21.15.69:0/3802734806 conn(0x555e98db4790 msgr2=0x555e98dbf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 -- 172.21.15.69:0/3802734806 shutdown_connections 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 -- 172.21.15.69:0/3802734806 wait complete. 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 -- start start 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 Processor -- start 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98dcb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98dc68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 0x555e98dc6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301af5a0 con 0x555e98db8b20 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301b0a10 con 0x555e98db8200 2024-08-02T17:47:21.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd1301afc70 con 0x555e98dc33e0 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd13a307740 10 client.0 _ll_register_callbacks cb 0x7fd12800bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd1354a4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98dc68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd1354a4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98dc68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50002/0 (socket says 172.21.15.69:50002) 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd1354a4640 1 -- 172.21.15.69:0/2191850486 learned_addr learned my addr 172.21.15.69:0/2191850486 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.814+0000 7fd1364a6640 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 0x555e98dc6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98dcb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1354a4640 1 -- 172.21.15.69:0/2191850486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 msgr2=0x555e98dc6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1354a4640 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 0x555e98dc6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1354a4640 1 -- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 msgr2=0x555e98dcb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1354a4640 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98dcb5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1354a4640 1 -- 172.21.15.69:0/2191850486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e98dc7640 con 0x555e98db8b20 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98dcb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:21.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1354a4640 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98dc68d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd12401b040 tx=0x7fd124005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6158d9929b0687f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1157fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1157fa640 1 -- 172.21.15.69:0/2191850486 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124027020 con 0x555e98db8b20 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1157fa640 1 -- 172.21.15.69:0/2191850486 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd124022650 con 0x555e98db8b20 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 1 -- 172.21.15.69:0/2191850486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd124005500 con 0x555e98db8b20 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1157fa640 1 -- 172.21.15.69:0/2191850486 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd124032050 con 0x555e98db8b20 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 10 client.15675 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 1 -- 172.21.15.69:0/2191850486 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e98e3de70 con 0x555e98db8b20 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1157fa640 1 -- 172.21.15.69:0/2191850486 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd124005500 con 0x555e98db8b20 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd1157fa640 1 -- 172.21.15.69:0/2191850486 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd1240195c0 con 0x555e98db8b20 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 10 client.15675 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 20 client.15675 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e98e3e640 0x555e98e40a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 4 client.15675 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:47:21.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.815+0000 7fd13a307740 1 -- 172.21.15.69:0/2191850486 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555e98d25270 con 0x555e98e3e640 2024-08-02T17:47:21.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.816+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e98e3e640 0x555e98e40a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:21.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.816+0000 7fd1157fa640 10 client.15675 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:21.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.816+0000 7fd135ca5640 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e98e3e640 0x555e98e40a60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b420a636c69b1f0 server_cookie=c9f79ddc175e120 in_seq=0 out_seq=0 2024-08-02T17:47:21.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.817+0000 7fd1157fa640 1 -- 172.21.15.69:0/2191850486 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555e98d25270 con 0x555e98e3e640 2024-08-02T17:47:21.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.817+0000 7fd1157fa640 10 client.15675 handle_command_reply: tid=0 2024-08-02T17:47:21.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.817+0000 7fd13a307740 1 client.15675 shutdown 2024-08-02T17:47:21.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 -- 172.21.15.69:0/2191850486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 msgr2=0x555e98dc68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98dc68d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd12401b040 tx=0x7fd124005230 comp rx=0 tx=0).stop 2024-08-02T17:47:21.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 -- 172.21.15.69:0/2191850486 shutdown_connections 2024-08-02T17:47:21.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e98db8b20 0x555e98dc68d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e98e3e640 0x555e98e40a60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e98dc33e0 0x555e98dc6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 --2- 172.21.15.69:0/2191850486 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e98db8200 0x555e98dcb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd1157fa640 0 client.15675 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:21.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 -- 172.21.15.69:0/2191850486 >> 172.21.15.69:0/2191850486 conn(0x555e98db4790 msgr2=0x555e98db30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:21.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 -- 172.21.15.69:0/2191850486 shutdown_connections 2024-08-02T17:47:21.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 1 -- 172.21.15.69:0/2191850486 wait complete. 2024-08-02T17:47:21.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.818+0000 7fd13a307740 20 client.15675 trim_cache size 0 max 16384 2024-08-02T17:47:21.837 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:47:21.837 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:47:21.837 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "a17e094a-6385-4f4a-8cb1-77ed2ab2a253", 2024-08-02T17:47:21.837 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:47:21.837 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 -- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1180a2ea0 msgr2=0x7fd1180a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1180a2ea0 0x7fd1180a5360 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1280058c0 tx=0x7fd12800b9c0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 -- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301018c0 msgr2=0x7fd130074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301018c0 0x7fd130074220 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd12002ae00 tx=0x7fd120003970 comp rx=0 tx=0).stop 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 -- 172.21.15.69:0/4090575951 shutdown_connections 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1180a2ea0 0x7fd1180a5360 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd1301078c0 0x7fd130074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd130102260 0x7fd130074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 --2- 172.21.15.69:0/4090575951 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd1301018c0 0x7fd130074220 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 -- 172.21.15.69:0/4090575951 >> 172.21.15.69:0/4090575951 conn(0x7fd1300fd620 msgr2=0x7fd1300fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:21.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 -- 172.21.15.69:0/4090575951 shutdown_connections 2024-08-02T17:47:21.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:21.819+0000 7fd137f09640 1 -- 172.21.15.69:0/4090575951 wait complete. 2024-08-02T17:47:22.558 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:47:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:22 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:47:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:22 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:47:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:22 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:47:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:22 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:47:22.812 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:22.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:22 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:47:22.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:22 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:47:22.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:22 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:47:22.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:22 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:47:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:22 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:47:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:22 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:47:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:22 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:47:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:22 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:47:23.702 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:23 smithi069 ceph-mon[32746]: pgmap v32: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-08-02T17:47:23.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:23 smithi083 ceph-mon[37031]: pgmap v32: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-08-02T17:47:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 -- 172.21.15.69:0/2679821984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2598101150 msgr2=0x7f259810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2679821984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2598101150 0x7f259810d680 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2590009930 tx=0x7f259002a420 comp rx=0 tx=0).stop 2024-08-02T17:47:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 -- 172.21.15.69:0/2679821984 shutdown_connections 2024-08-02T17:47:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2679821984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2598101150 0x7f259810d680 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2679821984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f25981007b0 0x7f2598100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2679821984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25981067b0 0x7f2598106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 -- 172.21.15.69:0/2679821984 >> 172.21.15.69:0/2679821984 conn(0x7f25980fc4d0 msgr2=0x7f25980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 -- 172.21.15.69:0/2679821984 shutdown_connections 2024-08-02T17:47:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.783+0000 7f259ca8f640 1 -- 172.21.15.69:0/2679821984 wait complete. 2024-08-02T17:47:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259ca8f640 1 Processor -- start 2024-08-02T17:47:23.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259ca8f640 1 -- start start 2024-08-02T17:47:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259ca8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25981007b0 0x7f2598102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259ca8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2598101150 0x7f25981028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259ca8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25981067b0 0x7f2598102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259ca8f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2598110bd0 con 0x7f25981007b0 2024-08-02T17:47:23.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259ca8f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2598110a50 con 0x7f25981067b0 2024-08-02T17:47:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259ca8f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2598110d50 con 0x7f2598101150 2024-08-02T17:47:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25981007b0 0x7f2598102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25981007b0 0x7f2598102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50022/0 (socket says 172.21.15.69:50022) 2024-08-02T17:47:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259659c640 1 -- 172.21.15.69:0/2816582313 learned_addr learned my addr 172.21.15.69:0/2816582313 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f2596d9d640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25981067b0 0x7f2598102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f2595d9b640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2598101150 0x7f25981028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259659c640 1 -- 172.21.15.69:0/2816582313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2598101150 msgr2=0x7f25981028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259659c640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2598101150 0x7f25981028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259659c640 1 -- 172.21.15.69:0/2816582313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25981067b0 msgr2=0x7f2598102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259659c640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25981067b0 0x7f2598102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f259659c640 1 -- 172.21.15.69:0/2816582313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25981a6670 con 0x7f25981007b0 2024-08-02T17:47:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.784+0000 7f2596d9d640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25981067b0 0x7f2598102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.785+0000 7f259659c640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25981007b0 0x7f2598102380 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f258c00d950 tx=0x7f258c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.785+0000 7f25877fe640 1 -- 172.21.15.69:0/2816582313 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c017070 con 0x7f25981007b0 2024-08-02T17:47:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.785+0000 7f25877fe640 1 -- 172.21.15.69:0/2816582313 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c009c30 con 0x7f25981007b0 2024-08-02T17:47:23.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.785+0000 7f25877fe640 1 -- 172.21.15.69:0/2816582313 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c00f6b0 con 0x7f25981007b0 2024-08-02T17:47:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.785+0000 7f259ca8f640 1 -- 172.21.15.69:0/2816582313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25981a6960 con 0x7f25981007b0 2024-08-02T17:47:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.785+0000 7f259ca8f640 1 -- 172.21.15.69:0/2816582313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25981a6ef0 con 0x7f25981007b0 2024-08-02T17:47:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.787+0000 7f25877fe640 1 -- 172.21.15.69:0/2816582313 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f258c0109d0 con 0x7f25981007b0 2024-08-02T17:47:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.787+0000 7f259ee8d740 1 Processor -- start 2024-08-02T17:47:23.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259ee8d740 1 -- start start 2024-08-02T17:47:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd3975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 0x556bbd3a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259ee8d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556bbd2fc980 con 0x556bbd397b30 2024-08-02T17:47:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259ee8d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bbd294a70 con 0x556bbd397210 2024-08-02T17:47:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259ee8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556bbd2f8940 con 0x556bbd3a23f0 2024-08-02T17:47:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3a1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50028/0 (socket says 172.21.15.69:50028) 2024-08-02T17:47:23.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259659c640 1 -- 172.21.15.69:0/216758096 learned_addr learned my addr 172.21.15.69:0/216758096 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:23.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f25877fe640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25800a2f20 0x7f25800a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f2596d9d640 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd3975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f2595d9b640 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 0x556bbd3a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f2596d9d640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25800a2f20 0x7f25800a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f25877fe640 1 -- 172.21.15.69:0/2816582313 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f258c0cf810 con 0x7f25981007b0 2024-08-02T17:47:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259659c640 1 -- 172.21.15.69:0/216758096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 msgr2=0x556bbd3a47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259659c640 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 0x556bbd3a47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259659c640 1 -- 172.21.15.69:0/216758096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 msgr2=0x556bbd3975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259659c640 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd3975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f259659c640 1 -- 172.21.15.69:0/216758096 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bbd2f76e0 con 0x556bbd397b30 2024-08-02T17:47:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.788+0000 7f2596d9d640 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd3975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.789+0000 7f259659c640 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3a1eb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ccaa0 tx=0x7f258c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9141ddf2b1d80ad server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:23.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.789+0000 7f2584ff9640 1 -- 172.21.15.69:0/216758096 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c020030 con 0x556bbd397b30 2024-08-02T17:47:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.789+0000 7f2596d9d640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25800a2f20 0x7f25800a53e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f259002a940 tx=0x7f2590005fd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.789+0000 7f2584ff9640 1 -- 172.21.15.69:0/216758096 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c009de0 con 0x556bbd397b30 2024-08-02T17:47:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.789+0000 7f2584ff9640 1 -- 172.21.15.69:0/216758096 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c0973a0 con 0x556bbd397b30 2024-08-02T17:47:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.789+0000 7f259ee8d740 1 -- 172.21.15.69:0/216758096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 msgr2=0x556bbd3a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3a1eb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ccaa0 tx=0x7f258c000bc0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 -- 172.21.15.69:0/216758096 shutdown_connections 2024-08-02T17:47:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3a1eb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 0x556bbd3a47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 --2- 172.21.15.69:0/216758096 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd3975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 -- 172.21.15.69:0/216758096 >> 172.21.15.69:0/216758096 conn(0x556bbd3937a0 msgr2=0x556bbd393b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 -- 172.21.15.69:0/216758096 shutdown_connections 2024-08-02T17:47:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 -- 172.21.15.69:0/216758096 wait complete. 2024-08-02T17:47:23.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 -- start start 2024-08-02T17:47:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 Processor -- start 2024-08-02T17:47:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd39f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397b30 0x556bbd39f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd3a23f0 0x556bbd39fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556bbd3a4f60 con 0x556bbd3a23f0 2024-08-02T17:47:23.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556bbd2b7aa0 con 0x556bbd397210 2024-08-02T17:47:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556bbd3a50e0 con 0x556bbd397b30 2024-08-02T17:47:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f259ee8d740 10 client.0 _ll_register_callbacks cb 0x556bbd2a6e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.790+0000 7f2596d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd3a23f0 0x556bbd39fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2596d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd3a23f0 0x556bbd39fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50030/0 (socket says 172.21.15.69:50030) 2024-08-02T17:47:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2596d9d640 1 -- 172.21.15.69:0/929495374 learned_addr learned my addr 172.21.15.69:0/929495374 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:23.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2595d9b640 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397b30 0x556bbd39f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f259659c640 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd39f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2596d9d640 1 -- 172.21.15.69:0/929495374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397b30 msgr2=0x556bbd39f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2596d9d640 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397b30 0x556bbd39f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2596d9d640 1 -- 172.21.15.69:0/929495374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 msgr2=0x556bbd39f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2596d9d640 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd39f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2596d9d640 1 -- 172.21.15.69:0/929495374 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bbd29b650 con 0x556bbd3a23f0 2024-08-02T17:47:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f2596d9d640 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd3a23f0 0x556bbd39fd50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2590003870 tx=0x7f2590009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a503b4969898574 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f257dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f257dffb640 1 -- 172.21.15.69:0/929495374 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259004d020 con 0x556bbd3a23f0 2024-08-02T17:47:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f257dffb640 1 -- 172.21.15.69:0/929495374 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2590049430 con 0x556bbd3a23f0 2024-08-02T17:47:23.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f259ee8d740 1 -- 172.21.15.69:0/929495374 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f258c009de0 con 0x556bbd3a23f0 2024-08-02T17:47:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f257dffb640 1 -- 172.21.15.69:0/929495374 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2590058050 con 0x556bbd3a23f0 2024-08-02T17:47:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f259ee8d740 10 client.15687 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.791+0000 7f259ee8d740 1 -- 172.21.15.69:0/929495374 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556bbd06e3c0 con 0x556bbd3a23f0 2024-08-02T17:47:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f257dffb640 1 -- 172.21.15.69:0/929495374 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c009de0 con 0x556bbd3a23f0 2024-08-02T17:47:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f257dffb640 1 -- 172.21.15.69:0/929495374 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2590048070 con 0x556bbd3a23f0 2024-08-02T17:47:23.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f259ee8d740 10 client.15687 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f259ee8d740 10 client.15687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f259ee8d740 20 client.15687 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f259ee8d740 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556bbd3b1ef0 0x556bbd3b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f259ee8d740 4 client.15687 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f259ee8d740 1 -- 172.21.15.69:0/929495374 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bbd309ef0 con 0x556bbd3b1ef0 2024-08-02T17:47:23.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f259659c640 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556bbd3b1ef0 0x556bbd3b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f259659c640 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556bbd3b1ef0 0x556bbd3b4350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9497e8992ff22cd6 server_cookie=4b2da00e145234 in_seq=0 out_seq=0 2024-08-02T17:47:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.792+0000 7f257dffb640 10 client.15687 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f257dffb640 1 -- 172.21.15.69:0/929495374 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556bbd309ef0 con 0x556bbd3b1ef0 2024-08-02T17:47:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f257dffb640 10 client.15687 handle_command_reply: tid=0 2024-08-02T17:47:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 client.15687 shutdown 2024-08-02T17:47:23.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 -- 172.21.15.69:0/929495374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd3a23f0 msgr2=0x556bbd39fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd3a23f0 0x556bbd39fd50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2590003870 tx=0x7f2590009260 comp rx=0 tx=0).stop 2024-08-02T17:47:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 -- 172.21.15.69:0/929495374 shutdown_connections 2024-08-02T17:47:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd3a23f0 0x556bbd39fd50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556bbd3b1ef0 0x556bbd3b4350 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397b30 0x556bbd39f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 --2- 172.21.15.69:0/929495374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397210 0x556bbd39f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f257dffb640 0 client.15687 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:23.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 -- 172.21.15.69:0/929495374 >> 172.21.15.69:0/929495374 conn(0x556bbd3937a0 msgr2=0x556bbd392900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:23.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 -- 172.21.15.69:0/929495374 shutdown_connections 2024-08-02T17:47:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.793+0000 7f259ee8d740 1 -- 172.21.15.69:0/929495374 wait complete. 2024-08-02T17:47:23.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.794+0000 7f259ee8d740 20 client.15687 trim_cache size 0 max 16384 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.810+0000 7f259ee8d740 1 Processor -- start 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.810+0000 7f259ee8d740 1 -- start start 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397210 0x556bbd399cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397b30 0x556bbd393b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 0x556bbd3b9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259ee8d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25981b3880 con 0x556bbd397210 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259ee8d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f25981b1150 con 0x556bbd397b30 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259ee8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25981aefa0 con 0x556bbd3a23f0 2024-08-02T17:47:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397210 0x556bbd399cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397210 0x556bbd399cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50040/0 (socket says 172.21.15.69:50040) 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 -- 172.21.15.69:0/2182768913 learned_addr learned my addr 172.21.15.69:0/2182768913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f2595d9b640 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397b30 0x556bbd393b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f2596d9d640 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 0x556bbd3b9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 -- 172.21.15.69:0/2182768913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 msgr2=0x556bbd3b9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 0x556bbd3b9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 -- 172.21.15.69:0/2182768913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397b30 msgr2=0x556bbd393b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397b30 0x556bbd393b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 -- 172.21.15.69:0/2182768913 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bbd39d820 con 0x556bbd397210 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f2595d9b640 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397b30 0x556bbd393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f259659c640 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397210 0x556bbd399cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f258c0cc9c0 tx=0x7f258c06a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72fa9ca0fa9967b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f257ffff640 1 -- 172.21.15.69:0/2182768913 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c06a020 con 0x556bbd397210 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f257ffff640 1 -- 172.21.15.69:0/2182768913 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c009de0 con 0x556bbd397210 2024-08-02T17:47:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.811+0000 7f257ffff640 1 -- 172.21.15.69:0/2182768913 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c020040 con 0x556bbd397210 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 -- 172.21.15.69:0/2182768913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397210 msgr2=0x556bbd399cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397210 0x556bbd399cc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f258c0cc9c0 tx=0x7f258c06a580 comp rx=0 tx=0).stop 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 -- 172.21.15.69:0/2182768913 shutdown_connections 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd3a23f0 0x556bbd3b9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd397b30 0x556bbd393b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 --2- 172.21.15.69:0/2182768913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397210 0x556bbd399cc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 -- 172.21.15.69:0/2182768913 >> 172.21.15.69:0/2182768913 conn(0x556bbd3937a0 msgr2=0x556bbd39e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 -- 172.21.15.69:0/2182768913 shutdown_connections 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 -- 172.21.15.69:0/2182768913 wait complete. 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.812+0000 7f259ee8d740 1 -- start start 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259ee8d740 1 Processor -- start 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397210 0x556bbd3aa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259ee8d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd3a23f0 0x556bbd3a58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259ee8d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f25981af610 con 0x556bbd397b30 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259ee8d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f25981b0a80 con 0x556bbd3a23f0 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259ee8d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f25981afce0 con 0x556bbd397210 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259ee8d740 10 client.0 _ll_register_callbacks cb 0x7f2590009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3aaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50046/0 (socket says 172.21.15.69:50046) 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 -- 172.21.15.69:0/3163262543 learned_addr learned my addr 172.21.15.69:0/3163262543 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f259659c640 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397210 0x556bbd3aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 -- 172.21.15.69:0/3163262543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397210 msgr2=0x556bbd3aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397210 0x556bbd3aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 -- 172.21.15.69:0/3163262543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd3a23f0 msgr2=0x556bbd3a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd3a23f0 0x556bbd3a58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 -- 172.21.15.69:0/3163262543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bbd3a6100 con 0x556bbd397b30 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f2595d9b640 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3aaa50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x556bbd394b20 tx=0x7f2588013d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e90f59103d37f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.813+0000 7f257dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f257dffb640 1 -- 172.21.15.69:0/3163262543 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588011cc0 con 0x556bbd397b30 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f257dffb640 1 -- 172.21.15.69:0/3163262543 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f25880143f0 con 0x556bbd397b30 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 1 -- 172.21.15.69:0/3163262543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f258c009de0 con 0x556bbd397b30 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f257dffb640 1 -- 172.21.15.69:0/3163262543 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2588011e60 con 0x556bbd397b30 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 10 client.15693 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 1 -- 172.21.15.69:0/3163262543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556bbd41c790 con 0x556bbd397b30 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f257dffb640 1 -- 172.21.15.69:0/3163262543 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258c009de0 con 0x556bbd397b30 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f257dffb640 1 -- 172.21.15.69:0/3163262543 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f258801f870 con 0x556bbd397b30 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 10 client.15693 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 20 client.15693 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556bbd41cf40 0x556bbd41f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 4 client.15693 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.814+0000 7f259ee8d740 1 -- 172.21.15.69:0/3163262543 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556bbd304400 con 0x556bbd41cf40 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259659c640 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556bbd41cf40 0x556bbd41f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f257dffb640 10 client.15693 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259659c640 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556bbd41cf40 0x556bbd41f360 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd78c84593698c5f server_cookie=c1ab746b3825bf13 in_seq=0 out_seq=0 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f257dffb640 1 -- 172.21.15.69:0/3163262543 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556bbd304400 con 0x556bbd41cf40 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f257dffb640 10 client.15693 handle_command_reply: tid=0 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 client.15693 shutdown 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 -- 172.21.15.69:0/3163262543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 msgr2=0x556bbd3aaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3aaa50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x556bbd394b20 tx=0x7f2588013d40 comp rx=0 tx=0).stop 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 -- 172.21.15.69:0/3163262543 shutdown_connections 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556bbd397b30 0x556bbd3aaa50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556bbd41cf40 0x556bbd41f360 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f257dffb640 0 client.15693 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556bbd3a23f0 0x556bbd3a58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 --2- 172.21.15.69:0/3163262543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556bbd397210 0x556bbd3aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 -- 172.21.15.69:0/3163262543 >> 172.21.15.69:0/3163262543 conn(0x556bbd3937a0 msgr2=0x556bbd3920e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 -- 172.21.15.69:0/3163262543 shutdown_connections 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.815+0000 7f259ee8d740 1 -- 172.21.15.69:0/3163262543 wait complete. 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ee8d740 20 client.15693 trim_cache size 0 max 16384 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:47:23.841 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:47:23.842 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:47:23.842 INFO:teuthology.orchestra.run.smithi069.stdout: "a17e094a-6385-4f4a-8cb1-77ed2ab2a253": { 2024-08-02T17:47:23.842 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:47:23.842 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a17e094a-6385-4f4a-8cb1-77ed2ab2a253", 2024-08-02T17:47:23.842 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:47:23.842 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:23.842 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:23.842 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 -- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25800a2f20 msgr2=0x7f25800a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25800a2f20 0x7f25800a53e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f259002a940 tx=0x7f2590005fd0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 -- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25981007b0 msgr2=0x7f2598102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25981007b0 0x7f2598102380 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f258c00d950 tx=0x7f258c00de10 comp rx=0 tx=0).stop 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 -- 172.21.15.69:0/2816582313 shutdown_connections 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25800a2f20 0x7f25800a53e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f25981067b0 0x7f2598102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2598101150 0x7f25981028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 --2- 172.21.15.69:0/2816582313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f25981007b0 0x7f2598102380 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 -- 172.21.15.69:0/2816582313 >> 172.21.15.69:0/2816582313 conn(0x7f25980fc4d0 msgr2=0x7f2598104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 -- 172.21.15.69:0/2816582313 shutdown_connections 2024-08-02T17:47:23.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:23.816+0000 7f259ca8f640 1 -- 172.21.15.69:0/2816582313 wait complete. 2024-08-02T17:47:23.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:23 smithi077 ceph-mon[37108]: pgmap v32: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-08-02T17:47:24.625 INFO:tasks.cephfs.filesystem:scrub status for tag:a17e094a-6385-4f4a-8cb1-77ed2ab2a253 - {'path': '/', 'tag': 'a17e094a-6385-4f4a-8cb1-77ed2ab2a253', 'options': 'recursive,force'} 2024-08-02T17:47:24.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:24 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:47:24.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:24 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:47:24.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:24 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:47:24.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 15.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:47:25.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:25 smithi083 ceph-mon[37031]: pgmap v33: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-08-02T17:47:25.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:25 smithi077 ceph-mon[37108]: pgmap v33: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-08-02T17:47:25.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:25 smithi069 ceph-mon[32746]: pgmap v33: 129 pgs: 129 active+clean; 557 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.4 MiB/s wr, 253 op/s 2024-08-02T17:47:27.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:27 smithi077 ceph-mon[37108]: pgmap v34: 129 pgs: 129 active+clean; 740 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 17 MiB/s wr, 349 op/s 2024-08-02T17:47:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:27 smithi069 ceph-mon[32746]: pgmap v34: 129 pgs: 129 active+clean; 740 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 17 MiB/s wr, 349 op/s 2024-08-02T17:47:27.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:27 smithi083 ceph-mon[37031]: pgmap v34: 129 pgs: 129 active+clean; 740 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 17 MiB/s wr, 349 op/s 2024-08-02T17:47:29.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:29 smithi077 ceph-mon[37108]: pgmap v35: 129 pgs: 129 active+clean; 740 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 194 op/s 2024-08-02T17:47:29.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:29 smithi069 ceph-mon[32746]: pgmap v35: 129 pgs: 129 active+clean; 740 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 194 op/s 2024-08-02T17:47:29.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:29 smithi083 ceph-mon[37031]: pgmap v35: 129 pgs: 129 active+clean; 740 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 194 op/s 2024-08-02T17:47:29.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 20.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:47:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 -- 172.21.15.69:0/3894147699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 msgr2=0x7fb52c1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3894147699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 0x7fb52c1009b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb520009900 tx=0x7fb52002a410 comp rx=0 tx=0).stop 2024-08-02T17:47:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 -- 172.21.15.69:0/3894147699 shutdown_connections 2024-08-02T17:47:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3894147699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb52c10e0f0 0x7fb52c1104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3894147699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c100ef0 0x7fb52c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3894147699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 0x7fb52c1009b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 -- 172.21.15.69:0/3894147699 >> 172.21.15.69:0/3894147699 conn(0x7fb52c0fc410 msgr2=0x7fb52c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 -- 172.21.15.69:0/3894147699 shutdown_connections 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.979+0000 7fb532ef0640 1 -- 172.21.15.69:0/3894147699 wait complete. 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb532ef0640 1 Processor -- start 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb532ef0640 1 -- start start 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb532ef0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 0x7fb52c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb532ef0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c100ef0 0x7fb52c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb532ef0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb52c10e0f0 0x7fb52c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb532ef0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c113df0 con 0x7fb52c1005d0 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb532ef0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c113c70 con 0x7fb52c10e0f0 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb532ef0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c113f70 con 0x7fb52c100ef0 2024-08-02T17:47:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb530c8c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 0x7fb52c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb530c8c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 0x7fb52c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34898/0 (socket says 172.21.15.69:34898) 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb530c8c640 1 -- 172.21.15.69:0/3196969691 learned_addr learned my addr 172.21.15.69:0/3196969691 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.980+0000 7fb52bfff640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c100ef0 0x7fb52c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb530c8c640 1 -- 172.21.15.69:0/3196969691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c100ef0 msgr2=0x7fb52c19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb530c8c640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c100ef0 0x7fb52c19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb530c8c640 1 -- 172.21.15.69:0/3196969691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb52c10e0f0 msgr2=0x7fb52c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb53148d640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb52c10e0f0 0x7fb52c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb530c8c640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb52c10e0f0 0x7fb52c19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb530c8c640 1 -- 172.21.15.69:0/3196969691 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c19f310 con 0x7fb52c1005d0 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb52bfff640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c100ef0 0x7fb52c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb530c8c640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 0x7fb52c19a320 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb520006010 tx=0x7fb5200042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb529ffb640 1 -- 172.21.15.69:0/3196969691 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52003f030 con 0x7fb52c1005d0 2024-08-02T17:47:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb529ffb640 1 -- 172.21.15.69:0/3196969691 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb520003ce0 con 0x7fb52c1005d0 2024-08-02T17:47:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb532ef0640 1 -- 172.21.15.69:0/3196969691 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c19f5a0 con 0x7fb52c1005d0 2024-08-02T17:47:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb529ffb640 1 -- 172.21.15.69:0/3196969691 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520035070 con 0x7fb52c1005d0 2024-08-02T17:47:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.981+0000 7fb532ef0640 1 -- 172.21.15.69:0/3196969691 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a6e90 con 0x7fb52c1005d0 2024-08-02T17:47:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.983+0000 7fb529ffb640 1 -- 172.21.15.69:0/3196969691 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7fb520042030 con 0x7fb52c1005d0 2024-08-02T17:47:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb535c40740 1 Processor -- start 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb535c40740 1 -- start start 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb529ffb640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb51c0a2ea0 0x7fb51c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb535c40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8abdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb535c40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb535c40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 0x55d7f8acad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb535c40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d7f8a219b0 con 0x55d7f8abe0d0 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb535c40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d7f89c9800 con 0x55d7f8ac8970 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb535c40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7f8a1d970 con 0x55d7f8abd7b0 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb53148d640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb51c0a2ea0 0x7fb51c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb529ffb640 1 -- 172.21.15.69:0/3196969691 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb5200f0180 con 0x7fb52c1005d0 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34910/0 (socket says 172.21.15.69:34910) 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb530c8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8abdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb52bfff640 1 -- 172.21.15.69:0/1478302094 learned_addr learned my addr 172.21.15.69:0/1478302094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb53148d640 1 --2- 172.21.15.69:0/1478302094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 0x55d7f8acad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb52bfff640 1 -- 172.21.15.69:0/1478302094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 msgr2=0x55d7f8abdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1478302094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8abdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb52bfff640 1 -- 172.21.15.69:0/1478302094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 msgr2=0x55d7f8acad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1478302094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 0x55d7f8acad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb52bfff640 1 -- 172.21.15.69:0/1478302094 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f8735210 con 0x55d7f8abe0d0 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.984+0000 7fb53148d640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb51c0a2ea0 0x7fb51c0a5360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb52c102720 tx=0x7fb524014950 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1478302094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac8430 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb518009ce0 tx=0x7fb5180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4a667d53f646b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb516ffd640 1 -- 172.21.15.69:0/1478302094 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518012450 con 0x55d7f8abe0d0 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb516ffd640 1 -- 172.21.15.69:0/1478302094 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb518012ad0 con 0x55d7f8abe0d0 2024-08-02T17:47:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb516ffd640 1 -- 172.21.15.69:0/1478302094 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51800fb60 con 0x55d7f8abe0d0 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb535c40740 1 -- 172.21.15.69:0/1478302094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 msgr2=0x55d7f8ac8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb535c40740 1 --2- 172.21.15.69:0/1478302094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac8430 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb518009ce0 tx=0x7fb5180117b0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb535c40740 1 -- 172.21.15.69:0/1478302094 shutdown_connections 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb535c40740 1 --2- 172.21.15.69:0/1478302094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac8430 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb535c40740 1 --2- 172.21.15.69:0/1478302094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 0x55d7f8acad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb535c40740 1 --2- 172.21.15.69:0/1478302094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8abdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb535c40740 1 -- 172.21.15.69:0/1478302094 >> 172.21.15.69:0/1478302094 conn(0x55d7f8ab9d20 msgr2=0x55d7f8aba100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.985+0000 7fb535c40740 1 -- 172.21.15.69:0/1478302094 shutdown_connections 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 -- 172.21.15.69:0/1478302094 wait complete. 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 -- start start 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 Processor -- start 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 0x55d7f8ac6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d7f8acb540 con 0x55d7f8abe0d0 2024-08-02T17:47:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d7f89dd500 con 0x55d7f8ac8970 2024-08-02T17:47:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d7f8acb6c0 con 0x55d7f8abd7b0 2024-08-02T17:47:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb535c40740 10 client.0 _ll_register_callbacks cb 0x55d7f8734a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34916/0 (socket says 172.21.15.69:34916) 2024-08-02T17:47:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb52bfff640 1 -- 172.21.15.69:0/1480682122 learned_addr learned my addr 172.21.15.69:0/1480682122 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb53148d640 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 0x55d7f8ac6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.986+0000 7fb530c8c640 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb52bfff640 1 -- 172.21.15.69:0/1480682122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 msgr2=0x55d7f8ac57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb52bfff640 1 -- 172.21.15.69:0/1480682122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 msgr2=0x55d7f8ac6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 0x55d7f8ac6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb52bfff640 1 -- 172.21.15.69:0/1480682122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f8801620 con 0x55d7f8abe0d0 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac5d20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb51800f740 tx=0x7fb518009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e46774f4f6e9960 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb50bfff640 1 -- 172.21.15.69:0/1480682122 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518029020 con 0x55d7f8abe0d0 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb50bfff640 1 -- 172.21.15.69:0/1480682122 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb5180126e0 con 0x55d7f8abe0d0 2024-08-02T17:47:29.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb535c40740 1 -- 172.21.15.69:0/1480682122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb518012ad0 con 0x55d7f8abe0d0 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb50bfff640 1 -- 172.21.15.69:0/1480682122 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb518034050 con 0x55d7f8abe0d0 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb535c40740 10 client.15705 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb535c40740 1 -- 172.21.15.69:0/1480682122 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d7f8ad7160 con 0x55d7f8abe0d0 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb50bfff640 1 -- 172.21.15.69:0/1480682122 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb518012ad0 con 0x55d7f8abe0d0 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb50bfff640 1 -- 172.21.15.69:0/1480682122 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb518005480 con 0x55d7f8abe0d0 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb535c40740 10 client.15705 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb535c40740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb535c40740 20 client.15705 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.987+0000 7fb535c40740 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d7f8ad8560 0x55d7f8ada950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.988+0000 7fb535c40740 4 client.15705 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.988+0000 7fb535c40740 1 -- 172.21.15.69:0/1480682122 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7f8a2f0e0 con 0x55d7f8ad8560 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.988+0000 7fb530c8c640 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d7f8ad8560 0x55d7f8ada950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:30.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.988+0000 7fb50bfff640 10 client.15705 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.988+0000 7fb530c8c640 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d7f8ad8560 0x55d7f8ada950 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670ec04256a8d6a server_cookie=aaf772e70b562855 in_seq=0 out_seq=0 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb50bfff640 1 -- 172.21.15.69:0/1480682122 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d7f8a2f0e0 con 0x55d7f8ad8560 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb50bfff640 10 client.15705 handle_command_reply: tid=0 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 client.15705 shutdown 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 -- 172.21.15.69:0/1480682122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 msgr2=0x55d7f8ac5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac5d20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb51800f740 tx=0x7fb518009dd0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 -- 172.21.15.69:0/1480682122 shutdown_connections 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8abe0d0 0x55d7f8ac5d20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d7f8ad8560 0x55d7f8ada950 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8ac8970 0x55d7f8ac6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 --2- 172.21.15.69:0/1480682122 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb50bfff640 0 client.15705 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 -- 172.21.15.69:0/1480682122 >> 172.21.15.69:0/1480682122 conn(0x55d7f8ab9d20 msgr2=0x55d7f8abfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:30.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 -- 172.21.15.69:0/1480682122 shutdown_connections 2024-08-02T17:47:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 1 -- 172.21.15.69:0/1480682122 wait complete. 2024-08-02T17:47:30.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:29.989+0000 7fb535c40740 20 client.15705 trim_cache size 0 max 16384 2024-08-02T17:47:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb535c40740 1 Processor -- start 2024-08-02T17:47:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb535c40740 1 -- start start 2024-08-02T17:47:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb535c40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb535c40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abe0d0 0x55d7f8adffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb535c40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8ae0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb535c40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b37d0 con 0x55d7f8ac8970 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb535c40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b10a0 con 0x55d7f8abd7b0 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb535c40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1aeef0 con 0x55d7f8abe0d0 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb53148d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8ae0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb53148d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8ae0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34928/0 (socket says 172.21.15.69:34928) 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.004+0000 7fb53148d640 1 -- 172.21.15.69:0/2805005090 learned_addr learned my addr 172.21.15.69:0/2805005090 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb530c8c640 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb52bfff640 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abe0d0 0x55d7f8adffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb53148d640 1 -- 172.21.15.69:0/2805005090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abe0d0 msgr2=0x55d7f8adffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb53148d640 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abe0d0 0x55d7f8adffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb53148d640 1 -- 172.21.15.69:0/2805005090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abd7b0 msgr2=0x55d7f8ac4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb53148d640 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb53148d640 1 -- 172.21.15.69:0/2805005090 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f8ab4f80 con 0x55d7f8ac8970 2024-08-02T17:47:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb530c8c640 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb53148d640 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8ae0520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb524008e20 tx=0x7fb524017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4674dd4e2c87d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb5167fc640 1 -- 172.21.15.69:0/2805005090 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524025070 con 0x55d7f8ac8970 2024-08-02T17:47:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb5167fc640 1 -- 172.21.15.69:0/2805005090 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb524004d20 con 0x55d7f8ac8970 2024-08-02T17:47:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.005+0000 7fb5167fc640 1 -- 172.21.15.69:0/2805005090 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb524021800 con 0x55d7f8ac8970 2024-08-02T17:47:30.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- 172.21.15.69:0/2805005090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 msgr2=0x55d7f8ae0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8ae0520 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb524008e20 tx=0x7fb524017000 comp rx=0 tx=0).stop 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- 172.21.15.69:0/2805005090 shutdown_connections 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8ae0520 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abe0d0 0x55d7f8adffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 --2- 172.21.15.69:0/2805005090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abd7b0 0x55d7f8ac4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- 172.21.15.69:0/2805005090 >> 172.21.15.69:0/2805005090 conn(0x55d7f8ab9d20 msgr2=0x55d7f8ac7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- 172.21.15.69:0/2805005090 shutdown_connections 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- 172.21.15.69:0/2805005090 wait complete. 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- start start 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 Processor -- start 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8acc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:30.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abe0d0 0x55d7f8accc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8acd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1af560 con 0x55d7f8ac8970 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b09d0 con 0x55d7f8abe0d0 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1afc30 con 0x55d7f8abd7b0 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.006+0000 7fb535c40740 10 client.0 _ll_register_callbacks cb 0x7fb520005750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb53148d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8acd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8acc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8acc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55474/0 (socket says 172.21.15.69:55474) 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb53148d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8acd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34934/0 (socket says 172.21.15.69:34934) 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 -- 172.21.15.69:0/1578109568 learned_addr learned my addr 172.21.15.69:0/1578109568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abe0d0 0x55d7f8accc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 -- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abe0d0 msgr2=0x55d7f8accc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abe0d0 0x55d7f8accc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 -- 172.21.15.69:0/1578109568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 msgr2=0x55d7f8acd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8acd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 -- 172.21.15.69:0/1578109568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f8b42000 con 0x55d7f8abd7b0 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb53148d640 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8acd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abe0d0 0x55d7f8accc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb530c8c640 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8acc700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5200b8b30 tx=0x7fb520031a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc3a10e61f7fc0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb50bfff640 1 -- 172.21.15.69:0/1578109568 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520049040 con 0x55d7f8abd7b0 2024-08-02T17:47:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb50bfff640 1 -- 172.21.15.69:0/1578109568 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb5200b87d0 con 0x55d7f8abd7b0 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb535c40740 1 -- 172.21.15.69:0/1578109568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb524004d20 con 0x55d7f8abd7b0 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb50bfff640 1 -- 172.21.15.69:0/1578109568 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb5200bc860 con 0x55d7f8abd7b0 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb535c40740 10 client.25249 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.007+0000 7fb535c40740 1 -- 172.21.15.69:0/1578109568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d7f8b43330 con 0x55d7f8abd7b0 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb50bfff640 1 -- 172.21.15.69:0/1578109568 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb524004d20 con 0x55d7f8abd7b0 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb50bfff640 1 -- 172.21.15.69:0/1578109568 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb5200d7050 con 0x55d7f8abd7b0 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb535c40740 10 client.25249 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb535c40740 10 client.25249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb535c40740 20 client.25249 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb535c40740 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d7f8b43a50 0x55d7f8b45e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb535c40740 4 client.25249 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb535c40740 1 -- 172.21.15.69:0/1578109568 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d7f8a29430 con 0x55d7f8b43a50 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d7f8b43a50 0x55d7f8b45e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb50bfff640 10 client.25249 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.008+0000 7fb52bfff640 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d7f8b43a50 0x55d7f8b45e70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb5e4fa5ec427ee server_cookie=b42238039490c179 in_seq=0 out_seq=0 2024-08-02T17:47:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb50bfff640 1 -- 172.21.15.69:0/1578109568 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d7f8a29430 con 0x55d7f8b43a50 2024-08-02T17:47:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb50bfff640 10 client.25249 handle_command_reply: tid=0 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 client.25249 shutdown 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 -- 172.21.15.69:0/1578109568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 msgr2=0x55d7f8acc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8acc700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb5200b8b30 tx=0x7fb520031a60 comp rx=0 tx=0).stop 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 -- 172.21.15.69:0/1578109568 shutdown_connections 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d7f8ac8970 0x55d7f8acd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d7f8b43a50 0x55d7f8b45e70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d7f8abe0d0 0x55d7f8accc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 --2- 172.21.15.69:0/1578109568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d7f8abd7b0 0x55d7f8acc700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb50bfff640 0 client.25249 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 -- 172.21.15.69:0/1578109568 >> 172.21.15.69:0/1578109568 conn(0x55d7f8ab9d20 msgr2=0x55d7f8ab8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 -- 172.21.15.69:0/1578109568 shutdown_connections 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.038+0000 7fb535c40740 1 -- 172.21.15.69:0/1578109568 wait complete. 2024-08-02T17:47:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb535c40740 20 client.25249 trim_cache size 0 max 16384 2024-08-02T17:47:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:47:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:47:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:47:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:47:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 -- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb51c0a2ea0 msgr2=0x7fb51c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb51c0a2ea0 0x7fb51c0a5360 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb52c102720 tx=0x7fb524014950 comp rx=0 tx=0).stop 2024-08-02T17:47:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 -- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 msgr2=0x7fb52c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 0x7fb52c19a320 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb520006010 tx=0x7fb5200042f0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 -- 172.21.15.69:0/3196969691 shutdown_connections 2024-08-02T17:47:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb51c0a2ea0 0x7fb51c0a5360 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb52c10e0f0 0x7fb52c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb52c100ef0 0x7fb52c19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 --2- 172.21.15.69:0/3196969691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb52c1005d0 0x7fb52c19a320 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 -- 172.21.15.69:0/3196969691 >> 172.21.15.69:0/3196969691 conn(0x7fb52c0fc410 msgr2=0x7fb52c0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 -- 172.21.15.69:0/3196969691 shutdown_connections 2024-08-02T17:47:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:30.039+0000 7fb532ef0640 1 -- 172.21.15.69:0/3196969691 wait complete. 2024-08-02T17:47:31.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:31 smithi077 ceph-mon[37108]: pgmap v36: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 40 MiB/s wr, 231 op/s 2024-08-02T17:47:31.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:31 smithi069 ceph-mon[32746]: pgmap v36: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 40 MiB/s wr, 231 op/s 2024-08-02T17:47:31.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:31 smithi083 ceph-mon[37031]: pgmap v36: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 40 MiB/s wr, 231 op/s 2024-08-02T17:47:31.902 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:47:32.172 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:32.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 -- 172.21.15.69:0/3085502359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 msgr2=0x7f375c101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:32.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 --2- 172.21.15.69:0/3085502359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 0x7f375c101d20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3750009930 tx=0x7f375002a420 comp rx=0 tx=0).stop 2024-08-02T17:47:32.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 -- 172.21.15.69:0/3085502359 shutdown_connections 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 --2- 172.21.15.69:0/3085502359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 0x7f375c101d20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 --2- 172.21.15.69:0/3085502359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f375c102260 0x7f375c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 --2- 172.21.15.69:0/3085502359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c1078c0 0x7f375c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 -- 172.21.15.69:0/3085502359 >> 172.21.15.69:0/3085502359 conn(0x7f375c0fd620 msgr2=0x7f375c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 -- 172.21.15.69:0/3085502359 shutdown_connections 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 -- 172.21.15.69:0/3085502359 wait complete. 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 Processor -- start 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.632+0000 7f3764577640 1 -- start start 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3764577640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 0x7f375c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3764577640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f375c102260 0x7f375c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3764577640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c1078c0 0x7f375c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3764577640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f375c111cc0 con 0x7f375c1018c0 2024-08-02T17:47:32.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3764577640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f375c111b40 con 0x7f375c102260 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3764577640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f375c111e40 con 0x7f375c1078c0 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 0x7f375c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 0x7f375c074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34964/0 (socket says 172.21.15.69:34964) 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 -- 172.21.15.69:0/1705255641 learned_addr learned my addr 172.21.15.69:0/1705255641 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3761b12640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f375c102260 0x7f375c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762b14640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c1078c0 0x7f375c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 -- 172.21.15.69:0/1705255641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c1078c0 msgr2=0x7f375c074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c1078c0 0x7f375c074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 -- 172.21.15.69:0/1705255641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f375c102260 msgr2=0x7f375c074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f375c102260 0x7f375c074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 -- 172.21.15.69:0/1705255641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375c1a6640 con 0x7f375c1018c0 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f3762313640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 0x7f375c074220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f375400eb90 tx=0x7f375400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:32.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.633+0000 7f375b7fe640 1 -- 172.21.15.69:0/1705255641 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754004d20 con 0x7f375c1018c0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.634+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375c1a6930 con 0x7f375c1018c0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.634+0000 7f375b7fe640 1 -- 172.21.15.69:0/1705255641 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f375400ee90 con 0x7f375c1018c0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.634+0000 7f375b7fe640 1 -- 172.21.15.69:0/1705255641 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3754010830 con 0x7f375c1018c0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.634+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f375c1a6e40 con 0x7f375c1018c0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.635+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375c101d20 con 0x7f375c1018c0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.636+0000 7f375b7fe640 1 -- 172.21.15.69:0/1705255641 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f3754005020 con 0x7f375c1018c0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.636+0000 7f375b7fe640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37440a2ea0 0x7f37440a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.636+0000 7f3761b12640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37440a2ea0 0x7f37440a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.636+0000 7f375b7fe640 1 -- 172.21.15.69:0/1705255641 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f37540d00b0 con 0x7f375c1018c0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.637+0000 7f3761b12640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37440a2ea0 0x7f37440a5360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3750009900 tx=0x7f37500058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:32.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.640+0000 7f375b7fe640 1 -- 172.21.15.69:0/1705255641 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f375409c6d0 con 0x7f375c1018c0 2024-08-02T17:47:32.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.840+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f375c109e30 con 0x7f375c1018c0 2024-08-02T17:47:32.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.841+0000 7f375b7fe640 1 -- 172.21.15.69:0/1705255641 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f375409bfd0 con 0x7f375c1018c0 2024-08-02T17:47:32.843 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:47:32.843 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:47:32.843 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:47:32.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37440a2ea0 msgr2=0x7f37440a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:32.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37440a2ea0 0x7f37440a5360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3750009900 tx=0x7f37500058d0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 msgr2=0x7f375c074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:32.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 0x7f375c074220 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f375400eb90 tx=0x7f375400c6a0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 shutdown_connections 2024-08-02T17:47:32.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37440a2ea0 0x7f37440a5360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f375c1078c0 0x7f375c074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f375c102260 0x7f375c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 --2- 172.21.15.69:0/1705255641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f375c1018c0 0x7f375c074220 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:32.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 >> 172.21.15.69:0/1705255641 conn(0x7f375c0fd620 msgr2=0x7f375c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:32.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 shutdown_connections 2024-08-02T17:47:32.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:32.843+0000 7f3764577640 1 -- 172.21.15.69:0/1705255641 wait complete. 2024-08-02T17:47:33.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:32 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1705255641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:32 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1705255641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:32 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1705255641' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:34.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:33 smithi083 ceph-mon[37031]: pgmap v37: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 133 op/s 2024-08-02T17:47:34.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:33 smithi077 ceph-mon[37108]: pgmap v37: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 133 op/s 2024-08-02T17:47:34.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:33 smithi069 ceph-mon[32746]: pgmap v37: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 133 op/s 2024-08-02T17:47:34.610 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 14b377d1 --release --await' 2024-08-02T17:47:34.610 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 14b377d1 --release --await 2024-08-02T17:47:34.873 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:34.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 25.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:47:35.311 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:35 smithi069 ceph-mon[32746]: pgmap v38: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 133 op/s 2024-08-02T17:47:35.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 -- 172.21.15.69:0/1277960143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 msgr2=0x7f9c281026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 --2- 172.21.15.69:0/1277960143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 0x7f9c281026e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009900 tx=0x7f9c1802a480 comp rx=0 tx=0).stop 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 -- 172.21.15.69:0/1277960143 shutdown_connections 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 --2- 172.21.15.69:0/1277960143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 0x7f9c281026e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 --2- 172.21.15.69:0/1277960143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c28102c20 0x7f9c2810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 --2- 172.21.15.69:0/1277960143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c28100b50 0x7f9c28100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 -- 172.21.15.69:0/1277960143 >> 172.21.15.69:0/1277960143 conn(0x7f9c280fc500 msgr2=0x7f9c280fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 -- 172.21.15.69:0/1277960143 shutdown_connections 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.322+0000 7f9c2db42640 1 -- 172.21.15.69:0/1277960143 wait complete. 2024-08-02T17:47:35.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c2db42640 1 Processor -- start 2024-08-02T17:47:35.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c2db42640 1 -- start start 2024-08-02T17:47:35.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c2db42640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c28100b50 0x7f9c28198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c2db42640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 0x7f9c28198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c2db42640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c28102c20 0x7f9c2819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c2db42640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28110c70 con 0x7f9c281022a0 2024-08-02T17:47:35.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c2db42640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28110af0 con 0x7f9c28100b50 2024-08-02T17:47:35.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c2db42640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28110df0 con 0x7f9c28102c20 2024-08-02T17:47:35.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 0x7f9c28198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 0x7f9c28198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49648/0 (socket says 172.21.15.69:49648) 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c26ffd640 1 -- 172.21.15.69:0/2202139742 learned_addr learned my addr 172.21.15.69:0/2202139742 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c27fff640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c28102c20 0x7f9c2819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c277fe640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c28100b50 0x7f9c28198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c26ffd640 1 -- 172.21.15.69:0/2202139742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c28102c20 msgr2=0x7f9c2819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c28102c20 0x7f9c2819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c26ffd640 1 -- 172.21.15.69:0/2202139742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c28100b50 msgr2=0x7f9c28198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c28100b50 0x7f9c28198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c26ffd640 1 -- 172.21.15.69:0/2202139742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2819d1f0 con 0x7f9c281022a0 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.323+0000 7f9c277fe640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c28100b50 0x7f9c28198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.324+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 0x7f9c28198780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9c18006010 tx=0x7f9c18003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.324+0000 7f9c24ff9640 1 -- 172.21.15.69:0/2202139742 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1803f070 con 0x7f9c281022a0 2024-08-02T17:47:35.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.324+0000 7f9c2db42640 1 -- 172.21.15.69:0/2202139742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2819d480 con 0x7f9c281022a0 2024-08-02T17:47:35.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.324+0000 7f9c24ff9640 1 -- 172.21.15.69:0/2202139742 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c18031c30 con 0x7f9c281022a0 2024-08-02T17:47:35.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.324+0000 7f9c24ff9640 1 -- 172.21.15.69:0/2202139742 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18035030 con 0x7f9c281022a0 2024-08-02T17:47:35.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.324+0000 7f9c2db42640 1 -- 172.21.15.69:0/2202139742 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c281a4dc0 con 0x7f9c281022a0 2024-08-02T17:47:35.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.326+0000 7f9c24ff9640 1 -- 172.21.15.69:0/2202139742 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f9c18042070 con 0x7f9c281022a0 2024-08-02T17:47:35.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.326+0000 7f9c2ff40740 1 Processor -- start 2024-08-02T17:47:35.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c2ff40740 1 -- start start 2024-08-02T17:47:35.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e79159ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915a7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e791500c60 con 0x55e79159b2b0 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e791498b90 con 0x55e79159a990 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7914fcc20 con 0x55e7915a5b70 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c24ff9640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c140a2ea0 0x7f9c140a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49656/0 (socket says 172.21.15.69:49656) 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c277fe640 1 -- 172.21.15.69:0/1602829305 learned_addr learned my addr 172.21.15.69:0/1602829305 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c27fff640 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e79159ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c27fff640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c140a2ea0 0x7f9c140a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c24ff9640 1 -- 172.21.15.69:0/2202139742 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f9c180f01f0 con 0x7f9c281022a0 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915a7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c277fe640 1 -- 172.21.15.69:0/1602829305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 msgr2=0x55e7915a7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915a7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c277fe640 1 -- 172.21.15.69:0/1602829305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 msgr2=0x55e79159ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e79159ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c277fe640 1 -- 172.21.15.69:0/1602829305 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e791210580 con 0x55e79159b2b0 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.327+0000 7f9c27fff640 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e79159ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.328+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a5630 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009d50 tx=0x7f9c100117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d054017f3532b22 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.328+0000 7f9c27fff640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c140a2ea0 0x7f9c140a5360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009fd0 tx=0x7f9c20009600 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.328+0000 7f9c1dffb640 1 -- 172.21.15.69:0/1602829305 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10012650 con 0x55e79159b2b0 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.328+0000 7f9c1dffb640 1 -- 172.21.15.69:0/1602829305 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c10009e40 con 0x55e79159b2b0 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.328+0000 7f9c1dffb640 1 -- 172.21.15.69:0/1602829305 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000fd00 con 0x55e79159b2b0 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1602829305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 msgr2=0x55e7915a5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a5630 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009d50 tx=0x7f9c100117b0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1602829305 shutdown_connections 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a5630 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915a7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1602829305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e79159ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1602829305 >> 172.21.15.69:0/1602829305 conn(0x55e791596f20 msgr2=0x55e791597300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:35.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1602829305 shutdown_connections 2024-08-02T17:47:35.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1602829305 wait complete. 2024-08-02T17:47:35.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 -- start start 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 Processor -- start 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159a990 0x55e7915a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159b2b0 0x55e7915a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.329+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915a3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e7915a8740 con 0x55e79159a990 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e79149e7c0 con 0x55e79159b2b0 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7915a88c0 con 0x55e7915a5b70 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c2ff40740 10 client.0 _ll_register_callbacks cb 0x55e7914a7f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159a990 0x55e7915a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159a990 0x55e7915a29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49670/0 (socket says 172.21.15.69:49670) 2024-08-02T17:47:35.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 -- 172.21.15.69:0/1824698983 learned_addr learned my addr 172.21.15.69:0/1824698983 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159b2b0 0x55e7915a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c27fff640 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 -- 172.21.15.69:0/1824698983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 msgr2=0x55e7915a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 -- 172.21.15.69:0/1824698983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159b2b0 msgr2=0x55e7915a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159b2b0 0x55e7915a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 -- 172.21.15.69:0/1824698983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7912da960 con 0x55e79159a990 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159a990 0x55e7915a29e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9c10011cb0 tx=0x7f9c10005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcbf0406ce5cc98 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1824698983 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c10027030 con 0x55e79159a990 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1824698983 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f9c10004250 con 0x55e79159a990 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1824698983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e79150e1d0 con 0x55e79159a990 2024-08-02T17:47:35.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1824698983 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c1000f440 con 0x55e79159a990 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c2ff40740 10 client.15729 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.330+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1824698983 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7915b4700 con 0x55e79159a990 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1824698983 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e79150e1d0 con 0x55e79159a990 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1824698983 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c10004c50 con 0x55e79159a990 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c2ff40740 10 client.15729 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c2ff40740 10 client.15729 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c2ff40740 20 client.15729 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7915b5b00 0x55e7915b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c2ff40740 4 client.15729 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1824698983 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7915086e0 con 0x55e7915b5b00 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7915b5b00 0x55e7915b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7915b5b00 0x55e7915b7ef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c323906d9b36f99 server_cookie=6d5756cb4cabf86 in_seq=0 out_seq=0 2024-08-02T17:47:35.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.331+0000 7f9c06ffd640 10 client.15729 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1824698983 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e7915086e0 con 0x55e7915b5b00 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c06ffd640 10 client.15729 handle_command_reply: tid=0 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 client.15729 shutdown 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1824698983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159a990 msgr2=0x55e7915a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159a990 0x55e7915a29e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9c10011cb0 tx=0x7f9c10005710 comp rx=0 tx=0).stop 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1824698983 shutdown_connections 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7915b5b00 0x55e7915b7ef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159b2b0 0x55e7915a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1824698983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159a990 0x55e7915a29e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c06ffd640 0 client.15729 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:35.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1824698983 >> 172.21.15.69:0/1824698983 conn(0x55e791596f20 msgr2=0x55e79159cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:35.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.332+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1824698983 shutdown_connections 2024-08-02T17:47:35.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.333+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1824698983 wait complete. 2024-08-02T17:47:35.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.333+0000 7f9c2ff40740 20 client.15729 trim_cache size 0 max 16384 2024-08-02T17:47:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:35 smithi077 ceph-mon[37108]: pgmap v38: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 133 op/s 2024-08-02T17:47:35.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.349+0000 7f9c2ff40740 1 Processor -- start 2024-08-02T17:47:35.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.349+0000 7f9c2ff40740 1 -- start start 2024-08-02T17:47:35.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79159a990 0x55e7915a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7915a5b70 0x55e7915bdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b1640 con 0x55e79159b2b0 2024-08-02T17:47:35.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281aef10 con 0x55e7915a5b70 2024-08-02T17:47:35.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281acd60 con 0x55e79159a990 2024-08-02T17:47:35.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915bd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49686/0 (socket says 172.21.15.69:49686) 2024-08-02T17:47:35.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 -- 172.21.15.69:0/816451703 learned_addr learned my addr 172.21.15.69:0/816451703 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:35.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c27fff640 1 --2- 172.21.15.69:0/816451703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7915a5b70 0x55e7915bdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 -- 172.21.15.69:0/816451703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79159a990 msgr2=0x55e7915a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:35.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/816451703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79159a990 0x55e7915a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 -- 172.21.15.69:0/816451703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7915a5b70 msgr2=0x55e7915bdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:35.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/816451703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7915a5b70 0x55e7915bdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 -- 172.21.15.69:0/816451703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e791596d50 con 0x55e79159b2b0 2024-08-02T17:47:35.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/816451703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915bd580 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9c180bbe10 tx=0x7f9c1803d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa266fda7a5cf763 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:35.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c1d7fa640 1 -- 172.21.15.69:0/816451703 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18049040 con 0x55e79159b2b0 2024-08-02T17:47:35.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c1d7fa640 1 -- 172.21.15.69:0/816451703 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c180d50d0 con 0x55e79159b2b0 2024-08-02T17:47:35.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.350+0000 7f9c1d7fa640 1 -- 172.21.15.69:0/816451703 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1803b460 con 0x55e79159b2b0 2024-08-02T17:47:35.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 -- 172.21.15.69:0/816451703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 msgr2=0x55e7915bd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/816451703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915bd580 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9c180bbe10 tx=0x7f9c1803d040 comp rx=0 tx=0).stop 2024-08-02T17:47:35.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 -- 172.21.15.69:0/816451703 shutdown_connections 2024-08-02T17:47:35.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/816451703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915bd580 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/816451703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7915a5b70 0x55e7915bdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/816451703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e79159a990 0x55e7915a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 -- 172.21.15.69:0/816451703 >> 172.21.15.69:0/816451703 conn(0x55e791596f20 msgr2=0x55e7915a19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 -- 172.21.15.69:0/816451703 shutdown_connections 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 -- 172.21.15.69:0/816451703 wait complete. 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 -- start start 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.351+0000 7f9c2ff40740 1 Processor -- start 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e7915a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c2ff40740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915aa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281ad3d0 con 0x55e79159b2b0 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281ae840 con 0x55e79159a990 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c2ff40740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281adaa0 con 0x55e7915a5b70 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c2ff40740 10 client.0 _ll_register_callbacks cb 0x7f9c1000fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49690/0 (socket says 172.21.15.69:49690) 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 -- 172.21.15.69:0/1172408735 learned_addr learned my addr 172.21.15.69:0/1172408735 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c27fff640 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e7915a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 -- 172.21.15.69:0/1172408735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 msgr2=0x55e7915aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 -- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 msgr2=0x55e7915a9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:35.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e7915a9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 -- 172.21.15.69:0/1172408735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79161f750 con 0x55e79159b2b0 2024-08-02T17:47:35.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e7915a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:35.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c26ffd640 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a9fd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9c180bc0a0 tx=0x7f9c180c04c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff51f4cf3dfaca83 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:35.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:35.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1172408735 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18049030 con 0x55e79159b2b0 2024-08-02T17:47:35.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1172408735 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f9c1803b460 con 0x55e79159b2b0 2024-08-02T17:47:35.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.352+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1172408735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c180d50d0 con 0x55e79159b2b0 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1172408735 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c180bc110 con 0x55e79159b2b0 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c2ff40740 10 client.15735 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1172408735 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e791620a30 con 0x55e79159b2b0 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1172408735 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c180d50d0 con 0x55e79159b2b0 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1172408735 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c180eece0 con 0x55e79159b2b0 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c2ff40740 10 client.15735 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c2ff40740 10 client.15735 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c2ff40740 20 client.15735 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e791621100 0x55e7916234f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c2ff40740 4 client.15735 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "14b377d1", "release": true, "await": true}] 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1172408735 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "14b377d1", "release": true, "await": true}) -- 0x55e7914855a0 con 0x55e791621100 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.353+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e791621100 0x55e7916234f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.354+0000 7f9c277fe640 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e791621100 0x55e7916234f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95204bdf222b6ff8 server_cookie=beaabde87c8d7b1 in_seq=0 out_seq=0 2024-08-02T17:47:35.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.354+0000 7f9c06ffd640 10 client.15735 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:35.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.398+0000 7f9c06ffd640 1 -- 172.21.15.69:0/1172408735 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e7914855a0 con 0x55e791621100 2024-08-02T17:47:35.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.398+0000 7f9c06ffd640 10 client.15735 handle_command_reply: tid=0 2024-08-02T17:47:35.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.398+0000 7f9c2ff40740 1 client.15735 shutdown 2024-08-02T17:47:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.399+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1172408735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 msgr2=0x55e7915a9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.399+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a9fd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9c180bc0a0 tx=0x7f9c180c04c0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.400+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1172408735 shutdown_connections 2024-08-02T17:47:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.400+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e79159b2b0 0x55e7915a9fd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.400+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e791621100 0x55e7916234f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.400+0000 7f9c06ffd640 0 client.15735 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.400+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7915a5b70 0x55e7915aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.400+0000 7f9c2ff40740 1 --2- 172.21.15.69:0/1172408735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e79159a990 0x55e7915a9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.400+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1172408735 >> 172.21.15.69:0/1172408735 conn(0x55e791596f20 msgr2=0x55e79159c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.401+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1172408735 shutdown_connections 2024-08-02T17:47:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.401+0000 7f9c2ff40740 1 -- 172.21.15.69:0/1172408735 wait complete. 2024-08-02T17:47:35.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.401+0000 7f9c2ff40740 20 client.15735 trim_cache size 0 max 16384 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 4, 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "14b377d1": { 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 4, 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:47:35.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 141.9, 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:35.405 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:35.406 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:35.406 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:47:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 -- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c140a2ea0 msgr2=0x7f9c140a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c140a2ea0 0x7f9c140a5360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009fd0 tx=0x7f9c20009600 comp rx=0 tx=0).stop 2024-08-02T17:47:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 -- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 msgr2=0x7f9c28198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 0x7f9c28198780 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9c18006010 tx=0x7f9c18003c10 comp rx=0 tx=0).stop 2024-08-02T17:47:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 -- 172.21.15.69:0/2202139742 shutdown_connections 2024-08-02T17:47:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c281022a0 0x7f9c28198780 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c140a2ea0 0x7f9c140a5360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c28102c20 0x7f9c2819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 --2- 172.21.15.69:0/2202139742 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c28100b50 0x7f9c28198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 -- 172.21.15.69:0/2202139742 >> 172.21.15.69:0/2202139742 conn(0x7f9c280fc500 msgr2=0x7f9c28069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 -- 172.21.15.69:0/2202139742 shutdown_connections 2024-08-02T17:47:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:35.403+0000 7f9c2db42640 1 -- 172.21.15.69:0/2202139742 wait complete. 2024-08-02T17:47:35.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:35 smithi083 ceph-mon[37031]: pgmap v38: 129 pgs: 129 active+clean; 969 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 40 MiB/s wr, 133 op/s 2024-08-02T17:47:36.130 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '14b377d1', total seconds elapsed: 26.1 2024-08-02T17:47:36.130 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.95 2024-08-02T17:47:36.130 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:47:36.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:36 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:47:36.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:36 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:47:36.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:36 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:47:36.586 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 -- 172.21.15.69:0/2105110614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 msgr2=0x7feb7010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 --2- 172.21.15.69:0/2105110614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 0x7feb7010e790 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb68009c10 tx=0x7feb6802a700 comp rx=0 tx=0).stop 2024-08-02T17:47:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 -- 172.21.15.69:0/2105110614 shutdown_connections 2024-08-02T17:47:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 --2- 172.21.15.69:0/2105110614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb701018c0 0x7feb70101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 --2- 172.21.15.69:0/2105110614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 0x7feb7010e790 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 --2- 172.21.15.69:0/2105110614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb701078c0 0x7feb70107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 -- 172.21.15.69:0/2105110614 >> 172.21.15.69:0/2105110614 conn(0x7feb700fd5e0 msgr2=0x7feb700ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:37.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 -- 172.21.15.69:0/2105110614 shutdown_connections 2024-08-02T17:47:37.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.152+0000 7feb78348640 1 -- 172.21.15.69:0/2105110614 wait complete. 2024-08-02T17:47:37.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb78348640 1 Processor -- start 2024-08-02T17:47:37.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb78348640 1 -- start start 2024-08-02T17:47:37.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb78348640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb701018c0 0x7feb7010db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:37.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb78348640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 0x7feb7010e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:37.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb78348640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb701078c0 0x7feb701a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:37.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb78348640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb70111cc0 con 0x7feb701018c0 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb78348640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feb70111b40 con 0x7feb701078c0 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb78348640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb70111e40 con 0x7feb70102260 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb758e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 0x7feb7010e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.153+0000 7feb758e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 0x7feb7010e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38892/0 (socket says 172.21.15.69:38892) 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb758e3640 1 -- 172.21.15.69:0/2558943681 learned_addr learned my addr 172.21.15.69:0/2558943681 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb768e5640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb701078c0 0x7feb701a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb768e5640 1 -- 172.21.15.69:0/2558943681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 msgr2=0x7feb7010e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb768e5640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 0x7feb7010e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb768e5640 1 -- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb701018c0 msgr2=0x7feb7010db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb760e4640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb701018c0 0x7feb7010db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb768e5640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb701018c0 0x7feb7010db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb768e5640 1 -- 172.21.15.69:0/2558943681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb701a3780 con 0x7feb701078c0 2024-08-02T17:47:37.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb760e4640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb701018c0 0x7feb7010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:37.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.154+0000 7feb768e5640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb701078c0 0x7feb701a3050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feb68034040 tx=0x7feb68003b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:37.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.155+0000 7feb677fe640 1 -- 172.21.15.69:0/2558943681 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68041020 con 0x7feb701078c0 2024-08-02T17:47:37.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.155+0000 7feb78348640 1 -- 172.21.15.69:0/2558943681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb701a3a10 con 0x7feb701078c0 2024-08-02T17:47:37.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.155+0000 7feb677fe640 1 -- 172.21.15.69:0/2558943681 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb68030880 con 0x7feb701078c0 2024-08-02T17:47:37.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.155+0000 7feb78348640 1 -- 172.21.15.69:0/2558943681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb701ab2c0 con 0x7feb701078c0 2024-08-02T17:47:37.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.155+0000 7feb677fe640 1 -- 172.21.15.69:0/2558943681 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68036040 con 0x7feb701078c0 2024-08-02T17:47:37.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.158+0000 7feb657fa640 1 -- 172.21.15.69:0/2558943681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb70101d20 con 0x7feb701078c0 2024-08-02T17:47:37.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.158+0000 7feb677fe640 1 -- 172.21.15.69:0/2558943681 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7feb68044020 con 0x7feb701078c0 2024-08-02T17:47:37.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.160+0000 7feb677fe640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb5c0a2fa0 0x7feb5c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:37.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.160+0000 7feb760e4640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb5c0a2fa0 0x7feb5c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:37.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.160+0000 7feb677fe640 1 -- 172.21.15.69:0/2558943681 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7feb680f0720 con 0x7feb701078c0 2024-08-02T17:47:37.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.160+0000 7feb760e4640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb5c0a2fa0 0x7feb5c0a5460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009ad0 tx=0x7feb6c005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:37.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.165+0000 7feb677fe640 1 -- 172.21.15.69:0/2558943681 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feb6804b020 con 0x7feb701078c0 2024-08-02T17:47:37.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:47:37.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:47:37.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:47:37.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:37 smithi077 ceph-mon[37108]: pgmap v39: 129 pgs: 129 active+clean; 971 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 40 MiB/s wr, 157 op/s 2024-08-02T17:47:37.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:47:37.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:47:37.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:47:37.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:37 smithi069 ceph-mon[32746]: pgmap v39: 129 pgs: 129 active+clean; 971 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 40 MiB/s wr, 157 op/s 2024-08-02T17:47:37.422 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:47:37.422 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:47:37.422 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:47:37.422 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:37 smithi083 ceph-mon[37031]: pgmap v39: 129 pgs: 129 active+clean; 971 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 40 MiB/s wr, 157 op/s 2024-08-02T17:47:37.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.559+0000 7feb657fa640 1 -- 172.21.15.69:0/2558943681 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb70062b20 con 0x7feb701078c0 2024-08-02T17:47:37.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.560+0000 7feb677fe640 1 -- 172.21.15.69:0/2558943681 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7feb680b8030 con 0x7feb701078c0 2024-08-02T17:47:37.564 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:47:37.564 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:47:37.564 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:47:37.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 -- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb5c0a2fa0 msgr2=0x7feb5c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:37.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb5c0a2fa0 0x7feb5c0a5460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009ad0 tx=0x7feb6c005f90 comp rx=0 tx=0).stop 2024-08-02T17:47:37.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 -- 172.21.15.69:0/2558943681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb701078c0 msgr2=0x7feb701a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb701078c0 0x7feb701a3050 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feb68034040 tx=0x7feb68003b80 comp rx=0 tx=0).stop 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 -- 172.21.15.69:0/2558943681 shutdown_connections 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb5c0a2fa0 0x7feb5c0a5460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb701078c0 0x7feb701a3050 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb70102260 0x7feb7010e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 --2- 172.21.15.69:0/2558943681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb701018c0 0x7feb7010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.567+0000 7feb657fa640 1 -- 172.21.15.69:0/2558943681 >> 172.21.15.69:0/2558943681 conn(0x7feb700fd5e0 msgr2=0x7feb700fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.568+0000 7feb657fa640 1 -- 172.21.15.69:0/2558943681 shutdown_connections 2024-08-02T17:47:37.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:37.568+0000 7feb657fa640 1 -- 172.21.15.69:0/2558943681 wait complete. 2024-08-02T17:47:38.571 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:38 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2558943681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:38.621 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 141.95 --await-for 45.0' 2024-08-02T17:47:38.622 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 141.95 --await-for 45.0 2024-08-02T17:47:38.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:38 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2558943681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:38.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:38 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2558943681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:39.065 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:39.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.650+0000 7fc7b9182640 1 -- 172.21.15.69:0/1614330769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ac0a4830 msgr2=0x7fc7ac0a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.650+0000 7fc7b9182640 1 --2- 172.21.15.69:0/1614330769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ac0a4830 0x7fc7ac0a4c10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc7a801a910 tx=0x7fc7a803b4e0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.651+0000 7fc7b9182640 1 -- 172.21.15.69:0/1614330769 shutdown_connections 2024-08-02T17:47:39.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.651+0000 7fc7b9182640 1 --2- 172.21.15.69:0/1614330769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7ac0b7550 0x7fc7ac0b9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.651+0000 7fc7b9182640 1 --2- 172.21.15.69:0/1614330769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7ac0a5150 0x7fc7ac0b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.651+0000 7fc7b9182640 1 --2- 172.21.15.69:0/1614330769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ac0a4830 0x7fc7ac0a4c10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.651+0000 7fc7b9182640 1 -- 172.21.15.69:0/1614330769 >> 172.21.15.69:0/1614330769 conn(0x7fc7ac01a0f0 msgr2=0x7fc7ac01a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:39.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.651+0000 7fc7b9182640 1 -- 172.21.15.69:0/1614330769 shutdown_connections 2024-08-02T17:47:39.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.651+0000 7fc7b9182640 1 -- 172.21.15.69:0/1614330769 wait complete. 2024-08-02T17:47:39.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.652+0000 7fc7b9182640 1 Processor -- start 2024-08-02T17:47:39.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.652+0000 7fc7b9182640 1 -- start start 2024-08-02T17:47:39.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b9182640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7ac0a4830 0x7fc7ac0b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b9182640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ac0a5150 0x7fc7ac0b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b9182640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7ac0b7550 0x7fc7ac0b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b9182640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac0af1f0 con 0x7fc7ac0a5150 2024-08-02T17:47:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b9182640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac0af070 con 0x7fc7ac0b7550 2024-08-02T17:47:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b9182640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac0af370 con 0x7fc7ac0a4830 2024-08-02T17:47:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b8981640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7ac0b7550 0x7fc7ac0b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7ac0a4830 0x7fc7ac0b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b8981640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7ac0b7550 0x7fc7ac0b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33074/0 (socket says 172.21.15.69:33074) 2024-08-02T17:47:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b8981640 1 -- 172.21.15.69:0/2804546708 learned_addr learned my addr 172.21.15.69:0/2804546708 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:39.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ac0a5150 0x7fc7ac0b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b3fff640 1 -- 172.21.15.69:0/2804546708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7ac0b7550 msgr2=0x7fc7ac0b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b3fff640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7ac0b7550 0x7fc7ac0b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b3fff640 1 -- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ac0a5150 msgr2=0x7fc7ac0b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b3fff640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ac0a5150 0x7fc7ac0b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.653+0000 7fc7b3fff640 1 -- 172.21.15.69:0/2804546708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac0b39b0 con 0x7fc7ac0a4830 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.654+0000 7fc7b3fff640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7ac0a4830 0x7fc7ac0b6ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a8004c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.654+0000 7fc7b17fa640 1 -- 172.21.15.69:0/2804546708 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a804f020 con 0x7fc7ac0a4830 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.655+0000 7fc7b17fa640 1 -- 172.21.15.69:0/2804546708 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a8004260 con 0x7fc7ac0a4830 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.655+0000 7fc7b17fa640 1 -- 172.21.15.69:0/2804546708 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8046070 con 0x7fc7ac0a4830 2024-08-02T17:47:39.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.655+0000 7fc7b9182640 1 -- 172.21.15.69:0/2804546708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac147370 con 0x7fc7ac0a4830 2024-08-02T17:47:39.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.655+0000 7fc7b9182640 1 -- 172.21.15.69:0/2804546708 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac147710 con 0x7fc7ac0a4830 2024-08-02T17:47:39.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.659+0000 7fc7b17fa640 1 -- 172.21.15.69:0/2804546708 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7fc7a8052020 con 0x7fc7ac0a4830 2024-08-02T17:47:39.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.659+0000 7fc7bc7e2740 1 Processor -- start 2024-08-02T17:47:39.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.659+0000 7fc7b17fa640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7a40a2fe0 0x7fc7a40a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.659+0000 7fc7bc7e2740 1 -- start start 2024-08-02T17:47:39.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b8981640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7a40a2fe0 0x7fc7a40a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e0157870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0157db0 0x55e1e0162130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0162670 0x55e1e0164a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7bc7e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e1e00bc810 con 0x55e1e0157db0 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7bc7e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e1e0063c70 con 0x55e1e0162670 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7bc7e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1e00b87d0 con 0x55e1e0157490 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0157db0 0x55e1e0162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e0157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0157db0 0x55e1e0162130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49744/0 (socket says 172.21.15.69:49744) 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b37fe640 1 -- 172.21.15.69:0/3369833799 learned_addr learned my addr 172.21.15.69:0/3369833799 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b8981640 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0162670 0x55e1e0164a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e0157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38922/0 (socket says 172.21.15.69:38922) 2024-08-02T17:47:39.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b17fa640 1 -- 172.21.15.69:0/2804546708 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc7a8100f30 con 0x7fc7ac0a4830 2024-08-02T17:47:39.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b37fe640 1 -- 172.21.15.69:0/3369833799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 msgr2=0x55e1e0157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:39.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e0157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b37fe640 1 -- 172.21.15.69:0/3369833799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0162670 msgr2=0x55e1e0164a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:39.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0162670 0x55e1e0164a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b37fe640 1 -- 172.21.15.69:0/3369833799 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1dfe2eba0 con 0x55e1e0157db0 2024-08-02T17:47:39.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.660+0000 7fc7b8981640 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0162670 0x55e1e0164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:39.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.661+0000 7fc7b8981640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7a40a2fe0 0x7fc7a40a54a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4050420 tx=0x7fc7b4068400 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:39.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.661+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0157db0 0x55e1e0162130 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000abb0 tx=0x7fc7a000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155cd6e4aeceed1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.662+0000 7fc79e7fc640 1 -- 172.21.15.69:0/3369833799 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a001d070 con 0x55e1e0157db0 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.662+0000 7fc79e7fc640 1 -- 172.21.15.69:0/3369833799 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a0018b20 con 0x55e1e0157db0 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.663+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/3369833799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0157db0 msgr2=0x55e1e0162130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.663+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0157db0 0x55e1e0162130 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000abb0 tx=0x7fc7a000f790 comp rx=0 tx=0).stop 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.664+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/3369833799 shutdown_connections 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.664+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0157db0 0x55e1e0162130 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.664+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0162670 0x55e1e0164a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.664+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/3369833799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e0157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.664+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/3369833799 >> 172.21.15.69:0/3369833799 conn(0x55e1e0153a20 msgr2=0x55e1e0153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.664+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/3369833799 shutdown_connections 2024-08-02T17:47:39.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.664+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/3369833799 wait complete. 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 1 -- start start 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 1 Processor -- start 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e1e01872a0 con 0x55e1e0162670 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e1e00779a0 con 0x55e1e0157db0 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e1e0187420 con 0x55e1e0157490 2024-08-02T17:47:39.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7bc7e2740 10 client.0 _ll_register_callbacks cb 0x55e1e0065c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38926/0 (socket says 172.21.15.69:38926) 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.665+0000 7fc7b3fff640 1 -- 172.21.15.69:0/4135832734 learned_addr learned my addr 172.21.15.69:0/4135832734 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b8981640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b3fff640 1 -- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 msgr2=0x55e1e015f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b3fff640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b3fff640 1 -- 172.21.15.69:0/4135832734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 msgr2=0x55e1e015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b3fff640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b3fff640 1 -- 172.21.15.69:0/4135832734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e0083220 con 0x55e1e0157490 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b8981640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:47:39.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7b3fff640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015f4a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80c9820 tx=0x7fc7a80ccce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e9e28b388b569b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7937fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.666+0000 7fc7937fe640 1 -- 172.21.15.69:0/4135832734 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a80cd550 con 0x55e1e0157490 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7937fe640 1 -- 172.21.15.69:0/4135832734 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc7a80a70b0 con 0x55e1e0157490 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4135832734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a0018b20 con 0x55e1e0157490 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7937fe640 1 -- 172.21.15.69:0/4135832734 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7a80cce50 con 0x55e1e0157490 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 10 client.25285 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4135832734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1e01872a0 con 0x55e1e0157490 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7937fe640 1 -- 172.21.15.69:0/4135832734 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a0018b20 con 0x55e1e0157490 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7937fe640 1 -- 172.21.15.69:0/4135832734 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7a80d5630 con 0x55e1e0157490 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 10 client.25285 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 10 client.25285 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 20 client.25285 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1e01942d0 0x55e1e01966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 4 client.25285 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:39.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.667+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4135832734 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1e00c9f40 con 0x55e1e01942d0 2024-08-02T17:47:39.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.668+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1e01942d0 0x55e1e01966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.668+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1e01942d0 0x55e1e01966e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2da6e8b99b8af2f server_cookie=acf9e86de2334c4 in_seq=0 out_seq=0 2024-08-02T17:47:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.668+0000 7fc7937fe640 10 client.25285 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7937fe640 1 -- 172.21.15.69:0/4135832734 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e1e00c9f40 con 0x55e1e01942d0 2024-08-02T17:47:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7937fe640 10 client.25285 handle_command_reply: tid=0 2024-08-02T17:47:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 client.25285 shutdown 2024-08-02T17:47:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4135832734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 msgr2=0x55e1e015f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015f4a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc7a80c9820 tx=0x7fc7a80ccce0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4135832734 shutdown_connections 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1e01942d0 0x55e1e01966e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4135832734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015f4a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4135832734 >> 172.21.15.69:0/4135832734 conn(0x55e1e0153a20 msgr2=0x55e1e0152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7937fe640 0 client.25285 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4135832734 shutdown_connections 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4135832734 wait complete. 2024-08-02T17:47:39.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.669+0000 7fc7bc7e2740 20 client.25285 trim_cache size 0 max 16384 2024-08-02T17:47:39.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.697+0000 7fc7bc7e2740 1 Processor -- start 2024-08-02T17:47:39.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.697+0000 7fc7bc7e2740 1 -- start start 2024-08-02T17:47:39.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.697+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e0161370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38938/0 (socket says 172.21.15.69:38938) 2024-08-02T17:47:39.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e0152c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7b3fff640 1 -- 172.21.15.69:0/2688690733 learned_addr learned my addr 172.21.15.69:0/2688690733 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/2688690733 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac012190 con 0x55e1e0162670 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/2688690733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac00faa0 con 0x55e1e0157db0 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/2688690733 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac0b4270 con 0x55e1e0157490 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e0161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.698+0000 7fc7b8981640 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e0152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.699+0000 7fc7b37fe640 1 -- 172.21.15.69:0/2688690733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 msgr2=0x55e1e015e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.699+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.699+0000 7fc7b37fe640 1 -- 172.21.15.69:0/2688690733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 msgr2=0x55e1e0152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.699+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e0152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.699+0000 7fc7b37fe640 1 -- 172.21.15.69:0/2688690733 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e0153850 con 0x55e1e0157db0 2024-08-02T17:47:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.699+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e0161370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7a001b040 tx=0x7fc7a0010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba044a63f77d947 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.699+0000 7fc79dffb640 1 -- 172.21.15.69:0/2688690733 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a001d040 con 0x55e1e0157db0 2024-08-02T17:47:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.700+0000 7fc79dffb640 1 -- 172.21.15.69:0/2688690733 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a001e040 con 0x55e1e0157db0 2024-08-02T17:47:39.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.700+0000 7fc79dffb640 1 -- 172.21.15.69:0/2688690733 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a0005630 con 0x55e1e0157db0 2024-08-02T17:47:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.700+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/2688690733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 msgr2=0x55e1e0161370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.700+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e0161370 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7a001b040 tx=0x7fc7a0010430 comp rx=0 tx=0).stop 2024-08-02T17:47:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.701+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/2688690733 shutdown_connections 2024-08-02T17:47:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.701+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e0152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.701+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e0161370 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.701+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/2688690733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e015e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.701+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/2688690733 >> 172.21.15.69:0/2688690733 conn(0x55e1e0153a20 msgr2=0x55e1e0152560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.701+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/2688690733 shutdown_connections 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.701+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/2688690733 wait complete. 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.702+0000 7fc7bc7e2740 1 -- start start 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.702+0000 7fc7bc7e2740 1 Processor -- start 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e018ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e018ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7bc7e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7b3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e018ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38940/0 (socket says 172.21.15.69:38940) 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7b37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33124/0 (socket says 172.21.15.69:33124) 2024-08-02T17:47:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7b3fff640 1 -- 172.21.15.69:0/4176944475 learned_addr learned my addr 172.21.15.69:0/4176944475 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7b8981640 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac00df60 con 0x55e1e0162670 2024-08-02T17:47:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac00f3d0 con 0x55e1e0157db0 2024-08-02T17:47:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.703+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac00e630 con 0x55e1e0157490 2024-08-02T17:47:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.704+0000 7fc7bc7e2740 10 client.0 _ll_register_callbacks cb 0x7fc7a801a620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.704+0000 7fc7b37fe640 1 -- 172.21.15.69:0/4176944475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 msgr2=0x55e1e018ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.704+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e018ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.705+0000 7fc7b37fe640 1 -- 172.21.15.69:0/4176944475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 msgr2=0x55e1e015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.705+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.705+0000 7fc7b37fe640 1 -- 172.21.15.69:0/4176944475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e015cae0 con 0x55e1e0157db0 2024-08-02T17:47:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.705+0000 7fc7b37fe640 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015bfb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55e1e014f730 tx=0x7fc7a000a2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fac2f1673de591c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.705+0000 7fc7937fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:47:39.708 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:39 smithi069 ceph-mon[32746]: pgmap v40: 129 pgs: 129 active+clean; 971 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 60 op/s 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.705+0000 7fc7937fe640 1 -- 172.21.15.69:0/4176944475 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a001d050 con 0x55e1e0157db0 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.705+0000 7fc7937fe640 1 -- 172.21.15.69:0/4176944475 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc7a0005040 con 0x55e1e0157db0 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.705+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a001e040 con 0x55e1e0157db0 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7937fe640 1 -- 172.21.15.69:0/4176944475 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc7a0010570 con 0x55e1e0157db0 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7bc7e2740 10 client.25397 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1e018b500 con 0x55e1e0157db0 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7937fe640 1 -- 172.21.15.69:0/4176944475 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7a001e040 con 0x55e1e0157db0 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7937fe640 1 -- 172.21.15.69:0/4176944475 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc7a0041030 con 0x55e1e0157db0 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7bc7e2740 10 client.25397 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7bc7e2740 10 client.25397 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:47:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7bc7e2740 20 client.25397 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1e01ff170 0x55e1e0201560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7bc7e2740 4 client.25397 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.95, "await_for": 45.0}] 2024-08-02T17:47:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.95, "await_for": 45.0}) -- 0x55e1e00c4290 con 0x55e1e01ff170 2024-08-02T17:47:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.706+0000 7fc7b3fff640 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1e01ff170 0x55e1e0201560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.707+0000 7fc7937fe640 10 client.25397 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.708+0000 7fc7b3fff640 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1e01ff170 0x55e1e0201560 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1550bc15f995f7e0 server_cookie=43735c7f07af057f in_seq=0 out_seq=0 2024-08-02T17:47:39.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:39 smithi083 ceph-mon[37031]: pgmap v40: 129 pgs: 129 active+clean; 971 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 60 op/s 2024-08-02T17:47:39.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:39 smithi077 ceph-mon[37108]: pgmap v40: 129 pgs: 129 active+clean; 971 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 MiB/s wr, 60 op/s 2024-08-02T17:47:39.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7937fe640 1 -- 172.21.15.69:0/4176944475 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e1e00c4290 con 0x55e1e01ff170 2024-08-02T17:47:39.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7937fe640 10 client.25397 handle_command_reply: tid=0 2024-08-02T17:47:39.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 client.25397 shutdown 2024-08-02T17:47:39.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 msgr2=0x55e1e015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015bfb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55e1e014f730 tx=0x7fc7a000a2f0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 shutdown_connections 2024-08-02T17:47:39.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1e0162670 0x55e1e015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1e01ff170 0x55e1e0201560 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e1e0157db0 0x55e1e015bfb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 --2- 172.21.15.69:0/4176944475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1e0157490 0x55e1e018ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7937fe640 0 client.25397 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 >> 172.21.15.69:0/4176944475 conn(0x55e1e0153a20 msgr2=0x55e1e0152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 shutdown_connections 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.950+0000 7fc7bc7e2740 1 -- 172.21.15.69:0/4176944475 wait complete. 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7bc7e2740 20 client.25397 trim_cache size 0 max 16384 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:47:39.953 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 6, 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "2319cd26": { 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 6, 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 141.9, 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:47:39.954 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:47:39.955 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 -- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7a40a2fe0 msgr2=0x7fc7a40a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7a40a2fe0 0x7fc7a40a54a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4050420 tx=0x7fc7b4068400 comp rx=0 tx=0).stop 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 -- 172.21.15.69:0/2804546708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7ac0a4830 msgr2=0x7fc7ac0b6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7ac0a4830 0x7fc7ac0b6ca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8000c00 tx=0x7fc7a8004c20 comp rx=0 tx=0).stop 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 -- 172.21.15.69:0/2804546708 shutdown_connections 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc7ac0a5150 0x7fc7ac0b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7a40a2fe0 0x7fc7a40a54a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc7ac0b7550 0x7fc7ac0b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 --2- 172.21.15.69:0/2804546708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7ac0a4830 0x7fc7ac0b6ca0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:39.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 -- 172.21.15.69:0/2804546708 >> 172.21.15.69:0/2804546708 conn(0x7fc7ac01a0f0 msgr2=0x7fc7ac0b7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 -- 172.21.15.69:0/2804546708 shutdown_connections 2024-08-02T17:47:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:39.951+0000 7fc7b9182640 1 -- 172.21.15.69:0/2804546708 wait complete. 2024-08-02T17:47:39.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 30.0 s, 3258.3 tps, lat 149.899 ms stddev 1571.529 2024-08-02T17:47:40.627 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2319cd26', quiesce took 4.5 seconds. Will release after: 17.45 2024-08-02T17:47:41.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:41 smithi077 ceph-mon[37108]: pgmap v41: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 311 op/s 2024-08-02T17:47:41.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:41 smithi069 ceph-mon[32746]: pgmap v41: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 311 op/s 2024-08-02T17:47:41.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:41 smithi083 ceph-mon[37031]: pgmap v41: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 27 MiB/s wr, 311 op/s 2024-08-02T17:47:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.136+0000 7fe9e5924640 1 -- 172.21.15.69:0/316544396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 msgr2=0x7fe9e010e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.136+0000 7fe9e5924640 1 --2- 172.21.15.69:0/316544396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 0x7fe9e010e6c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8009cd0 tx=0x7fe9d802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.136+0000 7fe9e5924640 1 -- 172.21.15.69:0/316544396 shutdown_connections 2024-08-02T17:47:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.136+0000 7fe9e5924640 1 --2- 172.21.15.69:0/316544396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 0x7fe9e010e6c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.136+0000 7fe9e5924640 1 --2- 172.21.15.69:0/316544396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9e01017f0 0x7fe9e0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.136+0000 7fe9e5924640 1 --2- 172.21.15.69:0/316544396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e0107800 0x7fe9e0107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.136+0000 7fe9e5924640 1 -- 172.21.15.69:0/316544396 >> 172.21.15.69:0/316544396 conn(0x7fe9e00fd4d0 msgr2=0x7fe9e00ff8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.136+0000 7fe9e5924640 1 -- 172.21.15.69:0/316544396 shutdown_connections 2024-08-02T17:47:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.137+0000 7fe9e5924640 1 -- 172.21.15.69:0/316544396 wait complete. 2024-08-02T17:47:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.137+0000 7fe9e5924640 1 Processor -- start 2024-08-02T17:47:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.137+0000 7fe9e5924640 1 -- start start 2024-08-02T17:47:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e5924640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9e01017f0 0x7fe9e0074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e5924640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 0x7fe9e0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e5924640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e0107800 0x7fe9e0074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e5924640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e0111c40 con 0x7fe9e0102190 2024-08-02T17:47:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e5924640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e0111ac0 con 0x7fe9e0107800 2024-08-02T17:47:43.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e5924640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e0111dc0 con 0x7fe9e01017f0 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 0x7fe9e0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 0x7fe9e0074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49794/0 (socket says 172.21.15.69:49794) 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 -- 172.21.15.69:0/102115723 learned_addr learned my addr 172.21.15.69:0/102115723 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e0107800 0x7fe9e0074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9deffd640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9e01017f0 0x7fe9e0074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 -- 172.21.15.69:0/102115723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9e01017f0 msgr2=0x7fe9e0074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9e01017f0 0x7fe9e0074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 -- 172.21.15.69:0/102115723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e0107800 msgr2=0x7fe9e0074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e0107800 0x7fe9e0074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 -- 172.21.15.69:0/102115723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e01a6610 con 0x7fe9e0102190 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9deffd640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9e01017f0 0x7fe9e0074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 0x7fe9e0074760 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d950 tx=0x7fe9c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:43.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e4922640 1 -- 172.21.15.69:0/102115723 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c8017030 con 0x7fe9e0102190 2024-08-02T17:47:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e4922640 1 -- 172.21.15.69:0/102115723 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c8004020 con 0x7fe9e0102190 2024-08-02T17:47:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e4922640 1 -- 172.21.15.69:0/102115723 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c80047a0 con 0x7fe9e0102190 2024-08-02T17:47:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.138+0000 7fe9e5924640 1 -- 172.21.15.69:0/102115723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e01a6900 con 0x7fe9e0102190 2024-08-02T17:47:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.139+0000 7fe9e5924640 1 -- 172.21.15.69:0/102115723 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e01a6e00 con 0x7fe9e0102190 2024-08-02T17:47:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e4922640 1 -- 172.21.15.69:0/102115723 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7fe9c80041d0 con 0x7fe9e0102190 2024-08-02T17:47:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e867d740 1 Processor -- start 2024-08-02T17:47:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e867d740 1 -- start start 2024-08-02T17:47:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8363b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba836e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8370d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e867d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560ba82c79b0 con 0x560ba83637b0 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e867d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ba826f800 con 0x560ba83640d0 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e867d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560ba82c3970 con 0x560ba836e970 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9e4922640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9cc0a2ea0 0x7fe9cc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.141+0000 7fe9df7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8363b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8363b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49800/0 (socket says 172.21.15.69:49800) 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 -- 172.21.15.69:0/2938612697 learned_addr learned my addr 172.21.15.69:0/2938612697 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8370d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9cc0a2ea0 0x7fe9cc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9e4922640 1 -- 172.21.15.69:0/102115723 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe9c80cf4d0 con 0x7fe9e0102190 2024-08-02T17:47:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9deffd640 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba836e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 -- 172.21.15.69:0/2938612697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 msgr2=0x560ba8370d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8370d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 -- 172.21.15.69:0/2938612697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 msgr2=0x560ba836e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba836e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 -- 172.21.15.69:0/2938612697 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba7fdb210 con 0x560ba83637b0 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8363b90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005fe0 tx=0x7fe9d80317d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4871db473420086e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9d5ffb640 1 -- 172.21.15.69:0/2938612697 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8003d40 con 0x560ba83637b0 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9d5ffb640 1 -- 172.21.15.69:0/2938612697 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d804e070 con 0x560ba83637b0 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9d5ffb640 1 -- 172.21.15.69:0/2938612697 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d8030920 con 0x560ba83637b0 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.142+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9cc0a2ea0 0x7fe9cc0a5360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9d804d000 tx=0x7fe9d804b000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 -- 172.21.15.69:0/2938612697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 msgr2=0x560ba8363b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8363b90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005fe0 tx=0x7fe9d80317d0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 -- 172.21.15.69:0/2938612697 shutdown_connections 2024-08-02T17:47:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8370d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba836e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2938612697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8363b90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 -- 172.21.15.69:0/2938612697 >> 172.21.15.69:0/2938612697 conn(0x560ba835fd20 msgr2=0x560ba8360100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 -- 172.21.15.69:0/2938612697 shutdown_connections 2024-08-02T17:47:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 -- 172.21.15.69:0/2938612697 wait complete. 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.143+0000 7fe9e867d740 1 -- start start 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9e867d740 1 Processor -- start 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba836b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba836bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba836e970 0x560ba836c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9e867d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560ba8371540 con 0x560ba83637b0 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9e867d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ba8283500 con 0x560ba836e970 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9e867d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560ba83716c0 con 0x560ba83640d0 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9e867d740 10 client.0 _ll_register_callbacks cb 0x560ba7fdaa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba836b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba836b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49802/0 (socket says 172.21.15.69:49802) 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 -- 172.21.15.69:0/2554364421 learned_addr learned my addr 172.21.15.69:0/2554364421 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba836e970 0x560ba836c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba836bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 -- 172.21.15.69:0/2554364421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 msgr2=0x560ba836bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba836bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 -- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba836e970 msgr2=0x560ba836c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba836e970 0x560ba836c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 -- 172.21.15.69:0/2554364421 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba80a7620 con 0x560ba83637b0 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba836bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba836e970 0x560ba836c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9deffd640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba836b840 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009ed0 tx=0x7fe9d00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd124bf5442c08b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.144+0000 7fe9beffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9beffd640 1 -- 172.21.15.69:0/2554364421 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0012860 con 0x560ba83637b0 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9beffd640 1 -- 172.21.15.69:0/2554364421 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe9d0012a00 con 0x560ba83637b0 2024-08-02T17:47:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 1 -- 172.21.15.69:0/2554364421 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d804e070 con 0x560ba83637b0 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9beffd640 1 -- 172.21.15.69:0/2554364421 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9d0025050 con 0x560ba83637b0 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 10 client.15768 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 1 -- 172.21.15.69:0/2554364421 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ba837d1e0 con 0x560ba83637b0 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9beffd640 1 -- 172.21.15.69:0/2554364421 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d804e070 con 0x560ba83637b0 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9beffd640 1 -- 172.21.15.69:0/2554364421 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe9d000f040 con 0x560ba83637b0 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 10 client.15768 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 20 client.15768 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560ba837e6a0 0x560ba8380af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 4 client.15768 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.145+0000 7fe9e867d740 1 -- 172.21.15.69:0/2554364421 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ba82d50e0 con 0x560ba837e6a0 2024-08-02T17:47:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.146+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560ba837e6a0 0x560ba8380af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.146+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560ba837e6a0 0x560ba8380af0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3023a07fa56e5808 server_cookie=621e64baea563a4c in_seq=0 out_seq=0 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.146+0000 7fe9beffd640 10 client.15768 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9beffd640 1 -- 172.21.15.69:0/2554364421 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560ba82d50e0 con 0x560ba837e6a0 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9beffd640 10 client.15768 handle_command_reply: tid=0 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 client.15768 shutdown 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 -- 172.21.15.69:0/2554364421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 msgr2=0x560ba836b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba836b840 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009ed0 tx=0x7fe9d00117b0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 -- 172.21.15.69:0/2554364421 shutdown_connections 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560ba837e6a0 0x560ba8380af0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba836e970 0x560ba836c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba836bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9beffd640 0 client.15768 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 --2- 172.21.15.69:0/2554364421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba836b840 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 -- 172.21.15.69:0/2554364421 >> 172.21.15.69:0/2554364421 conn(0x560ba835fd20 msgr2=0x560ba8365a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 -- 172.21.15.69:0/2554364421 shutdown_connections 2024-08-02T17:47:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 1 -- 172.21.15.69:0/2554364421 wait complete. 2024-08-02T17:47:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.147+0000 7fe9e867d740 20 client.15768 trim_cache size 0 max 16384 2024-08-02T17:47:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.162+0000 7fe9e867d740 1 Processor -- start 2024-08-02T17:47:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.162+0000 7fe9e867d740 1 -- start start 2024-08-02T17:47:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.162+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8366220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.162+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba8386450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8386990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.168 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9e867d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e01b38a0 con 0x560ba83637b0 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9e867d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e01b1170 con 0x560ba83640d0 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9e867d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e01aefc0 con 0x560ba836e970 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8366220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8366220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49812/0 (socket says 172.21.15.69:49812) 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 -- 172.21.15.69:0/3768318388 learned_addr learned my addr 172.21.15.69:0/3768318388 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba8386450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8386990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 -- 172.21.15.69:0/3768318388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 msgr2=0x560ba8386990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8386990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 -- 172.21.15.69:0/3768318388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 msgr2=0x560ba8386450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba8386450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 -- 172.21.15.69:0/3768318388 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba8360dd0 con 0x560ba83637b0 2024-08-02T17:47:43.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8386990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba8386450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9deffd640 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8366220 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0000c00 tx=0x7fe9d0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a945840600a9f68e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9d57fa640 1 -- 172.21.15.69:0/3768318388 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d002d070 con 0x560ba83637b0 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9d57fa640 1 -- 172.21.15.69:0/3768318388 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d0011cb0 con 0x560ba83637b0 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.163+0000 7fe9d57fa640 1 -- 172.21.15.69:0/3768318388 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00128c0 con 0x560ba83637b0 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 -- 172.21.15.69:0/3768318388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 msgr2=0x560ba8366220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8366220 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0000c00 tx=0x7fe9d0005e50 comp rx=0 tx=0).stop 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 -- 172.21.15.69:0/3768318388 shutdown_connections 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba836e970 0x560ba8386990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83640d0 0x560ba8386450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 --2- 172.21.15.69:0/3768318388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba83637b0 0x560ba8366220 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 -- 172.21.15.69:0/3768318388 >> 172.21.15.69:0/3768318388 conn(0x560ba835fd20 msgr2=0x560ba836a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:43.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 -- 172.21.15.69:0/3768318388 shutdown_connections 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 -- 172.21.15.69:0/3768318388 wait complete. 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 -- start start 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.164+0000 7fe9e867d740 1 Processor -- start 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83637b0 0x560ba8377070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba8368500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9e867d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba836e970 0x560ba8368a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9e867d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e01af630 con 0x560ba836e970 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9e867d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e01b0aa0 con 0x560ba83637b0 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9e867d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e01afd00 con 0x560ba83640d0 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9e867d740 10 client.0 _ll_register_callbacks cb 0x7fe9d0009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba836e970 0x560ba8368a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba836e970 0x560ba8368a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49816/0 (socket says 172.21.15.69:49816) 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 -- 172.21.15.69:0/305275566 learned_addr learned my addr 172.21.15.69:0/305275566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9deffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83637b0 0x560ba8377070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9d7fff640 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba8368500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 -- 172.21.15.69:0/305275566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 msgr2=0x560ba8368500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba8368500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 -- 172.21.15.69:0/305275566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83637b0 msgr2=0x560ba8377070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83637b0 0x560ba8377070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 -- 172.21.15.69:0/305275566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba8369240 con 0x560ba836e970 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9df7fe640 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba836e970 0x560ba8368a40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005e30 tx=0x7fe9d8031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4244219e08edbf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9beffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9beffd640 1 -- 172.21.15.69:0/305275566 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d805f060 con 0x560ba836e970 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9beffd640 1 -- 172.21.15.69:0/305275566 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe9d802e040 con 0x560ba836e970 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.165+0000 7fe9e867d740 1 -- 172.21.15.69:0/305275566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9d0011cb0 con 0x560ba836e970 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9beffd640 1 -- 172.21.15.69:0/305275566 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe9d8056070 con 0x560ba836e970 2024-08-02T17:47:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9e867d740 10 client.15774 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9e867d740 1 -- 172.21.15.69:0/305275566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ba83e9400 con 0x560ba836e970 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9beffd640 1 -- 172.21.15.69:0/305275566 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9d0011cb0 con 0x560ba836e970 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9beffd640 1 -- 172.21.15.69:0/305275566 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe9d8030070 con 0x560ba836e970 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9e867d740 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9e867d740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9e867d740 20 client.15774 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9e867d740 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560ba83e9680 0x560ba83eba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9e867d740 4 client.15774 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9e867d740 1 -- 172.21.15.69:0/305275566 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ba82cf430 con 0x560ba83e9680 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.166+0000 7fe9deffd640 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560ba83e9680 0x560ba83eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.167+0000 7fe9beffd640 10 client.15774 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.167+0000 7fe9deffd640 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560ba83e9680 0x560ba83eba70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60bf1415beb5b6c server_cookie=44b0f7b469ea0316 in_seq=0 out_seq=0 2024-08-02T17:47:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9beffd640 1 -- 172.21.15.69:0/305275566 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ba82cf430 con 0x560ba83e9680 2024-08-02T17:47:43.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9beffd640 10 client.15774 handle_command_reply: tid=0 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 client.15774 shutdown 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 -- 172.21.15.69:0/305275566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba836e970 msgr2=0x560ba8368a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba836e970 0x560ba8368a40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8005e30 tx=0x7fe9d8031940 comp rx=0 tx=0).stop 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 -- 172.21.15.69:0/305275566 shutdown_connections 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ba836e970 0x560ba8368a40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560ba83e9680 0x560ba83eba70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ba83640d0 0x560ba8368500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 --2- 172.21.15.69:0/305275566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ba83637b0 0x560ba8377070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9beffd640 0 client.15774 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.187+0000 7fe9e867d740 1 -- 172.21.15.69:0/305275566 >> 172.21.15.69:0/305275566 conn(0x560ba835fd20 msgr2=0x560ba835e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e867d740 1 -- 172.21.15.69:0/305275566 shutdown_connections 2024-08-02T17:47:43.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e867d740 1 -- 172.21.15.69:0/305275566 wait complete. 2024-08-02T17:47:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e867d740 20 client.15774 trim_cache size 0 max 16384 2024-08-02T17:47:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:47:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:47:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:47:43.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:47:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 -- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9cc0a2ea0 msgr2=0x7fe9cc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9cc0a2ea0 0x7fe9cc0a5360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe9d804d000 tx=0x7fe9d804b000 comp rx=0 tx=0).stop 2024-08-02T17:47:43.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 -- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 msgr2=0x7fe9e0074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 0x7fe9e0074760 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800d950 tx=0x7fe9c800de10 comp rx=0 tx=0).stop 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 -- 172.21.15.69:0/102115723 shutdown_connections 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe9e0102190 0x7fe9e0074760 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9cc0a2ea0 0x7fe9cc0a5360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe9e0107800 0x7fe9e0074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 --2- 172.21.15.69:0/102115723 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9e01017f0 0x7fe9e0074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 -- 172.21.15.69:0/102115723 >> 172.21.15.69:0/102115723 conn(0x7fe9e00fd4d0 msgr2=0x7fe9e00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 -- 172.21.15.69:0/102115723 shutdown_connections 2024-08-02T17:47:43.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:43.188+0000 7fe9e5924640 1 -- 172.21.15.69:0/102115723 wait complete. 2024-08-02T17:47:43.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:43 smithi077 ceph-mon[37108]: pgmap v42: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-08-02T17:47:43.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:43 smithi069 ceph-mon[32746]: pgmap v42: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-08-02T17:47:43.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:43 smithi083 ceph-mon[37031]: pgmap v42: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-08-02T17:47:44.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 35.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:47:45.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:45 smithi077 ceph-mon[37108]: pgmap v43: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-08-02T17:47:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:45 smithi069 ceph-mon[32746]: pgmap v43: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-08-02T17:47:45.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:45 smithi083 ceph-mon[37031]: pgmap v43: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 274 op/s 2024-08-02T17:47:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:47 smithi077 ceph-mon[37108]: pgmap v44: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2024-08-02T17:47:47.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:47 smithi069 ceph-mon[32746]: pgmap v44: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2024-08-02T17:47:47.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:47 smithi083 ceph-mon[37031]: pgmap v44: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.9 MiB/s wr, 283 op/s 2024-08-02T17:47:49.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:49 smithi077 ceph-mon[37108]: pgmap v45: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-08-02T17:47:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:49 smithi069 ceph-mon[32746]: pgmap v45: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-08-02T17:47:49.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:49 smithi083 ceph-mon[37031]: pgmap v45: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.6 MiB/s wr, 260 op/s 2024-08-02T17:47:49.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 40.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:47:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:51 smithi077 ceph-mon[37108]: pgmap v46: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-08-02T17:47:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:51 smithi069 ceph-mon[32746]: pgmap v46: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-08-02T17:47:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:51 smithi083 ceph-mon[37031]: pgmap v46: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-08-02T17:47:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:53 smithi077 ceph-mon[37108]: pgmap v47: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-08-02T17:47:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:53 smithi069 ceph-mon[32746]: pgmap v47: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-08-02T17:47:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:53 smithi083 ceph-mon[37031]: pgmap v47: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-08-02T17:47:54.627 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:47:54.892 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:54.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 45.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:47:55.286 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:55 smithi069 ceph-mon[32746]: pgmap v48: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-08-02T17:47:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:55 smithi077 ceph-mon[37108]: pgmap v48: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-08-02T17:47:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 -- 172.21.15.69:0/2713236572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650102ac0 msgr2=0x7fb650102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 --2- 172.21.15.69:0/2713236572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650102ac0 0x7fb650102ea0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb640009930 tx=0x7fb64002a420 comp rx=0 tx=0).stop 2024-08-02T17:47:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 -- 172.21.15.69:0/2713236572 shutdown_connections 2024-08-02T17:47:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 --2- 172.21.15.69:0/2713236572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb650107fc0 0x7fb65010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 --2- 172.21.15.69:0/2713236572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb650103470 0x7fb6501078b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 --2- 172.21.15.69:0/2713236572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650102ac0 0x7fb650102ea0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 -- 172.21.15.69:0/2713236572 >> 172.21.15.69:0/2713236572 conn(0x7fb6500fc500 msgr2=0x7fb6500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 -- 172.21.15.69:0/2713236572 shutdown_connections 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.369+0000 7fb655d0a640 1 -- 172.21.15.69:0/2713236572 wait complete. 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.370+0000 7fb655d0a640 1 Processor -- start 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.370+0000 7fb655d0a640 1 -- start start 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.370+0000 7fb655d0a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb650102ac0 0x7fb650198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.370+0000 7fb655d0a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650103470 0x7fb650198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.370+0000 7fb655d0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb650107fc0 0x7fb65019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.370+0000 7fb655d0a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb650111dc0 con 0x7fb650103470 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.370+0000 7fb655d0a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb650111c40 con 0x7fb650102ac0 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.370+0000 7fb655d0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb650111f40 con 0x7fb650107fc0 2024-08-02T17:47:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650103470 0x7fb650198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650103470 0x7fb650198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51338/0 (socket says 172.21.15.69:51338) 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 -- 172.21.15.69:0/1672965399 learned_addr learned my addr 172.21.15.69:0/1672965399 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64ffff640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb650107fc0 0x7fb65019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb650102ac0 0x7fb650198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 -- 172.21.15.69:0/1672965399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb650107fc0 msgr2=0x7fb65019cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb650107fc0 0x7fb65019cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 -- 172.21.15.69:0/1672965399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb650102ac0 msgr2=0x7fb650198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb650102ac0 0x7fb650198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 -- 172.21.15.69:0/1672965399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65019d1f0 con 0x7fb650103470 2024-08-02T17:47:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64effd640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650103470 0x7fb650198780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c970 tx=0x7fb64400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64cff9640 1 -- 172.21.15.69:0/1672965399 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644007bb0 con 0x7fb650103470 2024-08-02T17:47:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb655d0a640 1 -- 172.21.15.69:0/1672965399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65019d4e0 con 0x7fb650103470 2024-08-02T17:47:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64cff9640 1 -- 172.21.15.69:0/1672965399 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb644007d50 con 0x7fb650103470 2024-08-02T17:47:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb64cff9640 1 -- 172.21.15.69:0/1672965399 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64400f450 con 0x7fb650103470 2024-08-02T17:47:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.371+0000 7fb655d0a640 1 -- 172.21.15.69:0/1672965399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6501a4dc0 con 0x7fb650103470 2024-08-02T17:47:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.374+0000 7fb64cff9640 1 -- 172.21.15.69:0/1672965399 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7fb644004f90 con 0x7fb650103470 2024-08-02T17:47:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb658108740 1 Processor -- start 2024-08-02T17:47:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb658108740 1 -- start start 2024-08-02T17:47:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb658108740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 0x5573a2f725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb658108740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb658108740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f7d3f0 0x5573a2f7f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb658108740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5573a2ed7980 con 0x5573a2f72b30 2024-08-02T17:47:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb658108740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5573a2e6fa70 con 0x5573a2f72210 2024-08-02T17:47:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb658108740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5573a2ed3940 con 0x5573a2f7d3f0 2024-08-02T17:47:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51340/0 (socket says 172.21.15.69:51340) 2024-08-02T17:47:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64f7fe640 1 -- 172.21.15.69:0/3019383414 learned_addr learned my addr 172.21.15.69:0/3019383414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64cff9640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb6380a2ea0 0x7fb6380a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64ffff640 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 0x5573a2f725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64effd640 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f7d3f0 0x5573a2f7f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64f7fe640 1 -- 172.21.15.69:0/3019383414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f7d3f0 msgr2=0x5573a2f7f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64ffff640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb6380a2ea0 0x7fb6380a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.375+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f7d3f0 0x5573a2f7f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb64f7fe640 1 -- 172.21.15.69:0/3019383414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 msgr2=0x5573a2f725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:55.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 0x5573a2f725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb64f7fe640 1 -- 172.21.15.69:0/3019383414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a2ed26e0 con 0x5573a2f72b30 2024-08-02T17:47:55.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb64cff9640 1 -- 172.21.15.69:0/1672965399 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb6440cf780 con 0x7fb650103470 2024-08-02T17:47:55.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7ceb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb64002a940 tx=0x7fb6400316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cec1380ea3a6eac server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:55.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb64ffff640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb6380a2ea0 0x7fb6380a5360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb648014610 tx=0x7fb648004ab0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:55.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb635ffb640 1 -- 172.21.15.69:0/3019383414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640040070 con 0x5573a2f72b30 2024-08-02T17:47:55.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb635ffb640 1 -- 172.21.15.69:0/3019383414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb640005020 con 0x5573a2f72b30 2024-08-02T17:47:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.376+0000 7fb635ffb640 1 -- 172.21.15.69:0/3019383414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640035040 con 0x5573a2f72b30 2024-08-02T17:47:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 -- 172.21.15.69:0/3019383414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 msgr2=0x5573a2f7ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7ceb0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb64002a940 tx=0x7fb6400316e0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 -- 172.21.15.69:0/3019383414 shutdown_connections 2024-08-02T17:47:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7ceb0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f7d3f0 0x5573a2f7f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 --2- 172.21.15.69:0/3019383414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 0x5573a2f725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 -- 172.21.15.69:0/3019383414 >> 172.21.15.69:0/3019383414 conn(0x5573a2f6e7a0 msgr2=0x5573a2f6eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 -- 172.21.15.69:0/3019383414 shutdown_connections 2024-08-02T17:47:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 -- 172.21.15.69:0/3019383414 wait complete. 2024-08-02T17:47:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.377+0000 7fb658108740 1 -- start start 2024-08-02T17:47:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb658108740 1 Processor -- start 2024-08-02T17:47:55.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb658108740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72210 0x5573a2f7a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb658108740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb658108740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 0x5573a2f7ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb658108740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5573a2f7ffc0 con 0x5573a2f72b30 2024-08-02T17:47:55.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb658108740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5573a2e92aa0 con 0x5573a2f7d3f0 2024-08-02T17:47:55.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb658108740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5573a2f80140 con 0x5573a2f72210 2024-08-02T17:47:55.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb658108740 10 client.0 _ll_register_callbacks cb 0x5573a2e81e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:55.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51346/0 (socket says 172.21.15.69:51346) 2024-08-02T17:47:55.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 -- 172.21.15.69:0/3838272460 learned_addr learned my addr 172.21.15.69:0/3838272460 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:55.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72210 0x5573a2f7a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64ffff640 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 0x5573a2f7ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 -- 172.21.15.69:0/3838272460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72210 msgr2=0x5573a2f7a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:55.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72210 0x5573a2f7a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 -- 172.21.15.69:0/3838272460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 msgr2=0x5573a2f7ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:55.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 0x5573a2f7ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 -- 172.21.15.69:0/3838272460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a2e76650 con 0x5573a2f72b30 2024-08-02T17:47:55.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb64effd640 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7a7e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb644098120 tx=0x7fb64409b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b1e6c20c85586e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:55.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.378+0000 7fb62effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:55.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb62effd640 1 -- 172.21.15.69:0/3838272460 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64401d040 con 0x5573a2f72b30 2024-08-02T17:47:55.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb62effd640 1 -- 172.21.15.69:0/3838272460 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb644097dc0 con 0x5573a2f72b30 2024-08-02T17:47:55.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 1 -- 172.21.15.69:0/3838272460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb640005020 con 0x5573a2f72b30 2024-08-02T17:47:55.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb62effd640 1 -- 172.21.15.69:0/3838272460 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb6440046c0 con 0x5573a2f72b30 2024-08-02T17:47:55.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 10 client.15786 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:55.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 1 -- 172.21.15.69:0/3838272460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5573a2c493c0 con 0x5573a2f72b30 2024-08-02T17:47:55.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb62effd640 1 -- 172.21.15.69:0/3838272460 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb640005020 con 0x5573a2f72b30 2024-08-02T17:47:55.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb62effd640 1 -- 172.21.15.69:0/3838272460 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb6440cfbb0 con 0x5573a2f72b30 2024-08-02T17:47:55.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 10 client.15786 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:55.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:55.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 20 client.15786 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:55.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5573a2f8cdd0 0x5573a2f8f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 4 client.15786 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.379+0000 7fb658108740 1 -- 172.21.15.69:0/3838272460 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573a2ee4ef0 con 0x5573a2f8cdd0 2024-08-02T17:47:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.380+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5573a2f8cdd0 0x5573a2f8f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.380+0000 7fb62effd640 10 client.15786 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.380+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5573a2f8cdd0 0x5573a2f8f1c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=937c8e925ef890ad server_cookie=86351e8a79fe08be in_seq=0 out_seq=0 2024-08-02T17:47:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb62effd640 1 -- 172.21.15.69:0/3838272460 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5573a2ee4ef0 con 0x5573a2f8cdd0 2024-08-02T17:47:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb62effd640 10 client.15786 handle_command_reply: tid=0 2024-08-02T17:47:55.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 client.15786 shutdown 2024-08-02T17:47:55.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 -- 172.21.15.69:0/3838272460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 msgr2=0x5573a2f7a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7a7e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb644098120 tx=0x7fb64409b520 comp rx=0 tx=0).stop 2024-08-02T17:47:55.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 -- 172.21.15.69:0/3838272460 shutdown_connections 2024-08-02T17:47:55.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72b30 0x5573a2f7a7e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5573a2f8cdd0 0x5573a2f8f1c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 0x5573a2f7ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 --2- 172.21.15.69:0/3838272460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72210 0x5573a2f7a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb62effd640 0 client.15786 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:55.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 -- 172.21.15.69:0/3838272460 >> 172.21.15.69:0/3838272460 conn(0x5573a2f6e7a0 msgr2=0x5573a2f744d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:55.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 -- 172.21.15.69:0/3838272460 shutdown_connections 2024-08-02T17:47:55.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 1 -- 172.21.15.69:0/3838272460 wait complete. 2024-08-02T17:47:55.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.381+0000 7fb658108740 20 client.15786 trim_cache size 0 max 16384 2024-08-02T17:47:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.398+0000 7fb658108740 1 Processor -- start 2024-08-02T17:47:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.398+0000 7fb658108740 1 -- start start 2024-08-02T17:47:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb658108740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72210 0x5573a2f7c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb658108740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 0x5573a2f94a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb658108740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 0x5573a2f94fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb658108740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501b1680 con 0x5573a2f72210 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb658108740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501aef50 con 0x5573a2f7d3f0 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb658108740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501acda0 con 0x5573a2f72b30 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72210 0x5573a2f7c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72210 0x5573a2f7c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51362/0 (socket says 172.21.15.69:51362) 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 -- 172.21.15.69:0/3170851850 learned_addr learned my addr 172.21.15.69:0/3170851850 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64ffff640 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 0x5573a2f94fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64effd640 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 0x5573a2f94a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 -- 172.21.15.69:0/3170851850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 msgr2=0x5573a2f94a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 0x5573a2f94a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 -- 172.21.15.69:0/3170851850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 msgr2=0x5573a2f94fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 0x5573a2f94fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 -- 172.21.15.69:0/3170851850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573a2f787f0 con 0x5573a2f72210 2024-08-02T17:47:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72210 0x5573a2f7c100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb640004810 tx=0x7fb640004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae656f785182410 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb6357fa640 1 -- 172.21.15.69:0/3170851850 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640040040 con 0x5573a2f72210 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb6357fa640 1 -- 172.21.15.69:0/3170851850 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64003c620 con 0x5573a2f72210 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.399+0000 7fb6357fa640 1 -- 172.21.15.69:0/3170851850 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640035040 con 0x5573a2f72210 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 -- 172.21.15.69:0/3170851850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72210 msgr2=0x5573a2f7c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72210 0x5573a2f7c100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb640004810 tx=0x7fb640004840 comp rx=0 tx=0).stop 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 -- 172.21.15.69:0/3170851850 shutdown_connections 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f7d3f0 0x5573a2f94fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 0x5573a2f94a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 --2- 172.21.15.69:0/3170851850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f72210 0x5573a2f7c100 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 -- 172.21.15.69:0/3170851850 >> 172.21.15.69:0/3170851850 conn(0x5573a2f6e7a0 msgr2=0x5573a2f7b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 -- 172.21.15.69:0/3170851850 shutdown_connections 2024-08-02T17:47:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.400+0000 7fb658108740 1 -- 172.21.15.69:0/3170851850 wait complete. 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 1 -- start start 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 1 Processor -- start 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 0x5573a2f85c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 0x5573a2f80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f7d3f0 0x5573a2f81520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501ad410 con 0x5573a2f7d3f0 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501ae880 con 0x5573a2f72210 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501adae0 con 0x5573a2f72b30 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb658108740 10 client.0 _ll_register_callbacks cb 0x7fb64400c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f7d3f0 0x5573a2f81520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb64ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f7d3f0 0x5573a2f81520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51376/0 (socket says 172.21.15.69:51376) 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb64ffff640 1 -- 172.21.15.69:0/993704924 learned_addr learned my addr 172.21.15.69:0/993704924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.401+0000 7fb64effd640 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 0x5573a2f80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 0x5573a2f85c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb64ffff640 1 -- 172.21.15.69:0/993704924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 msgr2=0x5573a2f80f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb64ffff640 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 0x5573a2f80f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb64ffff640 1 -- 172.21.15.69:0/993704924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 msgr2=0x5573a2f85c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb64ffff640 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 0x5573a2f85c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb64ffff640 1 -- 172.21.15.69:0/993704924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573a2f81d50 con 0x5573a2f7d3f0 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb64ffff640 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f7d3f0 0x5573a2f81520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb648011ea0 tx=0x7fb648004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3495bdaad3580dab server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb62effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb62effd640 1 -- 172.21.15.69:0/993704924 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64802d060 con 0x5573a2f7d3f0 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb62effd640 1 -- 172.21.15.69:0/993704924 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb64801fc70 con 0x5573a2f7d3f0 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb658108740 1 -- 172.21.15.69:0/993704924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64003c620 con 0x5573a2f7d3f0 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb62effd640 1 -- 172.21.15.69:0/993704924 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb648034050 con 0x5573a2f7d3f0 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb658108740 10 client.15792 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:55.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.402+0000 7fb658108740 1 -- 172.21.15.69:0/993704924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5573a2ff7be0 con 0x5573a2f7d3f0 2024-08-02T17:47:55.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb62effd640 1 -- 172.21.15.69:0/993704924 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb64003c620 con 0x5573a2f7d3f0 2024-08-02T17:47:55.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb62effd640 1 -- 172.21.15.69:0/993704924 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb648027070 con 0x5573a2f7d3f0 2024-08-02T17:47:55.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb658108740 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:55.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb658108740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:55.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb658108740 20 client.15792 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:55.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb658108740 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5573a2ff8360 0x5573a2ffa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:55.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb658108740 4 client.15792 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:47:55.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb658108740 1 -- 172.21.15.69:0/993704924 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573a2edf400 con 0x5573a2ff8360 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.403+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5573a2ff8360 0x5573a2ffa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb64f7fe640 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5573a2ff8360 0x5573a2ffa780 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f61170da0dcdfb8e server_cookie=b078ba704a50383a in_seq=0 out_seq=0 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb62effd640 10 client.15792 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb62effd640 1 -- 172.21.15.69:0/993704924 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5573a2edf400 con 0x5573a2ff8360 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb62effd640 10 client.15792 handle_command_reply: tid=0 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 client.15792 shutdown 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 -- 172.21.15.69:0/993704924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f7d3f0 msgr2=0x5573a2f81520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f7d3f0 0x5573a2f81520 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb648011ea0 tx=0x7fb648004a40 comp rx=0 tx=0).stop 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 -- 172.21.15.69:0/993704924 shutdown_connections 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5573a2f7d3f0 0x5573a2f81520 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5573a2ff8360 0x5573a2ffa780 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5573a2f72b30 0x5573a2f80f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 --2- 172.21.15.69:0/993704924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5573a2f72210 0x5573a2f85c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 -- 172.21.15.69:0/993704924 >> 172.21.15.69:0/993704924 conn(0x5573a2f6e7a0 msgr2=0x5573a2f7aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:55.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb62effd640 0 client.15792 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:55.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 -- 172.21.15.69:0/993704924 shutdown_connections 2024-08-02T17:47:55.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 1 -- 172.21.15.69:0/993704924 wait complete. 2024-08-02T17:47:55.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.404+0000 7fb658108740 20 client.15792 trim_cache size 0 max 16384 2024-08-02T17:47:55.431 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:47:55.432 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:47:55.432 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:47:55.432 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 -- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb6380a2ea0 msgr2=0x7fb6380a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb6380a2ea0 0x7fb6380a5360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb648014610 tx=0x7fb648004ab0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 -- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650103470 msgr2=0x7fb650198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650103470 0x7fb650198780 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c970 tx=0x7fb64400ce30 comp rx=0 tx=0).stop 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 -- 172.21.15.69:0/1672965399 shutdown_connections 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb650103470 0x7fb650198780 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb6380a2ea0 0x7fb6380a5360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb650107fc0 0x7fb65019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 --2- 172.21.15.69:0/1672965399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb650102ac0 0x7fb650198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:55.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 -- 172.21.15.69:0/1672965399 >> 172.21.15.69:0/1672965399 conn(0x7fb6500fc500 msgr2=0x7fb650105a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:55.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 -- 172.21.15.69:0/1672965399 shutdown_connections 2024-08-02T17:47:55.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:55.405+0000 7fb655d0a640 1 -- 172.21.15.69:0/1672965399 wait complete. 2024-08-02T17:47:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:55 smithi083 ceph-mon[37031]: pgmap v48: 129 pgs: 129 active+clean; 983 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 9.3 KiB/s wr, 11 op/s 2024-08-02T17:47:56.118 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:47:56.118 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:47:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.277+0000 7f07c8487640 1 -- 172.21.15.69:0/2556831338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 msgr2=0x7f07c010b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.277+0000 7f07c8487640 1 --2- 172.21.15.69:0/2556831338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 0x7f07c010b890 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009900 tx=0x7f07b002a480 comp rx=0 tx=0).stop 2024-08-02T17:47:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 -- 172.21.15.69:0/2556831338 shutdown_connections 2024-08-02T17:47:56.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 --2- 172.21.15.69:0/2556831338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 0x7f07c010b890 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 --2- 172.21.15.69:0/2556831338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c010bdd0 0x7f07c010e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 --2- 172.21.15.69:0/2556831338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c0102a90 0x7f07c0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 -- 172.21.15.69:0/2556831338 >> 172.21.15.69:0/2556831338 conn(0x7f07c00776d0 msgr2=0x7f07c00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 -- 172.21.15.69:0/2556831338 shutdown_connections 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 -- 172.21.15.69:0/2556831338 wait complete. 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 Processor -- start 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 -- start start 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0102a90 0x7f07c0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 0x7f07c01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c010bdd0 0x7f07c019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0111c20 con 0x7f07c01033b0 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0111aa0 con 0x7f07c010bdd0 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.278+0000 7f07c8487640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0111da0 con 0x7f07c0102a90 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 0x7f07c01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c6223640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0102a90 0x7f07c0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 0x7f07c01985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51386/0 (socket says 172.21.15.69:51386) 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 -- 172.21.15.69:0/1320016706 learned_addr learned my addr 172.21.15.69:0/1320016706 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c6a24640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c010bdd0 0x7f07c019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 -- 172.21.15.69:0/1320016706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0102a90 msgr2=0x7f07c0198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0102a90 0x7f07c0198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 -- 172.21.15.69:0/1320016706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c010bdd0 msgr2=0x7f07c019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c010bdd0 0x7f07c019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 -- 172.21.15.69:0/1320016706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c019d070 con 0x7f07c01033b0 2024-08-02T17:47:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c5a22640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 0x7f07c01985c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f07b0004770 tx=0x7f07b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07b77fe640 1 -- 172.21.15.69:0/1320016706 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b003f070 con 0x7f07c01033b0 2024-08-02T17:47:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c8487640 1 -- 172.21.15.69:0/1320016706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c019d300 con 0x7f07c01033b0 2024-08-02T17:47:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07b77fe640 1 -- 172.21.15.69:0/1320016706 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b0031ca0 con 0x7f07c01033b0 2024-08-02T17:47:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07b77fe640 1 -- 172.21.15.69:0/1320016706 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0035030 con 0x7f07c01033b0 2024-08-02T17:47:56.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.279+0000 7f07c8487640 1 -- 172.21.15.69:0/1320016706 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c01a4bf0 con 0x7f07c01033b0 2024-08-02T17:47:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.282+0000 7f07b77fe640 1 -- 172.21.15.69:0/1320016706 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7f07b0042070 con 0x7f07c01033b0 2024-08-02T17:47:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.282+0000 7f07b77fe640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f07ac0a2ea0 0x7f07ac0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.283+0000 7f07c6223640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f07ac0a2ea0 0x7f07ac0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.283+0000 7f07b77fe640 1 -- 172.21.15.69:0/1320016706 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f07b00f0170 con 0x7f07c01033b0 2024-08-02T17:47:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.283+0000 7f07cb1d7740 1 Processor -- start 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.283+0000 7f07c6223640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f07ac0a2ea0 0x7f07ac0a5360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009bd0 tx=0x7f07bc005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.283+0000 7f07cb1d7740 1 -- start start 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 0x55c25814cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 0x55c258157430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c258159d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2580b09b0 con 0x55c258157970 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c258058800 con 0x55c25814d0d0 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c2580ac970 con 0x55c25814c7b0 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c258159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c258159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51390/0 (socket says 172.21.15.69:51390) 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 -- 172.21.15.69:0/3614934227 learned_addr learned my addr 172.21.15.69:0/3614934227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6223640 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 0x55c25814cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c5a22640 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 0x55c258157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 -- 172.21.15.69:0/3614934227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 msgr2=0x55c25814cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 0x55c25814cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 -- 172.21.15.69:0/3614934227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 msgr2=0x55c258157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 0x55c258157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 -- 172.21.15.69:0/3614934227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c257dc4210 con 0x55c258157970 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07c6a24640 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c258159d60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f07b8007ec0 tx=0x7f07b800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dfa129fa1f64e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07b4bfe640 1 -- 172.21.15.69:0/3614934227 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8010840 con 0x55c258157970 2024-08-02T17:47:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07b4bfe640 1 -- 172.21.15.69:0/3614934227 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b80183f0 con 0x55c258157970 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.284+0000 7f07b4bfe640 1 -- 172.21.15.69:0/3614934227 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8017600 con 0x55c258157970 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.285+0000 7f07cb1d7740 1 -- 172.21.15.69:0/3614934227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 msgr2=0x55c258159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.285+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c258159d60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f07b8007ec0 tx=0x7f07b800f790 comp rx=0 tx=0).stop 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.285+0000 7f07cb1d7740 1 -- 172.21.15.69:0/3614934227 shutdown_connections 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.285+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c258159d60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.285+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 0x55c258157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.285+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/3614934227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 0x55c25814cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.285+0000 7f07cb1d7740 1 -- 172.21.15.69:0/3614934227 >> 172.21.15.69:0/3614934227 conn(0x55c258148d20 msgr2=0x55c258149100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 -- 172.21.15.69:0/3614934227 shutdown_connections 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 -- 172.21.15.69:0/3614934227 wait complete. 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 -- start start 2024-08-02T17:47:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 Processor -- start 2024-08-02T17:47:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c258154810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c258154d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c258155290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c25815a540 con 0x55c25814d0d0 2024-08-02T17:47:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c25806c500 con 0x55c25814c7b0 2024-08-02T17:47:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.286+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c25815a6c0 con 0x55c258157970 2024-08-02T17:47:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07cb1d7740 10 client.0 _ll_register_callbacks cb 0x55c257dc3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c258154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c258154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51400/0 (socket says 172.21.15.69:51400) 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 -- 172.21.15.69:0/4283220214 learned_addr learned my addr 172.21.15.69:0/4283220214 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c6223640 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c258154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c6a24640 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c258155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 -- 172.21.15.69:0/4283220214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 msgr2=0x55c258155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c258155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 -- 172.21.15.69:0/4283220214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 msgr2=0x55c258154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c258154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 -- 172.21.15.69:0/4283220214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c257e90620 con 0x55c25814d0d0 2024-08-02T17:47:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c6a24640 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c258155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07c5a22640 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c258154d50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f07b0031550 tx=0x7f07b00bbaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bfa6d793c692ff9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07a9ffb640 1 -- 172.21.15.69:0/4283220214 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0049040 con 0x55c25814d0d0 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07a9ffb640 1 -- 172.21.15.69:0/4283220214 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f07b00b87b0 con 0x55c25814d0d0 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07cb1d7740 1 -- 172.21.15.69:0/4283220214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b80183f0 con 0x55c25814d0d0 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07a9ffb640 1 -- 172.21.15.69:0/4283220214 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07b00c0440 con 0x55c25814d0d0 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07cb1d7740 10 client.15804 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.287+0000 7f07cb1d7740 1 -- 172.21.15.69:0/4283220214 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c258166150 con 0x55c25814d0d0 2024-08-02T17:47:56.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07a9ffb640 1 -- 172.21.15.69:0/4283220214 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b80183f0 con 0x55c25814d0d0 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07a9ffb640 1 -- 172.21.15.69:0/4283220214 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f07b003b4f0 con 0x55c25814d0d0 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07cb1d7740 10 client.15804 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07cb1d7740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07cb1d7740 20 client.15804 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c258167600 0x55c258169a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07cb1d7740 4 client.15804 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07cb1d7740 1 -- 172.21.15.69:0/4283220214 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2580be0e0 con 0x55c258167600 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.288+0000 7f07c6223640 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c258167600 0x55c258169a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07a9ffb640 10 client.15804 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07c6223640 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c258167600 0x55c258169a40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70277e96213314a5 server_cookie=eae924e7b54ae483 in_seq=0 out_seq=0 2024-08-02T17:47:56.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07a9ffb640 1 -- 172.21.15.69:0/4283220214 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c2580be0e0 con 0x55c258167600 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07a9ffb640 10 client.15804 handle_command_reply: tid=0 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 client.15804 shutdown 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 -- 172.21.15.69:0/4283220214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 msgr2=0x55c258154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c258154d50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f07b0031550 tx=0x7f07b00bbaa0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 -- 172.21.15.69:0/4283220214 shutdown_connections 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c258154d50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c258167600 0x55c258169a40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c258155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/4283220214 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c258154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 -- 172.21.15.69:0/4283220214 >> 172.21.15.69:0/4283220214 conn(0x55c258148d20 msgr2=0x55c25814ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07a9ffb640 0 client.15804 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:56.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 -- 172.21.15.69:0/4283220214 shutdown_connections 2024-08-02T17:47:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.289+0000 7f07cb1d7740 1 -- 172.21.15.69:0/4283220214 wait complete. 2024-08-02T17:47:56.299 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.290+0000 7f07cb1d7740 20 client.15804 trim_cache size 0 max 16384 2024-08-02T17:47:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07cb1d7740 1 Processor -- start 2024-08-02T17:47:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07cb1d7740 1 -- start start 2024-08-02T17:47:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c25816f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c25816f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c25814e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01b1590 con 0x55c25814d0d0 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01aee60 con 0x55c25814c7b0 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01accb0 con 0x55c258157970 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c5a22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c25816f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c5a22640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c25816f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51416/0 (socket says 172.21.15.69:51416) 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c5a22640 1 -- 172.21.15.69:0/1213184824 learned_addr learned my addr 172.21.15.69:0/1213184824 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c6a24640 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c25814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c6223640 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c25816f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c5a22640 1 -- 172.21.15.69:0/1213184824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 msgr2=0x55c25814e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:56.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c5a22640 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c25814e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c5a22640 1 -- 172.21.15.69:0/1213184824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 msgr2=0x55c25816f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c5a22640 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c25816f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c5a22640 1 -- 172.21.15.69:0/1213184824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c258152d40 con 0x55c25814d0d0 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.305+0000 7f07c6223640 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c25816f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.306+0000 7f07c5a22640 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c25816f890 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f07b00f0410 tx=0x7f07b00bbbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d723ad0571e0137 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.306+0000 7f07abfff640 1 -- 172.21.15.69:0/1213184824 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0049070 con 0x55c25814d0d0 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.306+0000 7f07abfff640 1 -- 172.21.15.69:0/1213184824 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b00c4d00 con 0x55c25814d0d0 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.306+0000 7f07abfff640 1 -- 172.21.15.69:0/1213184824 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00b87b0 con 0x55c25814d0d0 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- 172.21.15.69:0/1213184824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 msgr2=0x55c25816f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c25816f890 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f07b00f0410 tx=0x7f07b00bbbd0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- 172.21.15.69:0/1213184824 shutdown_connections 2024-08-02T17:47:56.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c25814d0d0 0x55c25816f890 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c258157970 0x55c25814e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/1213184824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814c7b0 0x55c25816f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- 172.21.15.69:0/1213184824 >> 172.21.15.69:0/1213184824 conn(0x55c258148d20 msgr2=0x55c2581538e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- 172.21.15.69:0/1213184824 shutdown_connections 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- 172.21.15.69:0/1213184824 wait complete. 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- start start 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 Processor -- start 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 0x55c25815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 0x55c258151460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c2581519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01ad320 con 0x55c258157970 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01ae790 con 0x55c25814d0d0 2024-08-02T17:47:56.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01ad9f0 con 0x55c25814c7b0 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07cb1d7740 10 client.0 _ll_register_callbacks cb 0x7f07b0031e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07c6a24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c2581519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07c6a24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c2581519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51432/0 (socket says 172.21.15.69:51432) 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.307+0000 7f07c6a24640 1 -- 172.21.15.69:0/2715689478 learned_addr learned my addr 172.21.15.69:0/2715689478 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07c6223640 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 0x55c25815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07c5a22640 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 0x55c258151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07c6a24640 1 -- 172.21.15.69:0/2715689478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 msgr2=0x55c25815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07c6a24640 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 0x55c25815ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07c6a24640 1 -- 172.21.15.69:0/2715689478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 msgr2=0x55c258151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07c6a24640 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 0x55c258151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07c6a24640 1 -- 172.21.15.69:0/2715689478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2581521a0 con 0x55c258157970 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07c6a24640 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c2581519a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f07b8007ec0 tx=0x7f07b8007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3778a452c4f8c27 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:56.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07a9ffb640 1 -- 172.21.15.69:0/2715689478 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8027070 con 0x55c258157970 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07a9ffb640 1 -- 172.21.15.69:0/2715689478 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f07b8010a50 con 0x55c258157970 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07cb1d7740 1 -- 172.21.15.69:0/2715689478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b00c4d00 con 0x55c258157970 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07a9ffb640 1 -- 172.21.15.69:0/2715689478 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f07b8033050 con 0x55c258157970 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07cb1d7740 10 client.15810 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07cb1d7740 1 -- 172.21.15.69:0/2715689478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2581d21e0 con 0x55c258157970 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.308+0000 7f07a9ffb640 1 -- 172.21.15.69:0/2715689478 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b00c4d00 con 0x55c258157970 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07a9ffb640 1 -- 172.21.15.69:0/2715689478 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f07b8023440 con 0x55c258157970 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07cb1d7740 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07cb1d7740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07cb1d7740 20 client.15810 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2581d26a0 0x55c2581d4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07cb1d7740 4 client.15810 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:47:56.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07cb1d7740 1 -- 172.21.15.69:0/2715689478 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2580b8430 con 0x55c2581d26a0 2024-08-02T17:47:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07c6223640 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2581d26a0 0x55c2581d4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07c6223640 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2581d26a0 0x55c2581d4ac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feca695a3a6f9b26 server_cookie=3b5a7b20f52a3ab8 in_seq=0 out_seq=0 2024-08-02T17:47:56.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.309+0000 7f07a9ffb640 10 client.15810 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07a9ffb640 1 -- 172.21.15.69:0/2715689478 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2580b8430 con 0x55c2581d26a0 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07a9ffb640 10 client.15810 handle_command_reply: tid=0 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 client.15810 shutdown 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 -- 172.21.15.69:0/2715689478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 msgr2=0x55c2581519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c2581519a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f07b8007ec0 tx=0x7f07b8007dc0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 -- 172.21.15.69:0/2715689478 shutdown_connections 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c258157970 0x55c2581519a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2581d26a0 0x55c2581d4ac0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c25814d0d0 0x55c258151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 --2- 172.21.15.69:0/2715689478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c25814c7b0 0x55c25815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 -- 172.21.15.69:0/2715689478 >> 172.21.15.69:0/2715689478 conn(0x55c258148d20 msgr2=0x55c2581547d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07a9ffb640 0 client.15810 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 -- 172.21.15.69:0/2715689478 shutdown_connections 2024-08-02T17:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 1 -- 172.21.15.69:0/2715689478 wait complete. 2024-08-02T17:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.323+0000 7f07cb1d7740 20 client.15810 trim_cache size 0 max 16384 2024-08-02T17:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:47:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:47:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:47:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 -- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f07ac0a2ea0 msgr2=0x7f07ac0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f07ac0a2ea0 0x7f07ac0a5360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009bd0 tx=0x7f07bc005f90 comp rx=0 tx=0).stop 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 -- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 msgr2=0x7f07c01985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 0x7f07c01985c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f07b0004770 tx=0x7f07b0003c10 comp rx=0 tx=0).stop 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 -- 172.21.15.69:0/1320016706 shutdown_connections 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07c01033b0 0x7f07c01985c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f07ac0a2ea0 0x7f07ac0a5360 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07c010bdd0 0x7f07c019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 --2- 172.21.15.69:0/1320016706 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07c0102a90 0x7f07c0198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 -- 172.21.15.69:0/1320016706 >> 172.21.15.69:0/1320016706 conn(0x7f07c00776d0 msgr2=0x7f07c00fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 -- 172.21.15.69:0/1320016706 shutdown_connections 2024-08-02T17:47:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:47:56.324+0000 7f07c8487640 1 -- 172.21.15.69:0/1320016706 wait complete. 2024-08-02T17:47:56.388 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:56.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 -- 172.21.15.69:0/1205270137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 msgr2=0x7fdaa0102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/1205270137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 0x7fdaa0102eb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009900 tx=0x7fda9c02a480 comp rx=0 tx=0).stop 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 -- 172.21.15.69:0/1205270137 shutdown_connections 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/1205270137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaa010beb0 0x7fdaa010e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/1205270137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa01033f0 0x7fdaa010b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/1205270137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 0x7fdaa0102eb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 -- 172.21.15.69:0/1205270137 >> 172.21.15.69:0/1205270137 conn(0x7fdaa00fc4d0 msgr2=0x7fdaa00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 -- 172.21.15.69:0/1205270137 shutdown_connections 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.813+0000 7fdaa7c00640 1 -- 172.21.15.69:0/1205270137 wait complete. 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa7c00640 1 Processor -- start 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa7c00640 1 -- start start 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa7c00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 0x7fdaa01980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa7c00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaa01033f0 0x7fdaa0198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa7c00640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa010beb0 0x7fdaa019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa7c00640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0111c90 con 0x7fdaa0102ad0 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa7c00640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0111b10 con 0x7fdaa010beb0 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa7c00640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa0111e10 con 0x7fdaa01033f0 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa599c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 0x7fdaa01980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa599c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 0x7fdaa01980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51452/0 (socket says 172.21.15.69:51452) 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.814+0000 7fdaa599c640 1 -- 172.21.15.69:0/3101678324 learned_addr learned my addr 172.21.15.69:0/3101678324 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa619d640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa010beb0 0x7fdaa019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa519b640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaa01033f0 0x7fdaa0198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa599c640 1 -- 172.21.15.69:0/3101678324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaa01033f0 msgr2=0x7fdaa0198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa599c640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaa01033f0 0x7fdaa0198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa599c640 1 -- 172.21.15.69:0/3101678324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa010beb0 msgr2=0x7fdaa019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa599c640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa010beb0 0x7fdaa019c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:56.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa599c640 1 -- 172.21.15.69:0/3101678324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa019d0a0 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa519b640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaa01033f0 0x7fdaa0198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa599c640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 0x7fdaa01980f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005ab0 tx=0x7fda9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fda96ffd640 1 -- 172.21.15.69:0/3101678324 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c03f030 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa019d330 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fda96ffd640 1 -- 172.21.15.69:0/3101678324 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9c003ce0 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fda96ffd640 1 -- 172.21.15.69:0/3101678324 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c035070 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.815+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa01a4c70 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.816+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa006ad40 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.817+0000 7fda96ffd640 1 -- 172.21.15.69:0/3101678324 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 24) ==== 138117+0+0 (secure 0 0 0) 0x7fda9c042030 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.818+0000 7fda96ffd640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda8c0a2ea0 0x7fda8c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.818+0000 7fdaa519b640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda8c0a2ea0 0x7fda8c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.818+0000 7fda96ffd640 1 -- 172.21.15.69:0/3101678324 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fda9c0f05f0 con 0x7fdaa0102ad0 2024-08-02T17:47:56.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.819+0000 7fdaa519b640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda8c0a2ea0 0x7fda8c0a5360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdaa010d2d0 tx=0x7fda9000a5c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:56.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.821+0000 7fda96ffd640 1 -- 172.21.15.69:0/3101678324 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fda9c0bcc50 con 0x7fdaa0102ad0 2024-08-02T17:47:56.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:56.994+0000 7fda96ffd640 1 -- 172.21.15.69:0/3101678324 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fda9c003800 con 0x7fdaa0102ad0 2024-08-02T17:47:57.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.032+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaa0102eb0 con 0x7fdaa0102ad0 2024-08-02T17:47:57.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.033+0000 7fda96ffd640 1 -- 172.21.15.69:0/3101678324 <== mon.0 v2:172.21.15.69:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fda9c0c51c0 con 0x7fdaa0102ad0 2024-08-02T17:47:57.143 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:47:57.144 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda8c0a2ea0 msgr2=0x7fda8c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda8c0a2ea0 0x7fda8c0a5360 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdaa010d2d0 tx=0x7fda9000a5c0 comp rx=0 tx=0).stop 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 msgr2=0x7fdaa01980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 0x7fdaa01980f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fda9c005ab0 tx=0x7fda9c0042f0 comp rx=0 tx=0).stop 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 shutdown_connections 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda8c0a2ea0 0x7fda8c0a5360 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdaa010beb0 0x7fdaa019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:57.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdaa01033f0 0x7fdaa0198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:57.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 --2- 172.21.15.69:0/3101678324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdaa0102ad0 0x7fdaa01980f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:57.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 >> 172.21.15.69:0/3101678324 conn(0x7fdaa00fc4d0 msgr2=0x7fdaa00fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:57.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 shutdown_connections 2024-08-02T17:47:57.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:57.035+0000 7fdaa7c00640 1 -- 172.21.15.69:0/3101678324 wait complete. 2024-08-02T17:47:57.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:57 smithi077 ceph-mon[37108]: pgmap v49: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-08-02T17:47:57.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:57 smithi077 ceph-mon[37108]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-08-02T17:47:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:57 smithi069 ceph-mon[32746]: pgmap v49: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-08-02T17:47:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:57 smithi069 ceph-mon[32746]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-08-02T17:47:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:57 smithi083 ceph-mon[37031]: pgmap v49: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 51 KiB/s wr, 15 op/s 2024-08-02T17:47:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:57 smithi083 ceph-mon[37031]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-08-02T17:47:57.752 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:47:58.011 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:58.078 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:47:58.293 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:58 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3101678324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:58.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:58 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3101678324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:58.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 -- 172.21.15.69:0/807913284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c48101150 msgr2=0x7f3c4810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/807913284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c48101150 0x7f3c4810d680 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009cd0 tx=0x7f3c4002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 -- 172.21.15.69:0/807913284 shutdown_connections 2024-08-02T17:47:58.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/807913284 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c48101150 0x7f3c4810d680 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/807913284 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c481007b0 0x7f3c48100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/807913284 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c481067b0 0x7f3c48106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 -- 172.21.15.69:0/807913284 >> 172.21.15.69:0/807913284 conn(0x7f3c480fc4d0 msgr2=0x7f3c480fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:58.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 -- 172.21.15.69:0/807913284 shutdown_connections 2024-08-02T17:47:58.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.480+0000 7f3c4fec1640 1 -- 172.21.15.69:0/807913284 wait complete. 2024-08-02T17:47:58.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4fec1640 1 Processor -- start 2024-08-02T17:47:58.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4fec1640 1 -- start start 2024-08-02T17:47:58.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4fec1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c481007b0 0x7f3c4810e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4fec1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48101150 0x7f3c4810e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4fec1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c481067b0 0x7f3c4810eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4fec1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48111d50 con 0x7f3c481067b0 2024-08-02T17:47:58.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4fec1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48111bd0 con 0x7f3c48101150 2024-08-02T17:47:58.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4fec1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c48111ed0 con 0x7f3c481007b0 2024-08-02T17:47:58.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c481067b0 0x7f3c4810eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c481067b0 0x7f3c4810eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51470/0 (socket says 172.21.15.69:51470) 2024-08-02T17:47:58.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 -- 172.21.15.69:0/1468863513 learned_addr learned my addr 172.21.15.69:0/1468863513 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:58.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c481007b0 0x7f3c4810e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 -- 172.21.15.69:0/1468863513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c481007b0 msgr2=0x7f3c4810e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48101150 0x7f3c4810e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c481007b0 0x7f3c4810e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 -- 172.21.15.69:0/1468863513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48101150 msgr2=0x7f3c4810e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48101150 0x7f3c4810e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 -- 172.21.15.69:0/1468863513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c481a6790 con 0x7f3c481067b0 2024-08-02T17:47:58.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.481+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c481067b0 0x7f3c4810eb90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3c40006010 tx=0x7f3c40003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:58.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.482+0000 7f3c3affd640 1 -- 172.21.15.69:0/1468863513 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40004450 con 0x7f3c481067b0 2024-08-02T17:47:58.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.482+0000 7f3c4fec1640 1 -- 172.21.15.69:0/1468863513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c481a6a20 con 0x7f3c481067b0 2024-08-02T17:47:58.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.482+0000 7f3c3affd640 1 -- 172.21.15.69:0/1468863513 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c40003e10 con 0x7f3c481067b0 2024-08-02T17:47:58.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.482+0000 7f3c3affd640 1 -- 172.21.15.69:0/1468863513 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40046480 con 0x7f3c481067b0 2024-08-02T17:47:58.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.482+0000 7f3c4fec1640 1 -- 172.21.15.69:0/1468863513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c481a6fb0 con 0x7f3c481067b0 2024-08-02T17:47:58.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.484+0000 7f3c3affd640 1 -- 172.21.15.69:0/1468863513 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3c400045f0 con 0x7f3c481067b0 2024-08-02T17:47:58.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.484+0000 7f3c522bf740 1 Processor -- start 2024-08-02T17:47:58.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.484+0000 7f3c522bf740 1 -- start start 2024-08-02T17:47:58.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.484+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x5581642953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.484+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x55816429fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.484+0000 7f3c3affd640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c340a2cf0 0x7f3c340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x5581642a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c522bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5581641faa40 con 0x5581642a01d0 2024-08-02T17:47:58.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c522bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5581641929d0 con 0x558164295910 2024-08-02T17:47:58.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c522bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581641f6a00 con 0x558164294ff0 2024-08-02T17:47:58.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4e45e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x5581642a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4d45c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x5581642953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4d45c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x5581642953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:36704/0 (socket says 172.21.15.69:36704) 2024-08-02T17:47:58.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4d45c640 1 -- 172.21.15.69:0/3887050794 learned_addr learned my addr 172.21.15.69:0/3887050794 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:58.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c3affd640 1 -- 172.21.15.69:0/1468863513 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3c4002e030 con 0x7f3c481067b0 2024-08-02T17:47:58.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/3887050794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x55816429fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c340a2cf0 0x7f3c340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4e45e640 1 -- 172.21.15.69:0/3887050794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 msgr2=0x5581642953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:58.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/3887050794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x5581642953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4e45e640 1 -- 172.21.15.69:0/3887050794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 msgr2=0x55816429fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:58.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/3887050794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x55816429fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4e45e640 1 -- 172.21.15.69:0/3887050794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558164199650 con 0x5581642a01d0 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/3887050794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x5581642a25c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3c400ee550 tx=0x7f3c40043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bcbf5e327d694a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c340a2cf0 0x7f3c340a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400cfd0 tx=0x7f3c4400c470 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c33fff640 1 -- 172.21.15.69:0/3887050794 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400d0040 con 0x5581642a01d0 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c33fff640 1 -- 172.21.15.69:0/3887050794 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c40036070 con 0x5581642a01d0 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.485+0000 7f3c33fff640 1 -- 172.21.15.69:0/3887050794 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400a0150 con 0x5581642a01d0 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 -- 172.21.15.69:0/3887050794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 msgr2=0x5581642a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3887050794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x5581642a25c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3c400ee550 tx=0x7f3c40043000 comp rx=0 tx=0).stop 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 -- 172.21.15.69:0/3887050794 shutdown_connections 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3887050794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x5581642a25c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3887050794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x55816429fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3887050794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x5581642953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 -- 172.21.15.69:0/3887050794 >> 172.21.15.69:0/3887050794 conn(0x558164291580 msgr2=0x558164291960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:58.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 -- 172.21.15.69:0/3887050794 shutdown_connections 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.486+0000 7f3c522bf740 1 -- 172.21.15.69:0/3887050794 wait complete. 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 1 -- start start 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 1 Processor -- start 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x55816429d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164295910 0x55816429d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581642a01d0 0x55816429daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5581642a2b60 con 0x558164295910 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5581641b5b20 con 0x5581642a01d0 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581642a2ce0 con 0x558164294ff0 2024-08-02T17:47:58.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c522bf740 10 client.0 _ll_register_callbacks cb 0x5581641b3bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c4d45c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164295910 0x55816429d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c4d45c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164295910 0x55816429d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51480/0 (socket says 172.21.15.69:51480) 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c4d45c640 1 -- 172.21.15.69:0/3363836512 learned_addr learned my addr 172.21.15.69:0/3363836512 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.487+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x55816429d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581642a01d0 0x55816429daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c4d45c640 1 -- 172.21.15.69:0/3363836512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 msgr2=0x55816429d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x55816429d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c4d45c640 1 -- 172.21.15.69:0/3363836512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581642a01d0 msgr2=0x55816429daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581642a01d0 0x55816429daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c4d45c640 1 -- 172.21.15.69:0/3363836512 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558163f6c3c0 con 0x558164295910 2024-08-02T17:47:58.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581642a01d0 0x55816429daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164295910 0x55816429d5b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3c44017df0 tx=0x7f3c4400c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8130d74e564d769b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c317fa640 1 -- 172.21.15.69:0/3363836512 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44029060 con 0x558164295910 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c522bf740 1 -- 172.21.15.69:0/3363836512 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c40036070 con 0x558164295910 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c317fa640 1 -- 172.21.15.69:0/3363836512 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3c44024420 con 0x558164295910 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c317fa640 1 -- 172.21.15.69:0/3363836512 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c44030050 con 0x558164295910 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c522bf740 10 client.15828 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c522bf740 1 -- 172.21.15.69:0/3363836512 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581642ae6a0 con 0x558164295910 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.488+0000 7f3c317fa640 1 -- 172.21.15.69:0/3363836512 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c40036070 con 0x558164295910 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c317fa640 1 -- 172.21.15.69:0/3363836512 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c44016370 con 0x558164295910 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c522bf740 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c522bf740 10 client.15828 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:47:58.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c522bf740 20 client.15828 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5581642afb60 0x5581642b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c522bf740 4 client.15828 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c522bf740 1 -- 172.21.15.69:0/3363836512 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558164207fb0 con 0x5581642afb60 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5581642afb60 0x5581642b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5581642afb60 0x5581642b1fc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ff6e60cf5187f61 server_cookie=546077ffc14c0d26 in_seq=0 out_seq=0 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.489+0000 7f3c317fa640 10 client.15828 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c317fa640 1 -- 172.21.15.69:0/3363836512 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558164207fb0 con 0x5581642afb60 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c317fa640 10 client.15828 handle_command_reply: tid=0 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 client.15828 shutdown 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 -- 172.21.15.69:0/3363836512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164295910 msgr2=0x55816429d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164295910 0x55816429d5b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3c44017df0 tx=0x7f3c4400c170 comp rx=0 tx=0).stop 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 -- 172.21.15.69:0/3363836512 shutdown_connections 2024-08-02T17:47:58.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5581642afb60 0x5581642b1fc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164295910 0x55816429d5b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581642a01d0 0x55816429daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 --2- 172.21.15.69:0/3363836512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164294ff0 0x55816429d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c317fa640 0 client.15828 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:47:58.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 -- 172.21.15.69:0/3363836512 >> 172.21.15.69:0/3363836512 conn(0x558164291580 msgr2=0x5581642972b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:58.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.490+0000 7f3c522bf740 1 -- 172.21.15.69:0/3363836512 shutdown_connections 2024-08-02T17:47:58.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.491+0000 7f3c522bf740 1 -- 172.21.15.69:0/3363836512 wait complete. 2024-08-02T17:47:58.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.491+0000 7f3c522bf740 20 client.15828 trim_cache size 0 max 16384 2024-08-02T17:47:58.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c522bf740 1 Processor -- start 2024-08-02T17:47:58.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c522bf740 1 -- start start 2024-08-02T17:47:58.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164294ff0 0x5581642b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164295910 0x5581642b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x55816429b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c522bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b38e0 con 0x5581642a01d0 2024-08-02T17:47:58.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c522bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b11b0 con 0x558164294ff0 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c522bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481af000 con 0x558164295910 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4e45e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x55816429b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4e45e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x55816429b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51496/0 (socket says 172.21.15.69:51496) 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4e45e640 1 -- 172.21.15.69:0/2564870271 learned_addr learned my addr 172.21.15.69:0/2564870271 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164294ff0 0x5581642b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164295910 0x5581642b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4e45e640 1 -- 172.21.15.69:0/2564870271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164295910 msgr2=0x5581642b7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164295910 0x5581642b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4e45e640 1 -- 172.21.15.69:0/2564870271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164294ff0 msgr2=0x5581642b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164294ff0 0x5581642b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4e45e640 1 -- 172.21.15.69:0/2564870271 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55816429fb20 con 0x5581642a01d0 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.504+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164294ff0 0x5581642b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:58.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.505+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x55816429b810 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3c400eea40 tx=0x7f3c400048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a889f3544f670c59 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.505+0000 7f3c337fe640 1 -- 172.21.15.69:0/2564870271 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400cfe60 con 0x5581642a01d0 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.505+0000 7f3c337fe640 1 -- 172.21.15.69:0/2564870271 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c40036040 con 0x5581642a01d0 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.505+0000 7f3c337fe640 1 -- 172.21.15.69:0/2564870271 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400c5f90 con 0x5581642a01d0 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.505+0000 7f3c522bf740 1 -- 172.21.15.69:0/2564870271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 msgr2=0x55816429b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.505+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x55816429b810 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3c400eea40 tx=0x7f3c400048e0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 -- 172.21.15.69:0/2564870271 shutdown_connections 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581642a01d0 0x55816429b810 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558164295910 0x5581642b7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2564870271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164294ff0 0x5581642b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 -- 172.21.15.69:0/2564870271 >> 172.21.15.69:0/2564870271 conn(0x558164291580 msgr2=0x55816429efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 -- 172.21.15.69:0/2564870271 shutdown_connections 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 -- 172.21.15.69:0/2564870271 wait complete. 2024-08-02T17:47:58.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 -- start start 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.506+0000 7f3c522bf740 1 Processor -- start 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164294ff0 0x5581642a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x5581642a3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581642a01d0 0x5581642a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481af670 con 0x558164294ff0 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481b0ae0 con 0x558164295910 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c481afd40 con 0x5581642a01d0 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 10 client.0 _ll_register_callbacks cb 0x7f3c4400c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164294ff0 0x5581642a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164294ff0 0x5581642a83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51502/0 (socket says 172.21.15.69:51502) 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 -- 172.21.15.69:0/2043038562 learned_addr learned my addr 172.21.15.69:0/2043038562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581642a01d0 0x5581642a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x5581642a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 -- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581642a01d0 msgr2=0x5581642a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581642a01d0 0x5581642a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 -- 172.21.15.69:0/2043038562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 msgr2=0x5581642a3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x5581642a3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 -- 172.21.15.69:0/2043038562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581642a4240 con 0x558164294ff0 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4e45e640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581642a01d0 0x5581642a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x5581642a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c4dc5d640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164294ff0 0x5581642a83e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5581642b2800 tx=0x7f3c3c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a8cc4ac555c379 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c317fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c317fa640 1 -- 172.21.15.69:0/2043038562 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c023070 con 0x558164294ff0 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c317fa640 1 -- 172.21.15.69:0/2043038562 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3c3c015c80 con 0x558164294ff0 2024-08-02T17:47:58.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 1 -- 172.21.15.69:0/2043038562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c40036040 con 0x558164294ff0 2024-08-02T17:47:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c317fa640 1 -- 172.21.15.69:0/2043038562 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c3c02f050 con 0x558164294ff0 2024-08-02T17:47:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 10 client.15834 fetch_fsmap learned FSMap version 26 2024-08-02T17:47:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.507+0000 7f3c522bf740 1 -- 172.21.15.69:0/2043038562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55816431a6c0 con 0x558164294ff0 2024-08-02T17:47:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c317fa640 1 -- 172.21.15.69:0/2043038562 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c40036040 con 0x558164294ff0 2024-08-02T17:47:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c317fa640 1 -- 172.21.15.69:0/2043038562 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c3c01f330 con 0x558164294ff0 2024-08-02T17:47:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c522bf740 10 client.15834 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:47:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c522bf740 10 client.15834 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:47:58.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c522bf740 20 client.15834 populate_metadata read hostname 'smithi069' 2024-08-02T17:47:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55816431adf0 0x55816431d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c522bf740 4 client.15834 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:47:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c522bf740 1 -- 172.21.15.69:0/2043038562 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581642024c0 con 0x55816431adf0 2024-08-02T17:47:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.508+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55816431adf0 0x55816431d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c317fa640 10 client.15834 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:47:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c4d45c640 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55816431adf0 0x55816431d1e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ff1c9d8962fdd27 server_cookie=1f967f8f24c7b009 in_seq=0 out_seq=0 2024-08-02T17:47:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c317fa640 1 -- 172.21.15.69:0/2043038562 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581642024c0 con 0x55816431adf0 2024-08-02T17:47:58.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c317fa640 10 client.15834 handle_command_reply: tid=0 2024-08-02T17:47:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 client.15834 shutdown 2024-08-02T17:47:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 -- 172.21.15.69:0/2043038562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164294ff0 msgr2=0x5581642a83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164294ff0 0x5581642a83e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5581642b2800 tx=0x7f3c3c009f80 comp rx=0 tx=0).stop 2024-08-02T17:47:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 -- 172.21.15.69:0/2043038562 shutdown_connections 2024-08-02T17:47:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55816431adf0 0x55816431d1e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581642a01d0 0x5581642a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558164295910 0x5581642a3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 --2- 172.21.15.69:0/2043038562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558164294ff0 0x5581642a83e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 -- 172.21.15.69:0/2043038562 >> 172.21.15.69:0/2043038562 conn(0x558164291580 msgr2=0x558164296a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c317fa640 0 client.15834 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:47:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 -- 172.21.15.69:0/2043038562 shutdown_connections 2024-08-02T17:47:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 1 -- 172.21.15.69:0/2043038562 wait complete. 2024-08-02T17:47:58.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.509+0000 7f3c522bf740 20 client.15834 trim_cache size 0 max 16384 2024-08-02T17:47:58.530 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:47:58.532 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:58 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3101678324' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:47:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 -- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c340a2cf0 msgr2=0x7f3c340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c340a2cf0 0x7f3c340a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400cfd0 tx=0x7f3c4400c470 comp rx=0 tx=0).stop 2024-08-02T17:47:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 -- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c481067b0 msgr2=0x7f3c4810eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:58.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c481067b0 0x7f3c4810eb90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3c40006010 tx=0x7f3c40003c20 comp rx=0 tx=0).stop 2024-08-02T17:47:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 -- 172.21.15.69:0/1468863513 shutdown_connections 2024-08-02T17:47:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c481067b0 0x7f3c4810eb90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c340a2cf0 0x7f3c340a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c48101150 0x7f3c4810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 --2- 172.21.15.69:0/1468863513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c481007b0 0x7f3c4810e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 -- 172.21.15.69:0/1468863513 >> 172.21.15.69:0/1468863513 conn(0x7f3c480fc4d0 msgr2=0x7f3c480fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:58.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 -- 172.21.15.69:0/1468863513 shutdown_connections 2024-08-02T17:47:58.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:58.510+0000 7f3c4fec1640 1 -- 172.21.15.69:0/1468863513 wait complete. 2024-08-02T17:47:59.264 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:47:59.268 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:59.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:47:59 smithi069 ceph-mon[32746]: pgmap v50: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 43 KiB/s wr, 6 op/s 2024-08-02T17:47:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:47:59 smithi077 ceph-mon[37108]: pgmap v50: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 43 KiB/s wr, 6 op/s 2024-08-02T17:47:59.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:47:59 smithi083 ceph-mon[37031]: pgmap v50: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 43 KiB/s wr, 6 op/s 2024-08-02T17:47:59.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 -- 172.21.15.69:0/4241902033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc102a60 msgr2=0x7f68dc102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:59.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 --2- 172.21.15.69:0/4241902033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc102a60 0x7f68dc102e40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009930 tx=0x7f68d002a420 comp rx=0 tx=0).stop 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 -- 172.21.15.69:0/4241902033 shutdown_connections 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 --2- 172.21.15.69:0/4241902033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc107fd0 0x7f68dc10e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 --2- 172.21.15.69:0/4241902033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68dc103410 0x7f68dc1078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 --2- 172.21.15.69:0/4241902033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc102a60 0x7f68dc102e40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 -- 172.21.15.69:0/4241902033 >> 172.21.15.69:0/4241902033 conn(0x7f68dc0fc500 msgr2=0x7f68dc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 -- 172.21.15.69:0/4241902033 shutdown_connections 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.721+0000 7f68e2c40640 1 -- 172.21.15.69:0/4241902033 wait complete. 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e2c40640 1 Processor -- start 2024-08-02T17:47:59.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e2c40640 1 -- start start 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e2c40640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102a60 0x7f68dc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e2c40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68dc103410 0x7f68dc19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e2c40640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc107fd0 0x7f68dc19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e2c40640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc111dd0 con 0x7f68dc107fd0 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e2c40640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc111c50 con 0x7f68dc102a60 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e2c40640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68dc111f50 con 0x7f68dc103410 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e11dd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc107fd0 0x7f68dc19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68dbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68dc103410 0x7f68dc19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e11dd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc107fd0 0x7f68dc19ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51526/0 (socket says 172.21.15.69:51526) 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e11dd640 1 -- 172.21.15.69:0/735105528 learned_addr learned my addr 172.21.15.69:0/735105528 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e09dc640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102a60 0x7f68dc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:59.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e11dd640 1 -- 172.21.15.69:0/735105528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68dc103410 msgr2=0x7f68dc19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e11dd640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68dc103410 0x7f68dc19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e11dd640 1 -- 172.21.15.69:0/735105528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102a60 msgr2=0x7f68dc19a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e11dd640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102a60 0x7f68dc19a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.722+0000 7f68e11dd640 1 -- 172.21.15.69:0/735105528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68dc19f340 con 0x7f68dc107fd0 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.723+0000 7f68e11dd640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc107fd0 0x7f68dc19ec60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f68d400cce0 tx=0x7f68d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.723+0000 7f68d9ffb640 1 -- 172.21.15.69:0/735105528 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d400f450 con 0x7f68dc107fd0 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.723+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68dc19f630 con 0x7f68dc107fd0 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.723+0000 7f68d9ffb640 1 -- 172.21.15.69:0/735105528 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68d400fad0 con 0x7f68dc107fd0 2024-08-02T17:47:59.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.723+0000 7f68d9ffb640 1 -- 172.21.15.69:0/735105528 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d4018b60 con 0x7f68dc107fd0 2024-08-02T17:47:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.723+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68dc1a6f10 con 0x7f68dc107fd0 2024-08-02T17:47:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.724+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68dc06a580 con 0x7f68dc107fd0 2024-08-02T17:47:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.725+0000 7f68d9ffb640 1 -- 172.21.15.69:0/735105528 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f68d400fc80 con 0x7f68dc107fd0 2024-08-02T17:47:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.726+0000 7f68d9ffb640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68cc0a2cf0 0x7f68cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:47:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.726+0000 7f68d9ffb640 1 -- 172.21.15.69:0/735105528 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f68d40cfcc0 con 0x7f68dc107fd0 2024-08-02T17:47:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.726+0000 7f68e09dc640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68cc0a2cf0 0x7f68cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:47:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.726+0000 7f68e09dc640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68cc0a2cf0 0x7f68cc0a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f68d0000c00 tx=0x7f68d0031490 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:47:59.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.731+0000 7f68d9ffb640 1 -- 172.21.15.69:0/735105528 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f68d4014030 con 0x7f68dc107fd0 2024-08-02T17:47:59.749 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:47:59.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.940+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f68dc10bff0 con 0x7f68dc107fd0 2024-08-02T17:47:59.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.941+0000 7f68d9ffb640 1 -- 172.21.15.69:0/735105528 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f68d409c520 con 0x7f68dc107fd0 2024-08-02T17:47:59.943 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:47:59.943 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:47:59.943 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:47:59.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68cc0a2cf0 msgr2=0x7f68cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:59.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68cc0a2cf0 0x7f68cc0a51b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f68d0000c00 tx=0x7f68d0031490 comp rx=0 tx=0).stop 2024-08-02T17:47:59.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc107fd0 msgr2=0x7f68dc19ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:47:59.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc107fd0 0x7f68dc19ec60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f68d400cce0 tx=0x7f68d400a620 comp rx=0 tx=0).stop 2024-08-02T17:47:59.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 shutdown_connections 2024-08-02T17:47:59.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68dc107fd0 0x7f68dc19ec60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68cc0a2cf0 0x7f68cc0a51b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68dc103410 0x7f68dc19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 --2- 172.21.15.69:0/735105528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68dc102a60 0x7f68dc19a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:47:59.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 >> 172.21.15.69:0/735105528 conn(0x7f68dc0fc500 msgr2=0x7f68dc100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:47:59.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 shutdown_connections 2024-08-02T17:47:59.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:47:59.943+0000 7f68e2c40640 1 -- 172.21.15.69:0/735105528 wait complete. 2024-08-02T17:47:59.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 50.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:48:00.301 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:00 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/735105528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:48:00.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:00 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/735105528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:48:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:00 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/735105528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:48:00.672 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 2319cd26 --release --await' 2024-08-02T17:48:00.672 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 2319cd26 --release --await 2024-08-02T17:48:00.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 -- 172.21.15.69:0/361595754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 msgr2=0x7f12cc100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 --2- 172.21.15.69:0/361595754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 0x7f12cc100f30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009900 tx=0x7f12bc02a480 comp rx=0 tx=0).stop 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 -- 172.21.15.69:0/361595754 shutdown_connections 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 --2- 172.21.15.69:0/361595754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc102c20 0x7f12cc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 --2- 172.21.15.69:0/361595754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12cc1022a0 0x7f12cc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 --2- 172.21.15.69:0/361595754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 0x7f12cc100f30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 -- 172.21.15.69:0/361595754 >> 172.21.15.69:0/361595754 conn(0x7f12cc0fc500 msgr2=0x7f12cc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 -- 172.21.15.69:0/361595754 shutdown_connections 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.886+0000 7f12d08e4640 1 -- 172.21.15.69:0/361595754 wait complete. 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12d08e4640 1 Processor -- start 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12d08e4640 1 -- start start 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12d08e4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 0x7f12cc197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12d08e4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12cc1022a0 0x7f12cc1976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12d08e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc102c20 0x7f12cc19ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12d08e4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc110cb0 con 0x7f12cc100b50 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12d08e4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc110b30 con 0x7f12cc1022a0 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12d08e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc110e30 con 0x7f12cc102c20 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 0x7f12cc197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 0x7f12cc197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51538/0 (socket says 172.21.15.69:51538) 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12ca59c640 1 -- 172.21.15.69:0/1394732121 learned_addr learned my addr 172.21.15.69:0/1394732121 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.887+0000 7f12cad9d640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc102c20 0x7f12cc19ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12cc1022a0 0x7f12cc1976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12ca59c640 1 -- 172.21.15.69:0/1394732121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc102c20 msgr2=0x7f12cc19ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12ca59c640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc102c20 0x7f12cc19ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12ca59c640 1 -- 172.21.15.69:0/1394732121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12cc1022a0 msgr2=0x7f12cc1976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12ca59c640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12cc1022a0 0x7f12cc1976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12ca59c640 1 -- 172.21.15.69:0/1394732121 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc19c140 con 0x7f12cc100b50 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12cc1022a0 0x7f12cc1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12ca59c640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 0x7f12cc197190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f12bc006010 tx=0x7f12bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12c37fe640 1 -- 172.21.15.69:0/1394732121 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc03f070 con 0x7f12cc100b50 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12d08e4640 1 -- 172.21.15.69:0/1394732121 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc19c3d0 con 0x7f12cc100b50 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12c37fe640 1 -- 172.21.15.69:0/1394732121 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12bc031c30 con 0x7f12cc100b50 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12c37fe640 1 -- 172.21.15.69:0/1394732121 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc035030 con 0x7f12cc100b50 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.888+0000 7f12d08e4640 1 -- 172.21.15.69:0/1394732121 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc1a3d10 con 0x7f12cc100b50 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12d2ce2740 1 Processor -- start 2024-08-02T17:48:00.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12d2ce2740 1 -- start start 2024-08-02T17:48:00.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d2284ea40 con 0x560d228f8290 2024-08-02T17:48:00.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560d227e69d0 con 0x560d228e9410 2024-08-02T17:48:00.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d2284aa00 con 0x560d228ea640 2024-08-02T17:48:00.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.891+0000 7f12c37fe640 1 -- 172.21.15.69:0/1394732121 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f12bc042070 con 0x7f12cc100b50 2024-08-02T17:48:00.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51548/0 (socket says 172.21.15.69:51548) 2024-08-02T17:48:00.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 -- 172.21.15.69:0/472255269 learned_addr learned my addr 172.21.15.69:0/472255269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:00.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12cad9d640 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12ca59c640 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 -- 172.21.15.69:0/472255269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 msgr2=0x560d228e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:00.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228e8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 -- 172.21.15.69:0/472255269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 msgr2=0x560d228e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228e9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 -- 172.21.15.69:0/472255269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d227ed650 con 0x560d228f8290 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228fa680 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f12b400bd30 tx=0x7f12b400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7ec9cd1dedf091 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c0ff9640 1 -- 172.21.15.69:0/472255269 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4015070 con 0x560d228f8290 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c0ff9640 1 -- 172.21.15.69:0/472255269 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b400be20 con 0x560d228f8290 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.892+0000 7f12c0ff9640 1 -- 172.21.15.69:0/472255269 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4011aa0 con 0x560d228f8290 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.893+0000 7f12c37fe640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12b80a2cf0 0x7f12b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.893+0000 7f12c37fe640 1 -- 172.21.15.69:0/1394732121 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f12bc0f0070 con 0x7f12cc100b50 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.893+0000 7f12cad9d640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12b80a2cf0 0x7f12b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.893+0000 7f12d2ce2740 1 -- 172.21.15.69:0/472255269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 msgr2=0x560d228fa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.893+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228fa680 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f12b400bd30 tx=0x7f12b400f730 comp rx=0 tx=0).stop 2024-08-02T17:48:00.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.893+0000 7f12cad9d640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12b80a2cf0 0x7f12b80a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12c4005fd0 tx=0x7f12c4009600 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 -- 172.21.15.69:0/472255269 shutdown_connections 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228fa680 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/472255269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 -- 172.21.15.69:0/472255269 >> 172.21.15.69:0/472255269 conn(0x560d228e5580 msgr2=0x560d228e5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 -- 172.21.15.69:0/472255269 shutdown_connections 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 -- 172.21.15.69:0/472255269 wait complete. 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 -- start start 2024-08-02T17:48:00.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.894+0000 7f12d2ce2740 1 Processor -- start 2024-08-02T17:48:00.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228f5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d228fb1a0 con 0x560d228f8290 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560d22809b20 con 0x560d228e9410 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d228fb320 con 0x560d228ea640 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12d2ce2740 10 client.0 _ll_register_callbacks cb 0x560d22807bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12cad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12ca59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12cad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51556/0 (socket says 172.21.15.69:51556) 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12cad9d640 1 -- 172.21.15.69:0/338046482 learned_addr learned my addr 172.21.15.69:0/338046482 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12cad9d640 1 -- 172.21.15.69:0/338046482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 msgr2=0x560d228f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12cad9d640 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12cad9d640 1 -- 172.21.15.69:0/338046482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 msgr2=0x560d228f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12cad9d640 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12cad9d640 1 -- 172.21.15.69:0/338046482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d225c03c0 con 0x560d228f8290 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.895+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12cad9d640 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228f5c60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12c401d910 tx=0x7f12c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e83b2103666257 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12b1ffb640 1 -- 172.21.15.69:0/338046482 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4004d20 con 0x560d228f8290 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12d2ce2740 1 -- 172.21.15.69:0/338046482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b400be20 con 0x560d228f8290 2024-08-02T17:48:00.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12b1ffb640 1 -- 172.21.15.69:0/338046482 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f12c401e6e0 con 0x560d228f8290 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12b1ffb640 1 -- 172.21.15.69:0/338046482 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12c4030050 con 0x560d228f8290 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12d2ce2740 10 client.15852 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12d2ce2740 1 -- 172.21.15.69:0/338046482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d22906f10 con 0x560d228f8290 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.896+0000 7f12b1ffb640 1 -- 172.21.15.69:0/338046482 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b400be20 con 0x560d228f8290 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.897+0000 7f12b1ffb640 1 -- 172.21.15.69:0/338046482 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f12c4027070 con 0x560d228f8290 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.897+0000 7f12d2ce2740 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.897+0000 7f12d2ce2740 10 client.15852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.897+0000 7f12d2ce2740 20 client.15852 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.897+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560d22908700 0x560d2290aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.897+0000 7f12d2ce2740 4 client.15852 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.897+0000 7f12d2ce2740 1 -- 172.21.15.69:0/338046482 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d2285bfb0 con 0x560d22908700 2024-08-02T17:48:00.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.897+0000 7f12ca59c640 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560d22908700 0x560d2290aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.898+0000 7f12b1ffb640 10 client.15852 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.898+0000 7f12ca59c640 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560d22908700 0x560d2290aba0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=caccde4a74fbc18c server_cookie=eda6d5b191975fb7 in_seq=0 out_seq=0 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.898+0000 7f12b1ffb640 1 -- 172.21.15.69:0/338046482 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d2285bfb0 con 0x560d22908700 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.898+0000 7f12b1ffb640 10 client.15852 handle_command_reply: tid=0 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.898+0000 7f12d2ce2740 1 client.15852 shutdown 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.898+0000 7f12d2ce2740 1 -- 172.21.15.69:0/338046482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 msgr2=0x560d228f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.898+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228f5c60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f12c401d910 tx=0x7f12c4004a40 comp rx=0 tx=0).stop 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 1 -- 172.21.15.69:0/338046482 shutdown_connections 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560d22908700 0x560d2290aba0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228f8290 0x560d228f5c60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228ea640 0x560d228f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/338046482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 1 -- 172.21.15.69:0/338046482 >> 172.21.15.69:0/338046482 conn(0x560d228e5580 msgr2=0x560d228eb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:00.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12b1ffb640 0 client.15852 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:48:00.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 1 -- 172.21.15.69:0/338046482 shutdown_connections 2024-08-02T17:48:00.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 1 -- 172.21.15.69:0/338046482 wait complete. 2024-08-02T17:48:00.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.899+0000 7f12d2ce2740 20 client.15852 trim_cache size 0 max 16384 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.914+0000 7f12d2ce2740 1 Processor -- start 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.914+0000 7f12d2ce2740 1 -- start start 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.914+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.914+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d228f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.914+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.914+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1af480 con 0x560d228ea640 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.914+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1acd50 con 0x560d228e9410 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.914+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1aaba0 con 0x560d228f8290 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d228f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d228f9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51564/0 (socket says 172.21.15.69:51564) 2024-08-02T17:48:00.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 -- 172.21.15.69:0/2417289869 learned_addr learned my addr 172.21.15.69:0/2417289869 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12ca59c640 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12cad9d640 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 -- 172.21.15.69:0/2417289869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 msgr2=0x560d228f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 -- 172.21.15.69:0/2417289869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 msgr2=0x560d228f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 -- 172.21.15.69:0/2417289869 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d228f3750 con 0x560d228ea640 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12ca59c640 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12cad9d640 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d228f9330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f12b400bd30 tx=0x7f12b4011680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd4114c90288a67 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12b3fff640 1 -- 172.21.15.69:0/2417289869 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4015040 con 0x560d228ea640 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12b3fff640 1 -- 172.21.15.69:0/2417289869 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b4005040 con 0x560d228ea640 2024-08-02T17:48:00.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.915+0000 7f12b3fff640 1 -- 172.21.15.69:0/2417289869 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4011950 con 0x560d228ea640 2024-08-02T17:48:00.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 -- 172.21.15.69:0/2417289869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 msgr2=0x560d228f9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d228f9330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f12b400bd30 tx=0x7f12b4011680 comp rx=0 tx=0).stop 2024-08-02T17:48:00.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 -- 172.21.15.69:0/2417289869 shutdown_connections 2024-08-02T17:48:00.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d228f9330 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/2417289869 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d228f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 -- 172.21.15.69:0/2417289869 >> 172.21.15.69:0/2417289869 conn(0x560d228e5580 msgr2=0x560d228e01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 -- 172.21.15.69:0/2417289869 shutdown_connections 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 -- 172.21.15.69:0/2417289869 wait complete. 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.916+0000 7f12d2ce2740 1 -- start start 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12d2ce2740 1 Processor -- start 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d22900980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d22900ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12d2ce2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1ab210 con 0x560d228ea640 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1ac680 con 0x560d228e9410 2024-08-02T17:48:00.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12d2ce2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1ab8e0 con 0x560d228f8290 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12d2ce2740 10 client.0 _ll_register_callbacks cb 0x7f12c4009c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12c9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d22900ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12c9d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d22900ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51576/0 (socket says 172.21.15.69:51576) 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12c9d9b640 1 -- 172.21.15.69:0/1703650358 learned_addr learned my addr 172.21.15.69:0/1703650358 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12ca59c640 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d22900980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12cad9d640 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12c9d9b640 1 -- 172.21.15.69:0/1703650358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 msgr2=0x560d228f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12c9d9b640 1 -- 172.21.15.69:0/1703650358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 msgr2=0x560d22900980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d22900980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12c9d9b640 1 -- 172.21.15.69:0/1703650358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d228f2c60 con 0x560d228ea640 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.917+0000 7f12cad9d640 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:00.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12c9d9b640 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d22900ec0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f12b40054a0 tx=0x7f12b4011570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665cacff83527199 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12b1ffb640 1 -- 172.21.15.69:0/1703650358 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4015070 con 0x560d228ea640 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12b1ffb640 1 -- 172.21.15.69:0/1703650358 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f12b4011b00 con 0x560d228ea640 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12d2ce2740 1 -- 172.21.15.69:0/1703650358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b4005040 con 0x560d228ea640 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12b1ffb640 1 -- 172.21.15.69:0/1703650358 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12b403a050 con 0x560d228ea640 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12d2ce2740 10 client.15858 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12d2ce2740 1 -- 172.21.15.69:0/1703650358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d22972020 con 0x560d228ea640 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12b1ffb640 1 -- 172.21.15.69:0/1703650358 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b4005040 con 0x560d228ea640 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12b1ffb640 1 -- 172.21.15.69:0/1703650358 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f12b4035030 con 0x560d228ea640 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12d2ce2740 10 client.15858 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12d2ce2740 10 client.15858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.918+0000 7f12d2ce2740 20 client.15858 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:00.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560d229724c0 0x560d229748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12d2ce2740 4 client.15858 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12d2ce2740 1 -- 172.21.15.69:0/1703650358 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d228564c0 con 0x560d229724c0 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12ca59c640 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560d229724c0 0x560d229748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12ca59c640 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560d229724c0 0x560d229748e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0f8eb0e083fe55c server_cookie=3422480a9cc6eb4f in_seq=0 out_seq=0 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12b1ffb640 10 client.15858 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12b1ffb640 1 -- 172.21.15.69:0/1703650358 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d228564c0 con 0x560d229724c0 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12b1ffb640 10 client.15858 handle_command_reply: tid=0 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12d2ce2740 1 client.15858 shutdown 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12d2ce2740 1 -- 172.21.15.69:0/1703650358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 msgr2=0x560d22900ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d22900ec0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f12b40054a0 tx=0x7f12b4011570 comp rx=0 tx=0).stop 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12d2ce2740 1 -- 172.21.15.69:0/1703650358 shutdown_connections 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.919+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560d229724c0 0x560d229748e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d228ea640 0x560d22900ec0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d228f8290 0x560d228f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d2ce2740 1 --2- 172.21.15.69:0/1703650358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d228e9410 0x560d22900980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d2ce2740 1 -- 172.21.15.69:0/1703650358 >> 172.21.15.69:0/1703650358 conn(0x560d228e5580 msgr2=0x560d228f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:00.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12b1ffb640 0 client.15858 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:48:00.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d2ce2740 1 -- 172.21.15.69:0/1703650358 shutdown_connections 2024-08-02T17:48:00.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d2ce2740 1 -- 172.21.15.69:0/1703650358 wait complete. 2024-08-02T17:48:00.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d2ce2740 20 client.15858 trim_cache size 0 max 16384 2024-08-02T17:48:00.927 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:48:00.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 -- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12b80a2cf0 msgr2=0x7f12b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12b80a2cf0 0x7f12b80a51b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f12c4005fd0 tx=0x7f12c4009600 comp rx=0 tx=0).stop 2024-08-02T17:48:00.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 -- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 msgr2=0x7f12cc197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:00.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 0x7f12cc197190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f12bc006010 tx=0x7f12bc003c10 comp rx=0 tx=0).stop 2024-08-02T17:48:00.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 -- 172.21.15.69:0/1394732121 shutdown_connections 2024-08-02T17:48:00.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12b80a2cf0 0x7f12b80a51b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12cc102c20 0x7f12cc19ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12cc1022a0 0x7f12cc1976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 --2- 172.21.15.69:0/1394732121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12cc100b50 0x7f12cc197190 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:00.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.920+0000 7f12d08e4640 1 -- 172.21.15.69:0/1394732121 >> 172.21.15.69:0/1394732121 conn(0x7f12cc0fc500 msgr2=0x7f12cc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:00.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.921+0000 7f12d08e4640 1 -- 172.21.15.69:0/1394732121 shutdown_connections 2024-08-02T17:48:00.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:00.921+0000 7f12d08e4640 1 -- 172.21.15.69:0/1394732121 wait complete. 2024-08-02T17:48:01.289 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:01 smithi069 ceph-mon[32746]: pgmap v51: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-08-02T17:48:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:01 smithi077 ceph-mon[37108]: pgmap v51: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-08-02T17:48:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:01 smithi083 ceph-mon[37031]: pgmap v51: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-08-02T17:48:01.638 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:01.648 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:48:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 -- 172.21.15.69:0/3394334027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c001022a0 msgr2=0x7f3c001026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 --2- 172.21.15.69:0/3394334027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c001022a0 0x7f3c001026e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f3be8009900 tx=0x7f3be802a480 comp rx=0 tx=0).stop 2024-08-02T17:48:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 -- 172.21.15.69:0/3394334027 shutdown_connections 2024-08-02T17:48:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 --2- 172.21.15.69:0/3394334027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c001022a0 0x7f3c001026e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 --2- 172.21.15.69:0/3394334027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c00102c20 0x7f3c0010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 --2- 172.21.15.69:0/3394334027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c00100b50 0x7f3c00100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 -- 172.21.15.69:0/3394334027 >> 172.21.15.69:0/3394334027 conn(0x7f3c000fc500 msgr2=0x7f3c000fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 -- 172.21.15.69:0/3394334027 shutdown_connections 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.154+0000 7f3c05a24640 1 -- 172.21.15.69:0/3394334027 wait complete. 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3c05a24640 1 Processor -- start 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3c05a24640 1 -- start start 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3c05a24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c00100b50 0x7f3c00197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3c05a24640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c001022a0 0x7f3c001975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3c05a24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c00102c20 0x7f3c0019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3c05a24640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00110cb0 con 0x7f3c00102c20 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3c05a24640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00110b30 con 0x7f3c001022a0 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3c05a24640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c00110e30 con 0x7f3c00100b50 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c00102c20 0x7f3c0019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c00102c20 0x7f3c0019b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51592/0 (socket says 172.21.15.69:51592) 2024-08-02T17:48:02.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bff7fe640 1 -- 172.21.15.69:0/1808360864 learned_addr learned my addr 172.21.15.69:0/1808360864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c001022a0 0x7f3c001975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bff7fe640 1 -- 172.21.15.69:0/1808360864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c00100b50 msgr2=0x7f3c00197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c00100b50 0x7f3c00197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bff7fe640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c00100b50 0x7f3c00197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bff7fe640 1 -- 172.21.15.69:0/1808360864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c001022a0 msgr2=0x7f3c001975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bff7fe640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c001022a0 0x7f3c001975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bff7fe640 1 -- 172.21.15.69:0/1808360864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0019c040 con 0x7f3c00102c20 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.155+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c00100b50 0x7f3c00197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.156+0000 7f3bff7fe640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c00102c20 0x7f3c0019b960 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800cce0 tx=0x7f3bf800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.156+0000 7f3c04a22640 1 -- 172.21.15.69:0/1808360864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf800f450 con 0x7f3c00102c20 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.156+0000 7f3c05a24640 1 -- 172.21.15.69:0/1808360864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0019c330 con 0x7f3c00102c20 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.156+0000 7f3c04a22640 1 -- 172.21.15.69:0/1808360864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bf800fad0 con 0x7f3c00102c20 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.156+0000 7f3c04a22640 1 -- 172.21.15.69:0/1808360864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8018c10 con 0x7f3c00102c20 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.156+0000 7f3c05a24640 1 -- 172.21.15.69:0/1808360864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c001a3c10 con 0x7f3c00102c20 2024-08-02T17:48:02.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.158+0000 7f3c04a22640 1 -- 172.21.15.69:0/1808360864 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3bf800f5f0 con 0x7f3c00102c20 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.158+0000 7f3c07e22740 1 Processor -- start 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.158+0000 7f3c07e22740 1 -- start start 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.158+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd125d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 0x563cbd130630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd132f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3c07e22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563cbd08bc60 con 0x563cbd125990 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3c07e22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563cbd023b90 con 0x563cbd130b70 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3c07e22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563cbd087c20 con 0x563cbd1262b0 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd125d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd125d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51602/0 (socket says 172.21.15.69:51602) 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 -- 172.21.15.69:0/3769944445 learned_addr learned my addr 172.21.15.69:0/3769944445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 0x563cbd130630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3c04a22640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bec0a2cf0 0x7f3bec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 -- 172.21.15.69:0/3769944445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 msgr2=0x563cbd130630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 0x563cbd130630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 -- 172.21.15.69:0/3769944445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 msgr2=0x563cbd132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd132f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 -- 172.21.15.69:0/3769944445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cbcd9b580 con 0x563cbd125990 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bff7fe640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bec0a2cf0 0x7f3bec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd125d70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3be802a9a0 tx=0x7f3be8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7416fb47517776f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3c04a22640 1 -- 172.21.15.69:0/1808360864 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3bf80cff00 con 0x7f3c00102c20 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf5ffb640 1 -- 172.21.15.69:0/3769944445 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8035070 con 0x563cbd125990 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf5ffb640 1 -- 172.21.15.69:0/3769944445 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be8003cd0 con 0x563cbd125990 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.159+0000 7f3bf5ffb640 1 -- 172.21.15.69:0/3769944445 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8031d60 con 0x563cbd125990 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 -- 172.21.15.69:0/3769944445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 msgr2=0x563cbd125d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3bff7fe640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bec0a2cf0 0x7f3bec0a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80cd890 tx=0x7f3bf800af30 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd125d70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3be802a9a0 tx=0x7f3be8000bc0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 -- 172.21.15.69:0/3769944445 shutdown_connections 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd132f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 0x563cbd130630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 --2- 172.21.15.69:0/3769944445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd125d70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 -- 172.21.15.69:0/3769944445 >> 172.21.15.69:0/3769944445 conn(0x563cbd121f20 msgr2=0x563cbd122300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 -- 172.21.15.69:0/3769944445 shutdown_connections 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.160+0000 7f3c07e22740 1 -- 172.21.15.69:0/3769944445 wait complete. 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 1 -- start start 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 1 Processor -- start 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 0x563cbd133b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd1262b0 0x563cbd134090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd1345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563cbd133740 con 0x563cbd1262b0 2024-08-02T17:48:02.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563cbd0297c0 con 0x563cbd130b70 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563cbd1338c0 con 0x563cbd125990 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3c07e22740 10 client.0 _ll_register_callbacks cb 0x563cbd032f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bfeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd1262b0 0x563cbd134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bfeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd1262b0 0x563cbd134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51606/0 (socket says 172.21.15.69:51606) 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bfeffd640 1 -- 172.21.15.69:0/2145721494 learned_addr learned my addr 172.21.15.69:0/2145721494 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 0x563cbd133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bfeffd640 1 -- 172.21.15.69:0/2145721494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 msgr2=0x563cbd133b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 0x563cbd133b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bfeffd640 1 -- 172.21.15.69:0/2145721494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 msgr2=0x563cbd1345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd1345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.161+0000 7f3bfeffd640 1 -- 172.21.15.69:0/2145721494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cbce65960 con 0x563cbd1262b0 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd1262b0 0x563cbd134090 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00113e0 tx=0x7f3bf00118a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf4fcb232233116 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:02.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2145721494 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf0012980 con 0x563cbd1262b0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2145721494 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3bf0012b20 con 0x563cbd1262b0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 1 -- 172.21.15.69:0/2145721494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be8003cd0 con 0x563cbd1262b0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2145721494 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3bf0025050 con 0x563cbd1262b0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 10 client.15870 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 1 -- 172.21.15.69:0/2145721494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563cbd141a40 con 0x563cbd1262b0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2145721494 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be8003cd0 con 0x563cbd1262b0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2145721494 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3bf000f040 con 0x563cbd1262b0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 10 client.15870 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 20 client.15870 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563cbd142fa0 0x563cbd1453c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 4 client.15870 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.162+0000 7f3c07e22740 1 -- 172.21.15.69:0/2145721494 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cbd0991d0 con 0x563cbd142fa0 2024-08-02T17:48:02.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.163+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563cbd142fa0 0x563cbd1453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.163+0000 7f3bdeffd640 10 client.15870 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.163+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563cbd142fa0 0x563cbd1453c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41733ea6e460995c server_cookie=4b18f4667a486cc2 in_seq=0 out_seq=0 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2145721494 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563cbd0991d0 con 0x563cbd142fa0 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3bdeffd640 10 client.15870 handle_command_reply: tid=0 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 client.15870 shutdown 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 -- 172.21.15.69:0/2145721494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd1262b0 msgr2=0x563cbd134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd1262b0 0x563cbd134090 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3bf00113e0 tx=0x7f3bf00118a0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 -- 172.21.15.69:0/2145721494 shutdown_connections 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd1262b0 0x563cbd134090 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563cbd142fa0 0x563cbd1453c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd1345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2145721494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 0x563cbd133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 -- 172.21.15.69:0/2145721494 >> 172.21.15.69:0/2145721494 conn(0x563cbd121f20 msgr2=0x563cbd127c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3bdeffd640 0 client.15870 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 -- 172.21.15.69:0/2145721494 shutdown_connections 2024-08-02T17:48:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 1 -- 172.21.15.69:0/2145721494 wait complete. 2024-08-02T17:48:02.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.164+0000 7f3c07e22740 20 client.15870 trim_cache size 0 max 16384 2024-08-02T17:48:02.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.181+0000 7f3c07e22740 1 Processor -- start 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3c07e22740 1 -- start start 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd14ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 0x563cbd14b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd12ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3c07e22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001af2d0 con 0x563cbd125990 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3c07e22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001acba0 con 0x563cbd130b70 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3c07e22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001aa9f0 con 0x563cbd1262b0 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd14ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd14ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51608/0 (socket says 172.21.15.69:51608) 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 -- 172.21.15.69:0/651691304 learned_addr learned my addr 172.21.15.69:0/651691304 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 0x563cbd14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd12ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 -- 172.21.15.69:0/651691304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 msgr2=0x563cbd14b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 0x563cbd14b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 -- 172.21.15.69:0/651691304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 msgr2=0x563cbd12ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd12ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.182+0000 7f3bf7fff640 1 -- 172.21.15.69:0/651691304 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cbd1257c0 con 0x563cbd125990 2024-08-02T17:48:02.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.183+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd14ac50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3be8004260 tx=0x7f3be8031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ac279cc40988d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.183+0000 7f3bf57fa640 1 -- 172.21.15.69:0/651691304 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8000e60 con 0x563cbd125990 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.183+0000 7f3bf57fa640 1 -- 172.21.15.69:0/651691304 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be8005520 con 0x563cbd125990 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.183+0000 7f3bf57fa640 1 -- 172.21.15.69:0/651691304 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8035040 con 0x563cbd125990 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 -- 172.21.15.69:0/651691304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 msgr2=0x563cbd14ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd14ac50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3be8004260 tx=0x7f3be8031940 comp rx=0 tx=0).stop 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 -- 172.21.15.69:0/651691304 shutdown_connections 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd130b70 0x563cbd12ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd1262b0 0x563cbd14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 --2- 172.21.15.69:0/651691304 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd125990 0x563cbd14ac50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 -- 172.21.15.69:0/651691304 >> 172.21.15.69:0/651691304 conn(0x563cbd121f20 msgr2=0x563cbd121080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 -- 172.21.15.69:0/651691304 shutdown_connections 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 -- 172.21.15.69:0/651691304 wait complete. 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 -- start start 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.184+0000 7f3c07e22740 1 Processor -- start 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 0x563cbd1a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd1262b0 0x563cbd1a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3c07e22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd130b70 0x563cbd12ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3c07e22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001ab060 con 0x563cbd130b70 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3c07e22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001ac4d0 con 0x563cbd1262b0 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3c07e22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c001ab730 con 0x563cbd125990 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3c07e22740 10 client.0 _ll_register_callbacks cb 0x7f3bf0009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd130b70 0x563cbd12ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd130b70 0x563cbd12ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51614/0 (socket says 172.21.15.69:51614) 2024-08-02T17:48:02.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 -- 172.21.15.69:0/2093830973 learned_addr learned my addr 172.21.15.69:0/2093830973 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 0x563cbd1a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd1262b0 0x563cbd1a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 -- 172.21.15.69:0/2093830973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 msgr2=0x563cbd1a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 0x563cbd1a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 -- 172.21.15.69:0/2093830973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd1262b0 msgr2=0x563cbd1a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd1262b0 0x563cbd1a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 -- 172.21.15.69:0/2093830973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cbd12b210 con 0x563cbd130b70 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bf7fff640 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd130b70 0x563cbd12ab20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3be80316a0 tx=0x7f3be8053040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97b9d107f1e376e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.185+0000 7f3bdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2093830973 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be8035030 con 0x563cbd130b70 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 1 -- 172.21.15.69:0/2093830973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3be8005520 con 0x563cbd130b70 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2093830973 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3be80319b0 con 0x563cbd130b70 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2093830973 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3be8031bb0 con 0x563cbd130b70 2024-08-02T17:48:02.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 10 client.15876 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 1 -- 172.21.15.69:0/2093830973 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563cbd1afdf0 con 0x563cbd130b70 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2093830973 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3be8005520 con 0x563cbd130b70 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2093830973 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3be8048070 con 0x563cbd130b70 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 10 client.15876 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 10 client.15876 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 20 client.15876 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563cbd1b04c0 0x563cbd1b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 4 client.15876 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2319cd26", "release": true, "await": true}] 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.186+0000 7f3c07e22740 1 -- 172.21.15.69:0/2093830973 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2319cd26", "release": true, "await": true}) -- 0x563cbd0936e0 con 0x563cbd1b04c0 2024-08-02T17:48:02.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.187+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563cbd1b04c0 0x563cbd1b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:02.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.187+0000 7f3bfeffd640 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563cbd1b04c0 0x563cbd1b28b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86bb7807e09ede69 server_cookie=ab65bd9b1b71dcce in_seq=0 out_seq=0 2024-08-02T17:48:02.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.187+0000 7f3bdeffd640 10 client.15876 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:02.208 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:02.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3bdeffd640 1 -- 172.21.15.69:0/2093830973 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x563cbd0936e0 con 0x563cbd1b04c0 2024-08-02T17:48:02.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3bdeffd640 10 client.15876 handle_command_reply: tid=0 2024-08-02T17:48:02.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 client.15876 shutdown 2024-08-02T17:48:02.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 -- 172.21.15.69:0/2093830973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd130b70 msgr2=0x563cbd12ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd130b70 0x563cbd12ab20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3be80316a0 tx=0x7f3be8053040 comp rx=0 tx=0).stop 2024-08-02T17:48:02.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 -- 172.21.15.69:0/2093830973 shutdown_connections 2024-08-02T17:48:02.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563cbd130b70 0x563cbd12ab20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563cbd1b04c0 0x563cbd1b28b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563cbd1262b0 0x563cbd1a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 --2- 172.21.15.69:0/2093830973 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563cbd125990 0x563cbd1a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 -- 172.21.15.69:0/2093830973 >> 172.21.15.69:0/2093830973 conn(0x563cbd121f20 msgr2=0x563cbd1273e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3bdeffd640 0 client.15876 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 -- 172.21.15.69:0/2093830973 shutdown_connections 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.234+0000 7f3c07e22740 1 -- 172.21.15.69:0/2093830973 wait complete. 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c07e22740 20 client.15876 trim_cache size 0 max 16384 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:48:02.237 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 8, 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "2319cd26": { 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 8, 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 141.9, 2024-08-02T17:48:02.238 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:48:02.239 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:48:02.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 -- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bec0a2cf0 msgr2=0x7f3bec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bec0a2cf0 0x7f3bec0a51b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80cd890 tx=0x7f3bf800af30 comp rx=0 tx=0).stop 2024-08-02T17:48:02.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 -- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c00102c20 msgr2=0x7f3c0019b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:02.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c00102c20 0x7f3c0019b960 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800cce0 tx=0x7f3bf800a620 comp rx=0 tx=0).stop 2024-08-02T17:48:02.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 -- 172.21.15.69:0/1808360864 shutdown_connections 2024-08-02T17:48:02.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c00102c20 0x7f3c0019b960 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bec0a2cf0 0x7f3bec0a51b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c001022a0 0x7f3c001975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 --2- 172.21.15.69:0/1808360864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c00100b50 0x7f3c00197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:02.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.235+0000 7f3c05a24640 1 -- 172.21.15.69:0/1808360864 >> 172.21.15.69:0/1808360864 conn(0x7f3c000fc500 msgr2=0x7f3c000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:02.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.236+0000 7f3c05a24640 1 -- 172.21.15.69:0/1808360864 shutdown_connections 2024-08-02T17:48:02.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:02.236+0000 7f3c05a24640 1 -- 172.21.15.69:0/1808360864 wait complete. 2024-08-02T17:48:03.211 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2319cd26', total seconds elapsed: 27.1 2024-08-02T17:48:03.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:03 smithi069 ceph-mon[32746]: pgmap v52: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 68 KiB/s wr, 24 op/s 2024-08-02T17:48:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:03 smithi077 ceph-mon[37108]: pgmap v52: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 68 KiB/s wr, 24 op/s 2024-08-02T17:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.423+0000 7fd8788c0640 1 -- 172.21.15.69:0/3563040576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd874109180 msgr2=0x7fd87410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.424+0000 7fd8788c0640 1 --2- 172.21.15.69:0/3563040576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd874109180 0x7fd87410ff80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd864009860 tx=0x7fd86402a340 comp rx=0 tx=0).stop 2024-08-02T17:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.424+0000 7fd8788c0640 1 -- 172.21.15.69:0/3563040576 shutdown_connections 2024-08-02T17:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.424+0000 7fd8788c0640 1 --2- 172.21.15.69:0/3563040576 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd87411ae70 0x7fd87411d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.424+0000 7fd8788c0640 1 --2- 172.21.15.69:0/3563040576 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd874109560 0x7fd8741099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.424+0000 7fd8788c0640 1 --2- 172.21.15.69:0/3563040576 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd874109180 0x7fd87410ff80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.424+0000 7fd8788c0640 1 -- 172.21.15.69:0/3563040576 >> 172.21.15.69:0/3563040576 conn(0x7fd87406de70 msgr2=0x7fd87406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:03.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.424+0000 7fd8788c0640 1 -- 172.21.15.69:0/3563040576 shutdown_connections 2024-08-02T17:48:03.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.426+0000 7fd8788c0640 1 -- 172.21.15.69:0/3563040576 wait complete. 2024-08-02T17:48:03.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.427+0000 7fd8788c0640 1 Processor -- start 2024-08-02T17:48:03.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.427+0000 7fd8788c0640 1 -- start start 2024-08-02T17:48:03.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd8788c0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd874109180 0x7fd8741a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd8788c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd874109560 0x7fd8741a7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd8788c0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd87411ae70 0x7fd8741ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd8788c0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd874120b20 con 0x7fd87411ae70 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd8788c0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd8741209a0 con 0x7fd874109180 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd8788c0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd874120ca0 con 0x7fd874109560 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd87259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd874109180 0x7fd8741a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd87259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd874109180 0x7fd8741a6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:36266/0 (socket says 172.21.15.69:36266) 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd87259c640 1 -- 172.21.15.69:0/149845659 learned_addr learned my addr 172.21.15.69:0/149845659 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.428+0000 7fd871d9b640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd874109560 0x7fd8741a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.429+0000 7fd87259c640 1 -- 172.21.15.69:0/149845659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd874109560 msgr2=0x7fd8741a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.429+0000 7fd87259c640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd874109560 0x7fd8741a7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.429+0000 7fd87259c640 1 -- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd87411ae70 msgr2=0x7fd8741ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.429+0000 7fd87259c640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd87411ae70 0x7fd8741ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.429+0000 7fd87259c640 1 -- 172.21.15.69:0/149845659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8741abef0 con 0x7fd874109180 2024-08-02T17:48:03.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.430+0000 7fd87259c640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd874109180 0x7fd8741a6f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd864030920 tx=0x7fd864030460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:03.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.431+0000 7fd8637fe640 1 -- 172.21.15.69:0/149845659 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864037040 con 0x7fd874109180 2024-08-02T17:48:03.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.431+0000 7fd8637fe640 1 -- 172.21.15.69:0/149845659 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd864040070 con 0x7fd874109180 2024-08-02T17:48:03.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.431+0000 7fd8788c0640 1 -- 172.21.15.69:0/149845659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8741ac180 con 0x7fd874109180 2024-08-02T17:48:03.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.431+0000 7fd8788c0640 1 -- 172.21.15.69:0/149845659 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8741b3ac0 con 0x7fd874109180 2024-08-02T17:48:03.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.433+0000 7fd8637fe640 1 -- 172.21.15.69:0/149845659 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86403b430 con 0x7fd874109180 2024-08-02T17:48:03.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.438+0000 7fd87acbe740 1 Processor -- start 2024-08-02T17:48:03.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.438+0000 7fd87acbe740 1 -- start start 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf503d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf5ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf5b1d0 0x55a6faf5d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd8637fe640 1 -- 172.21.15.69:0/149845659 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd8640035f0 con 0x7fd874109180 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf5ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf5ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:36276/0 (socket says 172.21.15.69:36276) 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87259c640 1 -- 172.21.15.69:0/617337236 learned_addr learned my addr 172.21.15.69:0/617337236 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87acbe740 1 -- 172.21.15.69:0/617337236 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a6faeb5a40 con 0x55a6faf5b1d0 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87acbe740 1 -- 172.21.15.69:0/617337236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a6fae4d9d0 con 0x55a6faf50910 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd87acbe740 1 -- 172.21.15.69:0/617337236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a6faeb1a00 con 0x55a6faf4fff0 2024-08-02T17:48:03.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.439+0000 7fd872d9d640 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.440+0000 7fd87259c640 1 -- 172.21.15.69:0/617337236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 msgr2=0x55a6faf503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.440+0000 7fd87259c640 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.440+0000 7fd87259c640 1 -- 172.21.15.69:0/617337236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf5b1d0 msgr2=0x55a6faf5d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.440+0000 7fd87259c640 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf5b1d0 0x55a6faf5d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.440+0000 7fd87259c640 1 -- 172.21.15.69:0/617337236 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6fae54650 con 0x55a6faf50910 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.440+0000 7fd872d9d640 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.440+0000 7fd87259c640 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf5ac90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd8640ec7b0 tx=0x7fd8640f7490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3cae568ac0e8580 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.440+0000 7fd860ff9640 1 -- 172.21.15.69:0/617337236 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640f1310 con 0x55a6faf50910 2024-08-02T17:48:03.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.441+0000 7fd8637fe640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd85c0a2ce0 0x7fd85c0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.441+0000 7fd8637fe640 1 -- 172.21.15.69:0/149845659 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd8640ef400 con 0x7fd874109180 2024-08-02T17:48:03.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.441+0000 7fd860ff9640 1 -- 172.21.15.69:0/617337236 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8640bbd10 con 0x55a6faf50910 2024-08-02T17:48:03.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.441+0000 7fd860ff9640 1 -- 172.21.15.69:0/617337236 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640313a0 con 0x55a6faf50910 2024-08-02T17:48:03.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.442+0000 7fd872d9d640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd85c0a2ce0 0x7fd85c0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.443+0000 7fd87acbe740 1 -- 172.21.15.69:0/617337236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 msgr2=0x55a6faf5ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.443+0000 7fd87acbe740 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf5ac90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd8640ec7b0 tx=0x7fd8640f7490 comp rx=0 tx=0).stop 2024-08-02T17:48:03.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.443+0000 7fd87acbe740 1 -- 172.21.15.69:0/617337236 shutdown_connections 2024-08-02T17:48:03.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.443+0000 7fd87acbe740 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf5b1d0 0x55a6faf5d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.444+0000 7fd87acbe740 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf5ac90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.444+0000 7fd87acbe740 1 --2- 172.21.15.69:0/617337236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.444+0000 7fd87acbe740 1 -- 172.21.15.69:0/617337236 >> 172.21.15.69:0/617337236 conn(0x55a6faf4c580 msgr2=0x55a6faf4c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:03.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.444+0000 7fd872d9d640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd85c0a2ce0 0x7fd85c0a51a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd86c005fd0 tx=0x7fd86c009d00 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:03.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.445+0000 7fd87acbe740 1 -- 172.21.15.69:0/617337236 shutdown_connections 2024-08-02T17:48:03.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.445+0000 7fd87acbe740 1 -- 172.21.15.69:0/617337236 wait complete. 2024-08-02T17:48:03.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.445+0000 7fd87acbe740 1 -- start start 2024-08-02T17:48:03.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.445+0000 7fd87acbe740 1 Processor -- start 2024-08-02T17:48:03.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.446+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf58060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.446+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf50910 0x55a6faf585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.446+0000 7fd87259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf58060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.446+0000 7fd87259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf58060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:36924/0 (socket says 172.21.15.69:36924) 2024-08-02T17:48:03.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.446+0000 7fd87259c640 1 -- 172.21.15.69:0/3290582237 learned_addr learned my addr 172.21.15.69:0/3290582237 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:03.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.446+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 0x55a6faf58ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a6faf5db60 con 0x55a6faf50910 2024-08-02T17:48:03.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd872d9d640 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 0x55a6faf58ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a6fae70b20 con 0x55a6faf5b1d0 2024-08-02T17:48:03.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a6faf5dce0 con 0x55a6faf4fff0 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd871d9b640 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf50910 0x55a6faf585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd87259c640 1 -- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 msgr2=0x55a6faf58ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd87259c640 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 0x55a6faf58ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd87259c640 1 -- 172.21.15.69:0/3290582237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf50910 msgr2=0x55a6faf585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.447+0000 7fd87259c640 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf50910 0x55a6faf585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.448+0000 7fd87259c640 1 -- 172.21.15.69:0/3290582237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fac273c0 con 0x55a6faf4fff0 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.448+0000 7fd871d9b640 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf50910 0x55a6faf585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.448+0000 7fd87259c640 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf58060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd864049040 tx=0x7fd8640f7ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58577ef9cf46037b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:03.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.448+0000 7fd859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.448+0000 7fd859ffb640 1 -- 172.21.15.69:0/3290582237 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd864030950 con 0x55a6faf4fff0 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.448+0000 7fd859ffb640 1 -- 172.21.15.69:0/3290582237 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd8640b9910 con 0x55a6faf4fff0 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.448+0000 7fd87acbe740 10 client.0 _ll_register_callbacks cb 0x55a6fae6ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.448+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8640bbd10 con 0x55a6faf4fff0 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd859ffb640 1 -- 172.21.15.69:0/3290582237 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd86404a200 con 0x55a6faf4fff0 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd87acbe740 10 client.25387 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a6faf68f70 con 0x55a6faf4fff0 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd859ffb640 1 -- 172.21.15.69:0/3290582237 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd86403a070 con 0x55a6faf4fff0 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd87acbe740 10 client.25387 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd87acbe740 10 client.25387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd87acbe740 20 client.25387 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a6faf6a5e0 0x55a6faf6caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd87acbe740 4 client.25387 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:03.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.449+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd8640bbd10 con 0x55a6faf6a5e0 2024-08-02T17:48:03.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.450+0000 7fd871d9b640 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a6faf6a5e0 0x55a6faf6caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.450+0000 7fd871d9b640 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a6faf6a5e0 0x55a6faf6caa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64adabc4f91b2576 server_cookie=d2cfd0e0c7f8ecc5 in_seq=0 out_seq=0 2024-08-02T17:48:03.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.450+0000 7fd859ffb640 10 client.25387 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.452+0000 7fd859ffb640 1 -- 172.21.15.69:0/3290582237 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fd8640bbd10 con 0x55a6faf6a5e0 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.452+0000 7fd859ffb640 10 client.25387 handle_command_reply: tid=0 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.452+0000 7fd87acbe740 1 client.25387 shutdown 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 msgr2=0x55a6faf58060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf58060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd864049040 tx=0x7fd8640f7ea0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 shutdown_connections 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf50910 0x55a6faf585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a6faf6a5e0 0x55a6faf6caa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 0x55a6faf58ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3290582237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf4fff0 0x55a6faf58060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 >> 172.21.15.69:0/3290582237 conn(0x55a6faf4c580 msgr2=0x55a6faf52260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:03.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.453+0000 7fd859ffb640 0 client.25387 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:03.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.454+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 shutdown_connections 2024-08-02T17:48:03.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.454+0000 7fd87acbe740 1 -- 172.21.15.69:0/3290582237 wait complete. 2024-08-02T17:48:03.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.454+0000 7fd87acbe740 20 client.25387 trim_cache size 0 max 16384 2024-08-02T17:48:03.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.476+0000 7fd87acbe740 1 Processor -- start 2024-08-02T17:48:03.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.476+0000 7fd87acbe740 1 -- start start 2024-08-02T17:48:03.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf52a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf50910 0x55a6faf58930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 0x55a6faf723b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd87acbe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd854012760 con 0x55a6faf4fff0 2024-08-02T17:48:03.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd87acbe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd854010030 con 0x55a6faf5b1d0 2024-08-02T17:48:03.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd87acbe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd85400de80 con 0x55a6faf50910 2024-08-02T17:48:03.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd871d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf50910 0x55a6faf58930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd872d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 0x55a6faf723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd871d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf50910 0x55a6faf58930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:36926/0 (socket says 172.21.15.69:36926) 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd871d9b640 1 -- 172.21.15.69:0/3906624254 learned_addr learned my addr 172.21.15.69:0/3906624254 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd87259c640 1 --2- 172.21.15.69:0/3906624254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf52a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd871d9b640 1 -- 172.21.15.69:0/3906624254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 msgr2=0x55a6faf723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd871d9b640 1 --2- 172.21.15.69:0/3906624254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 0x55a6faf723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd871d9b640 1 -- 172.21.15.69:0/3906624254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 msgr2=0x55a6faf52a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd871d9b640 1 --2- 172.21.15.69:0/3906624254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf52a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.477+0000 7fd871d9b640 1 -- 172.21.15.69:0/3906624254 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6faf6d120 con 0x55a6faf50910 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.478+0000 7fd871d9b640 1 --2- 172.21.15.69:0/3906624254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf50910 0x55a6faf58930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55a6faf4d900 tx=0x7fd8680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cea5d74a993fd43 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.478+0000 7fd85bfff640 1 -- 172.21.15.69:0/3906624254 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86801f020 con 0x55a6faf50910 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.478+0000 7fd85bfff640 1 -- 172.21.15.69:0/3906624254 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd868003d20 con 0x55a6faf50910 2024-08-02T17:48:03.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.478+0000 7fd85bfff640 1 -- 172.21.15.69:0/3906624254 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86801b5f0 con 0x55a6faf50910 2024-08-02T17:48:03.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.479+0000 7fd87acbe740 1 -- 172.21.15.69:0/3906624254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf50910 msgr2=0x55a6faf58930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.479+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3906624254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf50910 0x55a6faf58930 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x55a6faf4d900 tx=0x7fd8680046c0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.480+0000 7fd87acbe740 1 -- 172.21.15.69:0/3906624254 shutdown_connections 2024-08-02T17:48:03.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.480+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3906624254 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf5b1d0 0x55a6faf723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.480+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3906624254 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf50910 0x55a6faf58930 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.480+0000 7fd87acbe740 1 --2- 172.21.15.69:0/3906624254 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf52a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.480+0000 7fd87acbe740 1 -- 172.21.15.69:0/3906624254 >> 172.21.15.69:0/3906624254 conn(0x55a6faf4c580 msgr2=0x55a6faf59fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:03.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.481+0000 7fd87acbe740 1 -- 172.21.15.69:0/3906624254 shutdown_connections 2024-08-02T17:48:03.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.481+0000 7fd87acbe740 1 -- 172.21.15.69:0/3906624254 wait complete. 2024-08-02T17:48:03.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.481+0000 7fd87acbe740 1 -- start start 2024-08-02T17:48:03.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.482+0000 7fd87acbe740 1 Processor -- start 2024-08-02T17:48:03.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.482+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf55080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.482+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd87acbe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf5b1d0 0x55a6faf55b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd87259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf55080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd87259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf55080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51674/0 (socket says 172.21.15.69:51674) 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd871d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd871d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:36320/0 (socket says 172.21.15.69:36320) 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd85400e4f0 con 0x55a6faf4fff0 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd87259c640 1 -- 172.21.15.69:0/319585457 learned_addr learned my addr 172.21.15.69:0/319585457 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd872d9d640 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf5b1d0 0x55a6faf55b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd85400f960 con 0x55a6faf50910 2024-08-02T17:48:03.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.483+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd85400ebc0 con 0x55a6faf5b1d0 2024-08-02T17:48:03.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87259c640 1 -- 172.21.15.69:0/319585457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf5b1d0 msgr2=0x55a6faf55b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:03.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87259c640 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf5b1d0 0x55a6faf55b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87259c640 1 -- 172.21.15.69:0/319585457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 msgr2=0x55a6faf555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87259c640 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87259c640 1 -- 172.21.15.69:0/319585457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6fafd4380 con 0x55a6faf4fff0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd871d9b640 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87259c640 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf55080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd864035040 tx=0x7fd8640f2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee21c9641fdb8bf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87acbe740 10 client.0 _ll_register_callbacks cb 0x7fd8640f7250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd859ffb640 1 -- 172.21.15.69:0/319585457 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8640bb150 con 0x55a6faf4fff0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd859ffb640 1 -- 172.21.15.69:0/319585457 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd8640b9910 con 0x55a6faf4fff0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd868003d20 con 0x55a6faf4fff0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd859ffb640 1 -- 172.21.15.69:0/319585457 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd8640bb3e0 con 0x55a6faf4fff0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87acbe740 10 client.15885 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.484+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a6fafd52a0 con 0x55a6faf4fff0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd859ffb640 1 -- 172.21.15.69:0/319585457 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd86403a040 con 0x55a6faf4fff0 2024-08-02T17:48:03.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd87acbe740 10 client.15885 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:03.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd87acbe740 10 client.15885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:03.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd87acbe740 20 client.15885 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:03.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd87acbe740 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a6fafd5780 0x55a6fafd7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:03.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd87acbe740 4 client.15885 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:48:03.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a6faec2fb0 con 0x55a6fafd5780 2024-08-02T17:48:03.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd871d9b640 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a6fafd5780 0x55a6fafd7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:03.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd871d9b640 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a6fafd5780 0x55a6fafd7c40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d39225baf313b9 server_cookie=c636f9666d86e6f0 in_seq=0 out_seq=0 2024-08-02T17:48:03.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.485+0000 7fd859ffb640 10 client.15885 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:03.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.487+0000 7fd859ffb640 1 -- 172.21.15.69:0/319585457 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a6faec2fb0 con 0x55a6fafd5780 2024-08-02T17:48:03.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.487+0000 7fd859ffb640 10 client.15885 handle_command_reply: tid=0 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.487+0000 7fd87acbe740 1 client.15885 shutdown 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 msgr2=0x55a6faf55080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd87acbe740 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf55080 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd864035040 tx=0x7fd8640f2e40 comp rx=0 tx=0).stop 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 shutdown_connections 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd87acbe740 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a6fafd5780 0x55a6fafd7c40 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd87acbe740 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a6faf5b1d0 0x55a6faf55b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd859ffb640 0 client.15885 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd87acbe740 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a6faf50910 0x55a6faf555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd87acbe740 1 --2- 172.21.15.69:0/319585457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a6faf4fff0 0x55a6faf55080 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.488+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 >> 172.21.15.69:0/319585457 conn(0x55a6faf4c580 msgr2=0x55a6faf65db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.489+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 shutdown_connections 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.489+0000 7fd87acbe740 1 -- 172.21.15.69:0/319585457 wait complete. 2024-08-02T17:48:03.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.489+0000 7fd87acbe740 20 client.15885 trim_cache size 0 max 16384 2024-08-02T17:48:03.492 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 -- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd85c0a2ce0 msgr2=0x7fd85c0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd85c0a2ce0 0x7fd85c0a51a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd86c005fd0 tx=0x7fd86c009d00 comp rx=0 tx=0).stop 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 -- 172.21.15.69:0/149845659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd874109180 msgr2=0x7fd8741a6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd874109180 0x7fd8741a6f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd864030920 tx=0x7fd864030460 comp rx=0 tx=0).stop 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 -- 172.21.15.69:0/149845659 shutdown_connections 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd87411ae70 0x7fd8741ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd85c0a2ce0 0x7fd85c0a51a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd874109560 0x7fd8741a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 --2- 172.21.15.69:0/149845659 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd874109180 0x7fd8741a6f40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 -- 172.21.15.69:0/149845659 >> 172.21.15.69:0/149845659 conn(0x7fd87406de70 msgr2=0x7fd87411bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 -- 172.21.15.69:0/149845659 shutdown_connections 2024-08-02T17:48:03.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:03.490+0000 7fd8788c0640 1 -- 172.21.15.69:0/149845659 wait complete. 2024-08-02T17:48:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:03 smithi083 ceph-mon[37031]: pgmap v52: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 68 KiB/s wr, 24 op/s 2024-08-02T17:48:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 55.0 s, 1142.9 tps, lat 441.894 ms stddev 3028.380 2024-08-02T17:48:05.325 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:48:05.325 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:48:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:05 smithi077 ceph-mon[37108]: pgmap v53: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 68 KiB/s wr, 24 op/s 2024-08-02T17:48:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:05 smithi069 ceph-mon[32746]: pgmap v53: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 68 KiB/s wr, 24 op/s 2024-08-02T17:48:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:05 smithi083 ceph-mon[37031]: pgmap v53: 129 pgs: 129 active+clean; 984 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 68 KiB/s wr, 24 op/s 2024-08-02T17:48:05.775 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:07.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:07 smithi083 ceph-mon[37031]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:48:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:07 smithi077 ceph-mon[37108]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:48:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:07 smithi069 ceph-mon[32746]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:48:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.336+0000 7fced1110640 1 -- 172.21.15.69:0/481143376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc1022a0 msgr2=0x7fcecc1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.336+0000 7fced1110640 1 --2- 172.21.15.69:0/481143376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc1022a0 0x7fcecc1026e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009cd0 tx=0x7fcec402a980 comp rx=0 tx=0).stop 2024-08-02T17:48:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.336+0000 7fced1110640 1 -- 172.21.15.69:0/481143376 shutdown_connections 2024-08-02T17:48:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.336+0000 7fced1110640 1 --2- 172.21.15.69:0/481143376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc102c20 0x7fcecc10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.336+0000 7fced1110640 1 --2- 172.21.15.69:0/481143376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc1022a0 0x7fcecc1026e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.336+0000 7fced1110640 1 --2- 172.21.15.69:0/481143376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcecc100b50 0x7fcecc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.336+0000 7fced1110640 1 -- 172.21.15.69:0/481143376 >> 172.21.15.69:0/481143376 conn(0x7fcecc0fc500 msgr2=0x7fcecc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:08.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.337+0000 7fced1110640 1 -- 172.21.15.69:0/481143376 shutdown_connections 2024-08-02T17:48:08.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.337+0000 7fced1110640 1 -- 172.21.15.69:0/481143376 wait complete. 2024-08-02T17:48:08.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.338+0000 7fced1110640 1 Processor -- start 2024-08-02T17:48:08.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.338+0000 7fced1110640 1 -- start start 2024-08-02T17:48:08.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fced1110640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc100b50 0x7fcecc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fced1110640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcecc1022a0 0x7fcecc19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fced1110640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc102c20 0x7fcecc19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fced1110640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc10fbf0 con 0x7fcecc1022a0 2024-08-02T17:48:08.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fced1110640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc10fa70 con 0x7fcecc100b50 2024-08-02T17:48:08.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fced1110640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc10fd70 con 0x7fcecc102c20 2024-08-02T17:48:08.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc100b50 0x7fcecc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc102c20 0x7fcecc19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc102c20 0x7fcecc19edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42760/0 (socket says 172.21.15.69:42760) 2024-08-02T17:48:08.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc100b50 0x7fcecc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43798/0 (socket says 172.21.15.69:43798) 2024-08-02T17:48:08.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecb59e640 1 -- 172.21.15.69:0/2337391641 learned_addr learned my addr 172.21.15.69:0/2337391641 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:08.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fceca59c640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcecc1022a0 0x7fcecc19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecb59e640 1 -- 172.21.15.69:0/2337391641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc100b50 msgr2=0x7fcecc19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecb59e640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc100b50 0x7fcecc19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecb59e640 1 -- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcecc1022a0 msgr2=0x7fcecc19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecb59e640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcecc1022a0 0x7fcecc19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecb59e640 1 -- 172.21.15.69:0/2337391641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc19f490 con 0x7fcecc102c20 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fceca59c640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcecc1022a0 0x7fcecc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.339+0000 7fcecad9d640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc100b50 0x7fcecc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.340+0000 7fcecb59e640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc102c20 0x7fcecc19edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e970 tx=0x7fcebc00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.340+0000 7fceb3fff640 1 -- 172.21.15.69:0/2337391641 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc00ccc0 con 0x7fcecc102c20 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.340+0000 7fceb3fff640 1 -- 172.21.15.69:0/2337391641 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc00ce60 con 0x7fcecc102c20 2024-08-02T17:48:08.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.340+0000 7fceb3fff640 1 -- 172.21.15.69:0/2337391641 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc010490 con 0x7fcecc102c20 2024-08-02T17:48:08.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.340+0000 7fced1110640 1 -- 172.21.15.69:0/2337391641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc19f780 con 0x7fcecc102c20 2024-08-02T17:48:08.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.340+0000 7fced1110640 1 -- 172.21.15.69:0/2337391641 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc1a7060 con 0x7fcecc102c20 2024-08-02T17:48:08.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.343+0000 7fceb3fff640 1 -- 172.21.15.69:0/2337391641 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcebc005020 con 0x7fcecc102c20 2024-08-02T17:48:08.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.344+0000 7fceb3fff640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fceb40a2e30 0x7fceb40a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.344+0000 7fceb3fff640 1 -- 172.21.15.69:0/2337391641 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcebc0cfce0 con 0x7fcecc102c20 2024-08-02T17:48:08.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.345+0000 7fcecad9d640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fceb40a2e30 0x7fceb40a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.345+0000 7fcecad9d640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fceb40a2e30 0x7fceb40a52f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009fc0 tx=0x7fcec0005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:08.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.345+0000 7fced350e740 1 Processor -- start 2024-08-02T17:48:08.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fced350e740 1 -- start start 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fced350e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fced350e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033deb20 0x555a033e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fced350e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033e93e0 0x555a033eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fceca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033deb20 0x555a033e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fceca59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033deb20 0x555a033e8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42766/0 (socket says 172.21.15.69:42766) 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fcecad9d640 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033e93e0 0x555a033eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fceca59c640 1 -- 172.21.15.69:0/1006878690 learned_addr learned my addr 172.21.15.69:0/1006878690 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fced350e740 1 -- 172.21.15.69:0/1006878690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555a033a1e40 con 0x555a033de200 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fced350e740 1 -- 172.21.15.69:0/1006878690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a033437f0 con 0x555a033e93e0 2024-08-02T17:48:08.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.346+0000 7fced350e740 1 -- 172.21.15.69:0/1006878690 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a032eafb0 con 0x555a033deb20 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fcecb59e640 1 -- 172.21.15.69:0/1006878690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033deb20 msgr2=0x555a033e8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033deb20 0x555a033e8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fcecb59e640 1 -- 172.21.15.69:0/1006878690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 msgr2=0x555a033de5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033de5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fcecb59e640 1 -- 172.21.15.69:0/1006878690 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a030b56a0 con 0x555a033e93e0 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fcecad9d640 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033de5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033e93e0 0x555a033eb7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0cd010 tx=0x7fcebc09b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cf4b86bd91d550b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fceb17fa640 1 -- 172.21.15.69:0/1006878690 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc09f070 con 0x555a033e93e0 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fceb17fa640 1 -- 172.21.15.69:0/1006878690 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc0a4a50 con 0x555a033e93e0 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.347+0000 7fceb17fa640 1 -- 172.21.15.69:0/1006878690 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc0cd290 con 0x555a033e93e0 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.348+0000 7fced350e740 1 -- 172.21.15.69:0/1006878690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033e93e0 msgr2=0x555a033eb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.348+0000 7fced350e740 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033e93e0 0x555a033eb7d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0cd010 tx=0x7fcebc09b800 comp rx=0 tx=0).stop 2024-08-02T17:48:08.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.348+0000 7fced350e740 1 -- 172.21.15.69:0/1006878690 shutdown_connections 2024-08-02T17:48:08.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.348+0000 7fced350e740 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033e93e0 0x555a033eb7d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.348+0000 7fced350e740 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033deb20 0x555a033e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.348+0000 7fced350e740 1 --2- 172.21.15.69:0/1006878690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033de5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.348+0000 7fced350e740 1 -- 172.21.15.69:0/1006878690 >> 172.21.15.69:0/1006878690 conn(0x555a033da790 msgr2=0x555a033dab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:08.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.350+0000 7fced350e740 1 -- 172.21.15.69:0/1006878690 shutdown_connections 2024-08-02T17:48:08.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.350+0000 7fced350e740 1 -- 172.21.15.69:0/1006878690 wait complete. 2024-08-02T17:48:08.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.350+0000 7fced350e740 1 -- start start 2024-08-02T17:48:08.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fced350e740 1 Processor -- start 2024-08-02T17:48:08.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fced350e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a033e6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fced350e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a033e6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fced350e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a033e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fced350e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555a032fecb0 con 0x555a033deb20 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fced350e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a0333f7b0 con 0x555a033de200 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fced350e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a033ebfb0 con 0x555a033e93e0 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fcecad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a033e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fced350e740 10 client.0 _ll_register_callbacks cb 0x555a03052ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fcecad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a033e6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43828/0 (socket says 172.21.15.69:43828) 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fcecad9d640 1 -- 172.21.15.69:0/1362971691 learned_addr learned my addr 172.21.15.69:0/1362971691 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fceca59c640 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a033e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.351+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a033e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fcecb59e640 1 -- 172.21.15.69:0/1362971691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 msgr2=0x555a033e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a033e6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fcecb59e640 1 -- 172.21.15.69:0/1362971691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 msgr2=0x555a033e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:08.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a033e6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fcecb59e640 1 -- 172.21.15.69:0/1362971691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a032fdbc0 con 0x555a033e93e0 2024-08-02T17:48:08.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a033e6da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x555a033df140 tx=0x7fcebc004a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1023b327eeff93c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:08.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fceaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:08.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fceaeff5640 1 -- 172.21.15.69:0/1362971691 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc09f070 con 0x555a033e93e0 2024-08-02T17:48:08.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fced350e740 1 -- 172.21.15.69:0/1362971691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcebc0a4a50 con 0x555a033e93e0 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.352+0000 7fceaeff5640 1 -- 172.21.15.69:0/1362971691 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcebc098450 con 0x555a033e93e0 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.353+0000 7fceaeff5640 1 -- 172.21.15.69:0/1362971691 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcebc0a45c0 con 0x555a033e93e0 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.353+0000 7fced350e740 10 client.25405 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.353+0000 7fced350e740 1 -- 172.21.15.69:0/1362971691 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a034006d0 con 0x555a033e93e0 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.353+0000 7fceaeff5640 1 -- 172.21.15.69:0/1362971691 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc0a4a50 con 0x555a033e93e0 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.353+0000 7fceaeff5640 1 -- 172.21.15.69:0/1362971691 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcebc09adb0 con 0x555a033e93e0 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.354+0000 7fced350e740 10 client.25405 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.354+0000 7fced350e740 10 client.25405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.354+0000 7fced350e740 20 client.25405 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.354+0000 7fced350e740 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a03401a20 0x555a03403e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.354+0000 7fced350e740 4 client.25405 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:08.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.354+0000 7fced350e740 1 -- 172.21.15.69:0/1362971691 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a03350d60 con 0x555a03401a20 2024-08-02T17:48:08.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.355+0000 7fcecad9d640 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a03401a20 0x555a03403e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.355+0000 7fcecad9d640 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a03401a20 0x555a03403e10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d73bbc174887d28f server_cookie=a5cbcf6eb81238b2 in_seq=0 out_seq=0 2024-08-02T17:48:08.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.355+0000 7fceaeff5640 10 client.25405 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:08.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.356+0000 7fceaeff5640 1 -- 172.21.15.69:0/1362971691 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a03350d60 con 0x555a03401a20 2024-08-02T17:48:08.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.356+0000 7fceaeff5640 10 client.25405 handle_command_reply: tid=0 2024-08-02T17:48:08.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.356+0000 7fced350e740 1 client.25405 shutdown 2024-08-02T17:48:08.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.356+0000 7fced350e740 1 -- 172.21.15.69:0/1362971691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 msgr2=0x555a033e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.356+0000 7fced350e740 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a033e6da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x555a033df140 tx=0x7fcebc004a50 comp rx=0 tx=0).stop 2024-08-02T17:48:08.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 1 -- 172.21.15.69:0/1362971691 shutdown_connections 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a033e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a03401a20 0x555a03403e10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a033e6da0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 1 --2- 172.21.15.69:0/1362971691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a033e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fceaeff5640 0 client.25405 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 1 -- 172.21.15.69:0/1362971691 >> 172.21.15.69:0/1362971691 conn(0x555a033da790 msgr2=0x555a033e04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 1 -- 172.21.15.69:0/1362971691 shutdown_connections 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 1 -- 172.21.15.69:0/1362971691 wait complete. 2024-08-02T17:48:08.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.357+0000 7fced350e740 20 client.25405 trim_cache size 0 max 16384 2024-08-02T17:48:08.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.387+0000 7fced350e740 1 Processor -- start 2024-08-02T17:48:08.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.388+0000 7fced350e740 1 -- start start 2024-08-02T17:48:08.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.388+0000 7fced350e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.388+0000 7fced350e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033deb20 0x555a03406a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.388+0000 7fced350e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a03406f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.388+0000 7fced350e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1af5b0 con 0x555a033de200 2024-08-02T17:48:08.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.388+0000 7fced350e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1ace80 con 0x555a033deb20 2024-08-02T17:48:08.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.388+0000 7fced350e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1aacf0 con 0x555a033e93e0 2024-08-02T17:48:08.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36958/0 (socket says 172.21.15.69:36958) 2024-08-02T17:48:08.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 -- 172.21.15.69:0/3488851618 learned_addr learned my addr 172.21.15.69:0/3488851618 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:08.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecb59e640 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a03406f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fceca59c640 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033deb20 0x555a03406a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 -- 172.21.15.69:0/3488851618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 msgr2=0x555a03406f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:08.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a03406f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 -- 172.21.15.69:0/3488851618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033deb20 msgr2=0x555a03406a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:08.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033deb20 0x555a03406a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 -- 172.21.15.69:0/3488851618 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a033e76a0 con 0x555a033de200 2024-08-02T17:48:08.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecb59e640 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a03406f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:08.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.389+0000 7fcecad9d640 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033e81f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcec0005d00 tx=0x7fcec0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa6d317d23f0c66 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:08.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.390+0000 7fceb0ff9640 1 -- 172.21.15.69:0/3488851618 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec000f480 con 0x555a033de200 2024-08-02T17:48:08.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.390+0000 7fceb0ff9640 1 -- 172.21.15.69:0/3488851618 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec000fb00 con 0x555a033de200 2024-08-02T17:48:08.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.390+0000 7fceb0ff9640 1 -- 172.21.15.69:0/3488851618 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec001db70 con 0x555a033de200 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.391+0000 7fced350e740 1 -- 172.21.15.69:0/3488851618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 msgr2=0x555a033e81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.391+0000 7fced350e740 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033e81f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcec0005d00 tx=0x7fcec0014000 comp rx=0 tx=0).stop 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 -- 172.21.15.69:0/3488851618 shutdown_connections 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a03406f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033deb20 0x555a03406a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 --2- 172.21.15.69:0/3488851618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033de200 0x555a033e81f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 -- 172.21.15.69:0/3488851618 >> 172.21.15.69:0/3488851618 conn(0x555a033da790 msgr2=0x555a033e5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 -- 172.21.15.69:0/3488851618 shutdown_connections 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 -- 172.21.15.69:0/3488851618 wait complete. 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 -- start start 2024-08-02T17:48:08.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.392+0000 7fced350e740 1 Processor -- start 2024-08-02T17:48:08.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fced350e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a03407050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fced350e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a034043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fced350e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a034048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fced350e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1ab340 con 0x555a033deb20 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fcecb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a034048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fcecb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a034048e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42800/0 (socket says 172.21.15.69:42800) 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fcecb59e640 1 -- 172.21.15.69:0/1380267242 learned_addr learned my addr 172.21.15.69:0/1380267242 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1ac7b0 con 0x555a033de200 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fceca59c640 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a034043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1aba10 con 0x555a033e93e0 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.393+0000 7fced350e740 10 client.0 _ll_register_callbacks cb 0x7fcebc09b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fcecad9d640 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a03407050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fcecb59e640 1 -- 172.21.15.69:0/1380267242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 msgr2=0x555a03407050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:08.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a03407050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fcecb59e640 1 -- 172.21.15.69:0/1380267242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 msgr2=0x555a034043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a034043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fcecb59e640 1 -- 172.21.15.69:0/1380267242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a03404f80 con 0x555a033e93e0 2024-08-02T17:48:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fceca59c640 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a034043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fcecb59e640 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a034048e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0a1370 tx=0x7fcebc09b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26f0211622587d99 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:08.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.394+0000 7fceaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:08.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.395+0000 7fceaeff5640 1 -- 172.21.15.69:0/1380267242 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc09f070 con 0x555a033e93e0 2024-08-02T17:48:08.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.395+0000 7fceaeff5640 1 -- 172.21.15.69:0/1380267242 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcebc09aca0 con 0x555a033e93e0 2024-08-02T17:48:08.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.395+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec000fb00 con 0x555a033e93e0 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.396+0000 7fceaeff5640 1 -- 172.21.15.69:0/1380267242 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcebc0a4a50 con 0x555a033e93e0 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.396+0000 7fced350e740 10 client.25411 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.396+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a03404f80 con 0x555a033e93e0 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.396+0000 7fceaeff5640 1 -- 172.21.15.69:0/1380267242 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc09aea0 con 0x555a033e93e0 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.397+0000 7fceaeff5640 1 -- 172.21.15.69:0/1380267242 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcebc0cd760 con 0x555a033e93e0 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.397+0000 7fced350e740 10 client.25411 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.397+0000 7fced350e740 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.397+0000 7fced350e740 20 client.25411 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.397+0000 7fced350e740 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a0346c510 0x555a0346e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.397+0000 7fced350e740 4 client.25411 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.397+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fcec000fb00 con 0x555a0346c510 2024-08-02T17:48:08.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.397+0000 7fcecad9d640 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a0346c510 0x555a0346e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:08.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.398+0000 7fcecad9d640 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a0346c510 0x555a0346e900 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406e04376bf6a595 server_cookie=f1440ae38f4159e0 in_seq=0 out_seq=0 2024-08-02T17:48:08.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.398+0000 7fceaeff5640 10 client.25411 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:08.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.399+0000 7fceaeff5640 1 -- 172.21.15.69:0/1380267242 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fcec000fb00 con 0x555a0346c510 2024-08-02T17:48:08.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.399+0000 7fceaeff5640 10 client.25411 handle_command_reply: tid=0 2024-08-02T17:48:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.399+0000 7fced350e740 1 client.25411 shutdown 2024-08-02T17:48:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 msgr2=0x555a034048e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a034048e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0a1370 tx=0x7fcebc09b800 comp rx=0 tx=0).stop 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 shutdown_connections 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a033deb20 0x555a034043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a0346c510 0x555a0346e900 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a033e93e0 0x555a034048e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 --2- 172.21.15.69:0/1380267242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a033de200 0x555a03407050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 >> 172.21.15.69:0/1380267242 conn(0x555a033da790 msgr2=0x555a033f4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fceaeff5640 0 client.25411 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 shutdown_connections 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 1 -- 172.21.15.69:0/1380267242 wait complete. 2024-08-02T17:48:08.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.400+0000 7fced350e740 20 client.25411 trim_cache size 0 max 16384 2024-08-02T17:48:08.403 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:48:08.403 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:48:08.403 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "99930d6d-669e-42fe-8edf-ef49ee1fd42e", 2024-08-02T17:48:08.403 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:48:08.403 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:48:08.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 -- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fceb40a2e30 msgr2=0x7fceb40a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fceb40a2e30 0x7fceb40a52f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009fc0 tx=0x7fcec0005f90 comp rx=0 tx=0).stop 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 -- 172.21.15.69:0/2337391641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc102c20 msgr2=0x7fcecc19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc102c20 0x7fcecc19edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00e970 tx=0x7fcebc00ed70 comp rx=0 tx=0).stop 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 -- 172.21.15.69:0/2337391641 shutdown_connections 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcecc1022a0 0x7fcecc19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fceb40a2e30 0x7fceb40a52f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcecc102c20 0x7fcecc19edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 --2- 172.21.15.69:0/2337391641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcecc100b50 0x7fcecc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 -- 172.21.15.69:0/2337391641 >> 172.21.15.69:0/2337391641 conn(0x7fcecc0fc500 msgr2=0x7fcecc104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 -- 172.21.15.69:0/2337391641 shutdown_connections 2024-08-02T17:48:08.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:08.401+0000 7fced1110640 1 -- 172.21.15.69:0/2337391641 wait complete. 2024-08-02T17:48:09.112 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:48:09.337 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:09 smithi069 ceph-mon[32746]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:48:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:09 smithi077 ceph-mon[37108]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:48:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.466+0000 7facf3a46640 1 -- 172.21.15.69:0/974946305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec1007c0 msgr2=0x7facec100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.466+0000 7facf3a46640 1 --2- 172.21.15.69:0/974946305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec1007c0 0x7facec100ba0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7facdc009900 tx=0x7facdc02a480 comp rx=0 tx=0).stop 2024-08-02T17:48:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.466+0000 7facf3a46640 1 -- 172.21.15.69:0/974946305 shutdown_connections 2024-08-02T17:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.466+0000 7facf3a46640 1 --2- 172.21.15.69:0/974946305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec105bf0 0x7facec106060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.466+0000 7facf3a46640 1 --2- 172.21.15.69:0/974946305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facec1010e0 0x7facec1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.466+0000 7facf3a46640 1 --2- 172.21.15.69:0/974946305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec1007c0 0x7facec100ba0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.466+0000 7facf3a46640 1 -- 172.21.15.69:0/974946305 >> 172.21.15.69:0/974946305 conn(0x7facec0fc3e0 msgr2=0x7facec0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.467+0000 7facf3a46640 1 -- 172.21.15.69:0/974946305 shutdown_connections 2024-08-02T17:48:09.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.467+0000 7facf3a46640 1 -- 172.21.15.69:0/974946305 wait complete. 2024-08-02T17:48:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.467+0000 7facf3a46640 1 Processor -- start 2024-08-02T17:48:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.467+0000 7facf3a46640 1 -- start start 2024-08-02T17:48:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf3a46640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facec1007c0 0x7facec19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf3a46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1010e0 0x7facec19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf3a46640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec105bf0 0x7facec1a3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf3a46640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7facec118230 con 0x7facec105bf0 2024-08-02T17:48:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf3a46640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7facec1180b0 con 0x7facec1007c0 2024-08-02T17:48:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf3a46640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facec1183b0 con 0x7facec1010e0 2024-08-02T17:48:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf1fe3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec105bf0 0x7facec1a3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf1fe3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec105bf0 0x7facec1a3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36990/0 (socket says 172.21.15.69:36990) 2024-08-02T17:48:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf1fe3640 1 -- 172.21.15.69:0/3485677087 learned_addr learned my addr 172.21.15.69:0/3485677087 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf0fe1640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1010e0 0x7facec19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.468+0000 7facf17e2640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facec1007c0 0x7facec19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.469+0000 7facf1fe3640 1 -- 172.21.15.69:0/3485677087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1010e0 msgr2=0x7facec19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.469+0000 7facf1fe3640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1010e0 0x7facec19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.469+0000 7facf1fe3640 1 -- 172.21.15.69:0/3485677087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facec1007c0 msgr2=0x7facec19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.469+0000 7facf1fe3640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facec1007c0 0x7facec19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.469+0000 7facf1fe3640 1 -- 172.21.15.69:0/3485677087 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec1a3730 con 0x7facec105bf0 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.469+0000 7facf0fe1640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1010e0 0x7facec19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.469+0000 7facf17e2640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facec1007c0 0x7facec19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.469+0000 7facf1fe3640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec105bf0 0x7facec1a3000 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7face400eb10 tx=0x7face400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.470+0000 7facea7fc640 1 -- 172.21.15.69:0/3485677087 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4004b20 con 0x7facec105bf0 2024-08-02T17:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.470+0000 7facea7fc640 1 -- 172.21.15.69:0/3485677087 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face4004cc0 con 0x7facec105bf0 2024-08-02T17:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.470+0000 7facea7fc640 1 -- 172.21.15.69:0/3485677087 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4010760 con 0x7facec105bf0 2024-08-02T17:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.470+0000 7facf3a46640 1 -- 172.21.15.69:0/3485677087 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facec1a3a20 con 0x7facec105bf0 2024-08-02T17:48:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.470+0000 7facf3a46640 1 -- 172.21.15.69:0/3485677087 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facec1ab2b0 con 0x7facec105bf0 2024-08-02T17:48:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.474+0000 7facf67a6740 1 Processor -- start 2024-08-02T17:48:09.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.474+0000 7facea7fc640 1 -- 172.21.15.69:0/3485677087 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7face4005020 con 0x7facec105bf0 2024-08-02T17:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.475+0000 7facf67a6740 1 -- start start 2024-08-02T17:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.475+0000 7facf67a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 0x5561f87dbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.475+0000 7facf67a6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.475+0000 7facf67a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf67a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5561f873f9b0 con 0x5561f87db7b0 2024-08-02T17:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf67a6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5561f86e7800 con 0x5561f87dc0d0 2024-08-02T17:48:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf67a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561f873b970 con 0x5561f87e6970 2024-08-02T17:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43882/0 (socket says 172.21.15.69:43882) 2024-08-02T17:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 -- 172.21.15.69:0/3749451238 learned_addr learned my addr 172.21.15.69:0/3749451238 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facea7fc640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7facd40a2e40 0x7facd40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf0fe1640 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf0fe1640 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 0x5561f87dbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 -- 172.21.15.69:0/3749451238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 msgr2=0x5561f87e8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 -- 172.21.15.69:0/3749451238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 msgr2=0x5561f87dbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 0x5561f87dbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 -- 172.21.15.69:0/3749451238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f8453210 con 0x5561f87dc0d0 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facea7fc640 1 -- 172.21.15.69:0/3485677087 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7face40cfd90 con 0x7facec105bf0 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf1fe3640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7facd40a2e40 0x7facd40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.476+0000 7facf17e2640 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e6430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facdc009230 tx=0x7facdc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae418e23eab24257 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.477+0000 7facd37fe640 1 -- 172.21.15.69:0/3749451238 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc035030 con 0x5561f87dc0d0 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.477+0000 7facd37fe640 1 -- 172.21.15.69:0/3749451238 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facdc005520 con 0x5561f87dc0d0 2024-08-02T17:48:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.477+0000 7facd37fe640 1 -- 172.21.15.69:0/3749451238 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0035f0 con 0x5561f87dc0d0 2024-08-02T17:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.478+0000 7facf67a6740 1 -- 172.21.15.69:0/3749451238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 msgr2=0x5561f87e6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.478+0000 7facf67a6740 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e6430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facdc009230 tx=0x7facdc0040c0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.478+0000 7facf1fe3640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7facd40a2e40 0x7facd40a5300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7face40cd1d0 tx=0x7face4005b40 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.478+0000 7facf67a6740 1 -- 172.21.15.69:0/3749451238 shutdown_connections 2024-08-02T17:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.478+0000 7facf67a6740 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.478+0000 7facf67a6740 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e6430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.478+0000 7facf67a6740 1 --2- 172.21.15.69:0/3749451238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 0x5561f87dbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.478+0000 7facf67a6740 1 -- 172.21.15.69:0/3749451238 >> 172.21.15.69:0/3749451238 conn(0x5561f87d7d20 msgr2=0x5561f87d8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.479+0000 7facf67a6740 1 -- 172.21.15.69:0/3749451238 shutdown_connections 2024-08-02T17:48:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.479+0000 7facf67a6740 1 -- 172.21.15.69:0/3749451238 wait complete. 2024-08-02T17:48:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.479+0000 7facf67a6740 1 -- start start 2024-08-02T17:48:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.479+0000 7facf67a6740 1 Processor -- start 2024-08-02T17:48:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.480+0000 7facf67a6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.480+0000 7facf67a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87dc0d0 0x5561f87e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.480+0000 7facf67a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.480+0000 7facf67a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5561f87e9540 con 0x5561f87dc0d0 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.480+0000 7facf67a6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5561f86fb500 con 0x5561f87db7b0 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.480+0000 7facf67a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5561f87e96c0 con 0x5561f87e6970 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.480+0000 7facf67a6740 10 client.0 _ll_register_callbacks cb 0x5561f8452a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf17e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87dc0d0 0x5561f87e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf17e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87dc0d0 0x5561f87e3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37014/0 (socket says 172.21.15.69:37014) 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf17e2640 1 -- 172.21.15.69:0/3551661058 learned_addr learned my addr 172.21.15.69:0/3551661058 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf0fe1640 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf0fe1640 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf17e2640 1 -- 172.21.15.69:0/3551661058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 msgr2=0x5561f87e4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf17e2640 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf17e2640 1 -- 172.21.15.69:0/3551661058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 msgr2=0x5561f87e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf17e2640 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.481+0000 7facf17e2640 1 -- 172.21.15.69:0/3551661058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f851f620 con 0x5561f87dc0d0 2024-08-02T17:48:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.482+0000 7facf17e2640 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87dc0d0 0x5561f87e3e00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7facdc02aa70 tx=0x7facdc0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115ccb4300d65dd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.482+0000 7facd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.482+0000 7facd0ff9640 1 -- 172.21.15.69:0/3551661058 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc035070 con 0x5561f87dc0d0 2024-08-02T17:48:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.482+0000 7facd0ff9640 1 -- 172.21.15.69:0/3551661058 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7facdc004f40 con 0x5561f87dc0d0 2024-08-02T17:48:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.483+0000 7facf67a6740 1 -- 172.21.15.69:0/3551661058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facdc005520 con 0x5561f87dc0d0 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.483+0000 7facd0ff9640 1 -- 172.21.15.69:0/3551661058 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7facdc005140 con 0x5561f87dc0d0 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.483+0000 7facf67a6740 10 client.15915 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.483+0000 7facf67a6740 1 -- 172.21.15.69:0/3551661058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561f87f51e0 con 0x5561f87dc0d0 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.484+0000 7facd0ff9640 1 -- 172.21.15.69:0/3551661058 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facdc005520 con 0x5561f87dc0d0 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.484+0000 7facd0ff9640 1 -- 172.21.15.69:0/3551661058 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7facdc0038d0 con 0x5561f87dc0d0 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.484+0000 7facf67a6740 10 client.15915 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.484+0000 7facf67a6740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.484+0000 7facf67a6740 20 client.15915 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.484+0000 7facf67a6740 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5561f87f6530 0x5561f87f8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.484+0000 7facf67a6740 4 client.15915 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.484+0000 7facf67a6740 1 -- 172.21.15.69:0/3551661058 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561f874d0e0 con 0x5561f87f6530 2024-08-02T17:48:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.485+0000 7facf0fe1640 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5561f87f6530 0x5561f87f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.485+0000 7facd0ff9640 10 client.15915 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.485+0000 7facf0fe1640 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5561f87f6530 0x5561f87f8920 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f4cf685ee82ef2 server_cookie=9cde08c076ab7598 in_seq=0 out_seq=0 2024-08-02T17:48:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facd0ff9640 1 -- 172.21.15.69:0/3551661058 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561f874d0e0 con 0x5561f87f6530 2024-08-02T17:48:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facd0ff9640 10 client.15915 handle_command_reply: tid=0 2024-08-02T17:48:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 client.15915 shutdown 2024-08-02T17:48:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 -- 172.21.15.69:0/3551661058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87dc0d0 msgr2=0x5561f87e3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87dc0d0 0x5561f87e3e00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7facdc02aa70 tx=0x7facdc0045a0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 -- 172.21.15.69:0/3551661058 shutdown_connections 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87dc0d0 0x5561f87e3e00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5561f87f6530 0x5561f87f8920 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 --2- 172.21.15.69:0/3551661058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.486+0000 7facf67a6740 1 -- 172.21.15.69:0/3551661058 >> 172.21.15.69:0/3551661058 conn(0x5561f87d7d20 msgr2=0x5561f87dda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.487+0000 7facd0ff9640 0 client.15915 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.487+0000 7facf67a6740 1 -- 172.21.15.69:0/3551661058 shutdown_connections 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.487+0000 7facf67a6740 1 -- 172.21.15.69:0/3551661058 wait complete. 2024-08-02T17:48:09.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.487+0000 7facf67a6740 20 client.15915 trim_cache size 0 max 16384 2024-08-02T17:48:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:09 smithi083 ceph-mon[37031]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:48:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.525+0000 7facf67a6740 1 Processor -- start 2024-08-02T17:48:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.526+0000 7facf67a6740 1 -- start start 2024-08-02T17:48:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.526+0000 7facf67a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 0x5561f87e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.526+0000 7facf67a6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf67a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf67a6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7facec1b7dc0 con 0x5561f87db7b0 2024-08-02T17:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf67a6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7facec1b5690 con 0x5561f87dc0d0 2024-08-02T17:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf17e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf67a6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facec1b34e0 con 0x5561f87e6970 2024-08-02T17:48:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf17e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43900/0 (socket says 172.21.15.69:43900) 2024-08-02T17:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf17e2640 1 -- 172.21.15.69:0/1601988555 learned_addr learned my addr 172.21.15.69:0/1601988555 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf17e2640 1 -- 172.21.15.69:0/1601988555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 msgr2=0x5561f87e5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf17e2640 1 --2- 172.21.15.69:0/1601988555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf17e2640 1 -- 172.21.15.69:0/1601988555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 msgr2=0x5561f87e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf17e2640 1 --2- 172.21.15.69:0/1601988555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 0x5561f87e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.527+0000 7facf17e2640 1 -- 172.21.15.69:0/1601988555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561f87dbf00 con 0x5561f87dc0d0 2024-08-02T17:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.528+0000 7facf17e2640 1 --2- 172.21.15.69:0/1601988555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e1040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7facdc009230 tx=0x7facdc004cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef88237de791014 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.528+0000 7facd2ffd640 1 -- 172.21.15.69:0/1601988555 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc035040 con 0x5561f87dc0d0 2024-08-02T17:48:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.530+0000 7facd2ffd640 1 -- 172.21.15.69:0/1601988555 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facdc003ad0 con 0x5561f87dc0d0 2024-08-02T17:48:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.530+0000 7facd2ffd640 1 -- 172.21.15.69:0/1601988555 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc031810 con 0x5561f87dc0d0 2024-08-02T17:48:09.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.533+0000 7facf67a6740 1 -- 172.21.15.69:0/1601988555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 msgr2=0x5561f87e1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.533+0000 7facf67a6740 1 --2- 172.21.15.69:0/1601988555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e1040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7facdc009230 tx=0x7facdc004cc0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.535+0000 7facf67a6740 1 -- 172.21.15.69:0/1601988555 shutdown_connections 2024-08-02T17:48:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.536+0000 7facf67a6740 1 --2- 172.21.15.69:0/1601988555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87e6970 0x5561f87e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.536+0000 7facf67a6740 1 --2- 172.21.15.69:0/1601988555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87dc0d0 0x5561f87e1040 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.536+0000 7facf67a6740 1 --2- 172.21.15.69:0/1601988555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87db7b0 0x5561f87e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.536+0000 7facf67a6740 1 -- 172.21.15.69:0/1601988555 >> 172.21.15.69:0/1601988555 conn(0x5561f87d7d20 msgr2=0x5561f87debb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.543+0000 7facf67a6740 1 -- 172.21.15.69:0/1601988555 shutdown_connections 2024-08-02T17:48:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.543+0000 7facf67a6740 1 -- 172.21.15.69:0/1601988555 wait complete. 2024-08-02T17:48:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.543+0000 7facf67a6740 1 -- start start 2024-08-02T17:48:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.544+0000 7facf67a6740 1 Processor -- start 2024-08-02T17:48:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.544+0000 7facf67a6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87fb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf67a6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87dc0d0 0x5561f87f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf67a6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87e6970 0x5561f87f9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf0fe1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf0fe1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43906/0 (socket says 172.21.15.69:43906) 2024-08-02T17:48:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf0fe1640 1 -- 172.21.15.69:0/2327640883 learned_addr learned my addr 172.21.15.69:0/2327640883 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7facec1b3b50 con 0x5561f87e6970 2024-08-02T17:48:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7facec1b4fc0 con 0x5561f87db7b0 2024-08-02T17:48:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7facec1b4220 con 0x5561f87dc0d0 2024-08-02T17:48:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.545+0000 7facf67a6740 10 client.0 _ll_register_callbacks cb 0x7facdc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.547+0000 7facf17e2640 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87dc0d0 0x5561f87f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.548+0000 7facf0fe1640 1 -- 172.21.15.69:0/2327640883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87dc0d0 msgr2=0x5561f87f8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.548+0000 7facf0fe1640 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87dc0d0 0x5561f87f8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.548+0000 7facf0fe1640 1 -- 172.21.15.69:0/2327640883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87e6970 msgr2=0x5561f87f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.548+0000 7facf0fe1640 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87e6970 0x5561f87f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.548+0000 7facf0fe1640 1 -- 172.21.15.69:0/2327640883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561f87f9790 con 0x5561f87db7b0 2024-08-02T17:48:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.550+0000 7facf0fe1640 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87fb980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7face000bc00 tx=0x7face00039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a39080e239bca2f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.550+0000 7facd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.550+0000 7facd0ff9640 1 -- 172.21.15.69:0/2327640883 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0005020 con 0x5561f87db7b0 2024-08-02T17:48:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.550+0000 7facd0ff9640 1 -- 172.21.15.69:0/2327640883 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7face00051c0 con 0x5561f87db7b0 2024-08-02T17:48:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.552+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facdc003ad0 con 0x5561f87db7b0 2024-08-02T17:48:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.552+0000 7facd0ff9640 1 -- 172.21.15.69:0/2327640883 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7face00053c0 con 0x5561f87db7b0 2024-08-02T17:48:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.552+0000 7facf67a6740 10 client.25541 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.552+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5561f8860bb0 con 0x5561f87db7b0 2024-08-02T17:48:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.553+0000 7facd0ff9640 1 -- 172.21.15.69:0/2327640883 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facdc003ad0 con 0x5561f87db7b0 2024-08-02T17:48:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.554+0000 7facd0ff9640 1 -- 172.21.15.69:0/2327640883 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7face0020730 con 0x5561f87db7b0 2024-08-02T17:48:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.554+0000 7facf67a6740 10 client.25541 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.554+0000 7facf67a6740 10 client.25541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.554+0000 7facf67a6740 20 client.25541 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.554+0000 7facf67a6740 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5561f8861350 0x5561f8863770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.554+0000 7facf67a6740 4 client.25541 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:48:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.554+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561f8747430 con 0x5561f8861350 2024-08-02T17:48:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.560+0000 7facf17e2640 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5561f8861350 0x5561f8863770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.564+0000 7facf17e2640 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5561f8861350 0x5561f8863770 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682a02491b51b731 server_cookie=a0bcbdf78bc7421c in_seq=0 out_seq=0 2024-08-02T17:48:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.564+0000 7facd0ff9640 10 client.25541 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.593+0000 7facd0ff9640 1 -- 172.21.15.69:0/2327640883 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5561f8747430 con 0x5561f8861350 2024-08-02T17:48:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.593+0000 7facd0ff9640 10 client.25541 handle_command_reply: tid=0 2024-08-02T17:48:09.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.593+0000 7facf67a6740 1 client.25541 shutdown 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 msgr2=0x5561f87fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87fb980 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7face000bc00 tx=0x7face00039c0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 shutdown_connections 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5561f87e6970 0x5561f87f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5561f8861350 0x5561f8863770 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5561f87dc0d0 0x5561f87f8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 --2- 172.21.15.69:0/2327640883 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5561f87db7b0 0x5561f87fb980 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 >> 172.21.15.69:0/2327640883 conn(0x5561f87d7d20 msgr2=0x5561f87e3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:09.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facd0ff9640 0 client.25541 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 shutdown_connections 2024-08-02T17:48:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 1 -- 172.21.15.69:0/2327640883 wait complete. 2024-08-02T17:48:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.594+0000 7facf67a6740 20 client.25541 trim_cache size 0 max 16384 2024-08-02T17:48:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:48:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:48:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:48:09.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 -- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7facd40a2e40 msgr2=0x7facd40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7facd40a2e40 0x7facd40a5300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7face40cd1d0 tx=0x7face4005b40 comp rx=0 tx=0).stop 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 -- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec105bf0 msgr2=0x7facec1a3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec105bf0 0x7facec1a3000 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7face400eb10 tx=0x7face400c2d0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 -- 172.21.15.69:0/3485677087 shutdown_connections 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7facec105bf0 0x7facec1a3000 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7facd40a2e40 0x7facd40a5300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7facec1010e0 0x7facec19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 --2- 172.21.15.69:0/3485677087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7facec1007c0 0x7facec19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 -- 172.21.15.69:0/3485677087 >> 172.21.15.69:0/3485677087 conn(0x7facec0fc3e0 msgr2=0x7facec10f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 -- 172.21.15.69:0/3485677087 shutdown_connections 2024-08-02T17:48:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:09.597+0000 7facf3a46640 1 -- 172.21.15.69:0/3485677087 wait complete. 2024-08-02T17:48:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 60.0 s, 4162.8 tps, lat 23.954 ms stddev 34.314 2024-08-02T17:48:10.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:10 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:48:10.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:10 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:10.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:10 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:48:10.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:10 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:10.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:10 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:48:10.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:10 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:48:10.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:10 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:10.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:10 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:48:10.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:10 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:10.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:10 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:48:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:10 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:48:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:10 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:10 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:48:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:10 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:10 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:48:11.205 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:11 smithi069 ceph-mon[32746]: pgmap v56: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-08-02T17:48:11.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:11 smithi077 ceph-mon[37108]: pgmap v56: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-08-02T17:48:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:11 smithi083 ceph-mon[37031]: pgmap v56: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 17 MiB/s wr, 1.49k op/s 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.120+0000 7f74db55a640 1 -- 172.21.15.69:0/1944709497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d411ae70 msgr2=0x7f74d411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.120+0000 7f74db55a640 1 --2- 172.21.15.69:0/1944709497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d411ae70 0x7f74d411d260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009ca0 tx=0x7f74cc02a8b0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.124+0000 7f74db55a640 1 -- 172.21.15.69:0/1944709497 shutdown_connections 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.124+0000 7f74db55a640 1 --2- 172.21.15.69:0/1944709497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74d4109560 0x7f74d41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.124+0000 7f74db55a640 1 --2- 172.21.15.69:0/1944709497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d411ae70 0x7f74d411d260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.124+0000 7f74db55a640 1 --2- 172.21.15.69:0/1944709497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d4109180 0x7f74d410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.124+0000 7f74db55a640 1 -- 172.21.15.69:0/1944709497 >> 172.21.15.69:0/1944709497 conn(0x7f74d406de70 msgr2=0x7f74d406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.125+0000 7f74db55a640 1 -- 172.21.15.69:0/1944709497 shutdown_connections 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.127+0000 7f74db55a640 1 -- 172.21.15.69:0/1944709497 wait complete. 2024-08-02T17:48:11.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.128+0000 7f74db55a640 1 Processor -- start 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.128+0000 7f74db55a640 1 -- start start 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.128+0000 7f74db55a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4109180 0x7f74d4118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.128+0000 7f74db55a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74d4109560 0x7f74d4110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.128+0000 7f74db55a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d411ae70 0x7f74d4110c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.128+0000 7f74db55a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4120af0 con 0x7f74d4109560 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.128+0000 7f74db55a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4120970 con 0x7f74d4109180 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.128+0000 7f74db55a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74d4120c70 con 0x7f74d411ae70 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.129+0000 7f74d8af5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74d4109560 0x7f74d4110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.129+0000 7f74d8af5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74d4109560 0x7f74d4110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37050/0 (socket says 172.21.15.69:37050) 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.129+0000 7f74d8af5640 1 -- 172.21.15.69:0/3173876668 learned_addr learned my addr 172.21.15.69:0/3173876668 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.129+0000 7f74d92f6640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4109180 0x7f74d4118750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.129+0000 7f74d9af7640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d411ae70 0x7f74d4110c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.130+0000 7f74d8af5640 1 -- 172.21.15.69:0/3173876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d411ae70 msgr2=0x7f74d4110c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.130+0000 7f74d8af5640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d411ae70 0x7f74d4110c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.130+0000 7f74d8af5640 1 -- 172.21.15.69:0/3173876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4109180 msgr2=0x7f74d4118750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.130+0000 7f74d8af5640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4109180 0x7f74d4118750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.130+0000 7f74d8af5640 1 -- 172.21.15.69:0/3173876668 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d4111480 con 0x7f74d4109560 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.130+0000 7f74d92f6640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4109180 0x7f74d4118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.130+0000 7f74d9af7640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d411ae70 0x7f74d4110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.130+0000 7f74d8af5640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74d4109560 0x7f74d4110720 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b4a0 tx=0x7f74c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.131+0000 7f74c67fc640 1 -- 172.21.15.69:0/3173876668 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8014070 con 0x7f74d4109560 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.131+0000 7f74c67fc640 1 -- 172.21.15.69:0/3173876668 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c8010af0 con 0x7f74d4109560 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.131+0000 7f74c67fc640 1 -- 172.21.15.69:0/3173876668 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c8017d10 con 0x7f74d4109560 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.131+0000 7f74db55a640 1 -- 172.21.15.69:0/3173876668 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d41bbe00 con 0x7f74d4109560 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.131+0000 7f74db55a640 1 -- 172.21.15.69:0/3173876668 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d41bc260 con 0x7f74d4109560 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.135+0000 7f74dd958740 1 Processor -- start 2024-08-02T17:48:11.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.136+0000 7f74dd958740 1 -- start start 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.136+0000 7f74dd958740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89210 0x557279c895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.136+0000 7f74dd958740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89b30 0x557279c93eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.136+0000 7f74dd958740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 0x557279c967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.136+0000 7f74dd958740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557279bee980 con 0x557279c89210 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.136+0000 7f74dd958740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557279b86a70 con 0x557279c89b30 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.136+0000 7f74dd958740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557279bea940 con 0x557279c943f0 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.136+0000 7f74c67fc640 1 -- 172.21.15.69:0/3173876668 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f74c8017070 con 0x7f74d4109560 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89b30 0x557279c93eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89b30 0x557279c93eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43950/0 (socket says 172.21.15.69:43950) 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 -- 172.21.15.69:0/2266385045 learned_addr learned my addr 172.21.15.69:0/2266385045 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d9af7640 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89210 0x557279c895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d8af5640 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 0x557279c967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 -- 172.21.15.69:0/2266385045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 msgr2=0x557279c967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 0x557279c967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 -- 172.21.15.69:0/2266385045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89210 msgr2=0x557279c895f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89210 0x557279c895f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 -- 172.21.15.69:0/2266385045 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557279be96e0 con 0x557279c89b30 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d8af5640 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 0x557279c967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74d92f6640 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89b30 0x557279c93eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74d000f3c0 tx=0x7f74d000f880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6363721f1e268827 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.137+0000 7f74bf7fe640 1 -- 172.21.15.69:0/2266385045 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d001d070 con 0x557279c89b30 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.138+0000 7f74bf7fe640 1 -- 172.21.15.69:0/2266385045 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74d0010de0 con 0x557279c89b30 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.138+0000 7f74bf7fe640 1 -- 172.21.15.69:0/2266385045 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0016340 con 0x557279c89b30 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.138+0000 7f74c67fc640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f74c00a2f00 0x7f74c00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.138+0000 7f74c67fc640 1 -- 172.21.15.69:0/3173876668 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f74c80ce990 con 0x7f74d4109560 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.138+0000 7f74d9af7640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f74c00a2f00 0x7f74c00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.138+0000 7f74dd958740 1 -- 172.21.15.69:0/2266385045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89b30 msgr2=0x557279c93eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.138+0000 7f74dd958740 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89b30 0x557279c93eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74d000f3c0 tx=0x7f74d000f880 comp rx=0 tx=0).stop 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74dd958740 1 -- 172.21.15.69:0/2266385045 shutdown_connections 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74dd958740 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 0x557279c967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74dd958740 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89b30 0x557279c93eb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74dd958740 1 --2- 172.21.15.69:0/2266385045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89210 0x557279c895f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74dd958740 1 -- 172.21.15.69:0/2266385045 >> 172.21.15.69:0/2266385045 conn(0x557279c857a0 msgr2=0x557279c85b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74dd958740 1 -- 172.21.15.69:0/2266385045 shutdown_connections 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74d9af7640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f74c00a2f00 0x7f74c00a53c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009bf0 tx=0x7f74cc0314d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74dd958740 1 -- 172.21.15.69:0/2266385045 wait complete. 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.139+0000 7f74dd958740 1 -- start start 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.140+0000 7f74dd958740 1 Processor -- start 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.140+0000 7f74dd958740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c91280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74dd958740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c917c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d92f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c91280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d92f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c91280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43952/0 (socket says 172.21.15.69:43952) 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d92f6640 1 -- 172.21.15.69:0/301294972 learned_addr learned my addr 172.21.15.69:0/301294972 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74dd958740 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 0x557279c91d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557279c96fc0 con 0x557279c89b30 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557279ba9aa0 con 0x557279c89210 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d8af5640 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c917c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557279c97140 con 0x557279c943f0 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74dd958740 10 client.0 _ll_register_callbacks cb 0x557279b98e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d92f6640 1 -- 172.21.15.69:0/301294972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 msgr2=0x557279c91d00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d92f6640 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 0x557279c91d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d92f6640 1 -- 172.21.15.69:0/301294972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 msgr2=0x557279c917c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:11.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d92f6640 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c917c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.141+0000 7f74d92f6640 1 -- 172.21.15.69:0/301294972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557279b8d650 con 0x557279c89210 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74d8af5640 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74d92f6640 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c91280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f74d0010fd0 tx=0x7f74d0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16dc4ad155fa420 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d0010de0 con 0x557279c89210 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74bcff9640 10 client.25553 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74bcff9640 1 -- 172.21.15.69:0/301294972 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0026020 con 0x557279c89210 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74bcff9640 1 -- 172.21.15.69:0/301294972 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f74d0010bd0 con 0x557279c89210 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74bcff9640 1 -- 172.21.15.69:0/301294972 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74d001d070 con 0x557279c89210 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74dd958740 10 client.25553 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.142+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5572799603c0 con 0x557279c89210 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.143+0000 7f74bcff9640 1 -- 172.21.15.69:0/301294972 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f74d0005040 con 0x557279c89210 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.143+0000 7f74dd958740 10 client.25553 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.143+0000 7f74dd958740 10 client.25553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:11.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.143+0000 7f74dd958740 20 client.25553 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.143+0000 7f74dd958740 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557279ca3b00 0x557279ca5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.143+0000 7f74dd958740 4 client.25553 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.143+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557279bfbef0 con 0x557279ca3b00 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.143+0000 7f74d8af5640 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557279ca3b00 0x557279ca5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.144+0000 7f74d8af5640 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557279ca3b00 0x557279ca5ef0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c51cb852adc460 server_cookie=e7deb98b7f57c4cd in_seq=0 out_seq=0 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.144+0000 7f74bcff9640 10 client.25553 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.145+0000 7f74bcff9640 1 -- 172.21.15.69:0/301294972 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557279bfbef0 con 0x557279ca3b00 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.145+0000 7f74bcff9640 10 client.25553 handle_command_reply: tid=0 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.145+0000 7f74dd958740 1 client.25553 shutdown 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.145+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 msgr2=0x557279c91280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c91280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f74d0010fd0 tx=0x7f74d0005dc0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 shutdown_connections 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c917c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557279ca3b00 0x557279ca5ef0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c943f0 0x557279c91d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 --2- 172.21.15.69:0/301294972 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c91280 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 >> 172.21.15.69:0/301294972 conn(0x557279c857a0 msgr2=0x557279c8b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74bcff9640 0 client.25553 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 shutdown_connections 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 1 -- 172.21.15.69:0/301294972 wait complete. 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.146+0000 7f74dd958740 20 client.25553 trim_cache size 0 max 16384 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74dd958740 1 Processor -- start 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74dd958740 1 -- start start 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74dd958740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89210 0x557279c8bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74dd958740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c91b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74dd958740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c943f0 0x557279cab520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74dd958740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41c8d30 con 0x557279c89b30 2024-08-02T17:48:11.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74dd958740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41c6600 con 0x557279c943f0 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74dd958740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41c4430 con 0x557279c89210 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74d8af5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c91b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74d92f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89210 0x557279c8bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74d92f6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89210 0x557279c8bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42888/0 (socket says 172.21.15.69:42888) 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74d92f6640 1 -- 172.21.15.69:0/3898389354 learned_addr learned my addr 172.21.15.69:0/3898389354 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.175+0000 7f74d8af5640 1 --2- 172.21.15.69:0/3898389354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c91b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37086/0 (socket says 172.21.15.69:37086) 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74d92f6640 1 -- 172.21.15.69:0/3898389354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c943f0 msgr2=0x557279cab520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74d92f6640 1 --2- 172.21.15.69:0/3898389354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c943f0 0x557279cab520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74d92f6640 1 -- 172.21.15.69:0/3898389354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 msgr2=0x557279c91b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74d92f6640 1 --2- 172.21.15.69:0/3898389354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c91b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74d92f6640 1 -- 172.21.15.69:0/3898389354 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557279c85240 con 0x557279c89210 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74d92f6640 1 --2- 172.21.15.69:0/3898389354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89210 0x557279c8bc70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74d00058a0 tx=0x7f74d001d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c12bfb8958722de server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74beffd640 1 -- 172.21.15.69:0/3898389354 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0026050 con 0x557279c89210 2024-08-02T17:48:11.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74beffd640 1 -- 172.21.15.69:0/3898389354 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74d0010de0 con 0x557279c89210 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.176+0000 7f74beffd640 1 -- 172.21.15.69:0/3898389354 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0034b10 con 0x557279c89210 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.177+0000 7f74dd958740 1 -- 172.21.15.69:0/3898389354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89210 msgr2=0x557279c8bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.177+0000 7f74dd958740 1 --2- 172.21.15.69:0/3898389354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89210 0x557279c8bc70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74d00058a0 tx=0x7f74d001d000 comp rx=0 tx=0).stop 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.178+0000 7f74dd958740 1 -- 172.21.15.69:0/3898389354 shutdown_connections 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.178+0000 7f74dd958740 1 --2- 172.21.15.69:0/3898389354 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c89b30 0x557279c91b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.178+0000 7f74dd958740 1 --2- 172.21.15.69:0/3898389354 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c943f0 0x557279cab520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.178+0000 7f74dd958740 1 --2- 172.21.15.69:0/3898389354 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89210 0x557279c8bc70 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.178+0000 7f74dd958740 1 -- 172.21.15.69:0/3898389354 >> 172.21.15.69:0/3898389354 conn(0x557279c857a0 msgr2=0x557279c931f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.180+0000 7f74dd958740 1 -- 172.21.15.69:0/3898389354 shutdown_connections 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.180+0000 7f74dd958740 1 -- 172.21.15.69:0/3898389354 wait complete. 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.180+0000 7f74dd958740 1 -- start start 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74dd958740 1 Processor -- start 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74dd958740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c8d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74dd958740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89b30 0x557279c8ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74dd958740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c943f0 0x557279c8e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74dd958740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41c4ac0 con 0x557279c943f0 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74dd958740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41c5f30 con 0x557279c89210 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74dd958740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74d41c5190 con 0x557279c89b30 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74d92f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c8d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74d92f6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c8d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43978/0 (socket says 172.21.15.69:43978) 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74d92f6640 1 -- 172.21.15.69:0/1822002785 learned_addr learned my addr 172.21.15.69:0/1822002785 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74dd958740 10 client.0 _ll_register_callbacks cb 0x7f74d001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.181+0000 7f74d8af5640 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89b30 0x557279c8ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74d9af7640 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c943f0 0x557279c8e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74d92f6640 1 -- 172.21.15.69:0/1822002785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89b30 msgr2=0x557279c8ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74d92f6640 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89b30 0x557279c8ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74d92f6640 1 -- 172.21.15.69:0/1822002785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c943f0 msgr2=0x557279c8e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74d92f6640 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c943f0 0x557279c8e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74d92f6640 1 -- 172.21.15.69:0/1822002785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557279d0c780 con 0x557279c89210 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74d9af7640 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c943f0 0x557279c8e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74d92f6640 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c8d8b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f74d000a310 tx=0x7f74d0005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d206edd3c1b7a335 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.182+0000 7f74bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.183+0000 7f74bcff9640 1 -- 172.21.15.69:0/1822002785 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0042020 con 0x557279c89210 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.183+0000 7f74bcff9640 1 -- 172.21.15.69:0/1822002785 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f74d0034820 con 0x557279c89210 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.183+0000 7f74dd958740 1 -- 172.21.15.69:0/1822002785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d0010de0 con 0x557279c89210 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.183+0000 7f74bcff9640 1 -- 172.21.15.69:0/1822002785 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f74d004d050 con 0x557279c89210 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.183+0000 7f74dd958740 10 client.25556 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.183+0000 7f74dd958740 1 -- 172.21.15.69:0/1822002785 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557279d0d9f0 con 0x557279c89210 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.183+0000 7f74bcff9640 1 -- 172.21.15.69:0/1822002785 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74d001e040 con 0x557279c89210 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.184+0000 7f74bcff9640 1 -- 172.21.15.69:0/1822002785 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f74d0022a40 con 0x557279c89210 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.184+0000 7f74dd958740 10 client.25556 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.184+0000 7f74dd958740 10 client.25556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.184+0000 7f74dd958740 20 client.25556 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:11.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.184+0000 7f74dd958740 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557279d0e0b0 0x557279d104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:11.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.184+0000 7f74dd958740 4 client.25556 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:48:11.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.184+0000 7f74dd958740 1 -- 172.21.15.69:0/1822002785 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f74d0010de0 con 0x557279d0e0b0 2024-08-02T17:48:11.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.184+0000 7f74d8af5640 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557279d0e0b0 0x557279d104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:11.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74d8af5640 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557279d0e0b0 0x557279d104a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd0a7b8d8244f518 server_cookie=26537c9fcce84f37 in_seq=0 out_seq=0 2024-08-02T17:48:11.970 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:48:11.970 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:48:11.970 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:48:11.970 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:48:11.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74bcff9640 10 client.25556 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:11.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74bcff9640 1 -- 172.21.15.69:0/1822002785 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f74d0010de0 con 0x557279d0e0b0 2024-08-02T17:48:11.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74bcff9640 10 client.25556 handle_command_reply: tid=0 2024-08-02T17:48:11.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74dd958740 1 client.25556 shutdown 2024-08-02T17:48:11.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74dd958740 1 -- 172.21.15.69:0/1822002785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 msgr2=0x557279c8d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74dd958740 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c8d8b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f74d000a310 tx=0x7f74d0005e00 comp rx=0 tx=0).stop 2024-08-02T17:48:11.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74dd958740 1 -- 172.21.15.69:0/1822002785 shutdown_connections 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74dd958740 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557279c943f0 0x557279c8e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.187+0000 7f74dd958740 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557279d0e0b0 0x557279d104a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74dd958740 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557279c89b30 0x557279c8ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74bcff9640 0 client.25556 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74dd958740 1 --2- 172.21.15.69:0/1822002785 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557279c89210 0x557279c8d8b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74dd958740 1 -- 172.21.15.69:0/1822002785 >> 172.21.15.69:0/1822002785 conn(0x557279c857a0 msgr2=0x557279c92810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74dd958740 1 -- 172.21.15.69:0/1822002785 shutdown_connections 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74dd958740 1 -- 172.21.15.69:0/1822002785 wait complete. 2024-08-02T17:48:11.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74dd958740 20 client.25556 trim_cache size 0 max 16384 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74db55a640 1 -- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f74c00a2f00 msgr2=0x7f74c00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74db55a640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f74c00a2f00 0x7f74c00a53c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009bf0 tx=0x7f74cc0314d0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74db55a640 1 -- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74d4109560 msgr2=0x7f74d4110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.188+0000 7f74db55a640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74d4109560 0x7f74d4110720 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b4a0 tx=0x7f74c800b960 comp rx=0 tx=0).stop 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.189+0000 7f74db55a640 1 -- 172.21.15.69:0/3173876668 shutdown_connections 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.189+0000 7f74db55a640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f74d4109560 0x7f74d4110720 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.189+0000 7f74db55a640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f74c00a2f00 0x7f74c00a53c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.189+0000 7f74db55a640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f74d411ae70 0x7f74d4110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.189+0000 7f74db55a640 1 --2- 172.21.15.69:0/3173876668 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f74d4109180 0x7f74d4118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.189+0000 7f74db55a640 1 -- 172.21.15.69:0/3173876668 >> 172.21.15.69:0/3173876668 conn(0x7f74d406de70 msgr2=0x7f74d411bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.189+0000 7f74db55a640 1 -- 172.21.15.69:0/3173876668 shutdown_connections 2024-08-02T17:48:11.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:11.189+0000 7f74db55a640 1 -- 172.21.15.69:0/3173876668 wait complete. 2024-08-02T17:48:13.165 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:48:13.165 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:48:13.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:13 smithi077 ceph-mon[37108]: pgmap v57: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-08-02T17:48:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:13 smithi069 ceph-mon[32746]: pgmap v57: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-08-02T17:48:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:13 smithi083 ceph-mon[37031]: pgmap v57: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-08-02T17:48:13.593 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.178+0000 7f6c4fb31640 1 -- 172.21.15.69:0/2208964202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 msgr2=0x7f6c48109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.178+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/2208964202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 0x7f6c48109390 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009860 tx=0x7f6c4402a340 comp rx=0 tx=0).stop 2024-08-02T17:48:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.179+0000 7f6c4fb31640 1 -- 172.21.15.69:0/2208964202 shutdown_connections 2024-08-02T17:48:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.179+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/2208964202 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c481147e0 0x7f6c48116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.179+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/2208964202 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c48109960 0x7f6c481142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.179+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/2208964202 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 0x7f6c48109390 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.179+0000 7f6c4fb31640 1 -- 172.21.15.69:0/2208964202 >> 172.21.15.69:0/2208964202 conn(0x7f6c4806de70 msgr2=0x7f6c4806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.179+0000 7f6c4fb31640 1 -- 172.21.15.69:0/2208964202 shutdown_connections 2024-08-02T17:48:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.179+0000 7f6c4fb31640 1 -- 172.21.15.69:0/2208964202 wait complete. 2024-08-02T17:48:14.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.180+0000 7f6c4fb31640 1 Processor -- start 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4fb31640 1 -- start start 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4fb31640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 0x7f6c4811a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4fb31640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c48109960 0x7f6c4811a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4fb31640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c481147e0 0x7f6c4811d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4fb31640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4810a3c0 con 0x7f6c481147e0 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4fb31640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4810a240 con 0x7f6c48109960 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4fb31640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4810a540 con 0x7f6c48108fb0 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d0cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c48109960 0x7f6c4811a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d8cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 0x7f6c4811a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d0cc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c48109960 0x7f6c4811a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43994/0 (socket says 172.21.15.69:43994) 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d0cc640 1 -- 172.21.15.69:0/4275704275 learned_addr learned my addr 172.21.15.69:0/4275704275 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:14.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4e0ce640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c481147e0 0x7f6c4811d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:14.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d8cd640 1 -- 172.21.15.69:0/4275704275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c48109960 msgr2=0x7f6c4811a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:14.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d8cd640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c48109960 0x7f6c4811a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d8cd640 1 -- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c481147e0 msgr2=0x7f6c4811d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d8cd640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c481147e0 0x7f6c4811d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.181+0000 7f6c4d8cd640 1 -- 172.21.15.69:0/4275704275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4811da90 con 0x7f6c48108fb0 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.182+0000 7f6c4d0cc640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c48109960 0x7f6c4811a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.182+0000 7f6c4d8cd640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 0x7f6c4811a2d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c4402a860 tx=0x7f6c44033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.183+0000 7f6c3effd640 1 -- 172.21.15.69:0/4275704275 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44037040 con 0x7f6c48108fb0 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.183+0000 7f6c3effd640 1 -- 172.21.15.69:0/4275704275 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c44030550 con 0x7f6c48108fb0 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.183+0000 7f6c3effd640 1 -- 172.21.15.69:0/4275704275 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44003b30 con 0x7f6c48108fb0 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.183+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4811dd20 con 0x7f6c48108fb0 2024-08-02T17:48:14.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.183+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4811e1b0 con 0x7f6c48108fb0 2024-08-02T17:48:14.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.185+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c48109390 con 0x7f6c48108fb0 2024-08-02T17:48:14.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.186+0000 7f6c3effd640 1 -- 172.21.15.69:0/4275704275 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6c44042020 con 0x7f6c48108fb0 2024-08-02T17:48:14.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.187+0000 7f6c3effd640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c340a2e40 0x7f6c340a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:14.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.187+0000 7f6c3effd640 1 -- 172.21.15.69:0/4275704275 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f6c440eed00 con 0x7f6c48108fb0 2024-08-02T17:48:14.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.188+0000 7f6c4d0cc640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c340a2e40 0x7f6c340a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:14.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.189+0000 7f6c4d0cc640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c340a2e40 0x7f6c340a5300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6c48114420 tx=0x7f6c3800b480 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.192+0000 7f6c3effd640 1 -- 172.21.15.69:0/4275704275 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6c44049020 con 0x7f6c48108fb0 2024-08-02T17:48:14.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.577+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c481b3ca0 con 0x7f6c48108fb0 2024-08-02T17:48:14.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.578+0000 7f6c3effd640 1 -- 172.21.15.69:0/4275704275 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f6c440bb670 con 0x7f6c48108fb0 2024-08-02T17:48:14.580 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:48:14.580 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:48:14.581 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:48:14.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c340a2e40 msgr2=0x7f6c340a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:14.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c340a2e40 0x7f6c340a5300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6c48114420 tx=0x7f6c3800b480 comp rx=0 tx=0).stop 2024-08-02T17:48:14.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 msgr2=0x7f6c4811a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:14.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 0x7f6c4811a2d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6c4402a860 tx=0x7f6c44033d90 comp rx=0 tx=0).stop 2024-08-02T17:48:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 shutdown_connections 2024-08-02T17:48:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6c481147e0 0x7f6c4811d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6c340a2e40 0x7f6c340a5300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6c48109960 0x7f6c4811a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 --2- 172.21.15.69:0/4275704275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c48108fb0 0x7f6c4811a2d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 >> 172.21.15.69:0/4275704275 conn(0x7f6c4806de70 msgr2=0x7f6c48070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 shutdown_connections 2024-08-02T17:48:14.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:14.582+0000 7f6c4fb31640 1 -- 172.21.15.69:0/4275704275 wait complete. 2024-08-02T17:48:14.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:14 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4275704275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:48:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 65.0 s, 3168.4 tps, lat 31.466 ms stddev 37.829 2024-08-02T17:48:15.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:14 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4275704275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:48:15.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:14 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4275704275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:48:15.379 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:48:15.779 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:15.922 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:15 smithi069 ceph-mon[32746]: pgmap v58: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-08-02T17:48:16.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:15 smithi083 ceph-mon[37031]: pgmap v58: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-08-02T17:48:16.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:15 smithi077 ceph-mon[37108]: pgmap v58: 129 pgs: 129 active+clean; 1.1 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 18 MiB/s wr, 1.68k op/s 2024-08-02T17:48:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.363+0000 7f995746a640 1 -- 172.21.15.69:0/3852189788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950100b50 msgr2=0x7f9950100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.363+0000 7f995746a640 1 --2- 172.21.15.69:0/3852189788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950100b50 0x7f9950100f30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f994c009900 tx=0x7f994c02a4b0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.363+0000 7f995746a640 1 -- 172.21.15.69:0/3852189788 shutdown_connections 2024-08-02T17:48:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.363+0000 7f995746a640 1 --2- 172.21.15.69:0/3852189788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9950102c20 0x7f995010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.363+0000 7f995746a640 1 --2- 172.21.15.69:0/3852189788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f99501022a0 0x7f99501026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.363+0000 7f995746a640 1 --2- 172.21.15.69:0/3852189788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950100b50 0x7f9950100f30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.363+0000 7f995746a640 1 -- 172.21.15.69:0/3852189788 >> 172.21.15.69:0/3852189788 conn(0x7f99500fc500 msgr2=0x7f99500fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.364+0000 7f995746a640 1 -- 172.21.15.69:0/3852189788 shutdown_connections 2024-08-02T17:48:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.364+0000 7f995746a640 1 -- 172.21.15.69:0/3852189788 wait complete. 2024-08-02T17:48:16.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.364+0000 7f995746a640 1 Processor -- start 2024-08-02T17:48:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.365+0000 7f995746a640 1 -- start start 2024-08-02T17:48:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f995746a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9950100b50 0x7f99501a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f995746a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99501022a0 0x7f99501a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f995746a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950102c20 0x7f995019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f995746a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9950110cb0 con 0x7f9950102c20 2024-08-02T17:48:16.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f995746a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9950110b30 con 0x7f9950100b50 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f995746a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9950110e30 con 0x7f99501022a0 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f9955a07640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950102c20 0x7f995019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f9955a07640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950102c20 0x7f995019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36930/0 (socket says 172.21.15.69:36930) 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f9955a07640 1 -- 172.21.15.69:0/3401779261 learned_addr learned my addr 172.21.15.69:0/3401779261 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f9954a05640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99501022a0 0x7f99501a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.366+0000 7f9955206640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9950100b50 0x7f99501a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f9954a05640 1 -- 172.21.15.69:0/3401779261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9950100b50 msgr2=0x7f99501a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f9954a05640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9950100b50 0x7f99501a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f9954a05640 1 -- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950102c20 msgr2=0x7f995019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f9954a05640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950102c20 0x7f995019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f9954a05640 1 -- 172.21.15.69:0/3401779261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995019f1f0 con 0x7f99501022a0 2024-08-02T17:48:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f9954a05640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99501022a0 0x7f99501a3930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f994000e9f0 tx=0x7f994000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f99467fc640 1 -- 172.21.15.69:0/3401779261 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994000ccc0 con 0x7f99501022a0 2024-08-02T17:48:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f99467fc640 1 -- 172.21.15.69:0/3401779261 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9940010910 con 0x7f99501022a0 2024-08-02T17:48:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f995746a640 1 -- 172.21.15.69:0/3401779261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995019f4e0 con 0x7f99501022a0 2024-08-02T17:48:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f995746a640 1 -- 172.21.15.69:0/3401779261 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99501ab4f0 con 0x7f99501022a0 2024-08-02T17:48:16.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.367+0000 7f99467fc640 1 -- 172.21.15.69:0/3401779261 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994000ccc0 con 0x7f99501022a0 2024-08-02T17:48:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.371+0000 7f9959868740 1 Processor -- start 2024-08-02T17:48:16.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9959868740 1 -- start start 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9959868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a9183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9959868740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a922c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9959868740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 0x55988a9255c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9959868740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55988a87da40 con 0x55988a9231d0 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9959868740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55988a8159d0 con 0x55988a918910 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9959868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55988a879a00 con 0x55988a917ff0 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9955a07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a9183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9955a07640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a9183d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47408/0 (socket says 172.21.15.69:47408) 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9955a07640 1 -- 172.21.15.69:0/1228942135 learned_addr learned my addr 172.21.15.69:0/1228942135 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.372+0000 7f9955206640 1 --2- 172.21.15.69:0/1228942135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a922c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f9955a07640 1 -- 172.21.15.69:0/1228942135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 msgr2=0x55988a922c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f9955a07640 1 --2- 172.21.15.69:0/1228942135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a922c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f9955a07640 1 -- 172.21.15.69:0/1228942135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 msgr2=0x55988a9255c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f9955a07640 1 --2- 172.21.15.69:0/1228942135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 0x55988a9255c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f9955a07640 1 -- 172.21.15.69:0/1228942135 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988a81c650 con 0x55988a917ff0 2024-08-02T17:48:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f9955a07640 1 --2- 172.21.15.69:0/1228942135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a9183d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9948011410 tx=0x7f99480118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f00490b3b7671b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f993b7fe640 1 -- 172.21.15.69:0/1228942135 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9948012770 con 0x55988a917ff0 2024-08-02T17:48:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f993b7fe640 1 -- 172.21.15.69:0/1228942135 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9948012df0 con 0x55988a917ff0 2024-08-02T17:48:16.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.373+0000 7f993b7fe640 1 -- 172.21.15.69:0/1228942135 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994801a3f0 con 0x55988a917ff0 2024-08-02T17:48:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.374+0000 7f99467fc640 1 -- 172.21.15.69:0/3401779261 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9940010430 con 0x7f99501022a0 2024-08-02T17:48:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.374+0000 7f9959868740 1 -- 172.21.15.69:0/1228942135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 msgr2=0x55988a9183d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.374+0000 7f9959868740 1 --2- 172.21.15.69:0/1228942135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a9183d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9948011410 tx=0x7f99480118d0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.374+0000 7f9959868740 1 -- 172.21.15.69:0/1228942135 shutdown_connections 2024-08-02T17:48:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.374+0000 7f9959868740 1 --2- 172.21.15.69:0/1228942135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 0x55988a9255c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.374+0000 7f9959868740 1 --2- 172.21.15.69:0/1228942135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a922c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.374+0000 7f9959868740 1 --2- 172.21.15.69:0/1228942135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a9183d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.375+0000 7f9959868740 1 -- 172.21.15.69:0/1228942135 >> 172.21.15.69:0/1228942135 conn(0x55988a914580 msgr2=0x55988a914960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.375+0000 7f9959868740 1 -- 172.21.15.69:0/1228942135 shutdown_connections 2024-08-02T17:48:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.375+0000 7f99467fc640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f993c0a2ea0 0x7f993c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.375+0000 7f9959868740 1 -- 172.21.15.69:0/1228942135 wait complete. 2024-08-02T17:48:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.376+0000 7f99467fc640 1 -- 172.21.15.69:0/3401779261 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f99400cf750 con 0x7f99501022a0 2024-08-02T17:48:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.376+0000 7f9959868740 1 -- start start 2024-08-02T17:48:16.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.376+0000 7f9959868740 1 Processor -- start 2024-08-02T17:48:16.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.376+0000 7f9955206640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f993c0a2ea0 0x7f993c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9959868740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a921840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9959868740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 0x55988a921d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9959868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 0x55988a9222c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9955206640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a921840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9955206640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a921840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59338/0 (socket says 172.21.15.69:59338) 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9955206640 1 -- 172.21.15.69:0/638344911 learned_addr learned my addr 172.21.15.69:0/638344911 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9959868740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55988a925df0 con 0x55988a918910 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55988a838b20 con 0x55988a917ff0 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55988a925f70 con 0x55988a9231d0 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9955a07640 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 0x55988a9222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9959868740 10 client.0 _ll_register_callbacks cb 0x55988a836bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.377+0000 7f9955206640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f993c0a2ea0 0x7f993c0a5360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f994c006010 tx=0x7f994c005a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9955206640 1 -- 172.21.15.69:0/638344911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 msgr2=0x55988a9222c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9955206640 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 0x55988a9222c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9955206640 1 -- 172.21.15.69:0/638344911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 msgr2=0x55988a921d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9955206640 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 0x55988a921d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9955206640 1 -- 172.21.15.69:0/638344911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988a5ef3c0 con 0x55988a917ff0 2024-08-02T17:48:16.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9955206640 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a921840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f994c009900 tx=0x7f994c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493b3701f8453e4b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:16.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:16.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9938ff9640 1 -- 172.21.15.69:0/638344911 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c04d070 con 0x55988a917ff0 2024-08-02T17:48:16.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9948012df0 con 0x55988a917ff0 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9938ff9640 1 -- 172.21.15.69:0/638344911 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f994c031740 con 0x55988a917ff0 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.378+0000 7f9938ff9640 1 -- 172.21.15.69:0/638344911 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f994c059050 con 0x55988a917ff0 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.379+0000 7f9959868740 10 client.25574 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.379+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55988a931a60 con 0x55988a917ff0 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.379+0000 7f9938ff9640 1 -- 172.21.15.69:0/638344911 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9948012df0 con 0x55988a917ff0 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.379+0000 7f9938ff9640 1 -- 172.21.15.69:0/638344911 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f994c03da90 con 0x55988a917ff0 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.379+0000 7f9959868740 10 client.25574 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.379+0000 7f9959868740 10 client.25574 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.379+0000 7f9959868740 20 client.25574 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.380+0000 7f9959868740 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55988a933310 0x55988a9357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.380+0000 7f9959868740 4 client.25574 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.380+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55988a88afb0 con 0x55988a933310 2024-08-02T17:48:16.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.380+0000 7f9954a05640 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55988a933310 0x55988a9357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.381+0000 7f9954a05640 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55988a933310 0x55988a9357d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c075ef01fb743e5 server_cookie=9b00efc5c8fa0bc6 in_seq=0 out_seq=0 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.381+0000 7f9938ff9640 10 client.25574 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9938ff9640 1 -- 172.21.15.69:0/638344911 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55988a88afb0 con 0x55988a933310 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9938ff9640 10 client.25574 handle_command_reply: tid=0 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 client.25574 shutdown 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 msgr2=0x55988a921840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a921840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f994c009900 tx=0x7f994c009260 comp rx=0 tx=0).stop 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 shutdown_connections 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55988a933310 0x55988a9357d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 0x55988a921d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 0x55988a9222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 --2- 172.21.15.69:0/638344911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a921840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 >> 172.21.15.69:0/638344911 conn(0x55988a914580 msgr2=0x55988a91a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9938ff9640 0 client.25574 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:48:16.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 shutdown_connections 2024-08-02T17:48:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 1 -- 172.21.15.69:0/638344911 wait complete. 2024-08-02T17:48:16.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.382+0000 7f9959868740 20 client.25574 trim_cache size 0 max 16384 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9959868740 1 Processor -- start 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9959868740 1 -- start start 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9959868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a920950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9959868740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a93b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9959868740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 0x55988a93b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9959868740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9934012800 con 0x55988a9231d0 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9959868740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f99340100d0 con 0x55988a918910 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9959868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f993400df20 con 0x55988a917ff0 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a920950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a920950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47416/0 (socket says 172.21.15.69:47416) 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955206640 1 -- 172.21.15.69:0/3991050164 learned_addr learned my addr 172.21.15.69:0/3991050164 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955a07640 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 0x55988a93b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9954a05640 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a93b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955206640 1 -- 172.21.15.69:0/3991050164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 msgr2=0x55988a93b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955206640 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a93b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955206640 1 -- 172.21.15.69:0/3991050164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 msgr2=0x55988a93b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955206640 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 0x55988a93b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.403+0000 7f9955206640 1 -- 172.21.15.69:0/3991050164 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988a935e50 con 0x55988a917ff0 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.404+0000 7f9954a05640 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a93b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:16.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.404+0000 7f9955206640 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a920950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f994c005ab0 tx=0x7f994c003a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d3d43196d0c2393 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.404+0000 7f993affd640 1 -- 172.21.15.69:0/3991050164 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c04d040 con 0x55988a917ff0 2024-08-02T17:48:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.404+0000 7f993affd640 1 -- 172.21.15.69:0/3991050164 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c056030 con 0x55988a917ff0 2024-08-02T17:48:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.404+0000 7f993affd640 1 -- 172.21.15.69:0/3991050164 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c035040 con 0x55988a917ff0 2024-08-02T17:48:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.405+0000 7f9959868740 1 -- 172.21.15.69:0/3991050164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 msgr2=0x55988a920950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.405+0000 7f9959868740 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a920950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f994c005ab0 tx=0x7f994c003a80 comp rx=0 tx=0).stop 2024-08-02T17:48:16.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.405+0000 7f9959868740 1 -- 172.21.15.69:0/3991050164 shutdown_connections 2024-08-02T17:48:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.405+0000 7f9959868740 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a9231d0 0x55988a93b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.405+0000 7f9959868740 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a918910 0x55988a93b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.405+0000 7f9959868740 1 --2- 172.21.15.69:0/3991050164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a917ff0 0x55988a920950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.405+0000 7f9959868740 1 -- 172.21.15.69:0/3991050164 >> 172.21.15.69:0/3991050164 conn(0x55988a914580 msgr2=0x55988a91f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.406+0000 7f9959868740 1 -- 172.21.15.69:0/3991050164 shutdown_connections 2024-08-02T17:48:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.406+0000 7f9959868740 1 -- 172.21.15.69:0/3991050164 wait complete. 2024-08-02T17:48:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.406+0000 7f9959868740 1 -- start start 2024-08-02T17:48:16.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.407+0000 7f9959868740 1 Processor -- start 2024-08-02T17:48:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9959868740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a9267a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9959868740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 0x55988a926ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9959868740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 0x55988a99c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9959868740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f993400e590 con 0x55988a918910 2024-08-02T17:48:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9959868740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f993400fa00 con 0x55988a917ff0 2024-08-02T17:48:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9959868740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f993400ec60 con 0x55988a9231d0 2024-08-02T17:48:16.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9959868740 10 client.0 _ll_register_callbacks cb 0x7f994c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9955206640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a9267a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9955206640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a9267a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59346/0 (socket says 172.21.15.69:59346) 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9955206640 1 -- 172.21.15.69:0/3815053982 learned_addr learned my addr 172.21.15.69:0/3815053982 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9954a05640 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 0x55988a926ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9955206640 1 -- 172.21.15.69:0/3815053982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 msgr2=0x55988a99c060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9955206640 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 0x55988a99c060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9955206640 1 -- 172.21.15.69:0/3815053982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 msgr2=0x55988a926ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9955206640 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 0x55988a926ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.408+0000 7f9955206640 1 -- 172.21.15.69:0/3815053982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988a99c5a0 con 0x55988a917ff0 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.409+0000 7f9955206640 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a9267a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f994c0096c0 tx=0x7f994c031890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568f1f00423d9708 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.409+0000 7f9938ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.409+0000 7f9938ff9640 1 -- 172.21.15.69:0/3815053982 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c04d070 con 0x55988a917ff0 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9938ff9640 1 -- 172.21.15.69:0/3815053982 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f994c03d650 con 0x55988a917ff0 2024-08-02T17:48:16.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 1 -- 172.21.15.69:0/3815053982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f994c056030 con 0x55988a917ff0 2024-08-02T17:48:16.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9938ff9640 1 -- 172.21.15.69:0/3815053982 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f994c06c050 con 0x55988a917ff0 2024-08-02T17:48:16.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 10 client.25577 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:16.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 1 -- 172.21.15.69:0/3815053982 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55988a99d810 con 0x55988a917ff0 2024-08-02T17:48:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9938ff9640 1 -- 172.21.15.69:0/3815053982 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f994c056030 con 0x55988a917ff0 2024-08-02T17:48:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9938ff9640 1 -- 172.21.15.69:0/3815053982 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f994c0499c0 con 0x55988a917ff0 2024-08-02T17:48:16.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 10 client.25577 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 10 client.25577 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 20 client.25577 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55988a99dea0 0x55988a9a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 4 client.25577 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.410+0000 7f9959868740 1 -- 172.21.15.69:0/3815053982 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55988a8854c0 con 0x55988a99dea0 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.411+0000 7f9954a05640 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55988a99dea0 0x55988a9a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9954a05640 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55988a99dea0 0x55988a9a02c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd69a75c91326b7d server_cookie=3310444b15a6d65b in_seq=0 out_seq=0 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9938ff9640 10 client.25577 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9938ff9640 1 -- 172.21.15.69:0/3815053982 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55988a8854c0 con 0x55988a99dea0 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9938ff9640 10 client.25577 handle_command_reply: tid=0 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 client.25577 shutdown 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 -- 172.21.15.69:0/3815053982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 msgr2=0x55988a9267a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a9267a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f994c0096c0 tx=0x7f994c031890 comp rx=0 tx=0).stop 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 -- 172.21.15.69:0/3815053982 shutdown_connections 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55988a99dea0 0x55988a9a02c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55988a918910 0x55988a926ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55988a9231d0 0x55988a99c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 --2- 172.21.15.69:0/3815053982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55988a917ff0 0x55988a9267a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 -- 172.21.15.69:0/3815053982 >> 172.21.15.69:0/3815053982 conn(0x55988a914580 msgr2=0x55988a919a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9938ff9640 0 client.25577 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.412+0000 7f9959868740 1 -- 172.21.15.69:0/3815053982 shutdown_connections 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.413+0000 7f9959868740 1 -- 172.21.15.69:0/3815053982 wait complete. 2024-08-02T17:48:16.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.413+0000 7f9959868740 20 client.25577 trim_cache size 0 max 16384 2024-08-02T17:48:16.417 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:48:16.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 -- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f993c0a2ea0 msgr2=0x7f993c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f993c0a2ea0 0x7f993c0a5360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f994c006010 tx=0x7f994c005a20 comp rx=0 tx=0).stop 2024-08-02T17:48:16.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 -- 172.21.15.69:0/3401779261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99501022a0 msgr2=0x7f99501a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99501022a0 0x7f99501a3930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f994000e9f0 tx=0x7f994000eeb0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 -- 172.21.15.69:0/3401779261 shutdown_connections 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9950102c20 0x7f995019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f993c0a2ea0 0x7f993c0a5360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99501022a0 0x7f99501a3930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 --2- 172.21.15.69:0/3401779261 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9950100b50 0x7f99501a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 -- 172.21.15.69:0/3401779261 >> 172.21.15.69:0/3401779261 conn(0x7f99500fc500 msgr2=0x7f9950069d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 -- 172.21.15.69:0/3401779261 shutdown_connections 2024-08-02T17:48:16.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:16.414+0000 7f995746a640 1 -- 172.21.15.69:0/3401779261 wait complete. 2024-08-02T17:48:17.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:17 smithi077 ceph-mon[37108]: pgmap v59: 129 pgs: 129 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 23 MiB/s wr, 2.03k op/s 2024-08-02T17:48:17.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:17 smithi069 ceph-mon[32746]: pgmap v59: 129 pgs: 129 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 23 MiB/s wr, 2.03k op/s 2024-08-02T17:48:17.490 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:48:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:17 smithi083 ceph-mon[37031]: pgmap v59: 129 pgs: 129 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 23 MiB/s wr, 2.03k op/s 2024-08-02T17:48:17.935 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:18.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 -- 172.21.15.69:0/2376756884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8901098d0 msgr2=0x7fc8901142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 --2- 172.21.15.69:0/2376756884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8901098d0 0x7fc8901142a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc88400fb10 tx=0x7fc884032970 comp rx=0 tx=0).stop 2024-08-02T17:48:18.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 -- 172.21.15.69:0/2376756884 shutdown_connections 2024-08-02T17:48:18.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 --2- 172.21.15.69:0/2376756884 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc8901098d0 0x7fc8901142a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 --2- 172.21.15.69:0/2376756884 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8901147e0 0x7fc890116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 --2- 172.21.15.69:0/2376756884 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc890108f20 0x7fc890109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 -- 172.21.15.69:0/2376756884 >> 172.21.15.69:0/2376756884 conn(0x7fc89006e000 msgr2=0x7fc89006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:18.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 -- 172.21.15.69:0/2376756884 shutdown_connections 2024-08-02T17:48:18.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.516+0000 7fc896a35640 1 -- 172.21.15.69:0/2376756884 wait complete. 2024-08-02T17:48:18.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.517+0000 7fc896a35640 1 Processor -- start 2024-08-02T17:48:18.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.517+0000 7fc896a35640 1 -- start start 2024-08-02T17:48:18.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc896a35640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc890108f20 0x7fc89019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc896a35640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8901098d0 0x7fc89019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc896a35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8901147e0 0x7fc8901a2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc896a35640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc890118370 con 0x7fc890108f20 2024-08-02T17:48:18.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc896a35640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901181f0 con 0x7fc8901098d0 2024-08-02T17:48:18.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc896a35640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901184f0 con 0x7fc8901147e0 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc894fd2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8901147e0 0x7fc8901a2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc894fd2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8901147e0 0x7fc8901a2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47446/0 (socket says 172.21.15.69:47446) 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc894fd2640 1 -- 172.21.15.69:0/3345843939 learned_addr learned my addr 172.21.15.69:0/3345843939 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.518+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8901098d0 0x7fc89019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.519+0000 7fc894fd2640 1 -- 172.21.15.69:0/3345843939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8901098d0 msgr2=0x7fc89019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.519+0000 7fc894fd2640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8901098d0 0x7fc89019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.519+0000 7fc894fd2640 1 -- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc890108f20 msgr2=0x7fc89019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.519+0000 7fc894fd2640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc890108f20 0x7fc89019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.519+0000 7fc894fd2640 1 -- 172.21.15.69:0/3345843939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8901a3760 con 0x7fc8901147e0 2024-08-02T17:48:18.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.519+0000 7fc894fd2640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8901147e0 0x7fc8901a2fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc88800b9e0 tx=0x7fc88800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:18.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.520+0000 7fc88d7fa640 1 -- 172.21.15.69:0/3345843939 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888019070 con 0x7fc8901147e0 2024-08-02T17:48:18.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.520+0000 7fc88d7fa640 1 -- 172.21.15.69:0/3345843939 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc888014440 con 0x7fc8901147e0 2024-08-02T17:48:18.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.520+0000 7fc88d7fa640 1 -- 172.21.15.69:0/3345843939 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888012620 con 0x7fc8901147e0 2024-08-02T17:48:18.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.520+0000 7fc896a35640 1 -- 172.21.15.69:0/3345843939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8901a3a50 con 0x7fc8901147e0 2024-08-02T17:48:18.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.520+0000 7fc896a35640 1 -- 172.21.15.69:0/3345843939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8901ab290 con 0x7fc8901147e0 2024-08-02T17:48:18.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.523+0000 7fc88d7fa640 1 -- 172.21.15.69:0/3345843939 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc88801c070 con 0x7fc8901147e0 2024-08-02T17:48:18.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.524+0000 7fc88d7fa640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8780a2e40 0x7fc8780a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.525+0000 7fc88d7fa640 1 -- 172.21.15.69:0/3345843939 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc8880ceb80 con 0x7fc8901147e0 2024-08-02T17:48:18.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.525+0000 7fc88ffff640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8780a2e40 0x7fc8780a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.525+0000 7fc898e33740 1 Processor -- start 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc898e33740 1 -- start start 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc898e33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd6b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc898e33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd75c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc898e33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd785c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc898e33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5631fdcd0a40 con 0x5631fdd761d0 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc898e33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5631fdc689d0 con 0x5631fdd6b910 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc898e33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631fdccca00 con 0x5631fdd6aff0 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd75c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd6b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc88f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd75c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59366/0 (socket says 172.21.15.69:59366) 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc88f7fe640 1 -- 172.21.15.69:0/3230229371 learned_addr learned my addr 172.21.15.69:0/3230229371 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd6b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47458/0 (socket says 172.21.15.69:47458) 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.526+0000 7fc894fd2640 1 --2- 172.21.15.69:0/3230229371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.527+0000 7fc88ffff640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8780a2e40 0x7fc8780a5300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8800058c0 tx=0x7fc880005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.527+0000 7fc88ffff640 1 -- 172.21.15.69:0/3230229371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 msgr2=0x5631fdd75c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.527+0000 7fc88ffff640 1 --2- 172.21.15.69:0/3230229371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd75c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.527+0000 7fc88ffff640 1 -- 172.21.15.69:0/3230229371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 msgr2=0x5631fdd785c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.527+0000 7fc88ffff640 1 --2- 172.21.15.69:0/3230229371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd785c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.527+0000 7fc88ffff640 1 -- 172.21.15.69:0/3230229371 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631fdc6f650 con 0x5631fdd6aff0 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.527+0000 7fc88ffff640 1 --2- 172.21.15.69:0/3230229371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd6b3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc88000ee50 tx=0x7fc88001b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8df38692a681c6a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.527+0000 7fc8767fc640 1 -- 172.21.15.69:0/3230229371 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880020070 con 0x5631fdd6aff0 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.528+0000 7fc8767fc640 1 -- 172.21.15.69:0/3230229371 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc880005500 con 0x5631fdd6aff0 2024-08-02T17:48:18.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.528+0000 7fc8767fc640 1 -- 172.21.15.69:0/3230229371 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880004e40 con 0x5631fdd6aff0 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.528+0000 7fc898e33740 1 -- 172.21.15.69:0/3230229371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 msgr2=0x5631fdd6b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.528+0000 7fc898e33740 1 --2- 172.21.15.69:0/3230229371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd6b3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc88000ee50 tx=0x7fc88001b000 comp rx=0 tx=0).stop 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.529+0000 7fc898e33740 1 -- 172.21.15.69:0/3230229371 shutdown_connections 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.529+0000 7fc898e33740 1 --2- 172.21.15.69:0/3230229371 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.529+0000 7fc898e33740 1 --2- 172.21.15.69:0/3230229371 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd75c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.529+0000 7fc898e33740 1 --2- 172.21.15.69:0/3230229371 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd6b3d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.529+0000 7fc898e33740 1 -- 172.21.15.69:0/3230229371 >> 172.21.15.69:0/3230229371 conn(0x5631fdd67580 msgr2=0x5631fdd67960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.529+0000 7fc898e33740 1 -- 172.21.15.69:0/3230229371 shutdown_connections 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.529+0000 7fc898e33740 1 -- 172.21.15.69:0/3230229371 wait complete. 2024-08-02T17:48:18.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.529+0000 7fc898e33740 1 -- start start 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc898e33740 1 Processor -- start 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc898e33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc898e33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd736d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc898e33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd73c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd73190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd73190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47472/0 (socket says 172.21.15.69:47472) 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc88ffff640 1 -- 172.21.15.69:0/2945270725 learned_addr learned my addr 172.21.15.69:0/2945270725 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5631fdd78da0 con 0x5631fdd761d0 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5631fdc8bb20 con 0x5631fdd6b910 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5631fdd78f20 con 0x5631fdd6aff0 2024-08-02T17:48:18.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.530+0000 7fc898e33740 10 client.0 _ll_register_callbacks cb 0x5631fdc89bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.532+0000 7fc894fd2640 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd73c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.532+0000 7fc88f7fe640 1 -- 172.21.15.69:0/2945270725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 msgr2=0x5631fdd73190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.532+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd73190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.532+0000 7fc88f7fe640 1 -- 172.21.15.69:0/2945270725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 msgr2=0x5631fdd73c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.532+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd73c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.532+0000 7fc88f7fe640 1 -- 172.21.15.69:0/2945270725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631fda423c0 con 0x5631fdd6b910 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.532+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd736d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5631fdd683e0 tx=0x7fc8840043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ec3b64e0add3f67 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.532+0000 7fc86f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.533+0000 7fc86f7fe640 1 -- 172.21.15.69:0/2945270725 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88400eb70 con 0x5631fdd6b910 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.533+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc880005500 con 0x5631fdd6b910 2024-08-02T17:48:18.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.533+0000 7fc86f7fe640 1 -- 172.21.15.69:0/2945270725 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc88403d070 con 0x5631fdd6b910 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.533+0000 7fc86f7fe640 1 -- 172.21.15.69:0/2945270725 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc88400ed10 con 0x5631fdd6b910 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc898e33740 10 client.25589 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631fdd84ad0 con 0x5631fdd6b910 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc86f7fe640 1 -- 172.21.15.69:0/2945270725 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc880005500 con 0x5631fdd6b910 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc86f7fe640 1 -- 172.21.15.69:0/2945270725 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc884003aa0 con 0x5631fdd6b910 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc898e33740 10 client.25589 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc898e33740 10 client.25589 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc898e33740 20 client.25589 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc898e33740 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5631fdd85f30 0x5631fdd88320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc898e33740 4 client.25589 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:18.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631fdcddfb0 con 0x5631fdd85f30 2024-08-02T17:48:18.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.534+0000 7fc88ffff640 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5631fdd85f30 0x5631fdd88320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.535+0000 7fc88ffff640 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5631fdd85f30 0x5631fdd88320 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=318ac4e6d4dfd5e4 server_cookie=7113d12e53c6ff2d in_seq=0 out_seq=0 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.535+0000 7fc86f7fe640 10 client.25589 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.536+0000 7fc86f7fe640 1 -- 172.21.15.69:0/2945270725 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5631fdcddfb0 con 0x5631fdd85f30 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.536+0000 7fc86f7fe640 10 client.25589 handle_command_reply: tid=0 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.536+0000 7fc898e33740 1 client.25589 shutdown 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.536+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 msgr2=0x5631fdd736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.536+0000 7fc898e33740 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd736d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x5631fdd683e0 tx=0x7fc8840043b0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 shutdown_connections 2024-08-02T17:48:18.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5631fdd85f30 0x5631fdd88320 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd73c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd736d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 1 --2- 172.21.15.69:0/2945270725 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd73190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 >> 172.21.15.69:0/2945270725 conn(0x5631fdd67580 msgr2=0x5631fdd6d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc86f7fe640 0 client.25589 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:48:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 shutdown_connections 2024-08-02T17:48:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 1 -- 172.21.15.69:0/2945270725 wait complete. 2024-08-02T17:48:18.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.537+0000 7fc898e33740 20 client.25589 trim_cache size 0 max 16384 2024-08-02T17:48:18.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.555+0000 7fc898e33740 1 Processor -- start 2024-08-02T17:48:18.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.555+0000 7fc898e33740 1 -- start start 2024-08-02T17:48:18.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc898e33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6aff0 0x5631fdd71a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc898e33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd6b910 0x5631fdd71fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc898e33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd761d0 0x5631fdd8dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc898e33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901b7b90 con 0x5631fdd6b910 2024-08-02T17:48:18.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc898e33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901b5460 con 0x5631fdd6aff0 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc898e33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901b32b0 con 0x5631fdd761d0 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6aff0 0x5631fdd71a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6aff0 0x5631fdd71a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59376/0 (socket says 172.21.15.69:59376) 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88ffff640 1 -- 172.21.15.69:0/2298626836 learned_addr learned my addr 172.21.15.69:0/2298626836 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd6b910 0x5631fdd71fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc894fd2640 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd761d0 0x5631fdd8dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88ffff640 1 -- 172.21.15.69:0/2298626836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd761d0 msgr2=0x5631fdd8dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88ffff640 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd761d0 0x5631fdd8dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88ffff640 1 -- 172.21.15.69:0/2298626836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd6b910 msgr2=0x5631fdd71fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88ffff640 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd6b910 0x5631fdd71fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88ffff640 1 -- 172.21.15.69:0/2298626836 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631fdd88900 con 0x5631fdd6aff0 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.556+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd6b910 0x5631fdd71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.557+0000 7fc88ffff640 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6aff0 0x5631fdd71a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc88000e720 tx=0x7fc8800093f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dc5689adf99570f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:18.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.557+0000 7fc875ffb640 1 -- 172.21.15.69:0/2298626836 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880020040 con 0x5631fdd6aff0 2024-08-02T17:48:18.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.557+0000 7fc875ffb640 1 -- 172.21.15.69:0/2298626836 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc880035070 con 0x5631fdd6aff0 2024-08-02T17:48:18.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.557+0000 7fc875ffb640 1 -- 172.21.15.69:0/2298626836 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88001cc10 con 0x5631fdd6aff0 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 -- 172.21.15.69:0/2298626836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6aff0 msgr2=0x5631fdd71a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6aff0 0x5631fdd71a60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc88000e720 tx=0x7fc8800093f0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 -- 172.21.15.69:0/2298626836 shutdown_connections 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd6b910 0x5631fdd71fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd761d0 0x5631fdd8dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 --2- 172.21.15.69:0/2298626836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6aff0 0x5631fdd71a60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 -- 172.21.15.69:0/2298626836 >> 172.21.15.69:0/2298626836 conn(0x5631fdd67580 msgr2=0x5631fdd6e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 -- 172.21.15.69:0/2298626836 shutdown_connections 2024-08-02T17:48:18.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.558+0000 7fc898e33740 1 -- 172.21.15.69:0/2298626836 wait complete. 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.559+0000 7fc898e33740 1 -- start start 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.559+0000 7fc898e33740 1 Processor -- start 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc898e33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd7e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc898e33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd79130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd7e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc898e33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd79740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc88ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd7e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47498/0 (socket says 172.21.15.69:47498) 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc88ffff640 1 -- 172.21.15.69:0/3672779561 learned_addr learned my addr 172.21.15.69:0/3672779561 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd79130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901b3920 con 0x5631fdd761d0 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901b4d90 con 0x5631fdd6b910 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8901b3ff0 con 0x5631fdd6aff0 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.560+0000 7fc898e33740 10 client.0 _ll_register_callbacks cb 0x7fc884005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:18.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.561+0000 7fc88ffff640 1 -- 172.21.15.69:0/3672779561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 msgr2=0x5631fdd79130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.561+0000 7fc88ffff640 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd79130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.561+0000 7fc88ffff640 1 -- 172.21.15.69:0/3672779561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 msgr2=0x5631fdd79740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.561+0000 7fc88ffff640 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd79740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.561+0000 7fc88ffff640 1 -- 172.21.15.69:0/3672779561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631fdd79f40 con 0x5631fdd6aff0 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.561+0000 7fc88ffff640 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd7e960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc880009460 tx=0x7fc88000e870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dd08509b25cae41 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.562+0000 7fc86f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.562+0000 7fc86f7fe640 1 -- 172.21.15.69:0/3672779561 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88003d030 con 0x5631fdd6aff0 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.562+0000 7fc86f7fe640 1 -- 172.21.15.69:0/3672779561 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc88001c650 con 0x5631fdd6aff0 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.562+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc880035070 con 0x5631fdd6aff0 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.562+0000 7fc86f7fe640 1 -- 172.21.15.69:0/3672779561 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc880048050 con 0x5631fdd6aff0 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.562+0000 7fc898e33740 10 client.25468 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.562+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631fddefc90 con 0x5631fdd6aff0 2024-08-02T17:48:18.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.562+0000 7fc86f7fe640 1 -- 172.21.15.69:0/3672779561 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc880035070 con 0x5631fdd6aff0 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.563+0000 7fc86f7fe640 1 -- 172.21.15.69:0/3672779561 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc880039420 con 0x5631fdd6aff0 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.563+0000 7fc898e33740 10 client.25468 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.563+0000 7fc898e33740 10 client.25468 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.563+0000 7fc898e33740 20 client.25468 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.563+0000 7fc898e33740 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5631fddf0110 0x5631fddf2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.563+0000 7fc898e33740 4 client.25468 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.563+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631fdcd84c0 con 0x5631fddf0110 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.563+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5631fddf0110 0x5631fddf2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.564+0000 7fc88f7fe640 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5631fddf0110 0x5631fddf2500 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce0980fbebc3a0ad server_cookie=e9d15e6eb9db996a in_seq=0 out_seq=0 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.564+0000 7fc86f7fe640 10 client.25468 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:48:18.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.564+0000 7fc86f7fe640 1 -- 172.21.15.69:0/3672779561 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631fdcd84c0 con 0x5631fddf0110 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.564+0000 7fc86f7fe640 10 client.25468 handle_command_reply: tid=0 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.564+0000 7fc898e33740 1 client.25468 shutdown 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 msgr2=0x5631fdd7e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd7e960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc880009460 tx=0x7fc88000e870 comp rx=0 tx=0).stop 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 shutdown_connections 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5631fddf0110 0x5631fddf2500 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631fdd761d0 0x5631fdd79740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631fdd6b910 0x5631fdd79130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc86f7fe640 0 client.25468 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 --2- 172.21.15.69:0/3672779561 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631fdd6aff0 0x5631fdd7e960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 >> 172.21.15.69:0/3672779561 conn(0x5631fdd67580 msgr2=0x5631fdd71a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 shutdown_connections 2024-08-02T17:48:18.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.565+0000 7fc898e33740 1 -- 172.21.15.69:0/3672779561 wait complete. 2024-08-02T17:48:18.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.566+0000 7fc898e33740 20 client.25468 trim_cache size 0 max 16384 2024-08-02T17:48:18.570 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 -- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8780a2e40 msgr2=0x7fc8780a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8780a2e40 0x7fc8780a5300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc8800058c0 tx=0x7fc880005850 comp rx=0 tx=0).stop 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 -- 172.21.15.69:0/3345843939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8901147e0 msgr2=0x7fc8901a2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8901147e0 0x7fc8901a2fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc88800b9e0 tx=0x7fc88800bea0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 -- 172.21.15.69:0/3345843939 shutdown_connections 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8780a2e40 0x7fc8780a5300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8901147e0 0x7fc8901a2fe0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc8901098d0 0x7fc89019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 --2- 172.21.15.69:0/3345843939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc890108f20 0x7fc89019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 -- 172.21.15.69:0/3345843939 >> 172.21.15.69:0/3345843939 conn(0x7fc89006e000 msgr2=0x7fc890070620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:18.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 -- 172.21.15.69:0/3345843939 shutdown_connections 2024-08-02T17:48:18.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:18.567+0000 7fc896a35640 1 -- 172.21.15.69:0/3345843939 wait complete. 2024-08-02T17:48:19.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:19 smithi077 ceph-mon[37108]: pgmap v60: 129 pgs: 129 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-08-02T17:48:19.353 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:19 smithi069 ceph-mon[32746]: pgmap v60: 129 pgs: 129 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-08-02T17:48:19.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:48:19.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:19 smithi083 ceph-mon[37031]: pgmap v60: 129 pgs: 129 active+clean; 1.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 10 MiB/s wr, 918 op/s 2024-08-02T17:48:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 70.0 s, 4513.7 tps, lat 19.735 ms stddev 23.852 2024-08-02T17:48:19.957 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.574+0000 7f41241ef640 1 -- 172.21.15.69:0/3670987874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c1005f0 msgr2=0x7f411c1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.574+0000 7f41241ef640 1 --2- 172.21.15.69:0/3670987874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c1005f0 0x7f411c1009d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f410c009900 tx=0x7f410c02a440 comp rx=0 tx=0).stop 2024-08-02T17:48:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.576+0000 7f41241ef640 1 -- 172.21.15.69:0/3670987874 shutdown_connections 2024-08-02T17:48:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.576+0000 7f41241ef640 1 --2- 172.21.15.69:0/3670987874 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f411c105bf0 0x7f411c106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.576+0000 7f41241ef640 1 --2- 172.21.15.69:0/3670987874 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c100f10 0x7f411c1056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.576+0000 7f41241ef640 1 --2- 172.21.15.69:0/3670987874 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c1005f0 0x7f411c1009d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.576+0000 7f41241ef640 1 -- 172.21.15.69:0/3670987874 >> 172.21.15.69:0/3670987874 conn(0x7f411c0fc500 msgr2=0x7f411c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:20.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.577+0000 7f41241ef640 1 -- 172.21.15.69:0/3670987874 shutdown_connections 2024-08-02T17:48:20.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.578+0000 7f41241ef640 1 -- 172.21.15.69:0/3670987874 wait complete. 2024-08-02T17:48:20.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.579+0000 7f41241ef640 1 Processor -- start 2024-08-02T17:48:20.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.583+0000 7f41241ef640 1 -- start start 2024-08-02T17:48:20.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.583+0000 7f41241ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1005f0 0x7f411c073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.583+0000 7f41241ef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c100f10 0x7f411c073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.583+0000 7f41241ef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f411c105bf0 0x7f411c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.583+0000 7f41241ef640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f411c1184d0 con 0x7f411c100f10 2024-08-02T17:48:20.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.583+0000 7f41241ef640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f411c118350 con 0x7f411c1005f0 2024-08-02T17:48:20.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.583+0000 7f41241ef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f411c118650 con 0x7f411c105bf0 2024-08-02T17:48:20.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.584+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1005f0 0x7f411c073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.584+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1005f0 0x7f411c073650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59406/0 (socket says 172.21.15.69:59406) 2024-08-02T17:48:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.584+0000 7f4121f8b640 1 -- 172.21.15.69:0/17070198 learned_addr learned my addr 172.21.15.69:0/17070198 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:20.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.585+0000 7f412178a640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c100f10 0x7f411c073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.585+0000 7f412278c640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f411c105bf0 0x7f411c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.587+0000 7f412178a640 1 -- 172.21.15.69:0/17070198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f411c105bf0 msgr2=0x7f411c071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.587+0000 7f412178a640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f411c105bf0 0x7f411c071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.587+0000 7f412178a640 1 -- 172.21.15.69:0/17070198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1005f0 msgr2=0x7f411c073650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.587+0000 7f412178a640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1005f0 0x7f411c073650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.587+0000 7f412178a640 1 -- 172.21.15.69:0/17070198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f411c072200 con 0x7f411c100f10 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.587+0000 7f412178a640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c100f10 0x7f411c073b90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4118009fd0 tx=0x7f411800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.588+0000 7f4112ffd640 1 -- 172.21.15.69:0/17070198 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4118015020 con 0x7f411c100f10 2024-08-02T17:48:20.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.588+0000 7f41241ef640 1 -- 172.21.15.69:0/17070198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f411c0724f0 con 0x7f411c100f10 2024-08-02T17:48:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.588+0000 7f41241ef640 1 -- 172.21.15.69:0/17070198 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f411c112030 con 0x7f411c100f10 2024-08-02T17:48:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.588+0000 7f4112ffd640 1 -- 172.21.15.69:0/17070198 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4118011d10 con 0x7f411c100f10 2024-08-02T17:48:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.588+0000 7f4112ffd640 1 -- 172.21.15.69:0/17070198 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411800f2e0 con 0x7f411c100f10 2024-08-02T17:48:20.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.588+0000 7f4121f8b640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1005f0 0x7f411c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:20.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.595+0000 7f41265ed740 1 Processor -- start 2024-08-02T17:48:20.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.596+0000 7f4112ffd640 1 -- 172.21.15.69:0/17070198 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4118018020 con 0x7f411c100f10 2024-08-02T17:48:20.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.597+0000 7f4112ffd640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41080a2ec0 0x7f41080a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.597+0000 7f4112ffd640 1 -- 172.21.15.69:0/17070198 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f41180ceea0 con 0x7f411c100f10 2024-08-02T17:48:20.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.597+0000 7f41265ed740 1 -- start start 2024-08-02T17:48:20.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.598+0000 7f41265ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef1ff0 0x55c729ef23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.598+0000 7f41265ed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef2910 0x55c729efcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.598+0000 7f41265ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729efd1d0 0x55c729eff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.598+0000 7f41265ed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c729e57a40 con 0x55c729ef2910 2024-08-02T17:48:20.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.598+0000 7f41265ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c729def9d0 con 0x55c729ef1ff0 2024-08-02T17:48:20.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.598+0000 7f41265ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c729e53a00 con 0x55c729efd1d0 2024-08-02T17:48:20.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.600+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef1ff0 0x55c729ef23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.600+0000 7f412178a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef2910 0x55c729efcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.600+0000 7f412178a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef2910 0x55c729efcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37008/0 (socket says 172.21.15.69:37008) 2024-08-02T17:48:20.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.600+0000 7f412178a640 1 -- 172.21.15.69:0/36171630 learned_addr learned my addr 172.21.15.69:0/36171630 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:20.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.601+0000 7f412178a640 1 -- 172.21.15.69:0/36171630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729efd1d0 msgr2=0x55c729eff5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:48:20.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.601+0000 7f412178a640 1 --2- 172.21.15.69:0/36171630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729efd1d0 0x55c729eff5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.601+0000 7f412178a640 1 -- 172.21.15.69:0/36171630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef1ff0 msgr2=0x55c729ef23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:20.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.601+0000 7f412178a640 1 --2- 172.21.15.69:0/36171630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef1ff0 0x55c729ef23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.601+0000 7f412178a640 1 -- 172.21.15.69:0/36171630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c729df6650 con 0x55c729ef2910 2024-08-02T17:48:20.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.601+0000 7f412178a640 1 --2- 172.21.15.69:0/36171630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef2910 0x55c729efcc90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4118097a80 tx=0x7f411809afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d3a59baaecc45c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:20.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.603+0000 7f412278c640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41080a2ec0 0x7f41080a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.604+0000 7f4107fff640 1 -- 172.21.15.69:0/36171630 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41180a3d20 con 0x55c729ef2910 2024-08-02T17:48:20.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.605+0000 7f4107fff640 1 -- 172.21.15.69:0/36171630 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41180bb0b0 con 0x55c729ef2910 2024-08-02T17:48:20.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.606+0000 7f41265ed740 1 -- 172.21.15.69:0/36171630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef2910 msgr2=0x55c729efcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.606+0000 7f41265ed740 1 --2- 172.21.15.69:0/36171630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef2910 0x55c729efcc90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4118097a80 tx=0x7f411809afb0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.607+0000 7f41265ed740 1 -- 172.21.15.69:0/36171630 shutdown_connections 2024-08-02T17:48:20.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.607+0000 7f41265ed740 1 --2- 172.21.15.69:0/36171630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef2910 0x55c729efcc90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.607+0000 7f41265ed740 1 --2- 172.21.15.69:0/36171630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729efd1d0 0x55c729eff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.607+0000 7f41265ed740 1 --2- 172.21.15.69:0/36171630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef1ff0 0x55c729ef23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.607+0000 7f41265ed740 1 -- 172.21.15.69:0/36171630 >> 172.21.15.69:0/36171630 conn(0x55c729eee580 msgr2=0x55c729eee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:20.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.607+0000 7f41265ed740 1 -- 172.21.15.69:0/36171630 shutdown_connections 2024-08-02T17:48:20.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.607+0000 7f412278c640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41080a2ec0 0x7f41080a5380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55c729eef900 tx=0x7f411400d040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:20.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.608+0000 7f41265ed740 1 -- 172.21.15.69:0/36171630 wait complete. 2024-08-02T17:48:20.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.608+0000 7f41265ed740 1 -- start start 2024-08-02T17:48:20.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.608+0000 7f41265ed740 1 Processor -- start 2024-08-02T17:48:20.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.609+0000 7f41265ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729efa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.609+0000 7f41265ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729efa6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.609+0000 7f41265ed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729efac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.609+0000 7f41265ed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c729effdf0 con 0x55c729efd1d0 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.609+0000 7f41265ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c729e12b20 con 0x55c729ef2910 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.609+0000 7f41265ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c729efff70 con 0x55c729ef1ff0 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.609+0000 7f41265ed740 10 client.0 _ll_register_callbacks cb 0x55c729e10bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.609+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729efa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729efa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47528/0 (socket says 172.21.15.69:47528) 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f4121f8b640 1 -- 172.21.15.69:0/1303173825 learned_addr learned my addr 172.21.15.69:0/1303173825 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f412178a640 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729efa6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f412278c640 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729efac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f4121f8b640 1 -- 172.21.15.69:0/1303173825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 msgr2=0x55c729efa6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f4121f8b640 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729efa6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f4121f8b640 1 -- 172.21.15.69:0/1303173825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 msgr2=0x55c729efac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f4121f8b640 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729efac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.610+0000 7f4121f8b640 1 -- 172.21.15.69:0/1303173825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c729bc93c0 con 0x55c729ef1ff0 2024-08-02T17:48:20.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.612+0000 7f4121f8b640 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729efa180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f410c0038e0 tx=0x7f410c003910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b31ee33096c41a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:20.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.612+0000 7f41057fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:20.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.612+0000 7f41057fa640 1 -- 172.21.15.69:0/1303173825 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c03f070 con 0x55c729ef1ff0 2024-08-02T17:48:20.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.612+0000 7f41057fa640 1 -- 172.21.15.69:0/1303173825 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f410c005020 con 0x55c729ef1ff0 2024-08-02T17:48:20.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.612+0000 7f41265ed740 1 -- 172.21.15.69:0/1303173825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c729e64fb0 con 0x55c729ef1ff0 2024-08-02T17:48:20.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.613+0000 7f41057fa640 1 -- 172.21.15.69:0/1303173825 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f410c04b050 con 0x55c729ef1ff0 2024-08-02T17:48:20.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.613+0000 7f41265ed740 10 client.25474 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:20.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.613+0000 7f41265ed740 1 -- 172.21.15.69:0/1303173825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c729f0bc10 con 0x55c729ef1ff0 2024-08-02T17:48:20.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.613+0000 7f41057fa640 1 -- 172.21.15.69:0/1303173825 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c729e64fb0 con 0x55c729ef1ff0 2024-08-02T17:48:20.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.613+0000 7f41057fa640 1 -- 172.21.15.69:0/1303173825 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f410c031540 con 0x55c729ef1ff0 2024-08-02T17:48:20.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.614+0000 7f41265ed740 10 client.25474 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:20.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.614+0000 7f41265ed740 10 client.25474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:20.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.614+0000 7f41265ed740 20 client.25474 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:20.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.614+0000 7f41265ed740 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c729f0d120 0x55c729f0f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.614+0000 7f41265ed740 4 client.25474 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:20.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.614+0000 7f41265ed740 1 -- 172.21.15.69:0/1303173825 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c729e5f4c0 con 0x55c729f0d120 2024-08-02T17:48:20.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.614+0000 7f412178a640 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c729f0d120 0x55c729f0f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.615+0000 7f412178a640 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c729f0d120 0x55c729f0f540 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e99399640f9679 server_cookie=1982424b0793e67f in_seq=0 out_seq=0 2024-08-02T17:48:20.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.615+0000 7f41057fa640 10 client.25474 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:20.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.616+0000 7f41057fa640 1 -- 172.21.15.69:0/1303173825 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c729e5f4c0 con 0x55c729f0d120 2024-08-02T17:48:20.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.616+0000 7f41057fa640 10 client.25474 handle_command_reply: tid=0 2024-08-02T17:48:20.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.616+0000 7f41265ed740 1 client.25474 shutdown 2024-08-02T17:48:20.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 -- 172.21.15.69:0/1303173825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 msgr2=0x55c729efa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729efa180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f410c0038e0 tx=0x7f410c003910 comp rx=0 tx=0).stop 2024-08-02T17:48:20.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 -- 172.21.15.69:0/1303173825 shutdown_connections 2024-08-02T17:48:20.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729efac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c729f0d120 0x55c729f0f540 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729efa6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 --2- 172.21.15.69:0/1303173825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729efa180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 -- 172.21.15.69:0/1303173825 >> 172.21.15.69:0/1303173825 conn(0x55c729eee580 msgr2=0x55c729ef42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:20.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41057fa640 0 client.25474 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:20.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 -- 172.21.15.69:0/1303173825 shutdown_connections 2024-08-02T17:48:20.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 1 -- 172.21.15.69:0/1303173825 wait complete. 2024-08-02T17:48:20.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.618+0000 7f41265ed740 20 client.25474 trim_cache size 0 max 16384 2024-08-02T17:48:20.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.641+0000 7f41265ed740 1 Processor -- start 2024-08-02T17:48:20.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.641+0000 7f41265ed740 1 -- start start 2024-08-02T17:48:20.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.641+0000 7f41265ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729ef8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.641+0000 7f41265ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729ef8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.641+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729ef8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.641+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729ef8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47538/0 (socket says 172.21.15.69:47538) 2024-08-02T17:48:20.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f4121f8b640 1 -- 172.21.15.69:0/612950448 learned_addr learned my addr 172.21.15.69:0/612950448 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f41265ed740 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729f14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f412178a640 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729ef8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f41265ed740 1 -- 172.21.15.69:0/612950448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4100012850 con 0x55c729efd1d0 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f41265ed740 1 -- 172.21.15.69:0/612950448 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4100010120 con 0x55c729ef2910 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f41265ed740 1 -- 172.21.15.69:0/612950448 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f410000df70 con 0x55c729ef1ff0 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f412278c640 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729f14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f412278c640 1 -- 172.21.15.69:0/612950448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 msgr2=0x55c729ef8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f412278c640 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729ef8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f412278c640 1 -- 172.21.15.69:0/612950448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 msgr2=0x55c729ef8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f412278c640 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729ef8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f412278c640 1 -- 172.21.15.69:0/612950448 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c729efb5f0 con 0x55c729efd1d0 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f4121f8b640 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729ef8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:20.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.642+0000 7f412178a640 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729ef8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:48:20.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.644+0000 7f412278c640 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729f14e10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4114016770 tx=0x7f4114009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7c31a933518e66 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:20.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.644+0000 7f41077fe640 1 -- 172.21.15.69:0/612950448 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140139a0 con 0x55c729efd1d0 2024-08-02T17:48:20.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.644+0000 7f41077fe640 1 -- 172.21.15.69:0/612950448 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4114027070 con 0x55c729efd1d0 2024-08-02T17:48:20.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.644+0000 7f41077fe640 1 -- 172.21.15.69:0/612950448 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41140213d0 con 0x55c729efd1d0 2024-08-02T17:48:20.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 -- 172.21.15.69:0/612950448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 msgr2=0x55c729f14e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729f14e10 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4114016770 tx=0x7f4114009f90 comp rx=0 tx=0).stop 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 -- 172.21.15.69:0/612950448 shutdown_connections 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729efd1d0 0x55c729f14e10 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729ef2910 0x55c729ef8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 --2- 172.21.15.69:0/612950448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef1ff0 0x55c729ef8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 -- 172.21.15.69:0/612950448 >> 172.21.15.69:0/612950448 conn(0x55c729eee580 msgr2=0x55c729ee91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 -- 172.21.15.69:0/612950448 shutdown_connections 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 -- 172.21.15.69:0/612950448 wait complete. 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 -- start start 2024-08-02T17:48:20.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.645+0000 7f41265ed740 1 Processor -- start 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f41265ed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef1ff0 0x55c729f00230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f41265ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef2910 0x55c729f00770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f41265ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729efd1d0 0x55c729f00cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f41265ed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f410000e5e0 con 0x55c729ef1ff0 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef1ff0 0x55c729f00230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f4121f8b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef1ff0 0x55c729f00230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37032/0 (socket says 172.21.15.69:37032) 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f4121f8b640 1 -- 172.21.15.69:0/2040500438 learned_addr learned my addr 172.21.15.69:0/2040500438 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f410000fa50 con 0x55c729efd1d0 2024-08-02T17:48:20.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.646+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f410000ecb0 con 0x55c729ef2910 2024-08-02T17:48:20.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f41265ed740 10 client.0 _ll_register_callbacks cb 0x7f410c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:20.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f412178a640 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef2910 0x55c729f00770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f412278c640 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729efd1d0 0x55c729f00cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f4121f8b640 1 -- 172.21.15.69:0/2040500438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef2910 msgr2=0x55c729f00770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:20.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f4121f8b640 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef2910 0x55c729f00770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f4121f8b640 1 -- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729efd1d0 msgr2=0x55c729f00cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:20.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f4121f8b640 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729efd1d0 0x55c729f00cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f4121f8b640 1 -- 172.21.15.69:0/2040500438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c729f75e20 con 0x55c729ef1ff0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.647+0000 7f412278c640 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729efd1d0 0x55c729f00cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.648+0000 7f4121f8b640 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef1ff0 0x55c729f00230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f410c0096b0 tx=0x7f410c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91b2e1bea767328 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.648+0000 7f41057fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.648+0000 7f41057fa640 1 -- 172.21.15.69:0/2040500438 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c03f030 con 0x55c729ef1ff0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.648+0000 7f41057fa640 1 -- 172.21.15.69:0/2040500438 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f410c02ac40 con 0x55c729ef1ff0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.648+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4114027070 con 0x55c729ef1ff0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.649+0000 7f41057fa640 1 -- 172.21.15.69:0/2040500438 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f410c035070 con 0x55c729ef1ff0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.649+0000 7f41265ed740 10 client.15975 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.649+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c729f770b0 con 0x55c729ef1ff0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.650+0000 7f41057fa640 1 -- 172.21.15.69:0/2040500438 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4114027070 con 0x55c729ef1ff0 2024-08-02T17:48:20.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.650+0000 7f41057fa640 1 -- 172.21.15.69:0/2040500438 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f410c058920 con 0x55c729ef1ff0 2024-08-02T17:48:20.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.650+0000 7f41265ed740 10 client.15975 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:20.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.650+0000 7f41265ed740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:20.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.650+0000 7f41265ed740 20 client.15975 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:20.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.650+0000 7f41265ed740 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c729f77800 0x55c729f79c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:20.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.650+0000 7f41265ed740 4 client.15975 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:48:20.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.650+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c729ddc170 con 0x55c729f77800 2024-08-02T17:48:20.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.651+0000 7f412178a640 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c729f77800 0x55c729f79c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:20.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.652+0000 7f41057fa640 10 client.15975 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:20.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.652+0000 7f412178a640 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c729f77800 0x55c729f79c20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1d352e26a88b7c server_cookie=eb33b85df44f2891 in_seq=0 out_seq=0 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.652+0000 7f41057fa640 1 -- 172.21.15.69:0/2040500438 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c729ddc170 con 0x55c729f77800 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.652+0000 7f41057fa640 10 client.15975 handle_command_reply: tid=0 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.652+0000 7f41265ed740 1 client.15975 shutdown 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.652+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef1ff0 msgr2=0x55c729f00230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.652+0000 7f41265ed740 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef1ff0 0x55c729f00230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f410c0096b0 tx=0x7f410c009260 comp rx=0 tx=0).stop 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.653+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 shutdown_connections 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.653+0000 7f41265ed740 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c729f77800 0x55c729f79c20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.653+0000 7f41265ed740 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c729efd1d0 0x55c729f00cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.653+0000 7f41057fa640 0 client.15975 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.653+0000 7f41265ed740 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c729ef2910 0x55c729f00770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.653+0000 7f41265ed740 1 --2- 172.21.15.69:0/2040500438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c729ef1ff0 0x55c729f00230 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.653+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 >> 172.21.15.69:0/2040500438 conn(0x55c729eee580 msgr2=0x55c729ef8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:20.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.654+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 shutdown_connections 2024-08-02T17:48:20.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.654+0000 7f41265ed740 1 -- 172.21.15.69:0/2040500438 wait complete. 2024-08-02T17:48:20.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.654+0000 7f41265ed740 20 client.15975 trim_cache size 0 max 16384 2024-08-02T17:48:20.657 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:48:20.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 -- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41080a2ec0 msgr2=0x7f41080a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41080a2ec0 0x7f41080a5380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55c729eef900 tx=0x7f411400d040 comp rx=0 tx=0).stop 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 -- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c100f10 msgr2=0x7f411c073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c100f10 0x7f411c073b90 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4118009fd0 tx=0x7f411800b870 comp rx=0 tx=0).stop 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 -- 172.21.15.69:0/17070198 shutdown_connections 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f411c100f10 0x7f411c073b90 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41080a2ec0 0x7f41080a5380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f411c105bf0 0x7f411c071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 --2- 172.21.15.69:0/17070198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f411c1005f0 0x7f411c073650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.655+0000 7f41241ef640 1 -- 172.21.15.69:0/17070198 >> 172.21.15.69:0/17070198 conn(0x7f411c0fc500 msgr2=0x7f411c106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.656+0000 7f41241ef640 1 -- 172.21.15.69:0/17070198 shutdown_connections 2024-08-02T17:48:20.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:20.656+0000 7f41241ef640 1 -- 172.21.15.69:0/17070198 wait complete. 2024-08-02T17:48:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:21 smithi077 ceph-mon[37108]: pgmap v61: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2024-08-02T17:48:21.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:21 smithi069 ceph-mon[32746]: pgmap v61: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2024-08-02T17:48:21.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:21 smithi083 ceph-mon[37031]: pgmap v61: 129 pgs: 129 active+clean; 1.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 23 MiB/s wr, 1.63k op/s 2024-08-02T17:48:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 -- 172.21.15.69:0/1774076989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da0106710 msgr2=0x7f8da0106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 --2- 172.21.15.69:0/1774076989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da0106710 0x7f8da0106af0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009ca0 tx=0x7f8d9002a8b0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 -- 172.21.15.69:0/1774076989 shutdown_connections 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 --2- 172.21.15.69:0/1774076989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da01010a0 0x7f8da010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 --2- 172.21.15.69:0/1774076989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da0100700 0x7f8da0100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 --2- 172.21.15.69:0/1774076989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da0106710 0x7f8da0106af0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 -- 172.21.15.69:0/1774076989 >> 172.21.15.69:0/1774076989 conn(0x7f8da00fc3e0 msgr2=0x7f8da00fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 -- 172.21.15.69:0/1774076989 shutdown_connections 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.746+0000 7f8da598b640 1 -- 172.21.15.69:0/1774076989 wait complete. 2024-08-02T17:48:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.747+0000 7f8da598b640 1 Processor -- start 2024-08-02T17:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.747+0000 7f8da598b640 1 -- start start 2024-08-02T17:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8da598b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da0100700 0x7f8da019a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8da598b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da01010a0 0x7f8da019a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8da598b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da0106710 0x7f8da019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8da598b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8da0111c90 con 0x7f8da01010a0 2024-08-02T17:48:22.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8da598b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8da0111b10 con 0x7f8da0106710 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8da598b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8da0111e10 con 0x7f8da0100700 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da0100700 0x7f8da019a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da0100700 0x7f8da019a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47574/0 (socket says 172.21.15.69:47574) 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8d9effd640 1 -- 172.21.15.69:0/1203623518 learned_addr learned my addr 172.21.15.69:0/1203623518 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8d9effd640 1 -- 172.21.15.69:0/1203623518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da0106710 msgr2=0x7f8da019ebf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8d9effd640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da0106710 0x7f8da019ebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8d9effd640 1 -- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da01010a0 msgr2=0x7f8da019a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8d9effd640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da01010a0 0x7f8da019a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.748+0000 7f8d9effd640 1 -- 172.21.15.69:0/1203623518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da019f2f0 con 0x7f8da0100700 2024-08-02T17:48:22.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.749+0000 7f8d9effd640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da0100700 0x7f8da019a330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d900097b0 tx=0x7f8d90003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.749+0000 7f8da4989640 1 -- 172.21.15.69:0/1203623518 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90030b20 con 0x7f8da0100700 2024-08-02T17:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.749+0000 7f8da598b640 1 -- 172.21.15.69:0/1203623518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da019f580 con 0x7f8da0100700 2024-08-02T17:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.749+0000 7f8da4989640 1 -- 172.21.15.69:0/1203623518 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d90040070 con 0x7f8da0100700 2024-08-02T17:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.749+0000 7f8da598b640 1 -- 172.21.15.69:0/1203623518 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da006b940 con 0x7f8da0100700 2024-08-02T17:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.750+0000 7f8da4989640 1 -- 172.21.15.69:0/1203623518 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90004460 con 0x7f8da0100700 2024-08-02T17:48:22.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.752+0000 7f8da4989640 1 -- 172.21.15.69:0/1203623518 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8d90030cc0 con 0x7f8da0100700 2024-08-02T17:48:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.753+0000 7f8da4989640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d8c0a2d40 0x7f8d8c0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.754+0000 7f8da4989640 1 -- 172.21.15.69:0/1203623518 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8d900f09b0 con 0x7f8da0100700 2024-08-02T17:48:22.755 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.754+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d8c0a2d40 0x7f8d8c0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8da87b9740 1 Processor -- start 2024-08-02T17:48:22.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8da87b9740 1 -- start start 2024-08-02T17:48:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6bb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 0x55a8ae6bdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8da87b9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8ae6149b0 con 0x55a8ae6b10d0 2024-08-02T17:48:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8da87b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a8ae5bc800 con 0x55a8ae6b07b0 2024-08-02T17:48:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8da87b9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8ae610970 con 0x55a8ae6bb970 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59472/0 (socket says 172.21.15.69:59472) 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.755+0000 7f8d9effd640 1 -- 172.21.15.69:0/4035904373 learned_addr learned my addr 172.21.15.69:0/4035904373 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d9effd640 1 -- 172.21.15.69:0/4035904373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 msgr2=0x55a8ae6bdd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d9effd640 1 --2- 172.21.15.69:0/4035904373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 0x55a8ae6bdd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d9effd640 1 -- 172.21.15.69:0/4035904373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b10d0 msgr2=0x55a8ae6bb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d9effd640 1 --2- 172.21.15.69:0/4035904373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6bb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d9effd640 1 -- 172.21.15.69:0/4035904373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ae328210 con 0x55a8ae6b07b0 2024-08-02T17:48:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d8c0a2d40 0x7f8d8c0a5200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d880058c0 tx=0x7f8d88005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d9effd640 1 --2- 172.21.15.69:0/4035904373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b0b90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d900edeb0 tx=0x7f8d9002a8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9152e3296f2fe7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d95ffb640 1 -- 172.21.15.69:0/4035904373 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d900c0fd0 con 0x55a8ae6b07b0 2024-08-02T17:48:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d95ffb640 1 -- 172.21.15.69:0/4035904373 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d90036020 con 0x55a8ae6b07b0 2024-08-02T17:48:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.756+0000 7f8d95ffb640 1 -- 172.21.15.69:0/4035904373 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90043370 con 0x55a8ae6b07b0 2024-08-02T17:48:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.757+0000 7f8da87b9740 1 -- 172.21.15.69:0/4035904373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 msgr2=0x55a8ae6b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.757+0000 7f8da87b9740 1 --2- 172.21.15.69:0/4035904373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b0b90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8d900edeb0 tx=0x7f8d9002a8b0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.758+0000 7f8da87b9740 1 -- 172.21.15.69:0/4035904373 shutdown_connections 2024-08-02T17:48:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.758+0000 7f8da87b9740 1 --2- 172.21.15.69:0/4035904373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6bb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.758+0000 7f8da87b9740 1 --2- 172.21.15.69:0/4035904373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 0x55a8ae6bdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.758+0000 7f8da87b9740 1 --2- 172.21.15.69:0/4035904373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b0b90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.758+0000 7f8da87b9740 1 -- 172.21.15.69:0/4035904373 >> 172.21.15.69:0/4035904373 conn(0x55a8ae6acd20 msgr2=0x55a8ae6ad100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.758+0000 7f8da87b9740 1 -- 172.21.15.69:0/4035904373 shutdown_connections 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.758+0000 7f8da87b9740 1 -- 172.21.15.69:0/4035904373 wait complete. 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.758+0000 7f8da87b9740 1 -- start start 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.759+0000 7f8da87b9740 1 Processor -- start 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6b8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6bb970 0x55a8ae6b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8d9e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6b8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8da87b9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8ae6be300 con 0x55a8ae6b07b0 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8da87b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a8ae5d0500 con 0x55a8ae6bb970 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6bb970 0x55a8ae6b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8d9e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6b8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47588/0 (socket says 172.21.15.69:47588) 2024-08-02T17:48:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37060/0 (socket says 172.21.15.69:37060) 2024-08-02T17:48:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8d9e7fc640 1 -- 172.21.15.69:0/1586189754 learned_addr learned my addr 172.21.15.69:0/1586189754 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8ae6be480 con 0x55a8ae6b10d0 2024-08-02T17:48:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.760+0000 7f8da87b9740 10 client.0 _ll_register_callbacks cb 0x55a8ae327a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.761+0000 7f8d9e7fc640 1 -- 172.21.15.69:0/1586189754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6bb970 msgr2=0x55a8ae6b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.761+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6bb970 0x55a8ae6b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.761+0000 7f8d9e7fc640 1 -- 172.21.15.69:0/1586189754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 msgr2=0x55a8ae6b8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.761+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.761+0000 7f8d9e7fc640 1 -- 172.21.15.69:0/1586189754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ae3f4620 con 0x55a8ae6b10d0 2024-08-02T17:48:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.761+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6b8e80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d88010710 tx=0x7f8d880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8838339fa2abe1e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.761+0000 7f8d7effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.762+0000 7f8d7effd640 1 -- 172.21.15.69:0/1586189754 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88020070 con 0x55a8ae6b10d0 2024-08-02T17:48:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.762+0000 7f8d7effd640 1 -- 172.21.15.69:0/1586189754 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8d8801c8f0 con 0x55a8ae6b10d0 2024-08-02T17:48:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.762+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d90036020 con 0x55a8ae6b10d0 2024-08-02T17:48:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.762+0000 7f8d7effd640 1 -- 172.21.15.69:0/1586189754 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d8801caf0 con 0x55a8ae6b10d0 2024-08-02T17:48:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.762+0000 7f8da87b9740 10 client.25486 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.762+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8ae6c9fe0 con 0x55a8ae6b10d0 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.762+0000 7f8d7effd640 1 -- 172.21.15.69:0/1586189754 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8801cc90 con 0x55a8ae6b10d0 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.763+0000 7f8d7effd640 1 -- 172.21.15.69:0/1586189754 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8d8802f680 con 0x55a8ae6b10d0 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.763+0000 7f8da87b9740 10 client.25486 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.763+0000 7f8da87b9740 10 client.25486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.763+0000 7f8da87b9740 20 client.25486 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.763+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8ae6cb7a0 0x55a8ae6cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.763+0000 7f8da87b9740 4 client.25486 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.763+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8d90036020 con 0x55a8ae6cb7a0 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.763+0000 7f8d9effd640 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8ae6cb7a0 0x55a8ae6cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.764+0000 7f8d7effd640 10 client.25486 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.764+0000 7f8d9effd640 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8ae6cb7a0 0x55a8ae6cdb90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c710d8cc29d26ba6 server_cookie=af92f22bf2694601 in_seq=0 out_seq=0 2024-08-02T17:48:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.765+0000 7f8d7effd640 1 -- 172.21.15.69:0/1586189754 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f8d90036020 con 0x55a8ae6cb7a0 2024-08-02T17:48:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.765+0000 7f8d7effd640 10 client.25486 handle_command_reply: tid=0 2024-08-02T17:48:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.765+0000 7f8da87b9740 1 client.25486 shutdown 2024-08-02T17:48:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 msgr2=0x55a8ae6b8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6b8e80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d88010710 tx=0x7f8d880046c0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 shutdown_connections 2024-08-02T17:48:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8ae6cb7a0 0x55a8ae6cdb90 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6bb970 0x55a8ae6b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8d7effd640 0 client.25486 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6b8e80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1586189754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6b8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.766+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 >> 172.21.15.69:0/1586189754 conn(0x55a8ae6acd20 msgr2=0x55a8ae6b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.768+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 shutdown_connections 2024-08-02T17:48:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.768+0000 7f8da87b9740 1 -- 172.21.15.69:0/1586189754 wait complete. 2024-08-02T17:48:22.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.768+0000 7f8da87b9740 20 client.25486 trim_cache size 0 max 16384 2024-08-02T17:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.794+0000 7f8da87b9740 1 Processor -- start 2024-08-02T17:48:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.795+0000 7f8da87b9740 1 -- start start 2024-08-02T17:48:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.795+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.795+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6d0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.795+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6bb970 0x55a8ae6d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.795+0000 7f8da87b9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01bc160 con 0x55a8ae6bb970 2024-08-02T17:48:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.795+0000 7f8da87b9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b9a30 con 0x55a8ae6b07b0 2024-08-02T17:48:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.795+0000 7f8da87b9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8da0101e80 con 0x55a8ae6b10d0 2024-08-02T17:48:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.795+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6bb970 0x55a8ae6d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59486/0 (socket says 172.21.15.69:59486) 2024-08-02T17:48:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 -- 172.21.15.69:0/1336636453 learned_addr learned my addr 172.21.15.69:0/1336636453 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6bb970 0x55a8ae6d10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37076/0 (socket says 172.21.15.69:37076) 2024-08-02T17:48:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6d0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 -- 172.21.15.69:0/1336636453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 msgr2=0x55a8ae6d0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6d0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 -- 172.21.15.69:0/1336636453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6bb970 msgr2=0x55a8ae6d10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6bb970 0x55a8ae6d10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 -- 172.21.15.69:0/1336636453 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8ae6bdf90 con 0x55a8ae6b07b0 2024-08-02T17:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9f7fe640 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6bb970 0x55a8ae6d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d9effd640 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6c6de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8d900ee340 tx=0x7f8d900bcda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a47b5bc8abbc3a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.796+0000 7f8d957fa640 1 -- 172.21.15.69:0/1336636453 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d900c0fd0 con 0x55a8ae6b07b0 2024-08-02T17:48:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.797+0000 7f8d957fa640 1 -- 172.21.15.69:0/1336636453 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d90036070 con 0x55a8ae6b07b0 2024-08-02T17:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.797+0000 7f8d957fa640 1 -- 172.21.15.69:0/1336636453 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d900bce80 con 0x55a8ae6b07b0 2024-08-02T17:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.798+0000 7f8da87b9740 1 -- 172.21.15.69:0/1336636453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 msgr2=0x55a8ae6c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.798+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6c6de0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8d900ee340 tx=0x7f8d900bcda0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.798+0000 7f8da87b9740 1 -- 172.21.15.69:0/1336636453 shutdown_connections 2024-08-02T17:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.798+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6bb970 0x55a8ae6d10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.798+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6d0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.798+0000 7f8da87b9740 1 --2- 172.21.15.69:0/1336636453 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6c6de0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.798+0000 7f8da87b9740 1 -- 172.21.15.69:0/1336636453 >> 172.21.15.69:0/1336636453 conn(0x55a8ae6acd20 msgr2=0x55a8ae6c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.798+0000 7f8da87b9740 1 -- 172.21.15.69:0/1336636453 shutdown_connections 2024-08-02T17:48:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.799+0000 7f8da87b9740 1 -- 172.21.15.69:0/1336636453 wait complete. 2024-08-02T17:48:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.799+0000 7f8da87b9740 1 -- start start 2024-08-02T17:48:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.800+0000 7f8da87b9740 1 Processor -- start 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.800+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.800+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8da87b9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 0x55a8ae7350e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37086/0 (socket says 172.21.15.69:37086) 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8da87b9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b7f30 con 0x55a8ae6b07b0 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6cf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59488/0 (socket says 172.21.15.69:59488) 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 -- 172.21.15.69:0/987574947 learned_addr learned my addr 172.21.15.69:0/987574947 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b9360 con 0x55a8ae6b10d0 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b85c0 con 0x55a8ae6bb970 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8da87b9740 10 client.0 _ll_register_callbacks cb 0x7f8d88014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9f7fe640 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 0x55a8ae7350e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 -- 172.21.15.69:0/987574947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 msgr2=0x55a8ae7350e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 0x55a8ae7350e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 -- 172.21.15.69:0/987574947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 msgr2=0x55a8ae6cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6cecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 -- 172.21.15.69:0/987574947 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8ae735620 con 0x55a8ae6b10d0 2024-08-02T17:48:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9effd640 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.801+0000 7f8d9e7fc640 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6cf230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005fd0 tx=0x7f8d8801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f86bc17af341fef server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.802+0000 7f8d7effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.802+0000 7f8d7effd640 1 -- 172.21.15.69:0/987574947 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88020040 con 0x55a8ae6b10d0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.802+0000 7f8d7effd640 1 -- 172.21.15.69:0/987574947 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8d88034070 con 0x55a8ae6b10d0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.802+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d90036070 con 0x55a8ae6b10d0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.803+0000 7f8d7effd640 1 -- 172.21.15.69:0/987574947 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d88009810 con 0x55a8ae6b10d0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.803+0000 7f8da87b9740 10 client.25619 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.803+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8ae736790 con 0x55a8ae6b10d0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.803+0000 7f8d7effd640 1 -- 172.21.15.69:0/987574947 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8802f630 con 0x55a8ae6b10d0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.803+0000 7f8d7effd640 1 -- 172.21.15.69:0/987574947 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8d880394d0 con 0x55a8ae6b10d0 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.803+0000 7f8da87b9740 10 client.25619 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.803+0000 7f8da87b9740 10 client.25619 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.803+0000 7f8da87b9740 20 client.25619 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.804+0000 7f8da87b9740 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8ae6cb7a0 0x55a8ae738f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.804+0000 7f8da87b9740 4 client.25619 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:48:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.804+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f8d90036070 con 0x55a8ae6cb7a0 2024-08-02T17:48:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.804+0000 7f8d9effd640 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8ae6cb7a0 0x55a8ae738f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.805+0000 7f8d9effd640 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8ae6cb7a0 0x55a8ae738f50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0adf53176e708b server_cookie=b341b17ceed30a32 in_seq=0 out_seq=0 2024-08-02T17:48:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.805+0000 7f8d7effd640 10 client.25619 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.821+0000 7f8d7effd640 1 -- 172.21.15.69:0/987574947 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f8d90036070 con 0x55a8ae6cb7a0 2024-08-02T17:48:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.821+0000 7f8d7effd640 10 client.25619 handle_command_reply: tid=0 2024-08-02T17:48:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.821+0000 7f8da87b9740 1 client.25619 shutdown 2024-08-02T17:48:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b10d0 msgr2=0x55a8ae6cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6cf230 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005fd0 tx=0x7f8d8801c9a0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 shutdown_connections 2024-08-02T17:48:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8ae6cb7a0 0x55a8ae738f50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8ae6bb970 0x55a8ae7350e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8ae6b10d0 0x55a8ae6cf230 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 --2- 172.21.15.69:0/987574947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8ae6b07b0 0x55a8ae6cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 >> 172.21.15.69:0/987574947 conn(0x55a8ae6acd20 msgr2=0x55a8ae6b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8d7effd640 0 client.25619 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 shutdown_connections 2024-08-02T17:48:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 1 -- 172.21.15.69:0/987574947 wait complete. 2024-08-02T17:48:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.822+0000 7f8da87b9740 20 client.25619 trim_cache size 0 max 16384 2024-08-02T17:48:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:48:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:48:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:48:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:48:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 -- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d8c0a2d40 msgr2=0x7f8d8c0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d8c0a2d40 0x7f8d8c0a5200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8d880058c0 tx=0x7f8d88005850 comp rx=0 tx=0).stop 2024-08-02T17:48:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 -- 172.21.15.69:0/1203623518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da0100700 msgr2=0x7f8da019a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da0100700 0x7f8da019a330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d900097b0 tx=0x7f8d90003ba0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 -- 172.21.15.69:0/1203623518 shutdown_connections 2024-08-02T17:48:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8da01010a0 0x7f8da019a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d8c0a2d40 0x7f8d8c0a5200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8da0106710 0x7f8da019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 --2- 172.21.15.69:0/1203623518 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8da0100700 0x7f8da019a330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.823+0000 7f8da598b640 1 -- 172.21.15.69:0/1203623518 >> 172.21.15.69:0/1203623518 conn(0x7f8da00fc3e0 msgr2=0x7f8da00fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.824+0000 7f8da598b640 1 -- 172.21.15.69:0/1203623518 shutdown_connections 2024-08-02T17:48:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:22.824+0000 7f8da598b640 1 -- 172.21.15.69:0/1203623518 wait complete. 2024-08-02T17:48:22.918 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:48:22.918 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:48:23.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:23 smithi077 ceph-mon[37108]: pgmap v62: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 31 MiB/s wr, 1.87k op/s 2024-08-02T17:48:23.351 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:23.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:23 smithi069 ceph-mon[32746]: pgmap v62: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 31 MiB/s wr, 1.87k op/s 2024-08-02T17:48:23.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:23 smithi083 ceph-mon[37031]: pgmap v62: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 31 MiB/s wr, 1.87k op/s 2024-08-02T17:48:24.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.831+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3698898671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc108180 msgr2=0x7f09cc108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.831+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3698898671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc108180 0x7f09cc108560 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009860 tx=0x7f09bc02a360 comp rx=0 tx=0).stop 2024-08-02T17:48:24.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.832+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3698898671 shutdown_connections 2024-08-02T17:48:24.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.832+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3698898671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09cc10d260 0x7f09cc114de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.832+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3698898671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09cc108b30 0x7f09cc10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.832+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3698898671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc108180 0x7f09cc108560 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.832+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3698898671 >> 172.21.15.69:0/3698898671 conn(0x7f09cc0778d0 msgr2=0x7f09cc079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:24.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.832+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3698898671 shutdown_connections 2024-08-02T17:48:24.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.832+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3698898671 wait complete. 2024-08-02T17:48:24.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.833+0000 7f09d1b0f640 1 Processor -- start 2024-08-02T17:48:24.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.833+0000 7f09d1b0f640 1 -- start start 2024-08-02T17:48:24.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09d1b0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09cc108180 0x7f09cc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09d1b0f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09cc108b30 0x7f09cc19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09d1b0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc10d260 0x7f09cc1a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09d1b0f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc118260 con 0x7f09cc10d260 2024-08-02T17:48:24.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09d1b0f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc1180e0 con 0x7f09cc108b30 2024-08-02T17:48:24.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09d1b0f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc1183e0 con 0x7f09cc108180 2024-08-02T17:48:24.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09c2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09cc108b30 0x7f09cc19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09cc108180 0x7f09cc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09cc108180 0x7f09cc19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33104/0 (socket says 172.21.15.69:33104) 2024-08-02T17:48:24.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09cb7fe640 1 -- 172.21.15.69:0/3771145927 learned_addr learned my addr 172.21.15.69:0/3771145927 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:24.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.834+0000 7f09cbfff640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc10d260 0x7f09cc1a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.835+0000 7f09cb7fe640 1 -- 172.21.15.69:0/3771145927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09cc108b30 msgr2=0x7f09cc19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.835+0000 7f09cb7fe640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09cc108b30 0x7f09cc19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.835+0000 7f09cb7fe640 1 -- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc10d260 msgr2=0x7f09cc1a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.835+0000 7f09cb7fe640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc10d260 0x7f09cc1a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.835+0000 7f09cb7fe640 1 -- 172.21.15.69:0/3771145927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc1a26b0 con 0x7f09cc108180 2024-08-02T17:48:24.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.835+0000 7f09cb7fe640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09cc108180 0x7f09cc19e760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f09bc02af50 tx=0x7f09bc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:24.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.836+0000 7f09c97fa640 1 -- 172.21.15.69:0/3771145927 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc041020 con 0x7f09cc108180 2024-08-02T17:48:24.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.836+0000 7f09c97fa640 1 -- 172.21.15.69:0/3771145927 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09bc0304f0 con 0x7f09cc108180 2024-08-02T17:48:24.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.836+0000 7f09c97fa640 1 -- 172.21.15.69:0/3771145927 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc037070 con 0x7f09cc108180 2024-08-02T17:48:24.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.836+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3771145927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09cc1a2940 con 0x7f09cc108180 2024-08-02T17:48:24.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.837+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3771145927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09cc1ab310 con 0x7f09cc108180 2024-08-02T17:48:24.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.840+0000 7f09c97fa640 1 -- 172.21.15.69:0/3771145927 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09bc044020 con 0x7f09cc108180 2024-08-02T17:48:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.841+0000 7f09d3f0d740 1 Processor -- start 2024-08-02T17:48:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.841+0000 7f09d3f0d740 1 -- start start 2024-08-02T17:48:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.841+0000 7f09c97fa640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09ac0a2ec0 0x7f09ac0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.841+0000 7f09cbfff640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09ac0a2ec0 0x7f09ac0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.841+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814d75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09c97fa640 1 -- 172.21.15.69:0/3771145927 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f09bc0efcb0 con 0x7f09cc108180 2024-08-02T17:48:24.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814d7b20 0x55d2814e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09d3f0d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d28149ae40 con 0x55d2814e23e0 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cbfff640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09ac0a2ec0 0x7f09ac0a5380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f09c400bab0 tx=0x7f09c400b260 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09d3f0d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d28143c7f0 con 0x55d2814d7b20 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09d3f0d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2813e3fb0 con 0x55d2814d7200 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814d75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09c2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814d7b20 0x55d2814e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814d75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33116/0 (socket says 172.21.15.69:33116) 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814e47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42230/0 (socket says 172.21.15.69:42230) 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cb7fe640 1 -- 172.21.15.69:0/1466049878 learned_addr learned my addr 172.21.15.69:0/1466049878 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cbfff640 1 -- 172.21.15.69:0/1466049878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 msgr2=0x55d2814d75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cbfff640 1 --2- 172.21.15.69:0/1466049878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814d75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cbfff640 1 -- 172.21.15.69:0/1466049878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814d7b20 msgr2=0x55d2814e1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cbfff640 1 --2- 172.21.15.69:0/1466049878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814d7b20 0x55d2814e1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.842+0000 7f09cbfff640 1 -- 172.21.15.69:0/1466049878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2811ae6a0 con 0x55d2814e23e0 2024-08-02T17:48:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.843+0000 7f09cbfff640 1 --2- 172.21.15.69:0/1466049878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814e47d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f09c40107a0 tx=0x7f09c4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65e0469f372e09b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.843+0000 7f09c1ffb640 1 -- 172.21.15.69:0/1466049878 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c401e420 con 0x55d2814e23e0 2024-08-02T17:48:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.843+0000 7f09c1ffb640 1 -- 172.21.15.69:0/1466049878 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c401eaa0 con 0x55d2814e23e0 2024-08-02T17:48:24.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.843+0000 7f09c1ffb640 1 -- 172.21.15.69:0/1466049878 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c401cb30 con 0x55d2814e23e0 2024-08-02T17:48:24.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 -- 172.21.15.69:0/1466049878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 msgr2=0x55d2814e47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/1466049878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814e47d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f09c40107a0 tx=0x7f09c4013000 comp rx=0 tx=0).stop 2024-08-02T17:48:24.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 -- 172.21.15.69:0/1466049878 shutdown_connections 2024-08-02T17:48:24.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/1466049878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814e47d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/1466049878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814d7b20 0x55d2814e1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/1466049878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814d75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 -- 172.21.15.69:0/1466049878 >> 172.21.15.69:0/1466049878 conn(0x55d2814d3790 msgr2=0x55d2814d3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 -- 172.21.15.69:0/1466049878 shutdown_connections 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.844+0000 7f09d3f0d740 1 -- 172.21.15.69:0/1466049878 wait complete. 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09d3f0d740 1 -- start start 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09d3f0d740 1 Processor -- start 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814df350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7b20 0x55d2814df890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814df350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814df350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42232/0 (socket says 172.21.15.69:42232) 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d2813f7cb0 con 0x55d2814d7200 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.845+0000 7f09cb7fe640 1 -- 172.21.15.69:0/2606839630 learned_addr learned my addr 172.21.15.69:0/2606839630 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d2814387b0 con 0x55d2814e23e0 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d2814e4fb0 con 0x55d2814d7b20 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09c2ffd640 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7b20 0x55d2814df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09cbfff640 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09d3f0d740 10 client.0 _ll_register_callbacks cb 0x55d28114bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:24.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09c2ffd640 1 -- 172.21.15.69:0/2606839630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 msgr2=0x55d2814dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09c2ffd640 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09c2ffd640 1 -- 172.21.15.69:0/2606839630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 msgr2=0x55d2814df350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09c2ffd640 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814df350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09c2ffd640 1 -- 172.21.15.69:0/2606839630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2813f6bc0 con 0x55d2814d7b20 2024-08-02T17:48:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09c2ffd640 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7b20 0x55d2814df890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09cc107b70 tx=0x7f09b400af80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc89f7071987c6c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.846+0000 7f09baffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.847+0000 7f09baffd640 1 -- 172.21.15.69:0/2606839630 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b400fd40 con 0x55d2814d7b20 2024-08-02T17:48:24.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.847+0000 7f09baffd640 1 -- 172.21.15.69:0/2606839630 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f09b4004b20 con 0x55d2814d7b20 2024-08-02T17:48:24.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.848+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c401eaa0 con 0x55d2814d7b20 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.849+0000 7f09baffd640 1 -- 172.21.15.69:0/2606839630 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f09b4004d20 con 0x55d2814d7b20 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.849+0000 7f09d3f0d740 10 client.25501 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.849+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2814f0f10 con 0x55d2814d7b20 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.849+0000 7f09baffd640 1 -- 172.21.15.69:0/2606839630 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c401eaa0 con 0x55d2814d7b20 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.849+0000 7f09baffd640 1 -- 172.21.15.69:0/2606839630 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f09b4019410 con 0x55d2814d7b20 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.849+0000 7f09d3f0d740 10 client.25501 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.849+0000 7f09d3f0d740 10 client.25501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.850+0000 7f09d3f0d740 20 client.25501 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.850+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d2814f23d0 0x55d2814f4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.850+0000 7f09d3f0d740 4 client.25501 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:24.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.850+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d281449d60 con 0x55d2814f23d0 2024-08-02T17:48:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.851+0000 7f09cb7fe640 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d2814f23d0 0x55d2814f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.851+0000 7f09cb7fe640 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d2814f23d0 0x55d2814f4800 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bcfab6b4ec1faf1 server_cookie=e09ba16f909c7142 in_seq=0 out_seq=0 2024-08-02T17:48:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.851+0000 7f09baffd640 10 client.25501 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.852+0000 7f09baffd640 1 -- 172.21.15.69:0/2606839630 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d281449d60 con 0x55d2814f23d0 2024-08-02T17:48:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.852+0000 7f09baffd640 10 client.25501 handle_command_reply: tid=0 2024-08-02T17:48:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.852+0000 7f09d3f0d740 1 client.25501 shutdown 2024-08-02T17:48:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.852+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7b20 msgr2=0x55d2814df890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.852+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7b20 0x55d2814df890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09cc107b70 tx=0x7f09b400af80 comp rx=0 tx=0).stop 2024-08-02T17:48:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.853+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 shutdown_connections 2024-08-02T17:48:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.853+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d2814f23d0 0x55d2814f4800 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.853+0000 7f09baffd640 0 client.25501 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.853+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.853+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7b20 0x55d2814df890 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f09cc107b70 tx=0x7f09b400af80 comp rx=0 tx=0).stop 2024-08-02T17:48:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.853+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/2606839630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814df350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.853+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 >> 172.21.15.69:0/2606839630 conn(0x55d2814d3790 msgr2=0x55d2814d9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.854+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 shutdown_connections 2024-08-02T17:48:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.855+0000 7f09d3f0d740 1 -- 172.21.15.69:0/2606839630 wait complete. 2024-08-02T17:48:24.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.855+0000 7f09d3f0d740 20 client.25501 trim_cache size 0 max 16384 2024-08-02T17:48:24.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.879+0000 7f09d3f0d740 1 Processor -- start 2024-08-02T17:48:24.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.879+0000 7f09d3f0d740 1 -- start start 2024-08-02T17:48:24.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.879+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.879+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814fa660 0x55d2814dc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09c2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814e12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33148/0 (socket says 172.21.15.69:33148) 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cb7fe640 1 -- 172.21.15.69:0/580068062 learned_addr learned my addr 172.21.15.69:0/580068062 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09d3f0d740 1 -- 172.21.15.69:0/580068062 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc1b7cd0 con 0x55d2814e23e0 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09d3f0d740 1 -- 172.21.15.69:0/580068062 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc1b55a0 con 0x55d2814fa660 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09d3f0d740 1 -- 172.21.15.69:0/580068062 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc1b33f0 con 0x55d2814d7200 2024-08-02T17:48:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cbfff640 1 --2- 172.21.15.69:0/580068062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814fa660 0x55d2814dc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cbfff640 1 -- 172.21.15.69:0/580068062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 msgr2=0x55d2814e12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:24.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cbfff640 1 --2- 172.21.15.69:0/580068062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814e12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cbfff640 1 -- 172.21.15.69:0/580068062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 msgr2=0x55d2814fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:24.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cbfff640 1 --2- 172.21.15.69:0/580068062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.880+0000 7f09cbfff640 1 -- 172.21.15.69:0/580068062 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2814f4e80 con 0x55d2814fa660 2024-08-02T17:48:24.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.881+0000 7f09cbfff640 1 --2- 172.21.15.69:0/580068062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814fa660 0x55d2814dc5b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f09c400b4a0 tx=0x7f09c400b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38894bbc99afe49 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:24.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.881+0000 7f09c17fa640 1 -- 172.21.15.69:0/580068062 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c4010ea0 con 0x55d2814fa660 2024-08-02T17:48:24.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.881+0000 7f09c17fa640 1 -- 172.21.15.69:0/580068062 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c401c3b0 con 0x55d2814fa660 2024-08-02T17:48:24.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.881+0000 7f09c17fa640 1 -- 172.21.15.69:0/580068062 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c401ce80 con 0x55d2814fa660 2024-08-02T17:48:24.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.882+0000 7f09d3f0d740 1 -- 172.21.15.69:0/580068062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814fa660 msgr2=0x55d2814dc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.882+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/580068062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814fa660 0x55d2814dc5b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f09c400b4a0 tx=0x7f09c400b700 comp rx=0 tx=0).stop 2024-08-02T17:48:24.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.882+0000 7f09d3f0d740 1 -- 172.21.15.69:0/580068062 shutdown_connections 2024-08-02T17:48:24.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.882+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/580068062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814e23e0 0x55d2814fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.882+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/580068062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814fa660 0x55d2814dc5b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.882+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/580068062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814d7200 0x55d2814e12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.882+0000 7f09d3f0d740 1 -- 172.21.15.69:0/580068062 >> 172.21.15.69:0/580068062 conn(0x55d2814d3790 msgr2=0x55d2814e0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.883+0000 7f09d3f0d740 1 -- 172.21.15.69:0/580068062 shutdown_connections 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.883+0000 7f09d3f0d740 1 -- 172.21.15.69:0/580068062 wait complete. 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.883+0000 7f09d3f0d740 1 -- start start 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.883+0000 7f09d3f0d740 1 Processor -- start 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09d3f0d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814fa660 0x55d2814e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814ea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42244/0 (socket says 172.21.15.69:42244) 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09cb7fe640 1 -- 172.21.15.69:0/3469780698 learned_addr learned my addr 172.21.15.69:0/3469780698 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09c2ffd640 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc1b3a60 con 0x55d2814d7200 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc1b4ed0 con 0x55d2814e23e0 2024-08-02T17:48:24.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc1b4130 con 0x55d2814fa660 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09d3f0d740 10 client.0 _ll_register_callbacks cb 0x7f09b400aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.884+0000 7f09cbfff640 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814fa660 0x55d2814e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09c2ffd640 1 -- 172.21.15.69:0/3469780698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814fa660 msgr2=0x55d2814e71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09c2ffd640 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814fa660 0x55d2814e71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09c2ffd640 1 -- 172.21.15.69:0/3469780698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 msgr2=0x55d2814ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09c2ffd640 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814ea950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09c2ffd640 1 -- 172.21.15.69:0/3469780698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2814e79b0 con 0x55d2814e23e0 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09c2ffd640 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814e6c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55d2814e0440 tx=0x7f09b400ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e86c6ecdac0ef072 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09baffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09baffd640 1 -- 172.21.15.69:0/3469780698 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4031060 con 0x55d2814e23e0 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09baffd640 1 -- 172.21.15.69:0/3469780698 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f09b4019aa0 con 0x55d2814e23e0 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.885+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09c401c3b0 con 0x55d2814e23e0 2024-08-02T17:48:24.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09baffd640 1 -- 172.21.15.69:0/3469780698 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f09b4027070 con 0x55d2814e23e0 2024-08-02T17:48:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09d3f0d740 10 client.25634 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:24.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d28155e1b0 con 0x55d2814e23e0 2024-08-02T17:48:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09baffd640 1 -- 172.21.15.69:0/3469780698 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09c401c3b0 con 0x55d2814e23e0 2024-08-02T17:48:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09baffd640 1 -- 172.21.15.69:0/3469780698 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f09b4023b70 con 0x55d2814e23e0 2024-08-02T17:48:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09d3f0d740 10 client.25634 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09d3f0d740 10 client.25634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09d3f0d740 20 client.25634 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d28155e8b0 0x55d281560cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.886+0000 7f09d3f0d740 4 client.25634 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:48:24.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.887+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d281444270 con 0x55d28155e8b0 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.887+0000 7f09cb7fe640 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d28155e8b0 0x55d281560cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.887+0000 7f09cb7fe640 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d28155e8b0 0x55d281560cd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d607cbfcdba6da2a server_cookie=a8c07f0fc8ac6344 in_seq=0 out_seq=0 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.887+0000 7f09baffd640 10 client.25634 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09baffd640 1 -- 172.21.15.69:0/3469780698 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d281444270 con 0x55d28155e8b0 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09baffd640 10 client.25634 handle_command_reply: tid=0 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 client.25634 shutdown 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 msgr2=0x55d2814e6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814e6c90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55d2814e0440 tx=0x7f09b400ff70 comp rx=0 tx=0).stop 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 shutdown_connections 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d28155e8b0 0x55d281560cd0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d2814fa660 0x55d2814e71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d2814e23e0 0x55d2814e6c90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 --2- 172.21.15.69:0/3469780698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d2814d7200 0x55d2814ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 >> 172.21.15.69:0/3469780698 conn(0x55d2814d3790 msgr2=0x55d2814d28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:24.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.888+0000 7f09baffd640 0 client.25634 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:24.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.890+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 shutdown_connections 2024-08-02T17:48:24.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.890+0000 7f09d3f0d740 1 -- 172.21.15.69:0/3469780698 wait complete. 2024-08-02T17:48:24.894 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:48:24.894 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:48:24.894 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "284a76cd-d315-47df-9037-e1a64cca2221", 2024-08-02T17:48:24.894 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:48:24.895 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:48:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.890+0000 7f09d3f0d740 20 client.25634 trim_cache size 0 max 16384 2024-08-02T17:48:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09ac0a2ec0 msgr2=0x7f09ac0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09ac0a2ec0 0x7f09ac0a5380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f09c400bab0 tx=0x7f09c400b260 comp rx=0 tx=0).stop 2024-08-02T17:48:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3771145927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09cc108180 msgr2=0x7f09cc19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09cc108180 0x7f09cc19e760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f09bc02af50 tx=0x7f09bc0042f0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3771145927 shutdown_connections 2024-08-02T17:48:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09cc10d260 0x7f09cc1a1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09ac0a2ec0 0x7f09ac0a5380 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09cc108b30 0x7f09cc19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 --2- 172.21.15.69:0/3771145927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09cc108180 0x7f09cc19e760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3771145927 >> 172.21.15.69:0/3771145927 conn(0x7f09cc0778d0 msgr2=0x7f09cc078bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3771145927 shutdown_connections 2024-08-02T17:48:24.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:24.891+0000 7f09d1b0f640 1 -- 172.21.15.69:0/3771145927 wait complete. 2024-08-02T17:48:24.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 75.0 s, 3298.4 tps, lat 33.741 ms stddev 103.529 2024-08-02T17:48:25.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:25 smithi077 ceph-mon[37108]: pgmap v63: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 1.66k op/s 2024-08-02T17:48:25.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:25 smithi069 ceph-mon[32746]: pgmap v63: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 1.66k op/s 2024-08-02T17:48:25.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:25 smithi083 ceph-mon[37031]: pgmap v63: 129 pgs: 129 active+clean; 1.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 29 MiB/s wr, 1.66k op/s 2024-08-02T17:48:25.684 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:48:26.087 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:26 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:48:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:26 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:26 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:48:26.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:26 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:26.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:26 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:48:26.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:26 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:26.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:26 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:48:26.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:26 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:26 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:48:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:26 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:26 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:48:26.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:26 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:48:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 -- 172.21.15.69:0/374387106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c809a4c0 msgr2=0x7f50c809a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 --2- 172.21.15.69:0/374387106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c809a4c0 0x7f50c809a8a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f50cc009860 tx=0x7f50cc02a340 comp rx=0 tx=0).stop 2024-08-02T17:48:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 -- 172.21.15.69:0/374387106 shutdown_connections 2024-08-02T17:48:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 --2- 172.21.15.69:0/374387106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50c8005740 0x7f50c8097fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 --2- 172.21.15.69:0/374387106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50c8004cb0 0x7f50c80050f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 --2- 172.21.15.69:0/374387106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c809a4c0 0x7f50c809a8a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 -- 172.21.15.69:0/374387106 >> 172.21.15.69:0/374387106 conn(0x7f50c800a1f0 msgr2=0x7f50c800a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:26.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 -- 172.21.15.69:0/374387106 shutdown_connections 2024-08-02T17:48:26.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.652+0000 7f50d725d640 1 -- 172.21.15.69:0/374387106 wait complete. 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.653+0000 7f50d725d640 1 Processor -- start 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.653+0000 7f50d725d640 1 -- start start 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d725d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50c8004cb0 0x7f50c8132420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d725d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8005740 0x7f50c8132960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d725d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50c809a4c0 0x7f50c8136cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d725d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80abfa0 con 0x7f50c8004cb0 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d725d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80abe20 con 0x7f50c809a4c0 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d725d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50c80ac120 con 0x7f50c8005740 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d625b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50c8004cb0 0x7f50c8132420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d5a5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8005740 0x7f50c8132960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d6a5c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50c809a4c0 0x7f50c8136cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d6a5c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50c809a4c0 0x7f50c8136cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37690/0 (socket says 172.21.15.69:37690) 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d5a5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8005740 0x7f50c8132960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33174/0 (socket says 172.21.15.69:33174) 2024-08-02T17:48:26.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.654+0000 7f50d6a5c640 1 -- 172.21.15.69:0/2837178359 learned_addr learned my addr 172.21.15.69:0/2837178359 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.655+0000 7f50d5a5a640 1 -- 172.21.15.69:0/2837178359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50c809a4c0 msgr2=0x7f50c8136cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.655+0000 7f50d5a5a640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50c809a4c0 0x7f50c8136cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.655+0000 7f50d5a5a640 1 -- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50c8004cb0 msgr2=0x7f50c8132420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.655+0000 7f50d5a5a640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50c8004cb0 0x7f50c8132420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.655+0000 7f50d5a5a640 1 -- 172.21.15.69:0/2837178359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c81373d0 con 0x7f50c8005740 2024-08-02T17:48:26.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.655+0000 7f50d5a5a640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8005740 0x7f50c8132960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f50c0005fd0 tx=0x7f50c000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:26.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.656+0000 7f50c77fe640 1 -- 172.21.15.69:0/2837178359 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c000cd00 con 0x7f50c8005740 2024-08-02T17:48:26.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.656+0000 7f50c77fe640 1 -- 172.21.15.69:0/2837178359 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c00035f0 con 0x7f50c8005740 2024-08-02T17:48:26.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.656+0000 7f50c77fe640 1 -- 172.21.15.69:0/2837178359 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0010650 con 0x7f50c8005740 2024-08-02T17:48:26.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.656+0000 7f50d725d640 1 -- 172.21.15.69:0/2837178359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c81376c0 con 0x7f50c8005740 2024-08-02T17:48:26.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.656+0000 7f50d725d640 1 -- 172.21.15.69:0/2837178359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c813efa0 con 0x7f50c8005740 2024-08-02T17:48:26.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.659+0000 7f50da8bd740 1 Processor -- start 2024-08-02T17:48:26.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50da8bd740 1 -- start start 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c531750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c53c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c53c550 0x55736c53e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50da8bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55736c496980 con 0x55736c53c550 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50da8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55736c42ea70 con 0x55736c531370 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50da8bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55736c492940 con 0x55736c531c90 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50c77fe640 1 -- 172.21.15.69:0/2837178359 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f50c00107f0 con 0x7f50c8005740 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50d6a5c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c531750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50d6a5c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c531750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37702/0 (socket says 172.21.15.69:37702) 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.660+0000 7f50d6a5c640 1 -- 172.21.15.69:0/3914278485 learned_addr learned my addr 172.21.15.69:0/3914278485 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.661+0000 7f50d625b640 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c53c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.661+0000 7f50d6a5c640 1 -- 172.21.15.69:0/3914278485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 msgr2=0x55736c53c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.661+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c53c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.661+0000 7f50d6a5c640 1 -- 172.21.15.69:0/3914278485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c53c550 msgr2=0x55736c53e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.661+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c53c550 0x55736c53e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.661+0000 7f50d6a5c640 1 -- 172.21.15.69:0/3914278485 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55736c4916e0 con 0x55736c531370 2024-08-02T17:48:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.661+0000 7f50d625b640 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c53c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:48:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.661+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c531750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f50d0067920 tx=0x7f50d006b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c43feeea71c06914 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.662+0000 7f50c4ff9640 1 -- 172.21.15.69:0/3914278485 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d007b660 con 0x55736c531370 2024-08-02T17:48:26.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.662+0000 7f50c4ff9640 1 -- 172.21.15.69:0/3914278485 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d007bce0 con 0x55736c531370 2024-08-02T17:48:26.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.662+0000 7f50c4ff9640 1 -- 172.21.15.69:0/3914278485 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d00773e0 con 0x55736c531370 2024-08-02T17:48:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.662+0000 7f50c77fe640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50bc0a2e50 0x7f50bc0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.663+0000 7f50c77fe640 1 -- 172.21.15.69:0/2837178359 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f50c00cf790 con 0x7f50c8005740 2024-08-02T17:48:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.663+0000 7f50da8bd740 1 -- 172.21.15.69:0/3914278485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 msgr2=0x55736c531750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.663+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c531750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f50d0067920 tx=0x7f50d006b420 comp rx=0 tx=0).stop 2024-08-02T17:48:26.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.663+0000 7f50da8bd740 1 -- 172.21.15.69:0/3914278485 shutdown_connections 2024-08-02T17:48:26.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.663+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c53c550 0x55736c53e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.663+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c53c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.663+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3914278485 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c531750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.663+0000 7f50da8bd740 1 -- 172.21.15.69:0/3914278485 >> 172.21.15.69:0/3914278485 conn(0x55736c52d900 msgr2=0x55736c52dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.665+0000 7f50da8bd740 1 -- 172.21.15.69:0/3914278485 shutdown_connections 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.665+0000 7f50d625b640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50bc0a2e50 0x7f50bc0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.665+0000 7f50da8bd740 1 -- 172.21.15.69:0/3914278485 wait complete. 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.666+0000 7f50da8bd740 1 -- start start 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.666+0000 7f50da8bd740 1 Processor -- start 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.666+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c5394d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.666+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531c90 0x55736c539a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.666+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c539f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.666+0000 7f50da8bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55736c53f120 con 0x55736c531370 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.666+0000 7f50da8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55736c451aa0 con 0x55736c531c90 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.666+0000 7f50da8bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55736c53f2a0 con 0x55736c53c550 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d5a5a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531c90 0x55736c539a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50da8bd740 10 client.0 _ll_register_callbacks cb 0x55736c440e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d625b640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50bc0a2e50 0x7f50bc0a5310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f50cc000c00 tx=0x7f50cc005d20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d5a5a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531c90 0x55736c539a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37718/0 (socket says 172.21.15.69:37718) 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d5a5a640 1 -- 172.21.15.69:0/613921775 learned_addr learned my addr 172.21.15.69:0/613921775 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c539f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d625b640 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c5394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d6a5c640 1 -- 172.21.15.69:0/613921775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531c90 msgr2=0x55736c539a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531c90 0x55736c539a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d6a5c640 1 -- 172.21.15.69:0/613921775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 msgr2=0x55736c5394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c5394d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.667+0000 7f50d6a5c640 1 -- 172.21.15.69:0/613921775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55736c435650 con 0x55736c53c550 2024-08-02T17:48:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.668+0000 7f50b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:26.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.668+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c539f50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f50d0065770 tx=0x7f50d0068140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba001cc9d2c67c5f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.668+0000 7f50b9ffb640 1 -- 172.21.15.69:0/613921775 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d007f040 con 0x55736c53c550 2024-08-02T17:48:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.668+0000 7f50b9ffb640 1 -- 172.21.15.69:0/613921775 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f50d007b660 con 0x55736c53c550 2024-08-02T17:48:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.668+0000 7f50da8bd740 1 -- 172.21.15.69:0/613921775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d007bce0 con 0x55736c53c550 2024-08-02T17:48:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.668+0000 7f50b9ffb640 1 -- 172.21.15.69:0/613921775 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50d0096050 con 0x55736c53c550 2024-08-02T17:48:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.668+0000 7f50da8bd740 10 client.25519 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.668+0000 7f50da8bd740 1 -- 172.21.15.69:0/613921775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55736c2083c0 con 0x55736c53c550 2024-08-02T17:48:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.669+0000 7f50b9ffb640 1 -- 172.21.15.69:0/613921775 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d007bce0 con 0x55736c53c550 2024-08-02T17:48:26.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.669+0000 7f50b9ffb640 1 -- 172.21.15.69:0/613921775 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f50d00773e0 con 0x55736c53c550 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.669+0000 7f50da8bd740 10 client.25519 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.669+0000 7f50da8bd740 10 client.25519 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.669+0000 7f50da8bd740 20 client.25519 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.669+0000 7f50da8bd740 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55736c54c430 0x55736c54e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.669+0000 7f50da8bd740 4 client.25519 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.669+0000 7f50da8bd740 1 -- 172.21.15.69:0/613921775 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55736c4a3ef0 con 0x55736c54c430 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.670+0000 7f50d625b640 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55736c54c430 0x55736c54e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.671+0000 7f50b9ffb640 10 client.25519 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.671+0000 7f50d625b640 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55736c54c430 0x55736c54e8b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5336d1b31b66a1a server_cookie=6ffd7ecf172fa57 in_seq=0 out_seq=0 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50b9ffb640 1 -- 172.21.15.69:0/613921775 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55736c4a3ef0 con 0x55736c54c430 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50b9ffb640 10 client.25519 handle_command_reply: tid=0 2024-08-02T17:48:26.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 client.25519 shutdown 2024-08-02T17:48:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 -- 172.21.15.69:0/613921775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 msgr2=0x55736c539f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c539f50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f50d0065770 tx=0x7f50d0068140 comp rx=0 tx=0).stop 2024-08-02T17:48:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 -- 172.21.15.69:0/613921775 shutdown_connections 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55736c54c430 0x55736c54e8b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c539f50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531c90 0x55736c539a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 --2- 172.21.15.69:0/613921775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c5394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.672+0000 7f50da8bd740 1 -- 172.21.15.69:0/613921775 >> 172.21.15.69:0/613921775 conn(0x55736c52d900 msgr2=0x55736c533680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.673+0000 7f50b9ffb640 0 client.25519 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.673+0000 7f50da8bd740 1 -- 172.21.15.69:0/613921775 shutdown_connections 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.673+0000 7f50da8bd740 1 -- 172.21.15.69:0/613921775 wait complete. 2024-08-02T17:48:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.673+0000 7f50da8bd740 20 client.25519 trim_cache size 0 max 16384 2024-08-02T17:48:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.700+0000 7f50da8bd740 1 Processor -- start 2024-08-02T17:48:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.700+0000 7f50da8bd740 1 -- start start 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c553f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c5544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c53c550 0x55736c536770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50d625b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c553f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50d625b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c553f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42294/0 (socket says 172.21.15.69:42294) 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50d5a5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c5544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50d5a5a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c5544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33206/0 (socket says 172.21.15.69:33206) 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50d625b640 1 -- 172.21.15.69:0/3915711137 learned_addr learned my addr 172.21.15.69:0/3915711137 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.701+0000 7f50da8bd740 1 -- 172.21.15.69:0/3915711137 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4012780 con 0x55736c531370 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50da8bd740 1 -- 172.21.15.69:0/3915711137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50b4010050 con 0x55736c53c550 2024-08-02T17:48:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50da8bd740 1 -- 172.21.15.69:0/3915711137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50b400de60 con 0x55736c531c90 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c53c550 0x55736c536770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50d6a5c640 1 -- 172.21.15.69:0/3915711137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 msgr2=0x55736c5544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c5544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50d6a5c640 1 -- 172.21.15.69:0/3915711137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 msgr2=0x55736c553f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c553f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50d6a5c640 1 -- 172.21.15.69:0/3915711137 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55736c5483b0 con 0x55736c53c550 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50d5a5a640 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c5544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50d6a5c640 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c53c550 0x55736c536770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f50d0076c20 tx=0x7f50d00679d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62be9234101a8e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50bbfff640 1 -- 172.21.15.69:0/3915711137 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0096050 con 0x55736c53c550 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50bbfff640 1 -- 172.21.15.69:0/3915711137 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d007aa00 con 0x55736c53c550 2024-08-02T17:48:26.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.702+0000 7f50bbfff640 1 -- 172.21.15.69:0/3915711137 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d008e370 con 0x55736c53c550 2024-08-02T17:48:26.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.703+0000 7f50da8bd740 1 -- 172.21.15.69:0/3915711137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c53c550 msgr2=0x55736c536770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.703+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c53c550 0x55736c536770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f50d0076c20 tx=0x7f50d00679d0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.704+0000 7f50da8bd740 1 -- 172.21.15.69:0/3915711137 shutdown_connections 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.704+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c53c550 0x55736c536770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.704+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c531c90 0x55736c5544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.704+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3915711137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531370 0x55736c553f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.704+0000 7f50da8bd740 1 -- 172.21.15.69:0/3915711137 >> 172.21.15.69:0/3915711137 conn(0x55736c52d900 msgr2=0x55736c533680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.705+0000 7f50da8bd740 1 -- 172.21.15.69:0/3915711137 shutdown_connections 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.705+0000 7f50da8bd740 1 -- 172.21.15.69:0/3915711137 wait complete. 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.705+0000 7f50da8bd740 1 -- start start 2024-08-02T17:48:26.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.705+0000 7f50da8bd740 1 Processor -- start 2024-08-02T17:48:26.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c540550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531c90 0x55736c540a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50da8bd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c540ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50da8bd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50b400e510 con 0x55736c531c90 2024-08-02T17:48:26.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50da8bd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50b400f980 con 0x55736c531370 2024-08-02T17:48:26.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50da8bd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50b400ebe0 con 0x55736c53c550 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50da8bd740 10 client.0 _ll_register_callbacks cb 0x7f50d0068b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50d625b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c540550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50d6a5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c540ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50d625b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c540550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37736/0 (socket says 172.21.15.69:37736) 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50d625b640 1 -- 172.21.15.69:0/3372295006 learned_addr learned my addr 172.21.15.69:0/3372295006 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.706+0000 7f50d5a5a640 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531c90 0x55736c540a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50d5a5a640 1 -- 172.21.15.69:0/3372295006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 msgr2=0x55736c540ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50d5a5a640 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c540ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50d5a5a640 1 -- 172.21.15.69:0/3372295006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 msgr2=0x55736c540550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:26.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50d5a5a640 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c540550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50d5a5a640 1 -- 172.21.15.69:0/3372295006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55736c5b6200 con 0x55736c531c90 2024-08-02T17:48:26.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50d625b640 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c540550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:26.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50d5a5a640 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531c90 0x55736c540a90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f50c00ccbd0 tx=0x7f50c009bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235be0eabca7aada server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:26.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:26.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50b9ffb640 1 -- 172.21.15.69:0/3372295006 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c00b3070 con 0x55736c531c90 2024-08-02T17:48:26.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50b9ffb640 1 -- 172.21.15.69:0/3372295006 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f50c00cf350 con 0x55736c531c90 2024-08-02T17:48:26.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50da8bd740 1 -- 172.21.15.69:0/3372295006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50d007aa00 con 0x55736c531c90 2024-08-02T17:48:26.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.707+0000 7f50b9ffb640 1 -- 172.21.15.69:0/3372295006 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50c009fca0 con 0x55736c531c90 2024-08-02T17:48:26.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50da8bd740 10 client.16020 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:26.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50da8bd740 1 -- 172.21.15.69:0/3372295006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55736c5b7510 con 0x55736c531c90 2024-08-02T17:48:26.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50b9ffb640 1 -- 172.21.15.69:0/3372295006 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50d007aa00 con 0x55736c531c90 2024-08-02T17:48:26.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50b9ffb640 1 -- 172.21.15.69:0/3372295006 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f50c009add0 con 0x55736c531c90 2024-08-02T17:48:26.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50da8bd740 10 client.16020 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50da8bd740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50da8bd740 20 client.16020 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55736c5b7c30 0x55736c5ba050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50da8bd740 4 client.16020 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.708+0000 7f50da8bd740 1 -- 172.21.15.69:0/3372295006 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55736c49e400 con 0x55736c5b7c30 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.709+0000 7f50d625b640 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55736c5b7c30 0x55736c5ba050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.709+0000 7f50d625b640 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55736c5b7c30 0x55736c5ba050 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0620619a5789894 server_cookie=c4020777b26ba896 in_seq=0 out_seq=0 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.709+0000 7f50b9ffb640 10 client.16020 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.710+0000 7f50b9ffb640 1 -- 172.21.15.69:0/3372295006 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55736c49e400 con 0x55736c5b7c30 2024-08-02T17:48:26.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.710+0000 7f50b9ffb640 10 client.16020 handle_command_reply: tid=0 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.710+0000 7f50da8bd740 1 client.16020 shutdown 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50da8bd740 1 -- 172.21.15.69:0/3372295006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531c90 msgr2=0x55736c540a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531c90 0x55736c540a90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f50c00ccbd0 tx=0x7f50c009bff0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50da8bd740 1 -- 172.21.15.69:0/3372295006 shutdown_connections 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55736c531c90 0x55736c540a90 secure :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f50c00ccbd0 tx=0x7f50c009bff0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55736c5b7c30 0x55736c5ba050 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55736c53c550 0x55736c540ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50da8bd740 1 --2- 172.21.15.69:0/3372295006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55736c531370 0x55736c540550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50da8bd740 1 -- 172.21.15.69:0/3372295006 >> 172.21.15.69:0/3372295006 conn(0x55736c52d900 msgr2=0x55736c5473a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.711+0000 7f50b9ffb640 0 client.16020 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:26.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.713+0000 7f50da8bd740 1 -- 172.21.15.69:0/3372295006 shutdown_connections 2024-08-02T17:48:26.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.717+0000 7f50da8bd740 1 -- 172.21.15.69:0/3372295006 wait complete. 2024-08-02T17:48:26.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.718+0000 7f50da8bd740 20 client.16020 trim_cache size 0 max 16384 2024-08-02T17:48:26.724 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:48:26.724 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:48:26.724 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:48:26.724 INFO:teuthology.orchestra.run.smithi069.stdout: "284a76cd-d315-47df-9037-e1a64cca2221": { 2024-08-02T17:48:26.724 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:48:26.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "284a76cd-d315-47df-9037-e1a64cca2221", 2024-08-02T17:48:26.724 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:48:26.725 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:48:26.725 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:48:26.725 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:48:26.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.725+0000 7f50d725d640 1 -- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50bc0a2e50 msgr2=0x7f50bc0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.725+0000 7f50d725d640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50bc0a2e50 0x7f50bc0a5310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f50cc000c00 tx=0x7f50cc005d20 comp rx=0 tx=0).stop 2024-08-02T17:48:26.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.725+0000 7f50d725d640 1 -- 172.21.15.69:0/2837178359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8005740 msgr2=0x7f50c8132960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:26.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.725+0000 7f50d725d640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8005740 0x7f50c8132960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f50c0005fd0 tx=0x7f50c000ee50 comp rx=0 tx=0).stop 2024-08-02T17:48:26.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.726+0000 7f50d725d640 1 -- 172.21.15.69:0/2837178359 shutdown_connections 2024-08-02T17:48:26.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.726+0000 7f50d725d640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50bc0a2e50 0x7f50bc0a5310 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.726+0000 7f50d725d640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50c809a4c0 0x7f50c8136cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.726+0000 7f50d725d640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50c8005740 0x7f50c8132960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.726+0000 7f50d725d640 1 --2- 172.21.15.69:0/2837178359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50c8004cb0 0x7f50c8132420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:26.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.726+0000 7f50d725d640 1 -- 172.21.15.69:0/2837178359 >> 172.21.15.69:0/2837178359 conn(0x7f50c800a1f0 msgr2=0x7f50c8094fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:26.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.728+0000 7f50d725d640 1 -- 172.21.15.69:0/2837178359 shutdown_connections 2024-08-02T17:48:26.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:48:26.728+0000 7f50d725d640 1 -- 172.21.15.69:0/2837178359 wait complete. 2024-08-02T17:48:27.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:27 smithi077 ceph-mon[37108]: pgmap v64: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 38 MiB/s wr, 2.19k op/s 2024-08-02T17:48:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:27 smithi069 ceph-mon[32746]: pgmap v64: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 38 MiB/s wr, 2.19k op/s 2024-08-02T17:48:27.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:27 smithi083 ceph-mon[37031]: pgmap v64: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 38 MiB/s wr, 2.19k op/s 2024-08-02T17:48:29.330 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:29 smithi069 ceph-mon[32746]: pgmap v65: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 33 MiB/s wr, 1.84k op/s 2024-08-02T17:48:29.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:29 smithi077 ceph-mon[37108]: pgmap v65: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 33 MiB/s wr, 1.84k op/s 2024-08-02T17:48:29.374 INFO:tasks.cephfs.filesystem:scrub status for tag:284a76cd-d315-47df-9037-e1a64cca2221 - {'path': '/', 'tag': '284a76cd-d315-47df-9037-e1a64cca2221', 'options': 'recursive,force'} 2024-08-02T17:48:29.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:29 smithi083 ceph-mon[37031]: pgmap v65: 129 pgs: 129 active+clean; 1.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 33 MiB/s wr, 1.84k op/s 2024-08-02T17:48:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 80.0 s, 3040.6 tps, lat 32.746 ms stddev 36.388 2024-08-02T17:48:30.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:30 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:48:30.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:30 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:48:30.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:30 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:48:31.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:31 smithi077 ceph-mon[37108]: pgmap v66: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 35 MiB/s wr, 2.02k op/s 2024-08-02T17:48:31.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:31 smithi069 ceph-mon[32746]: pgmap v66: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 35 MiB/s wr, 2.02k op/s 2024-08-02T17:48:31.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:31 smithi083 ceph-mon[37031]: pgmap v66: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 35 MiB/s wr, 2.02k op/s 2024-08-02T17:48:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:33 smithi077 ceph-mon[37108]: pgmap v67: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 23 MiB/s wr, 1.43k op/s 2024-08-02T17:48:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:33 smithi069 ceph-mon[32746]: pgmap v67: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 23 MiB/s wr, 1.43k op/s 2024-08-02T17:48:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:33 smithi083 ceph-mon[37031]: pgmap v67: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 23 MiB/s wr, 1.43k op/s 2024-08-02T17:48:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 85.0 s, 3468.0 tps, lat 28.955 ms stddev 34.444 2024-08-02T17:48:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:35 smithi077 ceph-mon[37108]: pgmap v68: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-08-02T17:48:35.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:35 smithi069 ceph-mon[32746]: pgmap v68: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-08-02T17:48:35.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:35 smithi083 ceph-mon[37031]: pgmap v68: 129 pgs: 129 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 12 MiB/s wr, 837 op/s 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 -- 172.21.15.69:0/1763329054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c108a60 msgr2=0x7fba7c10caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 --2- 172.21.15.69:0/1763329054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c108a60 0x7fba7c10caa0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009860 tx=0x7fba6c02a340 comp rx=0 tx=0).stop 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 -- 172.21.15.69:0/1763329054 shutdown_connections 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 --2- 172.21.15.69:0/1763329054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c108a60 0x7fba7c10caa0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 --2- 172.21.15.69:0/1763329054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c10d200 0x7fba7c114d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 --2- 172.21.15.69:0/1763329054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7c1080b0 0x7fba7c108490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 -- 172.21.15.69:0/1763329054 >> 172.21.15.69:0/1763329054 conn(0x7fba7c077810 msgr2=0x7fba7c079c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 -- 172.21.15.69:0/1763329054 shutdown_connections 2024-08-02T17:48:35.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.963+0000 7fba84291640 1 -- 172.21.15.69:0/1763329054 wait complete. 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.964+0000 7fba84291640 1 Processor -- start 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.964+0000 7fba84291640 1 -- start start 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba84291640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c1080b0 0x7fba7c1a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba84291640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c10d200 0x7fba7c1a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba84291640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7c19e5a0 0x7fba7c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba84291640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c118250 con 0x7fba7c10d200 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba84291640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1180d0 con 0x7fba7c19e5a0 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba84291640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1183d0 con 0x7fba7c1080b0 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8182c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c10d200 0x7fba7c1a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8182c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c10d200 0x7fba7c1a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40554/0 (socket says 172.21.15.69:40554) 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8182c640 1 -- 172.21.15.69:0/3161788494 learned_addr learned my addr 172.21.15.69:0/3161788494 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8202d640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c1080b0 0x7fba7c1a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8282e640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7c19e5a0 0x7fba7c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8182c640 1 -- 172.21.15.69:0/3161788494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c1080b0 msgr2=0x7fba7c1a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8182c640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c1080b0 0x7fba7c1a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8182c640 1 -- 172.21.15.69:0/3161788494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7c19e5a0 msgr2=0x7fba7c19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8182c640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7c19e5a0 0x7fba7c19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.965+0000 7fba8182c640 1 -- 172.21.15.69:0/3161788494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7c19f0c0 con 0x7fba7c10d200 2024-08-02T17:48:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.966+0000 7fba8182c640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c10d200 0x7fba7c1a4930 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02a8b0 tx=0x7fba6c033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.966+0000 7fba737fe640 1 -- 172.21.15.69:0/3161788494 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037040 con 0x7fba7c10d200 2024-08-02T17:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.966+0000 7fba84291640 1 -- 172.21.15.69:0/3161788494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7c19f350 con 0x7fba7c10d200 2024-08-02T17:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.966+0000 7fba84291640 1 -- 172.21.15.69:0/3161788494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7c1ab5a0 con 0x7fba7c10d200 2024-08-02T17:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.966+0000 7fba737fe640 1 -- 172.21.15.69:0/3161788494 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6c040070 con 0x7fba7c10d200 2024-08-02T17:48:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.966+0000 7fba737fe640 1 -- 172.21.15.69:0/3161788494 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c003a20 con 0x7fba7c10d200 2024-08-02T17:48:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.970+0000 7fba737fe640 1 -- 172.21.15.69:0/3161788494 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fba6c003ca0 con 0x7fba7c10d200 2024-08-02T17:48:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.971+0000 7fba86fdf740 1 Processor -- start 2024-08-02T17:48:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.971+0000 7fba737fe640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba680a2dc0 0x7fba680a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.971+0000 7fba737fe640 1 -- 172.21.15.69:0/3161788494 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fba6c0ef6b0 con 0x7fba7c10d200 2024-08-02T17:48:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.972+0000 7fba86fdf740 1 -- start start 2024-08-02T17:48:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.972+0000 7fba8282e640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba680a2dc0 0x7fba680a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.972+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039048b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.972+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390490d0 0x564039053430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.972+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x564039055d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba86fdf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564038fac9b0 con 0x5640390490d0 2024-08-02T17:48:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba86fdf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564038f54800 con 0x564039053970 2024-08-02T17:48:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba86fdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564038fa8970 con 0x5640390487b0 2024-08-02T17:48:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x564039055d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x564039055d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50306/0 (socket says 172.21.15.69:50306) 2024-08-02T17:48:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 -- 172.21.15.69:0/4096321446 learned_addr learned my addr 172.21.15.69:0/4096321446 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8182c640 1 --2- 172.21.15.69:0/4096321446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390490d0 0x564039053430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba680a2dc0 0x7fba680a5280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba78005fd0 tx=0x7fba78011810 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 -- 172.21.15.69:0/4096321446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 msgr2=0x564039048b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:48:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 --2- 172.21.15.69:0/4096321446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039048b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 -- 172.21.15.69:0/4096321446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390490d0 msgr2=0x564039053430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 --2- 172.21.15.69:0/4096321446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390490d0 0x564039053430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.973+0000 7fba8282e640 1 -- 172.21.15.69:0/4096321446 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564038cc0210 con 0x564039053970 2024-08-02T17:48:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.974+0000 7fba8282e640 1 --2- 172.21.15.69:0/4096321446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x564039055d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba7801e760 tx=0x7fba7801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85659ca78ec64e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.974+0000 7fba70bfe640 1 -- 172.21.15.69:0/4096321446 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7801d480 con 0x564039053970 2024-08-02T17:48:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.974+0000 7fba70bfe640 1 -- 172.21.15.69:0/4096321446 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba7801db00 con 0x564039053970 2024-08-02T17:48:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.974+0000 7fba70bfe640 1 -- 172.21.15.69:0/4096321446 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7800fce0 con 0x564039053970 2024-08-02T17:48:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.975+0000 7fba86fdf740 1 -- 172.21.15.69:0/4096321446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 msgr2=0x564039055d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.975+0000 7fba86fdf740 1 --2- 172.21.15.69:0/4096321446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x564039055d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba7801e760 tx=0x7fba7801d000 comp rx=0 tx=0).stop 2024-08-02T17:48:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.976+0000 7fba86fdf740 1 -- 172.21.15.69:0/4096321446 shutdown_connections 2024-08-02T17:48:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.976+0000 7fba86fdf740 1 --2- 172.21.15.69:0/4096321446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390490d0 0x564039053430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.976+0000 7fba86fdf740 1 --2- 172.21.15.69:0/4096321446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x564039055d60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.976+0000 7fba86fdf740 1 --2- 172.21.15.69:0/4096321446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039048b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.976+0000 7fba86fdf740 1 -- 172.21.15.69:0/4096321446 >> 172.21.15.69:0/4096321446 conn(0x564039044d20 msgr2=0x564039045100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.976+0000 7fba86fdf740 1 -- 172.21.15.69:0/4096321446 shutdown_connections 2024-08-02T17:48:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.976+0000 7fba86fdf740 1 -- 172.21.15.69:0/4096321446 wait complete. 2024-08-02T17:48:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.977+0000 7fba86fdf740 1 -- start start 2024-08-02T17:48:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.977+0000 7fba86fdf740 1 Processor -- start 2024-08-02T17:48:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.977+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039050930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.977+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390490d0 0x564039050e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.977+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564039053970 0x5640390513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.977+0000 7fba8202d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039050930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8202d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039050930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51728/0 (socket says 172.21.15.69:51728) 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8182c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390490d0 0x564039050e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8182c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390490d0 0x564039050e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50322/0 (socket says 172.21.15.69:50322) 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8202d640 1 -- 172.21.15.69:0/3763332895 learned_addr learned my addr 172.21.15.69:0/3763332895 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564039056540 con 0x564039053970 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564038f68500 con 0x5640390490d0 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5640390566c0 con 0x5640390487b0 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba86fdf740 10 client.0 _ll_register_callbacks cb 0x564038cbfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8202d640 1 -- 172.21.15.69:0/3763332895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390490d0 msgr2=0x564039050e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8202d640 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390490d0 0x564039050e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8202d640 1 -- 172.21.15.69:0/3763332895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564039053970 msgr2=0x5640390513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8202d640 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564039053970 0x5640390513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8202d640 1 -- 172.21.15.69:0/3763332895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564038d8c620 con 0x5640390487b0 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8182c640 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390490d0 0x564039050e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba7801db00 con 0x5640390487b0 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba8202d640 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039050930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fba74003c50 tx=0x7fba74004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ef568d507cd956 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.978+0000 7fba65ffb640 10 client.25534 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba65ffb640 1 -- 172.21.15.69:0/3763332895 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74011850 con 0x5640390487b0 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba65ffb640 1 -- 172.21.15.69:0/3763332895 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fba740119f0 con 0x5640390487b0 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba65ffb640 1 -- 172.21.15.69:0/3763332895 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba74011c50 con 0x5640390487b0 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba86fdf740 10 client.25534 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x564039062160 con 0x5640390487b0 2024-08-02T17:48:35.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba65ffb640 1 -- 172.21.15.69:0/3763332895 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fba7401a470 con 0x5640390487b0 2024-08-02T17:48:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba86fdf740 10 client.25534 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba86fdf740 10 client.25534 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba86fdf740 20 client.25534 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5640390635d0 0x564039065a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba86fdf740 4 client.25534 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.979+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564038fba0e0 con 0x5640390635d0 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.980+0000 7fba8182c640 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5640390635d0 0x564039065a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.981+0000 7fba8182c640 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5640390635d0 0x564039065a20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916eae792d48bab2 server_cookie=a0da47b7b1ce53ba in_seq=0 out_seq=0 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.981+0000 7fba65ffb640 10 client.25534 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba65ffb640 1 -- 172.21.15.69:0/3763332895 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564038fba0e0 con 0x5640390635d0 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba65ffb640 10 client.25534 handle_command_reply: tid=0 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 client.25534 shutdown 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 msgr2=0x564039050930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:35.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039050930 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fba74003c50 tx=0x7fba74004410 comp rx=0 tx=0).stop 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 shutdown_connections 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564039053970 0x5640390513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5640390635d0 0x564039065a20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390490d0 0x564039050e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3763332895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390487b0 0x564039050930 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 >> 172.21.15.69:0/3763332895 conn(0x564039044d20 msgr2=0x56403904aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.982+0000 7fba65ffb640 0 client.25534 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.983+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 shutdown_connections 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.983+0000 7fba86fdf740 1 -- 172.21.15.69:0/3763332895 wait complete. 2024-08-02T17:48:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:35.983+0000 7fba86fdf740 20 client.25534 trim_cache size 0 max 16384 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.013+0000 7fba86fdf740 1 Processor -- start 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.013+0000 7fba86fdf740 1 -- start start 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.013+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403904bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.013+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390490d0 0x56403906b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.013+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x56403906b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.013+0000 7fba86fdf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1b7ca0 con 0x5640390487b0 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.013+0000 7fba86fdf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1b5570 con 0x564039053970 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.013+0000 7fba86fdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1b33c0 con 0x5640390490d0 2024-08-02T17:48:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390490d0 0x56403906b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390490d0 0x56403906b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51742/0 (socket says 172.21.15.69:51742) 2024-08-02T17:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 -- 172.21.15.69:0/2710510601 learned_addr learned my addr 172.21.15.69:0/2710510601 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 -- 172.21.15.69:0/2710510601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 msgr2=0x56403906b870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:48:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 --2- 172.21.15.69:0/2710510601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x56403906b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 -- 172.21.15.69:0/2710510601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 msgr2=0x56403904bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 --2- 172.21.15.69:0/2710510601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403904bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 -- 172.21.15.69:0/2710510601 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564039048f00 con 0x5640390490d0 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba8182c640 1 --2- 172.21.15.69:0/2710510601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390490d0 0x56403906b330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0ecc00 tx=0x7fba6c0a6040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a37d570fe5cacdde server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba67fff640 1 -- 172.21.15.69:0/2710510601 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0a4e60 con 0x5640390490d0 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba67fff640 1 -- 172.21.15.69:0/2710510601 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6c0bb920 con 0x5640390490d0 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.014+0000 7fba67fff640 1 -- 172.21.15.69:0/2710510601 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004330 con 0x5640390490d0 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.015+0000 7fba86fdf740 1 -- 172.21.15.69:0/2710510601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390490d0 msgr2=0x56403906b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.015+0000 7fba86fdf740 1 --2- 172.21.15.69:0/2710510601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390490d0 0x56403906b330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0ecc00 tx=0x7fba6c0a6040 comp rx=0 tx=0).stop 2024-08-02T17:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.016+0000 7fba86fdf740 1 -- 172.21.15.69:0/2710510601 shutdown_connections 2024-08-02T17:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.016+0000 7fba86fdf740 1 --2- 172.21.15.69:0/2710510601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564039053970 0x56403906b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.016+0000 7fba86fdf740 1 --2- 172.21.15.69:0/2710510601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5640390490d0 0x56403906b330 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0ecc00 tx=0x7fba6c0a6040 comp rx=0 tx=0).stop 2024-08-02T17:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.016+0000 7fba86fdf740 1 --2- 172.21.15.69:0/2710510601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403904bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.016+0000 7fba86fdf740 1 -- 172.21.15.69:0/2710510601 >> 172.21.15.69:0/2710510601 conn(0x564039044d20 msgr2=0x5640390528a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.017+0000 7fba86fdf740 1 -- 172.21.15.69:0/2710510601 shutdown_connections 2024-08-02T17:48:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.017+0000 7fba86fdf740 1 -- 172.21.15.69:0/2710510601 wait complete. 2024-08-02T17:48:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.017+0000 7fba86fdf740 1 -- start start 2024-08-02T17:48:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.018+0000 7fba86fdf740 1 Processor -- start 2024-08-02T17:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.018+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403905bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.018+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564039053970 0x564039058060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.018+0000 7fba86fdf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390585a0 0x564039058a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.018+0000 7fba86fdf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1b3a30 con 0x5640390487b0 2024-08-02T17:48:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.018+0000 7fba86fdf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1b4ea0 con 0x5640390585a0 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.018+0000 7fba86fdf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba7c1b4100 con 0x564039053970 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.018+0000 7fba86fdf740 10 client.0 _ll_register_callbacks cb 0x7fba74009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.019+0000 7fba8202d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403905bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.019+0000 7fba8202d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403905bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40586/0 (socket says 172.21.15.69:40586) 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.019+0000 7fba8202d640 1 -- 172.21.15.69:0/3586296860 learned_addr learned my addr 172.21.15.69:0/3586296860 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.019+0000 7fba8202d640 1 -- 172.21.15.69:0/3586296860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564039053970 msgr2=0x564039058060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.019+0000 7fba8202d640 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564039053970 0x564039058060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.019+0000 7fba8202d640 1 -- 172.21.15.69:0/3586296860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390585a0 msgr2=0x564039058a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.019+0000 7fba8202d640 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390585a0 0x564039058a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.019+0000 7fba8202d640 1 -- 172.21.15.69:0/3586296860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640390ce310 con 0x5640390487b0 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.020+0000 7fba8202d640 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403905bdd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fba74009b50 tx=0x7fba74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefb333f1fd9d47b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.020+0000 7fba65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:48:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.021+0000 7fba65ffb640 1 -- 172.21.15.69:0/3586296860 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74004b20 con 0x5640390487b0 2024-08-02T17:48:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.021+0000 7fba65ffb640 1 -- 172.21.15.69:0/3586296860 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fba74004cc0 con 0x5640390487b0 2024-08-02T17:48:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.021+0000 7fba86fdf740 1 -- 172.21.15.69:0/3586296860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba6c0bb920 con 0x5640390487b0 2024-08-02T17:48:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.021+0000 7fba65ffb640 1 -- 172.21.15.69:0/3586296860 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba74011650 con 0x5640390487b0 2024-08-02T17:48:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.026+0000 7fba86fdf740 10 client.16032 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.027+0000 7fba86fdf740 1 -- 172.21.15.69:0/3586296860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5640390cf420 con 0x5640390487b0 2024-08-02T17:48:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.032+0000 7fba65ffb640 1 -- 172.21.15.69:0/3586296860 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba6c0bb920 con 0x5640390487b0 2024-08-02T17:48:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.032+0000 7fba65ffb640 1 -- 172.21.15.69:0/3586296860 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fba74040a80 con 0x5640390487b0 2024-08-02T17:48:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.032+0000 7fba86fdf740 10 client.16032 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.032+0000 7fba86fdf740 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.032+0000 7fba86fdf740 20 client.16032 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.032+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5640390cfb20 0x5640390d1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.032+0000 7fba86fdf740 4 client.16032 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:48:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.032+0000 7fba86fdf740 1 -- 172.21.15.69:0/3586296860 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564038fb4430 con 0x5640390cfb20 2024-08-02T17:48:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.034+0000 7fba8182c640 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5640390cfb20 0x5640390d1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.042+0000 7fba8182c640 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5640390cfb20 0x5640390d1f80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d63904c899f167af server_cookie=91cbcf00efd9ba62 in_seq=0 out_seq=0 2024-08-02T17:48:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.043+0000 7fba65ffb640 10 client.16032 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba65ffb640 1 -- 172.21.15.69:0/3586296860 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564038fb4430 con 0x5640390cfb20 2024-08-02T17:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba65ffb640 10 client.16032 handle_command_reply: tid=0 2024-08-02T17:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 client.16032 shutdown 2024-08-02T17:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 -- 172.21.15.69:0/3586296860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 msgr2=0x56403905bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403905bdd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fba74009b50 tx=0x7fba74004a40 comp rx=0 tx=0).stop 2024-08-02T17:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 -- 172.21.15.69:0/3586296860 shutdown_connections 2024-08-02T17:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5640390cfb20 0x5640390d1f80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5640390585a0 0x564039058a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564039053970 0x564039058060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 --2- 172.21.15.69:0/3586296860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5640390487b0 0x56403905bdd0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 -- 172.21.15.69:0/3586296860 >> 172.21.15.69:0/3586296860 conn(0x564039044d20 msgr2=0x564039051ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba65ffb640 0 client.16032 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 -- 172.21.15.69:0/3586296860 shutdown_connections 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 1 -- 172.21.15.69:0/3586296860 wait complete. 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.633+0000 7fba86fdf740 20 client.16032 trim_cache size 0 max 16384 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:48:36.636 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:48:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:48:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 -- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba680a2dc0 msgr2=0x7fba680a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba680a2dc0 0x7fba680a5280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fba78005fd0 tx=0x7fba78011810 comp rx=0 tx=0).stop 2024-08-02T17:48:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 -- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c10d200 msgr2=0x7fba7c1a4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c10d200 0x7fba7c1a4930 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fba6c02a8b0 tx=0x7fba6c033d10 comp rx=0 tx=0).stop 2024-08-02T17:48:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 -- 172.21.15.69:0/3161788494 shutdown_connections 2024-08-02T17:48:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba7c10d200 0x7fba7c1a4930 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.637 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba680a2dc0 0x7fba680a5280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba7c19e5a0 0x7fba7c19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 --2- 172.21.15.69:0/3161788494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba7c1080b0 0x7fba7c1a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.634+0000 7fba84291640 1 -- 172.21.15.69:0/3161788494 >> 172.21.15.69:0/3161788494 conn(0x7fba7c077810 msgr2=0x7fba7c078ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.635+0000 7fba84291640 1 -- 172.21.15.69:0/3161788494 shutdown_connections 2024-08-02T17:48:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:36.635+0000 7fba84291640 1 -- 172.21.15.69:0/3161788494 wait complete. 2024-08-02T17:48:37.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:36 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:48:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:36 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:48:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:36 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:48:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-08-02T17:48:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:48:38.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:48:38.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:37 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:48:38.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:37 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-08-02T17:48:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:48:38.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:48:38.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:37 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:48:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 90.0 s, 2558.8 tps, lat 39.099 ms stddev 43.006 2024-08-02T17:48:40.274 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:39 smithi083 ceph-mon[37031]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 MiB/s wr, 606 op/s 2024-08-02T17:48:40.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:39 smithi077 ceph-mon[37108]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 MiB/s wr, 606 op/s 2024-08-02T17:48:40.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:39 smithi069 ceph-mon[32746]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.3 MiB/s wr, 606 op/s 2024-08-02T17:48:42.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:41 smithi083 ceph-mon[37031]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.6 MiB/s wr, 832 op/s 2024-08-02T17:48:42.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:41 smithi077 ceph-mon[37108]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.6 MiB/s wr, 832 op/s 2024-08-02T17:48:42.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:41 smithi069 ceph-mon[32746]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 9.6 MiB/s wr, 832 op/s 2024-08-02T17:48:43.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:42 smithi083 ceph-mon[37031]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-08-02T17:48:43.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:42 smithi077 ceph-mon[37108]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-08-02T17:48:43.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:42 smithi069 ceph-mon[32746]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.9 MiB/s wr, 764 op/s 2024-08-02T17:48:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 95.0 s, 3314.0 tps, lat 30.172 ms stddev 34.074 2024-08-02T17:48:45.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:45 smithi077 ceph-mon[37108]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 637 op/s 2024-08-02T17:48:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:45 smithi069 ceph-mon[32746]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 637 op/s 2024-08-02T17:48:45.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:45 smithi083 ceph-mon[37031]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 637 op/s 2024-08-02T17:48:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:47 smithi077 ceph-mon[37108]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 991 op/s 2024-08-02T17:48:47.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:47 smithi069 ceph-mon[32746]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 991 op/s 2024-08-02T17:48:47.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:47 smithi083 ceph-mon[37031]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 991 op/s 2024-08-02T17:48:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:49 smithi077 ceph-mon[37108]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 689 op/s 2024-08-02T17:48:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:49 smithi069 ceph-mon[32746]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 689 op/s 2024-08-02T17:48:49.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:49 smithi083 ceph-mon[37031]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.7 MiB/s wr, 689 op/s 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.778+0000 7f228d6cd640 1 -- 172.21.15.69:0/527091251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288103a00 msgr2=0x7f2288103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.779+0000 7f228d6cd640 1 --2- 172.21.15.69:0/527091251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288103a00 0x7f2288103e60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2270009860 tx=0x7f227002a340 comp rx=0 tx=0).stop 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.779+0000 7f228d6cd640 1 -- 172.21.15.69:0/527091251 shutdown_connections 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.779+0000 7f228d6cd640 1 --2- 172.21.15.69:0/527091251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22881043a0 0x7f22881108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.779+0000 7f228d6cd640 1 --2- 172.21.15.69:0/527091251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288103a00 0x7f2288103e60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.779+0000 7f228d6cd640 1 --2- 172.21.15.69:0/527091251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2288109a10 0x7f2288109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.779+0000 7f228d6cd640 1 -- 172.21.15.69:0/527091251 >> 172.21.15.69:0/527091251 conn(0x7f22880ff6a0 msgr2=0x7f2288101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.779+0000 7f228d6cd640 1 -- 172.21.15.69:0/527091251 shutdown_connections 2024-08-02T17:48:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.779+0000 7f228d6cd640 1 -- 172.21.15.69:0/527091251 wait complete. 2024-08-02T17:48:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.780+0000 7f228d6cd640 1 Processor -- start 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f228d6cd640 1 -- start start 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f228d6cd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2288103a00 0x7f228819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f228d6cd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22881043a0 0x7f228819a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f228d6cd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288109a10 0x7f228819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f228d6cd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2288113e50 con 0x7f2288103a00 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f228d6cd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2288113cd0 con 0x7f2288109a10 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f228d6cd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2288113fd0 con 0x7f22881043a0 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f227ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22881043a0 0x7f228819a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f22877fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288109a10 0x7f228819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f22877fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288109a10 0x7f228819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:32842/0 (socket says 172.21.15.69:32842) 2024-08-02T17:48:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f227ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22881043a0 0x7f228819a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:46170/0 (socket says 172.21.15.69:46170) 2024-08-02T17:48:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f22877fe640 1 -- 172.21.15.69:0/3827745309 learned_addr learned my addr 172.21.15.69:0/3827745309 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.781+0000 7f2286ffd640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2288103a00 0x7f228819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f22877fe640 1 -- 172.21.15.69:0/3827745309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22881043a0 msgr2=0x7f228819a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f22877fe640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22881043a0 0x7f228819a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f22877fe640 1 -- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2288103a00 msgr2=0x7f228819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f22877fe640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2288103a00 0x7f228819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f22877fe640 1 -- 172.21.15.69:0/3827745309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228819f300 con 0x7f2288109a10 2024-08-02T17:48:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f22877fe640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288109a10 0x7f228819ebd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2280009fd0 tx=0x7f228000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f2284ff9640 1 -- 172.21.15.69:0/3827745309 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280004b20 con 0x7f2288109a10 2024-08-02T17:48:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f2284ff9640 1 -- 172.21.15.69:0/3827745309 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2280003e50 con 0x7f2288109a10 2024-08-02T17:48:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f228d6cd640 1 -- 172.21.15.69:0/3827745309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228819f5f0 con 0x7f2288109a10 2024-08-02T17:48:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f2284ff9640 1 -- 172.21.15.69:0/3827745309 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228000b630 con 0x7f2288109a10 2024-08-02T17:48:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.782+0000 7f228d6cd640 1 -- 172.21.15.69:0/3827745309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228806d950 con 0x7f2288109a10 2024-08-02T17:48:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.786+0000 7f2284ff9640 1 -- 172.21.15.69:0/3827745309 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f228001d020 con 0x7f2288109a10 2024-08-02T17:48:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.786+0000 7f2290428740 1 Processor -- start 2024-08-02T17:48:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.786+0000 7f2290428740 1 -- start start 2024-08-02T17:48:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.787+0000 7f2284ff9640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22740a2cf0 0x7f22740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.787+0000 7f227ffff640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22740a2cf0 0x7f22740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.787+0000 7f2284ff9640 1 -- 172.21.15.69:0/3827745309 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f22800cf7c0 con 0x7f2288109a10 2024-08-02T17:48:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.788+0000 7f2290428740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c4626f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.788+0000 7f2290428740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c46279c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.788+0000 7f2290428740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c4627c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.788+0000 7f2290428740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562c461ca9b0 con 0x562c4626f8e0 2024-08-02T17:48:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.788+0000 7f2290428740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562c46172800 con 0x562c4627a180 2024-08-02T17:48:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.788+0000 7f2290428740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c461c6970 con 0x562c4626efc0 2024-08-02T17:48:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c4626f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c4626f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:46186/0 (socket says 172.21.15.69:46186) 2024-08-02T17:48:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f2286ffd640 1 -- 172.21.15.69:0/4250640038 learned_addr learned my addr 172.21.15.69:0/4250640038 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f22877fe640 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c4627c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f2286ffd640 1 -- 172.21.15.69:0/4250640038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 msgr2=0x562c4627c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f2286ffd640 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c4627c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f2286ffd640 1 -- 172.21.15.69:0/4250640038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 msgr2=0x562c46279c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f2286ffd640 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c46279c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.789+0000 7f2286ffd640 1 -- 172.21.15.69:0/4250640038 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c45ede210 con 0x562c4626efc0 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.790+0000 7f22877fe640 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c4627c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.790+0000 7f227ffff640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22740a2cf0 0x7f22740a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f227002a930 tx=0x7f22700058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.790+0000 7f2286ffd640 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c4626f3a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2278009c70 tx=0x7f2278011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a68cb3fe24e35e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.790+0000 7f227d7fa640 1 -- 172.21.15.69:0/4250640038 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227801e070 con 0x562c4626efc0 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.790+0000 7f227d7fa640 1 -- 172.21.15.69:0/4250640038 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2278019b10 con 0x562c4626efc0 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.790+0000 7f227d7fa640 1 -- 172.21.15.69:0/4250640038 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227800ea90 con 0x562c4626efc0 2024-08-02T17:48:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 -- 172.21.15.69:0/4250640038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 msgr2=0x562c4626f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c4626f3a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2278009c70 tx=0x7f2278011810 comp rx=0 tx=0).stop 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 -- 172.21.15.69:0/4250640038 shutdown_connections 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c46279c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c4627c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 --2- 172.21.15.69:0/4250640038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c4626f3a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 -- 172.21.15.69:0/4250640038 >> 172.21.15.69:0/4250640038 conn(0x562c4626b530 msgr2=0x562c4626b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 -- 172.21.15.69:0/4250640038 shutdown_connections 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.791+0000 7f2290428740 1 -- 172.21.15.69:0/4250640038 wait complete. 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.792+0000 7f2290428740 1 -- start start 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.792+0000 7f2290428740 1 Processor -- start 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2290428740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c46277050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2290428740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c46277590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2290428740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c46277ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2290428740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562c4627cda0 con 0x562c4626f8e0 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2290428740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562c46186500 con 0x562c4627a180 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2290428740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562c4627cf20 con 0x562c4626efc0 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2290428740 10 client.0 _ll_register_callbacks cb 0x562c45edda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c46277ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c46277050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c46277ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:32862/0 (socket says 172.21.15.69:32862) 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c46277050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:46194/0 (socket says 172.21.15.69:46194) 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 -- 172.21.15.69:0/293002108 learned_addr learned my addr 172.21.15.69:0/293002108 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f227ffff640 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c46277590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 -- 172.21.15.69:0/293002108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 msgr2=0x562c46277050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c46277050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 -- 172.21.15.69:0/293002108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 msgr2=0x562c46277590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c46277590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 -- 172.21.15.69:0/293002108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c45faa620 con 0x562c4627a180 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f22877fe640 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c46277ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f22800982f0 tx=0x7f228009b520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b769a33d9314719a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.793+0000 7f226e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.794+0000 7f226e7fc640 1 -- 172.21.15.69:0/293002108 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22800b4060 con 0x562c4627a180 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.794+0000 7f226e7fc640 1 -- 172.21.15.69:0/293002108 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f228009bd80 con 0x562c4627a180 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.794+0000 7f2290428740 1 -- 172.21.15.69:0/293002108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2278019b10 con 0x562c4627a180 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.794+0000 7f226e7fc640 1 -- 172.21.15.69:0/293002108 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2280084e60 con 0x562c4627a180 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.794+0000 7f2290428740 10 client.25676 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.794+0000 7f2290428740 1 -- 172.21.15.69:0/293002108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c46288a40 con 0x562c4627a180 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.795+0000 7f226e7fc640 1 -- 172.21.15.69:0/293002108 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2278019b10 con 0x562c4627a180 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.795+0000 7f226e7fc640 1 -- 172.21.15.69:0/293002108 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f228000b040 con 0x562c4627a180 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.795+0000 7f2290428740 10 client.25676 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.795+0000 7f2290428740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.795+0000 7f2290428740 20 client.25676 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.795+0000 7f2290428740 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562c4628a3d0 0x562c4628c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.795+0000 7f2290428740 4 client.25676 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.795+0000 7f2290428740 1 -- 172.21.15.69:0/293002108 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c461d80e0 con 0x562c4628a3d0 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.796+0000 7f2286ffd640 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562c4628a3d0 0x562c4628c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.796+0000 7f226e7fc640 10 client.25676 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.796+0000 7f2286ffd640 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562c4628a3d0 0x562c4628c820 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a841f3cb62d4551b server_cookie=5f3b1254c17633da in_seq=0 out_seq=0 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.797+0000 7f226e7fc640 1 -- 172.21.15.69:0/293002108 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c461d80e0 con 0x562c4628a3d0 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.797+0000 7f226e7fc640 10 client.25676 handle_command_reply: tid=0 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.797+0000 7f2290428740 1 client.25676 shutdown 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 -- 172.21.15.69:0/293002108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 msgr2=0x562c46277ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c46277ad0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f22800982f0 tx=0x7f228009b520 comp rx=0 tx=0).stop 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 -- 172.21.15.69:0/293002108 shutdown_connections 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c46277590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562c4628a3d0 0x562c4628c820 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4627a180 0x562c46277ad0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 --2- 172.21.15.69:0/293002108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626efc0 0x562c46277050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 -- 172.21.15.69:0/293002108 >> 172.21.15.69:0/293002108 conn(0x562c4626b530 msgr2=0x562c46271290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f226e7fc640 0 client.25676 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 -- 172.21.15.69:0/293002108 shutdown_connections 2024-08-02T17:48:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 1 -- 172.21.15.69:0/293002108 wait complete. 2024-08-02T17:48:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.798+0000 7f2290428740 20 client.25676 trim_cache size 0 max 16384 2024-08-02T17:48:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.823+0000 7f2290428740 1 Processor -- start 2024-08-02T17:48:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.824+0000 7f2290428740 1 -- start start 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.824+0000 7f2290428740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626b530 0x562c462858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.824+0000 7f2290428740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c46274290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.824+0000 7f2290428740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c462747d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.824+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626b530 0x562c462858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.824+0000 7f227ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c46274290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.824+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626b530 0x562c462858c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:46200/0 (socket says 172.21.15.69:46200) 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2286ffd640 1 -- 172.21.15.69:0/3619652399 learned_addr learned my addr 172.21.15.69:0/3619652399 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.824+0000 7f2290428740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f22881b7cc0 con 0x562c4626f8e0 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2290428740 1 -- 172.21.15.69:0/3619652399 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2288073950 con 0x562c4626efc0 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2290428740 1 -- 172.21.15.69:0/3619652399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22880717c0 con 0x562c4626b530 2024-08-02T17:48:49.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f22877fe640 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c462747d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2286ffd640 1 -- 172.21.15.69:0/3619652399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 msgr2=0x562c46274290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2286ffd640 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c46274290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2286ffd640 1 -- 172.21.15.69:0/3619652399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 msgr2=0x562c462747d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2286ffd640 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c462747d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2286ffd640 1 -- 172.21.15.69:0/3619652399 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c4626b360 con 0x562c4626b530 2024-08-02T17:48:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f227ffff640 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c46274290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:48:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.825+0000 7f2286ffd640 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626b530 0x562c462858c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2278009250 tx=0x7f22780051e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d526e7f685f59be2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.826+0000 7f227cff9640 1 -- 172.21.15.69:0/3619652399 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278033070 con 0x562c4626b530 2024-08-02T17:48:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.826+0000 7f227cff9640 1 -- 172.21.15.69:0/3619652399 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227801f040 con 0x562c4626b530 2024-08-02T17:48:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.826+0000 7f227cff9640 1 -- 172.21.15.69:0/3619652399 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227801e040 con 0x562c4626b530 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 -- 172.21.15.69:0/3619652399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626b530 msgr2=0x562c462858c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626b530 0x562c462858c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2278009250 tx=0x7f22780051e0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 -- 172.21.15.69:0/3619652399 shutdown_connections 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626f8e0 0x562c462747d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c46274290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 --2- 172.21.15.69:0/3619652399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626b530 0x562c462858c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 -- 172.21.15.69:0/3619652399 >> 172.21.15.69:0/3619652399 conn(0x562c4628c980 msgr2=0x562c462851b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 -- 172.21.15.69:0/3619652399 shutdown_connections 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 -- 172.21.15.69:0/3619652399 wait complete. 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.827+0000 7f2290428740 1 -- start start 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2290428740 1 Processor -- start 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2290428740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626b530 0x562c4628f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2290428740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c4628fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2290428740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626f8e0 0x562c462ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2290428740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2288071e10 con 0x562c4626b530 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2290428740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2288073280 con 0x562c4626efc0 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2290428740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22880724e0 con 0x562c4626f8e0 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2290428740 10 client.0 _ll_register_callbacks cb 0x7f228009b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f2286ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626b530 0x562c4628f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f227ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c4628fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f227ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c4628fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:32882/0 (socket says 172.21.15.69:32882) 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.828+0000 7f227ffff640 1 -- 172.21.15.69:0/1817501612 learned_addr learned my addr 172.21.15.69:0/1817501612 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f227ffff640 1 -- 172.21.15.69:0/1817501612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626f8e0 msgr2=0x562c462ed080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f227ffff640 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626f8e0 0x562c462ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f227ffff640 1 -- 172.21.15.69:0/1817501612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626b530 msgr2=0x562c4628f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f227ffff640 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626b530 0x562c4628f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f227ffff640 1 -- 172.21.15.69:0/1817501612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c462ed6c0 con 0x562c4626efc0 2024-08-02T17:48:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f227ffff640 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c4628fbe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f227002a860 tx=0x7f2270005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78908312ccda6068 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:48:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f226e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:48:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f226e7fc640 1 -- 172.21.15.69:0/1817501612 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270037040 con 0x562c4626efc0 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.829+0000 7f226e7fc640 1 -- 172.21.15.69:0/1817501612 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f22700037f0 con 0x562c4626efc0 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.830+0000 7f2290428740 1 -- 172.21.15.69:0/1817501612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f227801f040 con 0x562c4626efc0 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.830+0000 7f226e7fc640 1 -- 172.21.15.69:0/1817501612 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f22700039f0 con 0x562c4626efc0 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.830+0000 7f2290428740 10 client.25682 fetch_fsmap learned FSMap version 26 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.830+0000 7f2290428740 1 -- 172.21.15.69:0/1817501612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c462f4a70 con 0x562c4626efc0 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.831+0000 7f226e7fc640 1 -- 172.21.15.69:0/1817501612 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f227801f040 con 0x562c4626efc0 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.831+0000 7f226e7fc640 1 -- 172.21.15.69:0/1817501612 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f227003d440 con 0x562c4626efc0 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.831+0000 7f2290428740 10 client.25682 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.831+0000 7f2290428740 10 client.25682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.831+0000 7f2290428740 20 client.25682 populate_metadata read hostname 'smithi069' 2024-08-02T17:48:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.831+0000 7f2290428740 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562c4627a180 0x562c462f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:48:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.831+0000 7f2290428740 4 client.25682 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:48:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.831+0000 7f2290428740 1 -- 172.21.15.69:0/1817501612 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c461d2430 con 0x562c4627a180 2024-08-02T17:48:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.832+0000 7f2286ffd640 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562c4627a180 0x562c462f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:48:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.832+0000 7f2286ffd640 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562c4627a180 0x562c462f7230 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4498be8bfde7911 server_cookie=ad8af488c54f4828 in_seq=0 out_seq=0 2024-08-02T17:48:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.832+0000 7f226e7fc640 10 client.25682 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.848+0000 7f226e7fc640 1 -- 172.21.15.69:0/1817501612 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562c461d2430 con 0x562c4627a180 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.848+0000 7f226e7fc640 10 client.25682 handle_command_reply: tid=0 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 client.25682 shutdown 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 -- 172.21.15.69:0/1817501612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 msgr2=0x562c4628fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c4628fbe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f227002a860 tx=0x7f2270005f90 comp rx=0 tx=0).stop 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 -- 172.21.15.69:0/1817501612 shutdown_connections 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562c4627a180 0x562c462f7230 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562c4626f8e0 0x562c462ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f226e7fc640 0 client.25682 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562c4626efc0 0x562c4628fbe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 --2- 172.21.15.69:0/1817501612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562c4626b530 0x562c4628f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 -- 172.21.15.69:0/1817501612 >> 172.21.15.69:0/1817501612 conn(0x562c4628c980 msgr2=0x562c462859d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 -- 172.21.15.69:0/1817501612 shutdown_connections 2024-08-02T17:48:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 1 -- 172.21.15.69:0/1817501612 wait complete. 2024-08-02T17:48:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.849+0000 7f2290428740 20 client.25682 trim_cache size 0 max 16384 2024-08-02T17:48:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:48:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:48:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:48:49.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:48:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 -- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22740a2cf0 msgr2=0x7f22740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22740a2cf0 0x7f22740a51b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f227002a930 tx=0x7f22700058d0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 -- 172.21.15.69:0/3827745309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288109a10 msgr2=0x7f228819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:48:49.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288109a10 0x7f228819ebd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2280009fd0 tx=0x7f228000c800 comp rx=0 tx=0).stop 2024-08-02T17:48:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 -- 172.21.15.69:0/3827745309 shutdown_connections 2024-08-02T17:48:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22740a2cf0 0x7f22740a51b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2288109a10 0x7f228819ebd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22881043a0 0x7f228819a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 --2- 172.21.15.69:0/3827745309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2288103a00 0x7f228819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:48:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.850+0000 7f228d6cd640 1 -- 172.21.15.69:0/3827745309 >> 172.21.15.69:0/3827745309 conn(0x7f22880ff6a0 msgr2=0x7f22880ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:48:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.851+0000 7f228d6cd640 1 -- 172.21.15.69:0/3827745309 shutdown_connections 2024-08-02T17:48:49.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:48:49.851+0000 7f228d6cd640 1 -- 172.21.15.69:0/3827745309 wait complete. 2024-08-02T17:48:49.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 100.0 s, 2776.6 tps, lat 35.817 ms stddev 44.116 2024-08-02T17:48:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:51 smithi077 ceph-mon[37108]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-02T17:48:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:51 smithi069 ceph-mon[32746]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-02T17:48:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:51 smithi083 ceph-mon[37031]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2024-08-02T17:48:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:53 smithi077 ceph-mon[37108]: pgmap v77: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-08-02T17:48:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:53 smithi069 ceph-mon[32746]: pgmap v77: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-08-02T17:48:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:53 smithi083 ceph-mon[37031]: pgmap v77: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 13 MiB/s wr, 959 op/s 2024-08-02T17:48:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 105.0 s, 3994.0 tps, lat 25.125 ms stddev 31.351 2024-08-02T17:48:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:55 smithi077 ceph-mon[37108]: pgmap v78: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 849 op/s 2024-08-02T17:48:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:55 smithi069 ceph-mon[32746]: pgmap v78: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 849 op/s 2024-08-02T17:48:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:55 smithi083 ceph-mon[37031]: pgmap v78: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 849 op/s 2024-08-02T17:48:57.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:57 smithi077 ceph-mon[37108]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-08-02T17:48:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:57 smithi069 ceph-mon[32746]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-08-02T17:48:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:57 smithi083 ceph-mon[37031]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 18 MiB/s wr, 1.24k op/s 2024-08-02T17:48:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:48:59 smithi077 ceph-mon[37108]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 887 op/s 2024-08-02T17:48:59.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:48:59 smithi069 ceph-mon[32746]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 887 op/s 2024-08-02T17:48:59.376 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:48:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:48:59 smithi083 ceph-mon[37031]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 MiB/s wr, 887 op/s 2024-08-02T17:48:59.810 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:48:59.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 110.0 s, 2793.2 tps, lat 35.747 ms stddev 42.444 2024-08-02T17:49:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 -- 172.21.15.69:0/3595595008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8680ff930 msgr2=0x7ff8680ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 --2- 172.21.15.69:0/3595595008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8680ff930 0x7ff8680ffd90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff854009890 tx=0x7ff85402a380 comp rx=0 tx=0).stop 2024-08-02T17:49:00.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 -- 172.21.15.69:0/3595595008 shutdown_connections 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 --2- 172.21.15.69:0/3595595008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8681002d0 0x7ff86810e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 --2- 172.21.15.69:0/3595595008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8680ff930 0x7ff8680ffd90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 --2- 172.21.15.69:0/3595595008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8681068b0 0x7ff868106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 -- 172.21.15.69:0/3595595008 >> 172.21.15.69:0/3595595008 conn(0x7ff8680fb7c0 msgr2=0x7ff8680fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 -- 172.21.15.69:0/3595595008 shutdown_connections 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 -- 172.21.15.69:0/3595595008 wait complete. 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.375+0000 7ff86ec69640 1 Processor -- start 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ec69640 1 -- start start 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ec69640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8680ff930 0x7ff8681981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ec69640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8681002d0 0x7ff8681986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ec69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8681068b0 0x7ff86819ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ec69640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff868111cd0 con 0x7ff8681002d0 2024-08-02T17:49:00.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ec69640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff868111b50 con 0x7ff8680ff930 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ec69640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff868111e50 con 0x7ff8681068b0 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8680ff930 0x7ff8681981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86d206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8681068b0 0x7ff86819ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff867fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8681002d0 0x7ff8681986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86d206640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8681068b0 0x7ff86819ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57922/0 (socket says 172.21.15.69:57922) 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8680ff930 0x7ff8681981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:56452/0 (socket says 172.21.15.69:56452) 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86d206640 1 -- 172.21.15.69:0/886927754 learned_addr learned my addr 172.21.15.69:0/886927754 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86d206640 1 -- 172.21.15.69:0/886927754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8680ff930 msgr2=0x7ff8681981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86d206640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8680ff930 0x7ff8681981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86d206640 1 -- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8681002d0 msgr2=0x7ff8681986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86d206640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8681002d0 0x7ff8681986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.376+0000 7ff86d206640 1 -- 172.21.15.69:0/886927754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86819d160 con 0x7ff8681068b0 2024-08-02T17:49:00.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.377+0000 7ff86d206640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8681068b0 0x7ff86819ca80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff860009fa0 tx=0x7ff86000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.377+0000 7ff865ffb640 1 -- 172.21.15.69:0/886927754 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860004b20 con 0x7ff8681068b0 2024-08-02T17:49:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.377+0000 7ff86ec69640 1 -- 172.21.15.69:0/886927754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86819d450 con 0x7ff8681068b0 2024-08-02T17:49:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.377+0000 7ff865ffb640 1 -- 172.21.15.69:0/886927754 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff860003e50 con 0x7ff8681068b0 2024-08-02T17:49:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.377+0000 7ff865ffb640 1 -- 172.21.15.69:0/886927754 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86000b570 con 0x7ff8681068b0 2024-08-02T17:49:00.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.377+0000 7ff86ec69640 1 -- 172.21.15.69:0/886927754 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681012d0 con 0x7ff8681068b0 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.379+0000 7ff865ffb640 1 -- 172.21.15.69:0/886927754 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff86001d020 con 0x7ff8681068b0 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff871067740 1 Processor -- start 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff871067740 1 -- start start 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff865ffb640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff8580a2cf0 0x7ff8580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff871067740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ee35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff865ffb640 1 -- 172.21.15.69:0/886927754 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff8600cf830 con 0x7ff8681068b0 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff867fff640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff8580a2cf0 0x7ff8580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff871067740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60ee3b30 0x55bc60eedeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff871067740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60eee3f0 0x55bc60ef07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff871067740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bc60e48980 con 0x55bc60ee3210 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ee35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff871067740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bc60de0a70 con 0x55bc60ee3b30 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff871067740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc60e44940 con 0x55bc60eee3f0 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ee35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59580/0 (socket says 172.21.15.69:59580) 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff86ca05640 1 -- 172.21.15.69:0/2667769837 learned_addr learned my addr 172.21.15.69:0/2667769837 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff867fff640 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60ee3b30 0x55bc60eedeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.380+0000 7ff86d206640 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60eee3f0 0x55bc60ef07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff86ca05640 1 -- 172.21.15.69:0/2667769837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60eee3f0 msgr2=0x55bc60ef07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff867fff640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff8580a2cf0 0x7ff8580a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff854000c00 tx=0x7ff854030e50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff86ca05640 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60eee3f0 0x55bc60ef07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff86ca05640 1 -- 172.21.15.69:0/2667769837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60ee3b30 msgr2=0x55bc60eedeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:00.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff86ca05640 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60ee3b30 0x55bc60eedeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff86ca05640 1 -- 172.21.15.69:0/2667769837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc60e436e0 con 0x55bc60ee3210 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff867fff640 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60ee3b30 0x55bc60eedeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff86d206640 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60eee3f0 0x55bc60ef07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff86ca05640 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ee35f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009d20 tx=0x7ff85c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be541bb00ea6ea server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff852ffd640 1 -- 172.21.15.69:0/2667769837 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c01e070 con 0x55bc60ee3210 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff852ffd640 1 -- 172.21.15.69:0/2667769837 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85c019b10 con 0x55bc60ee3210 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.381+0000 7ff852ffd640 1 -- 172.21.15.69:0/2667769837 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c00ea60 con 0x55bc60ee3210 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 -- 172.21.15.69:0/2667769837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 msgr2=0x55bc60ee35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ee35f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009d20 tx=0x7ff85c011810 comp rx=0 tx=0).stop 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 -- 172.21.15.69:0/2667769837 shutdown_connections 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60eee3f0 0x55bc60ef07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60ee3b30 0x55bc60eedeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 --2- 172.21.15.69:0/2667769837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ee35f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 -- 172.21.15.69:0/2667769837 >> 172.21.15.69:0/2667769837 conn(0x55bc60edf7a0 msgr2=0x55bc60edfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 -- 172.21.15.69:0/2667769837 shutdown_connections 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 -- 172.21.15.69:0/2667769837 wait complete. 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 -- start start 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.382+0000 7ff871067740 1 Processor -- start 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff871067740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60eeb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff871067740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60eeb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60eeb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60eeb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59588/0 (socket says 172.21.15.69:59588) 2024-08-02T17:49:00.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff871067740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60eebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 -- 172.21.15.69:0/1969065841 learned_addr learned my addr 172.21.15.69:0/1969065841 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bc60ef0fc0 con 0x55bc60ee3210 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bc60e03aa0 con 0x55bc60eee3f0 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bc60ef1140 con 0x55bc60ee3b30 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff871067740 10 client.0 _ll_register_callbacks cb 0x55bc60df2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff867fff640 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60eeb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86d206640 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60eebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 -- 172.21.15.69:0/1969065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 msgr2=0x55bc60eeb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60eeb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 -- 172.21.15.69:0/1969065841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 msgr2=0x55bc60eebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60eebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 -- 172.21.15.69:0/1969065841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc60de7650 con 0x55bc60ee3210 2024-08-02T17:49:00.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff867fff640 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60eeb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.383+0000 7ff86ca05640 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60eeb2d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0044f0 tx=0x7ff85c004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b538fd908f9859 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff847fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff847fff640 1 -- 172.21.15.69:0/1969065841 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c027020 con 0x55bc60ee3210 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85c019b10 con 0x55bc60ee3210 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff847fff640 1 -- 172.21.15.69:0/1969065841 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff85c019840 con 0x55bc60ee3210 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff847fff640 1 -- 172.21.15.69:0/1969065841 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff85c01e070 con 0x55bc60ee3210 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff871067740 10 client.16056 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc60bba3c0 con 0x55bc60ee3210 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff847fff640 1 -- 172.21.15.69:0/1969065841 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85c01f040 con 0x55bc60ee3210 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.384+0000 7ff847fff640 1 -- 172.21.15.69:0/1969065841 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff85c034020 con 0x55bc60ee3210 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff871067740 10 client.16056 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff871067740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff871067740 20 client.16056 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:00.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff871067740 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bc60efde00 0x55bc60f001f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff871067740 4 client.16056 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff85c019b10 con 0x55bc60efde00 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff867fff640 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bc60efde00 0x55bc60f001f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff847fff640 10 client.16056 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.385+0000 7ff867fff640 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bc60efde00 0x55bc60f001f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68360b2647555e1f server_cookie=faa86f7da5c187ac in_seq=0 out_seq=0 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.386+0000 7ff847fff640 1 -- 172.21.15.69:0/1969065841 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ff85c019b10 con 0x55bc60efde00 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.386+0000 7ff847fff640 10 client.16056 handle_command_reply: tid=0 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.386+0000 7ff871067740 1 client.16056 shutdown 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 msgr2=0x55bc60eeb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60eeb2d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0044f0 tx=0x7ff85c004520 comp rx=0 tx=0).stop 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 shutdown_connections 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bc60efde00 0x55bc60f001f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60eebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60eeb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 --2- 172.21.15.69:0/1969065841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60eeb2d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 >> 172.21.15.69:0/1969065841 conn(0x55bc60edf7a0 msgr2=0x55bc60ee54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff847fff640 0 client.16056 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 shutdown_connections 2024-08-02T17:49:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 1 -- 172.21.15.69:0/1969065841 wait complete. 2024-08-02T17:49:00.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.387+0000 7ff871067740 20 client.16056 trim_cache size 0 max 16384 2024-08-02T17:49:00.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff871067740 1 Processor -- start 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff871067740 1 -- start start 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff871067740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ede7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff871067740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f05980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff871067740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60f05ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff871067740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681af430 con 0x55bc60ee3210 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff871067740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681acd00 con 0x55bc60eee3f0 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff871067740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681aab50 con 0x55bc60ee3b30 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ede7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff867fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f05980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ede7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59596/0 (socket says 172.21.15.69:59596) 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff86ca05640 1 -- 172.21.15.69:0/3031368964 learned_addr learned my addr 172.21.15.69:0/3031368964 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:00.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.407+0000 7ff86d206640 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60f05ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff86ca05640 1 -- 172.21.15.69:0/3031368964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 msgr2=0x55bc60f05980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff86ca05640 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f05980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff86ca05640 1 -- 172.21.15.69:0/3031368964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 msgr2=0x55bc60f05ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff86ca05640 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60f05ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff86ca05640 1 -- 172.21.15.69:0/3031368964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc60edf5d0 con 0x55bc60ee3210 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff867fff640 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f05980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff86d206640 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60f05ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff86ca05640 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ede7f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e610 tx=0x7ff85c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9419f03618b2bc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff8527fc640 1 -- 172.21.15.69:0/3031368964 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c027040 con 0x55bc60ee3210 2024-08-02T17:49:00.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff8527fc640 1 -- 172.21.15.69:0/3031368964 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85c015030 con 0x55bc60ee3210 2024-08-02T17:49:00.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.408+0000 7ff8527fc640 1 -- 172.21.15.69:0/3031368964 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0199a0 con 0x55bc60ee3210 2024-08-02T17:49:00.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.409+0000 7ff871067740 1 -- 172.21.15.69:0/3031368964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 msgr2=0x55bc60ede7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.409+0000 7ff871067740 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ede7f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff85c00e610 tx=0x7ff85c009ed0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.409+0000 7ff871067740 1 -- 172.21.15.69:0/3031368964 shutdown_connections 2024-08-02T17:49:00.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.409+0000 7ff871067740 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60f05ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.409+0000 7ff871067740 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f05980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.409+0000 7ff871067740 1 --2- 172.21.15.69:0/3031368964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60ede7f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.409+0000 7ff871067740 1 -- 172.21.15.69:0/3031368964 >> 172.21.15.69:0/3031368964 conn(0x55bc60edf7a0 msgr2=0x55bc60ede0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.409+0000 7ff871067740 1 -- 172.21.15.69:0/3031368964 shutdown_connections 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 -- 172.21.15.69:0/3031368964 wait complete. 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 -- start start 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 Processor -- start 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60f60c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f61180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60ef18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ab1c0 con 0x55bc60ee3210 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ac630 con 0x55bc60eee3f0 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ab890 con 0x55bc60ee3b30 2024-08-02T17:49:00.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.410+0000 7ff871067740 10 client.0 _ll_register_callbacks cb 0x7ff85c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60f60c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60f60c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59604/0 (socket says 172.21.15.69:59604) 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 -- 172.21.15.69:0/861493670 learned_addr learned my addr 172.21.15.69:0/861493670 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86d206640 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60ef18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff867fff640 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 -- 172.21.15.69:0/861493670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 msgr2=0x55bc60f61180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f61180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 -- 172.21.15.69:0/861493670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 msgr2=0x55bc60ef18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60ef18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 -- 172.21.15.69:0/861493670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc60ef2120 con 0x55bc60ee3210 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff86ca05640 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60f60c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff85c005bc0 tx=0x7ff85c004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf93ba84f928cb25 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff847fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.411+0000 7ff847fff640 1 -- 172.21.15.69:0/861493670 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c027040 con 0x55bc60ee3210 2024-08-02T17:49:00.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 1 -- 172.21.15.69:0/861493670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85c015030 con 0x55bc60ee3210 2024-08-02T17:49:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff847fff640 1 -- 172.21.15.69:0/861493670 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff85c019720 con 0x55bc60ee3210 2024-08-02T17:49:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff847fff640 1 -- 172.21.15.69:0/861493670 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff85c009b70 con 0x55bc60ee3210 2024-08-02T17:49:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 10 client.16062 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 1 -- 172.21.15.69:0/861493670 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bc60f68460 con 0x55bc60ee3210 2024-08-02T17:49:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff847fff640 1 -- 172.21.15.69:0/861493670 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff85c015030 con 0x55bc60ee3210 2024-08-02T17:49:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff847fff640 1 -- 172.21.15.69:0/861493670 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff85c034070 con 0x55bc60ee3210 2024-08-02T17:49:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 10 client.16062 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:00.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 20 client.16062 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bc60f68950 0x55bc60f6ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 4 client.16062 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.412+0000 7ff871067740 1 -- 172.21.15.69:0/861493670 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc60e55ef0 con 0x55bc60f68950 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.413+0000 7ff867fff640 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bc60f68950 0x55bc60f6ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.413+0000 7ff847fff640 10 client.16062 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.413+0000 7ff867fff640 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bc60f68950 0x55bc60f6ad40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6d27a568dae9f7 server_cookie=e819635f153416ac in_seq=0 out_seq=0 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.413+0000 7ff847fff640 1 -- 172.21.15.69:0/861493670 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bc60e55ef0 con 0x55bc60f68950 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.413+0000 7ff847fff640 10 client.16062 handle_command_reply: tid=0 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.413+0000 7ff871067740 1 client.16062 shutdown 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 -- 172.21.15.69:0/861493670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 msgr2=0x55bc60f60c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60f60c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff85c005bc0 tx=0x7ff85c004900 comp rx=0 tx=0).stop 2024-08-02T17:49:00.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 -- 172.21.15.69:0/861493670 shutdown_connections 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bc60f68950 0x55bc60f6ad40 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bc60eee3f0 0x55bc60ef18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff847fff640 0 client.16062 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bc60ee3b30 0x55bc60f61180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 --2- 172.21.15.69:0/861493670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bc60ee3210 0x55bc60f60c40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 -- 172.21.15.69:0/861493670 >> 172.21.15.69:0/861493670 conn(0x55bc60edf7a0 msgr2=0x55bc60ede900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 -- 172.21.15.69:0/861493670 shutdown_connections 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 1 -- 172.21.15.69:0/861493670 wait complete. 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.414+0000 7ff871067740 20 client.16062 trim_cache size 0 max 16384 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:49:00.421 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:49:00.422 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:49:00.422 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:49:00.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 -- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff8580a2cf0 msgr2=0x7ff8580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff8580a2cf0 0x7ff8580a51b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff854000c00 tx=0x7ff854030e50 comp rx=0 tx=0).stop 2024-08-02T17:49:00.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 -- 172.21.15.69:0/886927754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8681068b0 msgr2=0x7ff86819ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:00.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8681068b0 0x7ff86819ca80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff860009fa0 tx=0x7ff86000c800 comp rx=0 tx=0).stop 2024-08-02T17:49:00.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 -- 172.21.15.69:0/886927754 shutdown_connections 2024-08-02T17:49:00.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff8681002d0 0x7ff8681986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff8580a2cf0 0x7ff8580a51b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff8681068b0 0x7ff86819ca80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 --2- 172.21.15.69:0/886927754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff8680ff930 0x7ff8681981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:00.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 -- 172.21.15.69:0/886927754 >> 172.21.15.69:0/886927754 conn(0x7ff8680fb7c0 msgr2=0x7ff8680fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:00.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 -- 172.21.15.69:0/886927754 shutdown_connections 2024-08-02T17:49:00.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:00.415+0000 7ff86ec69640 1 -- 172.21.15.69:0/886927754 wait complete. 2024-08-02T17:49:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:01 smithi077 ceph-mon[37108]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-08-02T17:49:01.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:01 smithi069 ceph-mon[32746]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-08-02T17:49:01.501 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:49:01.502 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:49:01.520 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:01 smithi083 ceph-mon[37031]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-08-02T17:49:01.806 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:02.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.361+0000 7f620f8c2640 1 -- 172.21.15.69:0/2315843392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 msgr2=0x7f6208108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.361+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2315843392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 0x7f6208108500 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009900 tx=0x7f61f802a440 comp rx=0 tx=0).stop 2024-08-02T17:49:02.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.364+0000 7f620f8c2640 1 -- 172.21.15.69:0/2315843392 shutdown_connections 2024-08-02T17:49:02.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.364+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2315843392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620810d250 0x7f6208114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.364+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2315843392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6208108ad0 0x7f620810cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.364+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2315843392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 0x7f6208108500 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.364+0000 7f620f8c2640 1 -- 172.21.15.69:0/2315843392 >> 172.21.15.69:0/2315843392 conn(0x7f6208077820 msgr2=0x7f6208079c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:02.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.370+0000 7f620f8c2640 1 -- 172.21.15.69:0/2315843392 shutdown_connections 2024-08-02T17:49:02.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.370+0000 7f620f8c2640 1 -- 172.21.15.69:0/2315843392 wait complete. 2024-08-02T17:49:02.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.371+0000 7f620f8c2640 1 Processor -- start 2024-08-02T17:49:02.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620f8c2640 1 -- start start 2024-08-02T17:49:02.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620f8c2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 0x7f6208073890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620f8c2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6208108ad0 0x7f620806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620f8c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620810d250 0x7f620806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620f8c2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f62081183e0 con 0x7f6208108120 2024-08-02T17:49:02.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620f8c2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6208118260 con 0x7f6208108ad0 2024-08-02T17:49:02.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620f8c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6208118560 con 0x7f620810d250 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620d65e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 0x7f6208073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620d65e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 0x7f6208073890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59622/0 (socket says 172.21.15.69:59622) 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620d65e640 1 -- 172.21.15.69:0/2311038969 learned_addr learned my addr 172.21.15.69:0/2311038969 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.372+0000 7f620de5f640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620810d250 0x7f620806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.373+0000 7f620ce5d640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6208108ad0 0x7f620806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.373+0000 7f620ce5d640 1 -- 172.21.15.69:0/2311038969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620810d250 msgr2=0x7f620806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.373+0000 7f620ce5d640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620810d250 0x7f620806ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.373+0000 7f620ce5d640 1 -- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 msgr2=0x7f6208073890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.373+0000 7f620ce5d640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 0x7f6208073890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.373+0000 7f620ce5d640 1 -- 172.21.15.69:0/2311038969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620806d590 con 0x7f6208108ad0 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.373+0000 7f620ce5d640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6208108ad0 0x7f620806c920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6204009fd0 tx=0x7f620400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:02.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.374+0000 7f61fe7fc640 1 -- 172.21.15.69:0/2311038969 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6204015020 con 0x7f6208108ad0 2024-08-02T17:49:02.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.374+0000 7f61fe7fc640 1 -- 172.21.15.69:0/2311038969 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6204011d10 con 0x7f6208108ad0 2024-08-02T17:49:02.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.375+0000 7f61fe7fc640 1 -- 172.21.15.69:0/2311038969 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620400f2e0 con 0x7f6208108ad0 2024-08-02T17:49:02.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.376+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62080713c0 con 0x7f6208108ad0 2024-08-02T17:49:02.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.376+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6208071880 con 0x7f6208108ad0 2024-08-02T17:49:02.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.381+0000 7f61fe7fc640 1 -- 172.21.15.69:0/2311038969 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6204018020 con 0x7f6208108ad0 2024-08-02T17:49:02.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.381+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6208109600 con 0x7f6208108ad0 2024-08-02T17:49:02.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.382+0000 7f61fe7fc640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f61f40a2f00 0x7f61f40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.382+0000 7f620d65e640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f61f40a2f00 0x7f61f40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.383+0000 7f61fe7fc640 1 -- 172.21.15.69:0/2311038969 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f62040ced10 con 0x7f6208108ad0 2024-08-02T17:49:02.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.383+0000 7f620d65e640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f61f40a2f00 0x7f61f40a53c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61f80096c0 tx=0x7f61f802a940 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:02.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.388+0000 7f61fe7fc640 1 -- 172.21.15.69:0/2311038969 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f620409b620 con 0x7f6208108ad0 2024-08-02T17:49:02.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.657+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f620806de50 con 0x7f6208108ad0 2024-08-02T17:49:02.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.658+0000 7f61fe7fc640 1 -- 172.21.15.69:0/2311038969 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f620409af20 con 0x7f6208108ad0 2024-08-02T17:49:02.660 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:49:02.660 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:49:02.660 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f61f40a2f00 msgr2=0x7f61f40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f61f40a2f00 0x7f61f40a53c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f61f80096c0 tx=0x7f61f802a940 comp rx=0 tx=0).stop 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6208108ad0 msgr2=0x7f620806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6208108ad0 0x7f620806c920 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6204009fd0 tx=0x7f620400b870 comp rx=0 tx=0).stop 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 shutdown_connections 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f61f40a2f00 0x7f61f40a53c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f620810d250 0x7f620806ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6208108ad0 0x7f620806c920 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 --2- 172.21.15.69:0/2311038969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6208108120 0x7f6208073890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 >> 172.21.15.69:0/2311038969 conn(0x7f6208077820 msgr2=0x7f6208078c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 shutdown_connections 2024-08-02T17:49:02.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:02.661+0000 7f620f8c2640 1 -- 172.21.15.69:0/2311038969 wait complete. 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 -- 172.21.15.69:0/2346659365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 msgr2=0x7f1cc410b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/2346659365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 0x7f1cc410b7b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009900 tx=0x7f1cb402a480 comp rx=0 tx=0).stop 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 -- 172.21.15.69:0/2346659365 shutdown_connections 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/2346659365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 0x7f1cc410b7b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/2346659365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc410bdd0 0x7f1cc410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/2346659365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc41029b0 0x7f1cc4102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 -- 172.21.15.69:0/2346659365 >> 172.21.15.69:0/2346659365 conn(0x7f1cc40776d0 msgr2=0x7f1cc40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 -- 172.21.15.69:0/2346659365 shutdown_connections 2024-08-02T17:49:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.960+0000 7f1ccbdee640 1 -- 172.21.15.69:0/2346659365 wait complete. 2024-08-02T17:49:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1ccbdee640 1 Processor -- start 2024-08-02T17:49:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1ccbdee640 1 -- start start 2024-08-02T17:49:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1ccbdee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc41029b0 0x7f1cc4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1ccbdee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 0x7f1cc41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1ccbdee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc410bdd0 0x7f1cc419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1ccbdee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4111bd0 con 0x7f1cc41032d0 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1ccbdee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4111a50 con 0x7f1cc41029b0 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1ccbdee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4111d50 con 0x7f1cc410bdd0 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1cc9389640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 0x7f1cc41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1cc9389640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 0x7f1cc41986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59646/0 (socket says 172.21.15.69:59646) 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1cc9389640 1 -- 172.21.15.69:0/1948575805 learned_addr learned my addr 172.21.15.69:0/1948575805 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc41029b0 0x7f1cc4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.961+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc410bdd0 0x7f1cc419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cc9389640 1 -- 172.21.15.69:0/1948575805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc410bdd0 msgr2=0x7f1cc419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cc9389640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc410bdd0 0x7f1cc419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cc9389640 1 -- 172.21.15.69:0/1948575805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc41029b0 msgr2=0x7f1cc4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cc9389640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc41029b0 0x7f1cc4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cc9389640 1 -- 172.21.15.69:0/1948575805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc419d170 con 0x7f1cc41032d0 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cc9389640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 0x7f1cc41986c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4006010 tx=0x7f1cb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cbaffd640 1 -- 172.21.15.69:0/1948575805 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb403f030 con 0x7f1cc41032d0 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cbaffd640 1 -- 172.21.15.69:0/1948575805 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb4031990 con 0x7f1cc41032d0 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1cbaffd640 1 -- 172.21.15.69:0/1948575805 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4035070 con 0x7f1cc41032d0 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1ccbdee640 1 -- 172.21.15.69:0/1948575805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc419d400 con 0x7f1cc41032d0 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.962+0000 7f1ccbdee640 1 -- 172.21.15.69:0/1948575805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc41a4cf0 con 0x7f1cc41032d0 2024-08-02T17:49:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.965+0000 7f1cbaffd640 1 -- 172.21.15.69:0/1948575805 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1cb4042030 con 0x7f1cc41032d0 2024-08-02T17:49:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1ccec23740 1 Processor -- start 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1ccec23740 1 -- start start 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1cbaffd640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1cb00a2cf0 0x7f1cb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443ef67b0 0x556443ef6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f01430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443f01970 0x556443f03d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1ccec23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556443e5a9b0 con 0x556443ef67b0 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1ccec23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556443e02800 con 0x556443f01970 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1ccec23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556443e56970 con 0x556443ef70d0 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1cbaffd640 1 -- 172.21.15.69:0/1948575805 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f1cb40efa40 con 0x7f1cc41032d0 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1cc9b8a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443ef67b0 0x556443ef6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1cca38b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443f01970 0x556443f03d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1cc9389640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f01430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1cc9b8a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443ef67b0 0x556443ef6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59658/0 (socket says 172.21.15.69:59658) 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1cc9b8a640 1 -- 172.21.15.69:0/1793601829 learned_addr learned my addr 172.21.15.69:0/1793601829 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.966+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1cb00a2cf0 0x7f1cb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1cc9b8a640 1 -- 172.21.15.69:0/1793601829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 msgr2=0x556443f01430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1793601829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f01430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1cc9b8a640 1 -- 172.21.15.69:0/1793601829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443f01970 msgr2=0x556443f03d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1793601829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443f01970 0x556443f03d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1cc9b8a640 1 -- 172.21.15.69:0/1793601829 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556443b6e210 con 0x556443ef67b0 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1cc9389640 1 --2- 172.21.15.69:0/1793601829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f01430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1cb00a2cf0 0x7f1cb00a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009fd0 tx=0x7f1cbc009b10 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1793601829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443ef67b0 0x556443ef6b90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009cf0 tx=0x7f1cc0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382234fbbb0101b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1caffff640 1 -- 172.21.15.69:0/1793601829 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc001e070 con 0x556443ef67b0 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1caffff640 1 -- 172.21.15.69:0/1793601829 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc0019b10 con 0x556443ef67b0 2024-08-02T17:49:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.967+0000 7f1caffff640 1 -- 172.21.15.69:0/1793601829 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000e990 con 0x556443ef67b0 2024-08-02T17:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 -- 172.21.15.69:0/1793601829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443ef67b0 msgr2=0x556443ef6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1793601829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443ef67b0 0x556443ef6b90 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009cf0 tx=0x7f1cc0011810 comp rx=0 tx=0).stop 2024-08-02T17:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 -- 172.21.15.69:0/1793601829 shutdown_connections 2024-08-02T17:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1793601829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443f01970 0x556443f03d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1793601829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f01430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1793601829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443ef67b0 0x556443ef6b90 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 -- 172.21.15.69:0/1793601829 >> 172.21.15.69:0/1793601829 conn(0x556443ef2d20 msgr2=0x556443ef3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:02.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 -- 172.21.15.69:0/1793601829 shutdown_connections 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 -- 172.21.15.69:0/1793601829 wait complete. 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 -- start start 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.968+0000 7f1ccec23740 1 Processor -- start 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 0x556443efe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443efed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443eff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1ccec23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556443f08950 con 0x556443f01970 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1ccec23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556443e16500 con 0x556443ef70d0 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1ccec23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556443f08ad0 con 0x556443ef67b0 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1ccec23740 10 client.0 _ll_register_callbacks cb 0x556443b6da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443efed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cca38b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443eff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9b8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 0x556443efe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443efed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:56526/0 (socket says 172.21.15.69:56526) 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cca38b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443eff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59674/0 (socket says 172.21.15.69:59674) 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 -- 172.21.15.69:0/1196290260 learned_addr learned my addr 172.21.15.69:0/1196290260 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 -- 172.21.15.69:0/1196290260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 msgr2=0x556443efe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 0x556443efe840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 -- 172.21.15.69:0/1196290260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 msgr2=0x556443eff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443eff2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 -- 172.21.15.69:0/1196290260 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556443c3a620 con 0x556443ef70d0 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cc9389640 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443efed80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40ece80 tx=0x7f1cb40bba30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d57a277eb788d32d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1196290260 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4049040 con 0x556443ef70d0 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1ccec23740 1 -- 172.21.15.69:0/1196290260 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc0019b10 con 0x556443ef70d0 2024-08-02T17:49:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.969+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1196290260 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f1cb40a63e0 con 0x556443ef70d0 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1196290260 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cb40044e0 con 0x556443ef70d0 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1ccec23740 10 client.25715 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1ccec23740 1 -- 172.21.15.69:0/1196290260 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556443f14590 con 0x556443ef70d0 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1196290260 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cc0019b10 con 0x556443ef70d0 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1196290260 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1cb40ed050 con 0x556443ef70d0 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1ccec23740 10 client.25715 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1ccec23740 10 client.25715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1ccec23740 20 client.25715 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556443f15a40 0x556443f17e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1ccec23740 4 client.25715 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1ccec23740 1 -- 172.21.15.69:0/1196290260 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556443e680e0 con 0x556443f15a40 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.970+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556443f15a40 0x556443f17e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556443f15a40 0x556443f17e90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7039e5319106502b server_cookie=c23c60c18afa5429 in_seq=0 out_seq=0 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1cad7fa640 10 client.25715 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1196290260 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556443e680e0 con 0x556443f15a40 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1cad7fa640 10 client.25715 handle_command_reply: tid=0 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 client.25715 shutdown 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 -- 172.21.15.69:0/1196290260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 msgr2=0x556443efed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443efed80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40ece80 tx=0x7f1cb40bba30 comp rx=0 tx=0).stop 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 -- 172.21.15.69:0/1196290260 shutdown_connections 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443eff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556443f15a40 0x556443f17e90 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1cad7fa640 0 client.25715 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443efed80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1196290260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 0x556443efe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 -- 172.21.15.69:0/1196290260 >> 172.21.15.69:0/1196290260 conn(0x556443ef2d20 msgr2=0x556443ef8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 -- 172.21.15.69:0/1196290260 shutdown_connections 2024-08-02T17:49:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.971+0000 7f1ccec23740 1 -- 172.21.15.69:0/1196290260 wait complete. 2024-08-02T17:49:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.972+0000 7f1ccec23740 20 client.25715 trim_cache size 0 max 16384 2024-08-02T17:49:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1ccec23740 1 Processor -- start 2024-08-02T17:49:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1ccec23740 1 -- start start 2024-08-02T17:49:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 0x556443f1d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443f1dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443ef8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1ccec23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41af370 con 0x556443f01970 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1ccec23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41acc40 con 0x556443ef70d0 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1ccec23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41aaa90 con 0x556443ef67b0 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cca38b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443ef8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cca38b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443ef8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59678/0 (socket says 172.21.15.69:59678) 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cca38b640 1 -- 172.21.15.69:0/1696569744 learned_addr learned my addr 172.21.15.69:0/1696569744 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cc9389640 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443f1dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 0x556443f1d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cca38b640 1 -- 172.21.15.69:0/1696569744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 msgr2=0x556443f1d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 0x556443f1d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cca38b640 1 -- 172.21.15.69:0/1696569744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 msgr2=0x556443f1dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:02.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443f1dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.988+0000 7f1cca38b640 1 -- 172.21.15.69:0/1696569744 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556443ef2b50 con 0x556443f01970 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.989+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443ef8210 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009b80 tx=0x7f1cbc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894a4b1eb3ecaaaf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.989+0000 7f1caf7fe640 1 -- 172.21.15.69:0/1696569744 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc019040 con 0x556443f01970 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.989+0000 7f1caf7fe640 1 -- 172.21.15.69:0/1696569744 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc00fe70 con 0x556443f01970 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.989+0000 7f1caf7fe640 1 -- 172.21.15.69:0/1696569744 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc01fd90 con 0x556443f01970 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.989+0000 7f1ccec23740 1 -- 172.21.15.69:0/1696569744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 msgr2=0x556443ef8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.989+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443ef8210 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009b80 tx=0x7f1cbc009500 comp rx=0 tx=0).stop 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 -- 172.21.15.69:0/1696569744 shutdown_connections 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443ef8210 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef70d0 0x556443f1dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1696569744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef67b0 0x556443f1d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 -- 172.21.15.69:0/1696569744 >> 172.21.15.69:0/1696569744 conn(0x556443ef2d20 msgr2=0x556443efd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 -- 172.21.15.69:0/1696569744 shutdown_connections 2024-08-02T17:49:02.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 -- 172.21.15.69:0/1696569744 wait complete. 2024-08-02T17:49:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 -- start start 2024-08-02T17:49:02.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 Processor -- start 2024-08-02T17:49:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef67b0 0x556443f0e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443f099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ab100 con 0x556443f01970 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ac570 con 0x556443ef67b0 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41ab7d0 con 0x556443ef70d0 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.990+0000 7f1ccec23740 10 client.0 _ll_register_callbacks cb 0x7f1cb40b7e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443f099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cc9389640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cc9b8a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef67b0 0x556443f0e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443f099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59694/0 (socket says 172.21.15.69:59694) 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cc9389640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:58040/0 (socket says 172.21.15.69:58040) 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 -- 172.21.15.69:0/1615396873 learned_addr learned my addr 172.21.15.69:0/1615396873 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 -- 172.21.15.69:0/1615396873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 msgr2=0x556443f094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f094b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 -- 172.21.15.69:0/1615396873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef67b0 msgr2=0x556443f0e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef67b0 0x556443f0e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 -- 172.21.15.69:0/1615396873 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556443f09f30 con 0x556443f01970 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef67b0 0x556443f0e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cca38b640 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443f099f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc02c040 tx=0x7f1cbc010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee1412331726017 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1615396873 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc038020 con 0x556443f01970 2024-08-02T17:49:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1615396873 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f1cbc01fa10 con 0x556443f01970 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1ccec23740 1 -- 172.21.15.69:0/1615396873 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cbc00fe70 con 0x556443f01970 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1615396873 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc043050 con 0x556443f01970 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1ccec23740 10 client.16086 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.991+0000 7f1ccec23740 1 -- 172.21.15.69:0/1615396873 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556443f80440 con 0x556443f01970 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1615396873 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cbc00fe70 con 0x556443f01970 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1615396873 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1cbc0345f0 con 0x556443f01970 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1ccec23740 10 client.16086 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1ccec23740 10 client.16086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1ccec23740 20 client.16086 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556443f80b80 0x556443f82fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1ccec23740 4 client.16086 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1ccec23740 1 -- 172.21.15.69:0/1615396873 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556443e62430 con 0x556443f80b80 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.992+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556443f80b80 0x556443f82fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.993+0000 7f1cad7fa640 10 client.16086 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:02.993+0000 7f1cc9b8a640 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556443f80b80 0x556443f82fa0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d3a5edd31a2b98 server_cookie=e6dce76e464beebd in_seq=0 out_seq=0 2024-08-02T17:49:03.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.017+0000 7f1cad7fa640 1 -- 172.21.15.69:0/1615396873 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556443e62430 con 0x556443f80b80 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.017+0000 7f1cad7fa640 10 client.16086 handle_command_reply: tid=0 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.017+0000 7f1ccec23740 1 client.16086 shutdown 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 -- 172.21.15.69:0/1615396873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 msgr2=0x556443f099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443f099f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc02c040 tx=0x7f1cbc010890 comp rx=0 tx=0).stop 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 -- 172.21.15.69:0/1615396873 shutdown_connections 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556443f01970 0x556443f099f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556443f80b80 0x556443f82fa0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556443ef70d0 0x556443f094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 --2- 172.21.15.69:0/1615396873 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556443ef67b0 0x556443f0e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1cad7fa640 0 client.16086 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 -- 172.21.15.69:0/1615396873 >> 172.21.15.69:0/1615396873 conn(0x556443ef2d20 msgr2=0x556443ef8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 -- 172.21.15.69:0/1615396873 shutdown_connections 2024-08-02T17:49:03.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 1 -- 172.21.15.69:0/1615396873 wait complete. 2024-08-02T17:49:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.018+0000 7f1ccec23740 20 client.16086 trim_cache size 0 max 16384 2024-08-02T17:49:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:49:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:49:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:49:03.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 -- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1cb00a2cf0 msgr2=0x7f1cb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1cb00a2cf0 0x7f1cb00a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009fd0 tx=0x7f1cbc009b10 comp rx=0 tx=0).stop 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 -- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 msgr2=0x7f1cc41986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 0x7f1cc41986c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4006010 tx=0x7f1cb40042f0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 -- 172.21.15.69:0/1948575805 shutdown_connections 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1cc41032d0 0x7f1cc41986c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1cb00a2cf0 0x7f1cb00a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cc410bdd0 0x7f1cc419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 --2- 172.21.15.69:0/1948575805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1cc41029b0 0x7f1cc4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 -- 172.21.15.69:0/1948575805 >> 172.21.15.69:0/1948575805 conn(0x7f1cc40776d0 msgr2=0x7f1cc40fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 -- 172.21.15.69:0/1948575805 shutdown_connections 2024-08-02T17:49:03.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:03.019+0000 7f1ccbdee640 1 -- 172.21.15.69:0/1948575805 wait complete. 2024-08-02T17:49:03.023 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:02 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2311038969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:03.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:02 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2311038969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:03.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:02 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2311038969' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:03.807 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:49:04.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:03 smithi083 ceph-mon[37031]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 840 op/s 2024-08-02T17:49:04.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:03 smithi077 ceph-mon[37108]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 840 op/s 2024-08-02T17:49:04.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:03 smithi069 ceph-mon[32746]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 11 MiB/s wr, 840 op/s 2024-08-02T17:49:04.114 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 -- 172.21.15.69:0/2092623484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06941022a0 msgr2=0x7f06941026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 --2- 172.21.15.69:0/2092623484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06941022a0 0x7f06941026e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0684009900 tx=0x7f068402a480 comp rx=0 tx=0).stop 2024-08-02T17:49:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 -- 172.21.15.69:0/2092623484 shutdown_connections 2024-08-02T17:49:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 --2- 172.21.15.69:0/2092623484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06941022a0 0x7f06941026e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 --2- 172.21.15.69:0/2092623484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0694102c20 0x7f069410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 --2- 172.21.15.69:0/2092623484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0694100b50 0x7f0694100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 -- 172.21.15.69:0/2092623484 >> 172.21.15.69:0/2092623484 conn(0x7f06940fc500 msgr2=0x7f06940fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 -- 172.21.15.69:0/2092623484 shutdown_connections 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 -- 172.21.15.69:0/2092623484 wait complete. 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 Processor -- start 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.623+0000 7f069b27a640 1 -- start start 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f069b27a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0694100b50 0x7f06941982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f069b27a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06941022a0 0x7f06941987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f069b27a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0694102c20 0x7f069419cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f0699016640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0694100b50 0x7f06941982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f0698815640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06941022a0 0x7f06941987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f0699016640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0694100b50 0x7f06941982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:58058/0 (socket says 172.21.15.69:58058) 2024-08-02T17:49:04.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f0699016640 1 -- 172.21.15.69:0/3422867734 learned_addr learned my addr 172.21.15.69:0/3422867734 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f0699817640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0694102c20 0x7f069419cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0694110c70 con 0x7f06941022a0 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0694110af0 con 0x7f0694102c20 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.624+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0694110df0 con 0x7f0694100b50 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f0699817640 1 -- 172.21.15.69:0/3422867734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0694100b50 msgr2=0x7f06941982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f0699817640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0694100b50 0x7f06941982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f0699817640 1 -- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06941022a0 msgr2=0x7f06941987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f0699817640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06941022a0 0x7f06941987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f0699817640 1 -- 172.21.15.69:0/3422867734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069419d250 con 0x7f0694102c20 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f0699817640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0694102c20 0x7f069419cb70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f068c009fd0 tx=0x7f068c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f068a7fc640 1 -- 172.21.15.69:0/3422867734 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c00ce20 con 0x7f0694102c20 2024-08-02T17:49:04.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f068a7fc640 1 -- 172.21.15.69:0/3422867734 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c005020 con 0x7f0694102c20 2024-08-02T17:49:04.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f068a7fc640 1 -- 172.21.15.69:0/3422867734 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c010720 con 0x7f0694102c20 2024-08-02T17:49:04.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.625+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069419d540 con 0x7f0694102c20 2024-08-02T17:49:04.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.626+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069419d9a0 con 0x7f0694102c20 2024-08-02T17:49:04.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.628+0000 7f068a7fc640 1 -- 172.21.15.69:0/3422867734 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f068c0051d0 con 0x7f0694102c20 2024-08-02T17:49:04.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.629+0000 7f069d678740 1 Processor -- start 2024-08-02T17:49:04.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.629+0000 7f068a7fc640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06800a2d70 0x7f06800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.629+0000 7f068a7fc640 1 -- 172.21.15.69:0/3422867734 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f068c0cfd10 con 0x7f0694102c20 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.629+0000 7f069d678740 1 -- start start 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f069d678740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949ae23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f069d678740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949aecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f069d678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949aef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f069d678740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e949a47a40 con 0x55e949ae1ff0 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f069d678740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9499df9d0 con 0x55e949ae2910 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f069d678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e949a43a00 con 0x55e949aed1d0 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699016640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949ae23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699016640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949ae23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59718/0 (socket says 172.21.15.69:59718) 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699016640 1 -- 172.21.15.69:0/1362991367 learned_addr learned my addr 172.21.15.69:0/1362991367 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0698815640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06800a2d70 0x7f06800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699817640 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949aef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0698815640 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949aecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699817640 1 -- 172.21.15.69:0/1362991367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 msgr2=0x55e949aecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699817640 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949aecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699817640 1 -- 172.21.15.69:0/1362991367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 msgr2=0x55e949ae23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699817640 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949ae23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699817640 1 -- 172.21.15.69:0/1362991367 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9499e6650 con 0x55e949aed1d0 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.630+0000 7f0699817640 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949aef5c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f068c0cd040 tx=0x7f068c09b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82444f171c39f2d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.631+0000 7f067f7fe640 1 -- 172.21.15.69:0/1362991367 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c0a4a80 con 0x55e949aed1d0 2024-08-02T17:49:04.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.631+0000 7f067f7fe640 1 -- 172.21.15.69:0/1362991367 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c005520 con 0x55e949aed1d0 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.631+0000 7f067f7fe640 1 -- 172.21.15.69:0/1362991367 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c0a4850 con 0x55e949aed1d0 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.631+0000 7f0698815640 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949aecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 -- 172.21.15.69:0/1362991367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 msgr2=0x55e949aef5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949aef5c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f068c0cd040 tx=0x7f068c09b830 comp rx=0 tx=0).stop 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 -- 172.21.15.69:0/1362991367 shutdown_connections 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949aef5c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949aecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 --2- 172.21.15.69:0/1362991367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949ae23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 -- 172.21.15.69:0/1362991367 >> 172.21.15.69:0/1362991367 conn(0x55e949ade580 msgr2=0x55e949ade960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f0698815640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06800a2d70 0x7f06800a5230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0684009230 tx=0x7f0684009560 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 -- 172.21.15.69:0/1362991367 shutdown_connections 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 -- 172.21.15.69:0/1362991367 wait complete. 2024-08-02T17:49:04.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.632+0000 7f069d678740 1 -- start start 2024-08-02T17:49:04.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f069d678740 1 Processor -- start 2024-08-02T17:49:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f069d678740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949aea040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f069d678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 0x55e949aea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f069d678740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f069d678740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e949aefda0 con 0x55e949aed1d0 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f069d678740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e949a02b20 con 0x55e949ae1ff0 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f069d678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e949aeff20 con 0x55e949ae2910 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f069d678740 10 client.0 _ll_register_callbacks cb 0x55e949a00bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f0699016640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949aea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f0699016640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949aea040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:56582/0 (socket says 172.21.15.69:56582) 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f0699016640 1 -- 172.21.15.69:0/1073469387 learned_addr learned my addr 172.21.15.69:0/1073469387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f0698815640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 0x55e949aea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.633+0000 7f0699817640 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f0699016640 1 -- 172.21.15.69:0/1073469387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 msgr2=0x55e949aea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f0699016640 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 0x55e949aea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f0699016640 1 -- 172.21.15.69:0/1073469387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 msgr2=0x55e949aeaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f0699016640 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f0699016640 1 -- 172.21.15.69:0/1073469387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9497b93c0 con 0x55e949ae1ff0 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f0699817640 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f0699016640 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949aea040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f06900126d0 tx=0x7f0690012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fee13e5cf01375c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f067cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f067cff9640 1 -- 172.21.15.69:0/1073469387 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690011bb0 con 0x55e949ae1ff0 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f069d678740 1 -- 172.21.15.69:0/1073469387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f068c005520 con 0x55e949ae1ff0 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f067cff9640 1 -- 172.21.15.69:0/1073469387 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0690011d50 con 0x55e949ae1ff0 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f067cff9640 1 -- 172.21.15.69:0/1073469387 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f069001c420 con 0x55e949ae1ff0 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f069d678740 10 client.25730 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:04.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f069d678740 1 -- 172.21.15.69:0/1073469387 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e949afb900 con 0x55e949ae1ff0 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f067cff9640 1 -- 172.21.15.69:0/1073469387 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c005520 con 0x55e949ae1ff0 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f067cff9640 1 -- 172.21.15.69:0/1073469387 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f069000f700 con 0x55e949ae1ff0 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f069d678740 10 client.25730 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f069d678740 10 client.25730 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.634+0000 7f069d678740 20 client.25730 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.635+0000 7f069d678740 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e949afcdc0 0x55e949aff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.635+0000 7f069d678740 4 client.25730 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.635+0000 7f069d678740 1 -- 172.21.15.69:0/1073469387 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e949a54fb0 con 0x55e949afcdc0 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.635+0000 7f0698815640 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e949afcdc0 0x55e949aff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.635+0000 7f0698815640 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e949afcdc0 0x55e949aff240 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2aa978cd0e06ec0 server_cookie=af966dd63bb30328 in_seq=0 out_seq=0 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.635+0000 7f067cff9640 10 client.25730 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f067cff9640 1 -- 172.21.15.69:0/1073469387 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e949a54fb0 con 0x55e949afcdc0 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f067cff9640 10 client.25730 handle_command_reply: tid=0 2024-08-02T17:49:04.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 client.25730 shutdown 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 -- 172.21.15.69:0/1073469387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 msgr2=0x55e949aea040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949aea040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f06900126d0 tx=0x7f0690012b90 comp rx=0 tx=0).stop 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 -- 172.21.15.69:0/1073469387 shutdown_connections 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e949afcdc0 0x55e949aff240 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f067cff9640 0 client.25730 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 0x55e949aea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 --2- 172.21.15.69:0/1073469387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949aea040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 -- 172.21.15.69:0/1073469387 >> 172.21.15.69:0/1073469387 conn(0x55e949ade580 msgr2=0x55e949ae4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 -- 172.21.15.69:0/1073469387 shutdown_connections 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.636+0000 7f069d678740 1 -- 172.21.15.69:0/1073469387 wait complete. 2024-08-02T17:49:04.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.637+0000 7f069d678740 20 client.25730 trim_cache size 0 max 16384 2024-08-02T17:49:04.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f069d678740 1 Processor -- start 2024-08-02T17:49:04.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f069d678740 1 -- start start 2024-08-02T17:49:04.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f069d678740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949b04b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f069d678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 0x55e949b050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f069d678740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f069d678740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06941af520 con 0x55e949aed1d0 2024-08-02T17:49:04.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f069d678740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06941acdf0 con 0x55e949ae1ff0 2024-08-02T17:49:04.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f069d678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06941aac40 con 0x55e949ae2910 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f0699817640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f0699817640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59724/0 (socket says 172.21.15.69:59724) 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f0699817640 1 -- 172.21.15.69:0/2350825811 learned_addr learned my addr 172.21.15.69:0/2350825811 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f0699016640 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949b04b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.652+0000 7f0698815640 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 0x55e949b050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f0699817640 1 -- 172.21.15.69:0/2350825811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 msgr2=0x55e949b050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f0699817640 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 0x55e949b050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f0699817640 1 -- 172.21.15.69:0/2350825811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 msgr2=0x55e949b04b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:04.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f0699817640 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949b04b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f0699817640 1 -- 172.21.15.69:0/2350825811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e949ae8590 con 0x55e949aed1d0 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f0699016640 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949b04b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f0699817640 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeb790 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f068c098270 tx=0x7f068c0981e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda86cc0025985d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f067effd640 1 -- 172.21.15.69:0/2350825811 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c07e070 con 0x55e949aed1d0 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f067effd640 1 -- 172.21.15.69:0/2350825811 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c005520 con 0x55e949aed1d0 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.653+0000 7f067effd640 1 -- 172.21.15.69:0/2350825811 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c09b260 con 0x55e949aed1d0 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 -- 172.21.15.69:0/2350825811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 msgr2=0x55e949aeb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeb790 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f068c098270 tx=0x7f068c0981e0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 -- 172.21.15.69:0/2350825811 shutdown_connections 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949aed1d0 0x55e949aeb790 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949ae2910 0x55e949b050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 --2- 172.21.15.69:0/2350825811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae1ff0 0x55e949b04b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 -- 172.21.15.69:0/2350825811 >> 172.21.15.69:0/2350825811 conn(0x55e949ade580 msgr2=0x55e949ae8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:04.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 -- 172.21.15.69:0/2350825811 shutdown_connections 2024-08-02T17:49:04.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 -- 172.21.15.69:0/2350825811 wait complete. 2024-08-02T17:49:04.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.654+0000 7f069d678740 1 -- start start 2024-08-02T17:49:04.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f069d678740 1 Processor -- start 2024-08-02T17:49:04.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f069d678740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949af5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f069d678740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949af5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f069d678740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949af0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f069d678740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06941ab2b0 con 0x55e949ae1ff0 2024-08-02T17:49:04.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f069d678740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06941ac720 con 0x55e949ae2910 2024-08-02T17:49:04.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f069d678740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06941ab980 con 0x55e949aed1d0 2024-08-02T17:49:04.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f069d678740 10 client.0 _ll_register_callbacks cb 0x7f0690009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f0698815640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949af5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f0698815640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949af5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:56610/0 (socket says 172.21.15.69:56610) 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f0698815640 1 -- 172.21.15.69:0/2315501321 learned_addr learned my addr 172.21.15.69:0/2315501321 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f0699817640 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949af0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.655+0000 7f0699016640 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949af5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f0698815640 1 -- 172.21.15.69:0/2315501321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 msgr2=0x55e949af0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f0698815640 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949af0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f0698815640 1 -- 172.21.15.69:0/2315501321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 msgr2=0x55e949af5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f0698815640 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949af5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f0698815640 1 -- 172.21.15.69:0/2315501321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e949af12b0 con 0x55e949ae2910 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f0699016640 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949af5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:04.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f0698815640 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949af5a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0684000c00 tx=0x7f0684009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec2003aca350281 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f067cff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f067cff9640 1 -- 172.21.15.69:0/2315501321 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0684035030 con 0x55e949ae2910 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f067cff9640 1 -- 172.21.15.69:0/2315501321 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0684031950 con 0x55e949ae2910 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f069d678740 1 -- 172.21.15.69:0/2315501321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f068c005520 con 0x55e949ae2910 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f067cff9640 1 -- 172.21.15.69:0/2315501321 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0684064050 con 0x55e949ae2910 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f069d678740 10 client.25736 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.656+0000 7f069d678740 1 -- 172.21.15.69:0/2315501321 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e949b67840 con 0x55e949ae2910 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f067cff9640 1 -- 172.21.15.69:0/2315501321 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f068c005520 con 0x55e949ae2910 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f067cff9640 1 -- 172.21.15.69:0/2315501321 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0684053070 con 0x55e949ae2910 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f069d678740 10 client.25736 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f069d678740 10 client.25736 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f069d678740 20 client.25736 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f069d678740 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e949afcdc0 0x55e949b6a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:04.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f069d678740 4 client.25736 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f069d678740 1 -- 172.21.15.69:0/2315501321 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e949a4f4c0 con 0x55e949afcdc0 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.657+0000 7f0699016640 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e949afcdc0 0x55e949b6a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f0699016640 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e949afcdc0 0x55e949b6a030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=568b65e5d97185db server_cookie=f3d45e5427ac0c86 in_seq=0 out_seq=0 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f067cff9640 10 client.25736 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f067cff9640 1 -- 172.21.15.69:0/2315501321 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e949a4f4c0 con 0x55e949afcdc0 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f067cff9640 10 client.25736 handle_command_reply: tid=0 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 client.25736 shutdown 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 -- 172.21.15.69:0/2315501321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 msgr2=0x55e949af5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949af5a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0684000c00 tx=0x7f0684009260 comp rx=0 tx=0).stop 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 -- 172.21.15.69:0/2315501321 shutdown_connections 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e949afcdc0 0x55e949b6a030 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e949aed1d0 0x55e949af0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e949ae2910 0x55e949af5a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f067cff9640 0 client.25736 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:49:04.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 --2- 172.21.15.69:0/2315501321 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e949ae1ff0 0x55e949af5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 -- 172.21.15.69:0/2315501321 >> 172.21.15.69:0/2315501321 conn(0x55e949ade580 msgr2=0x55e949adcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:04.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 -- 172.21.15.69:0/2315501321 shutdown_connections 2024-08-02T17:49:04.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 1 -- 172.21.15.69:0/2315501321 wait complete. 2024-08-02T17:49:04.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.658+0000 7f069d678740 20 client.25736 trim_cache size 0 max 16384 2024-08-02T17:49:04.665 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:49:04.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06800a2d70 msgr2=0x7f06800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06800a2d70 0x7f06800a5230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0684009230 tx=0x7f0684009560 comp rx=0 tx=0).stop 2024-08-02T17:49:04.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0694102c20 msgr2=0x7f069419cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:04.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0694102c20 0x7f069419cb70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f068c009fd0 tx=0x7f068c00ef10 comp rx=0 tx=0).stop 2024-08-02T17:49:04.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 shutdown_connections 2024-08-02T17:49:04.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06941022a0 0x7f06941987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06800a2d70 0x7f06800a5230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0694102c20 0x7f069419cb70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 --2- 172.21.15.69:0/3422867734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0694100b50 0x7f06941982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:04.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 >> 172.21.15.69:0/3422867734 conn(0x7f06940fc500 msgr2=0x7f0694069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:04.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 shutdown_connections 2024-08-02T17:49:04.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:04.659+0000 7f069b27a640 1 -- 172.21.15.69:0/3422867734 wait complete. 2024-08-02T17:49:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 115.0 s, 336.2 tps, lat 280.022 ms stddev 312.844 2024-08-02T17:49:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:05 smithi077 ceph-mon[37108]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.5 MiB/s wr, 666 op/s 2024-08-02T17:49:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:05 smithi069 ceph-mon[32746]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.5 MiB/s wr, 666 op/s 2024-08-02T17:49:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:05 smithi083 ceph-mon[37031]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 8.5 MiB/s wr, 666 op/s 2024-08-02T17:49:05.580 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:49:05.872 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:07.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:07 smithi083 ceph-mon[37031]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-08-02T17:49:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:07 smithi077 ceph-mon[37108]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-08-02T17:49:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:07 smithi069 ceph-mon[32746]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2024-08-02T17:49:08.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 -- 172.21.15.69:0/2980951921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb720102980 msgr2=0x7fb720102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 --2- 172.21.15.69:0/2980951921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb720102980 0x7fb720102d60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009900 tx=0x7fb71c02a490 comp rx=0 tx=0).stop 2024-08-02T17:49:08.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 -- 172.21.15.69:0/2980951921 shutdown_connections 2024-08-02T17:49:08.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 --2- 172.21.15.69:0/2980951921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb72010bd20 0x7fb72010e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 --2- 172.21.15.69:0/2980951921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7201032a0 0x7fb72010b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 --2- 172.21.15.69:0/2980951921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb720102980 0x7fb720102d60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 -- 172.21.15.69:0/2980951921 >> 172.21.15.69:0/2980951921 conn(0x7fb7200fc340 msgr2=0x7fb7200fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 -- 172.21.15.69:0/2980951921 shutdown_connections 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.557+0000 7fb7267a1640 1 -- 172.21.15.69:0/2980951921 wait complete. 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb7267a1640 1 Processor -- start 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb7267a1640 1 -- start start 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb7267a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb720102980 0x7fb72019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb7267a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7201032a0 0x7fb72019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb7267a1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72010bd20 0x7fb72010a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb7267a1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb720111b40 con 0x7fb72010bd20 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb7267a1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201119c0 con 0x7fb7201032a0 2024-08-02T17:49:08.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb7267a1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb720111cc0 con 0x7fb720102980 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb725fa0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72010bd20 0x7fb72010a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb72579f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb720102980 0x7fb72019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb725fa0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72010bd20 0x7fb72010a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49158/0 (socket says 172.21.15.69:49158) 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.558+0000 7fb725fa0640 1 -- 172.21.15.69:0/866401625 learned_addr learned my addr 172.21.15.69:0/866401625 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb724f9e640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7201032a0 0x7fb72019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb72579f640 1 -- 172.21.15.69:0/866401625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7201032a0 msgr2=0x7fb72019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb72579f640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7201032a0 0x7fb72019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb72579f640 1 -- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72010bd20 msgr2=0x7fb72010a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb72579f640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72010bd20 0x7fb72010a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb72579f640 1 -- 172.21.15.69:0/866401625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72010b1f0 con 0x7fb720102980 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb72579f640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb720102980 0x7fb72019ec00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb71c02a9b0 tx=0x7fb71c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb7167fc640 1 -- 172.21.15.69:0/866401625 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c03f070 con 0x7fb720102980 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb7167fc640 1 -- 172.21.15.69:0/866401625 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb71c031c90 con 0x7fb720102980 2024-08-02T17:49:08.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb7167fc640 1 -- 172.21.15.69:0/866401625 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c035030 con 0x7fb720102980 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb7267a1640 1 -- 172.21.15.69:0/866401625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72010b480 con 0x7fb720102980 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.559+0000 7fb7267a1640 1 -- 172.21.15.69:0/866401625 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72019a650 con 0x7fb720102980 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.562+0000 7fb7167fc640 1 -- 172.21.15.69:0/866401625 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb71c042070 con 0x7fb720102980 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.563+0000 7fb729e01740 1 Processor -- start 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.563+0000 7fb729e01740 1 -- start start 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb729e01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9e1c0 0x560f9ee9e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb729e01740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eea8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb729e01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9eea93a0 0x560f9eeab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb729e01740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f9ee03a40 con 0x560f9ee9eae0 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb729e01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560f9ed9b9d0 con 0x560f9eea93a0 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb729e01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f9edffa00 con 0x560f9ee9e1c0 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb7167fc640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb70c0a2df0 0x7fb70c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb724f9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9eea93a0 0x560f9eeab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9e1c0 0x560f9ee9e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9e1c0 0x560f9ee9e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54234/0 (socket says 172.21.15.69:54234) 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 -- 172.21.15.69:0/3847715696 learned_addr learned my addr 172.21.15.69:0/3847715696 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb7167fc640 1 -- 172.21.15.69:0/866401625 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb71c0eff00 con 0x7fb720102980 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb72579f640 1 --2- 172.21.15.69:0/3847715696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eea8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb70c0a2df0 0x7fb70c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 -- 172.21.15.69:0/3847715696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9eea93a0 msgr2=0x560f9eeab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 --2- 172.21.15.69:0/3847715696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9eea93a0 0x560f9eeab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 -- 172.21.15.69:0/3847715696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 msgr2=0x560f9eea8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 --2- 172.21.15.69:0/3847715696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eea8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.564+0000 7fb725fa0640 1 -- 172.21.15.69:0/3847715696 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9eda2650 con 0x560f9ee9e1c0 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.565+0000 7fb725fa0640 1 --2- 172.21.15.69:0/3847715696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9e1c0 0x560f9ee9e5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb718013bb0 tx=0x7fb7180117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eed3017b02c3e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:08.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.565+0000 7fb70b7fe640 1 -- 172.21.15.69:0/3847715696 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71801e810 con 0x560f9ee9e1c0 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.565+0000 7fb70b7fe640 1 -- 172.21.15.69:0/3847715696 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb71800e3e0 con 0x560f9ee9e1c0 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.565+0000 7fb70b7fe640 1 -- 172.21.15.69:0/3847715696 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71800f5e0 con 0x560f9ee9e1c0 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.565+0000 7fb725fa0640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb70c0a2df0 0x7fb70c0a52b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb718023000 tx=0x7fb718020030 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 -- 172.21.15.69:0/3847715696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9e1c0 msgr2=0x560f9ee9e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 --2- 172.21.15.69:0/3847715696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9e1c0 0x560f9ee9e5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb718013bb0 tx=0x7fb7180117b0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 -- 172.21.15.69:0/3847715696 shutdown_connections 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 --2- 172.21.15.69:0/3847715696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eea8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 --2- 172.21.15.69:0/3847715696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9eea93a0 0x560f9eeab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 --2- 172.21.15.69:0/3847715696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9e1c0 0x560f9ee9e5a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 -- 172.21.15.69:0/3847715696 >> 172.21.15.69:0/3847715696 conn(0x560f9ee9a750 msgr2=0x560f9ee9ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 -- 172.21.15.69:0/3847715696 shutdown_connections 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.566+0000 7fb729e01740 1 -- 172.21.15.69:0/3847715696 wait complete. 2024-08-02T17:49:08.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 1 -- start start 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 1 Processor -- start 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9e1c0 0x560f9eea62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9eae0 0x560f9eea6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 0x560f9eea6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f9eeabf70 con 0x560f9ee9e1c0 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560f9edbeb20 con 0x560f9ee9eae0 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f9eeac0f0 con 0x560f9eea93a0 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb729e01740 10 client.0 _ll_register_callbacks cb 0x560f9edbcbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb72579f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9e1c0 0x560f9eea62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb72579f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9e1c0 0x560f9eea62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49172/0 (socket says 172.21.15.69:49172) 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.567+0000 7fb72579f640 1 -- 172.21.15.69:0/2509431794 learned_addr learned my addr 172.21.15.69:0/2509431794 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb725fa0640 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 0x560f9eea6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb724f9e640 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9eae0 0x560f9eea6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb72579f640 1 -- 172.21.15.69:0/2509431794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 msgr2=0x560f9eea6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb72579f640 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 0x560f9eea6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb72579f640 1 -- 172.21.15.69:0/2509431794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9eae0 msgr2=0x560f9eea6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb72579f640 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9eae0 0x560f9eea6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb72579f640 1 -- 172.21.15.69:0/2509431794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9eb753c0 con 0x560f9ee9e1c0 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb72579f640 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9e1c0 0x560f9eea62d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0b8460 tx=0x7fb71c005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dae3b033c9246cc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb708ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb708ff9640 1 -- 172.21.15.69:0/2509431794 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c049040 con 0x560f9ee9e1c0 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb708ff9640 1 -- 172.21.15.69:0/2509431794 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb71c0ed4b0 con 0x560f9ee9e1c0 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb729e01740 1 -- 172.21.15.69:0/2509431794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb71800e3e0 con 0x560f9ee9e1c0 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb708ff9640 1 -- 172.21.15.69:0/2509431794 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb71c0c03b0 con 0x560f9ee9e1c0 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb729e01740 10 client.16116 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.568+0000 7fb729e01740 1 -- 172.21.15.69:0/2509431794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f9eeb7c50 con 0x560f9ee9e1c0 2024-08-02T17:49:08.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb708ff9640 1 -- 172.21.15.69:0/2509431794 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb71800e3e0 con 0x560f9ee9e1c0 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb708ff9640 1 -- 172.21.15.69:0/2509431794 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb71c0d8070 con 0x560f9ee9e1c0 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb729e01740 10 client.16116 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb729e01740 10 client.16116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb729e01740 20 client.16116 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb729e01740 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560f9eeb90e0 0x560f9eebb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb729e01740 4 client.16116 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb729e01740 1 -- 172.21.15.69:0/2509431794 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f9ee10fb0 con 0x560f9eeb90e0 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb724f9e640 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560f9eeb90e0 0x560f9eebb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.569+0000 7fb708ff9640 10 client.16116 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb724f9e640 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560f9eeb90e0 0x560f9eebb570 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70fc89aa6373c6ed server_cookie=93878c3a81f2db48 in_seq=0 out_seq=0 2024-08-02T17:49:08.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb708ff9640 1 -- 172.21.15.69:0/2509431794 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560f9ee10fb0 con 0x560f9eeb90e0 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb708ff9640 10 client.16116 handle_command_reply: tid=0 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 client.16116 shutdown 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 -- 172.21.15.69:0/2509431794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9e1c0 msgr2=0x560f9eea62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9e1c0 0x560f9eea62d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0b8460 tx=0x7fb71c005e50 comp rx=0 tx=0).stop 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 -- 172.21.15.69:0/2509431794 shutdown_connections 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560f9eeb90e0 0x560f9eebb570 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb708ff9640 0 client.16116 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 0x560f9eea6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9eae0 0x560f9eea6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 --2- 172.21.15.69:0/2509431794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9e1c0 0x560f9eea62d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.570+0000 7fb729e01740 1 -- 172.21.15.69:0/2509431794 >> 172.21.15.69:0/2509431794 conn(0x560f9ee9a750 msgr2=0x560f9ee99910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.571+0000 7fb729e01740 1 -- 172.21.15.69:0/2509431794 shutdown_connections 2024-08-02T17:49:08.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.571+0000 7fb729e01740 1 -- 172.21.15.69:0/2509431794 wait complete. 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.571+0000 7fb729e01740 20 client.16116 trim_cache size 0 max 16384 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb729e01740 1 Processor -- start 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb729e01740 1 -- start start 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb729e01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 0x560f9eea0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb729e01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9eae0 0x560f9eec0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb729e01740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9eea93a0 0x560f9eec13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb729e01740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201c0260 con 0x560f9eea93a0 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb729e01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201bdb30 con 0x560f9ee9e1c0 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb729e01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201bb980 con 0x560f9ee9eae0 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb725fa0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9eea93a0 0x560f9eec13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb725fa0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9eea93a0 0x560f9eec13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49178/0 (socket says 172.21.15.69:49178) 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb725fa0640 1 -- 172.21.15.69:0/305254278 learned_addr learned my addr 172.21.15.69:0/305254278 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:08.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb724f9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9eae0 0x560f9eec0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb72579f640 1 --2- 172.21.15.69:0/305254278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 0x560f9eea0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb725fa0640 1 -- 172.21.15.69:0/305254278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9eae0 msgr2=0x560f9eec0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb725fa0640 1 --2- 172.21.15.69:0/305254278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9eae0 0x560f9eec0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb725fa0640 1 -- 172.21.15.69:0/305254278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 msgr2=0x560f9eea0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb725fa0640 1 --2- 172.21.15.69:0/305254278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 0x560f9eea0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.584+0000 7fb725fa0640 1 -- 172.21.15.69:0/305254278 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f9ee9e910 con 0x560f9eea93a0 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.585+0000 7fb725fa0640 1 --2- 172.21.15.69:0/305254278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9eea93a0 0x560f9eec13c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb71800ed90 tx=0x7fb718015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8712c3cf25a1dcee server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.585+0000 7fb70affd640 1 -- 172.21.15.69:0/305254278 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb718036020 con 0x560f9eea93a0 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.585+0000 7fb70affd640 1 -- 172.21.15.69:0/305254278 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb71802e040 con 0x560f9eea93a0 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.585+0000 7fb70affd640 1 -- 172.21.15.69:0/305254278 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71802d050 con 0x560f9eea93a0 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 -- 172.21.15.69:0/305254278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9eea93a0 msgr2=0x560f9eec13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 --2- 172.21.15.69:0/305254278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9eea93a0 0x560f9eec13c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb71800ed90 tx=0x7fb718015000 comp rx=0 tx=0).stop 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 -- 172.21.15.69:0/305254278 shutdown_connections 2024-08-02T17:49:08.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 --2- 172.21.15.69:0/305254278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9eea93a0 0x560f9eec13c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 --2- 172.21.15.69:0/305254278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9ee9eae0 0x560f9eec0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 --2- 172.21.15.69:0/305254278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 0x560f9eea0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 -- 172.21.15.69:0/305254278 >> 172.21.15.69:0/305254278 conn(0x560f9ee9a750 msgr2=0x560f9ee9fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 -- 172.21.15.69:0/305254278 shutdown_connections 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 -- 172.21.15.69:0/305254278 wait complete. 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 -- start start 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.586+0000 7fb729e01740 1 Processor -- start 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb729e01740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 0x560f9eeacfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb729e01740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eead510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb729e01740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 0x560f9eeada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb729e01740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201bbff0 con 0x560f9ee9eae0 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb729e01740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201bd460 con 0x560f9ee9e1c0 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb729e01740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201bc6c0 con 0x560f9eea93a0 2024-08-02T17:49:08.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb724f9e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eead510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb729e01740 10 client.0 _ll_register_callbacks cb 0x7fb71c031e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb724f9e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eead510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:49190/0 (socket says 172.21.15.69:49190) 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb724f9e640 1 -- 172.21.15.69:0/3161809370 learned_addr learned my addr 172.21.15.69:0/3161809370 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb72579f640 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 0x560f9eeacfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb725fa0640 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 0x560f9eeada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb724f9e640 1 -- 172.21.15.69:0/3161809370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 msgr2=0x560f9eeada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb724f9e640 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 0x560f9eeada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb724f9e640 1 -- 172.21.15.69:0/3161809370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 msgr2=0x560f9eeacfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb724f9e640 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 0x560f9eeacfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.587+0000 7fb724f9e640 1 -- 172.21.15.69:0/3161809370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f9ef22e50 con 0x560f9ee9eae0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb724f9e640 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eead510 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb720197380 tx=0x7fb71001c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495620eac5bed51b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb708ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb708ff9640 1 -- 172.21.15.69:0/3161809370 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7100119d0 con 0x560f9ee9eae0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb708ff9640 1 -- 172.21.15.69:0/3161809370 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb710011b70 con 0x560f9ee9eae0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb729e01740 1 -- 172.21.15.69:0/3161809370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb71802e040 con 0x560f9ee9eae0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb708ff9640 1 -- 172.21.15.69:0/3161809370 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb710011d70 con 0x560f9ee9eae0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb729e01740 10 client.16122 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb729e01740 1 -- 172.21.15.69:0/3161809370 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f9ef24100 con 0x560f9ee9eae0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb708ff9640 1 -- 172.21.15.69:0/3161809370 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb71802e040 con 0x560f9ee9eae0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.588+0000 7fb708ff9640 1 -- 172.21.15.69:0/3161809370 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb710010460 con 0x560f9ee9eae0 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb729e01740 10 client.16122 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb729e01740 10 client.16122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:49:08.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb729e01740 20 client.16122 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb729e01740 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560f9eeb90e0 0x560f9ef268f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb729e01740 4 client.16122 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb729e01740 1 -- 172.21.15.69:0/3161809370 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f9ee0b4c0 con 0x560f9eeb90e0 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb72579f640 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560f9eeb90e0 0x560f9ef268f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb72579f640 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560f9eeb90e0 0x560f9ef268f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e315e765d9e4f27 server_cookie=feae4bf4cf46fd6 in_seq=0 out_seq=0 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.589+0000 7fb708ff9640 10 client.16122 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb708ff9640 1 -- 172.21.15.69:0/3161809370 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f9ee0b4c0 con 0x560f9eeb90e0 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb708ff9640 10 client.16122 handle_command_reply: tid=0 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 client.16122 shutdown 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 -- 172.21.15.69:0/3161809370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 msgr2=0x560f9eead510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eead510 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb720197380 tx=0x7fb71001c7a0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 -- 172.21.15.69:0/3161809370 shutdown_connections 2024-08-02T17:49:08.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560f9eeb90e0 0x560f9ef268f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f9ee9eae0 0x560f9eead510 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f9eea93a0 0x560f9eeada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 --2- 172.21.15.69:0/3161809370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f9ee9e1c0 0x560f9eeacfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 -- 172.21.15.69:0/3161809370 >> 172.21.15.69:0/3161809370 conn(0x560f9ee9a750 msgr2=0x560f9eea7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:08.433 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:49:08.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb708ff9640 0 client.16122 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:49:08.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 -- 172.21.15.69:0/3161809370 shutdown_connections 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 1 -- 172.21.15.69:0/3161809370 wait complete. 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.590+0000 7fb729e01740 20 client.16122 trim_cache size 0 max 16384 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 -- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb70c0a2df0 msgr2=0x7fb70c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb70c0a2df0 0x7fb70c0a52b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb718023000 tx=0x7fb718020030 comp rx=0 tx=0).stop 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 -- 172.21.15.69:0/866401625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb720102980 msgr2=0x7fb72019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb720102980 0x7fb72019ec00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb71c02a9b0 tx=0x7fb71c003b90 comp rx=0 tx=0).stop 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 -- 172.21.15.69:0/866401625 shutdown_connections 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb72010bd20 0x7fb72010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb70c0a2df0 0x7fb70c0a52b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb7201032a0 0x7fb72019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 --2- 172.21.15.69:0/866401625 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb720102980 0x7fb72019ec00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 -- 172.21.15.69:0/866401625 >> 172.21.15.69:0/866401625 conn(0x7fb7200fc340 msgr2=0x7fb7200fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:08.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 -- 172.21.15.69:0/866401625 shutdown_connections 2024-08-02T17:49:08.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:07.591+0000 7fb7267a1640 1 -- 172.21.15.69:0/866401625 wait complete. 2024-08-02T17:49:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:09 smithi077 ceph-mon[37108]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 769 op/s 2024-08-02T17:49:09.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:09 smithi069 ceph-mon[32746]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 769 op/s 2024-08-02T17:49:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:09 smithi083 ceph-mon[37031]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 769 op/s 2024-08-02T17:49:09.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 120.0 s, 1272.8 tps, lat 83.156 ms stddev 270.658 2024-08-02T17:49:11.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:11 smithi077 ceph-mon[37108]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-08-02T17:49:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:11 smithi069 ceph-mon[32746]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-08-02T17:49:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:11 smithi083 ceph-mon[37031]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 17 MiB/s wr, 1.04k op/s 2024-08-02T17:49:13.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:13 smithi077 ceph-mon[37108]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 1.51k op/s 2024-08-02T17:49:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:13 smithi069 ceph-mon[32746]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 1.51k op/s 2024-08-02T17:49:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:13 smithi083 ceph-mon[37031]: pgmap v87: 129 pgs: 129 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 30 MiB/s wr, 1.51k op/s 2024-08-02T17:49:14.342 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:49:14.734 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 125.0 s, 2771.2 tps, lat 36.146 ms stddev 45.453 2024-08-02T17:49:15.171 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:15 smithi069 ceph-mon[32746]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2024-08-02T17:49:15.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 -- 172.21.15.69:0/3077384961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9010a0c0 msgr2=0x7efd9010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 --2- 172.21.15.69:0/3077384961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9010a0c0 0x7efd9010a4a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7efd88009ca0 tx=0x7efd8802a950 comp rx=0 tx=0).stop 2024-08-02T17:49:15.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 -- 172.21.15.69:0/3077384961 shutdown_connections 2024-08-02T17:49:15.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 --2- 172.21.15.69:0/3077384961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd901135e0 0x7efd90115a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 --2- 172.21.15.69:0/3077384961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd9010aa70 0x7efd901130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 --2- 172.21.15.69:0/3077384961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd9010a0c0 0x7efd9010a4a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 -- 172.21.15.69:0/3077384961 >> 172.21.15.69:0/3077384961 conn(0x7efd9006de70 msgr2=0x7efd9006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 -- 172.21.15.69:0/3077384961 shutdown_connections 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.298+0000 7efd96c2c640 1 -- 172.21.15.69:0/3077384961 wait complete. 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.299+0000 7efd96c2c640 1 Processor -- start 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.299+0000 7efd96c2c640 1 -- start start 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.299+0000 7efd96c2c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd9010a0c0 0x7efd901a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd96c2c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd9010aa70 0x7efd901a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd96c2c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd901135e0 0x7efd9019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd96c2c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efd901193a0 con 0x7efd901135e0 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd96c2c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd90119220 con 0x7efd9010aa70 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd96c2c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd90119520 con 0x7efd9010a0c0 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd9010a0c0 0x7efd901a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd8ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd9010aa70 0x7efd901a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd9010a0c0 0x7efd901a65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51822/0 (socket says 172.21.15.69:51822) 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 -- 172.21.15.69:0/1423558794 learned_addr learned my addr 172.21.15.69:0/1423558794 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd8ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd9010aa70 0x7efd901a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37818/0 (socket says 172.21.15.69:37818) 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd951c9640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd901135e0 0x7efd9019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 -- 172.21.15.69:0/1423558794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd9010aa70 msgr2=0x7efd901a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd9010aa70 0x7efd901a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 -- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd901135e0 msgr2=0x7efd9019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd901135e0 0x7efd9019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 -- 172.21.15.69:0/1423558794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9019f250 con 0x7efd9010a0c0 2024-08-02T17:49:15.304 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd8ffff640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd9010aa70 0x7efd901a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:15.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.300+0000 7efd949c8640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd9010a0c0 0x7efd901a65f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd8802ae70 tx=0x7efd88030870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:15.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.301+0000 7efd8dffb640 1 -- 172.21.15.69:0/1423558794 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88048070 con 0x7efd9010a0c0 2024-08-02T17:49:15.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.301+0000 7efd8dffb640 1 -- 172.21.15.69:0/1423558794 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd88003e50 con 0x7efd9010a0c0 2024-08-02T17:49:15.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.301+0000 7efd8dffb640 1 -- 172.21.15.69:0/1423558794 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88034020 con 0x7efd9010a0c0 2024-08-02T17:49:15.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.301+0000 7efd96c2c640 1 -- 172.21.15.69:0/1423558794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9019f4e0 con 0x7efd9010a0c0 2024-08-02T17:49:15.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.301+0000 7efd96c2c640 1 -- 172.21.15.69:0/1423558794 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901bc280 con 0x7efd9010a0c0 2024-08-02T17:49:15.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.304+0000 7efd8dffb640 1 -- 172.21.15.69:0/1423558794 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efd8804b070 con 0x7efd9010a0c0 2024-08-02T17:49:15.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.304+0000 7efd9902a740 1 Processor -- start 2024-08-02T17:49:15.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd9902a740 1 -- start start 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd9902a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ede9ff0 0x55707edea3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd9902a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 0x55707edf4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd9902a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd9902a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55707ed4fa40 con 0x55707edf51d0 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd9902a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55707ece79d0 con 0x55707ede9ff0 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd9902a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55707ed4ba00 con 0x55707edea910 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd8ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd951c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ede9ff0 0x55707edea3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd8ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35780/0 (socket says 172.21.15.69:35780) 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd8ffff640 1 -- 172.21.15.69:0/701507478 learned_addr learned my addr 172.21.15.69:0/701507478 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.305+0000 7efd949c8640 1 --2- 172.21.15.69:0/701507478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 0x55707edf4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd8ffff640 1 -- 172.21.15.69:0/701507478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 msgr2=0x55707edf4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd8ffff640 1 --2- 172.21.15.69:0/701507478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 0x55707edf4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd8ffff640 1 -- 172.21.15.69:0/701507478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ede9ff0 msgr2=0x55707edea3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd8ffff640 1 --2- 172.21.15.69:0/701507478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ede9ff0 0x55707edea3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd8ffff640 1 -- 172.21.15.69:0/701507478 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707ecee650 con 0x55707edf51d0 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd8ffff640 1 --2- 172.21.15.69:0/701507478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf75c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efd840114f0 tx=0x7efd840119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc6892b6796a3f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd8dffb640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efd780a2df0 0x7efd780a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd8dffb640 1 -- 172.21.15.69:0/1423558794 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7efd880f0250 con 0x7efd9010a0c0 2024-08-02T17:49:15.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd951c9640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efd780a2df0 0x7efd780a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd76ffd640 1 -- 172.21.15.69:0/701507478 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84005020 con 0x55707edf51d0 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd76ffd640 1 -- 172.21.15.69:0/701507478 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd840047a0 con 0x55707edf51d0 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.306+0000 7efd76ffd640 1 -- 172.21.15.69:0/701507478 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8400f9e0 con 0x55707edf51d0 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd9902a740 1 -- 172.21.15.69:0/701507478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 msgr2=0x55707edf75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd9902a740 1 --2- 172.21.15.69:0/701507478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf75c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efd840114f0 tx=0x7efd840119b0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd951c9640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efd780a2df0 0x7efd780a52b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd80009d20 tx=0x7efd80005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd9902a740 1 -- 172.21.15.69:0/701507478 shutdown_connections 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd9902a740 1 --2- 172.21.15.69:0/701507478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf75c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd9902a740 1 --2- 172.21.15.69:0/701507478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 0x55707edf4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd9902a740 1 --2- 172.21.15.69:0/701507478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ede9ff0 0x55707edea3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd9902a740 1 -- 172.21.15.69:0/701507478 >> 172.21.15.69:0/701507478 conn(0x55707ede6580 msgr2=0x55707ede6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:15.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.307+0000 7efd9902a740 1 -- 172.21.15.69:0/701507478 shutdown_connections 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.308+0000 7efd9902a740 1 -- 172.21.15.69:0/701507478 wait complete. 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.308+0000 7efd9902a740 1 -- start start 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.308+0000 7efd9902a740 1 Processor -- start 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.309+0000 7efd9902a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ede9ff0 0x55707edf2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd9902a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edea910 0x55707edf2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd9902a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edf51d0 0x55707edf2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd9902a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55707edf7da0 con 0x55707edea910 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd9902a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55707ed0ab20 con 0x55707edf51d0 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd9902a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55707edf7f20 con 0x55707ede9ff0 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd949c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ede9ff0 0x55707edf2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd8ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edea910 0x55707edf2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd8ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edea910 0x55707edf2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35796/0 (socket says 172.21.15.69:35796) 2024-08-02T17:49:15.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd8ffff640 1 -- 172.21.15.69:0/2755507015 learned_addr learned my addr 172.21.15.69:0/2755507015 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd949c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ede9ff0 0x55707edf2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51834/0 (socket says 172.21.15.69:51834) 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd951c9640 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edf51d0 0x55707edf2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd9902a740 10 client.0 _ll_register_callbacks cb 0x55707ed08bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd949c8640 1 -- 172.21.15.69:0/2755507015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edf51d0 msgr2=0x55707edf2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd949c8640 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edf51d0 0x55707edf2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd949c8640 1 -- 172.21.15.69:0/2755507015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edea910 msgr2=0x55707edf2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd949c8640 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edea910 0x55707edf2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd949c8640 1 -- 172.21.15.69:0/2755507015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707eac13c0 con 0x55707ede9ff0 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd8ffff640 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edea910 0x55707edf2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd949c8640 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ede9ff0 0x55707edf2000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd880ed580 tx=0x7efd880095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea94cbd8f356ba7c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.310+0000 7efd6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:15.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd6ffff640 1 -- 172.21.15.69:0/2755507015 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880bc2f0 con 0x55707ede9ff0 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 1 -- 172.21.15.69:0/2755507015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd840047a0 con 0x55707ede9ff0 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd6ffff640 1 -- 172.21.15.69:0/2755507015 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7efd88004670 con 0x55707ede9ff0 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd6ffff640 1 -- 172.21.15.69:0/2755507015 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd880bbe80 con 0x55707ede9ff0 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 10 client.25636 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 1 -- 172.21.15.69:0/2755507015 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55707edf7da0 con 0x55707ede9ff0 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd6ffff640 1 -- 172.21.15.69:0/2755507015 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd880bc520 con 0x55707ede9ff0 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd6ffff640 1 -- 172.21.15.69:0/2755507015 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efd88042030 con 0x55707ede9ff0 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 10 client.25636 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 10 client.25636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 20 client.25636 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55707ee04d30 0x55707ee071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 4 client.25636 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:15.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.311+0000 7efd9902a740 1 -- 172.21.15.69:0/2755507015 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efd840047a0 con 0x55707ee04d30 2024-08-02T17:49:15.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.312+0000 7efd8ffff640 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55707ee04d30 0x55707ee071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.313+0000 7efd8ffff640 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55707ee04d30 0x55707ee071b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb79c247ff9fce4e server_cookie=16c44fda099bfc05 in_seq=0 out_seq=0 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.313+0000 7efd6ffff640 10 client.25636 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd6ffff640 1 -- 172.21.15.69:0/2755507015 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7efd840047a0 con 0x55707ee04d30 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd6ffff640 10 client.25636 handle_command_reply: tid=0 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 client.25636 shutdown 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 -- 172.21.15.69:0/2755507015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ede9ff0 msgr2=0x55707edf2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ede9ff0 0x55707edf2000 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd880ed580 tx=0x7efd880095c0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 -- 172.21.15.69:0/2755507015 shutdown_connections 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edea910 0x55707edf2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55707ee04d30 0x55707ee071b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edf51d0 0x55707edf2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 --2- 172.21.15.69:0/2755507015 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ede9ff0 0x55707edf2000 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd880ed580 tx=0x7efd880095c0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd9902a740 1 -- 172.21.15.69:0/2755507015 >> 172.21.15.69:0/2755507015 conn(0x55707ede6580 msgr2=0x55707edec260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:15.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.314+0000 7efd6ffff640 0 client.25636 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:15.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.315+0000 7efd9902a740 1 -- 172.21.15.69:0/2755507015 shutdown_connections 2024-08-02T17:49:15.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.315+0000 7efd9902a740 1 -- 172.21.15.69:0/2755507015 wait complete. 2024-08-02T17:49:15.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.315+0000 7efd9902a740 20 client.25636 trim_cache size 0 max 16384 2024-08-02T17:49:15.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.331+0000 7efd9902a740 1 Processor -- start 2024-08-02T17:49:15.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd9902a740 1 -- start start 2024-08-02T17:49:15.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd9902a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 0x55707edf08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd9902a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd9902a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ee0ca80 0x55707ee0cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd9902a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efd901c8cb0 con 0x55707edf51d0 2024-08-02T17:49:15.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd9902a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd901c6580 con 0x55707ee0ca80 2024-08-02T17:49:15.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd949c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 0x55707edf08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd951c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ee0ca80 0x55707ee0cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd9902a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd901c43d0 con 0x55707edea910 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd951c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ee0ca80 0x55707ee0cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37850/0 (socket says 172.21.15.69:37850) 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd951c9640 1 -- 172.21.15.69:0/2842615592 learned_addr learned my addr 172.21.15.69:0/2842615592 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.332+0000 7efd8ffff640 1 --2- 172.21.15.69:0/2842615592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.333+0000 7efd951c9640 1 -- 172.21.15.69:0/2842615592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 msgr2=0x55707edf08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.333+0000 7efd951c9640 1 --2- 172.21.15.69:0/2842615592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 0x55707edf08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.333+0000 7efd951c9640 1 -- 172.21.15.69:0/2842615592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 msgr2=0x55707edf0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.333+0000 7efd951c9640 1 --2- 172.21.15.69:0/2842615592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.333+0000 7efd951c9640 1 -- 172.21.15.69:0/2842615592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55707ee07830 con 0x55707ee0ca80 2024-08-02T17:49:15.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.333+0000 7efd951c9640 1 --2- 172.21.15.69:0/2842615592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ee0ca80 0x55707ee0cf00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd80004350 tx=0x7efd8001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63fe2f25799b8027 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:15.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.333+0000 7efd767fc640 1 -- 172.21.15.69:0/2842615592 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8002e070 con 0x55707ee0ca80 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd767fc640 1 -- 172.21.15.69:0/2842615592 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd80004380 con 0x55707ee0ca80 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd767fc640 1 -- 172.21.15.69:0/2842615592 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80029820 con 0x55707ee0ca80 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd9902a740 1 -- 172.21.15.69:0/2842615592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ee0ca80 msgr2=0x55707ee0cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd9902a740 1 --2- 172.21.15.69:0/2842615592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ee0ca80 0x55707ee0cf00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efd80004350 tx=0x7efd8001b000 comp rx=0 tx=0).stop 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd9902a740 1 -- 172.21.15.69:0/2842615592 shutdown_connections 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd9902a740 1 --2- 172.21.15.69:0/2842615592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707edf0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd9902a740 1 --2- 172.21.15.69:0/2842615592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707ee0ca80 0x55707ee0cf00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd9902a740 1 --2- 172.21.15.69:0/2842615592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707edea910 0x55707edf08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.334+0000 7efd9902a740 1 -- 172.21.15.69:0/2842615592 >> 172.21.15.69:0/2842615592 conn(0x55707ede6580 msgr2=0x55707ede11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.335+0000 7efd9902a740 1 -- 172.21.15.69:0/2842615592 shutdown_connections 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.335+0000 7efd9902a740 1 -- 172.21.15.69:0/2842615592 wait complete. 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.335+0000 7efd9902a740 1 -- start start 2024-08-02T17:49:15.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.335+0000 7efd9902a740 1 Processor -- start 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd9902a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edea910 0x55707edfd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd9902a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707ee00080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd9902a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ee0ca80 0x55707ee005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd949c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edea910 0x55707edfd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd949c8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edea910 0x55707edfd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37864/0 (socket says 172.21.15.69:37864) 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd949c8640 1 -- 172.21.15.69:0/1670730546 learned_addr learned my addr 172.21.15.69:0/1670730546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efd901c4a40 con 0x55707edf51d0 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd951c9640 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ee0ca80 0x55707ee005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efd901c5eb0 con 0x55707edea910 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd901c5110 con 0x55707ee0ca80 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd9902a740 10 client.0 _ll_register_callbacks cb 0x7efd880b8100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:15.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd949c8640 1 -- 172.21.15.69:0/1670730546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ee0ca80 msgr2=0x55707ee005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd949c8640 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ee0ca80 0x55707ee005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd949c8640 1 -- 172.21.15.69:0/1670730546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 msgr2=0x55707ee00080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd949c8640 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707ee00080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.336+0000 7efd949c8640 1 -- 172.21.15.69:0/1670730546 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55707ee00dc0 con 0x55707edea910 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.337+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd80004380 con 0x55707edea910 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.337+0000 7efd949c8640 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edea910 0x55707edfd980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd880bc4b0 tx=0x7efd880048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3776d97fd9d3b37 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.337+0000 7efd6ffff640 10 client.25772 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.337+0000 7efd6ffff640 1 -- 172.21.15.69:0/1670730546 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880bc060 con 0x55707edea910 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.337+0000 7efd6ffff640 1 -- 172.21.15.69:0/1670730546 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7efd88030b20 con 0x55707edea910 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.337+0000 7efd6ffff640 1 -- 172.21.15.69:0/1670730546 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efd880bc2f0 con 0x55707edea910 2024-08-02T17:49:15.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.337+0000 7efd9902a740 10 client.25772 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.337+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55707ee6f8e0 con 0x55707edea910 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.338+0000 7efd6ffff640 1 -- 172.21.15.69:0/1670730546 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efd88042070 con 0x55707edea910 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.338+0000 7efd9902a740 10 client.25772 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.338+0000 7efd9902a740 10 client.25772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.338+0000 7efd9902a740 20 client.25772 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.338+0000 7efd9902a740 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55707ee6fd10 0x55707ee72160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.338+0000 7efd9902a740 4 client.25772 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.338+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55707ed5cfb0 con 0x55707ee6fd10 2024-08-02T17:49:15.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.338+0000 7efd8ffff640 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55707ee6fd10 0x55707ee72160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:15.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd8ffff640 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55707ee6fd10 0x55707ee72160 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d693004f3fb2253 server_cookie=4873e580bff2698f in_seq=0 out_seq=0 2024-08-02T17:49:15.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd6ffff640 10 client.25772 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd6ffff640 1 -- 172.21.15.69:0/1670730546 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55707ed5cfb0 con 0x55707ee6fd10 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd6ffff640 10 client.25772 handle_command_reply: tid=0 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 client.25772 shutdown 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edea910 msgr2=0x55707edfd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edea910 0x55707edfd980 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7efd880bc4b0 tx=0x7efd880048a0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 shutdown_connections 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55707edf51d0 0x55707ee00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55707ee6fd10 0x55707ee72160 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55707ee0ca80 0x55707ee005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 --2- 172.21.15.69:0/1670730546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55707edea910 0x55707edfd980 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 >> 172.21.15.69:0/1670730546 conn(0x55707ede6580 msgr2=0x55707eded310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd6ffff640 0 client.25772 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 shutdown_connections 2024-08-02T17:49:15.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 1 -- 172.21.15.69:0/1670730546 wait complete. 2024-08-02T17:49:15.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.339+0000 7efd9902a740 20 client.25772 trim_cache size 0 max 16384 2024-08-02T17:49:15.345 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 -- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efd780a2df0 msgr2=0x7efd780a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efd780a2df0 0x7efd780a52b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd80009d20 tx=0x7efd80005f90 comp rx=0 tx=0).stop 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 -- 172.21.15.69:0/1423558794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd9010a0c0 msgr2=0x7efd901a65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd9010a0c0 0x7efd901a65f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd8802ae70 tx=0x7efd88030870 comp rx=0 tx=0).stop 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 -- 172.21.15.69:0/1423558794 shutdown_connections 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efd901135e0 0x7efd9019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efd780a2df0 0x7efd780a52b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efd9010aa70 0x7efd901a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 --2- 172.21.15.69:0/1423558794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd9010a0c0 0x7efd901a65f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:15.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 -- 172.21.15.69:0/1423558794 >> 172.21.15.69:0/1423558794 conn(0x7efd9006de70 msgr2=0x7efd90071420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:15.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 -- 172.21.15.69:0/1423558794 shutdown_connections 2024-08-02T17:49:15.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:15.341+0000 7efd96c2c640 1 -- 172.21.15.69:0/1423558794 wait complete. 2024-08-02T17:49:15.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:15 smithi077 ceph-mon[37108]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2024-08-02T17:49:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:15 smithi083 ceph-mon[37031]: pgmap v88: 129 pgs: 129 active+clean; 1.6 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 29 MiB/s wr, 1.34k op/s 2024-08-02T17:49:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 -- 172.21.15.69:0/2542290295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe73c102a10 msgr2=0x7fe73c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 --2- 172.21.15.69:0/2542290295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe73c102a10 0x7fe73c102df0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009860 tx=0x7fe72c02a3d0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 -- 172.21.15.69:0/2542290295 shutdown_connections 2024-08-02T17:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 --2- 172.21.15.69:0/2542290295 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe73c10bde0 0x7fe73c10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 --2- 172.21.15.69:0/2542290295 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c103330 0x7fe73c10b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 --2- 172.21.15.69:0/2542290295 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe73c102a10 0x7fe73c102df0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 -- 172.21.15.69:0/2542290295 >> 172.21.15.69:0/2542290295 conn(0x7fe73c0776d0 msgr2=0x7fe73c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 -- 172.21.15.69:0/2542290295 shutdown_connections 2024-08-02T17:49:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.152+0000 7fe7426a5640 1 -- 172.21.15.69:0/2542290295 wait complete. 2024-08-02T17:49:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.153+0000 7fe7426a5640 1 Processor -- start 2024-08-02T17:49:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.153+0000 7fe7426a5640 1 -- start start 2024-08-02T17:49:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe7426a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe73c102a10 0x7fe73c195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe7426a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c103330 0x7fe73c196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe7426a5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe73c10bde0 0x7fe73c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe7426a5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c10fa10 con 0x7fe73c10bde0 2024-08-02T17:49:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe7426a5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c10f890 con 0x7fe73c102a10 2024-08-02T17:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe73c102a10 0x7fe73c195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe7426a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c10fb90 con 0x7fe73c103330 2024-08-02T17:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe73c102a10 0x7fe73c195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37884/0 (socket says 172.21.15.69:37884) 2024-08-02T17:49:16.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe73bfff640 1 -- 172.21.15.69:0/3668283041 learned_addr learned my addr 172.21.15.69:0/3668283041 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe740c42640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe73c10bde0 0x7fe73c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe73bfff640 1 -- 172.21.15.69:0/3668283041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c103330 msgr2=0x7fe73c196500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe73bfff640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c103330 0x7fe73c196500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe73bfff640 1 -- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe73c10bde0 msgr2=0x7fe73c19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe73bfff640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe73c10bde0 0x7fe73c19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.154+0000 7fe73bfff640 1 -- 172.21.15.69:0/3668283041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73c19afb0 con 0x7fe73c102a10 2024-08-02T17:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.155+0000 7fe73bfff640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe73c102a10 0x7fe73c195fc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe72c02a8f0 tx=0x7fe72c030910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.155+0000 7fe7397fa640 1 -- 172.21.15.69:0/3668283041 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c040070 con 0x7fe73c102a10 2024-08-02T17:49:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.155+0000 7fe7397fa640 1 -- 172.21.15.69:0/3668283041 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c004460 con 0x7fe73c102a10 2024-08-02T17:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.155+0000 7fe7397fa640 1 -- 172.21.15.69:0/3668283041 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0046c0 con 0x7fe73c102a10 2024-08-02T17:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.155+0000 7fe7426a5640 1 -- 172.21.15.69:0/3668283041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73c19b240 con 0x7fe73c102a10 2024-08-02T17:49:16.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.155+0000 7fe7426a5640 1 -- 172.21.15.69:0/3668283041 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73c1a2b30 con 0x7fe73c102a10 2024-08-02T17:49:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.159+0000 7fe7397fa640 1 -- 172.21.15.69:0/3668283041 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe72c043070 con 0x7fe73c102a10 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.159+0000 7fe7453ff740 1 Processor -- start 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.159+0000 7fe7453ff740 1 -- start start 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cc6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc6780 0x561398cc1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cc6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cd9bf0 0x561398cdbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cc6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35846/0 (socket says 172.21.15.69:35846) 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe7453ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561398c219b0 con 0x561398cc5e60 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 -- 172.21.15.69:0/1678802479 learned_addr learned my addr 172.21.15.69:0/1678802479 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe7453ff740 1 -- 172.21.15.69:0/1678802479 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561398bc9800 con 0x561398cc6780 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe7453ff740 1 -- 172.21.15.69:0/1678802479 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561398c1d970 con 0x561398cd9bf0 2024-08-02T17:49:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe73bfff640 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc6780 0x561398cc1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe7397fa640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7240a2df0 0x7fe7240a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe73b7fe640 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cd9bf0 0x561398cdbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7240a2df0 0x7fe7240a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 -- 172.21.15.69:0/1678802479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cd9bf0 msgr2=0x561398cdbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cd9bf0 0x561398cdbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 -- 172.21.15.69:0/1678802479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc6780 msgr2=0x561398cc1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc6780 0x561398cc1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.160+0000 7fe740c42640 1 -- 172.21.15.69:0/1678802479 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561398935210 con 0x561398cc5e60 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.161+0000 7fe7397fa640 1 -- 172.21.15.69:0/3668283041 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe72c0f03e0 con 0x7fe73c102a10 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.161+0000 7fe740c42640 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cc6240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe734003fd0 tx=0x7fe7340044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4591c6fbb869ee server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.161+0000 7fe7227fc640 1 -- 172.21.15.69:0/1678802479 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340039d0 con 0x561398cc5e60 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.161+0000 7fe7227fc640 1 -- 172.21.15.69:0/1678802479 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe73401b920 con 0x561398cc5e60 2024-08-02T17:49:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.161+0000 7fe740c42640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7240a2df0 0x7fe7240a52b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe734022000 tx=0x7fe734020000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.161+0000 7fe7227fc640 1 -- 172.21.15.69:0/1678802479 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7340235f0 con 0x561398cc5e60 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.162+0000 7fe7453ff740 1 -- 172.21.15.69:0/1678802479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 msgr2=0x561398cc6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.162+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cc6240 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe734003fd0 tx=0x7fe7340044d0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.163+0000 7fe7453ff740 1 -- 172.21.15.69:0/1678802479 shutdown_connections 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.163+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cd9bf0 0x561398cdbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.163+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc6780 0x561398cc1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.163+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1678802479 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cc6240 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.163+0000 7fe7453ff740 1 -- 172.21.15.69:0/1678802479 >> 172.21.15.69:0/1678802479 conn(0x561398cc2530 msgr2=0x561398cc2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.163+0000 7fe7453ff740 1 -- 172.21.15.69:0/1678802479 shutdown_connections 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.163+0000 7fe7453ff740 1 -- 172.21.15.69:0/1678802479 wait complete. 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.164+0000 7fe7453ff740 1 -- start start 2024-08-02T17:49:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.164+0000 7fe7453ff740 1 Processor -- start 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cbecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cbf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cbecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cbecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35862/0 (socket says 172.21.15.69:35862) 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe73bfff640 1 -- 172.21.15.69:0/1578308054 learned_addr learned my addr 172.21.15.69:0/1578308054 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 0x561398cbf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561398cdc5e0 con 0x561398cc5e60 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561398bdd500 con 0x561398cd9bf0 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe73b7fe640 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cbf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561398cdc760 con 0x561398cc6780 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe740c42640 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 0x561398cbf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.165+0000 7fe7453ff740 10 client.0 _ll_register_callbacks cb 0x561398934a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.166+0000 7fe73bfff640 1 -- 172.21.15.69:0/1578308054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 msgr2=0x561398cbf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.166+0000 7fe73bfff640 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cbf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.166+0000 7fe73bfff640 1 -- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 msgr2=0x561398cbf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.166+0000 7fe73bfff640 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 0x561398cbf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.166+0000 7fe73bfff640 1 -- 172.21.15.69:0/1578308054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561398a01620 con 0x561398cc5e60 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.166+0000 7fe73bfff640 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cbecf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009830 tx=0x7fe72c032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41438a89ed6bb1c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.166+0000 7fe71b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.166+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1578308054 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0ede60 con 0x561398cc5e60 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.167+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1578308054 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe72c0c0580 con 0x561398cc5e60 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.167+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe73401b920 con 0x561398cc5e60 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.167+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1578308054 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe72c04a040 con 0x561398cc5e60 2024-08-02T17:49:16.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.167+0000 7fe7453ff740 10 client.16149 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.167+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561398ce82f0 con 0x561398cc5e60 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1578308054 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe73401b920 con 0x561398cc5e60 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1578308054 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe72c0b8b50 con 0x561398cc5e60 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe7453ff740 10 client.16149 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe7453ff740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe7453ff740 20 client.16149 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561398ce9960 0x561398cebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe7453ff740 4 client.16149 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561398c2f0e0 con 0x561398ce9960 2024-08-02T17:49:16.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.168+0000 7fe73b7fe640 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561398ce9960 0x561398cebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.169+0000 7fe71b7fe640 10 client.16149 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.169+0000 7fe73b7fe640 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561398ce9960 0x561398cebd50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab58e68605e7702 server_cookie=3f799d4794a41339 in_seq=0 out_seq=0 2024-08-02T17:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.170+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1578308054 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561398c2f0e0 con 0x561398ce9960 2024-08-02T17:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.170+0000 7fe71b7fe640 10 client.16149 handle_command_reply: tid=0 2024-08-02T17:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.170+0000 7fe7453ff740 1 client.16149 shutdown 2024-08-02T17:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 msgr2=0x561398cbecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:16.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cbecf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe72c009830 tx=0x7fe72c032f80 comp rx=0 tx=0).stop 2024-08-02T17:49:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 shutdown_connections 2024-08-02T17:49:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561398ce9960 0x561398cebd50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 0x561398cbf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe71b7fe640 0 client.16149 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:16.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cbf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1578308054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc5e60 0x561398cbecf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.171+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 >> 172.21.15.69:0/1578308054 conn(0x561398cc2530 msgr2=0x561398cc8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.172+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 shutdown_connections 2024-08-02T17:49:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.172+0000 7fe7453ff740 1 -- 172.21.15.69:0/1578308054 wait complete. 2024-08-02T17:49:16.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.172+0000 7fe7453ff740 20 client.16149 trim_cache size 0 max 16384 2024-08-02T17:49:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.195+0000 7fe7453ff740 1 Processor -- start 2024-08-02T17:49:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.195+0000 7fe7453ff740 1 -- start start 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.195+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc5e60 0x561398cb8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.195+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cf14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.195+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cd9bf0 0x561398cf1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.195+0000 7fe7453ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1af580 con 0x561398cd9bf0 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.195+0000 7fe7453ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1ace50 con 0x561398cc5e60 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.195+0000 7fe7453ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1aaca0 con 0x561398cc6780 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc5e60 0x561398cb8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cf14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cf14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51898/0 (socket says 172.21.15.69:51898) 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73b7fe640 1 -- 172.21.15.69:0/856566673 learned_addr learned my addr 172.21.15.69:0/856566673 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe740c42640 1 --2- 172.21.15.69:0/856566673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cd9bf0 0x561398cf1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73bfff640 1 -- 172.21.15.69:0/856566673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 msgr2=0x561398cf14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73bfff640 1 --2- 172.21.15.69:0/856566673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cf14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73bfff640 1 -- 172.21.15.69:0/856566673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cd9bf0 msgr2=0x561398cf1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73bfff640 1 --2- 172.21.15.69:0/856566673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cd9bf0 0x561398cf1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73bfff640 1 -- 172.21.15.69:0/856566673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561398cc6da0 con 0x561398cc5e60 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.196+0000 7fe73bfff640 1 --2- 172.21.15.69:0/856566673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc5e60 0x561398cb8d70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005940 tx=0x7fe72c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=904e4cfd480a5958 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.197+0000 7fe721ffb640 1 -- 172.21.15.69:0/856566673 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0ede60 con 0x561398cc5e60 2024-08-02T17:49:16.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.197+0000 7fe721ffb640 1 -- 172.21.15.69:0/856566673 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c0bcbe0 con 0x561398cc5e60 2024-08-02T17:49:16.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.197+0000 7fe721ffb640 1 -- 172.21.15.69:0/856566673 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0bb930 con 0x561398cc5e60 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.198+0000 7fe7453ff740 1 -- 172.21.15.69:0/856566673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc5e60 msgr2=0x561398cb8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.198+0000 7fe7453ff740 1 --2- 172.21.15.69:0/856566673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc5e60 0x561398cb8d70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe72c005940 tx=0x7fe72c005f90 comp rx=0 tx=0).stop 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.198+0000 7fe7453ff740 1 -- 172.21.15.69:0/856566673 shutdown_connections 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.198+0000 7fe7453ff740 1 --2- 172.21.15.69:0/856566673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cd9bf0 0x561398cf1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.198+0000 7fe7453ff740 1 --2- 172.21.15.69:0/856566673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc6780 0x561398cf14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.198+0000 7fe7453ff740 1 --2- 172.21.15.69:0/856566673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cc5e60 0x561398cb8d70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.198+0000 7fe7453ff740 1 -- 172.21.15.69:0/856566673 >> 172.21.15.69:0/856566673 conn(0x561398cc2530 msgr2=0x561398cb8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.199+0000 7fe7453ff740 1 -- 172.21.15.69:0/856566673 shutdown_connections 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.199+0000 7fe7453ff740 1 -- 172.21.15.69:0/856566673 wait complete. 2024-08-02T17:49:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.199+0000 7fe7453ff740 1 -- start start 2024-08-02T17:49:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.199+0000 7fe7453ff740 1 Processor -- start 2024-08-02T17:49:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc5e60 0x561398ce1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc6780 0x561398ce2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe7453ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 0x561398cde9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe7453ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1ab310 con 0x561398cc6780 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe7453ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1ac780 con 0x561398cd9bf0 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc5e60 0x561398ce1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc5e60 0x561398ce1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51906/0 (socket says 172.21.15.69:51906) 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73bfff640 1 -- 172.21.15.69:0/1335808119 learned_addr learned my addr 172.21.15.69:0/1335808119 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe7453ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe73c1ab9e0 con 0x561398cc5e60 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73b7fe640 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc6780 0x561398ce2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe740c42640 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 0x561398cde9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe7453ff740 10 client.0 _ll_register_callbacks cb 0x7fe72c0056e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73bfff640 1 -- 172.21.15.69:0/1335808119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 msgr2=0x561398cde9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73bfff640 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 0x561398cde9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73bfff640 1 -- 172.21.15.69:0/1335808119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc6780 msgr2=0x561398ce2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73bfff640 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc6780 0x561398ce2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.200+0000 7fe73bfff640 1 -- 172.21.15.69:0/1335808119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561398cdf060 con 0x561398cc5e60 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe73bfff640 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc5e60 0x561398ce1be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0bcd90 tx=0x7fe72c0ba9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=745dddefabb57556 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe71b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1335808119 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0ede60 con 0x561398cc5e60 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1335808119 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe72c0a5a20 con 0x561398cc5e60 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 1 -- 172.21.15.69:0/1335808119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe72c0bcbe0 con 0x561398cc5e60 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1335808119 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe72c0bbe40 con 0x561398cc5e60 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 10 client.25654 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:16.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 1 -- 172.21.15.69:0/1335808119 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561398d54e50 con 0x561398cc5e60 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1335808119 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe72c0bcbe0 con 0x561398cc5e60 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1335808119 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe72c0b8230 con 0x561398cc5e60 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 10 client.25654 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 20 client.25654 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561398ce9960 0x561398d573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 4 client.25654 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.201+0000 7fe7453ff740 1 -- 172.21.15.69:0/1335808119 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561398c29430 con 0x561398ce9960 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.202+0000 7fe73b7fe640 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561398ce9960 0x561398d573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:16.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.203+0000 7fe73b7fe640 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561398ce9960 0x561398d573f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f998465292045d81 server_cookie=82e8aa69e67790d1 in_seq=0 out_seq=0 2024-08-02T17:49:16.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.203+0000 7fe71b7fe640 10 client.25654 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:16.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe71b7fe640 1 -- 172.21.15.69:0/1335808119 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561398c29430 con 0x561398ce9960 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe71b7fe640 10 client.25654 handle_command_reply: tid=0 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 client.25654 shutdown 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 -- 172.21.15.69:0/1335808119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc5e60 msgr2=0x561398ce1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc5e60 0x561398ce1be0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe72c0bcd90 tx=0x7fe72c0ba9f0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 -- 172.21.15.69:0/1335808119 shutdown_connections 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561398cc6780 0x561398ce2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561398ce9960 0x561398d573f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561398cd9bf0 0x561398cde9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 --2- 172.21.15.69:0/1335808119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561398cc5e60 0x561398ce1be0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe71b7fe640 0 client.25654 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:16.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 -- 172.21.15.69:0/1335808119 >> 172.21.15.69:0/1335808119 conn(0x561398cc2530 msgr2=0x561398cbecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 -- 172.21.15.69:0/1335808119 shutdown_connections 2024-08-02T17:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 1 -- 172.21.15.69:0/1335808119 wait complete. 2024-08-02T17:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.225+0000 7fe7453ff740 20 client.25654 trim_cache size 0 max 16384 2024-08-02T17:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:49:16.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:49:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 -- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7240a2df0 msgr2=0x7fe7240a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7240a2df0 0x7fe7240a52b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe734022000 tx=0x7fe734020000 comp rx=0 tx=0).stop 2024-08-02T17:49:16.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 -- 172.21.15.69:0/3668283041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe73c102a10 msgr2=0x7fe73c195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe73c102a10 0x7fe73c195fc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe72c02a8f0 tx=0x7fe72c030910 comp rx=0 tx=0).stop 2024-08-02T17:49:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 -- 172.21.15.69:0/3668283041 shutdown_connections 2024-08-02T17:49:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe73c10bde0 0x7fe73c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe7240a2df0 0x7fe7240a52b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe73c103330 0x7fe73c196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 --2- 172.21.15.69:0/3668283041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe73c102a10 0x7fe73c195fc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 -- 172.21.15.69:0/3668283041 >> 172.21.15.69:0/3668283041 conn(0x7fe73c0776d0 msgr2=0x7fe73c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 -- 172.21.15.69:0/3668283041 shutdown_connections 2024-08-02T17:49:16.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:16.226+0000 7fe7426a5640 1 -- 172.21.15.69:0/3668283041 wait complete. 2024-08-02T17:49:17.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:49:17.318 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:49:17.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:17 smithi077 ceph-mon[37108]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-08-02T17:49:17.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:17 smithi069 ceph-mon[32746]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-08-02T17:49:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:17 smithi083 ceph-mon[37031]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 33 MiB/s wr, 1.68k op/s 2024-08-02T17:49:17.716 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:18.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.616+0000 7f549f91a640 1 -- 172.21.15.69:0/1010313695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5498102c20 msgr2=0x7f549810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:18.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.616+0000 7f549f91a640 1 --2- 172.21.15.69:0/1010313695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5498102c20 0x7f549810d770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5490009cd0 tx=0x7f549002a9c0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.617+0000 7f549f91a640 1 -- 172.21.15.69:0/1010313695 shutdown_connections 2024-08-02T17:49:18.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.617+0000 7f549f91a640 1 --2- 172.21.15.69:0/1010313695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54981022a0 0x7f54981026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.617+0000 7f549f91a640 1 --2- 172.21.15.69:0/1010313695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5498102c20 0x7f549810d770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.617+0000 7f549f91a640 1 --2- 172.21.15.69:0/1010313695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5498100b50 0x7f5498100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.617+0000 7f549f91a640 1 -- 172.21.15.69:0/1010313695 >> 172.21.15.69:0/1010313695 conn(0x7f54980fc500 msgr2=0x7f54980fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:18.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.619+0000 7f549f91a640 1 -- 172.21.15.69:0/1010313695 shutdown_connections 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.620+0000 7f549f91a640 1 -- 172.21.15.69:0/1010313695 wait complete. 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.621+0000 7f549f91a640 1 Processor -- start 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549f91a640 1 -- start start 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549f91a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5498100b50 0x7f54980736a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549f91a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54981022a0 0x7f5498073be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549f91a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5498102c20 0x7f549806d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549f91a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f549806c390 con 0x7f54981022a0 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549f91a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f549806c210 con 0x7f5498102c20 2024-08-02T17:49:18.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549f91a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f549806c510 con 0x7f5498100b50 2024-08-02T17:49:18.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549deb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5498102c20 0x7f549806d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549deb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5498102c20 0x7f549806d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37934/0 (socket says 172.21.15.69:37934) 2024-08-02T17:49:18.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.622+0000 7f549deb7640 1 -- 172.21.15.69:0/441652232 learned_addr learned my addr 172.21.15.69:0/441652232 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:18.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.623+0000 7f549deb7640 1 -- 172.21.15.69:0/441652232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5498100b50 msgr2=0x7f54980736a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:49:18.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.623+0000 7f549deb7640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5498100b50 0x7f54980736a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.623+0000 7f549deb7640 1 -- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54981022a0 msgr2=0x7f5498073be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:18.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.623+0000 7f549deb7640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54981022a0 0x7f5498073be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.623+0000 7f549deb7640 1 -- 172.21.15.69:0/441652232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549806e1e0 con 0x7f5498102c20 2024-08-02T17:49:18.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.624+0000 7f549deb7640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5498102c20 0x7f549806d920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54900309f0 tx=0x7f5490030a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:18.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.624+0000 7f548e7fc640 1 -- 172.21.15.69:0/441652232 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490049070 con 0x7f5498102c20 2024-08-02T17:49:18.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.625+0000 7f549f91a640 1 -- 172.21.15.69:0/441652232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549806e470 con 0x7f5498102c20 2024-08-02T17:49:18.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.625+0000 7f548e7fc640 1 -- 172.21.15.69:0/441652232 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f549002f8b0 con 0x7f5498102c20 2024-08-02T17:49:18.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.626+0000 7f548e7fc640 1 -- 172.21.15.69:0/441652232 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490034050 con 0x7f5498102c20 2024-08-02T17:49:18.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.628+0000 7f548e7fc640 1 -- 172.21.15.69:0/441652232 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f549004c070 con 0x7f5498102c20 2024-08-02T17:49:18.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.629+0000 7f548e7fc640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54840a2c20 0x7f54840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.633+0000 7f549f91a640 1 -- 172.21.15.69:0/441652232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5498069e70 con 0x7f5498102c20 2024-08-02T17:49:18.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.634+0000 7f548e7fc640 1 -- 172.21.15.69:0/441652232 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f54900bcd40 con 0x7f5498102c20 2024-08-02T17:49:18.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.638+0000 7f54a1d18740 1 Processor -- start 2024-08-02T17:49:18.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.638+0000 7f54a1d18740 1 -- start start 2024-08-02T17:49:18.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.636+0000 7f549d6b6640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54840a2c20 0x7f54840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.642+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.642+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 0x55d36ab43180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.643+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab45ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.643+0000 7f54a1d18740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d36aafbe40 con 0x55d36ab384e0 2024-08-02T17:49:18.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.643+0000 7f549ceb5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 0x55d36ab43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.643+0000 7f549deb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab45ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.643+0000 7f549deb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab45ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51912/0 (socket says 172.21.15.69:51912) 2024-08-02T17:49:18.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.643+0000 7f549deb7640 1 -- 172.21.15.69:0/3219874027 learned_addr learned my addr 172.21.15.69:0/3219874027 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:18.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.644+0000 7f54a1d18740 1 -- 172.21.15.69:0/3219874027 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d36aa9d7f0 con 0x55d36ab38e00 2024-08-02T17:49:18.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.644+0000 7f54a1d18740 1 -- 172.21.15.69:0/3219874027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d36aa44fb0 con 0x55d36ab436c0 2024-08-02T17:49:18.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.645+0000 7f549deb7640 1 -- 172.21.15.69:0/3219874027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 msgr2=0x55d36ab43180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:18.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.645+0000 7f549deb7640 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 0x55d36ab43180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.645+0000 7f549d6b6640 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.645+0000 7f549deb7640 1 -- 172.21.15.69:0/3219874027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 msgr2=0x55d36ab388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:18.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.645+0000 7f549deb7640 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.645+0000 7f549deb7640 1 -- 172.21.15.69:0/3219874027 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36a80f6a0 con 0x55d36ab436c0 2024-08-02T17:49:18.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.645+0000 7f549d6b6640 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:18.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.647+0000 7f549deb7640 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab45ab0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f54900bb410 tx=0x7f5490009890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3396efa02640dc77 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:18.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.647+0000 7f549d6b6640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54840a2c20 0x7f54840a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54940099e0 tx=0x7f5494009de0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:18.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.647+0000 7f5482ffd640 1 -- 172.21.15.69:0/3219874027 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54900dc070 con 0x55d36ab436c0 2024-08-02T17:49:18.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.647+0000 7f5482ffd640 1 -- 172.21.15.69:0/3219874027 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54900bf140 con 0x55d36ab436c0 2024-08-02T17:49:18.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.647+0000 7f5482ffd640 1 -- 172.21.15.69:0/3219874027 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490004da0 con 0x55d36ab436c0 2024-08-02T17:49:18.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.651+0000 7f54a1d18740 1 -- 172.21.15.69:0/3219874027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 msgr2=0x55d36ab45ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:18.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.651+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab45ab0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f54900bb410 tx=0x7f5490009890 comp rx=0 tx=0).stop 2024-08-02T17:49:18.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.651+0000 7f54a1d18740 1 -- 172.21.15.69:0/3219874027 shutdown_connections 2024-08-02T17:49:18.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.652+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab45ab0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.652+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 0x55d36ab43180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.652+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3219874027 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.652+0000 7f54a1d18740 1 -- 172.21.15.69:0/3219874027 >> 172.21.15.69:0/3219874027 conn(0x55d36ab34a70 msgr2=0x55d36ab34e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:18.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.654+0000 7f54a1d18740 1 -- 172.21.15.69:0/3219874027 shutdown_connections 2024-08-02T17:49:18.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.654+0000 7f54a1d18740 1 -- 172.21.15.69:0/3219874027 wait complete. 2024-08-02T17:49:18.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.654+0000 7f54a1d18740 1 -- start start 2024-08-02T17:49:18.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.654+0000 7f54a1d18740 1 Processor -- start 2024-08-02T17:49:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.655+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.655+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab38e00 0x55d36ab403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.655+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab436c0 0x55d36ab409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.655+0000 7f54a1d18740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d36aa58cb0 con 0x55d36ab436c0 2024-08-02T17:49:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.655+0000 7f54a1d18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d36aa997b0 con 0x55d36ab384e0 2024-08-02T17:49:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.655+0000 7f54a1d18740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d36aa44fb0 con 0x55d36ab38e00 2024-08-02T17:49:18.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.655+0000 7f54a1d18740 10 client.0 _ll_register_callbacks cb 0x55d36a7acba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.656+0000 7f549d6b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.656+0000 7f549d6b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab430f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37948/0 (socket says 172.21.15.69:37948) 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.656+0000 7f549d6b6640 1 -- 172.21.15.69:0/3240700928 learned_addr learned my addr 172.21.15.69:0/3240700928 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.657+0000 7f549ceb5640 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab38e00 0x55d36ab403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.657+0000 7f549deb7640 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab436c0 0x55d36ab409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.657+0000 7f549d6b6640 1 -- 172.21.15.69:0/3240700928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab38e00 msgr2=0x55d36ab403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.657+0000 7f549d6b6640 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab38e00 0x55d36ab403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.657+0000 7f549d6b6640 1 -- 172.21.15.69:0/3240700928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab436c0 msgr2=0x55d36ab409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.657+0000 7f549d6b6640 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab436c0 0x55d36ab409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.657+0000 7f549d6b6640 1 -- 172.21.15.69:0/3240700928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36aa57bc0 con 0x55d36ab384e0 2024-08-02T17:49:18.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.658+0000 7f549d6b6640 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab430f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5494017740 tx=0x7f5494004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f065611e5e04770a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:18.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.658+0000 7f5480ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:18.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.658+0000 7f5480ff9640 1 -- 172.21.15.69:0/3240700928 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494027060 con 0x55d36ab384e0 2024-08-02T17:49:18.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.658+0000 7f54a1d18740 1 -- 172.21.15.69:0/3240700928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54900bf140 con 0x55d36ab384e0 2024-08-02T17:49:18.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.659+0000 7f5480ff9640 1 -- 172.21.15.69:0/3240700928 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5494014680 con 0x55d36ab384e0 2024-08-02T17:49:18.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.659+0000 7f5480ff9640 1 -- 172.21.15.69:0/3240700928 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f549402e050 con 0x55d36ab384e0 2024-08-02T17:49:18.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.659+0000 7f54a1d18740 10 client.25799 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:18.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.659+0000 7f54a1d18740 1 -- 172.21.15.69:0/3240700928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d36ab55f80 con 0x55d36ab384e0 2024-08-02T17:49:18.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.660+0000 7f5480ff9640 1 -- 172.21.15.69:0/3240700928 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54900bf140 con 0x55d36ab384e0 2024-08-02T17:49:18.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.660+0000 7f5480ff9640 1 -- 172.21.15.69:0/3240700928 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5494022a60 con 0x55d36ab384e0 2024-08-02T17:49:18.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.660+0000 7f54a1d18740 10 client.25799 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:18.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.660+0000 7f54a1d18740 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:18.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.660+0000 7f54a1d18740 20 client.25799 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:18.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.661+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d36ab578e0 0x55d36ab59d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.661+0000 7f54a1d18740 4 client.25799 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:18.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.661+0000 7f54a1d18740 1 -- 172.21.15.69:0/3240700928 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d36aaaad60 con 0x55d36ab578e0 2024-08-02T17:49:18.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.662+0000 7f549ceb5640 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d36ab578e0 0x55d36ab59d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.665+0000 7f549ceb5640 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d36ab578e0 0x55d36ab59d20 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9459f5316a7b9886 server_cookie=208f0ba09c4d2297 in_seq=0 out_seq=0 2024-08-02T17:49:18.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.665+0000 7f5480ff9640 10 client.25799 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:18.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.666+0000 7f5480ff9640 1 -- 172.21.15.69:0/3240700928 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d36aaaad60 con 0x55d36ab578e0 2024-08-02T17:49:18.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.666+0000 7f5480ff9640 10 client.25799 handle_command_reply: tid=0 2024-08-02T17:49:18.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.666+0000 7f54a1d18740 1 client.25799 shutdown 2024-08-02T17:49:18.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.667+0000 7f54a1d18740 1 -- 172.21.15.69:0/3240700928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 msgr2=0x55d36ab430f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:18.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.667+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab430f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5494017740 tx=0x7f5494004770 comp rx=0 tx=0).stop 2024-08-02T17:49:18.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.669+0000 7f54a1d18740 1 -- 172.21.15.69:0/3240700928 shutdown_connections 2024-08-02T17:49:18.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.669+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab436c0 0x55d36ab409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.669+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d36ab578e0 0x55d36ab59d20 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.669+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab38e00 0x55d36ab403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.670+0000 7f54a1d18740 1 --2- 172.21.15.69:0/3240700928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab430f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.670+0000 7f54a1d18740 1 -- 172.21.15.69:0/3240700928 >> 172.21.15.69:0/3240700928 conn(0x55d36ab34a70 msgr2=0x55d36ab3a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:18.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.670+0000 7f5480ff9640 0 client.25799 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:18.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.671+0000 7f54a1d18740 1 -- 172.21.15.69:0/3240700928 shutdown_connections 2024-08-02T17:49:18.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.671+0000 7f54a1d18740 1 -- 172.21.15.69:0/3240700928 wait complete. 2024-08-02T17:49:18.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.671+0000 7f54a1d18740 20 client.25799 trim_cache size 0 max 16384 2024-08-02T17:49:18.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.700+0000 7f54a1d18740 1 Processor -- start 2024-08-02T17:49:18.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.700+0000 7f54a1d18740 1 -- start start 2024-08-02T17:49:18.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.700+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.700+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 0x55d36ab5f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.700+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab5fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.701+0000 7f549d6b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.701+0000 7f549d6b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35918/0 (socket says 172.21.15.69:35918) 2024-08-02T17:49:18.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.701+0000 7f549d6b6640 1 -- 172.21.15.69:0/1132850527 learned_addr learned my addr 172.21.15.69:0/1132850527 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:18.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.701+0000 7f549deb7640 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab5fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.701+0000 7f549ceb5640 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 0x55d36ab5f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.702+0000 7f54a1d18740 1 -- 172.21.15.69:0/1132850527 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f54981b7c30 con 0x55d36ab384e0 2024-08-02T17:49:18.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.702+0000 7f54a1d18740 1 -- 172.21.15.69:0/1132850527 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54981b5500 con 0x55d36ab38e00 2024-08-02T17:49:18.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.702+0000 7f54a1d18740 1 -- 172.21.15.69:0/1132850527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54981b3350 con 0x55d36ab436c0 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.703+0000 7f549deb7640 1 -- 172.21.15.69:0/1132850527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 msgr2=0x55d36ab5f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.703+0000 7f549deb7640 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 0x55d36ab5f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.703+0000 7f549deb7640 1 -- 172.21.15.69:0/1132850527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 msgr2=0x55d36ab598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.703+0000 7f549deb7640 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.703+0000 7f549deb7640 1 -- 172.21.15.69:0/1132850527 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d36ab38c30 con 0x55d36ab436c0 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.703+0000 7f549deb7640 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab5fb70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55d36ab394a0 tx=0x7f54900be4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b0c1771817f3f4c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.704+0000 7f54827fc640 1 -- 172.21.15.69:0/1132850527 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54900dc070 con 0x55d36ab436c0 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.704+0000 7f54827fc640 1 -- 172.21.15.69:0/1132850527 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54900bf140 con 0x55d36ab436c0 2024-08-02T17:49:18.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.704+0000 7f54827fc640 1 -- 172.21.15.69:0/1132850527 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490004b40 con 0x55d36ab436c0 2024-08-02T17:49:18.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 -- 172.21.15.69:0/1132850527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 msgr2=0x55d36ab5fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:18.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab5fb70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55d36ab394a0 tx=0x7f54900be4a0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 -- 172.21.15.69:0/1132850527 shutdown_connections 2024-08-02T17:49:18.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36ab5fb70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab38e00 0x55d36ab5f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1132850527 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab384e0 0x55d36ab598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 -- 172.21.15.69:0/1132850527 >> 172.21.15.69:0/1132850527 conn(0x55d36ab34a70 msgr2=0x55d36ab3ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:18.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 -- 172.21.15.69:0/1132850527 shutdown_connections 2024-08-02T17:49:18.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 -- 172.21.15.69:0/1132850527 wait complete. 2024-08-02T17:49:18.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 -- start start 2024-08-02T17:49:18.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.705+0000 7f54a1d18740 1 Processor -- start 2024-08-02T17:49:18.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.706+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.706+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab38e00 0x55d36ab47e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.706+0000 7f54a1d18740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36abba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.706+0000 7f54a1d18740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f54981b39c0 con 0x55d36ab38e00 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f54a1d18740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54981b4e30 con 0x55d36ab384e0 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f54a1d18740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54981b4090 con 0x55d36ab436c0 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f54a1d18740 10 client.0 _ll_register_callbacks cb 0x7f549400f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549d6b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549deb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36abba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549d6b6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab478c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37956/0 (socket says 172.21.15.69:37956) 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549deb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36abba5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51948/0 (socket says 172.21.15.69:51948) 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549d6b6640 1 -- 172.21.15.69:0/1973572390 learned_addr learned my addr 172.21.15.69:0/1973572390 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:18.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549ceb5640 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab38e00 0x55d36ab47e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549deb7640 1 -- 172.21.15.69:0/1973572390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 msgr2=0x55d36ab478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549deb7640 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549deb7640 1 -- 172.21.15.69:0/1973572390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab38e00 msgr2=0x55d36ab47e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549deb7640 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab38e00 0x55d36ab47e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.707+0000 7f549deb7640 1 -- 172.21.15.69:0/1973572390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d36ab0eb40 con 0x55d36ab436c0 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.708+0000 7f549ceb5640 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab38e00 0x55d36ab47e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.708+0000 7f549deb7640 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36abba5c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54900be510 tx=0x7f54900f0220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e98c41404815801 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.708+0000 7f5477fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.708+0000 7f5477fff640 1 -- 172.21.15.69:0/1973572390 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54900dc070 con 0x55d36ab436c0 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.708+0000 7f5477fff640 1 -- 172.21.15.69:0/1973572390 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5490009890 con 0x55d36ab436c0 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.708+0000 7f54a1d18740 1 -- 172.21.15.69:0/1973572390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54900bf140 con 0x55d36ab436c0 2024-08-02T17:49:18.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.708+0000 7f5477fff640 1 -- 172.21.15.69:0/1973572390 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54900bea40 con 0x55d36ab436c0 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f54a1d18740 10 client.25669 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f54a1d18740 1 -- 172.21.15.69:0/1973572390 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d36ab0eb40 con 0x55d36ab436c0 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f5477fff640 1 -- 172.21.15.69:0/1973572390 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54900bf140 con 0x55d36ab436c0 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f5477fff640 1 -- 172.21.15.69:0/1973572390 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f54900ba790 con 0x55d36ab436c0 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f54a1d18740 10 client.25669 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f54a1d18740 10 client.25669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f54a1d18740 20 client.25669 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d36abc1e30 0x55d36abc4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f54a1d18740 4 client.25669 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:49:18.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.709+0000 7f54a1d18740 1 -- 172.21.15.69:0/1973572390 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d36ab2fe10 con 0x55d36abc1e30 2024-08-02T17:49:18.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.710+0000 7f549d6b6640 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d36abc1e30 0x55d36abc4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:18.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.710+0000 7f549d6b6640 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d36abc1e30 0x55d36abc4220 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21c689a45c0b42c server_cookie=ae1ddf139dcc35ed in_seq=0 out_seq=0 2024-08-02T17:49:18.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.710+0000 7f5477fff640 10 client.25669 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:18.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.712+0000 7f5477fff640 1 -- 172.21.15.69:0/1973572390 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d36ab2fe10 con 0x55d36abc1e30 2024-08-02T17:49:18.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.712+0000 7f5477fff640 10 client.25669 handle_command_reply: tid=0 2024-08-02T17:49:18.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 client.25669 shutdown 2024-08-02T17:49:18.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 -- 172.21.15.69:0/1973572390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 msgr2=0x55d36abba5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:18.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36abba5c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54900be510 tx=0x7f54900f0220 comp rx=0 tx=0).stop 2024-08-02T17:49:18.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 -- 172.21.15.69:0/1973572390 shutdown_connections 2024-08-02T17:49:18.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d36ab38e00 0x55d36ab47e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d36abc1e30 0x55d36abc4220 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d36ab436c0 0x55d36abba5c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 --2- 172.21.15.69:0/1973572390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d36ab384e0 0x55d36ab478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f5477fff640 0 client.25669 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.713+0000 7f54a1d18740 1 -- 172.21.15.69:0/1973572390 >> 172.21.15.69:0/1973572390 conn(0x55d36ab34a70 msgr2=0x55d36ab39f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.714+0000 7f54a1d18740 1 -- 172.21.15.69:0/1973572390 shutdown_connections 2024-08-02T17:49:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.714+0000 7f54a1d18740 1 -- 172.21.15.69:0/1973572390 wait complete. 2024-08-02T17:49:18.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.714+0000 7f54a1d18740 20 client.25669 trim_cache size 0 max 16384 2024-08-02T17:49:18.716 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:49:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:49:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "c6c78eb4-b1f8-420f-98f4-6691309b9f3d", 2024-08-02T17:49:18.716 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:49:18.716 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.714+0000 7f5483fff640 1 -- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54840a2c20 msgr2=0x7f54840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.714+0000 7f5483fff640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54840a2c20 0x7f54840a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54940099e0 tx=0x7f5494009de0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 -- 172.21.15.69:0/441652232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5498102c20 msgr2=0x7f549806d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5498102c20 0x7f549806d920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54900309f0 tx=0x7f5490030a20 comp rx=0 tx=0).stop 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 -- 172.21.15.69:0/441652232 shutdown_connections 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54981022a0 0x7f5498073be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54840a2c20 0x7f54840a50e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5498102c20 0x7f549806d920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 --2- 172.21.15.69:0/441652232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5498100b50 0x7f54980736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 -- 172.21.15.69:0/441652232 >> 172.21.15.69:0/441652232 conn(0x7f54980fc500 msgr2=0x7f5498103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 -- 172.21.15.69:0/441652232 shutdown_connections 2024-08-02T17:49:18.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:18.715+0000 7f5483fff640 1 -- 172.21.15.69:0/441652232 wait complete. 2024-08-02T17:49:19.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:19 smithi077 ceph-mon[37108]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.18k op/s 2024-08-02T17:49:19.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:19 smithi069 ceph-mon[32746]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.18k op/s 2024-08-02T17:49:19.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:19 smithi083 ceph-mon[37031]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 1.18k op/s 2024-08-02T17:49:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 130.0 s, 3090.4 tps, lat 32.319 ms stddev 37.282 2024-08-02T17:49:20.104 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:49:20.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:20 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:49:20.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:20 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:20.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:20 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:49:20.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:20 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:20.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:20 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:49:20.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:20 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:49:20.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:20 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:20.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:20 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:49:20.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:20 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:20.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:20 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:49:20.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:20 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:49:20.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:20 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:20.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:20 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:49:20.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:20 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:20.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:20 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:49:20.574 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:21.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.085+0000 7f7d6a7fc640 1 -- 172.21.15.69:0/2139728921 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d64031cc0 con 0x7f7d6c100f10 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 -- 172.21.15.69:0/2139728921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c100f10 msgr2=0x7f7d6c1056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/2139728921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c100f10 0x7f7d6c1056b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009cd0 tx=0x7f7d6402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 -- 172.21.15.69:0/2139728921 shutdown_connections 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/2139728921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d6c105bf0 0x7f7d6c106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/2139728921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c100f10 0x7f7d6c1056b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/2139728921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d6c1005f0 0x7f7d6c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 -- 172.21.15.69:0/2139728921 >> 172.21.15.69:0/2139728921 conn(0x7f7d6c0fc500 msgr2=0x7f7d6c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 -- 172.21.15.69:0/2139728921 shutdown_connections 2024-08-02T17:49:21.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.086+0000 7f7d71f3a640 1 -- 172.21.15.69:0/2139728921 wait complete. 2024-08-02T17:49:21.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.087+0000 7f7d71f3a640 1 Processor -- start 2024-08-02T17:49:21.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.087+0000 7f7d71f3a640 1 -- start start 2024-08-02T17:49:21.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d71f3a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c1005f0 0x7f7d6c1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c1005f0 0x7f7d6c1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d6b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c1005f0 0x7f7d6c1a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37970/0 (socket says 172.21.15.69:37970) 2024-08-02T17:49:21.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d71f3a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d6c100f10 0x7f7d6c1a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d71f3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d6c105bf0 0x7f7d6c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d71f3a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c118390 con 0x7f7d6c100f10 2024-08-02T17:49:21.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d71f3a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c118210 con 0x7f7d6c1005f0 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d71f3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c118510 con 0x7f7d6c105bf0 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.088+0000 7f7d6b7fe640 1 -- 172.21.15.69:0/967607288 learned_addr learned my addr 172.21.15.69:0/967607288 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d6c105bf0 0x7f7d6c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d6b7fe640 1 -- 172.21.15.69:0/967607288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d6c105bf0 msgr2=0x7f7d6c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d6c105bf0 0x7f7d6c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d6b7fe640 1 -- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d6c100f10 msgr2=0x7f7d6c1a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d6c100f10 0x7f7d6c1a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d6b7fe640 1 -- 172.21.15.69:0/967607288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c19ef30 con 0x7f7d6c1005f0 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c1005f0 0x7f7d6c1a45f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00ea60 tx=0x7f7d5c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d68ff9640 1 -- 172.21.15.69:0/967607288 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00cde0 con 0x7f7d6c1005f0 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d68ff9640 1 -- 172.21.15.69:0/967607288 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c010910 con 0x7f7d6c1005f0 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.089+0000 7f7d68ff9640 1 -- 172.21.15.69:0/967607288 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c018a60 con 0x7f7d6c1005f0 2024-08-02T17:49:21.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.090+0000 7f7d71f3a640 1 -- 172.21.15.69:0/967607288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c19f220 con 0x7f7d6c1005f0 2024-08-02T17:49:21.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.090+0000 7f7d71f3a640 1 -- 172.21.15.69:0/967607288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c1ab3a0 con 0x7f7d6c1005f0 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.093+0000 7f7d68ff9640 1 -- 172.21.15.69:0/967607288 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7d5c018c00 con 0x7f7d6c1005f0 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.093+0000 7f7d74338740 1 Processor -- start 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.094+0000 7f7d74338740 1 -- start start 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.094+0000 7f7d74338740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435bd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.094+0000 7f7d74338740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.094+0000 7f7d74338740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.094+0000 7f7d74338740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a43522980 con 0x560a435bdb30 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.094+0000 7f7d74338740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560a434baa70 con 0x560a435bd210 2024-08-02T17:49:21.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.094+0000 7f7d74338740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a4351e940 con 0x560a435c83f0 2024-08-02T17:49:21.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.095+0000 7f7d68ff9640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7d540a2d70 0x7f7d540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.095+0000 7f7d6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.095+0000 7f7d6affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.096+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7d540a2d70 0x7f7d540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.097+0000 7f7d6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435bd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.097+0000 7f7d6bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435bd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:37980/0 (socket says 172.21.15.69:37980) 2024-08-02T17:49:21.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.097+0000 7f7d6bfff640 1 -- 172.21.15.69:0/2838728954 learned_addr learned my addr 172.21.15.69:0/2838728954 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:21.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.098+0000 7f7d68ff9640 1 -- 172.21.15.69:0/967607288 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f7d5c0cfaf0 con 0x7f7d6c1005f0 2024-08-02T17:49:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.099+0000 7f7d6bfff640 1 -- 172.21.15.69:0/2838728954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 msgr2=0x560a435ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.099+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/2838728954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435ca7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.099+0000 7f7d6bfff640 1 -- 172.21.15.69:0/2838728954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 msgr2=0x560a435c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.099+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/2838728954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.099+0000 7f7d6bfff640 1 -- 172.21.15.69:0/2838728954 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4351d6e0 con 0x560a435bd210 2024-08-02T17:49:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.099+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/2838728954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.099+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7d540a2d70 0x7f7d540a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d60012890 tx=0x7f7d60009f00 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:21.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.100+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/2838728954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435bd5f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009a70 tx=0x7f7d6001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f46eaaf1c97b0cd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:21.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.105+0000 7f7d51ffb640 1 -- 172.21.15.69:0/2838728954 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60022070 con 0x560a435bd210 2024-08-02T17:49:21.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.105+0000 7f7d51ffb640 1 -- 172.21.15.69:0/2838728954 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6001dc40 con 0x560a435bd210 2024-08-02T17:49:21.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.105+0000 7f7d51ffb640 1 -- 172.21.15.69:0/2838728954 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60026b00 con 0x560a435bd210 2024-08-02T17:49:21.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 -- 172.21.15.69:0/2838728954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 msgr2=0x560a435bd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 --2- 172.21.15.69:0/2838728954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435bd5f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009a70 tx=0x7f7d6001c000 comp rx=0 tx=0).stop 2024-08-02T17:49:21.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 -- 172.21.15.69:0/2838728954 shutdown_connections 2024-08-02T17:49:21.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 --2- 172.21.15.69:0/2838728954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 --2- 172.21.15.69:0/2838728954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435ca7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 --2- 172.21.15.69:0/2838728954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435bd5f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 -- 172.21.15.69:0/2838728954 >> 172.21.15.69:0/2838728954 conn(0x560a435b97a0 msgr2=0x560a435b9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:21.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 -- 172.21.15.69:0/2838728954 shutdown_connections 2024-08-02T17:49:21.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.106+0000 7f7d74338740 1 -- 172.21.15.69:0/2838728954 wait complete. 2024-08-02T17:49:21.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.107+0000 7f7d74338740 1 -- start start 2024-08-02T17:49:21.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.107+0000 7f7d74338740 1 Processor -- start 2024-08-02T17:49:21.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.108+0000 7f7d74338740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.108+0000 7f7d74338740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.108+0000 7f7d74338740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.108+0000 7f7d74338740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560a435caf60 con 0x560a435bdb30 2024-08-02T17:49:21.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.108+0000 7f7d74338740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560a434ddaa0 con 0x560a435bd210 2024-08-02T17:49:21.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.108+0000 7f7d74338740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560a435cb0e0 con 0x560a435c83f0 2024-08-02T17:49:21.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.109+0000 7f7d74338740 10 client.0 _ll_register_callbacks cb 0x560a434cce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:21.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.109+0000 7f7d6bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.109+0000 7f7d6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.109+0000 7f7d6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35974/0 (socket says 172.21.15.69:35974) 2024-08-02T17:49:21.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.110+0000 7f7d6affd640 1 -- 172.21.15.69:0/3435718530 learned_addr learned my addr 172.21.15.69:0/3435718530 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:21.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.110+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.111+0000 7f7d6b7fe640 1 -- 172.21.15.69:0/3435718530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 msgr2=0x560a435c5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:21.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.111+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.111+0000 7f7d6b7fe640 1 -- 172.21.15.69:0/3435718530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 msgr2=0x560a435c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:21.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.111+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.111+0000 7f7d6b7fe640 1 -- 172.21.15.69:0/3435718530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a434c1650 con 0x560a435bd210 2024-08-02T17:49:21.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.111+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435c5270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0cce80 tx=0x7f7d5c06dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84849fbf2c173daf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.117+0000 7f7d4affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.117+0000 7f7d4affd640 1 -- 172.21.15.69:0/3435718530 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c067d40 con 0x560a435bd210 2024-08-02T17:49:21.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.117+0000 7f7d4affd640 1 -- 172.21.15.69:0/3435718530 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f7d5c09b0a0 con 0x560a435bd210 2024-08-02T17:49:21.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.118+0000 7f7d74338740 1 -- 172.21.15.69:0/3435718530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6001dc40 con 0x560a435bd210 2024-08-02T17:49:21.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.120+0000 7f7d4affd640 1 -- 172.21.15.69:0/3435718530 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d5c0152d0 con 0x560a435bd210 2024-08-02T17:49:21.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.120+0000 7f7d74338740 10 client.25817 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:21.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.120+0000 7f7d74338740 1 -- 172.21.15.69:0/3435718530 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a432943c0 con 0x560a435bd210 2024-08-02T17:49:21.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.121+0000 7f7d4affd640 1 -- 172.21.15.69:0/3435718530 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d6001dc40 con 0x560a435bd210 2024-08-02T17:49:21.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.121+0000 7f7d4affd640 1 -- 172.21.15.69:0/3435718530 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7d5c010430 con 0x560a435bd210 2024-08-02T17:49:21.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.121+0000 7f7d74338740 10 client.25817 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:21.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.121+0000 7f7d74338740 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:21.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.121+0000 7f7d74338740 20 client.25817 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:21.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.121+0000 7f7d74338740 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560a435d8290 0x560a435da710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.121+0000 7f7d74338740 4 client.25817 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:21.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.121+0000 7f7d74338740 1 -- 172.21.15.69:0/3435718530 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a4352fef0 con 0x560a435d8290 2024-08-02T17:49:21.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.122+0000 7f7d6affd640 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560a435d8290 0x560a435da710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.123+0000 7f7d6affd640 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560a435d8290 0x560a435da710 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f20a836158492c7 server_cookie=53e816008459fa4f in_seq=0 out_seq=0 2024-08-02T17:49:21.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.123+0000 7f7d4affd640 10 client.25817 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:21.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.126+0000 7f7d4affd640 1 -- 172.21.15.69:0/3435718530 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560a4352fef0 con 0x560a435d8290 2024-08-02T17:49:21.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.126+0000 7f7d4affd640 10 client.25817 handle_command_reply: tid=0 2024-08-02T17:49:21.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.126+0000 7f7d74338740 1 client.25817 shutdown 2024-08-02T17:49:21.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d74338740 1 -- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 msgr2=0x560a435c5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d74338740 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435c5270 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0cce80 tx=0x7f7d5c06dc10 comp rx=0 tx=0).stop 2024-08-02T17:49:21.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d74338740 1 -- 172.21.15.69:0/3435718530 shutdown_connections 2024-08-02T17:49:21.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d74338740 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bdb30 0x560a435c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d74338740 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560a435d8290 0x560a435da710 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d74338740 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d74338740 1 --2- 172.21.15.69:0/3435718530 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bd210 0x560a435c5270 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d74338740 1 -- 172.21.15.69:0/3435718530 >> 172.21.15.69:0/3435718530 conn(0x560a435b97a0 msgr2=0x560a435bf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:21.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.132+0000 7f7d4affd640 0 client.25817 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:21.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.133+0000 7f7d74338740 1 -- 172.21.15.69:0/3435718530 shutdown_connections 2024-08-02T17:49:21.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.136+0000 7f7d74338740 1 -- 172.21.15.69:0/3435718530 wait complete. 2024-08-02T17:49:21.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.137+0000 7f7d74338740 20 client.25817 trim_cache size 0 max 16384 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.171+0000 7f7d74338740 1 Processor -- start 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.171+0000 7f7d74338740 1 -- start start 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d74338740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d74338740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435c3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d74338740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d74338740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7d58012920 con 0x560a435bd210 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d74338740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d580101f0 con 0x560a435bdb30 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d74338740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5800e040 con 0x560a435c83f0 2024-08-02T17:49:21.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51984/0 (socket says 172.21.15.69:51984) 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35976/0 (socket says 172.21.15.69:35976) 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6bfff640 1 -- 172.21.15.69:0/1213929484 learned_addr learned my addr 172.21.15.69:0/1213929484 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6affd640 1 --2- 172.21.15.69:0/1213929484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435c3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6bfff640 1 -- 172.21.15.69:0/1213929484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 msgr2=0x560a435c3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/1213929484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435c3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6bfff640 1 -- 172.21.15.69:0/1213929484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 msgr2=0x560a435bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/1213929484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.172+0000 7f7d6bfff640 1 -- 172.21.15.69:0/1213929484 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a435bd920 con 0x560a435c83f0 2024-08-02T17:49:21.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.173+0000 7f7d6bfff640 1 --2- 172.21.15.69:0/1213929484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5b40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009570 tx=0x7f7d60005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=642591002c1098f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:21.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.174+0000 7f7d517fa640 1 -- 172.21.15.69:0/1213929484 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60033070 con 0x560a435c83f0 2024-08-02T17:49:21.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.174+0000 7f7d517fa640 1 -- 172.21.15.69:0/1213929484 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d60005550 con 0x560a435c83f0 2024-08-02T17:49:21.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.174+0000 7f7d517fa640 1 -- 172.21.15.69:0/1213929484 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60022040 con 0x560a435c83f0 2024-08-02T17:49:21.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.175+0000 7f7d74338740 1 -- 172.21.15.69:0/1213929484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 msgr2=0x560a435c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.175+0000 7f7d74338740 1 --2- 172.21.15.69:0/1213929484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5b40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009570 tx=0x7f7d60005fd0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.175+0000 7f7d74338740 1 -- 172.21.15.69:0/1213929484 shutdown_connections 2024-08-02T17:49:21.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.175+0000 7f7d74338740 1 --2- 172.21.15.69:0/1213929484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c5b40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.175+0000 7f7d74338740 1 --2- 172.21.15.69:0/1213929484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435c3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.175+0000 7f7d74338740 1 --2- 172.21.15.69:0/1213929484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.175+0000 7f7d74338740 1 -- 172.21.15.69:0/1213929484 >> 172.21.15.69:0/1213929484 conn(0x560a435b97a0 msgr2=0x560a435b8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:21.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.175+0000 7f7d74338740 1 -- 172.21.15.69:0/1213929484 shutdown_connections 2024-08-02T17:49:21.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.176+0000 7f7d74338740 1 -- 172.21.15.69:0/1213929484 wait complete. 2024-08-02T17:49:21.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.176+0000 7f7d74338740 1 -- start start 2024-08-02T17:49:21.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.176+0000 7f7d74338740 1 Processor -- start 2024-08-02T17:49:21.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d74338740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d74338740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d74338740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d74338740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5800e6b0 con 0x560a435bd210 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d74338740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5800fb20 con 0x560a435bdb30 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d6bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d6bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:51990/0 (socket says 172.21.15.69:51990) 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d6bfff640 1 -- 172.21.15.69:0/2414340942 learned_addr learned my addr 172.21.15.69:0/2414340942 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5800ed80 con 0x560a435c83f0 2024-08-02T17:49:21.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.177+0000 7f7d74338740 10 client.0 _ll_register_callbacks cb 0x7f7d5c09b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.179+0000 7f7d6affd640 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.179+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.180+0000 7f7d6affd640 1 -- 172.21.15.69:0/2414340942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 msgr2=0x560a435c18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:21.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.180+0000 7f7d6affd640 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.180+0000 7f7d6affd640 1 -- 172.21.15.69:0/2414340942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 msgr2=0x560a435d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.180+0000 7f7d6affd640 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.180+0000 7f7d6affd640 1 -- 172.21.15.69:0/2414340942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a435c1de0 con 0x560a435bdb30 2024-08-02T17:49:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.181+0000 7f7d6affd640 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435d08a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x560a435c4b00 tx=0x7f7d64031500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa56532b89871ffe server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.181+0000 7f7d4affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.181+0000 7f7d4affd640 1 -- 172.21.15.69:0/2414340942 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6404c070 con 0x560a435bdb30 2024-08-02T17:49:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.181+0000 7f7d4affd640 1 -- 172.21.15.69:0/2414340942 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f7d64031cd0 con 0x560a435bdb30 2024-08-02T17:49:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.181+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d60005550 con 0x560a435bdb30 2024-08-02T17:49:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.181+0000 7f7d4affd640 1 -- 172.21.15.69:0/2414340942 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7d64003970 con 0x560a435bdb30 2024-08-02T17:49:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.181+0000 7f7d74338740 10 client.25820 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:21.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.182+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560a43653870 con 0x560a435bdb30 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.182+0000 7f7d4affd640 1 -- 172.21.15.69:0/2414340942 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d64036040 con 0x560a435bdb30 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.182+0000 7f7d4affd640 1 -- 172.21.15.69:0/2414340942 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7d64030070 con 0x560a435bdb30 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.182+0000 7f7d74338740 10 client.25820 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.182+0000 7f7d74338740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.182+0000 7f7d74338740 20 client.25820 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.182+0000 7f7d74338740 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560a43653fc0 0x560a436563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.183+0000 7f7d74338740 4 client.25820 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.183+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7d60005550 con 0x560a43653fc0 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.183+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560a43653fc0 0x560a436563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:21.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.183+0000 7f7d6b7fe640 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560a43653fc0 0x560a436563e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a381646c7d402e server_cookie=f9b44db6f1e6ee2b in_seq=0 out_seq=0 2024-08-02T17:49:21.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.183+0000 7f7d4affd640 10 client.25820 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:21.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.183+0000 7f7d4affd640 1 -- 172.21.15.69:0/2414340942 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f7d60005550 con 0x560a43653fc0 2024-08-02T17:49:21.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.184+0000 7f7d4affd640 10 client.25820 handle_command_reply: tid=0 2024-08-02T17:49:21.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.184+0000 7f7d74338740 1 client.25820 shutdown 2024-08-02T17:49:21.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 msgr2=0x560a435d08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d74338740 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435d08a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x560a435c4b00 tx=0x7f7d64031500 comp rx=0 tx=0).stop 2024-08-02T17:49:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 shutdown_connections 2024-08-02T17:49:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d74338740 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560a43653fc0 0x560a436563e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d74338740 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560a435c83f0 0x560a435c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d74338740 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560a435bdb30 0x560a435d08a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d74338740 1 --2- 172.21.15.69:0/2414340942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560a435bd210 0x560a435d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 >> 172.21.15.69:0/2414340942 conn(0x560a435b97a0 msgr2=0x560a435d31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:21.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.185+0000 7f7d4affd640 0 client.25820 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:21.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.186+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 shutdown_connections 2024-08-02T17:49:21.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.186+0000 7f7d74338740 1 -- 172.21.15.69:0/2414340942 wait complete. 2024-08-02T17:49:21.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.186+0000 7f7d74338740 20 client.25820 trim_cache size 0 max 16384 2024-08-02T17:49:21.188 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:49:21.188 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:49:21.189 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:49:21.189 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:49:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 -- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7d540a2d70 msgr2=0x7f7d540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7d540a2d70 0x7f7d540a5230 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7d60012890 tx=0x7f7d60009f00 comp rx=0 tx=0).stop 2024-08-02T17:49:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 -- 172.21.15.69:0/967607288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c1005f0 msgr2=0x7f7d6c1a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:21.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c1005f0 0x7f7d6c1a45f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00ea60 tx=0x7f7d5c00ef20 comp rx=0 tx=0).stop 2024-08-02T17:49:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 -- 172.21.15.69:0/967607288 shutdown_connections 2024-08-02T17:49:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7d6c100f10 0x7f7d6c1a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7d540a2d70 0x7f7d540a5230 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7d6c105bf0 0x7f7d6c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 --2- 172.21.15.69:0/967607288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7d6c1005f0 0x7f7d6c1a45f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 -- 172.21.15.69:0/967607288 >> 172.21.15.69:0/967607288 conn(0x7f7d6c0fc500 msgr2=0x7f7d6c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 -- 172.21.15.69:0/967607288 shutdown_connections 2024-08-02T17:49:21.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:21.187+0000 7f7d71f3a640 1 -- 172.21.15.69:0/967607288 wait complete. 2024-08-02T17:49:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:21 smithi077 ceph-mon[37108]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 25 MiB/s wr, 1.27k op/s 2024-08-02T17:49:21.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:21 smithi069 ceph-mon[32746]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 25 MiB/s wr, 1.27k op/s 2024-08-02T17:49:21.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:21 smithi083 ceph-mon[37031]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 25 MiB/s wr, 1.27k op/s 2024-08-02T17:49:22.139 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:49:22.140 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:49:22.554 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.104+0000 7f0e133e5640 1 -- 172.21.15.69:0/363361375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c076d70 msgr2=0x7f0e0c112220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.104+0000 7f0e133e5640 1 --2- 172.21.15.69:0/363361375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c076d70 0x7f0e0c112220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009860 tx=0x7f0dfc02a340 comp rx=0 tx=0).stop 2024-08-02T17:49:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.105+0000 7f0e133e5640 1 -- 172.21.15.69:0/363361375 shutdown_connections 2024-08-02T17:49:23.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.105+0000 7f0e133e5640 1 --2- 172.21.15.69:0/363361375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e0c112760 0x7f0e0c114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.105+0000 7f0e133e5640 1 --2- 172.21.15.69:0/363361375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c076d70 0x7f0e0c112220 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009860 tx=0x7f0dfc02a340 comp rx=0 tx=0).stop 2024-08-02T17:49:23.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.105+0000 7f0e133e5640 1 --2- 172.21.15.69:0/363361375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c076450 0x7f0e0c076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.105+0000 7f0e133e5640 1 -- 172.21.15.69:0/363361375 >> 172.21.15.69:0/363361375 conn(0x7f0e0c0ff7e0 msgr2=0x7f0e0c101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:23.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.107+0000 7f0e133e5640 1 -- 172.21.15.69:0/363361375 shutdown_connections 2024-08-02T17:49:23.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.107+0000 7f0e133e5640 1 -- 172.21.15.69:0/363361375 wait complete. 2024-08-02T17:49:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e133e5640 1 Processor -- start 2024-08-02T17:49:23.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e133e5640 1 -- start start 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e133e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c076450 0x7f0e0c1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e133e5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c112760 0x7f0e0c1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e133e5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e0c19e810 0x7f0e0c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e133e5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c1182d0 con 0x7f0e0c19e810 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e133e5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c118150 con 0x7f0e0c112760 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e133e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c118450 con 0x7f0e0c076450 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e11982640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e0c19e810 0x7f0e0c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e11982640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e0c19e810 0x7f0e0c19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35998/0 (socket says 172.21.15.69:35998) 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e11982640 1 -- 172.21.15.69:0/636671699 learned_addr learned my addr 172.21.15.69:0/636671699 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:23.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.108+0000 7f0e11181640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c076450 0x7f0e0c1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e10980640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c112760 0x7f0e0c1a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e11982640 1 -- 172.21.15.69:0/636671699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c076450 msgr2=0x7f0e0c1a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:23.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e11982640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c076450 0x7f0e0c1a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e11982640 1 -- 172.21.15.69:0/636671699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c112760 msgr2=0x7f0e0c1a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e11982640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c112760 0x7f0e0c1a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e11982640 1 -- 172.21.15.69:0/636671699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c19f3f0 con 0x7f0e0c19e810 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e10980640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c112760 0x7f0e0c1a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e11982640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e0c19e810 0x7f0e0c19ecc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400ec50 tx=0x7f0e0400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e027fc640 1 -- 172.21.15.69:0/636671699 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e04003b70 con 0x7f0e0c19e810 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c19f680 con 0x7f0e0c19e810 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e027fc640 1 -- 172.21.15.69:0/636671699 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e04003d10 con 0x7f0e0c19e810 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e027fc640 1 -- 172.21.15.69:0/636671699 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e040109e0 con 0x7f0e0c19e810 2024-08-02T17:49:23.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.109+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c1ab830 con 0x7f0e0c19e810 2024-08-02T17:49:23.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.112+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c077b90 con 0x7f0e0c19e810 2024-08-02T17:49:23.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.115+0000 7f0e027fc640 1 -- 172.21.15.69:0/636671699 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0e04010b80 con 0x7f0e0c19e810 2024-08-02T17:49:23.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.116+0000 7f0e027fc640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0df80a2d70 0x7f0df80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:23.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.116+0000 7f0e027fc640 1 -- 172.21.15.69:0/636671699 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0e040cfe40 con 0x7f0e0c19e810 2024-08-02T17:49:23.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.116+0000 7f0e11181640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0df80a2d70 0x7f0df80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:23.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.118+0000 7f0e11181640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0df80a2d70 0x7f0df80a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009530 tx=0x7f0e08009260 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:23.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.120+0000 7f0e027fc640 1 -- 172.21.15.69:0/636671699 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0e0409c6d0 con 0x7f0e0c19e810 2024-08-02T17:49:23.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:23 smithi077 ceph-mon[37108]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2024-08-02T17:49:23.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:23 smithi069 ceph-mon[32746]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2024-08-02T17:49:23.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.486+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e0c076830 con 0x7f0e0c19e810 2024-08-02T17:49:23.489 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:49:23.489 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:49:23.490 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:49:23.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.488+0000 7f0e027fc640 1 -- 172.21.15.69:0/636671699 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f0e0c076830 con 0x7f0e0c19e810 2024-08-02T17:49:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0df80a2d70 msgr2=0x7f0df80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:23.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0df80a2d70 0x7f0df80a5230 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009530 tx=0x7f0e08009260 comp rx=0 tx=0).stop 2024-08-02T17:49:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e0c19e810 msgr2=0x7f0e0c19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e0c19e810 0x7f0e0c19ecc0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400ec50 tx=0x7f0e0400c6a0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 shutdown_connections 2024-08-02T17:49:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0e0c19e810 0x7f0e0c19ecc0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0df80a2d70 0x7f0df80a5230 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0e0c112760 0x7f0e0c1a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 --2- 172.21.15.69:0/636671699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e0c076450 0x7f0e0c1a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:23.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.491+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 >> 172.21.15.69:0/636671699 conn(0x7f0e0c0ff7e0 msgr2=0x7f0e0c114520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.492+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 shutdown_connections 2024-08-02T17:49:23.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:23.492+0000 7f0e133e5640 1 -- 172.21.15.69:0/636671699 wait complete. 2024-08-02T17:49:23.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:23 smithi083 ceph-mon[37031]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 22 MiB/s wr, 1.30k op/s 2024-08-02T17:49:24.228 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:24 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/636671699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:24.281 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:49:24.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:24 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/636671699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:24.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:24 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/636671699' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:24.712 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:24.740 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-02T17:49:24.741 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.85 2024-08-02T17:49:24.741 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:49:24.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 135.0 s, 1918.6 tps, lat 51.534 ms stddev 57.842 2024-08-02T17:49:25.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:25 smithi077 ceph-mon[37108]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.3 MiB/s wr, 727 op/s 2024-08-02T17:49:25.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:25 smithi069 ceph-mon[32746]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.3 MiB/s wr, 727 op/s 2024-08-02T17:49:25.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:25 smithi083 ceph-mon[37031]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.3 MiB/s wr, 727 op/s 2024-08-02T17:49:26.675 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:49:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.476+0000 7fe98936e640 1 -- 172.21.15.69:0/54194095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 msgr2=0x7fe98411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.476+0000 7fe98936e640 1 --2- 172.21.15.69:0/54194095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 0x7fe98411d260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe974009ca0 tx=0x7fe97402a8c0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.476+0000 7fe98936e640 1 -- 172.21.15.69:0/54194095 shutdown_connections 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.476+0000 7fe98936e640 1 --2- 172.21.15.69:0/54194095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe984109560 0x7fe9841099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.476+0000 7fe98936e640 1 --2- 172.21.15.69:0/54194095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 0x7fe98411d260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.476+0000 7fe98936e640 1 --2- 172.21.15.69:0/54194095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984109180 0x7fe98410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.476+0000 7fe98936e640 1 -- 172.21.15.69:0/54194095 >> 172.21.15.69:0/54194095 conn(0x7fe98406de70 msgr2=0x7fe98406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.478+0000 7fe98936e640 1 -- 172.21.15.69:0/54194095 shutdown_connections 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.478+0000 7fe98936e640 1 -- 172.21.15.69:0/54194095 wait complete. 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.479+0000 7fe98936e640 1 Processor -- start 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.479+0000 7fe98936e640 1 -- start start 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe98936e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe984109180 0x7fe9841a6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe98936e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984109560 0x7fe9841a7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe98936e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 0x7fe9841ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe98936e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe984120ad0 con 0x7fe984109180 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe98936e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe984120950 con 0x7fe98411ae70 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe98936e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe984120c50 con 0x7fe984109560 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9837fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 0x7fe9841ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe982ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe984109180 0x7fe9841a6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9837fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 0x7fe9841ab7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49146/0 (socket says 172.21.15.69:49146) 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe982ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe984109180 0x7fe9841a6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42814/0 (socket says 172.21.15.69:42814) 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9837fe640 1 -- 172.21.15.69:0/171608236 learned_addr learned my addr 172.21.15.69:0/171608236 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9827fc640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984109560 0x7fe9841a7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9827fc640 1 -- 172.21.15.69:0/171608236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 msgr2=0x7fe9841ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9827fc640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 0x7fe9841ab7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9827fc640 1 -- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe984109180 msgr2=0x7fe9841a6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9827fc640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe984109180 0x7fe9841a6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9827fc640 1 -- 172.21.15.69:0/171608236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9841abe80 con 0x7fe984109560 2024-08-02T17:49:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.480+0000 7fe9837fe640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 0x7fe9841ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.481+0000 7fe9827fc640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984109560 0x7fe9841a7410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00e9f0 tx=0x7fe96c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.483+0000 7fe97bfff640 1 -- 172.21.15.69:0/171608236 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00ccc0 con 0x7fe984109560 2024-08-02T17:49:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.483+0000 7fe97bfff640 1 -- 172.21.15.69:0/171608236 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe96c010910 con 0x7fe984109560 2024-08-02T17:49:26.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.483+0000 7fe98936e640 1 -- 172.21.15.69:0/171608236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9841ac170 con 0x7fe984109560 2024-08-02T17:49:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.483+0000 7fe97bfff640 1 -- 172.21.15.69:0/171608236 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c018960 con 0x7fe984109560 2024-08-02T17:49:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.483+0000 7fe98936e640 1 -- 172.21.15.69:0/171608236 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9841b3a50 con 0x7fe984109560 2024-08-02T17:49:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.486+0000 7fe98b76c740 1 Processor -- start 2024-08-02T17:49:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.486+0000 7fe98b76c740 1 -- start start 2024-08-02T17:49:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.486+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.486+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c14c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.486+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c175c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.486+0000 7fe98b76c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564fa9b6fa40 con 0x564fa9c09ff0 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.486+0000 7fe98b76c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564fa9b079d0 con 0x564fa9c0a910 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.486+0000 7fe98b76c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564fa9b6ba00 con 0x564fa9c151d0 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.487+0000 7fe97bfff640 1 -- 172.21.15.69:0/171608236 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe96c010ac0 con 0x7fe984109560 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.487+0000 7fe982ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c14c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.487+0000 7fe9827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c175c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.487+0000 7fe982ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c14c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49148/0 (socket says 172.21.15.69:49148) 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.487+0000 7fe982ffd640 1 -- 172.21.15.69:0/2126315104 learned_addr learned my addr 172.21.15.69:0/2126315104 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:26.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.488+0000 7fe9837fe640 1 --2- 172.21.15.69:0/2126315104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.488+0000 7fe982ffd640 1 -- 172.21.15.69:0/2126315104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 msgr2=0x564fa9c175c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.488+0000 7fe982ffd640 1 --2- 172.21.15.69:0/2126315104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c175c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.488+0000 7fe982ffd640 1 -- 172.21.15.69:0/2126315104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 msgr2=0x564fa9c0a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.488+0000 7fe982ffd640 1 --2- 172.21.15.69:0/2126315104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.488+0000 7fe982ffd640 1 -- 172.21.15.69:0/2126315104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa9b0e650 con 0x564fa9c0a910 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.489+0000 7fe982ffd640 1 --2- 172.21.15.69:0/2126315104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c14c90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009cf0 tx=0x7fe97c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=881e56d15f346294 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.489+0000 7fe97bfff640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9700a2e40 0x7fe9700a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.489+0000 7fe9797fa640 1 -- 172.21.15.69:0/2126315104 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c01e070 con 0x564fa9c0a910 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.489+0000 7fe9837fe640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9700a2e40 0x7fe9700a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.489+0000 7fe97bfff640 1 -- 172.21.15.69:0/171608236 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe96c0cfa40 con 0x7fe984109560 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.489+0000 7fe9797fa640 1 -- 172.21.15.69:0/2126315104 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c019b10 con 0x564fa9c0a910 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.489+0000 7fe9797fa640 1 -- 172.21.15.69:0/2126315104 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00ea60 con 0x564fa9c0a910 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 -- 172.21.15.69:0/2126315104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 msgr2=0x564fa9c14c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe9837fe640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9700a2e40 0x7fe9700a5300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe974009a60 tx=0x7fe974004a60 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 --2- 172.21.15.69:0/2126315104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c14c90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe97c009cf0 tx=0x7fe97c011810 comp rx=0 tx=0).stop 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 -- 172.21.15.69:0/2126315104 shutdown_connections 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 --2- 172.21.15.69:0/2126315104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c175c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 --2- 172.21.15.69:0/2126315104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c14c90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 --2- 172.21.15.69:0/2126315104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 -- 172.21.15.69:0/2126315104 >> 172.21.15.69:0/2126315104 conn(0x564fa9c06580 msgr2=0x564fa9c06960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:26.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 -- 172.21.15.69:0/2126315104 shutdown_connections 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 -- 172.21.15.69:0/2126315104 wait complete. 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.490+0000 7fe98b76c740 1 -- start start 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe98b76c740 1 Processor -- start 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c09ff0 0x564fa9c12110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c12650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c151d0 0x564fa9c12b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe98b76c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564fa9c17da0 con 0x564fa9c151d0 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe98b76c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564fa9b2ab20 con 0x564fa9c09ff0 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe98b76c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564fa9c17f20 con 0x564fa9c0a910 2024-08-02T17:49:26.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe9827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c12650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe9827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c12650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50618/0 (socket says 172.21.15.69:50618) 2024-08-02T17:49:26.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe9827fc640 1 -- 172.21.15.69:0/4250449149 learned_addr learned my addr 172.21.15.69:0/4250449149 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:26.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.491+0000 7fe98b76c740 10 client.0 _ll_register_callbacks cb 0x564fa9b28bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:26.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.492+0000 7fe9827fc640 1 -- 172.21.15.69:0/4250449149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c09ff0 msgr2=0x564fa9c12110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:49:26.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.492+0000 7fe9827fc640 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c09ff0 0x564fa9c12110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.492+0000 7fe9827fc640 1 -- 172.21.15.69:0/4250449149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c151d0 msgr2=0x564fa9c12b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:26.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.492+0000 7fe9827fc640 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c151d0 0x564fa9c12b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.493+0000 7fe9827fc640 1 -- 172.21.15.69:0/4250449149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa98e13c0 con 0x564fa9c0a910 2024-08-02T17:49:26.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.493+0000 7fe9827fc640 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c12650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe96c09b950 tx=0x7fe96c09b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a6dd2cdf69f085e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:26.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.493+0000 7fe96a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:26.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.493+0000 7fe96a7fc640 1 -- 172.21.15.69:0/4250449149 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c09b2a0 con 0x564fa9c0a910 2024-08-02T17:49:26.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.494+0000 7fe98b76c740 1 -- 172.21.15.69:0/4250449149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe97c019b10 con 0x564fa9c0a910 2024-08-02T17:49:26.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.494+0000 7fe96a7fc640 1 -- 172.21.15.69:0/4250449149 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe96c010430 con 0x564fa9c0a910 2024-08-02T17:49:26.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.494+0000 7fe96a7fc640 1 -- 172.21.15.69:0/4250449149 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe96c09ff50 con 0x564fa9c0a910 2024-08-02T17:49:26.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.494+0000 7fe98b76c740 10 client.25705 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:26.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.494+0000 7fe98b76c740 1 -- 172.21.15.69:0/4250449149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564fa9c23a00 con 0x564fa9c0a910 2024-08-02T17:49:26.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.494+0000 7fe96a7fc640 1 -- 172.21.15.69:0/4250449149 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe96c0a4810 con 0x564fa9c0a910 2024-08-02T17:49:26.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.495+0000 7fe96a7fc640 1 -- 172.21.15.69:0/4250449149 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe96c0979a0 con 0x564fa9c0a910 2024-08-02T17:49:26.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.495+0000 7fe98b76c740 10 client.25705 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:26.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.495+0000 7fe98b76c740 10 client.25705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:49:26.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.495+0000 7fe98b76c740 20 client.25705 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:26.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.495+0000 7fe98b76c740 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x564fa9c252e0 0x564fa9c27780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.495+0000 7fe98b76c740 4 client.25705 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:26.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.495+0000 7fe98b76c740 1 -- 172.21.15.69:0/4250449149 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe97c019b10 con 0x564fa9c252e0 2024-08-02T17:49:26.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.495+0000 7fe982ffd640 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x564fa9c252e0 0x564fa9c27780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe982ffd640 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x564fa9c252e0 0x564fa9c27780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28ab7c4c022d7efc server_cookie=b852eaeb363134d6 in_seq=0 out_seq=0 2024-08-02T17:49:26.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe96a7fc640 10 client.25705 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:49:26.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe96a7fc640 1 -- 172.21.15.69:0/4250449149 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe97c019b10 con 0x564fa9c252e0 2024-08-02T17:49:26.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe96a7fc640 10 client.25705 handle_command_reply: tid=0 2024-08-02T17:49:26.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 client.25705 shutdown 2024-08-02T17:49:26.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 -- 172.21.15.69:0/4250449149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 msgr2=0x564fa9c12650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c12650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe96c09b950 tx=0x7fe96c09b790 comp rx=0 tx=0).stop 2024-08-02T17:49:26.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 -- 172.21.15.69:0/4250449149 shutdown_connections 2024-08-02T17:49:26.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x564fa9c252e0 0x564fa9c27780 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c151d0 0x564fa9c12b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe96a7fc640 0 client.25705 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:49:26.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c12650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 --2- 172.21.15.69:0/4250449149 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c09ff0 0x564fa9c12110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.496+0000 7fe98b76c740 1 -- 172.21.15.69:0/4250449149 >> 172.21.15.69:0/4250449149 conn(0x564fa9c06580 msgr2=0x564fa9c0c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:26.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.497+0000 7fe98b76c740 1 -- 172.21.15.69:0/4250449149 shutdown_connections 2024-08-02T17:49:26.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.497+0000 7fe98b76c740 1 -- 172.21.15.69:0/4250449149 wait complete. 2024-08-02T17:49:26.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.498+0000 7fe98b76c740 20 client.25705 trim_cache size 0 max 16384 2024-08-02T17:49:26.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe98b76c740 1 Processor -- start 2024-08-02T17:49:26.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe98b76c740 1 -- start start 2024-08-02T17:49:26.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c129e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c151d0 0x564fa9c2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe98b76c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841c05c0 con 0x564fa9c09ff0 2024-08-02T17:49:26.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe98b76c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841bde90 con 0x564fa9c151d0 2024-08-02T17:49:26.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe98b76c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841bbce0 con 0x564fa9c0a910 2024-08-02T17:49:26.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe9827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c129e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe982ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe9827fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c129e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50632/0 (socket says 172.21.15.69:50632) 2024-08-02T17:49:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.519+0000 7fe9827fc640 1 -- 172.21.15.69:0/2989493199 learned_addr learned my addr 172.21.15.69:0/2989493199 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe9837fe640 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c151d0 0x564fa9c2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe982ffd640 1 -- 172.21.15.69:0/2989493199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 msgr2=0x564fa9c129e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe982ffd640 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c129e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe982ffd640 1 -- 172.21.15.69:0/2989493199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c151d0 msgr2=0x564fa9c2d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:26.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe982ffd640 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c151d0 0x564fa9c2d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe982ffd640 1 -- 172.21.15.69:0/2989493199 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa9c14bc0 con 0x564fa9c09ff0 2024-08-02T17:49:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe9837fe640 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c151d0 0x564fa9c2d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe9827fc640 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.520+0000 7fe982ffd640 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0caa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe97c01c040 tx=0x7fe97c005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c885ee327fb4e30 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.522+0000 7fe978ff9640 1 -- 172.21.15.69:0/2989493199 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c01e040 con 0x564fa9c09ff0 2024-08-02T17:49:26.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.523+0000 7fe978ff9640 1 -- 172.21.15.69:0/2989493199 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c01f040 con 0x564fa9c09ff0 2024-08-02T17:49:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.523+0000 7fe978ff9640 1 -- 172.21.15.69:0/2989493199 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c00e6b0 con 0x564fa9c09ff0 2024-08-02T17:49:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 -- 172.21.15.69:0/2989493199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 msgr2=0x564fa9c0caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0caa0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe97c01c040 tx=0x7fe97c005870 comp rx=0 tx=0).stop 2024-08-02T17:49:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 -- 172.21.15.69:0/2989493199 shutdown_connections 2024-08-02T17:49:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c151d0 0x564fa9c2d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c0a910 0x564fa9c129e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 --2- 172.21.15.69:0/2989493199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c0caa0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 -- 172.21.15.69:0/2989493199 >> 172.21.15.69:0/2989493199 conn(0x564fa9c06580 msgr2=0x564fa9c011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 -- 172.21.15.69:0/2989493199 shutdown_connections 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.524+0000 7fe98b76c740 1 -- 172.21.15.69:0/2989493199 wait complete. 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.525+0000 7fe98b76c740 1 -- start start 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.525+0000 7fe98b76c740 1 Processor -- start 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.525+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c88890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe98b76c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe98b76c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841bc350 con 0x564fa9c09ff0 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe9827fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c88890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe9827fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c88890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49158/0 (socket says 172.21.15.69:49158) 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe9827fc640 1 -- 172.21.15.69:0/1686661589 learned_addr learned my addr 172.21.15.69:0/1686661589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe9837fe640 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe98b76c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841bd7c0 con 0x564fa9c0a910 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9841bca20 con 0x564fa9c151d0 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe98b76c740 10 client.0 _ll_register_callbacks cb 0x7fe96c00ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.526+0000 7fe982ffd640 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.527+0000 7fe9837fe640 1 -- 172.21.15.69:0/1686661589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 msgr2=0x564fa9c88890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:26.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.527+0000 7fe9837fe640 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c88890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.527+0000 7fe9837fe640 1 -- 172.21.15.69:0/1686661589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 msgr2=0x564fa9c88350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.527+0000 7fe9837fe640 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c88350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.527+0000 7fe9837fe640 1 -- 172.21.15.69:0/1686661589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa9c18de0 con 0x564fa9c151d0 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.527+0000 7fe9837fe640 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c186f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe974004af0 tx=0x7fe974003d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ce93d69f5d9c06b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.527+0000 7fe96a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.528+0000 7fe96a7fc640 1 -- 172.21.15.69:0/1686661589 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974004040 con 0x564fa9c151d0 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.528+0000 7fe96a7fc640 1 -- 172.21.15.69:0/1686661589 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe9740041e0 con 0x564fa9c151d0 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.528+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe97c01f040 con 0x564fa9c151d0 2024-08-02T17:49:26.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.528+0000 7fe96a7fc640 1 -- 172.21.15.69:0/1686661589 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe974004420 con 0x564fa9c151d0 2024-08-02T17:49:26.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.528+0000 7fe98b76c740 10 client.25711 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:26.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.528+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564fa9c8fb20 con 0x564fa9c151d0 2024-08-02T17:49:26.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.528+0000 7fe96a7fc640 1 -- 172.21.15.69:0/1686661589 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97c01f040 con 0x564fa9c151d0 2024-08-02T17:49:26.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.528+0000 7fe96a7fc640 1 -- 172.21.15.69:0/1686661589 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe974052020 con 0x564fa9c151d0 2024-08-02T17:49:26.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe98b76c740 10 client.25711 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe98b76c740 10 client.25711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:49:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe98b76c740 20 client.25711 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe98b76c740 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x564fa9c90220 0x564fa9c92640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe98b76c740 4 client.25711 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:49:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fa9b7cfb0 con 0x564fa9c90220 2024-08-02T17:49:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe982ffd640 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x564fa9c90220 0x564fa9c92640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:26.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe982ffd640 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x564fa9c90220 0x564fa9c92640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d67e36ee169b174d server_cookie=9d3eaba486225b in_seq=0 out_seq=0 2024-08-02T17:49:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe96a7fc640 10 client.25711 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:49:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe96a7fc640 1 -- 172.21.15.69:0/1686661589 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fa9b7cfb0 con 0x564fa9c90220 2024-08-02T17:49:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.529+0000 7fe96a7fc640 10 client.25711 handle_command_reply: tid=0 2024-08-02T17:49:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 client.25711 shutdown 2024-08-02T17:49:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 msgr2=0x564fa9c186f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c186f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe974004af0 tx=0x7fe974003d40 comp rx=0 tx=0).stop 2024-08-02T17:49:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 shutdown_connections 2024-08-02T17:49:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x564fa9c90220 0x564fa9c92640 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564fa9c151d0 0x564fa9c186f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564fa9c0a910 0x564fa9c88890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 --2- 172.21.15.69:0/1686661589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564fa9c09ff0 0x564fa9c88350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 >> 172.21.15.69:0/1686661589 conn(0x564fa9c06580 msgr2=0x564fa9c04d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.530+0000 7fe96a7fc640 0 client.25711 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:49:26.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.531+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 shutdown_connections 2024-08-02T17:49:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.531+0000 7fe98b76c740 1 -- 172.21.15.69:0/1686661589 wait complete. 2024-08-02T17:49:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.531+0000 7fe98b76c740 20 client.25711 trim_cache size 0 max 16384 2024-08-02T17:49:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.537+0000 7fe98936e640 1 -- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9700a2e40 msgr2=0x7fe9700a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.537+0000 7fe98936e640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9700a2e40 0x7fe9700a5300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe974009a60 tx=0x7fe974004a60 comp rx=0 tx=0).stop 2024-08-02T17:49:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.537+0000 7fe98936e640 1 -- 172.21.15.69:0/171608236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984109560 msgr2=0x7fe9841a7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:26.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.537+0000 7fe98936e640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984109560 0x7fe9841a7410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00e9f0 tx=0x7fe96c00eeb0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.540+0000 7fe98936e640 1 -- 172.21.15.69:0/171608236 shutdown_connections 2024-08-02T17:49:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.540+0000 7fe98936e640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9700a2e40 0x7fe9700a5300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.540+0000 7fe98936e640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe98411ae70 0x7fe9841ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.540+0000 7fe98936e640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe984109560 0x7fe9841a7410 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.540+0000 7fe98936e640 1 --2- 172.21.15.69:0/171608236 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe984109180 0x7fe9841a6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:26.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.540+0000 7fe98936e640 1 -- 172.21.15.69:0/171608236 >> 172.21.15.69:0/171608236 conn(0x7fe98406de70 msgr2=0x7fe98411bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.543+0000 7fe98936e640 1 -- 172.21.15.69:0/171608236 shutdown_connections 2024-08-02T17:49:26.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:26.543+0000 7fe98936e640 1 -- 172.21.15.69:0/171608236 wait complete. 2024-08-02T17:49:27.146 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:27.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:27 smithi077 ceph-mon[37108]: pgmap v94: 129 pgs: 129 active+clean; 1.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 9.6 MiB/s wr, 971 op/s 2024-08-02T17:49:27.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:27 smithi069 ceph-mon[32746]: pgmap v94: 129 pgs: 129 active+clean; 1.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 9.6 MiB/s wr, 971 op/s 2024-08-02T17:49:27.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:27 smithi083 ceph-mon[37031]: pgmap v94: 129 pgs: 129 active+clean; 1.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 9.6 MiB/s wr, 971 op/s 2024-08-02T17:49:27.669 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:49:28.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.224+0000 7fab3918b640 1 -- 172.21.15.69:0/3445601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab2c0a0f50 msgr2=0x7fab2c0a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.224+0000 7fab3918b640 1 --2- 172.21.15.69:0/3445601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab2c0a0f50 0x7fab2c0a8af0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fab34067690 tx=0x7fab34096fb0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.226+0000 7fab3918b640 1 -- 172.21.15.69:0/3445601 shutdown_connections 2024-08-02T17:49:28.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.226+0000 7fab3918b640 1 --2- 172.21.15.69:0/3445601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab2c0a0f50 0x7fab2c0a8af0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.226+0000 7fab3918b640 1 --2- 172.21.15.69:0/3445601 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab2c09d080 0x7fab2c0a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.226+0000 7fab3918b640 1 --2- 172.21.15.69:0/3445601 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab2c09c6d0 0x7fab2c09cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.226+0000 7fab3918b640 1 -- 172.21.15.69:0/3445601 >> 172.21.15.69:0/3445601 conn(0x7fab2c009f40 msgr2=0x7fab2c00a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:28.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.227+0000 7fab3918b640 1 -- 172.21.15.69:0/3445601 shutdown_connections 2024-08-02T17:49:28.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.232+0000 7fab3918b640 1 -- 172.21.15.69:0/3445601 wait complete. 2024-08-02T17:49:28.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.233+0000 7fab3918b640 1 Processor -- start 2024-08-02T17:49:28.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.233+0000 7fab3918b640 1 -- start start 2024-08-02T17:49:28.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3918b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab2c09c6d0 0x7fab2c014bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3918b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab2c09d080 0x7fab2c015130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3918b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab2c0a0f50 0x7fab2c00de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab33fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab2c09c6d0 0x7fab2c014bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab33fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab2c09c6d0 0x7fab2c014bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42850/0 (socket says 172.21.15.69:42850) 2024-08-02T17:49:28.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3918b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c0abed0 con 0x7fab2c09c6d0 2024-08-02T17:49:28.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3918b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c0abd50 con 0x7fab2c0a0f50 2024-08-02T17:49:28.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3918b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c0ac050 con 0x7fab2c09d080 2024-08-02T17:49:28.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3898a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab2c0a0f50 0x7fab2c00de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3898a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab2c0a0f50 0x7fab2c00de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49180/0 (socket says 172.21.15.69:49180) 2024-08-02T17:49:28.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.234+0000 7fab3898a640 1 -- 172.21.15.69:0/3550461840 learned_addr learned my addr 172.21.15.69:0/3550461840 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:28.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.235+0000 7fab3898a640 1 -- 172.21.15.69:0/3550461840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab2c09d080 msgr2=0x7fab2c015130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.235+0000 7fab3898a640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab2c09d080 0x7fab2c015130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.235+0000 7fab3898a640 1 -- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab2c09c6d0 msgr2=0x7fab2c014bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.235+0000 7fab3898a640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab2c09c6d0 0x7fab2c014bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.235+0000 7fab3898a640 1 -- 172.21.15.69:0/3550461840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c00e700 con 0x7fab2c0a0f50 2024-08-02T17:49:28.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.235+0000 7fab3898a640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab2c0a0f50 0x7fab2c00de40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab34065770 tx=0x7fab340a0b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:28.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.236+0000 7fab33fff640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab2c09c6d0 0x7fab2c014bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-08-02T17:49:28.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.238+0000 7fab317fa640 1 -- 172.21.15.69:0/3550461840 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab340ae070 con 0x7fab2c0a0f50 2024-08-02T17:49:28.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.238+0000 7fab317fa640 1 -- 172.21.15.69:0/3550461840 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab340a98f0 con 0x7fab2c0a0f50 2024-08-02T17:49:28.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.238+0000 7fab317fa640 1 -- 172.21.15.69:0/3550461840 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3409f830 con 0x7fab2c0a0f50 2024-08-02T17:49:28.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.238+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c00e990 con 0x7fab2c0a0f50 2024-08-02T17:49:28.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.238+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c0128f0 con 0x7fab2c0a0f50 2024-08-02T17:49:28.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.240+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c09d480 con 0x7fab2c0a0f50 2024-08-02T17:49:28.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.243+0000 7fab317fa640 1 -- 172.21.15.69:0/3550461840 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fab340b1070 con 0x7fab2c0a0f50 2024-08-02T17:49:28.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.244+0000 7fab317fa640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fab240a2f00 0x7fab240a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.244+0000 7fab317fa640 1 -- 172.21.15.69:0/3550461840 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fab3415bb90 con 0x7fab2c0a0f50 2024-08-02T17:49:28.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.247+0000 7fab317fa640 1 -- 172.21.15.69:0/3550461840 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fab34128520 con 0x7fab2c0a0f50 2024-08-02T17:49:28.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.250+0000 7fab33fff640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fab240a2f00 0x7fab240a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.251+0000 7fab33fff640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fab240a2f00 0x7fab240a53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab28009b40 tx=0x7fab28005e50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:28.305 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:28.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.608+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab2c013370 con 0x7fab2c0a0f50 2024-08-02T17:49:28.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.613+0000 7fab317fa640 1 -- 172.21.15.69:0/3550461840 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fab34127e20 con 0x7fab2c0a0f50 2024-08-02T17:49:28.619 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:49:28.622 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:49:28.622 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:49:28.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.622+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fab240a2f00 msgr2=0x7fab240a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.622+0000 7fab3918b640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fab240a2f00 0x7fab240a53c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fab28009b40 tx=0x7fab28005e50 comp rx=0 tx=0).stop 2024-08-02T17:49:28.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.622+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab2c0a0f50 msgr2=0x7fab2c00de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.622+0000 7fab3918b640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab2c0a0f50 0x7fab2c00de40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fab34065770 tx=0x7fab340a0b00 comp rx=0 tx=0).stop 2024-08-02T17:49:28.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.628+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 shutdown_connections 2024-08-02T17:49:28.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.628+0000 7fab3918b640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fab240a2f00 0x7fab240a53c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.628+0000 7fab3918b640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fab2c0a0f50 0x7fab2c00de40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.628+0000 7fab3918b640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab2c09d080 0x7fab2c015130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.628+0000 7fab3918b640 1 --2- 172.21.15.69:0/3550461840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fab2c09c6d0 0x7fab2c014bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.628+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 >> 172.21.15.69:0/3550461840 conn(0x7fab2c009f40 msgr2=0x7fab2c09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:28.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.628+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 shutdown_connections 2024-08-02T17:49:28.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.628+0000 7fab3918b640 1 -- 172.21.15.69:0/3550461840 wait complete. 2024-08-02T17:49:28.843 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:28 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3550461840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:28.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.860+0000 7f210e757640 1 -- 172.21.15.69:0/2244965635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2108100780 msgr2=0x7f2108105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.860+0000 7f210e757640 1 --2- 172.21.15.69:0/2244965635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2108100780 0x7f2108105510 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009900 tx=0x7f20f802a420 comp rx=0 tx=0).stop 2024-08-02T17:49:28.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.861+0000 7f210e757640 1 -- 172.21.15.69:0/2244965635 shutdown_connections 2024-08-02T17:49:28.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.861+0000 7f210e757640 1 --2- 172.21.15.69:0/2244965635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2108100780 0x7f2108105510 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.861+0000 7f210e757640 1 --2- 172.21.15.69:0/2244965635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21080ffdc0 0x7f2108100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.861+0000 7f210e757640 1 --2- 172.21.15.69:0/2244965635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21081068a0 0x7f2108106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.861+0000 7f210e757640 1 -- 172.21.15.69:0/2244965635 >> 172.21.15.69:0/2244965635 conn(0x7f2108076f40 msgr2=0x7f21080fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:28.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.861+0000 7f210e757640 1 -- 172.21.15.69:0/2244965635 shutdown_connections 2024-08-02T17:49:28.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.862+0000 7f210e757640 1 -- 172.21.15.69:0/2244965635 wait complete. 2024-08-02T17:49:28.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.862+0000 7f210e757640 1 Processor -- start 2024-08-02T17:49:28.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.863+0000 7f210e757640 1 -- start start 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.863+0000 7f210e757640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21080ffdc0 0x7f21081152a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.863+0000 7f210e757640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2108100780 0x7f21081157e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.863+0000 7f210e757640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21081068a0 0x7f210810e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.863+0000 7f210e757640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2108118390 con 0x7f21081068a0 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.863+0000 7f210e757640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2108118210 con 0x7f2108100780 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.863+0000 7f210e757640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2108118510 con 0x7f21080ffdc0 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.864+0000 7f21077fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2108100780 0x7f21081157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.864+0000 7f2107fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21080ffdc0 0x7f21081152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.864+0000 7f21077fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2108100780 0x7f21081157e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49202/0 (socket says 172.21.15.69:49202) 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.864+0000 7f21077fe640 1 -- 172.21.15.69:0/1372873032 learned_addr learned my addr 172.21.15.69:0/1372873032 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.864+0000 7f210ccf4640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21081068a0 0x7f210810e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.865+0000 7f21077fe640 1 -- 172.21.15.69:0/1372873032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21080ffdc0 msgr2=0x7f21081152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.865+0000 7f21077fe640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21080ffdc0 0x7f21081152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.865+0000 7f21077fe640 1 -- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21081068a0 msgr2=0x7f210810e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.865+0000 7f21077fe640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21081068a0 0x7f210810e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.865+0000 7f21077fe640 1 -- 172.21.15.69:0/1372873032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210810edd0 con 0x7f2108100780 2024-08-02T17:49:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.865+0000 7f21077fe640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2108100780 0x7f21081157e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b450 tx=0x7f20fc00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.865+0000 7f21057fa640 1 -- 172.21.15.69:0/1372873032 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc015020 con 0x7f2108100780 2024-08-02T17:49:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.866+0000 7f210e757640 1 -- 172.21.15.69:0/1372873032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210810f0c0 con 0x7f2108100780 2024-08-02T17:49:28.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.866+0000 7f210e757640 1 -- 172.21.15.69:0/1372873032 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081130e0 con 0x7f2108100780 2024-08-02T17:49:28.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.869+0000 7f21057fa640 1 -- 172.21.15.69:0/1372873032 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20fc011d70 con 0x7f2108100780 2024-08-02T17:49:28.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.869+0000 7f21057fa640 1 -- 172.21.15.69:0/1372873032 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00f340 con 0x7f2108100780 2024-08-02T17:49:28.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.872+0000 7f21057fa640 1 -- 172.21.15.69:0/1372873032 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f20fc018020 con 0x7f2108100780 2024-08-02T17:49:28.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.873+0000 7f2110b55740 1 Processor -- start 2024-08-02T17:49:28.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.873+0000 7f2110b55740 1 -- start start 2024-08-02T17:49:28.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f21057fa640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f00a2d40 0x7f20f00a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f210ccf4640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f00a2d40 0x7f20f00a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f21057fa640 1 -- 172.21.15.69:0/1372873032 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f20fc0cf110 con 0x7f2108100780 2024-08-02T17:49:28.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2110b55740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2dfff0 0x55a86b2e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2110b55740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 0x55a86b2eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2110b55740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2eb1d0 0x55a86b2ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2110b55740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a86b245a40 con 0x55a86b2dfff0 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2110b55740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a86b1dd9d0 con 0x55a86b2e0910 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2110b55740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a86b241a00 con 0x55a86b2eb1d0 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2107fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2dfff0 0x55a86b2e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2107fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2dfff0 0x55a86b2e03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42880/0 (socket says 172.21.15.69:42880) 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.874+0000 7f2107fff640 1 -- 172.21.15.69:0/527049681 learned_addr learned my addr 172.21.15.69:0/527049681 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f210ccf4640 1 --2- 172.21.15.69:0/527049681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2eb1d0 0x55a86b2ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f2107fff640 1 -- 172.21.15.69:0/527049681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2eb1d0 msgr2=0x55a86b2ed5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f2107fff640 1 --2- 172.21.15.69:0/527049681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2eb1d0 0x55a86b2ed5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f2107fff640 1 -- 172.21.15.69:0/527049681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 msgr2=0x55a86b2eac90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:49:28.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f2107fff640 1 --2- 172.21.15.69:0/527049681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 0x55a86b2eac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f2107fff640 1 -- 172.21.15.69:0/527049681 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86b1e4650 con 0x55a86b2dfff0 2024-08-02T17:49:28.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f2107fff640 1 --2- 172.21.15.69:0/527049681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2dfff0 0x55a86b2e03d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f210000a4b0 tx=0x7f21000096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e8150fade05f72 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:28.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f210ccf4640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f00a2d40 0x7f20f00a5200 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005ab0 tx=0x7f20f80041c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:28.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.875+0000 7f20ee7fc640 1 -- 172.21.15.69:0/527049681 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100018020 con 0x55a86b2dfff0 2024-08-02T17:49:28.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.876+0000 7f20ee7fc640 1 -- 172.21.15.69:0/527049681 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100014cf0 con 0x55a86b2dfff0 2024-08-02T17:49:28.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.876+0000 7f20ee7fc640 1 -- 172.21.15.69:0/527049681 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100013d90 con 0x55a86b2dfff0 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.876+0000 7f2110b55740 1 -- 172.21.15.69:0/527049681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2dfff0 msgr2=0x55a86b2e03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.876+0000 7f2110b55740 1 --2- 172.21.15.69:0/527049681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2dfff0 0x55a86b2e03d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f210000a4b0 tx=0x7f21000096d0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 -- 172.21.15.69:0/527049681 shutdown_connections 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 --2- 172.21.15.69:0/527049681 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2eb1d0 0x55a86b2ed5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 --2- 172.21.15.69:0/527049681 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 0x55a86b2eac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 --2- 172.21.15.69:0/527049681 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2dfff0 0x55a86b2e03d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 -- 172.21.15.69:0/527049681 >> 172.21.15.69:0/527049681 conn(0x55a86b2dc580 msgr2=0x55a86b2dc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 -- 172.21.15.69:0/527049681 shutdown_connections 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 -- 172.21.15.69:0/527049681 wait complete. 2024-08-02T17:49:28.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 -- start start 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.877+0000 7f2110b55740 1 Processor -- start 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2110b55740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2110b55740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 0x55a86b2e8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2110b55740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2eb1d0 0x55a86b2e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2107fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2107fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50680/0 (socket says 172.21.15.69:50680) 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2107fff640 1 -- 172.21.15.69:0/2581114929 learned_addr learned my addr 172.21.15.69:0/2581114929 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a86b2edda0 con 0x55a86b2eb1d0 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a86b200b20 con 0x55a86b2e0910 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a86b2edf20 con 0x55a86b2dfff0 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f21077fe640 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 0x55a86b2e8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f2110b55740 10 client.0 _ll_register_callbacks cb 0x55a86b1febb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:28.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f210ccf4640 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2eb1d0 0x55a86b2e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f21077fe640 1 -- 172.21.15.69:0/2581114929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 msgr2=0x55a86b2e8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f21077fe640 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f21077fe640 1 -- 172.21.15.69:0/2581114929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2eb1d0 msgr2=0x55a86b2e8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f21077fe640 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2eb1d0 0x55a86b2e8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f21077fe640 1 -- 172.21.15.69:0/2581114929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86afb73c0 con 0x55a86b2e0910 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f21077fe640 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 0x55a86b2e8570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0cc5b0 tx=0x7f20fc09ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1f428cd017809b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.878+0000 7f20e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.879+0000 7f20e77fe640 1 -- 172.21.15.69:0/2581114929 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0b4060 con 0x55a86b2e0910 2024-08-02T17:49:28.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.879+0000 7f20e77fe640 1 -- 172.21.15.69:0/2581114929 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f20fc0a3b60 con 0x55a86b2e0910 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.879+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2100014cf0 con 0x55a86b2e0910 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.879+0000 7f20e77fe640 1 -- 172.21.15.69:0/2581114929 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20fc0b9050 con 0x55a86b2e0910 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.879+0000 7f2110b55740 10 client.25850 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.879+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a86b2f99a0 con 0x55a86b2e0910 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.879+0000 7f20e77fe640 1 -- 172.21.15.69:0/2581114929 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100014cf0 con 0x55a86b2e0910 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.879+0000 7f20e77fe640 1 -- 172.21.15.69:0/2581114929 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f20fc07e020 con 0x55a86b2e0910 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.880+0000 7f2110b55740 10 client.25850 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.880+0000 7f2110b55740 10 client.25850 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.880+0000 7f2110b55740 20 client.25850 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.880+0000 7f2110b55740 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a86b2faf00 0x55a86b2fd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.880+0000 7f2110b55740 4 client.25850 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.880+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a86b252fb0 con 0x55a86b2faf00 2024-08-02T17:49:28.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.880+0000 7f2107fff640 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a86b2faf00 0x55a86b2fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.881+0000 7f2107fff640 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a86b2faf00 0x55a86b2fd310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b056793f3920055 server_cookie=e113e99ac04f4f2b in_seq=0 out_seq=0 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.881+0000 7f20e77fe640 10 client.25850 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f20e77fe640 1 -- 172.21.15.69:0/2581114929 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a86b252fb0 con 0x55a86b2faf00 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f20e77fe640 10 client.25850 handle_command_reply: tid=0 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 client.25850 shutdown 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 msgr2=0x55a86b2e8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 0x55a86b2e8570 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0cc5b0 tx=0x7f20fc09ba80 comp rx=0 tx=0).stop 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 shutdown_connections 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a86b2faf00 0x55a86b2fd310 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2eb1d0 0x55a86b2e8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2e0910 0x55a86b2e8570 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f20e77fe640 0 client.25850 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:49:28.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 --2- 172.21.15.69:0/2581114929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.882+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 >> 172.21.15.69:0/2581114929 conn(0x55a86b2dc580 msgr2=0x55a86b2e22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:28.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.883+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 shutdown_connections 2024-08-02T17:49:28.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.883+0000 7f2110b55740 1 -- 172.21.15.69:0/2581114929 wait complete. 2024-08-02T17:49:28.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.883+0000 7f2110b55740 20 client.25850 trim_cache size 0 max 16384 2024-08-02T17:49:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.904+0000 7f2110b55740 1 Processor -- start 2024-08-02T17:49:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.904+0000 7f2110b55740 1 -- start start 2024-08-02T17:49:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2110b55740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2110b55740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2db5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2110b55740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 0x55a86b302c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2110b55740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20f40127f0 con 0x55a86b2e0910 2024-08-02T17:49:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2110b55740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20f40100c0 con 0x55a86b2eb1d0 2024-08-02T17:49:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2110b55740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20f400df10 con 0x55a86b2dfff0 2024-08-02T17:49:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f21077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2db5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2107fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f21077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2db5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42892/0 (socket says 172.21.15.69:42892) 2024-08-02T17:49:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f21077fe640 1 -- 172.21.15.69:0/987348897 learned_addr learned my addr 172.21.15.69:0/987348897 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:28.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f210ccf4640 1 --2- 172.21.15.69:0/987348897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 0x55a86b302c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2107fff640 1 -- 172.21.15.69:0/987348897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 msgr2=0x55a86b302c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2107fff640 1 --2- 172.21.15.69:0/987348897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 0x55a86b302c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2107fff640 1 -- 172.21.15.69:0/987348897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 msgr2=0x55a86b2db5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2107fff640 1 --2- 172.21.15.69:0/987348897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2db5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.905+0000 7f2107fff640 1 -- 172.21.15.69:0/987348897 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a86b2f70c0 con 0x55a86b2dfff0 2024-08-02T17:49:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.906+0000 7f2107fff640 1 --2- 172.21.15.69:0/987348897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e2aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f210000a480 tx=0x7f210000c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5fdc534174c965 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:28.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.906+0000 7f20edffb640 1 -- 172.21.15.69:0/987348897 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100018040 con 0x55a86b2dfff0 2024-08-02T17:49:28.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.906+0000 7f20edffb640 1 -- 172.21.15.69:0/987348897 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100013810 con 0x55a86b2dfff0 2024-08-02T17:49:28.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.906+0000 7f20edffb640 1 -- 172.21.15.69:0/987348897 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100014b20 con 0x55a86b2dfff0 2024-08-02T17:49:28.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.907+0000 7f2110b55740 1 -- 172.21.15.69:0/987348897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 msgr2=0x55a86b2e2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.907+0000 7f2110b55740 1 --2- 172.21.15.69:0/987348897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e2aa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f210000a480 tx=0x7f210000c040 comp rx=0 tx=0).stop 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.907+0000 7f2110b55740 1 -- 172.21.15.69:0/987348897 shutdown_connections 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.907+0000 7f2110b55740 1 --2- 172.21.15.69:0/987348897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2db5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.907+0000 7f2110b55740 1 --2- 172.21.15.69:0/987348897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 0x55a86b302c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.907+0000 7f2110b55740 1 --2- 172.21.15.69:0/987348897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2e2aa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.907+0000 7f2110b55740 1 -- 172.21.15.69:0/987348897 >> 172.21.15.69:0/987348897 conn(0x55a86b2dc580 msgr2=0x55a86b2d71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.908+0000 7f2110b55740 1 -- 172.21.15.69:0/987348897 shutdown_connections 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.908+0000 7f2110b55740 1 -- 172.21.15.69:0/987348897 wait complete. 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.908+0000 7f2110b55740 1 -- start start 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.909+0000 7f2110b55740 1 Processor -- start 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.909+0000 7f2110b55740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.909+0000 7f2110b55740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2f3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.909+0000 7f2110b55740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 0x55a86b2e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.909+0000 7f2110b55740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f20f400e580 con 0x55a86b2e0910 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2110b55740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f20f400f9f0 con 0x55a86b2eb1d0 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2110b55740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f20f400ec50 con 0x55a86b2dfff0 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2110b55740 10 client.0 _ll_register_callbacks cb 0x7f20fc097620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f21077fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50700/0 (socket says 172.21.15.69:50700) 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 -- 172.21.15.69:0/364599159 learned_addr learned my addr 172.21.15.69:0/364599159 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f210ccf4640 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 0x55a86b2e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 -- 172.21.15.69:0/364599159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 msgr2=0x55a86b2e4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 0x55a86b2e4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 -- 172.21.15.69:0/364599159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 msgr2=0x55a86b2f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 -- 172.21.15.69:0/364599159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a86b2e4cf0 con 0x55a86b2dfff0 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f2107fff640 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2f33e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2100004b90 tx=0x7f2100013bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a842bd12ca0dd8a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f20e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f20e77fe640 1 -- 172.21.15.69:0/364599159 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100018040 con 0x55a86b2dfff0 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.910+0000 7f20e77fe640 1 -- 172.21.15.69:0/364599159 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2100014720 con 0x55a86b2dfff0 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 1 -- 172.21.15.69:0/364599159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2100013810 con 0x55a86b2dfff0 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f20e77fe640 1 -- 172.21.15.69:0/364599159 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2100009c60 con 0x55a86b2dfff0 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 10 client.25732 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:28.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 1 -- 172.21.15.69:0/364599159 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a86b365550 con 0x55a86b2dfff0 2024-08-02T17:49:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f20e77fe640 1 -- 172.21.15.69:0/364599159 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100013810 con 0x55a86b2dfff0 2024-08-02T17:49:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f20e77fe640 1 -- 172.21.15.69:0/364599159 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2100027040 con 0x55a86b2dfff0 2024-08-02T17:49:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 10 client.25732 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 10 client.25732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:49:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 20 client.25732 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:28.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a86b3659c0 0x55a86b367db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 4 client.25732 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f2110b55740 1 -- 172.21.15.69:0/364599159 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a86b24d4c0 con 0x55a86b3659c0 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.911+0000 7f21077fe640 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a86b3659c0 0x55a86b367db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.912+0000 7f20e77fe640 10 client.25732 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.912+0000 7f21077fe640 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a86b3659c0 0x55a86b367db0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e683f288278661 server_cookie=564a723f09fca252 in_seq=0 out_seq=0 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.912+0000 7f20e77fe640 1 -- 172.21.15.69:0/364599159 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a86b24d4c0 con 0x55a86b3659c0 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.912+0000 7f20e77fe640 10 client.25732 handle_command_reply: tid=0 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.912+0000 7f2110b55740 1 client.25732 shutdown 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 -- 172.21.15.69:0/364599159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 msgr2=0x55a86b2f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2f33e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2100004b90 tx=0x7f2100013bf0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 -- 172.21.15.69:0/364599159 shutdown_connections 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a86b3659c0 0x55a86b367db0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f20e77fe640 0 client.25732 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a86b2e0910 0x55a86b2f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a86b2eb1d0 0x55a86b2e4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 --2- 172.21.15.69:0/364599159 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a86b2dfff0 0x55a86b2f33e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 -- 172.21.15.69:0/364599159 >> 172.21.15.69:0/364599159 conn(0x55a86b2dc580 msgr2=0x55a86b2db5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 -- 172.21.15.69:0/364599159 shutdown_connections 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 1 -- 172.21.15.69:0/364599159 wait complete. 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.913+0000 7f2110b55740 20 client.25732 trim_cache size 0 max 16384 2024-08-02T17:49:28.920 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:49:28.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 -- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f00a2d40 msgr2=0x7f20f00a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f00a2d40 0x7f20f00a5200 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f20f8005ab0 tx=0x7f20f80041c0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 -- 172.21.15.69:0/1372873032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2108100780 msgr2=0x7f21081157e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2108100780 0x7f21081157e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b450 tx=0x7f20fc00b910 comp rx=0 tx=0).stop 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 -- 172.21.15.69:0/1372873032 shutdown_connections 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21081068a0 0x7f210810e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f00a2d40 0x7f20f00a5200 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2108100780 0x7f21081157e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 --2- 172.21.15.69:0/1372873032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21080ffdc0 0x7f21081152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 -- 172.21.15.69:0/1372873032 >> 172.21.15.69:0/1372873032 conn(0x7f2108076f40 msgr2=0x7f21081034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 -- 172.21.15.69:0/1372873032 shutdown_connections 2024-08-02T17:49:28.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:28.914+0000 7f210e757640 1 -- 172.21.15.69:0/1372873032 wait complete. 2024-08-02T17:49:29.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:28 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3550461840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:29.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:28 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3550461840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.366+0000 7f37be1d0640 1 -- 172.21.15.69:0/761664169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b8107800 msgr2=0x7f37b8107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.366+0000 7f37be1d0640 1 --2- 172.21.15.69:0/761664169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b8107800 0x7f37b8107be0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009900 tx=0x7f37a802a400 comp rx=0 tx=0).stop 2024-08-02T17:49:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.367+0000 7f37be1d0640 1 -- 172.21.15.69:0/761664169 shutdown_connections 2024-08-02T17:49:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.367+0000 7f37be1d0640 1 --2- 172.21.15.69:0/761664169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b81017f0 0x7f37b8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.367+0000 7f37be1d0640 1 --2- 172.21.15.69:0/761664169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b8102190 0x7f37b810e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.367+0000 7f37be1d0640 1 --2- 172.21.15.69:0/761664169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b8107800 0x7f37b8107be0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.367+0000 7f37be1d0640 1 -- 172.21.15.69:0/761664169 >> 172.21.15.69:0/761664169 conn(0x7f37b80fd510 msgr2=0x7f37b80ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.367+0000 7f37be1d0640 1 -- 172.21.15.69:0/761664169 shutdown_connections 2024-08-02T17:49:29.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.367+0000 7f37be1d0640 1 -- 172.21.15.69:0/761664169 wait complete. 2024-08-02T17:49:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.367+0000 7f37be1d0640 1 Processor -- start 2024-08-02T17:49:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37be1d0640 1 -- start start 2024-08-02T17:49:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37be1d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b81017f0 0x7f37b8074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37be1d0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8102190 0x7f37b8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37be1d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b8107800 0x7f37b8074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37be1d0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111c90 con 0x7f37b8102190 2024-08-02T17:49:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37be1d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111b10 con 0x7f37b81017f0 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37be1d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8111e10 con 0x7f37b8107800 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b81017f0 0x7f37b8074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8102190 0x7f37b8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b81017f0 0x7f37b8074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49256/0 (socket says 172.21.15.69:49256) 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b77fe640 1 -- 172.21.15.69:0/93696070 learned_addr learned my addr 172.21.15.69:0/93696070 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b7fff640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b8107800 0x7f37b8074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b77fe640 1 -- 172.21.15.69:0/93696070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b8107800 msgr2=0x7f37b8074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b77fe640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b8107800 0x7f37b8074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b77fe640 1 -- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8102190 msgr2=0x7f37b8074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.368+0000 7f37b77fe640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8102190 0x7f37b8074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.369+0000 7f37b77fe640 1 -- 172.21.15.69:0/93696070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b81aa8a0 con 0x7f37b81017f0 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.369+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8102190 0x7f37b8074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.369+0000 7f37b77fe640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b81017f0 0x7f37b8074220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009850 tx=0x7f37a8030640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.369+0000 7f37b4ff9640 1 -- 172.21.15.69:0/93696070 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a803f070 con 0x7f37b81017f0 2024-08-02T17:49:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.369+0000 7f37b4ff9640 1 -- 172.21.15.69:0/93696070 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a8004460 con 0x7f37b81017f0 2024-08-02T17:49:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.369+0000 7f37be1d0640 1 -- 172.21.15.69:0/93696070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b81aaa70 con 0x7f37b81017f0 2024-08-02T17:49:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.369+0000 7f37be1d0640 1 -- 172.21.15.69:0/93696070 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81aafb0 con 0x7f37b81017f0 2024-08-02T17:49:29.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.369+0000 7f37b4ff9640 1 -- 172.21.15.69:0/93696070 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8035030 con 0x7f37b81017f0 2024-08-02T17:49:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.374+0000 7f37b4ff9640 1 -- 172.21.15.69:0/93696070 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f37a8042070 con 0x7f37b81017f0 2024-08-02T17:49:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.374+0000 7f37b4ff9640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37a00a2dc0 0x7f37a00a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37c1002740 1 Processor -- start 2024-08-02T17:49:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37c1002740 1 -- start start 2024-08-02T17:49:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37a00a2dc0 0x7f37a00a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b4ff9640 1 -- 172.21.15.69:0/93696070 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f37a80effa0 con 0x7f37b81017f0 2024-08-02T17:49:29.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37c1002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e9927b0 0x56337e992b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37c1002740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 0x56337e99d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37c1002740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e99d970 0x56337e99fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37c1002740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56337e8f69b0 con 0x56337e9930d0 2024-08-02T17:49:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37c1002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56337e89e800 con 0x56337e9927b0 2024-08-02T17:49:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37c1002740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56337e8f2970 con 0x56337e99d970 2024-08-02T17:49:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e9927b0 0x56337e992b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 0x56337e99d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e9927b0 0x56337e992b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49268/0 (socket says 172.21.15.69:49268) 2024-08-02T17:49:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 0x56337e99d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42932/0 (socket says 172.21.15.69:42932) 2024-08-02T17:49:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b77fe640 1 -- 172.21.15.69:0/2444157091 learned_addr learned my addr 172.21.15.69:0/2444157091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b7fff640 1 --2- 172.21.15.69:0/2444157091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e99d970 0x56337e99fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b77fe640 1 -- 172.21.15.69:0/2444157091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e99d970 msgr2=0x56337e99fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b77fe640 1 --2- 172.21.15.69:0/2444157091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e99d970 0x56337e99fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b77fe640 1 -- 172.21.15.69:0/2444157091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 msgr2=0x56337e99d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b77fe640 1 --2- 172.21.15.69:0/2444157091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 0x56337e99d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b77fe640 1 -- 172.21.15.69:0/2444157091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337e60a210 con 0x56337e9927b0 2024-08-02T17:49:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.375+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37a00a2dc0 0x7f37a00a5280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f37b80759e0 tx=0x7f37ac013c30 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.376+0000 7f37b77fe640 1 --2- 172.21.15.69:0/2444157091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e9927b0 0x56337e992b90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37a80b8610 tx=0x7f37a80bbdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=983cd1dba0faa54c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.376+0000 7f379dffb640 1 -- 172.21.15.69:0/2444157091 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8049070 con 0x56337e9927b0 2024-08-02T17:49:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.376+0000 7f379dffb640 1 -- 172.21.15.69:0/2444157091 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a80c04b0 con 0x56337e9927b0 2024-08-02T17:49:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.376+0000 7f379dffb640 1 -- 172.21.15.69:0/2444157091 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80c48e0 con 0x56337e9927b0 2024-08-02T17:49:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 -- 172.21.15.69:0/2444157091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e9927b0 msgr2=0x56337e992b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 --2- 172.21.15.69:0/2444157091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e9927b0 0x56337e992b90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37a80b8610 tx=0x7f37a80bbdb0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 -- 172.21.15.69:0/2444157091 shutdown_connections 2024-08-02T17:49:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 --2- 172.21.15.69:0/2444157091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 0x56337e99d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 --2- 172.21.15.69:0/2444157091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e99d970 0x56337e99fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 --2- 172.21.15.69:0/2444157091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e9927b0 0x56337e992b90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 -- 172.21.15.69:0/2444157091 >> 172.21.15.69:0/2444157091 conn(0x56337e98ed20 msgr2=0x56337e98f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 -- 172.21.15.69:0/2444157091 shutdown_connections 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 -- 172.21.15.69:0/2444157091 wait complete. 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.377+0000 7f37c1002740 1 -- start start 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.378+0000 7f37c1002740 1 Processor -- start 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.378+0000 7f37c1002740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e99a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.378+0000 7f37c1002740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e99ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.378+0000 7f37c1002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e99b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.378+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e99a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.378+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e99a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42946/0 (socket says 172.21.15.69:42946) 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.378+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e99ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.378+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e99ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50742/0 (socket says 172.21.15.69:50742) 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37b77fe640 1 -- 172.21.15.69:0/2567290009 learned_addr learned my addr 172.21.15.69:0/2567290009 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56337e9a0300 con 0x56337e9927b0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56337e8b2500 con 0x56337e99d970 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37b7fff640 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e99b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56337e9a0480 con 0x56337e9930d0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37c1002740 10 client.0 _ll_register_callbacks cb 0x56337e609a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37b6ffd640 1 -- 172.21.15.69:0/2567290009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 msgr2=0x56337e99b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e99b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37b6ffd640 1 -- 172.21.15.69:0/2567290009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 msgr2=0x56337e99a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e99a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.379+0000 7f37b6ffd640 1 -- 172.21.15.69:0/2567290009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337e6d6620 con 0x56337e9930d0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.380+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e99ae00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x56337e98fb80 tx=0x7f37ac004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6acf9cba1d383192 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.381+0000 7f3796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.381+0000 7f3796ffd640 1 -- 172.21.15.69:0/2567290009 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac003d80 con 0x56337e9930d0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.381+0000 7f3796ffd640 1 -- 172.21.15.69:0/2567290009 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f37ac010430 con 0x56337e9930d0 2024-08-02T17:49:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.381+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a80c04b0 con 0x56337e9930d0 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f3796ffd640 1 -- 172.21.15.69:0/2567290009 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37ac010630 con 0x56337e9930d0 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f37c1002740 10 client.25744 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56337e9abf90 con 0x56337e9930d0 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f3796ffd640 1 -- 172.21.15.69:0/2567290009 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a80c04b0 con 0x56337e9930d0 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f3796ffd640 1 -- 172.21.15.69:0/2567290009 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37ac01f070 con 0x56337e9930d0 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f37c1002740 10 client.25744 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f37c1002740 10 client.25744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f37c1002740 20 client.25744 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f37c1002740 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56337e9ad4a0 0x56337e9af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f37c1002740 4 client.25744 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:29.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.382+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56337e9040e0 con 0x56337e9ad4a0 2024-08-02T17:49:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.383+0000 7f37b77fe640 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56337e9ad4a0 0x56337e9af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.383+0000 7f3796ffd640 10 client.25744 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.383+0000 7f37b77fe640 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56337e9ad4a0 0x56337e9af890 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8cebe8eb7490d18 server_cookie=4c7d960d60179d76 in_seq=0 out_seq=0 2024-08-02T17:49:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.384+0000 7f3796ffd640 1 -- 172.21.15.69:0/2567290009 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56337e9040e0 con 0x56337e9ad4a0 2024-08-02T17:49:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.384+0000 7f3796ffd640 10 client.25744 handle_command_reply: tid=0 2024-08-02T17:49:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.384+0000 7f37c1002740 1 client.25744 shutdown 2024-08-02T17:49:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.384+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 msgr2=0x56337e99ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.391 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.384+0000 7f37c1002740 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e99ae00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x56337e98fb80 tx=0x7f37ac004730 comp rx=0 tx=0).stop 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 shutdown_connections 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56337e9ad4a0 0x56337e9af890 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f3796ffd640 0 client.25744 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e99b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e99ae00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 1 --2- 172.21.15.69:0/2567290009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e99a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 >> 172.21.15.69:0/2567290009 conn(0x56337e98ed20 msgr2=0x56337e994a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 shutdown_connections 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 1 -- 172.21.15.69:0/2567290009 wait complete. 2024-08-02T17:49:29.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.385+0000 7f37c1002740 20 client.25744 trim_cache size 0 max 16384 2024-08-02T17:49:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.411+0000 7f37c1002740 1 Processor -- start 2024-08-02T17:49:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.412+0000 7f37c1002740 1 -- start start 2024-08-02T17:49:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.412+0000 7f37c1002740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9927b0 0x56337e98f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.412+0000 7f37c1002740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 0x56337e9b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.412+0000 7f37c1002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e9b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.412+0000 7f37c1002740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37a4012920 con 0x56337e9930d0 2024-08-02T17:49:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.412+0000 7f37c1002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37a40101f0 con 0x56337e99d970 2024-08-02T17:49:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.412+0000 7f37c1002740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37a400e040 con 0x56337e9927b0 2024-08-02T17:49:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.413+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9927b0 0x56337e98f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.413+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9927b0 0x56337e98f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50750/0 (socket says 172.21.15.69:50750) 2024-08-02T17:49:29.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.413+0000 7f37b77fe640 1 -- 172.21.15.69:0/2473297272 learned_addr learned my addr 172.21.15.69:0/2473297272 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.413+0000 7f37b77fe640 1 -- 172.21.15.69:0/2473297272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 msgr2=0x56337e9b56a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:49:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.413+0000 7f37b77fe640 1 --2- 172.21.15.69:0/2473297272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e9b56a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.413+0000 7f37b77fe640 1 -- 172.21.15.69:0/2473297272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 msgr2=0x56337e9b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.413+0000 7f37b77fe640 1 --2- 172.21.15.69:0/2473297272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 0x56337e9b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.413+0000 7f37b77fe640 1 -- 172.21.15.69:0/2473297272 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337e98fdd0 con 0x56337e9927b0 2024-08-02T17:49:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.414+0000 7f37b77fe640 1 --2- 172.21.15.69:0/2473297272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9927b0 0x56337e98f560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f37a80096c0 tx=0x7f37a80dc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23b7301910915a29 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.414+0000 7f379d7fa640 1 -- 172.21.15.69:0/2473297272 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a80d4070 con 0x56337e9927b0 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.414+0000 7f379d7fa640 1 -- 172.21.15.69:0/2473297272 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a80c0160 con 0x56337e9927b0 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.414+0000 7f379d7fa640 1 -- 172.21.15.69:0/2473297272 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8049040 con 0x56337e9927b0 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 -- 172.21.15.69:0/2473297272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9927b0 msgr2=0x56337e98f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 --2- 172.21.15.69:0/2473297272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9927b0 0x56337e98f560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f37a80096c0 tx=0x7f37a80dc040 comp rx=0 tx=0).stop 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 -- 172.21.15.69:0/2473297272 shutdown_connections 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 --2- 172.21.15.69:0/2473297272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9930d0 0x56337e9b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 --2- 172.21.15.69:0/2473297272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e9b56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 --2- 172.21.15.69:0/2473297272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9927b0 0x56337e98f560 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 -- 172.21.15.69:0/2473297272 >> 172.21.15.69:0/2473297272 conn(0x56337e98ed20 msgr2=0x56337e99c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 -- 172.21.15.69:0/2473297272 shutdown_connections 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 -- 172.21.15.69:0/2473297272 wait complete. 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.415+0000 7f37c1002740 1 -- start start 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37c1002740 1 Processor -- start 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37c1002740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e9a1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37c1002740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e9a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37c1002740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e9a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37c1002740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37a400e6b0 con 0x56337e9927b0 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37c1002740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37a400fb20 con 0x56337e99d970 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37c1002740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37a400ed80 con 0x56337e9930d0 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37c1002740 10 client.0 _ll_register_callbacks cb 0x7f37ac00fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e9a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37b6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e9a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37b77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e9a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42950/0 (socket says 172.21.15.69:42950) 2024-08-02T17:49:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37b77fe640 1 -- 172.21.15.69:0/3826811977 learned_addr learned my addr 172.21.15.69:0/3826811977 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.416+0000 7f37b7fff640 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e9a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37b6ffd640 1 -- 172.21.15.69:0/3826811977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 msgr2=0x56337e9a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e9a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37b6ffd640 1 -- 172.21.15.69:0/3826811977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 msgr2=0x56337e9a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e9a1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37b6ffd640 1 -- 172.21.15.69:0/3826811977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337ea28410 con 0x56337e9930d0 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37b77fe640 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e9a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37b6ffd640 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e9a1c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x56337e99d060 tx=0x7f37ac020c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df791ad211beee11 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f3796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f3796ffd640 1 -- 172.21.15.69:0/3826811977 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ac036030 con 0x56337e9930d0 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f3796ffd640 1 -- 172.21.15.69:0/3826811977 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f37ac009050 con 0x56337e9930d0 2024-08-02T17:49:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37c1002740 1 -- 172.21.15.69:0/3826811977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a80c0160 con 0x56337e9930d0 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f3796ffd640 1 -- 172.21.15.69:0/3826811977 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37ac010430 con 0x56337e9930d0 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37c1002740 10 client.25750 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f37c1002740 1 -- 172.21.15.69:0/3826811977 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56337ea29780 con 0x56337e9930d0 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f3796ffd640 1 -- 172.21.15.69:0/3826811977 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37a80c0160 con 0x56337e9930d0 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.417+0000 7f3796ffd640 1 -- 172.21.15.69:0/3826811977 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37ac01f040 con 0x56337e9930d0 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f37c1002740 10 client.25750 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f37c1002740 10 client.25750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f37c1002740 20 client.25750 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f37c1002740 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56337e9ad4a0 0x56337ea2bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f37c1002740 4 client.25750 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f37c1002740 1 -- 172.21.15.69:0/3826811977 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56337e8fe430 con 0x56337e9ad4a0 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f37b77fe640 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56337e9ad4a0 0x56337ea2bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f3796ffd640 10 client.25750 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.418+0000 7f37b77fe640 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56337e9ad4a0 0x56337ea2bf90 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf3a09fffa015a3 server_cookie=c627108044bfa0be in_seq=0 out_seq=0 2024-08-02T17:49:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f3796ffd640 1 -- 172.21.15.69:0/3826811977 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56337e8fe430 con 0x56337e9ad4a0 2024-08-02T17:49:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f3796ffd640 10 client.25750 handle_command_reply: tid=0 2024-08-02T17:49:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 client.25750 shutdown 2024-08-02T17:49:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 -- 172.21.15.69:0/3826811977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 msgr2=0x56337e9a1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e9a1c90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x56337e99d060 tx=0x7f37ac020c20 comp rx=0 tx=0).stop 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 -- 172.21.15.69:0/3826811977 shutdown_connections 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56337e9ad4a0 0x56337ea2bf90 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56337e99d970 0x56337e9a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56337e9930d0 0x56337e9a1c90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 --2- 172.21.15.69:0/3826811977 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56337e9927b0 0x56337e9a1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.430+0000 7f37c1002740 1 -- 172.21.15.69:0/3826811977 >> 172.21.15.69:0/3826811977 conn(0x56337e98ed20 msgr2=0x56337e98d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f3796ffd640 0 client.25750 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37c1002740 1 -- 172.21.15.69:0/3826811977 shutdown_connections 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37c1002740 1 -- 172.21.15.69:0/3826811977 wait complete. 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37c1002740 20 client.25750 trim_cache size 0 max 16384 2024-08-02T17:49:29.433 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:49:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:49:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:49:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:49:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 -- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37a00a2dc0 msgr2=0x7f37a00a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.434 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37a00a2dc0 0x7f37a00a5280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f37b80759e0 tx=0x7f37ac013c30 comp rx=0 tx=0).stop 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 -- 172.21.15.69:0/93696070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b81017f0 msgr2=0x7f37b8074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b81017f0 0x7f37b8074220 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009850 tx=0x7f37a8030640 comp rx=0 tx=0).stop 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 -- 172.21.15.69:0/93696070 shutdown_connections 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37b8102190 0x7f37b8074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37a00a2dc0 0x7f37a00a5280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37b8107800 0x7f37b8074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 --2- 172.21.15.69:0/93696070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37b81017f0 0x7f37b8074220 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.431+0000 7f37be1d0640 1 -- 172.21.15.69:0/93696070 >> 172.21.15.69:0/93696070 conn(0x7f37b80fd510 msgr2=0x7f37b80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.432+0000 7f37be1d0640 1 -- 172.21.15.69:0/93696070 shutdown_connections 2024-08-02T17:49:29.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:29.432+0000 7f37be1d0640 1 -- 172.21.15.69:0/93696070 wait complete. 2024-08-02T17:49:29.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:29 smithi069 ceph-mon[32746]: pgmap v95: 129 pgs: 129 active+clean; 1.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.4 MiB/s wr, 656 op/s 2024-08-02T17:49:29.812 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 140.85 --await-for 45.0' 2024-08-02T17:49:29.813 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 140.85 --await-for 45.0 2024-08-02T17:49:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 140.0 s, 1958.6 tps, lat 51.431 ms stddev 56.877 2024-08-02T17:49:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:29 smithi083 ceph-mon[37031]: pgmap v95: 129 pgs: 129 active+clean; 1.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.4 MiB/s wr, 656 op/s 2024-08-02T17:49:30.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:29 smithi077 ceph-mon[37108]: pgmap v95: 129 pgs: 129 active+clean; 1.7 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 5.4 MiB/s wr, 656 op/s 2024-08-02T17:49:30.602 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:49:30.761 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:31.192 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:31 smithi069 ceph-mon[32746]: pgmap v96: 129 pgs: 129 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.7 MiB/s wr, 705 op/s 2024-08-02T17:49:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.323+0000 7f252113f640 1 -- 172.21.15.69:0/1400492472 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251802f300 con 0x7f251c100b50 2024-08-02T17:49:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.324+0000 7f25243a5640 1 -- 172.21.15.69:0/1400492472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 msgr2=0x7f251c100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.324+0000 7f25243a5640 1 --2- 172.21.15.69:0/1400492472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 0x7f251c100f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2518009890 tx=0x7f251802a410 comp rx=0 tx=0).stop 2024-08-02T17:49:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.324+0000 7f25243a5640 1 -- 172.21.15.69:0/1400492472 shutdown_connections 2024-08-02T17:49:31.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.324+0000 7f25243a5640 1 --2- 172.21.15.69:0/1400492472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f251c102c20 0x7f251c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.324+0000 7f25243a5640 1 --2- 172.21.15.69:0/1400492472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c1022a0 0x7f251c1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.324+0000 7f25243a5640 1 --2- 172.21.15.69:0/1400492472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 0x7f251c100f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.324+0000 7f25243a5640 1 -- 172.21.15.69:0/1400492472 >> 172.21.15.69:0/1400492472 conn(0x7f251c0fc500 msgr2=0x7f251c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.325+0000 7f25243a5640 1 -- 172.21.15.69:0/1400492472 shutdown_connections 2024-08-02T17:49:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.326+0000 7f25243a5640 1 -- 172.21.15.69:0/1400492472 wait complete. 2024-08-02T17:49:31.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.327+0000 7f25243a5640 1 Processor -- start 2024-08-02T17:49:31.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.328+0000 7f25243a5640 1 -- start start 2024-08-02T17:49:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.328+0000 7f25243a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 0x7f251c1a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.328+0000 7f25243a5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f251c1022a0 0x7f251c1a4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.328+0000 7f25243a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102c20 0x7f251c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.328+0000 7f25243a5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10fbf0 con 0x7f251c1022a0 2024-08-02T17:49:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.328+0000 7f25243a5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10fa70 con 0x7f251c102c20 2024-08-02T17:49:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.328+0000 7f25243a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f251c10fd70 con 0x7f251c100b50 2024-08-02T17:49:31.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.328+0000 7f2521940640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f251c1022a0 0x7f251c1a4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.329+0000 7f2522141640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 0x7f251c1a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.329+0000 7f2522141640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 0x7f251c1a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50766/0 (socket says 172.21.15.69:50766) 2024-08-02T17:49:31.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.329+0000 7f2521940640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f251c1022a0 0x7f251c1a4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42964/0 (socket says 172.21.15.69:42964) 2024-08-02T17:49:31.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.329+0000 7f2522141640 1 -- 172.21.15.69:0/1558381735 learned_addr learned my addr 172.21.15.69:0/1558381735 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.329+0000 7f2522942640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102c20 0x7f251c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f2522942640 1 -- 172.21.15.69:0/1558381735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 msgr2=0x7f251c1a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f2522942640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 0x7f251c1a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f2522942640 1 -- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f251c1022a0 msgr2=0x7f251c1a4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f2522942640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f251c1022a0 0x7f251c1a4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f2522942640 1 -- 172.21.15.69:0/1558381735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251c19eef0 con 0x7f251c102c20 2024-08-02T17:49:31.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f2522942640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102c20 0x7f251c19e680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f251400ef00 tx=0x7f251400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f25137fe640 1 -- 172.21.15.69:0/1558381735 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514004b20 con 0x7f251c102c20 2024-08-02T17:49:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f25137fe640 1 -- 172.21.15.69:0/1558381735 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2514003e50 con 0x7f251c102c20 2024-08-02T17:49:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.330+0000 7f25137fe640 1 -- 172.21.15.69:0/1558381735 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251400b570 con 0x7f251c102c20 2024-08-02T17:49:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.331+0000 7f25243a5640 1 -- 172.21.15.69:0/1558381735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f251c19f1e0 con 0x7f251c102c20 2024-08-02T17:49:31.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.331+0000 7f25243a5640 1 -- 172.21.15.69:0/1558381735 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f251c1ab360 con 0x7f251c102c20 2024-08-02T17:49:31.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.335+0000 7f25137fe640 1 -- 172.21.15.69:0/1558381735 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f251401d020 con 0x7f251c102c20 2024-08-02T17:49:31.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.336+0000 7f25137fe640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25080a2df0 0x7f25080a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.336+0000 7f25137fe640 1 -- 172.21.15.69:0/1558381735 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f25140cf690 con 0x7f251c102c20 2024-08-02T17:49:31.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.340+0000 7f25267a3740 1 Processor -- start 2024-08-02T17:49:31.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.341+0000 7f25267a3740 1 -- start start 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.341+0000 7f25267a3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad662f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.341+0000 7f25267a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6639f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.341+0000 7f25267a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 0x55cad663c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.341+0000 7f25267a3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cad6594810 con 0x55cad662f2c0 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.341+0000 7f25267a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cad653bc70 con 0x55cad662fbe0 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.341+0000 7f25267a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cad65907d0 con 0x55cad663a4a0 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.344+0000 7f2521940640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6639f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.344+0000 7f2521940640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6639f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49320/0 (socket says 172.21.15.69:49320) 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.344+0000 7f2521940640 1 -- 172.21.15.69:0/3226615515 learned_addr learned my addr 172.21.15.69:0/3226615515 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.344+0000 7f2522141640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25080a2df0 0x7f25080a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:31 smithi077 ceph-mon[37108]: pgmap v96: 129 pgs: 129 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.7 MiB/s wr, 705 op/s 2024-08-02T17:49:31.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.345+0000 7f2521940640 1 -- 172.21.15.69:0/3226615515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 msgr2=0x55cad663c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.345+0000 7f2521940640 1 --2- 172.21.15.69:0/3226615515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 0x55cad663c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.345+0000 7f2521940640 1 -- 172.21.15.69:0/3226615515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 msgr2=0x55cad662f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.345+0000 7f2521940640 1 --2- 172.21.15.69:0/3226615515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad662f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.345+0000 7f2521940640 1 -- 172.21.15.69:0/3226615515 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad6306ba0 con 0x55cad662fbe0 2024-08-02T17:49:31.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.346+0000 7f2521940640 1 --2- 172.21.15.69:0/3226615515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6639f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f250c009ca0 tx=0x7f250c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ec5afcbee59180 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.348+0000 7f2510dea640 1 -- 172.21.15.69:0/3226615515 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c005220 con 0x55cad662fbe0 2024-08-02T17:49:31.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.348+0000 7f2510dea640 1 -- 172.21.15.69:0/3226615515 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c004c80 con 0x55cad662fbe0 2024-08-02T17:49:31.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.349+0000 7f25267a3740 1 -- 172.21.15.69:0/3226615515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 msgr2=0x55cad6639f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.349+0000 7f25267a3740 1 --2- 172.21.15.69:0/3226615515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6639f60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f250c009ca0 tx=0x7f250c011810 comp rx=0 tx=0).stop 2024-08-02T17:49:31.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.350+0000 7f25267a3740 1 -- 172.21.15.69:0/3226615515 shutdown_connections 2024-08-02T17:49:31.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.350+0000 7f25267a3740 1 --2- 172.21.15.69:0/3226615515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 0x55cad663c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.350+0000 7f25267a3740 1 --2- 172.21.15.69:0/3226615515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6639f60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.350+0000 7f25267a3740 1 --2- 172.21.15.69:0/3226615515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad662f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.350+0000 7f25267a3740 1 -- 172.21.15.69:0/3226615515 >> 172.21.15.69:0/3226615515 conn(0x55cad662b850 msgr2=0x55cad662bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.352+0000 7f2522141640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25080a2df0 0x7f25080a52b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25180095f0 tx=0x7f2518032ec0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.356+0000 7f25267a3740 1 -- 172.21.15.69:0/3226615515 shutdown_connections 2024-08-02T17:49:31.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.359+0000 7f25267a3740 1 -- 172.21.15.69:0/3226615515 wait complete. 2024-08-02T17:49:31.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.359+0000 7f25267a3740 1 -- start start 2024-08-02T17:49:31.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.359+0000 7f25267a3740 1 Processor -- start 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f25267a3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad66373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f25267a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6637910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f25267a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 0x55cad6637e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f25267a3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cad664e120 con 0x55cad662f2c0 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f25267a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cad654f9a0 con 0x55cad662fbe0 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f25267a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cad664e2a0 con 0x55cad663a4a0 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f2521940640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6637910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f2521940640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6637910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49334/0 (socket says 172.21.15.69:49334) 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f2521940640 1 -- 172.21.15.69:0/1072473798 learned_addr learned my addr 172.21.15.69:0/1072473798 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f2522942640 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 0x55cad6637e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.360+0000 7f25267a3740 10 client.0 _ll_register_callbacks cb 0x55cad653dc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:31.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.361+0000 7f2521940640 1 -- 172.21.15.69:0/1072473798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 msgr2=0x55cad6637e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.361+0000 7f2521940640 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 0x55cad6637e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.361+0000 7f2521940640 1 -- 172.21.15.69:0/1072473798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 msgr2=0x55cad66373d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:49:31.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.361+0000 7f2521940640 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad66373d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.361+0000 7f2521940640 1 -- 172.21.15.69:0/1072473798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad655b220 con 0x55cad662fbe0 2024-08-02T17:49:31.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.365+0000 7f2521940640 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6637910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f250c0092e0 tx=0x7f250c009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b836f215ddc60682 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.365+0000 7f2505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:31.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.366+0000 7f2505ffb640 1 -- 172.21.15.69:0/1072473798 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c029070 con 0x55cad662fbe0 2024-08-02T17:49:31.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.366+0000 7f2505ffb640 1 -- 172.21.15.69:0/1072473798 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f250c005430 con 0x55cad662fbe0 2024-08-02T17:49:31.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.366+0000 7f25267a3740 1 -- 172.21.15.69:0/1072473798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cad65a1f40 con 0x55cad662fbe0 2024-08-02T17:49:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.367+0000 7f2505ffb640 1 -- 172.21.15.69:0/1072473798 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f250c005630 con 0x55cad662fbe0 2024-08-02T17:49:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.367+0000 7f25267a3740 10 client.25883 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:31.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.367+0000 7f25267a3740 1 -- 172.21.15.69:0/1072473798 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cad6659f10 con 0x55cad662fbe0 2024-08-02T17:49:31.372 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:31.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.370+0000 7f2505ffb640 1 -- 172.21.15.69:0/1072473798 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f250c004b00 con 0x55cad662fbe0 2024-08-02T17:49:31.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.370+0000 7f2505ffb640 1 -- 172.21.15.69:0/1072473798 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f250c023a60 con 0x55cad662fbe0 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.371+0000 7f25267a3740 10 client.25883 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.371+0000 7f25267a3740 10 client.25883 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.371+0000 7f25267a3740 20 client.25883 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.371+0000 7f25267a3740 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cad665b260 0x55cad665d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.371+0000 7f25267a3740 4 client.25883 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.371+0000 7f25267a3740 1 -- 172.21.15.69:0/1072473798 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cad65a1f40 con 0x55cad665b260 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.371+0000 7f2522141640 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cad665b260 0x55cad665d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.372+0000 7f2522141640 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cad665b260 0x55cad665d650 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a5640918013829 server_cookie=bb03008a1827d48e in_seq=0 out_seq=0 2024-08-02T17:49:31.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.372+0000 7f2505ffb640 10 client.25883 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:31.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.376+0000 7f2505ffb640 1 -- 172.21.15.69:0/1072473798 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cad65a1f40 con 0x55cad665b260 2024-08-02T17:49:31.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.376+0000 7f2505ffb640 10 client.25883 handle_command_reply: tid=0 2024-08-02T17:49:31.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.377+0000 7f25267a3740 1 client.25883 shutdown 2024-08-02T17:49:31.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.377+0000 7f25267a3740 1 -- 172.21.15.69:0/1072473798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 msgr2=0x55cad6637910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.377+0000 7f25267a3740 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6637910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f250c0092e0 tx=0x7f250c009f10 comp rx=0 tx=0).stop 2024-08-02T17:49:31.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.379+0000 7f25267a3740 1 -- 172.21.15.69:0/1072473798 shutdown_connections 2024-08-02T17:49:31.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.379+0000 7f25267a3740 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cad665b260 0x55cad665d650 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.379+0000 7f25267a3740 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad663a4a0 0x55cad6637e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.379+0000 7f25267a3740 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad662fbe0 0x55cad6637910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.379+0000 7f2505ffb640 0 client.25883 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:31.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.379+0000 7f25267a3740 1 --2- 172.21.15.69:0/1072473798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad66373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.379+0000 7f25267a3740 1 -- 172.21.15.69:0/1072473798 >> 172.21.15.69:0/1072473798 conn(0x55cad662b850 msgr2=0x55cad662ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.383+0000 7f25267a3740 1 -- 172.21.15.69:0/1072473798 shutdown_connections 2024-08-02T17:49:31.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.383+0000 7f25267a3740 1 -- 172.21.15.69:0/1072473798 wait complete. 2024-08-02T17:49:31.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.384+0000 7f25267a3740 20 client.25883 trim_cache size 0 max 16384 2024-08-02T17:49:31.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.413+0000 7f25267a3740 1 Processor -- start 2024-08-02T17:49:31.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.413+0000 7f25267a3740 1 -- start start 2024-08-02T17:49:31.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f25267a3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad6637ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f25267a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662fbe0 0x55cad6662de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f25267a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad6663320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f25267a3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1af550 con 0x55cad662f2c0 2024-08-02T17:49:31.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f25267a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ace20 con 0x55cad663a4a0 2024-08-02T17:49:31.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f25267a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f251c073880 con 0x55cad662fbe0 2024-08-02T17:49:31.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f2522942640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad6663320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f2522942640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad6663320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49342/0 (socket says 172.21.15.69:49342) 2024-08-02T17:49:31.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f2522942640 1 -- 172.21.15.69:0/2897902808 learned_addr learned my addr 172.21.15.69:0/2897902808 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.414+0000 7f2521940640 1 --2- 172.21.15.69:0/2897902808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662fbe0 0x55cad6662de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.415+0000 7f2522942640 1 -- 172.21.15.69:0/2897902808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662fbe0 msgr2=0x55cad6662de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.415+0000 7f2522942640 1 --2- 172.21.15.69:0/2897902808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662fbe0 0x55cad6662de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.415+0000 7f2522942640 1 -- 172.21.15.69:0/2897902808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 msgr2=0x55cad6637ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.415+0000 7f2522942640 1 --2- 172.21.15.69:0/2897902808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad6637ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.415+0000 7f2522942640 1 -- 172.21.15.69:0/2897902808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cad662b680 con 0x55cad663a4a0 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.415+0000 7f2522942640 1 --2- 172.21.15.69:0/2897902808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad6663320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f251400cc40 tx=0x7f251409b1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174c4ca2c745e9ab server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.416+0000 7f2507fff640 1 -- 172.21.15.69:0/2897902808 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251401a040 con 0x55cad663a4a0 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.416+0000 7f2507fff640 1 -- 172.21.15.69:0/2897902808 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25140cf050 con 0x55cad663a4a0 2024-08-02T17:49:31.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.416+0000 7f2507fff640 1 -- 172.21.15.69:0/2897902808 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514097e00 con 0x55cad663a4a0 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 -- 172.21.15.69:0/2897902808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 msgr2=0x55cad6663320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 --2- 172.21.15.69:0/2897902808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad6663320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f251400cc40 tx=0x7f251409b1b0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 -- 172.21.15.69:0/2897902808 shutdown_connections 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 --2- 172.21.15.69:0/2897902808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad6663320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 --2- 172.21.15.69:0/2897902808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662fbe0 0x55cad6662de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 --2- 172.21.15.69:0/2897902808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662f2c0 0x55cad6637ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 -- 172.21.15.69:0/2897902808 >> 172.21.15.69:0/2897902808 conn(0x55cad662b850 msgr2=0x55cad662ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 -- 172.21.15.69:0/2897902808 shutdown_connections 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.417+0000 7f25267a3740 1 -- 172.21.15.69:0/2897902808 wait complete. 2024-08-02T17:49:31.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 1 -- start start 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 1 Processor -- start 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662f2c0 0x55cad664f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662fbe0 0x55cad664faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad664ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f251c073ef0 con 0x55cad662fbe0 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ac750 con 0x55cad663a4a0 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f251c1ab9b0 con 0x55cad662f2c0 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f25267a3740 10 client.0 _ll_register_callbacks cb 0x7f250c009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f2521940640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662fbe0 0x55cad664faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f2522141640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662f2c0 0x55cad664f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f2522141640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662f2c0 0x55cad664f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50812/0 (socket says 172.21.15.69:50812) 2024-08-02T17:49:31.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.418+0000 7f2522141640 1 -- 172.21.15.69:0/4045620685 learned_addr learned my addr 172.21.15.69:0/4045620685 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2522942640 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad664ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2522141640 1 -- 172.21.15.69:0/4045620685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 msgr2=0x55cad664ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2522141640 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad664ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2522141640 1 -- 172.21.15.69:0/4045620685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662fbe0 msgr2=0x55cad664faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2522141640 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662fbe0 0x55cad664faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2522141640 1 -- 172.21.15.69:0/4045620685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cad665dbb0 con 0x55cad662f2c0 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2522141640 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662f2c0 0x55cad664f560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2518005940 tx=0x7f2518005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15eaec6d8ce5bef server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2505ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2505ffb640 1 -- 172.21.15.69:0/4045620685 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251805a060 con 0x55cad662f2c0 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f25267a3740 1 -- 172.21.15.69:0/4045620685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25140cf050 con 0x55cad662f2c0 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.419+0000 7f2505ffb640 1 -- 172.21.15.69:0/4045620685 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f25180303a0 con 0x55cad662f2c0 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f2505ffb640 1 -- 172.21.15.69:0/4045620685 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2518061050 con 0x55cad662f2c0 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f25267a3740 10 client.25765 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:31.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f25267a3740 1 -- 172.21.15.69:0/4045620685 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cad662c9e0 con 0x55cad662f2c0 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f2505ffb640 1 -- 172.21.15.69:0/4045620685 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25140cf050 con 0x55cad662f2c0 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f2505ffb640 1 -- 172.21.15.69:0/4045620685 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f251804a040 con 0x55cad662f2c0 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f25267a3740 10 client.25765 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f25267a3740 10 client.25765 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f25267a3740 20 client.25765 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f25267a3740 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cad66c66c0 0x55cad66c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f25267a3740 4 client.25765 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.85, "await_for": 45.0}] 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.420+0000 7f25267a3740 1 -- 172.21.15.69:0/4045620685 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.85, "await_for": 45.0}) -- 0x55cad659c290 con 0x55cad66c66c0 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.421+0000 7f2521940640 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cad66c66c0 0x55cad66c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.421+0000 7f2521940640 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cad66c66c0 0x55cad66c8ab0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f40bd2789844267 server_cookie=75d4e1687ee362d1 in_seq=0 out_seq=0 2024-08-02T17:49:31.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.421+0000 7f2505ffb640 10 client.25765 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:31.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:31 smithi083 ceph-mon[37031]: pgmap v96: 129 pgs: 129 active+clean; 1.7 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 5.7 MiB/s wr, 705 op/s 2024-08-02T17:49:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f2505ffb640 1 -- 172.21.15.69:0/4045620685 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cad659c290 con 0x55cad66c66c0 2024-08-02T17:49:31.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f2505ffb640 10 client.25765 handle_command_reply: tid=0 2024-08-02T17:49:31.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 client.25765 shutdown 2024-08-02T17:49:31.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 -- 172.21.15.69:0/4045620685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662f2c0 msgr2=0x55cad664f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662f2c0 0x55cad664f560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2518005940 tx=0x7f2518005f90 comp rx=0 tx=0).stop 2024-08-02T17:49:31.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 -- 172.21.15.69:0/4045620685 shutdown_connections 2024-08-02T17:49:31.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cad662fbe0 0x55cad664faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cad66c66c0 0x55cad66c8ab0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cad663a4a0 0x55cad664ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 --2- 172.21.15.69:0/4045620685 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cad662f2c0 0x55cad664f560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f2505ffb640 0 client.25765 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 -- 172.21.15.69:0/4045620685 >> 172.21.15.69:0/4045620685 conn(0x55cad662b850 msgr2=0x55cad6630d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 -- 172.21.15.69:0/4045620685 shutdown_connections 2024-08-02T17:49:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 1 -- 172.21.15.69:0/4045620685 wait complete. 2024-08-02T17:49:31.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25267a3740 20 client.25765 trim_cache size 0 max 16384 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 10, 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "d272e2ef": { 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 10, 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:49:31.676 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.8, 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:31.677 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:49:31.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 -- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25080a2df0 msgr2=0x7f25080a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25080a2df0 0x7f25080a52b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25180095f0 tx=0x7f2518032ec0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 -- 172.21.15.69:0/1558381735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102c20 msgr2=0x7f251c19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102c20 0x7f251c19e680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f251400ef00 tx=0x7f251400c740 comp rx=0 tx=0).stop 2024-08-02T17:49:31.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 -- 172.21.15.69:0/1558381735 shutdown_connections 2024-08-02T17:49:31.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f251c1022a0 0x7f251c1a4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25080a2df0 0x7f25080a52b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f251c102c20 0x7f251c19e680 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 --2- 172.21.15.69:0/1558381735 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f251c100b50 0x7f251c1a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 -- 172.21.15.69:0/1558381735 >> 172.21.15.69:0/1558381735 conn(0x7f251c0fc500 msgr2=0x7f251c0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 -- 172.21.15.69:0/1558381735 shutdown_connections 2024-08-02T17:49:31.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.669+0000 7f25243a5640 1 -- 172.21.15.69:0/1558381735 wait complete. 2024-08-02T17:49:31.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 -- 172.21.15.69:0/319556201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc074be0 msgr2=0x7f5dfc112380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/319556201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc074be0 0x7f5dfc112380 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009900 tx=0x7f5dec02a480 comp rx=0 tx=0).stop 2024-08-02T17:49:31.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 -- 172.21.15.69:0/319556201 shutdown_connections 2024-08-02T17:49:31.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/319556201 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc074be0 0x7f5dfc112380 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/319556201 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc1128c0 0x7f5dfc114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/319556201 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dfc0742c0 0x7f5dfc0746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 -- 172.21.15.69:0/319556201 >> 172.21.15.69:0/319556201 conn(0x7f5dfc0fd5f0 msgr2=0x7f5dfc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 -- 172.21.15.69:0/319556201 shutdown_connections 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.842+0000 7f5e03a7d640 1 -- 172.21.15.69:0/319556201 wait complete. 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e03a7d640 1 Processor -- start 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e03a7d640 1 -- start start 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e03a7d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc0742c0 0x7f5dfc1a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e03a7d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dfc074be0 0x7f5dfc1a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e03a7d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc1128c0 0x7f5dfc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e03a7d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc118270 con 0x7f5dfc0742c0 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e03a7d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1180f0 con 0x7f5dfc074be0 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e03a7d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1183f0 con 0x7f5dfc1128c0 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e01819640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc0742c0 0x7f5dfc1a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e01819640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc0742c0 0x7f5dfc1a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43008/0 (socket says 172.21.15.69:43008) 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e01819640 1 -- 172.21.15.69:0/1893964543 learned_addr learned my addr 172.21.15.69:0/1893964543 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.843+0000 7f5e0201a640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc1128c0 0x7f5dfc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e01819640 1 -- 172.21.15.69:0/1893964543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc1128c0 msgr2=0x7f5dfc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e01819640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc1128c0 0x7f5dfc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e01018640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dfc074be0 0x7f5dfc1a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e01819640 1 -- 172.21.15.69:0/1893964543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dfc074be0 msgr2=0x7f5dfc1a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e01819640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dfc074be0 0x7f5dfc1a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e01819640 1 -- 172.21.15.69:0/1893964543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc19ecf0 con 0x7f5dfc0742c0 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e0201a640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc1128c0 0x7f5dfc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e01018640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dfc074be0 0x7f5dfc1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e01819640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc0742c0 0x7f5dfc1a4400 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5df800c920 tx=0x7f5df800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5df2ffd640 1 -- 172.21.15.69:0/1893964543 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8007c70 con 0x7f5dfc0742c0 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e03a7d640 1 -- 172.21.15.69:0/1893964543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc19efe0 con 0x7f5dfc0742c0 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5df2ffd640 1 -- 172.21.15.69:0/1893964543 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df8007e10 con 0x7f5dfc0742c0 2024-08-02T17:49:31.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5df2ffd640 1 -- 172.21.15.69:0/1893964543 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8018890 con 0x7f5dfc0742c0 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.844+0000 7f5e03a7d640 1 -- 172.21.15.69:0/1893964543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc1ab2d0 con 0x7f5dfc0742c0 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.846+0000 7f5df2ffd640 1 -- 172.21.15.69:0/1893964543 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5df8018a70 con 0x7f5dfc0742c0 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e05e7b740 1 Processor -- start 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e05e7b740 1 -- start start 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5df2ffd640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5de80a2cf0 0x7f5de80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c6910 0x55b5634d0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b56342ba40 con 0x55b5634d11d0 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5633c39d0 con 0x55b5634c5ff0 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b563427a00 con 0x55b5634c6910 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e01819640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5de80a2cf0 0x7f5de80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e0201a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.847+0000 7f5e0201a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634d35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43018/0 (socket says 172.21.15.69:43018) 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e0201a640 1 -- 172.21.15.69:0/303001399 learned_addr learned my addr 172.21.15.69:0/303001399 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e0201a640 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5df2ffd640 1 -- 172.21.15.69:0/1893964543 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5df80cfa10 con 0x7f5dfc0742c0 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e01018640 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c6910 0x55b5634d0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e0201a640 1 -- 172.21.15.69:0/303001399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c6910 msgr2=0x55b5634d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e0201a640 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c6910 0x55b5634d0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e0201a640 1 -- 172.21.15.69:0/303001399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 msgr2=0x55b5634c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e0201a640 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e0201a640 1 -- 172.21.15.69:0/303001399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5633ca650 con 0x55b5634d11d0 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e01819640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5de80a2cf0 0x7f5de80a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5df809b870 tx=0x7f5df8091000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5e0201a640 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634d35c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5df40129e0 tx=0x7f5df4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04f9494700c5697 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5de7fff640 1 -- 172.21.15.69:0/303001399 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4004b20 con 0x55b5634d11d0 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5de7fff640 1 -- 172.21.15.69:0/303001399 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df4010d10 con 0x55b5634d11d0 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.848+0000 7f5de7fff640 1 -- 172.21.15.69:0/303001399 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df40055b0 con 0x55b5634d11d0 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 -- 172.21.15.69:0/303001399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 msgr2=0x55b5634d35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634d35c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5df40129e0 tx=0x7f5df4009750 comp rx=0 tx=0).stop 2024-08-02T17:49:31.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 -- 172.21.15.69:0/303001399 shutdown_connections 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634d35c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c6910 0x55b5634d0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/303001399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 -- 172.21.15.69:0/303001399 >> 172.21.15.69:0/303001399 conn(0x55b5634c2580 msgr2=0x55b5634c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 -- 172.21.15.69:0/303001399 shutdown_connections 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 -- 172.21.15.69:0/303001399 wait complete. 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.849+0000 7f5e05e7b740 1 -- start start 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e05e7b740 1 Processor -- start 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b5634d3da0 con 0x55b5634c6910 2024-08-02T17:49:31.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5633e6b20 con 0x55b5634c5ff0 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5634d3f20 con 0x55b5634d11d0 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e05e7b740 10 client.0 _ll_register_callbacks cb 0x55b5633e4bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e01018640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e01018640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43028/0 (socket says 172.21.15.69:43028) 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e01018640 1 -- 172.21.15.69:0/4067487860 learned_addr learned my addr 172.21.15.69:0/4067487860 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e01018640 1 -- 172.21.15.69:0/4067487860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 msgr2=0x55b5634cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e01018640 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e01018640 1 -- 172.21.15.69:0/4067487860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 msgr2=0x55b5634ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e01018640 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e01018640 1 -- 172.21.15.69:0/4067487860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56319d3c0 con 0x55b5634c6910 2024-08-02T17:49:31.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.850+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:31.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e01018640 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634ce590 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009230 tx=0x7f5dec0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc5a78f95e18f34 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5de57fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:31.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5de57fa640 1 -- 172.21.15.69:0/4067487860 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec03f070 con 0x55b5634c6910 2024-08-02T17:49:31.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4067487860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df4010d10 con 0x55b5634c6910 2024-08-02T17:49:31.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5de57fa640 1 -- 172.21.15.69:0/4067487860 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5dec004450 con 0x55b5634c6910 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5de57fa640 1 -- 172.21.15.69:0/4067487860 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dec04c050 con 0x55b5634c6910 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 10 client.16254 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4067487860 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5634df950 con 0x55b5634c6910 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5de57fa640 1 -- 172.21.15.69:0/4067487860 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df4010d10 con 0x55b5634c6910 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5de57fa640 1 -- 172.21.15.69:0/4067487860 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5dec004c50 con 0x55b5634c6910 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 10 client.16254 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 20 client.16254 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:31.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5634e0e10 0x55b5634e3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 4 client.16254 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.851+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4067487860 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b563438fb0 con 0x55b5634e0e10 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.852+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5634e0e10 0x55b5634e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.852+0000 7f5de57fa640 10 client.16254 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.852+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5634e0e10 0x55b5634e3280 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ec5f328b0c1d4d server_cookie=456cb639b2ad66ee in_seq=0 out_seq=0 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5de57fa640 1 -- 172.21.15.69:0/4067487860 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b563438fb0 con 0x55b5634e0e10 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5de57fa640 10 client.16254 handle_command_reply: tid=0 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 client.16254 shutdown 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4067487860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 msgr2=0x55b5634ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634ce590 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009230 tx=0x7f5dec0319a0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4067487860 shutdown_connections 2024-08-02T17:49:31.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634ce590 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5634e0e10 0x55b5634e3280 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4067487860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5de57fa640 0 client.16254 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4067487860 >> 172.21.15.69:0/4067487860 conn(0x55b5634c2580 msgr2=0x55b5634c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4067487860 shutdown_connections 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4067487860 wait complete. 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.853+0000 7f5e05e7b740 20 client.16254 trim_cache size 0 max 16384 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e05e7b740 1 Processor -- start 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e05e7b740 1 -- start start 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c5ff0 0x55b5634cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c6910 0x55b5634e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1b7c90 con 0x55b5634d11d0 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1b5560 con 0x55b5634c6910 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1b33b0 con 0x55b5634c5ff0 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43032/0 (socket says 172.21.15.69:43032) 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 -- 172.21.15.69:0/4155045063 learned_addr learned my addr 172.21.15.69:0/4155045063 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c5ff0 0x55b5634cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e01018640 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c6910 0x55b5634e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 -- 172.21.15.69:0/4155045063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c5ff0 msgr2=0x55b5634cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c5ff0 0x55b5634cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 -- 172.21.15.69:0/4155045063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c6910 msgr2=0x55b5634e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c6910 0x55b5634e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.866+0000 7f5e0201a640 1 -- 172.21.15.69:0/4155045063 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5634c23b0 con 0x55b5634d11d0 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e0201a640 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634e9090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5df4011010 tx=0x7f5df4021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acd88dacc73b894 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5de77fe640 1 -- 172.21.15.69:0/4155045063 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4032030 con 0x55b5634d11d0 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5de77fe640 1 -- 172.21.15.69:0/4155045063 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df4030070 con 0x55b5634d11d0 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5de77fe640 1 -- 172.21.15.69:0/4155045063 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4005580 con 0x55b5634d11d0 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4155045063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 msgr2=0x55b5634e9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634e9090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5df4011010 tx=0x7f5df4021040 comp rx=0 tx=0).stop 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4155045063 shutdown_connections 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634d11d0 0x55b5634e9090 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c6910 0x55b5634e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/4155045063 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634c5ff0 0x55b5634cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4155045063 >> 172.21.15.69:0/4155045063 conn(0x55b5634c2580 msgr2=0x55b5634cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4155045063 shutdown_connections 2024-08-02T17:49:31.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.867+0000 7f5e05e7b740 1 -- 172.21.15.69:0/4155045063 wait complete. 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 1 -- start start 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 1 Processor -- start 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1b3a20 con 0x55b5634c6910 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1b4e90 con 0x55b5634c5ff0 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1b40f0 con 0x55b5634d11d0 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e05e7b740 10 client.0 _ll_register_callbacks cb 0x7f5dec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e01018640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e01018640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43048/0 (socket says 172.21.15.69:43048) 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e01018640 1 -- 172.21.15.69:0/2176059834 learned_addr learned my addr 172.21.15.69:0/2176059834 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:31.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.868+0000 7f5e0201a640 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e0201a640 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634d8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e01018640 1 -- 172.21.15.69:0/2176059834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 msgr2=0x55b5634d8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e01018640 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634d8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e01018640 1 -- 172.21.15.69:0/2176059834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 msgr2=0x55b5634d1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e01018640 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634d1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e01018640 1 -- 172.21.15.69:0/2176059834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5634c3210 con 0x55b5634c6910 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e0201a640 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e01018640 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634d2290 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0037b0 tx=0x7f5dec003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504c985368181b9a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5de57fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5de57fa640 1 -- 172.21.15.69:0/2176059834 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec054070 con 0x55b5634c6910 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5de57fa640 1 -- 172.21.15.69:0/2176059834 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5dec003830 con 0x55b5634c6910 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e05e7b740 1 -- 172.21.15.69:0/2176059834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df4030070 con 0x55b5634c6910 2024-08-02T17:49:31.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5de57fa640 1 -- 172.21.15.69:0/2176059834 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5dec05d050 con 0x55b5634c6910 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e05e7b740 10 client.16260 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5e05e7b740 1 -- 172.21.15.69:0/2176059834 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5634c1e30 con 0x55b5634c6910 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5de57fa640 1 -- 172.21.15.69:0/2176059834 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df4030070 con 0x55b5634c6910 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.869+0000 7f5de57fa640 1 -- 172.21.15.69:0/2176059834 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5dec057020 con 0x55b5634c6910 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 10 client.16260 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 10 client.16260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 20 client.16260 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b56354bb20 0x55b56354df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 4 client.16260 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 1 -- 172.21.15.69:0/2176059834 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5634334c0 con 0x55b56354bb20 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e0201a640 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b56354bb20 0x55b56354df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5de57fa640 10 client.16260 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e0201a640 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b56354bb20 0x55b56354df10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc1801af27ea418 server_cookie=6c941a45ac42e035 in_seq=0 out_seq=0 2024-08-02T17:49:31.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5de57fa640 1 -- 172.21.15.69:0/2176059834 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5634334c0 con 0x55b56354bb20 2024-08-02T17:49:31.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5de57fa640 10 client.16260 handle_command_reply: tid=0 2024-08-02T17:49:31.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 1 client.16260 shutdown 2024-08-02T17:49:31.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 1 -- 172.21.15.69:0/2176059834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 msgr2=0x55b5634d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.870+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634d2290 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0037b0 tx=0x7f5dec003f80 comp rx=0 tx=0).stop 2024-08-02T17:49:31.973 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 1 -- 172.21.15.69:0/2176059834 shutdown_connections 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5634c6910 0x55b5634d2290 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b56354bb20 0x55b56354df10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5634d11d0 0x55b5634d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 1 --2- 172.21.15.69:0/2176059834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5634c5ff0 0x55b5634d1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5de57fa640 0 client.16260 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 1 -- 172.21.15.69:0/2176059834 >> 172.21.15.69:0/2176059834 conn(0x55b5634c2580 msgr2=0x55b5634c8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 1 -- 172.21.15.69:0/2176059834 shutdown_connections 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 1 -- 172.21.15.69:0/2176059834 wait complete. 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e05e7b740 20 client.16260 trim_cache size 0 max 16384 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 -- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5de80a2cf0 msgr2=0x7f5de80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5de80a2cf0 0x7f5de80a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5df809b870 tx=0x7f5df8091000 comp rx=0 tx=0).stop 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 -- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc0742c0 msgr2=0x7f5dfc1a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc0742c0 0x7f5dfc1a4400 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f5df800c920 tx=0x7f5df800cde0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 -- 172.21.15.69:0/1893964543 shutdown_connections 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5de80a2cf0 0x7f5de80a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5dfc1128c0 0x7f5dfc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5dfc074be0 0x7f5dfc1a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 --2- 172.21.15.69:0/1893964543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5dfc0742c0 0x7f5dfc1a4400 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 -- 172.21.15.69:0/1893964543 >> 172.21.15.69:0/1893964543 conn(0x7f5dfc0fd5f0 msgr2=0x7f5dfc0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 -- 172.21.15.69:0/1893964543 shutdown_connections 2024-08-02T17:49:31.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:31.871+0000 7f5e03a7d640 1 -- 172.21.15.69:0/1893964543 wait complete. 2024-08-02T17:49:33.329 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:33 smithi069 ceph-mon[32746]: pgmap v97: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 8.5 MiB/s wr, 834 op/s 2024-08-02T17:49:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:33 smithi077 ceph-mon[37108]: pgmap v97: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 8.5 MiB/s wr, 834 op/s 2024-08-02T17:49:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:33 smithi083 ceph-mon[37031]: pgmap v97: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 8.5 MiB/s wr, 834 op/s 2024-08-02T17:49:34.775 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd272e2ef', quiesce took 10.0 seconds. Will release after: 10.850000000000001 2024-08-02T17:49:34.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 145.0 s, 651.7 tps, lat 46.526 ms stddev 53.556 2024-08-02T17:49:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:35 smithi077 ceph-mon[37108]: pgmap v98: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 533 op/s 2024-08-02T17:49:35.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:35 smithi069 ceph-mon[32746]: pgmap v98: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 533 op/s 2024-08-02T17:49:35.360 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:49:35.360 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:49:35.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:35 smithi083 ceph-mon[37031]: pgmap v98: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.2 MiB/s wr, 533 op/s 2024-08-02T17:49:35.625 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:36.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 -- 172.21.15.69:0/266108189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 msgr2=0x7fa2a810e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/266108189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 0x7fa2a810e340 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009ca0 tx=0x7fa2a002a950 comp rx=0 tx=0).stop 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 -- 172.21.15.69:0/266108189 shutdown_connections 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/266108189 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 0x7fa2a810e340 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/266108189 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2a8103390 0x7fa2a810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/266108189 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2a8102a70 0x7fa2a8102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 -- 172.21.15.69:0/266108189 >> 172.21.15.69:0/266108189 conn(0x7fa2a80fc4d0 msgr2=0x7fa2a80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 -- 172.21.15.69:0/266108189 shutdown_connections 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.654+0000 7fa2ad32a640 1 -- 172.21.15.69:0/266108189 wait complete. 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.655+0000 7fa2ad32a640 1 Processor -- start 2024-08-02T17:49:36.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.655+0000 7fa2ad32a640 1 -- start start 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.655+0000 7fa2ad32a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2a8102a70 0x7fa2a8198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.655+0000 7fa2ad32a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2a8103390 0x7fa2a8198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.655+0000 7fa2ad32a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 0x7fa2a819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.655+0000 7fa2ad32a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111c80 con 0x7fa2a810bea0 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.655+0000 7fa2ad32a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111b00 con 0x7fa2a8102a70 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.655+0000 7fa2ad32a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111e00 con 0x7fa2a8103390 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 0x7fa2a819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 0x7fa2a819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36094/0 (socket says 172.21.15.69:36094) 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 -- 172.21.15.69:0/809324651 learned_addr learned my addr 172.21.15.69:0/809324651 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2a8103390 0x7fa2a8198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2a8102a70 0x7fa2a8198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 -- 172.21.15.69:0/809324651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2a8103390 msgr2=0x7fa2a8198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2a8103390 0x7fa2a8198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 -- 172.21.15.69:0/809324651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2a8102a70 msgr2=0x7fa2a8198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2a8102a70 0x7fa2a8198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 -- 172.21.15.69:0/809324651 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a819d0b0 con 0x7fa2a810bea0 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a77fe640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 0x7fa2a819c9d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa2a002ae70 tx=0x7fa2a002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a4ff9640 1 -- 172.21.15.69:0/809324651 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0034040 con 0x7fa2a810bea0 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a4ff9640 1 -- 172.21.15.69:0/809324651 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a0030920 con 0x7fa2a810bea0 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2ad32a640 1 -- 172.21.15.69:0/809324651 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a819d340 con 0x7fa2a810bea0 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2a4ff9640 1 -- 172.21.15.69:0/809324651 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0045480 con 0x7fa2a810bea0 2024-08-02T17:49:36.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.656+0000 7fa2ad32a640 1 -- 172.21.15.69:0/809324651 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a4c80 con 0x7fa2a810bea0 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.659+0000 7fa2a4ff9640 1 -- 172.21.15.69:0/809324651 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2a0037020 con 0x7fa2a810bea0 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.659+0000 7fa2af728740 1 Processor -- start 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.659+0000 7fa2af728740 1 -- start start 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2af728740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2af728740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a19e0 0x5559803b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2af728740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 0x5559803b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2af728740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555980364e40 con 0x5559803a10c0 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2af728740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5559803067f0 con 0x5559803b07f0 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2af728740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5559802adfb0 con 0x5559803a19e0 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa29e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa29e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36104/0 (socket says 172.21.15.69:36104) 2024-08-02T17:49:36.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa29e7fc640 1 -- 172.21.15.69:0/3530657568 learned_addr learned my addr 172.21.15.69:0/3530657568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2a4ff9640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2940a2cf0 0x7fa2940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/3530657568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a19e0 0x5559803b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2a77fe640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2940a2cf0 0x7fa2940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa29e7fc640 1 -- 172.21.15.69:0/3530657568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a19e0 msgr2=0x5559803b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/3530657568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a19e0 0x5559803b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa29e7fc640 1 -- 172.21.15.69:0/3530657568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 msgr2=0x5559803b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/3530657568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 0x5559803b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa2a4ff9640 1 -- 172.21.15.69:0/809324651 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa2a00f05b0 con 0x7fa2a810bea0 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.660+0000 7fa29e7fc640 1 -- 172.21.15.69:0/3530657568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559800786a0 con 0x5559803a10c0 2024-08-02T17:49:36.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.661+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/3530657568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803a14a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa290004610 tx=0x7fa29000bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32882e7ccb281e44 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.661+0000 7fa2a77fe640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2940a2cf0 0x7fa2940a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00bc520 tx=0x7fa2a002a950 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.661+0000 7fa29d7fa640 1 -- 172.21.15.69:0/3530657568 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290003dd0 con 0x5559803a10c0 2024-08-02T17:49:36.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.661+0000 7fa29d7fa640 1 -- 172.21.15.69:0/3530657568 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290016030 con 0x5559803a10c0 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.661+0000 7fa29d7fa640 1 -- 172.21.15.69:0/3530657568 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290004d20 con 0x5559803a10c0 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- 172.21.15.69:0/3530657568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 msgr2=0x5559803a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 --2- 172.21.15.69:0/3530657568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803a14a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa290004610 tx=0x7fa29000bc10 comp rx=0 tx=0).stop 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- 172.21.15.69:0/3530657568 shutdown_connections 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 --2- 172.21.15.69:0/3530657568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 0x5559803b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 --2- 172.21.15.69:0/3530657568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a19e0 0x5559803b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 --2- 172.21.15.69:0/3530657568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803a14a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- 172.21.15.69:0/3530657568 >> 172.21.15.69:0/3530657568 conn(0x55598039d790 msgr2=0x55598039db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- 172.21.15.69:0/3530657568 shutdown_connections 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- 172.21.15.69:0/3530657568 wait complete. 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- start start 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 Processor -- start 2024-08-02T17:49:36.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a10c0 0x5559803ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803adca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 0x5559803ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5559802c1cb0 con 0x5559803a19e0 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5559803027b0 con 0x5559803a10c0 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5559803b3360 con 0x5559803b07f0 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.662+0000 7fa2af728740 10 client.0 _ll_register_callbacks cb 0x555980015ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa29e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 0x5559803ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36106/0 (socket says 172.21.15.69:36106) 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 -- 172.21.15.69:0/1914027228 learned_addr learned my addr 172.21.15.69:0/1914027228 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a10c0 0x5559803ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 -- 172.21.15.69:0/1914027228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 msgr2=0x5559803ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 0x5559803ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 -- 172.21.15.69:0/1914027228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a10c0 msgr2=0x5559803ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:36.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a10c0 0x5559803ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 -- 172.21.15.69:0/1914027228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559802c0bc0 con 0x5559803a19e0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803adca0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa2980127d0 tx=0x7fa298012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5ba3c7974e1526 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa28e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa28e7fc640 1 -- 172.21.15.69:0/1914027228 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29801f060 con 0x5559803a19e0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa28e7fc640 1 -- 172.21.15.69:0/1914027228 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa298010dc0 con 0x5559803a19e0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2af728740 1 -- 172.21.15.69:0/1914027228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa290016030 con 0x5559803a19e0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa28e7fc640 1 -- 172.21.15.69:0/1914027228 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa298026050 con 0x5559803a19e0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2af728740 10 client.16272 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.663+0000 7fa2af728740 1 -- 172.21.15.69:0/1914027228 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5559803bf340 con 0x5559803a19e0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa28e7fc640 1 -- 172.21.15.69:0/1914027228 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290016030 con 0x5559803a19e0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa28e7fc640 1 -- 172.21.15.69:0/1914027228 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa29800d440 con 0x5559803a19e0 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa2af728740 10 client.16272 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:36.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa2af728740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa2af728740 20 client.16272 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa2af728740 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5559803c0800 0x5559803c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa2af728740 4 client.16272 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa2af728740 1 -- 172.21.15.69:0/1914027228 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555980313d60 con 0x5559803c0800 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5559803c0800 0x5559803c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa28e7fc640 10 client.16272 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.664+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5559803c0800 0x5559803c2c50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cab783d681a961d server_cookie=26b5891a15a29a85 in_seq=0 out_seq=0 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa28e7fc640 1 -- 172.21.15.69:0/1914027228 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555980313d60 con 0x5559803c0800 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa28e7fc640 10 client.16272 handle_command_reply: tid=0 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 client.16272 shutdown 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 -- 172.21.15.69:0/1914027228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 msgr2=0x5559803adca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803adca0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa2980127d0 tx=0x7fa298012c90 comp rx=0 tx=0).stop 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 -- 172.21.15.69:0/1914027228 shutdown_connections 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803adca0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5559803c0800 0x5559803c2c50 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 0x5559803ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa28e7fc640 0 client.16272 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 --2- 172.21.15.69:0/1914027228 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a10c0 0x5559803ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 -- 172.21.15.69:0/1914027228 >> 172.21.15.69:0/1914027228 conn(0x55598039d790 msgr2=0x55598039c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 -- 172.21.15.69:0/1914027228 shutdown_connections 2024-08-02T17:49:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.665+0000 7fa2af728740 1 -- 172.21.15.69:0/1914027228 wait complete. 2024-08-02T17:49:36.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.666+0000 7fa2af728740 20 client.16272 trim_cache size 0 max 16384 2024-08-02T17:49:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.682+0000 7fa2af728740 1 Processor -- start 2024-08-02T17:49:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2af728740 1 -- start start 2024-08-02T17:49:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2af728740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2af728740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a19e0 0x5559803c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2af728740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 0x5559803c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2af728740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81af410 con 0x5559803a10c0 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2af728740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81acce0 con 0x5559803a19e0 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2af728740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81aab30 con 0x5559803b07f0 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36112/0 (socket says 172.21.15.69:36112) 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 -- 172.21.15.69:0/3847851934 learned_addr learned my addr 172.21.15.69:0/3847851934 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a19e0 0x5559803c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 -- 172.21.15.69:0/3847851934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 msgr2=0x5559803c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 0x5559803c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 -- 172.21.15.69:0/3847851934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a19e0 msgr2=0x5559803c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a19e0 0x5559803c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 -- 172.21.15.69:0/3847851934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559803a1810 con 0x5559803a10c0 2024-08-02T17:49:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a19e0 0x5559803c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.683+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803af6d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa290004390 tx=0x7fa290004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b0578c8bc7f61a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa29cff9640 1 -- 172.21.15.69:0/3847851934 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa290024040 con 0x5559803a10c0 2024-08-02T17:49:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa29cff9640 1 -- 172.21.15.69:0/3847851934 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290016070 con 0x5559803a10c0 2024-08-02T17:49:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa29cff9640 1 -- 172.21.15.69:0/3847851934 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900107b0 con 0x5559803a10c0 2024-08-02T17:49:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 -- 172.21.15.69:0/3847851934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 msgr2=0x5559803af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803af6d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa290004390 tx=0x7fa290004040 comp rx=0 tx=0).stop 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 -- 172.21.15.69:0/3847851934 shutdown_connections 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803b07f0 0x5559803c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803a19e0 0x5559803c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 --2- 172.21.15.69:0/3847851934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a10c0 0x5559803af6d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 -- 172.21.15.69:0/3847851934 >> 172.21.15.69:0/3847851934 conn(0x55598039d790 msgr2=0x5559803aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 -- 172.21.15.69:0/3847851934 shutdown_connections 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.684+0000 7fa2af728740 1 -- 172.21.15.69:0/3847851934 wait complete. 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 1 -- start start 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 1 Processor -- start 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a10c0 0x5559803b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 0x5559803b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81ab1a0 con 0x5559803a19e0 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81ac610 con 0x5559803b07f0 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81ab870 con 0x5559803a10c0 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2af728740 10 client.0 _ll_register_callbacks cb 0x7fa29800a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36120/0 (socket says 172.21.15.69:36120) 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2a6ffd640 1 -- 172.21.15.69:0/31722414 learned_addr learned my addr 172.21.15.69:0/31722414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a10c0 0x5559803b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 0x5559803b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2a6ffd640 1 -- 172.21.15.69:0/31722414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a10c0 msgr2=0x5559803b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a10c0 0x5559803b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2a6ffd640 1 -- 172.21.15.69:0/31722414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 msgr2=0x5559803b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 0x5559803b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.685+0000 7fa2a6ffd640 1 -- 172.21.15.69:0/31722414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55598042a6d0 con 0x5559803a19e0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2a6ffd640 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803b4a60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa298000c00 tx=0x7fa29800dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b406f7c71ad13a60 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa28e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa28e7fc640 1 -- 172.21.15.69:0/31722414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2980107a0 con 0x5559803a19e0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa28e7fc640 1 -- 172.21.15.69:0/31722414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa29800d340 con 0x5559803a19e0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 1 -- 172.21.15.69:0/31722414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa290016070 con 0x5559803a19e0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa28e7fc640 1 -- 172.21.15.69:0/31722414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa29801f020 con 0x5559803a19e0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 10 client.16278 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 1 -- 172.21.15.69:0/31722414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55598042b940 con 0x5559803a19e0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa28e7fc640 1 -- 172.21.15.69:0/31722414 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290016070 con 0x5559803a19e0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa28e7fc640 1 -- 172.21.15.69:0/31722414 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa298032030 con 0x5559803a19e0 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 10 client.16278 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 20 client.16278 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55598042c050 0x55598042e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 4 client.16278 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.686+0000 7fa2af728740 1 -- 172.21.15.69:0/31722414 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55598030e270 con 0x55598042c050 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.687+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55598042c050 0x55598042e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.687+0000 7fa29e7fc640 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55598042c050 0x55598042e470 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a735de40e6503a7f server_cookie=878c0e82d9eb4fcd in_seq=0 out_seq=0 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.687+0000 7fa28e7fc640 10 client.16278 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa28e7fc640 1 -- 172.21.15.69:0/31722414 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55598030e270 con 0x55598042c050 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa28e7fc640 10 client.16278 handle_command_reply: tid=0 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 client.16278 shutdown 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 -- 172.21.15.69:0/31722414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 msgr2=0x5559803b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803b4a60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa298000c00 tx=0x7fa29800dd30 comp rx=0 tx=0).stop 2024-08-02T17:49:36.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 -- 172.21.15.69:0/31722414 shutdown_connections 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5559803a19e0 0x5559803b4a60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55598042c050 0x55598042e470 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5559803b07f0 0x5559803b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 --2- 172.21.15.69:0/31722414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5559803a10c0 0x5559803b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa28e7fc640 0 client.16278 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 -- 172.21.15.69:0/31722414 >> 172.21.15.69:0/31722414 conn(0x55598039d790 msgr2=0x5559803af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 -- 172.21.15.69:0/31722414 shutdown_connections 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 1 -- 172.21.15.69:0/31722414 wait complete. 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.688+0000 7fa2af728740 20 client.16278 trim_cache size 0 max 16384 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:49:36.695 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:49:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "cd23ba6b-bc81-477c-8814-025be2e89f6b", 2024-08-02T17:49:36.696 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:49:36.696 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:49:36.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 -- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2940a2cf0 msgr2=0x7fa2940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2940a2cf0 0x7fa2940a51b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa2a00bc520 tx=0x7fa2a002a950 comp rx=0 tx=0).stop 2024-08-02T17:49:36.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 -- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 msgr2=0x7fa2a819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 0x7fa2a819c9d0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa2a002ae70 tx=0x7fa2a002f5b0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 -- 172.21.15.69:0/809324651 shutdown_connections 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2a810bea0 0x7fa2a819c9d0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2940a2cf0 0x7fa2940a51b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2a8103390 0x7fa2a8198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 --2- 172.21.15.69:0/809324651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2a8102a70 0x7fa2a8198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 -- 172.21.15.69:0/809324651 >> 172.21.15.69:0/809324651 conn(0x7fa2a80fc4d0 msgr2=0x7fa2a80fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 -- 172.21.15.69:0/809324651 shutdown_connections 2024-08-02T17:49:36.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:36.689+0000 7fa2ad32a640 1 -- 172.21.15.69:0/809324651 wait complete. 2024-08-02T17:49:37.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:37 smithi083 ceph-mon[37031]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 549 op/s 2024-08-02T17:49:37.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:37 smithi077 ceph-mon[37108]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 549 op/s 2024-08-02T17:49:37.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:37 smithi069 ceph-mon[32746]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 6.3 MiB/s wr, 549 op/s 2024-08-02T17:49:37.462 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:49:37.713 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:38.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.256+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/2167245709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b5406d8d0 msgr2=0x7f4b5406dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:38.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.256+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/2167245709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b5406d8d0 0x7f4b5406dcb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009860 tx=0x7f4b4402a360 comp rx=0 tx=0).stop 2024-08-02T17:49:38.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.258+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/2167245709 shutdown_connections 2024-08-02T17:49:38.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.258+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/2167245709 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b5411af40 0x7f4b5411d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.258+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/2167245709 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5406e1f0 0x7f4b5411aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.258+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/2167245709 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b5406d8d0 0x7f4b5406dcb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.258+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/2167245709 >> 172.21.15.69:0/2167245709 conn(0x7f4b5406caf0 msgr2=0x7f4b5406cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:38.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.265+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/2167245709 shutdown_connections 2024-08-02T17:49:38.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.269+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/2167245709 wait complete. 2024-08-02T17:49:38.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.270+0000 7f4b5a5d9640 1 Processor -- start 2024-08-02T17:49:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.272+0000 7f4b5a5d9640 1 -- start start 2024-08-02T17:49:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.272+0000 7f4b5a5d9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b5406d8d0 0x7f4b5411a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.272+0000 7f4b5a5d9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b5406e1f0 0x7f4b54116790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.272+0000 7f4b5a5d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5411af40 0x7f4b54116cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.272+0000 7f4b5a5d9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5410ec30 con 0x7f4b5406d8d0 2024-08-02T17:49:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.272+0000 7f4b5a5d9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5410eab0 con 0x7f4b5406e1f0 2024-08-02T17:49:38.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.272+0000 7f4b5a5d9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5410edb0 con 0x7f4b5411af40 2024-08-02T17:49:38.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.274+0000 7f4b58b76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5411af40 0x7f4b54116cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.274+0000 7f4b58b76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5411af40 0x7f4b54116cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40290/0 (socket says 172.21.15.69:40290) 2024-08-02T17:49:38.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.274+0000 7f4b58b76640 1 -- 172.21.15.69:0/3679793940 learned_addr learned my addr 172.21.15.69:0/3679793940 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:38.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.274+0000 7f4b58b76640 1 -- 172.21.15.69:0/3679793940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b5406e1f0 msgr2=0x7f4b54116790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:49:38.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.274+0000 7f4b58b76640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b5406e1f0 0x7f4b54116790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.274+0000 7f4b58b76640 1 -- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b5406d8d0 msgr2=0x7f4b5411a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:38.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.274+0000 7f4b58b76640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b5406d8d0 0x7f4b5411a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.274+0000 7f4b58b76640 1 -- 172.21.15.69:0/3679793940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b54117450 con 0x7f4b5411af40 2024-08-02T17:49:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.275+0000 7f4b58b76640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5411af40 0x7f4b54116cd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c01acf0 tx=0x7f4b4c01b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.275+0000 7f4b517fa640 1 -- 172.21.15.69:0/3679793940 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c013030 con 0x7f4b5411af40 2024-08-02T17:49:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.276+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/3679793940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b541cce80 con 0x7f4b5411af40 2024-08-02T17:49:38.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.276+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/3679793940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b541cd330 con 0x7f4b5411af40 2024-08-02T17:49:38.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.276+0000 7f4b517fa640 1 -- 172.21.15.69:0/3679793940 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b4c005020 con 0x7f4b5411af40 2024-08-02T17:49:38.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.276+0000 7f4b517fa640 1 -- 172.21.15.69:0/3679793940 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00f6e0 con 0x7f4b5411af40 2024-08-02T17:49:38.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.283+0000 7f4b517fa640 1 -- 172.21.15.69:0/3679793940 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4b4c016070 con 0x7f4b5411af40 2024-08-02T17:49:38.286 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.284+0000 7f4b5c9d7740 1 Processor -- start 2024-08-02T17:49:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.284+0000 7f4b5c9d7740 1 -- start start 2024-08-02T17:49:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.284+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e8895f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.284+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e893eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.284+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e8967e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.284+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55934e7ee980 con 0x55934e889210 2024-08-02T17:49:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.284+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55934e786a70 con 0x55934e889b30 2024-08-02T17:49:38.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.284+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55934e7ea940 con 0x55934e8943f0 2024-08-02T17:49:38.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.287+0000 7f4b537fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e8895f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.287+0000 7f4b537fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e8895f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36140/0 (socket says 172.21.15.69:36140) 2024-08-02T17:49:38.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.287+0000 7f4b537fe640 1 -- 172.21.15.69:0/3691537061 learned_addr learned my addr 172.21.15.69:0/3691537061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:38.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.288+0000 7f4b537fe640 1 --2- 172.21.15.69:0/3691537061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e8967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.292+0000 7f4b537fe640 1 -- 172.21.15.69:0/3691537061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 msgr2=0x55934e8967e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:38.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.292+0000 7f4b537fe640 1 --2- 172.21.15.69:0/3691537061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e8967e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.292+0000 7f4b537fe640 1 -- 172.21.15.69:0/3691537061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 msgr2=0x55934e893eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:49:38.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.292+0000 7f4b537fe640 1 --2- 172.21.15.69:0/3691537061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e893eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.292+0000 7f4b537fe640 1 -- 172.21.15.69:0/3691537061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934e7e96e0 con 0x55934e889210 2024-08-02T17:49:38.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.293+0000 7f4b517fa640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4b3c0a2f00 0x7f4b3c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.293+0000 7f4b517fa640 1 -- 172.21.15.69:0/3679793940 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4b4c0cfac0 con 0x7f4b5411af40 2024-08-02T17:49:38.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.296+0000 7f4b537fe640 1 --2- 172.21.15.69:0/3691537061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e8895f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009c80 tx=0x7f4b48009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabbac69637c8f51 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:38.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.299+0000 7f4b53fff640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4b3c0a2f00 0x7f4b3c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.303+0000 7f4b3a7fc640 1 -- 172.21.15.69:0/3691537061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48009e30 con 0x55934e889210 2024-08-02T17:49:38.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.303+0000 7f4b3a7fc640 1 -- 172.21.15.69:0/3691537061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b48010040 con 0x55934e889210 2024-08-02T17:49:38.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.303+0000 7f4b3a7fc640 1 -- 172.21.15.69:0/3691537061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48003d90 con 0x55934e889210 2024-08-02T17:49:38.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.305+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/3691537061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 msgr2=0x55934e8895f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:38.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.305+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/3691537061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e8895f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009c80 tx=0x7f4b48009410 comp rx=0 tx=0).stop 2024-08-02T17:49:38.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.306+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/3691537061 shutdown_connections 2024-08-02T17:49:38.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.306+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/3691537061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e8967e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.306+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/3691537061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e893eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.306+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/3691537061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e8895f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.306+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/3691537061 >> 172.21.15.69:0/3691537061 conn(0x55934e8857a0 msgr2=0x55934e885b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:38.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.308+0000 7f4b53fff640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4b3c0a2f00 0x7f4b3c0a53c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b440095f0 tx=0x7f4b440304c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:38.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.309+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/3691537061 shutdown_connections 2024-08-02T17:49:38.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.309+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/3691537061 wait complete. 2024-08-02T17:49:38.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 1 -- start start 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 1 Processor -- start 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e891340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e891880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e891dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55934e896fc0 con 0x55934e889210 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55934e7a9aa0 con 0x55934e889b30 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55934e897140 con 0x55934e8943f0 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.310+0000 7f4b5c9d7740 10 client.0 _ll_register_callbacks cb 0x55934e798e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.311+0000 7f4b58b76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e891dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.311+0000 7f4b58b76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e891dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40322/0 (socket says 172.21.15.69:40322) 2024-08-02T17:49:38.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.311+0000 7f4b58b76640 1 -- 172.21.15.69:0/129176478 learned_addr learned my addr 172.21.15.69:0/129176478 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:38.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.313+0000 7f4b58b76640 1 -- 172.21.15.69:0/129176478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 msgr2=0x55934e891880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.313+0000 7f4b58b76640 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e891880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.313+0000 7f4b58b76640 1 -- 172.21.15.69:0/129176478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 msgr2=0x55934e891340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.313+0000 7f4b58b76640 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e891340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.313+0000 7f4b58b76640 1 -- 172.21.15.69:0/129176478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934e78d650 con 0x55934e8943f0 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.313+0000 7f4b58b76640 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e891dc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c01aa60 tx=0x7f4b4c09bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb140f8137c5278 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.313+0000 7f4b337fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.315+0000 7f4b337fe640 1 -- 172.21.15.69:0/129176478 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0cd280 con 0x55934e8943f0 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.315+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/129176478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b48010040 con 0x55934e8943f0 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.315+0000 7f4b337fe640 1 -- 172.21.15.69:0/129176478 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4b4c09b010 con 0x55934e8943f0 2024-08-02T17:49:38.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.318+0000 7f4b337fe640 1 -- 172.21.15.69:0/129176478 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b4c09b210 con 0x55934e8943f0 2024-08-02T17:49:38.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.318+0000 7f4b5c9d7740 10 client.25801 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:38.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.318+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/129176478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55934e5603c0 con 0x55934e8943f0 2024-08-02T17:49:38.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.321+0000 7f4b337fe640 1 -- 172.21.15.69:0/129176478 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b48010040 con 0x55934e8943f0 2024-08-02T17:49:38.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.322+0000 7f4b337fe640 1 -- 172.21.15.69:0/129176478 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4b4c0a1260 con 0x55934e8943f0 2024-08-02T17:49:38.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.322+0000 7f4b5c9d7740 10 client.25801 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:38.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.322+0000 7f4b5c9d7740 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:38.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.322+0000 7f4b5c9d7740 20 client.25801 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:38.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.322+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55934e8a44a0 0x55934e8a6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.322+0000 7f4b5c9d7740 4 client.25801 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:38.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.322+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/129176478 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55934e7fbef0 con 0x55934e8a44a0 2024-08-02T17:49:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.325+0000 7f4b53fff640 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55934e8a44a0 0x55934e8a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.327+0000 7f4b53fff640 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55934e8a44a0 0x55934e8a6960 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173eaa4e4943bfb0 server_cookie=c7afe3bbba642404 in_seq=0 out_seq=0 2024-08-02T17:49:38.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.327+0000 7f4b337fe640 10 client.25801 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:38.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.330+0000 7f4b337fe640 1 -- 172.21.15.69:0/129176478 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f4b44030cd0 con 0x55934e8a44a0 2024-08-02T17:49:38.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.330+0000 7f4b337fe640 10 client.25801 handle_command_reply: tid=0 2024-08-02T17:49:38.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.330+0000 7f4b5c9d7740 1 client.25801 shutdown 2024-08-02T17:49:38.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.334+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/129176478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 msgr2=0x55934e891dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:38.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.334+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e891dc0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c01aa60 tx=0x7f4b4c09bc80 comp rx=0 tx=0).stop 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/129176478 shutdown_connections 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55934e8a44a0 0x55934e8a6960 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e8943f0 0x55934e891dc0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e891880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/129176478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889210 0x55934e891340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b337fe640 0 client.25801 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/129176478 >> 172.21.15.69:0/129176478 conn(0x55934e8857a0 msgr2=0x55934e88b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/129176478 shutdown_connections 2024-08-02T17:49:38.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/129176478 wait complete. 2024-08-02T17:49:38.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:38 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:49:38.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:38 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:38.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:38 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:49:38.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:38 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:38.338 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:38 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:49:38.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.335+0000 7f4b5c9d7740 20 client.25801 trim_cache size 0 max 16384 2024-08-02T17:49:38.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.381+0000 7f4b5c9d7740 1 Processor -- start 2024-08-02T17:49:38.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.381+0000 7f4b5c9d7740 1 -- start start 2024-08-02T17:49:38.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.381+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e88bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.381+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e885b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.381+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e8943f0 0x55934e8931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.381+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40012790 con 0x55934e8943f0 2024-08-02T17:49:38.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.382+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b40010060 con 0x55934e889b30 2024-08-02T17:49:38.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.382+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4000df60 con 0x55934e889210 2024-08-02T17:49:38.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.385+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e88bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.385+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e88bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40330/0 (socket says 172.21.15.69:40330) 2024-08-02T17:49:38.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.385+0000 7f4b53fff640 1 -- 172.21.15.69:0/1649265250 learned_addr learned my addr 172.21.15.69:0/1649265250 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:38.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.385+0000 7f4b58b76640 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e8943f0 0x55934e8931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.388+0000 7f4b537fe640 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e885b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.389+0000 7f4b58b76640 1 -- 172.21.15.69:0/1649265250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 msgr2=0x55934e88bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:38.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.389+0000 7f4b58b76640 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e88bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.390+0000 7f4b58b76640 1 -- 172.21.15.69:0/1649265250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 msgr2=0x55934e885b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:38.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.390+0000 7f4b58b76640 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e885b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.390+0000 7f4b58b76640 1 -- 172.21.15.69:0/1649265250 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55934e88cf20 con 0x55934e8943f0 2024-08-02T17:49:38.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.394+0000 7f4b58b76640 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e8943f0 0x55934e8931a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c099e70 tx=0x7f4b4c01a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45d377bd65a4aa3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:38.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.398+0000 7f4b39ffb640 1 -- 172.21.15.69:0/1649265250 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0cd280 con 0x55934e8943f0 2024-08-02T17:49:38.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.398+0000 7f4b39ffb640 1 -- 172.21.15.69:0/1649265250 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b4c0b2020 con 0x55934e8943f0 2024-08-02T17:49:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.399+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/1649265250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e8943f0 msgr2=0x55934e8931a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:38.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.399+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e8943f0 0x55934e8931a0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c099e70 tx=0x7f4b4c01a790 comp rx=0 tx=0).stop 2024-08-02T17:49:38.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.401+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/1649265250 shutdown_connections 2024-08-02T17:49:38.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.401+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e8943f0 0x55934e8931a0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.401+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e889b30 0x55934e885b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.401+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/1649265250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e88bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.401+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/1649265250 >> 172.21.15.69:0/1649265250 conn(0x55934e8857a0 msgr2=0x55934e892a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:38.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.405+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/1649265250 shutdown_connections 2024-08-02T17:49:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.407+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/1649265250 wait complete. 2024-08-02T17:49:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.408+0000 7f4b5c9d7740 1 -- start start 2024-08-02T17:49:38.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.408+0000 7f4b5c9d7740 1 Processor -- start 2024-08-02T17:49:38.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.412+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e89c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.412+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889b30 0x55934e897bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.412+0000 7f4b5c9d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e8943f0 0x55934e8980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.412+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4000e520 con 0x55934e889b30 2024-08-02T17:49:38.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.412+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4000f990 con 0x55934e8943f0 2024-08-02T17:49:38.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.412+0000 7f4b5c9d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4000ebf0 con 0x55934e889210 2024-08-02T17:49:38.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.412+0000 7f4b5c9d7740 10 client.0 _ll_register_callbacks cb 0x7f4b4c09b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:38.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.413+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e89c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.413+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e89c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40338/0 (socket says 172.21.15.69:40338) 2024-08-02T17:49:38.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.413+0000 7f4b53fff640 1 -- 172.21.15.69:0/2314321862 learned_addr learned my addr 172.21.15.69:0/2314321862 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:38.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.414+0000 7f4b53fff640 1 -- 172.21.15.69:0/2314321862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e8943f0 msgr2=0x55934e8980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.414+0000 7f4b53fff640 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e8943f0 0x55934e8980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.414+0000 7f4b53fff640 1 -- 172.21.15.69:0/2314321862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889b30 msgr2=0x55934e897bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.414+0000 7f4b53fff640 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889b30 0x55934e897bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.414+0000 7f4b53fff640 1 -- 172.21.15.69:0/2314321862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934e898630 con 0x55934e889210 2024-08-02T17:49:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.416+0000 7f4b53fff640 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e89c880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b440057c0 tx=0x7f4b44005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d081e90ebff651f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:38.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.416+0000 7f4b337fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:49:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.417+0000 7f4b337fe640 1 -- 172.21.15.69:0/2314321862 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44037040 con 0x55934e889210 2024-08-02T17:49:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.417+0000 7f4b337fe640 1 -- 172.21.15.69:0/2314321862 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4b44030580 con 0x55934e889210 2024-08-02T17:49:38.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.419+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/2314321862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b4c0b2020 con 0x55934e889210 2024-08-02T17:49:38.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.423+0000 7f4b337fe640 1 -- 172.21.15.69:0/2314321862 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b440044b0 con 0x55934e889210 2024-08-02T17:49:38.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.423+0000 7f4b5c9d7740 10 client.25807 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.423+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/2314321862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55934e90ef60 con 0x55934e889210 2024-08-02T17:49:38.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.425+0000 7f4b337fe640 1 -- 172.21.15.69:0/2314321862 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55934e7fbef0 con 0x55934e889210 2024-08-02T17:49:38.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.427+0000 7f4b337fe640 1 -- 172.21.15.69:0/2314321862 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4b4403a030 con 0x55934e889210 2024-08-02T17:49:38.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.427+0000 7f4b5c9d7740 10 client.25807 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:38.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.427+0000 7f4b5c9d7740 10 client.25807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:38.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.427+0000 7f4b5c9d7740 20 client.25807 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:38.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.427+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55934e90f680 0x55934e911b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:38.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.427+0000 7f4b5c9d7740 4 client.25807 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:49:38.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.427+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/2314321862 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4b4c0b2020 con 0x55934e90f680 2024-08-02T17:49:38.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.429+0000 7f4b537fe640 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55934e90f680 0x55934e911b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:38.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.435+0000 7f4b537fe640 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55934e90f680 0x55934e911b20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6436faff9a56fb65 server_cookie=869b5a17e804a88e in_seq=0 out_seq=0 2024-08-02T17:49:38.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.435+0000 7f4b337fe640 10 client.25807 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:38.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.436+0000 7f4b337fe640 1 -- 172.21.15.69:0/2314321862 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f4b4c0b2020 con 0x55934e90f680 2024-08-02T17:49:38.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.436+0000 7f4b337fe640 10 client.25807 handle_command_reply: tid=0 2024-08-02T17:49:38.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.437+0000 7f4b5c9d7740 1 client.25807 shutdown 2024-08-02T17:49:38.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.441+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/2314321862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 msgr2=0x55934e89c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:38.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.441+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e89c880 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4b440057c0 tx=0x7f4b44005f90 comp rx=0 tx=0).stop 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/2314321862 shutdown_connections 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55934e889b30 0x55934e897bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55934e90f680 0x55934e911b20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55934e8943f0 0x55934e8980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b337fe640 0 client.25807 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 1 --2- 172.21.15.69:0/2314321862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55934e889210 0x55934e89c880 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/2314321862 >> 172.21.15.69:0/2314321862 conn(0x55934e8857a0 msgr2=0x55934e884900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/2314321862 shutdown_connections 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 1 -- 172.21.15.69:0/2314321862 wait complete. 2024-08-02T17:49:38.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.444+0000 7f4b5c9d7740 20 client.25807 trim_cache size 0 max 16384 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout: "cd23ba6b-bc81-477c-8814-025be2e89f6b": { 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "cd23ba6b-bc81-477c-8814-025be2e89f6b", 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:38.448 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:49:38.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4b3c0a2f00 msgr2=0x7f4b3c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:38.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4b3c0a2f00 0x7f4b3c0a53c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4b440095f0 tx=0x7f4b440304c0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/3679793940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5411af40 msgr2=0x7f4b54116cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:38.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5411af40 0x7f4b54116cd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c01acf0 tx=0x7f4b4c01b6d0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/3679793940 shutdown_connections 2024-08-02T17:49:38.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4b3c0a2f00 0x7f4b3c0a53c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b5411af40 0x7f4b54116cd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4b5406e1f0 0x7f4b54116790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 --2- 172.21.15.69:0/3679793940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4b5406d8d0 0x7f4b5411a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:38.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/3679793940 >> 172.21.15.69:0/3679793940 conn(0x7f4b5406caf0 msgr2=0x7f4b5411b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:38.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/3679793940 shutdown_connections 2024-08-02T17:49:38.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:38.445+0000 7f4b5a5d9640 1 -- 172.21.15.69:0/3679793940 wait complete. 2024-08-02T17:49:38.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:38 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:49:38.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:38 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:38.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:38 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:49:38.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:38 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:38.528 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:38 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:49:38.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:38 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:49:38.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:38 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:38.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:38 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:49:38.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:38 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:49:38.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:38 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:49:39.190 INFO:tasks.cephfs.filesystem:scrub status for tag:cd23ba6b-bc81-477c-8814-025be2e89f6b - {'path': '/', 'tag': 'cd23ba6b-bc81-477c-8814-025be2e89f6b', 'options': 'recursive,force'} 2024-08-02T17:49:39.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:39 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:49:39.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:39 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:49:39.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:39 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:49:39.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:39 smithi083 ceph-mon[37031]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.0 MiB/s wr, 314 op/s 2024-08-02T17:49:39.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:39 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:49:39.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:39 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:49:39.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:39 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:49:39.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:39 smithi077 ceph-mon[37108]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.0 MiB/s wr, 314 op/s 2024-08-02T17:49:39.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:39 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:49:39.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:39 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:49:39.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:39 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:49:39.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:39 smithi069 ceph-mon[32746]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.0 MiB/s wr, 314 op/s 2024-08-02T17:49:39.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 150.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:49:40.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:40 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:49:40.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:40 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:49:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:40 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:49:41.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:41 smithi083 ceph-mon[37031]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.0 MiB/s wr, 460 op/s 2024-08-02T17:49:41.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:41 smithi077 ceph-mon[37108]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.0 MiB/s wr, 460 op/s 2024-08-02T17:49:41.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:41 smithi069 ceph-mon[32746]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.0 MiB/s wr, 460 op/s 2024-08-02T17:49:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 -- 172.21.15.69:0/1142948006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4100e80 msgr2=0x7f21f41076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 --2- 172.21.15.69:0/1142948006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4100e80 0x7f21f41076c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009930 tx=0x7f21e802a4d0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 -- 172.21.15.69:0/1142948006 shutdown_connections 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 --2- 172.21.15.69:0/1142948006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4100e80 0x7f21f41076c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 --2- 172.21.15.69:0/1142948006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4107e20 0x7f21f410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 --2- 172.21.15.69:0/1142948006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21f4100560 0x7f21f4100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 -- 172.21.15.69:0/1142948006 >> 172.21.15.69:0/1142948006 conn(0x7f21f4075bf0 msgr2=0x7f21f4076000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 -- 172.21.15.69:0/1142948006 shutdown_connections 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 -- 172.21.15.69:0/1142948006 wait complete. 2024-08-02T17:49:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 Processor -- start 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.534+0000 7f21fa487640 1 -- start start 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21fa487640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21f4100560 0x7f21f41981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21fa487640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4100e80 0x7f21f41986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21fa487640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4107e20 0x7f21f419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21fa487640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111c50 con 0x7f21f4107e20 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21fa487640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111ad0 con 0x7f21f4100560 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21fa487640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21f4111dd0 con 0x7f21f4100e80 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4107e20 0x7f21f419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4107e20 0x7f21f419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36204/0 (socket says 172.21.15.69:36204) 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 -- 172.21.15.69:0/1701169629 learned_addr learned my addr 172.21.15.69:0/1701169629 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f37fe640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4100e80 0x7f21f41986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f3fff640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21f4100560 0x7f21f41981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 -- 172.21.15.69:0/1701169629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4100e80 msgr2=0x7f21f41986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4100e80 0x7f21f41986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 -- 172.21.15.69:0/1701169629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21f4100560 msgr2=0x7f21f41981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21f4100560 0x7f21f41981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 -- 172.21.15.69:0/1701169629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f419d190 con 0x7f21f4107e20 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f37fe640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4100e80 0x7f21f41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f3fff640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21f4100560 0x7f21f41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.535+0000 7f21f8a24640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4107e20 0x7f21f419ca60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00eaa0 tx=0x7f21ec00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.536+0000 7f21f17fa640 1 -- 172.21.15.69:0/1701169629 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00cdb0 con 0x7f21f4107e20 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.536+0000 7f21f17fa640 1 -- 172.21.15.69:0/1701169629 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21ec005020 con 0x7f21f4107e20 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.536+0000 7f21f17fa640 1 -- 172.21.15.69:0/1701169629 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0116b0 con 0x7f21f4107e20 2024-08-02T17:49:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.536+0000 7f21fa487640 1 -- 172.21.15.69:0/1701169629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f419d480 con 0x7f21f4107e20 2024-08-02T17:49:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.536+0000 7f21fa487640 1 -- 172.21.15.69:0/1701169629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f41a4d10 con 0x7f21f4107e20 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.538+0000 7f21f17fa640 1 -- 172.21.15.69:0/1701169629 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f21ec005560 con 0x7f21f4107e20 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.538+0000 7f21fd1d9740 1 Processor -- start 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.538+0000 7f21fd1d9740 1 -- start start 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 0x56366e4d9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4da0d0 0x56366e4e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4e4970 0x56366e4e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56366e43d9b0 con 0x56366e4da0d0 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56366e3e5800 con 0x56366e4d97b0 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56366e439970 con 0x56366e4e4970 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4da0d0 0x56366e4e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f17fa640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4da0d0 0x56366e4e4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36216/0 (socket says 172.21.15.69:36216) 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 -- 172.21.15.69:0/17627716 learned_addr learned my addr 172.21.15.69:0/17627716 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f37fe640 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 0x56366e4d9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f37fe640 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4e4970 0x56366e4e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f8a24640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 -- 172.21.15.69:0/17627716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4e4970 msgr2=0x56366e4e6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4e4970 0x56366e4e6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 -- 172.21.15.69:0/17627716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 msgr2=0x56366e4d9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 0x56366e4d9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f17fa640 1 -- 172.21.15.69:0/1701169629 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f21ec0cfa60 con 0x7f21f4107e20 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 -- 172.21.15.69:0/17627716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366e151210 con 0x56366e4da0d0 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.539+0000 7f21f3fff640 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4da0d0 0x56366e4e4430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f21e400bb70 tx=0x7f21e400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d8b89a151f996f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21da7fc640 1 -- 172.21.15.69:0/17627716 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4017620 con 0x56366e4da0d0 2024-08-02T17:49:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21f8a24640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0ccea0 tx=0x7f21ec0cb040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21da7fc640 1 -- 172.21.15.69:0/17627716 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e4017ca0 con 0x56366e4da0d0 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21da7fc640 1 -- 172.21.15.69:0/17627716 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e400fc30 con 0x56366e4da0d0 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 -- 172.21.15.69:0/17627716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4da0d0 msgr2=0x56366e4e4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4da0d0 0x56366e4e4430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f21e400bb70 tx=0x7f21e400d710 comp rx=0 tx=0).stop 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 -- 172.21.15.69:0/17627716 shutdown_connections 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4da0d0 0x56366e4e4430 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4e4970 0x56366e4e6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/17627716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 0x56366e4d9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 -- 172.21.15.69:0/17627716 >> 172.21.15.69:0/17627716 conn(0x56366e4d5d20 msgr2=0x56366e4d6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 -- 172.21.15.69:0/17627716 shutdown_connections 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.540+0000 7f21fd1d9740 1 -- 172.21.15.69:0/17627716 wait complete. 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 1 -- start start 2024-08-02T17:49:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 1 Processor -- start 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4d97b0 0x56366e4e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4da0d0 0x56366e4e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4e22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56366e4e7540 con 0x56366e4e4970 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56366e3f9500 con 0x56366e4da0d0 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56366e4e76c0 con 0x56366e4d97b0 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21fd1d9740 10 client.0 _ll_register_callbacks cb 0x56366e150a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4e22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4e22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36224/0 (socket says 172.21.15.69:36224) 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21f37fe640 1 -- 172.21.15.69:0/4061686387 learned_addr learned my addr 172.21.15.69:0/4061686387 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.541+0000 7f21f37fe640 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4d97b0 0x56366e4e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21f3fff640 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4da0d0 0x56366e4e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21f37fe640 1 -- 172.21.15.69:0/4061686387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4d97b0 msgr2=0x56366e4e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21f37fe640 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4d97b0 0x56366e4e1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21f37fe640 1 -- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4da0d0 msgr2=0x56366e4e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21f37fe640 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4da0d0 0x56366e4e1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21f37fe640 1 -- 172.21.15.69:0/4061686387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366e21d620 con 0x56366e4e4970 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21f3fff640 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4da0d0 0x56366e4e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21f37fe640 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4e22b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f21f4076f70 tx=0x7f21e8040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f372dcdaa63b7283 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21d37fe640 1 -- 172.21.15.69:0/4061686387 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8035040 con 0x56366e4e4970 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21d37fe640 1 -- 172.21.15.69:0/4061686387 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f21e8003a20 con 0x56366e4e4970 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21fd1d9740 1 -- 172.21.15.69:0/4061686387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e4017ca0 con 0x56366e4e4970 2024-08-02T17:49:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21d37fe640 1 -- 172.21.15.69:0/4061686387 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f21e8056050 con 0x56366e4e4970 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21fd1d9740 10 client.16299 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21fd1d9740 1 -- 172.21.15.69:0/4061686387 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56366e4f3180 con 0x56366e4e4970 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.542+0000 7f21d37fe640 1 -- 172.21.15.69:0/4061686387 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e4017ca0 con 0x56366e4e4970 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21d37fe640 1 -- 172.21.15.69:0/4061686387 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f21e803a040 con 0x56366e4e4970 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21fd1d9740 10 client.16299 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21fd1d9740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21fd1d9740 20 client.16299 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56366e4f4640 0x56366e4f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21fd1d9740 4 client.16299 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21fd1d9740 1 -- 172.21.15.69:0/4061686387 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56366e44b0e0 con 0x56366e4f4640 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21f3fff640 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56366e4f4640 0x56366e4f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21d37fe640 10 client.16299 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.543+0000 7f21f3fff640 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56366e4f4640 0x56366e4f6a30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94207e537233200 server_cookie=199806c3b1689c90 in_seq=0 out_seq=0 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21d37fe640 1 -- 172.21.15.69:0/4061686387 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56366e44b0e0 con 0x56366e4f4640 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21d37fe640 10 client.16299 handle_command_reply: tid=0 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 client.16299 shutdown 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 -- 172.21.15.69:0/4061686387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 msgr2=0x56366e4e22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4e22b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f21f4076f70 tx=0x7f21e8040040 comp rx=0 tx=0).stop 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 -- 172.21.15.69:0/4061686387 shutdown_connections 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4e22b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56366e4f4640 0x56366e4f6a30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4da0d0 0x56366e4e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21d37fe640 0 client.16299 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/4061686387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4d97b0 0x56366e4e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 -- 172.21.15.69:0/4061686387 >> 172.21.15.69:0/4061686387 conn(0x56366e4d5d20 msgr2=0x56366e4dba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 -- 172.21.15.69:0/4061686387 shutdown_connections 2024-08-02T17:49:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.544+0000 7f21fd1d9740 1 -- 172.21.15.69:0/4061686387 wait complete. 2024-08-02T17:49:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.545+0000 7f21fd1d9740 20 client.16299 trim_cache size 0 max 16384 2024-08-02T17:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21fd1d9740 1 Processor -- start 2024-08-02T17:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21fd1d9740 1 -- start start 2024-08-02T17:49:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 0x56366e4dc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4fc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4fc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41b1790 con 0x56366e4e4970 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41af060 con 0x56366e4d97b0 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41aceb0 con 0x56366e4da0d0 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4fc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4fc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36230/0 (socket says 172.21.15.69:36230) 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 -- 172.21.15.69:0/2227051255 learned_addr learned my addr 172.21.15.69:0/2227051255 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f3fff640 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 -- 172.21.15.69:0/2227051255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 msgr2=0x56366e4fc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4fc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 -- 172.21.15.69:0/2227051255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 msgr2=0x56366e4dc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 0x56366e4dc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 -- 172.21.15.69:0/2227051255 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56366e4d69b0 con 0x56366e4e4970 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f3fff640 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.560+0000 7f21f37fe640 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4fc860 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f21e8006010 tx=0x7f21e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880cac0ecfb77ec9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21d9ffb640 1 -- 172.21.15.69:0/2227051255 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8035040 con 0x56366e4e4970 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21d9ffb640 1 -- 172.21.15.69:0/2227051255 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e8009520 con 0x56366e4e4970 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21d9ffb640 1 -- 172.21.15.69:0/2227051255 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8003ad0 con 0x56366e4e4970 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 -- 172.21.15.69:0/2227051255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 msgr2=0x56366e4fc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4fc860 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f21e8006010 tx=0x7f21e8000bc0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 -- 172.21.15.69:0/2227051255 shutdown_connections 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4e4970 0x56366e4fc860 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4fc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/2227051255 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4d97b0 0x56366e4dc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 -- 172.21.15.69:0/2227051255 >> 172.21.15.69:0/2227051255 conn(0x56366e4d5d20 msgr2=0x56366e4e0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 -- 172.21.15.69:0/2227051255 shutdown_connections 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 -- 172.21.15.69:0/2227051255 wait complete. 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.561+0000 7f21fd1d9740 1 -- start start 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21fd1d9740 1 Processor -- start 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4d97b0 0x56366e4ecd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21fd1d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4e4970 0x56366e4de6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41ad520 con 0x56366e4d97b0 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41ae990 con 0x56366e4e4970 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21fd1d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f21f41adbf0 con 0x56366e4da0d0 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21fd1d9740 10 client.0 _ll_register_callbacks cb 0x7f21e80044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:42.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4d97b0 0x56366e4ecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4d97b0 0x56366e4ecd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36242/0 (socket says 172.21.15.69:36242) 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f37fe640 1 -- 172.21.15.69:0/1696277686 learned_addr learned my addr 172.21.15.69:0/1696277686 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f3fff640 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f37fe640 1 -- 172.21.15.69:0/1696277686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 msgr2=0x56366e4de1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f37fe640 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4de1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f37fe640 1 -- 172.21.15.69:0/1696277686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4e4970 msgr2=0x56366e4de6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f37fe640 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4e4970 0x56366e4de6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f37fe640 1 -- 172.21.15.69:0/1696277686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56366e4dec30 con 0x56366e4d97b0 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.562+0000 7f21f3fff640 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21f37fe640 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4d97b0 0x56366e4ecd80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009850 tx=0x7f21e8031c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831550bf4c7d6636 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21d37fe640 1 -- 172.21.15.69:0/1696277686 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e8067020 con 0x56366e4d97b0 2024-08-02T17:49:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21d37fe640 1 -- 172.21.15.69:0/1696277686 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f21e803d860 con 0x56366e4d97b0 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 1 -- 172.21.15.69:0/1696277686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e8009520 con 0x56366e4d97b0 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21d37fe640 1 -- 172.21.15.69:0/1696277686 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f21e8072050 con 0x56366e4d97b0 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 10 client.16305 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 1 -- 172.21.15.69:0/1696277686 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56366e55ef80 con 0x56366e4d97b0 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21d37fe640 1 -- 172.21.15.69:0/1696277686 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21e8009520 con 0x56366e4d97b0 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21d37fe640 1 -- 172.21.15.69:0/1696277686 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f21e804ba20 con 0x56366e4d97b0 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 10 client.16305 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 20 client.16305 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56366e55f5e0 0x56366e5619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 4 client.16305 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.563+0000 7f21fd1d9740 1 -- 172.21.15.69:0/1696277686 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56366e445430 con 0x56366e55f5e0 2024-08-02T17:49:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.564+0000 7f21f3fff640 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56366e55f5e0 0x56366e5619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.564+0000 7f21d37fe640 10 client.16305 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.564+0000 7f21f3fff640 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56366e55f5e0 0x56366e5619d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dbca764a23278f2 server_cookie=f7491bbde4580eb6 in_seq=0 out_seq=0 2024-08-02T17:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.581+0000 7f21d37fe640 1 -- 172.21.15.69:0/1696277686 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56366e445430 con 0x56366e55f5e0 2024-08-02T17:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.581+0000 7f21d37fe640 10 client.16305 handle_command_reply: tid=0 2024-08-02T17:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.581+0000 7f21fd1d9740 1 client.16305 shutdown 2024-08-02T17:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.581+0000 7f21fd1d9740 1 -- 172.21.15.69:0/1696277686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4d97b0 msgr2=0x56366e4ecd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.581+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4d97b0 0x56366e4ecd80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009850 tx=0x7f21e8031c70 comp rx=0 tx=0).stop 2024-08-02T17:49:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.581+0000 7f21fd1d9740 1 -- 172.21.15.69:0/1696277686 shutdown_connections 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.581+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56366e55f5e0 0x56366e5619d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56366e4e4970 0x56366e4de6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56366e4da0d0 0x56366e4de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21d37fe640 0 client.16305 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fd1d9740 1 --2- 172.21.15.69:0/1696277686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56366e4d97b0 0x56366e4ecd80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fd1d9740 1 -- 172.21.15.69:0/1696277686 >> 172.21.15.69:0/1696277686 conn(0x56366e4d5d20 msgr2=0x56366e4e17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fd1d9740 1 -- 172.21.15.69:0/1696277686 shutdown_connections 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fd1d9740 1 -- 172.21.15.69:0/1696277686 wait complete. 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fd1d9740 20 client.16305 trim_cache size 0 max 16384 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:49:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:49:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:49:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:49:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 -- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f21dc0a2cf0 msgr2=0x7f21dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0ccea0 tx=0x7f21ec0cb040 comp rx=0 tx=0).stop 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 -- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4107e20 msgr2=0x7f21f419ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4107e20 0x7f21f419ca60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00eaa0 tx=0x7f21ec00ef60 comp rx=0 tx=0).stop 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 -- 172.21.15.69:0/1701169629 shutdown_connections 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f21f4107e20 0x7f21f419ca60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f21dc0a2cf0 0x7f21dc0a51b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f21f4100e80 0x7f21f41986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 --2- 172.21.15.69:0/1701169629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f21f4100560 0x7f21f41981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 -- 172.21.15.69:0/1701169629 >> 172.21.15.69:0/1701169629 conn(0x7f21f4075bf0 msgr2=0x7f21f4105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 -- 172.21.15.69:0/1701169629 shutdown_connections 2024-08-02T17:49:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:42.582+0000 7f21fa487640 1 -- 172.21.15.69:0/1701169629 wait complete. 2024-08-02T17:49:43.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:43 smithi077 ceph-mon[37108]: pgmap v102: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-08-02T17:49:43.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:43 smithi069 ceph-mon[32746]: pgmap v102: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-08-02T17:49:43.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:43 smithi083 ceph-mon[37031]: pgmap v102: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 15 MiB/s wr, 1.02k op/s 2024-08-02T17:49:44.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 155.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:49:45.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:45 smithi077 ceph-mon[37108]: pgmap v103: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-08-02T17:49:45.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:45 smithi069 ceph-mon[32746]: pgmap v103: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-08-02T17:49:45.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:45 smithi083 ceph-mon[37031]: pgmap v103: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 806 op/s 2024-08-02T17:49:45.626 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:49:45.891 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:46.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 -- 172.21.15.69:0/1319228588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 msgr2=0x7f39e4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:46.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 --2- 172.21.15.69:0/1319228588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 0x7f39e4101d10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009930 tx=0x7f39d802a420 comp rx=0 tx=0).stop 2024-08-02T17:49:46.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 -- 172.21.15.69:0/1319228588 shutdown_connections 2024-08-02T17:49:46.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 --2- 172.21.15.69:0/1319228588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 0x7f39e4101d10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 --2- 172.21.15.69:0/1319228588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e4102250 0x7f39e410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 --2- 172.21.15.69:0/1319228588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e41078b0 0x7f39e4107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 -- 172.21.15.69:0/1319228588 >> 172.21.15.69:0/1319228588 conn(0x7f39e40fd5f0 msgr2=0x7f39e40ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 -- 172.21.15.69:0/1319228588 shutdown_connections 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.372+0000 7f39e97ac640 1 -- 172.21.15.69:0/1319228588 wait complete. 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e97ac640 1 Processor -- start 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e97ac640 1 -- start start 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e97ac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 0x7f39e4074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e97ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e4102250 0x7f39e4074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e97ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e41078b0 0x7f39e4074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e97ac640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4111cb0 con 0x7f39e41018b0 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e97ac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4111b30 con 0x7f39e4102250 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e97ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39e4111e30 con 0x7f39e41078b0 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 0x7f39e4074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 0x7f39e4074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55226/0 (socket says 172.21.15.69:55226) 2024-08-02T17:49:46.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e2ffd640 1 -- 172.21.15.69:0/2182368242 learned_addr learned my addr 172.21.15.69:0/2182368242 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e27fc640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e4102250 0x7f39e4074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.373+0000 7f39e37fe640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e41078b0 0x7f39e4074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39e2ffd640 1 -- 172.21.15.69:0/2182368242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e41078b0 msgr2=0x7f39e4074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39e2ffd640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e41078b0 0x7f39e4074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39e2ffd640 1 -- 172.21.15.69:0/2182368242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e4102250 msgr2=0x7f39e4074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39e2ffd640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e4102250 0x7f39e4074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39e2ffd640 1 -- 172.21.15.69:0/2182368242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e41a66b0 con 0x7f39e41018b0 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39e2ffd640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 0x7f39e4074220 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f39d400c970 tx=0x7f39d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39cbfff640 1 -- 172.21.15.69:0/2182368242 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4007bb0 con 0x7f39e41018b0 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e41a69a0 con 0x7f39e41018b0 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39cbfff640 1 -- 172.21.15.69:0/2182368242 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d4007d50 con 0x7f39e41018b0 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39cbfff640 1 -- 172.21.15.69:0/2182368242 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d400f450 con 0x7f39e41018b0 2024-08-02T17:49:46.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.374+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e41a6eb0 con 0x7f39e41018b0 2024-08-02T17:49:46.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.375+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e4101d10 con 0x7f39e41018b0 2024-08-02T17:49:46.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.376+0000 7f39cbfff640 1 -- 172.21.15.69:0/2182368242 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f39d4005020 con 0x7f39e41018b0 2024-08-02T17:49:46.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.377+0000 7f39cbfff640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f39cc0a2cf0 0x7f39cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:46.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.377+0000 7f39e27fc640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f39cc0a2cf0 0x7f39cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:46.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.377+0000 7f39cbfff640 1 -- 172.21.15.69:0/2182368242 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f39d40cf580 con 0x7f39e41018b0 2024-08-02T17:49:46.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.377+0000 7f39e27fc640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f39cc0a2cf0 0x7f39cc0a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009900 tx=0x7f39d80058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:46.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.381+0000 7f39cbfff640 1 -- 172.21.15.69:0/2182368242 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f39d4014030 con 0x7f39e41018b0 2024-08-02T17:49:46.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.583+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39e4109e20 con 0x7f39e41018b0 2024-08-02T17:49:46.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.584+0000 7f39cbfff640 1 -- 172.21.15.69:0/2182368242 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f39d409bde0 con 0x7f39e41018b0 2024-08-02T17:49:46.585 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:49:46.585 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:49:46.586 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:49:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f39cc0a2cf0 msgr2=0x7f39cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f39cc0a2cf0 0x7f39cc0a51b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f39d8009900 tx=0x7f39d80058d0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 msgr2=0x7f39e4074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 0x7f39e4074220 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f39d400c970 tx=0x7f39d400ce30 comp rx=0 tx=0).stop 2024-08-02T17:49:46.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 shutdown_connections 2024-08-02T17:49:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f39cc0a2cf0 0x7f39cc0a51b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39e41078b0 0x7f39e4074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39e4102250 0x7f39e4074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 --2- 172.21.15.69:0/2182368242 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39e41018b0 0x7f39e4074220 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 >> 172.21.15.69:0/2182368242 conn(0x7f39e40fd5f0 msgr2=0x7f39e40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 shutdown_connections 2024-08-02T17:49:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:46.586+0000 7f39e97ac640 1 -- 172.21.15.69:0/2182368242 wait complete. 2024-08-02T17:49:46.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:46 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2182368242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:47.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:46 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2182368242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:47.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:46 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2182368242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:48.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:47 smithi083 ceph-mon[37031]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-08-02T17:49:48.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:47 smithi077 ceph-mon[37108]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-08-02T17:49:48.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:47 smithi069 ceph-mon[32746]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 12 MiB/s wr, 815 op/s 2024-08-02T17:49:48.438 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id d272e2ef --release --await' 2024-08-02T17:49:48.439 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id d272e2ef --release --await 2024-08-02T17:49:48.708 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:49 smithi069 ceph-mon[32746]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 800 op/s 2024-08-02T17:49:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:49 smithi077 ceph-mon[37108]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 800 op/s 2024-08-02T17:49:49.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 -- 172.21.15.69:0/3681196990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8103410 msgr2=0x7f73f810b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 --2- 172.21.15.69:0/3681196990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8103410 0x7f73f810b900 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009900 tx=0x7f73e002a480 comp rx=0 tx=0).stop 2024-08-02T17:49:49.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 -- 172.21.15.69:0/3681196990 shutdown_connections 2024-08-02T17:49:49.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 --2- 172.21.15.69:0/3681196990 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8103410 0x7f73f810b900 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 --2- 172.21.15.69:0/3681196990 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f810be90 0x7f73f810e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 --2- 172.21.15.69:0/3681196990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f8102af0 0x7f73f8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 -- 172.21.15.69:0/3681196990 >> 172.21.15.69:0/3681196990 conn(0x7f73f80fc4b0 msgr2=0x7f73f80fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 -- 172.21.15.69:0/3681196990 shutdown_connections 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 -- 172.21.15.69:0/3681196990 wait complete. 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.485+0000 7f73fd369640 1 Processor -- start 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73fd369640 1 -- start start 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73fd369640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8102af0 0x7f73f8100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73fd369640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f8103410 0x7f73f8100c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73fd369640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f810be90 0x7f73f81011d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73fd369640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8111d40 con 0x7f73f8102af0 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73fd369640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8111bc0 con 0x7f73f8103410 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73fd369640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73f8111ec0 con 0x7f73f810be90 2024-08-02T17:49:49.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8102af0 0x7f73f8100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8102af0 0x7f73f8100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55236/0 (socket says 172.21.15.69:55236) 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f6ffd640 1 -- 172.21.15.69:0/3738305531 learned_addr learned my addr 172.21.15.69:0/3738305531 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f77fe640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f810be90 0x7f73f81011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f67fc640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f8103410 0x7f73f8100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f6ffd640 1 -- 172.21.15.69:0/3738305531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f810be90 msgr2=0x7f73f81011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f810be90 0x7f73f81011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f6ffd640 1 -- 172.21.15.69:0/3738305531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f8103410 msgr2=0x7f73f8100c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f8103410 0x7f73f8100c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f6ffd640 1 -- 172.21.15.69:0/3738305531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73f81a67c0 con 0x7f73f8102af0 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.486+0000 7f73f67fc640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f8103410 0x7f73f8100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.487+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8102af0 0x7f73f8100750 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f73e800ba50 tx=0x7f73e800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:49.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.487+0000 7f73effff640 1 -- 172.21.15.69:0/3738305531 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8019070 con 0x7f73f8102af0 2024-08-02T17:49:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.487+0000 7f73fd369640 1 -- 172.21.15.69:0/3738305531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73f81a6ab0 con 0x7f73f8102af0 2024-08-02T17:49:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.487+0000 7f73effff640 1 -- 172.21.15.69:0/3738305531 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e800cbe0 con 0x7f73f8102af0 2024-08-02T17:49:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.487+0000 7f73effff640 1 -- 172.21.15.69:0/3738305531 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e80076e0 con 0x7f73f8102af0 2024-08-02T17:49:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.487+0000 7f73fd369640 1 -- 172.21.15.69:0/3738305531 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73f81a6fa0 con 0x7f73f8102af0 2024-08-02T17:49:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.489+0000 7f73effff640 1 -- 172.21.15.69:0/3738305531 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f73e801c070 con 0x7f73f8102af0 2024-08-02T17:49:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.489+0000 7f73ff767740 1 Processor -- start 2024-08-02T17:49:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.489+0000 7f73ff767740 1 -- start start 2024-08-02T17:49:49.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.489+0000 7f73effff640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73e40a2cf0 0x7f73e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73e40a2cf0 0x7f73e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73effff640 1 -- 172.21.15.69:0/3738305531 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f73e80ced20 con 0x7f73f8102af0 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73ff767740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95070d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73ff767740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e9507b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73ff767740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e9507df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73ff767740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555e94fd6c60 con 0x555e9507bb70 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73ff767740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e94f6eb90 con 0x555e950712b0 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73ff767740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e94fd2c20 con 0x555e95070990 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e9507df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e9507df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55240/0 (socket says 172.21.15.69:55240) 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 -- 172.21.15.69:0/1333392613 learned_addr learned my addr 172.21.15.69:0/1333392613 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95070d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f67fc640 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e9507b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73e40a2cf0 0x7f73e40a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009950 tx=0x7f73f0009d50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 -- 172.21.15.69:0/1333392613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 msgr2=0x555e95070d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95070d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 -- 172.21.15.69:0/1333392613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 msgr2=0x555e9507b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e9507b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 -- 172.21.15.69:0/1333392613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e94ce6580 con 0x555e9507bb70 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.490+0000 7f73f77fe640 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e9507df60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f73f000ebe0 tx=0x7f73f0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28584d2659016baa server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ed7fa640 1 -- 172.21.15.69:0/1333392613 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0022070 con 0x555e9507bb70 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ed7fa640 1 -- 172.21.15.69:0/1333392613 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f0015e20 con 0x555e9507bb70 2024-08-02T17:49:49.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ed7fa640 1 -- 172.21.15.69:0/1333392613 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f001a070 con 0x555e9507bb70 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ff767740 1 -- 172.21.15.69:0/1333392613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 msgr2=0x555e9507df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ff767740 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e9507df60 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f73f000ebe0 tx=0x7f73f0019000 comp rx=0 tx=0).stop 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ff767740 1 -- 172.21.15.69:0/1333392613 shutdown_connections 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ff767740 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e9507df60 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ff767740 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e9507b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ff767740 1 --2- 172.21.15.69:0/1333392613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95070d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.491+0000 7f73ff767740 1 -- 172.21.15.69:0/1333392613 >> 172.21.15.69:0/1333392613 conn(0x555e9506cf20 msgr2=0x555e9506d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 -- 172.21.15.69:0/1333392613 shutdown_connections 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 -- 172.21.15.69:0/1333392613 wait complete. 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 -- start start 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 Processor -- start 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e95070990 0x555e95078a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e950712b0 0x555e95078f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95079490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555e9507e740 con 0x555e950712b0 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e94f747c0 con 0x555e95070990 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e9507e8c0 con 0x555e9507bb70 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73ff767740 10 client.0 _ll_register_callbacks cb 0x555e94f7df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73f67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e950712b0 0x555e95078f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73f67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e950712b0 0x555e95078f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55256/0 (socket says 172.21.15.69:55256) 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73f67fc640 1 -- 172.21.15.69:0/339075542 learned_addr learned my addr 172.21.15.69:0/339075542 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.492+0000 7f73f77fe640 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95079490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e95070990 0x555e95078a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73f67fc640 1 -- 172.21.15.69:0/339075542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 msgr2=0x555e95079490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73f67fc640 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95079490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73f67fc640 1 -- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e95070990 msgr2=0x555e95078a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73f67fc640 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e95070990 0x555e95078a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73f67fc640 1 -- 172.21.15.69:0/339075542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e94db0960 con 0x555e950712b0 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e95070990 0x555e95078a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73f67fc640 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e950712b0 0x555e95078f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009230 tx=0x7f73e00317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbffe2aee91d559 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73de7fc640 1 -- 172.21.15.69:0/339075542 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e0035040 con 0x555e950712b0 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73de7fc640 1 -- 172.21.15.69:0/339075542 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f73e0004460 con 0x555e950712b0 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73ff767740 1 -- 172.21.15.69:0/339075542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f0015e20 con 0x555e950712b0 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73de7fc640 1 -- 172.21.15.69:0/339075542 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73e004c050 con 0x555e950712b0 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73ff767740 10 client.16323 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73ff767740 1 -- 172.21.15.69:0/339075542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e9508a730 con 0x555e950712b0 2024-08-02T17:49:49.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.493+0000 7f73de7fc640 1 -- 172.21.15.69:0/339075542 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f0015e20 con 0x555e950712b0 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73de7fc640 1 -- 172.21.15.69:0/339075542 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f73e0004bb0 con 0x555e950712b0 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73ff767740 10 client.16323 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73ff767740 10 client.16323 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73ff767740 20 client.16323 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73ff767740 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e9508bbf0 0x555e9508e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73ff767740 4 client.16323 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73ff767740 1 -- 172.21.15.69:0/339075542 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e94fe41d0 con 0x555e9508bbf0 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e9508bbf0 0x555e9508e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e9508bbf0 0x555e9508e040 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4466799035d52062 server_cookie=df3a0ff7bcf7fb27 in_seq=0 out_seq=0 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.494+0000 7f73de7fc640 10 client.16323 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:49.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73de7fc640 1 -- 172.21.15.69:0/339075542 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555e94fe41d0 con 0x555e9508bbf0 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73de7fc640 10 client.16323 handle_command_reply: tid=0 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 client.16323 shutdown 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 -- 172.21.15.69:0/339075542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e950712b0 msgr2=0x555e95078f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e950712b0 0x555e95078f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f73e0009230 tx=0x7f73e00317e0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 -- 172.21.15.69:0/339075542 shutdown_connections 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e950712b0 0x555e95078f50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e9508bbf0 0x555e9508e040 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95079490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73de7fc640 0 client.16323 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:49.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 --2- 172.21.15.69:0/339075542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e95070990 0x555e95078a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 -- 172.21.15.69:0/339075542 >> 172.21.15.69:0/339075542 conn(0x555e9506cf20 msgr2=0x555e95072c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 -- 172.21.15.69:0/339075542 shutdown_connections 2024-08-02T17:49:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 1 -- 172.21.15.69:0/339075542 wait complete. 2024-08-02T17:49:49.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.495+0000 7f73ff767740 20 client.16323 trim_cache size 0 max 16384 2024-08-02T17:49:49.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.512+0000 7f73ff767740 1 Processor -- start 2024-08-02T17:49:49.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.512+0000 7f73ff767740 1 -- start start 2024-08-02T17:49:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.512+0000 7f73ff767740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95093910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.512+0000 7f73ff767740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95093e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.512+0000 7f73ff767740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e95075cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73ff767740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81b39d0 con 0x555e9507bb70 2024-08-02T17:49:49.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73ff767740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81b12a0 con 0x555e950712b0 2024-08-02T17:49:49.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73ff767740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81af0f0 con 0x555e95070990 2024-08-02T17:49:49.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e95075cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e95075cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55262/0 (socket says 172.21.15.69:55262) 2024-08-02T17:49:49.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 -- 172.21.15.69:0/3495531270 learned_addr learned my addr 172.21.15.69:0/3495531270 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95093910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f67fc640 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95093e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 -- 172.21.15.69:0/3495531270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 msgr2=0x555e95093910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95093910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 -- 172.21.15.69:0/3495531270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 msgr2=0x555e95093e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95093e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 -- 172.21.15.69:0/3495531270 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e950779e0 con 0x555e9507bb70 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95093910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f67fc640 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95093e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73f77fe640 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e95075cb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f73f001ef20 tx=0x7f73f001eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b3548e141de3a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73ecff9640 1 -- 172.21.15.69:0/3495531270 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0035070 con 0x555e9507bb70 2024-08-02T17:49:49.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73ecff9640 1 -- 172.21.15.69:0/3495531270 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f0015e20 con 0x555e9507bb70 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.513+0000 7f73ecff9640 1 -- 172.21.15.69:0/3495531270 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0022040 con 0x555e9507bb70 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 -- 172.21.15.69:0/3495531270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 msgr2=0x555e95075cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e95075cb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f73f001ef20 tx=0x7f73f001eeb0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 -- 172.21.15.69:0/3495531270 shutdown_connections 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e9507bb70 0x555e95075cb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95093e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 --2- 172.21.15.69:0/3495531270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e95070990 0x555e95093910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 -- 172.21.15.69:0/3495531270 >> 172.21.15.69:0/3495531270 conn(0x555e9506cf20 msgr2=0x555e9507aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 -- 172.21.15.69:0/3495531270 shutdown_connections 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 -- 172.21.15.69:0/3495531270 wait complete. 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.514+0000 7f73ff767740 1 -- start start 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73ff767740 1 Processor -- start 2024-08-02T17:49:49.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73ff767740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e95070990 0x555e95083b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73ff767740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95084050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73ff767740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95080ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73ff767740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81af760 con 0x555e95070990 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73ff767740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81b0bd0 con 0x555e950712b0 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73ff767740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73f81afe30 con 0x555e9507bb70 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73ff767740 10 client.0 _ll_register_callbacks cb 0x7f73e0032e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e95070990 0x555e95083b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e95070990 0x555e95083b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55278/0 (socket says 172.21.15.69:55278) 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 -- 172.21.15.69:0/3148900605 learned_addr learned my addr 172.21.15.69:0/3148900605 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f67fc640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95084050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f77fe640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95080ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 -- 172.21.15.69:0/3148900605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 msgr2=0x555e95080ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95080ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 -- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 msgr2=0x555e95084050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95084050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 -- 172.21.15.69:0/3148900605 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e95081020 con 0x555e95070990 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f67fc640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95084050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f77fe640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95080ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73f6ffd640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e95070990 0x555e95083b10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f73e809acf0 tx=0x7f73e80bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fb62586a012103 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.515+0000 7f73de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73de7fc640 1 -- 172.21.15.69:0/3148900605 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8079e60 con 0x555e95070990 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73de7fc640 1 -- 172.21.15.69:0/3148900605 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f73e80b9bd0 con 0x555e95070990 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 1 -- 172.21.15.69:0/3148900605 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f0015e20 con 0x555e95070990 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73de7fc640 1 -- 172.21.15.69:0/3148900605 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73e8082050 con 0x555e95070990 2024-08-02T17:49:49.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 10 client.16329 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 1 -- 172.21.15.69:0/3148900605 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e950f6ff0 con 0x555e95070990 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73de7fc640 1 -- 172.21.15.69:0/3148900605 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e80a3e50 con 0x555e95070990 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73de7fc640 1 -- 172.21.15.69:0/3148900605 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f73e8095bb0 con 0x555e95070990 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 10 client.16329 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 10 client.16329 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 20 client.16329 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e950f7740 0x555e950f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 4 client.16329 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d272e2ef", "release": true, "await": true}] 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.516+0000 7f73ff767740 1 -- 172.21.15.69:0/3148900605 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d272e2ef", "release": true, "await": true}) -- 0x555e94fde6e0 con 0x555e950f7740 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.517+0000 7f73f67fc640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e950f7740 0x555e950f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.517+0000 7f73f67fc640 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e950f7740 0x555e950f9b30 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44157b6cac5f7f4c server_cookie=717c67e3f721708f in_seq=0 out_seq=0 2024-08-02T17:49:49.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.517+0000 7f73de7fc640 10 client.16329 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:49.533 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:49 smithi083 ceph-mon[37031]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 12 MiB/s wr, 800 op/s 2024-08-02T17:49:49.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73de7fc640 1 -- 172.21.15.69:0/3148900605 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555e94fde6e0 con 0x555e950f7740 2024-08-02T17:49:49.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73de7fc640 10 client.16329 handle_command_reply: tid=0 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 client.16329 shutdown 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 -- 172.21.15.69:0/3148900605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e95070990 msgr2=0x555e95083b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e95070990 0x555e95083b10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f73e809acf0 tx=0x7f73e80bb040 comp rx=0 tx=0).stop 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 -- 172.21.15.69:0/3148900605 shutdown_connections 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e950f7740 0x555e950f9b30 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e9507bb70 0x555e95080ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e950712b0 0x555e95084050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 --2- 172.21.15.69:0/3148900605 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e95070990 0x555e95083b10 secure :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f73e809acf0 tx=0x7f73e80bb040 comp rx=0 tx=0).stop 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73de7fc640 0 client.16329 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:49.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.575+0000 7f73ff767740 1 -- 172.21.15.69:0/3148900605 >> 172.21.15.69:0/3148900605 conn(0x555e9506cf20 msgr2=0x555e950723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:49.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.576+0000 7f73ff767740 1 -- 172.21.15.69:0/3148900605 shutdown_connections 2024-08-02T17:49:49.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.576+0000 7f73ff767740 1 -- 172.21.15.69:0/3148900605 wait complete. 2024-08-02T17:49:49.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.576+0000 7f73ff767740 20 client.16329 trim_cache size 0 max 16384 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 12, 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "d272e2ef": { 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 12, 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:49:49.579 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.8, 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:49.580 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:49.581 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:49:49.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.577+0000 7f73fd369640 1 -- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73e40a2cf0 msgr2=0x7f73e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.577+0000 7f73fd369640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73e40a2cf0 0x7f73e40a51b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009950 tx=0x7f73f0009d50 comp rx=0 tx=0).stop 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.577+0000 7f73fd369640 1 -- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8102af0 msgr2=0x7f73f8100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.577+0000 7f73fd369640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8102af0 0x7f73f8100750 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f73e800ba50 tx=0x7f73e800bf10 comp rx=0 tx=0).stop 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.579+0000 7f73fd369640 1 -- 172.21.15.69:0/3738305531 shutdown_connections 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.579+0000 7f73fd369640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73e40a2cf0 0x7f73e40a51b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.579+0000 7f73fd369640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73f810be90 0x7f73f81011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.579+0000 7f73fd369640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73f8103410 0x7f73f8100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.579+0000 7f73fd369640 1 --2- 172.21.15.69:0/3738305531 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73f8102af0 0x7f73f8100750 secure :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f73e800ba50 tx=0x7f73e800bf10 comp rx=0 tx=0).stop 2024-08-02T17:49:49.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.579+0000 7f73fd369640 1 -- 172.21.15.69:0/3738305531 >> 172.21.15.69:0/3738305531 conn(0x7f73f80fc4b0 msgr2=0x7f73f80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:49.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.583+0000 7f73fd369640 1 -- 172.21.15.69:0/3738305531 shutdown_connections 2024-08-02T17:49:49.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:49.584+0000 7f73fd369640 1 -- 172.21.15.69:0/3738305531 wait complete. 2024-08-02T17:49:49.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 160.0 s, 191.4 tps, lat 1936.947 ms stddev 5575.496 2024-08-02T17:49:50.517 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd272e2ef', total seconds elapsed: 25.8 2024-08-02T17:49:50.518 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.85 2024-08-02T17:49:50.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:49:50.937 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:51 smithi077 ceph-mon[37108]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 804 op/s 2024-08-02T17:49:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:51 smithi069 ceph-mon[32746]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 804 op/s 2024-08-02T17:49:51.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.500+0000 7f088ed2d640 1 -- 172.21.15.69:0/293062065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 msgr2=0x7f088811ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:51.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.500+0000 7f088ed2d640 1 --2- 172.21.15.69:0/293062065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 0x7f088811ad40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0880009be0 tx=0x7f088002a790 comp rx=0 tx=0).stop 2024-08-02T17:49:51.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.505+0000 7f088ed2d640 1 -- 172.21.15.69:0/293062065 shutdown_connections 2024-08-02T17:49:51.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.505+0000 7f088ed2d640 1 --2- 172.21.15.69:0/293062065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 0x7f088811ad40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:51.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.505+0000 7f088ed2d640 1 --2- 172.21.15.69:0/293062065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f088811b280 0x7f088811d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:51.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.505+0000 7f088ed2d640 1 --2- 172.21.15.69:0/293062065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f088806b610 0x7f088806b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:51.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.505+0000 7f088ed2d640 1 -- 172.21.15.69:0/293062065 >> 172.21.15.69:0/293062065 conn(0x7f0888071360 msgr2=0x7f0888073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:51.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.513+0000 7f088ed2d640 1 -- 172.21.15.69:0/293062065 shutdown_connections 2024-08-02T17:49:51.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.513+0000 7f088ed2d640 1 -- 172.21.15.69:0/293062065 wait complete. 2024-08-02T17:49:51.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.514+0000 7f088ed2d640 1 Processor -- start 2024-08-02T17:49:51.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.514+0000 7f088ed2d640 1 -- start start 2024-08-02T17:49:51.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.514+0000 7f088ed2d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f088806b610 0x7f08881a7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:51.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.514+0000 7f088ed2d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 0x7f08881a75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f088cac9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f088806b610 0x7f08881a7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f088cac9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f088806b610 0x7f08881a7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:41546/0 (socket says 172.21.15.69:41546) 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f0887fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 0x7f08881a75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f0887fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 0x7f08881a75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55306/0 (socket says 172.21.15.69:55306) 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f088cac9640 1 -- 172.21.15.69:0/1628285498 learned_addr learned my addr 172.21.15.69:0/1628285498 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f088ed2d640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f088811b280 0x7f08881ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f088ed2d640 1 -- 172.21.15.69:0/1628285498 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0888120ae0 con 0x7f088806bf30 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f088ed2d640 1 -- 172.21.15.69:0/1628285498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0888120960 con 0x7f088806b610 2024-08-02T17:49:51.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f088ed2d640 1 -- 172.21.15.69:0/1628285498 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0888120c60 con 0x7f088811b280 2024-08-02T17:49:51.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.515+0000 7f088cac9640 1 -- 172.21.15.69:0/1628285498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f088811b280 msgr2=0x7f08881ab930 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:49:51.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.516+0000 7f088cac9640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f088811b280 0x7f08881ab930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.516+0000 7f088cac9640 1 -- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 msgr2=0x7f08881a75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.516+0000 7f088cac9640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 0x7f08881a75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.516+0000 7f088cac9640 1 -- 172.21.15.69:0/1628285498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08881ac010 con 0x7f088806b610 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.516+0000 7f088cac9640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f088806b610 0x7f08881a7060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f087800b4a0 tx=0x7f087800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.516+0000 7f0885ffb640 1 -- 172.21.15.69:0/1628285498 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878014070 con 0x7f088806b610 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.516+0000 7f088ed2d640 1 -- 172.21.15.69:0/1628285498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08881ac300 con 0x7f088806b610 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.517+0000 7f088ed2d640 1 -- 172.21.15.69:0/1628285498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881b3be0 con 0x7f088806b610 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.517+0000 7f0885ffb640 1 -- 172.21.15.69:0/1628285498 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0878010af0 con 0x7f088806b610 2024-08-02T17:49:51.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.517+0000 7f0885ffb640 1 -- 172.21.15.69:0/1628285498 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878017d10 con 0x7f088806b610 2024-08-02T17:49:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:51 smithi083 ceph-mon[37031]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 12 MiB/s wr, 804 op/s 2024-08-02T17:49:51.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.519+0000 7f0885ffb640 1 -- 172.21.15.69:0/1628285498 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0878017070 con 0x7f088806b610 2024-08-02T17:49:51.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.520+0000 7f0885ffb640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f08700a2df0 0x7f08700a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:51.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.520+0000 7f0885ffb640 1 -- 172.21.15.69:0/1628285498 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0878010610 con 0x7f088806b610 2024-08-02T17:49:51.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.523+0000 7f088ed2d640 1 -- 172.21.15.69:0/1628285498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0874005290 con 0x7f088806b610 2024-08-02T17:49:51.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.525+0000 7f0887fff640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f08700a2df0 0x7f08700a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:51.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.528+0000 7f0887fff640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f08700a2df0 0x7f08700a52b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f088810e180 tx=0x7f0880000f40 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:51.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.530+0000 7f0885ffb640 1 -- 172.21.15.69:0/1628285498 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f087809c440 con 0x7f088806b610 2024-08-02T17:49:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.874+0000 7f088ed2d640 1 -- 172.21.15.69:0/1628285498 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0874005850 con 0x7f088806b610 2024-08-02T17:49:51.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.880+0000 7f0885ffb640 1 -- 172.21.15.69:0/1628285498 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f087809bd40 con 0x7f088806b610 2024-08-02T17:49:51.881 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:49:51.881 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:49:51.882 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:49:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 -- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f08700a2df0 msgr2=0x7f08700a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f08700a2df0 0x7f08700a52b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f088810e180 tx=0x7f0880000f40 comp rx=0 tx=0).stop 2024-08-02T17:49:51.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 -- 172.21.15.69:0/1628285498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f088806b610 msgr2=0x7f08881a7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:51.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f088806b610 0x7f08881a7060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f087800b4a0 tx=0x7f087800b960 comp rx=0 tx=0).stop 2024-08-02T17:49:51.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 -- 172.21.15.69:0/1628285498 shutdown_connections 2024-08-02T17:49:51.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f088806bf30 0x7f08881a75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:51.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f08700a2df0 0x7f08700a52b0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f088810e180 tx=0x7f0880000f40 comp rx=0 tx=0).stop 2024-08-02T17:49:51.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f088811b280 0x7f08881ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:51.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 --2- 172.21.15.69:0/1628285498 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f088806b610 0x7f08881a7060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:51.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.888+0000 7f086f7fe640 1 -- 172.21.15.69:0/1628285498 >> 172.21.15.69:0/1628285498 conn(0x7f0888071360 msgr2=0x7f0888072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:51.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.891+0000 7f086f7fe640 1 -- 172.21.15.69:0/1628285498 shutdown_connections 2024-08-02T17:49:51.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:51.895+0000 7f086f7fe640 1 -- 172.21.15.69:0/1628285498 wait complete. 2024-08-02T17:49:52.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:52 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1628285498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:52 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1628285498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:52.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:52 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1628285498' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:49:52.837 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 140.85 --await-for 45.0' 2024-08-02T17:49:52.837 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 140.85 --await-for 45.0 2024-08-02T17:49:53.258 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:49:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:53 smithi077 ceph-mon[37108]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 722 op/s 2024-08-02T17:49:53.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:53 smithi069 ceph-mon[32746]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 722 op/s 2024-08-02T17:49:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:53 smithi083 ceph-mon[37031]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 722 op/s 2024-08-02T17:49:53.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.825+0000 7fbf57aca640 1 -- 172.21.15.69:0/843842192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010d6e0 msgr2=0x7fbf50115240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:53.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.825+0000 7fbf57aca640 1 --2- 172.21.15.69:0/843842192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010d6e0 0x7fbf50115240 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009930 tx=0x7fbf4c02a460 comp rx=0 tx=0).stop 2024-08-02T17:49:53.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.825+0000 7fbf57aca640 1 -- 172.21.15.69:0/843842192 shutdown_connections 2024-08-02T17:49:53.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.825+0000 7fbf57aca640 1 --2- 172.21.15.69:0/843842192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010d6e0 0x7fbf50115240 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.825+0000 7fbf57aca640 1 --2- 172.21.15.69:0/843842192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf5010ec60 0x7fbf5010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.825+0000 7fbf57aca640 1 --2- 172.21.15.69:0/843842192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf5010e2b0 0x7fbf5010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.825+0000 7fbf57aca640 1 -- 172.21.15.69:0/843842192 >> 172.21.15.69:0/843842192 conn(0x7fbf5006de70 msgr2=0x7fbf5006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:53.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.827+0000 7fbf57aca640 1 -- 172.21.15.69:0/843842192 shutdown_connections 2024-08-02T17:49:53.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.827+0000 7fbf57aca640 1 -- 172.21.15.69:0/843842192 wait complete. 2024-08-02T17:49:53.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.828+0000 7fbf57aca640 1 Processor -- start 2024-08-02T17:49:53.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.828+0000 7fbf57aca640 1 -- start start 2024-08-02T17:49:53.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.829+0000 7fbf57aca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf5010d6e0 0x7fbf5011e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.829+0000 7fbf57aca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010e2b0 0x7fbf5011ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.829+0000 7fbf57aca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf50119a20 0x7fbf50119ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.829+0000 7fbf57aca640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf50118300 con 0x7fbf5010e2b0 2024-08-02T17:49:53.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.829+0000 7fbf57aca640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf50118180 con 0x7fbf5010d6e0 2024-08-02T17:49:53.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.829+0000 7fbf55065640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010e2b0 0x7fbf5011ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.829+0000 7fbf56067640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf50119a20 0x7fbf50119ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.830+0000 7fbf55065640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010e2b0 0x7fbf5011ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55318/0 (socket says 172.21.15.69:55318) 2024-08-02T17:49:53.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.830+0000 7fbf55065640 1 -- 172.21.15.69:0/1695327917 learned_addr learned my addr 172.21.15.69:0/1695327917 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:53.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.830+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf50118480 con 0x7fbf50119a20 2024-08-02T17:49:53.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.831+0000 7fbf56067640 1 -- 172.21.15.69:0/1695327917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf5010d6e0 msgr2=0x7fbf5011e900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:49:53.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.831+0000 7fbf56067640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf5010d6e0 0x7fbf5011e900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.831+0000 7fbf56067640 1 -- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010e2b0 msgr2=0x7fbf5011ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:53.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.831+0000 7fbf56067640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010e2b0 0x7fbf5011ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.831+0000 7fbf56067640 1 -- 172.21.15.69:0/1695327917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5011a790 con 0x7fbf50119a20 2024-08-02T17:49:53.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.831+0000 7fbf56067640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf50119a20 0x7fbf50119ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009850 tx=0x7fbf4c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:53.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.833+0000 7fbf46ffd640 1 -- 172.21.15.69:0/1695327917 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c03f070 con 0x7fbf50119a20 2024-08-02T17:49:53.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.833+0000 7fbf46ffd640 1 -- 172.21.15.69:0/1695327917 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c031c90 con 0x7fbf50119a20 2024-08-02T17:49:53.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.833+0000 7fbf46ffd640 1 -- 172.21.15.69:0/1695327917 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c035030 con 0x7fbf50119a20 2024-08-02T17:49:53.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.835+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf50124620 con 0x7fbf50119a20 2024-08-02T17:49:53.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.835+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf50124ae0 con 0x7fbf50119a20 2024-08-02T17:49:53.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.844+0000 7fbf59ec8740 1 Processor -- start 2024-08-02T17:49:53.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.844+0000 7fbf59ec8740 1 -- start start 2024-08-02T17:49:53.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.844+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823862c0 0x5590823866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.844+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082390f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.844+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823914a0 0x559082393890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.845+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5590822eb810 con 0x5590823914a0 2024-08-02T17:49:53.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.845+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559082292c70 con 0x559082386be0 2024-08-02T17:49:53.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.845+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590822e77d0 con 0x5590823862c0 2024-08-02T17:49:53.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.845+0000 7fbf46ffd640 1 -- 172.21.15.69:0/1695327917 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbf4c042070 con 0x7fbf50119a20 2024-08-02T17:49:53.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.845+0000 7fbf55866640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082390f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.845+0000 7fbf55866640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082390f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:41558/0 (socket says 172.21.15.69:41558) 2024-08-02T17:49:53.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.845+0000 7fbf55866640 1 -- 172.21.15.69:0/3675055373 learned_addr learned my addr 172.21.15.69:0/3675055373 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:53.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.846+0000 7fbf55866640 1 -- 172.21.15.69:0/3675055373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823862c0 msgr2=0x5590823866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:53.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.846+0000 7fbf55866640 1 --2- 172.21.15.69:0/3675055373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823862c0 0x5590823866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.846+0000 7fbf55866640 1 -- 172.21.15.69:0/3675055373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823914a0 msgr2=0x559082393890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:53.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.846+0000 7fbf55866640 1 --2- 172.21.15.69:0/3675055373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823914a0 0x559082393890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.846+0000 7fbf55866640 1 -- 172.21.15.69:0/3675055373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908205dba0 con 0x559082386be0 2024-08-02T17:49:53.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.846+0000 7fbf55866640 1 --2- 172.21.15.69:0/3675055373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082390f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009940 tx=0x7fbf4800b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee8a25cda55dd6c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:53.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.847+0000 7fbf46ffd640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf3c0a2f00 0x7fbf3c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.847+0000 7fbf46ffd640 1 -- 172.21.15.69:0/1695327917 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbf4c0ef9d0 con 0x7fbf50119a20 2024-08-02T17:49:53.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.847+0000 7fbf3bfff640 1 -- 172.21.15.69:0/3675055373 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf48004b20 con 0x559082386be0 2024-08-02T17:49:53.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.848+0000 7fbf3bfff640 1 -- 172.21.15.69:0/3675055373 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf48003e50 con 0x559082386be0 2024-08-02T17:49:53.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.848+0000 7fbf55065640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf3c0a2f00 0x7fbf3c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3675055373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 msgr2=0x559082390f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/3675055373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082390f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009940 tx=0x7fbf4800b7b0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3675055373 shutdown_connections 2024-08-02T17:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/3675055373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823914a0 0x559082393890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/3675055373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082390f60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/3675055373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823862c0 0x5590823866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3675055373 >> 172.21.15.69:0/3675055373 conn(0x559082382850 msgr2=0x559082382c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3675055373 shutdown_connections 2024-08-02T17:49:53.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf55065640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf3c0a2f00 0x7fbf3c0a53c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbf40006e10 tx=0x7fbf40006d60 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:53.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.849+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3675055373 wait complete. 2024-08-02T17:49:53.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.850+0000 7fbf59ec8740 1 -- start start 2024-08-02T17:49:53.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.850+0000 7fbf59ec8740 1 Processor -- start 2024-08-02T17:49:53.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.852+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823862c0 0x55908238e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.852+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559082386be0 0x55908238e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.852+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.852+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559082394060 con 0x559082386be0 2024-08-02T17:49:53.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.852+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5590822a69a0 con 0x5590823914a0 2024-08-02T17:49:53.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.852+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5590823941e0 con 0x5590823862c0 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.852+0000 7fbf59ec8740 10 client.0 _ll_register_callbacks cb 0x559082294c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.852+0000 7fbf55065640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559082386be0 0x55908238e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.853+0000 7fbf55065640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559082386be0 0x55908238e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55334/0 (socket says 172.21.15.69:55334) 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.853+0000 7fbf55065640 1 -- 172.21.15.69:0/2042724654 learned_addr learned my addr 172.21.15.69:0/2042724654 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.853+0000 7fbf55065640 1 -- 172.21.15.69:0/2042724654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823862c0 msgr2=0x55908238e3d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.853+0000 7fbf55065640 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823862c0 0x55908238e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.853+0000 7fbf55065640 1 -- 172.21.15.69:0/2042724654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 msgr2=0x55908238ee50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.853+0000 7fbf55065640 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238ee50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.853+0000 7fbf55065640 1 -- 172.21.15.69:0/2042724654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590822b2220 con 0x559082386be0 2024-08-02T17:49:53.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.854+0000 7fbf55065640 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559082386be0 0x55908238e910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf40015700 tx=0x7fbf40009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e479149e03d8e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:53.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.854+0000 7fbf397fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:53.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.854+0000 7fbf397fa640 1 -- 172.21.15.69:0/2042724654 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40023020 con 0x559082386be0 2024-08-02T17:49:53.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.855+0000 7fbf59ec8740 1 -- 172.21.15.69:0/2042724654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5590822f8f40 con 0x559082386be0 2024-08-02T17:49:53.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.854+0000 7fbf397fa640 1 -- 172.21.15.69:0/2042724654 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbf40013510 con 0x559082386be0 2024-08-02T17:49:53.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.855+0000 7fbf397fa640 1 -- 172.21.15.69:0/2042724654 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf4001b040 con 0x559082386be0 2024-08-02T17:49:53.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.856+0000 7fbf59ec8740 10 client.16344 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:53.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.856+0000 7fbf59ec8740 1 -- 172.21.15.69:0/2042724654 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55908239ff60 con 0x559082386be0 2024-08-02T17:49:53.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.857+0000 7fbf397fa640 1 -- 172.21.15.69:0/2042724654 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5590822f8f40 con 0x559082386be0 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.857+0000 7fbf397fa640 1 -- 172.21.15.69:0/2042724654 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf4001e030 con 0x559082386be0 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.857+0000 7fbf59ec8740 10 client.16344 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.857+0000 7fbf59ec8740 10 client.16344 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.857+0000 7fbf59ec8740 20 client.16344 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.857+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5590823a18c0 0x5590823a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.857+0000 7fbf59ec8740 4 client.16344 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.857+0000 7fbf59ec8740 1 -- 172.21.15.69:0/2042724654 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590822f3290 con 0x5590823a18c0 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.858+0000 7fbf55866640 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5590823a18c0 0x5590823a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.858+0000 7fbf397fa640 10 client.16344 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:53.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.859+0000 7fbf55866640 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5590823a18c0 0x5590823a3d60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=586d99aca5535450 server_cookie=53c897695d781d89 in_seq=0 out_seq=0 2024-08-02T17:49:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.860+0000 7fbf397fa640 1 -- 172.21.15.69:0/2042724654 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5590822f3290 con 0x5590823a18c0 2024-08-02T17:49:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.860+0000 7fbf397fa640 10 client.16344 handle_command_reply: tid=0 2024-08-02T17:49:53.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.860+0000 7fbf59ec8740 1 client.16344 shutdown 2024-08-02T17:49:53.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.861+0000 7fbf59ec8740 1 -- 172.21.15.69:0/2042724654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559082386be0 msgr2=0x55908238e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:53.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.861+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559082386be0 0x55908238e910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbf40015700 tx=0x7fbf40009110 comp rx=0 tx=0).stop 2024-08-02T17:49:53.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.861+0000 7fbf59ec8740 1 -- 172.21.15.69:0/2042724654 shutdown_connections 2024-08-02T17:49:53.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.861+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559082386be0 0x55908238e910 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.861+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5590823a18c0 0x5590823a3d60 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.861+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.861+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/2042724654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823862c0 0x55908238e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.861+0000 7fbf59ec8740 1 -- 172.21.15.69:0/2042724654 >> 172.21.15.69:0/2042724654 conn(0x559082382850 msgr2=0x559082381b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:53.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.862+0000 7fbf397fa640 0 client.16344 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.862+0000 7fbf59ec8740 1 -- 172.21.15.69:0/2042724654 shutdown_connections 2024-08-02T17:49:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.862+0000 7fbf59ec8740 1 -- 172.21.15.69:0/2042724654 wait complete. 2024-08-02T17:49:53.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.862+0000 7fbf59ec8740 20 client.16344 trim_cache size 0 max 16384 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf59ec8740 1 Processor -- start 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf59ec8740 1 -- start start 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823862c0 0x5590823a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559082386be0 0x5590823a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf501c9280 con 0x5590823862c0 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf501c6b50 con 0x5590823914a0 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf501c49a0 con 0x559082386be0 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf56067640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf56067640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:41568/0 (socket says 172.21.15.69:41568) 2024-08-02T17:49:53.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf56067640 1 -- 172.21.15.69:0/3785870347 learned_addr learned my addr 172.21.15.69:0/3785870347 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:53.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.889+0000 7fbf55065640 1 --2- 172.21.15.69:0/3785870347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559082386be0 0x5590823a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf56067640 1 -- 172.21.15.69:0/3785870347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559082386be0 msgr2=0x5590823a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf56067640 1 --2- 172.21.15.69:0/3785870347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559082386be0 0x5590823a9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf56067640 1 -- 172.21.15.69:0/3785870347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823862c0 msgr2=0x5590823a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:53.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf56067640 1 --2- 172.21.15.69:0/3785870347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823862c0 0x5590823a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf56067640 1 -- 172.21.15.69:0/3785870347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559082386a10 con 0x5590823914a0 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf56067640 1 --2- 172.21.15.69:0/3785870347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238b630 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0c4210 tx=0x7fbf4c0b7e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5f541aa1982c791 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf3b7fe640 1 -- 172.21.15.69:0/3785870347 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0bc1d0 con 0x5590823914a0 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf3b7fe640 1 -- 172.21.15.69:0/3785870347 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c0bbad0 con 0x5590823914a0 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.890+0000 7fbf3b7fe640 1 -- 172.21.15.69:0/3785870347 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0c42b0 con 0x5590823914a0 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3785870347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 msgr2=0x55908238b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/3785870347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238b630 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0c4210 tx=0x7fbf4c0b7e90 comp rx=0 tx=0).stop 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3785870347 shutdown_connections 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/3785870347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5590823914a0 0x55908238b630 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/3785870347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559082386be0 0x5590823a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/3785870347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823862c0 0x5590823a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3785870347 >> 172.21.15.69:0/3785870347 conn(0x559082382850 msgr2=0x559082389680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:53.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3785870347 shutdown_connections 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.891+0000 7fbf59ec8740 1 -- 172.21.15.69:0/3785870347 wait complete. 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 1 -- start start 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 1 Processor -- start 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823862c0 0x559082394bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082395130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823914a0 0x559082404600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf501c5010 con 0x5590823862c0 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf501c6480 con 0x559082386be0 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf501c56e0 con 0x5590823914a0 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf59ec8740 10 client.0 _ll_register_callbacks cb 0x7fbf40009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.892+0000 7fbf55065640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082395130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf55065640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082395130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:41570/0 (socket says 172.21.15.69:41570) 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf55065640 1 -- 172.21.15.69:0/143325819 learned_addr learned my addr 172.21.15.69:0/143325819 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf56067640 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823914a0 0x559082404600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf55065640 1 -- 172.21.15.69:0/143325819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823914a0 msgr2=0x559082404600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf55065640 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823914a0 0x559082404600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf55065640 1 -- 172.21.15.69:0/143325819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823862c0 msgr2=0x559082394bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf55065640 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823862c0 0x559082394bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf55065640 1 -- 172.21.15.69:0/143325819 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559082404bf0 con 0x559082386be0 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf55065640 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082395130 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf40003870 tx=0x7fbf40004010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb2a9de3691fdb70 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf397fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf397fa640 1 -- 172.21.15.69:0/143325819 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf40023040 con 0x559082386be0 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf59ec8740 1 -- 172.21.15.69:0/143325819 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf4c0bbad0 con 0x559082386be0 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf397fa640 1 -- 172.21.15.69:0/143325819 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbf40013510 con 0x559082386be0 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.893+0000 7fbf397fa640 1 -- 172.21.15.69:0/143325819 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbf40013710 con 0x559082386be0 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf59ec8740 10 client.25946 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf59ec8740 1 -- 172.21.15.69:0/143325819 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55908240bf60 con 0x559082386be0 2024-08-02T17:49:53.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf397fa640 1 -- 172.21.15.69:0/143325819 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c0bbad0 con 0x559082386be0 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf397fa640 1 -- 172.21.15.69:0/143325819 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbf4001e050 con 0x559082386be0 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf59ec8740 10 client.25946 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf59ec8740 10 client.25946 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf59ec8740 20 client.25946 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55908240c780 0x55908240eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf59ec8740 4 client.25946 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.85, "await_for": 45.0}] 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.894+0000 7fbf59ec8740 1 -- 172.21.15.69:0/143325819 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.85, "await_for": 45.0}) -- 0x55908226ff40 con 0x55908240c780 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.895+0000 7fbf55866640 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55908240c780 0x55908240eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:53.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.895+0000 7fbf55866640 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55908240c780 0x55908240eba0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1712f0b4dca68ca4 server_cookie=d7ebbf960d329154 in_seq=0 out_seq=0 2024-08-02T17:49:53.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:53.895+0000 7fbf397fa640 10 client.25946 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:54.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf397fa640 1 -- 172.21.15.69:0/143325819 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55908226ff40 con 0x55908240c780 2024-08-02T17:49:54.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf397fa640 10 client.25946 handle_command_reply: tid=0 2024-08-02T17:49:54.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 client.25946 shutdown 2024-08-02T17:49:54.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 -- 172.21.15.69:0/143325819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 msgr2=0x559082395130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:54.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082395130 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf40003870 tx=0x7fbf40004010 comp rx=0 tx=0).stop 2024-08-02T17:49:54.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 -- 172.21.15.69:0/143325819 shutdown_connections 2024-08-02T17:49:54.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55908240c780 0x55908240eba0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:54.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5590823914a0 0x559082404600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:54.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559082386be0 0x559082395130 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:54.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf397fa640 0 client.25946 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:54.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 --2- 172.21.15.69:0/143325819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5590823862c0 0x559082394bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:54.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 -- 172.21.15.69:0/143325819 >> 172.21.15.69:0/143325819 conn(0x559082382850 msgr2=0x559082381310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:54.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 -- 172.21.15.69:0/143325819 shutdown_connections 2024-08-02T17:49:54.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.151+0000 7fbf59ec8740 1 -- 172.21.15.69:0/143325819 wait complete. 2024-08-02T17:49:54.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf59ec8740 20 client.25946 trim_cache size 0 max 16384 2024-08-02T17:49:54.154 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 14, 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "48023e9": { 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 14, 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.8, 2024-08-02T17:49:54.155 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:49:54.156 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:49:54.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf3c0a2f00 msgr2=0x7fbf3c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:54.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf3c0a2f00 0x7fbf3c0a53c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbf40006e10 tx=0x7fbf40006d60 comp rx=0 tx=0).stop 2024-08-02T17:49:54.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf50119a20 msgr2=0x7fbf50119ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:54.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf50119a20 0x7fbf50119ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009850 tx=0x7fbf4c003b90 comp rx=0 tx=0).stop 2024-08-02T17:49:54.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 shutdown_connections 2024-08-02T17:49:54.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf5010e2b0 0x7fbf5011ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:54.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf3c0a2f00 0x7fbf3c0a53c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:54.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf50119a20 0x7fbf50119ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:54.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 --2- 172.21.15.69:0/1695327917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf5010d6e0 0x7fbf5011e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:54.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 >> 172.21.15.69:0/1695327917 conn(0x7fbf5006de70 msgr2=0x7fbf50072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:54.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 shutdown_connections 2024-08-02T17:49:54.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:49:54.152+0000 7fbf57aca640 1 -- 172.21.15.69:0/1695327917 wait complete. 2024-08-02T17:49:54.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 165.0 s, 1617.9 tps, lat 50.198 ms stddev 58.815 2024-08-02T17:49:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:55 smithi077 ceph-mon[37108]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 941 KiB/s wr, 108 op/s 2024-08-02T17:49:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:55 smithi069 ceph-mon[32746]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 941 KiB/s wr, 108 op/s 2024-08-02T17:49:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:55 smithi083 ceph-mon[37031]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 941 KiB/s wr, 108 op/s 2024-08-02T17:49:55.665 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '48023e9', quiesce took 5.1 seconds. Will release after: 15.750000000000002 2024-08-02T17:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 -- 172.21.15.69:0/510175089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 msgr2=0x7fcc6c100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 --2- 172.21.15.69:0/510175089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 0x7fcc6c100b70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcc68009930 tx=0x7fcc6802a420 comp rx=0 tx=0).stop 2024-08-02T17:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 -- 172.21.15.69:0/510175089 shutdown_connections 2024-08-02T17:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 --2- 172.21.15.69:0/510175089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 0x7fcc6c100b70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 --2- 172.21.15.69:0/510175089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc6c1010b0 0x7fcc6c10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 --2- 172.21.15.69:0/510175089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c106720 0x7fcc6c106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 -- 172.21.15.69:0/510175089 >> 172.21.15.69:0/510175089 conn(0x7fcc6c0fc3f0 msgr2=0x7fcc6c0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 -- 172.21.15.69:0/510175089 shutdown_connections 2024-08-02T17:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.683+0000 7fcc74106640 1 -- 172.21.15.69:0/510175089 wait complete. 2024-08-02T17:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc74106640 1 Processor -- start 2024-08-02T17:49:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc74106640 1 -- start start 2024-08-02T17:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc74106640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 0x7fcc6c198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc74106640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc6c1010b0 0x7fcc6c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc74106640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c106720 0x7fcc6c1a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc74106640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c110b20 con 0x7fcc6c100710 2024-08-02T17:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc74106640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1109a0 con 0x7fcc6c106720 2024-08-02T17:49:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc74106640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c110ca0 con 0x7fcc6c1010b0 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc71ea2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 0x7fcc6c198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc71ea2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 0x7fcc6c198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35508/0 (socket says 172.21.15.69:35508) 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc71ea2640 1 -- 172.21.15.69:0/1216408057 learned_addr learned my addr 172.21.15.69:0/1216408057 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.684+0000 7fcc716a1640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc6c1010b0 0x7fcc6c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc726a3640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c106720 0x7fcc6c1a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc71ea2640 1 -- 172.21.15.69:0/1216408057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc6c1010b0 msgr2=0x7fcc6c199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc6c1010b0 0x7fcc6c199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc71ea2640 1 -- 172.21.15.69:0/1216408057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c106720 msgr2=0x7fcc6c1a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c106720 0x7fcc6c1a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc71ea2640 1 -- 172.21.15.69:0/1216408057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c1a4590 con 0x7fcc6c100710 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 0x7fcc6c198fd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00d950 tx=0x7fcc5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc62ffd640 1 -- 172.21.15.69:0/1216408057 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c017070 con 0x7fcc6c100710 2024-08-02T17:49:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc62ffd640 1 -- 172.21.15.69:0/1216408057 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc5c009c30 con 0x7fcc6c100710 2024-08-02T17:49:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc74106640 1 -- 172.21.15.69:0/1216408057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6c1a4760 con 0x7fcc6c100710 2024-08-02T17:49:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc62ffd640 1 -- 172.21.15.69:0/1216408057 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c00f6b0 con 0x7fcc6c100710 2024-08-02T17:49:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.685+0000 7fcc74106640 1 -- 172.21.15.69:0/1216408057 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6c1a4c80 con 0x7fcc6c100710 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.687+0000 7fcc62ffd640 1 -- 172.21.15.69:0/1216408057 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcc5c0109d0 con 0x7fcc6c100710 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc76e56740 1 Processor -- start 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc76e56740 1 -- start start 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730917b0 0x555e73091b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7309c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc76e56740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555e72ff59b0 con 0x555e730917b0 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc76e56740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e72f9d800 con 0x555e7309c970 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc76e56740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e72ff1970 con 0x555e730920d0 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc726a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730917b0 0x555e73091b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc726a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730917b0 0x555e73091b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35510/0 (socket says 172.21.15.69:35510) 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc726a3640 1 -- 172.21.15.69:0/3820980609 learned_addr learned my addr 172.21.15.69:0/3820980609 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc62ffd640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc580a2cf0 0x7fcc580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc716a1640 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.688+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7309c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc726a3640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc580a2cf0 0x7fcc580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc62ffd640 1 -- 172.21.15.69:0/1216408057 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcc5c0cf3b0 con 0x7fcc6c100710 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc726a3640 1 -- 172.21.15.69:0/3820980609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 msgr2=0x555e7309c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc726a3640 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7309c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc726a3640 1 -- 172.21.15.69:0/3820980609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 msgr2=0x555e7309ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc726a3640 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc726a3640 1 -- 172.21.15.69:0/3820980609 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e72d09210 con 0x555e730917b0 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7309c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:49:55.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc716a1640 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc726a3640 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730917b0 0x555e73091b90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcc640149c0 tx=0x7fcc64014dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756ed1a72752cff9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc57fff640 1 -- 172.21.15.69:0/3820980609 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6401eb40 con 0x555e730917b0 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc57fff640 1 -- 172.21.15.69:0/3820980609 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc64023070 con 0x555e730917b0 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.689+0000 7fcc57fff640 1 -- 172.21.15.69:0/3820980609 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400cdd0 con 0x555e730917b0 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 -- 172.21.15.69:0/3820980609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730917b0 msgr2=0x555e73091b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc726a3640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc580a2cf0 0x7fcc580a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc64022000 tx=0x7fcc64020000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730917b0 0x555e73091b90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcc640149c0 tx=0x7fcc64014dc0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 -- 172.21.15.69:0/3820980609 shutdown_connections 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7309c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3820980609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730917b0 0x555e73091b90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 -- 172.21.15.69:0/3820980609 >> 172.21.15.69:0/3820980609 conn(0x555e7308dd20 msgr2=0x555e7308e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:55.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 -- 172.21.15.69:0/3820980609 shutdown_connections 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 -- 172.21.15.69:0/3820980609 wait complete. 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 -- start start 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.690+0000 7fcc76e56740 1 Processor -- start 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730917b0 0x555e73099890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e73099dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc76e56740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555e7309f540 con 0x555e730920d0 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc76e56740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555e72fb1500 con 0x555e7309c970 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc76e56740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555e7309f6c0 con 0x555e730917b0 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc76e56740 10 client.0 _ll_register_callbacks cb 0x555e72d08a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:55.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc716a1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e73099dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc716a1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e73099dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35520/0 (socket says 172.21.15.69:35520) 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc716a1640 1 -- 172.21.15.69:0/3930624037 learned_addr learned my addr 172.21.15.69:0/3930624037 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730917b0 0x555e73099890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc726a3640 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc716a1640 1 -- 172.21.15.69:0/3930624037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730917b0 msgr2=0x555e73099890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc716a1640 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730917b0 0x555e73099890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc716a1640 1 -- 172.21.15.69:0/3930624037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 msgr2=0x555e7309a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc716a1640 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.691+0000 7fcc716a1640 1 -- 172.21.15.69:0/3930624037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e72dd5620 con 0x555e730920d0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc716a1640 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e73099dd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcc6802a940 tx=0x7fcc68031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982af165f5f3611d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc557fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc557fa640 1 -- 172.21.15.69:0/3930624037 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68035040 con 0x555e730920d0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc557fa640 1 -- 172.21.15.69:0/3930624037 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcc68004240 con 0x555e730920d0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 1 -- 172.21.15.69:0/3930624037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc64023070 con 0x555e730920d0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc557fa640 1 -- 172.21.15.69:0/3930624037 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcc6804f050 con 0x555e730920d0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 10 client.16356 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 1 -- 172.21.15.69:0/3930624037 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e730af620 con 0x555e730920d0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc557fa640 1 -- 172.21.15.69:0/3930624037 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc64023070 con 0x555e730920d0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc557fa640 1 -- 172.21.15.69:0/3930624037 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcc680465f0 con 0x555e730920d0 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 10 client.16356 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:55.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 20 client.16356 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e730b0ae0 0x555e730b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 4 client.16356 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.692+0000 7fcc76e56740 1 -- 172.21.15.69:0/3930624037 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e730030e0 con 0x555e730b0ae0 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.693+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e730b0ae0 0x555e730b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.693+0000 7fcc557fa640 10 client.16356 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.693+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e730b0ae0 0x555e730b2ed0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c12fa5fdb9ba0a6d server_cookie=a68534b24c7b0973 in_seq=0 out_seq=0 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc557fa640 1 -- 172.21.15.69:0/3930624037 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555e730030e0 con 0x555e730b0ae0 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc557fa640 10 client.16356 handle_command_reply: tid=0 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 client.16356 shutdown 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 -- 172.21.15.69:0/3930624037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 msgr2=0x555e73099dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e73099dd0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcc6802a940 tx=0x7fcc68031920 comp rx=0 tx=0).stop 2024-08-02T17:49:55.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 -- 172.21.15.69:0/3930624037 shutdown_connections 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e73099dd0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e730b0ae0 0x555e730b2ed0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e7309c970 0x555e7309a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc557fa640 0 client.16356 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3930624037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730917b0 0x555e73099890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 -- 172.21.15.69:0/3930624037 >> 172.21.15.69:0/3930624037 conn(0x555e7308dd20 msgr2=0x555e7308ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 -- 172.21.15.69:0/3930624037 shutdown_connections 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 1 -- 172.21.15.69:0/3930624037 wait complete. 2024-08-02T17:49:55.704 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.694+0000 7fcc76e56740 20 client.16356 trim_cache size 0 max 16384 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.709+0000 7fcc76e56740 1 Processor -- start 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.709+0000 7fcc76e56740 1 -- start start 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e7309a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7308cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e7309c970 0x555e730b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc76e56740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1af390 con 0x555e7309c970 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc76e56740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1acc60 con 0x555e730917b0 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc76e56740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1aaab0 con 0x555e730920d0 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e7309c970 0x555e730b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e7309c970 0x555e730b87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35524/0 (socket says 172.21.15.69:35524) 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 -- 172.21.15.69:0/3946351984 learned_addr learned my addr 172.21.15.69:0/3946351984 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:55.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e7309a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc716a1640 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7308cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 -- 172.21.15.69:0/3946351984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 msgr2=0x555e7308cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7308cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 -- 172.21.15.69:0/3946351984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 msgr2=0x555e7309a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e7309a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 -- 172.21.15.69:0/3946351984 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e73087e80 con 0x555e7309c970 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e7309a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc726a3640 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e7309c970 0x555e730b87c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009640 tx=0x7fcc64011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acdab321e0ec1a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc577fe640 1 -- 172.21.15.69:0/3946351984 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64037070 con 0x555e7309c970 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc577fe640 1 -- 172.21.15.69:0/3946351984 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc64023070 con 0x555e7309c970 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.710+0000 7fcc577fe640 1 -- 172.21.15.69:0/3946351984 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64032480 con 0x555e7309c970 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 -- 172.21.15.69:0/3946351984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e7309c970 msgr2=0x555e730b87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e7309c970 0x555e730b87c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009640 tx=0x7fcc64011000 comp rx=0 tx=0).stop 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 -- 172.21.15.69:0/3946351984 shutdown_connections 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e7309c970 0x555e730b87c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e730920d0 0x555e7308cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 --2- 172.21.15.69:0/3946351984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e7309a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 -- 172.21.15.69:0/3946351984 >> 172.21.15.69:0/3946351984 conn(0x555e7308dd20 msgr2=0x555e7308c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 -- 172.21.15.69:0/3946351984 shutdown_connections 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 -- 172.21.15.69:0/3946351984 wait complete. 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 -- start start 2024-08-02T17:49:55.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.711+0000 7fcc76e56740 1 Processor -- start 2024-08-02T17:49:55.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e730a9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e730a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc76e56740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e7309c970 0x555e730a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc76e56740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1ab120 con 0x555e730920d0 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc76e56740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1ac590 con 0x555e730917b0 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc76e56740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c1ab7f0 con 0x555e7309c970 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc76e56740 10 client.0 _ll_register_callbacks cb 0x7fcc68009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e730a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e730a49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35526/0 (socket says 172.21.15.69:35526) 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 -- 172.21.15.69:0/591939815 learned_addr learned my addr 172.21.15.69:0/591939815 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e730a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 -- 172.21.15.69:0/591939815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e7309c970 msgr2=0x555e730a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e7309c970 0x555e730a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 -- 172.21.15.69:0/591939815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 msgr2=0x555e730a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e730a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 -- 172.21.15.69:0/591939815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e7311a790 con 0x555e730920d0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc716a1640 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e730a49a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcc68046b60 tx=0x7fcc68005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c4df0f471085c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.712+0000 7fcc557fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc557fa640 1 -- 172.21.15.69:0/591939815 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68035040 con 0x555e730920d0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc557fa640 1 -- 172.21.15.69:0/591939815 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcc68004450 con 0x555e730920d0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 1 -- 172.21.15.69:0/591939815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc64023070 con 0x555e730920d0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc557fa640 1 -- 172.21.15.69:0/591939815 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcc68061050 con 0x555e730920d0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 10 client.16362 fetch_fsmap learned FSMap version 26 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 1 -- 172.21.15.69:0/591939815 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555e7311b860 con 0x555e730920d0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc557fa640 1 -- 172.21.15.69:0/591939815 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc64023070 con 0x555e730920d0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc557fa640 1 -- 172.21.15.69:0/591939815 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcc6803d480 con 0x555e730920d0 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 10 client.16362 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:49:55.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:49:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 20 client.16362 populate_metadata read hostname 'smithi069' 2024-08-02T17:49:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e7311c000 0x555e7311e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:49:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 4 client.16362 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:49:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc76e56740 1 -- 172.21.15.69:0/591939815 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e72ffd430 con 0x555e7311c000 2024-08-02T17:49:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.713+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e7311c000 0x555e7311e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:49:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.714+0000 7fcc557fa640 10 client.16362 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:55.721 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.714+0000 7fcc71ea2640 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e7311c000 0x555e7311e420 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f397b2b50ac9e3 server_cookie=8fe050225fdb3099 in_seq=0 out_seq=0 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc557fa640 1 -- 172.21.15.69:0/591939815 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e72ffd430 con 0x555e7311c000 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc557fa640 10 client.16362 handle_command_reply: tid=0 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 client.16362 shutdown 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 -- 172.21.15.69:0/591939815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 msgr2=0x555e730a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e730a49a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcc68046b60 tx=0x7fcc68005fd0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 -- 172.21.15.69:0/591939815 shutdown_connections 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555e730920d0 0x555e730a49a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555e7311c000 0x555e7311e420 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555e7309c970 0x555e730a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc557fa640 0 client.16362 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:49:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 --2- 172.21.15.69:0/591939815 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555e730917b0 0x555e730a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 -- 172.21.15.69:0/591939815 >> 172.21.15.69:0/591939815 conn(0x555e7308dd20 msgr2=0x555e7308ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 -- 172.21.15.69:0/591939815 shutdown_connections 2024-08-02T17:49:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 1 -- 172.21.15.69:0/591939815 wait complete. 2024-08-02T17:49:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.729+0000 7fcc76e56740 20 client.16362 trim_cache size 0 max 16384 2024-08-02T17:49:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:49:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:49:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:49:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:49:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 -- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc580a2cf0 msgr2=0x7fcc580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc580a2cf0 0x7fcc580a51b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc64022000 tx=0x7fcc64020000 comp rx=0 tx=0).stop 2024-08-02T17:49:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 -- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 msgr2=0x7fcc6c198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 0x7fcc6c198fd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00d950 tx=0x7fcc5c00de10 comp rx=0 tx=0).stop 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 -- 172.21.15.69:0/1216408057 shutdown_connections 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc580a2cf0 0x7fcc580a51b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc6c106720 0x7fcc6c1a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc6c1010b0 0x7fcc6c199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 --2- 172.21.15.69:0/1216408057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc6c100710 0x7fcc6c198fd0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 -- 172.21.15.69:0/1216408057 >> 172.21.15.69:0/1216408057 conn(0x7fcc6c0fc3f0 msgr2=0x7fcc6c0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 -- 172.21.15.69:0/1216408057 shutdown_connections 2024-08-02T17:49:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:49:55.730+0000 7fcc74106640 1 -- 172.21.15.69:0/1216408057 wait complete. 2024-08-02T17:49:57.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:57 smithi077 ceph-mon[37108]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-08-02T17:49:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:57 smithi069 ceph-mon[32746]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-08-02T17:49:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:57 smithi083 ceph-mon[37031]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-08-02T17:49:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:49:59 smithi077 ceph-mon[37108]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-08-02T17:49:59.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:49:59 smithi069 ceph-mon[32746]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-08-02T17:49:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:49:59 smithi083 ceph-mon[37031]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-08-02T17:49:59.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 170.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:50:00.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:00 smithi077 ceph-mon[37108]: overall HEALTH_OK 2024-08-02T17:50:00.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:00 smithi069 ceph-mon[32746]: overall HEALTH_OK 2024-08-02T17:50:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:00 smithi083 ceph-mon[37031]: overall HEALTH_OK 2024-08-02T17:50:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:01 smithi077 ceph-mon[37108]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-08-02T17:50:01.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:01 smithi069 ceph-mon[32746]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-08-02T17:50:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:01 smithi083 ceph-mon[37031]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 2.3 MiB/s wr, 259 op/s 2024-08-02T17:50:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:03 smithi077 ceph-mon[37108]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-08-02T17:50:03.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:03 smithi069 ceph-mon[32746]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-08-02T17:50:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:03 smithi083 ceph-mon[37031]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-08-02T17:50:04.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 175.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:50:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:05 smithi077 ceph-mon[37108]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-08-02T17:50:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:05 smithi069 ceph-mon[32746]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-08-02T17:50:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:05 smithi083 ceph-mon[37031]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-08-02T17:50:07.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:07 smithi083 ceph-mon[37031]: pgmap v114: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-08-02T17:50:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:07 smithi077 ceph-mon[37108]: pgmap v114: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-08-02T17:50:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:07 smithi069 ceph-mon[32746]: pgmap v114: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.9 MiB/s wr, 269 op/s 2024-08-02T17:50:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.908+0000 7fbbf53d0640 1 -- 172.21.15.69:0/1481231262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 msgr2=0x7fbbe80a8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:08.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.908+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/1481231262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 0x7fbbe80a8b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0067690 tx=0x7fbbf009eea0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.914+0000 7fbbf53d0640 1 -- 172.21.15.69:0/1481231262 shutdown_connections 2024-08-02T17:50:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.914+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/1481231262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbe809c830 0x7fbbe80a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.914+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/1481231262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 0x7fbbe80a8b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.914+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/1481231262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe809be80 0x7fbbe809c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.914+0000 7fbbf53d0640 1 -- 172.21.15.69:0/1481231262 >> 172.21.15.69:0/1481231262 conn(0x7fbbe800d3e0 msgr2=0x7fbbe809a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.920+0000 7fbbf53d0640 1 -- 172.21.15.69:0/1481231262 shutdown_connections 2024-08-02T17:50:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.921+0000 7fbbf53d0640 1 -- 172.21.15.69:0/1481231262 wait complete. 2024-08-02T17:50:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.921+0000 7fbbf53d0640 1 Processor -- start 2024-08-02T17:50:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.922+0000 7fbbf53d0640 1 -- start start 2024-08-02T17:50:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.922+0000 7fbbf53d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe809be80 0x7fbbe8132620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.922+0000 7fbbf53d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 0x7fbbe8132b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.922+0000 7fbbf53d0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbe8136ee0 0x7fbbe8137390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.922+0000 7fbbf53d0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80abff0 con 0x7fbbe8136ee0 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.922+0000 7fbbf53d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80abe70 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.922+0000 7fbbf53d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80ac170 con 0x7fbbe809be80 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.923+0000 7fbbeeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe809be80 0x7fbbe8132620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.924+0000 7fbbee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 0x7fbbe8132b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.924+0000 7fbbeeffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe809be80 0x7fbbe8132620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57328/0 (socket says 172.21.15.69:57328) 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.924+0000 7fbbeeffd640 1 -- 172.21.15.69:0/3358323877 learned_addr learned my addr 172.21.15.69:0/3358323877 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.924+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbe8136ee0 0x7fbbe8137390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.927+0000 7fbbee7fc640 1 -- 172.21.15.69:0/3358323877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe809be80 msgr2=0x7fbbe8132620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.927+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe809be80 0x7fbbe8132620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.927+0000 7fbbee7fc640 1 -- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbe8136ee0 msgr2=0x7fbbe8137390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.931+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbe8136ee0 0x7fbbe8137390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.931+0000 7fbbee7fc640 1 -- 172.21.15.69:0/3358323877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe81379d0 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.931+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbe8136ee0 0x7fbbe8137390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.935+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 0x7fbbe8132b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009fd0 tx=0x7fbbe000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.936+0000 7fbbd7fff640 1 -- 172.21.15.69:0/3358323877 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0015020 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.936+0000 7fbbf53d0640 1 -- 172.21.15.69:0/3358323877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe8137cc0 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.936+0000 7fbbf53d0640 1 -- 172.21.15.69:0/3358323877 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81614c0 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.936+0000 7fbbd7fff640 1 -- 172.21.15.69:0/3358323877 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe0011d10 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.936+0000 7fbbd7fff640 1 -- 172.21.15.69:0/3358323877 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000f2e0 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.947+0000 7fbbf81ff740 1 Processor -- start 2024-08-02T17:50:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.947+0000 7fbbd7fff640 1 -- 172.21.15.69:0/3358323877 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbbe0018020 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.948+0000 7fbbd7fff640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbd80a2f00 0x7fbbd80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.948+0000 7fbbd7fff640 1 -- 172.21.15.69:0/3358323877 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbbe00cfe40 con 0x7fbbe80a0fd0 2024-08-02T17:50:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.950+0000 7fbbf81ff740 1 -- start start 2024-08-02T17:50:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.951+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3b940 0x559f15d3bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.951+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.951+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d46b00 0x559f15d48ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.951+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f15c9f9b0 con 0x559f15d46b00 2024-08-02T17:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.951+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f15c47800 con 0x559f15d3c260 2024-08-02T17:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.951+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f15c9b970 con 0x559f15d3b940 2024-08-02T17:50:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.951+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbd80a2f00 0x7fbbd80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.953+0000 7fbbee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.953+0000 7fbbef7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d46b00 0x559f15d48ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.953+0000 7fbbef7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d46b00 0x559f15d48ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41196/0 (socket says 172.21.15.69:41196) 2024-08-02T17:50:08.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.953+0000 7fbbef7fe640 1 -- 172.21.15.69:0/2708998215 learned_addr learned my addr 172.21.15.69:0/2708998215 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:08.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.953+0000 7fbbeeffd640 1 --2- 172.21.15.69:0/2708998215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3b940 0x559f15d3bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.957 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.956+0000 7fbbee7fc640 1 -- 172.21.15.69:0/2708998215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3b940 msgr2=0x559f15d3bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.956+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/2708998215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3b940 0x559f15d3bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.956+0000 7fbbee7fc640 1 -- 172.21.15.69:0/2708998215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d46b00 msgr2=0x559f15d48ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.956+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/2708998215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d46b00 0x559f15d48ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.956+0000 7fbbee7fc640 1 -- 172.21.15.69:0/2708998215 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f159b3210 con 0x559f15d3c260 2024-08-02T17:50:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.956+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/2708998215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d465c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00cd280 tx=0x7fbbe009ba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4633b7a3c1ed0e5e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:08.958 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.957+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbd80a2f00 0x7fbbd80a53c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0051230 tx=0x7fbbf0068000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.959+0000 7fbbd57fa640 1 -- 172.21.15.69:0/2708998215 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe009c050 con 0x559f15d3c260 2024-08-02T17:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.959+0000 7fbbd57fa640 1 -- 172.21.15.69:0/2708998215 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe00a0400 con 0x559f15d3c260 2024-08-02T17:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.960+0000 7fbbf81ff740 1 -- 172.21.15.69:0/2708998215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 msgr2=0x559f15d465c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:08.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.960+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/2708998215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d465c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00cd280 tx=0x7fbbe009ba70 comp rx=0 tx=0).stop 2024-08-02T17:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.965+0000 7fbbf81ff740 1 -- 172.21.15.69:0/2708998215 shutdown_connections 2024-08-02T17:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.965+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/2708998215 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d46b00 0x559f15d48ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.965+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/2708998215 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d465c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.966 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.965+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/2708998215 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3b940 0x559f15d3bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.967 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.965+0000 7fbbf81ff740 1 -- 172.21.15.69:0/2708998215 >> 172.21.15.69:0/2708998215 conn(0x559f15d37eb0 msgr2=0x559f15d38290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.971+0000 7fbbf81ff740 1 -- 172.21.15.69:0/2708998215 shutdown_connections 2024-08-02T17:50:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.971+0000 7fbbf81ff740 1 -- 172.21.15.69:0/2708998215 wait complete. 2024-08-02T17:50:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.971+0000 7fbbf81ff740 1 -- start start 2024-08-02T17:50:08.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.971+0000 7fbbf81ff740 1 Processor -- start 2024-08-02T17:50:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d46b00 0x559f15d44070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f15d49720 con 0x559f15d3b940 2024-08-02T17:50:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f15c5b500 con 0x559f15d3c260 2024-08-02T17:50:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f15d498a0 con 0x559f15d46b00 2024-08-02T17:50:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbf81ff740 10 client.0 _ll_register_callbacks cb 0x559f159b2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d46b00 0x559f15d44070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d46b00 0x559f15d44070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57348/0 (socket says 172.21.15.69:57348) 2024-08-02T17:50:08.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbef7fe640 1 -- 172.21.15.69:0/3825973278 learned_addr learned my addr 172.21.15.69:0/3825973278 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.972+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.973+0000 7fbbeeffd640 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.973+0000 7fbbef7fe640 1 -- 172.21.15.69:0/3825973278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 msgr2=0x559f15d438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.973+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.973+0000 7fbbef7fe640 1 -- 172.21.15.69:0/3825973278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 msgr2=0x559f15d465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.973+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:08.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.973+0000 7fbbef7fe640 1 -- 172.21.15.69:0/3825973278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f15a7f620 con 0x559f15d46b00 2024-08-02T17:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.973+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d46b00 0x559f15d44070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0065090 tx=0x7fbbf009eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0edb73e297f6ac7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.974+0000 7fbbd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.974+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/3825973278 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00b6060 con 0x559f15d46b00 2024-08-02T17:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.974+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/3825973278 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbbf00a14e0 con 0x559f15d46b00 2024-08-02T17:50:08.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.974+0000 7fbbf81ff740 1 -- 172.21.15.69:0/3825973278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe00a0400 con 0x559f15d46b00 2024-08-02T17:50:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.981+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/3825973278 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbf0082040 con 0x559f15d46b00 2024-08-02T17:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.983+0000 7fbbf81ff740 10 client.25870 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.983+0000 7fbbf81ff740 1 -- 172.21.15.69:0/3825973278 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f15d4b4c0 con 0x559f15d46b00 2024-08-02T17:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.986+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/3825973278 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe00a0400 con 0x559f15d46b00 2024-08-02T17:50:08.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.986+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/3825973278 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbbf0085040 con 0x559f15d46b00 2024-08-02T17:50:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.986+0000 7fbbf81ff740 10 client.25870 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.986+0000 7fbbf81ff740 10 client.25870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.986+0000 7fbbf81ff740 20 client.25870 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.986+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f15d4cc10 0x559f15d4f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.986+0000 7fbbf81ff740 4 client.25870 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:08.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.986+0000 7fbbf81ff740 1 -- 172.21.15.69:0/3825973278 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f15cad0e0 con 0x559f15d4cc10 2024-08-02T17:50:08.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.987+0000 7fbbeeffd640 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f15d4cc10 0x559f15d4f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.994+0000 7fbbeeffd640 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f15d4cc10 0x559f15d4f070 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740b4de1d420e89b server_cookie=e520e9fd1343d74c in_seq=0 out_seq=0 2024-08-02T17:50:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.995+0000 7fbbd2ff5640 10 client.25870 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.996+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/3825973278 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f15cad0e0 con 0x559f15d4cc10 2024-08-02T17:50:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.996+0000 7fbbd2ff5640 10 client.25870 handle_command_reply: tid=0 2024-08-02T17:50:08.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:08.996+0000 7fbbf81ff740 1 client.25870 shutdown 2024-08-02T17:50:09.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.004+0000 7fbbf81ff740 1 -- 172.21.15.69:0/3825973278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d46b00 msgr2=0x559f15d44070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:09.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.004+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d46b00 0x559f15d44070 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0065090 tx=0x7fbbf009eea0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.008+0000 7fbbf81ff740 1 -- 172.21.15.69:0/3825973278 shutdown_connections 2024-08-02T17:50:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.008+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f15d4cc10 0x559f15d4f070 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.008+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d46b00 0x559f15d44070 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.008+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.008+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/3825973278 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.008+0000 7fbbf81ff740 1 -- 172.21.15.69:0/3825973278 >> 172.21.15.69:0/3825973278 conn(0x559f15d37eb0 msgr2=0x559f15d3dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:09.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.008+0000 7fbbd2ff5640 0 client.25870 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.011+0000 7fbbf81ff740 1 -- 172.21.15.69:0/3825973278 shutdown_connections 2024-08-02T17:50:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.011+0000 7fbbf81ff740 1 -- 172.21.15.69:0/3825973278 wait complete. 2024-08-02T17:50:09.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.012+0000 7fbbf81ff740 20 client.25870 trim_cache size 0 max 16384 2024-08-02T17:50:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.054+0000 7fbbf81ff740 1 Processor -- start 2024-08-02T17:50:09.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.054+0000 7fbbf81ff740 1 -- start start 2024-08-02T17:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.054+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d37eb0 0x559f15d386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.055+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d42920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.055+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d54940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:09.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.055+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc012900 con 0x559f15d3b940 2024-08-02T17:50:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.055+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc0101d0 con 0x559f15d3c260 2024-08-02T17:50:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.055+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc00e020 con 0x559f15d37eb0 2024-08-02T17:50:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.055+0000 7fbbee7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d42920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.055+0000 7fbbee7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d42920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41206/0 (socket says 172.21.15.69:41206) 2024-08-02T17:50:09.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.055+0000 7fbbee7fc640 1 -- 172.21.15.69:0/833083765 learned_addr learned my addr 172.21.15.69:0/833083765 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.056+0000 7fbbeeffd640 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d37eb0 0x559f15d386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.056+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d54940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.056+0000 7fbbee7fc640 1 -- 172.21.15.69:0/833083765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d37eb0 msgr2=0x559f15d386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.056+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d37eb0 0x559f15d386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.056+0000 7fbbee7fc640 1 -- 172.21.15.69:0/833083765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 msgr2=0x559f15d54940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:09.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.056+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d54940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.056+0000 7fbbee7fc640 1 -- 172.21.15.69:0/833083765 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f15d63a60 con 0x559f15d3b940 2024-08-02T17:50:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.056+0000 7fbbee7fc640 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d42920 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00984b0 tx=0x7fbbe00109f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34647f24921d2717 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.060+0000 7fbbd4ff9640 1 -- 172.21.15.69:0/833083765 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe009c050 con 0x559f15d3b940 2024-08-02T17:50:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.060+0000 7fbbd4ff9640 1 -- 172.21.15.69:0/833083765 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe00a4cc0 con 0x559f15d3b940 2024-08-02T17:50:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.060+0000 7fbbd4ff9640 1 -- 172.21.15.69:0/833083765 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00a4a90 con 0x559f15d3b940 2024-08-02T17:50:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.062+0000 7fbbf81ff740 1 -- 172.21.15.69:0/833083765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 msgr2=0x559f15d42920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.062+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d42920 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00984b0 tx=0x7fbbe00109f0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.064+0000 7fbbf81ff740 1 -- 172.21.15.69:0/833083765 shutdown_connections 2024-08-02T17:50:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.064+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d3b940 0x559f15d42920 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.064+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3c260 0x559f15d54940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.064+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/833083765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d37eb0 0x559f15d386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.064+0000 7fbbf81ff740 1 -- 172.21.15.69:0/833083765 >> 172.21.15.69:0/833083765 conn(0x559f15d46700 msgr2=0x559f15d42210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:09.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.070+0000 7fbbf81ff740 1 -- 172.21.15.69:0/833083765 shutdown_connections 2024-08-02T17:50:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.073+0000 7fbbf81ff740 1 -- 172.21.15.69:0/833083765 wait complete. 2024-08-02T17:50:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.074+0000 7fbbf81ff740 1 -- start start 2024-08-02T17:50:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.074+0000 7fbbf81ff740 1 Processor -- start 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.078+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d37eb0 0x559f15d54b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.078+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3b940 0x559f15d55080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.078+0000 7fbbf81ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3c260 0x559f15d51920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.078+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc00e690 con 0x559f15d37eb0 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.078+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc00fb00 con 0x559f15d3b940 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.078+0000 7fbbf81ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc00ed60 con 0x559f15d3c260 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.078+0000 7fbbf81ff740 10 client.0 _ll_register_callbacks cb 0x7fbbf006c5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.079+0000 7fbbee7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3b940 0x559f15d55080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.079+0000 7fbbef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3c260 0x559f15d51920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:09.085 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.079+0000 7fbbef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3c260 0x559f15d51920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57374/0 (socket says 172.21.15.69:57374) 2024-08-02T17:50:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.079+0000 7fbbef7fe640 1 -- 172.21.15.69:0/1469536842 learned_addr learned my addr 172.21.15.69:0/1469536842 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.080+0000 7fbbeeffd640 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d37eb0 0x559f15d54b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.080+0000 7fbbef7fe640 1 -- 172.21.15.69:0/1469536842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3b940 msgr2=0x559f15d55080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.080+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3b940 0x559f15d55080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.080+0000 7fbbef7fe640 1 -- 172.21.15.69:0/1469536842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d37eb0 msgr2=0x559f15d54b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.080+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d37eb0 0x559f15d54b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.080+0000 7fbbef7fe640 1 -- 172.21.15.69:0/1469536842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f15d51fc0 con 0x559f15d3c260 2024-08-02T17:50:09.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.083+0000 7fbbef7fe640 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3c260 0x559f15d51920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0050a90 tx=0x7fbbf0067ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=869bb36795d22809 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.084+0000 7fbbd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.084+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/1469536842 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00b6070 con 0x559f15d3c260 2024-08-02T17:50:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.085+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/1469536842 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbbf0079d10 con 0x559f15d3c260 2024-08-02T17:50:09.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.086+0000 7fbbf81ff740 1 -- 172.21.15.69:0/1469536842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe00a4cc0 con 0x559f15d3c260 2024-08-02T17:50:09.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.090+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/1469536842 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbbf0075000 con 0x559f15d3c260 2024-08-02T17:50:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.093+0000 7fbbf81ff740 10 client.25876 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.093+0000 7fbbf81ff740 1 -- 172.21.15.69:0/1469536842 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f15de34f0 con 0x559f15d3c260 2024-08-02T17:50:09.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.094+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/1469536842 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbe00a4cc0 con 0x559f15d3c260 2024-08-02T17:50:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.096+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/1469536842 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbbf00bf020 con 0x559f15d3c260 2024-08-02T17:50:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.096+0000 7fbbf81ff740 10 client.25876 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.096+0000 7fbbf81ff740 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.096+0000 7fbbf81ff740 20 client.25876 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.096+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f15d46b00 0x559f15de5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.096+0000 7fbbf81ff740 4 client.25876 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:50:09.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.096+0000 7fbbf81ff740 1 -- 172.21.15.69:0/1469536842 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f15ca7430 con 0x559f15d46b00 2024-08-02T17:50:09.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.098+0000 7fbbeeffd640 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f15d46b00 0x559f15de5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.108+0000 7fbbeeffd640 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f15d46b00 0x559f15de5c20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf10bf88cce4fb4 server_cookie=7b0b04153447f52c in_seq=0 out_seq=0 2024-08-02T17:50:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.108+0000 7fbbd2ff5640 10 client.25876 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:09.192 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:50:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:09 smithi077 ceph-mon[37108]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-08-02T17:50:09.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:09 smithi069 ceph-mon[32746]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-08-02T17:50:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:09 smithi083 ceph-mon[37031]: pgmap v115: 129 pgs: 129 active+clean; 1.9 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-08-02T17:50:09.694 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbd2ff5640 1 -- 172.21.15.69:0/1469536842 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f15ca7430 con 0x559f15d46b00 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbd2ff5640 10 client.25876 handle_command_reply: tid=0 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 client.25876 shutdown 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 -- 172.21.15.69:0/1469536842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3c260 msgr2=0x559f15d51920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3c260 0x559f15d51920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0050a90 tx=0x7fbbf0067ee0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 -- 172.21.15.69:0/1469536842 shutdown_connections 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f15d46b00 0x559f15de5c20 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f15d3c260 0x559f15d51920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f15d3b940 0x559f15d55080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 --2- 172.21.15.69:0/1469536842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f15d37eb0 0x559f15d54b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbd2ff5640 0 client.25876 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 -- 172.21.15.69:0/1469536842 >> 172.21.15.69:0/1469536842 conn(0x559f15d46700 msgr2=0x559f15d42a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 -- 172.21.15.69:0/1469536842 shutdown_connections 2024-08-02T17:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.934+0000 7fbbf81ff740 1 -- 172.21.15.69:0/1469536842 wait complete. 2024-08-02T17:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf81ff740 20 client.25876 trim_cache size 0 max 16384 2024-08-02T17:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:50:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:50:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 -- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbd80a2f00 msgr2=0x7fbbd80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbd80a2f00 0x7fbbd80a53c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0051230 tx=0x7fbbf0068000 comp rx=0 tx=0).stop 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 -- 172.21.15.69:0/3358323877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 msgr2=0x7fbbe8132b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 0x7fbbe8132b60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009fd0 tx=0x7fbbe000b870 comp rx=0 tx=0).stop 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 -- 172.21.15.69:0/3358323877 shutdown_connections 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbbe8136ee0 0x7fbbe8137390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbd80a2f00 0x7fbbd80a53c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbbe80a0fd0 0x7fbbe8132b60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 --2- 172.21.15.69:0/3358323877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbbe809be80 0x7fbbe8132620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 -- 172.21.15.69:0/3358323877 >> 172.21.15.69:0/3358323877 conn(0x7fbbe800d3e0 msgr2=0x7fbbe8008a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 -- 172.21.15.69:0/3358323877 shutdown_connections 2024-08-02T17:50:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:09.935+0000 7fbbf53d0640 1 -- 172.21.15.69:0/3358323877 wait complete. 2024-08-02T17:50:09.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 180.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:50:10.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.189+0000 7f0bda270640 1 -- 172.21.15.69:0/1088808472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4100790 msgr2=0x7f0bd4100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.189+0000 7f0bda270640 1 --2- 172.21.15.69:0/1088808472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4100790 0x7f0bd4100bf0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009930 tx=0x7f0bbc02a420 comp rx=0 tx=0).stop 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 -- 172.21.15.69:0/1088808472 shutdown_connections 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 --2- 172.21.15.69:0/1088808472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4100790 0x7f0bd4100bf0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 --2- 172.21.15.69:0/1088808472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd4101130 0x7f0bd410d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 --2- 172.21.15.69:0/1088808472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd4106790 0x7f0bd4106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 -- 172.21.15.69:0/1088808472 >> 172.21.15.69:0/1088808472 conn(0x7f0bd40fc4b0 msgr2=0x7f0bd40fe8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 -- 172.21.15.69:0/1088808472 shutdown_connections 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 -- 172.21.15.69:0/1088808472 wait complete. 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 Processor -- start 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.190+0000 7f0bda270640 1 -- start start 2024-08-02T17:50:10.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bda270640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd4100790 0x7f0bd4199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bda270640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4101130 0x7f0bd41995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bda270640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd4106790 0x7f0bd41a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bda270640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4110b90 con 0x7f0bd4101130 2024-08-02T17:50:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bda270640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4110a10 con 0x7f0bd4100790 2024-08-02T17:50:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bda270640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4110d10 con 0x7f0bd4106790 2024-08-02T17:50:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4101130 0x7f0bd41995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4101130 0x7f0bd41995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41242/0 (socket says 172.21.15.69:41242) 2024-08-02T17:50:10.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 -- 172.21.15.69:0/2061932160 learned_addr learned my addr 172.21.15.69:0/2061932160 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd4100790 0x7f0bd4199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bd880d640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd4106790 0x7f0bd41a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 -- 172.21.15.69:0/2061932160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd4106790 msgr2=0x7f0bd41a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd4106790 0x7f0bd41a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 -- 172.21.15.69:0/2061932160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd4100790 msgr2=0x7f0bd4199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd4100790 0x7f0bd4199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 -- 172.21.15.69:0/2061932160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd41a4640 con 0x7f0bd4101130 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd4100790 0x7f0bd4199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bd880d640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd4106790 0x7f0bd41a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4101130 0x7f0bd41995c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009230 tx=0x7f0bbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bd1ffb640 1 -- 172.21.15.69:0/2061932160 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc03f030 con 0x7f0bd4101130 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bd1ffb640 1 -- 172.21.15.69:0/2061932160 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bbc003ad0 con 0x7f0bd4101130 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bda270640 1 -- 172.21.15.69:0/2061932160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd41a4810 con 0x7f0bd4101130 2024-08-02T17:50:10.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bd1ffb640 1 -- 172.21.15.69:0/2061932160 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc035070 con 0x7f0bd4101130 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.191+0000 7f0bda270640 1 -- 172.21.15.69:0/2061932160 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd41a4c60 con 0x7f0bd4101130 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bd1ffb640 1 -- 172.21.15.69:0/2061932160 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0bbc042030 con 0x7f0bd4101130 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bdc66e740 1 Processor -- start 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bdc66e740 1 -- start start 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5556ba237980 con 0x5556ba2dd3f0 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5556ba1cfa70 con 0x5556ba2d2210 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5556ba233940 con 0x5556ba2d2b30 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bd880d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bcbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bd880d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2d25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49188/0 (socket says 172.21.15.69:49188) 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bd880d640 1 -- 172.21.15.69:0/2143948998 learned_addr learned my addr 172.21.15.69:0/2143948998 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bd1ffb640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bc00a2cf0 0x7f0bc00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.194+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/2143948998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd880d640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bc00a2cf0 0x7f0bc00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd880d640 1 -- 172.21.15.69:0/2143948998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 msgr2=0x5556ba2dceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd880d640 1 --2- 172.21.15.69:0/2143948998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2dceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd880d640 1 -- 172.21.15.69:0/2143948998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 msgr2=0x5556ba2df7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd880d640 1 --2- 172.21.15.69:0/2143948998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2df7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd1ffb640 1 -- 172.21.15.69:0/2061932160 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0bbc0efdb0 con 0x7f0bd4101130 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd880d640 1 -- 172.21.15.69:0/2143948998 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ba2326e0 con 0x5556ba2d2210 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd880d640 1 --2- 172.21.15.69:0/2143948998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2d25f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc012a00 tx=0x7f0bcc012ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1da0d8645f3a5a98 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bd880d640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bc00a2cf0 0x7f0bc00a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c000 tx=0x7f0bcc01b000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:10.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bca7fc640 1 -- 172.21.15.69:0/2143948998 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc022070 con 0x5556ba2d2210 2024-08-02T17:50:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bca7fc640 1 -- 172.21.15.69:0/2143948998 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bcc010de0 con 0x5556ba2d2210 2024-08-02T17:50:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.195+0000 7f0bca7fc640 1 -- 172.21.15.69:0/2143948998 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0059d0 con 0x5556ba2d2210 2024-08-02T17:50:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 -- 172.21.15.69:0/2143948998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 msgr2=0x5556ba2d25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/2143948998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2d25f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc012a00 tx=0x7f0bcc012ec0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 -- 172.21.15.69:0/2143948998 shutdown_connections 2024-08-02T17:50:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/2143948998 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2df7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/2143948998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2dceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/2143948998 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2d25f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 -- 172.21.15.69:0/2143948998 >> 172.21.15.69:0/2143948998 conn(0x5556ba2ce7a0 msgr2=0x5556ba2ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 -- 172.21.15.69:0/2143948998 shutdown_connections 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 -- 172.21.15.69:0/2143948998 wait complete. 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.196+0000 7f0bdc66e740 1 -- start start 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bdc66e740 1 Processor -- start 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2210 0x5556ba2da2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2b30 0x5556ba2da7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5556ba2dffc0 con 0x5556ba2dd3f0 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5556ba1f2aa0 con 0x5556ba2d2b30 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5556ba2e0140 con 0x5556ba2d2210 2024-08-02T17:50:10.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bdc66e740 10 client.0 _ll_register_callbacks cb 0x5556ba1e1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2dad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41260/0 (socket says 172.21.15.69:41260) 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 -- 172.21.15.69:0/1916195382 learned_addr learned my addr 172.21.15.69:0/1916195382 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2b30 0x5556ba2da7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2210 0x5556ba2da2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 -- 172.21.15.69:0/1916195382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2210 msgr2=0x5556ba2da2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2210 0x5556ba2da2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 -- 172.21.15.69:0/1916195382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2b30 msgr2=0x5556ba2da7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2b30 0x5556ba2da7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 -- 172.21.15.69:0/1916195382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ba1d6650 con 0x5556ba2dd3f0 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bd880d640 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2dad30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc010fd0 tx=0x7f0bcc004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c9d66a171183bf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.197+0000 7f0bb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bb37fe640 1 -- 172.21.15.69:0/1916195382 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc02b070 con 0x5556ba2dd3f0 2024-08-02T17:50:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bb37fe640 1 -- 172.21.15.69:0/1916195382 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0bcc005960 con 0x5556ba2dd3f0 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 1 -- 172.21.15.69:0/1916195382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bcc010de0 con 0x5556ba2dd3f0 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bb37fe640 1 -- 172.21.15.69:0/1916195382 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc037050 con 0x5556ba2dd3f0 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 10 client.16389 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 1 -- 172.21.15.69:0/1916195382 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5556b9fa93c0 con 0x5556ba2dd3f0 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bb37fe640 1 -- 172.21.15.69:0/1916195382 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bcc010de0 con 0x5556ba2dd3f0 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bb37fe640 1 -- 172.21.15.69:0/1916195382 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0bcc01d940 con 0x5556ba2dd3f0 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 10 client.16389 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 20 client.16389 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ba2ece30 0x5556ba2ef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 4 client.16389 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bdc66e740 1 -- 172.21.15.69:0/1916195382 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556ba244ef0 con 0x5556ba2ece30 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.198+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ba2ece30 0x5556ba2ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.199+0000 7f0bb37fe640 10 client.16389 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:10.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.199+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ba2ece30 0x5556ba2ef220 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=deccbbfcab644329 server_cookie=7d376b4e0deff8e1 in_seq=0 out_seq=0 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.199+0000 7f0bb37fe640 1 -- 172.21.15.69:0/1916195382 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556ba244ef0 con 0x5556ba2ece30 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.199+0000 7f0bb37fe640 10 client.16389 handle_command_reply: tid=0 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.199+0000 7f0bdc66e740 1 client.16389 shutdown 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 -- 172.21.15.69:0/1916195382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 msgr2=0x5556ba2dad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2dad30 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc010fd0 tx=0x7f0bcc004dc0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 -- 172.21.15.69:0/1916195382 shutdown_connections 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2dd3f0 0x5556ba2dad30 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ba2ece30 0x5556ba2ef220 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2b30 0x5556ba2da7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/1916195382 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2210 0x5556ba2da2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bb37fe640 0 client.16389 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:10.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 -- 172.21.15.69:0/1916195382 >> 172.21.15.69:0/1916195382 conn(0x5556ba2ce7a0 msgr2=0x5556ba2d44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:10.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 -- 172.21.15.69:0/1916195382 shutdown_connections 2024-08-02T17:50:10.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 1 -- 172.21.15.69:0/1916195382 wait complete. 2024-08-02T17:50:10.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.200+0000 7f0bdc66e740 20 client.16389 trim_cache size 0 max 16384 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bdc66e740 1 Processor -- start 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bdc66e740 1 -- start start 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2b30 0x5556ba2f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2dd3f0 0x5556ba2f4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41b1660 con 0x5556ba2d2b30 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41aef30 con 0x5556ba2d2210 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41acd80 con 0x5556ba2dd3f0 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bcbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2b30 0x5556ba2f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bcbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2b30 0x5556ba2f49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41264/0 (socket says 172.21.15.69:41264) 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.217+0000 7f0bcbfff640 1 -- 172.21.15.69:0/3008739779 learned_addr learned my addr 172.21.15.69:0/3008739779 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:10.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bd880d640 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2dd3f0 0x5556ba2f4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bcbfff640 1 -- 172.21.15.69:0/3008739779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2dd3f0 msgr2=0x5556ba2f4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2dd3f0 0x5556ba2f4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bcbfff640 1 -- 172.21.15.69:0/3008739779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 msgr2=0x5556ba2dc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2dc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bcbfff640 1 -- 172.21.15.69:0/3008739779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ba2cf850 con 0x5556ba2d2b30 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2b30 0x5556ba2f49e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ed1a0 tx=0x7f0bbc0d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e3f65438f58ed5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bc9ffb640 1 -- 172.21.15.69:0/3008739779 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc049040 con 0x5556ba2d2b30 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bc9ffb640 1 -- 172.21.15.69:0/3008739779 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bbc0bbf10 con 0x5556ba2d2b30 2024-08-02T17:50:10.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.218+0000 7f0bc9ffb640 1 -- 172.21.15.69:0/3008739779 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0ed3c0 con 0x5556ba2d2b30 2024-08-02T17:50:10.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 -- 172.21.15.69:0/3008739779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2b30 msgr2=0x5556ba2f49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2b30 0x5556ba2f49e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ed1a0 tx=0x7f0bbc0d0040 comp rx=0 tx=0).stop 2024-08-02T17:50:10.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 -- 172.21.15.69:0/3008739779 shutdown_connections 2024-08-02T17:50:10.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2b30 0x5556ba2f49e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2dd3f0 0x5556ba2f4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.226 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/3008739779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2d2210 0x5556ba2dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 -- 172.21.15.69:0/3008739779 >> 172.21.15.69:0/3008739779 conn(0x5556ba2ce7a0 msgr2=0x5556ba2d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 -- 172.21.15.69:0/3008739779 shutdown_connections 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 -- 172.21.15.69:0/3008739779 wait complete. 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 -- start start 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.219+0000 7f0bdc66e740 1 Processor -- start 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2210 0x5556ba2e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2e16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bdc66e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2dd3f0 0x5556ba2e1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41ad3f0 con 0x5556ba2d2210 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41ae860 con 0x5556ba2dd3f0 2024-08-02T17:50:10.227 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bdc66e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41adac0 con 0x5556ba2d2b30 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bdc66e740 10 client.0 _ll_register_callbacks cb 0x7f0bcc020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2210 0x5556ba2e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2210 0x5556ba2e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41276/0 (socket says 172.21.15.69:41276) 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 -- 172.21.15.69:0/340208646 learned_addr learned my addr 172.21.15.69:0/340208646 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd880d640 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2dd3f0 0x5556ba2e1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2e16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 -- 172.21.15.69:0/340208646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 msgr2=0x5556ba2e16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2e16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 -- 172.21.15.69:0/340208646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2dd3f0 msgr2=0x5556ba2e1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2dd3f0 0x5556ba2e1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 -- 172.21.15.69:0/340208646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ba2cf430 con 0x5556ba2d2210 2024-08-02T17:50:10.228 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2e16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bd3fff640 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2210 0x5556ba2e1180 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x5556ba2ef950 tx=0x7f0bc4010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c8d5f7f38d7fac server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bb37fe640 1 -- 172.21.15.69:0/340208646 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4004220 con 0x5556ba2d2210 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bb37fe640 1 -- 172.21.15.69:0/340208646 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f0bc40043c0 con 0x5556ba2d2210 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.220+0000 7f0bdc66e740 1 -- 172.21.15.69:0/340208646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bbc0bbf10 con 0x5556ba2d2210 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bb37fe640 1 -- 172.21.15.69:0/340208646 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bc402b050 con 0x5556ba2d2210 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bdc66e740 10 client.16395 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bdc66e740 1 -- 172.21.15.69:0/340208646 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5556ba2ce5d0 con 0x5556ba2d2210 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bb37fe640 1 -- 172.21.15.69:0/340208646 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bbc0bbf10 con 0x5556ba2d2210 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bb37fe640 1 -- 172.21.15.69:0/340208646 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0bc4003c00 con 0x5556ba2d2210 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bdc66e740 10 client.16395 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bdc66e740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bdc66e740 20 client.16395 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:10.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ba35c550 0x5556ba35e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bdc66e740 4 client.16395 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bdc66e740 1 -- 172.21.15.69:0/340208646 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556ba23f400 con 0x5556ba35c550 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.221+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ba35c550 0x5556ba35e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bcbfff640 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ba35c550 0x5556ba35e970 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8429db619f5ea1cb server_cookie=1598a277d942b67e in_seq=0 out_seq=0 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bb37fe640 10 client.16395 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bb37fe640 1 -- 172.21.15.69:0/340208646 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5556ba23f400 con 0x5556ba35c550 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bb37fe640 10 client.16395 handle_command_reply: tid=0 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 client.16395 shutdown 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 -- 172.21.15.69:0/340208646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2210 msgr2=0x5556ba2e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2210 0x5556ba2e1180 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x5556ba2ef950 tx=0x7f0bc4010af0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 -- 172.21.15.69:0/340208646 shutdown_connections 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ba35c550 0x5556ba35e970 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556ba2dd3f0 0x5556ba2e1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556ba2d2b30 0x5556ba2e16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bb37fe640 0 client.16395 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 --2- 172.21.15.69:0/340208646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556ba2d2210 0x5556ba2e1180 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 -- 172.21.15.69:0/340208646 >> 172.21.15.69:0/340208646 conn(0x5556ba2ce7a0 msgr2=0x5556ba2d8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 -- 172.21.15.69:0/340208646 shutdown_connections 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 1 -- 172.21.15.69:0/340208646 wait complete. 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.222+0000 7f0bdc66e740 20 client.16395 trim_cache size 0 max 16384 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:50:10.231 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:50:10.232 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:50:10.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 -- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bc00a2cf0 msgr2=0x7f0bc00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bc00a2cf0 0x7f0bc00a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc00c000 tx=0x7f0bcc01b000 comp rx=0 tx=0).stop 2024-08-02T17:50:10.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 -- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4101130 msgr2=0x7f0bd41995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:10.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4101130 0x7f0bd41995c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009230 tx=0x7f0bbc004270 comp rx=0 tx=0).stop 2024-08-02T17:50:10.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 -- 172.21.15.69:0/2061932160 shutdown_connections 2024-08-02T17:50:10.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd4101130 0x7f0bd41995c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bc00a2cf0 0x7f0bc00a51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd4106790 0x7f0bd41a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 --2- 172.21.15.69:0/2061932160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd4100790 0x7f0bd4199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:10.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 -- 172.21.15.69:0/2061932160 >> 172.21.15.69:0/2061932160 conn(0x7f0bd40fc4b0 msgr2=0x7f0bd40fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:10.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 -- 172.21.15.69:0/2061932160 shutdown_connections 2024-08-02T17:50:10.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:10.223+0000 7f0bda270640 1 -- 172.21.15.69:0/2061932160 wait complete. 2024-08-02T17:50:11.146 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:11 smithi069 ceph-mon[32746]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2024-08-02T17:50:11.174 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:50:11.175 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:50:11.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:11 smithi077 ceph-mon[37108]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2024-08-02T17:50:11.416 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:50:11.435 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:11 smithi083 ceph-mon[37031]: pgmap v116: 129 pgs: 129 active+clean; 1.9 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.2 MiB/s wr, 290 op/s 2024-08-02T17:50:11.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.929+0000 7fdbc2207640 1 -- 172.21.15.69:0/1657043585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 msgr2=0x7fdbbc10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:11.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.929+0000 7fdbc2207640 1 --2- 172.21.15.69:0/1657043585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 0x7fdbbc10d770 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdbb4009ca0 tx=0x7fdbb402a950 comp rx=0 tx=0).stop 2024-08-02T17:50:11.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 -- 172.21.15.69:0/1657043585 shutdown_connections 2024-08-02T17:50:11.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 --2- 172.21.15.69:0/1657043585 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 0x7fdbbc10d770 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:11.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 --2- 172.21.15.69:0/1657043585 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbbc1022a0 0x7fdbbc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:11.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 --2- 172.21.15.69:0/1657043585 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbbc100b50 0x7fdbbc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:11.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 -- 172.21.15.69:0/1657043585 >> 172.21.15.69:0/1657043585 conn(0x7fdbbc0fc500 msgr2=0x7fdbbc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:11.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 -- 172.21.15.69:0/1657043585 shutdown_connections 2024-08-02T17:50:11.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 -- 172.21.15.69:0/1657043585 wait complete. 2024-08-02T17:50:11.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 Processor -- start 2024-08-02T17:50:11.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 -- start start 2024-08-02T17:50:11.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbbc100b50 0x7fdbbc0fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:11.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbbc1022a0 0x7fdbbc0fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:11.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 0x7fdbbc0fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:11.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc110cb0 con 0x7fdbbc102c20 2024-08-02T17:50:11.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc110b30 con 0x7fdbbc1022a0 2024-08-02T17:50:11.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.930+0000 7fdbc2207640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdbbc110e30 con 0x7fdbbc100b50 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 0x7fdbbc0fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 0x7fdbbc0fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41296/0 (socket says 172.21.15.69:41296) 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 -- 172.21.15.69:0/3534544782 learned_addr learned my addr 172.21.15.69:0/3534544782 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbb3fff640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbbc1022a0 0x7fdbbc0fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbb7fe640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbbc100b50 0x7fdbbc0fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 -- 172.21.15.69:0/3534544782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbbc100b50 msgr2=0x7fdbbc0fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbbc100b50 0x7fdbbc0fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 -- 172.21.15.69:0/3534544782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbbc1022a0 msgr2=0x7fdbbc0fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbbc1022a0 0x7fdbbc0fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 -- 172.21.15.69:0/3534544782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbbc1a65e0 con 0x7fdbbc102c20 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbbbfff640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 0x7fdbbc0fd430 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdbb402ae70 tx=0x7fdbb402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:11.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbb97fa640 1 -- 172.21.15.69:0/3534544782 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4034040 con 0x7fdbbc102c20 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbbc1a6870 con 0x7fdbbc102c20 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbb97fa640 1 -- 172.21.15.69:0/3534544782 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbb4030920 con 0x7fdbbc102c20 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbb97fa640 1 -- 172.21.15.69:0/3534544782 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4045480 con 0x7fdbbc102c20 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.931+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbbc1a6da0 con 0x7fdbbc102c20 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.932+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbbc100f30 con 0x7fdbbc102c20 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.933+0000 7fdbb97fa640 1 -- 172.21.15.69:0/3534544782 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdbb4037020 con 0x7fdbbc102c20 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.934+0000 7fdbb97fa640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdba80a2cf0 0x7fdba80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.934+0000 7fdbbb7fe640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdba80a2cf0 0x7fdba80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.934+0000 7fdbb97fa640 1 -- 172.21.15.69:0/3534544782 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fdbb40f0610 con 0x7fdbbc102c20 2024-08-02T17:50:11.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.934+0000 7fdbbb7fe640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdba80a2cf0 0x7fdba80a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005e10 tx=0x7fdbac005da0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:11.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:11.938+0000 7fdbb97fa640 1 -- 172.21.15.69:0/3534544782 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdbb40bcf80 con 0x7fdbbc102c20 2024-08-02T17:50:11.986 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:12.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.150+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbbc109dd0 con 0x7fdbbc102c20 2024-08-02T17:50:12.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.150+0000 7fdbb97fa640 1 -- 172.21.15.69:0/3534544782 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fdbb40bc880 con 0x7fdbbc102c20 2024-08-02T17:50:12.152 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:50:12.152 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:50:12.152 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:50:12.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdba80a2cf0 msgr2=0x7fdba80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:12.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdba80a2cf0 0x7fdba80a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdbac005e10 tx=0x7fdbac005da0 comp rx=0 tx=0).stop 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 msgr2=0x7fdbbc0fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 0x7fdbbc0fd430 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdbb402ae70 tx=0x7fdbb402f5d0 comp rx=0 tx=0).stop 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 shutdown_connections 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdbbc102c20 0x7fdbbc0fd430 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdba80a2cf0 0x7fdba80a51b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdbbc1022a0 0x7fdbbc0fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 --2- 172.21.15.69:0/3534544782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdbbc100b50 0x7fdbbc0fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.152+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 >> 172.21.15.69:0/3534544782 conn(0x7fdbbc0fc500 msgr2=0x7fdbbc069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.153+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 shutdown_connections 2024-08-02T17:50:12.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:12.153+0000 7fdbc2207640 1 -- 172.21.15.69:0/3534544782 wait complete. 2024-08-02T17:50:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:12 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3534544782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:12.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:12 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3534544782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:12 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3534544782' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:12.978 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:50:13.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 -- 172.21.15.69:0/2173824450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 msgr2=0x7fdfc4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:13.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 --2- 172.21.15.69:0/2173824450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 0x7fdfc4100f30 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009900 tx=0x7fdfb402a480 comp rx=0 tx=0).stop 2024-08-02T17:50:13.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 -- 172.21.15.69:0/2173824450 shutdown_connections 2024-08-02T17:50:13.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 --2- 172.21.15.69:0/2173824450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc4102c20 0x7fdfc410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 --2- 172.21.15.69:0/2173824450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfc41022a0 0x7fdfc41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 --2- 172.21.15.69:0/2173824450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 0x7fdfc4100f30 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 -- 172.21.15.69:0/2173824450 >> 172.21.15.69:0/2173824450 conn(0x7fdfc40fc500 msgr2=0x7fdfc40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 -- 172.21.15.69:0/2173824450 shutdown_connections 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 -- 172.21.15.69:0/2173824450 wait complete. 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.199+0000 7fdfca291640 1 Processor -- start 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfca291640 1 -- start start 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfca291640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 0x7fdfc41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfca291640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc41022a0 0x7fdfc4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfca291640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfc4102c20 0x7fdfc41a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfca291640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4110d20 con 0x7fdfc4100b50 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfca291640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4110ba0 con 0x7fdfc4102c20 2024-08-02T17:50:13.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfca291640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc4110ea0 con 0x7fdfc41022a0 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 0x7fdfc41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 0x7fdfc41990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41328/0 (socket says 172.21.15.69:41328) 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc3fff640 1 -- 172.21.15.69:0/3443728545 learned_addr learned my addr 172.21.15.69:0/3443728545 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc882e640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfc4102c20 0x7fdfc41a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc37fe640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc41022a0 0x7fdfc4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc3fff640 1 -- 172.21.15.69:0/3443728545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc41022a0 msgr2=0x7fdfc4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc3fff640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc41022a0 0x7fdfc4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc3fff640 1 -- 172.21.15.69:0/3443728545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfc4102c20 msgr2=0x7fdfc41a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:13.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc3fff640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfc4102c20 0x7fdfc41a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.200+0000 7fdfc3fff640 1 -- 172.21.15.69:0/3443728545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc41a4680 con 0x7fdfc4100b50 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.201+0000 7fdfc3fff640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 0x7fdfc41990c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402aa70 tx=0x7fdfb4003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.201+0000 7fdfc17fa640 1 -- 172.21.15.69:0/3443728545 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb403f070 con 0x7fdfc4100b50 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.201+0000 7fdfc17fa640 1 -- 172.21.15.69:0/3443728545 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfb4031c20 con 0x7fdfc4100b50 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.201+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc41a4850 con 0x7fdfc4100b50 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.201+0000 7fdfc17fa640 1 -- 172.21.15.69:0/3443728545 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4035030 con 0x7fdfc4100b50 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.201+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc41a4ca0 con 0x7fdfc4100b50 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.202+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc4100f30 con 0x7fdfc4100b50 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.203+0000 7fdfc17fa640 1 -- 172.21.15.69:0/3443728545 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdfb4042070 con 0x7fdfc4100b50 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.204+0000 7fdfc17fa640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfac0a2cf0 0x7fdfac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.204+0000 7fdfc37fe640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfac0a2cf0 0x7fdfac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:13.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.204+0000 7fdfc17fa640 1 -- 172.21.15.69:0/3443728545 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fdfb40f00e0 con 0x7fdfc4100b50 2024-08-02T17:50:13.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.205+0000 7fdfc37fe640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfac0a2cf0 0x7fdfac0a51b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdfc406a7f0 tx=0x7fdfb800b590 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:13.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.212+0000 7fdfc17fa640 1 -- 172.21.15.69:0/3443728545 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdfb40bc8e0 con 0x7fdfc4100b50 2024-08-02T17:50:13.279 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:13.401 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:13 smithi069 ceph-mon[32746]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 774 op/s 2024-08-02T17:50:13.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.424+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfc41a4fc0 con 0x7fdfc4100b50 2024-08-02T17:50:13.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.425+0000 7fdfc17fa640 1 -- 172.21.15.69:0/3443728545 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fdfb40dae20 con 0x7fdfc4100b50 2024-08-02T17:50:13.427 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:50:13.427 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:50:13.427 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfac0a2cf0 msgr2=0x7fdfac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfac0a2cf0 0x7fdfac0a51b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdfc406a7f0 tx=0x7fdfb800b590 comp rx=0 tx=0).stop 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 msgr2=0x7fdfc41990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 0x7fdfc41990c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdfb402aa70 tx=0x7fdfb4003b60 comp rx=0 tx=0).stop 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 shutdown_connections 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfac0a2cf0 0x7fdfac0a51b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdfc4102c20 0x7fdfc41a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdfc41022a0 0x7fdfc4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 --2- 172.21.15.69:0/3443728545 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdfc4100b50 0x7fdfc41990c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 >> 172.21.15.69:0/3443728545 conn(0x7fdfc40fc500 msgr2=0x7fdfc40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:13.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 shutdown_connections 2024-08-02T17:50:13.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:13.427+0000 7fdfca291640 1 -- 172.21.15.69:0/3443728545 wait complete. 2024-08-02T17:50:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:13 smithi083 ceph-mon[37031]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 774 op/s 2024-08-02T17:50:13.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:13 smithi077 ceph-mon[37108]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 11 MiB/s wr, 774 op/s 2024-08-02T17:50:14.516 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:14 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3443728545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:14.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:14 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3443728545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:14.542 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 48023e9 --release --await' 2024-08-02T17:50:14.542 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 48023e9 --release --await 2024-08-02T17:50:14.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:14 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3443728545' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 -- 172.21.15.69:0/3151118687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 msgr2=0x7f93a010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3151118687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 0x7f93a010d770 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9398009ca0 tx=0x7f939802a950 comp rx=0 tx=0).stop 2024-08-02T17:50:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 -- 172.21.15.69:0/3151118687 shutdown_connections 2024-08-02T17:50:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3151118687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 0x7f93a010d770 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3151118687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93a01022a0 0x7f93a01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3151118687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a0100b50 0x7f93a0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 -- 172.21.15.69:0/3151118687 >> 172.21.15.69:0/3151118687 conn(0x7f93a00fc500 msgr2=0x7f93a00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 -- 172.21.15.69:0/3151118687 shutdown_connections 2024-08-02T17:50:14.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 -- 172.21.15.69:0/3151118687 wait complete. 2024-08-02T17:50:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.709+0000 7f93a5c99640 1 Processor -- start 2024-08-02T17:50:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f93a5c99640 1 -- start start 2024-08-02T17:50:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f93a5c99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a0100b50 0x7f93a0195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f93a5c99640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93a01022a0 0x7f93a0196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f93a5c99640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 0x7f93a019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f93a5c99640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93a010fbb0 con 0x7f93a0102c20 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f93a5c99640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93a010fa30 con 0x7f93a01022a0 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f93a5c99640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93a010fd30 con 0x7f93a0100b50 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 0x7f93a019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 0x7f93a019a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41336/0 (socket says 172.21.15.69:41336) 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 -- 172.21.15.69:0/3361493696 learned_addr learned my addr 172.21.15.69:0/3361493696 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f9397fff640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93a01022a0 0x7f93a0196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939f7fe640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a0100b50 0x7f93a0195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 -- 172.21.15.69:0/3361493696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a0100b50 msgr2=0x7f93a0195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a0100b50 0x7f93a0195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 -- 172.21.15.69:0/3361493696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93a01022a0 msgr2=0x7f93a0196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93a01022a0 0x7f93a0196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 -- 172.21.15.69:0/3361493696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a019ae80 con 0x7f93a0102c20 2024-08-02T17:50:14.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939f7fe640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a0100b50 0x7f93a0195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:14.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.710+0000 7f939ffff640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 0x7f93a019a7a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f939802ae70 tx=0x7f939802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:14.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.711+0000 7f939d7fa640 1 -- 172.21.15.69:0/3361493696 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398034040 con 0x7f93a0102c20 2024-08-02T17:50:14.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.711+0000 7f939d7fa640 1 -- 172.21.15.69:0/3361493696 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9398030920 con 0x7f93a0102c20 2024-08-02T17:50:14.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.711+0000 7f939d7fa640 1 -- 172.21.15.69:0/3361493696 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93980047f0 con 0x7f93a0102c20 2024-08-02T17:50:14.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.711+0000 7f93a5c99640 1 -- 172.21.15.69:0/3361493696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a019b110 con 0x7f93a0102c20 2024-08-02T17:50:14.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.711+0000 7f93a5c99640 1 -- 172.21.15.69:0/3361493696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a01a2a50 con 0x7f93a0102c20 2024-08-02T17:50:14.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.713+0000 7f939d7fa640 1 -- 172.21.15.69:0/3361493696 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9398037020 con 0x7f93a0102c20 2024-08-02T17:50:14.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.713+0000 7f93a8097740 1 Processor -- start 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f93a8097740 1 -- start start 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f93a8097740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be4f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f93a8097740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be59c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f93a8097740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be5c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f93a8097740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56103bdb4a40 con 0x56103be5a1d0 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f93a8097740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56103bd4c9d0 con 0x56103be4f910 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f93a8097740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56103bdb0a00 con 0x56103be4eff0 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f939d7fa640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f938c0a4f00 0x7f938c0a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be5c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be5c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41350/0 (socket says 172.21.15.69:41350) 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 -- 172.21.15.69:0/3978951116 learned_addr learned my addr 172.21.15.69:0/3978951116 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:14.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f939f7fe640 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be59c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f939d7fa640 1 -- 172.21.15.69:0/3361493696 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f93980f06a0 con 0x7f93a0102c20 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f939ffff640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f938c0a4f00 0x7f938c0a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be4f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 -- 172.21.15.69:0/3978951116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 msgr2=0x56103be4f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be4f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 -- 172.21.15.69:0/3978951116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 msgr2=0x56103be59c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be59c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.714+0000 7f9397fff640 1 -- 172.21.15.69:0/3978951116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103bd53650 con 0x56103be5a1d0 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.715+0000 7f939ffff640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f938c0a4f00 0x7f938c0a73c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f93980b92e0 tx=0x7f939802a950 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.715+0000 7f9397fff640 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be5c5c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9388011560 tx=0x7f938800a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344f9f205d7259da server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.715+0000 7f9395ffb640 1 -- 172.21.15.69:0/3978951116 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388005220 con 0x56103be5a1d0 2024-08-02T17:50:14.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.715+0000 7f9395ffb640 1 -- 172.21.15.69:0/3978951116 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9388004c80 con 0x56103be5a1d0 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.715+0000 7f9395ffb640 1 -- 172.21.15.69:0/3978951116 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800fc50 con 0x56103be5a1d0 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 -- 172.21.15.69:0/3978951116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 msgr2=0x56103be5c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be5c5c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9388011560 tx=0x7f938800a810 comp rx=0 tx=0).stop 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 -- 172.21.15.69:0/3978951116 shutdown_connections 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be5c5c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be59c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 --2- 172.21.15.69:0/3978951116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be4f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 -- 172.21.15.69:0/3978951116 >> 172.21.15.69:0/3978951116 conn(0x56103be4b580 msgr2=0x56103be4b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 -- 172.21.15.69:0/3978951116 shutdown_connections 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 -- 172.21.15.69:0/3978951116 wait complete. 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 -- start start 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.716+0000 7f93a8097740 1 Processor -- start 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f93a8097740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be5e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f93a8097740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4f910 0x56103be5eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f93a8097740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 0x56103be62c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f93a8097740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56103be5cda0 con 0x56103be4f910 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f93a8097740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56103bd6fb20 con 0x56103be5a1d0 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f93a8097740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56103be5cf20 con 0x56103be4eff0 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f93a8097740 10 client.0 _ll_register_callbacks cb 0x56103bd6dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4f910 0x56103be5eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4f910 0x56103be5eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41356/0 (socket says 172.21.15.69:41356) 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 -- 172.21.15.69:0/3733011038 learned_addr learned my addr 172.21.15.69:0/3733011038 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f9397fff640 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 0x56103be62c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f9397fff640 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be5e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 -- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 msgr2=0x56103be5e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:14.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be5e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 -- 172.21.15.69:0/3733011038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 msgr2=0x56103be62c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 0x56103be62c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 -- 172.21.15.69:0/3733011038 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103bb263c0 con 0x56103be4f910 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f9397fff640 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be5e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f939f7fe640 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4f910 0x56103be5eb80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f939000bfb0 tx=0x7f93900128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6604d30e75671e95 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f937effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f937effd640 1 -- 172.21.15.69:0/3733011038 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93900035f0 con 0x56103be4f910 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f937effd640 1 -- 172.21.15.69:0/3733011038 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f9390003790 con 0x56103be4f910 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.717+0000 7f93a8097740 1 -- 172.21.15.69:0/3733011038 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9388004c80 con 0x56103be4f910 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f937effd640 1 -- 172.21.15.69:0/3733011038 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9390003990 con 0x56103be4f910 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f93a8097740 10 client.16419 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f93a8097740 1 -- 172.21.15.69:0/3733011038 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56103be6ce00 con 0x56103be4f910 2024-08-02T17:50:14.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f937effd640 1 -- 172.21.15.69:0/3733011038 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9388004c80 con 0x56103be4f910 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f937effd640 1 -- 172.21.15.69:0/3733011038 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9390019070 con 0x56103be4f910 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f93a8097740 10 client.16419 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f93a8097740 10 client.16419 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f93a8097740 20 client.16419 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f93a8097740 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56103be6e360 0x56103be70780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f93a8097740 4 client.16419 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f93a8097740 1 -- 172.21.15.69:0/3733011038 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56103bdc1fb0 con 0x56103be6e360 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.718+0000 7f9397fff640 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56103be6e360 0x56103be70780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.719+0000 7f9397fff640 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56103be6e360 0x56103be70780 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbf4111979506ffa server_cookie=78d0806d1dfeec53 in_seq=0 out_seq=0 2024-08-02T17:50:14.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.719+0000 7f937effd640 10 client.16419 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.719+0000 7f937effd640 1 -- 172.21.15.69:0/3733011038 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56103bdc1fb0 con 0x56103be6e360 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.719+0000 7f937effd640 10 client.16419 handle_command_reply: tid=0 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.719+0000 7f93a8097740 1 client.16419 shutdown 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 -- 172.21.15.69:0/3733011038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4f910 msgr2=0x56103be5eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4f910 0x56103be5eb80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f939000bfb0 tx=0x7f93900128e0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 -- 172.21.15.69:0/3733011038 shutdown_connections 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56103be6e360 0x56103be70780 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4f910 0x56103be5eb80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 0x56103be62c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 --2- 172.21.15.69:0/3733011038 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be5e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f937effd640 0 client.16419 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:50:14.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 -- 172.21.15.69:0/3733011038 >> 172.21.15.69:0/3733011038 conn(0x56103be4b580 msgr2=0x56103be51260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:14.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 -- 172.21.15.69:0/3733011038 shutdown_connections 2024-08-02T17:50:14.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 1 -- 172.21.15.69:0/3733011038 wait complete. 2024-08-02T17:50:14.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.720+0000 7f93a8097740 20 client.16419 trim_cache size 0 max 16384 2024-08-02T17:50:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.733+0000 7f93a8097740 1 Processor -- start 2024-08-02T17:50:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.733+0000 7f93a8097740 1 -- start start 2024-08-02T17:50:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.733+0000 7f93a8097740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4eff0 0x56103be4a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f93a8097740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4f910 0x56103be4a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f93a8097740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 0x56103be76030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f93a8097740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01af240 con 0x56103be4eff0 2024-08-02T17:50:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f93a8097740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01acb10 con 0x56103be5a1d0 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f93a8097740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01aa960 con 0x56103be4f910 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4eff0 0x56103be4a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4eff0 0x56103be4a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54972/0 (socket says 172.21.15.69:54972) 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 -- 172.21.15.69:0/173436902 learned_addr learned my addr 172.21.15.69:0/173436902 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f939f7fe640 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4f910 0x56103be4a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 0x56103be76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 -- 172.21.15.69:0/173436902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4f910 msgr2=0x56103be4a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4f910 0x56103be4a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 -- 172.21.15.69:0/173436902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 msgr2=0x56103be76030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 0x56103be76030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 -- 172.21.15.69:0/173436902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56103be4b3b0 con 0x56103be4eff0 2024-08-02T17:50:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.734+0000 7f9397fff640 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4eff0 0x56103be4a4a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f938800f780 tx=0x7f938800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2183f63437a536 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.735+0000 7f93957fa640 1 -- 172.21.15.69:0/173436902 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388021040 con 0x56103be4eff0 2024-08-02T17:50:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.735+0000 7f93957fa640 1 -- 172.21.15.69:0/173436902 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93880054e0 con 0x56103be4eff0 2024-08-02T17:50:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.735+0000 7f93957fa640 1 -- 172.21.15.69:0/173436902 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388004c60 con 0x56103be4eff0 2024-08-02T17:50:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.735+0000 7f93a8097740 1 -- 172.21.15.69:0/173436902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4eff0 msgr2=0x56103be4a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.735+0000 7f93a8097740 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4eff0 0x56103be4a4a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f938800f780 tx=0x7f938800f850 comp rx=0 tx=0).stop 2024-08-02T17:50:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.736+0000 7f93a8097740 1 -- 172.21.15.69:0/173436902 shutdown_connections 2024-08-02T17:50:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.736+0000 7f93a8097740 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be5a1d0 0x56103be76030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.736+0000 7f93a8097740 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4f910 0x56103be4a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.736+0000 7f93a8097740 1 --2- 172.21.15.69:0/173436902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be4eff0 0x56103be4a4a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.736+0000 7f93a8097740 1 -- 172.21.15.69:0/173436902 >> 172.21.15.69:0/173436902 conn(0x56103be4b580 msgr2=0x56103be49d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.736+0000 7f93a8097740 1 -- 172.21.15.69:0/173436902 shutdown_connections 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.736+0000 7f93a8097740 1 -- 172.21.15.69:0/173436902 wait complete. 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.736+0000 7f93a8097740 1 -- start start 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f93a8097740 1 Processor -- start 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f93a8097740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be6ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f93a8097740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be66050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f93a8097740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f93a8097740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01aafd0 con 0x56103be5a1d0 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f93a8097740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01ac440 con 0x56103be4f910 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f93a8097740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93a01ab6a0 con 0x56103be4eff0 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f93a8097740 10 client.0 _ll_register_callbacks cb 0x7f939000b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f939f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be66050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f9397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f939f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be66050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59764/0 (socket says 172.21.15.69:59764) 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f939f7fe640 1 -- 172.21.15.69:0/1465843291 learned_addr learned my addr 172.21.15.69:0/1465843291 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f9397fff640 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be6ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f939f7fe640 1 -- 172.21.15.69:0/1465843291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 msgr2=0x56103be6ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f939f7fe640 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be6ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f939f7fe640 1 -- 172.21.15.69:0/1465843291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 msgr2=0x56103be667e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f939f7fe640 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be667e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.737+0000 7f939f7fe640 1 -- 172.21.15.69:0/1465843291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56103be56270 con 0x56103be4f910 2024-08-02T17:50:14.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.738+0000 7f939f7fe640 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be66050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9390000c00 tx=0x7f9390003760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea736d0fd630379a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.738+0000 7f937effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.738+0000 7f937effd640 1 -- 172.21.15.69:0/1465843291 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9390005030 con 0x56103be4f910 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.738+0000 7f93a8097740 1 -- 172.21.15.69:0/1465843291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93880054e0 con 0x56103be4f910 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.738+0000 7f937effd640 1 -- 172.21.15.69:0/1465843291 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f93900051d0 con 0x56103be4f910 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f937effd640 1 -- 172.21.15.69:0/1465843291 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9390038050 con 0x56103be4f910 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f93a8097740 10 client.26018 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f93a8097740 1 -- 172.21.15.69:0/1465843291 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56103be51d60 con 0x56103be4f910 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f937effd640 1 -- 172.21.15.69:0/1465843291 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93880054e0 con 0x56103be4f910 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f937effd640 1 -- 172.21.15.69:0/1465843291 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9390019040 con 0x56103be4f910 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f93a8097740 10 client.26018 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f93a8097740 10 client.26018 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f93a8097740 20 client.26018 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:14.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f93a8097740 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56103be52560 0x56103be54980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f93a8097740 4 client.26018 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:50:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f93a8097740 1 -- 172.21.15.69:0/1465843291 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56103bdbc4c0 con 0x56103be52560 2024-08-02T17:50:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.739+0000 7f9397fff640 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56103be52560 0x56103be54980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f937effd640 10 client.26018 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:50:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f9397fff640 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56103be52560 0x56103be54980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3294e4bad016863d server_cookie=ea61dc7a541febd1 in_seq=0 out_seq=0 2024-08-02T17:50:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f937effd640 1 -- 172.21.15.69:0/1465843291 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56103bdbc4c0 con 0x56103be52560 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f937effd640 10 client.26018 handle_command_reply: tid=0 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 client.26018 shutdown 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 -- 172.21.15.69:0/1465843291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 msgr2=0x56103be66050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be66050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9390000c00 tx=0x7f9390003760 comp rx=0 tx=0).stop 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 -- 172.21.15.69:0/1465843291 shutdown_connections 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56103be52560 0x56103be54980 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56103be5a1d0 0x56103be667e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56103be4f910 0x56103be66050 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 --2- 172.21.15.69:0/1465843291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56103be4eff0 0x56103be6ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f93a8097740 1 -- 172.21.15.69:0/1465843291 >> 172.21.15.69:0/1465843291 conn(0x56103be4b580 msgr2=0x56103be4a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.740+0000 7f937effd640 0 client.26018 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a8097740 1 -- 172.21.15.69:0/1465843291 shutdown_connections 2024-08-02T17:50:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a8097740 1 -- 172.21.15.69:0/1465843291 wait complete. 2024-08-02T17:50:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a8097740 20 client.26018 trim_cache size 0 max 16384 2024-08-02T17:50:14.747 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:50:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 -- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f938c0a4f00 msgr2=0x7f938c0a73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f938c0a4f00 0x7f938c0a73c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f93980b92e0 tx=0x7f939802a950 comp rx=0 tx=0).stop 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 -- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 msgr2=0x7f93a019a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 0x7f93a019a7a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f939802ae70 tx=0x7f939802f5d0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 -- 172.21.15.69:0/3361493696 shutdown_connections 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93a0102c20 0x7f93a019a7a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f938c0a4f00 0x7f938c0a73c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93a01022a0 0x7f93a0196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 --2- 172.21.15.69:0/3361493696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a0100b50 0x7f93a0195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.741+0000 7f93a5c99640 1 -- 172.21.15.69:0/3361493696 >> 172.21.15.69:0/3361493696 conn(0x7f93a00fc500 msgr2=0x7f93a01064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.742+0000 7f93a5c99640 1 -- 172.21.15.69:0/3361493696 shutdown_connections 2024-08-02T17:50:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:14.742+0000 7f93a5c99640 1 -- 172.21.15.69:0/3361493696 wait complete. 2024-08-02T17:50:14.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 185.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:50:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:15 smithi083 ceph-mon[37031]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.5 MiB/s wr, 668 op/s 2024-08-02T17:50:15.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:15 smithi077 ceph-mon[37108]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.5 MiB/s wr, 668 op/s 2024-08-02T17:50:15.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:15 smithi069 ceph-mon[32746]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 9.5 MiB/s wr, 668 op/s 2024-08-02T17:50:17.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:17 smithi077 ceph-mon[37108]: pgmap v119: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.5 MiB/s wr, 672 op/s 2024-08-02T17:50:17.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:17 smithi069 ceph-mon[32746]: pgmap v119: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.5 MiB/s wr, 672 op/s 2024-08-02T17:50:17.515 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:17 smithi083 ceph-mon[37031]: pgmap v119: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.5 MiB/s wr, 672 op/s 2024-08-02T17:50:17.545 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 -- 172.21.15.69:0/2373157837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c5010e0e0 msgr2=0x7f3c50110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 --2- 172.21.15.69:0/2373157837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c5010e0e0 0x7f3c50110530 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f3c48009ca0 tx=0x7f3c4802a950 comp rx=0 tx=0).stop 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 -- 172.21.15.69:0/2373157837 shutdown_connections 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 --2- 172.21.15.69:0/2373157837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c5010e0e0 0x7f3c50110530 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 --2- 172.21.15.69:0/2373157837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c50074c70 0x7f3c5010dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 --2- 172.21.15.69:0/2373157837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c50074350 0x7f3c50074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 -- 172.21.15.69:0/2373157837 >> 172.21.15.69:0/2373157837 conn(0x7f3c500fd5e0 msgr2=0x7f3c500ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 -- 172.21.15.69:0/2373157837 shutdown_connections 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 -- 172.21.15.69:0/2373157837 wait complete. 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.980+0000 7f3c56264640 1 Processor -- start 2024-08-02T17:50:17.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c56264640 1 -- start start 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c56264640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c50074350 0x7f3c5019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c56264640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c50074c70 0x7f3c5019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c56264640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c5010e0e0 0x7f3c5019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c56264640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c50113e70 con 0x7f3c50074c70 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c56264640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c50113cf0 con 0x7f3c5010e0e0 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c56264640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c50113ff0 con 0x7f3c50074350 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c50074c70 0x7f3c5019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c50074c70 0x7f3c5019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54994/0 (socket says 172.21.15.69:54994) 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4f7fe640 1 -- 172.21.15.69:0/3199750474 learned_addr learned my addr 172.21.15.69:0/3199750474 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c54801640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c5010e0e0 0x7f3c5019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c50074350 0x7f3c5019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4f7fe640 1 -- 172.21.15.69:0/3199750474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c50074350 msgr2=0x7f3c5019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c50074350 0x7f3c5019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4f7fe640 1 -- 172.21.15.69:0/3199750474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c5010e0e0 msgr2=0x7f3c5019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c5010e0e0 0x7f3c5019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.981+0000 7f3c4f7fe640 1 -- 172.21.15.69:0/3199750474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5019f300 con 0x7f3c50074c70 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.982+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c50074c70 0x7f3c5019a890 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400d770 tx=0x7f3c4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.982+0000 7f3c4d7fa640 1 -- 172.21.15.69:0/3199750474 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44017070 con 0x7f3c50074c70 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.982+0000 7f3c56264640 1 -- 172.21.15.69:0/3199750474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5019f5f0 con 0x7f3c50074c70 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.982+0000 7f3c4d7fa640 1 -- 172.21.15.69:0/3199750474 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c44005500 con 0x7f3c50074c70 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.982+0000 7f3c4d7fa640 1 -- 172.21.15.69:0/3199750474 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44004e00 con 0x7f3c50074c70 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.982+0000 7f3c56264640 1 -- 172.21.15.69:0/3199750474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c501a6ed0 con 0x7f3c50074c70 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.984+0000 7f3c4d7fa640 1 -- 172.21.15.69:0/3199750474 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3c44004fa0 con 0x7f3c50074c70 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.984+0000 7f3c58662740 1 Processor -- start 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.984+0000 7f3c58662740 1 -- start start 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c58662740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0377990 0x5603b0377d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c58662740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b0382630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c58662740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b0384f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c58662740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5603b02ddc60 con 0x5603b0377990 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c58662740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5603b0275b90 con 0x5603b0382b70 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c58662740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603b02d9c20 con 0x5603b03782b0 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4d7fa640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c380a2cf0 0x7f3c380a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c54801640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0377990 0x5603b0377d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c54801640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0377990 0x5603b0377d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55004/0 (socket says 172.21.15.69:55004) 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c54801640 1 -- 172.21.15.69:0/2483955948 learned_addr learned my addr 172.21.15.69:0/2483955948 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b0382630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4d7fa640 1 -- 172.21.15.69:0/3199750474 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3c440cf2e0 con 0x7f3c50074c70 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b0384f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c54801640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c380a2cf0 0x7f3c380a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4f7fe640 1 -- 172.21.15.69:0/2483955948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 msgr2=0x5603b0382630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b0382630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4f7fe640 1 -- 172.21.15.69:0/2483955948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0377990 msgr2=0x5603b0377d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0377990 0x5603b0377d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.985+0000 7f3c4f7fe640 1 -- 172.21.15.69:0/2483955948 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603affed580 con 0x5603b0382b70 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.986+0000 7f3c54801640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c380a2cf0 0x7f3c380a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c48005fa0 tx=0x7f3c480095c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.986+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b0384f60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c440cc670 tx=0x7f3c4409b340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52cb4669f3f4fbf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.986+0000 7f3c367fc640 1 -- 172.21.15.69:0/2483955948 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c44020040 con 0x5603b0382b70 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.986+0000 7f3c367fc640 1 -- 172.21.15.69:0/2483955948 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4409bb40 con 0x5603b0382b70 2024-08-02T17:50:17.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.986+0000 7f3c367fc640 1 -- 172.21.15.69:0/2483955948 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c440cc8f0 con 0x5603b0382b70 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 -- 172.21.15.69:0/2483955948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 msgr2=0x5603b0384f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b0384f60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c440cc670 tx=0x7f3c4409b340 comp rx=0 tx=0).stop 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 -- 172.21.15.69:0/2483955948 shutdown_connections 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b0384f60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b0382630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 --2- 172.21.15.69:0/2483955948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0377990 0x5603b0377d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 -- 172.21.15.69:0/2483955948 >> 172.21.15.69:0/2483955948 conn(0x5603b0373f20 msgr2=0x5603b0374300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 -- 172.21.15.69:0/2483955948 shutdown_connections 2024-08-02T17:50:17.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 -- 172.21.15.69:0/2483955948 wait complete. 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.987+0000 7f3c58662740 1 -- start start 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c58662740 1 Processor -- start 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c58662740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b037fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c58662740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b037fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c58662740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b0380530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c58662740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5603b0385740 con 0x5603b0382b70 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c58662740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5603b027b7c0 con 0x5603b0377990 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c58662740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5603b03858c0 con 0x5603b03782b0 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c58662740 10 client.0 _ll_register_callbacks cb 0x5603b0284f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c54801640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b0380530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c54801640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b0380530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55018/0 (socket says 172.21.15.69:55018) 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c54801640 1 -- 172.21.15.69:0/777317413 learned_addr learned my addr 172.21.15.69:0/777317413 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:17.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b037fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b037fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c54801640 1 -- 172.21.15.69:0/777317413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 msgr2=0x5603b037fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c54801640 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b037fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c54801640 1 -- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 msgr2=0x5603b037fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c54801640 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b037fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c54801640 1 -- 172.21.15.69:0/777317413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b00b7960 con 0x5603b0382b70 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.988+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b037fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c54801640 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b0380530 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3c4802fb10 tx=0x7f3c4802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187c7646c480e04b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:50:17.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/777317413 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4804f030 con 0x5603b0382b70 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/777317413 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3c48003d00 con 0x5603b0382b70 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c58662740 1 -- 172.21.15.69:0/777317413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c4409bb40 con 0x5603b0382b70 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/777317413 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c48034040 con 0x5603b0382b70 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c58662740 10 client.16437 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c58662740 1 -- 172.21.15.69:0/777317413 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5603b03917d0 con 0x5603b0382b70 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/777317413 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4409bb40 con 0x5603b0382b70 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.989+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/777317413 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c4804b460 con 0x5603b0382b70 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c58662740 10 client.16437 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c58662740 10 client.16437 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c58662740 20 client.16437 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c58662740 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5603b0392c90 0x5603b03950c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c58662740 4 client.16437 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:17.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c58662740 1 -- 172.21.15.69:0/777317413 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603b02eb1d0 con 0x5603b0392c90 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5603b0392c90 0x5603b03950c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5603b0392c90 0x5603b03950c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f163b3e84be5c473 server_cookie=3fffa55f5186710d in_seq=0 out_seq=0 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.990+0000 7f3c2f7fe640 10 client.16437 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/777317413 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5603b02eb1d0 con 0x5603b0392c90 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c2f7fe640 10 client.16437 handle_command_reply: tid=0 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 client.16437 shutdown 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 -- 172.21.15.69:0/777317413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 msgr2=0x5603b0380530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b0380530 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f3c4802fb10 tx=0x7f3c4802a950 comp rx=0 tx=0).stop 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 -- 172.21.15.69:0/777317413 shutdown_connections 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b0380530 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5603b0392c90 0x5603b03950c0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b037fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 --2- 172.21.15.69:0/777317413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b037fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c2f7fe640 0 client.16437 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:18.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 -- 172.21.15.69:0/777317413 >> 172.21.15.69:0/777317413 conn(0x5603b0373f20 msgr2=0x5603b0379c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:18.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 -- 172.21.15.69:0/777317413 shutdown_connections 2024-08-02T17:50:18.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 1 -- 172.21.15.69:0/777317413 wait complete. 2024-08-02T17:50:18.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:17.991+0000 7f3c58662740 20 client.16437 trim_cache size 0 max 16384 2024-08-02T17:50:18.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c58662740 1 Processor -- start 2024-08-02T17:50:18.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c58662740 1 -- start start 2024-08-02T17:50:18.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c58662740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b039a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:18.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c58662740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b039af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:18.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c58662740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b037cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:18.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c58662740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501b38e0 con 0x5603b0382b70 2024-08-02T17:50:18.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c58662740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501b11b0 con 0x5603b0377990 2024-08-02T17:50:18.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c58662740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501af000 con 0x5603b03782b0 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c54801640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b037cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c54801640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b037cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55028/0 (socket says 172.21.15.69:55028) 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c54801640 1 -- 172.21.15.69:0/3215270102 learned_addr learned my addr 172.21.15.69:0/3215270102 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b039a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.009+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b039af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c54801640 1 -- 172.21.15.69:0/3215270102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 msgr2=0x5603b039af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c54801640 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b039af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c54801640 1 -- 172.21.15.69:0/3215270102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 msgr2=0x5603b039a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c54801640 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b039a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c54801640 1 -- 172.21.15.69:0/3215270102 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603b0372470 con 0x5603b0382b70 2024-08-02T17:50:18.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b039a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:18.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c54801640 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b037cd10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3c48004af0 tx=0x7f3c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afed467f45c6478 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:18.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c35ffb640 1 -- 172.21.15.69:0/3215270102 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4804f070 con 0x5603b0382b70 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c35ffb640 1 -- 172.21.15.69:0/3215270102 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4804bb40 con 0x5603b0382b70 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.010+0000 7f3c35ffb640 1 -- 172.21.15.69:0/3215270102 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c48004da0 con 0x5603b0382b70 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 -- 172.21.15.69:0/3215270102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 msgr2=0x5603b037cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b037cd10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f3c48004af0 tx=0x7f3c48004a40 comp rx=0 tx=0).stop 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 -- 172.21.15.69:0/3215270102 shutdown_connections 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b0382b70 0x5603b037cd10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b03782b0 0x5603b039af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 --2- 172.21.15.69:0/3215270102 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0377990 0x5603b039a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 -- 172.21.15.69:0/3215270102 >> 172.21.15.69:0/3215270102 conn(0x5603b0373f20 msgr2=0x5603b0381b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 -- 172.21.15.69:0/3215270102 shutdown_connections 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.011+0000 7f3c58662740 1 -- 172.21.15.69:0/3215270102 wait complete. 2024-08-02T17:50:18.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 1 -- start start 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 1 Processor -- start 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b0377990 0x5603b0386c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b03782b0 0x5603b0387160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b03fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501af670 con 0x5603b03782b0 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501b0ae0 con 0x5603b0382b70 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c501afd40 con 0x5603b0377990 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c58662740 10 client.0 _ll_register_callbacks cb 0x7f3c4802fed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:18.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b0377990 0x5603b0386c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c4ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b0377990 0x5603b0386c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57288/0 (socket says 172.21.15.69:57288) 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.012+0000 7f3c4ffff640 1 -- 172.21.15.69:0/641031359 learned_addr learned my addr 172.21.15.69:0/641031359 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b03782b0 0x5603b0387160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c54801640 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b03fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c4ffff640 1 -- 172.21.15.69:0/641031359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 msgr2=0x5603b03fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b03fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c4ffff640 1 -- 172.21.15.69:0/641031359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b03782b0 msgr2=0x5603b0387160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b03782b0 0x5603b0387160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c4ffff640 1 -- 172.21.15.69:0/641031359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603b03fcef0 con 0x5603b0377990 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c4ffff640 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b0377990 0x5603b0386c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5603b0395900 tx=0x7f3c40009b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e70eed126da4bdfe server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/641031359 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40012650 con 0x5603b0377990 2024-08-02T17:50:18.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c58662740 1 -- 172.21.15.69:0/641031359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c4804bb40 con 0x5603b0377990 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/641031359 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3c400127f0 con 0x5603b0377990 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/641031359 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3c400129f0 con 0x5603b0377990 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c58662740 10 client.25924 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.013+0000 7f3c58662740 1 -- 172.21.15.69:0/641031359 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5603b03fe0e0 con 0x5603b0377990 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/641031359 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c4804bb40 con 0x5603b0377990 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/641031359 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3c40022070 con 0x5603b0377990 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c58662740 10 client.25924 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c58662740 10 client.25924 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c58662740 20 client.25924 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:18.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c58662740 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5603b03fe830 0x5603b0400c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:18.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c58662740 4 client.25924 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "48023e9", "release": true, "await": true}] 2024-08-02T17:50:18.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c58662740 1 -- 172.21.15.69:0/641031359 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "48023e9", "release": true, "await": true}) -- 0x5603b02e56e0 con 0x5603b03fe830 2024-08-02T17:50:18.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.014+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5603b03fe830 0x5603b0400c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:18.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.015+0000 7f3c4f7fe640 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5603b03fe830 0x5603b0400c50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=823b8d4a1e9fee server_cookie=aa9b21f5333aec02 in_seq=0 out_seq=0 2024-08-02T17:50:18.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.015+0000 7f3c2f7fe640 10 client.25924 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:18.026 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:18.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.071+0000 7f3c2f7fe640 1 -- 172.21.15.69:0/641031359 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5603b02e56e0 con 0x5603b03fe830 2024-08-02T17:50:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.071+0000 7f3c2f7fe640 10 client.25924 handle_command_reply: tid=0 2024-08-02T17:50:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.071+0000 7f3c58662740 1 client.25924 shutdown 2024-08-02T17:50:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 -- 172.21.15.69:0/641031359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b0377990 msgr2=0x5603b0386c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b0377990 0x5603b0386c20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5603b0395900 tx=0x7f3c40009b00 comp rx=0 tx=0).stop 2024-08-02T17:50:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 -- 172.21.15.69:0/641031359 shutdown_connections 2024-08-02T17:50:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5603b03782b0 0x5603b0387160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5603b03fe830 0x5603b0400c50 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5603b0382b70 0x5603b03fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c2f7fe640 0 client.25924 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 --2- 172.21.15.69:0/641031359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5603b0377990 0x5603b0386c20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 -- 172.21.15.69:0/641031359 >> 172.21.15.69:0/641031359 conn(0x5603b0373f20 msgr2=0x5603b03793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 -- 172.21.15.69:0/641031359 shutdown_connections 2024-08-02T17:50:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 1 -- 172.21.15.69:0/641031359 wait complete. 2024-08-02T17:50:18.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.072+0000 7f3c58662740 20 client.25924 trim_cache size 0 max 16384 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 16, 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "48023e9": { 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 16, 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:50:18.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.8, 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:50:18.077 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:50:18.078 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:50:18.078 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:50:18.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 -- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c380a2cf0 msgr2=0x7f3c380a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:18.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c380a2cf0 0x7f3c380a51b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c48005fa0 tx=0x7f3c480095c0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 -- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c50074c70 msgr2=0x7f3c5019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c50074c70 0x7f3c5019a890 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400d770 tx=0x7f3c4400dc30 comp rx=0 tx=0).stop 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 -- 172.21.15.69:0/3199750474 shutdown_connections 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3c50074c70 0x7f3c5019a890 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3c380a2cf0 0x7f3c380a51b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3c5010e0e0 0x7f3c5019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 --2- 172.21.15.69:0/3199750474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c50074350 0x7f3c5019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 -- 172.21.15.69:0/3199750474 >> 172.21.15.69:0/3199750474 conn(0x7f3c500fd5e0 msgr2=0x7f3c500fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.073+0000 7f3c56264640 1 -- 172.21.15.69:0/3199750474 shutdown_connections 2024-08-02T17:50:18.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:18.074+0000 7f3c56264640 1 -- 172.21.15.69:0/3199750474 wait complete. 2024-08-02T17:50:19.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:19 smithi077 ceph-mon[37108]: pgmap v120: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 669 op/s 2024-08-02T17:50:19.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:19 smithi069 ceph-mon[32746]: pgmap v120: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 669 op/s 2024-08-02T17:50:19.451 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '48023e9', total seconds elapsed: 28.9 2024-08-02T17:50:19.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:19 smithi083 ceph-mon[37031]: pgmap v120: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 9.5 MiB/s wr, 669 op/s 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 -- 172.21.15.69:0/3001509040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff4103ad0 msgr2=0x7f5ff4103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/3001509040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff4103ad0 0x7f5ff4103f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009930 tx=0x7f5fe802a440 comp rx=0 tx=0).stop 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 -- 172.21.15.69:0/3001509040 shutdown_connections 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/3001509040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff4103ad0 0x7f5ff4103f30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/3001509040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ff4104470 0x7f5ff41109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/3001509040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff4109b60 0x7f5ff4109f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 -- 172.21.15.69:0/3001509040 >> 172.21.15.69:0/3001509040 conn(0x7f5ff40ff750 msgr2=0x7f5ff4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 -- 172.21.15.69:0/3001509040 shutdown_connections 2024-08-02T17:50:19.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.649+0000 7f5ff9a98640 1 -- 172.21.15.69:0/3001509040 wait complete. 2024-08-02T17:50:19.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.650+0000 7f5ff9a98640 1 Processor -- start 2024-08-02T17:50:19.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.650+0000 7f5ff9a98640 1 -- start start 2024-08-02T17:50:19.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff9a98640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff4103ad0 0x7f5ff419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff9a98640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ff4104470 0x7f5ff419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff9a98640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff4109b60 0x7f5ff419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff9a98640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113ee0 con 0x7f5ff4103ad0 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff9a98640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113d60 con 0x7f5ff4104470 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ff4104470 0x7f5ff419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ff4104470 0x7f5ff419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59838/0 (socket says 172.21.15.69:59838) 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/4190315307 learned_addr learned my addr 172.21.15.69:0/4190315307 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4114060 con 0x7f5ff4109b60 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff4103ad0 0x7f5ff419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff4109b60 0x7f5ff419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/4190315307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff4109b60 msgr2=0x7f5ff419eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff4109b60 0x7f5ff419eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff4103ad0 msgr2=0x7f5ff419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff4103ad0 0x7f5ff419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/4190315307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff419f190 con 0x7f5ff4104470 2024-08-02T17:50:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.651+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ff4104470 0x7f5ff419a720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fe802a960 tx=0x7f5fe8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.652+0000 7f5ff0ff9640 1 -- 172.21.15.69:0/4190315307 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8035030 con 0x7f5ff4104470 2024-08-02T17:50:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.652+0000 7f5ff0ff9640 1 -- 172.21.15.69:0/4190315307 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe8045070 con 0x7f5ff4104470 2024-08-02T17:50:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.652+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff419f420 con 0x7f5ff4104470 2024-08-02T17:50:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.652+0000 7f5ff0ff9640 1 -- 172.21.15.69:0/4190315307 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8030e90 con 0x7f5ff4104470 2024-08-02T17:50:19.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.652+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff419f840 con 0x7f5ff4104470 2024-08-02T17:50:19.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.655+0000 7f5ff0ff9640 1 -- 172.21.15.69:0/4190315307 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5fe80307f0 con 0x7f5ff4104470 2024-08-02T17:50:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ffbe96740 1 Processor -- start 2024-08-02T17:50:19.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ffbe96740 1 -- start start 2024-08-02T17:50:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ff0ff9640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fdc0a2df0 0x7f5fdc0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd62c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ffbe96740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560ddccbda40 con 0x560ddcd631d0 2024-08-02T17:50:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ffbe96740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ddcc559d0 con 0x560ddcd57ff0 2024-08-02T17:50:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ffbe96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560ddccb9a00 con 0x560ddcd58910 2024-08-02T17:50:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.656+0000 7f5ff0ff9640 1 -- 172.21.15.69:0/4190315307 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5fe80f8760 con 0x7f5ff4104470 2024-08-02T17:50:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.657+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.657+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fdc0a2df0 0x7f5fdc0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.657+0000 7f5ff2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd62c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.657+0000 7f5ff2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd62c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57304/0 (socket says 172.21.15.69:57304) 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.657+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/1270985538 learned_addr learned my addr 172.21.15.69:0/1270985538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fdc0a2df0 0x7f5fdc0a52b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009270 tx=0x7f5fec009200 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5ff37fe640 1 -- 172.21.15.69:0/1270985538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 msgr2=0x560ddcd62c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/1270985538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd62c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5ff37fe640 1 -- 172.21.15.69:0/1270985538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 msgr2=0x560ddcd655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/1270985538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5ff37fe640 1 -- 172.21.15.69:0/1270985538 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ddcc5c650 con 0x560ddcd57ff0 2024-08-02T17:50:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/1270985538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd583d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009af0 tx=0x7f5fe40113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490df263c1124ccb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5fd9ffb640 1 -- 172.21.15.69:0/1270985538 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4011e10 con 0x560ddcd57ff0 2024-08-02T17:50:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5fd9ffb640 1 -- 172.21.15.69:0/1270985538 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe4016070 con 0x560ddcd57ff0 2024-08-02T17:50:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.658+0000 7f5fd9ffb640 1 -- 172.21.15.69:0/1270985538 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4011e10 con 0x560ddcd57ff0 2024-08-02T17:50:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.659+0000 7f5ffbe96740 1 -- 172.21.15.69:0/1270985538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 msgr2=0x560ddcd583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.659+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/1270985538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd583d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009af0 tx=0x7f5fe40113e0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.659+0000 7f5ffbe96740 1 -- 172.21.15.69:0/1270985538 shutdown_connections 2024-08-02T17:50:19.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.659+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/1270985538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.659+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/1270985538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd62c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.659+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/1270985538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd583d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.659+0000 7f5ffbe96740 1 -- 172.21.15.69:0/1270985538 >> 172.21.15.69:0/1270985538 conn(0x560ddcd54580 msgr2=0x560ddcd54960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:19.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.660+0000 7f5ffbe96740 1 -- 172.21.15.69:0/1270985538 shutdown_connections 2024-08-02T17:50:19.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.660+0000 7f5ffbe96740 1 -- 172.21.15.69:0/1270985538 wait complete. 2024-08-02T17:50:19.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.661+0000 7f5ffbe96740 1 -- start start 2024-08-02T17:50:19.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.661+0000 7f5ffbe96740 1 Processor -- start 2024-08-02T17:50:19.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.661+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd57ff0 0x560ddcd62c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd58910 0x560ddcd5ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd57ff0 0x560ddcd62c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd57ff0 0x560ddcd62c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57314/0 (socket says 172.21.15.69:57314) 2024-08-02T17:50:19.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd58910 0x560ddcd5ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd58910 0x560ddcd5ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55044/0 (socket says 172.21.15.69:55044) 2024-08-02T17:50:19.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff37fe640 1 -- 172.21.15.69:0/2984281635 learned_addr learned my addr 172.21.15.69:0/2984281635 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:19.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd631d0 0x560ddcd60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560ddcd76db0 con 0x560ddcd58910 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560ddcc78b20 con 0x560ddcd631d0 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560ddcd76f30 con 0x560ddcd57ff0 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd631d0 0x560ddcd60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ffbe96740 10 client.0 _ll_register_callbacks cb 0x560ddcc76bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff3fff640 1 -- 172.21.15.69:0/2984281635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd57ff0 msgr2=0x560ddcd62c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd57ff0 0x560ddcd62c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff3fff640 1 -- 172.21.15.69:0/2984281635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd58910 msgr2=0x560ddcd5ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd58910 0x560ddcd5ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.662+0000 7f5ff3fff640 1 -- 172.21.15.69:0/2984281635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ddca2f3c0 con 0x560ddcd631d0 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.663+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd57ff0 0x560ddcd62c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.663+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd58910 0x560ddcd5ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:19.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.663+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd631d0 0x560ddcd60760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x560ddcd55900 tx=0x7f5fec009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16a5880d40aad87 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.663+0000 7f5fd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.664+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/2984281635 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fec029060 con 0x560ddcd631d0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.664+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/2984281635 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5fec0108a0 con 0x560ddcd631d0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.664+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe4016070 con 0x560ddcd631d0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.664+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/2984281635 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5fec030050 con 0x560ddcd631d0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.664+0000 7f5ffbe96740 10 client.26048 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.664+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ddcd82970 con 0x560ddcd631d0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.664+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/2984281635 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fec024440 con 0x560ddcd631d0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.665+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/2984281635 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5fec01c400 con 0x560ddcd631d0 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.665+0000 7f5ffbe96740 10 client.26048 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.665+0000 7f5ffbe96740 10 client.26048 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.665+0000 7f5ffbe96740 20 client.26048 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:19.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.665+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560ddcd83e30 0x560ddcd86220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.665+0000 7f5ffbe96740 4 client.26048 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.665+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5fe4016070 con 0x560ddcd83e30 2024-08-02T17:50:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.665+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560ddcd83e30 0x560ddcd86220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.666+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560ddcd83e30 0x560ddcd86220 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9707466661588a8 server_cookie=59e48ce7f02cd5de in_seq=0 out_seq=0 2024-08-02T17:50:19.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.666+0000 7f5fd2ffd640 10 client.26048 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/2984281635 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f5fe4016070 con 0x560ddcd83e30 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5fd2ffd640 10 client.26048 handle_command_reply: tid=0 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 client.26048 shutdown 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd631d0 msgr2=0x560ddcd60760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd631d0 0x560ddcd60760 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x560ddcd55900 tx=0x7f5fec009500 comp rx=0 tx=0).stop 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 shutdown_connections 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560ddcd83e30 0x560ddcd86220 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5fd2ffd640 0 client.26048 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:50:19.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd58910 0x560ddcd5ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd631d0 0x560ddcd60760 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/2984281635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd57ff0 0x560ddcd62c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.667+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 >> 172.21.15.69:0/2984281635 conn(0x560ddcd54580 msgr2=0x560ddcd5a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:19.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.668+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 shutdown_connections 2024-08-02T17:50:19.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.668+0000 7f5ffbe96740 1 -- 172.21.15.69:0/2984281635 wait complete. 2024-08-02T17:50:19.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.668+0000 7f5ffbe96740 20 client.26048 trim_cache size 0 max 16384 2024-08-02T17:50:19.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.695+0000 7f5ffbe96740 1 Processor -- start 2024-08-02T17:50:19.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.695+0000 7f5ffbe96740 1 -- start start 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd5f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd5fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ffbe96740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff41a6db0 con 0x560ddcd631d0 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd5fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59858/0 (socket says 172.21.15.69:59858) 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff37fe640 1 -- 172.21.15.69:0/98050442 learned_addr learned my addr 172.21.15.69:0/98050442 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ffbe96740 1 -- 172.21.15.69:0/98050442 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4071a40 con 0x560ddcd57ff0 2024-08-02T17:50:19.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ffbe96740 1 -- 172.21.15.69:0/98050442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff406f8d0 con 0x560ddcd58910 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd5f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/98050442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 msgr2=0x560ddcd606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/98050442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 msgr2=0x560ddcd5fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd5fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.696+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/98050442 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ddcd58740 con 0x560ddcd58910 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.697+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd5fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.697+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd5f890 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80c1340 tx=0x7f5fe800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71939f125f142f98 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:19.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.697+0000 7f5fd97fa640 1 -- 172.21.15.69:0/98050442 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80c5070 con 0x560ddcd58910 2024-08-02T17:50:19.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.698+0000 7f5fd97fa640 1 -- 172.21.15.69:0/98050442 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe80c4970 con 0x560ddcd58910 2024-08-02T17:50:19.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.698+0000 7f5fd97fa640 1 -- 172.21.15.69:0/98050442 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80f5d10 con 0x560ddcd58910 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 -- 172.21.15.69:0/98050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 msgr2=0x560ddcd5f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd5f890 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80c1340 tx=0x7f5fe800b040 comp rx=0 tx=0).stop 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 -- 172.21.15.69:0/98050442 shutdown_connections 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd5fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd5f890 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/98050442 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 -- 172.21.15.69:0/98050442 >> 172.21.15.69:0/98050442 conn(0x560ddcd54580 msgr2=0x560ddcd5e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 -- 172.21.15.69:0/98050442 shutdown_connections 2024-08-02T17:50:19.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.700+0000 7f5ffbe96740 1 -- 172.21.15.69:0/98050442 wait complete. 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.701+0000 7f5ffbe96740 1 -- start start 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.701+0000 7f5ffbe96740 1 Processor -- start 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.701+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd7c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.701+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd7c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ffbe96740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd77710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ffbe96740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff406ff20 con 0x560ddcd631d0 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ffbe96740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4071370 con 0x560ddcd57ff0 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ffbe96740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff40705d0 con 0x560ddcd58910 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ffbe96740 10 client.0 _ll_register_callbacks cb 0x7f5fec003e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:19.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd7c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd7c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57340/0 (socket says 172.21.15.69:57340) 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/87507402 learned_addr learned my addr 172.21.15.69:0/87507402 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd7c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 msgr2=0x560ddcd7c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd7c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/87507402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 msgr2=0x560ddcd77710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd77710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd77710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.702+0000 7f5ff2ffd640 1 -- 172.21.15.69:0/87507402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ddcd77f40 con 0x560ddcd58910 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd7c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5ff3fff640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd77710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:19.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5ff2ffd640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd7c7c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8053040 tx=0x7f5fe80305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7ff425aaef27a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5fd2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/87507402 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80e1070 con 0x560ddcd58910 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5ffbe96740 1 -- 172.21.15.69:0/87507402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe80c4970 con 0x560ddcd58910 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/87507402 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f5fe80e3310 con 0x560ddcd58910 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/87507402 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5fe80ac050 con 0x560ddcd58910 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5ffbe96740 10 client.25939 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.703+0000 7f5ffbe96740 1 -- 172.21.15.69:0/87507402 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560ddcdee550 con 0x560ddcd58910 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/87507402 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe80c4970 con 0x560ddcd58910 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/87507402 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5fe8052030 con 0x560ddcd58910 2024-08-02T17:50:19.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5ffbe96740 10 client.25939 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5ffbe96740 10 client.25939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:50:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5ffbe96740 20 client.25939 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560ddcd83e30 0x560ddcdf0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5ffbe96740 4 client.25939 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:50:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5ffbe96740 1 -- 172.21.15.69:0/87507402 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ddcccafb0 con 0x560ddcd83e30 2024-08-02T17:50:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560ddcd83e30 0x560ddcdf0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:19.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.704+0000 7f5fd2ffd640 10 client.25939 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.705+0000 7f5ff37fe640 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560ddcd83e30 0x560ddcdf0c20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f1fbd5e1d152eee server_cookie=809b122a87214f3f in_seq=0 out_seq=0 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.706+0000 7f5fd2ffd640 1 -- 172.21.15.69:0/87507402 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ddcccafb0 con 0x560ddcd83e30 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.706+0000 7f5fd2ffd640 10 client.25939 handle_command_reply: tid=0 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.706+0000 7f5ffbe96740 1 client.25939 shutdown 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.706+0000 7f5ffbe96740 1 -- 172.21.15.69:0/87507402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 msgr2=0x560ddcd7c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.706+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd7c7c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8053040 tx=0x7f5fe80305d0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.707+0000 7f5ffbe96740 1 -- 172.21.15.69:0/87507402 shutdown_connections 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.707+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x560ddcd83e30 0x560ddcdf0c20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.707+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560ddcd631d0 0x560ddcd77710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.707+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560ddcd58910 0x560ddcd7c7c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.707+0000 7f5ffbe96740 1 --2- 172.21.15.69:0/87507402 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560ddcd57ff0 0x560ddcd7c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.707+0000 7f5ffbe96740 1 -- 172.21.15.69:0/87507402 >> 172.21.15.69:0/87507402 conn(0x560ddcd54580 msgr2=0x560ddcd5e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:19.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.708+0000 7f5fd2ffd640 0 client.25939 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:50:19.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.708+0000 7f5ffbe96740 1 -- 172.21.15.69:0/87507402 shutdown_connections 2024-08-02T17:50:19.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.708+0000 7f5ffbe96740 1 -- 172.21.15.69:0/87507402 wait complete. 2024-08-02T17:50:19.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.708+0000 7f5ffbe96740 20 client.25939 trim_cache size 0 max 16384 2024-08-02T17:50:19.712 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:50:19.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fdc0a2df0 msgr2=0x7f5fdc0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fdc0a2df0 0x7f5fdc0a52b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009270 tx=0x7f5fec009200 comp rx=0 tx=0).stop 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ff4104470 msgr2=0x7f5ff419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ff4104470 0x7f5ff419a720 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fe802a960 tx=0x7f5fe8005d50 comp rx=0 tx=0).stop 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 shutdown_connections 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fdc0a2df0 0x7f5fdc0a52b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5ff4109b60 0x7f5ff419eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5ff4104470 0x7f5ff419a720 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 --2- 172.21.15.69:0/4190315307 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5ff4103ad0 0x7f5ff419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 >> 172.21.15.69:0/4190315307 conn(0x7f5ff40ff750 msgr2=0x7f5ff40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 shutdown_connections 2024-08-02T17:50:19.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:19.709+0000 7f5ff9a98640 1 -- 172.21.15.69:0/4190315307 wait complete. 2024-08-02T17:50:19.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 190.0 s, 422.5 tps, lat 1225.594 ms stddev 5148.816 2024-08-02T17:50:20.531 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:50:20.936 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:21 smithi077 ceph-mon[37108]: pgmap v121: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.8 MiB/s wr, 697 op/s 2024-08-02T17:50:21.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:21 smithi069 ceph-mon[32746]: pgmap v121: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.8 MiB/s wr, 697 op/s 2024-08-02T17:50:21.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:21 smithi083 ceph-mon[37031]: pgmap v121: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 9.8 MiB/s wr, 697 op/s 2024-08-02T17:50:21.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.543+0000 7fcc489e5640 1 -- 172.21.15.69:0/3748059290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc44100b50 msgr2=0x7fcc44100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.543+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3748059290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc44100b50 0x7fcc44100f30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009900 tx=0x7fcc3402a4a0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.545+0000 7fcc489e5640 1 -- 172.21.15.69:0/3748059290 shutdown_connections 2024-08-02T17:50:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.545+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3748059290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102c20 0x7fcc4410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.545+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3748059290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc441022a0 0x7fcc441026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.545+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3748059290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc44100b50 0x7fcc44100f30 secure :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009900 tx=0x7fcc3402a4a0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.545+0000 7fcc489e5640 1 -- 172.21.15.69:0/3748059290 >> 172.21.15.69:0/3748059290 conn(0x7fcc440fc500 msgr2=0x7fcc440fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.546+0000 7fcc489e5640 1 -- 172.21.15.69:0/3748059290 shutdown_connections 2024-08-02T17:50:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.546+0000 7fcc489e5640 1 -- 172.21.15.69:0/3748059290 wait complete. 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.547+0000 7fcc489e5640 1 Processor -- start 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.547+0000 7fcc489e5640 1 -- start start 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.547+0000 7fcc489e5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc441022a0 0x7fcc4410aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.547+0000 7fcc489e5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102c20 0x7fcc4410afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc489e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc4410b510 0x7fcc441aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc489e5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4410fc10 con 0x7fcc441022a0 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc489e5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4410fa90 con 0x7fcc44102c20 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc489e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4410fd90 con 0x7fcc4410b510 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc441022a0 0x7fcc4410aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc441022a0 0x7fcc4410aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55086/0 (socket says 172.21.15.69:55086) 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc4259c640 1 -- 172.21.15.69:0/3015316568 learned_addr learned my addr 172.21.15.69:0/3015316568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102c20 0x7fcc4410afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc4410b510 0x7fcc441aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc4259c640 1 -- 172.21.15.69:0/3015316568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc4410b510 msgr2=0x7fcc441aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc4259c640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc4410b510 0x7fcc441aab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc4259c640 1 -- 172.21.15.69:0/3015316568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102c20 msgr2=0x7fcc4410afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc4259c640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102c20 0x7fcc4410afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.548+0000 7fcc4259c640 1 -- 172.21.15.69:0/3015316568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc441ab130 con 0x7fcc441022a0 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.549+0000 7fcc4259c640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc441022a0 0x7fcc4410aa90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcc34006010 tx=0x7fcc34003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.549+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc4410b510 0x7fcc441aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.549+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102c20 0x7fcc4410afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.549+0000 7fcc337fe640 1 -- 172.21.15.69:0/3015316568 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3403f020 con 0x7fcc441022a0 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.549+0000 7fcc337fe640 1 -- 172.21.15.69:0/3015316568 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc34005d10 con 0x7fcc441022a0 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.549+0000 7fcc337fe640 1 -- 172.21.15.69:0/3015316568 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc34035070 con 0x7fcc441022a0 2024-08-02T17:50:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.549+0000 7fcc489e5640 1 -- 172.21.15.69:0/3015316568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc441ab3c0 con 0x7fcc441022a0 2024-08-02T17:50:21.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.549+0000 7fcc489e5640 1 -- 172.21.15.69:0/3015316568 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc441ab920 con 0x7fcc441022a0 2024-08-02T17:50:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.553+0000 7fcc337fe640 1 -- 172.21.15.69:0/3015316568 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcc34042020 con 0x7fcc441022a0 2024-08-02T17:50:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.553+0000 7fcc4ade3740 1 Processor -- start 2024-08-02T17:50:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.553+0000 7fcc337fe640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc2c0a2df0 0x7fcc2c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.554+0000 7fcc337fe640 1 -- 172.21.15.69:0/3015316568 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcc340effd0 con 0x7fcc441022a0 2024-08-02T17:50:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.556+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc2c0a2df0 0x7fcc2c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.556+0000 7fcc4ade3740 1 -- start start 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.556+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.556+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eadc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.556+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eb05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.556+0000 7fcc41d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eadc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc41d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eadc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59894/0 (socket says 172.21.15.69:59894) 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc42d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eb05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc42d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eb05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55100/0 (socket says 172.21.15.69:55100) 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57366/0 (socket says 172.21.15.69:57366) 2024-08-02T17:50:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc41d9b640 1 -- 172.21.15.69:0/1038203310 learned_addr learned my addr 172.21.15.69:0/1038203310 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1038203310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f022e08a40 con 0x55f022eae1d0 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1038203310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f022da09d0 con 0x55f022ea3910 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1038203310 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f022e04a00 con 0x55f022ea2ff0 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc41d9b640 1 -- 172.21.15.69:0/1038203310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 msgr2=0x55f022ea33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc41d9b640 1 -- 172.21.15.69:0/1038203310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 msgr2=0x55f022eb05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eb05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc41d9b640 1 -- 172.21.15.69:0/1038203310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f022da7650 con 0x55f022ea3910 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eb05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.557+0000 7fcc4259c640 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.558+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc2c0a2df0 0x7fcc2c0a52b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00c830 tx=0x7fcc3c00cc30 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.558+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eadc90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc380114f0 tx=0x7fcc380119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b12228efaed5269d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.558+0000 7fcc30ff9640 1 -- 172.21.15.69:0/1038203310 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc380110f0 con 0x55f022ea3910 2024-08-02T17:50:21.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.558+0000 7fcc30ff9640 1 -- 172.21.15.69:0/1038203310 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc38011290 con 0x55f022ea3910 2024-08-02T17:50:21.562 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.558+0000 7fcc30ff9640 1 -- 172.21.15.69:0/1038203310 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3800f560 con 0x55f022ea3910 2024-08-02T17:50:21.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.558+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1038203310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 msgr2=0x55f022eadc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.558+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eadc90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcc380114f0 tx=0x7fcc380119b0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.559+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1038203310 shutdown_connections 2024-08-02T17:50:21.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.559+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eb05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.559+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eadc90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.559+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1038203310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.559+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1038203310 >> 172.21.15.69:0/1038203310 conn(0x55f022e9f580 msgr2=0x55f022e9f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.559+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1038203310 shutdown_connections 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.559+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1038203310 wait complete. 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 1 -- start start 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 1 Processor -- start 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022eab010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eaba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f022eb0da0 con 0x55f022eae1d0 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f022dc3b20 con 0x55f022ea3910 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f022eb0f20 con 0x55f022ea2ff0 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.560+0000 7fcc4ade3740 10 client.0 _ll_register_callbacks cb 0x55f022dc1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022eab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc42d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eaba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59902/0 (socket says 172.21.15.69:59902) 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022eab010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57370/0 (socket says 172.21.15.69:57370) 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 -- 172.21.15.69:0/2100293670 learned_addr learned my addr 172.21.15.69:0/2100293670 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 -- 172.21.15.69:0/2100293670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 msgr2=0x55f022eab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022eab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 -- 172.21.15.69:0/2100293670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 msgr2=0x55f022eaba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eaba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 -- 172.21.15.69:0/2100293670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f022b7a3c0 con 0x55f022ea3910 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eaba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eab550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc3801b040 tx=0x7fcc38004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=380d4e8d030b988d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc29ffb640 1 -- 172.21.15.69:0/2100293670 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3803a020 con 0x55f022ea3910 2024-08-02T17:50:21.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2100293670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc38011290 con 0x55f022ea3910 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.561+0000 7fcc29ffb640 1 -- 172.21.15.69:0/2100293670 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcc38004c30 con 0x55f022ea3910 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc29ffb640 1 -- 172.21.15.69:0/2100293670 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcc38045050 con 0x55f022ea3910 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4ade3740 10 client.26063 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2100293670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f022ebc880 con 0x55f022ea3910 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc29ffb640 1 -- 172.21.15.69:0/2100293670 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc38011290 con 0x55f022ea3910 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc29ffb640 1 -- 172.21.15.69:0/2100293670 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcc38036480 con 0x55f022ea3910 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4ade3740 10 client.26063 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4ade3740 10 client.26063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4ade3740 20 client.26063 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f022ebdc30 0x55f022ec0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4ade3740 4 client.26063 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2100293670 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f022e15fb0 con 0x55f022ebdc30 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.562+0000 7fcc4259c640 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f022ebdc30 0x55f022ec0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.563+0000 7fcc29ffb640 10 client.26063 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:21.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.563+0000 7fcc4259c640 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f022ebdc30 0x55f022ec0020 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8774f3fc00d20d9 server_cookie=423b41b677c2acca in_seq=0 out_seq=0 2024-08-02T17:50:21.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.565+0000 7fcc29ffb640 1 -- 172.21.15.69:0/2100293670 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f022e15fb0 con 0x55f022ebdc30 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.565+0000 7fcc29ffb640 10 client.26063 handle_command_reply: tid=0 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.565+0000 7fcc4ade3740 1 client.26063 shutdown 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2100293670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 msgr2=0x55f022eab550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eab550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc3801b040 tx=0x7fcc38004700 comp rx=0 tx=0).stop 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2100293670 shutdown_connections 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022eae1d0 0x55f022eaba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f022ebdc30 0x55f022ec0020 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022ea3910 0x55f022eab550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2100293670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022eab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2100293670 >> 172.21.15.69:0/2100293670 conn(0x55f022e9f580 msgr2=0x55f022ea5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc29ffb640 0 client.26063 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:21.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2100293670 shutdown_connections 2024-08-02T17:50:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2100293670 wait complete. 2024-08-02T17:50:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.566+0000 7fcc4ade3740 20 client.26063 trim_cache size 0 max 16384 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4ade3740 1 Processor -- start 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4ade3740 1 -- start start 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea3910 0x55f022eab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022ec5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4ade3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441af950 con 0x55f022ea3910 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4ade3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441ad220 con 0x55f022eae1d0 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57382/0 (socket says 172.21.15.69:57382) 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4259c640 1 -- 172.21.15.69:0/2008544758 learned_addr learned my addr 172.21.15.69:0/2008544758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc41d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea3910 0x55f022eab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/2008544758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022ec5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.585+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2008544758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4411c210 con 0x55f022ea2ff0 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc42d9d640 1 -- 172.21.15.69:0/2008544758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 msgr2=0x55f022ea5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/2008544758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc42d9d640 1 -- 172.21.15.69:0/2008544758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea3910 msgr2=0x55f022eab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/2008544758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea3910 0x55f022eab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc42d9d640 1 -- 172.21.15.69:0/2008544758 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f022eb0900 con 0x55f022eae1d0 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/2008544758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022ec5740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55f022ea0900 tx=0x7fcc3c01c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d725132da98b8279 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc2bfff640 1 -- 172.21.15.69:0/2008544758 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c01c900 con 0x55f022eae1d0 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc2bfff640 1 -- 172.21.15.69:0/2008544758 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc3c016040 con 0x55f022eae1d0 2024-08-02T17:50:21.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.586+0000 7fcc2bfff640 1 -- 172.21.15.69:0/2008544758 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c027c10 con 0x55f022eae1d0 2024-08-02T17:50:21.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2008544758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 msgr2=0x55f022ec5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2008544758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022ec5740 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55f022ea0900 tx=0x7fcc3c01c000 comp rx=0 tx=0).stop 2024-08-02T17:50:21.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2008544758 shutdown_connections 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2008544758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea3910 0x55f022eab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2008544758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022ec5740 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/2008544758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea2ff0 0x55f022ea5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2008544758 >> 172.21.15.69:0/2008544758 conn(0x55f022e9f580 msgr2=0x55f022ea4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2008544758 shutdown_connections 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.587+0000 7fcc4ade3740 1 -- 172.21.15.69:0/2008544758 wait complete. 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.588+0000 7fcc4ade3740 1 -- start start 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.588+0000 7fcc4ade3740 1 Processor -- start 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.588+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea2ff0 0x55f022eb6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.588+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea3910 0x55f022eb1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc4ade3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022eb1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea2ff0 0x55f022eb6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc41d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea3910 0x55f022eb1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc4259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea2ff0 0x55f022eb6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55126/0 (socket says 172.21.15.69:55126) 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc4259c640 1 -- 172.21.15.69:0/1874365610 learned_addr learned my addr 172.21.15.69:0/1874365610 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea3910 0x55f022eb1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57396/0 (socket says 172.21.15.69:57396) 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4411c880 con 0x55f022ea2ff0 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441acb50 con 0x55f022eae1d0 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc42d9d640 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022eb1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcc441abdb0 con 0x55f022ea3910 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc4ade3740 10 client.0 _ll_register_callbacks cb 0x7fcc38009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.589+0000 7fcc41d9b640 1 -- 172.21.15.69:0/1874365610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 msgr2=0x55f022eb1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.590+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022eb1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.590+0000 7fcc41d9b640 1 -- 172.21.15.69:0/1874365610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea2ff0 msgr2=0x55f022eb6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.590+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea2ff0 0x55f022eb6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.590+0000 7fcc41d9b640 1 -- 172.21.15.69:0/1874365610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f022f26700 con 0x55f022ea3910 2024-08-02T17:50:21.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.590+0000 7fcc41d9b640 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea3910 0x55f022eb1130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcc380368d0 tx=0x7fcc380366b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b23df178b87b7c00 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:21.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.593+0000 7fcc29ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:21.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.593+0000 7fcc29ffb640 1 -- 172.21.15.69:0/1874365610 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3803a070 con 0x55f022ea3910 2024-08-02T17:50:21.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.593+0000 7fcc29ffb640 1 -- 172.21.15.69:0/1874365610 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fcc380110f0 con 0x55f022ea3910 2024-08-02T17:50:21.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.593+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc3c016040 con 0x55f022ea3910 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.593+0000 7fcc29ffb640 1 -- 172.21.15.69:0/1874365610 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcc38051050 con 0x55f022ea3910 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.593+0000 7fcc4ade3740 10 client.25954 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.593+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f022f27880 con 0x55f022ea3910 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.593+0000 7fcc29ffb640 1 -- 172.21.15.69:0/1874365610 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc3c016040 con 0x55f022ea3910 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc29ffb640 1 -- 172.21.15.69:0/1874365610 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcc3801a450 con 0x55f022ea3910 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc4ade3740 10 client.25954 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc4ade3740 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc4ade3740 20 client.25954 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f022f27e20 0x55f022f2a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:21.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc4ade3740 4 client.25954 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f022e104c0 con 0x55f022f27e20 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc4259c640 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f022f27e20 0x55f022f2a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc29ffb640 10 client.25954 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.594+0000 7fcc4259c640 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f022f27e20 0x55f022f2a240 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192d869c930b5a37 server_cookie=1151a6d649fa50ec in_seq=0 out_seq=0 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc29ffb640 1 -- 172.21.15.69:0/1874365610 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f022e104c0 con 0x55f022f27e20 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc29ffb640 10 client.25954 handle_command_reply: tid=0 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 client.25954 shutdown 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea3910 msgr2=0x55f022eb1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea3910 0x55f022eb1130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcc380368d0 tx=0x7fcc380366b0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 shutdown_connections 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f022f27e20 0x55f022f2a240 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f022eae1d0 0x55f022eb1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f022ea3910 0x55f022eb1130 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc29ffb640 0 client.25954 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:21.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 --2- 172.21.15.69:0/1874365610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f022ea2ff0 0x55f022eb6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 >> 172.21.15.69:0/1874365610 conn(0x55f022e9f580 msgr2=0x55f022ea5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:21.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 shutdown_connections 2024-08-02T17:50:21.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 1 -- 172.21.15.69:0/1874365610 wait complete. 2024-08-02T17:50:21.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.595+0000 7fcc4ade3740 20 client.25954 trim_cache size 0 max 16384 2024-08-02T17:50:21.599 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:50:21.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 -- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc2c0a2df0 msgr2=0x7fcc2c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc2c0a2df0 0x7fcc2c0a52b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00c830 tx=0x7fcc3c00cc30 comp rx=0 tx=0).stop 2024-08-02T17:50:21.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 -- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc441022a0 msgr2=0x7fcc4410aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:21.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc441022a0 0x7fcc4410aa90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcc34006010 tx=0x7fcc34003c10 comp rx=0 tx=0).stop 2024-08-02T17:50:21.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 -- 172.21.15.69:0/3015316568 shutdown_connections 2024-08-02T17:50:21.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcc2c0a2df0 0x7fcc2c0a52b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcc4410b510 0x7fcc441aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcc44102c20 0x7fcc4410afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 --2- 172.21.15.69:0/3015316568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcc441022a0 0x7fcc4410aa90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:21.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.596+0000 7fcc489e5640 1 -- 172.21.15.69:0/3015316568 >> 172.21.15.69:0/3015316568 conn(0x7fcc440fc500 msgr2=0x7fcc440fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:21.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.597+0000 7fcc489e5640 1 -- 172.21.15.69:0/3015316568 shutdown_connections 2024-08-02T17:50:21.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:21.597+0000 7fcc489e5640 1 -- 172.21.15.69:0/3015316568 wait complete. 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 -- 172.21.15.69:0/4063672429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 msgr2=0x7f3eb8006530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 --2- 172.21.15.69:0/4063672429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 0x7f3eb8006530 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0009900 tx=0x7f3eb002a480 comp rx=0 tx=0).stop 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 -- 172.21.15.69:0/4063672429 shutdown_connections 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 --2- 172.21.15.69:0/4063672429 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 0x7f3eb8006530 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 --2- 172.21.15.69:0/4063672429 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb8006a70 0x7f3eb80a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 --2- 172.21.15.69:0/4063672429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb809b6c0 0x7f3eb809baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 -- 172.21.15.69:0/4063672429 >> 172.21.15.69:0/4063672429 conn(0x7f3eb800a290 msgr2=0x7f3eb800a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 -- 172.21.15.69:0/4063672429 shutdown_connections 2024-08-02T17:50:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.068+0000 7f3ec6549640 1 -- 172.21.15.69:0/4063672429 wait complete. 2024-08-02T17:50:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.069+0000 7f3ec6549640 1 Processor -- start 2024-08-02T17:50:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.069+0000 7f3ec6549640 1 -- start start 2024-08-02T17:50:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.069+0000 7f3ec6549640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 0x7f3eb812e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.069+0000 7f3ec6549640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb8006a70 0x7f3eb812e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.069+0000 7f3ec6549640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb809b6c0 0x7f3eb81328f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.069+0000 7f3ec6549640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8095580 con 0x7f3eb80060f0 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.069+0000 7f3ec6549640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8095400 con 0x7f3eb809b6c0 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.069+0000 7f3ec6549640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8095700 con 0x7f3eb8006a70 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb8006a70 0x7f3eb812e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb8006a70 0x7f3eb812e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57406/0 (socket says 172.21.15.69:57406) 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 -- 172.21.15.69:0/1450707792 learned_addr learned my addr 172.21.15.69:0/1450707792 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec5d48640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb809b6c0 0x7f3eb81328f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec5547640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 0x7f3eb812e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 -- 172.21.15.69:0/1450707792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb809b6c0 msgr2=0x7f3eb81328f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb809b6c0 0x7f3eb81328f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 -- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 msgr2=0x7f3eb812e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 0x7f3eb812e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 -- 172.21.15.69:0/1450707792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb8133020 con 0x7f3eb8006a70 2024-08-02T17:50:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb809b6c0 0x7f3eb81328f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec5547640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 0x7f3eb812e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.070+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb8006a70 0x7f3eb812e580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005940 tx=0x7f3eb0031540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.075+0000 7f3eb67fc640 1 -- 172.21.15.69:0/1450707792 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0040070 con 0x7f3eb8006a70 2024-08-02T17:50:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.075+0000 7f3eb67fc640 1 -- 172.21.15.69:0/1450707792 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eb0031bc0 con 0x7f3eb8006a70 2024-08-02T17:50:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.075+0000 7f3ec6549640 1 -- 172.21.15.69:0/1450707792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb8133310 con 0x7f3eb8006a70 2024-08-02T17:50:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.075+0000 7f3eb67fc640 1 -- 172.21.15.69:0/1450707792 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb0035020 con 0x7f3eb8006a70 2024-08-02T17:50:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.076+0000 7f3ec6549640 1 -- 172.21.15.69:0/1450707792 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb813aba0 con 0x7f3eb8006a70 2024-08-02T17:50:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.080+0000 7f3eca5db740 1 Processor -- start 2024-08-02T17:50:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.081+0000 7f3eb67fc640 1 -- 172.21.15.69:0/1450707792 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3eb0043070 con 0x7f3eb8006a70 2024-08-02T17:50:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eb67fc640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3eac0a2ed0 0x7f3eac0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eca5db740 1 -- start start 2024-08-02T17:50:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eb67fc640 1 -- 172.21.15.69:0/1450707792 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3eb00ef9e0 con 0x7f3eb8006a70 2024-08-02T17:50:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197ea150 0x561b197ea530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eca5db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561b197459b0 con 0x561b197eaa70 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eca5db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b196ed800 con 0x561b197f5310 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3eca5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561b19741970 con 0x561b197ea150 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3eac0a2ed0 0x7f3eac0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.084+0000 7f3ec4d46640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec4d46640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55154/0 (socket says 172.21.15.69:55154) 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec4d46640 1 -- 172.21.15.69:0/3937022801 learned_addr learned my addr 172.21.15.69:0/3937022801 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec4d46640 1 -- 172.21.15.69:0/3937022801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197ea150 msgr2=0x561b197ea530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/3937022801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197ea150 0x561b197ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec4d46640 1 -- 172.21.15.69:0/3937022801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 msgr2=0x561b197f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/3937022801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec4d46640 1 -- 172.21.15.69:0/3937022801 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b19459210 con 0x561b197eaa70 2024-08-02T17:50:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3eac0a2ed0 0x7f3eac0a5390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0067970 tx=0x7f3ec0068820 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.085+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/3937022801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197f4dd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0004770 tx=0x7f3eb00bba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b98aa0af9261c52 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.213+0000 7f3eab7fe640 1 -- 172.21.15.69:0/3937022801 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00b9e90 con 0x561b197eaa70 2024-08-02T17:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.213+0000 7f3eab7fe640 1 -- 172.21.15.69:0/3937022801 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3eb00b8200 con 0x561b197eaa70 2024-08-02T17:50:23.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.214+0000 7f3eab7fe640 1 -- 172.21.15.69:0/3937022801 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb00b9e90 con 0x561b197eaa70 2024-08-02T17:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.214+0000 7f3eca5db740 1 -- 172.21.15.69:0/3937022801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 msgr2=0x561b197f4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.214+0000 7f3eca5db740 1 --2- 172.21.15.69:0/3937022801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197f4dd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0004770 tx=0x7f3eb00bba90 comp rx=0 tx=0).stop 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 -- 172.21.15.69:0/3937022801 shutdown_connections 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 --2- 172.21.15.69:0/3937022801 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197f4dd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 --2- 172.21.15.69:0/3937022801 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 --2- 172.21.15.69:0/3937022801 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197ea150 0x561b197ea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 -- 172.21.15.69:0/3937022801 >> 172.21.15.69:0/3937022801 conn(0x561b197e66c0 msgr2=0x561b197e6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 -- 172.21.15.69:0/3937022801 shutdown_connections 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 -- 172.21.15.69:0/3937022801 wait complete. 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 -- start start 2024-08-02T17:50:23.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.215+0000 7f3eca5db740 1 Processor -- start 2024-08-02T17:50:23.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.219+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 0x561b197f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.219+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3ec4d46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3ec4d46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57420/0 (socket says 172.21.15.69:57420) 2024-08-02T17:50:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3ec5d48640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3ec5d48640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59946/0 (socket says 172.21.15.69:59946) 2024-08-02T17:50:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3ec5d48640 1 -- 172.21.15.69:0/2038202577 learned_addr learned my addr 172.21.15.69:0/2038202577 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561b197f7ca0 con 0x561b197ea150 2024-08-02T17:50:23.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b19701500 con 0x561b197f5310 2024-08-02T17:50:23.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.220+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561b197f7e20 con 0x561b197eaa70 2024-08-02T17:50:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.221+0000 7f3ec5d48640 1 -- 172.21.15.69:0/2038202577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 msgr2=0x561b197f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.221+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.221+0000 7f3ec5d48640 1 -- 172.21.15.69:0/2038202577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 msgr2=0x561b197f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.221+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 0x561b197f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.221+0000 7f3ec5d48640 1 -- 172.21.15.69:0/2038202577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b19525620 con 0x561b197f5310 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.221+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f2d30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0065090 tx=0x7f3ec0067800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6259956845b412 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.221+0000 7f3eca5db740 10 client.0 _ll_register_callbacks cb 0x561b19458a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.222+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eb00b8200 con 0x561b197f5310 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.222+0000 7f3ea8ff9640 10 client.26078 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.222+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2038202577 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00890d0 con 0x561b197f5310 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.222+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2038202577 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3ec0089270 con 0x561b197f5310 2024-08-02T17:50:23.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.222+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2038202577 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ec0098670 con 0x561b197f5310 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.223+0000 7f3eca5db740 10 client.26078 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.223+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x561b198037c0 con 0x561b197f5310 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.225+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2038202577 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec0098250 con 0x561b197f5310 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.225+0000 7f3eca5db740 10 client.26078 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.225+0000 7f3eca5db740 10 client.26078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.225+0000 7f3eca5db740 20 client.26078 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.225+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561b19804b70 0x561b19807000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.226+0000 7f3eca5db740 4 client.26078 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:23.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.226+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3eb00b8200 con 0x561b19804b70 2024-08-02T17:50:23.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.228+0000 7f3ec5547640 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561b19804b70 0x561b19807000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.230+0000 7f3ec5547640 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561b19804b70 0x561b19807000 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da31ba038492ffd0 server_cookie=be1b84e088f65c3c in_seq=0 out_seq=0 2024-08-02T17:50:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.230+0000 7f3ea8ff9640 10 client.26078 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.234+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2038202577 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f3eb00b8200 con 0x561b19804b70 2024-08-02T17:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.234+0000 7f3ea8ff9640 10 client.26078 handle_command_reply: tid=0 2024-08-02T17:50:23.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.235+0000 7f3eca5db740 1 client.26078 shutdown 2024-08-02T17:50:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.237+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 msgr2=0x561b197f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.237+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f2d30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0065090 tx=0x7f3ec0067800 comp rx=0 tx=0).stop 2024-08-02T17:50:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.238+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 shutdown_connections 2024-08-02T17:50:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.238+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561b19804b70 0x561b19807000 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.238+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197f2d30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.238+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.238+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2038202577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 0x561b197f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.238+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 >> 172.21.15.69:0/2038202577 conn(0x561b197e66c0 msgr2=0x561b197ec3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:23.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.241+0000 7f3ea8ff9640 0 client.26078 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:23.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.241+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 shutdown_connections 2024-08-02T17:50:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.242+0000 7f3eca5db740 1 -- 172.21.15.69:0/2038202577 wait complete. 2024-08-02T17:50:23.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.243+0000 7f3eca5db740 20 client.26078 trim_cache size 0 max 16384 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.274+0000 7f3eca5db740 1 Processor -- start 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3eca5db740 1 -- start start 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 0x561b197ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b1980c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3eca5db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8143160 con 0x561b197ea150 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3eca5db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8140a30 con 0x561b197f5310 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3eca5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb813e8c0 con 0x561b197eaa70 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3ec5d48640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b1980c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3ec5d48640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b1980c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59954/0 (socket says 172.21.15.69:59954) 2024-08-02T17:50:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.275+0000 7f3ec5d48640 1 -- 172.21.15.69:0/522151117 learned_addr learned my addr 172.21.15.69:0/522151117 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.276+0000 7f3ec5547640 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 0x561b197ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.276+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.277+0000 7f3ec5d48640 1 -- 172.21.15.69:0/522151117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 msgr2=0x561b197f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.277+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.277+0000 7f3ec5d48640 1 -- 172.21.15.69:0/522151117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 msgr2=0x561b197ed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.277+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 0x561b197ed590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.277+0000 7f3ec5d48640 1 -- 172.21.15.69:0/522151117 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b197e9ed0 con 0x561b197f5310 2024-08-02T17:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.277+0000 7f3ec5547640 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 0x561b197ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.277+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.277+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b1980c910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00514c0 tx=0x7f3ec008b070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca697d3a7d9cebc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.279+0000 7f3eaaffd640 1 -- 172.21.15.69:0/522151117 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00890d0 con 0x561b197f5310 2024-08-02T17:50:23.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.279+0000 7f3eaaffd640 1 -- 172.21.15.69:0/522151117 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec0089270 con 0x561b197f5310 2024-08-02T17:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.280+0000 7f3eaaffd640 1 -- 172.21.15.69:0/522151117 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec0087810 con 0x561b197f5310 2024-08-02T17:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.280+0000 7f3eca5db740 1 -- 172.21.15.69:0/522151117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 msgr2=0x561b1980c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.280+0000 7f3eca5db740 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b1980c910 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00514c0 tx=0x7f3ec008b070 comp rx=0 tx=0).stop 2024-08-02T17:50:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.284+0000 7f3eca5db740 1 -- 172.21.15.69:0/522151117 shutdown_connections 2024-08-02T17:50:23.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.284+0000 7f3eca5db740 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b1980c910 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.284+0000 7f3eca5db740 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197eaa70 0x561b197f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.284+0000 7f3eca5db740 1 --2- 172.21.15.69:0/522151117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197ea150 0x561b197ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.284+0000 7f3eca5db740 1 -- 172.21.15.69:0/522151117 >> 172.21.15.69:0/522151117 conn(0x561b197e66c0 msgr2=0x561b197f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.284+0000 7f3eca5db740 1 -- 172.21.15.69:0/522151117 shutdown_connections 2024-08-02T17:50:23.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.285+0000 7f3eca5db740 1 -- 172.21.15.69:0/522151117 wait complete. 2024-08-02T17:50:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.285+0000 7f3eca5db740 1 -- start start 2024-08-02T17:50:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.286+0000 7f3eca5db740 1 Processor -- start 2024-08-02T17:50:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197ea150 0x561b197fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197fa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.291 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3eca5db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197faac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3eca5db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb813ef10 con 0x561b197eaa70 2024-08-02T17:50:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3eca5db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb8140360 con 0x561b197f5310 2024-08-02T17:50:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3eca5db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb813f5c0 con 0x561b197ea150 2024-08-02T17:50:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3eca5db740 10 client.0 _ll_register_callbacks cb 0x7f3ec0068510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3ec5d48640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197faac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3ec5d48640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197faac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59962/0 (socket says 172.21.15.69:59962) 2024-08-02T17:50:23.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.290+0000 7f3ec5d48640 1 -- 172.21.15.69:0/2517736251 learned_addr learned my addr 172.21.15.69:0/2517736251 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.293+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.292+0000 7f3ec5d48640 1 -- 172.21.15.69:0/2517736251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197ea150 msgr2=0x561b197fa040 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:50:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.293+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197ea150 0x561b197fa040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.293+0000 7f3ec5d48640 1 -- 172.21.15.69:0/2517736251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 msgr2=0x561b197fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.293+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197fa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.293+0000 7f3ec5d48640 1 -- 172.21.15.69:0/2517736251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b197e4710 con 0x561b197f5310 2024-08-02T17:50:23.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.296+0000 7f3ec4d46640 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.299+0000 7f3ec5d48640 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197faac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0000b60 tx=0x7f3ec0089420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=963368563942d21e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:23.300 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.299+0000 7f3ea8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.303+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2517736251 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec00890d0 con 0x561b197f5310 2024-08-02T17:50:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.303+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2517736251 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f3ec0086810 con 0x561b197f5310 2024-08-02T17:50:23.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.304+0000 7f3eca5db740 1 -- 172.21.15.69:0/2517736251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec0089270 con 0x561b197f5310 2024-08-02T17:50:23.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.308+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2517736251 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ec006be30 con 0x561b197f5310 2024-08-02T17:50:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.317+0000 7f3eca5db740 10 client.26084 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:23.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.317+0000 7f3eca5db740 1 -- 172.21.15.69:0/2517736251 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b197e5960 con 0x561b197f5310 2024-08-02T17:50:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.319+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2517736251 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ec0089270 con 0x561b197f5310 2024-08-02T17:50:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.319+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2517736251 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec00876c0 con 0x561b197f5310 2024-08-02T17:50:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.320+0000 7f3eca5db740 10 client.26084 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.320+0000 7f3eca5db740 10 client.26084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.320+0000 7f3eca5db740 20 client.26084 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.320+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561b198788b0 0x561b1987aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.320+0000 7f3eca5db740 4 client.26084 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:50:23.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.320+0000 7f3eca5db740 1 -- 172.21.15.69:0/2517736251 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b197530e0 con 0x561b198788b0 2024-08-02T17:50:23.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.327+0000 7f3ec5547640 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561b198788b0 0x561b1987aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.333+0000 7f3ec5547640 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561b198788b0 0x561b1987aca0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e164c75bb93f3a1 server_cookie=df16a4f292f77c91 in_seq=0 out_seq=0 2024-08-02T17:50:23.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.333+0000 7f3ea8ff9640 10 client.26084 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:23.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:23 smithi077 ceph-mon[37108]: pgmap v122: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.4 MiB/s wr, 662 op/s 2024-08-02T17:50:23.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:23 smithi069 ceph-mon[32746]: pgmap v122: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.4 MiB/s wr, 662 op/s 2024-08-02T17:50:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.365+0000 7f3ea8ff9640 1 -- 172.21.15.69:0/2517736251 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b197530e0 con 0x561b198788b0 2024-08-02T17:50:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.365+0000 7f3ea8ff9640 10 client.26084 handle_command_reply: tid=0 2024-08-02T17:50:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 client.26084 shutdown 2024-08-02T17:50:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 -- 172.21.15.69:0/2517736251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 msgr2=0x561b197faac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197faac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0000b60 tx=0x7f3ec0089420 comp rx=0 tx=0).stop 2024-08-02T17:50:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 -- 172.21.15.69:0/2517736251 shutdown_connections 2024-08-02T17:50:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b197eaa70 0x561b197fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561b198788b0 0x561b1987aca0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b197f5310 0x561b197faac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 --2- 172.21.15.69:0/2517736251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b197ea150 0x561b197fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3eca5db740 1 -- 172.21.15.69:0/2517736251 >> 172.21.15.69:0/2517736251 conn(0x561b197e66c0 msgr2=0x561b197f38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.366+0000 7f3ea8ff9640 0 client.26084 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3eca5db740 1 -- 172.21.15.69:0/2517736251 shutdown_connections 2024-08-02T17:50:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3eca5db740 1 -- 172.21.15.69:0/2517736251 wait complete. 2024-08-02T17:50:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3eca5db740 20 client.26084 trim_cache size 0 max 16384 2024-08-02T17:50:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:50:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:50:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:50:23.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:50:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 -- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3eac0a2ed0 msgr2=0x7f3eac0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3eac0a2ed0 0x7f3eac0a5390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0067970 tx=0x7f3ec0068820 comp rx=0 tx=0).stop 2024-08-02T17:50:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 -- 172.21.15.69:0/1450707792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb8006a70 msgr2=0x7f3eb812e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb8006a70 0x7f3eb812e580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3eb0005940 tx=0x7f3eb0031540 comp rx=0 tx=0).stop 2024-08-02T17:50:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 -- 172.21.15.69:0/1450707792 shutdown_connections 2024-08-02T17:50:23.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3eac0a2ed0 0x7f3eac0a5390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3eb809b6c0 0x7f3eb81328f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3eb8006a70 0x7f3eb812e580 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 --2- 172.21.15.69:0/1450707792 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3eb80060f0 0x7f3eb812e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.367+0000 7f3ec6549640 1 -- 172.21.15.69:0/1450707792 >> 172.21.15.69:0/1450707792 conn(0x7f3eb800a290 msgr2=0x7f3eb8092aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.368+0000 7f3ec6549640 1 -- 172.21.15.69:0/1450707792 shutdown_connections 2024-08-02T17:50:23.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:23.368+0000 7f3ec6549640 1 -- 172.21.15.69:0/1450707792 wait complete. 2024-08-02T17:50:23.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:23 smithi083 ceph-mon[37031]: pgmap v122: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 8.4 MiB/s wr, 662 op/s 2024-08-02T17:50:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 195.0 s, 1737.8 tps, lat 57.818 ms stddev 74.782 2024-08-02T17:50:25.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:25 smithi077 ceph-mon[37108]: pgmap v123: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-08-02T17:50:25.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:25 smithi069 ceph-mon[32746]: pgmap v123: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-08-02T17:50:25.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:25 smithi083 ceph-mon[37031]: pgmap v123: 129 pgs: 129 active+clean; 2.0 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-08-02T17:50:26.741 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:50:26.741 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:50:27.173 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:27.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:27 smithi077 ceph-mon[37108]: pgmap v124: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-08-02T17:50:27.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:27 smithi069 ceph-mon[32746]: pgmap v124: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-08-02T17:50:27.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:27 smithi083 ceph-mon[37031]: pgmap v124: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 3.8 MiB/s wr, 441 op/s 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.719+0000 7f44c1bec640 1 -- 172.21.15.69:0/2972427463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc06e760 msgr2=0x7f44bc06eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.719+0000 7f44c1bec640 1 --2- 172.21.15.69:0/2972427463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc06e760 0x7f44bc06eba0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009860 tx=0x7f44b002a360 comp rx=0 tx=0).stop 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.720+0000 7f44c1bec640 1 -- 172.21.15.69:0/2972427463 shutdown_connections 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.720+0000 7f44c1bec640 1 --2- 172.21.15.69:0/2972427463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44bc11ae70 0x7f44bc11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.720+0000 7f44c1bec640 1 --2- 172.21.15.69:0/2972427463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc06e760 0x7f44bc06eba0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.720+0000 7f44c1bec640 1 --2- 172.21.15.69:0/2972427463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc10fec0 0x7f44bc06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.720+0000 7f44c1bec640 1 -- 172.21.15.69:0/2972427463 >> 172.21.15.69:0/2972427463 conn(0x7f44bc072eb0 msgr2=0x7f44bc0732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.721+0000 7f44c1bec640 1 -- 172.21.15.69:0/2972427463 shutdown_connections 2024-08-02T17:50:27.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.721+0000 7f44c1bec640 1 -- 172.21.15.69:0/2972427463 wait complete. 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.722+0000 7f44c1bec640 1 Processor -- start 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.722+0000 7f44c1bec640 1 -- start start 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c1bec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc06e760 0x7f44bc1a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c1bec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44bc10fec0 0x7f44bc1a73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c1bec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc11ae70 0x7f44bc1ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c1bec640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc120900 con 0x7f44bc10fec0 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c1bec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc120780 con 0x7f44bc11ae70 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c1bec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc120a80 con 0x7f44bc06e760 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c13eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc11ae70 0x7f44bc1ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c13eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc11ae70 0x7f44bc1ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42124/0 (socket says 172.21.15.69:42124) 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c13eb640 1 -- 172.21.15.69:0/3323017013 learned_addr learned my addr 172.21.15.69:0/3323017013 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c0bea640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc06e760 0x7f44bc1a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.723+0000 7f44c13eb640 1 -- 172.21.15.69:0/3323017013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc06e760 msgr2=0x7f44bc1a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44c13eb640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc06e760 0x7f44bc1a6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44c13eb640 1 -- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44bc10fec0 msgr2=0x7f44bc1a73b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44c13eb640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44bc10fec0 0x7f44bc1a73b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44c13eb640 1 -- 172.21.15.69:0/3323017013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc1abe40 con 0x7f44bc11ae70 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44c13eb640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc11ae70 0x7f44bc1ab740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44b400ca00 tx=0x7f44b400ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44b9ffb640 1 -- 172.21.15.69:0/3323017013 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4004d20 con 0x7f44bc11ae70 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44b9ffb640 1 -- 172.21.15.69:0/3323017013 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b4005020 con 0x7f44bc11ae70 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44b9ffb640 1 -- 172.21.15.69:0/3323017013 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b400f690 con 0x7f44bc11ae70 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.724+0000 7f44c1bec640 1 -- 172.21.15.69:0/3323017013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc1ac130 con 0x7f44bc11ae70 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.725+0000 7f44c1bec640 1 -- 172.21.15.69:0/3323017013 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc110650 con 0x7f44bc11ae70 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.731+0000 7f44c524c740 1 Processor -- start 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.732+0000 7f44c524c740 1 -- start start 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.732+0000 7f44c524c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0210a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.732+0000 7f44c524c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf02115070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.732+0000 7f44c524c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf021179a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.732+0000 7f44c524c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cf020cde40 con 0x55cf0210a3d0 2024-08-02T17:50:27.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.732+0000 7f44c524c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf0206f7f0 con 0x55cf021155b0 2024-08-02T17:50:27.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.732+0000 7f44c524c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf02016fb0 con 0x55cf0210acf0 2024-08-02T17:50:27.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.741+0000 7f44c0bea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf02115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.741+0000 7f44c0bea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf02115070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:32874/0 (socket says 172.21.15.69:32874) 2024-08-02T17:50:27.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.741+0000 7f44c0bea640 1 -- 172.21.15.69:0/2702499826 learned_addr learned my addr 172.21.15.69:0/2702499826 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:27.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.742+0000 7f44bbfff640 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf021179a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.742+0000 7f44bbfff640 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0210a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.742+0000 7f44c0bea640 1 -- 172.21.15.69:0/2702499826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 msgr2=0x55cf021179a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:27.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.742+0000 7f44c0bea640 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf021179a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.742+0000 7f44c0bea640 1 -- 172.21.15.69:0/2702499826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 msgr2=0x55cf0210a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:27.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.742+0000 7f44c0bea640 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0210a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.742+0000 7f44c0bea640 1 -- 172.21.15.69:0/2702499826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf01de16a0 con 0x55cf0210acf0 2024-08-02T17:50:27.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.743+0000 7f44c0bea640 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf02115070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009cd0 tx=0x7f44ac011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73b8d617093926b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:27.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.744+0000 7f44b9ffb640 1 -- 172.21.15.69:0/3323017013 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f44b400ab00 con 0x7f44bc11ae70 2024-08-02T17:50:27.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.744+0000 7f44a2ffd640 1 -- 172.21.15.69:0/2702499826 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac01e070 con 0x55cf0210acf0 2024-08-02T17:50:27.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.744+0000 7f44a2ffd640 1 -- 172.21.15.69:0/2702499826 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44ac019b10 con 0x55cf0210acf0 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.744+0000 7f44a2ffd640 1 -- 172.21.15.69:0/2702499826 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00eae0 con 0x55cf0210acf0 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44b9ffb640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f44a40a2e40 0x7f44a40a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44c524c740 1 -- 172.21.15.69:0/2702499826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 msgr2=0x55cf02115070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44c524c740 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf02115070 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009cd0 tx=0x7f44ac011810 comp rx=0 tx=0).stop 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44c524c740 1 -- 172.21.15.69:0/2702499826 shutdown_connections 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44c524c740 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf021179a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44c524c740 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf02115070 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44c524c740 1 --2- 172.21.15.69:0/2702499826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0210a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44c524c740 1 -- 172.21.15.69:0/2702499826 >> 172.21.15.69:0/2702499826 conn(0x55cf02106960 msgr2=0x55cf02106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:27.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.745+0000 7f44b9ffb640 1 -- 172.21.15.69:0/3323017013 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f44b40cfa60 con 0x7f44bc11ae70 2024-08-02T17:50:27.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.747+0000 7f44bbfff640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f44a40a2e40 0x7f44a40a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.747+0000 7f44c524c740 1 -- 172.21.15.69:0/2702499826 shutdown_connections 2024-08-02T17:50:27.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.747+0000 7f44c524c740 1 -- 172.21.15.69:0/2702499826 wait complete. 2024-08-02T17:50:27.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 1 -- start start 2024-08-02T17:50:27.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 1 Processor -- start 2024-08-02T17:50:27.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210a3d0 0x55cf02112390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210acf0 0x55cf021128d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf02112e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cf0202acb0 con 0x55cf0210acf0 2024-08-02T17:50:27.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cf0206b7b0 con 0x55cf021155b0 2024-08-02T17:50:27.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cf02118220 con 0x55cf0210a3d0 2024-08-02T17:50:27.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.748+0000 7f44c524c740 10 client.0 _ll_register_callbacks cb 0x55cf01d7eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:27.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.749+0000 7f44c13eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf02112e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.749+0000 7f44c13eb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf02112e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42128/0 (socket says 172.21.15.69:42128) 2024-08-02T17:50:27.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.749+0000 7f44c13eb640 1 -- 172.21.15.69:0/2674778059 learned_addr learned my addr 172.21.15.69:0/2674778059 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:27.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.749+0000 7f44c0bea640 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210a3d0 0x55cf02112390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.750+0000 7f44c13eb640 1 -- 172.21.15.69:0/2674778059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210a3d0 msgr2=0x55cf02112390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:27.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.750+0000 7f44c13eb640 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210a3d0 0x55cf02112390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.750+0000 7f44c13eb640 1 -- 172.21.15.69:0/2674778059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210acf0 msgr2=0x55cf021128d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:27.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.750+0000 7f44c13eb640 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210acf0 0x55cf021128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.750+0000 7f44c13eb640 1 -- 172.21.15.69:0/2674778059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf02029bc0 con 0x55cf021155b0 2024-08-02T17:50:27.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.750+0000 7f44bbfff640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f44a40a2e40 0x7f44a40a5300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009830 tx=0x7f44b0005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:27.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.750+0000 7f44c13eb640 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf02112e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f44b4097fc0 tx=0x7f44b409c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991bbd2673d1c939 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:27.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.752+0000 7f449bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:27.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.752+0000 7f449bfff640 1 -- 172.21.15.69:0/2674778059 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40b3060 con 0x55cf021155b0 2024-08-02T17:50:27.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.752+0000 7f449bfff640 1 -- 172.21.15.69:0/2674778059 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f44b40051d0 con 0x55cf021155b0 2024-08-02T17:50:27.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.752+0000 7f44c524c740 1 -- 172.21.15.69:0/2674778059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44ac019b10 con 0x55cf021155b0 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.752+0000 7f449bfff640 1 -- 172.21.15.69:0/2674778059 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44b401d050 con 0x55cf021155b0 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c524c740 10 client.26096 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c524c740 1 -- 172.21.15.69:0/2674778059 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cf02124060 con 0x55cf021155b0 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f449bfff640 1 -- 172.21.15.69:0/2674778059 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44ac019b10 con 0x55cf021155b0 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f449bfff640 1 -- 172.21.15.69:0/2674778059 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f44b40cd010 con 0x55cf021155b0 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c524c740 10 client.26096 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c524c740 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c524c740 20 client.26096 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c524c740 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cf021258a0 0x55cf02127cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c524c740 4 client.26096 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c524c740 1 -- 172.21.15.69:0/2674778059 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf0207cd60 con 0x55cf021258a0 2024-08-02T17:50:27.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.753+0000 7f44c0bea640 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cf021258a0 0x55cf02127cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.754+0000 7f44c0bea640 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cf021258a0 0x55cf02127cf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ec7f5ca7896bea server_cookie=e6e1918caab2b713 in_seq=0 out_seq=0 2024-08-02T17:50:27.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.754+0000 7f449bfff640 10 client.26096 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:27.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f449bfff640 1 -- 172.21.15.69:0/2674778059 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cf0207cd60 con 0x55cf021258a0 2024-08-02T17:50:27.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f449bfff640 10 client.26096 handle_command_reply: tid=0 2024-08-02T17:50:27.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 client.26096 shutdown 2024-08-02T17:50:27.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 -- 172.21.15.69:0/2674778059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 msgr2=0x55cf02112e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:27.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf02112e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f44b4097fc0 tx=0x7f44b409c260 comp rx=0 tx=0).stop 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 -- 172.21.15.69:0/2674778059 shutdown_connections 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210acf0 0x55cf021128d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cf021258a0 0x55cf02127cf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf02112e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 --2- 172.21.15.69:0/2674778059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210a3d0 0x55cf02112390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 -- 172.21.15.69:0/2674778059 >> 172.21.15.69:0/2674778059 conn(0x55cf02106960 msgr2=0x55cf0210c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f449bfff640 0 client.26096 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.755+0000 7f44c524c740 1 -- 172.21.15.69:0/2674778059 shutdown_connections 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.756+0000 7f44c524c740 1 -- 172.21.15.69:0/2674778059 wait complete. 2024-08-02T17:50:27.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.756+0000 7f44c524c740 20 client.26096 trim_cache size 0 max 16384 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.786+0000 7f44c524c740 1 Processor -- start 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.786+0000 7f44c524c740 1 -- start start 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.786+0000 7f44c524c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf02112c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.786+0000 7f44c524c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf0212d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.786+0000 7f44c524c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf0212db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.786+0000 7f44c524c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1c8cc0 con 0x55cf0210a3d0 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.786+0000 7f44c0bea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf02112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44c0bea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf02112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52638/0 (socket says 172.21.15.69:52638) 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44c524c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1c6590 con 0x55cf021155b0 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44c524c740 1 -- 172.21.15.69:0/1105626324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc118840 con 0x55cf0210acf0 2024-08-02T17:50:27.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44c0bea640 1 -- 172.21.15.69:0/1105626324 learned_addr learned my addr 172.21.15.69:0/1105626324 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44bbfff640 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf0212d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44c13eb640 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf0212db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44bbfff640 1 -- 172.21.15.69:0/1105626324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 msgr2=0x55cf0212db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44bbfff640 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf0212db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44bbfff640 1 -- 172.21.15.69:0/1105626324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 msgr2=0x55cf02112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44bbfff640 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf02112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44bbfff640 1 -- 172.21.15.69:0/1105626324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf021108e0 con 0x55cf0210acf0 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.787+0000 7f44bbfff640 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf0212d600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44b0035040 tx=0x7f44b0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9edc2ba915fec9fe server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:27.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.788+0000 7f44a27fc640 1 -- 172.21.15.69:0/1105626324 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0050070 con 0x55cf0210acf0 2024-08-02T17:50:27.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.788+0000 7f44a27fc640 1 -- 172.21.15.69:0/1105626324 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b0004230 con 0x55cf0210acf0 2024-08-02T17:50:27.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.788+0000 7f44a27fc640 1 -- 172.21.15.69:0/1105626324 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0037040 con 0x55cf0210acf0 2024-08-02T17:50:27.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 -- 172.21.15.69:0/1105626324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 msgr2=0x55cf0212d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:27.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf0212d600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f44b0035040 tx=0x7f44b0000bc0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 -- 172.21.15.69:0/1105626324 shutdown_connections 2024-08-02T17:50:27.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf021155b0 0x55cf0212db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf0210acf0 0x55cf0212d600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 --2- 172.21.15.69:0/1105626324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf02112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 -- 172.21.15.69:0/1105626324 >> 172.21.15.69:0/1105626324 conn(0x55cf02106960 msgr2=0x55cf0210c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 -- 172.21.15.69:0/1105626324 shutdown_connections 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.789+0000 7f44c524c740 1 -- 172.21.15.69:0/1105626324 wait complete. 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.790+0000 7f44c524c740 1 -- start start 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.790+0000 7f44c524c740 1 Processor -- start 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c524c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0211d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c524c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf0210acf0 0x55cf02111cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c524c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf021155b0 0x55cf021121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c0bea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0211d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c0bea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0211d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52648/0 (socket says 172.21.15.69:52648) 2024-08-02T17:50:27.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c0bea640 1 -- 172.21.15.69:0/2614082406 learned_addr learned my addr 172.21.15.69:0/2614082406 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:27.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c13eb640 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf021155b0 0x55cf021121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44bbfff640 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf0210acf0 0x55cf02111cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1c4a50 con 0x55cf0210a3d0 2024-08-02T17:50:27.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1c5ec0 con 0x55cf0210acf0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.791+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc1c5120 con 0x55cf021155b0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c13eb640 1 -- 172.21.15.69:0/2614082406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf0210acf0 msgr2=0x55cf02111cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c13eb640 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf0210acf0 0x55cf02111cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c13eb640 1 -- 172.21.15.69:0/2614082406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 msgr2=0x55cf0211d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c13eb640 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0211d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c13eb640 1 -- 172.21.15.69:0/2614082406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf02112730 con 0x55cf021155b0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c524c740 10 client.0 _ll_register_callbacks cb 0x7f44b400cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c13eb640 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf021155b0 0x55cf021121f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f44b4007330 tx=0x7f44b4007500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2543d12f79a66fc1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f449bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b0004230 con 0x55cf021155b0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f449bfff640 1 -- 172.21.15.69:0/2614082406 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40b3030 con 0x55cf021155b0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f449bfff640 1 -- 172.21.15.69:0/2614082406 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f44b40a4340 con 0x55cf021155b0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f449bfff640 1 -- 172.21.15.69:0/2614082406 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44b409b6a0 con 0x55cf021155b0 2024-08-02T17:50:27.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c524c740 10 client.25984 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.792+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55cf0211c0b0 con 0x55cf021155b0 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.793+0000 7f449bfff640 1 -- 172.21.15.69:0/2614082406 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f44b409bdc0 con 0x55cf021155b0 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.793+0000 7f44c524c740 10 client.25984 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.793+0000 7f44c524c740 10 client.25984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.793+0000 7f44c524c740 20 client.25984 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.793+0000 7f44c524c740 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cf02190800 0x55cf02192c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.793+0000 7f44c524c740 4 client.25984 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.793+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f44b0004230 con 0x55cf02190800 2024-08-02T17:50:27.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.793+0000 7f44c0bea640 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cf02190800 0x55cf02192c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:27.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.794+0000 7f44c0bea640 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cf02190800 0x55cf02192c40 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e42a44a59580aea server_cookie=80d4f831389320db in_seq=0 out_seq=0 2024-08-02T17:50:27.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.794+0000 7f449bfff640 10 client.25984 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:27.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.795+0000 7f449bfff640 1 -- 172.21.15.69:0/2614082406 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f44b0004230 con 0x55cf02190800 2024-08-02T17:50:27.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.795+0000 7f449bfff640 10 client.25984 handle_command_reply: tid=0 2024-08-02T17:50:27.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.795+0000 7f44c524c740 1 client.25984 shutdown 2024-08-02T17:50:27.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.795+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf021155b0 msgr2=0x55cf021121f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.795+0000 7f44c524c740 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf021155b0 0x55cf021121f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f44b4007330 tx=0x7f44b4007500 comp rx=0 tx=0).stop 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.795+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 shutdown_connections 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.795+0000 7f44c524c740 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cf02190800 0x55cf02192c40 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.796+0000 7f449bfff640 0 client.25984 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.796+0000 7f44c524c740 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cf021155b0 0x55cf021121f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.796+0000 7f44c524c740 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cf0210acf0 0x55cf02111cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.796+0000 7f44c524c740 1 --2- 172.21.15.69:0/2614082406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cf0210a3d0 0x55cf0211d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.796+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 >> 172.21.15.69:0/2614082406 conn(0x55cf02106960 msgr2=0x55cf02105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.796+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 shutdown_connections 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.796+0000 7f44c524c740 1 -- 172.21.15.69:0/2614082406 wait complete. 2024-08-02T17:50:27.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.796+0000 7f44c524c740 20 client.25984 trim_cache size 0 max 16384 2024-08-02T17:50:27.804 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:50:27.804 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:50:27.804 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "8e8977fc-2990-4fdc-a326-2e5b34c45574", 2024-08-02T17:50:27.804 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:50:27.804 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:50:27.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.797+0000 7f44c1bec640 1 -- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f44a40a2e40 msgr2=0x7f44a40a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:27.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.797+0000 7f44c1bec640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f44a40a2e40 0x7f44a40a5300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009830 tx=0x7f44b0005f90 comp rx=0 tx=0).stop 2024-08-02T17:50:27.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 -- 172.21.15.69:0/3323017013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc11ae70 msgr2=0x7f44bc1ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:27.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc11ae70 0x7f44bc1ab740 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44b400ca00 tx=0x7f44b400ce00 comp rx=0 tx=0).stop 2024-08-02T17:50:27.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 -- 172.21.15.69:0/3323017013 shutdown_connections 2024-08-02T17:50:27.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f44bc10fec0 0x7f44bc1a73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f44a40a2e40 0x7f44a40a5300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f44bc11ae70 0x7f44bc1ab740 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 --2- 172.21.15.69:0/3323017013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f44bc06e760 0x7f44bc1a6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:27.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 -- 172.21.15.69:0/3323017013 >> 172.21.15.69:0/3323017013 conn(0x7f44bc072eb0 msgr2=0x7f44bc1190e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:27.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 -- 172.21.15.69:0/3323017013 shutdown_connections 2024-08-02T17:50:27.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:27.798+0000 7f44c1bec640 1 -- 172.21.15.69:0/3323017013 wait complete. 2024-08-02T17:50:28.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:28 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:50:28.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:28 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:28.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:28 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:50:28.604 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:28 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:28.605 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:50:28.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:28 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:50:28.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:28 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:28.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:28 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:50:28.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:28 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:28.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:28 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:50:28.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:28 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:28.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:28 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:50:28.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:28 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:29.013 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:29.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 -- 172.21.15.69:0/973303002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10aac0 msgr2=0x7fa20c116f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 --2- 172.21.15.69:0/973303002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10aac0 0x7fa20c116f50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa204009ca0 tx=0x7fa20402a950 comp rx=0 tx=0).stop 2024-08-02T17:50:29.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 -- 172.21.15.69:0/973303002 shutdown_connections 2024-08-02T17:50:29.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 --2- 172.21.15.69:0/973303002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10aac0 0x7fa20c116f50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 --2- 172.21.15.69:0/973303002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa20c10a140 0x7fa20c10a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 --2- 172.21.15.69:0/973303002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1101a0 0x7fa20c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 -- 172.21.15.69:0/973303002 >> 172.21.15.69:0/973303002 conn(0x7fa20c071360 msgr2=0x7fa20c073780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:29.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 -- 172.21.15.69:0/973303002 shutdown_connections 2024-08-02T17:50:29.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.516+0000 7fa212eea640 1 -- 172.21.15.69:0/973303002 wait complete. 2024-08-02T17:50:29.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.517+0000 7fa212eea640 1 Processor -- start 2024-08-02T17:50:29.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.518+0000 7fa212eea640 1 -- start start 2024-08-02T17:50:29.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.518+0000 7fa212eea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10a140 0x7fa20c1a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa212eea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa20c10aac0 0x7fa20c1a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa212eea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1101a0 0x7fa20c1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa212eea640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c11a480 con 0x7fa20c10a140 2024-08-02T17:50:29.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa212eea640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c11a300 con 0x7fa20c10aac0 2024-08-02T17:50:29.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa212eea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c11a600 con 0x7fa20c1101a0 2024-08-02T17:50:29.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa210c86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10a140 0x7fa20c1a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa210c86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10a140 0x7fa20c1a65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52662/0 (socket says 172.21.15.69:52662) 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa210c86640 1 -- 172.21.15.69:0/1329625608 learned_addr learned my addr 172.21.15.69:0/1329625608 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa211487640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1101a0 0x7fa20c1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa20bfff640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa20c10aac0 0x7fa20c1a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa211487640 1 -- 172.21.15.69:0/1329625608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa20c10aac0 msgr2=0x7fa20c1a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa211487640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa20c10aac0 0x7fa20c1a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa211487640 1 -- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10a140 msgr2=0x7fa20c1a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa211487640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10a140 0x7fa20c1a65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.519+0000 7fa211487640 1 -- 172.21.15.69:0/1329625608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c1a0f00 con 0x7fa20c1101a0 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.520+0000 7fa20bfff640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa20c10aac0 0x7fa20c1a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.520+0000 7fa211487640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1101a0 0x7fa20c1a0690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa204005cc0 tx=0x7fa20402f420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.520+0000 7fa209ffb640 1 -- 172.21.15.69:0/1329625608 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204034040 con 0x7fa20c1101a0 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.520+0000 7fa209ffb640 1 -- 172.21.15.69:0/1329625608 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa204030920 con 0x7fa20c1101a0 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.520+0000 7fa209ffb640 1 -- 172.21.15.69:0/1329625608 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa204004610 con 0x7fa20c1101a0 2024-08-02T17:50:29.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.520+0000 7fa212eea640 1 -- 172.21.15.69:0/1329625608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c1a1190 con 0x7fa20c1101a0 2024-08-02T17:50:29.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.520+0000 7fa212eea640 1 -- 172.21.15.69:0/1329625608 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c1ad370 con 0x7fa20c1101a0 2024-08-02T17:50:29.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.524+0000 7fa2152e8740 1 Processor -- start 2024-08-02T17:50:29.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.525+0000 7fa209ffb640 1 -- 172.21.15.69:0/1329625608 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa204037020 con 0x7fa20c1101a0 2024-08-02T17:50:29.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.526+0000 7fa2152e8740 1 -- start start 2024-08-02T17:50:29.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace978830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978d70 0x557ace972da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace973890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa2152e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557ace8ce980 con 0x557ace973410 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa2152e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557ace866a70 con 0x557ace978d70 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa2152e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557ace8ca940 con 0x557ace978450 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa209ffb640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1f80a2d70 0x7fa1f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace978830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace978830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:32928/0 (socket says 172.21.15.69:32928) 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa20bfff640 1 -- 172.21.15.69:0/3573076424 learned_addr learned my addr 172.21.15.69:0/3573076424 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa20bfff640 1 --2- 172.21.15.69:0/3573076424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace973890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa209ffb640 1 -- 172.21.15.69:0/1329625608 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa2040f05c0 con 0x7fa20c1101a0 2024-08-02T17:50:29.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.527+0000 7fa211487640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1f80a2d70 0x7fa1f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa20bfff640 1 -- 172.21.15.69:0/3573076424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978d70 msgr2=0x557ace972da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa20bfff640 1 --2- 172.21.15.69:0/3573076424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978d70 0x557ace972da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa20bfff640 1 -- 172.21.15.69:0/3573076424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 msgr2=0x557ace973890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa20bfff640 1 --2- 172.21.15.69:0/3573076424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace973890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa20bfff640 1 -- 172.21.15.69:0/3573076424 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ace8c96e0 con 0x557ace978450 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa20bfff640 1 --2- 172.21.15.69:0/3573076424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace978830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00bf90 tx=0x7fa1fc00b570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898a135de9d50bcc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa202ffd640 1 -- 172.21.15.69:0/3573076424 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc004020 con 0x557ace978450 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa202ffd640 1 -- 172.21.15.69:0/3573076424 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc005500 con 0x557ace978450 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa202ffd640 1 -- 172.21.15.69:0/3573076424 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc004e00 con 0x557ace978450 2024-08-02T17:50:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.528+0000 7fa211487640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1f80a2d70 0x7fa1f80a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa204009ca0 tx=0x7fa2040a7040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:29.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 -- 172.21.15.69:0/3573076424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 msgr2=0x557ace978830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 --2- 172.21.15.69:0/3573076424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace978830 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00bf90 tx=0x7fa1fc00b570 comp rx=0 tx=0).stop 2024-08-02T17:50:29.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 -- 172.21.15.69:0/3573076424 shutdown_connections 2024-08-02T17:50:29.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 --2- 172.21.15.69:0/3573076424 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace973890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 --2- 172.21.15.69:0/3573076424 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978d70 0x557ace972da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 --2- 172.21.15.69:0/3573076424 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace978830 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 -- 172.21.15.69:0/3573076424 >> 172.21.15.69:0/3573076424 conn(0x557ace965250 msgr2=0x557ace965630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 -- 172.21.15.69:0/3573076424 shutdown_connections 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 -- 172.21.15.69:0/3573076424 wait complete. 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.529+0000 7fa2152e8740 1 -- start start 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa2152e8740 1 Processor -- start 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 0x557ace97c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace97c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978d70 0x557ace97cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa2152e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557ace97f790 con 0x557ace978d70 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa2152e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557ace889aa0 con 0x557ace973410 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa2152e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557ace97f910 con 0x557ace978450 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa2152e8740 10 client.0 _ll_register_callbacks cb 0x557ace878e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:29.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978d70 0x557ace97cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978d70 0x557ace97cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52670/0 (socket says 172.21.15.69:52670) 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa20bfff640 1 -- 172.21.15.69:0/681141419 learned_addr learned my addr 172.21.15.69:0/681141419 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa210c86640 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace97c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.530+0000 7fa20bfff640 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 0x557ace97c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa210c86640 1 -- 172.21.15.69:0/681141419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 msgr2=0x557ace97c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa210c86640 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 0x557ace97c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa210c86640 1 -- 172.21.15.69:0/681141419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978d70 msgr2=0x557ace97cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa210c86640 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978d70 0x557ace97cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa210c86640 1 -- 172.21.15.69:0/681141419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ace86d650 con 0x557ace978450 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa20bfff640 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978d70 0x557ace97cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa210c86640 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace97c7c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x557ace9666a0 tx=0x7fa1f4011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d917259398f9cd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa1ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa1ebfff640 1 -- 172.21.15.69:0/681141419 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f400f920 con 0x557ace978450 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa2152e8740 1 -- 172.21.15.69:0/681141419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc005500 con 0x557ace978450 2024-08-02T17:50:29.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa1ebfff640 1 -- 172.21.15.69:0/681141419 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa1f400fac0 con 0x557ace978450 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa1ebfff640 1 -- 172.21.15.69:0/681141419 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1f4004520 con 0x557ace978450 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa2152e8740 10 client.25996 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.531+0000 7fa2152e8740 1 -- 172.21.15.69:0/681141419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ace6403c0 con 0x557ace978450 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.532+0000 7fa1ebfff640 1 -- 172.21.15.69:0/681141419 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc005500 con 0x557ace978450 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa1ebfff640 1 -- 172.21.15.69:0/681141419 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1f4018070 con 0x557ace978450 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa2152e8740 10 client.25996 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa2152e8740 10 client.25996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa2152e8740 20 client.25996 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa2152e8740 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557ace98cbd0 0x557ace98eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa2152e8740 4 client.25996 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa2152e8740 1 -- 172.21.15.69:0/681141419 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ace8dbef0 con 0x557ace98cbd0 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa20bfff640 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557ace98cbd0 0x557ace98eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa20bfff640 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557ace98cbd0 0x557ace98eff0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cac8cf31601b7f4 server_cookie=8a29c317cee9c48e in_seq=0 out_seq=0 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.533+0000 7fa1ebfff640 10 client.25996 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.534+0000 7fa1ebfff640 1 -- 172.21.15.69:0/681141419 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557ace8dbef0 con 0x557ace98cbd0 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.534+0000 7fa1ebfff640 10 client.25996 handle_command_reply: tid=0 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.534+0000 7fa2152e8740 1 client.25996 shutdown 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.534+0000 7fa2152e8740 1 -- 172.21.15.69:0/681141419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 msgr2=0x557ace97c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.534+0000 7fa2152e8740 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace97c7c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x557ace9666a0 tx=0x7fa1f4011b50 comp rx=0 tx=0).stop 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 1 -- 172.21.15.69:0/681141419 shutdown_connections 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978d70 0x557ace97cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557ace98cbd0 0x557ace98eff0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978450 0x557ace97c7c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 1 --2- 172.21.15.69:0/681141419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 0x557ace97c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 1 -- 172.21.15.69:0/681141419 >> 172.21.15.69:0/681141419 conn(0x557ace965250 msgr2=0x557ace964900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa1ebfff640 0 client.25996 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 1 -- 172.21.15.69:0/681141419 shutdown_connections 2024-08-02T17:50:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 1 -- 172.21.15.69:0/681141419 wait complete. 2024-08-02T17:50:29.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.535+0000 7fa2152e8740 20 client.25996 trim_cache size 0 max 16384 2024-08-02T17:50:29.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.563+0000 7fa2152e8740 1 Processor -- start 2024-08-02T17:50:29.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.563+0000 7fa2152e8740 1 -- start start 2024-08-02T17:50:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.563+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace97e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.563+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978450 0x557ace972da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.563+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 0x557ace975880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.563+0000 7fa2152e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b7b40 con 0x557ace973410 2024-08-02T17:50:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.563+0000 7fa2152e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b5410 con 0x557ace978450 2024-08-02T17:50:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.563+0000 7fa2152e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b3260 con 0x557ace978d70 2024-08-02T17:50:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.564+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace97e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.564+0000 7fa20bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace97e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52676/0 (socket says 172.21.15.69:52676) 2024-08-02T17:50:29.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.564+0000 7fa20bfff640 1 -- 172.21.15.69:0/3930421639 learned_addr learned my addr 172.21.15.69:0/3930421639 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:29.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.564+0000 7fa20bfff640 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 0x557ace975880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.565+0000 7fa210c86640 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978450 0x557ace972da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.565+0000 7fa20bfff640 1 -- 172.21.15.69:0/3930421639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 msgr2=0x557ace975880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:29.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.565+0000 7fa20bfff640 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 0x557ace975880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.565+0000 7fa20bfff640 1 -- 172.21.15.69:0/3930421639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978450 msgr2=0x557ace972da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:29.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.565+0000 7fa20bfff640 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978450 0x557ace972da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.565+0000 7fa20bfff640 1 -- 172.21.15.69:0/3930421639 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ace97b230 con 0x557ace973410 2024-08-02T17:50:29.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.568+0000 7fa20bfff640 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace97e9e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc004b50 tx=0x7fa1fc0049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862018b5b1ee5a16 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:29.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.572+0000 7fa2027fc640 1 -- 172.21.15.69:0/3930421639 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00b9f0 con 0x557ace973410 2024-08-02T17:50:29.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.572+0000 7fa2027fc640 1 -- 172.21.15.69:0/3930421639 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc030040 con 0x557ace973410 2024-08-02T17:50:29.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.572+0000 7fa2027fc640 1 -- 172.21.15.69:0/3930421639 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00ee60 con 0x557ace973410 2024-08-02T17:50:29.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.573+0000 7fa2152e8740 1 -- 172.21.15.69:0/3930421639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 msgr2=0x557ace97e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.573+0000 7fa2152e8740 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace97e9e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc004b50 tx=0x7fa1fc0049e0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.575+0000 7fa2152e8740 1 -- 172.21.15.69:0/3930421639 shutdown_connections 2024-08-02T17:50:29.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.575+0000 7fa2152e8740 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 0x557ace975880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.575+0000 7fa2152e8740 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace978450 0x557ace972da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.575+0000 7fa2152e8740 1 --2- 172.21.15.69:0/3930421639 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace973410 0x557ace97e9e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.575+0000 7fa2152e8740 1 -- 172.21.15.69:0/3930421639 >> 172.21.15.69:0/3930421639 conn(0x557ace965250 msgr2=0x557ace9640e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.579+0000 7fa2152e8740 1 -- 172.21.15.69:0/3930421639 shutdown_connections 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.579+0000 7fa2152e8740 1 -- 172.21.15.69:0/3930421639 wait complete. 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.579+0000 7fa2152e8740 1 -- start start 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.579+0000 7fa2152e8740 1 Processor -- start 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.580+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 0x557ace985260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.580+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978450 0x557ace981ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.580+0000 7fa2152e8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 0x557ace9820e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.580+0000 7fa2152e8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b38d0 con 0x557ace978450 2024-08-02T17:50:29.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.580+0000 7fa2152e8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b4d40 con 0x557ace973410 2024-08-02T17:50:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.580+0000 7fa2152e8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b3fa0 con 0x557ace978d70 2024-08-02T17:50:29.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.581+0000 7fa2152e8740 10 client.0 _ll_register_callbacks cb 0x7fa1f4009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:29.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.585+0000 7fa210c86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978450 0x557ace981ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.585+0000 7fa210c86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978450 0x557ace981ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52678/0 (socket says 172.21.15.69:52678) 2024-08-02T17:50:29.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.585+0000 7fa210c86640 1 -- 172.21.15.69:0/1212997004 learned_addr learned my addr 172.21.15.69:0/1212997004 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:29.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.589+0000 7fa20bfff640 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 0x557ace9820e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.589+0000 7fa20bfff640 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 0x557ace985260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.594+0000 7fa210c86640 1 -- 172.21.15.69:0/1212997004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 msgr2=0x557ace9820e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:29.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.594+0000 7fa210c86640 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 0x557ace9820e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.594+0000 7fa210c86640 1 -- 172.21.15.69:0/1212997004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 msgr2=0x557ace985260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:29.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.594+0000 7fa210c86640 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 0x557ace985260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.594+0000 7fa210c86640 1 -- 172.21.15.69:0/1212997004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ace98f360 con 0x557ace978450 2024-08-02T17:50:29.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.595+0000 7fa210c86640 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978450 0x557ace981ba0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x557ace97bb10 tx=0x7fa1f401aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc9bcdbf7f6e474 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:29.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.596+0000 7fa1ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:50:29.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.596+0000 7fa1ebfff640 1 -- 172.21.15.69:0/1212997004 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f402c020 con 0x557ace978450 2024-08-02T17:50:29.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.596+0000 7fa1ebfff640 1 -- 172.21.15.69:0/1212997004 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa1f40035f0 con 0x557ace978450 2024-08-02T17:50:29.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.597+0000 7fa2152e8740 1 -- 172.21.15.69:0/1212997004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc030040 con 0x557ace978450 2024-08-02T17:50:29.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.598+0000 7fa1ebfff640 1 -- 172.21.15.69:0/1212997004 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa1f4009c90 con 0x557ace978450 2024-08-02T17:50:29.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.606+0000 7fa2152e8740 10 client.16509 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.606+0000 7fa2152e8740 1 -- 172.21.15.69:0/1212997004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557ace9722c0 con 0x557ace978450 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.607+0000 7fa1ebfff640 1 -- 172.21.15.69:0/1212997004 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1f4003d10 con 0x557ace978450 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.607+0000 7fa1ebfff640 1 -- 172.21.15.69:0/1212997004 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa1f4018050 con 0x557ace978450 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.608+0000 7fa2152e8740 10 client.16509 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.608+0000 7fa2152e8740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.609+0000 7fa2152e8740 20 client.16509 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.609+0000 7fa2152e8740 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557ace9729e0 0x557ace9f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.609+0000 7fa2152e8740 4 client.16509 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:50:29.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.609+0000 7fa2152e8740 1 -- 172.21.15.69:0/1212997004 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa1fc030040 con 0x557ace9729e0 2024-08-02T17:50:29.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.610+0000 7fa20bfff640 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557ace9729e0 0x557ace9f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:29.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.614+0000 7fa20bfff640 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557ace9729e0 0x557ace9f18a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7368ca5afbd8cdee server_cookie=ad2f1a7a2a2cd419 in_seq=0 out_seq=0 2024-08-02T17:50:29.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.614+0000 7fa1ebfff640 10 client.16509 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:29.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.615+0000 7fa1ebfff640 1 -- 172.21.15.69:0/1212997004 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fa1fc030040 con 0x557ace9729e0 2024-08-02T17:50:29.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.615+0000 7fa1ebfff640 10 client.16509 handle_command_reply: tid=0 2024-08-02T17:50:29.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.615+0000 7fa2152e8740 1 client.16509 shutdown 2024-08-02T17:50:29.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.617+0000 7fa2152e8740 1 -- 172.21.15.69:0/1212997004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978450 msgr2=0x557ace981ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.617+0000 7fa2152e8740 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978450 0x557ace981ba0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x557ace97bb10 tx=0x7fa1f401aa90 comp rx=0 tx=0).stop 2024-08-02T17:50:29.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.619+0000 7fa2152e8740 1 -- 172.21.15.69:0/1212997004 shutdown_connections 2024-08-02T17:50:29.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa2152e8740 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557ace978450 0x557ace981ba0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa2152e8740 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557ace9729e0 0x557ace9f18a0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa2152e8740 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557ace978d70 0x557ace9820e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa2152e8740 1 --2- 172.21.15.69:0/1212997004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557ace973410 0x557ace985260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa2152e8740 1 -- 172.21.15.69:0/1212997004 >> 172.21.15.69:0/1212997004 conn(0x557ace965250 msgr2=0x557ace9640e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:29.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa1ebfff640 0 client.16509 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:29.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa2152e8740 1 -- 172.21.15.69:0/1212997004 shutdown_connections 2024-08-02T17:50:29.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa2152e8740 1 -- 172.21.15.69:0/1212997004 wait complete. 2024-08-02T17:50:29.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.620+0000 7fa2152e8740 20 client.16509 trim_cache size 0 max 16384 2024-08-02T17:50:29.623 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:50:29.623 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:50:29.623 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:50:29.623 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:50:29.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 -- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1f80a2d70 msgr2=0x7fa1f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1f80a2d70 0x7fa1f80a5230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa204009ca0 tx=0x7fa2040a7040 comp rx=0 tx=0).stop 2024-08-02T17:50:29.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 -- 172.21.15.69:0/1329625608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1101a0 msgr2=0x7fa20c1a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:29.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1101a0 0x7fa20c1a0690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa204005cc0 tx=0x7fa20402f420 comp rx=0 tx=0).stop 2024-08-02T17:50:29.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 -- 172.21.15.69:0/1329625608 shutdown_connections 2024-08-02T17:50:29.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1f80a2d70 0x7fa1f80a5230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa20c1101a0 0x7fa20c1a0690 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa20c10aac0 0x7fa20c1a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 --2- 172.21.15.69:0/1329625608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa20c10a140 0x7fa20c1a65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:29.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 -- 172.21.15.69:0/1329625608 >> 172.21.15.69:0/1329625608 conn(0x7fa20c071360 msgr2=0x7fa20c072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:29.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 -- 172.21.15.69:0/1329625608 shutdown_connections 2024-08-02T17:50:29.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:29.624+0000 7fa212eea640 1 -- 172.21.15.69:0/1329625608 wait complete. 2024-08-02T17:50:29.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:29 smithi083 ceph-mon[37031]: pgmap v125: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-08-02T17:50:29.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:29 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:50:29.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:29 smithi077 ceph-mon[37108]: pgmap v125: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-08-02T17:50:29.851 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:29 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:50:29.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:29 smithi069 ceph-mon[32746]: pgmap v125: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 3.8 MiB/s wr, 436 op/s 2024-08-02T17:50:29.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:29 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:50:29.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 200.0 s, 1595.8 tps, lat 62.054 ms stddev 73.110 2024-08-02T17:50:31.278 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:31 smithi069 ceph-mon[32746]: pgmap v126: 129 pgs: 129 active+clean; 2.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.2 MiB/s wr, 503 op/s 2024-08-02T17:50:31.331 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:50:31.331 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:50:31.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:31 smithi077 ceph-mon[37108]: pgmap v126: 129 pgs: 129 active+clean; 2.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.2 MiB/s wr, 503 op/s 2024-08-02T17:50:31.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:31 smithi083 ceph-mon[37031]: pgmap v126: 129 pgs: 129 active+clean; 2.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.2 MiB/s wr, 503 op/s 2024-08-02T17:50:31.769 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:32.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.270+0000 7fbf77cb4640 1 -- 172.21.15.69:0/873859512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf701007b0 msgr2=0x7fbf70100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:32.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.270+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/873859512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf701007b0 0x7fbf70100c10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009930 tx=0x7fbf6402a420 comp rx=0 tx=0).stop 2024-08-02T17:50:32.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.270+0000 7fbf77cb4640 1 -- 172.21.15.69:0/873859512 shutdown_connections 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.271+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/873859512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf701007b0 0x7fbf70100c10 secure :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009930 tx=0x7fbf6402a420 comp rx=0 tx=0).stop 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.271+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/873859512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf70101150 0x7fbf7010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.271+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/873859512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf701067b0 0x7fbf70106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.271+0000 7fbf77cb4640 1 -- 172.21.15.69:0/873859512 >> 172.21.15.69:0/873859512 conn(0x7fbf700fc4d0 msgr2=0x7fbf700fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.271+0000 7fbf77cb4640 1 -- 172.21.15.69:0/873859512 shutdown_connections 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.271+0000 7fbf77cb4640 1 -- 172.21.15.69:0/873859512 wait complete. 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.272+0000 7fbf77cb4640 1 Processor -- start 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.272+0000 7fbf77cb4640 1 -- start start 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.273+0000 7fbf77cb4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf70101150 0x7fbf70198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.273+0000 7fbf77cb4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf701067b0 0x7fbf701988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.273+0000 7fbf77cb4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7019cc40 0x7fbf7019d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:32.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.273+0000 7fbf77cb4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70111cd0 con 0x7fbf7019cc40 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.273+0000 7fbf77cb4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70111b50 con 0x7fbf70101150 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.273+0000 7fbf77cb4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70111e50 con 0x7fbf701067b0 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf70101150 0x7fbf70198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf70101150 0x7fbf70198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42218/0 (socket says 172.21.15.69:42218) 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 -- 172.21.15.69:0/1206673825 learned_addr learned my addr 172.21.15.69:0/1206673825 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf7524f640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf701067b0 0x7fbf701988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf76251640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7019cc40 0x7fbf7019d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 -- 172.21.15.69:0/1206673825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf701067b0 msgr2=0x7fbf701988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf701067b0 0x7fbf701988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 -- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7019cc40 msgr2=0x7fbf7019d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7019cc40 0x7fbf7019d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 -- 172.21.15.69:0/1206673825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7019d630 con 0x7fbf70101150 2024-08-02T17:50:32.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf76251640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7019cc40 0x7fbf7019d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.274+0000 7fbf75a50640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf70101150 0x7fbf70198370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00e9f0 tx=0x7fbf6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.275+0000 7fbf62ffd640 1 -- 172.21.15.69:0/1206673825 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c00cd80 con 0x7fbf70101150 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.275+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7019d920 con 0x7fbf70101150 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.275+0000 7fbf62ffd640 1 -- 172.21.15.69:0/1206673825 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf6c010910 con 0x7fbf70101150 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.275+0000 7fbf62ffd640 1 -- 172.21.15.69:0/1206673825 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0188f0 con 0x7fbf70101150 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.275+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf701a74c0 con 0x7fbf70101150 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.276+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf70100c10 con 0x7fbf70101150 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.277+0000 7fbf62ffd640 1 -- 172.21.15.69:0/1206673825 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbf6c010ac0 con 0x7fbf70101150 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.279+0000 7fbf62ffd640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf5c0a2cf0 0x7fbf5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.279+0000 7fbf62ffd640 1 -- 172.21.15.69:0/1206673825 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbf6c0cfcb0 con 0x7fbf70101150 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.279+0000 7fbf7524f640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf5c0a2cf0 0x7fbf5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.280+0000 7fbf7524f640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf5c0a2cf0 0x7fbf5c0a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009230 tx=0x7fbf640314b0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:32.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.284+0000 7fbf62ffd640 1 -- 172.21.15.69:0/1206673825 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbf6c09c510 con 0x7fbf70101150 2024-08-02T17:50:32.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.654+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf7002d5e0 con 0x7fbf70101150 2024-08-02T17:50:32.658 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:50:32.658 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:50:32.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.655+0000 7fbf62ffd640 1 -- 172.21.15.69:0/1206673825 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fbf6c09be10 con 0x7fbf70101150 2024-08-02T17:50:32.658 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:50:32.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf5c0a2cf0 msgr2=0x7fbf5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:32.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf5c0a2cf0 0x7fbf5c0a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009230 tx=0x7fbf640314b0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf70101150 msgr2=0x7fbf70198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:32.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf70101150 0x7fbf70198370 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00e9f0 tx=0x7fbf6c00eeb0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 shutdown_connections 2024-08-02T17:50:32.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbf7019cc40 0x7fbf7019d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbf5c0a2cf0 0x7fbf5c0a51b0 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009230 tx=0x7fbf640314b0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbf701067b0 0x7fbf701988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 --2- 172.21.15.69:0/1206673825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbf70101150 0x7fbf70198370 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:32.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.658+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 >> 172.21.15.69:0/1206673825 conn(0x7fbf700fc4d0 msgr2=0x7fbf700fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:32.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.659+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 shutdown_connections 2024-08-02T17:50:32.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:32.659+0000 7fbf77cb4640 1 -- 172.21.15.69:0/1206673825 wait complete. 2024-08-02T17:50:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:32 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1206673825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:32 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1206673825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:32 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1206673825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:50:33.623 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:50:34.020 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:34.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.582+0000 7fccdbe73640 1 -- 172.21.15.69:0/2829427377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd4101140 msgr2=0x7fccd410d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.582+0000 7fccdbe73640 1 --2- 172.21.15.69:0/2829427377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd4101140 0x7fccd410d670 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fcccc009cd0 tx=0x7fcccc02caf0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.583+0000 7fccdbe73640 1 -- 172.21.15.69:0/2829427377 shutdown_connections 2024-08-02T17:50:34.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.583+0000 7fccdbe73640 1 --2- 172.21.15.69:0/2829427377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd4101140 0x7fccd410d670 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.583+0000 7fccdbe73640 1 --2- 172.21.15.69:0/2829427377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccd41007a0 0x7fccd4100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.583+0000 7fccdbe73640 1 --2- 172.21.15.69:0/2829427377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd41067a0 0x7fccd4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.583+0000 7fccdbe73640 1 -- 172.21.15.69:0/2829427377 >> 172.21.15.69:0/2829427377 conn(0x7fccd40fc4c0 msgr2=0x7fccd40fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:34.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.584+0000 7fccdbe73640 1 -- 172.21.15.69:0/2829427377 shutdown_connections 2024-08-02T17:50:34.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.584+0000 7fccdbe73640 1 -- 172.21.15.69:0/2829427377 wait complete. 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.584+0000 7fccdbe73640 1 Processor -- start 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccdbe73640 1 -- start start 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccdbe73640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd41007a0 0x7fccd419a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccdbe73640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccd4101140 0x7fccd419a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccdbe73640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd41067a0 0x7fccd419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccdbe73640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410fa40 con 0x7fccd41067a0 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccdbe73640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410f8c0 con 0x7fccd4101140 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccdbe73640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccd410fbc0 con 0x7fccd41007a0 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccda410640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd41067a0 0x7fccd419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccda410640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd41067a0 0x7fccd419ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52726/0 (socket says 172.21.15.69:52726) 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccd9c0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd41007a0 0x7fccd419a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccda410640 1 -- 172.21.15.69:0/578702877 learned_addr learned my addr 172.21.15.69:0/578702877 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.585+0000 7fccd940e640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccd4101140 0x7fccd419a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccd9c0f640 1 -- 172.21.15.69:0/578702877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccd4101140 msgr2=0x7fccd419a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccd4101140 0x7fccd419a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccd9c0f640 1 -- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd41067a0 msgr2=0x7fccd419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd41067a0 0x7fccd419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccd9c0f640 1 -- 172.21.15.69:0/578702877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd419f340 con 0x7fccd41007a0 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccda410640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd41067a0 0x7fccd419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd41007a0 0x7fccd419a390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fccd000e9f0 tx=0x7fccd000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fcccaffd640 1 -- 172.21.15.69:0/578702877 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd000ccc0 con 0x7fccd41007a0 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fcccaffd640 1 -- 172.21.15.69:0/578702877 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd0005020 con 0x7fccd41007a0 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccdbe73640 1 -- 172.21.15.69:0/578702877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd419f630 con 0x7fccd41007a0 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fcccaffd640 1 -- 172.21.15.69:0/578702877 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd00049e0 con 0x7fccd41007a0 2024-08-02T17:50:34.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.586+0000 7fccdbe73640 1 -- 172.21.15.69:0/578702877 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd41a6f10 con 0x7fccd41007a0 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.590+0000 7fccde271740 1 Processor -- start 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.590+0000 7fccde271740 1 -- start start 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.590+0000 7fccde271740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34856aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.590+0000 7fccde271740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c34851fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.590+0000 7fccde271740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3486c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.590+0000 7fccde271740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561c347b3a40 con 0x561c34856fe0 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.590+0000 7fccde271740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c3474b9d0 con 0x561c3486a450 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.590+0000 7fccde271740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c347afa00 con 0x561c348566c0 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34856aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34856aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33004/0 (socket says 172.21.15.69:33004) 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 -- 172.21.15.69:0/3770887996 learned_addr learned my addr 172.21.15.69:0/3770887996 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 -- 172.21.15.69:0/3770887996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 msgr2=0x561c3486c840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:50:34.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 --2- 172.21.15.69:0/3770887996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3486c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 -- 172.21.15.69:0/3770887996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 msgr2=0x561c34851fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:50:34.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 --2- 172.21.15.69:0/3770887996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c34851fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 -- 172.21.15.69:0/3770887996 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c34752650 con 0x561c348566c0 2024-08-02T17:50:34.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.591+0000 7fccda410640 1 --2- 172.21.15.69:0/3770887996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34856aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005f20 tx=0x7fcccc005f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2af6e016c035569 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.592+0000 7fccbffff640 1 -- 172.21.15.69:0/3770887996 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc047070 con 0x561c348566c0 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.592+0000 7fccbffff640 1 -- 172.21.15.69:0/3770887996 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcccc003c30 con 0x561c348566c0 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.592+0000 7fcccaffd640 1 -- 172.21.15.69:0/578702877 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fccd0005560 con 0x7fccd41007a0 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.592+0000 7fccde271740 1 -- 172.21.15.69:0/3770887996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 msgr2=0x561c34856aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.592+0000 7fccde271740 1 --2- 172.21.15.69:0/3770887996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34856aa0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcccc005f20 tx=0x7fcccc005f50 comp rx=0 tx=0).stop 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 -- 172.21.15.69:0/3770887996 shutdown_connections 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 --2- 172.21.15.69:0/3770887996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c34851fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 --2- 172.21.15.69:0/3770887996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3486c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 --2- 172.21.15.69:0/3770887996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34856aa0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 -- 172.21.15.69:0/3770887996 >> 172.21.15.69:0/3770887996 conn(0x561c34852d90 msgr2=0x561c34853170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:34.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 -- 172.21.15.69:0/3770887996 shutdown_connections 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fcccaffd640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccc00a2cf0 0x7fccc00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 -- 172.21.15.69:0/3770887996 wait complete. 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fcccaffd640 1 -- 172.21.15.69:0/578702877 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fccd00cf6e0 con 0x7fccd41007a0 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccd940e640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccc00a2cf0 0x7fccc00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 -- start start 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.593+0000 7fccde271740 1 Processor -- start 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccde271740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c3484f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccde271740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c3484f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccde271740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccde271740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561c3486ce40 con 0x561c34856fe0 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccde271740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c3476eb20 con 0x561c3486a450 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccde271740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c3486cfc0 con 0x561c348566c0 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccde271740 10 client.0 _ll_register_callbacks cb 0x561c3476cbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:34.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccd9c0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c3484f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccd9c0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c3484f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33018/0 (socket says 172.21.15.69:33018) 2024-08-02T17:50:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccd940e640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccc00a2cf0 0x7fccc00a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x561c34853e80 tx=0x7fccc4005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccd9c0f640 1 -- 172.21.15.69:0/1958821204 learned_addr learned my addr 172.21.15.69:0/1958821204 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:34.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.594+0000 7fccda410640 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccd940e640 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c3484f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccd9c0f640 1 -- 172.21.15.69:0/1958821204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 msgr2=0x561c3484fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:34.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccd9c0f640 1 -- 172.21.15.69:0/1958821204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 msgr2=0x561c3484f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:34.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c3484f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccd9c0f640 1 -- 172.21.15.69:0/1958821204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c345253c0 con 0x561c348566c0 2024-08-02T17:50:34.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccd940e640 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c3484f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c3484f380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccd0098210 tx=0x7fccd009b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d967693ec2c525f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccbd7fa640 1 -- 172.21.15.69:0/1958821204 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd001c040 con 0x561c348566c0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.595+0000 7fccbd7fa640 1 -- 172.21.15.69:0/1958821204 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fccd00b4670 con 0x561c348566c0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 1 -- 172.21.15.69:0/1958821204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcccc003c30 con 0x561c348566c0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccbd7fa640 1 -- 172.21.15.69:0/1958821204 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fccd009b840 con 0x561c348566c0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 10 client.26020 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 1 -- 172.21.15.69:0/1958821204 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c34878ae0 con 0x561c348566c0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccbd7fa640 1 -- 172.21.15.69:0/1958821204 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcccc003c30 con 0x561c348566c0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccbd7fa640 1 -- 172.21.15.69:0/1958821204 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fccd00cf2d0 con 0x561c348566c0 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 10 client.26020 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 10 client.26020 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:50:34.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 20 client.26020 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3487a370 0x561c3487c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 4 client.26020 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.596+0000 7fccde271740 1 -- 172.21.15.69:0/1958821204 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c347c0fb0 con 0x561c3487a370 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.597+0000 7fccd940e640 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3487a370 0x561c3487c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.597+0000 7fccbd7fa640 10 client.26020 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.597+0000 7fccd940e640 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3487a370 0x561c3487c7c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8f004fae7f14028 server_cookie=2165084bda73ade0 in_seq=0 out_seq=0 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccbd7fa640 1 -- 172.21.15.69:0/1958821204 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561c347c0fb0 con 0x561c3487a370 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccbd7fa640 10 client.26020 handle_command_reply: tid=0 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 client.26020 shutdown 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 -- 172.21.15.69:0/1958821204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 msgr2=0x561c3484f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c3484f380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fccd0098210 tx=0x7fccd009b260 comp rx=0 tx=0).stop 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 -- 172.21.15.69:0/1958821204 shutdown_connections 2024-08-02T17:50:34.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3487a370 0x561c3487c7c0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c3484f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 --2- 172.21.15.69:0/1958821204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c3484f380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccbd7fa640 0 client.26020 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:50:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.598+0000 7fccde271740 1 -- 172.21.15.69:0/1958821204 >> 172.21.15.69:0/1958821204 conn(0x561c34852d90 msgr2=0x561c34858af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.599+0000 7fccde271740 1 -- 172.21.15.69:0/1958821204 shutdown_connections 2024-08-02T17:50:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.599+0000 7fccde271740 1 -- 172.21.15.69:0/1958821204 wait complete. 2024-08-02T17:50:34.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.599+0000 7fccde271740 20 client.26020 trim_cache size 0 max 16384 2024-08-02T17:50:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.621+0000 7fccde271740 1 Processor -- start 2024-08-02T17:50:34.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.621+0000 7fccde271740 1 -- start start 2024-08-02T17:50:34.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.621+0000 7fccde271740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c348566c0 0x561c34849410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.621+0000 7fccde271740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c34856fe0 0x561c348820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccde271740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3486a450 0x561c34882610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccde271740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41af470 con 0x561c348566c0 2024-08-02T17:50:34.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccde271740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41acd40 con 0x561c34856fe0 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccde271740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41aabd0 con 0x561c3486a450 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c348566c0 0x561c34849410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccda410640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3486a450 0x561c34882610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c348566c0 0x561c34849410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52744/0 (socket says 172.21.15.69:52744) 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 -- 172.21.15.69:0/1463822908 learned_addr learned my addr 172.21.15.69:0/1463822908 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd940e640 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c34856fe0 0x561c348820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 -- 172.21.15.69:0/1463822908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3486a450 msgr2=0x561c34882610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3486a450 0x561c34882610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 -- 172.21.15.69:0/1463822908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c34856fe0 msgr2=0x561c348820d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c34856fe0 0x561c348820d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 -- 172.21.15.69:0/1463822908 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3487ce40 con 0x561c348566c0 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccda410640 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3486a450 0x561c34882610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd940e640 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c34856fe0 0x561c348820d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:34.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.622+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c348566c0 0x561c34849410 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fccd00cd070 tx=0x7fccd009b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fbbd37569a2ce6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:34.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccbf7fe640 1 -- 172.21.15.69:0/1463822908 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd001c030 con 0x561c348566c0 2024-08-02T17:50:34.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccbf7fe640 1 -- 172.21.15.69:0/1463822908 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccd007e020 con 0x561c348566c0 2024-08-02T17:50:34.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccbf7fe640 1 -- 172.21.15.69:0/1463822908 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd00a4110 con 0x561c348566c0 2024-08-02T17:50:34.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccde271740 1 -- 172.21.15.69:0/1463822908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c348566c0 msgr2=0x561c34849410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccde271740 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c348566c0 0x561c34849410 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fccd00cd070 tx=0x7fccd009b2d0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccde271740 1 -- 172.21.15.69:0/1463822908 shutdown_connections 2024-08-02T17:50:34.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccde271740 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3486a450 0x561c34882610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccde271740 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c34856fe0 0x561c348820d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccde271740 1 --2- 172.21.15.69:0/1463822908 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c348566c0 0x561c34849410 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.623+0000 7fccde271740 1 -- 172.21.15.69:0/1463822908 >> 172.21.15.69:0/1463822908 conn(0x561c34852d90 msgr2=0x561c34848d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.624+0000 7fccde271740 1 -- 172.21.15.69:0/1463822908 shutdown_connections 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.624+0000 7fccde271740 1 -- 172.21.15.69:0/1463822908 wait complete. 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.624+0000 7fccde271740 1 -- start start 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccde271740 1 Processor -- start 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccde271740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34872210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccde271740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c34872750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccde271740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccde271740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41ab220 con 0x561c34856fe0 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccde271740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41ac670 con 0x561c3486a450 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccde271740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccd41ab8d0 con 0x561c348566c0 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccde271740 10 client.0 _ll_register_callbacks cb 0x7fccd00053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:34.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccda410640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccd9c0f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34872210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccd940e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c34872750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccda410640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42256/0 (socket says 172.21.15.69:42256) 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.625+0000 7fccda410640 1 -- 172.21.15.69:0/982121529 learned_addr learned my addr 172.21.15.69:0/982121529 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.626+0000 7fccd940e640 1 -- 172.21.15.69:0/982121529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 msgr2=0x561c34872210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.626+0000 7fccd940e640 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34872210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.626+0000 7fccd940e640 1 -- 172.21.15.69:0/982121529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 msgr2=0x561c3484b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.626+0000 7fccd940e640 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.626+0000 7fccd940e640 1 -- 172.21.15.69:0/982121529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3484d280 con 0x561c34856fe0 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.626+0000 7fccda410640 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.627+0000 7fccd940e640 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c34872750 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x561c3487d040 tx=0x7fccc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbac6731f8b8e9f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.627+0000 7fccbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.627+0000 7fccbd7fa640 1 -- 172.21.15.69:0/982121529 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc402b070 con 0x561c34856fe0 2024-08-02T17:50:34.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.627+0000 7fccbd7fa640 1 -- 172.21.15.69:0/982121529 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fccc4003920 con 0x561c34856fe0 2024-08-02T17:50:34.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.627+0000 7fccde271740 1 -- 172.21.15.69:0/982121529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccd007e020 con 0x561c34856fe0 2024-08-02T17:50:34.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.627+0000 7fccbd7fa640 1 -- 172.21.15.69:0/982121529 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fccc4036050 con 0x561c34856fe0 2024-08-02T17:50:34.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.627+0000 7fccde271740 10 client.16524 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:34.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.627+0000 7fccde271740 1 -- 172.21.15.69:0/982121529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c348e4510 con 0x561c34856fe0 2024-08-02T17:50:34.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccbd7fa640 1 -- 172.21.15.69:0/982121529 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc4005020 con 0x561c34856fe0 2024-08-02T17:50:34.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccbd7fa640 1 -- 172.21.15.69:0/982121529 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fccc4004aa0 con 0x561c34856fe0 2024-08-02T17:50:34.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccde271740 10 client.16524 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:34.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccde271740 10 client.16524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccde271740 20 client.16524 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccde271740 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c348e4c40 0x561c348e7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccde271740 4 client.16524 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccde271740 1 -- 172.21.15.69:0/982121529 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fccd007e020 con 0x561c348e4c40 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.628+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c348e4c40 0x561c348e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccbd7fa640 10 client.16524 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccd9c0f640 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c348e4c40 0x561c348e7060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bcf29d22c4c0f2 server_cookie=6822a09952326a49 in_seq=0 out_seq=0 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccbd7fa640 1 -- 172.21.15.69:0/982121529 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fccd007e020 con 0x561c348e4c40 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccbd7fa640 10 client.16524 handle_command_reply: tid=0 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 client.16524 shutdown 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 -- 172.21.15.69:0/982121529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 msgr2=0x561c34872750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c34872750 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x561c3487d040 tx=0x7fccc4004730 comp rx=0 tx=0).stop 2024-08-02T17:50:34.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 -- 172.21.15.69:0/982121529 shutdown_connections 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c348e4c40 0x561c348e7060 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c34856fe0 0x561c34872750 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3486a450 0x561c3484b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccbd7fa640 0 client.16524 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 --2- 172.21.15.69:0/982121529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c348566c0 0x561c34872210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 -- 172.21.15.69:0/982121529 >> 172.21.15.69:0/982121529 conn(0x561c34852d90 msgr2=0x561c348509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 -- 172.21.15.69:0/982121529 shutdown_connections 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.629+0000 7fccde271740 1 -- 172.21.15.69:0/982121529 wait complete. 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccde271740 20 client.16524 trim_cache size 0 max 16384 2024-08-02T17:50:34.635 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 -- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccc00a2cf0 msgr2=0x7fccc00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccc00a2cf0 0x7fccc00a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x561c34853e80 tx=0x7fccc4005f90 comp rx=0 tx=0).stop 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 -- 172.21.15.69:0/578702877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd41007a0 msgr2=0x7fccd419a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd41007a0 0x7fccd419a390 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fccd000e9f0 tx=0x7fccd000eeb0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 -- 172.21.15.69:0/578702877 shutdown_connections 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccd41067a0 0x7fccd419ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccc00a2cf0 0x7fccc00a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccd4101140 0x7fccd419a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 --2- 172.21.15.69:0/578702877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccd41007a0 0x7fccd419a390 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:34.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.630+0000 7fccdbe73640 1 -- 172.21.15.69:0/578702877 >> 172.21.15.69:0/578702877 conn(0x7fccd40fc4c0 msgr2=0x7fccd410be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:34.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.631+0000 7fccdbe73640 1 -- 172.21.15.69:0/578702877 shutdown_connections 2024-08-02T17:50:34.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:34.631+0000 7fccdbe73640 1 -- 172.21.15.69:0/578702877 wait complete. 2024-08-02T17:50:34.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 205.0 s, 2526.0 tps, lat 40.106 ms stddev 53.928 2024-08-02T17:50:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:34 smithi083 ceph-mon[37031]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.1 MiB/s wr, 744 op/s 2024-08-02T17:50:35.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:34 smithi077 ceph-mon[37108]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.1 MiB/s wr, 744 op/s 2024-08-02T17:50:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:34 smithi069 ceph-mon[32746]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.1 MiB/s wr, 744 op/s 2024-08-02T17:50:35.779 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:50:36.196 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.507+0000 7fb46bfe2640 1 -- 172.21.15.69:0/1168691915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb464100ef0 msgr2=0x7fb464104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.507+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/1168691915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb464100ef0 0x7fb464104ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb454009860 tx=0x7fb45402a340 comp rx=0 tx=0).stop 2024-08-02T17:50:36.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.508+0000 7fb46bfe2640 1 -- 172.21.15.69:0/1168691915 shutdown_connections 2024-08-02T17:50:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.508+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/1168691915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb464105230 0x7fb46410cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.508+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/1168691915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb464100ef0 0x7fb464104ad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.508+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/1168691915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb464100540 0x7fb464100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.508+0000 7fb46bfe2640 1 -- 172.21.15.69:0/1168691915 >> 172.21.15.69:0/1168691915 conn(0x7fb464075be0 msgr2=0x7fb464075ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.508+0000 7fb46bfe2640 1 -- 172.21.15.69:0/1168691915 shutdown_connections 2024-08-02T17:50:36.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.508+0000 7fb46bfe2640 1 -- 172.21.15.69:0/1168691915 wait complete. 2024-08-02T17:50:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.510+0000 7fb46bfe2640 1 Processor -- start 2024-08-02T17:50:36.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.510+0000 7fb46bfe2640 1 -- start start 2024-08-02T17:50:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.510+0000 7fb46bfe2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb464100540 0x7fb464196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.510+0000 7fb46bfe2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb464105230 0x7fb464197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46bfe2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46419b890 0x7fb46419bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46bfe2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb464110280 con 0x7fb46419b890 2024-08-02T17:50:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46bfe2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb464110100 con 0x7fb464100540 2024-08-02T17:50:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46419b890 0x7fb46419bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46bfe2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb464110400 con 0x7fb464105230 2024-08-02T17:50:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46419b890 0x7fb46419bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40972/0 (socket says 172.21.15.69:40972) 2024-08-02T17:50:36.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 -- 172.21.15.69:0/434390372 learned_addr learned my addr 172.21.15.69:0/434390372 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46957d640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb464105230 0x7fb464197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb469d7e640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb464100540 0x7fb464196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 -- 172.21.15.69:0/434390372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb464105230 msgr2=0x7fb464197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb464105230 0x7fb464197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 -- 172.21.15.69:0/434390372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb464100540 msgr2=0x7fb464196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb464100540 0x7fb464196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 -- 172.21.15.69:0/434390372 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46419c380 con 0x7fb46419b890 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb469d7e640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb464100540 0x7fb464196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.511+0000 7fb46a57f640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46419b890 0x7fb46419bd40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00b670 tx=0x7fb45c00bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.512+0000 7fb45affd640 1 -- 172.21.15.69:0/434390372 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c003970 con 0x7fb46419b890 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.512+0000 7fb46bfe2640 1 -- 172.21.15.69:0/434390372 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46419c670 con 0x7fb46419b890 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.512+0000 7fb45affd640 1 -- 172.21.15.69:0/434390372 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb45c00a850 con 0x7fb46419b890 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.512+0000 7fb45affd640 1 -- 172.21.15.69:0/434390372 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c004690 con 0x7fb46419b890 2024-08-02T17:50:36.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.512+0000 7fb46bfe2640 1 -- 172.21.15.69:0/434390372 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4640fe430 con 0x7fb46419b890 2024-08-02T17:50:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.516+0000 7fb46ee15740 1 Processor -- start 2024-08-02T17:50:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.516+0000 7fb45affd640 1 -- 172.21.15.69:0/434390372 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb45c003b10 con 0x7fb46419b890 2024-08-02T17:50:36.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.516+0000 7fb46ee15740 1 -- start start 2024-08-02T17:50:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.516+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356b7b0 0x562e6356bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.516+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e63576430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 0x562e63578d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb46ee15740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e634cf9b0 con 0x562e6356b7b0 2024-08-02T17:50:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb46ee15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562e63477800 con 0x562e6356c0d0 2024-08-02T17:50:36.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb46ee15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562e634cb970 con 0x562e63576970 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb469d7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e63576430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb46957d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356b7b0 0x562e6356bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb469d7e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e63576430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50322/0 (socket says 172.21.15.69:50322) 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb469d7e640 1 -- 172.21.15.69:0/3337744057 learned_addr learned my addr 172.21.15.69:0/3337744057 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb46957d640 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 0x562e63578d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb469d7e640 1 -- 172.21.15.69:0/3337744057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 msgr2=0x562e63578d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb469d7e640 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 0x562e63578d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb469d7e640 1 -- 172.21.15.69:0/3337744057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356b7b0 msgr2=0x562e6356bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb469d7e640 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356b7b0 0x562e6356bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb469d7e640 1 -- 172.21.15.69:0/3337744057 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e631e3210 con 0x562e6356c0d0 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb45affd640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4500a2e40 0x7fb4500a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb46957d640 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356b7b0 0x562e6356bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.517+0000 7fb45affd640 1 -- 172.21.15.69:0/434390372 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb45c0d38c0 con 0x7fb46419b890 2024-08-02T17:50:36.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.518+0000 7fb469d7e640 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e63576430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb46000bd40 tx=0x7fb46000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7565ff4aa120abb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.518+0000 7fb46957d640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4500a2e40 0x7fb4500a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.518+0000 7fb44ffff640 1 -- 172.21.15.69:0/3337744057 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460015070 con 0x562e6356c0d0 2024-08-02T17:50:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.518+0000 7fb44ffff640 1 -- 172.21.15.69:0/3337744057 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46000be30 con 0x562e6356c0d0 2024-08-02T17:50:36.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.518+0000 7fb44ffff640 1 -- 172.21.15.69:0/3337744057 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4600119a0 con 0x562e6356c0d0 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.519+0000 7fb46ee15740 1 -- 172.21.15.69:0/3337744057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 msgr2=0x562e63576430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.519+0000 7fb46ee15740 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e63576430 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb46000bd40 tx=0x7fb46000f730 comp rx=0 tx=0).stop 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.519+0000 7fb46957d640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4500a2e40 0x7fb4500a5300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb45402a930 tx=0x7fb45402c040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.519+0000 7fb46ee15740 1 -- 172.21.15.69:0/3337744057 shutdown_connections 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.519+0000 7fb46ee15740 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 0x562e63578d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.519+0000 7fb46ee15740 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e63576430 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.519+0000 7fb46ee15740 1 --2- 172.21.15.69:0/3337744057 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356b7b0 0x562e6356bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.519+0000 7fb46ee15740 1 -- 172.21.15.69:0/3337744057 >> 172.21.15.69:0/3337744057 conn(0x562e63567d20 msgr2=0x562e63568100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.520+0000 7fb46ee15740 1 -- 172.21.15.69:0/3337744057 shutdown_connections 2024-08-02T17:50:36.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.520+0000 7fb46ee15740 1 -- 172.21.15.69:0/3337744057 wait complete. 2024-08-02T17:50:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.520+0000 7fb46ee15740 1 -- start start 2024-08-02T17:50:36.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46ee15740 1 Processor -- start 2024-08-02T17:50:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356b7b0 0x562e635737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356c0d0 0x562e63573d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 0x562e63574250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46ee15740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562e63579540 con 0x562e6356c0d0 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46ee15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562e6348b500 con 0x562e6356b7b0 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46ee15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562e635796c0 con 0x562e63576970 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46ee15740 10 client.0 _ll_register_callbacks cb 0x562e631e2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46957d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356c0d0 0x562e63573d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46957d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356c0d0 0x562e63573d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40996/0 (socket says 172.21.15.69:40996) 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46957d640 1 -- 172.21.15.69:0/654563967 learned_addr learned my addr 172.21.15.69:0/654563967 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb469d7e640 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356b7b0 0x562e635737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.521+0000 7fb46a57f640 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 0x562e63574250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb46957d640 1 -- 172.21.15.69:0/654563967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 msgr2=0x562e63574250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb46957d640 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 0x562e63574250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb46957d640 1 -- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356b7b0 msgr2=0x562e635737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.526 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb46957d640 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356b7b0 0x562e635737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb46957d640 1 -- 172.21.15.69:0/654563967 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e632af620 con 0x562e6356c0d0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb469d7e640 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356b7b0 0x562e635737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb46957d640 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356c0d0 0x562e63573d10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb45402c010 tx=0x7fb45402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7349e2262242696 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb44d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.522+0000 7fb44d7fa640 1 -- 172.21.15.69:0/654563967 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454051060 con 0x562e6356c0d0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb44d7fa640 1 -- 172.21.15.69:0/654563967 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb454003d40 con 0x562e6356c0d0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 1 -- 172.21.15.69:0/654563967 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb46000be30 con 0x562e6356c0d0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb44d7fa640 1 -- 172.21.15.69:0/654563967 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb454058050 con 0x562e6356c0d0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 10 client.16536 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 1 -- 172.21.15.69:0/654563967 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e63585160 con 0x562e6356c0d0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb44d7fa640 1 -- 172.21.15.69:0/654563967 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb46000be30 con 0x562e6356c0d0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb44d7fa640 1 -- 172.21.15.69:0/654563967 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb45403d070 con 0x562e6356c0d0 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 10 client.16536 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 20 client.16536 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562e63586620 0x562e63588a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 4 client.16536 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.523+0000 7fb46ee15740 1 -- 172.21.15.69:0/654563967 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e634dd0e0 con 0x562e63586620 2024-08-02T17:50:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.524+0000 7fb469d7e640 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562e63586620 0x562e63588a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.524+0000 7fb469d7e640 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562e63586620 0x562e63588a10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54999f6f90ff821b server_cookie=65a7f3df54594bfa in_seq=0 out_seq=0 2024-08-02T17:50:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.524+0000 7fb44d7fa640 10 client.16536 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.525+0000 7fb44d7fa640 1 -- 172.21.15.69:0/654563967 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562e634dd0e0 con 0x562e63586620 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.525+0000 7fb44d7fa640 10 client.16536 handle_command_reply: tid=0 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.525+0000 7fb46ee15740 1 client.16536 shutdown 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.526+0000 7fb46ee15740 1 -- 172.21.15.69:0/654563967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356c0d0 msgr2=0x562e63573d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.526+0000 7fb46ee15740 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356c0d0 0x562e63573d10 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb45402c010 tx=0x7fb45402a340 comp rx=0 tx=0).stop 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.526+0000 7fb46ee15740 1 -- 172.21.15.69:0/654563967 shutdown_connections 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.526+0000 7fb46ee15740 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e6356c0d0 0x562e63573d10 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.526+0000 7fb46ee15740 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562e63586620 0x562e63588a10 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.526+0000 7fb46ee15740 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e63576970 0x562e63574250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.526+0000 7fb44d7fa640 0 client.16536 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.526+0000 7fb46ee15740 1 --2- 172.21.15.69:0/654563967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356b7b0 0x562e635737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.527+0000 7fb46ee15740 1 -- 172.21.15.69:0/654563967 >> 172.21.15.69:0/654563967 conn(0x562e63567d20 msgr2=0x562e6356da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.528+0000 7fb46ee15740 1 -- 172.21.15.69:0/654563967 shutdown_connections 2024-08-02T17:50:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.528+0000 7fb46ee15740 1 -- 172.21.15.69:0/654563967 wait complete. 2024-08-02T17:50:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.528+0000 7fb46ee15740 20 client.16536 trim_cache size 0 max 16384 2024-08-02T17:50:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.543+0000 7fb46ee15740 1 Processor -- start 2024-08-02T17:50:36.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.543+0000 7fb46ee15740 1 -- start start 2024-08-02T17:50:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 0x562e6358e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6358e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e63566660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46ee15740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641af510 con 0x562e63576970 2024-08-02T17:50:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46ee15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641acde0 con 0x562e6356c0d0 2024-08-02T17:50:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46ee15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641aac30 con 0x562e6356b7b0 2024-08-02T17:50:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e63566660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e63566660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40998/0 (socket says 172.21.15.69:40998) 2024-08-02T17:50:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 -- 172.21.15.69:0/1035444067 learned_addr learned my addr 172.21.15.69:0/1035444067 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb469d7e640 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 0x562e6358e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46957d640 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6358e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 -- 172.21.15.69:0/1035444067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 msgr2=0x562e6358e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 0x562e6358e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 -- 172.21.15.69:0/1035444067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 msgr2=0x562e6358e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6358e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 -- 172.21.15.69:0/1035444067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e635689b0 con 0x562e63576970 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46957d640 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6358e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb46a57f640 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e63566660 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb45c09f6c0 tx=0x7fb45c005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9f80ac44f48ae9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.544+0000 7fb44f7fe640 1 -- 172.21.15.69:0/1035444067 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0b8070 con 0x562e63576970 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb44f7fe640 1 -- 172.21.15.69:0/1035444067 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb45c0a3d70 con 0x562e63576970 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb44f7fe640 1 -- 172.21.15.69:0/1035444067 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00ab20 con 0x562e63576970 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 -- 172.21.15.69:0/1035444067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 msgr2=0x562e63566660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e63566660 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb45c09f6c0 tx=0x7fb45c005c80 comp rx=0 tx=0).stop 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 -- 172.21.15.69:0/1035444067 shutdown_connections 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e63566660 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6358e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 --2- 172.21.15.69:0/1035444067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 0x562e6358e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 -- 172.21.15.69:0/1035444067 >> 172.21.15.69:0/1035444067 conn(0x562e63567d20 msgr2=0x562e635728a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 -- 172.21.15.69:0/1035444067 shutdown_connections 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.545+0000 7fb46ee15740 1 -- 172.21.15.69:0/1035444067 wait complete. 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.546+0000 7fb46ee15740 1 -- start start 2024-08-02T17:50:36.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.546+0000 7fb46ee15740 1 Processor -- start 2024-08-02T17:50:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 0x562e6357f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6357a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46ee15740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e6357a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46ee15740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641ab2a0 con 0x562e63576970 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46ee15740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641ac710 con 0x562e6356c0d0 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46ee15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4641ab970 con 0x562e6356b7b0 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46ee15740 10 client.0 _ll_register_callbacks cb 0x7fb454030f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46957d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6357a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46957d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6357a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50334/0 (socket says 172.21.15.69:50334) 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46957d640 1 -- 172.21.15.69:0/2801214943 learned_addr learned my addr 172.21.15.69:0/2801214943 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.547+0000 7fb46a57f640 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e6357a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.548+0000 7fb46957d640 1 -- 172.21.15.69:0/2801214943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 msgr2=0x562e6357f040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.548+0000 7fb46957d640 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 0x562e6357f040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.548+0000 7fb46957d640 1 -- 172.21.15.69:0/2801214943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 msgr2=0x562e6357a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.548+0000 7fb46957d640 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e6357a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.548+0000 7fb46957d640 1 -- 172.21.15.69:0/2801214943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6357b0d0 con 0x562e6356c0d0 2024-08-02T17:50:36.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.548+0000 7fb46957d640 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6357a3b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb454009780 tx=0x7fb45403e6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cca94789c8c9b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.549+0000 7fb44d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.549+0000 7fb44d7fa640 1 -- 172.21.15.69:0/2801214943 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454051040 con 0x562e6356c0d0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.549+0000 7fb46ee15740 1 -- 172.21.15.69:0/2801214943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb45c0a3d70 con 0x562e6356c0d0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.549+0000 7fb44d7fa640 1 -- 172.21.15.69:0/2801214943 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb45403e950 con 0x562e6356c0d0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.549+0000 7fb44d7fa640 1 -- 172.21.15.69:0/2801214943 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb454037040 con 0x562e6356c0d0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.549+0000 7fb46ee15740 10 client.26147 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.549+0000 7fb46ee15740 1 -- 172.21.15.69:0/2801214943 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562e635f1200 con 0x562e6356c0d0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb44d7fa640 1 -- 172.21.15.69:0/2801214943 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb45c0a3d70 con 0x562e6356c0d0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb44d7fa640 1 -- 172.21.15.69:0/2801214943 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb45403ece0 con 0x562e6356c0d0 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb46ee15740 10 client.26147 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb46ee15740 10 client.26147 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb46ee15740 20 client.26147 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:36.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb46ee15740 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562e63586620 0x562e635f3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb46ee15740 4 client.26147 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:50:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb46ee15740 1 -- 172.21.15.69:0/2801214943 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e634d7430 con 0x562e63586620 2024-08-02T17:50:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.550+0000 7fb469d7e640 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562e63586620 0x562e635f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.551+0000 7fb469d7e640 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562e63586620 0x562e635f3990 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fa5a55389a4fef server_cookie=363b92722929a787 in_seq=0 out_seq=0 2024-08-02T17:50:36.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.551+0000 7fb44d7fa640 10 client.26147 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.618+0000 7fb44d7fa640 1 -- 172.21.15.69:0/2801214943 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e634d7430 con 0x562e63586620 2024-08-02T17:50:36.620 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.618+0000 7fb44d7fa640 10 client.26147 handle_command_reply: tid=0 2024-08-02T17:50:36.621 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.619+0000 7fb46ee15740 1 client.26147 shutdown 2024-08-02T17:50:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.625+0000 7fb46ee15740 1 -- 172.21.15.69:0/2801214943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 msgr2=0x562e6357a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.625+0000 7fb46ee15740 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6357a3b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb454009780 tx=0x7fb45403e6e0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.625+0000 7fb46ee15740 1 -- 172.21.15.69:0/2801214943 shutdown_connections 2024-08-02T17:50:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.626+0000 7fb46ee15740 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562e63576970 0x562e6357a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.626+0000 7fb46ee15740 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562e63586620 0x562e635f3990 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.626+0000 7fb46ee15740 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562e6356c0d0 0x562e6357a3b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.626+0000 7fb46ee15740 1 --2- 172.21.15.69:0/2801214943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562e6356b7b0 0x562e6357f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.626+0000 7fb46ee15740 1 -- 172.21.15.69:0/2801214943 >> 172.21.15.69:0/2801214943 conn(0x562e63567d20 msgr2=0x562e6356d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.626+0000 7fb44d7fa640 0 client.26147 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.629+0000 7fb46ee15740 1 -- 172.21.15.69:0/2801214943 shutdown_connections 2024-08-02T17:50:36.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.631+0000 7fb46ee15740 1 -- 172.21.15.69:0/2801214943 wait complete. 2024-08-02T17:50:36.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.633+0000 7fb46ee15740 20 client.26147 trim_cache size 0 max 16384 2024-08-02T17:50:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:50:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:50:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:50:36.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:50:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.637+0000 7fb46bfe2640 1 -- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4500a2e40 msgr2=0x7fb4500a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.637+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4500a2e40 0x7fb4500a5300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb45402a930 tx=0x7fb45402c040 comp rx=0 tx=0).stop 2024-08-02T17:50:36.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.637+0000 7fb46bfe2640 1 -- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46419b890 msgr2=0x7fb46419bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.639 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.637+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46419b890 0x7fb46419bd40 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00b670 tx=0x7fb45c00bb30 comp rx=0 tx=0).stop 2024-08-02T17:50:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.639+0000 7fb46bfe2640 1 -- 172.21.15.69:0/434390372 shutdown_connections 2024-08-02T17:50:36.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.639+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb46419b890 0x7fb46419bd40 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.639+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4500a2e40 0x7fb4500a5300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.639+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb464105230 0x7fb464197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.639+0000 7fb46bfe2640 1 --2- 172.21.15.69:0/434390372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb464100540 0x7fb464196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.639+0000 7fb46bfe2640 1 -- 172.21.15.69:0/434390372 >> 172.21.15.69:0/434390372 conn(0x7fb464075be0 msgr2=0x7fb4640ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.641+0000 7fb46bfe2640 1 -- 172.21.15.69:0/434390372 shutdown_connections 2024-08-02T17:50:36.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:36.642+0000 7fb46bfe2640 1 -- 172.21.15.69:0/434390372 wait complete. 2024-08-02T17:50:36.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.784+0000 7f79d15b5640 1 -- 172.21.15.69:0/1816992264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc109180 msgr2=0x7f79cc10ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.784+0000 7f79d15b5640 1 --2- 172.21.15.69:0/1816992264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc109180 0x7f79cc10ff80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009900 tx=0x7f79bc02a440 comp rx=0 tx=0).stop 2024-08-02T17:50:36.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.784+0000 7f79d15b5640 1 -- 172.21.15.69:0/1816992264 shutdown_connections 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.784+0000 7f79d15b5640 1 --2- 172.21.15.69:0/1816992264 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc11ae70 0x7f79cc11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.784+0000 7f79d15b5640 1 --2- 172.21.15.69:0/1816992264 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79cc109560 0x7f79cc1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.784+0000 7f79d15b5640 1 --2- 172.21.15.69:0/1816992264 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc109180 0x7f79cc10ff80 secure :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009900 tx=0x7f79bc02a440 comp rx=0 tx=0).stop 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.784+0000 7f79d15b5640 1 -- 172.21.15.69:0/1816992264 >> 172.21.15.69:0/1816992264 conn(0x7f79cc06de70 msgr2=0x7f79cc06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.785+0000 7f79d15b5640 1 -- 172.21.15.69:0/1816992264 shutdown_connections 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.785+0000 7f79d15b5640 1 -- 172.21.15.69:0/1816992264 wait complete. 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.786+0000 7f79d15b5640 1 Processor -- start 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79d15b5640 1 -- start start 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79d15b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc109560 0x7f79cc118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79d15b5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc11ae70 0x7f79cc110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79d15b5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79cc110c60 0x7f79cc111110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79d15b5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc120b60 con 0x7f79cc11ae70 2024-08-02T17:50:36.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79d15b5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1209e0 con 0x7f79cc110c60 2024-08-02T17:50:36.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79d15b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc120ce0 con 0x7f79cc109560 2024-08-02T17:50:36.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc11ae70 0x7f79cc110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc11ae70 0x7f79cc110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41022/0 (socket says 172.21.15.69:41022) 2024-08-02T17:50:36.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.787+0000 7f79ca7fc640 1 -- 172.21.15.69:0/3718417420 learned_addr learned my addr 172.21.15.69:0/3718417420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.788+0000 7f79ca7fc640 1 -- 172.21.15.69:0/3718417420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc109560 msgr2=0x7f79cc118750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:50:36.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.788+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc109560 0x7f79cc118750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.788+0000 7f79ca7fc640 1 -- 172.21.15.69:0/3718417420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79cc110c60 msgr2=0x7f79cc111110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.788+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79cc110c60 0x7f79cc111110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.788+0000 7f79ca7fc640 1 -- 172.21.15.69:0/3718417420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc1bc0e0 con 0x7f79cc11ae70 2024-08-02T17:50:36.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.789+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc11ae70 0x7f79cc110720 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f79c000e9f0 tx=0x7f79c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.794+0000 7f79b3fff640 1 -- 172.21.15.69:0/3718417420 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c000cd80 con 0x7f79cc11ae70 2024-08-02T17:50:36.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.794+0000 7f79d15b5640 1 -- 172.21.15.69:0/3718417420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79cc1bc3d0 con 0x7f79cc11ae70 2024-08-02T17:50:36.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.794+0000 7f79b3fff640 1 -- 172.21.15.69:0/3718417420 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c0010910 con 0x7f79cc11ae70 2024-08-02T17:50:36.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.794+0000 7f79b3fff640 1 -- 172.21.15.69:0/3718417420 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00188b0 con 0x7f79cc11ae70 2024-08-02T17:50:36.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.794+0000 7f79d15b5640 1 -- 172.21.15.69:0/3718417420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79cc1bc910 con 0x7f79cc11ae70 2024-08-02T17:50:36.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.805+0000 7f79d39b3740 1 Processor -- start 2024-08-02T17:50:36.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.805+0000 7f79d39b3740 1 -- start start 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182bafbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182bafe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79d39b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56182ba56a40 con 0x56182bafc1d0 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79d39b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56182b9ee9d0 con 0x56182baf0ff0 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79d39b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56182ba52a00 con 0x56182baf1910 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79caffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182bafbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79caffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182bafbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:36276/0 (socket says 172.21.15.69:36276) 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79caffd640 1 -- 172.21.15.69:0/3468154362 learned_addr learned my addr 172.21.15.69:0/3468154362 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79cb7fe640 1 --2- 172.21.15.69:0/3468154362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79caffd640 1 -- 172.21.15.69:0/3468154362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 msgr2=0x56182baf13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79caffd640 1 --2- 172.21.15.69:0/3468154362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79caffd640 1 -- 172.21.15.69:0/3468154362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 msgr2=0x56182bafe5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:50:36.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79caffd640 1 --2- 172.21.15.69:0/3468154362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182bafe5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.814+0000 7f79caffd640 1 -- 172.21.15.69:0/3468154362 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182b9f5650 con 0x56182baf1910 2024-08-02T17:50:36.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.815+0000 7f79caffd640 1 --2- 172.21.15.69:0/3468154362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182bafbc90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005ab0 tx=0x7f79bc009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f80abb3fb0efb111 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.815+0000 7f79b17fa640 1 -- 172.21.15.69:0/3468154362 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc03e070 con 0x56182baf1910 2024-08-02T17:50:36.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.815+0000 7f79b17fa640 1 -- 172.21.15.69:0/3468154362 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79bc031da0 con 0x56182baf1910 2024-08-02T17:50:36.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.815+0000 7f79b17fa640 1 -- 172.21.15.69:0/3468154362 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc035040 con 0x56182baf1910 2024-08-02T17:50:36.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.815+0000 7f79b3fff640 1 -- 172.21.15.69:0/3718417420 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f79c0010430 con 0x7f79cc11ae70 2024-08-02T17:50:36.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.816+0000 7f79d39b3740 1 -- 172.21.15.69:0/3468154362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 msgr2=0x56182bafbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.816+0000 7f79d39b3740 1 --2- 172.21.15.69:0/3468154362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182bafbc90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005ab0 tx=0x7f79bc009680 comp rx=0 tx=0).stop 2024-08-02T17:50:36.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.816+0000 7f79b3fff640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f79b40a2f00 0x7f79b40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.817+0000 7f79b3fff640 1 -- 172.21.15.69:0/3718417420 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f79c00cfb50 con 0x7f79cc11ae70 2024-08-02T17:50:36.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.817+0000 7f79cb7fe640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f79b40a2f00 0x7f79b40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.817+0000 7f79d39b3740 1 -- 172.21.15.69:0/3468154362 shutdown_connections 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.817+0000 7f79d39b3740 1 --2- 172.21.15.69:0/3468154362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182bafe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.817+0000 7f79d39b3740 1 --2- 172.21.15.69:0/3468154362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182bafbc90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.817+0000 7f79d39b3740 1 --2- 172.21.15.69:0/3468154362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.817+0000 7f79d39b3740 1 -- 172.21.15.69:0/3468154362 >> 172.21.15.69:0/3468154362 conn(0x56182baed580 msgr2=0x56182baed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.818+0000 7f79d39b3740 1 -- 172.21.15.69:0/3468154362 shutdown_connections 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.818+0000 7f79d39b3740 1 -- 172.21.15.69:0/3468154362 wait complete. 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.818+0000 7f79cb7fe640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f79b40a2f00 0x7f79b40a53c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009e00 tx=0x7f79c4009430 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.818+0000 7f79d39b3740 1 -- start start 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.818+0000 7f79d39b3740 1 Processor -- start 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182baf95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182baf9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79d39b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56182bafeda0 con 0x56182bafc1d0 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79d39b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56182ba11b20 con 0x56182baf0ff0 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79d39b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56182bafef20 con 0x56182baf1910 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79d39b3740 10 client.0 _ll_register_callbacks cb 0x56182ba0fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182baf95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182baf95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:36278/0 (socket says 172.21.15.69:36278) 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79caffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79ca7fc640 1 -- 172.21.15.69:0/1786454381 learned_addr learned my addr 172.21.15.69:0/1786454381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79ca7fc640 1 -- 172.21.15.69:0/1786454381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 msgr2=0x56182baf9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79ca7fc640 1 -- 172.21.15.69:0/1786454381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 msgr2=0x56182baf9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182baf9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.819+0000 7f79ca7fc640 1 -- 172.21.15.69:0/1786454381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182b7c83c0 con 0x56182baf1910 2024-08-02T17:50:36.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.820+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182baf95c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79c000c6a0 tx=0x7f79c009b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0dd4cbb3e3e9140 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.821+0000 7f79aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:36.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.821+0000 7f79aeff5640 1 -- 172.21.15.69:0/1786454381 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00b1070 con 0x56182baf1910 2024-08-02T17:50:36.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.821+0000 7f79aeff5640 1 -- 172.21.15.69:0/1786454381 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f79c009b2a0 con 0x56182baf1910 2024-08-02T17:50:36.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.821+0000 7f79d39b3740 1 -- 172.21.15.69:0/1786454381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79bc031da0 con 0x56182baf1910 2024-08-02T17:50:36.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.821+0000 7f79aeff5640 1 -- 172.21.15.69:0/1786454381 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f79c00cd310 con 0x56182baf1910 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79d39b3740 10 client.26047 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79d39b3740 1 -- 172.21.15.69:0/1786454381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56182bb0aa00 con 0x56182baf1910 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79aeff5640 1 -- 172.21.15.69:0/1786454381 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79bc031da0 con 0x56182baf1910 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79aeff5640 1 -- 172.21.15.69:0/1786454381 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f79c00b4020 con 0x56182baf1910 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79d39b3740 10 client.26047 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79d39b3740 10 client.26047 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79d39b3740 20 client.26047 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x56182bb0c350 0x56182bb0e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79d39b3740 4 client.26047 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.822+0000 7f79d39b3740 1 -- 172.21.15.69:0/1786454381 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56182ba63fb0 con 0x56182bb0c350 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.823+0000 7f79caffd640 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x56182bb0c350 0x56182bb0e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.824+0000 7f79caffd640 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x56182bb0c350 0x56182bb0e810 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33c552fe7082f178 server_cookie=53b829fcf0b9f69b in_seq=0 out_seq=0 2024-08-02T17:50:36.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.824+0000 7f79aeff5640 10 client.26047 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79aeff5640 1 -- 172.21.15.69:0/1786454381 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56182ba63fb0 con 0x56182bb0c350 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79aeff5640 10 client.26047 handle_command_reply: tid=0 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79d39b3740 1 client.26047 shutdown 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79d39b3740 1 -- 172.21.15.69:0/1786454381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 msgr2=0x56182baf95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182baf95c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f79c000c6a0 tx=0x7f79c009b880 comp rx=0 tx=0).stop 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79d39b3740 1 -- 172.21.15.69:0/1786454381 shutdown_connections 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x56182bb0c350 0x56182bb0e810 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182baf9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf1910 0x56182baf95c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.825+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1786454381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.826+0000 7f79d39b3740 1 -- 172.21.15.69:0/1786454381 >> 172.21.15.69:0/1786454381 conn(0x56182baed580 msgr2=0x56182baf32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.826+0000 7f79aeff5640 0 client.26047 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:50:36.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.826+0000 7f79d39b3740 1 -- 172.21.15.69:0/1786454381 shutdown_connections 2024-08-02T17:50:36.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.826+0000 7f79d39b3740 1 -- 172.21.15.69:0/1786454381 wait complete. 2024-08-02T17:50:36.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.826+0000 7f79d39b3740 20 client.26047 trim_cache size 0 max 16384 2024-08-02T17:50:36.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.845+0000 7f79d39b3740 1 Processor -- start 2024-08-02T17:50:36.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.845+0000 7f79d39b3740 1 -- start start 2024-08-02T17:50:36.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.845+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.845+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182baf1910 0x56182baf8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.845+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182bafc1d0 0x56182bb140e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.845+0000 7f79d39b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79b80128c0 con 0x56182baf1910 2024-08-02T17:50:36.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.845+0000 7f79d39b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79b8010190 con 0x56182baf0ff0 2024-08-02T17:50:36.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.845+0000 7f79d39b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79b800dfe0 con 0x56182bafc1d0 2024-08-02T17:50:36.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182baf1910 0x56182baf8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79caffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79caffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50400/0 (socket says 172.21.15.69:50400) 2024-08-02T17:50:36.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79caffd640 1 -- 172.21.15.69:0/1355111590 learned_addr learned my addr 172.21.15.69:0/1355111590 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79ca7fc640 1 -- 172.21.15.69:0/1355111590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182bafc1d0 msgr2=0x56182bb140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/1355111590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182bafc1d0 0x56182bb140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79ca7fc640 1 -- 172.21.15.69:0/1355111590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 msgr2=0x56182baf8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/1355111590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79ca7fc640 1 -- 172.21.15.69:0/1355111590 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182baf1740 con 0x56182baf1910 2024-08-02T17:50:36.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.846+0000 7f79caffd640 1 --2- 172.21.15.69:0/1355111590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:36.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.847+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/1355111590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182baf1910 0x56182baf8170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f79c009bbe0 tx=0x7f79c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f1a150a9679889 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.847+0000 7f79b0ff9640 1 -- 172.21.15.69:0/1355111590 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00b1070 con 0x56182baf1910 2024-08-02T17:50:36.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.847+0000 7f79b0ff9640 1 -- 172.21.15.69:0/1355111590 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c00b4e50 con 0x56182baf1910 2024-08-02T17:50:36.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.847+0000 7f79b0ff9640 1 -- 172.21.15.69:0/1355111590 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c009b300 con 0x56182baf1910 2024-08-02T17:50:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 -- 172.21.15.69:0/1355111590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182baf1910 msgr2=0x56182baf8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1355111590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182baf1910 0x56182baf8170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f79c009bbe0 tx=0x7f79c0000bc0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 -- 172.21.15.69:0/1355111590 shutdown_connections 2024-08-02T17:50:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1355111590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182baf1910 0x56182baf8170 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1355111590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182bafc1d0 0x56182bb140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 --2- 172.21.15.69:0/1355111590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf0ff0 0x56182baf8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 -- 172.21.15.69:0/1355111590 >> 172.21.15.69:0/1355111590 conn(0x56182baed580 msgr2=0x56182baed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 -- 172.21.15.69:0/1355111590 shutdown_connections 2024-08-02T17:50:36.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.849+0000 7f79d39b3740 1 -- 172.21.15.69:0/1355111590 wait complete. 2024-08-02T17:50:36.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 1 -- start start 2024-08-02T17:50:36.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 1 Processor -- start 2024-08-02T17:50:36.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf0ff0 0x56182baff200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf1910 0x56182baff740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182baffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f79b800e650 con 0x56182bafc1d0 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f79b800fac0 con 0x56182baf1910 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f79b800ed20 con 0x56182baf0ff0 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79d39b3740 10 client.0 _ll_register_callbacks cb 0x7f79c0005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf1910 0x56182baff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79ca7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf1910 0x56182baff740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50414/0 (socket says 172.21.15.69:50414) 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79ca7fc640 1 -- 172.21.15.69:0/2658786491 learned_addr learned my addr 172.21.15.69:0/2658786491 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.850+0000 7f79caffd640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf0ff0 0x56182baff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79cb7fe640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182baffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79ca7fc640 1 -- 172.21.15.69:0/2658786491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf0ff0 msgr2=0x56182baff200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf0ff0 0x56182baff200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79ca7fc640 1 -- 172.21.15.69:0/2658786491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 msgr2=0x56182baffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182baffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79ca7fc640 1 -- 172.21.15.69:0/2658786491 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182bb753e0 con 0x56182baf1910 2024-08-02T17:50:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79cb7fe640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182baffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79caffd640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf0ff0 0x56182baff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79ca7fc640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf1910 0x56182baff740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f79c00b6010 tx=0x7f79c009b2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45897d3074cf4f72 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:36.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.851+0000 7f79aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.853+0000 7f79aeff5640 1 -- 172.21.15.69:0/2658786491 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00a9020 con 0x56182baf1910 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.853+0000 7f79aeff5640 1 -- 172.21.15.69:0/2658786491 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f79c009ad10 con 0x56182baf1910 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.853+0000 7f79d39b3740 1 -- 172.21.15.69:0/2658786491 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79c00b4e50 con 0x56182baf1910 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.853+0000 7f79aeff5640 1 -- 172.21.15.69:0/2658786491 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f79c00b1040 con 0x56182baf1910 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.853+0000 7f79d39b3740 10 client.26162 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.853+0000 7f79d39b3740 1 -- 172.21.15.69:0/2658786491 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56182bb753e0 con 0x56182baf1910 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.853+0000 7f79aeff5640 1 -- 172.21.15.69:0/2658786491 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c00a48c0 con 0x56182baf1910 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79aeff5640 1 -- 172.21.15.69:0/2658786491 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f79c00b4040 con 0x56182baf1910 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79d39b3740 10 client.26162 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79d39b3740 10 client.26162 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79d39b3740 20 client.26162 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79d39b3740 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x56182bb76910 0x56182bb78d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:36.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79d39b3740 4 client.26162 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79d39b3740 1 -- 172.21.15.69:0/2658786491 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f79c00b4e50 con 0x56182bb76910 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79caffd640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x56182bb76910 0x56182bb78d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79caffd640 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x56182bb76910 0x56182bb78d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3c0a91bac3a39f3 server_cookie=97de2efb9846ae85 in_seq=0 out_seq=0 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79aeff5640 10 client.26162 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79aeff5640 1 -- 172.21.15.69:0/2658786491 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f79c00b4e50 con 0x56182bb76910 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79aeff5640 10 client.26162 handle_command_reply: tid=0 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.854+0000 7f79d39b3740 1 client.26162 shutdown 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 -- 172.21.15.69:0/2658786491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf1910 msgr2=0x56182baff740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf1910 0x56182baff740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f79c00b6010 tx=0x7f79c009b2d0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 -- 172.21.15.69:0/2658786491 shutdown_connections 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x56182bb76910 0x56182bb78d30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56182bafc1d0 0x56182baffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79aeff5640 0 client.26162 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:50:36.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56182baf1910 0x56182baff740 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 --2- 172.21.15.69:0/2658786491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56182baf0ff0 0x56182baff200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 -- 172.21.15.69:0/2658786491 >> 172.21.15.69:0/2658786491 conn(0x56182baed580 msgr2=0x56182baec5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 -- 172.21.15.69:0/2658786491 shutdown_connections 2024-08-02T17:50:36.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 1 -- 172.21.15.69:0/2658786491 wait complete. 2024-08-02T17:50:36.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.855+0000 7f79d39b3740 20 client.26162 trim_cache size 0 max 16384 2024-08-02T17:50:36.860 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:50:36.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 -- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f79b40a2f00 msgr2=0x7f79b40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f79b40a2f00 0x7f79b40a53c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009e00 tx=0x7f79c4009430 comp rx=0 tx=0).stop 2024-08-02T17:50:36.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 -- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc11ae70 msgr2=0x7f79cc110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:36.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc11ae70 0x7f79cc110720 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f79c000e9f0 tx=0x7f79c000eeb0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 -- 172.21.15.69:0/3718417420 shutdown_connections 2024-08-02T17:50:36.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f79cc11ae70 0x7f79cc110720 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f79b40a2f00 0x7f79b40a53c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f79cc110c60 0x7f79cc111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 --2- 172.21.15.69:0/3718417420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f79cc109560 0x7f79cc118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:36.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 -- 172.21.15.69:0/3718417420 >> 172.21.15.69:0/3718417420 conn(0x7f79cc06de70 msgr2=0x7f79cc11bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:36.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 -- 172.21.15.69:0/3718417420 shutdown_connections 2024-08-02T17:50:36.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:36.856+0000 7f79d15b5640 1 -- 172.21.15.69:0/3718417420 wait complete. 2024-08-02T17:50:37.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:36 smithi083 ceph-mon[37031]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.9 MiB/s wr, 605 op/s 2024-08-02T17:50:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:36 smithi077 ceph-mon[37108]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.9 MiB/s wr, 605 op/s 2024-08-02T17:50:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:36 smithi069 ceph-mon[32746]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 4.9 MiB/s wr, 605 op/s 2024-08-02T17:50:37.671 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:50:37.999 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:37 smithi077 ceph-mon[37108]: pgmap v129: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.1 MiB/s wr, 841 op/s 2024-08-02T17:50:38.024 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:37 smithi083 ceph-mon[37031]: pgmap v129: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.1 MiB/s wr, 841 op/s 2024-08-02T17:50:38.108 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:38.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:37 smithi069 ceph-mon[32746]: pgmap v129: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 8.1 MiB/s wr, 841 op/s 2024-08-02T17:50:38.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.693+0000 7f404701e640 1 -- 172.21.15.69:0/1700668870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4040109b30 msgr2=0x7f4040074000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.693+0000 7f404701e640 1 --2- 172.21.15.69:0/1700668870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4040109b30 0x7f4040074000 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4038009ca0 tx=0x7f403802a8f0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.695+0000 7f404701e640 1 -- 172.21.15.69:0/1700668870 shutdown_connections 2024-08-02T17:50:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.695+0000 7f404701e640 1 --2- 172.21.15.69:0/1700668870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4040109b30 0x7f4040074000 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.695+0000 7f404701e640 1 --2- 172.21.15.69:0/1700668870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40400746c0 0x7f404007b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.695+0000 7f404701e640 1 --2- 172.21.15.69:0/1700668870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040109180 0x7f4040109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.695+0000 7f404701e640 1 -- 172.21.15.69:0/1700668870 >> 172.21.15.69:0/1700668870 conn(0x7f404006de70 msgr2=0x7f404006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:38.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.696+0000 7f404701e640 1 -- 172.21.15.69:0/1700668870 shutdown_connections 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.696+0000 7f404701e640 1 -- 172.21.15.69:0/1700668870 wait complete. 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f404701e640 1 Processor -- start 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f404701e640 1 -- start start 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f404701e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40400746c0 0x7f404007afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f404701e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040109180 0x7f4040076060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f404701e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4040109b30 0x7f40400765e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f404701e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f404007e560 con 0x7f40400746c0 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f404701e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f404007e3e0 con 0x7f4040109b30 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f404701e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f404007e6e0 con 0x7f4040109180 2024-08-02T17:50:38.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.697+0000 7f4044dba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40400746c0 0x7f404007afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f4044dba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40400746c0 0x7f404007afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41056/0 (socket says 172.21.15.69:41056) 2024-08-02T17:50:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f4044dba640 1 -- 172.21.15.69:0/1900391105 learned_addr learned my addr 172.21.15.69:0/1900391105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f40455bb640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4040109b30 0x7f40400765e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f403ffff640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040109180 0x7f4040076060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f40455bb640 1 -- 172.21.15.69:0/1900391105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040109180 msgr2=0x7f4040076060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f40455bb640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040109180 0x7f4040076060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f40455bb640 1 -- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40400746c0 msgr2=0x7f404007afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f40455bb640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40400746c0 0x7f404007afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f40455bb640 1 -- 172.21.15.69:0/1900391105 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4040076ea0 con 0x7f4040109b30 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f40455bb640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4040109b30 0x7f40400765e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f403000b3e0 tx=0x7f403000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.698+0000 7f403dffb640 1 -- 172.21.15.69:0/1900391105 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4030015020 con 0x7f4040109b30 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.699+0000 7f403dffb640 1 -- 172.21.15.69:0/1900391105 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4030011d10 con 0x7f4040109b30 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.699+0000 7f403dffb640 1 -- 172.21.15.69:0/1900391105 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403000f2e0 con 0x7f4040109b30 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.699+0000 7f404701e640 1 -- 172.21.15.69:0/1900391105 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40401b4db0 con 0x7f4040109b30 2024-08-02T17:50:38.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.699+0000 7f404701e640 1 -- 172.21.15.69:0/1900391105 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40401b52f0 con 0x7f4040109b30 2024-08-02T17:50:38.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.703+0000 7f403dffb640 1 -- 172.21.15.69:0/1900391105 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4030018020 con 0x7f4040109b30 2024-08-02T17:50:38.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.704+0000 7f403dffb640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40280a2d60 0x7f40280a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.704+0000 7f403dffb640 1 -- 172.21.15.69:0/1900391105 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f40300ceea0 con 0x7f4040109b30 2024-08-02T17:50:38.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.705+0000 7f4044dba640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40280a2d60 0x7f40280a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f404941c740 1 Processor -- start 2024-08-02T17:50:38.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f404941c740 1 -- start start 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f404941c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393178ff0 0x5583931793d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f4044dba640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40280a2d60 0x7f40280a5220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4034006fd0 tx=0x7f4034008040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f404941c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393183c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f404941c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x5583931865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f404941c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5583930dea40 con 0x558393178ff0 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f404941c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5583930769d0 con 0x558393179910 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f404941c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5583930daa00 con 0x5583931841d0 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f403ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393183c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f4044dba640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393178ff0 0x5583931793d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f40455bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x5583931865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f403ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393183c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50444/0 (socket says 172.21.15.69:50444) 2024-08-02T17:50:38.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f403ffff640 1 -- 172.21.15.69:0/542441929 learned_addr learned my addr 172.21.15.69:0/542441929 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:38.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.706+0000 7f40455bb640 1 --2- 172.21.15.69:0/542441929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x5583931865c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:36318/0 (socket says 172.21.15.69:36318) 2024-08-02T17:50:38.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.707+0000 7f403ffff640 1 -- 172.21.15.69:0/542441929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 msgr2=0x5583931865c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.707+0000 7f403ffff640 1 --2- 172.21.15.69:0/542441929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x5583931865c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.707+0000 7f403ffff640 1 -- 172.21.15.69:0/542441929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393178ff0 msgr2=0x5583931793d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.707+0000 7f403ffff640 1 --2- 172.21.15.69:0/542441929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393178ff0 0x5583931793d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.707+0000 7f403ffff640 1 -- 172.21.15.69:0/542441929 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839307d650 con 0x558393179910 2024-08-02T17:50:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.707+0000 7f403ffff640 1 --2- 172.21.15.69:0/542441929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393183c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f404006f3a0 tx=0x7f4038031d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0cb62af6e4dbbe4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.707+0000 7f4026ffd640 1 -- 172.21.15.69:0/542441929 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380307f0 con 0x558393179910 2024-08-02T17:50:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.708+0000 7f4026ffd640 1 -- 172.21.15.69:0/542441929 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4038036030 con 0x558393179910 2024-08-02T17:50:38.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.708+0000 7f4026ffd640 1 -- 172.21.15.69:0/542441929 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038030ec0 con 0x558393179910 2024-08-02T17:50:38.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 -- 172.21.15.69:0/542441929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 msgr2=0x558393183c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 --2- 172.21.15.69:0/542441929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393183c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f404006f3a0 tx=0x7f4038031d10 comp rx=0 tx=0).stop 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 -- 172.21.15.69:0/542441929 shutdown_connections 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 --2- 172.21.15.69:0/542441929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x5583931865c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 --2- 172.21.15.69:0/542441929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393183c90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 --2- 172.21.15.69:0/542441929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393178ff0 0x5583931793d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 -- 172.21.15.69:0/542441929 >> 172.21.15.69:0/542441929 conn(0x558393175580 msgr2=0x558393175960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 -- 172.21.15.69:0/542441929 shutdown_connections 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.709+0000 7f404941c740 1 -- 172.21.15.69:0/542441929 wait complete. 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.710+0000 7f404941c740 1 -- start start 2024-08-02T17:50:38.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.710+0000 7f404941c740 1 Processor -- start 2024-08-02T17:50:38.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f404941c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393178ff0 0x558393183c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f404941c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393179910 0x558393180fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f404941c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393181760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f404941c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558393186df0 con 0x558393179910 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f404941c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558393099b20 con 0x558393178ff0 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f404941c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558393186f70 con 0x5583931841d0 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f4044dba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393178ff0 0x558393183c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f403ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393179910 0x558393180fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f4044dba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393178ff0 0x558393183c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50454/0 (socket says 172.21.15.69:50454) 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f4044dba640 1 -- 172.21.15.69:0/2721558373 learned_addr learned my addr 172.21.15.69:0/2721558373 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f403ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393179910 0x558393180fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41076/0 (socket says 172.21.15.69:41076) 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f404941c740 10 client.0 _ll_register_callbacks cb 0x558393097bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:38.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f40455bb640 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393181760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f4044dba640 1 -- 172.21.15.69:0/2721558373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 msgr2=0x558393181760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f4044dba640 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393181760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f4044dba640 1 -- 172.21.15.69:0/2721558373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393179910 msgr2=0x558393180fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f4044dba640 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393179910 0x558393180fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f4044dba640 1 -- 172.21.15.69:0/2721558373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558392e503c0 con 0x558393178ff0 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.711+0000 7f40455bb640 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393181760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f4044dba640 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393178ff0 0x558393183c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f40340146b0 tx=0x7f4034004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=531896459a025c6c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f401ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f401ffff640 1 -- 172.21.15.69:0/2721558373 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034024020 con 0x558393178ff0 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f401ffff640 1 -- 172.21.15.69:0/2721558373 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f403400f040 con 0x558393178ff0 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f404941c740 1 -- 172.21.15.69:0/2721558373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4038036030 con 0x558393178ff0 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f401ffff640 1 -- 172.21.15.69:0/2721558373 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f403402f050 con 0x558393178ff0 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f404941c740 10 client.26171 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:38.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f404941c740 1 -- 172.21.15.69:0/2721558373 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558393192990 con 0x558393178ff0 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f401ffff640 1 -- 172.21.15.69:0/2721558373 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4034012b90 con 0x558393178ff0 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f401ffff640 1 -- 172.21.15.69:0/2721558373 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4034013070 con 0x558393178ff0 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.712+0000 7f404941c740 10 client.26171 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.713+0000 7f404941c740 10 client.26171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.713+0000 7f404941c740 20 client.26171 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.713+0000 7f404941c740 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558393193e50 0x558393196310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.713+0000 7f404941c740 4 client.26171 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.713+0000 7f404941c740 1 -- 172.21.15.69:0/2721558373 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4038036030 con 0x558393193e50 2024-08-02T17:50:38.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.713+0000 7f403ffff640 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558393193e50 0x558393196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.714+0000 7f403ffff640 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558393193e50 0x558393196310 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce57dfea9d309ba server_cookie=32f125819fe6d37c in_seq=0 out_seq=0 2024-08-02T17:50:38.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.714+0000 7f401ffff640 10 client.26171 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:38.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.716+0000 7f401ffff640 1 -- 172.21.15.69:0/2721558373 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f4038036030 con 0x558393193e50 2024-08-02T17:50:38.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.716+0000 7f401ffff640 10 client.26171 handle_command_reply: tid=0 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 client.26171 shutdown 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 -- 172.21.15.69:0/2721558373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393178ff0 msgr2=0x558393183c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393178ff0 0x558393183c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f40340146b0 tx=0x7f4034004750 comp rx=0 tx=0).stop 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 -- 172.21.15.69:0/2721558373 shutdown_connections 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558393179910 0x558393180fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558393193e50 0x558393196310 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393181760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 --2- 172.21.15.69:0/2721558373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393178ff0 0x558393183c80 secure :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f40340146b0 tx=0x7f4034004750 comp rx=0 tx=0).stop 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f404941c740 1 -- 172.21.15.69:0/2721558373 >> 172.21.15.69:0/2721558373 conn(0x558393175580 msgr2=0x55839317b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.717+0000 7f401ffff640 0 client.26171 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.718+0000 7f404941c740 1 -- 172.21.15.69:0/2721558373 shutdown_connections 2024-08-02T17:50:38.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.718+0000 7f404941c740 1 -- 172.21.15.69:0/2721558373 wait complete. 2024-08-02T17:50:38.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.718+0000 7f404941c740 20 client.26171 trim_cache size 0 max 16384 2024-08-02T17:50:38.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.737+0000 7f404941c740 1 Processor -- start 2024-08-02T17:50:38.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f404941c740 1 -- start start 2024-08-02T17:50:38.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f404941c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393175960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f404941c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393175ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f404941c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55839317c310 0x55839317c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f404941c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40401d9cd0 con 0x55839317c310 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f404941c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40401d75a0 con 0x558393179910 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f404941c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40401d54f0 con 0x5583931841d0 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f403ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393175ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f4044dba640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393175960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f403ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393175ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:36336/0 (socket says 172.21.15.69:36336) 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f403ffff640 1 -- 172.21.15.69:0/1604604513 learned_addr learned my addr 172.21.15.69:0/1604604513 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.738+0000 7f40455bb640 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55839317c310 0x55839317c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f4044dba640 1 -- 172.21.15.69:0/1604604513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 msgr2=0x558393175ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:38.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f4044dba640 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393175ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f4044dba640 1 -- 172.21.15.69:0/1604604513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55839317c310 msgr2=0x55839317c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f4044dba640 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55839317c310 0x55839317c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f4044dba640 1 -- 172.21.15.69:0/1604604513 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558393179740 con 0x558393179910 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f403ffff640 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393175ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f4044dba640 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393175960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4034016000 tx=0x7f403400dad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb0c53dd607d6395 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f40267fc640 1 -- 172.21.15.69:0/1604604513 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034024040 con 0x558393179910 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f40267fc640 1 -- 172.21.15.69:0/1604604513 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4034004500 con 0x558393179910 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.739+0000 7f40267fc640 1 -- 172.21.15.69:0/1604604513 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4034020760 con 0x558393179910 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 -- 172.21.15.69:0/1604604513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 msgr2=0x558393175960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393175960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4034016000 tx=0x7f403400dad0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 -- 172.21.15.69:0/1604604513 shutdown_connections 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55839317c310 0x55839317c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583931841d0 0x558393175ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 --2- 172.21.15.69:0/1604604513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393175960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 -- 172.21.15.69:0/1604604513 >> 172.21.15.69:0/1604604513 conn(0x558393175580 msgr2=0x5583931701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 -- 172.21.15.69:0/1604604513 shutdown_connections 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 -- 172.21.15.69:0/1604604513 wait complete. 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 -- start start 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.740+0000 7f404941c740 1 Processor -- start 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f404941c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393187950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f404941c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55839317c310 0x558393187e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f404941c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583931841d0 0x5583931f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f404941c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40401d5a60 con 0x5583931841d0 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f404941c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40401d6ed0 con 0x558393179910 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f404941c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40401d6130 con 0x55839317c310 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f404941c740 10 client.0 _ll_register_callbacks cb 0x7f403400d8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f403ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55839317c310 0x558393187e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f403ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55839317c310 0x558393187e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:36340/0 (socket says 172.21.15.69:36340) 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f403ffff640 1 -- 172.21.15.69:0/3665517406 learned_addr learned my addr 172.21.15.69:0/3665517406 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f4044dba640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393187950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f40455bb640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583931841d0 0x5583931f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f403ffff640 1 -- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 msgr2=0x558393187950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:38.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f403ffff640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393187950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f403ffff640 1 -- 172.21.15.69:0/3665517406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583931841d0 msgr2=0x5583931f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f403ffff640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583931841d0 0x5583931f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f403ffff640 1 -- 172.21.15.69:0/3665517406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583931f72d0 con 0x55839317c310 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f40455bb640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583931841d0 0x5583931f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.741+0000 7f4044dba640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393187950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f403ffff640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55839317c310 0x558393187e90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f404006f3a0 tx=0x7f4038003aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b49767c3af5ca0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f401ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f401ffff640 1 -- 172.21.15.69:0/3665517406 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403804a040 con 0x55839317c310 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f401ffff640 1 -- 172.21.15.69:0/3665517406 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f40380307f0 con 0x55839317c310 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f404941c740 1 -- 172.21.15.69:0/3665517406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4034004500 con 0x55839317c310 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f401ffff640 1 -- 172.21.15.69:0/3665517406 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4038030b10 con 0x55839317c310 2024-08-02T17:50:38.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f404941c740 10 client.26065 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f404941c740 1 -- 172.21.15.69:0/3665517406 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583931fe6f0 con 0x55839317c310 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.742+0000 7f401ffff640 1 -- 172.21.15.69:0/3665517406 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4034004500 con 0x55839317c310 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.743+0000 7f401ffff640 1 -- 172.21.15.69:0/3665517406 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4038058070 con 0x55839317c310 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.744+0000 7f404941c740 10 client.26065 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.744+0000 7f404941c740 10 client.26065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.744+0000 7f404941c740 20 client.26065 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.744+0000 7f404941c740 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5583931fea30 0x558393200e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.744+0000 7f404941c740 4 client.26065 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.744+0000 7f404941c740 1 -- 172.21.15.69:0/3665517406 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5583930ebfb0 con 0x5583931fea30 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.744+0000 7f4044dba640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5583931fea30 0x558393200e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f401ffff640 10 client.26065 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f4044dba640 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5583931fea30 0x558393200e20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e956c33e5900eb67 server_cookie=9a63ef49c458734d in_seq=0 out_seq=0 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f401ffff640 1 -- 172.21.15.69:0/3665517406 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5583930ebfb0 con 0x5583931fea30 2024-08-02T17:50:38.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f401ffff640 10 client.26065 handle_command_reply: tid=0 2024-08-02T17:50:38.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 client.26065 shutdown 2024-08-02T17:50:38.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 -- 172.21.15.69:0/3665517406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55839317c310 msgr2=0x558393187e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55839317c310 0x558393187e90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f404006f3a0 tx=0x7f4038003aa0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 -- 172.21.15.69:0/3665517406 shutdown_connections 2024-08-02T17:50:38.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583931841d0 0x5583931f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5583931fea30 0x558393200e20 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f401ffff640 0 client.26065 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:38.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55839317c310 0x558393187e90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 --2- 172.21.15.69:0/3665517406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558393179910 0x558393187950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 -- 172.21.15.69:0/3665517406 >> 172.21.15.69:0/3665517406 conn(0x558393175580 msgr2=0x558393175960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:38.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 -- 172.21.15.69:0/3665517406 shutdown_connections 2024-08-02T17:50:38.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.745+0000 7f404941c740 1 -- 172.21.15.69:0/3665517406 wait complete. 2024-08-02T17:50:38.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404941c740 20 client.26065 trim_cache size 0 max 16384 2024-08-02T17:50:38.757 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 -- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40280a2d60 msgr2=0x7f40280a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40280a2d60 0x7f40280a5220 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4034006fd0 tx=0x7f4034008040 comp rx=0 tx=0).stop 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 -- 172.21.15.69:0/1900391105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4040109b30 msgr2=0x7f40400765e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4040109b30 0x7f40400765e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f403000b3e0 tx=0x7f403000b8a0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 -- 172.21.15.69:0/1900391105 shutdown_connections 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40280a2d60 0x7f40280a5220 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4040109b30 0x7f40400765e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4040109180 0x7f4040076060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 --2- 172.21.15.69:0/1900391105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40400746c0 0x7f404007afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.746+0000 7f404701e640 1 -- 172.21.15.69:0/1900391105 >> 172.21.15.69:0/1900391105 conn(0x7f404006de70 msgr2=0x7f40400722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.747+0000 7f404701e640 1 -- 172.21.15.69:0/1900391105 shutdown_connections 2024-08-02T17:50:38.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:38.747+0000 7f404701e640 1 -- 172.21.15.69:0/1900391105 wait complete. 2024-08-02T17:50:39.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:38 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:50:39.277 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:38 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:50:39.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:38 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:50:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 210.0 s, 1733.4 tps, lat 57.426 ms stddev 64.219 2024-08-02T17:50:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:39 smithi083 ceph-mon[37031]: pgmap v130: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-08-02T17:50:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:39 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:50:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:39 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:50:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:39 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:50:40.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:39 smithi077 ceph-mon[37108]: pgmap v130: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-08-02T17:50:40.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:39 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:50:40.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:39 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:50:40.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:39 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:50:40.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:39 smithi069 ceph-mon[32746]: pgmap v130: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.3 MiB/s wr, 624 op/s 2024-08-02T17:50:40.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:39 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:50:40.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:39 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:50:40.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:39 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:50:40.577 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:50:40.577 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:50:41.010 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:41.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:41 smithi077 ceph-mon[37108]: pgmap v131: 129 pgs: 129 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.6 MiB/s wr, 661 op/s 2024-08-02T17:50:41.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:41 smithi069 ceph-mon[32746]: pgmap v131: 129 pgs: 129 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.6 MiB/s wr, 661 op/s 2024-08-02T17:50:41.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:41 smithi083 ceph-mon[37031]: pgmap v131: 129 pgs: 129 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 6.6 MiB/s wr, 661 op/s 2024-08-02T17:50:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:44 smithi069 ceph-mon[32746]: pgmap v132: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.3 MiB/s wr, 819 op/s 2024-08-02T17:50:44.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:44 smithi077 ceph-mon[37108]: pgmap v132: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.3 MiB/s wr, 819 op/s 2024-08-02T17:50:44.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:44 smithi083 ceph-mon[37031]: pgmap v132: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 8.3 MiB/s wr, 819 op/s 2024-08-02T17:50:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.912+0000 7f733e6e5640 1 -- 172.21.15.69:0/1067182537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73381007b0 msgr2=0x7f7338100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.912+0000 7f733e6e5640 1 --2- 172.21.15.69:0/1067182537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73381007b0 0x7f7338100c10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f732c009930 tx=0x7f732c02a420 comp rx=0 tx=0).stop 2024-08-02T17:50:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.913+0000 7f733e6e5640 1 -- 172.21.15.69:0/1067182537 shutdown_connections 2024-08-02T17:50:44.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.913+0000 7f733e6e5640 1 --2- 172.21.15.69:0/1067182537 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73381007b0 0x7f7338100c10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.913+0000 7f733e6e5640 1 --2- 172.21.15.69:0/1067182537 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7338101150 0x7f733810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.913+0000 7f733e6e5640 1 --2- 172.21.15.69:0/1067182537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73381067b0 0x7f7338106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.913+0000 7f733e6e5640 1 -- 172.21.15.69:0/1067182537 >> 172.21.15.69:0/1067182537 conn(0x7f73380fc4d0 msgr2=0x7f73380fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:44.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.913+0000 7f733e6e5640 1 -- 172.21.15.69:0/1067182537 shutdown_connections 2024-08-02T17:50:44.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.913+0000 7f733e6e5640 1 -- 172.21.15.69:0/1067182537 wait complete. 2024-08-02T17:50:44.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.914+0000 7f733e6e5640 1 Processor -- start 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.914+0000 7f733e6e5640 1 -- start start 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.914+0000 7f733e6e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73381007b0 0x7f7338109b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f733e6e5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7338101150 0x7f733810a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f733e6e5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73381067b0 0x7f733810a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f733e6e5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f733810faf0 con 0x7f73381067b0 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f733e6e5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f733810f970 con 0x7f7338101150 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f7337fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73381007b0 0x7f7338109b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f7337fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73381007b0 0x7f7338109b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33914/0 (socket says 172.21.15.69:33914) 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f73377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7338101150 0x7f733810a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f7337fff640 1 -- 172.21.15.69:0/2576094168 learned_addr learned my addr 172.21.15.69:0/2576094168 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f733810fc70 con 0x7f73381007b0 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f73377fe640 1 -- 172.21.15.69:0/2576094168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73381007b0 msgr2=0x7f7338109b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f73377fe640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73381007b0 0x7f7338109b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f73377fe640 1 -- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73381067b0 msgr2=0x7f733810a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f73377fe640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73381067b0 0x7f733810a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f73377fe640 1 -- 172.21.15.69:0/2576094168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73381aaa60 con 0x7f7338101150 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.915+0000 7f73377fe640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7338101150 0x7f733810a050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f732c009850 tx=0x7f732c003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.916+0000 7f73357fa640 1 -- 172.21.15.69:0/2576094168 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c03f070 con 0x7f7338101150 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.916+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73381aacf0 con 0x7f7338101150 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.916+0000 7f73357fa640 1 -- 172.21.15.69:0/2576094168 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732c031c90 con 0x7f7338101150 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.916+0000 7f73357fa640 1 -- 172.21.15.69:0/2576094168 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c035030 con 0x7f7338101150 2024-08-02T17:50:44.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.916+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73381ab1f0 con 0x7f7338101150 2024-08-02T17:50:44.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.920+0000 7f73357fa640 1 -- 172.21.15.69:0/2576094168 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f732c042070 con 0x7f7338101150 2024-08-02T17:50:44.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.920+0000 7f7340ae3740 1 Processor -- start 2024-08-02T17:50:44.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.920+0000 7f7340ae3740 1 -- start start 2024-08-02T17:50:44.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675200 0x55ef4d6755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675b20 0x55ef4d67fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d6803e0 0x55ef4d6827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7340ae3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ef4d638e40 con 0x55ef4d6803e0 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7340ae3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef4d5da7f0 con 0x55ef4d675200 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7337fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675b20 0x55ef4d67fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7337fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675b20 0x55ef4d67fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33926/0 (socket says 172.21.15.69:33926) 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7337fff640 1 -- 172.21.15.69:0/2499993067 learned_addr learned my addr 172.21.15.69:0/2499993067 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.921+0000 7f7340ae3740 1 -- 172.21.15.69:0/2499993067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef4d581fb0 con 0x55ef4d675b20 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.922+0000 7f7337fff640 1 -- 172.21.15.69:0/2499993067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675200 msgr2=0x55ef4d6755e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.922+0000 7f7337fff640 1 --2- 172.21.15.69:0/2499993067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675200 0x55ef4d6755e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.922+0000 7f7337fff640 1 -- 172.21.15.69:0/2499993067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d6803e0 msgr2=0x55ef4d6827d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.922+0000 7f7337fff640 1 --2- 172.21.15.69:0/2499993067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d6803e0 0x55ef4d6827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.922+0000 7f7337fff640 1 -- 172.21.15.69:0/2499993067 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef4d34c6a0 con 0x55ef4d675b20 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.923+0000 7f7337fff640 1 --2- 172.21.15.69:0/2499993067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675b20 0x55ef4d67fea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7328009d20 tx=0x7f7328011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eefc8ed8521ddb5c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.923+0000 7f73357fa640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73200a2dc0 0x7f73200a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.923+0000 7f73357fa640 1 -- 172.21.15.69:0/2576094168 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f732c0efe80 con 0x7f7338101150 2024-08-02T17:50:44.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.923+0000 7f733cc82640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73200a2dc0 0x7f73200a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.924+0000 7f731e7fc640 1 -- 172.21.15.69:0/2499993067 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732802a620 con 0x55ef4d675b20 2024-08-02T17:50:44.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.924+0000 7f731e7fc640 1 -- 172.21.15.69:0/2499993067 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732802aca0 con 0x55ef4d675b20 2024-08-02T17:50:44.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.924+0000 7f731e7fc640 1 -- 172.21.15.69:0/2499993067 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7328029cf0 con 0x55ef4d675b20 2024-08-02T17:50:44.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 -- 172.21.15.69:0/2499993067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675b20 msgr2=0x55ef4d67fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 --2- 172.21.15.69:0/2499993067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675b20 0x55ef4d67fea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7328009d20 tx=0x7f7328011810 comp rx=0 tx=0).stop 2024-08-02T17:50:44.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f733cc82640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73200a2dc0 0x7f73200a5280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7330005c40 tx=0x7f7330005bd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:44.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 -- 172.21.15.69:0/2499993067 shutdown_connections 2024-08-02T17:50:44.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 --2- 172.21.15.69:0/2499993067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d6803e0 0x55ef4d6827d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 --2- 172.21.15.69:0/2499993067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675b20 0x55ef4d67fea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 --2- 172.21.15.69:0/2499993067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675200 0x55ef4d6755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 -- 172.21.15.69:0/2499993067 >> 172.21.15.69:0/2499993067 conn(0x55ef4d671790 msgr2=0x55ef4d671b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 -- 172.21.15.69:0/2499993067 shutdown_connections 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 -- 172.21.15.69:0/2499993067 wait complete. 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.925+0000 7f7340ae3740 1 -- start start 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7340ae3740 1 Processor -- start 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d67d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d67dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7340ae3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ef4d595cb0 con 0x55ef4d675b20 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7340ae3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ef4d5d67b0 con 0x55ef4d6803e0 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7340ae3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ef4d682fb0 con 0x55ef4d675200 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7337fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f733cc82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d67dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7337fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33938/0 (socket says 172.21.15.69:33938) 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7340ae3740 10 client.0 _ll_register_callbacks cb 0x55ef4d2e9ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f7337fff640 1 -- 172.21.15.69:0/1854690390 learned_addr learned my addr 172.21.15.69:0/1854690390 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f733cc82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d67dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60032/0 (socket says 172.21.15.69:60032) 2024-08-02T17:50:44.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f73377fe640 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d67d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f733cc82640 1 -- 172.21.15.69:0/1854690390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 msgr2=0x55ef4d67d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f733cc82640 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f733cc82640 1 -- 172.21.15.69:0/1854690390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 msgr2=0x55ef4d67d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f733cc82640 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d67d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.926+0000 7f733cc82640 1 -- 172.21.15.69:0/1854690390 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef4d594bc0 con 0x55ef4d6803e0 2024-08-02T17:50:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.927+0000 7f7337fff640 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.927+0000 7f733cc82640 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d67dd70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7330005fd0 tx=0x7f7330004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4f2593ed1cf0bb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:44.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.927+0000 7f73177fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.927+0000 7f73177fe640 1 -- 172.21.15.69:0/1854690390 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330020070 con 0x55ef4d6803e0 2024-08-02T17:50:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.927+0000 7f73177fe640 1 -- 172.21.15.69:0/1854690390 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f733000fb40 con 0x55ef4d6803e0 2024-08-02T17:50:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.927+0000 7f7340ae3740 1 -- 172.21.15.69:0/1854690390 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f732802aca0 con 0x55ef4d6803e0 2024-08-02T17:50:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.928+0000 7f73177fe640 1 -- 172.21.15.69:0/1854690390 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f733002e050 con 0x55ef4d6803e0 2024-08-02T17:50:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.928+0000 7f7340ae3740 10 client.26186 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.928+0000 7f7340ae3740 1 -- 172.21.15.69:0/1854690390 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef4d68ef80 con 0x55ef4d6803e0 2024-08-02T17:50:44.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.928+0000 7f73177fe640 1 -- 172.21.15.69:0/1854690390 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733000fd40 con 0x55ef4d6803e0 2024-08-02T17:50:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.929+0000 7f73177fe640 1 -- 172.21.15.69:0/1854690390 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f733001c5e0 con 0x55ef4d6803e0 2024-08-02T17:50:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.930+0000 7f7340ae3740 10 client.26186 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.930+0000 7f7340ae3740 10 client.26186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.930+0000 7f7340ae3740 20 client.26186 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.930+0000 7f7340ae3740 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ef4d690440 0x55ef4d692830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.930+0000 7f7340ae3740 4 client.26186 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.930+0000 7f7340ae3740 1 -- 172.21.15.69:0/1854690390 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f732802aca0 con 0x55ef4d690440 2024-08-02T17:50:44.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.930+0000 7f7337fff640 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ef4d690440 0x55ef4d692830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.931+0000 7f7337fff640 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ef4d690440 0x55ef4d692830 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26ee0c1f1104094 server_cookie=ce67a95ab5a3e99 in_seq=0 out_seq=0 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.931+0000 7f73177fe640 10 client.26186 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.931+0000 7f73177fe640 1 -- 172.21.15.69:0/1854690390 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f732802aca0 con 0x55ef4d690440 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f73177fe640 10 client.26186 handle_command_reply: tid=0 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 client.26186 shutdown 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 -- 172.21.15.69:0/1854690390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 msgr2=0x55ef4d67dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d67dd70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7330005fd0 tx=0x7f7330004a40 comp rx=0 tx=0).stop 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 -- 172.21.15.69:0/1854690390 shutdown_connections 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d67d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ef4d690440 0x55ef4d692830 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f73177fe640 0 client.26186 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:44.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d67dd70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 --2- 172.21.15.69:0/1854690390 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.932+0000 7f7340ae3740 1 -- 172.21.15.69:0/1854690390 >> 172.21.15.69:0/1854690390 conn(0x55ef4d671790 msgr2=0x55ef4d677470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.933+0000 7f7340ae3740 1 -- 172.21.15.69:0/1854690390 shutdown_connections 2024-08-02T17:50:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.933+0000 7f7340ae3740 1 -- 172.21.15.69:0/1854690390 wait complete. 2024-08-02T17:50:44.939 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.933+0000 7f7340ae3740 20 client.26186 trim_cache size 0 max 16384 2024-08-02T17:50:44.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 215.0 s, 2094.4 tps, lat 48.229 ms stddev 66.059 2024-08-02T17:50:44.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7340ae3740 1 Processor -- start 2024-08-02T17:50:44.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7340ae3740 1 -- start start 2024-08-02T17:50:44.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d698010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d698550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7340ae3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f73381af530 con 0x55ef4d675b20 2024-08-02T17:50:44.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7340ae3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f73381ace00 con 0x55ef4d6803e0 2024-08-02T17:50:44.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7340ae3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f733811d1b0 con 0x55ef4d675200 2024-08-02T17:50:44.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f73377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d698010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f73377fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d698010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55554/0 (socket says 172.21.15.69:55554) 2024-08-02T17:50:44.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f73377fe640 1 -- 172.21.15.69:0/899247141 learned_addr learned my addr 172.21.15.69:0/899247141 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:44.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f733cc82640 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d698550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f7337fff640 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f73377fe640 1 -- 172.21.15.69:0/899247141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 msgr2=0x55ef4d67dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:44.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f73377fe640 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f73377fe640 1 -- 172.21.15.69:0/899247141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 msgr2=0x55ef4d698550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f73377fe640 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d698550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.960+0000 7f73377fe640 1 -- 172.21.15.69:0/899247141 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef4d692ba0 con 0x55ef4d675b20 2024-08-02T17:50:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.961+0000 7f73377fe640 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d698010 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f732c0ed270 tx=0x7f732c0bbbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80910da7e3ecf856 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f731dffb640 1 -- 172.21.15.69:0/899247141 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c049070 con 0x55ef4d675b20 2024-08-02T17:50:44.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f731dffb640 1 -- 172.21.15.69:0/899247141 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732c0bc6e0 con 0x55ef4d675b20 2024-08-02T17:50:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f731dffb640 1 -- 172.21.15.69:0/899247141 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0b8650 con 0x55ef4d675b20 2024-08-02T17:50:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f7340ae3740 1 -- 172.21.15.69:0/899247141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 msgr2=0x55ef4d698010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f7340ae3740 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d698010 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f732c0ed270 tx=0x7f732c0bbbe0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f7340ae3740 1 -- 172.21.15.69:0/899247141 shutdown_connections 2024-08-02T17:50:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f7340ae3740 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675b20 0x55ef4d698010 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f7340ae3740 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d6803e0 0x55ef4d698550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f7340ae3740 1 --2- 172.21.15.69:0/899247141 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d675200 0x55ef4d67dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.962+0000 7f7340ae3740 1 -- 172.21.15.69:0/899247141 >> 172.21.15.69:0/899247141 conn(0x55ef4d671790 msgr2=0x55ef4d67f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 -- 172.21.15.69:0/899247141 shutdown_connections 2024-08-02T17:50:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 -- 172.21.15.69:0/899247141 wait complete. 2024-08-02T17:50:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 -- start start 2024-08-02T17:50:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 Processor -- start 2024-08-02T17:50:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675200 0x55ef4d67a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675b20 0x55ef4d67a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d6803e0 0x55ef4d67adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f733811d820 con 0x55ef4d675200 2024-08-02T17:50:44.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f73381ac730 con 0x55ef4d675b20 2024-08-02T17:50:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73381ab990 con 0x55ef4d6803e0 2024-08-02T17:50:44.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.963+0000 7f7340ae3740 10 client.0 _ll_register_callbacks cb 0x7f7330010500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f73377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675b20 0x55ef4d67a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f73377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675b20 0x55ef4d67a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60056/0 (socket says 172.21.15.69:60056) 2024-08-02T17:50:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f73377fe640 1 -- 172.21.15.69:0/326877630 learned_addr learned my addr 172.21.15.69:0/326877630 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f7337fff640 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675200 0x55ef4d67a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f733cc82640 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d6803e0 0x55ef4d67adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f733cc82640 1 -- 172.21.15.69:0/326877630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675b20 msgr2=0x55ef4d67a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f733cc82640 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675b20 0x55ef4d67a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f733cc82640 1 -- 172.21.15.69:0/326877630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675200 msgr2=0x55ef4d67a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f733cc82640 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675200 0x55ef4d67a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.964+0000 7f733cc82640 1 -- 172.21.15.69:0/326877630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef4d6fa010 con 0x55ef4d6803e0 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.965+0000 7f733cc82640 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d6803e0 0x55ef4d67adc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f733000a010 tx=0x7f733001f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0df3b069bbeca1e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.965+0000 7f73177fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.965+0000 7f73177fe640 1 -- 172.21.15.69:0/326877630 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330041060 con 0x55ef4d6803e0 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.965+0000 7f73177fe640 1 -- 172.21.15.69:0/326877630 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f733000fa50 con 0x55ef4d6803e0 2024-08-02T17:50:44.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.965+0000 7f7340ae3740 1 -- 172.21.15.69:0/326877630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f732c0bc6e0 con 0x55ef4d6803e0 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.966+0000 7f73177fe640 1 -- 172.21.15.69:0/326877630 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7330048050 con 0x55ef4d6803e0 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.966+0000 7f7340ae3740 10 client.26083 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.966+0000 7f7340ae3740 1 -- 172.21.15.69:0/326877630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef4d6fb310 con 0x55ef4d6803e0 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.966+0000 7f73177fe640 1 -- 172.21.15.69:0/326877630 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732c0bc6e0 con 0x55ef4d6803e0 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.967+0000 7f73177fe640 1 -- 172.21.15.69:0/326877630 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7330025940 con 0x55ef4d6803e0 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.967+0000 7f7340ae3740 10 client.26083 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.967+0000 7f7340ae3740 10 client.26083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.967+0000 7f7340ae3740 20 client.26083 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.967+0000 7f7340ae3740 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ef4d690440 0x55ef4d6fd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.967+0000 7f7340ae3740 4 client.26083 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.967+0000 7f7340ae3740 1 -- 172.21.15.69:0/326877630 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ef4d5e7d60 con 0x55ef4d690440 2024-08-02T17:50:44.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.967+0000 7f7337fff640 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ef4d690440 0x55ef4d6fd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.968+0000 7f7337fff640 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ef4d690440 0x55ef4d6fd9e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4158f2cb67f6e34 server_cookie=6440ee02349181a8 in_seq=0 out_seq=0 2024-08-02T17:50:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.968+0000 7f73177fe640 10 client.26083 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.975+0000 7f73177fe640 1 -- 172.21.15.69:0/326877630 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ef4d5e7d60 con 0x55ef4d690440 2024-08-02T17:50:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.975+0000 7f73177fe640 10 client.26083 handle_command_reply: tid=0 2024-08-02T17:50:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.975+0000 7f7340ae3740 1 client.26083 shutdown 2024-08-02T17:50:44.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.985+0000 7f7340ae3740 1 -- 172.21.15.69:0/326877630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d6803e0 msgr2=0x55ef4d67adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.985+0000 7f7340ae3740 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d6803e0 0x55ef4d67adc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f733000a010 tx=0x7f733001f000 comp rx=0 tx=0).stop 2024-08-02T17:50:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.987+0000 7f7340ae3740 1 -- 172.21.15.69:0/326877630 shutdown_connections 2024-08-02T17:50:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.987+0000 7f7340ae3740 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ef4d690440 0x55ef4d6fd9e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.987+0000 7f7340ae3740 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ef4d6803e0 0x55ef4d67adc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.987+0000 7f7340ae3740 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ef4d675b20 0x55ef4d67a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.987+0000 7f7340ae3740 1 --2- 172.21.15.69:0/326877630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ef4d675200 0x55ef4d67a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.987+0000 7f7340ae3740 1 -- 172.21.15.69:0/326877630 >> 172.21.15.69:0/326877630 conn(0x55ef4d671790 msgr2=0x55ef4d676c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:44.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.988+0000 7f73177fe640 0 client.26083 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:44.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.993+0000 7f7340ae3740 1 -- 172.21.15.69:0/326877630 shutdown_connections 2024-08-02T17:50:44.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.993+0000 7f7340ae3740 1 -- 172.21.15.69:0/326877630 wait complete. 2024-08-02T17:50:44.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.994+0000 7f7340ae3740 20 client.26083 trim_cache size 0 max 16384 2024-08-02T17:50:44.995 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:50:44.995 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:50:44.995 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "4b3cd9fe-932a-40af-853f-794c5cf3c1f4", 2024-08-02T17:50:44.995 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:50:44.996 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.994+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73200a2dc0 msgr2=0x7f73200a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.994+0000 7f733e6e5640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73200a2dc0 0x7f73200a5280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7330005c40 tx=0x7f7330005bd0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.994+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7338101150 msgr2=0x7f733810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.994+0000 7f733e6e5640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7338101150 0x7f733810a050 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f732c009850 tx=0x7f732c003b90 comp rx=0 tx=0).stop 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.995+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 shutdown_connections 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.995+0000 7f733e6e5640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73381067b0 0x7f733810a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.995+0000 7f733e6e5640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73200a2dc0 0x7f73200a5280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.995+0000 7f733e6e5640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7338101150 0x7f733810a050 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.995+0000 7f733e6e5640 1 --2- 172.21.15.69:0/2576094168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73381007b0 0x7f7338109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.995+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 >> 172.21.15.69:0/2576094168 conn(0x7f73380fc4d0 msgr2=0x7f73380fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:44.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.996+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 shutdown_connections 2024-08-02T17:50:44.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:44.996+0000 7f733e6e5640 1 -- 172.21.15.69:0/2576094168 wait complete. 2024-08-02T17:50:45.821 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:50:46.254 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:50:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:46 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:50:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:46 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:46.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:46 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:50:46.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:46 smithi077 ceph-mon[37108]: pgmap v133: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 550 op/s 2024-08-02T17:50:46.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:46 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:46 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:50:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:46 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:46 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:50:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:46 smithi069 ceph-mon[32746]: pgmap v133: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 550 op/s 2024-08-02T17:50:46.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:46 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:46.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:46 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:50:46.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:46 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:46.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:46 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:50:46.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:46 smithi083 ceph-mon[37031]: pgmap v133: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 MiB/s wr, 550 op/s 2024-08-02T17:50:46.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:46 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:50:47.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.124+0000 7f8d88f39640 1 -- 172.21.15.69:0/3411923302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d8411ae70 msgr2=0x7f8d8411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.124+0000 7f8d88f39640 1 --2- 172.21.15.69:0/3411923302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d8411ae70 0x7f8d8411d260 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009be0 tx=0x7f8d7c02a6f0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.124+0000 7f8d88f39640 1 -- 172.21.15.69:0/3411923302 shutdown_connections 2024-08-02T17:50:47.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.124+0000 7f8d88f39640 1 --2- 172.21.15.69:0/3411923302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d8411ae70 0x7f8d8411d260 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.124+0000 7f8d88f39640 1 --2- 172.21.15.69:0/3411923302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d84109560 0x7f8d841099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.124+0000 7f8d88f39640 1 --2- 172.21.15.69:0/3411923302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d84109180 0x7f8d8410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.124+0000 7f8d88f39640 1 -- 172.21.15.69:0/3411923302 >> 172.21.15.69:0/3411923302 conn(0x7f8d8406de70 msgr2=0x7f8d8406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:47.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.125+0000 7f8d88f39640 1 -- 172.21.15.69:0/3411923302 shutdown_connections 2024-08-02T17:50:47.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.125+0000 7f8d88f39640 1 -- 172.21.15.69:0/3411923302 wait complete. 2024-08-02T17:50:47.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.126+0000 7f8d88f39640 1 Processor -- start 2024-08-02T17:50:47.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.126+0000 7f8d88f39640 1 -- start start 2024-08-02T17:50:47.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.126+0000 7f8d88f39640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d84109180 0x7f8d84118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.126+0000 7f8d88f39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d84109560 0x7f8d84110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.126+0000 7f8d88f39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d8411ae70 0x7f8d84110c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.126+0000 7f8d88f39640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84120b40 con 0x7f8d84109180 2024-08-02T17:50:47.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.126+0000 7f8d88f39640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841209c0 con 0x7f8d8411ae70 2024-08-02T17:50:47.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.126+0000 7f8d88f39640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84120cc0 con 0x7f8d84109560 2024-08-02T17:50:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.127+0000 7f8d8259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d84109180 0x7f8d84118750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.128+0000 7f8d8259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d84109180 0x7f8d84118750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55580/0 (socket says 172.21.15.69:55580) 2024-08-02T17:50:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.128+0000 7f8d8259c640 1 -- 172.21.15.69:0/861658514 learned_addr learned my addr 172.21.15.69:0/861658514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.128+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d84109560 0x7f8d84110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.128+0000 7f8d8259c640 1 -- 172.21.15.69:0/861658514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d84109560 msgr2=0x7f8d84110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.128+0000 7f8d8259c640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d84109560 0x7f8d84110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.128+0000 7f8d8259c640 1 -- 172.21.15.69:0/861658514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d8411ae70 msgr2=0x7f8d84110c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.128+0000 7f8d8259c640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d8411ae70 0x7f8d84110c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.128+0000 7f8d8259c640 1 -- 172.21.15.69:0/861658514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d84111480 con 0x7f8d84109180 2024-08-02T17:50:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.129+0000 7f8d8259c640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d84109180 0x7f8d84118750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400e9c0 tx=0x7f8d7400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.129+0000 7f8d7b7fe640 1 -- 172.21.15.69:0/861658514 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400ccc0 con 0x7f8d84109180 2024-08-02T17:50:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.129+0000 7f8d7b7fe640 1 -- 172.21.15.69:0/861658514 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74005500 con 0x7f8d84109180 2024-08-02T17:50:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.129+0000 7f8d7b7fe640 1 -- 172.21.15.69:0/861658514 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74010430 con 0x7f8d84109180 2024-08-02T17:50:47.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.130+0000 7f8d88f39640 1 -- 172.21.15.69:0/861658514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d841bbdb0 con 0x7f8d84109180 2024-08-02T17:50:47.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.130+0000 7f8d88f39640 1 -- 172.21.15.69:0/861658514 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d841bc270 con 0x7f8d84109180 2024-08-02T17:50:47.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.137+0000 7f8d8b337740 1 Processor -- start 2024-08-02T17:50:47.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.142+0000 7f8d8b337740 1 -- start start 2024-08-02T17:50:47.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.142+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732a210 0x563f8732a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.142+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f87334eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.142+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 0x563f873377e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d8b337740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563f8728f980 con 0x563f873353f0 2024-08-02T17:50:47.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d8b337740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563f87227a70 con 0x563f8732ab30 2024-08-02T17:50:47.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d82d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732a210 0x563f8732a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d82d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732a210 0x563f8732a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33976/0 (socket says 172.21.15.69:33976) 2024-08-02T17:50:47.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d82d9d640 1 -- 172.21.15.69:0/1948720470 learned_addr learned my addr 172.21.15.69:0/1948720470 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:47.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d8b337740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563f8728b940 con 0x563f8732a210 2024-08-02T17:50:47.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/1948720470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 0x563f873377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d82d9d640 1 -- 172.21.15.69:0/1948720470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 msgr2=0x563f87334eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:50:47.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/1948720470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f87334eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d82d9d640 1 -- 172.21.15.69:0/1948720470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 msgr2=0x563f873377e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:47.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/1948720470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 0x563f873377e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.143+0000 7f8d82d9d640 1 -- 172.21.15.69:0/1948720470 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8728a6e0 con 0x563f8732a210 2024-08-02T17:50:47.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.144+0000 7f8d7b7fe640 1 -- 172.21.15.69:0/861658514 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8d74004fb0 con 0x7f8d84109180 2024-08-02T17:50:47.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.145+0000 7f8d7b7fe640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d700a2f00 0x7f8d700a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.146+0000 7f8d7b7fe640 1 -- 172.21.15.69:0/861658514 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8d740cfce0 con 0x7f8d84109180 2024-08-02T17:50:47.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.146+0000 7f8d8259c640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d700a2f00 0x7f8d700a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.146+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/1948720470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732a210 0x563f8732a5f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0314b0 tx=0x7f8d7c0314e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6591e503062517 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:47.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.147+0000 7f8d78ff9640 1 -- 172.21.15.69:0/1948720470 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c041020 con 0x563f8732a210 2024-08-02T17:50:47.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.147+0000 7f8d8259c640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d700a2f00 0x7f8d700a53c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x563f87327600 tx=0x7f8d7400eef0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:47.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.147+0000 7f8d78ff9640 1 -- 172.21.15.69:0/1948720470 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c030b80 con 0x563f8732a210 2024-08-02T17:50:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.147+0000 7f8d78ff9640 1 -- 172.21.15.69:0/1948720470 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c036040 con 0x563f8732a210 2024-08-02T17:50:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.148+0000 7f8d8b337740 1 -- 172.21.15.69:0/1948720470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732a210 msgr2=0x563f8732a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.148+0000 7f8d8b337740 1 --2- 172.21.15.69:0/1948720470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732a210 0x563f8732a5f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0314b0 tx=0x7f8d7c0314e0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.148+0000 7f8d8b337740 1 -- 172.21.15.69:0/1948720470 shutdown_connections 2024-08-02T17:50:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.148+0000 7f8d8b337740 1 --2- 172.21.15.69:0/1948720470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 0x563f873377e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.148+0000 7f8d8b337740 1 --2- 172.21.15.69:0/1948720470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f87334eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.148+0000 7f8d8b337740 1 --2- 172.21.15.69:0/1948720470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732a210 0x563f8732a5f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.149+0000 7f8d8b337740 1 -- 172.21.15.69:0/1948720470 >> 172.21.15.69:0/1948720470 conn(0x563f873267a0 msgr2=0x563f87326b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:47.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.149+0000 7f8d8b337740 1 -- 172.21.15.69:0/1948720470 shutdown_connections 2024-08-02T17:50:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.150+0000 7f8d8b337740 1 -- 172.21.15.69:0/1948720470 wait complete. 2024-08-02T17:50:47.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.150+0000 7f8d8b337740 1 -- start start 2024-08-02T17:50:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.150+0000 7f8d8b337740 1 Processor -- start 2024-08-02T17:50:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 0x563f87332270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f873327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f87332cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d8b337740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563f87338010 con 0x563f8732a210 2024-08-02T17:50:47.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d8b337740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563f8724aaa0 con 0x563f8732ab30 2024-08-02T17:50:47.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d8b337740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563f87338190 con 0x563f873353f0 2024-08-02T17:50:47.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d8b337740 10 client.0 _ll_register_callbacks cb 0x563f87239e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:47.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d81d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f873327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d81d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f873327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60074/0 (socket says 172.21.15.69:60074) 2024-08-02T17:50:47.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.151+0000 7f8d81d9b640 1 -- 172.21.15.69:0/2295176091 learned_addr learned my addr 172.21.15.69:0/2295176091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:47.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.153+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 0x563f87332270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.153+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f87332cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.156+0000 7f8d81d9b640 1 -- 172.21.15.69:0/2295176091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 msgr2=0x563f87332cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:47.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.156+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f87332cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.156+0000 7f8d81d9b640 1 -- 172.21.15.69:0/2295176091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 msgr2=0x563f87332270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:47.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.156+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 0x563f87332270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.156+0000 7f8d81d9b640 1 -- 172.21.15.69:0/2295176091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8722e650 con 0x563f8732ab30 2024-08-02T17:50:47.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.156+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f873327b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009f00 tx=0x7f8d6c011630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac902c77a88377a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:47.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.157+0000 7f8d69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:47.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.157+0000 7f8d69ffb640 1 -- 172.21.15.69:0/2295176091 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c019070 con 0x563f8732ab30 2024-08-02T17:50:47.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.157+0000 7f8d69ffb640 1 -- 172.21.15.69:0/2295176091 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8d6c015660 con 0x563f8732ab30 2024-08-02T17:50:47.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.157+0000 7f8d8b337740 1 -- 172.21.15.69:0/2295176091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d7c030b80 con 0x563f8732ab30 2024-08-02T17:50:47.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.158+0000 7f8d69ffb640 1 -- 172.21.15.69:0/2295176091 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c025050 con 0x563f8732ab30 2024-08-02T17:50:47.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.158+0000 7f8d8b337740 10 client.26198 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:47.159 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.158+0000 7f8d8b337740 1 -- 172.21.15.69:0/2295176091 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f870013c0 con 0x563f8732ab30 2024-08-02T17:50:47.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.159+0000 7f8d69ffb640 1 -- 172.21.15.69:0/2295176091 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c030b80 con 0x563f8732ab30 2024-08-02T17:50:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.159+0000 7f8d69ffb640 1 -- 172.21.15.69:0/2295176091 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8d6c014b10 con 0x563f8732ab30 2024-08-02T17:50:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.159+0000 7f8d8b337740 10 client.26198 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.159+0000 7f8d8b337740 10 client.26198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.159+0000 7f8d8b337740 20 client.26198 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.159+0000 7f8d8b337740 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563f87344eb0 0x563f873472e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.159+0000 7f8d8b337740 4 client.26198 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:47.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.159+0000 7f8d8b337740 1 -- 172.21.15.69:0/2295176091 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f8729cef0 con 0x563f87344eb0 2024-08-02T17:50:47.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.160+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563f87344eb0 0x563f873472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.161+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563f87344eb0 0x563f873472e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ce476db2306464 server_cookie=bbecd684843e9dc1 in_seq=0 out_seq=0 2024-08-02T17:50:47.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.161+0000 7f8d69ffb640 10 client.26198 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:47.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.162+0000 7f8d69ffb640 1 -- 172.21.15.69:0/2295176091 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563f8729cef0 con 0x563f87344eb0 2024-08-02T17:50:47.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.162+0000 7f8d69ffb640 10 client.26198 handle_command_reply: tid=0 2024-08-02T17:50:47.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.162+0000 7f8d8b337740 1 client.26198 shutdown 2024-08-02T17:50:47.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.162+0000 7f8d8b337740 1 -- 172.21.15.69:0/2295176091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 msgr2=0x563f873327b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.162+0000 7f8d8b337740 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f873327b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009f00 tx=0x7f8d6c011630 comp rx=0 tx=0).stop 2024-08-02T17:50:47.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.168+0000 7f8d8b337740 1 -- 172.21.15.69:0/2295176091 shutdown_connections 2024-08-02T17:50:47.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.168+0000 7f8d8b337740 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563f87344eb0 0x563f873472e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.168+0000 7f8d8b337740 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f87332cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.168+0000 7f8d8b337740 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f873327b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.168+0000 7f8d8b337740 1 --2- 172.21.15.69:0/2295176091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 0x563f87332270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.168+0000 7f8d8b337740 1 -- 172.21.15.69:0/2295176091 >> 172.21.15.69:0/2295176091 conn(0x563f873267a0 msgr2=0x563f87325900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:47.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.169+0000 7f8d69ffb640 0 client.26198 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:47.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.171+0000 7f8d8b337740 1 -- 172.21.15.69:0/2295176091 shutdown_connections 2024-08-02T17:50:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.174+0000 7f8d8b337740 1 -- 172.21.15.69:0/2295176091 wait complete. 2024-08-02T17:50:47.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.174+0000 7f8d8b337740 20 client.26198 trim_cache size 0 max 16384 2024-08-02T17:50:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.218+0000 7f8d8b337740 1 Processor -- start 2024-08-02T17:50:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.218+0000 7f8d8b337740 1 -- start start 2024-08-02T17:50:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 0x563f873340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f8734cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f8734d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d8b337740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841c8e00 con 0x563f8732a210 2024-08-02T17:50:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d8b337740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841c66d0 con 0x563f8732ab30 2024-08-02T17:50:47.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d8b337740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841c4520 con 0x563f873353f0 2024-08-02T17:50:47.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d82d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f8734d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d82d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f8734d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33992/0 (socket says 172.21.15.69:33992) 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d82d9d640 1 -- 172.21.15.69:0/382979408 learned_addr learned my addr 172.21.15.69:0/382979408 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d82d9d640 1 -- 172.21.15.69:0/382979408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 msgr2=0x563f8734cbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/382979408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f8734cbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d82d9d640 1 -- 172.21.15.69:0/382979408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 msgr2=0x563f873340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/382979408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 0x563f873340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.219+0000 7f8d82d9d640 1 -- 172.21.15.69:0/382979408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f8732a040 con 0x563f873353f0 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.220+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/382979408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f8734d0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0314b0 tx=0x7f8d7c031e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64bd6aa60c557590 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.220+0000 7f8d6bfff640 1 -- 172.21.15.69:0/382979408 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c036050 con 0x563f873353f0 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.220+0000 7f8d6bfff640 1 -- 172.21.15.69:0/382979408 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c0316e0 con 0x563f873353f0 2024-08-02T17:50:47.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.220+0000 7f8d6bfff640 1 -- 172.21.15.69:0/382979408 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c041040 con 0x563f873353f0 2024-08-02T17:50:47.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.221+0000 7f8d8b337740 1 -- 172.21.15.69:0/382979408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 msgr2=0x563f8734d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.221+0000 7f8d8b337740 1 --2- 172.21.15.69:0/382979408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f8734d0f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c0314b0 tx=0x7f8d7c031e20 comp rx=0 tx=0).stop 2024-08-02T17:50:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.222+0000 7f8d8b337740 1 -- 172.21.15.69:0/382979408 shutdown_connections 2024-08-02T17:50:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.222+0000 7f8d8b337740 1 --2- 172.21.15.69:0/382979408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f873353f0 0x563f8734d0f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.222+0000 7f8d8b337740 1 --2- 172.21.15.69:0/382979408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732ab30 0x563f8734cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.222+0000 7f8d8b337740 1 --2- 172.21.15.69:0/382979408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f8732a210 0x563f873340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.222+0000 7f8d8b337740 1 -- 172.21.15.69:0/382979408 >> 172.21.15.69:0/382979408 conn(0x563f873267a0 msgr2=0x563f873339c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:47.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.224+0000 7f8d8b337740 1 -- 172.21.15.69:0/382979408 shutdown_connections 2024-08-02T17:50:47.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.228+0000 7f8d8b337740 1 -- 172.21.15.69:0/382979408 wait complete. 2024-08-02T17:50:47.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.228+0000 7f8d8b337740 1 -- start start 2024-08-02T17:50:47.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.229+0000 7f8d8b337740 1 Processor -- start 2024-08-02T17:50:47.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.229+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732a210 0x563f8733db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.229+0000 7f8d8b337740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732ab30 0x563f8732ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.229+0000 7f8d82d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732a210 0x563f8733db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d82d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732a210 0x563f8733db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60080/0 (socket says 172.21.15.69:60080) 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d82d9d640 1 -- 172.21.15.69:0/376787145 learned_addr learned my addr 172.21.15.69:0/376787145 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d8b337740 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 0x563f8732f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841c4b90 con 0x563f873353f0 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841c6000 con 0x563f8732a210 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8d841c5260 con 0x563f8732ab30 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d8b337740 10 client.0 _ll_register_callbacks cb 0x7f8d6c0094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732ab30 0x563f8732ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 0x563f8732f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.230+0000 7f8d81d9b640 1 -- 172.21.15.69:0/376787145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732a210 msgr2=0x563f8733db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732a210 0x563f8733db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d81d9b640 1 -- 172.21.15.69:0/376787145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 msgr2=0x563f8732f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 0x563f8732f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d81d9b640 1 -- 172.21.15.69:0/376787145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f8732fc80 con 0x563f8732ab30 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732a210 0x563f8733db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d81d9b640 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732ab30 0x563f8732ef40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x563f873261f0 tx=0x7f8d6c006ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8482be4335299bd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d69ffb640 1 -- 172.21.15.69:0/376787145 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c019040 con 0x563f8732ab30 2024-08-02T17:50:47.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d7c0316e0 con 0x563f8732ab30 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d69ffb640 1 -- 172.21.15.69:0/376787145 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8d6c01db50 con 0x563f8732ab30 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d69ffb640 1 -- 172.21.15.69:0/376787145 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c01dd50 con 0x563f8732ab30 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d8b337740 10 client.26098 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.231+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563f873afb30 con 0x563f8732ab30 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d69ffb640 1 -- 172.21.15.69:0/376787145 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d7c0316e0 con 0x563f8732ab30 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d69ffb640 1 -- 172.21.15.69:0/376787145 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8d6c015460 con 0x563f8732ab30 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d8b337740 10 client.26098 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d8b337740 10 client.26098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d8b337740 20 client.26098 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d8b337740 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563f873b0280 0x563f873b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d8b337740 4 client.26098 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:50:47.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f87297400 con 0x563f873b0280 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.232+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563f873b0280 0x563f873b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d82d9d640 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563f873b0280 0x563f873b26a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4cdc6116fd8ebcc server_cookie=5785afb2b5904d1 in_seq=0 out_seq=0 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d69ffb640 10 client.26098 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d69ffb640 1 -- 172.21.15.69:0/376787145 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563f87297400 con 0x563f873b0280 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d69ffb640 10 client.26098 handle_command_reply: tid=0 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 client.26098 shutdown 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732ab30 msgr2=0x563f8732ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732ab30 0x563f8732ef40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x563f873261f0 tx=0x7f8d6c006ed0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 shutdown_connections 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563f873353f0 0x563f8732f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563f873b0280 0x563f873b26a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563f8732ab30 0x563f8732ef40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 --2- 172.21.15.69:0/376787145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563f8732a210 0x563f8733db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d69ffb640 0 client.26098 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:47.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 >> 172.21.15.69:0/376787145 conn(0x563f873267a0 msgr2=0x563f87325900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:47.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 shutdown_connections 2024-08-02T17:50:47.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.233+0000 7f8d8b337740 1 -- 172.21.15.69:0/376787145 wait complete. 2024-08-02T17:50:47.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d8b337740 20 client.26098 trim_cache size 0 max 16384 2024-08-02T17:50:47.238 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout: "4b3cd9fe-932a-40af-853f-794c5cf3c1f4": { 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "4b3cd9fe-932a-40af-853f-794c5cf3c1f4", 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:50:47.239 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:50:47.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 -- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d700a2f00 msgr2=0x7f8d700a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d700a2f00 0x7f8d700a53c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x563f87327600 tx=0x7f8d7400eef0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 -- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d84109180 msgr2=0x7f8d84118750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:47.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d84109180 0x7f8d84118750 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400e9c0 tx=0x7f8d7400edc0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 -- 172.21.15.69:0/861658514 shutdown_connections 2024-08-02T17:50:47.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8d700a2f00 0x7f8d700a53c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8d8411ae70 0x7f8d84110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8d84109560 0x7f8d84110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 --2- 172.21.15.69:0/861658514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8d84109180 0x7f8d84118750 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:47.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.234+0000 7f8d88f39640 1 -- 172.21.15.69:0/861658514 >> 172.21.15.69:0/861658514 conn(0x7f8d8406de70 msgr2=0x7f8d8411b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:47.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.235+0000 7f8d88f39640 1 -- 172.21.15.69:0/861658514 shutdown_connections 2024-08-02T17:50:47.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:50:47.235+0000 7f8d88f39640 1 -- 172.21.15.69:0/861658514 wait complete. 2024-08-02T17:50:48.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:48 smithi077 ceph-mon[37108]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-08-02T17:50:48.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:48 smithi069 ceph-mon[32746]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-08-02T17:50:48.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:48 smithi083 ceph-mon[37031]: pgmap v134: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 12 MiB/s wr, 1.11k op/s 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 -- 172.21.15.69:0/1084297441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 msgr2=0x7fb12c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 --2- 172.21.15.69:0/1084297441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 0x7fb12c10da40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb120009900 tx=0x7fb12002a3f0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 -- 172.21.15.69:0/1084297441 shutdown_connections 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 --2- 172.21.15.69:0/1084297441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb12c10e020 0x7fb12c110470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 --2- 172.21.15.69:0/1084297441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 0x7fb12c10da40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 --2- 172.21.15.69:0/1084297441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb12c100540 0x7fb12c100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 -- 172.21.15.69:0/1084297441 >> 172.21.15.69:0/1084297441 conn(0x7fb12c075be0 msgr2=0x7fb12c075ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 -- 172.21.15.69:0/1084297441 shutdown_connections 2024-08-02T17:50:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.780+0000 7fb131f5e640 1 -- 172.21.15.69:0/1084297441 wait complete. 2024-08-02T17:50:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.781+0000 7fb131f5e640 1 Processor -- start 2024-08-02T17:50:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.781+0000 7fb131f5e640 1 -- start start 2024-08-02T17:50:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb131f5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb12c100540 0x7fb12c1036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb131f5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 0x7fb12c103c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb131f5e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb12c10e020 0x7fb12c104140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb131f5e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c113e20 con 0x7fb12c10e020 2024-08-02T17:50:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb131f5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c113ca0 con 0x7fb12c100540 2024-08-02T17:50:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb131f5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c113fa0 con 0x7fb12c100e60 2024-08-02T17:50:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb12affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 0x7fb12c103c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb12affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 0x7fb12c103c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:34022/0 (socket says 172.21.15.69:34022) 2024-08-02T17:50:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb12affd640 1 -- 172.21.15.69:0/4140295213 learned_addr learned my addr 172.21.15.69:0/4140295213 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb12affd640 1 -- 172.21.15.69:0/4140295213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb12c100540 msgr2=0x7fb12c1036c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb12affd640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb12c100540 0x7fb12c1036c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb12affd640 1 -- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb12c10e020 msgr2=0x7fb12c104140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb12affd640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb12c10e020 0x7fb12c104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.782+0000 7fb12affd640 1 -- 172.21.15.69:0/4140295213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12c1aa8b0 con 0x7fb12c100e60 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.783+0000 7fb12affd640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 0x7fb12c103c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb12002a910 tx=0x7fb120003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.783+0000 7fb128ff9640 1 -- 172.21.15.69:0/4140295213 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12003f070 con 0x7fb12c100e60 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.783+0000 7fb128ff9640 1 -- 172.21.15.69:0/4140295213 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb120031c90 con 0x7fb12c100e60 2024-08-02T17:50:49.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.783+0000 7fb128ff9640 1 -- 172.21.15.69:0/4140295213 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120035030 con 0x7fb12c100e60 2024-08-02T17:50:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.783+0000 7fb131f5e640 1 -- 172.21.15.69:0/4140295213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12c1aab40 con 0x7fb12c100e60 2024-08-02T17:50:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.783+0000 7fb131f5e640 1 -- 172.21.15.69:0/4140295213 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb12c1aafe0 con 0x7fb12c100e60 2024-08-02T17:50:49.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.787+0000 7fb128ff9640 1 -- 172.21.15.69:0/4140295213 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb120042070 con 0x7fb12c100e60 2024-08-02T17:50:49.789 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.788+0000 7fb134d92740 1 Processor -- start 2024-08-02T17:50:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.788+0000 7fb134d92740 1 -- start start 2024-08-02T17:50:49.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.788+0000 7fb134d92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f7062cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.788+0000 7fb134d92740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 0x560f70637430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.788+0000 7fb134d92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70639d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.788+0000 7fb134d92740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f705909b0 con 0x560f7062d0d0 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.788+0000 7fb134d92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560f70538800 con 0x560f70637970 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb134d92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f7058c970 con 0x560f7062c7b0 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 0x560f70637430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f7062cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f7062cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:34034/0 (socket says 172.21.15.69:34034) 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12bfff640 1 -- 172.21.15.69:0/965520334 learned_addr learned my addr 172.21.15.69:0/965520334 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12affd640 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70639d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12bfff640 1 -- 172.21.15.69:0/965520334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 msgr2=0x560f70639d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12bfff640 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70639d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12bfff640 1 -- 172.21.15.69:0/965520334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 msgr2=0x560f70637430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12bfff640 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 0x560f70637430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.789+0000 7fb12bfff640 1 -- 172.21.15.69:0/965520334 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f702a4210 con 0x560f7062c7b0 2024-08-02T17:50:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.790+0000 7fb12bfff640 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f7062cb90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb124009d80 tx=0x7fb12400e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=281f5d1925b560ae server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.790+0000 7fb12affd640 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70639d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:50:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.790+0000 7fb128ff9640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1140a2e40 0x7fb1140a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.790+0000 7fb111ffb640 1 -- 172.21.15.69:0/965520334 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124016070 con 0x560f7062c7b0 2024-08-02T17:50:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.790+0000 7fb111ffb640 1 -- 172.21.15.69:0/965520334 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124005020 con 0x560f7062c7b0 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.790+0000 7fb128ff9640 1 -- 172.21.15.69:0/4140295213 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb1200efc70 con 0x7fb12c100e60 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.790+0000 7fb111ffb640 1 -- 172.21.15.69:0/965520334 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1240116d0 con 0x560f7062c7b0 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.790+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1140a2e40 0x7fb1140a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb134d92740 1 -- 172.21.15.69:0/965520334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 msgr2=0x560f7062cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1140a2e40 0x7fb1140a5300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb11c005fd0 tx=0x7fb11c005dc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb134d92740 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f7062cb90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb124009d80 tx=0x7fb12400e6c0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb134d92740 1 -- 172.21.15.69:0/965520334 shutdown_connections 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb134d92740 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 0x560f70637430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb134d92740 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70639d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb134d92740 1 --2- 172.21.15.69:0/965520334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f7062cb90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb134d92740 1 -- 172.21.15.69:0/965520334 >> 172.21.15.69:0/965520334 conn(0x560f70628d20 msgr2=0x560f70629100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.791+0000 7fb134d92740 1 -- 172.21.15.69:0/965520334 shutdown_connections 2024-08-02T17:50:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.792+0000 7fb134d92740 1 -- 172.21.15.69:0/965520334 wait complete. 2024-08-02T17:50:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.792+0000 7fb134d92740 1 -- start start 2024-08-02T17:50:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.792+0000 7fb134d92740 1 Processor -- start 2024-08-02T17:50:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb134d92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f706347d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb134d92740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 0x560f70634d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb134d92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70635250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb134d92740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560f7064b5f0 con 0x560f7062d0d0 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb134d92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560f7054c500 con 0x560f70637970 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f706347d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70635250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f706347d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:34046/0 (socket says 172.21.15.69:34046) 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12b7fe640 1 -- 172.21.15.69:0/1698896076 learned_addr learned my addr 172.21.15.69:0/1698896076 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb134d92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560f7064b770 con 0x560f7062c7b0 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12affd640 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 0x560f70634d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb134d92740 10 client.0 _ll_register_callbacks cb 0x560f702a3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12bfff640 1 -- 172.21.15.69:0/1698896076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 msgr2=0x560f706347d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12bfff640 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f706347d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12bfff640 1 -- 172.21.15.69:0/1698896076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 msgr2=0x560f70634d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:49.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12bfff640 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 0x560f70634d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.793+0000 7fb12bfff640 1 -- 172.21.15.69:0/1698896076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f70370620 con 0x560f70637970 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb12bfff640 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70635250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb124005ad0 tx=0x7fb124009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d2de2b56a48d5bc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb10affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb10affd640 1 -- 172.21.15.69:0/1698896076 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124016050 con 0x560f70637970 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb10affd640 1 -- 172.21.15.69:0/1698896076 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb1240051d0 con 0x560f70637970 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb134d92740 1 -- 172.21.15.69:0/1698896076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb124005020 con 0x560f70637970 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb10affd640 1 -- 172.21.15.69:0/1698896076 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb124003e40 con 0x560f70637970 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb134d92740 10 client.26207 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb134d92740 1 -- 172.21.15.69:0/1698896076 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f70657150 con 0x560f70637970 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb10affd640 1 -- 172.21.15.69:0/1698896076 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb124004040 con 0x560f70637970 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.794+0000 7fb10affd640 1 -- 172.21.15.69:0/1698896076 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb12400f040 con 0x560f70637970 2024-08-02T17:50:49.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.795+0000 7fb134d92740 10 client.26207 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.795+0000 7fb134d92740 10 client.26207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.795+0000 7fb134d92740 20 client.26207 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.795+0000 7fb134d92740 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560f706585b0 0x560f7065a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.795+0000 7fb134d92740 4 client.26207 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.795+0000 7fb134d92740 1 -- 172.21.15.69:0/1698896076 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb124005020 con 0x560f706585b0 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.795+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560f706585b0 0x560f7065a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.795+0000 7fb10affd640 10 client.26207 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.796+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560f706585b0 0x560f7065a9a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d1105cddcfb28f server_cookie=de10b6da3e6a1eef in_seq=0 out_seq=0 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb10affd640 1 -- 172.21.15.69:0/1698896076 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fb124005020 con 0x560f706585b0 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb10affd640 10 client.26207 handle_command_reply: tid=0 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 client.26207 shutdown 2024-08-02T17:50:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 -- 172.21.15.69:0/1698896076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 msgr2=0x560f70635250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70635250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb124005ad0 tx=0x7fb124009af0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 -- 172.21.15.69:0/1698896076 shutdown_connections 2024-08-02T17:50:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062d0d0 0x560f70634d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560f706585b0 0x560f7065a9a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb10affd640 0 client.26207 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70635250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 --2- 172.21.15.69:0/1698896076 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062c7b0 0x560f706347d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 -- 172.21.15.69:0/1698896076 >> 172.21.15.69:0/1698896076 conn(0x560f70628d20 msgr2=0x560f7062ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 -- 172.21.15.69:0/1698896076 shutdown_connections 2024-08-02T17:50:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 1 -- 172.21.15.69:0/1698896076 wait complete. 2024-08-02T17:50:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.797+0000 7fb134d92740 20 client.26207 trim_cache size 0 max 16384 2024-08-02T17:50:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.828+0000 7fb134d92740 1 Processor -- start 2024-08-02T17:50:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.828+0000 7fb134d92740 1 -- start start 2024-08-02T17:50:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.828+0000 7fb134d92740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f70635080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.829+0000 7fb134d92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062d0d0 0x560f70660210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.829+0000 7fb134d92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70660750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.831 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.829+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f70635080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.829+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f70635080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55642/0 (socket says 172.21.15.69:55642) 2024-08-02T17:50:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.829+0000 7fb12b7fe640 1 -- 172.21.15.69:0/126676771 learned_addr learned my addr 172.21.15.69:0/126676771 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.829+0000 7fb134d92740 1 -- 172.21.15.69:0/126676771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1af320 con 0x560f7062c7b0 2024-08-02T17:50:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.829+0000 7fb134d92740 1 -- 172.21.15.69:0/126676771 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1acbf0 con 0x560f70637970 2024-08-02T17:50:49.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.829+0000 7fb134d92740 1 -- 172.21.15.69:0/126676771 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c104bf0 con 0x560f7062d0d0 2024-08-02T17:50:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.832+0000 7fb12b7fe640 1 -- 172.21.15.69:0/126676771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062d0d0 msgr2=0x560f70660210 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:50:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.832+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/126676771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062d0d0 0x560f70660210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.832+0000 7fb12b7fe640 1 -- 172.21.15.69:0/126676771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 msgr2=0x560f70660750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:50:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.832+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/126676771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70660750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.832+0000 7fb12b7fe640 1 -- 172.21.15.69:0/126676771 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f7065ad10 con 0x560f7062c7b0 2024-08-02T17:50:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.836+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/126676771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f70635080 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x560f7065b150 tx=0x7fb11c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aef0a0f3b43a0c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.840+0000 7fb1117fa640 1 -- 172.21.15.69:0/126676771 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c0273b0 con 0x560f7062c7b0 2024-08-02T17:50:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.840+0000 7fb1117fa640 1 -- 172.21.15.69:0/126676771 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb11c027a30 con 0x560f7062c7b0 2024-08-02T17:50:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.840+0000 7fb1117fa640 1 -- 172.21.15.69:0/126676771 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11c010c20 con 0x560f7062c7b0 2024-08-02T17:50:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.841+0000 7fb134d92740 1 -- 172.21.15.69:0/126676771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 msgr2=0x560f70635080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.841+0000 7fb134d92740 1 --2- 172.21.15.69:0/126676771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f70635080 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x560f7065b150 tx=0x7fb11c014000 comp rx=0 tx=0).stop 2024-08-02T17:50:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.844+0000 7fb134d92740 1 -- 172.21.15.69:0/126676771 shutdown_connections 2024-08-02T17:50:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.844+0000 7fb134d92740 1 --2- 172.21.15.69:0/126676771 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f70637970 0x560f70660750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.844+0000 7fb134d92740 1 --2- 172.21.15.69:0/126676771 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f7062d0d0 0x560f70660210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.844+0000 7fb134d92740 1 --2- 172.21.15.69:0/126676771 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f70635080 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.844+0000 7fb134d92740 1 -- 172.21.15.69:0/126676771 >> 172.21.15.69:0/126676771 conn(0x560f70628d20 msgr2=0x560f706338a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:49.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.850+0000 7fb134d92740 1 -- 172.21.15.69:0/126676771 shutdown_connections 2024-08-02T17:50:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.855+0000 7fb134d92740 1 -- 172.21.15.69:0/126676771 wait complete. 2024-08-02T17:50:49.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.855+0000 7fb134d92740 1 -- start start 2024-08-02T17:50:49.857 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.856+0000 7fb134d92740 1 Processor -- start 2024-08-02T17:50:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.862+0000 7fb134d92740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f7064c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.863 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.862+0000 7fb134d92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f7062d0d0 0x560f7064c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.862+0000 7fb134d92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f70637970 0x560f70640260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.862+0000 7fb134d92740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1ab210 con 0x560f7062c7b0 2024-08-02T17:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.862+0000 7fb134d92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1ac520 con 0x560f7062d0d0 2024-08-02T17:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.862+0000 7fb134d92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb12c1ab780 con 0x560f70637970 2024-08-02T17:50:49.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.862+0000 7fb134d92740 10 client.0 _ll_register_callbacks cb 0x7fb124009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:50:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.864+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f7064c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.864+0000 7fb12b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f7064c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55648/0 (socket says 172.21.15.69:55648) 2024-08-02T17:50:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.864+0000 7fb12b7fe640 1 -- 172.21.15.69:0/3458471138 learned_addr learned my addr 172.21.15.69:0/3458471138 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:50:49.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.864+0000 7fb12affd640 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f7062d0d0 0x560f7064c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.865+0000 7fb12bfff640 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f70637970 0x560f70640260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.866+0000 7fb12affd640 1 -- 172.21.15.69:0/3458471138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f70637970 msgr2=0x560f70640260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.866+0000 7fb12affd640 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f70637970 0x560f70640260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.866+0000 7fb12affd640 1 -- 172.21.15.69:0/3458471138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 msgr2=0x560f7064c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:50:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.866+0000 7fb12affd640 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f7064c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.866+0000 7fb12affd640 1 -- 172.21.15.69:0/3458471138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f706408a0 con 0x560f7062d0d0 2024-08-02T17:50:49.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.867+0000 7fb12affd640 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f7062d0d0 0x560f7064c730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1200ecf70 tx=0x7fb1200ba280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0769e95b5742081 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:50:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.869+0000 7fb10affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:50:49.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.869+0000 7fb10affd640 1 -- 172.21.15.69:0/3458471138 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb120049040 con 0x560f7062d0d0 2024-08-02T17:50:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.869+0000 7fb10affd640 1 -- 172.21.15.69:0/3458471138 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb1200ed220 con 0x560f7062d0d0 2024-08-02T17:50:49.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.870+0000 7fb134d92740 1 -- 172.21.15.69:0/3458471138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb11c027a30 con 0x560f7062d0d0 2024-08-02T17:50:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.872+0000 7fb10affd640 1 -- 172.21.15.69:0/3458471138 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb1200ed420 con 0x560f7062d0d0 2024-08-02T17:50:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.874+0000 7fb134d92740 10 client.26210 fetch_fsmap learned FSMap version 26 2024-08-02T17:50:49.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.874+0000 7fb134d92740 1 -- 172.21.15.69:0/3458471138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560f70641c10 con 0x560f7062d0d0 2024-08-02T17:50:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.878+0000 7fb10affd640 1 -- 172.21.15.69:0/3458471138 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb11c027a30 con 0x560f7062d0d0 2024-08-02T17:50:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.878+0000 7fb10affd640 1 -- 172.21.15.69:0/3458471138 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb1200d2070 con 0x560f7062d0d0 2024-08-02T17:50:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.879+0000 7fb134d92740 10 client.26210 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:50:49.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.879+0000 7fb134d92740 10 client.26210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.879+0000 7fb134d92740 20 client.26210 populate_metadata read hostname 'smithi069' 2024-08-02T17:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.879+0000 7fb134d92740 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560f706422f0 0x560f70644710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.879+0000 7fb134d92740 4 client.26210 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:50:49.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.879+0000 7fb134d92740 1 -- 172.21.15.69:0/3458471138 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560f7059e0e0 con 0x560f706422f0 2024-08-02T17:50:49.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.884+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560f706422f0 0x560f70644710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:50:49.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.889+0000 7fb12b7fe640 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560f706422f0 0x560f70644710 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc75c67a5e9f7120 server_cookie=2898de5f75fbd30c in_seq=0 out_seq=0 2024-08-02T17:50:49.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.889+0000 7fb10affd640 10 client.26210 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb10affd640 1 -- 172.21.15.69:0/3458471138 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560f7059e0e0 con 0x560f706422f0 2024-08-02T17:50:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb10affd640 10 client.26210 handle_command_reply: tid=0 2024-08-02T17:50:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 client.26210 shutdown 2024-08-02T17:50:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 -- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f7062d0d0 msgr2=0x560f7064c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f7062d0d0 0x560f7064c730 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb1200ecf70 tx=0x7fb1200ba280 comp rx=0 tx=0).stop 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 -- 172.21.15.69:0/3458471138 shutdown_connections 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560f706422f0 0x560f70644710 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560f70637970 0x560f70640260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560f7062d0d0 0x560f7064c730 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 --2- 172.21.15.69:0/3458471138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560f7062c7b0 0x560f7064c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb134d92740 1 -- 172.21.15.69:0/3458471138 >> 172.21.15.69:0/3458471138 conn(0x560f70628d20 msgr2=0x560f70635090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.943+0000 7fb10affd640 0 client.26210 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.944+0000 7fb134d92740 1 -- 172.21.15.69:0/3458471138 shutdown_connections 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.944+0000 7fb134d92740 1 -- 172.21.15.69:0/3458471138 wait complete. 2024-08-02T17:50:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.944+0000 7fb134d92740 20 client.26210 trim_cache size 0 max 16384 2024-08-02T17:50:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:50:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:50:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:50:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 -- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1140a2e40 msgr2=0x7fb1140a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1140a2e40 0x7fb1140a5300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb11c005fd0 tx=0x7fb11c005dc0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 -- 172.21.15.69:0/4140295213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 msgr2=0x7fb12c103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 0x7fb12c103c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb12002a910 tx=0x7fb120003b90 comp rx=0 tx=0).stop 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 -- 172.21.15.69:0/4140295213 shutdown_connections 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb12c10e020 0x7fb12c104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb1140a2e40 0x7fb1140a5300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb12c100e60 0x7fb12c103c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 --2- 172.21.15.69:0/4140295213 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb12c100540 0x7fb12c1036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.945+0000 7fb131f5e640 1 -- 172.21.15.69:0/4140295213 >> 172.21.15.69:0/4140295213 conn(0x7fb12c075be0 msgr2=0x7fb12c10e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.946+0000 7fb131f5e640 1 -- 172.21.15.69:0/4140295213 shutdown_connections 2024-08-02T17:50:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:50:49.946+0000 7fb131f5e640 1 -- 172.21.15.69:0/4140295213 wait complete. 2024-08-02T17:50:49.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 220.0 s, 2158.0 tps, lat 45.435 ms stddev 62.416 2024-08-02T17:50:50.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:50 smithi077 ceph-mon[37108]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 9.0 MiB/s wr, 922 op/s 2024-08-02T17:50:50.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:50 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:50:50.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:50 smithi069 ceph-mon[32746]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 9.0 MiB/s wr, 922 op/s 2024-08-02T17:50:50.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:50 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:50:50.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:50 smithi083 ceph-mon[37031]: pgmap v135: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 9.0 MiB/s wr, 922 op/s 2024-08-02T17:50:50.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:50 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:50:50.903 INFO:tasks.cephfs.filesystem:scrub status for tag:4b3cd9fe-932a-40af-853f-794c5cf3c1f4 - {'path': '/', 'tag': '4b3cd9fe-932a-40af-853f-794c5cf3c1f4', 'options': 'recursive,force'} 2024-08-02T17:50:52.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:52 smithi077 ceph-mon[37108]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 8.7 MiB/s wr, 913 op/s 2024-08-02T17:50:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:52 smithi069 ceph-mon[32746]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 8.7 MiB/s wr, 913 op/s 2024-08-02T17:50:52.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:52 smithi083 ceph-mon[37031]: pgmap v136: 129 pgs: 129 active+clean; 2.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 8.7 MiB/s wr, 913 op/s 2024-08-02T17:50:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:54 smithi077 ceph-mon[37108]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-08-02T17:50:54.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:54 smithi069 ceph-mon[32746]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-08-02T17:50:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:54 smithi083 ceph-mon[37031]: pgmap v137: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 12 MiB/s wr, 1.09k op/s 2024-08-02T17:50:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 225.0 s, 1699.3 tps, lat 52.397 ms stddev 66.138 2024-08-02T17:50:56.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:56 smithi069 ceph-mon[32746]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.7 MiB/s wr, 870 op/s 2024-08-02T17:50:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:56 smithi083 ceph-mon[37031]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.7 MiB/s wr, 870 op/s 2024-08-02T17:50:56.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:56 smithi077 ceph-mon[37108]: pgmap v138: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.7 MiB/s wr, 870 op/s 2024-08-02T17:50:58.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:50:58 smithi069 ceph-mon[32746]: pgmap v139: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-08-02T17:50:58.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:50:58 smithi083 ceph-mon[37031]: pgmap v139: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-08-02T17:50:58.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:50:58 smithi077 ceph-mon[37108]: pgmap v139: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-08-02T17:50:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 230.0 s, 1845.7 tps, lat 60.997 ms stddev 186.334 2024-08-02T17:51:00.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:00 smithi069 ceph-mon[32746]: pgmap v140: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.58k op/s 2024-08-02T17:51:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:00 smithi083 ceph-mon[37031]: pgmap v140: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.58k op/s 2024-08-02T17:51:00.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:00 smithi077 ceph-mon[37108]: pgmap v140: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 1.58k op/s 2024-08-02T17:51:02.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:02 smithi083 ceph-mon[37031]: pgmap v141: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-08-02T17:51:02.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:02 smithi077 ceph-mon[37108]: pgmap v141: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-08-02T17:51:02.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:02 smithi069 ceph-mon[32746]: pgmap v141: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-08-02T17:51:03.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 -- 172.21.15.69:0/4220214154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c0a0f90 msgr2=0x7fa21c0a8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 --2- 172.21.15.69:0/4220214154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c0a0f90 0x7fa21c0a8ae0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa22404ffb0 tx=0x7fa224096d40 comp rx=0 tx=0).stop 2024-08-02T17:51:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 -- 172.21.15.69:0/4220214154 shutdown_connections 2024-08-02T17:51:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 --2- 172.21.15.69:0/4220214154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c0a0f90 0x7fa21c0a8ae0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 --2- 172.21.15.69:0/4220214154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c09c840 0x7fa21c0a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 --2- 172.21.15.69:0/4220214154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa21c09be90 0x7fa21c09c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 -- 172.21.15.69:0/4220214154 >> 172.21.15.69:0/4220214154 conn(0x7fa21c009020 msgr2=0x7fa21c09a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 -- 172.21.15.69:0/4220214154 shutdown_connections 2024-08-02T17:51:03.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 -- 172.21.15.69:0/4220214154 wait complete. 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.091+0000 7fa229301640 1 Processor -- start 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa229301640 1 -- start start 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa229301640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c09be90 0x7fa21c132230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa229301640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa21c0a0f90 0x7fa21c132770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa229301640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c136af0 0x7fa21c136fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa229301640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0abf20 con 0x7fa21c0a0f90 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa229301640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0abda0 con 0x7fa21c09be90 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa229301640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c0ac0a0 con 0x7fa21c136af0 2024-08-02T17:51:03.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa228b00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c136af0 0x7fa21c136fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa228b00640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c136af0 0x7fa21c136fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37838/0 (socket says 172.21.15.69:37838) 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.092+0000 7fa228b00640 1 -- 172.21.15.69:0/1021180927 learned_addr learned my addr 172.21.15.69:0/1021180927 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa223fff640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c09be90 0x7fa21c132230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa228b00640 1 -- 172.21.15.69:0/1021180927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c09be90 msgr2=0x7fa21c132230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa228b00640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c09be90 0x7fa21c132230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa228b00640 1 -- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa21c0a0f90 msgr2=0x7fa21c132770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa2237fe640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa21c0a0f90 0x7fa21c132770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa228b00640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa21c0a0f90 0x7fa21c132770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa228b00640 1 -- 172.21.15.69:0/1021180927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c1375e0 con 0x7fa21c136af0 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa223fff640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c09be90 0x7fa21c132230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa2237fe640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa21c0a0f90 0x7fa21c132770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:03.097 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa228b00640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c136af0 0x7fa21c136fa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa224097330 tx=0x7fa22406a290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa2217fa640 1 -- 172.21.15.69:0/1021180927 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2240ae070 con 0x7fa21c136af0 2024-08-02T17:51:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.093+0000 7fa229301640 1 -- 172.21.15.69:0/1021180927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21c137870 con 0x7fa21c136af0 2024-08-02T17:51:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.094+0000 7fa229301640 1 -- 172.21.15.69:0/1021180927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21c13f030 con 0x7fa21c136af0 2024-08-02T17:51:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.094+0000 7fa2217fa640 1 -- 172.21.15.69:0/1021180927 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa22406a610 con 0x7fa21c136af0 2024-08-02T17:51:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.094+0000 7fa2217fa640 1 -- 172.21.15.69:0/1021180927 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2240a9670 con 0x7fa21c136af0 2024-08-02T17:51:03.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.097+0000 7fa2217fa640 1 -- 172.21.15.69:0/1021180927 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2240b1070 con 0x7fa21c136af0 2024-08-02T17:51:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.097+0000 7fa22d38c740 1 Processor -- start 2024-08-02T17:51:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa22d38c740 1 -- start start 2024-08-02T17:51:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6170c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a618c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 0x55f9a618ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa22d38c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f9a60d49b0 con 0x55f9a6171170 2024-08-02T17:51:03.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa22d38c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9a607c800 con 0x55f9a618cb60 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa22d38c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f9a60d0970 con 0x55f9a6170850 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa2237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6170c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa223fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a618c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa2237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6170c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37850/0 (socket says 172.21.15.69:37850) 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa2237fe640 1 -- 172.21.15.69:0/2327861478 learned_addr learned my addr 172.21.15.69:0/2327861478 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.098+0000 7fa2237fe640 1 --2- 172.21.15.69:0/2327861478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 0x55f9a618ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa2217fa640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2140a2e00 0x7fa2140a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa2237fe640 1 -- 172.21.15.69:0/2327861478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 msgr2=0x55f9a618ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa2237fe640 1 --2- 172.21.15.69:0/2327861478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 0x55f9a618ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa2237fe640 1 -- 172.21.15.69:0/2327861478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 msgr2=0x55f9a618c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa2237fe640 1 --2- 172.21.15.69:0/2327861478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a618c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa2237fe640 1 -- 172.21.15.69:0/2327861478 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a5de8210 con 0x55f9a6170850 2024-08-02T17:51:03.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa228b00640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2140a2e00 0x7fa2140a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa2237fe640 1 --2- 172.21.15.69:0/2327861478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6170c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa210014880 tx=0x7fa2100095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6843b7c64a9f246d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa2217fa640 1 -- 172.21.15.69:0/1021180927 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa22415c860 con 0x7fa21c136af0 2024-08-02T17:51:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa20e7fc640 1 -- 172.21.15.69:0/2327861478 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210014110 con 0x55f9a6170850 2024-08-02T17:51:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa20e7fc640 1 -- 172.21.15.69:0/2327861478 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2100142b0 con 0x55f9a6170850 2024-08-02T17:51:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.099+0000 7fa20e7fc640 1 -- 172.21.15.69:0/2327861478 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2100035f0 con 0x55f9a6170850 2024-08-02T17:51:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.100+0000 7fa22d38c740 1 -- 172.21.15.69:0/2327861478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 msgr2=0x55f9a6170c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.100+0000 7fa22d38c740 1 --2- 172.21.15.69:0/2327861478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6170c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa210014880 tx=0x7fa2100095b0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.104 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.100+0000 7fa228b00640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2140a2e00 0x7fa2140a52c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa224065770 tx=0x7fa22412c040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.100+0000 7fa22d38c740 1 -- 172.21.15.69:0/2327861478 shutdown_connections 2024-08-02T17:51:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.100+0000 7fa22d38c740 1 --2- 172.21.15.69:0/2327861478 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a618c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.100+0000 7fa22d38c740 1 --2- 172.21.15.69:0/2327861478 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 0x55f9a618ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.100+0000 7fa22d38c740 1 --2- 172.21.15.69:0/2327861478 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6170c30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.100+0000 7fa22d38c740 1 -- 172.21.15.69:0/2327861478 >> 172.21.15.69:0/2327861478 conn(0x55f9a616cf00 msgr2=0x55f9a616d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.101+0000 7fa22d38c740 1 -- 172.21.15.69:0/2327861478 shutdown_connections 2024-08-02T17:51:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.101+0000 7fa22d38c740 1 -- 172.21.15.69:0/2327861478 wait complete. 2024-08-02T17:51:03.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.101+0000 7fa22d38c740 1 -- start start 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa22d38c740 1 Processor -- start 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6189a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a6189fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 0x55f9a618a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa22d38c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f9a618f6d0 con 0x55f9a6171170 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa22d38c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f9a6090500 con 0x55f9a618cb60 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa22d38c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f9a618f850 con 0x55f9a6170850 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa22d38c740 10 client.0 _ll_register_callbacks cb 0x55f9a5de7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.102+0000 7fa223fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a6189fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa223fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a6189fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52166/0 (socket says 172.21.15.69:52166) 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa223fff640 1 -- 172.21.15.69:0/1809856897 learned_addr learned my addr 172.21.15.69:0/1809856897 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa2237fe640 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6189a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa2237fe640 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 0x55f9a618a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa223fff640 1 -- 172.21.15.69:0/1809856897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 msgr2=0x55f9a6189a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa223fff640 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6189a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa223fff640 1 -- 172.21.15.69:0/1809856897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 msgr2=0x55f9a618a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa223fff640 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 0x55f9a618a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa223fff640 1 -- 172.21.15.69:0/1809856897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a5eb4620 con 0x55f9a6171170 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa223fff640 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a6189fd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa218013710 tx=0x7fa218013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9672d52a4048c3a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa2037fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.103+0000 7fa2037fe640 1 -- 172.21.15.69:0/1809856897 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218011dd0 con 0x55f9a6171170 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.104+0000 7fa22d38c740 1 -- 172.21.15.69:0/1809856897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2100142b0 con 0x55f9a6171170 2024-08-02T17:51:03.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.104+0000 7fa2037fe640 1 -- 172.21.15.69:0/1809856897 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa21801b3e0 con 0x55f9a6171170 2024-08-02T17:51:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.104+0000 7fa2037fe640 1 -- 172.21.15.69:0/1809856897 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa21801b5e0 con 0x55f9a6171170 2024-08-02T17:51:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.104+0000 7fa22d38c740 10 client.16620 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.104+0000 7fa22d38c740 1 -- 172.21.15.69:0/1809856897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9a619b4e0 con 0x55f9a6171170 2024-08-02T17:51:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.106+0000 7fa2037fe640 1 -- 172.21.15.69:0/1809856897 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2100142b0 con 0x55f9a6171170 2024-08-02T17:51:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.106+0000 7fa2037fe640 1 -- 172.21.15.69:0/1809856897 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa21801a070 con 0x55f9a6171170 2024-08-02T17:51:03.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.106+0000 7fa22d38c740 10 client.16620 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.106+0000 7fa22d38c740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.106+0000 7fa22d38c740 20 client.16620 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.106+0000 7fa22d38c740 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f9a619ca10 0x55f9a619ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.106+0000 7fa22d38c740 4 client.16620 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.106+0000 7fa22d38c740 1 -- 172.21.15.69:0/1809856897 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9a60e20e0 con 0x55f9a619ca10 2024-08-02T17:51:03.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.108+0000 7fa2237fe640 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f9a619ca10 0x55f9a619ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.111+0000 7fa2237fe640 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f9a619ca10 0x55f9a619ee30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc8f631a814263a server_cookie=24a5bdf4681d6d6b in_seq=0 out_seq=0 2024-08-02T17:51:03.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.111+0000 7fa2037fe640 10 client.16620 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.114+0000 7fa2037fe640 1 -- 172.21.15.69:0/1809856897 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f9a60e20e0 con 0x55f9a619ca10 2024-08-02T17:51:03.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.114+0000 7fa2037fe640 10 client.16620 handle_command_reply: tid=0 2024-08-02T17:51:03.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.116+0000 7fa22d38c740 1 client.16620 shutdown 2024-08-02T17:51:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.125+0000 7fa22d38c740 1 -- 172.21.15.69:0/1809856897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 msgr2=0x55f9a6189fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.125+0000 7fa22d38c740 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a6189fd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa218013710 tx=0x7fa218013bd0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.129+0000 7fa22d38c740 1 -- 172.21.15.69:0/1809856897 shutdown_connections 2024-08-02T17:51:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.129+0000 7fa22d38c740 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a6171170 0x55f9a6189fd0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.129+0000 7fa22d38c740 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f9a619ca10 0x55f9a619ee30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.129+0000 7fa22d38c740 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a618cb60 0x55f9a618a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.129+0000 7fa22d38c740 1 --2- 172.21.15.69:0/1809856897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6189a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.129+0000 7fa22d38c740 1 -- 172.21.15.69:0/1809856897 >> 172.21.15.69:0/1809856897 conn(0x55f9a616cf00 msgr2=0x55f9a6172b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:03.131 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.130+0000 7fa2037fe640 0 client.16620 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:03.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.136+0000 7fa22d38c740 1 -- 172.21.15.69:0/1809856897 shutdown_connections 2024-08-02T17:51:03.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.140+0000 7fa22d38c740 1 -- 172.21.15.69:0/1809856897 wait complete. 2024-08-02T17:51:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.144+0000 7fa22d38c740 20 client.16620 trim_cache size 0 max 16384 2024-08-02T17:51:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.175+0000 7fa22d38c740 1 Processor -- start 2024-08-02T17:51:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa22d38c740 1 -- start start 2024-08-02T17:51:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6170850 0x55f9a6188a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6171170 0x55f9a618a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 0x55f9a61a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa2237fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6170850 0x55f9a6188a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa2237fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6170850 0x55f9a6188a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48958/0 (socket says 172.21.15.69:48958) 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa2237fe640 1 -- 172.21.15.69:0/3114942719 learned_addr learned my addr 172.21.15.69:0/3114942719 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa22d38c740 1 -- 172.21.15.69:0/3114942719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa208012850 con 0x55f9a618cb60 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa22d38c740 1 -- 172.21.15.69:0/3114942719 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa208010120 con 0x55f9a6170850 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.176+0000 7fa22d38c740 1 -- 172.21.15.69:0/3114942719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa20800df70 con 0x55f9a6171170 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.177+0000 7fa223fff640 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6171170 0x55f9a618a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.177+0000 7fa2237fe640 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 0x55f9a61a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.177+0000 7fa223fff640 1 -- 172.21.15.69:0/3114942719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6170850 msgr2=0x55f9a6188a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.177+0000 7fa223fff640 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6170850 0x55f9a6188a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.177+0000 7fa223fff640 1 -- 172.21.15.69:0/3114942719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 msgr2=0x55f9a61a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.177+0000 7fa223fff640 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 0x55f9a61a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.177+0000 7fa223fff640 1 -- 172.21.15.69:0/3114942719 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9a618c540 con 0x55f9a6171170 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.178+0000 7fa223fff640 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6171170 0x55f9a618a340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa21801cb60 tx=0x7fa2180118e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b44e517632705fb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.178+0000 7fa20dffb640 1 -- 172.21.15.69:0/3114942719 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21802f070 con 0x55f9a6171170 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.179+0000 7fa20dffb640 1 -- 172.21.15.69:0/3114942719 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2180055f0 con 0x55f9a6171170 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.179+0000 7fa20dffb640 1 -- 172.21.15.69:0/3114942719 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21801b780 con 0x55f9a6171170 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.179+0000 7fa22d38c740 1 -- 172.21.15.69:0/3114942719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6171170 msgr2=0x55f9a618a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.179+0000 7fa22d38c740 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6171170 0x55f9a618a340 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa21801cb60 tx=0x7fa2180118e0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.180+0000 7fa22d38c740 1 -- 172.21.15.69:0/3114942719 shutdown_connections 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.180+0000 7fa22d38c740 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 0x55f9a61a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.180+0000 7fa22d38c740 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6171170 0x55f9a618a340 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.180+0000 7fa22d38c740 1 --2- 172.21.15.69:0/3114942719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6170850 0x55f9a6188a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.180+0000 7fa22d38c740 1 -- 172.21.15.69:0/3114942719 >> 172.21.15.69:0/3114942719 conn(0x55f9a616cf00 msgr2=0x55f9a6188340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:03.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.181+0000 7fa22d38c740 1 -- 172.21.15.69:0/3114942719 shutdown_connections 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.181+0000 7fa22d38c740 1 -- 172.21.15.69:0/3114942719 wait complete. 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.181+0000 7fa22d38c740 1 -- start start 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.181+0000 7fa22d38c740 1 Processor -- start 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.182+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6194ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.182+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6171170 0x55f9a6186430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.182+0000 7fa22d38c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 0x55f9a6186970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.182+0000 7fa22d38c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa20800e5e0 con 0x55f9a618cb60 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.182+0000 7fa22d38c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa20800fa50 con 0x55f9a6171170 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.182+0000 7fa22d38c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa20800ecb0 con 0x55f9a6170850 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.182+0000 7fa22d38c740 10 client.0 _ll_register_callbacks cb 0x7fa218009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6194ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6194ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37870/0 (socket says 172.21.15.69:37870) 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 -- 172.21.15.69:0/543344775 learned_addr learned my addr 172.21.15.69:0/543344775 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:03.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 0x55f9a6186970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa223fff640 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6171170 0x55f9a6186430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 -- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6171170 msgr2=0x55f9a6186430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6171170 0x55f9a6186430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 -- 172.21.15.69:0/543344775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 msgr2=0x55f9a6186970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 0x55f9a6186970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 -- 172.21.15.69:0/543344775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9a6186eb0 con 0x55f9a6170850 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa2237fe640 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6194ff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa21001c000 tx=0x7fa218009d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852d0688895e111b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.183+0000 7fa223fff640 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6171170 0x55f9a6186430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.184+0000 7fa2037fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.184+0000 7fa2037fe640 1 -- 172.21.15.69:0/543344775 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21000fdd0 con 0x55f9a6170850 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.184+0000 7fa2037fe640 1 -- 172.21.15.69:0/543344775 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa21002b040 con 0x55f9a6170850 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.184+0000 7fa22d38c740 1 -- 172.21.15.69:0/543344775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9a60dc430 con 0x55f9a6170850 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa2037fe640 1 -- 172.21.15.69:0/543344775 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa21003c050 con 0x55f9a6170850 2024-08-02T17:51:03.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa22d38c740 10 client.26128 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa22d38c740 1 -- 172.21.15.69:0/543344775 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f9a6218420 con 0x55f9a6170850 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa2037fe640 1 -- 172.21.15.69:0/543344775 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f9a60dc430 con 0x55f9a6170850 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa2037fe640 1 -- 172.21.15.69:0/543344775 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2100035f0 con 0x55f9a6170850 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa22d38c740 10 client.26128 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa22d38c740 10 client.26128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa22d38c740 20 client.26128 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa22d38c740 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f9a6218850 0x55f9a621ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa22d38c740 4 client.26128 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:51:03.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.185+0000 7fa22d38c740 1 -- 172.21.15.69:0/543344775 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9a60598e0 con 0x55f9a6218850 2024-08-02T17:51:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.186+0000 7fa223fff640 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f9a6218850 0x55f9a621ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.186+0000 7fa2037fe640 10 client.26128 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:03.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.186+0000 7fa223fff640 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f9a6218850 0x55f9a621ac40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a84cc55fef56c7d server_cookie=f1f285900a7715f0 in_seq=0 out_seq=0 2024-08-02T17:51:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.216+0000 7fa2037fe640 1 -- 172.21.15.69:0/543344775 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9a60598e0 con 0x55f9a6218850 2024-08-02T17:51:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.216+0000 7fa2037fe640 10 client.26128 handle_command_reply: tid=0 2024-08-02T17:51:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.216+0000 7fa22d38c740 1 client.26128 shutdown 2024-08-02T17:51:03.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 -- 172.21.15.69:0/543344775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 msgr2=0x55f9a6194ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6194ff0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa21001c000 tx=0x7fa218009d90 comp rx=0 tx=0).stop 2024-08-02T17:51:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 -- 172.21.15.69:0/543344775 shutdown_connections 2024-08-02T17:51:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f9a618cb60 0x55f9a6186970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f9a6218850 0x55f9a621ac40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f9a6171170 0x55f9a6186430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa2037fe640 0 client.26128 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 --2- 172.21.15.69:0/543344775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f9a6170850 0x55f9a6194ff0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 -- 172.21.15.69:0/543344775 >> 172.21.15.69:0/543344775 conn(0x55f9a616cf00 msgr2=0x55f9a6188b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 -- 172.21.15.69:0/543344775 shutdown_connections 2024-08-02T17:51:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.217+0000 7fa22d38c740 1 -- 172.21.15.69:0/543344775 wait complete. 2024-08-02T17:51:03.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.218+0000 7fa22d38c740 20 client.26128 trim_cache size 0 max 16384 2024-08-02T17:51:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:51:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:51:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:51:03.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:51:03.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 -- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2140a2e00 msgr2=0x7fa2140a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2140a2e00 0x7fa2140a52c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa224065770 tx=0x7fa22412c040 comp rx=0 tx=0).stop 2024-08-02T17:51:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 -- 172.21.15.69:0/1021180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c136af0 msgr2=0x7fa21c136fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c136af0 0x7fa21c136fa0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa224097330 tx=0x7fa22406a290 comp rx=0 tx=0).stop 2024-08-02T17:51:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 -- 172.21.15.69:0/1021180927 shutdown_connections 2024-08-02T17:51:03.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa21c0a0f90 0x7fa21c132770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2140a2e00 0x7fa2140a52c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa21c136af0 0x7fa21c136fa0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 --2- 172.21.15.69:0/1021180927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa21c09be90 0x7fa21c132230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 -- 172.21.15.69:0/1021180927 >> 172.21.15.69:0/1021180927 conn(0x7fa21c009020 msgr2=0x7fa21c009400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 -- 172.21.15.69:0/1021180927 shutdown_connections 2024-08-02T17:51:03.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:03.219+0000 7fa229301640 1 -- 172.21.15.69:0/1021180927 wait complete. 2024-08-02T17:51:04.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:04 smithi083 ceph-mon[37031]: pgmap v142: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 2.25k op/s 2024-08-02T17:51:04.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:04 smithi077 ceph-mon[37108]: pgmap v142: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 2.25k op/s 2024-08-02T17:51:04.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:04 smithi069 ceph-mon[32746]: pgmap v142: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 29 MiB/s wr, 2.25k op/s 2024-08-02T17:51:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 235.0 s, 1969.2 tps, lat 50.084 ms stddev 88.344 2024-08-02T17:51:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:06 smithi083 ceph-mon[37031]: pgmap v143: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 2.03k op/s 2024-08-02T17:51:06.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:06 smithi077 ceph-mon[37108]: pgmap v143: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 2.03k op/s 2024-08-02T17:51:06.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:06 smithi069 ceph-mon[32746]: pgmap v143: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 MiB/s wr, 2.03k op/s 2024-08-02T17:51:08.399 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:08 smithi077 ceph-mon[37108]: pgmap v144: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 28 MiB/s wr, 2.29k op/s 2024-08-02T17:51:08.523 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:08 smithi083 ceph-mon[37031]: pgmap v144: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 28 MiB/s wr, 2.29k op/s 2024-08-02T17:51:08.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:08 smithi069 ceph-mon[32746]: pgmap v144: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 28 MiB/s wr, 2.29k op/s 2024-08-02T17:51:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 240.0 s, 2236.8 tps, lat 45.201 ms stddev 56.745 2024-08-02T17:51:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:10 smithi083 ceph-mon[37031]: pgmap v145: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.19k op/s 2024-08-02T17:51:10.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:10 smithi077 ceph-mon[37108]: pgmap v145: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.19k op/s 2024-08-02T17:51:10.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:10 smithi069 ceph-mon[32746]: pgmap v145: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 12 MiB/s wr, 1.19k op/s 2024-08-02T17:51:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:12 smithi083 ceph-mon[37031]: pgmap v146: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-08-02T17:51:12.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:12 smithi077 ceph-mon[37108]: pgmap v146: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-08-02T17:51:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:12 smithi069 ceph-mon[32746]: pgmap v146: 129 pgs: 129 active+clean; 2.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 10 MiB/s wr, 1.08k op/s 2024-08-02T17:51:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:14 smithi083 ceph-mon[37031]: pgmap v147: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-08-02T17:51:14.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:14 smithi077 ceph-mon[37108]: pgmap v147: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-08-02T17:51:14.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:14 smithi069 ceph-mon[32746]: pgmap v147: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 12 MiB/s wr, 1.21k op/s 2024-08-02T17:51:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 245.0 s, 2376.2 tps, lat 40.561 ms stddev 50.887 2024-08-02T17:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.341+0000 7fee541ee640 1 -- 172.21.15.69:0/2262847160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c1010c0 msgr2=0x7fee4c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.341+0000 7fee541ee640 1 --2- 172.21.15.69:0/2262847160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c1010c0 0x7fee4c10d5f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fee40009930 tx=0x7fee4002a420 comp rx=0 tx=0).stop 2024-08-02T17:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 -- 172.21.15.69:0/2262847160 shutdown_connections 2024-08-02T17:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 --2- 172.21.15.69:0/2262847160 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c1010c0 0x7fee4c10d5f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 --2- 172.21.15.69:0/2262847160 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee4c100720 0x7fee4c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 --2- 172.21.15.69:0/2262847160 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4c106730 0x7fee4c106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 -- 172.21.15.69:0/2262847160 >> 172.21.15.69:0/2262847160 conn(0x7fee4c0fc420 msgr2=0x7fee4c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:16.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 -- 172.21.15.69:0/2262847160 shutdown_connections 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 -- 172.21.15.69:0/2262847160 wait complete. 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 Processor -- start 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.342+0000 7fee541ee640 1 -- start start 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee541ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee4c100720 0x7fee4c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee541ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4c1010c0 0x7fee4c199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee541ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c106730 0x7fee4c1a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee541ee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c110bc0 con 0x7fee4c106730 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee541ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c110a40 con 0x7fee4c100720 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee541ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c110d40 con 0x7fee4c1010c0 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee5278b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c106730 0x7fee4c1a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee5278b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c106730 0x7fee4c1a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58690/0 (socket says 172.21.15.69:58690) 2024-08-02T17:51:16.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee5278b640 1 -- 172.21.15.69:0/1929954864 learned_addr learned my addr 172.21.15.69:0/1929954864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.343+0000 7fee51f8a640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee4c100720 0x7fee4c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.344+0000 7fee5278b640 1 -- 172.21.15.69:0/1929954864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4c1010c0 msgr2=0x7fee4c199430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.344+0000 7fee5278b640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4c1010c0 0x7fee4c199430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.344+0000 7fee5278b640 1 -- 172.21.15.69:0/1929954864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee4c100720 msgr2=0x7fee4c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.344+0000 7fee5278b640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee4c100720 0x7fee4c198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.344+0000 7fee5278b640 1 -- 172.21.15.69:0/1929954864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c1a44b0 con 0x7fee4c106730 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.344+0000 7fee5278b640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c106730 0x7fee4c1a3f70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee40004770 tx=0x7fee40003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.345+0000 7fee4affd640 1 -- 172.21.15.69:0/1929954864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4003f030 con 0x7fee4c106730 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.345+0000 7fee541ee640 1 -- 172.21.15.69:0/1929954864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c1a4680 con 0x7fee4c106730 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.345+0000 7fee541ee640 1 -- 172.21.15.69:0/1929954864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c1a4a80 con 0x7fee4c106730 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.345+0000 7fee4affd640 1 -- 172.21.15.69:0/1929954864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee40003cb0 con 0x7fee4c106730 2024-08-02T17:51:16.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.345+0000 7fee4affd640 1 -- 172.21.15.69:0/1929954864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee40049060 con 0x7fee4c106730 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.347+0000 7fee4affd640 1 -- 172.21.15.69:0/1929954864 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fee40042030 con 0x7fee4c106730 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.348+0000 7fee57027740 1 Processor -- start 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.348+0000 7fee57027740 1 -- start start 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee57027740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef0fd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee57027740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 0x55e8ef0fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee57027740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef113400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee57027740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef0619b0 con 0x55e8ef0fdb90 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee57027740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef009800 con 0x55e8ef111010 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee57027740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef05d970 con 0x55e8ef0fed40 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee4affd640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee340a2e30 0x7fee340a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef0fd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef0fd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33722/0 (socket says 172.21.15.69:33722) 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 -- 172.21.15.69:0/2544626808 learned_addr learned my addr 172.21.15.69:0/2544626808 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 --2- 172.21.15.69:0/2544626808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef113400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee5278b640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee340a2e30 0x7fee340a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee4affd640 1 -- 172.21.15.69:0/1929954864 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fee400efb90 con 0x7fee4c106730 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 -- 172.21.15.69:0/2544626808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 msgr2=0x55e8ef113400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 --2- 172.21.15.69:0/2544626808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef113400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 -- 172.21.15.69:0/2544626808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 msgr2=0x55e8ef0fdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 --2- 172.21.15.69:0/2544626808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 0x55e8ef0fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.349+0000 7fee51789640 1 -- 172.21.15.69:0/2544626808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8eed75210 con 0x55e8ef0fed40 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.350+0000 7fee51789640 1 --2- 172.21.15.69:0/2544626808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef0fd650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0137b0 tx=0x7fee3c0096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8697a8e183448b49 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.350+0000 7fee5278b640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee340a2e30 0x7fee340a52f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fee400b80f0 tx=0x7fee400b1120 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.350+0000 7fee33fff640 1 -- 172.21.15.69:0/2544626808 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c013110 con 0x55e8ef0fed40 2024-08-02T17:51:16.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.350+0000 7fee33fff640 1 -- 172.21.15.69:0/2544626808 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c0132b0 con 0x55e8ef0fed40 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.350+0000 7fee33fff640 1 -- 172.21.15.69:0/2544626808 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c004a00 con 0x55e8ef0fed40 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 -- 172.21.15.69:0/2544626808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 msgr2=0x55e8ef0fd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 --2- 172.21.15.69:0/2544626808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef0fd650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0137b0 tx=0x7fee3c0096c0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 -- 172.21.15.69:0/2544626808 shutdown_connections 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 --2- 172.21.15.69:0/2544626808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 0x55e8ef0fdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 --2- 172.21.15.69:0/2544626808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef113400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 --2- 172.21.15.69:0/2544626808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef0fd650 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 -- 172.21.15.69:0/2544626808 >> 172.21.15.69:0/2544626808 conn(0x55e8ef0f9d20 msgr2=0x55e8ef0fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 -- 172.21.15.69:0/2544626808 shutdown_connections 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.351+0000 7fee57027740 1 -- 172.21.15.69:0/2544626808 wait complete. 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 1 -- start start 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 1 Processor -- start 2024-08-02T17:51:16.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 0x55e8ef10acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef1109d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef10b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef104be0 con 0x55e8ef0fdb90 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef01d500 con 0x55e8ef111010 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e8ef104d60 con 0x55e8ef0fed40 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee57027740 10 client.0 _ll_register_callbacks cb 0x55e8eed74a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee51789640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 0x55e8ef10acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee51f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef1109d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee51789640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 0x55e8ef10acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58702/0 (socket says 172.21.15.69:58702) 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee51789640 1 -- 172.21.15.69:0/3894032994 learned_addr learned my addr 172.21.15.69:0/3894032994 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.352+0000 7fee51789640 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef10b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee51f8a640 1 -- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 msgr2=0x55e8ef10b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee51f8a640 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef10b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee51f8a640 1 -- 172.21.15.69:0/3894032994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 msgr2=0x55e8ef10acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee51f8a640 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 0x55e8ef10acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee51f8a640 1 -- 172.21.15.69:0/3894032994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8eee41620 con 0x55e8ef0fed40 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee51789640 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef10b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee51f8a640 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef1109d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee44005fd0 tx=0x7fee44011dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1b08261efc13de8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee317fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee317fa640 1 -- 172.21.15.69:0/3894032994 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4400fb70 con 0x55e8ef0fed40 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee57027740 1 -- 172.21.15.69:0/3894032994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c0132b0 con 0x55e8ef0fed40 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee317fa640 1 -- 172.21.15.69:0/3894032994 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fee4400fd10 con 0x55e8ef0fed40 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee317fa640 1 -- 172.21.15.69:0/3894032994 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee44024050 con 0x55e8ef0fed40 2024-08-02T17:51:16.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee57027740 10 client.26137 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee57027740 1 -- 172.21.15.69:0/3894032994 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8ef1283a0 con 0x55e8ef0fed40 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.353+0000 7fee317fa640 1 -- 172.21.15.69:0/3894032994 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c0132b0 con 0x55e8ef0fed40 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.354+0000 7fee317fa640 1 -- 172.21.15.69:0/3894032994 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fee44018460 con 0x55e8ef0fed40 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.354+0000 7fee57027740 10 client.26137 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.354+0000 7fee57027740 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.354+0000 7fee57027740 20 client.26137 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.354+0000 7fee57027740 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e8ef129860 0x55e8ef12bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.354+0000 7fee57027740 4 client.26137 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.354+0000 7fee57027740 1 -- 172.21.15.69:0/3894032994 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8ef06f0e0 con 0x55e8ef129860 2024-08-02T17:51:16.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.354+0000 7fee51789640 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e8ef129860 0x55e8ef12bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.356+0000 7fee51789640 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e8ef129860 0x55e8ef12bc70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2556fb94519a89 server_cookie=8d0341596d640c8f in_seq=0 out_seq=0 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.356+0000 7fee317fa640 10 client.26137 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.356+0000 7fee317fa640 1 -- 172.21.15.69:0/3894032994 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e8ef06f0e0 con 0x55e8ef129860 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.356+0000 7fee317fa640 10 client.26137 handle_command_reply: tid=0 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.356+0000 7fee57027740 1 client.26137 shutdown 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 -- 172.21.15.69:0/3894032994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 msgr2=0x55e8ef1109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef1109d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee44005fd0 tx=0x7fee44011dd0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 -- 172.21.15.69:0/3894032994 shutdown_connections 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e8ef129860 0x55e8ef12bc70 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef111010 0x55e8ef10b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef1109d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 --2- 172.21.15.69:0/3894032994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fdb90 0x55e8ef10acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 -- 172.21.15.69:0/3894032994 >> 172.21.15.69:0/3894032994 conn(0x55e8ef0f9d20 msgr2=0x55e8ef0ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee317fa640 0 client.26137 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 -- 172.21.15.69:0/3894032994 shutdown_connections 2024-08-02T17:51:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 1 -- 172.21.15.69:0/3894032994 wait complete. 2024-08-02T17:51:16.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.357+0000 7fee57027740 20 client.26137 trim_cache size 0 max 16384 2024-08-02T17:51:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.381+0000 7fee57027740 1 Processor -- start 2024-08-02T17:51:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.382+0000 7fee57027740 1 -- start start 2024-08-02T17:51:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.382+0000 7fee57027740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef1109b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.382+0000 7fee57027740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef100830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.382+0000 7fee57027740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef111010 0x55e8ef0fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.382+0000 7fee57027740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c1af360 con 0x55e8ef111010 2024-08-02T17:51:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.382+0000 7fee57027740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c1acc30 con 0x55e8ef0fdb90 2024-08-02T17:51:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.382+0000 7fee57027740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c1aaa80 con 0x55e8ef0fed40 2024-08-02T17:51:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.386+0000 7fee51f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef100830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.386+0000 7fee51f8a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef100830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33734/0 (socket says 172.21.15.69:33734) 2024-08-02T17:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.386+0000 7fee51f8a640 1 -- 172.21.15.69:0/4251533870 learned_addr learned my addr 172.21.15.69:0/4251533870 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.386+0000 7fee51789640 1 --2- 172.21.15.69:0/4251533870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef1109b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.387+0000 7fee51789640 1 -- 172.21.15.69:0/4251533870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 msgr2=0x55e8ef100830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.387+0000 7fee51789640 1 --2- 172.21.15.69:0/4251533870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef100830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.387+0000 7fee51789640 1 -- 172.21.15.69:0/4251533870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef111010 msgr2=0x55e8ef0fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.387+0000 7fee51789640 1 --2- 172.21.15.69:0/4251533870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef111010 0x55e8ef0fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.387+0000 7fee51789640 1 -- 172.21.15.69:0/4251533870 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8ef103a10 con 0x55e8ef0fdb90 2024-08-02T17:51:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.387+0000 7fee51789640 1 --2- 172.21.15.69:0/4251533870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef1109b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee3c005550 tx=0x7fee3c023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df0535e8a21b459b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.390+0000 7fee337fe640 1 -- 172.21.15.69:0/4251533870 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c009930 con 0x55e8ef0fdb90 2024-08-02T17:51:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.390+0000 7fee337fe640 1 -- 172.21.15.69:0/4251533870 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c005110 con 0x55e8ef0fdb90 2024-08-02T17:51:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.391+0000 7fee57027740 1 -- 172.21.15.69:0/4251533870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 msgr2=0x55e8ef1109b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.391+0000 7fee57027740 1 --2- 172.21.15.69:0/4251533870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef1109b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee3c005550 tx=0x7fee3c023040 comp rx=0 tx=0).stop 2024-08-02T17:51:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.395+0000 7fee57027740 1 -- 172.21.15.69:0/4251533870 shutdown_connections 2024-08-02T17:51:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.395+0000 7fee57027740 1 --2- 172.21.15.69:0/4251533870 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef111010 0x55e8ef0fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.395+0000 7fee57027740 1 --2- 172.21.15.69:0/4251533870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef0fed40 0x55e8ef100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.395+0000 7fee57027740 1 --2- 172.21.15.69:0/4251533870 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef1109b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.395+0000 7fee57027740 1 -- 172.21.15.69:0/4251533870 >> 172.21.15.69:0/4251533870 conn(0x55e8ef0f9d20 msgr2=0x55e8ef12b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.398+0000 7fee57027740 1 -- 172.21.15.69:0/4251533870 shutdown_connections 2024-08-02T17:51:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.399+0000 7fee57027740 1 -- 172.21.15.69:0/4251533870 wait complete. 2024-08-02T17:51:16.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.399+0000 7fee57027740 1 -- start start 2024-08-02T17:51:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.399+0000 7fee57027740 1 Processor -- start 2024-08-02T17:51:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.400+0000 7fee57027740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.400+0000 7fee57027740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fed40 0x55e8ef13d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.400+0000 7fee57027740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef111010 0x55e8ef13d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.400+0000 7fee57027740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c1ab0f0 con 0x55e8ef0fed40 2024-08-02T17:51:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.400+0000 7fee57027740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c1ac560 con 0x55e8ef0fdb90 2024-08-02T17:51:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.400+0000 7fee57027740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c1ab7c0 con 0x55e8ef111010 2024-08-02T17:51:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.400+0000 7fee57027740 10 client.0 _ll_register_callbacks cb 0x7fee44009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.401+0000 7fee51789640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.401+0000 7fee51789640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef102ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33396/0 (socket says 172.21.15.69:33396) 2024-08-02T17:51:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.401+0000 7fee51789640 1 -- 172.21.15.69:0/853905101 learned_addr learned my addr 172.21.15.69:0/853905101 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.401+0000 7fee51789640 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef111010 0x55e8ef13d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.402+0000 7fee51f8a640 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fed40 0x55e8ef13d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.402+0000 7fee51789640 1 -- 172.21.15.69:0/853905101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 msgr2=0x55e8ef102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.402+0000 7fee51789640 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.402+0000 7fee51789640 1 -- 172.21.15.69:0/853905101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fed40 msgr2=0x55e8ef13d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.402+0000 7fee51789640 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fed40 0x55e8ef13d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.402+0000 7fee51789640 1 -- 172.21.15.69:0/853905101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8ef13dee0 con 0x55e8ef111010 2024-08-02T17:51:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.402+0000 7fee51f8a640 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fed40 0x55e8ef13d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.405+0000 7fee51789640 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef111010 0x55e8ef13d7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee3c013430 tx=0x7fee3c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68558ee2f0db29e2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.405+0000 7fee317fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:51:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.406+0000 7fee317fa640 1 -- 172.21.15.69:0/853905101 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c003b90 con 0x55e8ef111010 2024-08-02T17:51:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.406+0000 7fee57027740 1 -- 172.21.15.69:0/853905101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee3c005110 con 0x55e8ef111010 2024-08-02T17:51:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.407+0000 7fee317fa640 1 -- 172.21.15.69:0/853905101 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fee3c003d30 con 0x55e8ef111010 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.407+0000 7fee317fa640 1 -- 172.21.15.69:0/853905101 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee3c045050 con 0x55e8ef111010 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.407+0000 7fee57027740 10 client.26143 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.407+0000 7fee57027740 1 -- 172.21.15.69:0/853905101 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e8ef1a5440 con 0x55e8ef111010 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.408+0000 7fee317fa640 1 -- 172.21.15.69:0/853905101 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee3c033040 con 0x55e8ef111010 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.408+0000 7fee317fa640 1 -- 172.21.15.69:0/853905101 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fee3c02d7e0 con 0x55e8ef111010 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.408+0000 7fee57027740 10 client.26143 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.408+0000 7fee57027740 10 client.26143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.408+0000 7fee57027740 20 client.26143 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.408+0000 7fee57027740 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e8ef1a59b0 0x55e8ef1a7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.408+0000 7fee57027740 4 client.26143 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:51:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.408+0000 7fee57027740 1 -- 172.21.15.69:0/853905101 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fee3c005110 con 0x55e8ef1a59b0 2024-08-02T17:51:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.410+0000 7fee51f8a640 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e8ef1a59b0 0x55e8ef1a7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.414+0000 7fee51f8a640 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e8ef1a59b0 0x55e8ef1a7dd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1456da625315d31 server_cookie=3bb4adecb73b4e0d in_seq=0 out_seq=0 2024-08-02T17:51:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.414+0000 7fee317fa640 10 client.26143 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:16.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.450+0000 7fee317fa640 1 -- 172.21.15.69:0/853905101 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fee3c005110 con 0x55e8ef1a59b0 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.450+0000 7fee317fa640 10 client.26143 handle_command_reply: tid=0 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.450+0000 7fee57027740 1 client.26143 shutdown 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.450+0000 7fee57027740 1 -- 172.21.15.69:0/853905101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef111010 msgr2=0x55e8ef13d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.450+0000 7fee57027740 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef111010 0x55e8ef13d7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fee3c013430 tx=0x7fee3c00ef60 comp rx=0 tx=0).stop 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 1 -- 172.21.15.69:0/853905101 shutdown_connections 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e8ef0fed40 0x55e8ef13d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e8ef1a59b0 0x55e8ef1a7dd0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e8ef111010 0x55e8ef13d7f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee317fa640 0 client.26143 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 1 --2- 172.21.15.69:0/853905101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e8ef0fdb90 0x55e8ef102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 1 -- 172.21.15.69:0/853905101 >> 172.21.15.69:0/853905101 conn(0x55e8ef0f9d20 msgr2=0x55e8ef0fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:16.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 1 -- 172.21.15.69:0/853905101 shutdown_connections 2024-08-02T17:51:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 1 -- 172.21.15.69:0/853905101 wait complete. 2024-08-02T17:51:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.451+0000 7fee57027740 20 client.26143 trim_cache size 0 max 16384 2024-08-02T17:51:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:51:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:51:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:51:16.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:51:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 -- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee340a2e30 msgr2=0x7fee340a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee340a2e30 0x7fee340a52f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fee400b80f0 tx=0x7fee400b1120 comp rx=0 tx=0).stop 2024-08-02T17:51:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 -- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c106730 msgr2=0x7fee4c1a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c106730 0x7fee4c1a3f70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fee40004770 tx=0x7fee40003880 comp rx=0 tx=0).stop 2024-08-02T17:51:16.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 -- 172.21.15.69:0/1929954864 shutdown_connections 2024-08-02T17:51:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fee4c106730 0x7fee4c1a3f70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fee340a2e30 0x7fee340a52f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fee4c1010c0 0x7fee4c199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 --2- 172.21.15.69:0/1929954864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fee4c100720 0x7fee4c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 -- 172.21.15.69:0/1929954864 >> 172.21.15.69:0/1929954864 conn(0x7fee4c0fc420 msgr2=0x7fee4c0fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 -- 172.21.15.69:0/1929954864 shutdown_connections 2024-08-02T17:51:16.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:16.453+0000 7fee541ee640 1 -- 172.21.15.69:0/1929954864 wait complete. 2024-08-02T17:51:16.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:16 smithi083 ceph-mon[37031]: pgmap v148: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.5 MiB/s wr, 555 op/s 2024-08-02T17:51:16.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:16 smithi077 ceph-mon[37108]: pgmap v148: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.5 MiB/s wr, 555 op/s 2024-08-02T17:51:16.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:16 smithi069 ceph-mon[32746]: pgmap v148: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.5 MiB/s wr, 555 op/s 2024-08-02T17:51:18.409 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:18 smithi069 ceph-mon[32746]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 1.13k op/s 2024-08-02T17:51:18.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:18 smithi083 ceph-mon[37031]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 1.13k op/s 2024-08-02T17:51:18.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:18 smithi077 ceph-mon[37108]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 17 MiB/s wr, 1.13k op/s 2024-08-02T17:51:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 250.0 s, 1638.4 tps, lat 63.394 ms stddev 97.512 2024-08-02T17:51:20.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:20 smithi083 ceph-mon[37031]: pgmap v150: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 958 op/s 2024-08-02T17:51:20.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:20 smithi077 ceph-mon[37108]: pgmap v150: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 958 op/s 2024-08-02T17:51:20.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:20 smithi069 ceph-mon[32746]: pgmap v150: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 16 MiB/s wr, 958 op/s 2024-08-02T17:51:20.905 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:51:21.378 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.240+0000 7f6525ffb640 1 -- 172.21.15.69:0/3439647439 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c0319b0 con 0x7f6528102a60 2024-08-02T17:51:22.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.240+0000 7f652d92d640 1 -- 172.21.15.69:0/3439647439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6528102a60 msgr2=0x7f6528102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.240+0000 7f652d92d640 1 --2- 172.21.15.69:0/3439647439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6528102a60 0x7f6528102e40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f651c009900 tx=0x7f651c02a480 comp rx=0 tx=0).stop 2024-08-02T17:51:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.241+0000 7f652d92d640 1 -- 172.21.15.69:0/3439647439 shutdown_connections 2024-08-02T17:51:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.241+0000 7f652d92d640 1 --2- 172.21.15.69:0/3439647439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f652810be90 0x7f652810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.241+0000 7f652d92d640 1 --2- 172.21.15.69:0/3439647439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6528103380 0x7f652810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.241+0000 7f652d92d640 1 --2- 172.21.15.69:0/3439647439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6528102a60 0x7f6528102e40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.241+0000 7f652d92d640 1 -- 172.21.15.69:0/3439647439 >> 172.21.15.69:0/3439647439 conn(0x7f65280fc500 msgr2=0x7f65280fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.241+0000 7f652d92d640 1 -- 172.21.15.69:0/3439647439 shutdown_connections 2024-08-02T17:51:22.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.241+0000 7f652d92d640 1 -- 172.21.15.69:0/3439647439 wait complete. 2024-08-02T17:51:22.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.242+0000 7f652d92d640 1 Processor -- start 2024-08-02T17:51:22.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.242+0000 7f652d92d640 1 -- start start 2024-08-02T17:51:22.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.242+0000 7f652d92d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6528102a60 0x7f65281006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.242+0000 7f652d92d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6528103380 0x7f6528100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.242+0000 7f652d92d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f652810be90 0x7f6528101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.242+0000 7f652d92d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6528111ce0 con 0x7f6528103380 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.242+0000 7f652d92d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6528111b60 con 0x7f6528102a60 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.242+0000 7f652d92d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6528111e60 con 0x7f652810be90 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6528102a60 0x7f65281006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f65277fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f652810be90 0x7f6528101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6528102a60 0x7f65281006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33418/0 (socket says 172.21.15.69:33418) 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 -- 172.21.15.69:0/1727524732 learned_addr learned my addr 172.21.15.69:0/1727524732 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f65267fc640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6528103380 0x7f6528100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 -- 172.21.15.69:0/1727524732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f652810be90 msgr2=0x7f6528101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f652810be90 0x7f6528101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 -- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6528103380 msgr2=0x7f6528100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6528103380 0x7f6528100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 -- 172.21.15.69:0/1727524732 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65281a6600 con 0x7f6528102a60 2024-08-02T17:51:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.243+0000 7f6526ffd640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6528102a60 0x7f65281006c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f651c004770 tx=0x7f651c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.244+0000 7f652c92b640 1 -- 172.21.15.69:0/1727524732 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c03f070 con 0x7f6528102a60 2024-08-02T17:51:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.244+0000 7f652d92d640 1 -- 172.21.15.69:0/1727524732 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65281a6890 con 0x7f6528102a60 2024-08-02T17:51:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.244+0000 7f652d92d640 1 -- 172.21.15.69:0/1727524732 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65281a6dc0 con 0x7f6528102a60 2024-08-02T17:51:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.244+0000 7f652c92b640 1 -- 172.21.15.69:0/1727524732 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651c031ca0 con 0x7f6528102a60 2024-08-02T17:51:22.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.244+0000 7f652c92b640 1 -- 172.21.15.69:0/1727524732 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c049060 con 0x7f6528102a60 2024-08-02T17:51:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.248+0000 7f652c92b640 1 -- 172.21.15.69:0/1727524732 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f651c042070 con 0x7f6528102a60 2024-08-02T17:51:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.248+0000 7f652fd2b740 1 Processor -- start 2024-08-02T17:51:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f652fd2b740 1 -- start start 2024-08-02T17:51:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e01eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e01eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e01eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58744/0 (socket says 172.21.15.69:58744) 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f65277fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f65277fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33428/0 (socket says 172.21.15.69:33428) 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 0x563ed1e047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f6526ffd640 1 -- 172.21.15.69:0/2593692142 learned_addr learned my addr 172.21.15.69:0/2593692142 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f652fd2b740 1 -- 172.21.15.69:0/2593692142 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563ed1d5c980 con 0x563ed1df7b30 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f652fd2b740 1 -- 172.21.15.69:0/2593692142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563ed1cf4a70 con 0x563ed1df7210 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f652fd2b740 1 -- 172.21.15.69:0/2593692142 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563ed1d58940 con 0x563ed1e023f0 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.249+0000 7f652c92b640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65100a2cf0 0x7f65100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f65277fe640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65100a2cf0 0x7f65100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f65277fe640 1 -- 172.21.15.69:0/2593692142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 msgr2=0x563ed1e047e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f65277fe640 1 --2- 172.21.15.69:0/2593692142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 0x563ed1e047e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f65277fe640 1 -- 172.21.15.69:0/2593692142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 msgr2=0x563ed1e01eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f65277fe640 1 --2- 172.21.15.69:0/2593692142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e01eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f65277fe640 1 -- 172.21.15.69:0/2593692142 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed1d576e0 con 0x563ed1df7210 2024-08-02T17:51:22.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f6526ffd640 1 --2- 172.21.15.69:0/2593692142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e01eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:22.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f65277fe640 1 --2- 172.21.15.69:0/2593692142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df75f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6518013550 tx=0x7f6518013a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=992f46e978f664ab server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:22.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f652c92b640 1 -- 172.21.15.69:0/1727524732 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f651c0efa70 con 0x7f6528102a60 2024-08-02T17:51:22.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f650dffb640 1 -- 172.21.15.69:0/2593692142 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6518022020 con 0x563ed1df7210 2024-08-02T17:51:22.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f650dffb640 1 -- 172.21.15.69:0/2593692142 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651800fd50 con 0x563ed1df7210 2024-08-02T17:51:22.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.250+0000 7f650dffb640 1 -- 172.21.15.69:0/2593692142 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651801e420 con 0x563ed1df7210 2024-08-02T17:51:22.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.251+0000 7f65277fe640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65100a2cf0 0x7f65100a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6518023000 tx=0x7f6518021000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.251+0000 7f652fd2b740 1 -- 172.21.15.69:0/2593692142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 msgr2=0x563ed1df75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.251+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2593692142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df75f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6518013550 tx=0x7f6518013a10 comp rx=0 tx=0).stop 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.252+0000 7f652fd2b740 1 -- 172.21.15.69:0/2593692142 shutdown_connections 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.252+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2593692142 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e01eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.252+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2593692142 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 0x563ed1e047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.252+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2593692142 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df75f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.252+0000 7f652fd2b740 1 -- 172.21.15.69:0/2593692142 >> 172.21.15.69:0/2593692142 conn(0x563ed1df37a0 msgr2=0x563ed1df3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.252+0000 7f652fd2b740 1 -- 172.21.15.69:0/2593692142 shutdown_connections 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.252+0000 7f652fd2b740 1 -- 172.21.15.69:0/2593692142 wait complete. 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.252+0000 7f652fd2b740 1 -- start start 2024-08-02T17:51:22.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.253+0000 7f652fd2b740 1 Processor -- start 2024-08-02T17:51:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.253+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7210 0x563ed1dff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.253+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7b30 0x563ed1dff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.253+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 0x563ed1dffd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.253+0000 7f652fd2b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563ed1e04fc0 con 0x563ed1e023f0 2024-08-02T17:51:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.253+0000 7f652fd2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563ed1d17aa0 con 0x563ed1df7b30 2024-08-02T17:51:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f652fd2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563ed1e05140 con 0x563ed1df7210 2024-08-02T17:51:22.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65267fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7b30 0x563ed1dff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65267fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7b30 0x563ed1dff800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33438/0 (socket says 172.21.15.69:33438) 2024-08-02T17:51:22.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65267fc640 1 -- 172.21.15.69:0/2355798035 learned_addr learned my addr 172.21.15.69:0/2355798035 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:22.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65277fe640 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 0x563ed1dffd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f6526ffd640 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7210 0x563ed1dff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65267fc640 1 -- 172.21.15.69:0/2355798035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7210 msgr2=0x563ed1dff2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:22.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65267fc640 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7210 0x563ed1dff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65267fc640 1 -- 172.21.15.69:0/2355798035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 msgr2=0x563ed1dffd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65267fc640 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 0x563ed1dffd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.254+0000 7f65267fc640 1 -- 172.21.15.69:0/2355798035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed1cfb650 con 0x563ed1df7b30 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f652fd2b740 10 client.0 _ll_register_callbacks cb 0x563ed1d06e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f652fd2b740 1 -- 172.21.15.69:0/2355798035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f651800fd50 con 0x563ed1df7b30 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f65267fc640 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7b30 0x563ed1dff800 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65200118e0 tx=0x7f6520011da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69cb0479f7fb6e07 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f6506ffd640 10 client.26255 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f6506ffd640 1 -- 172.21.15.69:0/2355798035 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652001e060 con 0x563ed1df7b30 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f6506ffd640 1 -- 172.21.15.69:0/2355798035 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f652000f920 con 0x563ed1df7b30 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f6506ffd640 1 -- 172.21.15.69:0/2355798035 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6520025050 con 0x563ed1df7b30 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f652fd2b740 10 client.26255 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.255+0000 7f652fd2b740 1 -- 172.21.15.69:0/2355798035 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x563ed1ace3c0 con 0x563ed1df7b30 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f6506ffd640 1 -- 172.21.15.69:0/2355798035 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f652000fb20 con 0x563ed1df7b30 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f652fd2b740 10 client.26255 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:22.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f652fd2b740 10 client.26255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f652fd2b740 20 client.26255 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563ed1e11d10 0x563ed1e14140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f652fd2b740 4 client.26255 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f652fd2b740 1 -- 172.21.15.69:0/2355798035 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed1d69ef0 con 0x563ed1e11d10 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f6526ffd640 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563ed1e11d10 0x563ed1e14140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f6526ffd640 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563ed1e11d10 0x563ed1e14140 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261d2b1f71804000 server_cookie=c342861afbdded95 in_seq=0 out_seq=0 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.256+0000 7f6506ffd640 10 client.26255 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.257+0000 7f6506ffd640 1 -- 172.21.15.69:0/2355798035 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ed1d69ef0 con 0x563ed1e11d10 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.257+0000 7f6506ffd640 10 client.26255 handle_command_reply: tid=0 2024-08-02T17:51:22.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.257+0000 7f652fd2b740 1 client.26255 shutdown 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f652fd2b740 1 -- 172.21.15.69:0/2355798035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7b30 msgr2=0x563ed1dff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7b30 0x563ed1dff800 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65200118e0 tx=0x7f6520011da0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f652fd2b740 1 -- 172.21.15.69:0/2355798035 shutdown_connections 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 0x563ed1dffd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563ed1e11d10 0x563ed1e14140 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7b30 0x563ed1dff800 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f652fd2b740 1 --2- 172.21.15.69:0/2355798035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7210 0x563ed1dff2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f652fd2b740 1 -- 172.21.15.69:0/2355798035 >> 172.21.15.69:0/2355798035 conn(0x563ed1df37a0 msgr2=0x563ed1df94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:22.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.258+0000 7f6506ffd640 0 client.26255 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:22.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.259+0000 7f652fd2b740 1 -- 172.21.15.69:0/2355798035 shutdown_connections 2024-08-02T17:51:22.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.259+0000 7f652fd2b740 1 -- 172.21.15.69:0/2355798035 wait complete. 2024-08-02T17:51:22.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.259+0000 7f652fd2b740 20 client.26255 trim_cache size 0 max 16384 2024-08-02T17:51:22.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.286+0000 7f652fd2b740 1 Processor -- start 2024-08-02T17:51:22.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.286+0000 7f652fd2b740 1 -- start start 2024-08-02T17:51:22.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.286+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e19a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 0x563ed1e19f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f65277fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 0x563ed1e19f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f65277fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 0x563ed1e19f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33782/0 (socket says 172.21.15.69:33782) 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f65277fe640 1 -- 172.21.15.69:0/1469150808 learned_addr learned my addr 172.21.15.69:0/1469150808 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f652fd2b740 1 -- 172.21.15.69:0/1469150808 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6514012840 con 0x563ed1df7b30 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f652fd2b740 1 -- 172.21.15.69:0/1469150808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6514010110 con 0x563ed1df7210 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f65267fc640 1 --2- 172.21.15.69:0/1469150808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e19a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.287+0000 7f652fd2b740 1 -- 172.21.15.69:0/1469150808 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f651400df60 con 0x563ed1e023f0 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.288+0000 7f6526ffd640 1 -- 172.21.15.69:0/1469150808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 msgr2=0x563ed1e19f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.288+0000 7f6526ffd640 1 --2- 172.21.15.69:0/1469150808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 0x563ed1e19f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.288+0000 7f6526ffd640 1 -- 172.21.15.69:0/1469150808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 msgr2=0x563ed1e19a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:22.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.288+0000 7f6526ffd640 1 --2- 172.21.15.69:0/1469150808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e19a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.288+0000 7f6526ffd640 1 -- 172.21.15.69:0/1469150808 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed1df35d0 con 0x563ed1df7210 2024-08-02T17:51:22.291 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.288+0000 7f6526ffd640 1 --2- 172.21.15.69:0/1469150808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df9cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x563ed1df4600 tx=0x7f651c0bbad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f69eca4f0b70b6f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.289+0000 7f650d7fa640 1 -- 172.21.15.69:0/1469150808 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c0bbbd0 con 0x563ed1df7210 2024-08-02T17:51:22.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.289+0000 7f650d7fa640 1 -- 172.21.15.69:0/1469150808 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651c0bff80 con 0x563ed1df7210 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.289+0000 7f650d7fa640 1 -- 172.21.15.69:0/1469150808 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c0c43b0 con 0x563ed1df7210 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.290+0000 7f652fd2b740 1 -- 172.21.15.69:0/1469150808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 msgr2=0x563ed1df9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.290+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1469150808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df9cc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x563ed1df4600 tx=0x7f651c0bbad0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.290+0000 7f652fd2b740 1 -- 172.21.15.69:0/1469150808 shutdown_connections 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.290+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1469150808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1df7b30 0x563ed1e19a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.290+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1469150808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1e023f0 0x563ed1e19f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.290+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1469150808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1df9cc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.290+0000 7f652fd2b740 1 -- 172.21.15.69:0/1469150808 >> 172.21.15.69:0/1469150808 conn(0x563ed1df37a0 msgr2=0x563ed1dfe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.290+0000 7f652fd2b740 1 -- 172.21.15.69:0/1469150808 shutdown_connections 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.291+0000 7f652fd2b740 1 -- 172.21.15.69:0/1469150808 wait complete. 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.291+0000 7f652fd2b740 1 -- start start 2024-08-02T17:51:22.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.291+0000 7f652fd2b740 1 Processor -- start 2024-08-02T17:51:22.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1e05980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7b30 0x563ed1e05ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f652fd2b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 0x563ed1e7aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1e05980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f6526ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1e05980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33462/0 (socket says 172.21.15.69:33462) 2024-08-02T17:51:22.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f65267fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7b30 0x563ed1e05ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.297 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f65267fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7b30 0x563ed1e05ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33798/0 (socket says 172.21.15.69:33798) 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f6526ffd640 1 -- 172.21.15.69:0/1785601526 learned_addr learned my addr 172.21.15.69:0/1785601526 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f651400e5d0 con 0x563ed1e023f0 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f651400fa40 con 0x563ed1df7210 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f651400eca0 con 0x563ed1df7b30 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f65277fe640 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 0x563ed1e7aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f65267fc640 1 -- 172.21.15.69:0/1785601526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 msgr2=0x563ed1e05980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f65267fc640 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1e05980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f65267fc640 1 -- 172.21.15.69:0/1785601526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 msgr2=0x563ed1e7aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f65267fc640 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 0x563ed1e7aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f65267fc640 1 -- 172.21.15.69:0/1785601526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed1e7b000 con 0x563ed1df7b30 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.292+0000 7f652fd2b740 10 client.0 _ll_register_callbacks cb 0x7f6520009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:22.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f65267fc640 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7b30 0x563ed1e05ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x563ed1e00870 tx=0x7f6520004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baf9faa1f78625cf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f6506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f651c0bff80 con 0x563ed1df7b30 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f6506ffd640 1 -- 172.21.15.69:0/1785601526 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520003fa0 con 0x563ed1df7b30 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f6506ffd640 1 -- 172.21.15.69:0/1785601526 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f6520011030 con 0x563ed1df7b30 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f6506ffd640 1 -- 172.21.15.69:0/1785601526 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6520011290 con 0x563ed1df7b30 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f652fd2b740 10 client.26155 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x563ed1e7c1b0 con 0x563ed1df7b30 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f6506ffd640 1 -- 172.21.15.69:0/1785601526 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6520004210 con 0x563ed1df7b30 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.293+0000 7f652fd2b740 10 client.26155 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f652fd2b740 10 client.26155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f652fd2b740 20 client.26155 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563ed1e7c810 0x563ed1e7cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f652fd2b740 4 client.26155 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:51:22.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ed1d64400 con 0x563ed1e7c810 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f6526ffd640 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563ed1e7c810 0x563ed1e7cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f6506ffd640 10 client.26155 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f6526ffd640 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563ed1e7c810 0x563ed1e7cc20 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d654f8396f989976 server_cookie=a9eb467c03168f54 in_seq=0 out_seq=0 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.294+0000 7f6506ffd640 1 -- 172.21.15.69:0/1785601526 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563ed1d64400 con 0x563ed1e7c810 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f6506ffd640 10 client.26155 handle_command_reply: tid=0 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 client.26155 shutdown 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7b30 msgr2=0x563ed1e05ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7b30 0x563ed1e05ec0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x563ed1e00870 tx=0x7f6520004ab0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 shutdown_connections 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563ed1e023f0 0x563ed1e7aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.300 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563ed1e7c810 0x563ed1e7cc20 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f6506ffd640 0 client.26155 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563ed1df7b30 0x563ed1e05ec0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 --2- 172.21.15.69:0/1785601526 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563ed1df7210 0x563ed1e05980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 >> 172.21.15.69:0/1785601526 conn(0x563ed1df37a0 msgr2=0x563ed1df20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 shutdown_connections 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 1 -- 172.21.15.69:0/1785601526 wait complete. 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.295+0000 7f652fd2b740 20 client.26155 trim_cache size 0 max 16384 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:51:22.301 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:51:22.302 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:51:22.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 -- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65100a2cf0 msgr2=0x7f65100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65100a2cf0 0x7f65100a51b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6518023000 tx=0x7f6518021000 comp rx=0 tx=0).stop 2024-08-02T17:51:22.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 -- 172.21.15.69:0/1727524732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6528102a60 msgr2=0x7f65281006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6528102a60 0x7f65281006c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f651c004770 tx=0x7f651c003c10 comp rx=0 tx=0).stop 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 -- 172.21.15.69:0/1727524732 shutdown_connections 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6528103380 0x7f6528100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65100a2cf0 0x7f65100a51b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f652810be90 0x7f6528101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 --2- 172.21.15.69:0/1727524732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6528102a60 0x7f65281006c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 -- 172.21.15.69:0/1727524732 >> 172.21.15.69:0/1727524732 conn(0x7f65280fc500 msgr2=0x7f65280fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 -- 172.21.15.69:0/1727524732 shutdown_connections 2024-08-02T17:51:22.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:22.296+0000 7f652d92d640 1 -- 172.21.15.69:0/1727524732 wait complete. 2024-08-02T17:51:22.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:22 smithi083 ceph-mon[37031]: pgmap v151: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.00k op/s 2024-08-02T17:51:22.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:22 smithi077 ceph-mon[37108]: pgmap v151: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.00k op/s 2024-08-02T17:51:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:22 smithi069 ceph-mon[32746]: pgmap v151: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 MiB/s wr, 1.00k op/s 2024-08-02T17:51:23.475 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:51:23.475 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:51:23.894 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:24.281 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:24 smithi069 ceph-mon[32746]: pgmap v152: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-08-02T17:51:24.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.512+0000 7f42dd216640 1 -- 172.21.15.69:0/889383413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d8109180 msgr2=0x7f42d810ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.512+0000 7f42dd216640 1 --2- 172.21.15.69:0/889383413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d8109180 0x7f42d810ff80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009900 tx=0x7f42c802a410 comp rx=0 tx=0).stop 2024-08-02T17:51:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.512+0000 7f42dd216640 1 -- 172.21.15.69:0/889383413 shutdown_connections 2024-08-02T17:51:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.512+0000 7f42dd216640 1 --2- 172.21.15.69:0/889383413 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42d8109560 0x7f42d81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.512+0000 7f42dd216640 1 --2- 172.21.15.69:0/889383413 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d811ae70 0x7f42d811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.512+0000 7f42dd216640 1 --2- 172.21.15.69:0/889383413 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d8109180 0x7f42d810ff80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.512+0000 7f42dd216640 1 -- 172.21.15.69:0/889383413 >> 172.21.15.69:0/889383413 conn(0x7f42d806de70 msgr2=0x7f42d806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.513+0000 7f42dd216640 1 -- 172.21.15.69:0/889383413 shutdown_connections 2024-08-02T17:51:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.513+0000 7f42dd216640 1 -- 172.21.15.69:0/889383413 wait complete. 2024-08-02T17:51:24.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42dd216640 1 Processor -- start 2024-08-02T17:51:24.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42dd216640 1 -- start start 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42dd216640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d8109180 0x7f42d81a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42dd216640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42d8109560 0x7f42d81a7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42dd216640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d811ae70 0x7f42d81ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42dd216640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f42d8120b20 con 0x7f42d8109560 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42dd216640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f42d81209a0 con 0x7f42d8109180 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42dd216640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42d8120ca0 con 0x7f42d811ae70 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42d759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d811ae70 0x7f42d81ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42d6d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d8109180 0x7f42d81a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.514+0000 7f42d759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d811ae70 0x7f42d81ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:33816/0 (socket says 172.21.15.69:33816) 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d759e640 1 -- 172.21.15.69:0/1720459473 learned_addr learned my addr 172.21.15.69:0/1720459473 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d659c640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42d8109560 0x7f42d81a7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d6d9d640 1 -- 172.21.15.69:0/1720459473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d811ae70 msgr2=0x7f42d81ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:24.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d6d9d640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d811ae70 0x7f42d81ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d6d9d640 1 -- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42d8109560 msgr2=0x7f42d81a7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d6d9d640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42d8109560 0x7f42d81a7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d6d9d640 1 -- 172.21.15.69:0/1720459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d81abf90 con 0x7f42d8109180 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d759e640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d811ae70 0x7f42d81ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42d6d9d640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d8109180 0x7f42d81a6fe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f42c802a930 tx=0x7f42c8003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42bffff640 1 -- 172.21.15.69:0/1720459473 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c803f070 con 0x7f42d8109180 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42bffff640 1 -- 172.21.15.69:0/1720459473 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42c8031c90 con 0x7f42d8109180 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42dd216640 1 -- 172.21.15.69:0/1720459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d81ac220 con 0x7f42d8109180 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.515+0000 7f42bffff640 1 -- 172.21.15.69:0/1720459473 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c8035030 con 0x7f42d8109180 2024-08-02T17:51:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.516+0000 7f42dd216640 1 -- 172.21.15.69:0/1720459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d81b3b60 con 0x7f42d8109180 2024-08-02T17:51:24.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.517+0000 7f42dd216640 1 -- 172.21.15.69:0/1720459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d81a8340 con 0x7f42d8109180 2024-08-02T17:51:24.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.518+0000 7f42bffff640 1 -- 172.21.15.69:0/1720459473 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f42c8042070 con 0x7f42d8109180 2024-08-02T17:51:24.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:24 smithi083 ceph-mon[37031]: pgmap v152: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-08-02T17:51:24.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.519+0000 7f42bffff640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f42c00a2cf0 0x7f42c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:24.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.519+0000 7f42d659c640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f42c00a2cf0 0x7f42c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:24.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.520+0000 7f42bffff640 1 -- 172.21.15.69:0/1720459473 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f42c80efe80 con 0x7f42d8109180 2024-08-02T17:51:24.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.520+0000 7f42d659c640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f42c00a2cf0 0x7f42c00a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42d811d5d0 tx=0x7f42cc005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:24.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.524+0000 7f42bffff640 1 -- 172.21.15.69:0/1720459473 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f42c80bc680 con 0x7f42d8109180 2024-08-02T17:51:24.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:24 smithi077 ceph-mon[37108]: pgmap v152: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 17 MiB/s wr, 1.17k op/s 2024-08-02T17:51:24.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.866+0000 7f42dd216640 1 -- 172.21.15.69:0/1720459473 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42d802d5e0 con 0x7f42d8109180 2024-08-02T17:51:24.870 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:51:24.870 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:51:24.871 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:51:24.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.869+0000 7f42bffff640 1 -- 172.21.15.69:0/1720459473 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f42c80bbf80 con 0x7f42d8109180 2024-08-02T17:51:24.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.877+0000 7f42bdffb640 1 -- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f42c00a2cf0 msgr2=0x7f42c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:24.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.877+0000 7f42bdffb640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f42c00a2cf0 0x7f42c00a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f42d811d5d0 tx=0x7f42cc005f90 comp rx=0 tx=0).stop 2024-08-02T17:51:24.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.877+0000 7f42bdffb640 1 -- 172.21.15.69:0/1720459473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d8109180 msgr2=0x7f42d81a6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:24.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.877+0000 7f42bdffb640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d8109180 0x7f42d81a6fe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f42c802a930 tx=0x7f42c8003b20 comp rx=0 tx=0).stop 2024-08-02T17:51:24.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.878+0000 7f42bdffb640 1 -- 172.21.15.69:0/1720459473 shutdown_connections 2024-08-02T17:51:24.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.878+0000 7f42bdffb640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f42d8109560 0x7f42d81a7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:24.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.878+0000 7f42bdffb640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f42c00a2cf0 0x7f42c00a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:24.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.878+0000 7f42bdffb640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f42d811ae70 0x7f42d81ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:24.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.878+0000 7f42bdffb640 1 --2- 172.21.15.69:0/1720459473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f42d8109180 0x7f42d81a6fe0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f42c802a930 tx=0x7f42c8003b20 comp rx=0 tx=0).stop 2024-08-02T17:51:24.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.878+0000 7f42bdffb640 1 -- 172.21.15.69:0/1720459473 >> 172.21.15.69:0/1720459473 conn(0x7f42d806de70 msgr2=0x7f42d811ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:24.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.881+0000 7f42bdffb640 1 -- 172.21.15.69:0/1720459473 shutdown_connections 2024-08-02T17:51:24.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:24.881+0000 7f42bdffb640 1 -- 172.21.15.69:0/1720459473 wait complete. 2024-08-02T17:51:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 255.0 s, 1612.2 tps, lat 60.370 ms stddev 70.275 2024-08-02T17:51:25.508 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:25 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1720459473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:25.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:25 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1720459473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:25.544 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:51:25.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:25 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1720459473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:25.942 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:26.326 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:26 smithi069 ceph-mon[32746]: pgmap v153: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 923 op/s 2024-08-02T17:51:26.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 -- 172.21.15.69:0/1087691572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc1018b0 msgr2=0x7ff0bc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 --2- 172.21.15.69:0/1087691572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc1018b0 0x7ff0bc101d10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009930 tx=0x7ff0ac02a420 comp rx=0 tx=0).stop 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 -- 172.21.15.69:0/1087691572 shutdown_connections 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 --2- 172.21.15.69:0/1087691572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc1018b0 0x7ff0bc101d10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 --2- 172.21.15.69:0/1087691572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0bc102250 0x7ff0bc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 --2- 172.21.15.69:0/1087691572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0bc1078b0 0x7ff0bc107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 -- 172.21.15.69:0/1087691572 >> 172.21.15.69:0/1087691572 conn(0x7ff0bc0fd5b0 msgr2=0x7ff0bc0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 -- 172.21.15.69:0/1087691572 shutdown_connections 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.453+0000 7ff0c3659640 1 -- 172.21.15.69:0/1087691572 wait complete. 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c3659640 1 Processor -- start 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c3659640 1 -- start start 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c3659640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0bc1018b0 0x7ff0bc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c3659640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc102250 0x7ff0bc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c3659640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0bc1078b0 0x7ff0bc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c3659640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc111cb0 con 0x7ff0bc102250 2024-08-02T17:51:26.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c3659640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc111b30 con 0x7ff0bc1018b0 2024-08-02T17:51:26.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c3659640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc111e30 con 0x7ff0bc1078b0 2024-08-02T17:51:26.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c13f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0bc1018b0 0x7ff0bc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c0bf4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc102250 0x7ff0bc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c13f5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0bc1018b0 0x7ff0bc198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44046/0 (socket says 172.21.15.69:44046) 2024-08-02T17:51:26.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.454+0000 7ff0c13f5640 1 -- 172.21.15.69:0/378494528 learned_addr learned my addr 172.21.15.69:0/378494528 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:26.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.455+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0bc1078b0 0x7ff0bc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.456+0000 7ff0c0bf4640 1 -- 172.21.15.69:0/378494528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0bc1078b0 msgr2=0x7ff0bc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.456+0000 7ff0c0bf4640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0bc1078b0 0x7ff0bc19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.456+0000 7ff0c0bf4640 1 -- 172.21.15.69:0/378494528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0bc1018b0 msgr2=0x7ff0bc198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.456+0000 7ff0c0bf4640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0bc1018b0 0x7ff0bc198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.456+0000 7ff0c0bf4640 1 -- 172.21.15.69:0/378494528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc19d140 con 0x7ff0bc102250 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.456+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0bc1018b0 0x7ff0bc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.456+0000 7ff0c0bf4640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc102250 0x7ff0bc1986d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005940 tx=0x7ff0ac0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.457+0000 7ff0b27fc640 1 -- 172.21.15.69:0/378494528 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac03f070 con 0x7ff0bc102250 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.457+0000 7ff0c3659640 1 -- 172.21.15.69:0/378494528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0bc19d3d0 con 0x7ff0bc102250 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.457+0000 7ff0c3659640 1 -- 172.21.15.69:0/378494528 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0bc1a4d10 con 0x7ff0bc102250 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.457+0000 7ff0b27fc640 1 -- 172.21.15.69:0/378494528 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ac0044c0 con 0x7ff0bc102250 2024-08-02T17:51:26.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.457+0000 7ff0b27fc640 1 -- 172.21.15.69:0/378494528 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac03f070 con 0x7ff0bc102250 2024-08-02T17:51:26.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.460+0000 7ff0c5a57740 1 Processor -- start 2024-08-02T17:51:26.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.460+0000 7ff0c5a57740 1 -- start start 2024-08-02T17:51:26.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e212605a40 con 0x55e21269fff0 2024-08-02T17:51:26.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e21259d9d0 con 0x55e2126a0910 2024-08-02T17:51:26.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e212601a00 con 0x55e2126ab1d0 2024-08-02T17:51:26.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0b27fc640 1 -- 172.21.15.69:0/378494528 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff0ac0050a0 con 0x7ff0bc102250 2024-08-02T17:51:26.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56464/0 (socket says 172.21.15.69:56464) 2024-08-02T17:51:26.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c1bf6640 1 -- 172.21.15.69:0/439244976 learned_addr learned my addr 172.21.15.69:0/439244976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:26.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.461+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c13f5640 1 -- 172.21.15.69:0/439244976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 msgr2=0x55e2126ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:26.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c0bf4640 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126ad5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c13f5640 1 -- 172.21.15.69:0/439244976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 msgr2=0x55e2126a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c13f5640 1 -- 172.21.15.69:0/439244976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2125a4650 con 0x55e2126a0910 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0b27fc640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff0a80a2ef0 0x7ff0a80a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126aac90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8011440 tx=0x7ff0b8011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9edf1858162f397 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0b27fc640 1 -- 172.21.15.69:0/378494528 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff0ac0efab0 con 0x7ff0bc102250 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0a77fe640 1 -- 172.21.15.69:0/439244976 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8004030 con 0x55e2126a0910 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff0a80a2ef0 0x7ff0a80a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.462+0000 7ff0a77fe640 1 -- 172.21.15.69:0/439244976 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b80041d0 con 0x55e2126a0910 2024-08-02T17:51:26.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0a77fe640 1 -- 172.21.15.69:0/439244976 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b800f990 con 0x55e2126a0910 2024-08-02T17:51:26.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 -- 172.21.15.69:0/439244976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 msgr2=0x55e2126aac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff0a80a2ef0 0x7ff0a80a53b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009fd0 tx=0x7ff0b40097e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:26.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126aac90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8011440 tx=0x7ff0b8011900 comp rx=0 tx=0).stop 2024-08-02T17:51:26.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 -- 172.21.15.69:0/439244976 shutdown_connections 2024-08-02T17:51:26.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126aac90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/439244976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 -- 172.21.15.69:0/439244976 >> 172.21.15.69:0/439244976 conn(0x55e21269c580 msgr2=0x55e21269c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 -- 172.21.15.69:0/439244976 shutdown_connections 2024-08-02T17:51:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.463+0000 7ff0c5a57740 1 -- 172.21.15.69:0/439244976 wait complete. 2024-08-02T17:51:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.464+0000 7ff0c5a57740 1 -- start start 2024-08-02T17:51:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.464+0000 7ff0c5a57740 1 Processor -- start 2024-08-02T17:51:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.464+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.464+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.464+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126a8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.464+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e2126ade40 con 0x55e21269fff0 2024-08-02T17:51:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.464+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e2125c0b20 con 0x55e2126a0910 2024-08-02T17:51:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.464+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e2126adfc0 con 0x55e2126ab1d0 2024-08-02T17:51:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.465+0000 7ff0c0bf4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.465+0000 7ff0c0bf4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126a7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44062/0 (socket says 172.21.15.69:44062) 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.465+0000 7ff0c0bf4640 1 -- 172.21.15.69:0/2269839577 learned_addr learned my addr 172.21.15.69:0/2269839577 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.465+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.465+0000 7ff0c5a57740 10 client.0 _ll_register_callbacks cb 0x55e2125bebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.466+0000 7ff0c0bf4640 1 -- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 msgr2=0x55e2126a8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.466+0000 7ff0c0bf4640 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126a8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.466+0000 7ff0c0bf4640 1 -- 172.21.15.69:0/2269839577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 msgr2=0x55e2126aaa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.466+0000 7ff0c0bf4640 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.466+0000 7ff0c0bf4640 1 -- 172.21.15.69:0/2269839577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2123773c0 con 0x55e2126a0910 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.466+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0c0bf4640 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126a7fd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006010 tx=0x7ff0ac0bb810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3c2fc9e1da59b6f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/2269839577 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac035040 con 0x55e2126a0910 2024-08-02T17:51:26.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/2269839577 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff0ac0c43f0 con 0x55e2126a0910 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2269839577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b80041d0 con 0x55e2126a0910 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/2269839577 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0ac0ed2c0 con 0x55e2126a0910 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0c5a57740 10 client.26279 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2269839577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e2126b1f70 con 0x55e2126a0910 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.467+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/2269839577 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ac0d2030 con 0x55e2126a0910 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/2269839577 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0ac005390 con 0x55e2126a0910 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0c5a57740 10 client.26279 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0c5a57740 10 client.26279 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0c5a57740 20 client.26279 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e2126d47b0 0x55e2126d6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0c5a57740 4 client.26279 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2269839577 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e212612fb0 con 0x55e2126d47b0 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e2126d47b0 0x55e2126d6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0a4ff9640 10 client.26279 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.468+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e2126d47b0 0x55e2126d6c50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0313ba53fc40253 server_cookie=f71a845dff6a2cc6 in_seq=0 out_seq=0 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/2269839577 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e212612fb0 con 0x55e2126d47b0 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0a4ff9640 10 client.26279 handle_command_reply: tid=0 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 client.26279 shutdown 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2269839577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 msgr2=0x55e2126a7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126a7fd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac006010 tx=0x7ff0ac0bb810 comp rx=0 tx=0).stop 2024-08-02T17:51:26.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2269839577 shutdown_connections 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e2126d47b0 0x55e2126d6c50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126ab1d0 0x55e2126a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0a4ff9640 0 client.26279 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126a7fd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2269839577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126aaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.469+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2269839577 >> 172.21.15.69:0/2269839577 conn(0x55e21269c580 msgr2=0x55e2126a22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.470+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2269839577 shutdown_connections 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.470+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2269839577 wait complete. 2024-08-02T17:51:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.470+0000 7ff0c5a57740 20 client.26279 trim_cache size 0 max 16384 2024-08-02T17:51:26.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.486+0000 7ff0c5a57740 1 Processor -- start 2024-08-02T17:51:26.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.487+0000 7ff0c5a57740 1 -- start start 2024-08-02T17:51:26.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.487+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e21269fff0 0x55e2126dc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.487+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.487+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2126ab1d0 0x55e2126a9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a00127f0 con 0x55e2126ab1d0 2024-08-02T17:51:26.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2126ab1d0 0x55e2126a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c0bf4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c0bf4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126dca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44070/0 (socket says 172.21.15.69:44070) 2024-08-02T17:51:26.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2126ab1d0 0x55e2126a9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56474/0 (socket says 172.21.15.69:56474) 2024-08-02T17:51:26.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c0bf4640 1 -- 172.21.15.69:0/2907842566 learned_addr learned my addr 172.21.15.69:0/2907842566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:26.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/2907842566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e21269fff0 0x55e2126dc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a00100c0 con 0x55e2126a0910 2024-08-02T17:51:26.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.488+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2907842566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a000df10 con 0x55e21269fff0 2024-08-02T17:51:26.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.489+0000 7ff0c1bf6640 1 -- 172.21.15.69:0/2907842566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e21269fff0 msgr2=0x55e2126dc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:26.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.489+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/2907842566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e21269fff0 0x55e2126dc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.489+0000 7ff0c1bf6640 1 -- 172.21.15.69:0/2907842566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 msgr2=0x55e2126dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:26.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.489+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/2907842566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126dca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.489+0000 7ff0c1bf6640 1 -- 172.21.15.69:0/2907842566 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2126d72d0 con 0x55e2126ab1d0 2024-08-02T17:51:26.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.489+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/2907842566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2126ab1d0 0x55e2126a9720 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009af0 tx=0x7ff0b4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16e3156cd1cdd67 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:26.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.490+0000 7ff0a6ffd640 1 -- 172.21.15.69:0/2907842566 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b402c070 con 0x55e2126ab1d0 2024-08-02T17:51:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.491+0000 7ff0a6ffd640 1 -- 172.21.15.69:0/2907842566 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b4010de0 con 0x55e2126ab1d0 2024-08-02T17:51:26.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.491+0000 7ff0a6ffd640 1 -- 172.21.15.69:0/2907842566 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b401e650 con 0x55e2126ab1d0 2024-08-02T17:51:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.492+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2907842566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2126ab1d0 msgr2=0x55e2126a9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.492+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2907842566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2126ab1d0 0x55e2126a9720 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009af0 tx=0x7ff0b4015000 comp rx=0 tx=0).stop 2024-08-02T17:51:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.493+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2907842566 shutdown_connections 2024-08-02T17:51:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.493+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2907842566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2126ab1d0 0x55e2126a9720 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.493+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2907842566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126a0910 0x55e2126dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.493+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/2907842566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e21269fff0 0x55e2126dc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.493+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2907842566 >> 172.21.15.69:0/2907842566 conn(0x55e21269c580 msgr2=0x55e2126971b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:26.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.500+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2907842566 shutdown_connections 2024-08-02T17:51:26.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.502+0000 7ff0c5a57740 1 -- 172.21.15.69:0/2907842566 wait complete. 2024-08-02T17:51:26.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.502+0000 7ff0c5a57740 1 -- start start 2024-08-02T17:51:26.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.503+0000 7ff0c5a57740 1 Processor -- start 2024-08-02T17:51:26.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.503+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.504+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126a0910 0x55e2126a55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.504+0000 7ff0c5a57740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126ab1d0 0x55e2126a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.504+0000 7ff0c5a57740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a000e580 con 0x55e21269fff0 2024-08-02T17:51:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.504+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126ab1d0 0x55e2126a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.504+0000 7ff0c1bf6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126ab1d0 0x55e2126a5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44080/0 (socket says 172.21.15.69:44080) 2024-08-02T17:51:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.504+0000 7ff0c1bf6640 1 -- 172.21.15.69:0/1472368309 learned_addr learned my addr 172.21.15.69:0/1472368309 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.505+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a000f9f0 con 0x55e2126ab1d0 2024-08-02T17:51:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.505+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a000ec50 con 0x55e2126a0910 2024-08-02T17:51:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.505+0000 7ff0c5a57740 10 client.0 _ll_register_callbacks cb 0x7ff0ac009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.506+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.506+0000 7ff0c1bf6640 1 -- 172.21.15.69:0/1472368309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126a0910 msgr2=0x55e2126a55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.506+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126a0910 0x55e2126a55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.506+0000 7ff0c1bf6640 1 -- 172.21.15.69:0/1472368309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 msgr2=0x55e2126a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.506+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.506+0000 7ff0c1bf6640 1 -- 172.21.15.69:0/1472368309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2126be520 con 0x55e2126ab1d0 2024-08-02T17:51:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.507+0000 7ff0c1bf6640 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126ab1d0 0x55e2126a5b20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4027c10 tx=0x7ff0b4027c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89df10a50032929 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.507+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0b4010de0 con 0x55e2126ab1d0 2024-08-02T17:51:26.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.510+0000 7ff0a4ff9640 10 client.26285 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:26.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.510+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/1472368309 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b402c040 con 0x55e2126ab1d0 2024-08-02T17:51:26.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.510+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/1472368309 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff0b4024040 con 0x55e2126ab1d0 2024-08-02T17:51:26.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.510+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/1472368309 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0b403f050 con 0x55e2126ab1d0 2024-08-02T17:51:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.510+0000 7ff0c5a57740 10 client.26285 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.510+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55e21273f670 con 0x55e2126ab1d0 2024-08-02T17:51:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.513+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/1472368309 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0b400f040 con 0x55e2126ab1d0 2024-08-02T17:51:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.513+0000 7ff0c5a57740 10 client.26285 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.513+0000 7ff0c5a57740 10 client.26285 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:51:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.513+0000 7ff0c5a57740 20 client.26285 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.513+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e21273fb60 0x55e212741fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.513+0000 7ff0c5a57740 4 client.26285 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:51:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.513+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff0b4010de0 con 0x55e21273fb60 2024-08-02T17:51:26.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.514+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e21273fb60 0x55e212741fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:26.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.517+0000 7ff0c13f5640 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e21273fb60 0x55e212741fc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f59081be6e2dca97 server_cookie=b4440b35ed6aa5b5 in_seq=0 out_seq=0 2024-08-02T17:51:26.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.517+0000 7ff0a4ff9640 10 client.26285 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:51:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.517+0000 7ff0a4ff9640 1 -- 172.21.15.69:0/1472368309 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff0b4010de0 con 0x55e21273fb60 2024-08-02T17:51:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.517+0000 7ff0a4ff9640 10 client.26285 handle_command_reply: tid=0 2024-08-02T17:51:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 client.26285 shutdown 2024-08-02T17:51:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126ab1d0 msgr2=0x55e2126a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126ab1d0 0x55e2126a5b20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4027c10 tx=0x7ff0b4027c40 comp rx=0 tx=0).stop 2024-08-02T17:51:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 shutdown_connections 2024-08-02T17:51:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55e21273fb60 0x55e212741fc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2126ab1d0 0x55e2126a5b20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e2126a0910 0x55e2126a55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 --2- 172.21.15.69:0/1472368309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e21269fff0 0x55e2126a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 >> 172.21.15.69:0/1472368309 conn(0x55e21269c580 msgr2=0x55e21269ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:26.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0a4ff9640 0 client.26285 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:51:26.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.518+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 shutdown_connections 2024-08-02T17:51:26.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.519+0000 7ff0c5a57740 1 -- 172.21.15.69:0/1472368309 wait complete. 2024-08-02T17:51:26.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.519+0000 7ff0c5a57740 20 client.26285 trim_cache size 0 max 16384 2024-08-02T17:51:26.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:26 smithi083 ceph-mon[37031]: pgmap v153: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 923 op/s 2024-08-02T17:51:26.522 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:51:26.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 -- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff0a80a2ef0 msgr2=0x7ff0a80a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff0a80a2ef0 0x7ff0a80a53b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009fd0 tx=0x7ff0b40097e0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 -- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc102250 msgr2=0x7ff0bc1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc102250 0x7ff0bc1986d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005940 tx=0x7ff0ac0040c0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 -- 172.21.15.69:0/378494528 shutdown_connections 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff0bc102250 0x7ff0bc1986d0 secure :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005940 tx=0x7ff0ac0040c0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff0a80a2ef0 0x7ff0a80a53b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff0bc1078b0 0x7ff0bc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 --2- 172.21.15.69:0/378494528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff0bc1018b0 0x7ff0bc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 -- 172.21.15.69:0/378494528 >> 172.21.15.69:0/378494528 conn(0x7ff0bc0fd5b0 msgr2=0x7ff0bc0fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 -- 172.21.15.69:0/378494528 shutdown_connections 2024-08-02T17:51:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:26.520+0000 7ff0c3659640 1 -- 172.21.15.69:0/378494528 wait complete. 2024-08-02T17:51:26.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:26 smithi077 ceph-mon[37108]: pgmap v153: 129 pgs: 129 active+clean; 2.6 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 14 MiB/s wr, 923 op/s 2024-08-02T17:51:28.442 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:28 smithi069 ceph-mon[32746]: pgmap v154: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-08-02T17:51:28.444 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:51:28.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:28 smithi083 ceph-mon[37031]: pgmap v154: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-08-02T17:51:28.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:28 smithi077 ceph-mon[37108]: pgmap v154: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 17 MiB/s wr, 1.14k op/s 2024-08-02T17:51:28.854 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 -- 172.21.15.69:0/432251646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4c01006d0 msgr2=0x7fc4c0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 --2- 172.21.15.69:0/432251646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4c01006d0 0x7fc4c0100b30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009930 tx=0x7fc4b402a420 comp rx=0 tx=0).stop 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 -- 172.21.15.69:0/432251646 shutdown_connections 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 --2- 172.21.15.69:0/432251646 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4c01006d0 0x7fc4c0100b30 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 --2- 172.21.15.69:0/432251646 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c0101070 0x7fc4c010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 --2- 172.21.15.69:0/432251646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4c01066e0 0x7fc4c0106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 -- 172.21.15.69:0/432251646 >> 172.21.15.69:0/432251646 conn(0x7fc4c00776d0 msgr2=0x7fc4c00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 -- 172.21.15.69:0/432251646 shutdown_connections 2024-08-02T17:51:29.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.577+0000 7fc4c669e640 1 -- 172.21.15.69:0/432251646 wait complete. 2024-08-02T17:51:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.578+0000 7fc4c669e640 1 Processor -- start 2024-08-02T17:51:29.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.578+0000 7fc4c669e640 1 -- start start 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.578+0000 7fc4c669e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c01006d0 0x7fc4c01970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.578+0000 7fc4c669e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4c0101070 0x7fc4c01975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.578+0000 7fc4c669e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4c01066e0 0x7fc4c019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4c669e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0110b70 con 0x7fc4c01066e0 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4c669e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01109f0 con 0x7fc4c01006d0 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4c669e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0110cf0 con 0x7fc4c0101070 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c01006d0 0x7fc4c01970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4c4c3b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4c01066e0 0x7fc4c019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c01006d0 0x7fc4c01970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44096/0 (socket says 172.21.15.69:44096) 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bffff640 1 -- 172.21.15.69:0/1051318073 learned_addr learned my addr 172.21.15.69:0/1051318073 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:29.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4c0101070 0x7fc4c01975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bffff640 1 -- 172.21.15.69:0/1051318073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4c0101070 msgr2=0x7fc4c01975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bffff640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4c0101070 0x7fc4c01975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bffff640 1 -- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4c01066e0 msgr2=0x7fc4c019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bffff640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4c01066e0 0x7fc4c019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.579+0000 7fc4bffff640 1 -- 172.21.15.69:0/1051318073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c019c090 con 0x7fc4c01006d0 2024-08-02T17:51:29.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.580+0000 7fc4bffff640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c01006d0 0x7fc4c01970a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000d770 tx=0x7fc4b000db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.580+0000 7fc4bd7fa640 1 -- 172.21.15.69:0/1051318073 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0017070 con 0x7fc4c01006d0 2024-08-02T17:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.580+0000 7fc4bd7fa640 1 -- 172.21.15.69:0/1051318073 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b0009820 con 0x7fc4c01006d0 2024-08-02T17:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.580+0000 7fc4c669e640 1 -- 172.21.15.69:0/1051318073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c019c380 con 0x7fc4c01006d0 2024-08-02T17:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.580+0000 7fc4bd7fa640 1 -- 172.21.15.69:0/1051318073 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0004d00 con 0x7fc4c01006d0 2024-08-02T17:51:29.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.580+0000 7fc4c669e640 1 -- 172.21.15.69:0/1051318073 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c01a3c10 con 0x7fc4c01006d0 2024-08-02T17:51:29.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.583+0000 7fc4bd7fa640 1 -- 172.21.15.69:0/1051318073 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc4b0005220 con 0x7fc4c01006d0 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.584+0000 7fc4bd7fa640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc4a80a2df0 0x7fc4a80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.584+0000 7fc4c93fb740 1 Processor -- start 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.584+0000 7fc4c93fb740 1 -- start start 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.584+0000 7fc4bd7fa640 1 -- 172.21.15.69:0/1051318073 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc4b00cf200 con 0x7fc4c01006d0 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.584+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc4a80a2df0 0x7fc4a80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.584+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862be1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862beea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4c93fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564862b459b0 con 0x564862be1c00 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4c93fb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564862aed800 con 0x564862be2e50 2024-08-02T17:51:29.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4c93fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564862b41970 con 0x564862bec620 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862be1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862be1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44112/0 (socket says 172.21.15.69:44112) 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 -- 172.21.15.69:0/2031369336 learned_addr learned my addr 172.21.15.69:0/2031369336 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4c4c3b640 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862beea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 -- 172.21.15.69:0/2031369336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 msgr2=0x564862beea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862beea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 -- 172.21.15.69:0/2031369336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 msgr2=0x564862be2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 -- 172.21.15.69:0/2031369336 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564862859210 con 0x564862be2e50 2024-08-02T17:51:29.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc4a80a2df0 0x7fc4a80a52b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4006010 tx=0x7fc4b403fae0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4bffff640 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862be1630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000dfd0 tx=0x7fc4b00975a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c240cec9515f2d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4a67fc640 1 -- 172.21.15.69:0/2031369336 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0020070 con 0x564862be2e50 2024-08-02T17:51:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4a67fc640 1 -- 172.21.15.69:0/2031369336 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b009f6b0 con 0x564862be2e50 2024-08-02T17:51:29.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.585+0000 7fc4a67fc640 1 -- 172.21.15.69:0/2031369336 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00099d0 con 0x564862be2e50 2024-08-02T17:51:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2031369336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 msgr2=0x564862be1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862be1630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000dfd0 tx=0x7fc4b00975a0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2031369336 shutdown_connections 2024-08-02T17:51:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862beea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2031369336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862be1630 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2031369336 >> 172.21.15.69:0/2031369336 conn(0x564862bddd20 msgr2=0x564862bde100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2031369336 shutdown_connections 2024-08-02T17:51:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.586+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2031369336 wait complete. 2024-08-02T17:51:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.587+0000 7fc4c93fb740 1 -- start start 2024-08-02T17:51:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.587+0000 7fc4c93fb740 1 Processor -- start 2024-08-02T17:51:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.587+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.587+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862be9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.587+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862bec620 0x564862bea030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.587+0000 7fc4c93fb740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564862bef4f0 con 0x564862be1c00 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.587+0000 7fc4c93fb740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564862b01500 con 0x564862bec620 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.587+0000 7fc4c93fb740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564862bef670 con 0x564862be2e50 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862bec620 0x564862bea030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862bec620 0x564862bea030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44120/0 (socket says 172.21.15.69:44120) 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 -- 172.21.15.69:0/2821218351 learned_addr learned my addr 172.21.15.69:0/2821218351 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862be9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 -- 172.21.15.69:0/2821218351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 msgr2=0x564862be9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862be9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 -- 172.21.15.69:0/2821218351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 msgr2=0x564862be95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 -- 172.21.15.69:0/2821218351 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564862925620 con 0x564862bec620 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c93fb740 10 client.0 _ll_register_callbacks cb 0x564862858a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c4c3b640 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862bec620 0x564862bea030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8012d50 tx=0x7fc4b800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f05f5008418544a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc49f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc49f7fe640 1 -- 172.21.15.69:0/2821218351 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800fe60 con 0x564862bec620 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.588+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2821218351 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b009f6b0 con 0x564862bec620 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.589+0000 7fc49f7fe640 1 -- 172.21.15.69:0/2821218351 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc4b80120d0 con 0x564862bec620 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.589+0000 7fc49f7fe640 1 -- 172.21.15.69:0/2821218351 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8010430 con 0x564862bec620 2024-08-02T17:51:29.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.589+0000 7fc4c93fb740 10 client.26297 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.589+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2821218351 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x564862bfb0d0 con 0x564862bec620 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.590+0000 7fc49f7fe640 1 -- 172.21.15.69:0/2821218351 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc4b80105d0 con 0x564862bec620 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.590+0000 7fc4c93fb740 10 client.26297 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.590+0000 7fc4c93fb740 10 client.26297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.590+0000 7fc4c93fb740 20 client.26297 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.590+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564862bfc480 0x564862bfe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.590+0000 7fc4c93fb740 4 client.26297 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.590+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2821218351 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc4b009f6b0 con 0x564862bfc480 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.590+0000 7fc4bffff640 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564862bfc480 0x564862bfe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.591+0000 7fc49f7fe640 10 client.26297 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:29.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.591+0000 7fc4bffff640 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564862bfc480 0x564862bfe910 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b7dc7d2ef9e727f server_cookie=1b10b3395e8dac6 in_seq=0 out_seq=0 2024-08-02T17:51:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.591+0000 7fc49f7fe640 1 -- 172.21.15.69:0/2821218351 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fc4b009f6b0 con 0x564862bfc480 2024-08-02T17:51:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.591+0000 7fc49f7fe640 10 client.26297 handle_command_reply: tid=0 2024-08-02T17:51:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.591+0000 7fc4c93fb740 1 client.26297 shutdown 2024-08-02T17:51:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2821218351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862bec620 msgr2=0x564862bea030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862bec620 0x564862bea030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8012d50 tx=0x7fc4b800f790 comp rx=0 tx=0).stop 2024-08-02T17:51:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2821218351 shutdown_connections 2024-08-02T17:51:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564862bfc480 0x564862bfe910 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862bec620 0x564862bea030 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc49f7fe640 0 client.26297 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862be9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2821218351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2821218351 >> 172.21.15.69:0/2821218351 conn(0x564862bddd20 msgr2=0x564862be3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2821218351 shutdown_connections 2024-08-02T17:51:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2821218351 wait complete. 2024-08-02T17:51:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.593+0000 7fc4c93fb740 20 client.26297 trim_cache size 0 max 16384 2024-08-02T17:51:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.620+0000 7fc4c93fb740 1 Processor -- start 2024-08-02T17:51:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.620+0000 7fc4c93fb740 1 -- start start 2024-08-02T17:51:29.622 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.620+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.621+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.625 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.621+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56534/0 (socket says 172.21.15.69:56534) 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.621+0000 7fc4bffff640 1 -- 172.21.15.69:0/2382878468 learned_addr learned my addr 172.21.15.69:0/2382878468 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.621+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862bdc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.621+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862bdcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.621+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2382878468 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01af460 con 0x564862be1c00 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.621+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2382878468 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01acd30 con 0x564862be2e50 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.621+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2382878468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01aab80 con 0x564862bec620 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.622+0000 7fc4c4c3b640 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862bdcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.624+0000 7fc4c4c3b640 1 -- 172.21.15.69:0/2382878468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 msgr2=0x564862bdc660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.624+0000 7fc4c4c3b640 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862bdc660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.624+0000 7fc4c4c3b640 1 -- 172.21.15.69:0/2382878468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 msgr2=0x564862be9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.624+0000 7fc4c4c3b640 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.624+0000 7fc4c4c3b640 1 -- 172.21.15.69:0/2382878468 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564862be23c0 con 0x564862bec620 2024-08-02T17:51:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.625+0000 7fc4c4c3b640 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862bdcba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8006010 tx=0x7fc4b8014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c1612f34a162e7a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.625+0000 7fc4a5ffb640 1 -- 172.21.15.69:0/2382878468 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b802f070 con 0x564862bec620 2024-08-02T17:51:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.625+0000 7fc4a5ffb640 1 -- 172.21.15.69:0/2382878468 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b8010ad0 con 0x564862bec620 2024-08-02T17:51:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.626+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2382878468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 msgr2=0x564862bdcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.626+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862bdcba0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8006010 tx=0x7fc4b8014000 comp rx=0 tx=0).stop 2024-08-02T17:51:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.630+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2382878468 shutdown_connections 2024-08-02T17:51:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.630+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862bec620 0x564862bdcba0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.630+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be2e50 0x564862bdc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.630+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/2382878468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862be1c00 0x564862be9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.630+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2382878468 >> 172.21.15.69:0/2382878468 conn(0x564862bddd20 msgr2=0x564862beb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:29.638 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.637+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2382878468 shutdown_connections 2024-08-02T17:51:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.638+0000 7fc4c93fb740 1 -- 172.21.15.69:0/2382878468 wait complete. 2024-08-02T17:51:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.638+0000 7fc4c93fb740 1 -- start start 2024-08-02T17:51:29.640 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.638+0000 7fc4c93fb740 1 Processor -- start 2024-08-02T17:51:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.639+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be1c00 0x564862bf0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.639+0000 7fc4c93fb740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862bf12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.639+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be1c00 0x564862bf0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.639+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be1c00 0x564862bf0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44142/0 (socket says 172.21.15.69:44142) 2024-08-02T17:51:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.639+0000 7fc4bffff640 1 -- 172.21.15.69:0/495354344 learned_addr learned my addr 172.21.15.69:0/495354344 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.639+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862bf12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.641 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862bec620 0x564862c77200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01ab1f0 con 0x564862bec620 2024-08-02T17:51:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01ac660 con 0x564862be1c00 2024-08-02T17:51:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01ab8c0 con 0x564862be2e50 2024-08-02T17:51:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4c93fb740 10 client.0 _ll_register_callbacks cb 0x7fc4b8009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4bffff640 1 -- 172.21.15.69:0/495354344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 msgr2=0x564862bf12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:29.642 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4bffff640 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862bf12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4bffff640 1 -- 172.21.15.69:0/495354344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862bec620 msgr2=0x564862c77200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4bffff640 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862bec620 0x564862c77200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.640+0000 7fc4bffff640 1 -- 172.21.15.69:0/495354344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564862c77740 con 0x564862be1c00 2024-08-02T17:51:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.641+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862bf12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.641+0000 7fc4bffff640 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be1c00 0x564862bf0db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0097cd0 tx=0x7fc4b009bb00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6fa331aec7d6647 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.641+0000 7fc49f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.641+0000 7fc49f7fe640 1 -- 172.21.15.69:0/495354344 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00b4060 con 0x564862be1c00 2024-08-02T17:51:29.643 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.641+0000 7fc49f7fe640 1 -- 172.21.15.69:0/495354344 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc4b0009d60 con 0x564862be1c00 2024-08-02T17:51:29.644 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.641+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b8010ad0 con 0x564862be1c00 2024-08-02T17:51:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.645+0000 7fc49f7fe640 1 -- 172.21.15.69:0/495354344 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc4b00b9050 con 0x564862be1c00 2024-08-02T17:51:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.645+0000 7fc4c93fb740 10 client.26300 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:29.646 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.645+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564862c78a10 con 0x564862be1c00 2024-08-02T17:51:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.646+0000 7fc49f7fe640 1 -- 172.21.15.69:0/495354344 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b009f6b0 con 0x564862be1c00 2024-08-02T17:51:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.646+0000 7fc49f7fe640 1 -- 172.21.15.69:0/495354344 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc4b00099d0 con 0x564862be1c00 2024-08-02T17:51:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.646+0000 7fc4c93fb740 10 client.26300 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.646+0000 7fc4c93fb740 10 client.26300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.646+0000 7fc4c93fb740 20 client.26300 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.646+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564862bfc480 0x564862c7ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.647+0000 7fc4c93fb740 4 client.26300 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:51:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.647+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fc4b8010ad0 con 0x564862bfc480 2024-08-02T17:51:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.647+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564862bfc480 0x564862c7ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.647+0000 7fc4bf7fe640 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564862bfc480 0x564862c7ad10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89e20736850c661 server_cookie=660ca4de780734bd in_seq=0 out_seq=0 2024-08-02T17:51:29.649 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.647+0000 7fc49f7fe640 10 client.26300 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.742+0000 7fc49f7fe640 1 -- 172.21.15.69:0/495354344 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fc4b8010ad0 con 0x564862bfc480 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.742+0000 7fc49f7fe640 10 client.26300 handle_command_reply: tid=0 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.742+0000 7fc4c93fb740 1 client.26300 shutdown 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be1c00 msgr2=0x564862bf0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be1c00 0x564862bf0db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0097cd0 tx=0x7fc4b009bb00 comp rx=0 tx=0).stop 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 shutdown_connections 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564862bec620 0x564862c77200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564862bfc480 0x564862c7ad10 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564862be2e50 0x564862bf12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc49f7fe640 0 client.26300 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 --2- 172.21.15.69:0/495354344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564862be1c00 0x564862bf0db0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 >> 172.21.15.69:0/495354344 conn(0x564862bddd20 msgr2=0x564862be7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 shutdown_connections 2024-08-02T17:51:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 1 -- 172.21.15.69:0/495354344 wait complete. 2024-08-02T17:51:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c93fb740 20 client.26300 trim_cache size 0 max 16384 2024-08-02T17:51:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:51:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:51:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:51:29.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:51:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c669e640 1 -- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc4a80a2df0 msgr2=0x7fc4a80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.743+0000 7fc4c669e640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc4a80a2df0 0x7fc4a80a52b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4006010 tx=0x7fc4b403fae0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 -- 172.21.15.69:0/1051318073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c01006d0 msgr2=0x7fc4c01970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c01006d0 0x7fc4c01970a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000d770 tx=0x7fc4b000db70 comp rx=0 tx=0).stop 2024-08-02T17:51:29.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 -- 172.21.15.69:0/1051318073 shutdown_connections 2024-08-02T17:51:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc4c01066e0 0x7fc4c019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc4a80a2df0 0x7fc4a80a52b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4c0101070 0x7fc4c01975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 --2- 172.21.15.69:0/1051318073 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc4c01006d0 0x7fc4c01970a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 -- 172.21.15.69:0/1051318073 >> 172.21.15.69:0/1051318073 conn(0x7fc4c00776d0 msgr2=0x7fc4c0103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 -- 172.21.15.69:0/1051318073 shutdown_connections 2024-08-02T17:51:29.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:29.744+0000 7fc4c669e640 1 -- 172.21.15.69:0/1051318073 wait complete. 2024-08-02T17:51:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 260.0 s, 1057.6 tps, lat 96.363 ms stddev 114.240 2024-08-02T17:51:30.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:30 smithi069 ceph-mon[32746]: pgmap v155: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 592 op/s 2024-08-02T17:51:30.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:30 smithi083 ceph-mon[37031]: pgmap v155: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 592 op/s 2024-08-02T17:51:30.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:30 smithi077 ceph-mon[37108]: pgmap v155: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 592 op/s 2024-08-02T17:51:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:32 smithi069 ceph-mon[32746]: pgmap v156: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 596 op/s 2024-08-02T17:51:32.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:32 smithi083 ceph-mon[37031]: pgmap v156: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 596 op/s 2024-08-02T17:51:32.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:32 smithi077 ceph-mon[37108]: pgmap v156: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 MiB/s wr, 596 op/s 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 -- 172.21.15.69:0/656701252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40941067b0 msgr2=0x7f4094106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 --2- 172.21.15.69:0/656701252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40941067b0 0x7f4094106b90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4090009930 tx=0x7f409002a420 comp rx=0 tx=0).stop 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 -- 172.21.15.69:0/656701252 shutdown_connections 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 --2- 172.21.15.69:0/656701252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4094101150 0x7f409410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 --2- 172.21.15.69:0/656701252 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40941007b0 0x7f4094100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 --2- 172.21.15.69:0/656701252 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f40941067b0 0x7f4094106b90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 -- 172.21.15.69:0/656701252 >> 172.21.15.69:0/656701252 conn(0x7f40940fc4d0 msgr2=0x7f40940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 -- 172.21.15.69:0/656701252 shutdown_connections 2024-08-02T17:51:32.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.078+0000 7f409c611640 1 -- 172.21.15.69:0/656701252 wait complete. 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409c611640 1 Processor -- start 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409c611640 1 -- start start 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409c611640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40941007b0 0x7f4094197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409c611640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4094101150 0x7f4094197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409c611640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40941067b0 0x7f409419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409c611640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4094110bd0 con 0x7f4094101150 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409c611640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4094110a50 con 0x7f40941007b0 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409c611640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4094110d50 con 0x7f40941067b0 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4094101150 0x7f4094197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4094101150 0x7f4094197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56552/0 (socket says 172.21.15.69:56552) 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 -- 172.21.15.69:0/650459507 learned_addr learned my addr 172.21.15.69:0/650459507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409abae640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40941067b0 0x7f409419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f409a3ad640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40941007b0 0x7f4094197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 -- 172.21.15.69:0/650459507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40941067b0 msgr2=0x7f409419b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40941067b0 0x7f409419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 -- 172.21.15.69:0/650459507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40941007b0 msgr2=0x7f4094197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40941007b0 0x7f4094197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 -- 172.21.15.69:0/650459507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409419bfe0 con 0x7f4094101150 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.079+0000 7f4099bac640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4094101150 0x7f4094197570 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f408400c920 tx=0x7f408400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.080+0000 7f408b7fe640 1 -- 172.21.15.69:0/650459507 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084007c70 con 0x7f4094101150 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.080+0000 7f408b7fe640 1 -- 172.21.15.69:0/650459507 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084007e10 con 0x7f4094101150 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.080+0000 7f409c611640 1 -- 172.21.15.69:0/650459507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409419c2d0 con 0x7f4094101150 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.080+0000 7f408b7fe640 1 -- 172.21.15.69:0/650459507 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084018890 con 0x7f4094101150 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.080+0000 7f409c611640 1 -- 172.21.15.69:0/650459507 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40941a3bb0 con 0x7f4094101150 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.082+0000 7f408b7fe640 1 -- 172.21.15.69:0/650459507 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4084018a50 con 0x7f4094101150 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.082+0000 7f409ea0f740 1 Processor -- start 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.082+0000 7f409ea0f740 1 -- start start 2024-08-02T17:51:32.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f408b7fe640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40800a2d70 0x7f40800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf185e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf22ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409ea0f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558aabe7ba40 con 0x558aabf18b20 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409ea0f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558aabe139d0 con 0x558aabf233e0 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409ea0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558aabe77a00 con 0x558aabf18200 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409abae640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40800a2d70 0x7f40800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf22ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409a3ad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf185e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f408b7fe640 1 -- 172.21.15.69:0/650459507 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f40840cf990 con 0x7f4094101150 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf22ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56568/0 (socket says 172.21.15.69:56568) 2024-08-02T17:51:32.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 -- 172.21.15.69:0/293605067 learned_addr learned my addr 172.21.15.69:0/293605067 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409abae640 1 --2- 172.21.15.69:0/293605067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 -- 172.21.15.69:0/293605067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 msgr2=0x558aabf185e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 --2- 172.21.15.69:0/293605067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf185e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 -- 172.21.15.69:0/293605067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 msgr2=0x558aabf257d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 --2- 172.21.15.69:0/293605067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf257d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 -- 172.21.15.69:0/293605067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aabe1a650 con 0x558aabf18b20 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f409abae640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40800a2d70 0x7f40800a5230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f408c0148c0 tx=0x7f408c0068e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4099bac640 1 --2- 172.21.15.69:0/293605067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf22ea0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4084097f50 tx=0x7f408401f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71b3a371cac6608 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.083+0000 7f4088ff9640 1 -- 172.21.15.69:0/293605067 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408409fea0 con 0x558aabf18b20 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f4088ff9640 1 -- 172.21.15.69:0/293605067 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40840a4760 con 0x558aabf18b20 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f4088ff9640 1 -- 172.21.15.69:0/293605067 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084098160 con 0x558aabf18b20 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 -- 172.21.15.69:0/293605067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 msgr2=0x558aabf22ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 --2- 172.21.15.69:0/293605067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf22ea0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4084097f50 tx=0x7f408401f3c0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 -- 172.21.15.69:0/293605067 shutdown_connections 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 --2- 172.21.15.69:0/293605067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf22ea0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 --2- 172.21.15.69:0/293605067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 --2- 172.21.15.69:0/293605067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf185e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 -- 172.21.15.69:0/293605067 >> 172.21.15.69:0/293605067 conn(0x558aabf14790 msgr2=0x558aabf14b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 -- 172.21.15.69:0/293605067 shutdown_connections 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.084+0000 7f409ea0f740 1 -- 172.21.15.69:0/293605067 wait complete. 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 1 -- start start 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 1 Processor -- start 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf1e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 0x558aabf1eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf1f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558aabf121b0 con 0x558aabf233e0 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558aabe36b20 con 0x558aabf18200 2024-08-02T17:51:32.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558aabf12330 con 0x558aabf18b20 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409ea0f740 10 client.0 _ll_register_callbacks cb 0x558aabe34bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409abae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf1f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409abae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf1f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56570/0 (socket says 172.21.15.69:56570) 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409abae640 1 -- 172.21.15.69:0/3061796578 learned_addr learned my addr 172.21.15.69:0/3061796578 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f4099bac640 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 0x558aabf1eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409a3ad640 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf1e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409abae640 1 -- 172.21.15.69:0/3061796578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 msgr2=0x558aabf1eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409abae640 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 0x558aabf1eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409abae640 1 -- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 msgr2=0x558aabf1e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409abae640 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf1e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.085+0000 7f409abae640 1 -- 172.21.15.69:0/3061796578 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aabbed3c0 con 0x558aabf233e0 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409a3ad640 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf1e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:32.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409abae640 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf1f090 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f408c018690 tx=0x7f408c005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf97a401fac5404b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f407dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f407dffb640 1 -- 172.21.15.69:0/3061796578 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c027060 con 0x558aabf233e0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f407dffb640 1 -- 172.21.15.69:0/3061796578 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f408c022620 con 0x558aabf233e0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 1 -- 172.21.15.69:0/3061796578 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40840a4760 con 0x558aabf233e0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f407dffb640 1 -- 172.21.15.69:0/3061796578 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f408c02e050 con 0x558aabf233e0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 10 client.16698 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 1 -- 172.21.15.69:0/3061796578 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558aabf2f880 con 0x558aabf233e0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f407dffb640 1 -- 172.21.15.69:0/3061796578 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40840a4760 con 0x558aabf233e0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f407dffb640 1 -- 172.21.15.69:0/3061796578 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f408c021630 con 0x558aabf233e0 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 10 client.16698 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 10 client.16698 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 20 client.16698 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:32.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x558aabf30d40 0x558aabf33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 4 client.16698 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.086+0000 7f409ea0f740 1 -- 172.21.15.69:0/3061796578 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558aabe88fb0 con 0x558aabf30d40 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.087+0000 7f409a3ad640 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x558aabf30d40 0x558aabf33170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.087+0000 7f409a3ad640 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x558aabf30d40 0x558aabf33170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8e534a3fc349260 server_cookie=adf2ba1cfd696300 in_seq=0 out_seq=0 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.087+0000 7f407dffb640 10 client.16698 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.089+0000 7f407dffb640 1 -- 172.21.15.69:0/3061796578 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558aabe88fb0 con 0x558aabf30d40 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.089+0000 7f407dffb640 10 client.16698 handle_command_reply: tid=0 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.089+0000 7f409ea0f740 1 client.16698 shutdown 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f409ea0f740 1 -- 172.21.15.69:0/3061796578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 msgr2=0x558aabf1f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f409ea0f740 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf1f090 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f408c018690 tx=0x7f408c005df0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f409ea0f740 1 -- 172.21.15.69:0/3061796578 shutdown_connections 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f409ea0f740 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x558aabf30d40 0x558aabf33170 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f407dffb640 0 client.16698 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:51:32.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f409ea0f740 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf1f090 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f409ea0f740 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 0x558aabf1eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f409ea0f740 1 --2- 172.21.15.69:0/3061796578 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf1e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.092+0000 7f409ea0f740 1 -- 172.21.15.69:0/3061796578 >> 172.21.15.69:0/3061796578 conn(0x558aabf14790 msgr2=0x558aabf1a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.093+0000 7f409ea0f740 1 -- 172.21.15.69:0/3061796578 shutdown_connections 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.093+0000 7f409ea0f740 1 -- 172.21.15.69:0/3061796578 wait complete. 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.093+0000 7f409ea0f740 20 client.16698 trim_cache size 0 max 16384 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409ea0f740 1 Processor -- start 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409ea0f740 1 -- start start 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf13550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf1cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf1d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409ea0f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40941af4a0 con 0x558aabf18b20 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409ea0f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40941acd70 con 0x558aabf233e0 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409ea0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40941aabc0 con 0x558aabf18200 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f4099bac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf1cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f4099bac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf1cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56582/0 (socket says 172.21.15.69:56582) 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f4099bac640 1 -- 172.21.15.69:0/1558332317 learned_addr learned my addr 172.21.15.69:0/1558332317 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409a3ad640 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf13550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.107+0000 7f409abae640 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf1d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f4099bac640 1 -- 172.21.15.69:0/1558332317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 msgr2=0x558aabf13550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f4099bac640 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf13550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f4099bac640 1 -- 172.21.15.69:0/1558332317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 msgr2=0x558aabf1d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f4099bac640 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf1d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f4099bac640 1 -- 172.21.15.69:0/1558332317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aabf15420 con 0x558aabf18b20 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f409a3ad640 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf13550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:32.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f409abae640 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf1d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f4099bac640 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf1cea0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f408401f4b0 tx=0x7f408400fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8705f6f518f6a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f407ffff640 1 -- 172.21.15.69:0/1558332317 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084099b90 con 0x558aabf18b20 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f407ffff640 1 -- 172.21.15.69:0/1558332317 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084099df0 con 0x558aabf18b20 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.108+0000 7f407ffff640 1 -- 172.21.15.69:0/1558332317 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40840cd130 con 0x558aabf18b20 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- 172.21.15.69:0/1558332317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 msgr2=0x558aabf1cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf1cea0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f408401f4b0 tx=0x7f408400fe30 comp rx=0 tx=0).stop 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- 172.21.15.69:0/1558332317 shutdown_connections 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf18b20 0x558aabf1cea0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf233e0 0x558aabf1d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1558332317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18200 0x558aabf13550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- 172.21.15.69:0/1558332317 >> 172.21.15.69:0/1558332317 conn(0x558aabf14790 msgr2=0x558aabf24760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- 172.21.15.69:0/1558332317 shutdown_connections 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- 172.21.15.69:0/1558332317 wait complete. 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- start start 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 Processor -- start 2024-08-02T17:51:32.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf2bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 0x558aabf2c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f40941ab230 con 0x558aabf233e0 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f40941ac6a0 con 0x558aabf18200 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.109+0000 7f409ea0f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f40941ab900 con 0x558aabf18b20 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409ea0f740 10 client.0 _ll_register_callbacks cb 0x7f408c016e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf2c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56588/0 (socket says 172.21.15.69:56588) 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 -- 172.21.15.69:0/1970669365 learned_addr learned my addr 172.21.15.69:0/1970669365 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409a3ad640 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf2bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f4099bac640 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 0x558aabf2c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 -- 172.21.15.69:0/1970669365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 msgr2=0x558aabf2c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 0x558aabf2c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 -- 172.21.15.69:0/1970669365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 msgr2=0x558aabf2bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf2bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 -- 172.21.15.69:0/1970669365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aabf27f00 con 0x558aabf233e0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409abae640 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf2c840 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f408c021ea0 tx=0x7f408c016880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3caf23632b9064 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f407dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f407dffb640 1 -- 172.21.15.69:0/1970669365 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c027040 con 0x558aabf233e0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f407dffb640 1 -- 172.21.15.69:0/1970669365 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f408c0215c0 con 0x558aabf233e0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409ea0f740 1 -- 172.21.15.69:0/1970669365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4084099df0 con 0x558aabf233e0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f407dffb640 1 -- 172.21.15.69:0/1970669365 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f408c01e040 con 0x558aabf233e0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409ea0f740 10 client.16704 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.110+0000 7f409ea0f740 1 -- 172.21.15.69:0/1970669365 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558aabfa3690 con 0x558aabf233e0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f407dffb640 1 -- 172.21.15.69:0/1970669365 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4084099df0 con 0x558aabf233e0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f407dffb640 1 -- 172.21.15.69:0/1970669365 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f408c022c00 con 0x558aabf233e0 2024-08-02T17:51:32.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f409ea0f740 10 client.16704 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f409ea0f740 10 client.16704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f409ea0f740 20 client.16704 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x558aabfa3dd0 0x558aabfa61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f409ea0f740 4 client.16704 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f409ea0f740 1 -- 172.21.15.69:0/1970669365 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558aabe834c0 con 0x558aabfa3dd0 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f409a3ad640 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x558aabfa3dd0 0x558aabfa61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f407dffb640 10 client.16704 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f409a3ad640 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x558aabfa3dd0 0x558aabfa61f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c95ccefeb2c3701a server_cookie=594d9eee88c55ea4 in_seq=0 out_seq=0 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f407dffb640 1 -- 172.21.15.69:0/1970669365 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558aabe834c0 con 0x558aabfa3dd0 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.111+0000 7f407dffb640 10 client.16704 handle_command_reply: tid=0 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 client.16704 shutdown 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 -- 172.21.15.69:0/1970669365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 msgr2=0x558aabf2c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf2c840 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f408c021ea0 tx=0x7f408c016880 comp rx=0 tx=0).stop 2024-08-02T17:51:32.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 -- 172.21.15.69:0/1970669365 shutdown_connections 2024-08-02T17:51:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x558aabfa3dd0 0x558aabfa61f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558aabf233e0 0x558aabf2c840 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558aabf18b20 0x558aabf2c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 --2- 172.21.15.69:0/1970669365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558aabf18200 0x558aabf2bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 -- 172.21.15.69:0/1970669365 >> 172.21.15.69:0/1970669365 conn(0x558aabf14790 msgr2=0x558aabf237c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:32.704 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f407dffb640 0 client.16704 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 -- 172.21.15.69:0/1970669365 shutdown_connections 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 1 -- 172.21.15.69:0/1970669365 wait complete. 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409ea0f740 20 client.16704 trim_cache size 0 max 16384 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 -- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40800a2d70 msgr2=0x7f40800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40800a2d70 0x7f40800a5230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f408c0148c0 tx=0x7f408c0068e0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 -- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4094101150 msgr2=0x7f4094197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4094101150 0x7f4094197570 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f408400c920 tx=0x7f408400cde0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 -- 172.21.15.69:0/650459507 shutdown_connections 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4094101150 0x7f4094197570 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f40800a2d70 0x7f40800a5230 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f40941067b0 0x7f409419b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 --2- 172.21.15.69:0/650459507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f40941007b0 0x7f4094197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.112+0000 7f409c611640 1 -- 172.21.15.69:0/650459507 >> 172.21.15.69:0/650459507 conn(0x7f40940fc4d0 msgr2=0x7f40940fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.113+0000 7f409c611640 1 -- 172.21.15.69:0/650459507 shutdown_connections 2024-08-02T17:51:32.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:32.113+0000 7f409c611640 1 -- 172.21.15.69:0/650459507 wait complete. 2024-08-02T17:51:33.613 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:51:34.072 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:34.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:34 smithi083 ceph-mon[37031]: pgmap v157: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 934 op/s 2024-08-02T17:51:34.530 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:34 smithi069 ceph-mon[32746]: pgmap v157: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 934 op/s 2024-08-02T17:51:34.599 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:34 smithi077 ceph-mon[37108]: pgmap v157: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 12 MiB/s wr, 934 op/s 2024-08-02T17:51:34.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.664+0000 7fdaa1194640 1 -- 172.21.15.69:0/1725687007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda9c109560 msgr2=0x7fda9c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.664+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1725687007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda9c109560 0x7fda9c1099c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fda84009860 tx=0x7fda8402a380 comp rx=0 tx=0).stop 2024-08-02T17:51:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.665+0000 7fdaa1194640 1 -- 172.21.15.69:0/1725687007 shutdown_connections 2024-08-02T17:51:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.665+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1725687007 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda9c109560 0x7fda9c1099c0 secure :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fda84009860 tx=0x7fda8402a380 comp rx=0 tx=0).stop 2024-08-02T17:51:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.665+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1725687007 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda9c11ae70 0x7fda9c11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.665+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1725687007 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c109180 0x7fda9c10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.665+0000 7fdaa1194640 1 -- 172.21.15.69:0/1725687007 >> 172.21.15.69:0/1725687007 conn(0x7fda9c06de70 msgr2=0x7fda9c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:34.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.670+0000 7fdaa1194640 1 -- 172.21.15.69:0/1725687007 shutdown_connections 2024-08-02T17:51:34.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.673+0000 7fdaa1194640 1 -- 172.21.15.69:0/1725687007 wait complete. 2024-08-02T17:51:34.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.674+0000 7fdaa1194640 1 Processor -- start 2024-08-02T17:51:34.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.674+0000 7fdaa1194640 1 -- start start 2024-08-02T17:51:34.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.674+0000 7fdaa1194640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda9c109180 0x7fda9c11a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.674+0000 7fdaa1194640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c11ae70 0x7fda9c1148e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.675+0000 7fdaa1194640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda9c114e70 0x7fda9c115300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.675+0000 7fdaa1194640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1110a0 con 0x7fda9c114e70 2024-08-02T17:51:34.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.675+0000 7fdaa1194640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c110f20 con 0x7fda9c109180 2024-08-02T17:51:34.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.675+0000 7fdaa1194640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c111220 con 0x7fda9c11ae70 2024-08-02T17:51:34.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.678+0000 7fda9ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda9c109180 0x7fda9c11a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.678+0000 7fda9b59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda9c114e70 0x7fda9c115300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.678+0000 7fda9a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c11ae70 0x7fda9c1148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.678+0000 7fda9ad9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda9c109180 0x7fda9c11a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44202/0 (socket says 172.21.15.69:44202) 2024-08-02T17:51:34.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.678+0000 7fda9ad9d640 1 -- 172.21.15.69:0/1132236459 learned_addr learned my addr 172.21.15.69:0/1132236459 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:34.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.682+0000 7fda9ad9d640 1 -- 172.21.15.69:0/1132236459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c11ae70 msgr2=0x7fda9c1148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.682+0000 7fda9ad9d640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c11ae70 0x7fda9c1148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.682+0000 7fda9ad9d640 1 -- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda9c114e70 msgr2=0x7fda9c115300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.682+0000 7fda9ad9d640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda9c114e70 0x7fda9c115300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.682+0000 7fda9ad9d640 1 -- 172.21.15.69:0/1132236459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda9c1bc140 con 0x7fda9c109180 2024-08-02T17:51:34.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.685+0000 7fda9ad9d640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda9c109180 0x7fda9c11a860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00ba50 tx=0x7fda8c00bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:34.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.687+0000 7fda93fff640 1 -- 172.21.15.69:0/1132236459 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c019070 con 0x7fda9c109180 2024-08-02T17:51:34.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.687+0000 7fda93fff640 1 -- 172.21.15.69:0/1132236459 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda8c00cc10 con 0x7fda9c109180 2024-08-02T17:51:34.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.687+0000 7fda93fff640 1 -- 172.21.15.69:0/1132236459 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8c0076a0 con 0x7fda9c109180 2024-08-02T17:51:34.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.691+0000 7fdaa1194640 1 -- 172.21.15.69:0/1132236459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda9c1bc430 con 0x7fda9c109180 2024-08-02T17:51:34.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.691+0000 7fdaa1194640 1 -- 172.21.15.69:0/1132236459 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda9c1bc8f0 con 0x7fda9c109180 2024-08-02T17:51:34.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.695+0000 7fdaa3592740 1 Processor -- start 2024-08-02T17:51:34.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.695+0000 7fdaa3592740 1 -- start start 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.699+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.699+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 0x555a118a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.699+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a118a31d0 0x555a118a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.699+0000 7fdaa3592740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555a117fda40 con 0x555a118a31d0 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.699+0000 7fdaa3592740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a117959d0 con 0x555a11898910 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.699+0000 7fdaa3592740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a117f9a00 con 0x555a11897ff0 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.701+0000 7fda9a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a118a31d0 0x555a118a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.701+0000 7fda9a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a118a31d0 0x555a118a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56618/0 (socket says 172.21.15.69:56618) 2024-08-02T17:51:34.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.701+0000 7fda9a59c640 1 -- 172.21.15.69:0/2038544457 learned_addr learned my addr 172.21.15.69:0/2038544457 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:34.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.701+0000 7fda9b59e640 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.701+0000 7fda93fff640 1 -- 172.21.15.69:0/1132236459 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fda8c007840 con 0x7fda9c109180 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.701+0000 7fda9ad9d640 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 0x555a118a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.703+0000 7fda9b59e640 1 -- 172.21.15.69:0/2038544457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 msgr2=0x555a118a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.703+0000 7fda9b59e640 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 0x555a118a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.703+0000 7fda9b59e640 1 -- 172.21.15.69:0/2038544457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a118a31d0 msgr2=0x555a118a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.703+0000 7fda9b59e640 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a118a31d0 0x555a118a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.703+0000 7fda9b59e640 1 -- 172.21.15.69:0/2038544457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1179c650 con 0x555a11897ff0 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.703+0000 7fda93fff640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda880a2f00 0x7fda880a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.703+0000 7fda93fff640 1 -- 172.21.15.69:0/1132236459 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fda8c0cec10 con 0x7fda9c109180 2024-08-02T17:51:34.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.704+0000 7fda9b59e640 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118983d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fda94004560 tx=0x7fda9400b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5909ce91c18ddb6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:34.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.704+0000 7fda917fa640 1 -- 172.21.15.69:0/2038544457 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94003ab0 con 0x555a11897ff0 2024-08-02T17:51:34.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.705+0000 7fda917fa640 1 -- 172.21.15.69:0/2038544457 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda94015070 con 0x555a11897ff0 2024-08-02T17:51:34.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.706+0000 7fdaa3592740 1 -- 172.21.15.69:0/2038544457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 msgr2=0x555a118983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.706+0000 7fdaa3592740 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118983d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fda94004560 tx=0x7fda9400b9f0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.706+0000 7fdaa3592740 1 -- 172.21.15.69:0/2038544457 shutdown_connections 2024-08-02T17:51:34.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.706+0000 7fdaa3592740 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a118a31d0 0x555a118a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.706+0000 7fdaa3592740 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 0x555a118a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.706+0000 7fdaa3592740 1 --2- 172.21.15.69:0/2038544457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118983d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.706+0000 7fdaa3592740 1 -- 172.21.15.69:0/2038544457 >> 172.21.15.69:0/2038544457 conn(0x555a11894580 msgr2=0x555a11894960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:34.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.707+0000 7fda9ad9d640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda880a2f00 0x7fda880a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.708+0000 7fdaa3592740 1 -- 172.21.15.69:0/2038544457 shutdown_connections 2024-08-02T17:51:34.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.708+0000 7fdaa3592740 1 -- 172.21.15.69:0/2038544457 wait complete. 2024-08-02T17:51:34.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 1 -- start start 2024-08-02T17:51:34.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 1 Processor -- start 2024-08-02T17:51:34.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118a0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 0x555a118a0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a118a31d0 0x555a118a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555a118a5df0 con 0x555a11898910 2024-08-02T17:51:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555a117b8b20 con 0x555a118a31d0 2024-08-02T17:51:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555a118a5f70 con 0x555a11897ff0 2024-08-02T17:51:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.709+0000 7fdaa3592740 10 client.0 _ll_register_callbacks cb 0x555a117b6bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:34.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.710+0000 7fda9b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.710+0000 7fda9b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:41204/0 (socket says 172.21.15.69:41204) 2024-08-02T17:51:34.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.710+0000 7fda9b59e640 1 -- 172.21.15.69:0/1946273504 learned_addr learned my addr 172.21.15.69:0/1946273504 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:34.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.710+0000 7fda9b59e640 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a118a31d0 0x555a118a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.713+0000 7fda9a59c640 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 0x555a118a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.713+0000 7fda9b59e640 1 -- 172.21.15.69:0/1946273504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a118a31d0 msgr2=0x555a118a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.713+0000 7fda9b59e640 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a118a31d0 0x555a118a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.713+0000 7fda9b59e640 1 -- 172.21.15.69:0/1946273504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 msgr2=0x555a118a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.713+0000 7fda9b59e640 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 0x555a118a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.713+0000 7fda9b59e640 1 -- 172.21.15.69:0/1946273504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1156f3c0 con 0x555a11897ff0 2024-08-02T17:51:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.713+0000 7fda9b59e640 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118a0130 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fda94011e00 tx=0x7fda9400b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8142b99e82828b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.713+0000 7fda827fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:51:34.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.715+0000 7fda9ad9d640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda880a2f00 0x7fda880a53c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x555a118953e0 tx=0x7fda8c09adb0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.715+0000 7fda827fc640 1 -- 172.21.15.69:0/1946273504 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94003ac0 con 0x555a11897ff0 2024-08-02T17:51:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.715+0000 7fdaa3592740 1 -- 172.21.15.69:0/1946273504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555a1180afb0 con 0x555a11897ff0 2024-08-02T17:51:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.715+0000 7fda827fc640 1 -- 172.21.15.69:0/1946273504 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fda94003c60 con 0x555a11897ff0 2024-08-02T17:51:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.716+0000 7fda827fc640 1 -- 172.21.15.69:0/1946273504 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda94031050 con 0x555a11897ff0 2024-08-02T17:51:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.717+0000 7fdaa3592740 10 client.26212 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:34.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.717+0000 7fdaa3592740 1 -- 172.21.15.69:0/1946273504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a118b1b50 con 0x555a11897ff0 2024-08-02T17:51:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.717+0000 7fda827fc640 1 -- 172.21.15.69:0/1946273504 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555a1180afb0 con 0x555a11897ff0 2024-08-02T17:51:34.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.718+0000 7fda827fc640 1 -- 172.21.15.69:0/1946273504 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda94004880 con 0x555a11897ff0 2024-08-02T17:51:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.719+0000 7fdaa3592740 10 client.26212 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.719+0000 7fdaa3592740 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.719+0000 7fdaa3592740 20 client.26212 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.719+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a118b2f60 0x555a118b5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.719+0000 7fdaa3592740 4 client.26212 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:34.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.719+0000 7fdaa3592740 1 -- 172.21.15.69:0/1946273504 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a118054c0 con 0x555a118b2f60 2024-08-02T17:51:34.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.721+0000 7fda9a59c640 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a118b2f60 0x555a118b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.727+0000 7fda9a59c640 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a118b2f60 0x555a118b5350 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=302a997bb8db44e7 server_cookie=8971da3f7d99ed62 in_seq=0 out_seq=0 2024-08-02T17:51:34.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.727+0000 7fda827fc640 10 client.26212 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fda827fc640 1 -- 172.21.15.69:0/1946273504 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555a118054c0 con 0x555a118b2f60 2024-08-02T17:51:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fda827fc640 10 client.26212 handle_command_reply: tid=0 2024-08-02T17:51:34.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 client.26212 shutdown 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 -- 172.21.15.69:0/1946273504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 msgr2=0x555a118a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118a0130 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fda94011e00 tx=0x7fda9400b2e0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 -- 172.21.15.69:0/1946273504 shutdown_connections 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 0x555a118a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a118b2f60 0x555a118b5350 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a118a31d0 0x555a118a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1946273504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a11897ff0 0x555a118a0130 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fdaa3592740 1 -- 172.21.15.69:0/1946273504 >> 172.21.15.69:0/1946273504 conn(0x555a11894580 msgr2=0x555a1189a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:34.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.728+0000 7fda827fc640 0 client.26212 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.733+0000 7fdaa3592740 1 -- 172.21.15.69:0/1946273504 shutdown_connections 2024-08-02T17:51:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.733+0000 7fdaa3592740 1 -- 172.21.15.69:0/1946273504 wait complete. 2024-08-02T17:51:34.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.734+0000 7fdaa3592740 20 client.26212 trim_cache size 0 max 16384 2024-08-02T17:51:34.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.756+0000 7fdaa3592740 1 Processor -- start 2024-08-02T17:51:34.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.756+0000 7fdaa3592740 1 -- start start 2024-08-02T17:51:34.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.756+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11897ff0 0x555a1189aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.756+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 0x555a1189d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.756+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a1189d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.756+0000 7fda9b59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11897ff0 0x555a1189aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fdaa3592740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1c9410 con 0x555a11897ff0 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fdaa3592740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1c6ce0 con 0x555a11898910 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fdaa3592740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1c4b30 con 0x555a118a31d0 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11897ff0 0x555a1189aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43050/0 (socket says 172.21.15.69:43050) 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 -- 172.21.15.69:0/674247521 learned_addr learned my addr 172.21.15.69:0/674247521 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9a59c640 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 0x555a1189d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a1189d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 -- 172.21.15.69:0/674247521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 msgr2=0x555a1189d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 0x555a1189d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 -- 172.21.15.69:0/674247521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11897ff0 msgr2=0x555a1189aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11897ff0 0x555a1189aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 -- 172.21.15.69:0/674247521 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1189af90 con 0x555a118a31d0 2024-08-02T17:51:34.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.757+0000 7fda9b59e640 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a1189d8f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda94004c80 tx=0x7fda94004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66164b876b19dca1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:34.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.758+0000 7fda90ff9640 1 -- 172.21.15.69:0/674247521 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9400b3f0 con 0x555a118a31d0 2024-08-02T17:51:34.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.758+0000 7fda90ff9640 1 -- 172.21.15.69:0/674247521 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda94015070 con 0x555a118a31d0 2024-08-02T17:51:34.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.758+0000 7fda90ff9640 1 -- 172.21.15.69:0/674247521 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94003f30 con 0x555a118a31d0 2024-08-02T17:51:34.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 -- 172.21.15.69:0/674247521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 msgr2=0x555a1189d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a1189d8f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fda94004c80 tx=0x7fda94004180 comp rx=0 tx=0).stop 2024-08-02T17:51:34.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 -- 172.21.15.69:0/674247521 shutdown_connections 2024-08-02T17:51:34.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a1189d8f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11898910 0x555a1189d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 --2- 172.21.15.69:0/674247521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11897ff0 0x555a1189aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 -- 172.21.15.69:0/674247521 >> 172.21.15.69:0/674247521 conn(0x555a11894580 msgr2=0x555a11894960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 -- 172.21.15.69:0/674247521 shutdown_connections 2024-08-02T17:51:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.759+0000 7fdaa3592740 1 -- 172.21.15.69:0/674247521 wait complete. 2024-08-02T17:51:34.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.760+0000 7fdaa3592740 1 -- start start 2024-08-02T17:51:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.760+0000 7fdaa3592740 1 Processor -- start 2024-08-02T17:51:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11897ff0 0x555a118b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 0x555a118b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fdaa3592740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a118b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11897ff0 0x555a118b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11897ff0 0x555a118b8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45620/0 (socket says 172.21.15.69:45620) 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 -- 172.21.15.69:0/1264442912 learned_addr learned my addr 172.21.15.69:0/1264442912 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1c51a0 con 0x555a11898910 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9a59c640 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 0x555a118b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1c6610 con 0x555a11897ff0 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda9c1c5870 con 0x555a118a31d0 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fdaa3592740 10 client.0 _ll_register_callbacks cb 0x7fda840096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a118b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 -- 172.21.15.69:0/1264442912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 msgr2=0x555a118b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a118b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 -- 172.21.15.69:0/1264442912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 msgr2=0x555a118b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 0x555a118b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.761+0000 7fda9b59e640 1 -- 172.21.15.69:0/1264442912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a118b5660 con 0x555a11897ff0 2024-08-02T17:51:34.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.762+0000 7fda9b59e640 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11897ff0 0x555a118b8180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda9400b340 tx=0x7fda94004290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7b66ab6735905e3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:34.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.762+0000 7fda827fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:34.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.762+0000 7fda827fc640 1 -- 172.21.15.69:0/1264442912 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda94004a60 con 0x555a11897ff0 2024-08-02T17:51:34.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.762+0000 7fda827fc640 1 -- 172.21.15.69:0/1264442912 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fda94003ac0 con 0x555a11897ff0 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.763+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda94015070 con 0x555a11897ff0 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.763+0000 7fda827fc640 1 -- 172.21.15.69:0/1264442912 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda94049050 con 0x555a11897ff0 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.763+0000 7fdaa3592740 10 client.26327 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.763+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555a1191c1f0 con 0x555a11897ff0 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fda827fc640 1 -- 172.21.15.69:0/1264442912 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda940054e0 con 0x555a11897ff0 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fda827fc640 1 -- 172.21.15.69:0/1264442912 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda9401a660 con 0x555a11897ff0 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fdaa3592740 10 client.26327 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fdaa3592740 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fdaa3592740 20 client.26327 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a1191c7b0 0x555a1191ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fdaa3592740 4 client.26327 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fda94015070 con 0x555a1191c7b0 2024-08-02T17:51:34.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.764+0000 7fda9a59c640 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a1191c7b0 0x555a1191ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fda827fc640 10 client.26327 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fda9a59c640 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a1191c7b0 0x555a1191ebd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c18dd6a187e413b server_cookie=8de6a4d98b095078 in_seq=0 out_seq=0 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fda827fc640 1 -- 172.21.15.69:0/1264442912 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fda94015070 con 0x555a1191c7b0 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fda827fc640 10 client.26327 handle_command_reply: tid=0 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 client.26327 shutdown 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11897ff0 msgr2=0x555a118b8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11897ff0 0x555a118b8180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda9400b340 tx=0x7fda94004290 comp rx=0 tx=0).stop 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 shutdown_connections 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555a11898910 0x555a118b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555a1191c7b0 0x555a1191ebd0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555a118a31d0 0x555a118b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 --2- 172.21.15.69:0/1264442912 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555a11897ff0 0x555a118b8180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fda827fc640 0 client.26327 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:34.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 >> 172.21.15.69:0/1264442912 conn(0x555a11894580 msgr2=0x555a1189ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:34.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 shutdown_connections 2024-08-02T17:51:34.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 1 -- 172.21.15.69:0/1264442912 wait complete. 2024-08-02T17:51:34.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.765+0000 7fdaa3592740 20 client.26327 trim_cache size 0 max 16384 2024-08-02T17:51:34.770 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:51:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 -- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda880a2f00 msgr2=0x7fda880a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda880a2f00 0x7fda880a53c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x555a118953e0 tx=0x7fda8c09adb0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 -- 172.21.15.69:0/1132236459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda9c109180 msgr2=0x7fda9c11a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda9c109180 0x7fda9c11a860 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00ba50 tx=0x7fda8c00bf10 comp rx=0 tx=0).stop 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 -- 172.21.15.69:0/1132236459 shutdown_connections 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fda9c114e70 0x7fda9c115300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fda880a2f00 0x7fda880a53c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda9c11ae70 0x7fda9c1148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 --2- 172.21.15.69:0/1132236459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fda9c109180 0x7fda9c11a860 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 -- 172.21.15.69:0/1132236459 >> 172.21.15.69:0/1132236459 conn(0x7fda9c06de70 msgr2=0x7fda9c072180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 -- 172.21.15.69:0/1132236459 shutdown_connections 2024-08-02T17:51:34.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:34.766+0000 7fdaa1194640 1 -- 172.21.15.69:0/1132236459 wait complete. 2024-08-02T17:51:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 265.0 s, 1575.0 tps, lat 64.095 ms stddev 103.800 2024-08-02T17:51:36.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:36 smithi083 ceph-mon[37031]: pgmap v158: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 737 op/s 2024-08-02T17:51:36.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:36 smithi077 ceph-mon[37108]: pgmap v158: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 737 op/s 2024-08-02T17:51:36.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:36 smithi069 ceph-mon[32746]: pgmap v158: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 11 MiB/s wr, 737 op/s 2024-08-02T17:51:36.613 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:51:36.613 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:51:36.895 INFO:tasks.quiescer.fs.[cephfs]:Total duration (57.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-02T17:51:36.895 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.9 2024-08-02T17:51:36.895 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:51:37.067 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:38.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:38 smithi083 ceph-mon[37031]: pgmap v159: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 981 op/s 2024-08-02T17:51:38.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:38 smithi077 ceph-mon[37108]: pgmap v159: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 981 op/s 2024-08-02T17:51:38.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:38 smithi069 ceph-mon[32746]: pgmap v159: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 MiB/s wr, 981 op/s 2024-08-02T17:51:39.703 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:51:39.704 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:51:39.704 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "3be2ee35-d884-4b9b-80a6-39fc0586e3d8", 2024-08-02T17:51:39.704 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:51:39.704 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.812+0000 7fa2b597a640 1 -- 172.21.15.69:0/2292036209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0100b50 msgr2=0x7fa2b0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.812+0000 7fa2b597a640 1 --2- 172.21.15.69:0/2292036209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0100b50 0x7fa2b0100f30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009900 tx=0x7fa2a002a480 comp rx=0 tx=0).stop 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 -- 172.21.15.69:0/2292036209 shutdown_connections 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 --2- 172.21.15.69:0/2292036209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0102c20 0x7fa2b010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 --2- 172.21.15.69:0/2292036209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b01022a0 0x7fa2b01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 --2- 172.21.15.69:0/2292036209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b0100b50 0x7fa2b0100f30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 -- 172.21.15.69:0/2292036209 >> 172.21.15.69:0/2292036209 conn(0x7fa2b00fc500 msgr2=0x7fa2b00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 -- 172.21.15.69:0/2292036209 shutdown_connections 2024-08-02T17:51:39.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 -- 172.21.15.69:0/2292036209 wait complete. 2024-08-02T17:51:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 Processor -- start 2024-08-02T17:51:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.813+0000 7fa2b597a640 1 -- start start 2024-08-02T17:51:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b597a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0100b50 0x7fa2b0197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b597a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b01022a0 0x7fa2b0197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b597a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b0102c20 0x7fa2b019ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b597a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0110cb0 con 0x7fa2b01022a0 2024-08-02T17:51:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b597a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0110b30 con 0x7fa2b0102c20 2024-08-02T17:51:39.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b597a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b0110e30 con 0x7fa2b0100b50 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b01022a0 0x7fa2b0197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b01022a0 0x7fa2b0197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43074/0 (socket says 172.21.15.69:43074) 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 -- 172.21.15.69:0/1969911563 learned_addr learned my addr 172.21.15.69:0/1969911563 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b0102c20 0x7fa2b019ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0100b50 0x7fa2b0197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 -- 172.21.15.69:0/1969911563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0100b50 msgr2=0x7fa2b0197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0100b50 0x7fa2b0197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 -- 172.21.15.69:0/1969911563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b0102c20 msgr2=0x7fa2b019ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b0102c20 0x7fa2b019ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 -- 172.21.15.69:0/1969911563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b019c0e0 con 0x7fa2b01022a0 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b0102c20 0x7fa2b019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:39.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2ae7fc640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b01022a0 0x7fa2b0197670 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e9f0 tx=0x7fa2a400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b4978640 1 -- 172.21.15.69:0/1969911563 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400cd80 con 0x7fa2b01022a0 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b4978640 1 -- 172.21.15.69:0/1969911563 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a4010910 con 0x7fa2b01022a0 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b597a640 1 -- 172.21.15.69:0/1969911563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b019c3d0 con 0x7fa2b01022a0 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.814+0000 7fa2b4978640 1 -- 172.21.15.69:0/1969911563 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4018990 con 0x7fa2b01022a0 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.815+0000 7fa2b597a640 1 -- 172.21.15.69:0/1969911563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b01a3cb0 con 0x7fa2b01022a0 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.817+0000 7fa2b4978640 1 -- 172.21.15.69:0/1969911563 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa2a4010ac0 con 0x7fa2b01022a0 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.817+0000 7fa2b7d78740 1 Processor -- start 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.817+0000 7fa2b7d78740 1 -- start start 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.817+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec00a0 0x56214fec0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fecd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2b4978640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2980a2cf0 0x7fa2980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fecf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56214fe83e40 con 0x56214fecd5f0 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56214fe257f0 con 0x56214fec00a0 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56214fdccfb0 con 0x56214fec09c0 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fecf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fecf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43086/0 (socket says 172.21.15.69:43086) 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 -- 172.21.15.69:0/4111138152 learned_addr learned my addr 172.21.15.69:0/4111138152 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2ae7fc640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2980a2cf0 0x7fa2980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec00a0 0x56214fec0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fecd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2b4978640 1 -- 172.21.15.69:0/1969911563 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa2a40cfc80 con 0x7fa2b01022a0 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 -- 172.21.15.69:0/4111138152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 msgr2=0x56214fecd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:39.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fecd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 -- 172.21.15.69:0/4111138152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec00a0 msgr2=0x56214fec0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec00a0 0x56214fec0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 -- 172.21.15.69:0/4111138152 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214fb976a0 con 0x56214fecd5f0 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fecd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fecf9e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80124d0 tx=0x7fa2a80097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b87a453be4d8eb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa2ae7fc640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2980a2cf0 0x7fa2980a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4098240 tx=0x7fa2a4025360 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa295ffb640 1 -- 172.21.15.69:0/4111138152 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8004b20 con 0x56214fecd5f0 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa295ffb640 1 -- 172.21.15.69:0/4111138152 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a8009c90 con 0x56214fecd5f0 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.818+0000 7fa295ffb640 1 -- 172.21.15.69:0/4111138152 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a801e970 con 0x56214fecd5f0 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 -- 172.21.15.69:0/4111138152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 msgr2=0x56214fecf9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fecf9e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa2a80124d0 tx=0x7fa2a80097d0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 -- 172.21.15.69:0/4111138152 shutdown_connections 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fecf9e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fecd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/4111138152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec00a0 0x56214fec0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 -- 172.21.15.69:0/4111138152 >> 172.21.15.69:0/4111138152 conn(0x56214febc790 msgr2=0x56214febcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 -- 172.21.15.69:0/4111138152 shutdown_connections 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.819+0000 7fa2b7d78740 1 -- 172.21.15.69:0/4111138152 wait complete. 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 1 -- start start 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 1 Processor -- start 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 0x56214fec8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec09c0 0x56214fec8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fed6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56214fde0cb0 con 0x56214fecd5f0 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56214fe217b0 con 0x56214fec09c0 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56214fec3080 con 0x56214fec00a0 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2b7d78740 10 client.0 _ll_register_callbacks cb 0x56214fb34ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fed6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fed6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43090/0 (socket says 172.21.15.69:43090) 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2af7fe640 1 -- 172.21.15.69:0/3475906171 learned_addr learned my addr 172.21.15.69:0/3475906171 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec09c0 0x56214fec8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.820+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 0x56214fec8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2aeffd640 1 -- 172.21.15.69:0/3475906171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 msgr2=0x56214fec8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 0x56214fec8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2aeffd640 1 -- 172.21.15.69:0/3475906171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 msgr2=0x56214fed6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fed6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2aeffd640 1 -- 172.21.15.69:0/3475906171 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214fddfbc0 con 0x56214fec09c0 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fed6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec09c0 0x56214fec8d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009230 tx=0x7fa2a00052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f11ffe1313942 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa28effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa28effd640 1 -- 172.21.15.69:0/3475906171 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0035030 con 0x56214fec09c0 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 1 -- 172.21.15.69:0/3475906171 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a8009c90 con 0x56214fec09c0 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa28effd640 1 -- 172.21.15.69:0/3475906171 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa2a00048c0 con 0x56214fec09c0 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa28effd640 1 -- 172.21.15.69:0/3475906171 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2a004d050 con 0x56214fec09c0 2024-08-02T17:51:39.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 10 client.26339 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 1 -- 172.21.15.69:0/3475906171 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56214fed9f80 con 0x56214fec09c0 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa28effd640 1 -- 172.21.15.69:0/3475906171 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a8009c90 con 0x56214fec09c0 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa28effd640 1 -- 172.21.15.69:0/3475906171 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2a0003b50 con 0x56214fec09c0 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 10 client.26339 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 10 client.26339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 20 client.26339 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56214fedb5f0 0x56214fedda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 4 client.26339 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.821+0000 7fa2b7d78740 1 -- 172.21.15.69:0/3475906171 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56214fe32d60 con 0x56214fedb5f0 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.822+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56214fedb5f0 0x56214fedda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.822+0000 7fa28effd640 10 client.26339 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:39.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.822+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56214fedb5f0 0x56214fedda10 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9522d6aaefedf2f3 server_cookie=2479d658e3e7e7cb in_seq=0 out_seq=0 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.823+0000 7fa28effd640 1 -- 172.21.15.69:0/3475906171 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56214fe32d60 con 0x56214fedb5f0 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.823+0000 7fa28effd640 10 client.26339 handle_command_reply: tid=0 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.823+0000 7fa2b7d78740 1 client.26339 shutdown 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa2b7d78740 1 -- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec09c0 msgr2=0x56214fec8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec09c0 0x56214fec8d90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0009230 tx=0x7fa2a00052f0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa2b7d78740 1 -- 172.21.15.69:0/3475906171 shutdown_connections 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fecd5f0 0x56214fed6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56214fedb5f0 0x56214fedda10 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa28effd640 0 client.26339 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fec09c0 0x56214fec8d90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/3475906171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 0x56214fec8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.824+0000 7fa2b7d78740 1 -- 172.21.15.69:0/3475906171 >> 172.21.15.69:0/3475906171 conn(0x56214febc790 msgr2=0x56214febb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.825+0000 7fa2b7d78740 1 -- 172.21.15.69:0/3475906171 shutdown_connections 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.825+0000 7fa2b7d78740 1 -- 172.21.15.69:0/3475906171 wait complete. 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.825+0000 7fa2b7d78740 20 client.26339 trim_cache size 0 max 16384 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2b7d78740 1 Processor -- start 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2b7d78740 1 -- start start 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec00a0 0x56214febb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fed6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 0x56214fee3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01af370 con 0x56214fec00a0 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01acc40 con 0x56214fecd5f0 2024-08-02T17:51:39.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01aaa90 con 0x56214fec09c0 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec00a0 0x56214febb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec00a0 0x56214febb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43100/0 (socket says 172.21.15.69:43100) 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 -- 172.21.15.69:0/506243451 learned_addr learned my addr 172.21.15.69:0/506243451 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/506243451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fed6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 -- 172.21.15.69:0/506243451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 msgr2=0x56214fed6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/506243451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fed6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 -- 172.21.15.69:0/506243451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 msgr2=0x56214fee3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/506243451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 0x56214fee3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 -- 172.21.15.69:0/506243451 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214fec07f0 con 0x56214fec00a0 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.845+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/506243451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec00a0 0x56214febb7e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8012fd0 tx=0x7fa2a801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dee4ab21735b94 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2957fa640 1 -- 172.21.15.69:0/506243451 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8009c90 con 0x56214fec00a0 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2957fa640 1 -- 172.21.15.69:0/506243451 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a80125c0 con 0x56214fec00a0 2024-08-02T17:51:39.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2957fa640 1 -- 172.21.15.69:0/506243451 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a8010af0 con 0x56214fec00a0 2024-08-02T17:51:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2b7d78740 1 -- 172.21.15.69:0/506243451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec00a0 msgr2=0x56214febb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/506243451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec00a0 0x56214febb7e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8012fd0 tx=0x7fa2a801a040 comp rx=0 tx=0).stop 2024-08-02T17:51:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2b7d78740 1 -- 172.21.15.69:0/506243451 shutdown_connections 2024-08-02T17:51:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/506243451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 0x56214fee3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/506243451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec09c0 0x56214fed6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/506243451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec00a0 0x56214febb7e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2b7d78740 1 -- 172.21.15.69:0/506243451 >> 172.21.15.69:0/506243451 conn(0x56214febc790 msgr2=0x56214febb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.846+0000 7fa2b7d78740 1 -- 172.21.15.69:0/506243451 shutdown_connections 2024-08-02T17:51:39.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 -- 172.21.15.69:0/506243451 wait complete. 2024-08-02T17:51:39.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 -- start start 2024-08-02T17:51:39.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 Processor -- start 2024-08-02T17:51:39.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 0x56214fed1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec09c0 0x56214fed2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 0x56214ff44140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01ab100 con 0x56214fec09c0 2024-08-02T17:51:39.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01ac570 con 0x56214fecd5f0 2024-08-02T17:51:39.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b01ab7d0 con 0x56214fec00a0 2024-08-02T17:51:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2b7d78740 10 client.0 _ll_register_callbacks cb 0x7fa2a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.847+0000 7fa2aeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec09c0 0x56214fed2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2aeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec09c0 0x56214fed2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43116/0 (socket says 172.21.15.69:43116) 2024-08-02T17:51:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2aeffd640 1 -- 172.21.15.69:0/2688245806 learned_addr learned my addr 172.21.15.69:0/2688245806 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:39.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 0x56214fed1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 0x56214ff44140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2aeffd640 1 -- 172.21.15.69:0/2688245806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 msgr2=0x56214fed1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 0x56214fed1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2aeffd640 1 -- 172.21.15.69:0/2688245806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 msgr2=0x56214ff44140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:39.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 0x56214ff44140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2aeffd640 1 -- 172.21.15.69:0/2688245806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214ff44680 con 0x56214fec09c0 2024-08-02T17:51:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2aeffd640 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec09c0 0x56214fed2070 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x56214fecd5c0 tx=0x7fa2a0003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ea50f7132a7590 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa28effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa28effd640 1 -- 172.21.15.69:0/2688245806 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0035040 con 0x56214fec09c0 2024-08-02T17:51:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa28effd640 1 -- 172.21.15.69:0/2688245806 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fa2a00035f0 con 0x56214fec09c0 2024-08-02T17:51:39.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2b7d78740 1 -- 172.21.15.69:0/2688245806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a80125c0 con 0x56214fec09c0 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa28effd640 1 -- 172.21.15.69:0/2688245806 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa2a0005d50 con 0x56214fec09c0 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2b7d78740 10 client.16740 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.848+0000 7fa2b7d78740 1 -- 172.21.15.69:0/2688245806 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56214ff45970 con 0x56214fec09c0 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa28effd640 1 -- 172.21.15.69:0/2688245806 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a80125c0 con 0x56214fec09c0 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa28effd640 1 -- 172.21.15.69:0/2688245806 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa2a0059030 con 0x56214fec09c0 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa2b7d78740 10 client.16740 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa2b7d78740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa2b7d78740 20 client.16740 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56214fedb5f0 0x56214ff48180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa2b7d78740 4 client.16740 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa2b7d78740 1 -- 172.21.15.69:0/2688245806 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56214fe2d270 con 0x56214fedb5f0 2024-08-02T17:51:39.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.849+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56214fedb5f0 0x56214ff48180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.850+0000 7fa28effd640 10 client.16740 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.850+0000 7fa2af7fe640 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56214fedb5f0 0x56214ff48180 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14be787bf6b69ec6 server_cookie=7ea938be34f5acbc in_seq=0 out_seq=0 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa28effd640 1 -- 172.21.15.69:0/2688245806 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56214fe2d270 con 0x56214fedb5f0 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa28effd640 10 client.16740 handle_command_reply: tid=0 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 client.16740 shutdown 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 -- 172.21.15.69:0/2688245806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec09c0 msgr2=0x56214fed2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec09c0 0x56214fed2070 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x56214fecd5c0 tx=0x7fa2a0003fd0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 -- 172.21.15.69:0/2688245806 shutdown_connections 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56214fec09c0 0x56214fed2070 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56214fedb5f0 0x56214ff48180 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56214fecd5f0 0x56214ff44140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa28effd640 0 client.16740 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:39.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 --2- 172.21.15.69:0/2688245806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56214fec00a0 0x56214fed1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 -- 172.21.15.69:0/2688245806 >> 172.21.15.69:0/2688245806 conn(0x56214febc790 msgr2=0x56214febb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 -- 172.21.15.69:0/2688245806 shutdown_connections 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 1 -- 172.21.15.69:0/2688245806 wait complete. 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.851+0000 7fa2b7d78740 20 client.16740 trim_cache size 0 max 16384 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 -- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2980a2cf0 msgr2=0x7fa2980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2980a2cf0 0x7fa2980a51b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4098240 tx=0x7fa2a4025360 comp rx=0 tx=0).stop 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 -- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b01022a0 msgr2=0x7fa2b0197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b01022a0 0x7fa2b0197670 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e9f0 tx=0x7fa2a400eeb0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 -- 172.21.15.69:0/1969911563 shutdown_connections 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa2b01022a0 0x7fa2b0197670 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa2980a2cf0 0x7fa2980a51b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa2b0102c20 0x7fa2b019ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 --2- 172.21.15.69:0/1969911563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2b0100b50 0x7fa2b0197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:39.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 -- 172.21.15.69:0/1969911563 >> 172.21.15.69:0/1969911563 conn(0x7fa2b00fc500 msgr2=0x7fa2b00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:39.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 -- 172.21.15.69:0/1969911563 shutdown_connections 2024-08-02T17:51:39.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:38.852+0000 7fa2b597a640 1 -- 172.21.15.69:0/1969911563 wait complete. 2024-08-02T17:51:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 270.0 s, 1759.9 tps, lat 55.870 ms stddev 66.982 2024-08-02T17:51:40.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:40 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:51:40.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:40 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:40.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:40 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:51:40.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:40 smithi083 ceph-mon[37031]: pgmap v160: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.8 MiB/s wr, 784 op/s 2024-08-02T17:51:40.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:40 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:40.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:40 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:51:40.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:40 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:51:40.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:40 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:51:40.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:40 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:40.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:40 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:51:40.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:40 smithi077 ceph-mon[37108]: pgmap v160: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.8 MiB/s wr, 784 op/s 2024-08-02T17:51:40.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:40 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:40.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:40 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:51:40.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:40 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:51:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:40 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:51:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:40 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:40.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:40 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:51:40.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:40 smithi069 ceph-mon[32746]: pgmap v160: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.8 MiB/s wr, 784 op/s 2024-08-02T17:51:40.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:40 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:40.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:40 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:51:40.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:40 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:51:42.189 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:51:42.343 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:42.485 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:42 smithi069 ceph-mon[32746]: pgmap v161: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 834 op/s 2024-08-02T17:51:42.485 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:42 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:51:42.485 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:42 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:51:42.485 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:42 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:51:42.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:42 smithi083 ceph-mon[37031]: pgmap v161: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 834 op/s 2024-08-02T17:51:42.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:42 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:51:42.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:42 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:51:42.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:42 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:51:42.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:42 smithi077 ceph-mon[37108]: pgmap v161: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 10 MiB/s wr, 834 op/s 2024-08-02T17:51:42.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:42 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:51:42.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:42 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:51:42.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:42 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:51:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.892+0000 7f8ccceab640 1 -- 172.21.15.69:0/2067163084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 msgr2=0x7f8cc810ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:42.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.892+0000 7f8ccceab640 1 --2- 172.21.15.69:0/2067163084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 0x7f8cc810ff80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009900 tx=0x7f8cb802a440 comp rx=0 tx=0).stop 2024-08-02T17:51:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.896+0000 7f8ccceab640 1 -- 172.21.15.69:0/2067163084 shutdown_connections 2024-08-02T17:51:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.896+0000 7f8ccceab640 1 --2- 172.21.15.69:0/2067163084 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cc811ae70 0x7f8cc811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.896+0000 7f8ccceab640 1 --2- 172.21.15.69:0/2067163084 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc8109300 0x7f8cc8109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.896+0000 7f8ccceab640 1 --2- 172.21.15.69:0/2067163084 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 0x7f8cc810ff80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.896+0000 7f8ccceab640 1 -- 172.21.15.69:0/2067163084 >> 172.21.15.69:0/2067163084 conn(0x7f8cc806c900 msgr2=0x7f8cc806cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:42.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.898+0000 7f8ccceab640 1 -- 172.21.15.69:0/2067163084 shutdown_connections 2024-08-02T17:51:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.898+0000 7f8ccceab640 1 -- 172.21.15.69:0/2067163084 wait complete. 2024-08-02T17:51:42.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.899+0000 7f8ccceab640 1 Processor -- start 2024-08-02T17:51:42.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8ccceab640 1 -- start start 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8ccceab640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 0x7f8cc811a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8ccceab640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc8109300 0x7f8cc8114770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8ccceab640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cc811ae70 0x7f8cc8114cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8ccceab640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81110a0 con 0x7f8cc8108f20 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8ccceab640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8110f20 con 0x7f8cc811ae70 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8cc6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc8109300 0x7f8cc8114770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8cc6ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc8109300 0x7f8cc8114770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:46306/0 (socket says 172.21.15.69:46306) 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8cc6ffd640 1 -- 172.21.15.69:0/3873716899 learned_addr learned my addr 172.21.15.69:0/3873716899 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 0x7f8cc811a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cc811ae70 0x7f8cc8114cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.900+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc8111220 con 0x7f8cc8109300 2024-08-02T17:51:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.902+0000 7f8cc77fe640 1 -- 172.21.15.69:0/3873716899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc8109300 msgr2=0x7f8cc8114770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.902+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc8109300 0x7f8cc8114770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.902+0000 7f8cc77fe640 1 -- 172.21.15.69:0/3873716899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cc811ae70 msgr2=0x7f8cc8114cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.902+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cc811ae70 0x7f8cc8114cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.902+0000 7f8cc77fe640 1 -- 172.21.15.69:0/3873716899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cc8115570 con 0x7f8cc8108f20 2024-08-02T17:51:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.902+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 0x7f8cc811a700 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8cb802a9b0 tx=0x7f8cb8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.903+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cc811ae70 0x7f8cc8114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.903+0000 7f8cc4ff9640 1 -- 172.21.15.69:0/3873716899 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb803f070 con 0x7f8cc8108f20 2024-08-02T17:51:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.903+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cc81bbce0 con 0x7f8cc8108f20 2024-08-02T17:51:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.903+0000 7f8cc4ff9640 1 -- 172.21.15.69:0/3873716899 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cb803b460 con 0x7f8cc8108f20 2024-08-02T17:51:42.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.903+0000 7f8cc4ff9640 1 -- 172.21.15.69:0/3873716899 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8035030 con 0x7f8cc8108f20 2024-08-02T17:51:42.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.906+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cc81bc1a0 con 0x7f8cc8108f20 2024-08-02T17:51:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.909+0000 7f8cd0e68740 1 Processor -- start 2024-08-02T17:51:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.909+0000 7f8cd0e68740 1 -- start start 2024-08-02T17:51:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b6ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b7bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d3e3ad29b0 con 0x55d3e3b6e940 2024-08-02T17:51:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3e3a7a800 con 0x55d3e3b79b00 2024-08-02T17:51:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d3e3ace970 con 0x55d3e3b6f260 2024-08-02T17:51:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cc7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b6ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cc7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b6ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43140/0 (socket says 172.21.15.69:43140) 2024-08-02T17:51:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.910+0000 7f8cc7fff640 1 -- 172.21.15.69:0/111059765 learned_addr learned my addr 172.21.15.69:0/111059765 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.911+0000 7f8cc7fff640 1 -- 172.21.15.69:0/111059765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 msgr2=0x55d3e3b795c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:51:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.911+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/111059765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b795c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.911+0000 7f8cc7fff640 1 -- 172.21.15.69:0/111059765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 msgr2=0x55d3e3b7bef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:51:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.911+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/111059765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b7bef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.911+0000 7f8cc7fff640 1 -- 172.21.15.69:0/111059765 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e37e6210 con 0x55d3e3b6e940 2024-08-02T17:51:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.911+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/111059765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b6ed20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0007f00 tx=0x7f8cc000c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faad10a70590188a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.911+0000 7f8cadffb640 1 -- 172.21.15.69:0/111059765 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0014070 con 0x55d3e3b6e940 2024-08-02T17:51:42.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.912+0000 7f8cadffb640 1 -- 172.21.15.69:0/111059765 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc000fd30 con 0x55d3e3b6e940 2024-08-02T17:51:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.912+0000 7f8cc4ff9640 1 -- 172.21.15.69:0/3873716899 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8cb8042070 con 0x7f8cc8108f20 2024-08-02T17:51:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.913+0000 7f8cd0e68740 1 -- 172.21.15.69:0/111059765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 msgr2=0x55d3e3b6ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.913+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/111059765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b6ed20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0007f00 tx=0x7f8cc000c690 comp rx=0 tx=0).stop 2024-08-02T17:51:42.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.913+0000 7f8cc4ff9640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8cb00a2e40 0x7f8cb00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.913+0000 7f8cc4ff9640 1 -- 172.21.15.69:0/3873716899 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8cb80f01e0 con 0x7f8cc8108f20 2024-08-02T17:51:42.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.914+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8cb00a2e40 0x7f8cb00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.915+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8cb00a2e40 0x7f8cb00a5300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0007f00 tx=0x7f8cc0007bf0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.915+0000 7f8cd0e68740 1 -- 172.21.15.69:0/111059765 shutdown_connections 2024-08-02T17:51:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.915+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/111059765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b7bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.915+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/111059765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.915+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/111059765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b6ed20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.916+0000 7f8cd0e68740 1 -- 172.21.15.69:0/111059765 >> 172.21.15.69:0/111059765 conn(0x55d3e3b6aeb0 msgr2=0x55d3e3b6b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.916+0000 7f8cd0e68740 1 -- 172.21.15.69:0/111059765 shutdown_connections 2024-08-02T17:51:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.916+0000 7f8cd0e68740 1 -- 172.21.15.69:0/111059765 wait complete. 2024-08-02T17:51:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.916+0000 7f8cd0e68740 1 -- start start 2024-08-02T17:51:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.917+0000 7f8cd0e68740 1 Processor -- start 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.917+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b76960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6f260 0x55d3e3b76ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cc77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b76960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cc77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b76960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45688/0 (socket says 172.21.15.69:45688) 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cc77fe640 1 -- 172.21.15.69:0/3779838275 learned_addr learned my addr 172.21.15.69:0/3779838275 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b79b00 0x55d3e3b773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d3e3b7c720 con 0x55d3e3b6f260 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3e3a8e500 con 0x55d3e3b6e940 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b79b00 0x55d3e3b773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.918+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d3e3b7c8a0 con 0x55d3e3b79b00 2024-08-02T17:51:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.919+0000 7f8cd0e68740 10 client.0 _ll_register_callbacks cb 0x55d3e37e5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.919+0000 7f8cc6ffd640 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6f260 0x55d3e3b76ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.919+0000 7f8cc7fff640 1 -- 172.21.15.69:0/3779838275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 msgr2=0x55d3e3b76960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.919+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b76960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.920+0000 7f8cc7fff640 1 -- 172.21.15.69:0/3779838275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6f260 msgr2=0x55d3e3b76ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.920+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6f260 0x55d3e3b76ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.920+0000 7f8cc7fff640 1 -- 172.21.15.69:0/3779838275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e38b2620 con 0x55d3e3b79b00 2024-08-02T17:51:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.920+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b79b00 0x55d3e3b773e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000f040 tx=0x7f8cc00077d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=305633fc1585afb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.920+0000 7f8ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:51:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.921+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/3779838275 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0014040 con 0x55d3e3b79b00 2024-08-02T17:51:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.921+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc000fd30 con 0x55d3e3b79b00 2024-08-02T17:51:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.921+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/3779838275 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8cc000eb30 con 0x55d3e3b79b00 2024-08-02T17:51:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.921+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/3779838275 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8cc003a050 con 0x55d3e3b79b00 2024-08-02T17:51:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.925+0000 7f8cd0e68740 10 client.26242 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.925+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3e3b883c0 con 0x55d3e3b79b00 2024-08-02T17:51:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.925+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/3779838275 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc000fd30 con 0x55d3e3b79b00 2024-08-02T17:51:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.925+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/3779838275 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8cc0027920 con 0x55d3e3b79b00 2024-08-02T17:51:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.925+0000 7f8cd0e68740 10 client.26242 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.925+0000 7f8cd0e68740 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.925+0000 7f8cd0e68740 20 client.26242 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.925+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3e3b89c70 0x55d3e3b8c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.926+0000 7f8cd0e68740 4 client.26242 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.926+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3e3ae00e0 con 0x55d3e3b89c70 2024-08-02T17:51:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.926+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3e3b89c70 0x55d3e3b8c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.927+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3e3b89c70 0x55d3e3b8c0b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2fbd01d1860067 server_cookie=a13e731e90f894e7 in_seq=0 out_seq=0 2024-08-02T17:51:42.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.927+0000 7f8ca6ffd640 10 client.26242 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.928+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/3779838275 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d3e3ae00e0 con 0x55d3e3b89c70 2024-08-02T17:51:42.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.928+0000 7f8ca6ffd640 10 client.26242 handle_command_reply: tid=0 2024-08-02T17:51:42.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.928+0000 7f8cd0e68740 1 client.26242 shutdown 2024-08-02T17:51:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b79b00 msgr2=0x55d3e3b773e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b79b00 0x55d3e3b773e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000f040 tx=0x7f8cc00077d0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 shutdown_connections 2024-08-02T17:51:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6f260 0x55d3e3b76ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3e3b89c70 0x55d3e3b8c0b0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b79b00 0x55d3e3b773e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/3779838275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b76960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 >> 172.21.15.69:0/3779838275 conn(0x55d3e3b6aeb0 msgr2=0x55d3e3b70bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.933+0000 7f8ca6ffd640 0 client.26242 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.935+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 shutdown_connections 2024-08-02T17:51:42.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.935+0000 7f8cd0e68740 1 -- 172.21.15.69:0/3779838275 wait complete. 2024-08-02T17:51:42.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.937+0000 7f8cd0e68740 20 client.26242 trim_cache size 0 max 16384 2024-08-02T17:51:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.957+0000 7f34b99ca640 1 -- 172.21.15.69:0/3056275803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34b40ff920 msgr2=0x7f34b40ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.957+0000 7f34b99ca640 1 --2- 172.21.15.69:0/3056275803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34b40ff920 0x7f34b40ffd00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f34a801a910 tx=0x7f34a803b500 comp rx=0 tx=0).stop 2024-08-02T17:51:42.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.957+0000 7f34b99ca640 1 -- 172.21.15.69:0/3056275803 shutdown_connections 2024-08-02T17:51:42.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.957+0000 7f34b99ca640 1 --2- 172.21.15.69:0/3056275803 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34b4100240 0x7f34b4112070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.957+0000 7f34b99ca640 1 --2- 172.21.15.69:0/3056275803 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b41125b0 0x7f34b41149a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.957+0000 7f34b99ca640 1 --2- 172.21.15.69:0/3056275803 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34b40ff920 0x7f34b40ffd00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.957+0000 7f34b99ca640 1 -- 172.21.15.69:0/3056275803 >> 172.21.15.69:0/3056275803 conn(0x7f34b40fb7c0 msgr2=0x7f34b40fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:42.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.965+0000 7f34b99ca640 1 -- 172.21.15.69:0/3056275803 shutdown_connections 2024-08-02T17:51:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.965+0000 7f34b99ca640 1 -- 172.21.15.69:0/3056275803 wait complete. 2024-08-02T17:51:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b99ca640 1 Processor -- start 2024-08-02T17:51:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b99ca640 1 -- start start 2024-08-02T17:51:42.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b99ca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b40ff920 0x7f34b4111d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b99ca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34b4100240 0x7f34b410de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b99ca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34b41125b0 0x7f34b410e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b99ca640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f34b410a270 con 0x7f34b4100240 2024-08-02T17:51:42.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b99ca640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f34b410a0f0 con 0x7f34b40ff920 2024-08-02T17:51:42.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b99ca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f34b410a3f0 con 0x7f34b41125b0 2024-08-02T17:51:42.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34b4100240 0x7f34b410de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.966+0000 7f34b27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34b4100240 0x7f34b410de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43162/0 (socket says 172.21.15.69:43162) 2024-08-02T17:51:42.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.967+0000 7f34b27fc640 1 -- 172.21.15.69:0/1617084703 learned_addr learned my addr 172.21.15.69:0/1617084703 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:42.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.969+0000 7f34b37fe640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34b41125b0 0x7f34b410e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.969+0000 7f34b2ffd640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b40ff920 0x7f34b4111d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.970+0000 7f34b27fc640 1 -- 172.21.15.69:0/1617084703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34b41125b0 msgr2=0x7f34b410e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:42.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.970+0000 7f34b27fc640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34b41125b0 0x7f34b410e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.970+0000 7f34b27fc640 1 -- 172.21.15.69:0/1617084703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b40ff920 msgr2=0x7f34b4111d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.970+0000 7f34b27fc640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b40ff920 0x7f34b4111d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.970+0000 7f34b27fc640 1 -- 172.21.15.69:0/1617084703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b410ead0 con 0x7f34b4100240 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.971+0000 7f34b27fc640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34b4100240 0x7f34b410de10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00ba50 tx=0x7f34ac00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.971+0000 7f34b89c8640 1 -- 172.21.15.69:0/1617084703 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac019070 con 0x7f34b4100240 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.971+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b41a2650 con 0x7f34b4100240 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.971+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b41a2b50 con 0x7f34b4100240 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.971+0000 7f34b89c8640 1 -- 172.21.15.69:0/1617084703 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ac00cbe0 con 0x7f34b4100240 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.971+0000 7f34b89c8640 1 -- 172.21.15.69:0/1617084703 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0076e0 con 0x7f34b4100240 2024-08-02T17:51:42.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.974+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a0005290 con 0x7f34b4100240 2024-08-02T17:51:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.975+0000 7f34b89c8640 1 -- 172.21.15.69:0/1617084703 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f34ac01c070 con 0x7f34b4100240 2024-08-02T17:51:42.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.976+0000 7f34b89c8640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f349c0a2e40 0x7f349c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.976+0000 7f34b89c8640 1 -- 172.21.15.69:0/1617084703 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f34ac0cec30 con 0x7f34b4100240 2024-08-02T17:51:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.983+0000 7f34b2ffd640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f349c0a2e40 0x7f349c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.986+0000 7f34b89c8640 1 -- 172.21.15.69:0/1617084703 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f34ac09b540 con 0x7f34b4100240 2024-08-02T17:51:42.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:42.988+0000 7f34b2ffd640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f349c0a2e40 0x7f349c0a5300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f34a801a6d0 tx=0x7f34a8005fd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.995+0000 7f8cd0e68740 1 Processor -- start 2024-08-02T17:51:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.995+0000 7f8cd0e68740 1 -- start start 2024-08-02T17:51:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.995+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b77210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.999+0000 7f8cc77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b77210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:43.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:42.999+0000 7f8cc77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b77210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45722/0 (socket says 172.21.15.69:45722) 2024-08-02T17:51:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.003+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b919c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.003+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b79b00 0x55d3e3b91f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.003+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81c8bb0 con 0x55d3e3b79b00 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.003+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81c6480 con 0x55d3e3b6e940 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.003+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81c42d0 con 0x55d3e3b6f260 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.004+0000 7f8cc7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b79b00 0x55d3e3b91f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.004+0000 7f8cc7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b79b00 0x55d3e3b91f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43176/0 (socket says 172.21.15.69:43176) 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.004+0000 7f8cc7fff640 1 -- 172.21.15.69:0/1200936729 learned_addr learned my addr 172.21.15.69:0/1200936729 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.004+0000 7f8cc7fff640 1 -- 172.21.15.69:0/1200936729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 msgr2=0x55d3e3b919c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.004+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/1200936729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b919c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.004+0000 7f8cc7fff640 1 -- 172.21.15.69:0/1200936729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 msgr2=0x55d3e3b77210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.004+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/1200936729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b77210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.004+0000 7f8cc7fff640 1 -- 172.21.15.69:0/1200936729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e3b79410 con 0x55d3e3b79b00 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.005+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/1200936729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b79b00 0x55d3e3b91f00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c6d0 tx=0x7f8cc0007510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2181a9330ca7bc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.006+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/1200936729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b77210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-08-02T17:51:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.006+0000 7f8cad7fa640 1 -- 172.21.15.69:0/1200936729 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0014040 con 0x55d3e3b79b00 2024-08-02T17:51:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.006+0000 7f8cad7fa640 1 -- 172.21.15.69:0/1200936729 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc002c070 con 0x55d3e3b79b00 2024-08-02T17:51:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.006+0000 7f8cad7fa640 1 -- 172.21.15.69:0/1200936729 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000f8b0 con 0x55d3e3b79b00 2024-08-02T17:51:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.007+0000 7f8cd0e68740 1 -- 172.21.15.69:0/1200936729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b79b00 msgr2=0x55d3e3b91f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.007+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/1200936729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b79b00 0x55d3e3b91f00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000c6d0 tx=0x7f8cc0007510 comp rx=0 tx=0).stop 2024-08-02T17:51:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.009+0000 7f8cd0e68740 1 -- 172.21.15.69:0/1200936729 shutdown_connections 2024-08-02T17:51:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.009+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/1200936729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b79b00 0x55d3e3b91f00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.009+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/1200936729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b919c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.009+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/1200936729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b6e940 0x55d3e3b77210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.009+0000 7f8cd0e68740 1 -- 172.21.15.69:0/1200936729 >> 172.21.15.69:0/1200936729 conn(0x55d3e3b6aeb0 msgr2=0x55d3e3b75a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.011+0000 7f8cd0e68740 1 -- 172.21.15.69:0/1200936729 shutdown_connections 2024-08-02T17:51:43.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.013+0000 7f8cd0e68740 1 -- 172.21.15.69:0/1200936729 wait complete. 2024-08-02T17:51:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.013+0000 7f8cd0e68740 1 -- start start 2024-08-02T17:51:43.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.014+0000 7f8cd0e68740 1 Processor -- start 2024-08-02T17:51:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.014+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b82040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.014+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b73ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.014+0000 7f8cd0e68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.014+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81c4940 con 0x55d3e3b6e940 2024-08-02T17:51:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.014+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81c5db0 con 0x55d3e3b79b00 2024-08-02T17:51:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.014+0000 7f8cd0e68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8cc81c5010 con 0x55d3e3b6f260 2024-08-02T17:51:43.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.015+0000 7f8cd0e68740 10 client.0 _ll_register_callbacks cb 0x7f8cc0007d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45734/0 (socket says 172.21.15.69:45734) 2024-08-02T17:51:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 -- 172.21.15.69:0/2618928272 learned_addr learned my addr 172.21.15.69:0/2618928272 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:43.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b82040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 -- 172.21.15.69:0/2618928272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 msgr2=0x55d3e3b73ca0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:51:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b73ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 -- 172.21.15.69:0/2618928272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 msgr2=0x55d3e3b82040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b82040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 -- 172.21.15.69:0/2618928272 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e3b74750 con 0x55d3e3b79b00 2024-08-02T17:51:43.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b82040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.016+0000 7f8cc7fff640 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b741e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0000f80 tx=0x7f8cc000e910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4275ec180de7d1b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.017+0000 7f8ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.017+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/2618928272 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc0014040 con 0x55d3e3b79b00 2024-08-02T17:51:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.017+0000 7f8cd0e68740 1 -- 172.21.15.69:0/2618928272 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cc002c070 con 0x55d3e3b79b00 2024-08-02T17:51:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.017+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/2618928272 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8cc0027850 con 0x55d3e3b79b00 2024-08-02T17:51:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.017+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/2618928272 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8cc004b050 con 0x55d3e3b79b00 2024-08-02T17:51:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.017+0000 7f8cd0e68740 10 client.26351 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.017+0000 7f8cd0e68740 1 -- 172.21.15.69:0/2618928272 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3e3b739c0 con 0x55d3e3b79b00 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/2618928272 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc002c070 con 0x55d3e3b79b00 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/2618928272 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8cc003d030 con 0x55d3e3b79b00 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8cd0e68740 10 client.26351 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8cd0e68740 10 client.26351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8cd0e68740 20 client.26351 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3e3bf3d80 0x55d3e3bf6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8cd0e68740 4 client.26351 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8cd0e68740 1 -- 172.21.15.69:0/2618928272 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3e3ada430 con 0x55d3e3bf3d80 2024-08-02T17:51:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.018+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3e3bf3d80 0x55d3e3bf6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.019+0000 7f8cc77fe640 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3e3bf3d80 0x55d3e3bf6170 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc27574fa5104d61 server_cookie=daca7d985cd0c87f in_seq=0 out_seq=0 2024-08-02T17:51:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.019+0000 7f8ca6ffd640 10 client.26351 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:43.049 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8ca6ffd640 1 -- 172.21.15.69:0/2618928272 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3e3ada430 con 0x55d3e3bf3d80 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8ca6ffd640 10 client.26351 handle_command_reply: tid=0 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 client.26351 shutdown 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 -- 172.21.15.69:0/2618928272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 msgr2=0x55d3e3b741e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b741e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0000f80 tx=0x7f8cc000e910 comp rx=0 tx=0).stop 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 -- 172.21.15.69:0/2618928272 shutdown_connections 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3e3bf3d80 0x55d3e3bf6170 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3e3b79b00 0x55d3e3b741e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3e3b6f260 0x55d3e3b73ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 --2- 172.21.15.69:0/2618928272 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3e3b6e940 0x55d3e3b82040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 -- 172.21.15.69:0/2618928272 >> 172.21.15.69:0/2618928272 conn(0x55d3e3b6aeb0 msgr2=0x55d3e3b696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8ca6ffd640 0 client.26351 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 -- 172.21.15.69:0/2618928272 shutdown_connections 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 1 -- 172.21.15.69:0/2618928272 wait complete. 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.061+0000 7f8cd0e68740 20 client.26351 trim_cache size 0 max 16384 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:51:43.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:51:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8cb00a2e40 msgr2=0x7f8cb00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8cb00a2e40 0x7f8cb00a5300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0007f00 tx=0x7f8cc0007bf0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 msgr2=0x7f8cc811a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 0x7f8cc811a700 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8cb802a9b0 tx=0x7f8cb8003cd0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 shutdown_connections 2024-08-02T17:51:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8cb00a2e40 0x7f8cb00a5300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8cc811ae70 0x7f8cc8114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8cc8109300 0x7f8cc8114770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 --2- 172.21.15.69:0/3873716899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8cc8108f20 0x7f8cc811a700 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 >> 172.21.15.69:0/3873716899 conn(0x7f8cc806c900 msgr2=0x7f8cc8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 shutdown_connections 2024-08-02T17:51:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:43.062+0000 7f8ccceab640 1 -- 172.21.15.69:0/3873716899 wait complete. 2024-08-02T17:51:43.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.365+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34a0005850 con 0x7f34b4100240 2024-08-02T17:51:43.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.370+0000 7f34b89c8640 1 -- 172.21.15.69:0/1617084703 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f34ac09ae40 con 0x7f34b4100240 2024-08-02T17:51:43.373 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:51:43.373 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:51:43.376 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f349c0a2e40 msgr2=0x7f349c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f349c0a2e40 0x7f349c0a5300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f34a801a6d0 tx=0x7f34a8005fd0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34b4100240 msgr2=0x7f34b410de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34b4100240 0x7f34b410de10 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00ba50 tx=0x7f34ac00bf10 comp rx=0 tx=0).stop 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 shutdown_connections 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f34b4100240 0x7f34b410de10 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f349c0a2e40 0x7f349c0a5300 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f34a801a6d0 tx=0x7f34a8005fd0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f34b41125b0 0x7f34b410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 --2- 172.21.15.69:0/1617084703 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f34b40ff920 0x7f34b4111d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:43.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 >> 172.21.15.69:0/1617084703 conn(0x7f34b40fb7c0 msgr2=0x7f34b4112b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:43.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 shutdown_connections 2024-08-02T17:51:43.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:43.379+0000 7f34b99ca640 1 -- 172.21.15.69:0/1617084703 wait complete. 2024-08-02T17:51:44.396 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 148.9 --await-for 45.0' 2024-08-02T17:51:44.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 148.9 --await-for 45.0 2024-08-02T17:51:44.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:44 smithi083 ceph-mon[37031]: pgmap v162: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:51:44.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:44 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1617084703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:44.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:44 smithi069 ceph-mon[32746]: pgmap v162: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:51:44.549 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:44 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1617084703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:44.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:44 smithi077 ceph-mon[37108]: pgmap v162: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2024-08-02T17:51:44.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:44 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1617084703' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:44.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 275.0 s, 1222.9 tps, lat 82.917 ms stddev 101.815 2024-08-02T17:51:46.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.202+0000 7f85ea482640 1 -- 172.21.15.69:0/628624406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e411ae70 msgr2=0x7f85e411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.202+0000 7f85ea482640 1 --2- 172.21.15.69:0/628624406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e411ae70 0x7f85e411d260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f85cc01a910 tx=0x7f85cc03b470 comp rx=0 tx=0).stop 2024-08-02T17:51:46.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.202+0000 7f85ea482640 1 -- 172.21.15.69:0/628624406 shutdown_connections 2024-08-02T17:51:46.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.202+0000 7f85ea482640 1 --2- 172.21.15.69:0/628624406 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85e4109560 0x7f85e41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.202+0000 7f85ea482640 1 --2- 172.21.15.69:0/628624406 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e411ae70 0x7f85e411d260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.202+0000 7f85ea482640 1 --2- 172.21.15.69:0/628624406 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85e4109180 0x7f85e410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.202+0000 7f85ea482640 1 -- 172.21.15.69:0/628624406 >> 172.21.15.69:0/628624406 conn(0x7f85e406de70 msgr2=0x7f85e406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.204+0000 7f85ea482640 1 -- 172.21.15.69:0/628624406 shutdown_connections 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.204+0000 7f85ea482640 1 -- 172.21.15.69:0/628624406 wait complete. 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.205+0000 7f85ea482640 1 Processor -- start 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.208+0000 7f85ea482640 1 -- start start 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.208+0000 7f85ea482640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85e4109180 0x7f85e411a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.208+0000 7f85ea482640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e4109560 0x7f85e4116730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.208+0000 7f85ea482640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85e411ae70 0x7f85e4116c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.208+0000 7f85ea482640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4113110 con 0x7f85e411ae70 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.208+0000 7f85ea482640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4112f90 con 0x7f85e4109560 2024-08-02T17:51:46.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.208+0000 7f85ea482640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85e4113290 con 0x7f85e4109180 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.209+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85e4109180 0x7f85e411a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.209+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85e4109180 0x7f85e411a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:44954/0 (socket says 172.21.15.69:44954) 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.209+0000 7f85e3fff640 1 -- 172.21.15.69:0/2174160107 learned_addr learned my addr 172.21.15.69:0/2174160107 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.212+0000 7f85e8a1f640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85e411ae70 0x7f85e4116c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.215+0000 7f85e37fe640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e4109560 0x7f85e4116730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.217+0000 7f85e3fff640 1 -- 172.21.15.69:0/2174160107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e4109560 msgr2=0x7f85e4116730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.217+0000 7f85e3fff640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e4109560 0x7f85e4116730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.217+0000 7f85e3fff640 1 -- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85e411ae70 msgr2=0x7f85e4116c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.217+0000 7f85e3fff640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85e411ae70 0x7f85e4116c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.217+0000 7f85e3fff640 1 -- 172.21.15.69:0/2174160107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e4117530 con 0x7f85e4109180 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.217+0000 7f85e37fe640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e4109560 0x7f85e4116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.218+0000 7f85e3fff640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85e4109180 0x7f85e411a690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85d400b450 tx=0x7f85d400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:46.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.228+0000 7f85e17fa640 1 -- 172.21.15.69:0/2174160107 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4015020 con 0x7f85e4109180 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.228+0000 7f85ea482640 1 -- 172.21.15.69:0/2174160107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e407c1a0 con 0x7f85e4109180 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.228+0000 7f85ea482640 1 -- 172.21.15.69:0/2174160107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e407c6e0 con 0x7f85e4109180 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.228+0000 7f85e17fa640 1 -- 172.21.15.69:0/2174160107 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d4011d70 con 0x7f85e4109180 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.228+0000 7f85e17fa640 1 -- 172.21.15.69:0/2174160107 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d400f340 con 0x7f85e4109180 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.234+0000 7f85e17fa640 1 -- 172.21.15.69:0/2174160107 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f85d4018020 con 0x7f85e4109180 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.234+0000 7f85ec880740 1 Processor -- start 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85ec880740 1 -- start start 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85ec880740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x56321970a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85ec880740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x563219714eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85ec880740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x5632197177e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85ec880740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56321966f980 con 0x5632197153f0 2024-08-02T17:51:46.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85ec880740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563219607a70 con 0x56321970a210 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85ec880740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56321966b940 con 0x56321970ab30 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x5632197177e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85e8a1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x56321970a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85e8a1f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x56321970a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52662/0 (socket says 172.21.15.69:52662) 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.235+0000 7f85e8a1f640 1 -- 172.21.15.69:0/1357725539 learned_addr learned my addr 172.21.15.69:0/1357725539 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e17fa640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85d00a2e40 0x7f85d00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e8a1f640 1 -- 172.21.15.69:0/1357725539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 msgr2=0x563219714eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e8a1f640 1 --2- 172.21.15.69:0/1357725539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x563219714eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e8a1f640 1 -- 172.21.15.69:0/1357725539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 msgr2=0x5632197177e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e8a1f640 1 --2- 172.21.15.69:0/1357725539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x5632197177e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e8a1f640 1 -- 172.21.15.69:0/1357725539 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321966a6e0 con 0x56321970a210 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e17fa640 1 -- 172.21.15.69:0/2174160107 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f85d40d0210 con 0x7f85e4109180 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e8a1f640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85d00a2e40 0x7f85d00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.236+0000 7f85e8a1f640 1 --2- 172.21.15.69:0/1357725539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x56321970a5f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0051e0 tx=0x7f85cc005210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d526f4da8ca7cdf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.237+0000 7f85e8a1f640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85d00a2e40 0x7f85d00a5300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85cc05a000 tx=0x7f85cc05d000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.237+0000 7f85e37fe640 1 --2- 172.21.15.69:0/1357725539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x5632197177e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.240+0000 7f85da7fc640 1 -- 172.21.15.69:0/1357725539 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc046030 con 0x56321970a210 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.240+0000 7f85da7fc640 1 -- 172.21.15.69:0/1357725539 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85cc0409f0 con 0x56321970a210 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.241+0000 7f85ec880740 1 -- 172.21.15.69:0/1357725539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 msgr2=0x56321970a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.241+0000 7f85ec880740 1 --2- 172.21.15.69:0/1357725539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x56321970a5f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0051e0 tx=0x7f85cc005210 comp rx=0 tx=0).stop 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.242+0000 7f85ec880740 1 -- 172.21.15.69:0/1357725539 shutdown_connections 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.242+0000 7f85ec880740 1 --2- 172.21.15.69:0/1357725539 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x5632197177e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.242+0000 7f85ec880740 1 --2- 172.21.15.69:0/1357725539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x563219714eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.242+0000 7f85ec880740 1 --2- 172.21.15.69:0/1357725539 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x56321970a5f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.242+0000 7f85ec880740 1 -- 172.21.15.69:0/1357725539 >> 172.21.15.69:0/1357725539 conn(0x5632197067a0 msgr2=0x563219706b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.247+0000 7f85ec880740 1 -- 172.21.15.69:0/1357725539 shutdown_connections 2024-08-02T17:51:46.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.247+0000 7f85ec880740 1 -- 172.21.15.69:0/1357725539 wait complete. 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.248+0000 7f85ec880740 1 -- start start 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.248+0000 7f85ec880740 1 Processor -- start 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.249+0000 7f85ec880740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x563219714e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.249+0000 7f85ec880740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x5632197121e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.249+0000 7f85ec880740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x563219712970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.249+0000 7f85ec880740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563219718010 con 0x5632197153f0 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.249+0000 7f85ec880740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56321962aaa0 con 0x56321970a210 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.249+0000 7f85ec880740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563219718190 con 0x56321970ab30 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.249+0000 7f85ec880740 10 client.0 _ll_register_callbacks cb 0x563219619e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:46.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.250+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x563219714e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.250+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x563219714e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52672/0 (socket says 172.21.15.69:52672) 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.250+0000 7f85e3fff640 1 -- 172.21.15.69:0/2322942050 learned_addr learned my addr 172.21.15.69:0/2322942050 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.253+0000 7f85e37fe640 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x5632197121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.254+0000 7f85e3fff640 1 -- 172.21.15.69:0/2322942050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 msgr2=0x5632197121e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.254+0000 7f85e3fff640 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x5632197121e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.254+0000 7f85e3fff640 1 -- 172.21.15.69:0/2322942050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 msgr2=0x563219712970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.254+0000 7f85e3fff640 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x563219712970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.254+0000 7f85e3fff640 1 -- 172.21.15.69:0/2322942050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321960e650 con 0x56321970a210 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.255+0000 7f85e3fff640 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x563219714e90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f85d40cdba0 tx=0x7f85d409cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21ad0f28cac8c10c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.255+0000 7f85c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:46.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.255+0000 7f85c37fe640 1 -- 172.21.15.69:0/2322942050 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d40a50f0 con 0x56321970a210 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.255+0000 7f85ec880740 1 -- 172.21.15.69:0/2322942050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85cc0409f0 con 0x56321970a210 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.255+0000 7f85c37fe640 1 -- 172.21.15.69:0/2322942050 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f85d40119b0 con 0x56321970a210 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.255+0000 7f85c37fe640 1 -- 172.21.15.69:0/2322942050 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85d4011bb0 con 0x56321970a210 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.256+0000 7f85ec880740 10 client.26363 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.256+0000 7f85ec880740 1 -- 172.21.15.69:0/2322942050 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5632193e13c0 con 0x56321970a210 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.258+0000 7f85c37fe640 1 -- 172.21.15.69:0/2322942050 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85cc0409f0 con 0x56321970a210 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.258+0000 7f85c37fe640 1 -- 172.21.15.69:0/2322942050 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f85d409b8d0 con 0x56321970a210 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.258+0000 7f85ec880740 10 client.26363 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.258+0000 7f85ec880740 10 client.26363 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.258+0000 7f85ec880740 20 client.26363 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.258+0000 7f85ec880740 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563219724f10 0x563219727320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.258+0000 7f85ec880740 4 client.26363 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.258+0000 7f85ec880740 1 -- 172.21.15.69:0/2322942050 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321967cef0 con 0x563219724f10 2024-08-02T17:51:46.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.259+0000 7f85e37fe640 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563219724f10 0x563219727320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.262+0000 7f85e37fe640 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563219724f10 0x563219727320 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7607c8b21a28e97 server_cookie=42f29fa901d6ed1f in_seq=0 out_seq=0 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.263+0000 7f85c37fe640 10 client.26363 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.264+0000 7f85c37fe640 1 -- 172.21.15.69:0/2322942050 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56321967cef0 con 0x563219724f10 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.264+0000 7f85c37fe640 10 client.26363 handle_command_reply: tid=0 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.264+0000 7f85ec880740 1 client.26363 shutdown 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.265+0000 7f85ec880740 1 -- 172.21.15.69:0/2322942050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 msgr2=0x563219714e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.265+0000 7f85ec880740 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x563219714e90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f85d40cdba0 tx=0x7f85d409cb80 comp rx=0 tx=0).stop 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.268+0000 7f85ec880740 1 -- 172.21.15.69:0/2322942050 shutdown_connections 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.268+0000 7f85ec880740 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x563219712970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.268+0000 7f85ec880740 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563219724f10 0x563219727320 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.268+0000 7f85ec880740 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x5632197121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.268+0000 7f85ec880740 1 --2- 172.21.15.69:0/2322942050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970a210 0x563219714e90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.268+0000 7f85ec880740 1 -- 172.21.15.69:0/2322942050 >> 172.21.15.69:0/2322942050 conn(0x5632197067a0 msgr2=0x56321970c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.268+0000 7f85c37fe640 0 client.26363 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.269+0000 7f85ec880740 1 -- 172.21.15.69:0/2322942050 shutdown_connections 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.270+0000 7f85ec880740 1 -- 172.21.15.69:0/2322942050 wait complete. 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.274+0000 7f85ec880740 20 client.26363 trim_cache size 0 max 16384 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.307+0000 7f85ec880740 1 Processor -- start 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.308+0000 7f85ec880740 1 -- start start 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.308+0000 7f85ec880740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970a210 0x5632197121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.309+0000 7f85ec880740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970ab30 0x56321972cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.309+0000 7f85ec880740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x56321972d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.309+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970ab30 0x56321972cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.309+0000 7f85e37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970ab30 0x56321972cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52680/0 (socket says 172.21.15.69:52680) 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.309+0000 7f85e37fe640 1 -- 172.21.15.69:0/1330216533 learned_addr learned my addr 172.21.15.69:0/1330216533 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.309+0000 7f85ec880740 1 -- 172.21.15.69:0/1330216533 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85c40128c0 con 0x5632197153f0 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.309+0000 7f85ec880740 1 -- 172.21.15.69:0/1330216533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85c4010190 con 0x56321970ab30 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.309+0000 7f85ec880740 1 -- 172.21.15.69:0/1330216533 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85c400dfe0 con 0x56321970a210 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.310+0000 7f85e8a1f640 1 --2- 172.21.15.69:0/1330216533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x56321972d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.310+0000 7f85e37fe640 1 -- 172.21.15.69:0/1330216533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970a210 msgr2=0x5632197121b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.310+0000 7f85e37fe640 1 --2- 172.21.15.69:0/1330216533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970a210 0x5632197121b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.310+0000 7f85e37fe640 1 -- 172.21.15.69:0/1330216533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 msgr2=0x56321972d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.310+0000 7f85e37fe640 1 --2- 172.21.15.69:0/1330216533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x56321972d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.310+0000 7f85e37fe640 1 -- 172.21.15.69:0/1330216533 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563219712ce0 con 0x56321970ab30 2024-08-02T17:51:46.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.311+0000 7f85e37fe640 1 --2- 172.21.15.69:0/1330216533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970ab30 0x56321972cbf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f85e4117ef0 tx=0x7f85dc009a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87fc94620ef3615c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.311+0000 7f85d9ffb640 1 -- 172.21.15.69:0/1330216533 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00d2d0 con 0x56321970ab30 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.311+0000 7f85d9ffb640 1 -- 172.21.15.69:0/1330216533 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85dc00adf0 con 0x56321970ab30 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.311+0000 7f85d9ffb640 1 -- 172.21.15.69:0/1330216533 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0193f0 con 0x56321970ab30 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.313+0000 7f85ec880740 1 -- 172.21.15.69:0/1330216533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970ab30 msgr2=0x56321972cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.313+0000 7f85ec880740 1 --2- 172.21.15.69:0/1330216533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970ab30 0x56321972cbf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f85e4117ef0 tx=0x7f85dc009a00 comp rx=0 tx=0).stop 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.313+0000 7f85ec880740 1 -- 172.21.15.69:0/1330216533 shutdown_connections 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.313+0000 7f85ec880740 1 --2- 172.21.15.69:0/1330216533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x56321972d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.313+0000 7f85ec880740 1 --2- 172.21.15.69:0/1330216533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970ab30 0x56321972cbf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.313+0000 7f85ec880740 1 --2- 172.21.15.69:0/1330216533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970a210 0x5632197121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.313+0000 7f85ec880740 1 -- 172.21.15.69:0/1330216533 >> 172.21.15.69:0/1330216533 conn(0x5632197067a0 msgr2=0x563219711aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.313+0000 7f85ec880740 1 -- 172.21.15.69:0/1330216533 shutdown_connections 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.314+0000 7f85ec880740 1 -- 172.21.15.69:0/1330216533 wait complete. 2024-08-02T17:51:46.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.314+0000 7f85ec880740 1 -- start start 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.314+0000 7f85ec880740 1 Processor -- start 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.315+0000 7f85ec880740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x56321971dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.315+0000 7f85ec880740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x56321970efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.315+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x56321971dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.315+0000 7f85e3fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x56321971dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:44974/0 (socket says 172.21.15.69:44974) 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.316+0000 7f85e3fff640 1 -- 172.21.15.69:0/3687300233 learned_addr learned my addr 172.21.15.69:0/3687300233 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.316+0000 7f85ec880740 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970f540 0x56321970f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.316+0000 7f85e37fe640 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x56321970efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.316+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f85c400e650 con 0x5632197153f0 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.316+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f85c400fac0 con 0x56321970f540 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.316+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85c400ed20 con 0x56321970ab30 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.316+0000 7f85ec880740 10 client.0 _ll_register_callbacks cb 0x7f85d4010f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.316+0000 7f85e8a1f640 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970f540 0x56321970f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85e3fff640 1 -- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970f540 msgr2=0x56321970f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85e3fff640 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970f540 0x56321970f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85e3fff640 1 -- 172.21.15.69:0/3687300233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 msgr2=0x56321970efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85e3fff640 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x56321970efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85e3fff640 1 -- 172.21.15.69:0/3687300233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321978edf0 con 0x56321970ab30 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85e3fff640 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x56321971dd40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f85d409c640 tx=0x7f85d4006d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a75d86673ce6484 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85c37fe640 1 -- 172.21.15.69:0/3687300233 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d409bfc0 con 0x56321970ab30 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.317+0000 7f85c37fe640 1 -- 172.21.15.69:0/3687300233 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f85d4098af0 con 0x56321970ab30 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.318+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85dc00adf0 con 0x56321970ab30 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.318+0000 7f85c37fe640 1 -- 172.21.15.69:0/3687300233 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f85d409bfc0 con 0x56321970ab30 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.318+0000 7f85ec880740 10 client.26257 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.318+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5632197900e0 con 0x56321970ab30 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.320+0000 7f85c37fe640 1 -- 172.21.15.69:0/3687300233 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85dc00adf0 con 0x56321970ab30 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.320+0000 7f85c37fe640 1 -- 172.21.15.69:0/3687300233 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f85d409b520 con 0x56321970ab30 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.321+0000 7f85ec880740 10 client.26257 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:46.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.321+0000 7f85ec880740 10 client.26257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.321+0000 7f85ec880740 20 client.26257 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.321+0000 7f85ec880740 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5632197907f0 0x563219792be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.321+0000 7f85ec880740 4 client.26257 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.321+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563219677400 con 0x5632197907f0 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.321+0000 7f85e37fe640 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5632197907f0 0x563219792be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.322+0000 7f85e37fe640 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5632197907f0 0x563219792be0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81ef56648866b760 server_cookie=f8c344f3d4be9bd6 in_seq=0 out_seq=0 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.322+0000 7f85c37fe640 10 client.26257 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.322+0000 7f85c37fe640 1 -- 172.21.15.69:0/3687300233 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563219677400 con 0x5632197907f0 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.322+0000 7f85c37fe640 10 client.26257 handle_command_reply: tid=0 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.322+0000 7f85ec880740 1 client.26257 shutdown 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 msgr2=0x56321971dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x56321971dd40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f85d409c640 tx=0x7f85d4006d50 comp rx=0 tx=0).stop 2024-08-02T17:51:46.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 shutdown_connections 2024-08-02T17:51:46.203 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:51:46.203 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:51:46.203 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:51:46.203 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5632197153f0 0x56321970efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5632197907f0 0x563219792be0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56321970f540 0x56321970f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 --2- 172.21.15.69:0/3687300233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56321970ab30 0x56321971dd40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 >> 172.21.15.69:0/3687300233 conn(0x5632197067a0 msgr2=0x5632197050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85c37fe640 0 client.26257 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 shutdown_connections 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.324+0000 7f85ec880740 1 -- 172.21.15.69:0/3687300233 wait complete. 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.325+0000 7f85ec880740 20 client.26257 trim_cache size 0 max 16384 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 -- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85d00a2e40 msgr2=0x7f85d00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85d00a2e40 0x7f85d00a5300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85cc05a000 tx=0x7f85cc05d000 comp rx=0 tx=0).stop 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 -- 172.21.15.69:0/2174160107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85e4109180 msgr2=0x7f85e411a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:46.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85e4109180 0x7f85e411a690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f85d400b450 tx=0x7f85d400b910 comp rx=0 tx=0).stop 2024-08-02T17:51:46.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 -- 172.21.15.69:0/2174160107 shutdown_connections 2024-08-02T17:51:46.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f85e411ae70 0x7f85e4116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f85d00a2e40 0x7f85d00a5300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f85e4109560 0x7f85e4116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 --2- 172.21.15.69:0/2174160107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85e4109180 0x7f85e411a690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:46.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.326+0000 7f85ea482640 1 -- 172.21.15.69:0/2174160107 >> 172.21.15.69:0/2174160107 conn(0x7f85e406de70 msgr2=0x7f85e4070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:46.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.327+0000 7f85ea482640 1 -- 172.21.15.69:0/2174160107 shutdown_connections 2024-08-02T17:51:46.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:45.327+0000 7f85ea482640 1 -- 172.21.15.69:0/2174160107 wait complete. 2024-08-02T17:51:46.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:46 smithi083 ceph-mon[37031]: pgmap v163: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.3 MiB/s wr, 728 op/s 2024-08-02T17:51:46.565 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:46 smithi069 ceph-mon[32746]: pgmap v163: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.3 MiB/s wr, 728 op/s 2024-08-02T17:51:46.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:46 smithi077 ceph-mon[37108]: pgmap v163: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 7.3 MiB/s wr, 728 op/s 2024-08-02T17:51:46.998 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:51:46.998 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:51:47.122 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:47.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.741+0000 7f4a49d3a640 1 -- 172.21.15.69:0/3609859036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a44109560 msgr2=0x7f4a441099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:47.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.741+0000 7f4a49d3a640 1 --2- 172.21.15.69:0/3609859036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a44109560 0x7f4a441099c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c01a910 tx=0x7f4a2c03b450 comp rx=0 tx=0).stop 2024-08-02T17:51:47.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.744+0000 7f4a49d3a640 1 -- 172.21.15.69:0/3609859036 shutdown_connections 2024-08-02T17:51:47.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.744+0000 7f4a49d3a640 1 --2- 172.21.15.69:0/3609859036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a44109560 0x7f4a441099c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.744+0000 7f4a49d3a640 1 --2- 172.21.15.69:0/3609859036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a4411ae70 0x7f4a4411d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.744+0000 7f4a49d3a640 1 --2- 172.21.15.69:0/3609859036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44109180 0x7f4a4410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.744+0000 7f4a49d3a640 1 -- 172.21.15.69:0/3609859036 >> 172.21.15.69:0/3609859036 conn(0x7f4a4406de70 msgr2=0x7f4a4406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:47.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.754+0000 7f4a49d3a640 1 -- 172.21.15.69:0/3609859036 shutdown_connections 2024-08-02T17:51:47.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.757+0000 7f4a49d3a640 1 -- 172.21.15.69:0/3609859036 wait complete. 2024-08-02T17:51:47.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.758+0000 7f4a49d3a640 1 Processor -- start 2024-08-02T17:51:47.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.758+0000 7f4a49d3a640 1 -- start start 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.758+0000 7f4a49d3a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44109180 0x7f4a4411a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.758+0000 7f4a49d3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a44109560 0x7f4a441166e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.758+0000 7f4a49d3a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a4411ae70 0x7f4a44116c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.759+0000 7f4a49d3a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441130f0 con 0x7f4a4411ae70 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.759+0000 7f4a49d3a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44112f70 con 0x7f4a44109180 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.759+0000 7f4a49d3a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44113270 con 0x7f4a44109560 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.759+0000 7f4a437fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44109180 0x7f4a4411a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.759+0000 7f4a437fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44109180 0x7f4a4411a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52712/0 (socket says 172.21.15.69:52712) 2024-08-02T17:51:47.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.759+0000 7f4a437fe640 1 -- 172.21.15.69:0/3524777059 learned_addr learned my addr 172.21.15.69:0/3524777059 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:47.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.759+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a4411ae70 0x7f4a44116c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.760+0000 7f4a43fff640 1 -- 172.21.15.69:0/3524777059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a44109560 msgr2=0x7f4a441166e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:47.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.760+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a44109560 0x7f4a441166e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.760+0000 7f4a43fff640 1 -- 172.21.15.69:0/3524777059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44109180 msgr2=0x7f4a4411a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:47.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.760+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44109180 0x7f4a4411a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.760+0000 7f4a43fff640 1 -- 172.21.15.69:0/3524777059 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a441173a0 con 0x7f4a4411ae70 2024-08-02T17:51:47.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.761+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a4411ae70 0x7f4a44116c20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c410 tx=0x7f4a3c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:47.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.761+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3524777059 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c004020 con 0x7f4a4411ae70 2024-08-02T17:51:47.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.761+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3524777059 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3c0041c0 con 0x7f4a4411ae70 2024-08-02T17:51:47.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.761+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3524777059 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c003b10 con 0x7f4a4411ae70 2024-08-02T17:51:47.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.761+0000 7f4a49d3a640 1 -- 172.21.15.69:0/3524777059 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a441cceb0 con 0x7f4a4411ae70 2024-08-02T17:51:47.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.761+0000 7f4a49d3a640 1 -- 172.21.15.69:0/3524777059 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a441cd310 con 0x7f4a4411ae70 2024-08-02T17:51:47.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.765+0000 7f4a4c138740 1 Processor -- start 2024-08-02T17:51:47.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.765+0000 7f4a4c138740 1 -- start start 2024-08-02T17:51:47.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.765+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc2c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.765+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc2cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.765+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc39480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.765+0000 7f4a4c138740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55947cb91a70 con 0x55947cc2c730 2024-08-02T17:51:47.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.765+0000 7f4a4c138740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55947cb29780 con 0x55947cc2da20 2024-08-02T17:51:47.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.765+0000 7f4a4c138740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55947cb8da30 con 0x55947cc37090 2024-08-02T17:51:47.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.767+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc2c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.767+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc2c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52716/0 (socket says 172.21.15.69:52716) 2024-08-02T17:51:47.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.767+0000 7f4a42ffd640 1 -- 172.21.15.69:0/108102966 learned_addr learned my addr 172.21.15.69:0/108102966 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:47.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.767+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/108102966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc39480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.771+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3524777059 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4a3c019020 con 0x7f4a4411ae70 2024-08-02T17:51:47.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.772+0000 7f4a40ff9640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a300a2ec0 0x7f4a300a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.773+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3524777059 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4a3c0ce730 con 0x7f4a4411ae70 2024-08-02T17:51:47.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.773+0000 7f4a42ffd640 1 -- 172.21.15.69:0/108102966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 msgr2=0x55947cc39480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:47.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.773+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/108102966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc39480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.773+0000 7f4a42ffd640 1 -- 172.21.15.69:0/108102966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 msgr2=0x55947cc2cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:47.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.773+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/108102966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc2cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.773+0000 7f4a42ffd640 1 -- 172.21.15.69:0/108102966 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947c903b90 con 0x55947cc2da20 2024-08-02T17:51:47.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.774+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a300a2ec0 0x7f4a300a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.775+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a300a2ec0 0x7f4a300a5380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0cbbd0 tx=0x7f4a3c0035f0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:47.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.775+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/108102966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc2c1f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c004ee0 tx=0x7f4a2c01a570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8608ffc3d97d4e0a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:47.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.776+0000 7f4a49d3a640 1 -- 172.21.15.69:0/108102966 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c046030 con 0x55947cc2da20 2024-08-02T17:51:47.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.776+0000 7f4a49d3a640 1 -- 172.21.15.69:0/108102966 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a2c005560 con 0x55947cc2da20 2024-08-02T17:51:47.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.776+0000 7f4a49d3a640 1 -- 172.21.15.69:0/108102966 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c003970 con 0x55947cc2da20 2024-08-02T17:51:47.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.777+0000 7f4a4c138740 1 -- 172.21.15.69:0/108102966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 msgr2=0x55947cc2c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:47.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.777+0000 7f4a4c138740 1 --2- 172.21.15.69:0/108102966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc2c1f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c004ee0 tx=0x7f4a2c01a570 comp rx=0 tx=0).stop 2024-08-02T17:51:47.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.777+0000 7f4a4c138740 1 -- 172.21.15.69:0/108102966 shutdown_connections 2024-08-02T17:51:47.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.777+0000 7f4a4c138740 1 --2- 172.21.15.69:0/108102966 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc2cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.777+0000 7f4a4c138740 1 --2- 172.21.15.69:0/108102966 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc39480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.777+0000 7f4a4c138740 1 --2- 172.21.15.69:0/108102966 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc2c1f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.777+0000 7f4a4c138740 1 -- 172.21.15.69:0/108102966 >> 172.21.15.69:0/108102966 conn(0x55947cc28980 msgr2=0x55947cc28d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.783+0000 7f4a4c138740 1 -- 172.21.15.69:0/108102966 shutdown_connections 2024-08-02T17:51:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.783+0000 7f4a4c138740 1 -- 172.21.15.69:0/108102966 wait complete. 2024-08-02T17:51:47.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.783+0000 7f4a4c138740 1 -- start start 2024-08-02T17:51:47.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.784+0000 7f4a4c138740 1 Processor -- start 2024-08-02T17:51:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.784+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc34050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.784+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc34590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.784+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc34ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.784+0000 7f4a4c138740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55947cc39e40 con 0x55947cc2c730 2024-08-02T17:51:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.784+0000 7f4a4c138740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55947cb4c990 con 0x55947cc2da20 2024-08-02T17:51:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.784+0000 7f4a4c138740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55947cc39fc0 con 0x55947cc37090 2024-08-02T17:51:47.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.784+0000 7f4a4c138740 10 client.0 _ll_register_callbacks cb 0x55947c8a1090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.786+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc34ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.786+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc34ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:44990/0 (socket says 172.21.15.69:44990) 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.786+0000 7f4a42ffd640 1 -- 172.21.15.69:0/1929762095 learned_addr learned my addr 172.21.15.69:0/1929762095 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.786+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc34050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.787+0000 7f4a437fe640 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc34590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.790+0000 7f4a42ffd640 1 -- 172.21.15.69:0/1929762095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 msgr2=0x55947cc34590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.790+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc34590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.790+0000 7f4a42ffd640 1 -- 172.21.15.69:0/1929762095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 msgr2=0x55947cc34050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:47.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.790+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc34050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.790+0000 7f4a42ffd640 1 -- 172.21.15.69:0/1929762095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947cb4b6c0 con 0x55947cc37090 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.790+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc34ad0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005ab0 tx=0x7f4a2c004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a422c7eb3e79f5d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.792+0000 7f4a22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.792+0000 7f4a22ffd640 1 -- 172.21.15.69:0/1929762095 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c046040 con 0x55947cc37090 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.792+0000 7f4a4c138740 1 -- 172.21.15.69:0/1929762095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2c005560 con 0x55947cc37090 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.792+0000 7f4a22ffd640 1 -- 172.21.15.69:0/1929762095 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4a2c040920 con 0x55947cc37090 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.792+0000 7f4a22ffd640 1 -- 172.21.15.69:0/1929762095 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a2c05a050 con 0x55947cc37090 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.792+0000 7f4a4c138740 10 client.26263 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:47.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.792+0000 7f4a4c138740 1 -- 172.21.15.69:0/1929762095 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55947cc39e40 con 0x55947cc37090 2024-08-02T17:51:47.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.795+0000 7f4a22ffd640 1 -- 172.21.15.69:0/1929762095 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a2c005560 con 0x55947cc37090 2024-08-02T17:51:47.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.795+0000 7f4a22ffd640 1 -- 172.21.15.69:0/1929762095 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a2c04cad0 con 0x55947cc37090 2024-08-02T17:51:47.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.796+0000 7f4a4c138740 10 client.26263 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:47.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.796+0000 7f4a4c138740 10 client.26263 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:51:47.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.796+0000 7f4a4c138740 20 client.26263 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:47.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.796+0000 7f4a4c138740 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55947cc471c0 0x55947cc495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.796+0000 7f4a4c138740 4 client.26263 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:47.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.796+0000 7f4a4c138740 1 -- 172.21.15.69:0/1929762095 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55947cb9f1a0 con 0x55947cc471c0 2024-08-02T17:51:47.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.798+0000 7f4a437fe640 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55947cc471c0 0x55947cc495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.801+0000 7f4a437fe640 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55947cc471c0 0x55947cc495e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f0444f4a68c5a3 server_cookie=6a2f044fc0cda2a4 in_seq=0 out_seq=0 2024-08-02T17:51:47.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.801+0000 7f4a22ffd640 10 client.26263 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:47.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.801+0000 7f4a22ffd640 1 -- 172.21.15.69:0/1929762095 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55947cb9f1a0 con 0x55947cc471c0 2024-08-02T17:51:47.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.802+0000 7f4a22ffd640 10 client.26263 handle_command_reply: tid=0 2024-08-02T17:51:47.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 client.26263 shutdown 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 -- 172.21.15.69:0/1929762095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 msgr2=0x55947cc34ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc34ad0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005ab0 tx=0x7f4a2c004610 comp rx=0 tx=0).stop 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 -- 172.21.15.69:0/1929762095 shutdown_connections 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55947cc471c0 0x55947cc495e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc37090 0x55947cc34ad0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005ab0 tx=0x7f4a2c004610 comp rx=0 tx=0).stop 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc34590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 --2- 172.21.15.69:0/1929762095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc34050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a4c138740 1 -- 172.21.15.69:0/1929762095 >> 172.21.15.69:0/1929762095 conn(0x55947cc28980 msgr2=0x55947cc2e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:47.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.803+0000 7f4a22ffd640 0 client.26263 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:47.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.805+0000 7f4a4c138740 1 -- 172.21.15.69:0/1929762095 shutdown_connections 2024-08-02T17:51:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.805+0000 7f4a4c138740 1 -- 172.21.15.69:0/1929762095 wait complete. 2024-08-02T17:51:47.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.806+0000 7f4a4c138740 20 client.26263 trim_cache size 0 max 16384 2024-08-02T17:51:47.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.836+0000 7f4a4c138740 1 Processor -- start 2024-08-02T17:51:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.837+0000 7f4a4c138740 1 -- start start 2024-08-02T17:51:47.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.837+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc2f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.838+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc34920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.838+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.838+0000 7f4a4c138740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24012950 con 0x55947cc2c730 2024-08-02T17:51:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.838+0000 7f4a4c138740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a24010220 con 0x55947cc2da20 2024-08-02T17:51:47.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.838+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.838+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc2ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45000/0 (socket says 172.21.15.69:45000) 2024-08-02T17:51:47.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.838+0000 7f4a42ffd640 1 -- 172.21.15.69:0/612937694 learned_addr learned my addr 172.21.15.69:0/612937694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:47.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.838+0000 7f4a4c138740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2400e070 con 0x55947cc2e5a0 2024-08-02T17:51:47.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.839+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/612937694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc2f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.839+0000 7f4a42ffd640 1 -- 172.21.15.69:0/612937694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 msgr2=0x55947cc34920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:51:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.839+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/612937694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc34920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.839+0000 7f4a42ffd640 1 -- 172.21.15.69:0/612937694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 msgr2=0x55947cc2f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.839+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/612937694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc2f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.839+0000 7f4a42ffd640 1 -- 172.21.15.69:0/612937694 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55947cc35f90 con 0x55947cc2e5a0 2024-08-02T17:51:47.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.840+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/612937694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc2ea10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c012550 tx=0x7f4a2c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d6711f6ddfe4edd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.840+0000 7f4a397fa640 1 -- 172.21.15.69:0/612937694 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c046040 con 0x55947cc2e5a0 2024-08-02T17:51:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.840+0000 7f4a397fa640 1 -- 172.21.15.69:0/612937694 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a2c004a50 con 0x55947cc2e5a0 2024-08-02T17:51:47.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.840+0000 7f4a397fa640 1 -- 172.21.15.69:0/612937694 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c040980 con 0x55947cc2e5a0 2024-08-02T17:51:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 -- 172.21.15.69:0/612937694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 msgr2=0x55947cc2ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 --2- 172.21.15.69:0/612937694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc2ea10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c012550 tx=0x7f4a2c001e70 comp rx=0 tx=0).stop 2024-08-02T17:51:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 -- 172.21.15.69:0/612937694 shutdown_connections 2024-08-02T17:51:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 --2- 172.21.15.69:0/612937694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc2ea10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 --2- 172.21.15.69:0/612937694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc34920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 --2- 172.21.15.69:0/612937694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc2f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 -- 172.21.15.69:0/612937694 >> 172.21.15.69:0/612937694 conn(0x55947cc28980 msgr2=0x55947cc2de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:47.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 -- 172.21.15.69:0/612937694 shutdown_connections 2024-08-02T17:51:47.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 -- 172.21.15.69:0/612937694 wait complete. 2024-08-02T17:51:47.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.842+0000 7f4a4c138740 1 -- start start 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.843+0000 7f4a4c138740 1 Processor -- start 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.843+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc3f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.843+0000 7f4a4c138740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc3fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.843+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc3f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.843+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc3f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34954/0 (socket says 172.21.15.69:34954) 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.844+0000 7f4a437fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc3fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.844+0000 7f4a437fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc3fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52748/0 (socket says 172.21.15.69:52748) 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.844+0000 7f4a42ffd640 1 -- 172.21.15.69:0/3159192039 learned_addr learned my addr 172.21.15.69:0/3159192039 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.844+0000 7f4a4c138740 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc30cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.844+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2400e6e0 con 0x55947cc2c730 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.844+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2400fb50 con 0x55947cc2da20 2024-08-02T17:51:47.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.844+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2400edb0 con 0x55947cc2e5a0 2024-08-02T17:51:47.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.844+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc30cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.845+0000 7f4a437fe640 1 -- 172.21.15.69:0/3159192039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 msgr2=0x55947cc30cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.845+0000 7f4a437fe640 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc30cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.845+0000 7f4a437fe640 1 -- 172.21.15.69:0/3159192039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 msgr2=0x55947cc3f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.845+0000 7f4a437fe640 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc3f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.845+0000 7f4a437fe640 1 -- 172.21.15.69:0/3159192039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55947cc314e0 con 0x55947cc2da20 2024-08-02T17:51:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.845+0000 7f4a4c138740 10 client.0 _ll_register_callbacks cb 0x7f4a2c0049a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.845+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2c004a50 con 0x55947cc2da20 2024-08-02T17:51:47.848 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.846+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc3f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.847+0000 7f4a437fe640 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc3fc10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55947cc49e60 tx=0x7f4a34007720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2174821c4a6b1a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.847+0000 7f4a22ffd640 10 client.26381 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.847+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3159192039 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34025060 con 0x55947cc2da20 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.847+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3159192039 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4a34012040 con 0x55947cc2da20 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.847+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3159192039 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a34007ea0 con 0x55947cc2da20 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.847+0000 7f4a4c138740 10 client.26381 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.847+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55947ccb1b20 con 0x55947cc2da20 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.848+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3159192039 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a34017620 con 0x55947cc2da20 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.848+0000 7f4a4c138740 10 client.26381 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.848+0000 7f4a4c138740 10 client.26381 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:51:47.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.848+0000 7f4a4c138740 20 client.26381 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:47.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.848+0000 7f4a4c138740 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55947ccb2150 0x55947ccb4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:47.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.848+0000 7f4a4c138740 4 client.26381 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.9, "await_for": 45.0}] 2024-08-02T17:51:47.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.848+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.9, "await_for": 45.0}) -- 0x55947cb994f0 con 0x55947ccb2150 2024-08-02T17:51:47.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.849+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55947ccb2150 0x55947ccb4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:47.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.850+0000 7f4a22ffd640 10 client.26381 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:47.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:47.850+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55947ccb2150 0x55947ccb4540 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba1e9a7b761a248 server_cookie=ff9b6353e6c28a9b in_seq=0 out_seq=0 2024-08-02T17:51:48.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.140+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3159192039 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55947cb994f0 con 0x55947ccb2150 2024-08-02T17:51:48.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.140+0000 7f4a22ffd640 10 client.26381 handle_command_reply: tid=0 2024-08-02T17:51:48.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.140+0000 7f4a4c138740 1 client.26381 shutdown 2024-08-02T17:51:48.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 msgr2=0x55947cc3fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:48.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc3fc10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x55947cc49e60 tx=0x7f4a34007720 comp rx=0 tx=0).stop 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 shutdown_connections 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55947ccb2150 0x55947ccb4540 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55947cc2e5a0 0x55947cc30cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55947cc2da20 0x55947cc3fc10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a22ffd640 0 client.26381 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 --2- 172.21.15.69:0/3159192039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55947cc2c730 0x55947cc3f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 >> 172.21.15.69:0/3159192039 conn(0x55947cc28980 msgr2=0x55947cc42160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 shutdown_connections 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 1 -- 172.21.15.69:0/3159192039 wait complete. 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.141+0000 7f4a4c138740 20 client.26381 trim_cache size 0 max 16384 2024-08-02T17:51:48.144 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 18, 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "f4b08119": { 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 18, 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:51:48.145 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 148.9, 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:51:48.146 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:51:48.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 -- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a300a2ec0 msgr2=0x7f4a300a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:48.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a300a2ec0 0x7f4a300a5380 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0cbbd0 tx=0x7f4a3c0035f0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 -- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a4411ae70 msgr2=0x7f4a44116c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:48.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a4411ae70 0x7f4a44116c20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00c410 tx=0x7f4a3c00c8d0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 -- 172.21.15.69:0/3524777059 shutdown_connections 2024-08-02T17:51:48.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a4411ae70 0x7f4a44116c20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a300a2ec0 0x7f4a300a5380 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a44109560 0x7f4a441166e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 --2- 172.21.15.69:0/3524777059 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44109180 0x7f4a4411a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.142+0000 7f4a3a7fc640 1 -- 172.21.15.69:0/3524777059 >> 172.21.15.69:0/3524777059 conn(0x7f4a4406de70 msgr2=0x7f4a44071fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:48.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.143+0000 7f4a3a7fc640 1 -- 172.21.15.69:0/3524777059 shutdown_connections 2024-08-02T17:51:48.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.143+0000 7f4a3a7fc640 1 -- 172.21.15.69:0/3524777059 wait complete. 2024-08-02T17:51:48.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 -- 172.21.15.69:0/2688836180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 msgr2=0x7fbff010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:48.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2688836180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 0x7fbff010e970 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009cd0 tx=0x7fbfe802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 -- 172.21.15.69:0/2688836180 shutdown_connections 2024-08-02T17:51:48.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2688836180 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 0x7fbff010e970 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2688836180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff0103410 0x7fbff01078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2688836180 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbff0102a60 0x7fbff0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 -- 172.21.15.69:0/2688836180 >> 172.21.15.69:0/2688836180 conn(0x7fbff00fc500 msgr2=0x7fbff00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 -- 172.21.15.69:0/2688836180 shutdown_connections 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 -- 172.21.15.69:0/2688836180 wait complete. 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.313+0000 7fbff60a4640 1 Processor -- start 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbff60a4640 1 -- start start 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbff60a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff0102a60 0x7fbff01006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbff60a4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbff0103410 0x7fbff0100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbff60a4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 0x7fbff0101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbff60a4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0111dd0 con 0x7fbff0107fd0 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbff60a4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0111c50 con 0x7fbff0103410 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbff60a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0111f50 con 0x7fbff0102a60 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 0x7fbff0101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 0x7fbff0101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34970/0 (socket says 172.21.15.69:34970) 2024-08-02T17:51:48.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeffff640 1 -- 172.21.15.69:0/2257646320 learned_addr learned my addr 172.21.15.69:0/2257646320 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfef7fe640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff0102a60 0x7fbff01006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeeffd640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbff0103410 0x7fbff0100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeffff640 1 -- 172.21.15.69:0/2257646320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff0102a60 msgr2=0x7fbff01006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeffff640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff0102a60 0x7fbff01006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeffff640 1 -- 172.21.15.69:0/2257646320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbff0103410 msgr2=0x7fbff0100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeffff640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbff0103410 0x7fbff0100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.314+0000 7fbfeffff640 1 -- 172.21.15.69:0/2257646320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff01a66b0 con 0x7fbff0107fd0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.315+0000 7fbfeffff640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 0x7fbff0101140 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005cc0 tx=0x7fbfe8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.315+0000 7fbfecff9640 1 -- 172.21.15.69:0/2257646320 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8004040 con 0x7fbff0107fd0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.315+0000 7fbfecff9640 1 -- 172.21.15.69:0/2257646320 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe80046c0 con 0x7fbff0107fd0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.315+0000 7fbfecff9640 1 -- 172.21.15.69:0/2257646320 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8030e70 con 0x7fbff0107fd0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.315+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff01a6940 con 0x7fbff0107fd0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.315+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff01a6e80 con 0x7fbff0107fd0 2024-08-02T17:51:48.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.316+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff0102e40 con 0x7fbff0107fd0 2024-08-02T17:51:48.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.317+0000 7fbfecff9640 1 -- 172.21.15.69:0/2257646320 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbfe80305a0 con 0x7fbff0107fd0 2024-08-02T17:51:48.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.317+0000 7fbfecff9640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:48.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.318+0000 7fbfecff9640 1 -- 172.21.15.69:0/2257646320 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbfe802e030 con 0x7fbff0107fd0 2024-08-02T17:51:48.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.318+0000 7fbfef7fe640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:48.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.318+0000 7fbfef7fe640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000bbe0 tx=0x7fbfe000b390 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:48.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.322+0000 7fbfecff9640 1 -- 172.21.15.69:0/2257646320 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fbfe80bd0f0 con 0x7fbff0107fd0 2024-08-02T17:51:48.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:48 smithi069 ceph-mon[32746]: pgmap v164: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.4 MiB/s wr, 959 op/s 2024-08-02T17:51:48.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:48 smithi083 ceph-mon[37031]: pgmap v164: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.4 MiB/s wr, 959 op/s 2024-08-02T17:51:48.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.535+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbff0109fe0 con 0x7fbff0107fd0 2024-08-02T17:51:48.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.536+0000 7fbfecff9640 1 -- 172.21.15.69:0/2257646320 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fbfe80098b0 con 0x7fbff0107fd0 2024-08-02T17:51:48.538 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:51:48.538 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:51:48.538 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbfd80a2cf0 msgr2=0x7fbfd80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000bbe0 tx=0x7fbfe000b390 comp rx=0 tx=0).stop 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 msgr2=0x7fbff0101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 0x7fbff0101140 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8005cc0 tx=0x7fbfe8003970 comp rx=0 tx=0).stop 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 shutdown_connections 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbff0107fd0 0x7fbff0101140 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbfd80a2cf0 0x7fbfd80a51b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbff0103410 0x7fbff0100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 --2- 172.21.15.69:0/2257646320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbff0102a60 0x7fbff01006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 >> 172.21.15.69:0/2257646320 conn(0x7fbff00fc500 msgr2=0x7fbff01060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 shutdown_connections 2024-08-02T17:51:48.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:48.538+0000 7fbff60a4640 1 -- 172.21.15.69:0/2257646320 wait complete. 2024-08-02T17:51:48.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:48 smithi077 ceph-mon[37108]: pgmap v164: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 9.4 MiB/s wr, 959 op/s 2024-08-02T17:51:49.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:49 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2257646320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:49.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:49 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2257646320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:49.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:49 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2257646320' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:51:49.955 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 280.0 s, 871.9 tps, lat 69.022 ms stddev 73.545 2024-08-02T17:51:50.390 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:51:50.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:50 smithi083 ceph-mon[37031]: pgmap v165: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.3 MiB/s wr, 740 op/s 2024-08-02T17:51:50.526 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:50 smithi069 ceph-mon[32746]: pgmap v165: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.3 MiB/s wr, 740 op/s 2024-08-02T17:51:50.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:50 smithi077 ceph-mon[37108]: pgmap v165: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 7.3 MiB/s wr, 740 op/s 2024-08-02T17:51:50.910 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f4b08119', quiesce took 14.0 seconds. Will release after: 14.899999999999999 2024-08-02T17:51:51.075 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:51.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.576+0000 7f90559d4640 1 -- 172.21.15.69:0/3389386868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050101150 msgr2=0x7f905010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.576+0000 7f90559d4640 1 --2- 172.21.15.69:0/3389386868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050101150 0x7f905010d680 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9048009930 tx=0x7f904802a420 comp rx=0 tx=0).stop 2024-08-02T17:51:51.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.577+0000 7f90559d4640 1 -- 172.21.15.69:0/3389386868 shutdown_connections 2024-08-02T17:51:51.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.577+0000 7f90559d4640 1 --2- 172.21.15.69:0/3389386868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9050101150 0x7f905010d680 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.577+0000 7f90559d4640 1 --2- 172.21.15.69:0/3389386868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90501007b0 0x7f9050100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.577+0000 7f90559d4640 1 --2- 172.21.15.69:0/3389386868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90501067b0 0x7f9050106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.577+0000 7f90559d4640 1 -- 172.21.15.69:0/3389386868 >> 172.21.15.69:0/3389386868 conn(0x7f90500fc4d0 msgr2=0x7f90500fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.577+0000 7f90559d4640 1 -- 172.21.15.69:0/3389386868 shutdown_connections 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.577+0000 7f90559d4640 1 -- 172.21.15.69:0/3389386868 wait complete. 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.577+0000 7f90559d4640 1 Processor -- start 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f90559d4640 1 -- start start 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f90559d4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90501007b0 0x7f9050198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f90559d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9050101150 0x7f9050199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f90559d4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90501067b0 0x7f90501a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f90559d4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9050110bb0 con 0x7f90501007b0 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f90559d4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9050110a30 con 0x7f90501067b0 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f90559d4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9050110d30 con 0x7f9050101150 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90501007b0 0x7f9050198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90501007b0 0x7f9050198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34994/0 (socket says 172.21.15.69:34994) 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 -- 172.21.15.69:0/571972178 learned_addr learned my addr 172.21.15.69:0/571972178 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904e7fc640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9050101150 0x7f9050199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904f7fe640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90501067b0 0x7f90501a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 -- 172.21.15.69:0/571972178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9050101150 msgr2=0x7f9050199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9050101150 0x7f9050199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 -- 172.21.15.69:0/571972178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90501067b0 msgr2=0x7f90501a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90501067b0 0x7f90501a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 -- 172.21.15.69:0/571972178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90501a44c0 con 0x7f90501007b0 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.578+0000 7f904effd640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90501007b0 0x7f9050198f00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f904000cce0 tx=0x7f904000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.579+0000 7f90549d2640 1 -- 172.21.15.69:0/571972178 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904000f650 con 0x7f90501007b0 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.579+0000 7f90549d2640 1 -- 172.21.15.69:0/571972178 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f904000fcd0 con 0x7f90501007b0 2024-08-02T17:51:51.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.579+0000 7f90559d4640 1 -- 172.21.15.69:0/571972178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90501a4690 con 0x7f90501007b0 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.579+0000 7f90549d2640 1 -- 172.21.15.69:0/571972178 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040018c70 con 0x7f90501007b0 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.579+0000 7f90559d4640 1 -- 172.21.15.69:0/571972178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90501a4bd0 con 0x7f90501007b0 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.581+0000 7f90549d2640 1 -- 172.21.15.69:0/571972178 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f904000f7f0 con 0x7f90501007b0 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.581+0000 7f9057dd2740 1 Processor -- start 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f9057dd2740 1 -- start start 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c446aff0 0x5646c446b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c4475c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c44761d0 0x5646c44785c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f9057dd2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5646c43d0a40 con 0x5646c446b910 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f9057dd2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5646c43689d0 con 0x5646c446aff0 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f9057dd2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646c43cca00 con 0x5646c44761d0 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c4475c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c4475c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35000/0 (socket says 172.21.15.69:35000) 2024-08-02T17:51:51.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904effd640 1 -- 172.21.15.69:0/1714570540 learned_addr learned my addr 172.21.15.69:0/1714570540 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904e7fc640 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c44761d0 0x5646c44785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904f7fe640 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c446aff0 0x5646c446b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904effd640 1 -- 172.21.15.69:0/1714570540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c44761d0 msgr2=0x5646c44785c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904effd640 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c44761d0 0x5646c44785c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904effd640 1 -- 172.21.15.69:0/1714570540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c446aff0 msgr2=0x5646c446b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904effd640 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c446aff0 0x5646c446b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904effd640 1 -- 172.21.15.69:0/1714570540 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c436f650 con 0x5646c446b910 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f90549d2640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f903c0a2d70 0x7f903c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.582+0000 7f904e7fc640 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c44761d0 0x5646c44785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.583+0000 7f904f7fe640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f903c0a2d70 0x7f903c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.583+0000 7f904effd640 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c4475c90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f90400cce00 tx=0x7f90400d3dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdf25c5e8d89fdf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.583+0000 7f90549d2640 1 -- 172.21.15.69:0/571972178 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f90400cfb90 con 0x7f90501007b0 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.583+0000 7f9045ffb640 1 -- 172.21.15.69:0/1714570540 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90400d4040 con 0x5646c446b910 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.583+0000 7f9045ffb640 1 -- 172.21.15.69:0/1714570540 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f904009c390 con 0x5646c446b910 2024-08-02T17:51:51.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.583+0000 7f9045ffb640 1 -- 172.21.15.69:0/1714570540 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904009ba60 con 0x5646c446b910 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 -- 172.21.15.69:0/1714570540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 msgr2=0x5646c4475c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f904f7fe640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f903c0a2d70 0x7f903c0a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9048006010 tx=0x7f90480095a0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c4475c90 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f90400cce00 tx=0x7f90400d3dc0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 -- 172.21.15.69:0/1714570540 shutdown_connections 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c4475c90 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c44761d0 0x5646c44785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 --2- 172.21.15.69:0/1714570540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c446aff0 0x5646c446b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 -- 172.21.15.69:0/1714570540 >> 172.21.15.69:0/1714570540 conn(0x5646c4467580 msgr2=0x5646c4467960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 -- 172.21.15.69:0/1714570540 shutdown_connections 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 -- 172.21.15.69:0/1714570540 wait complete. 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 -- start start 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 Processor -- start 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c44730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.584+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c44735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c4473b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f9057dd2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5646c4478da0 con 0x5646c446aff0 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f9057dd2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5646c438bb20 con 0x5646c44761d0 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f9057dd2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646c4478f20 con 0x5646c446b910 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f9057dd2740 10 client.0 _ll_register_callbacks cb 0x5646c4389bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c44730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c44730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35012/0 (socket says 172.21.15.69:35012) 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 -- 172.21.15.69:0/3702986798 learned_addr learned my addr 172.21.15.69:0/3702986798 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904f7fe640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c4473b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904e7fc640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c44735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 -- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 msgr2=0x5646c44735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c44735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 -- 172.21.15.69:0/3702986798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 msgr2=0x5646c4473b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c4473b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 -- 172.21.15.69:0/3702986798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c41423c0 con 0x5646c446aff0 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904e7fc640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c44735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:51.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904f7fe640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c4473b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f904effd640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c44730b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f90400d8650 tx=0x7f90400cce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98222d79f728fadc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f902effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f902effd640 1 -- 172.21.15.69:0/3702986798 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904009b5f0 con 0x5646c446aff0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f902effd640 1 -- 172.21.15.69:0/3702986798 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f90400d3840 con 0x5646c446aff0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f9057dd2740 1 -- 172.21.15.69:0/3702986798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f904009c390 con 0x5646c446aff0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f902effd640 1 -- 172.21.15.69:0/3702986798 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f904009b7d0 con 0x5646c446aff0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f9057dd2740 10 client.16803 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.585+0000 7f9057dd2740 1 -- 172.21.15.69:0/3702986798 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646c44849a0 con 0x5646c446aff0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f902effd640 1 -- 172.21.15.69:0/3702986798 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f904009c390 con 0x5646c446aff0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f902effd640 1 -- 172.21.15.69:0/3702986798 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f90400da040 con 0x5646c446aff0 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f9057dd2740 10 client.16803 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f9057dd2740 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f9057dd2740 20 client.16803 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f9057dd2740 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5646c4485cf0 0x5646c44880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f9057dd2740 4 client.16803 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:51.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f9057dd2740 1 -- 172.21.15.69:0/3702986798 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c43ddfb0 con 0x5646c4485cf0 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f904e7fc640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5646c4485cf0 0x5646c44880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f904e7fc640 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5646c4485cf0 0x5646c44880e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f514653047d2b8b server_cookie=dcb853e120381694 in_seq=0 out_seq=0 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.586+0000 7f902effd640 10 client.16803 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f902effd640 1 -- 172.21.15.69:0/3702986798 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5646c43ddfb0 con 0x5646c4485cf0 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f902effd640 10 client.16803 handle_command_reply: tid=0 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 client.16803 shutdown 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 -- 172.21.15.69:0/3702986798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 msgr2=0x5646c44730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c44730b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f90400d8650 tx=0x7f90400cce30 comp rx=0 tx=0).stop 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 -- 172.21.15.69:0/3702986798 shutdown_connections 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5646c4485cf0 0x5646c44880e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c4473b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c44735f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 --2- 172.21.15.69:0/3702986798 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c44730b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 -- 172.21.15.69:0/3702986798 >> 172.21.15.69:0/3702986798 conn(0x5646c4467580 msgr2=0x5646c446d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f902effd640 0 client.16803 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:51:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 -- 172.21.15.69:0/3702986798 shutdown_connections 2024-08-02T17:51:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.587+0000 7f9057dd2740 1 -- 172.21.15.69:0/3702986798 wait complete. 2024-08-02T17:51:51.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.588+0000 7f9057dd2740 20 client.16803 trim_cache size 0 max 16384 2024-08-02T17:51:51.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.600+0000 7f9057dd2740 1 Processor -- start 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.600+0000 7f9057dd2740 1 -- start start 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446aff0 0x5646c448d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c448dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c446ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f9057dd2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f90501b1630 con 0x5646c446b910 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f9057dd2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90501aef00 con 0x5646c44761d0 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f9057dd2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90501acd50 con 0x5646c446aff0 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c448dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c448dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35016/0 (socket says 172.21.15.69:35016) 2024-08-02T17:51:51.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 -- 172.21.15.69:0/590742438 learned_addr learned my addr 172.21.15.69:0/590742438 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904effd640 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446aff0 0x5646c448d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904f7fe640 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c446ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 -- 172.21.15.69:0/590742438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446aff0 msgr2=0x5646c448d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446aff0 0x5646c448d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 -- 172.21.15.69:0/590742438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 msgr2=0x5646c446ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c446ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 -- 172.21.15.69:0/590742438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c4482000 con 0x5646c446b910 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f904e7fc640 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c448dc20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f905006b010 tx=0x7f903800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8cd10b7bad775e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f90457fa640 1 -- 172.21.15.69:0/590742438 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038003d60 con 0x5646c446b910 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f90457fa640 1 -- 172.21.15.69:0/590742438 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903801b810 con 0x5646c446b910 2024-08-02T17:51:51.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.601+0000 7f90457fa640 1 -- 172.21.15.69:0/590742438 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038012810 con 0x5646c446b910 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 -- 172.21.15.69:0/590742438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 msgr2=0x5646c448dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c448dc20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f905006b010 tx=0x7f903800b390 comp rx=0 tx=0).stop 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 -- 172.21.15.69:0/590742438 shutdown_connections 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446b910 0x5646c448dc20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c446ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 --2- 172.21.15.69:0/590742438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446aff0 0x5646c448d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 -- 172.21.15.69:0/590742438 >> 172.21.15.69:0/590742438 conn(0x5646c4467580 msgr2=0x5646c4471980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 -- 172.21.15.69:0/590742438 shutdown_connections 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 -- 172.21.15.69:0/590742438 wait complete. 2024-08-02T17:51:51.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.602+0000 7f9057dd2740 1 -- start start 2024-08-02T17:51:51.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f9057dd2740 1 Processor -- start 2024-08-02T17:51:51.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c447e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c447e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f9057dd2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c446f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f9057dd2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f90501ad3c0 con 0x5646c446aff0 2024-08-02T17:51:51.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f9057dd2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f90501ae830 con 0x5646c44761d0 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f9057dd2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90501ada90 con 0x5646c446b910 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c447e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c447e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35018/0 (socket says 172.21.15.69:35018) 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f9057dd2740 10 client.0 _ll_register_callbacks cb 0x7f90400d3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 -- 172.21.15.69:0/4005464191 learned_addr learned my addr 172.21.15.69:0/4005464191 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904f7fe640 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c446f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904e7fc640 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c447e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 -- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 msgr2=0x5646c447e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c447e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 -- 172.21.15.69:0/4005464191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 msgr2=0x5646c446f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c446f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 -- 172.21.15.69:0/4005464191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c446ff60 con 0x5646c446aff0 2024-08-02T17:51:51.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904e7fc640 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c447e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f904effd640 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c447e350 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f90400980f0 tx=0x7f9040097f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b108a0d49745c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.603+0000 7f902effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f902effd640 1 -- 172.21.15.69:0/4005464191 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f904009b5f0 con 0x5646c446aff0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f902effd640 1 -- 172.21.15.69:0/4005464191 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f90400a4680 con 0x5646c446aff0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 1 -- 172.21.15.69:0/4005464191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f903801b810 con 0x5646c446aff0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f902effd640 1 -- 172.21.15.69:0/4005464191 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f90400d3640 con 0x5646c446aff0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 10 client.16809 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 1 -- 172.21.15.69:0/4005464191 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646c44f4440 con 0x5646c446aff0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f902effd640 1 -- 172.21.15.69:0/4005464191 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903801b810 con 0x5646c446aff0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f902effd640 1 -- 172.21.15.69:0/4005464191 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f90400da070 con 0x5646c446aff0 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 10 client.16809 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 10 client.16809 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 20 client.16809 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:51.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5646c44f4870 0x5646c44f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:51.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 4 client.16809 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:51:51.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f9057dd2740 1 -- 172.21.15.69:0/4005464191 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646c43d84c0 con 0x5646c44f4870 2024-08-02T17:51:51.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.604+0000 7f904e7fc640 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5646c44f4870 0x5646c44f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:51.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f902effd640 10 client.16809 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:51:51.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f904e7fc640 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5646c44f4870 0x5646c44f6c80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2a5ae95cc3eb255 server_cookie=184a881f2933d22d in_seq=0 out_seq=0 2024-08-02T17:51:51.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f902effd640 1 -- 172.21.15.69:0/4005464191 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646c43d84c0 con 0x5646c44f4870 2024-08-02T17:51:51.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f902effd640 10 client.16809 handle_command_reply: tid=0 2024-08-02T17:51:51.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 client.16809 shutdown 2024-08-02T17:51:51.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 -- 172.21.15.69:0/4005464191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 msgr2=0x5646c447e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c447e350 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f90400980f0 tx=0x7f9040097f30 comp rx=0 tx=0).stop 2024-08-02T17:51:51.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 -- 172.21.15.69:0/4005464191 shutdown_connections 2024-08-02T17:51:51.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5646c44f4870 0x5646c44f6c80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646c44761d0 0x5646c446f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646c446b910 0x5646c447e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 --2- 172.21.15.69:0/4005464191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646c446aff0 0x5646c447e350 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f902effd640 0 client.16809 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:51:51.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 -- 172.21.15.69:0/4005464191 >> 172.21.15.69:0/4005464191 conn(0x5646c4467580 msgr2=0x5646c4480ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:51.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 -- 172.21.15.69:0/4005464191 shutdown_connections 2024-08-02T17:51:51.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 1 -- 172.21.15.69:0/4005464191 wait complete. 2024-08-02T17:51:51.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.605+0000 7f9057dd2740 20 client.16809 trim_cache size 0 max 16384 2024-08-02T17:51:51.616 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 -- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f903c0a2d70 msgr2=0x7f903c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f903c0a2d70 0x7f903c0a5230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9048006010 tx=0x7f90480095a0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 -- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90501007b0 msgr2=0x7f9050198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90501007b0 0x7f9050198f00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f904000cce0 tx=0x7f904000a620 comp rx=0 tx=0).stop 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 -- 172.21.15.69:0/571972178 shutdown_connections 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f903c0a2d70 0x7f903c0a5230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f90501067b0 0x7f90501a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9050101150 0x7f9050199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 --2- 172.21.15.69:0/571972178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f90501007b0 0x7f9050198f00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 -- 172.21.15.69:0/571972178 >> 172.21.15.69:0/571972178 conn(0x7f90500fc4d0 msgr2=0x7f90500fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 -- 172.21.15.69:0/571972178 shutdown_connections 2024-08-02T17:51:51.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:51.606+0000 7f90559d4640 1 -- 172.21.15.69:0/571972178 wait complete. 2024-08-02T17:51:52.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:52 smithi083 ceph-mon[37031]: pgmap v166: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.1 MiB/s wr, 723 op/s 2024-08-02T17:51:52.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:52 smithi077 ceph-mon[37108]: pgmap v166: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.1 MiB/s wr, 723 op/s 2024-08-02T17:51:52.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:52 smithi069 ceph-mon[32746]: pgmap v166: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 7.1 MiB/s wr, 723 op/s 2024-08-02T17:51:52.710 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:51:52.967 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:53.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 -- 172.21.15.69:0/3872648830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc1078b0 msgr2=0x7ff7bc107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/3872648830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc1078b0 0x7ff7bc107c90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009930 tx=0x7ff7ac02a420 comp rx=0 tx=0).stop 2024-08-02T17:51:53.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 -- 172.21.15.69:0/3872648830 shutdown_connections 2024-08-02T17:51:53.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/3872648830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc102250 0x7ff7bc10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/3872648830 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7bc1018b0 0x7ff7bc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/3872648830 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc1078b0 0x7ff7bc107c90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 -- 172.21.15.69:0/3872648830 >> 172.21.15.69:0/3872648830 conn(0x7ff7bc0fd5f0 msgr2=0x7ff7bc0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 -- 172.21.15.69:0/3872648830 shutdown_connections 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.395+0000 7ff7c16e2640 1 -- 172.21.15.69:0/3872648830 wait complete. 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7c16e2640 1 Processor -- start 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7c16e2640 1 -- start start 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7c16e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc1018b0 0x7ff7bc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7c16e2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102250 0x7ff7bc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7c16e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7bc1078b0 0x7ff7bc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7c16e2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc111cb0 con 0x7ff7bc102250 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7c16e2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc111b30 con 0x7ff7bc1078b0 2024-08-02T17:51:53.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7c16e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc111e30 con 0x7ff7bc1018b0 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7b27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102250 0x7ff7bc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7b27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102250 0x7ff7bc1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35026/0 (socket says 172.21.15.69:35026) 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.396+0000 7ff7b27fc640 1 -- 172.21.15.69:0/1301994915 learned_addr learned my addr 172.21.15.69:0/1301994915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7baffd640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc1018b0 0x7ff7bc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7bc1078b0 0x7ff7bc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b27fc640 1 -- 172.21.15.69:0/1301994915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc1018b0 msgr2=0x7ff7bc198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc1018b0 0x7ff7bc198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b27fc640 1 -- 172.21.15.69:0/1301994915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7bc1078b0 msgr2=0x7ff7bc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7bc1078b0 0x7ff7bc19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b27fc640 1 -- 172.21.15.69:0/1301994915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc19d140 con 0x7ff7bc102250 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7bc1078b0 0x7ff7bc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102250 0x7ff7bc1986d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400d950 tx=0x7ff7a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b8ff9640 1 -- 172.21.15.69:0/1301994915 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4017070 con 0x7ff7bc102250 2024-08-02T17:51:53.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b8ff9640 1 -- 172.21.15.69:0/1301994915 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7a4009c30 con 0x7ff7bc102250 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7c16e2640 1 -- 172.21.15.69:0/1301994915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc19d430 con 0x7ff7bc102250 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7b8ff9640 1 -- 172.21.15.69:0/1301994915 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a400f6b0 con 0x7ff7bc102250 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.397+0000 7ff7c16e2640 1 -- 172.21.15.69:0/1301994915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc1a4d10 con 0x7ff7bc102250 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.399+0000 7ff7b8ff9640 1 -- 172.21.15.69:0/1301994915 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff7a40109d0 con 0x7ff7bc102250 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7c3ae0740 1 Processor -- start 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7c3ae0740 1 -- start start 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798787290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x5567987960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5567986eca40 con 0x556798786eb0 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5567986849d0 con 0x5567987965e0 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7b8ff9640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567986e8a00 con 0x5567987877d0 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7bb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798787290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7bb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798787290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35036/0 (socket says 172.21.15.69:35036) 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7bb7fe640 1 -- 172.21.15.69:0/4165691556 learned_addr learned my addr 172.21.15.69:0/4165691556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7b8ff9640 1 -- 172.21.15.69:0/1301994915 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff7a40cf4a0 con 0x7ff7bc102250 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7baffd640 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x5567987960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.400+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7bb7fe640 1 -- 172.21.15.69:0/4165691556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 msgr2=0x5567987960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x5567987960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7bb7fe640 1 -- 172.21.15.69:0/4165691556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 msgr2=0x5567987989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7bb7fe640 1 -- 172.21.15.69:0/4165691556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55679868b650 con 0x556798786eb0 2024-08-02T17:51:53.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798787290 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4014ea0 tx=0x7ff7b4014f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1655884fb97aafb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4011c40 tx=0x7ff7b401c000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7b17fa640 1 -- 172.21.15.69:0/4165691556 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4022070 con 0x556798786eb0 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7b17fa640 1 -- 172.21.15.69:0/4165691556 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b4003e50 con 0x556798786eb0 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.401+0000 7ff7b17fa640 1 -- 172.21.15.69:0/4165691556 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4004290 con 0x556798786eb0 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4165691556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 msgr2=0x556798787290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798787290 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4014ea0 tx=0x7ff7b4014f90 comp rx=0 tx=0).stop 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4165691556 shutdown_connections 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x5567987960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/4165691556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798787290 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4165691556 >> 172.21.15.69:0/4165691556 conn(0x556798783580 msgr2=0x556798783960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4165691556 shutdown_connections 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4165691556 wait complete. 2024-08-02T17:51:53.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 -- start start 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.402+0000 7ff7c3ae0740 1 Processor -- start 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556798786eb0 0x556798793550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798793a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567987965e0 0x556798793fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556798799150 con 0x5567987965e0 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5567986a7b20 con 0x556798786eb0 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5567987992d0 con 0x5567987877d0 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7c3ae0740 10 client.0 _ll_register_callbacks cb 0x5567986a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567987965e0 0x556798793fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567987965e0 0x556798793fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35042/0 (socket says 172.21.15.69:35042) 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 -- 172.21.15.69:0/149665306 learned_addr learned my addr 172.21.15.69:0/149665306 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7baffd640 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556798786eb0 0x556798793550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798793a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 -- 172.21.15.69:0/149665306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 msgr2=0x556798793a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798793a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 -- 172.21.15.69:0/149665306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556798786eb0 msgr2=0x556798793550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556798786eb0 0x556798793550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 -- 172.21.15.69:0/149665306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55679845e3c0 con 0x5567987965e0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798793a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567987965e0 0x556798793fd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4014230 tx=0x7ff7b4014aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadf3e3f8167b97 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.403+0000 7ff7a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7a27fc640 1 -- 172.21.15.69:0/149665306 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4022070 con 0x5567987965e0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7a27fc640 1 -- 172.21.15.69:0/149665306 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff7b40050d0 con 0x5567987965e0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/149665306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7b4003e50 con 0x5567987965e0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7a27fc640 1 -- 172.21.15.69:0/149665306 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7b4034050 con 0x5567987965e0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 10 client.16821 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/149665306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5567987a4de0 con 0x5567987965e0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7a27fc640 1 -- 172.21.15.69:0/149665306 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7b4003e50 con 0x5567987965e0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7a27fc640 1 -- 172.21.15.69:0/149665306 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff7b402f030 con 0x5567987965e0 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 10 client.16821 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:53.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:51:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 20 client.16821 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5567987a6240 0x5567987a8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 4 client.16821 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/149665306 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567986f9fb0 con 0x5567987a6240 2024-08-02T17:51:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.404+0000 7ff7baffd640 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5567987a6240 0x5567987a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.405+0000 7ff7a27fc640 10 client.16821 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:51:53.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.405+0000 7ff7baffd640 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5567987a6240 0x5567987a8630 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b9b5e989c531834 server_cookie=6afb2c9ef471f8f1 in_seq=0 out_seq=0 2024-08-02T17:51:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.405+0000 7ff7a27fc640 1 -- 172.21.15.69:0/149665306 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5567986f9fb0 con 0x5567987a6240 2024-08-02T17:51:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.405+0000 7ff7a27fc640 10 client.16821 handle_command_reply: tid=0 2024-08-02T17:51:53.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 client.16821 shutdown 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/149665306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567987965e0 msgr2=0x556798793fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567987965e0 0x556798793fd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4014230 tx=0x7ff7b4014aa0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/149665306 shutdown_connections 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5567987a6240 0x5567987a8630 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5567987965e0 0x556798793fd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798793a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7a27fc640 0 client.16821 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/149665306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556798786eb0 0x556798793550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/149665306 >> 172.21.15.69:0/149665306 conn(0x556798783580 msgr2=0x5567987892e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/149665306 shutdown_connections 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/149665306 wait complete. 2024-08-02T17:51:53.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.406+0000 7ff7c3ae0740 20 client.16821 trim_cache size 0 max 16384 2024-08-02T17:51:53.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7c3ae0740 1 Processor -- start 2024-08-02T17:51:53.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7c3ae0740 1 -- start start 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x5567987953b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x5567987adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1af430 con 0x556798786eb0 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1acd00 con 0x5567987965e0 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1aab50 con 0x5567987877d0 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x5567987953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x5567987953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35054/0 (socket says 172.21.15.69:35054) 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7baffd640 1 -- 172.21.15.69:0/4163281570 learned_addr learned my addr 172.21.15.69:0/4163281570 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:53.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7baffd640 1 -- 172.21.15.69:0/4163281570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 msgr2=0x5567987adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7baffd640 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x5567987adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7baffd640 1 -- 172.21.15.69:0/4163281570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 msgr2=0x5567987ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7baffd640 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7baffd640 1 -- 172.21.15.69:0/4163281570 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556798784210 con 0x556798786eb0 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.419+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.420+0000 7ff7baffd640 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x5567987953b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac02a940 tx=0x7ff7ac0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5816353fd3ac8574 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.420+0000 7ff7b0ff9640 1 -- 172.21.15.69:0/4163281570 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac035040 con 0x556798786eb0 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.420+0000 7ff7b0ff9640 1 -- 172.21.15.69:0/4163281570 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac005500 con 0x556798786eb0 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.420+0000 7ff7b0ff9640 1 -- 172.21.15.69:0/4163281570 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac004bc0 con 0x556798786eb0 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.420+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4163281570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 msgr2=0x5567987953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.420+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x5567987953b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac02a940 tx=0x7ff7ac0319a0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4163281570 shutdown_connections 2024-08-02T17:51:53.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x5567987ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x5567987adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/4163281570 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x5567987953b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4163281570 >> 172.21.15.69:0/4163281570 conn(0x556798783580 msgr2=0x556798794ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:53.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4163281570 shutdown_connections 2024-08-02T17:51:53.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/4163281570 wait complete. 2024-08-02T17:51:53.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 -- start start 2024-08-02T17:51:53.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 Processor -- start 2024-08-02T17:51:53.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798799aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798799fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x55679880ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1ab1c0 con 0x556798786eb0 2024-08-02T17:51:53.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1ac630 con 0x5567987965e0 2024-08-02T17:51:53.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1ab890 con 0x5567987877d0 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7c3ae0740 10 client.0 _ll_register_callbacks cb 0x7ff7ac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798799aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798799aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35064/0 (socket says 172.21.15.69:35064) 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7baffd640 1 -- 172.21.15.69:0/3815724620 learned_addr learned my addr 172.21.15.69:0/3815724620 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.421+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798799fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x55679880ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7baffd640 1 -- 172.21.15.69:0/3815724620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 msgr2=0x556798799fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7baffd640 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798799fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7baffd640 1 -- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 msgr2=0x55679880ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7baffd640 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x55679880ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7baffd640 1 -- 172.21.15.69:0/3815724620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55679880f260 con 0x556798786eb0 2024-08-02T17:51:53.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7bb7fe640 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x55679880ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7baffd640 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798799aa0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009a50 tx=0x7ff7ac031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58962d067474774 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7a27fc640 1 -- 172.21.15.69:0/3815724620 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac035040 con 0x556798786eb0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7a27fc640 1 -- 172.21.15.69:0/3815724620 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff7ac004040 con 0x556798786eb0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/3815724620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7ac005500 con 0x556798786eb0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7a27fc640 1 -- 172.21.15.69:0/3815724620 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff7ac060050 con 0x556798786eb0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7c3ae0740 10 client.16827 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/3815724620 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556798810490 con 0x556798786eb0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7a27fc640 1 -- 172.21.15.69:0/3815724620 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac005500 con 0x556798786eb0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.422+0000 7ff7a27fc640 1 -- 172.21.15.69:0/3815724620 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff7ac003680 con 0x556798786eb0 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7c3ae0740 10 client.16827 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7c3ae0740 10 client.16827 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7c3ae0740 20 client.16827 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:53.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x556798810ba0 0x556798812fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7c3ae0740 4 client.16827 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/3815724620 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567986f44c0 con 0x556798810ba0 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x556798810ba0 0x556798812fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7a27fc640 10 client.16827 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7b27fc640 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x556798810ba0 0x556798812fc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e9a7084a03958f6 server_cookie=792dbbec6e641c68 in_seq=0 out_seq=0 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.423+0000 7ff7a27fc640 1 -- 172.21.15.69:0/3815724620 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567986f44c0 con 0x556798810ba0 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7a27fc640 10 client.16827 handle_command_reply: tid=0 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 client.16827 shutdown 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/3815724620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 msgr2=0x556798799aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798799aa0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009a50 tx=0x7ff7ac031a90 comp rx=0 tx=0).stop 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/3815724620 shutdown_connections 2024-08-02T17:51:53.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x556798810ba0 0x556798812fc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5567987965e0 0x55679880ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5567987877d0 0x556798799fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 --2- 172.21.15.69:0/3815724620 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556798786eb0 0x556798799aa0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/3815724620 >> 172.21.15.69:0/3815724620 conn(0x556798783580 msgr2=0x556798789170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7a27fc640 0 client.16827 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/3815724620 shutdown_connections 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 1 -- 172.21.15.69:0/3815724620 wait complete. 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c3ae0740 20 client.16827 trim_cache size 0 max 16384 2024-08-02T17:51:53.433 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:51:53.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c16e2640 1 -- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7a80a2cf0 msgr2=0x7ff7a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4011c40 tx=0x7ff7b401c000 comp rx=0 tx=0).stop 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c16e2640 1 -- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102250 msgr2=0x7ff7bc1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.424+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102250 0x7ff7bc1986d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400d950 tx=0x7ff7a400de10 comp rx=0 tx=0).stop 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.425+0000 7ff7c16e2640 1 -- 172.21.15.69:0/1301994915 shutdown_connections 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.425+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff7bc102250 0x7ff7bc1986d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.425+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7a80a2cf0 0x7ff7a80a51b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.425+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff7bc1078b0 0x7ff7bc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.425+0000 7ff7c16e2640 1 --2- 172.21.15.69:0/1301994915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff7bc1018b0 0x7ff7bc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.425+0000 7ff7c16e2640 1 -- 172.21.15.69:0/1301994915 >> 172.21.15.69:0/1301994915 conn(0x7ff7bc0fd5f0 msgr2=0x7ff7bc0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.425+0000 7ff7c16e2640 1 -- 172.21.15.69:0/1301994915 shutdown_connections 2024-08-02T17:51:53.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:53.425+0000 7ff7c16e2640 1 -- 172.21.15.69:0/1301994915 wait complete. 2024-08-02T17:51:54.154 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:51:54.410 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:54 smithi083 ceph-mon[37031]: pgmap v167: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-08-02T17:51:54.532 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:54 smithi069 ceph-mon[32746]: pgmap v167: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-08-02T17:51:54.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:54 smithi077 ceph-mon[37108]: pgmap v167: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 11 MiB/s wr, 929 op/s 2024-08-02T17:51:54.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 -- 172.21.15.69:0/2595923062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf81078b0 msgr2=0x7fadf8107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 --2- 172.21.15.69:0/2595923062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf81078b0 0x7fadf8107c90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fadec009930 tx=0x7fadec02a420 comp rx=0 tx=0).stop 2024-08-02T17:51:54.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 -- 172.21.15.69:0/2595923062 shutdown_connections 2024-08-02T17:51:54.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 --2- 172.21.15.69:0/2595923062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf8102250 0x7fadf810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 --2- 172.21.15.69:0/2595923062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf81018b0 0x7fadf8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 --2- 172.21.15.69:0/2595923062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf81078b0 0x7fadf8107c90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 -- 172.21.15.69:0/2595923062 >> 172.21.15.69:0/2595923062 conn(0x7fadf80fd5f0 msgr2=0x7fadf80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:54.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 -- 172.21.15.69:0/2595923062 shutdown_connections 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.894+0000 7fadff12f640 1 -- 172.21.15.69:0/2595923062 wait complete. 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadff12f640 1 Processor -- start 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadff12f640 1 -- start start 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadff12f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf81018b0 0x7fadf819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadff12f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf8102250 0x7fadf819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadff12f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81078b0 0x7fadf819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadff12f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fadf8111cd0 con 0x7fadf8102250 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadff12f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadf8111b50 con 0x7fadf81018b0 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadff12f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadf8111e50 con 0x7fadf81078b0 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf8102250 0x7fadf819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf8102250 0x7fadf819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54786/0 (socket says 172.21.15.69:54786) 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 -- 172.21.15.69:0/2458474877 learned_addr learned my addr 172.21.15.69:0/2458474877 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:54.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadfcecb640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf81018b0 0x7fadf819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81078b0 0x7fadf819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 -- 172.21.15.69:0/2458474877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81078b0 msgr2=0x7fadf819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81078b0 0x7fadf819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 -- 172.21.15.69:0/2458474877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf81018b0 msgr2=0x7fadf819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf81018b0 0x7fadf819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 -- 172.21.15.69:0/2458474877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf819f3b0 con 0x7fadf8102250 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81078b0 0x7fadf819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.895+0000 7fadf7fff640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf8102250 0x7fadf819a940 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fade400d950 tx=0x7fade400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.896+0000 7fadf5ffb640 1 -- 172.21.15.69:0/2458474877 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4017070 con 0x7fadf8102250 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.896+0000 7fadf5ffb640 1 -- 172.21.15.69:0/2458474877 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade4009c30 con 0x7fadf8102250 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.896+0000 7fadff12f640 1 -- 172.21.15.69:0/2458474877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf819f6a0 con 0x7fadf8102250 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.896+0000 7fadf5ffb640 1 -- 172.21.15.69:0/2458474877 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade400f6b0 con 0x7fadf8102250 2024-08-02T17:51:54.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.896+0000 7fadff12f640 1 -- 172.21.15.69:0/2458474877 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf81a6f80 con 0x7fadf8102250 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fadf5ffb640 1 -- 172.21.15.69:0/2458474877 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fade40109d0 con 0x7fadf8102250 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fae0152d740 1 Processor -- start 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fae0152d740 1 -- start start 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fae0152d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db96aff0 0x55d4db96b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fae0152d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db975c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fae0152d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db9761d0 0x55d4db9785c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fadf5ffb640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fade80a2cf0 0x7fade80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fae0152d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d4db8d0a40 con 0x55d4db96b910 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fae0152d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4db8689d0 con 0x55d4db96aff0 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.898+0000 7fae0152d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d4db8cca00 con 0x55d4db9761d0 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db975c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fade80a2cf0 0x7fade80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db975c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54790/0 (socket says 172.21.15.69:54790) 2024-08-02T17:51:54.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 -- 172.21.15.69:0/1011517269 learned_addr learned my addr 172.21.15.69:0/1011517269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadf5ffb640 1 -- 172.21.15.69:0/2458474877 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fade40cf4a0 con 0x7fadf8102250 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db96aff0 0x55d4db96b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadf7fff640 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db9761d0 0x55d4db9785c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 -- 172.21.15.69:0/1011517269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db9761d0 msgr2=0x55d4db9785c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db9761d0 0x55d4db9785c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 -- 172.21.15.69:0/1011517269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db96aff0 msgr2=0x55d4db96b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db96aff0 0x55d4db96b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 -- 172.21.15.69:0/1011517269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4db86f650 con 0x55d4db96b910 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fade80a2cf0 0x7fade80a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fadf000afd0 tx=0x7fadf000ad00 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fadfcecb640 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db975c90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fadec009930 tx=0x7fadec031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb6bcba056619d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fade2ffd640 1 -- 172.21.15.69:0/1011517269 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec040070 con 0x55d4db96b910 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fade2ffd640 1 -- 172.21.15.69:0/1011517269 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadec005020 con 0x55d4db96b910 2024-08-02T17:51:54.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.899+0000 7fade2ffd640 1 -- 172.21.15.69:0/1011517269 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec035040 con 0x55d4db96b910 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 -- 172.21.15.69:0/1011517269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 msgr2=0x55d4db975c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db975c90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fadec009930 tx=0x7fadec031630 comp rx=0 tx=0).stop 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 -- 172.21.15.69:0/1011517269 shutdown_connections 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db975c90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db9761d0 0x55d4db9785c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 --2- 172.21.15.69:0/1011517269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db96aff0 0x55d4db96b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 -- 172.21.15.69:0/1011517269 >> 172.21.15.69:0/1011517269 conn(0x55d4db967580 msgr2=0x55d4db967960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 -- 172.21.15.69:0/1011517269 shutdown_connections 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 -- 172.21.15.69:0/1011517269 wait complete. 2024-08-02T17:51:54.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.900+0000 7fae0152d740 1 -- start start 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fae0152d740 1 Processor -- start 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fae0152d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db975b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fae0152d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 0x55d4db972ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fae0152d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fae0152d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d4db978da0 con 0x55d4db96aff0 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fae0152d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4db88bb20 con 0x55d4db9761d0 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fae0152d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d4db978f20 con 0x55d4db96b910 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fae0152d740 10 client.0 _ll_register_callbacks cb 0x55d4db889bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfcecb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db975b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfcecb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db975b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54798/0 (socket says 172.21.15.69:54798) 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfcecb640 1 -- 172.21.15.69:0/3515273004 learned_addr learned my addr 172.21.15.69:0/3515273004 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadf7fff640 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 0x55d4db972ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfcecb640 1 -- 172.21.15.69:0/3515273004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 msgr2=0x55d4db972ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfcecb640 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 0x55d4db972ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfcecb640 1 -- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 msgr2=0x55d4db9734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfcecb640 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfcecb640 1 -- 172.21.15.69:0/3515273004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4db6423c0 con 0x55d4db96aff0 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.901+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fadfcecb640 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db975b90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fadec031ba0 tx=0x7fadec005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbeab7e52df70ae9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fadd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fadd7fff640 1 -- 172.21.15.69:0/3515273004 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec040070 con 0x55d4db96aff0 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fadd7fff640 1 -- 172.21.15.69:0/3515273004 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fadec0053e0 con 0x55d4db96aff0 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 1 -- 172.21.15.69:0/3515273004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadec005020 con 0x55d4db96aff0 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fadd7fff640 1 -- 172.21.15.69:0/3515273004 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fadec035050 con 0x55d4db96aff0 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 10 client.16839 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:54.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 1 -- 172.21.15.69:0/3515273004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4db9847c0 con 0x55d4db96aff0 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fadd7fff640 1 -- 172.21.15.69:0/3515273004 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadec005020 con 0x55d4db96aff0 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fadd7fff640 1 -- 172.21.15.69:0/3515273004 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fadec03c860 con 0x55d4db96aff0 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 10 client.16839 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 20 client.16839 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4db985b10 0x55d4db987f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 4 client.16839 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.902+0000 7fae0152d740 1 -- 172.21.15.69:0/3515273004 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4db8ddfb0 con 0x55d4db985b10 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.903+0000 7fadf7fff640 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4db985b10 0x55d4db987f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.903+0000 7fadd7fff640 10 client.16839 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.903+0000 7fadf7fff640 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4db985b10 0x55d4db987f00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6941bf6b91f70bc9 server_cookie=a7952caf46ff30ac in_seq=0 out_seq=0 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fadd7fff640 1 -- 172.21.15.69:0/3515273004 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d4db8ddfb0 con 0x55d4db985b10 2024-08-02T17:51:54.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fadd7fff640 10 client.16839 handle_command_reply: tid=0 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 client.16839 shutdown 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 -- 172.21.15.69:0/3515273004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 msgr2=0x55d4db975b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db975b90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fadec031ba0 tx=0x7fadec005b50 comp rx=0 tx=0).stop 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 -- 172.21.15.69:0/3515273004 shutdown_connections 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4db985b10 0x55d4db987f00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 0x55d4db972ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fadd7fff640 0 client.16839 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 --2- 172.21.15.69:0/3515273004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db975b90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 -- 172.21.15.69:0/3515273004 >> 172.21.15.69:0/3515273004 conn(0x55d4db967580 msgr2=0x55d4db96d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 -- 172.21.15.69:0/3515273004 shutdown_connections 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 1 -- 172.21.15.69:0/3515273004 wait complete. 2024-08-02T17:51:54.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.904+0000 7fae0152d740 20 client.16839 trim_cache size 0 max 16384 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fae0152d740 1 Processor -- start 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fae0152d740 1 -- start start 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fae0152d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96aff0 0x55d4db96e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fae0152d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db96e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fae0152d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fae0152d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81b38e0 con 0x55d4db96b910 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fae0152d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81b11b0 con 0x55d4db9761d0 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fae0152d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81af000 con 0x55d4db96aff0 2024-08-02T17:51:54.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db96e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db96e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54804/0 (socket says 172.21.15.69:54804) 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.917+0000 7fadf7fff640 1 -- 172.21.15.69:0/3797827061 learned_addr learned my addr 172.21.15.69:0/3797827061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadfcecb640 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96aff0 0x55d4db96e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadf7fff640 1 -- 172.21.15.69:0/3797827061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96aff0 msgr2=0x55d4db96e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadf7fff640 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96aff0 0x55d4db96e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadf7fff640 1 -- 172.21.15.69:0/3797827061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 msgr2=0x55d4db9775c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadf7fff640 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9775c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadf7fff640 1 -- 172.21.15.69:0/3797827061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4db96ad70 con 0x55d4db96b910 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9775c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fadf7fff640 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db96e710 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fade409b300 tx=0x7fade40b1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14013ecceb7f646e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fade27fc640 1 -- 172.21.15.69:0/3797827061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4097040 con 0x55d4db96b910 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fade27fc640 1 -- 172.21.15.69:0/3797827061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade409f9b0 con 0x55d4db96b910 2024-08-02T17:51:54.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.918+0000 7fade27fc640 1 -- 172.21.15.69:0/3797827061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade4020040 con 0x55d4db96b910 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 -- 172.21.15.69:0/3797827061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 msgr2=0x55d4db96e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db96e710 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fade409b300 tx=0x7fade40b1040 comp rx=0 tx=0).stop 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 -- 172.21.15.69:0/3797827061 shutdown_connections 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96b910 0x55d4db96e710 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db9775c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 --2- 172.21.15.69:0/3797827061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96aff0 0x55d4db96e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 -- 172.21.15.69:0/3797827061 >> 172.21.15.69:0/3797827061 conn(0x55d4db967580 msgr2=0x55d4db9621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 -- 172.21.15.69:0/3797827061 shutdown_connections 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 -- 172.21.15.69:0/3797827061 wait complete. 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 -- start start 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.919+0000 7fae0152d740 1 Processor -- start 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fae0152d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db970bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fae0152d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 0x55d4db98af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fae0152d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db98b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fae0152d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81af670 con 0x55d4db96aff0 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fae0152d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81b0ae0 con 0x55d4db9761d0 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fae0152d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81afd40 con 0x55d4db96b910 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fae0152d740 10 client.0 _ll_register_callbacks cb 0x7fadec005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db970bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db970bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54812/0 (socket says 172.21.15.69:54812) 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 -- 172.21.15.69:0/1720360209 learned_addr learned my addr 172.21.15.69:0/1720360209 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfd6cc640 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db98b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadf7fff640 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 0x55d4db98af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 -- 172.21.15.69:0/1720360209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 msgr2=0x55d4db98af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 0x55d4db98af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 -- 172.21.15.69:0/1720360209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 msgr2=0x55d4db98b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:54.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db98b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 -- 172.21.15.69:0/1720360209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4db96f1d0 con 0x55d4db96aff0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadfcecb640 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db970bc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fadec031b70 tx=0x7fadec03c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94696fd0dab4a931 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadd7fff640 1 -- 172.21.15.69:0/1720360209 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadec040040 con 0x55d4db96aff0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fae0152d740 1 -- 172.21.15.69:0/1720360209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade409f9b0 con 0x55d4db96aff0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.920+0000 7fadd7fff640 1 -- 172.21.15.69:0/1720360209 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fadec03c970 con 0x55d4db96aff0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fadd7fff640 1 -- 172.21.15.69:0/1720360209 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fadec005850 con 0x55d4db96aff0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fae0152d740 10 client.16845 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fae0152d740 1 -- 172.21.15.69:0/1720360209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4db9ef7c0 con 0x55d4db96aff0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fadd7fff640 1 -- 172.21.15.69:0/1720360209 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade409f9b0 con 0x55d4db96aff0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fadd7fff640 1 -- 172.21.15.69:0/1720360209 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fadec05b030 con 0x55d4db96aff0 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fae0152d740 10 client.16845 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:54.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fae0152d740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fae0152d740 20 client.16845 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fae0152d740 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4db9efcf0 0x55d4db9f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fae0152d740 4 client.16845 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fae0152d740 1 -- 172.21.15.69:0/1720360209 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4db8d84c0 con 0x55d4db9efcf0 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.921+0000 7fadf7fff640 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4db9efcf0 0x55d4db9f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fadd7fff640 10 client.16845 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fadf7fff640 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4db9efcf0 0x55d4db9f20e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8d9ab5c319e56fb server_cookie=f03ffefb31fdee20 in_seq=0 out_seq=0 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fadd7fff640 1 -- 172.21.15.69:0/1720360209 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4db8d84c0 con 0x55d4db9efcf0 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fadd7fff640 10 client.16845 handle_command_reply: tid=0 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 client.16845 shutdown 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 -- 172.21.15.69:0/1720360209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 msgr2=0x55d4db970bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db970bc0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fadec031b70 tx=0x7fadec03c670 comp rx=0 tx=0).stop 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 -- 172.21.15.69:0/1720360209 shutdown_connections 2024-08-02T17:51:54.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4db9efcf0 0x55d4db9f20e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d4db9761d0 0x55d4db98b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d4db96b910 0x55d4db98af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fadd7fff640 0 client.16845 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 --2- 172.21.15.69:0/1720360209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d4db96aff0 0x55d4db970bc0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 -- 172.21.15.69:0/1720360209 >> 172.21.15.69:0/1720360209 conn(0x55d4db967580 msgr2=0x55d4db96e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 -- 172.21.15.69:0/1720360209 shutdown_connections 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 1 -- 172.21.15.69:0/1720360209 wait complete. 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.922+0000 7fae0152d740 20 client.16845 trim_cache size 0 max 16384 2024-08-02T17:51:54.934 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:51:54.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 -- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fade80a2cf0 msgr2=0x7fade80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fade80a2cf0 0x7fade80a51b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fadf000afd0 tx=0x7fadf000ad00 comp rx=0 tx=0).stop 2024-08-02T17:51:54.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 -- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf8102250 msgr2=0x7fadf819a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:54.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf8102250 0x7fadf819a940 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fade400d950 tx=0x7fade400de10 comp rx=0 tx=0).stop 2024-08-02T17:51:54.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 -- 172.21.15.69:0/2458474877 shutdown_connections 2024-08-02T17:51:54.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadf8102250 0x7fadf819a940 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fade80a2cf0 0x7fade80a51b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadf81078b0 0x7fadf819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 --2- 172.21.15.69:0/2458474877 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadf81018b0 0x7fadf819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:54.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 -- 172.21.15.69:0/2458474877 >> 172.21.15.69:0/2458474877 conn(0x7fadf80fd5f0 msgr2=0x7fadf80fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:54.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 -- 172.21.15.69:0/2458474877 shutdown_connections 2024-08-02T17:51:54.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:54.923+0000 7fadff12f640 1 -- 172.21.15.69:0/2458474877 wait complete. 2024-08-02T17:51:54.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 285.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/2303663249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541006d0 msgr2=0x7f2d54100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/2303663249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541006d0 0x7f2d54100b30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009930 tx=0x7f2d4802a420 comp rx=0 tx=0).stop 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/2303663249 shutdown_connections 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/2303663249 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541006d0 0x7f2d54100b30 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/2303663249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d54101070 0x7f2d5410d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/2303663249 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d541066e0 0x7f2d54106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/2303663249 >> 172.21.15.69:0/2303663249 conn(0x7f2d540776d0 msgr2=0x7f2d540fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/2303663249 shutdown_connections 2024-08-02T17:51:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.168+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/2303663249 wait complete. 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5c3d7640 1 Processor -- start 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5c3d7640 1 -- start start 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5c3d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d541006d0 0x7f2d54197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5c3d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d54101070 0x7f2d54197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5c3d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541066e0 0x7f2d5419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5c3d7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54110b70 con 0x7f2d541066e0 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5c3d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541109f0 con 0x7f2d54101070 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5c3d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d54110cf0 con 0x7f2d541006d0 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a974640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541066e0 0x7f2d5419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a974640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541066e0 0x7f2d5419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54830/0 (socket says 172.21.15.69:54830) 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a974640 1 -- 172.21.15.69:0/333766684 learned_addr learned my addr 172.21.15.69:0/333766684 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a173640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d541006d0 0x7f2d54197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d59972640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d54101070 0x7f2d54197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a974640 1 -- 172.21.15.69:0/333766684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d541006d0 msgr2=0x7f2d54197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a974640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d541006d0 0x7f2d54197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a974640 1 -- 172.21.15.69:0/333766684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d54101070 msgr2=0x7f2d54197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a974640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d54101070 0x7f2d54197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d5a974640 1 -- 172.21.15.69:0/333766684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5419c030 con 0x7f2d541066e0 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.169+0000 7f2d59972640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d54101070 0x7f2d54197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.170+0000 7f2d5a974640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541066e0 0x7f2d5419b900 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400cad0 tx=0x7f2d4400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.170+0000 7f2d537fe640 1 -- 172.21.15.69:0/333766684 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400ae20 con 0x7f2d541066e0 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.170+0000 7f2d537fe640 1 -- 172.21.15.69:0/333766684 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d44005020 con 0x7f2d541066e0 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.170+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/333766684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d5419c320 con 0x7f2d541066e0 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.170+0000 7f2d537fe640 1 -- 172.21.15.69:0/333766684 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4400f6f0 con 0x7f2d541066e0 2024-08-02T17:51:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.170+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/333766684 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d541a3bb0 con 0x7f2d541066e0 2024-08-02T17:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.172+0000 7f2d537fe640 1 -- 172.21.15.69:0/333766684 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2d440051d0 con 0x7f2d541066e0 2024-08-02T17:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.172+0000 7f2d5f20a740 1 Processor -- start 2024-08-02T17:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5f20a740 1 -- start start 2024-08-02T17:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 0x5557d3304640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d3304b80 0x5557d3306f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5557d325b9b0 con 0x5557d3304b80 2024-08-02T17:51:56.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557d3203800 con 0x5557d32fa2e0 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5557d3257970 con 0x5557d32f99c0 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d537fe640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d3c0a2cf0 0x7f2d3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d59972640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d3304b80 0x5557d3306f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d59972640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d3304b80 0x5557d3306f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54834/0 (socket says 172.21.15.69:54834) 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d59972640 1 -- 172.21.15.69:0/3663387693 learned_addr learned my addr 172.21.15.69:0/3663387693 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5a974640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d3c0a2cf0 0x7f2d3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d5a173640 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 0x5557d3304640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d59972640 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.173+0000 7f2d537fe640 1 -- 172.21.15.69:0/333766684 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2d440cfb70 con 0x7f2d541066e0 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d59972640 1 -- 172.21.15.69:0/3663387693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 msgr2=0x5557d32f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d59972640 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d59972640 1 -- 172.21.15.69:0/3663387693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 msgr2=0x5557d3304640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:56.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d59972640 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 0x5557d3304640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d59972640 1 -- 172.21.15.69:0/3663387693 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d2f6f210 con 0x5557d3304b80 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d59972640 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d3304b80 0x5557d3306f70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005ab0 tx=0x7f2d48000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a80c5370e2690ec server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d5a974640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d3c0a2cf0 0x7f2d3c0a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2d440cd500 tx=0x7f2d44000f40 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d50bfe640 1 -- 172.21.15.69:0/3663387693 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48045070 con 0x5557d3304b80 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d50bfe640 1 -- 172.21.15.69:0/3663387693 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d480046c0 con 0x5557d3304b80 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.174+0000 7f2d50bfe640 1 -- 172.21.15.69:0/3663387693 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48035040 con 0x5557d3304b80 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3663387693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d3304b80 msgr2=0x5557d3306f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d3304b80 0x5557d3306f70 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2d48005ab0 tx=0x7f2d48000bc0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3663387693 shutdown_connections 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d3304b80 0x5557d3306f70 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 0x5557d3304640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/3663387693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3663387693 >> 172.21.15.69:0/3663387693 conn(0x5557d32f5f30 msgr2=0x5557d32f6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:56.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3663387693 shutdown_connections 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3663387693 wait complete. 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 -- start start 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.175+0000 7f2d5f20a740 1 Processor -- start 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d3301ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d3302010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 0x5557d3302550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5557d3307750 con 0x5557d32fa2e0 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5557d3217500 con 0x5557d3304b80 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5557d33078d0 con 0x5557d32f99c0 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5f20a740 10 client.0 _ll_register_callbacks cb 0x5557d2f6ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d3302010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d3302010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54840/0 (socket says 172.21.15.69:54840) 2024-08-02T17:51:56.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 -- 172.21.15.69:0/2206154203 learned_addr learned my addr 172.21.15.69:0/2206154203 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d59972640 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 0x5557d3302550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d59972640 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d3301ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 -- 172.21.15.69:0/2206154203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 msgr2=0x5557d3301ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d3301ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 -- 172.21.15.69:0/2206154203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 msgr2=0x5557d3302550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 0x5557d3302550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 -- 172.21.15.69:0/2206154203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d303b620 con 0x5557d32fa2e0 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d59972640 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d3301ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5a173640 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d3302010 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00ba20 tx=0x7f2d4c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9d1fa1409e3e05 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d39ffb640 1 -- 172.21.15.69:0/2206154203 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c003e80 con 0x5557d32fa2e0 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d39ffb640 1 -- 172.21.15.69:0/2206154203 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2d4c011440 con 0x5557d32fa2e0 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.176+0000 7f2d5f20a740 1 -- 172.21.15.69:0/2206154203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d480046c0 con 0x5557d32fa2e0 2024-08-02T17:51:56.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d39ffb640 1 -- 172.21.15.69:0/2206154203 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d4c011640 con 0x5557d32fa2e0 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d5f20a740 10 client.16857 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d5f20a740 1 -- 172.21.15.69:0/2206154203 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557d33134a0 con 0x5557d32fa2e0 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d39ffb640 1 -- 172.21.15.69:0/2206154203 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d480046c0 con 0x5557d32fa2e0 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d39ffb640 1 -- 172.21.15.69:0/2206154203 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d4c01ba70 con 0x5557d32fa2e0 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d5f20a740 10 client.16857 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d5f20a740 10 client.16857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d5f20a740 20 client.16857 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5557d3314a20 0x5557d3316e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d5f20a740 4 client.16857 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d5f20a740 1 -- 172.21.15.69:0/2206154203 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557d32690e0 con 0x5557d3314a20 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.177+0000 7f2d59972640 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5557d3314a20 0x5557d3316e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d39ffb640 10 client.16857 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:56.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d59972640 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5557d3314a20 0x5557d3316e10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee3e4c40bf31cd4 server_cookie=30e0aa2818045cff in_seq=0 out_seq=0 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d39ffb640 1 -- 172.21.15.69:0/2206154203 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5557d32690e0 con 0x5557d3314a20 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d39ffb640 10 client.16857 handle_command_reply: tid=0 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 client.16857 shutdown 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 -- 172.21.15.69:0/2206154203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 msgr2=0x5557d3302010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d3302010 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00ba20 tx=0x7f2d4c00bee0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 -- 172.21.15.69:0/2206154203 shutdown_connections 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d3302010 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5557d3314a20 0x5557d3316e10 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 0x5557d3302550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/2206154203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d3301ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d39ffb640 0 client.16857 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.178+0000 7f2d5f20a740 1 -- 172.21.15.69:0/2206154203 >> 172.21.15.69:0/2206154203 conn(0x5557d32f5f30 msgr2=0x5557d32fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:56.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.179+0000 7f2d5f20a740 1 -- 172.21.15.69:0/2206154203 shutdown_connections 2024-08-02T17:51:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.179+0000 7f2d5f20a740 1 -- 172.21.15.69:0/2206154203 wait complete. 2024-08-02T17:51:56.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.179+0000 7f2d5f20a740 20 client.16857 trim_cache size 0 max 16384 2024-08-02T17:51:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.194+0000 7f2d5f20a740 1 Processor -- start 2024-08-02T17:51:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.194+0000 7f2d5f20a740 1 -- start start 2024-08-02T17:51:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.194+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32f99c0 0x5557d3302380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.194+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 0x5557d32fbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.194+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d3304b80 0x5557d330cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.194+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541af530 con 0x5557d32f99c0 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.194+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541ace00 con 0x5557d32fa2e0 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.194+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541aac50 con 0x5557d3304b80 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32f99c0 0x5557d3302380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32f99c0 0x5557d3302380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54844/0 (socket says 172.21.15.69:54844) 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 -- 172.21.15.69:0/3943015795 learned_addr learned my addr 172.21.15.69:0/3943015795 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 -- 172.21.15.69:0/3943015795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d3304b80 msgr2=0x5557d330cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 --2- 172.21.15.69:0/3943015795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d3304b80 0x5557d330cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 -- 172.21.15.69:0/3943015795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 msgr2=0x5557d32fbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 --2- 172.21.15.69:0/3943015795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 0x5557d32fbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 -- 172.21.15.69:0/3943015795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d3317180 con 0x5557d32f99c0 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d59972640 1 --2- 172.21.15.69:0/3943015795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32f99c0 0x5557d3302380 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009930 tx=0x7f2d48004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5257688c87c5792c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d3bfff640 1 -- 172.21.15.69:0/3943015795 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48035070 con 0x5557d32f99c0 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d3bfff640 1 -- 172.21.15.69:0/3943015795 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d48050070 con 0x5557d32f99c0 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.195+0000 7f2d3bfff640 1 -- 172.21.15.69:0/3943015795 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d48045040 con 0x5557d32f99c0 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3943015795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32f99c0 msgr2=0x5557d3302380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/3943015795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32f99c0 0x5557d3302380 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2d48009930 tx=0x7f2d48004260 comp rx=0 tx=0).stop 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3943015795 shutdown_connections 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/3943015795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d3304b80 0x5557d330cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/3943015795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d32fa2e0 0x5557d32fbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/3943015795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32f99c0 0x5557d3302380 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3943015795 >> 172.21.15.69:0/3943015795 conn(0x5557d32f5f30 msgr2=0x5557d32fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3943015795 shutdown_connections 2024-08-02T17:51:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 -- 172.21.15.69:0/3943015795 wait complete. 2024-08-02T17:51:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 -- start start 2024-08-02T17:51:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 Processor -- start 2024-08-02T17:51:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d32f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 0x5557d32f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.196+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541ab2c0 con 0x5557d32fa2e0 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541ac730 con 0x5557d3304b80 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5f20a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d541ab990 con 0x5557d32f99c0 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5f20a740 10 client.0 _ll_register_callbacks cb 0x7f2d4c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d32f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d32f53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54854/0 (socket says 172.21.15.69:54854) 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 -- 172.21.15.69:0/1910018226 learned_addr learned my addr 172.21.15.69:0/1910018226 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d59972640 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 0x5557d32f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 -- 172.21.15.69:0/1910018226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 msgr2=0x5557d32f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d59972640 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 -- 172.21.15.69:0/1910018226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 msgr2=0x5557d32f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 0x5557d32f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 -- 172.21.15.69:0/1910018226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d32f3bf0 con 0x5557d32fa2e0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d59972640 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5a173640 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d32f53d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c01be30 tx=0x7f2d4c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c59cdd8a64df7a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d39ffb640 1 -- 172.21.15.69:0/1910018226 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c010830 con 0x5557d32fa2e0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5f20a740 1 -- 172.21.15.69:0/1910018226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d48050070 con 0x5557d32fa2e0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d39ffb640 1 -- 172.21.15.69:0/1910018226 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2d4c0109d0 con 0x5557d32fa2e0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d39ffb640 1 -- 172.21.15.69:0/1910018226 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d4c037050 con 0x5557d32fa2e0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5f20a740 10 client.16863 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.197+0000 7f2d5f20a740 1 -- 172.21.15.69:0/1910018226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5557d3382390 con 0x5557d32fa2e0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d39ffb640 1 -- 172.21.15.69:0/1910018226 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d48050070 con 0x5557d32fa2e0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d39ffb640 1 -- 172.21.15.69:0/1910018226 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d4c0050d0 con 0x5557d32fa2e0 2024-08-02T17:51:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d5f20a740 10 client.16863 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d5f20a740 10 client.16863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d5f20a740 20 client.16863 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5557d3382b30 0x5557d3384f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d5f20a740 4 client.16863 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d5f20a740 1 -- 172.21.15.69:0/1910018226 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557d3263430 con 0x5557d3382b30 2024-08-02T17:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d59972640 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5557d3382b30 0x5557d3384f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d39ffb640 10 client.16863 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.198+0000 7f2d59972640 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5557d3382b30 0x5557d3384f50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26c955723d27af3 server_cookie=270bf7e2e94f7bf8 in_seq=0 out_seq=0 2024-08-02T17:51:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.214+0000 7f2d39ffb640 1 -- 172.21.15.69:0/1910018226 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557d3263430 con 0x5557d3382b30 2024-08-02T17:51:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.214+0000 7f2d39ffb640 10 client.16863 handle_command_reply: tid=0 2024-08-02T17:51:56.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.214+0000 7f2d5f20a740 1 client.16863 shutdown 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.214+0000 7f2d5f20a740 1 -- 172.21.15.69:0/1910018226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 msgr2=0x5557d32f53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.214+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d32f53d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c01be30 tx=0x7f2d4c00bfd0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 1 -- 172.21.15.69:0/1910018226 shutdown_connections 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5557d32fa2e0 0x5557d32f53d0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5557d3382b30 0x5557d3384f50 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5557d3304b80 0x5557d32f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d39ffb640 0 client.16863 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 1 --2- 172.21.15.69:0/1910018226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5557d32f99c0 0x5557d32f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 1 -- 172.21.15.69:0/1910018226 >> 172.21.15.69:0/1910018226 conn(0x5557d32f5f30 msgr2=0x5557d32fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 1 -- 172.21.15.69:0/1910018226 shutdown_connections 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 1 -- 172.21.15.69:0/1910018226 wait complete. 2024-08-02T17:51:56.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5f20a740 20 client.16863 trim_cache size 0 max 16384 2024-08-02T17:51:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:51:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:51:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:51:56.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d3c0a2cf0 msgr2=0x7f2d3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d3c0a2cf0 0x7f2d3c0a51b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2d440cd500 tx=0x7f2d44000f40 comp rx=0 tx=0).stop 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541066e0 msgr2=0x7f2d5419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541066e0 0x7f2d5419b900 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400cad0 tx=0x7f2d4400cf90 comp rx=0 tx=0).stop 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/333766684 shutdown_connections 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d541066e0 0x7f2d5419b900 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d3c0a2cf0 0x7f2d3c0a51b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d54101070 0x7f2d54197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 --2- 172.21.15.69:0/333766684 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d541006d0 0x7f2d54197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:56.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/333766684 >> 172.21.15.69:0/333766684 conn(0x7f2d540776d0 msgr2=0x7f2d54103540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/333766684 shutdown_connections 2024-08-02T17:51:56.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:51:56.215+0000 7f2d5c3d7640 1 -- 172.21.15.69:0/333766684 wait complete. 2024-08-02T17:51:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:56 smithi083 ceph-mon[37031]: pgmap v168: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.4 MiB/s wr, 544 op/s 2024-08-02T17:51:56.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:56 smithi077 ceph-mon[37108]: pgmap v168: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.4 MiB/s wr, 544 op/s 2024-08-02T17:51:56.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:56 smithi069 ceph-mon[32746]: pgmap v168: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.4 MiB/s wr, 544 op/s 2024-08-02T17:51:56.714 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:51:56.714 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:51:56.971 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:57.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.733+0000 7fe38d30e640 1 -- 172.21.15.69:0/761113463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe388103390 msgr2=0x7fe38810b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.733+0000 7fe38d30e640 1 --2- 172.21.15.69:0/761113463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe388103390 0x7fe38810b880 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe370009900 tx=0x7fe37002a480 comp rx=0 tx=0).stop 2024-08-02T17:51:57.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.733+0000 7fe38d30e640 1 -- 172.21.15.69:0/761113463 shutdown_connections 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.733+0000 7fe38d30e640 1 --2- 172.21.15.69:0/761113463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe388103390 0x7fe38810b880 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.733+0000 7fe38d30e640 1 --2- 172.21.15.69:0/761113463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe38810bea0 0x7fe38810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.733+0000 7fe38d30e640 1 --2- 172.21.15.69:0/761113463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe388102a70 0x7fe388102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.734+0000 7fe38d30e640 1 -- 172.21.15.69:0/761113463 >> 172.21.15.69:0/761113463 conn(0x7fe3880fc4d0 msgr2=0x7fe3880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.734+0000 7fe38d30e640 1 -- 172.21.15.69:0/761113463 shutdown_connections 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.734+0000 7fe38d30e640 1 -- 172.21.15.69:0/761113463 wait complete. 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.734+0000 7fe38d30e640 1 Processor -- start 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.734+0000 7fe38d30e640 1 -- start start 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe38d30e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe388102a70 0x7fe388198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe38d30e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe388103390 0x7fe388198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe38d30e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe38810bea0 0x7fe38819c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe38d30e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe388111ca0 con 0x7fe38810bea0 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe38d30e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe388111b20 con 0x7fe388103390 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe38d30e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe388111e20 con 0x7fe388102a70 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe38810bea0 0x7fe38819c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe38810bea0 0x7fe38819c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54860/0 (socket says 172.21.15.69:54860) 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 -- 172.21.15.69:0/1463482579 learned_addr learned my addr 172.21.15.69:0/1463482579 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe37ffff640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe388103390 0x7fe388198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe386ffd640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe388102a70 0x7fe388198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 -- 172.21.15.69:0/1463482579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe388102a70 msgr2=0x7fe388198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe388102a70 0x7fe388198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 -- 172.21.15.69:0/1463482579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe388103390 msgr2=0x7fe388198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe388103390 0x7fe388198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 -- 172.21.15.69:0/1463482579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38819d0b0 con 0x7fe38810bea0 2024-08-02T17:51:57.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe3877fe640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe38810bea0 0x7fe38819c9d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe38000eaa0 tx=0x7fe38000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe384ff9640 1 -- 172.21.15.69:0/1463482579 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38000cdb0 con 0x7fe38810bea0 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe384ff9640 1 -- 172.21.15.69:0/1463482579 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe380005020 con 0x7fe38810bea0 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe38d30e640 1 -- 172.21.15.69:0/1463482579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38819d3a0 con 0x7fe38810bea0 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.735+0000 7fe384ff9640 1 -- 172.21.15.69:0/1463482579 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3800116b0 con 0x7fe38810bea0 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.736+0000 7fe38d30e640 1 -- 172.21.15.69:0/1463482579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3881a4c80 con 0x7fe38810bea0 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe384ff9640 1 -- 172.21.15.69:0/1463482579 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe380005560 con 0x7fe38810bea0 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe38f70c740 1 Processor -- start 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe38f70c740 1 -- start start 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02b20 0x559489a0cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a0d3e0 0x559489a0f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe38f70c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5594899c5e40 con 0x559489a02200 2024-08-02T17:51:57.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe38f70c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5594899677f0 con 0x559489a0d3e0 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.738+0000 7fe38f70c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55948990efb0 con 0x559489a02b20 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54874/0 (socket says 172.21.15.69:54874) 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 -- 172.21.15.69:0/669630649 learned_addr learned my addr 172.21.15.69:0/669630649 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe386ffd640 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02b20 0x559489a0cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a0d3e0 0x559489a0f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe384ff9640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3740a2cf0 0x7fe3740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 -- 172.21.15.69:0/669630649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02b20 msgr2=0x559489a0cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02b20 0x559489a0cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 -- 172.21.15.69:0/669630649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a0d3e0 msgr2=0x559489a0f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a0d3e0 0x559489a0f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 -- 172.21.15.69:0/669630649 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594896d96a0 con 0x559489a02200 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe3877fe640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3740a2cf0 0x7fe3740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37ffff640 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a025e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe37002a9a0 tx=0x7fe370000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd78cbefb549e4fb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe384ff9640 1 -- 172.21.15.69:0/1463482579 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe3800cf650 con 0x7fe38810bea0 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.739+0000 7fe37d7fa640 1 -- 172.21.15.69:0/669630649 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370044030 con 0x559489a02200 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe37d7fa640 1 -- 172.21.15.69:0/669630649 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe370031a00 con 0x559489a02200 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe37d7fa640 1 -- 172.21.15.69:0/669630649 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370035040 con 0x559489a02200 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 -- 172.21.15.69:0/669630649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 msgr2=0x559489a025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe3877fe640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3740a2cf0 0x7fe3740a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe380097c10 tx=0x7fe38006e820 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a025e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe37002a9a0 tx=0x7fe370000bc0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 -- 172.21.15.69:0/669630649 shutdown_connections 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a0d3e0 0x559489a0f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02b20 0x559489a0cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 --2- 172.21.15.69:0/669630649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a025e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 -- 172.21.15.69:0/669630649 >> 172.21.15.69:0/669630649 conn(0x5594899fe790 msgr2=0x5594899feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 -- 172.21.15.69:0/669630649 shutdown_connections 2024-08-02T17:51:57.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.740+0000 7fe38f70c740 1 -- 172.21.15.69:0/669630649 wait complete. 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 1 -- start start 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 1 Processor -- start 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02200 0x559489a0a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a0d3e0 0x559489a0adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559489922cb0 con 0x559489a0d3e0 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5594899637b0 con 0x559489a02b20 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559489a0ffb0 con 0x559489a02200 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe38f70c740 10 client.0 _ll_register_callbacks cb 0x559489676ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe37ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a0d3e0 0x559489a0adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe37ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a0d3e0 0x559489a0adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54888/0 (socket says 172.21.15.69:54888) 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe37ffff640 1 -- 172.21.15.69:0/1001683055 learned_addr learned my addr 172.21.15.69:0/1001683055 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:57.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe386ffd640 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.741+0000 7fe37ffff640 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02200 0x559489a0a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe37ffff640 1 -- 172.21.15.69:0/1001683055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02200 msgr2=0x559489a0a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe37ffff640 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02200 0x559489a0a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe37ffff640 1 -- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 msgr2=0x559489a0a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe37ffff640 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe37ffff640 1 -- 172.21.15.69:0/1001683055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559489921bc0 con 0x559489a0d3e0 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe386ffd640 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe37ffff640 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a0d3e0 0x559489a0adc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe37002ac60 tx=0x7fe370031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c5f97963e5afd2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe36e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe36e7fc640 1 -- 172.21.15.69:0/1001683055 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370051020 con 0x559489a0d3e0 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe36e7fc640 1 -- 172.21.15.69:0/1001683055 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe370004ad0 con 0x559489a0d3e0 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe38f70c740 1 -- 172.21.15.69:0/1001683055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe370031a00 con 0x559489a0d3e0 2024-08-02T17:51:57.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe36e7fc640 1 -- 172.21.15.69:0/1001683055 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe370035070 con 0x559489a0d3e0 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe38f70c740 10 client.16875 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe38f70c740 1 -- 172.21.15.69:0/1001683055 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559489a1bef0 con 0x559489a0d3e0 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe36e7fc640 1 -- 172.21.15.69:0/1001683055 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe370031a00 con 0x559489a0d3e0 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe36e7fc640 1 -- 172.21.15.69:0/1001683055 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe370048c90 con 0x559489a0d3e0 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe38f70c740 10 client.16875 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe38f70c740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.742+0000 7fe38f70c740 20 client.16875 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.743+0000 7fe38f70c740 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559489a1d240 0x559489a1f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.743+0000 7fe38f70c740 4 client.16875 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.743+0000 7fe38f70c740 1 -- 172.21.15.69:0/1001683055 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559489974d60 con 0x559489a1d240 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.743+0000 7fe386ffd640 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559489a1d240 0x559489a1f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.743+0000 7fe386ffd640 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559489a1d240 0x559489a1f630 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ee8e0617a2c4d0 server_cookie=aa96f6578fb4b7f7 in_seq=0 out_seq=0 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.743+0000 7fe36e7fc640 10 client.16875 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:57.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe36e7fc640 1 -- 172.21.15.69:0/1001683055 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559489974d60 con 0x559489a1d240 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe36e7fc640 10 client.16875 handle_command_reply: tid=0 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 client.16875 shutdown 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 -- 172.21.15.69:0/1001683055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a0d3e0 msgr2=0x559489a0adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a0d3e0 0x559489a0adc0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe37002ac60 tx=0x7fe370031400 comp rx=0 tx=0).stop 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 -- 172.21.15.69:0/1001683055 shutdown_connections 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a0d3e0 0x559489a0adc0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559489a1d240 0x559489a1f630 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 --2- 172.21.15.69:0/1001683055 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a02200 0x559489a0a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe36e7fc640 0 client.16875 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 -- 172.21.15.69:0/1001683055 >> 172.21.15.69:0/1001683055 conn(0x5594899fe790 msgr2=0x559489a044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 -- 172.21.15.69:0/1001683055 shutdown_connections 2024-08-02T17:51:57.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 1 -- 172.21.15.69:0/1001683055 wait complete. 2024-08-02T17:51:57.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.744+0000 7fe38f70c740 20 client.16875 trim_cache size 0 max 16384 2024-08-02T17:51:57.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.762+0000 7fe38f70c740 1 Processor -- start 2024-08-02T17:51:57.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.762+0000 7fe38f70c740 1 -- start start 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.762+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a0c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.762+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.762+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 0x559489a07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.762+0000 7fe38f70c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3881b15c0 con 0x559489a02200 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.762+0000 7fe38f70c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3881aee90 con 0x559489a02b20 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.762+0000 7fe38f70c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3881acce0 con 0x559489a0d3e0 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a0c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a0c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54900/0 (socket says 172.21.15.69:54900) 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 -- 172.21.15.69:0/2574664628 learned_addr learned my addr 172.21.15.69:0/2574664628 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe386ffd640 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 -- 172.21.15.69:0/2574664628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 msgr2=0x559489a07620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:57.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 0x559489a07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 -- 172.21.15.69:0/2574664628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 msgr2=0x559489a0aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:57.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 -- 172.21.15.69:0/2574664628 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559489a0fb40 con 0x559489a02200 2024-08-02T17:51:57.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe386ffd640 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:57.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37ffff640 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a0c210 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe370009900 tx=0x7fe370009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5ebc970be67950 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:57.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37cff9640 1 -- 172.21.15.69:0/2574664628 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370051070 con 0x559489a02200 2024-08-02T17:51:57.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37cff9640 1 -- 172.21.15.69:0/2574664628 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe370031900 con 0x559489a02200 2024-08-02T17:51:57.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.763+0000 7fe37cff9640 1 -- 172.21.15.69:0/2574664628 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370044040 con 0x559489a02200 2024-08-02T17:51:57.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 -- 172.21.15.69:0/2574664628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 msgr2=0x559489a0c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a0c210 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe370009900 tx=0x7fe370009780 comp rx=0 tx=0).stop 2024-08-02T17:51:57.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 -- 172.21.15.69:0/2574664628 shutdown_connections 2024-08-02T17:51:57.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 0x559489a07620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02b20 0x559489a0aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 --2- 172.21.15.69:0/2574664628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02200 0x559489a0c210 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 -- 172.21.15.69:0/2574664628 >> 172.21.15.69:0/2574664628 conn(0x5594899fe790 msgr2=0x559489a0a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 -- 172.21.15.69:0/2574664628 shutdown_connections 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 -- 172.21.15.69:0/2574664628 wait complete. 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.764+0000 7fe38f70c740 1 -- start start 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe38f70c740 1 Processor -- start 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02200 0x559489a20500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02b20 0x559489a20a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe38f70c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 0x559489a20f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe38f70c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3881ad350 con 0x559489a02b20 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe38f70c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3881ae7c0 con 0x559489a02200 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe38f70c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3881ada20 con 0x559489a0d3e0 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe38f70c740 10 client.0 _ll_register_callbacks cb 0x7fe37800c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02b20 0x559489a20a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02b20 0x559489a20a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54914/0 (socket says 172.21.15.69:54914) 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 -- 172.21.15.69:0/586015489 learned_addr learned my addr 172.21.15.69:0/586015489 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe37ffff640 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 0x559489a20f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe37ffff640 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02200 0x559489a20500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 -- 172.21.15.69:0/586015489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 msgr2=0x559489a20f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 0x559489a20f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 -- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02200 msgr2=0x559489a20500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02200 0x559489a20500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 -- 172.21.15.69:0/586015489 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559489a86ce0 con 0x559489a02b20 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe37ffff640 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02200 0x559489a20500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe386ffd640 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02b20 0x559489a20a40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe37800c710 tx=0x7fe3780039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4d5eb8a5d71fb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.765+0000 7fe36e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe36e7fc640 1 -- 172.21.15.69:0/586015489 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378005220 con 0x559489a02b20 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe36e7fc640 1 -- 172.21.15.69:0/586015489 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fe3780053c0 con 0x559489a02b20 2024-08-02T17:51:57.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 1 -- 172.21.15.69:0/586015489 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe370031900 con 0x559489a02b20 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe36e7fc640 1 -- 172.21.15.69:0/586015489 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe37802b050 con 0x559489a02b20 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 10 client.16881 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 1 -- 172.21.15.69:0/586015489 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559489a88050 con 0x559489a02b20 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe36e7fc640 1 -- 172.21.15.69:0/586015489 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe370031900 con 0x559489a02b20 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe36e7fc640 1 -- 172.21.15.69:0/586015489 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe37801c820 con 0x559489a02b20 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 10 client.16881 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 20 client.16881 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559489a88800 0x559489a8ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 4 client.16881 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.766+0000 7fe38f70c740 1 -- 172.21.15.69:0/586015489 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55948996f270 con 0x559489a88800 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.767+0000 7fe37ffff640 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559489a88800 0x559489a8ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.767+0000 7fe37ffff640 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559489a88800 0x559489a8ac20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3126992ff58c8d97 server_cookie=33cad083c1548a95 in_seq=0 out_seq=0 2024-08-02T17:51:57.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.767+0000 7fe36e7fc640 10 client.16881 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:57.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe36e7fc640 1 -- 172.21.15.69:0/586015489 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55948996f270 con 0x559489a88800 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe36e7fc640 10 client.16881 handle_command_reply: tid=0 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 client.16881 shutdown 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 -- 172.21.15.69:0/586015489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02b20 msgr2=0x559489a20a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02b20 0x559489a20a40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe37800c710 tx=0x7fe3780039c0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 -- 172.21.15.69:0/586015489 shutdown_connections 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559489a02b20 0x559489a20a40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559489a88800 0x559489a8ac20 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559489a0d3e0 0x559489a20f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 --2- 172.21.15.69:0/586015489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559489a02200 0x559489a20500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe36e7fc640 0 client.16881 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.768+0000 7fe38f70c740 1 -- 172.21.15.69:0/586015489 >> 172.21.15.69:0/586015489 conn(0x5594899fe790 msgr2=0x559489a0ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:57.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38f70c740 1 -- 172.21.15.69:0/586015489 shutdown_connections 2024-08-02T17:51:57.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38f70c740 1 -- 172.21.15.69:0/586015489 wait complete. 2024-08-02T17:51:57.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38f70c740 20 client.16881 trim_cache size 0 max 16384 2024-08-02T17:51:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:51:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:51:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "36943257-5629-47a9-8154-5fdecedc0cb8", 2024-08-02T17:51:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:51:57.775 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 -- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3740a2cf0 msgr2=0x7fe3740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3740a2cf0 0x7fe3740a51b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe380097c10 tx=0x7fe38006e820 comp rx=0 tx=0).stop 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 -- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe38810bea0 msgr2=0x7fe38819c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe38810bea0 0x7fe38819c9d0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe38000eaa0 tx=0x7fe38000ef60 comp rx=0 tx=0).stop 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 -- 172.21.15.69:0/1463482579 shutdown_connections 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe38810bea0 0x7fe38819c9d0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3740a2cf0 0x7fe3740a51b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe388103390 0x7fe388198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 --2- 172.21.15.69:0/1463482579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe388102a70 0x7fe388198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:57.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 -- 172.21.15.69:0/1463482579 >> 172.21.15.69:0/1463482579 conn(0x7fe3880fc4d0 msgr2=0x7fe3880fd170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:57.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.769+0000 7fe38d30e640 1 -- 172.21.15.69:0/1463482579 shutdown_connections 2024-08-02T17:51:57.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:57.770+0000 7fe38d30e640 1 -- 172.21.15.69:0/1463482579 wait complete. 2024-08-02T17:51:58.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:58 smithi083 ceph-mon[37031]: pgmap v169: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 549 op/s 2024-08-02T17:51:58.536 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:58 smithi069 ceph-mon[32746]: pgmap v169: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 549 op/s 2024-08-02T17:51:58.566 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:51:58.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:58 smithi077 ceph-mon[37108]: pgmap v169: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.4 MiB/s wr, 549 op/s 2024-08-02T17:51:58.818 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.286+0000 7fc90b2ac640 1 -- 172.21.15.69:0/1339050312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 msgr2=0x7fc904107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.286+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/1339050312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 0x7fc904107cc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009cd0 tx=0x7fc8f402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.286+0000 7fc90b2ac640 1 -- 172.21.15.69:0/1339050312 shutdown_connections 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.286+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/1339050312 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc904102280 0x7fc90410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.286+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/1339050312 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9041018e0 0x7fc904101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.286+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/1339050312 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 0x7fc904107cc0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.286+0000 7fc90b2ac640 1 -- 172.21.15.69:0/1339050312 >> 172.21.15.69:0/1339050312 conn(0x7fc9040fd600 msgr2=0x7fc9040ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 -- 172.21.15.69:0/1339050312 shutdown_connections 2024-08-02T17:51:59.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 -- 172.21.15.69:0/1339050312 wait complete. 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 Processor -- start 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 -- start start 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9041018e0 0x7fc904074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc904102280 0x7fc904074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 0x7fc904074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc904111ce0 con 0x7fc9041078e0 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc904111b60 con 0x7fc9041018e0 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.287+0000 7fc90b2ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc904111e60 con 0x7fc904102280 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 0x7fc904074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 0x7fc904074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54938/0 (socket says 172.21.15.69:54938) 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 -- 172.21.15.69:0/4041223976 learned_addr learned my addr 172.21.15.69:0/4041223976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:59.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909048640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9041018e0 0x7fc904074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc908847640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc904102280 0x7fc904074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 -- 172.21.15.69:0/4041223976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc904102280 msgr2=0x7fc904074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc904102280 0x7fc904074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 -- 172.21.15.69:0/4041223976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9041018e0 msgr2=0x7fc904074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9041018e0 0x7fc904074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 -- 172.21.15.69:0/4041223976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9041a66e0 con 0x7fc9041078e0 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc908847640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc904102280 0x7fc904074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc909849640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 0x7fc904074ca0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00e970 tx=0x7fc8fc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc9027fc640 1 -- 172.21.15.69:0/4041223976 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc00ccc0 con 0x7fc9041078e0 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc90b2ac640 1 -- 172.21.15.69:0/4041223976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9041a69d0 con 0x7fc9041078e0 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc9027fc640 1 -- 172.21.15.69:0/4041223976 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8fc00ce60 con 0x7fc9041078e0 2024-08-02T17:51:59.309 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc9027fc640 1 -- 172.21.15.69:0/4041223976 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc010490 con 0x7fc9041078e0 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.288+0000 7fc90b2ac640 1 -- 172.21.15.69:0/4041223976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041a6f10 con 0x7fc9041078e0 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.290+0000 7fc9027fc640 1 -- 172.21.15.69:0/4041223976 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc8fc005020 con 0x7fc9041078e0 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc90d6aa740 1 Processor -- start 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc90d6aa740 1 -- start start 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc9027fc640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2cda210 0x560dd2cda5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2ce53f0 0x560dd2ce77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560dd2c3f980 con 0x560dd2cdab30 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560dd2bd7a70 con 0x560dd2cda210 2024-08-02T17:51:59.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560dd2c3b940 con 0x560dd2ce53f0 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc9027fc640 1 -- 172.21.15.69:0/4041223976 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc8fc0cfd40 con 0x7fc9041078e0 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc908847640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc908847640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc908847640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54948/0 (socket says 172.21.15.69:54948) 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.291+0000 7fc908847640 1 -- 172.21.15.69:0/3023611827 learned_addr learned my addr 172.21.15.69:0/3023611827 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc909048640 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2cda210 0x560dd2cda5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc909849640 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2ce53f0 0x560dd2ce77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc908847640 1 -- 172.21.15.69:0/3023611827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2ce53f0 msgr2=0x560dd2ce77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc908847640 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2ce53f0 0x560dd2ce77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc908847640 1 -- 172.21.15.69:0/3023611827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2cda210 msgr2=0x560dd2cda5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc908847640 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2cda210 0x560dd2cda5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc908847640 1 -- 172.21.15.69:0/3023611827 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd2c3a6e0 con 0x560dd2cdab30 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc909048640 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2cda210 0x560dd2cda5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc909849640 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2ce53f0 0x560dd2ce77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc908847640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8012a00 tx=0x7fc8f8009ba0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc908847640 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce4eb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80107a0 tx=0x7fc8f801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b92a9b8853ed6c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc8eb7fe640 1 -- 172.21.15.69:0/3023611827 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8025070 con 0x560dd2cdab30 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc8eb7fe640 1 -- 172.21.15.69:0/3023611827 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f801fc50 con 0x560dd2cdab30 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.292+0000 7fc8eb7fe640 1 -- 172.21.15.69:0/3023611827 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8011730 con 0x560dd2cdab30 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3023611827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 msgr2=0x560dd2ce4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce4eb0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fc8f80107a0 tx=0x7fc8f801c000 comp rx=0 tx=0).stop 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3023611827 shutdown_connections 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce4eb0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2ce53f0 0x560dd2ce77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3023611827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2cda210 0x560dd2cda5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3023611827 >> 172.21.15.69:0/3023611827 conn(0x560dd2cd67a0 msgr2=0x560dd2cd6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3023611827 shutdown_connections 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3023611827 wait complete. 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.293+0000 7fc90d6aa740 1 -- start start 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc90d6aa740 1 Processor -- start 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cda210 0x560dd2ce22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560dd2ce7fc0 con 0x560dd2cdab30 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560dd2bfaaa0 con 0x560dd2ce53f0 2024-08-02T17:51:59.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560dd2ce8140 con 0x560dd2cda210 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc90d6aa740 10 client.0 _ll_register_callbacks cb 0x560dd2be9e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc908847640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc908847640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54956/0 (socket says 172.21.15.69:54956) 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc908847640 1 -- 172.21.15.69:0/3148709463 learned_addr learned my addr 172.21.15.69:0/3148709463 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc909849640 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc908847640 1 -- 172.21.15.69:0/3148709463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cda210 msgr2=0x560dd2ce22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc909048640 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cda210 0x560dd2ce22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc908847640 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cda210 0x560dd2ce22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc908847640 1 -- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 msgr2=0x560dd2ce2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc908847640 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.314 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc908847640 1 -- 172.21.15.69:0/3148709463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd2bde650 con 0x560dd2cdab30 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.294+0000 7fc909849640 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc908847640 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce2810 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8006010 tx=0x7fc8f8011760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7bb1a7ddab0277 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc8e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3148709463 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f8025040 con 0x560dd2cdab30 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3148709463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f801fc50 con 0x560dd2cdab30 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3148709463 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc8f801f940 con 0x560dd2cdab30 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3148709463 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8f8036050 con 0x560dd2cdab30 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 10 client.16893 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3148709463 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560dd29b13c0 con 0x560dd2cdab30 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3148709463 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f801fc50 con 0x560dd2cdab30 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3148709463 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc8f8031030 con 0x560dd2cdab30 2024-08-02T17:51:59.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 10 client.16893 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 10 client.16893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 20 client.16893 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560dd2cf4dd0 0x560dd2cf71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 4 client.16893 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.295+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3148709463 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd2c4cef0 con 0x560dd2cf4dd0 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.296+0000 7fc909048640 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560dd2cf4dd0 0x560dd2cf71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.296+0000 7fc8e8ff9640 10 client.16893 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.296+0000 7fc909048640 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560dd2cf4dd0 0x560dd2cf71c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c075f375fdd5822 server_cookie=81398b034cc707f6 in_seq=0 out_seq=0 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3148709463 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560dd2c4cef0 con 0x560dd2cf4dd0 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc8e8ff9640 10 client.16893 handle_command_reply: tid=0 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 client.16893 shutdown 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3148709463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 msgr2=0x560dd2ce2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce2810 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8006010 tx=0x7fc8f8011760 comp rx=0 tx=0).stop 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3148709463 shutdown_connections 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cdab30 0x560dd2ce2810 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560dd2cf4dd0 0x560dd2cf71c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3148709463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cda210 0x560dd2ce22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc8e8ff9640 0 client.16893 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3148709463 >> 172.21.15.69:0/3148709463 conn(0x560dd2cd67a0 msgr2=0x560dd2cd5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3148709463 shutdown_connections 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3148709463 wait complete. 2024-08-02T17:51:59.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.297+0000 7fc90d6aa740 20 client.16893 trim_cache size 0 max 16384 2024-08-02T17:51:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc90d6aa740 1 Processor -- start 2024-08-02T17:51:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc90d6aa740 1 -- start start 2024-08-02T17:51:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2cfca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cfcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2cdbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041b38d0 con 0x560dd2cda210 2024-08-02T17:51:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041b11a0 con 0x560dd2ce53f0 2024-08-02T17:51:59.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041aeff0 con 0x560dd2cdab30 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc909048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2cfca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc909048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2cfca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54968/0 (socket says 172.21.15.69:54968) 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.314+0000 7fc909048640 1 -- 172.21.15.69:0/3768904935 learned_addr learned my addr 172.21.15.69:0/3768904935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc908847640 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cfcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc909048640 1 -- 172.21.15.69:0/3768904935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 msgr2=0x560dd2cfcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc909048640 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cfcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc909048640 1 -- 172.21.15.69:0/3768904935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 msgr2=0x560dd2cdbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc909048640 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2cdbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc909048640 1 -- 172.21.15.69:0/3768904935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd2cda960 con 0x560dd2cda210 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc908847640 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cfcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc909048640 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2cfca20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009cd0 tx=0x7fc8f40317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5cb4e32d5d7681 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:59.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc8eaffd640 1 -- 172.21.15.69:0/3768904935 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40318c0 con 0x560dd2cda210 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc8eaffd640 1 -- 172.21.15.69:0/3768904935 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4036040 con 0x560dd2cda210 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.315+0000 7fc8eaffd640 1 -- 172.21.15.69:0/3768904935 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4004500 con 0x560dd2cda210 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3768904935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 msgr2=0x560dd2cfca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2cfca20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009cd0 tx=0x7fc8f40317a0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3768904935 shutdown_connections 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2cdbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cfcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3768904935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2cfca20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3768904935 >> 172.21.15.69:0/3768904935 conn(0x560dd2cd67a0 msgr2=0x560dd2ce4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3768904935 shutdown_connections 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3768904935 wait complete. 2024-08-02T17:51:59.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 -- start start 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 Processor -- start 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2ced680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.316+0000 7fc90d6aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041af660 con 0x560dd2cda210 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041b0ad0 con 0x560dd2ce53f0 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc90d6aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041afd30 con 0x560dd2cdab30 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc90d6aa740 10 client.0 _ll_register_callbacks cb 0x7fc8f8023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2ced680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2ced680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:54978/0 (socket says 172.21.15.69:54978) 2024-08-02T17:51:59.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 -- 172.21.15.69:0/3625560538 learned_addr learned my addr 172.21.15.69:0/3625560538 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc908847640 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909849640 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 -- 172.21.15.69:0/3625560538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 msgr2=0x560dd2cedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 -- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 msgr2=0x560dd2ce8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 -- 172.21.15.69:0/3625560538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd2ce9290 con 0x560dd2cda210 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909849640 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc909048640 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2ced680 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4000f80 tx=0x7fc8f4031560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab42784b04c7e38a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc8e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3625560538 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4031bd0 con 0x560dd2cda210 2024-08-02T17:51:59.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3625560538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8f4036040 con 0x560dd2cda210 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.317+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3625560538 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fc8f4003970 con 0x560dd2cda210 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3625560538 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc8f4063050 con 0x560dd2cda210 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc90d6aa740 10 client.16899 fetch_fsmap learned FSMap version 26 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3625560538 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560dd2d5f4f0 con 0x560dd2cda210 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3625560538 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8f4036040 con 0x560dd2cda210 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3625560538 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc8f4030070 con 0x560dd2cda210 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc90d6aa740 10 client.16899 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc90d6aa740 10 client.16899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc90d6aa740 20 client.16899 populate_metadata read hostname 'smithi069' 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560dd2d5fbe0 0x560dd2d62000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc90d6aa740 4 client.16899 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:51:59.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3625560538 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560dd2c47400 con 0x560dd2d5fbe0 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.318+0000 7fc908847640 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560dd2d5fbe0 0x560dd2d62000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc908847640 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560dd2d5fbe0 0x560dd2d62000 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ae3caedb6b28b29 server_cookie=a6794d8831bfb943 in_seq=0 out_seq=0 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc8e8ff9640 10 client.16899 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc8e8ff9640 1 -- 172.21.15.69:0/3625560538 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560dd2c47400 con 0x560dd2d5fbe0 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc8e8ff9640 10 client.16899 handle_command_reply: tid=0 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 client.16899 shutdown 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3625560538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 msgr2=0x560dd2ced680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2ced680 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4000f80 tx=0x7fc8f4031560 comp rx=0 tx=0).stop 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3625560538 shutdown_connections 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560dd2d5fbe0 0x560dd2d62000 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560dd2ce53f0 0x560dd2ce8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560dd2cdab30 0x560dd2cedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc8e8ff9640 0 client.16899 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:51:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 --2- 172.21.15.69:0/3625560538 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560dd2cda210 0x560dd2ced680 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3625560538 >> 172.21.15.69:0/3625560538 conn(0x560dd2cd67a0 msgr2=0x560dd2ce2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3625560538 shutdown_connections 2024-08-02T17:51:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 1 -- 172.21.15.69:0/3625560538 wait complete. 2024-08-02T17:51:59.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.319+0000 7fc90d6aa740 20 client.16899 trim_cache size 0 max 16384 2024-08-02T17:51:59.330 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: "36943257-5629-47a9-8154-5fdecedc0cb8": { 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "36943257-5629-47a9-8154-5fdecedc0cb8", 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:51:59.331 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:51:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 -- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8ec0a2cf0 msgr2=0x7fc8ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8012a00 tx=0x7fc8f8009ba0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 -- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 msgr2=0x7fc904074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:51:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 0x7fc904074ca0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00e970 tx=0x7fc8fc00ed70 comp rx=0 tx=0).stop 2024-08-02T17:51:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 -- 172.21.15.69:0/4041223976 shutdown_connections 2024-08-02T17:51:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc9041078e0 0x7fc904074ca0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc8ec0a2cf0 0x7fc8ec0a51b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc904102280 0x7fc904074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 --2- 172.21.15.69:0/4041223976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc9041018e0 0x7fc904074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:51:59.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 -- 172.21.15.69:0/4041223976 >> 172.21.15.69:0/4041223976 conn(0x7fc9040fd600 msgr2=0x7fc9040fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:51:59.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 -- 172.21.15.69:0/4041223976 shutdown_connections 2024-08-02T17:51:59.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:51:59.320+0000 7fc90b2ac640 1 -- 172.21.15.69:0/4041223976 wait complete. 2024-08-02T17:51:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:59 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:51:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:59 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:59 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:51:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:51:59 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:59.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:59 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:51:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:59 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:59 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:51:59.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:51:59 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:59.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:59 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:51:59.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:59 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:59.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:59 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:51:59.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:51:59 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:51:59.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 290.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:52:00.243 INFO:tasks.cephfs.filesystem:scrub status for tag:36943257-5629-47a9-8154-5fdecedc0cb8 - {'path': '/', 'tag': '36943257-5629-47a9-8154-5fdecedc0cb8', 'options': 'recursive,force'} 2024-08-02T17:52:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:00 smithi083 ceph-mon[37031]: pgmap v170: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.4 MiB/s wr, 320 op/s 2024-08-02T17:52:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:00 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:52:00.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:00 smithi077 ceph-mon[37108]: pgmap v170: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.4 MiB/s wr, 320 op/s 2024-08-02T17:52:00.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:00 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:52:00.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:00 smithi069 ceph-mon[32746]: pgmap v170: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.4 MiB/s wr, 320 op/s 2024-08-02T17:52:00.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:00 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:52:02.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:02 smithi083 ceph-mon[37031]: pgmap v171: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.2 MiB/s wr, 311 op/s 2024-08-02T17:52:02.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:02 smithi077 ceph-mon[37108]: pgmap v171: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.2 MiB/s wr, 311 op/s 2024-08-02T17:52:02.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:02 smithi069 ceph-mon[32746]: pgmap v171: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 5.2 MiB/s wr, 311 op/s 2024-08-02T17:52:04.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:04 smithi077 ceph-mon[37108]: pgmap v172: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.2 MiB/s wr, 395 op/s 2024-08-02T17:52:04.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:04 smithi069 ceph-mon[32746]: pgmap v172: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.2 MiB/s wr, 395 op/s 2024-08-02T17:52:04.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:04 smithi083 ceph-mon[37031]: pgmap v172: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.2 MiB/s wr, 395 op/s 2024-08-02T17:52:04.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 295.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:52:05.811 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:52:06.320 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:06.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:06 smithi077 ceph-mon[37108]: pgmap v173: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 38 KiB/s wr, 113 op/s 2024-08-02T17:52:06.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:06 smithi069 ceph-mon[32746]: pgmap v173: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 38 KiB/s wr, 113 op/s 2024-08-02T17:52:06.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:06 smithi083 ceph-mon[37031]: pgmap v173: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 38 KiB/s wr, 113 op/s 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 -- 172.21.15.69:0/663410799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 msgr2=0x7f06f0100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 --2- 172.21.15.69:0/663410799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 0x7f06f0100c10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009930 tx=0x7f06d802a420 comp rx=0 tx=0).stop 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 -- 172.21.15.69:0/663410799 shutdown_connections 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 --2- 172.21.15.69:0/663410799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 0x7f06f0100c10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 --2- 172.21.15.69:0/663410799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f0101150 0x7f06f010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 --2- 172.21.15.69:0/663410799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06f01067b0 0x7f06f0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 -- 172.21.15.69:0/663410799 >> 172.21.15.69:0/663410799 conn(0x7f06f00fc4d0 msgr2=0x7f06f00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 -- 172.21.15.69:0/663410799 shutdown_connections 2024-08-02T17:52:06.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 -- 172.21.15.69:0/663410799 wait complete. 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 Processor -- start 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.836+0000 7f06f5450640 1 -- start start 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06f5450640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 0x7f06f0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06f5450640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06f0101150 0x7f06f0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06f5450640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f01067b0 0x7f06f019cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06f5450640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0111ce0 con 0x7f06f01007b0 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06f5450640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0111b60 con 0x7f06f0101150 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06f5450640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0111e60 con 0x7f06f01067b0 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 0x7f06f0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:06.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 0x7f06f0198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41092/0 (socket says 172.21.15.69:41092) 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 -- 172.21.15.69:0/416372499 learned_addr learned my addr 172.21.15.69:0/416372499 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06ef7fe640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f01067b0 0x7f06f019cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06ee7fc640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06f0101150 0x7f06f0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 -- 172.21.15.69:0/416372499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f01067b0 msgr2=0x7f06f019cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f01067b0 0x7f06f019cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 -- 172.21.15.69:0/416372499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06f0101150 msgr2=0x7f06f0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06f0101150 0x7f06f0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 -- 172.21.15.69:0/416372499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f019d1b0 con 0x7f06f01007b0 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06ef7fe640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f01067b0 0x7f06f019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06ee7fc640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06f0101150 0x7f06f0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06eeffd640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 0x7f06f0198200 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f06e800d950 tx=0x7f06e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06e7fff640 1 -- 172.21.15.69:0/416372499 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8017070 con 0x7f06f01007b0 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.837+0000 7f06e7fff640 1 -- 172.21.15.69:0/416372499 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8009c30 con 0x7f06f01007b0 2024-08-02T17:52:06.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.838+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f019d4a0 con 0x7f06f01007b0 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.838+0000 7f06e7fff640 1 -- 172.21.15.69:0/416372499 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e800f6b0 con 0x7f06f01007b0 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.838+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f01a4d80 con 0x7f06f01007b0 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.839+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f0100c10 con 0x7f06f01007b0 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.840+0000 7f06e7fff640 1 -- 172.21.15.69:0/416372499 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f06e80109d0 con 0x7f06f01007b0 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.840+0000 7f06e7fff640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06dc0a2d70 0x7f06dc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.840+0000 7f06ee7fc640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06dc0a2d70 0x7f06dc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.840+0000 7f06e7fff640 1 -- 172.21.15.69:0/416372499 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f06e80cf4b0 con 0x7f06f01007b0 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.841+0000 7f06ee7fc640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06dc0a2d70 0x7f06dc0a5230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06d8000c00 tx=0x7f06d8032c70 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:06.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:06.845+0000 7f06e7fff640 1 -- 172.21.15.69:0/416372499 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f06e8009de0 con 0x7f06f01007b0 2024-08-02T17:52:07.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.044+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06f0199560 con 0x7f06f01007b0 2024-08-02T17:52:07.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.045+0000 7f06e7fff640 1 -- 172.21.15.69:0/416372499 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f06e809bd10 con 0x7f06f01007b0 2024-08-02T17:52:07.047 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:52:07.047 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:52:07.047 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:52:07.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06dc0a2d70 msgr2=0x7f06dc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:07.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06dc0a2d70 0x7f06dc0a5230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06d8000c00 tx=0x7f06d8032c70 comp rx=0 tx=0).stop 2024-08-02T17:52:07.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 msgr2=0x7f06f0198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:07.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 0x7f06f0198200 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f06e800d950 tx=0x7f06e800de10 comp rx=0 tx=0).stop 2024-08-02T17:52:07.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 shutdown_connections 2024-08-02T17:52:07.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f06dc0a2d70 0x7f06dc0a5230 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:07.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06f01067b0 0x7f06f019cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:07.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06f0101150 0x7f06f0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:07.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 --2- 172.21.15.69:0/416372499 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06f01007b0 0x7f06f0198200 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:07.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 >> 172.21.15.69:0/416372499 conn(0x7f06f00fc4d0 msgr2=0x7f06f00fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:07.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 shutdown_connections 2024-08-02T17:52:07.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:07.047+0000 7f06f5450640 1 -- 172.21.15.69:0/416372499 wait complete. 2024-08-02T17:52:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:07 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/416372499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:07.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:07 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/416372499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:07.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:07 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/416372499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:08.036 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id f4b08119 --release --await' 2024-08-02T17:52:08.037 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id f4b08119 --release --await 2024-08-02T17:52:08.294 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:08.537 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:08 smithi069 ceph-mon[32746]: pgmap v174: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-08-02T17:52:08.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:08 smithi077 ceph-mon[37108]: pgmap v174: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-08-02T17:52:08.672 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:08 smithi083 ceph-mon[37031]: pgmap v174: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 3.6 MiB/s wr, 297 op/s 2024-08-02T17:52:08.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 -- 172.21.15.69:0/3356858588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac102250 msgr2=0x7f09ac10e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 --2- 172.21.15.69:0/3356858588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac102250 0x7f09ac10e780 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009cd0 tx=0x7f09a402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 -- 172.21.15.69:0/3356858588 shutdown_connections 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 --2- 172.21.15.69:0/3356858588 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac102250 0x7f09ac10e780 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 --2- 172.21.15.69:0/3356858588 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ac1018b0 0x7f09ac101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 --2- 172.21.15.69:0/3356858588 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac1078b0 0x7f09ac107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 -- 172.21.15.69:0/3356858588 >> 172.21.15.69:0/3356858588 conn(0x7f09ac0fd5f0 msgr2=0x7f09ac0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 -- 172.21.15.69:0/3356858588 shutdown_connections 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.757+0000 7f09b3420640 1 -- 172.21.15.69:0/3356858588 wait complete. 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b3420640 1 Processor -- start 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b3420640 1 -- start start 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b3420640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac1018b0 0x7f09ac198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b3420640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ac102250 0x7f09ac198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b3420640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac1078b0 0x7f09ac19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b3420640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac111cb0 con 0x7f09ac1078b0 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b3420640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac111b30 con 0x7f09ac1018b0 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b3420640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac111e30 con 0x7f09ac102250 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b19bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac1078b0 0x7f09ac19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b19bd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac1078b0 0x7f09ac19ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41120/0 (socket says 172.21.15.69:41120) 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b19bd640 1 -- 172.21.15.69:0/4137090765 learned_addr learned my addr 172.21.15.69:0/4137090765 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b09bb640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ac102250 0x7f09ac198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.758+0000 7f09b11bc640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac1018b0 0x7f09ac198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b19bd640 1 -- 172.21.15.69:0/4137090765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ac102250 msgr2=0x7f09ac198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b19bd640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ac102250 0x7f09ac198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b19bd640 1 -- 172.21.15.69:0/4137090765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac1018b0 msgr2=0x7f09ac198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b19bd640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac1018b0 0x7f09ac198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b19bd640 1 -- 172.21.15.69:0/4137090765 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09ac19d100 con 0x7f09ac1078b0 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b09bb640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ac102250 0x7f09ac198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b19bd640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac1078b0 0x7f09ac19ca20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f09a4005cc0 tx=0x7f09a4003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09a27fc640 1 -- 172.21.15.69:0/4137090765 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4004040 con 0x7f09ac1078b0 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09a27fc640 1 -- 172.21.15.69:0/4137090765 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a40046c0 con 0x7f09ac1078b0 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b3420640 1 -- 172.21.15.69:0/4137090765 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09ac19d390 con 0x7f09ac1078b0 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09a27fc640 1 -- 172.21.15.69:0/4137090765 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4046480 con 0x7f09ac1078b0 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.759+0000 7f09b3420640 1 -- 172.21.15.69:0/4137090765 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09ac1a4cd0 con 0x7f09ac1078b0 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.761+0000 7f09a27fc640 1 -- 172.21.15.69:0/4137090765 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09a40305a0 con 0x7f09ac1078b0 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b581e740 1 Processor -- start 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b581e740 1 -- start start 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b581e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990b9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b581e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990ba2b0 0x55e5990c4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b581e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 0x55e5990c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.776 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b581e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e59901fc60 con 0x55e5990ba2b0 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b581e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e598fb7b90 con 0x55e5990b9990 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b581e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e59901bc20 con 0x55e5990c4b70 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b11bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990ba2b0 0x55e5990c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09a27fc640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09980a2cf0 0x7f09980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b11bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990ba2b0 0x55e5990c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41124/0 (socket says 172.21.15.69:41124) 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.762+0000 7f09b11bc640 1 -- 172.21.15.69:0/371895291 learned_addr learned my addr 172.21.15.69:0/371895291 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b09bb640 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 0x55e5990c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b09bb640 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990b9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b19bd640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09980a2cf0 0x7f09980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b11bc640 1 -- 172.21.15.69:0/371895291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 msgr2=0x55e5990c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b11bc640 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 0x55e5990c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09a27fc640 1 -- 172.21.15.69:0/4137090765 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f09a402e030 con 0x7f09ac1078b0 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b11bc640 1 -- 172.21.15.69:0/371895291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 msgr2=0x55e5990b9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:08.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b11bc640 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990b9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b11bc640 1 -- 172.21.15.69:0/371895291 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e598d2f580 con 0x55e5990ba2b0 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b11bc640 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990ba2b0 0x55e5990c4630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cb80 tx=0x7f09a800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21da61c209272e1a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09977fe640 1 -- 172.21.15.69:0/371895291 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8010630 con 0x55e5990ba2b0 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09b19bd640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09980a2cf0 0x7f09980a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09a4006010 tx=0x7f09a4043000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09977fe640 1 -- 172.21.15.69:0/371895291 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a8010cb0 con 0x55e5990ba2b0 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.763+0000 7f09977fe640 1 -- 172.21.15.69:0/371895291 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a800fbb0 con 0x55e5990ba2b0 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 -- 172.21.15.69:0/371895291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990ba2b0 msgr2=0x55e5990c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990ba2b0 0x55e5990c4630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cb80 tx=0x7f09a800e720 comp rx=0 tx=0).stop 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 -- 172.21.15.69:0/371895291 shutdown_connections 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990ba2b0 0x55e5990c4630 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 0x55e5990c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 --2- 172.21.15.69:0/371895291 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990b9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 -- 172.21.15.69:0/371895291 >> 172.21.15.69:0/371895291 conn(0x55e5990b5f20 msgr2=0x55e5990b6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 -- 172.21.15.69:0/371895291 shutdown_connections 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 -- 172.21.15.69:0/371895291 wait complete. 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.764+0000 7f09b581e740 1 -- start start 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b581e740 1 Processor -- start 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b581e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990c19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b581e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990c1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b581e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990c2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b581e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e5990c7740 con 0x55e5990c4b70 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b581e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e598fbd7c0 con 0x55e5990b9990 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b581e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5990c78c0 con 0x55e5990ba2b0 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b581e740 10 client.0 _ll_register_callbacks cb 0x55e598fc6f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990c2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41128/0 (socket says 172.21.15.69:41128) 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 -- 172.21.15.69:0/1433788938 learned_addr learned my addr 172.21.15.69:0/1433788938 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b11bc640 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990c1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990c19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 -- 172.21.15.69:0/1433788938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 msgr2=0x55e5990c1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990c1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 -- 172.21.15.69:0/1433788938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 msgr2=0x55e5990c19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990c19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.765+0000 7f09b09bb640 1 -- 172.21.15.69:0/1433788938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e598df9960 con 0x55e5990c4b70 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b09bb640 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990c2450 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f099c012ea0 tx=0x7f099c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e107acb4f99c02a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f0994ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f0994ff9640 1 -- 172.21.15.69:0/1433788938 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c00ce20 con 0x55e5990c4b70 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f0994ff9640 1 -- 172.21.15.69:0/1433788938 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f099c01f410 con 0x55e5990c4b70 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 1 -- 172.21.15.69:0/1433788938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a8010cb0 con 0x55e5990c4b70 2024-08-02T17:52:08.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f0994ff9640 1 -- 172.21.15.69:0/1433788938 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f099c027050 con 0x55e5990c4b70 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 10 client.16917 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 1 -- 172.21.15.69:0/1433788938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5990d37e0 con 0x55e5990c4b70 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f0994ff9640 1 -- 172.21.15.69:0/1433788938 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a8010cb0 con 0x55e5990c4b70 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f0994ff9640 1 -- 172.21.15.69:0/1433788938 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f099c011710 con 0x55e5990c4b70 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 10 client.16917 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 10 client.16917 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 20 client.16917 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e5990d4bf0 0x55e5990d7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 4 client.16917 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.766+0000 7f09b581e740 1 -- 172.21.15.69:0/1433788938 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e59902d1d0 con 0x55e5990d4bf0 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.767+0000 7f09b11bc640 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e5990d4bf0 0x55e5990d7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.767+0000 7f09b11bc640 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e5990d4bf0 0x55e5990d7050 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd803cf9e66a792 server_cookie=89caa80b8894ceb5 in_seq=0 out_seq=0 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.767+0000 7f0994ff9640 10 client.16917 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:08.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f0994ff9640 1 -- 172.21.15.69:0/1433788938 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e59902d1d0 con 0x55e5990d4bf0 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f0994ff9640 10 client.16917 handle_command_reply: tid=0 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 client.16917 shutdown 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 -- 172.21.15.69:0/1433788938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 msgr2=0x55e5990c2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990c2450 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f099c012ea0 tx=0x7f099c004730 comp rx=0 tx=0).stop 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 -- 172.21.15.69:0/1433788938 shutdown_connections 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990c2450 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e5990d4bf0 0x55e5990d7050 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990c1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 --2- 172.21.15.69:0/1433788938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990c19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f0994ff9640 0 client.16917 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 -- 172.21.15.69:0/1433788938 >> 172.21.15.69:0/1433788938 conn(0x55e5990b5f20 msgr2=0x55e5990b5080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 -- 172.21.15.69:0/1433788938 shutdown_connections 2024-08-02T17:52:08.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 1 -- 172.21.15.69:0/1433788938 wait complete. 2024-08-02T17:52:08.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.768+0000 7f09b581e740 20 client.16917 trim_cache size 0 max 16384 2024-08-02T17:52:08.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b581e740 1 Processor -- start 2024-08-02T17:52:08.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b581e740 1 -- start start 2024-08-02T17:52:08.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b581e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990bc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b581e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b581e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990dc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b581e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1b15e0 con 0x55e5990c4b70 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b581e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1aeeb0 con 0x55e5990b9990 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b581e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1acd00 con 0x55e5990ba2b0 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990dc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990dc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41138/0 (socket says 172.21.15.69:41138) 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 -- 172.21.15.69:0/4034456087 learned_addr learned my addr 172.21.15.69:0/4034456087 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990bc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 -- 172.21.15.69:0/4034456087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 msgr2=0x55e5990b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b11bc640 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990b4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 -- 172.21.15.69:0/4034456087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 msgr2=0x55e5990bc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:08.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990bc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 -- 172.21.15.69:0/4034456087 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5990b59c0 con 0x55e5990c4b70 2024-08-02T17:52:08.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b11bc640 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:08.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.785+0000 7f09b09bb640 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990dc970 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f099c011cd0 tx=0x7f099c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d5bbdf26ef253f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:08.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f0996ffd640 1 -- 172.21.15.69:0/4034456087 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c01fe60 con 0x55e5990c4b70 2024-08-02T17:52:08.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f0996ffd640 1 -- 172.21.15.69:0/4034456087 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c030020 con 0x55e5990c4b70 2024-08-02T17:52:08.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f0996ffd640 1 -- 172.21.15.69:0/4034456087 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c01f410 con 0x55e5990c4b70 2024-08-02T17:52:08.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f09b581e740 1 -- 172.21.15.69:0/4034456087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 msgr2=0x55e5990dc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f09b581e740 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990dc970 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f099c011cd0 tx=0x7f099c001e50 comp rx=0 tx=0).stop 2024-08-02T17:52:08.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f09b581e740 1 -- 172.21.15.69:0/4034456087 shutdown_connections 2024-08-02T17:52:08.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f09b581e740 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990c4b70 0x55e5990dc970 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f09b581e740 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990ba2b0 0x55e5990b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f09b581e740 1 --2- 172.21.15.69:0/4034456087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990b9990 0x55e5990bc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.786+0000 7f09b581e740 1 -- 172.21.15.69:0/4034456087 >> 172.21.15.69:0/4034456087 conn(0x55e5990b5f20 msgr2=0x55e5990b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 -- 172.21.15.69:0/4034456087 shutdown_connections 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 -- 172.21.15.69:0/4034456087 wait complete. 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 -- start start 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 Processor -- start 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990b9990 0x55e5990cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990ba2b0 0x55e5990cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 0x55e5990c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1ad370 con 0x55e5990b9990 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1ae7e0 con 0x55e5990ba2b0 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f09ac1ada40 con 0x55e5990c4b70 2024-08-02T17:52:08.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b581e740 10 client.0 _ll_register_callbacks cb 0x7f099c012a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.787+0000 7f09b09bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990b9990 0x55e5990cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990b9990 0x55e5990cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41144/0 (socket says 172.21.15.69:41144) 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 -- 172.21.15.69:0/354910705 learned_addr learned my addr 172.21.15.69:0/354910705 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b11bc640 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990ba2b0 0x55e5990cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 0x55e5990c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 -- 172.21.15.69:0/354910705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 msgr2=0x55e5990c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 0x55e5990c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 -- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990ba2b0 msgr2=0x55e5990cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990ba2b0 0x55e5990cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 -- 172.21.15.69:0/354910705 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5990ca150 con 0x55e5990b9990 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b11bc640 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990ba2b0 0x55e5990cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b09bb640 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990b9990 0x55e5990cccf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f099c00c5b0 tx=0x7f099c005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c561a97287f3ed8d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:08.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f0994ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f0994ff9640 1 -- 172.21.15.69:0/354910705 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c011440 con 0x55e5990b9990 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f0994ff9640 1 -- 172.21.15.69:0/354910705 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f099c0115e0 con 0x55e5990b9990 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.788+0000 7f09b581e740 1 -- 172.21.15.69:0/354910705 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099c030020 con 0x55e5990b9990 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f0994ff9640 1 -- 172.21.15.69:0/354910705 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f099c0117e0 con 0x55e5990b9990 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b581e740 10 client.16923 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b581e740 1 -- 172.21.15.69:0/354910705 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e599140240 con 0x55e5990b9990 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f0994ff9640 1 -- 172.21.15.69:0/354910705 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f099c030020 con 0x55e5990b9990 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f0994ff9640 1 -- 172.21.15.69:0/354910705 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f099c0049f0 con 0x55e5990b9990 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b581e740 10 client.16923 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b581e740 10 client.16923 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b581e740 20 client.16923 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b581e740 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e599140950 0x55e599142d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b581e740 4 client.16923 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f4b08119", "release": true, "await": true}] 2024-08-02T17:52:08.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b581e740 1 -- 172.21.15.69:0/354910705 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f4b08119", "release": true, "await": true}) -- 0x55e5990b11d0 con 0x55e599140950 2024-08-02T17:52:08.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.789+0000 7f09b11bc640 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e599140950 0x55e599142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:08.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.790+0000 7f09b11bc640 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e599140950 0x55e599142d40 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59326dcb16c3ffc5 server_cookie=a089408c23b6fe54 in_seq=0 out_seq=0 2024-08-02T17:52:08.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.790+0000 7f0994ff9640 10 client.16923 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:08.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f0994ff9640 1 -- 172.21.15.69:0/354910705 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e5990b11d0 con 0x55e599140950 2024-08-02T17:52:08.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f0994ff9640 10 client.16923 handle_command_reply: tid=0 2024-08-02T17:52:08.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 client.16923 shutdown 2024-08-02T17:52:08.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 -- 172.21.15.69:0/354910705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990b9990 msgr2=0x55e5990cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990b9990 0x55e5990cccf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f099c00c5b0 tx=0x7f099c005fb0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 -- 172.21.15.69:0/354910705 shutdown_connections 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e599140950 0x55e599142d40 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5990c4b70 0x55e5990c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5990ba2b0 0x55e5990cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f0994ff9640 0 client.16923 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 --2- 172.21.15.69:0/354910705 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5990b9990 0x55e5990cccf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 -- 172.21.15.69:0/354910705 >> 172.21.15.69:0/354910705 conn(0x55e5990b5f20 msgr2=0x55e5990c21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 -- 172.21.15.69:0/354910705 shutdown_connections 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 1 -- 172.21.15.69:0/354910705 wait complete. 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.838+0000 7f09b581e740 20 client.16923 trim_cache size 0 max 16384 2024-08-02T17:52:08.841 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:52:08.842 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:52:08.842 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:52:08.842 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 20, 2024-08-02T17:52:08.842 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:52:08.842 INFO:teuthology.orchestra.run.smithi069.stdout: "f4b08119": { 2024-08-02T17:52:08.842 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 20, 2024-08-02T17:52:08.842 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 148.9, 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:08.843 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:08.844 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:08.844 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:08.844 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:08.844 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:52:08.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 -- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09980a2cf0 msgr2=0x7f09980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09980a2cf0 0x7f09980a51b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f09a4006010 tx=0x7f09a4043000 comp rx=0 tx=0).stop 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 -- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac1078b0 msgr2=0x7f09ac19ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac1078b0 0x7f09ac19ca20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f09a4005cc0 tx=0x7f09a4003970 comp rx=0 tx=0).stop 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 -- 172.21.15.69:0/4137090765 shutdown_connections 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f09ac1078b0 0x7f09ac19ca20 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f09980a2cf0 0x7f09980a51b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f09ac102250 0x7f09ac198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 --2- 172.21.15.69:0/4137090765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f09ac1018b0 0x7f09ac198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.839+0000 7f09b3420640 1 -- 172.21.15.69:0/4137090765 >> 172.21.15.69:0/4137090765 conn(0x7f09ac0fd5f0 msgr2=0x7f09ac0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.840+0000 7f09b3420640 1 -- 172.21.15.69:0/4137090765 shutdown_connections 2024-08-02T17:52:08.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:08.840+0000 7f09b3420640 1 -- 172.21.15.69:0/4137090765 wait complete. 2024-08-02T17:52:09.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.369+0000 7f1acda87640 1 -- 172.21.15.69:0/3139695313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac81010c0 msgr2=0x7f1ac810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:09.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.369+0000 7f1acda87640 1 --2- 172.21.15.69:0/3139695313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac81010c0 0x7f1ac810d5f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009ca0 tx=0x7f1ac002a8f0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.369+0000 7f1acda87640 1 -- 172.21.15.69:0/3139695313 shutdown_connections 2024-08-02T17:52:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.369+0000 7f1acda87640 1 --2- 172.21.15.69:0/3139695313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac81010c0 0x7f1ac810d5f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.369+0000 7f1acda87640 1 --2- 172.21.15.69:0/3139695313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8100720 0x7f1ac8100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.369+0000 7f1acda87640 1 --2- 172.21.15.69:0/3139695313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac8106730 0x7f1ac8106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.369+0000 7f1acda87640 1 -- 172.21.15.69:0/3139695313 >> 172.21.15.69:0/3139695313 conn(0x7f1ac80fc3e0 msgr2=0x7f1ac80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:09.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.378+0000 7f1acda87640 1 -- 172.21.15.69:0/3139695313 shutdown_connections 2024-08-02T17:52:09.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.380+0000 7f1acda87640 1 -- 172.21.15.69:0/3139695313 wait complete. 2024-08-02T17:52:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.380+0000 7f1acda87640 1 Processor -- start 2024-08-02T17:52:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.381+0000 7f1acda87640 1 -- start start 2024-08-02T17:52:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.381+0000 7f1acda87640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8100720 0x7f1ac8116380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.381+0000 7f1acda87640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac81010c0 0x7f1ac81168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.381+0000 7f1acda87640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac8106730 0x7f1ac811c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.381+0000 7f1acda87640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810faf0 con 0x7f1ac8106730 2024-08-02T17:52:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.381+0000 7f1acda87640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810f970 con 0x7f1ac81010c0 2024-08-02T17:52:09.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.381+0000 7f1acda87640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac810fc70 con 0x7f1ac8100720 2024-08-02T17:52:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.383+0000 7f1ac7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac8106730 0x7f1ac811c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.383+0000 7f1ac7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac8106730 0x7f1ac811c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41162/0 (socket says 172.21.15.69:41162) 2024-08-02T17:52:09.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.383+0000 7f1ac7fff640 1 -- 172.21.15.69:0/778476438 learned_addr learned my addr 172.21.15.69:0/778476438 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.386+0000 7f1ac7fff640 1 -- 172.21.15.69:0/778476438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8100720 msgr2=0x7f1ac8116380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:52:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.386+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8100720 0x7f1ac8116380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.386+0000 7f1ac7fff640 1 -- 172.21.15.69:0/778476438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac81010c0 msgr2=0x7f1ac81168c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:52:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.386+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac81010c0 0x7f1ac81168c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.386+0000 7f1ac7fff640 1 -- 172.21.15.69:0/778476438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac8111490 con 0x7f1ac8106730 2024-08-02T17:52:09.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.386+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac8106730 0x7f1ac811c090 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00097b0 tx=0x7f1ac00316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.387+0000 7f1ac4ff9640 1 -- 172.21.15.69:0/778476438 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0031850 con 0x7f1ac8106730 2024-08-02T17:52:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.387+0000 7f1acda87640 1 -- 172.21.15.69:0/778476438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac8111720 con 0x7f1ac8106730 2024-08-02T17:52:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.387+0000 7f1acda87640 1 -- 172.21.15.69:0/778476438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac8111c70 con 0x7f1ac8106730 2024-08-02T17:52:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.387+0000 7f1ac4ff9640 1 -- 172.21.15.69:0/778476438 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac002d070 con 0x7f1ac8106730 2024-08-02T17:52:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.388+0000 7f1ac4ff9640 1 -- 172.21.15.69:0/778476438 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac002fc30 con 0x7f1ac8106730 2024-08-02T17:52:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.396+0000 7f1ac4ff9640 1 -- 172.21.15.69:0/778476438 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1ac0004040 con 0x7f1ac8106730 2024-08-02T17:52:09.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.396+0000 7f1ad07d7740 1 Processor -- start 2024-08-02T17:52:09.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.396+0000 7f1ad07d7740 1 -- start start 2024-08-02T17:52:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.400+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715daa7b0 0x55c715daab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.400+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715dab0d0 0x55c715db5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.403+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 0x55c715db7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.403+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c715d0e9b0 con 0x55c715db5970 2024-08-02T17:52:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.403+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c715cb6800 con 0x55c715daa7b0 2024-08-02T17:52:09.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.403+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c715d0a970 con 0x55c715dab0d0 2024-08-02T17:52:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.402+0000 7f1ac77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715dab0d0 0x55c715db5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.409+0000 7f1ac77fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715dab0d0 0x55c715db5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45870/0 (socket says 172.21.15.69:45870) 2024-08-02T17:52:09.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.409+0000 7f1ac77fe640 1 -- 172.21.15.69:0/1653840320 learned_addr learned my addr 172.21.15.69:0/1653840320 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:09.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.409+0000 7f1ac6ffd640 1 --2- 172.21.15.69:0/1653840320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715daa7b0 0x55c715daab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.411+0000 7f1ac77fe640 1 -- 172.21.15.69:0/1653840320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715daa7b0 msgr2=0x55c715daab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.411+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/1653840320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715daa7b0 0x55c715daab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.411+0000 7f1ac77fe640 1 -- 172.21.15.69:0/1653840320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 msgr2=0x55c715db7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.411+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/1653840320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 0x55c715db7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.411+0000 7f1ac77fe640 1 -- 172.21.15.69:0/1653840320 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c715a22210 con 0x55c715dab0d0 2024-08-02T17:52:09.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.411+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/1653840320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715dab0d0 0x55c715db5430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009780 tx=0x7f1ab8009c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc63403f7e948368 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.412+0000 7f1ac4ff9640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1ab00a2f00 0x7f1ab00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.412+0000 7f1ac4ff9640 1 -- 172.21.15.69:0/778476438 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f1ac00f0d70 con 0x7f1ac8106730 2024-08-02T17:52:09.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.413+0000 7f1ac6ffd640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1ab00a2f00 0x7f1ab00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.417+0000 7f1aadffb640 1 -- 172.21.15.69:0/1653840320 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8015020 con 0x55c715dab0d0 2024-08-02T17:52:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.417+0000 7f1aadffb640 1 -- 172.21.15.69:0/1653840320 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab8011420 con 0x55c715dab0d0 2024-08-02T17:52:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.417+0000 7f1aadffb640 1 -- 172.21.15.69:0/1653840320 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80105c0 con 0x55c715dab0d0 2024-08-02T17:52:09.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.417+0000 7f1ac6ffd640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1ab00a2f00 0x7f1ab00a53c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55c715da80a0 tx=0x7f1abc008040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.418+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1653840320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715dab0d0 msgr2=0x55c715db5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:09.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.418+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1653840320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715dab0d0 0x55c715db5430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009780 tx=0x7f1ab8009c40 comp rx=0 tx=0).stop 2024-08-02T17:52:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.419+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1653840320 shutdown_connections 2024-08-02T17:52:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.419+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1653840320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 0x55c715db7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.419+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1653840320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715dab0d0 0x55c715db5430 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009780 tx=0x7f1ab8009c40 comp rx=0 tx=0).stop 2024-08-02T17:52:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.419+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1653840320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715daa7b0 0x55c715daab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.419+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1653840320 >> 172.21.15.69:0/1653840320 conn(0x55c715da6d20 msgr2=0x55c715da7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:09.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.424+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1653840320 shutdown_connections 2024-08-02T17:52:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.425+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1653840320 wait complete. 2024-08-02T17:52:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.425+0000 7f1ad07d7740 1 -- start start 2024-08-02T17:52:09.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.425+0000 7f1ad07d7740 1 Processor -- start 2024-08-02T17:52:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.426+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715daa7b0 0x55c715db29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.426+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715db2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.426+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715db3450 0x55c715dd2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.426+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c715db8540 con 0x55c715daa7b0 2024-08-02T17:52:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.426+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c715cca500 con 0x55c715db5970 2024-08-02T17:52:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.426+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c715db86c0 con 0x55c715db3450 2024-08-02T17:52:09.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.426+0000 7f1ad07d7740 10 client.0 _ll_register_callbacks cb 0x55c715a21a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:09.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.430+0000 7f1ac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715db3450 0x55c715dd2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.430+0000 7f1ac7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715db3450 0x55c715dd2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45874/0 (socket says 172.21.15.69:45874) 2024-08-02T17:52:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.430+0000 7f1ac7fff640 1 -- 172.21.15.69:0/733845507 learned_addr learned my addr 172.21.15.69:0/733845507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:09.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.430+0000 7f1ac6ffd640 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715db2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.431+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715daa7b0 0x55c715db29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.438 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.436+0000 7f1ac7fff640 1 -- 172.21.15.69:0/733845507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 msgr2=0x55c715db2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.436+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715db2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.436+0000 7f1ac7fff640 1 -- 172.21.15.69:0/733845507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715daa7b0 msgr2=0x55c715db29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.436+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715daa7b0 0x55c715db29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.436+0000 7f1ac7fff640 1 -- 172.21.15.69:0/733845507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c715aee620 con 0x55c715db3450 2024-08-02T17:52:09.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.441+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715db3450 0x55c715dd2a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00b9950 tx=0x7f1ac0031ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d516724794fb0a3b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:09.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.442+0000 7f1aa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:52:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.446+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/733845507 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00bcf80 con 0x55c715db3450 2024-08-02T17:52:09.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.446+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/733845507 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f1ac003b410 con 0x55c715db3450 2024-08-02T17:52:09.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.448+0000 7f1ad07d7740 1 -- 172.21.15.69:0/733845507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab8011420 con 0x55c715db3450 2024-08-02T17:52:09.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.458+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/733845507 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ac00c1330 con 0x55c715db3450 2024-08-02T17:52:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.464+0000 7f1ad07d7740 10 client.26368 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.464+0000 7f1ad07d7740 1 -- 172.21.15.69:0/733845507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c715db8540 con 0x55c715db3450 2024-08-02T17:52:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.467+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/733845507 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab8011420 con 0x55c715db3450 2024-08-02T17:52:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.467+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/733845507 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1ac00c5760 con 0x55c715db3450 2024-08-02T17:52:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.471+0000 7f1ad07d7740 10 client.26368 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.471+0000 7f1ad07d7740 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.471+0000 7f1ad07d7740 20 client.26368 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.471+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c715dd6c40 0x55c715dd90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.471+0000 7f1ad07d7740 4 client.26368 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.471+0000 7f1ad07d7740 1 -- 172.21.15.69:0/733845507 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c715d1c0e0 con 0x55c715dd6c40 2024-08-02T17:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.471+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c715dd6c40 0x55c715dd90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.472+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c715dd6c40 0x55c715dd90a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62210de8c7adab9b server_cookie=2bf5c77e7a221675 in_seq=0 out_seq=0 2024-08-02T17:52:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.472+0000 7f1aa6ffd640 10 client.26368 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.475+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/733845507 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c715d1c0e0 con 0x55c715dd6c40 2024-08-02T17:52:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.475+0000 7f1aa6ffd640 10 client.26368 handle_command_reply: tid=0 2024-08-02T17:52:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.475+0000 7f1ad07d7740 1 client.26368 shutdown 2024-08-02T17:52:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.482+0000 7f1ad07d7740 1 -- 172.21.15.69:0/733845507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715db3450 msgr2=0x55c715dd2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.482+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715db3450 0x55c715dd2a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00b9950 tx=0x7f1ac0031ab0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.484+0000 7f1ad07d7740 1 -- 172.21.15.69:0/733845507 shutdown_connections 2024-08-02T17:52:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.484+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c715dd6c40 0x55c715dd90a0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.484+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715db3450 0x55c715dd2a90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.484+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715db2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.484+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/733845507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715daa7b0 0x55c715db29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.484+0000 7f1ad07d7740 1 -- 172.21.15.69:0/733845507 >> 172.21.15.69:0/733845507 conn(0x55c715da6d20 msgr2=0x55c715daca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.484+0000 7f1aa6ffd640 0 client.26368 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:09.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.487+0000 7f1ad07d7740 1 -- 172.21.15.69:0/733845507 shutdown_connections 2024-08-02T17:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.488+0000 7f1ad07d7740 1 -- 172.21.15.69:0/733845507 wait complete. 2024-08-02T17:52:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.489+0000 7f1ad07d7740 20 client.26368 trim_cache size 0 max 16384 2024-08-02T17:52:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.528+0000 7f1ad07d7740 1 Processor -- start 2024-08-02T17:52:09.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.528+0000 7f1ad07d7740 1 -- start start 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.528+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dd2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.528+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db3450 0x55c715db4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.528+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715dde9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.528+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4012880 con 0x55c715db3450 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.528+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4010150 con 0x55c715db5970 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.528+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab400dfa0 con 0x55c715daa7b0 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.529+0000 7f1ac6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db3450 0x55c715db4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.529+0000 7f1ac7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715dde9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.529+0000 7f1ac7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715dde9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:53250/0 (socket says 172.21.15.69:53250) 2024-08-02T17:52:09.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.529+0000 7f1ac7fff640 1 -- 172.21.15.69:0/482068372 learned_addr learned my addr 172.21.15.69:0/482068372 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:09.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.532+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dd2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.534+0000 7f1ac7fff640 1 -- 172.21.15.69:0/482068372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 msgr2=0x55c715dd2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.534+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dd2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.534+0000 7f1ac7fff640 1 -- 172.21.15.69:0/482068372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db3450 msgr2=0x55c715db4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.534+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db3450 0x55c715db4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.534+0000 7f1ac7fff640 1 -- 172.21.15.69:0/482068372 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c715db3280 con 0x55c715db5970 2024-08-02T17:52:09.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.536+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dd2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:09.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.540+0000 7f1ac7fff640 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715dde9b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00bcc80 tx=0x7f1ac0031b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9151c304d8704f0d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.542+0000 7f1aad7fa640 1 -- 172.21.15.69:0/482068372 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00bcf80 con 0x55c715db5970 2024-08-02T17:52:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.543+0000 7f1aad7fa640 1 -- 172.21.15.69:0/482068372 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac00d7060 con 0x55c715db5970 2024-08-02T17:52:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.543+0000 7f1aad7fa640 1 -- 172.21.15.69:0/482068372 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00b9710 con 0x55c715db5970 2024-08-02T17:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.543+0000 7f1ad07d7740 1 -- 172.21.15.69:0/482068372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 msgr2=0x55c715dde9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.543+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715dde9b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00bcc80 tx=0x7f1ac0031b20 comp rx=0 tx=0).stop 2024-08-02T17:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.544+0000 7f1ad07d7740 1 -- 172.21.15.69:0/482068372 shutdown_connections 2024-08-02T17:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.544+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db3450 0x55c715db4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.544+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db5970 0x55c715dde9b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.544+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/482068372 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dd2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.544+0000 7f1ad07d7740 1 -- 172.21.15.69:0/482068372 >> 172.21.15.69:0/482068372 conn(0x55c715da6d20 msgr2=0x55c715db4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.547+0000 7f1ad07d7740 1 -- 172.21.15.69:0/482068372 shutdown_connections 2024-08-02T17:52:09.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.549+0000 7f1ad07d7740 1 -- 172.21.15.69:0/482068372 wait complete. 2024-08-02T17:52:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.549+0000 7f1ad07d7740 1 -- start start 2024-08-02T17:52:09.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.549+0000 7f1ad07d7740 1 Processor -- start 2024-08-02T17:52:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.550+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dafa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.550+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db3450 0x55c715daff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.550+0000 7f1ad07d7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 0x55c715db04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.550+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab400e610 con 0x55c715db5970 2024-08-02T17:52:09.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.550+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab400fa80 con 0x55c715db3450 2024-08-02T17:52:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.550+0000 7f1ad07d7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab400ece0 con 0x55c715daa7b0 2024-08-02T17:52:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.550+0000 7f1ad07d7740 10 client.0 _ll_register_callbacks cb 0x7f1ac002f680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 0x55c715db04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 0x55c715db04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41208/0 (socket says 172.21.15.69:41208) 2024-08-02T17:52:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac7fff640 1 -- 172.21.15.69:0/1695416476 learned_addr learned my addr 172.21.15.69:0/1695416476 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac6ffd640 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db3450 0x55c715daff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dafa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac6ffd640 1 -- 172.21.15.69:0/1695416476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 msgr2=0x55c715dafa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac6ffd640 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dafa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac6ffd640 1 -- 172.21.15.69:0/1695416476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 msgr2=0x55c715db04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac6ffd640 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 0x55c715db04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac6ffd640 1 -- 172.21.15.69:0/1695416476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c715e40a30 con 0x55c715db3450 2024-08-02T17:52:09.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.551+0000 7f1ac6ffd640 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db3450 0x55c715daff80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006fd0 tx=0x7f1abc004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e087f7c5fb1b6809 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.552+0000 7f1aa6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:52:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.552+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/1695416476 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc025070 con 0x55c715db3450 2024-08-02T17:52:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.552+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/1695416476 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f1abc004020 con 0x55c715db3450 2024-08-02T17:52:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.552+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1695416476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac00d7060 con 0x55c715db3450 2024-08-02T17:52:09.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.552+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/1695416476 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1abc030050 con 0x55c715db3450 2024-08-02T17:52:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.554+0000 7f1ad07d7740 10 client.26492 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.554+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1695416476 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c715e41d80 con 0x55c715db3450 2024-08-02T17:52:09.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.555+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/1695416476 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ac00d7060 con 0x55c715db3450 2024-08-02T17:52:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.556+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/1695416476 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1abc021650 con 0x55c715db3450 2024-08-02T17:52:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.557+0000 7f1ad07d7740 10 client.26492 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.557+0000 7f1ad07d7740 10 client.26492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.557+0000 7f1ad07d7740 20 client.26492 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.557+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c715e424b0 0x55c715e448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.557+0000 7f1ad07d7740 4 client.26492 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:52:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.557+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1695416476 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c715d16430 con 0x55c715e424b0 2024-08-02T17:52:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.557+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c715e424b0 0x55c715e448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.561+0000 7f1ac77fe640 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c715e424b0 0x55c715e448d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee1dde491d65627c server_cookie=72bbaf703508c44e in_seq=0 out_seq=0 2024-08-02T17:52:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:09.561+0000 7f1aa6ffd640 10 client.26492 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:09.685 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f4b08119', total seconds elapsed: 32.8 2024-08-02T17:52:09.686 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 28.9 2024-08-02T17:52:09.686 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:52:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 300.0 s, 26.2 tps, lat 13607.859 ms stddev 9985.739 2024-08-02T17:52:10.216 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:10.581 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:10 smithi069 ceph-mon[32746]: pgmap v175: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.0 MiB/s wr, 324 op/s 2024-08-02T17:52:10.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:10 smithi077 ceph-mon[37108]: pgmap v175: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.0 MiB/s wr, 324 op/s 2024-08-02T17:52:10.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:10 smithi083 ceph-mon[37031]: pgmap v175: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.0 MiB/s wr, 324 op/s 2024-08-02T17:52:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.106+0000 7f2cc70e3640 1 -- 172.21.15.69:0/1860679534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80a4820 msgr2=0x7f2cb80a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.106+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/1860679534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80a4820 0x7f2cb80a4c60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009370 tx=0x7f2cb002e9c0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.106+0000 7f2cc70e3640 1 -- 172.21.15.69:0/1860679534 shutdown_connections 2024-08-02T17:52:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.106+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/1860679534 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb80a51a0 0x7f2cb80b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.106+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/1860679534 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80a4820 0x7f2cb80a4c60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.106+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/1860679534 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cb80aa810 0x7f2cb80aabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.106+0000 7f2cc70e3640 1 -- 172.21.15.69:0/1860679534 >> 172.21.15.69:0/1860679534 conn(0x7f2cb801a510 msgr2=0x7f2cb801a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:11.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.109+0000 7f2cc70e3640 1 -- 172.21.15.69:0/1860679534 shutdown_connections 2024-08-02T17:52:11.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.109+0000 7f2cc70e3640 1 -- 172.21.15.69:0/1860679534 wait complete. 2024-08-02T17:52:11.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.110+0000 7f2cc70e3640 1 Processor -- start 2024-08-02T17:52:11.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.110+0000 7f2cc70e3640 1 -- start start 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.110+0000 7f2cc70e3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cb80a51a0 0x7f2cb813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.110+0000 7f2cc70e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80aa810 0x7f2cb813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.110+0000 7f2cc70e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb813b430 0x7f2cb813f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.110+0000 7f2cc70e3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80b4820 con 0x7f2cb80a51a0 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.110+0000 7f2cc70e3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80b46a0 con 0x7f2cb813b430 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.110+0000 7f2cc70e3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2cb80b49a0 con 0x7f2cb80aa810 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc68e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb813b430 0x7f2cb813f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc68e2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb813b430 0x7f2cb813f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:53274/0 (socket says 172.21.15.69:53274) 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc68e2640 1 -- 172.21.15.69:0/2649395809 learned_addr learned my addr 172.21.15.69:0/2649395809 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc58e0640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80aa810 0x7f2cb813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc68e2640 1 -- 172.21.15.69:0/2649395809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80aa810 msgr2=0x7f2cb813ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc68e2640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80aa810 0x7f2cb813ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc68e2640 1 -- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cb80a51a0 msgr2=0x7f2cb813a8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc68e2640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cb80a51a0 0x7f2cb813a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.111+0000 7f2cc68e2640 1 -- 172.21.15.69:0/2649395809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cb813fc40 con 0x7f2cb813b430 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.112+0000 7f2cc68e2640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb813b430 0x7f2cb813f380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009990 tx=0x7f2cbc009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.112+0000 7f2cb77fe640 1 -- 172.21.15.69:0/2649395809 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc00c960 con 0x7f2cb813b430 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.112+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cb813ff30 con 0x7f2cb813b430 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.112+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cb8147a10 con 0x7f2cb813b430 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.112+0000 7f2cb77fe640 1 -- 172.21.15.69:0/2649395809 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cbc005020 con 0x7f2cb813b430 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.112+0000 7f2cb77fe640 1 -- 172.21.15.69:0/2649395809 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc004ce0 con 0x7f2cb813b430 2024-08-02T17:52:11.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.114+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca4005290 con 0x7f2cb813b430 2024-08-02T17:52:11.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.116+0000 7f2cb77fe640 1 -- 172.21.15.69:0/2649395809 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2cbc00cb00 con 0x7f2cb813b430 2024-08-02T17:52:11.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.117+0000 7f2cb77fe640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2cac0a2e40 0x7f2cac0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:11.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.117+0000 7f2cb77fe640 1 -- 172.21.15.69:0/2649395809 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2cbc0cf5b0 con 0x7f2cb813b430 2024-08-02T17:52:11.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.128+0000 7f2cc60e1640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2cac0a2e40 0x7f2cac0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:11.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.128+0000 7f2cb77fe640 1 -- 172.21.15.69:0/2649395809 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2cbc09bdb0 con 0x7f2cb813b430 2024-08-02T17:52:11.136 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.135+0000 7f2cc60e1640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2cac0a2e40 0x7f2cac0a5300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0065770 tx=0x7f2cc0050bd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:11.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.473+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ca4005580 con 0x7f2cb813b430 2024-08-02T17:52:11.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.474+0000 7f2cb77fe640 1 -- 172.21.15.69:0/2649395809 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f2cbc09b6b0 con 0x7f2cb813b430 2024-08-02T17:52:11.475 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:52:11.476 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:52:11.476 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:52:11.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2cac0a2e40 msgr2=0x7f2cac0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:11.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2cac0a2e40 0x7f2cac0a5300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0065770 tx=0x7f2cc0050bd0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb813b430 msgr2=0x7f2cb813f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:11.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb813b430 0x7f2cb813f380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009990 tx=0x7f2cbc009e50 comp rx=0 tx=0).stop 2024-08-02T17:52:11.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 shutdown_connections 2024-08-02T17:52:11.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2cac0a2e40 0x7f2cac0a5300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2cb813b430 0x7f2cb813f380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2cb80aa810 0x7f2cb813ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 --2- 172.21.15.69:0/2649395809 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2cb80a51a0 0x7f2cb813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 >> 172.21.15.69:0/2649395809 conn(0x7f2cb801a510 msgr2=0x7f2cb80a8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:11.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 shutdown_connections 2024-08-02T17:52:11.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:11.476+0000 7f2cc70e3640 1 -- 172.21.15.69:0/2649395809 wait complete. 2024-08-02T17:52:11.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1aa6ffd640 1 -- 172.21.15.69:0/1695416476 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c715d16430 con 0x55c715e424b0 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1aa6ffd640 10 client.26492 handle_command_reply: tid=0 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 client.26492 shutdown 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db3450 msgr2=0x55c715daff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db3450 0x55c715daff80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006fd0 tx=0x7f1abc004770 comp rx=0 tx=0).stop 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1695416476 shutdown_connections 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c715db5970 0x55c715db04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c715e424b0 0x55c715e448d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c715db3450 0x55c715daff80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 --2- 172.21.15.69:0/1695416476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c715daa7b0 0x55c715dafa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1aa6ffd640 0 client.26492 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.582+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1695416476 >> 172.21.15.69:0/1695416476 conn(0x55c715da6d20 msgr2=0x55c715daca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.583+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1695416476 shutdown_connections 2024-08-02T17:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.583+0000 7f1ad07d7740 1 -- 172.21.15.69:0/1695416476 wait complete. 2024-08-02T17:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.583+0000 7f1ad07d7740 20 client.26492 trim_cache size 0 max 16384 2024-08-02T17:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:52:11.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 -- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1ab00a2f00 msgr2=0x7f1ab00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1ab00a2f00 0x7f1ab00a53c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55c715da80a0 tx=0x7f1abc008040 comp rx=0 tx=0).stop 2024-08-02T17:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 -- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac8106730 msgr2=0x7f1ac811c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac8106730 0x7f1ac811c090 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00097b0 tx=0x7f1ac00316b0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 -- 172.21.15.69:0/778476438 shutdown_connections 2024-08-02T17:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1ac8106730 0x7f1ac811c090 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1ab00a2f00 0x7f1ab00a53c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1ac81010c0 0x7f1ac81168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 --2- 172.21.15.69:0/778476438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1ac8100720 0x7f1ac8116380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 -- 172.21.15.69:0/778476438 >> 172.21.15.69:0/778476438 conn(0x7f1ac80fc3e0 msgr2=0x7f1ac80fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 -- 172.21.15.69:0/778476438 shutdown_connections 2024-08-02T17:52:11.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:11.584+0000 7f1acda87640 1 -- 172.21.15.69:0/778476438 wait complete. 2024-08-02T17:52:12.435 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 148.9 --await-for 45.0' 2024-08-02T17:52:12.435 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 148.9 --await-for 45.0 2024-08-02T17:52:12.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:12 smithi077 ceph-mon[37108]: pgmap v176: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.7 MiB/s wr, 375 op/s 2024-08-02T17:52:12.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:12 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2649395809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:12 smithi069 ceph-mon[32746]: pgmap v176: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.7 MiB/s wr, 375 op/s 2024-08-02T17:52:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:12 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2649395809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:12.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:12 smithi083 ceph-mon[37031]: pgmap v176: 129 pgs: 129 active+clean; 2.8 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 4.7 MiB/s wr, 375 op/s 2024-08-02T17:52:12.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:12 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2649395809' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:12.910 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:14.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:14 smithi077 ceph-mon[37108]: pgmap v177: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 14 MiB/s wr, 990 op/s 2024-08-02T17:52:14.599 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:14 smithi069 ceph-mon[32746]: pgmap v177: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 14 MiB/s wr, 990 op/s 2024-08-02T17:52:14.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.625+0000 7f87828f8640 1 -- 172.21.15.69:0/1320356022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f877c102a80 msgr2=0x7f877c102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:14.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.625+0000 7f87828f8640 1 --2- 172.21.15.69:0/1320356022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f877c102a80 0x7f877c102e60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f876c009900 tx=0x7f876c02a480 comp rx=0 tx=0).stop 2024-08-02T17:52:14.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.625+0000 7f87828f8640 1 -- 172.21.15.69:0/1320356022 shutdown_connections 2024-08-02T17:52:14.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.626+0000 7f87828f8640 1 --2- 172.21.15.69:0/1320356022 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f877c10beb0 0x7f877c10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.626+0000 7f87828f8640 1 --2- 172.21.15.69:0/1320356022 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f877c1033a0 0x7f877c10b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.626+0000 7f87828f8640 1 --2- 172.21.15.69:0/1320356022 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f877c102a80 0x7f877c102e60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.626+0000 7f87828f8640 1 -- 172.21.15.69:0/1320356022 >> 172.21.15.69:0/1320356022 conn(0x7f877c0fc500 msgr2=0x7f877c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:14.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.626+0000 7f87828f8640 1 -- 172.21.15.69:0/1320356022 shutdown_connections 2024-08-02T17:52:14.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.626+0000 7f87828f8640 1 -- 172.21.15.69:0/1320356022 wait complete. 2024-08-02T17:52:14.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f87828f8640 1 Processor -- start 2024-08-02T17:52:14.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f87828f8640 1 -- start start 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f87828f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f877c102a80 0x7f877c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f87828f8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f877c1033a0 0x7f877c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f87828f8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f877c10beb0 0x7f877c19c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f87828f8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111d00 con 0x7f877c10beb0 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f87828f8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111b80 con 0x7f877c1033a0 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f87828f8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111e80 con 0x7f877c102a80 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.627+0000 7f8773fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f877c1033a0 0x7f877c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f877bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f877c102a80 0x7f877c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8773fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f877c1033a0 0x7f877c1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:53284/0 (socket says 172.21.15.69:53284) 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8773fff640 1 -- 172.21.15.69:0/1845567120 learned_addr learned my addr 172.21.15.69:0/1845567120 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8780e95640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f877c10beb0 0x7f877c19c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8773fff640 1 -- 172.21.15.69:0/1845567120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f877c102a80 msgr2=0x7f877c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:14.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8773fff640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f877c102a80 0x7f877c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8773fff640 1 -- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f877c10beb0 msgr2=0x7f877c19c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8773fff640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f877c10beb0 0x7f877c19c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8773fff640 1 -- 172.21.15.69:0/1845567120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c19cf50 con 0x7f877c1033a0 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f877bfff640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f877c102a80 0x7f877c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8773fff640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f877c1033a0 0x7f877c1984e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f876400e9f0 tx=0x7f876400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8779ffb640 1 -- 172.21.15.69:0/1845567120 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876400ccc0 con 0x7f877c1033a0 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f87828f8640 1 -- 172.21.15.69:0/1845567120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877c19d240 con 0x7f877c1033a0 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8779ffb640 1 -- 172.21.15.69:0/1845567120 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f876400ce60 con 0x7f877c1033a0 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f8779ffb640 1 -- 172.21.15.69:0/1845567120 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8764010690 con 0x7f877c1033a0 2024-08-02T17:52:14.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.628+0000 7f87828f8640 1 -- 172.21.15.69:0/1845567120 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877c100610 con 0x7f877c1033a0 2024-08-02T17:52:14.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8779ffb640 1 -- 172.21.15.69:0/1845567120 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8764004e80 con 0x7f877c1033a0 2024-08-02T17:52:14.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8784cf6740 1 Processor -- start 2024-08-02T17:52:14.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8784cf6740 1 -- start start 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x56423795a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237964dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237967700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8784cf6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5642378bfa70 con 0x564237965310 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8784cf6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564237857780 con 0x56423795aa50 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.632+0000 7f8784cf6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5642378bba30 con 0x56423795a130 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x56423795a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f877bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237964dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x56423795a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45932/0 (socket says 172.21.15.69:45932) 2024-08-02T17:52:14.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f877bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237964dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:53286/0 (socket says 172.21.15.69:53286) 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 -- 172.21.15.69:0/3254175632 learned_addr learned my addr 172.21.15.69:0/3254175632 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8773fff640 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237967700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8779ffb640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87680a2cf0 0x7f87680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 -- 172.21.15.69:0/3254175632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 msgr2=0x564237964dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237964dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 -- 172.21.15.69:0/3254175632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 msgr2=0x564237967700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237967700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 -- 172.21.15.69:0/3254175632 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564237631b90 con 0x56423795a130 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87680a2cf0 0x7f87680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8779ffb640 1 -- 172.21.15.69:0/1845567120 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f87640cfb70 con 0x7f877c1033a0 2024-08-02T17:52:14.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8773fff640 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237967700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:14.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f877bfff640 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237964dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:14.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.633+0000 7f8780e95640 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x56423795a510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87740104a0 tx=0x7f8774010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6898cba06c5fb16e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:14.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.634+0000 7f87727fc640 1 -- 172.21.15.69:0/3254175632 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774022070 con 0x56423795a130 2024-08-02T17:52:14.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.634+0000 7f87727fc640 1 -- 172.21.15.69:0/3254175632 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774013c60 con 0x56423795a130 2024-08-02T17:52:14.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.634+0000 7f87727fc640 1 -- 172.21.15.69:0/3254175632 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877400e430 con 0x56423795a130 2024-08-02T17:52:14.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.634+0000 7f8780e95640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87680a2cf0 0x7f87680a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f877401f000 tx=0x7f877401c000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:14.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.635+0000 7f8784cf6740 1 -- 172.21.15.69:0/3254175632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 msgr2=0x56423795a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:14.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.635+0000 7f8784cf6740 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x56423795a510 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87740104a0 tx=0x7f8774010960 comp rx=0 tx=0).stop 2024-08-02T17:52:14.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.641+0000 7f8784cf6740 1 -- 172.21.15.69:0/3254175632 shutdown_connections 2024-08-02T17:52:14.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.642+0000 7f8784cf6740 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237967700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.642+0000 7f8784cf6740 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237964dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.642+0000 7f8784cf6740 1 --2- 172.21.15.69:0/3254175632 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x56423795a510 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.642+0000 7f8784cf6740 1 -- 172.21.15.69:0/3254175632 >> 172.21.15.69:0/3254175632 conn(0x5642379566c0 msgr2=0x564237956aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.652+0000 7f8784cf6740 1 -- 172.21.15.69:0/3254175632 shutdown_connections 2024-08-02T17:52:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.652+0000 7f8784cf6740 1 -- 172.21.15.69:0/3254175632 wait complete. 2024-08-02T17:52:14.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.652+0000 7f8784cf6740 1 -- start start 2024-08-02T17:52:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.653+0000 7f8784cf6740 1 Processor -- start 2024-08-02T17:52:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.653+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x5642379621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.654+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x5642379626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.654+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564237965310 0x564237962c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.654+0000 7f8784cf6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564237967ee0 con 0x56423795aa50 2024-08-02T17:52:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.654+0000 7f8784cf6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56423787a990 con 0x564237965310 2024-08-02T17:52:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.654+0000 7f8784cf6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564237968060 con 0x56423795a130 2024-08-02T17:52:14.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.655+0000 7f8784cf6740 10 client.0 _ll_register_callbacks cb 0x5642375cf090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.656+0000 7f8773fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x5642379626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.656+0000 7f8773fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x5642379626e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41272/0 (socket says 172.21.15.69:41272) 2024-08-02T17:52:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.656+0000 7f8773fff640 1 -- 172.21.15.69:0/827587020 learned_addr learned my addr 172.21.15.69:0/827587020 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:14.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.656+0000 7f877bfff640 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x5642379621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.658+0000 7f8780e95640 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564237965310 0x564237962c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.659+0000 7f877bfff640 1 -- 172.21.15.69:0/827587020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564237965310 msgr2=0x564237962c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.659+0000 7f877bfff640 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564237965310 0x564237962c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.659+0000 7f877bfff640 1 -- 172.21.15.69:0/827587020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 msgr2=0x5642379626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.659+0000 7f877bfff640 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x5642379626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.660+0000 7f877bfff640 1 -- 172.21.15.69:0/827587020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642378796c0 con 0x56423795a130 2024-08-02T17:52:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.662+0000 7f877bfff640 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x5642379621a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f876c02a9a0 tx=0x7f876c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53a4b18ef478dcbb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.662+0000 7f875b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:52:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.663+0000 7f875b7fe640 1 -- 172.21.15.69:0/827587020 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c04c020 con 0x56423795a130 2024-08-02T17:52:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.663+0000 7f875b7fe640 1 -- 172.21.15.69:0/827587020 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f876c03c2d0 con 0x56423795a130 2024-08-02T17:52:14.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.664+0000 7f8784cf6740 1 -- 172.21.15.69:0/827587020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8774013c60 con 0x56423795a130 2024-08-02T17:52:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.671+0000 7f875b7fe640 1 -- 172.21.15.69:0/827587020 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f876c035040 con 0x56423795a130 2024-08-02T17:52:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.674+0000 7f8784cf6740 10 client.26392 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:14.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.674+0000 7f8784cf6740 1 -- 172.21.15.69:0/827587020 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564237973b20 con 0x56423795a130 2024-08-02T17:52:14.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.674+0000 7f875b7fe640 1 -- 172.21.15.69:0/827587020 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774013c60 con 0x56423795a130 2024-08-02T17:52:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.674+0000 7f875b7fe640 1 -- 172.21.15.69:0/827587020 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f876c005430 con 0x56423795a130 2024-08-02T17:52:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.675+0000 7f8784cf6740 10 client.26392 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.675+0000 7f8784cf6740 10 client.26392 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:52:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.675+0000 7f8784cf6740 20 client.26392 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.675+0000 7f8784cf6740 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564237974fe0 0x5642379773d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.675+0000 7f8784cf6740 4 client.26392 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:14.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.675+0000 7f8784cf6740 1 -- 172.21.15.69:0/827587020 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642378cd1a0 con 0x564237974fe0 2024-08-02T17:52:14.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.677+0000 7f8773fff640 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564237974fe0 0x5642379773d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.679+0000 7f875b7fe640 10 client.26392 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:14.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.679+0000 7f8773fff640 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564237974fe0 0x5642379773d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30810b480e48a38 server_cookie=cc838ce1ef21c199 in_seq=0 out_seq=0 2024-08-02T17:52:14.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.681+0000 7f875b7fe640 1 -- 172.21.15.69:0/827587020 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5642378cd1a0 con 0x564237974fe0 2024-08-02T17:52:14.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.681+0000 7f875b7fe640 10 client.26392 handle_command_reply: tid=0 2024-08-02T17:52:14.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.683+0000 7f8784cf6740 1 client.26392 shutdown 2024-08-02T17:52:14.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.684+0000 7f8784cf6740 1 -- 172.21.15.69:0/827587020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 msgr2=0x5642379621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:14.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.684+0000 7f8784cf6740 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x5642379621a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f876c02a9a0 tx=0x7f876c0045e0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.685+0000 7f8784cf6740 1 -- 172.21.15.69:0/827587020 shutdown_connections 2024-08-02T17:52:14.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.685+0000 7f8784cf6740 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x5642379626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.685+0000 7f8784cf6740 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564237974fe0 0x5642379773d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.685+0000 7f8784cf6740 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564237965310 0x564237962c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.685+0000 7f8784cf6740 1 --2- 172.21.15.69:0/827587020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423795a130 0x5642379621a0 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f876c02a9a0 tx=0x7f876c0045e0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.685+0000 7f8784cf6740 1 -- 172.21.15.69:0/827587020 >> 172.21.15.69:0/827587020 conn(0x5642379566c0 msgr2=0x56423795c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:14.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.685+0000 7f875b7fe640 0 client.26392 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:14.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.691+0000 7f8784cf6740 1 -- 172.21.15.69:0/827587020 shutdown_connections 2024-08-02T17:52:14.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.693+0000 7f8784cf6740 1 -- 172.21.15.69:0/827587020 wait complete. 2024-08-02T17:52:14.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.695+0000 7f8784cf6740 20 client.26392 trim_cache size 0 max 16384 2024-08-02T17:52:14.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.729+0000 7f8784cf6740 1 Processor -- start 2024-08-02T17:52:14.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f8784cf6740 1 -- start start 2024-08-02T17:52:14.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x56423795cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564237965310 0x564237964110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f877bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x56423795cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f877bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x56423795cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35406/0 (socket says 172.21.15.69:35406) 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423797cc50 0x56423797d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f877bfff640 1 -- 172.21.15.69:0/3231462902 learned_addr learned my addr 172.21.15.69:0/3231462902 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f8784cf6740 1 -- 172.21.15.69:0/3231462902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1af460 con 0x56423795aa50 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f8784cf6740 1 -- 172.21.15.69:0/3231462902 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1acd30 con 0x56423797cc50 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.730+0000 7f8784cf6740 1 -- 172.21.15.69:0/3231462902 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1aab80 con 0x564237965310 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.731+0000 7f877bfff640 1 -- 172.21.15.69:0/3231462902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564237965310 msgr2=0x564237964110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.731+0000 7f877bfff640 1 --2- 172.21.15.69:0/3231462902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564237965310 0x564237964110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.731+0000 7f877bfff640 1 -- 172.21.15.69:0/3231462902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423797cc50 msgr2=0x56423797d0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.731+0000 7f877bfff640 1 --2- 172.21.15.69:0/3231462902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423797cc50 0x56423797d0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.731+0000 7f877bfff640 1 -- 172.21.15.69:0/3231462902 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564237957180 con 0x56423795aa50 2024-08-02T17:52:14.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.731+0000 7f877bfff640 1 --2- 172.21.15.69:0/3231462902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x56423795cb90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f876c0038a0 tx=0x7f876c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af96b4c9e01661b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:14.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.732+0000 7f8771ffb640 1 -- 172.21.15.69:0/3231462902 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c05f070 con 0x56423795aa50 2024-08-02T17:52:14.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.732+0000 7f8771ffb640 1 -- 172.21.15.69:0/3231462902 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f876c009520 con 0x56423795aa50 2024-08-02T17:52:14.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.732+0000 7f8771ffb640 1 -- 172.21.15.69:0/3231462902 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c04c070 con 0x56423795aa50 2024-08-02T17:52:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.734+0000 7f8784cf6740 1 -- 172.21.15.69:0/3231462902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 msgr2=0x56423795cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:14.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.734+0000 7f8784cf6740 1 --2- 172.21.15.69:0/3231462902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x56423795cb90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f876c0038a0 tx=0x7f876c02a480 comp rx=0 tx=0).stop 2024-08-02T17:52:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.735+0000 7f8784cf6740 1 -- 172.21.15.69:0/3231462902 shutdown_connections 2024-08-02T17:52:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.735+0000 7f8784cf6740 1 --2- 172.21.15.69:0/3231462902 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423797cc50 0x56423797d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.735+0000 7f8784cf6740 1 --2- 172.21.15.69:0/3231462902 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564237965310 0x564237964110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.735+0000 7f8784cf6740 1 --2- 172.21.15.69:0/3231462902 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56423795aa50 0x56423795cb90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.735+0000 7f8784cf6740 1 -- 172.21.15.69:0/3231462902 >> 172.21.15.69:0/3231462902 conn(0x5642379566c0 msgr2=0x564237963a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.736+0000 7f8784cf6740 1 -- 172.21.15.69:0/3231462902 shutdown_connections 2024-08-02T17:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.736+0000 7f8784cf6740 1 -- 172.21.15.69:0/3231462902 wait complete. 2024-08-02T17:52:14.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.737+0000 7f8784cf6740 1 -- start start 2024-08-02T17:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.737+0000 7f8784cf6740 1 Processor -- start 2024-08-02T17:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.738+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237969640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.738+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237969b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.738+0000 7f8784cf6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423797cc50 0x5642379dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f8784cf6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1ab1f0 con 0x564237965310 2024-08-02T17:52:14.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f877bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237969640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f877bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237969640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49650/0 (socket says 172.21.15.69:49650) 2024-08-02T17:52:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f877bfff640 1 -- 172.21.15.69:0/1489011800 learned_addr learned my addr 172.21.15.69:0/1489011800 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f8780e95640 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423797cc50 0x5642379dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1ac660 con 0x56423795aa50 2024-08-02T17:52:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1ab8c0 con 0x56423797cc50 2024-08-02T17:52:14.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f8784cf6740 10 client.0 _ll_register_callbacks cb 0x7f876c031480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.739+0000 7f8773fff640 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237969b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.740+0000 7f877bfff640 1 -- 172.21.15.69:0/1489011800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423797cc50 msgr2=0x5642379dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.740+0000 7f877bfff640 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423797cc50 0x5642379dec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.740+0000 7f877bfff640 1 -- 172.21.15.69:0/1489011800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 msgr2=0x564237969b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.740+0000 7f877bfff640 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237969b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.740+0000 7f877bfff640 1 -- 172.21.15.69:0/1489011800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642379df160 con 0x56423795aa50 2024-08-02T17:52:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.740+0000 7f877bfff640 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237969640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f876c003fd0 tx=0x7f876c004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6efb0ba98b8ad594 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:14.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.740+0000 7f875b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:52:14.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.744+0000 7f875b7fe640 1 -- 172.21.15.69:0/1489011800 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c05f070 con 0x56423795aa50 2024-08-02T17:52:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.744+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f876c009520 con 0x56423795aa50 2024-08-02T17:52:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.744+0000 7f875b7fe640 1 -- 172.21.15.69:0/1489011800 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f876c03c4e0 con 0x56423795aa50 2024-08-02T17:52:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.744+0000 7f875b7fe640 1 -- 172.21.15.69:0/1489011800 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f876c06e050 con 0x56423795aa50 2024-08-02T17:52:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.744+0000 7f8784cf6740 10 client.26513 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:14.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.744+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5642379e0300 con 0x56423795aa50 2024-08-02T17:52:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.745+0000 7f875b7fe640 1 -- 172.21.15.69:0/1489011800 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f876c009520 con 0x56423795aa50 2024-08-02T17:52:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.745+0000 7f875b7fe640 1 -- 172.21.15.69:0/1489011800 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f876c03a510 con 0x56423795aa50 2024-08-02T17:52:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.745+0000 7f8784cf6740 10 client.26513 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.745+0000 7f8784cf6740 10 client.26513 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:52:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.745+0000 7f8784cf6740 20 client.26513 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.745+0000 7f8784cf6740 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5642379e0980 0x5642379e2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.745+0000 7f8784cf6740 4 client.26513 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.9, "await_for": 45.0}] 2024-08-02T17:52:14.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.745+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 148.9, "await_for": 45.0}) -- 0x5642378c74f0 con 0x5642379e0980 2024-08-02T17:52:14.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.746+0000 7f8773fff640 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5642379e0980 0x5642379e2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:14.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.748+0000 7f875b7fe640 10 client.26513 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:14.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:14.749+0000 7f8773fff640 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5642379e0980 0x5642379e2d70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950b38aa53400536 server_cookie=f1d6c5b4cf448f86 in_seq=0 out_seq=0 2024-08-02T17:52:14.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:14 smithi083 ceph-mon[37031]: pgmap v177: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 14 MiB/s wr, 990 op/s 2024-08-02T17:52:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 305.0 s, 799.6 tps, lat 220.404 ms stddev 1490.005 2024-08-02T17:52:15.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f875b7fe640 1 -- 172.21.15.69:0/1489011800 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5642378c74f0 con 0x5642379e0980 2024-08-02T17:52:15.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f875b7fe640 10 client.26513 handle_command_reply: tid=0 2024-08-02T17:52:15.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 client.26513 shutdown 2024-08-02T17:52:15.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 msgr2=0x564237969640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:15.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237969640 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f876c003fd0 tx=0x7f876c004250 comp rx=0 tx=0).stop 2024-08-02T17:52:15.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 shutdown_connections 2024-08-02T17:52:15.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564237965310 0x564237969b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5642379e0980 0x5642379e2d70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56423797cc50 0x5642379dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 --2- 172.21.15.69:0/1489011800 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56423795aa50 0x564237969640 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f875b7fe640 0 client.26513 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 >> 172.21.15.69:0/1489011800 conn(0x5642379566c0 msgr2=0x564237954ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 shutdown_connections 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 1 -- 172.21.15.69:0/1489011800 wait complete. 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.010+0000 7f8784cf6740 20 client.26513 trim_cache size 0 max 16384 2024-08-02T17:52:15.013 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 22, 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "4e57b63c": { 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 22, 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:52:15.014 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 148.9, 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:15.015 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:15.016 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:15.016 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:15.016 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:52:15.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 -- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87680a2cf0 msgr2=0x7f87680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:15.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87680a2cf0 0x7f87680a51b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f877401f000 tx=0x7f877401c000 comp rx=0 tx=0).stop 2024-08-02T17:52:15.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 -- 172.21.15.69:0/1845567120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f877c1033a0 msgr2=0x7f877c1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:15.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f877c1033a0 0x7f877c1984e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f876400e9f0 tx=0x7f876400eeb0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 -- 172.21.15.69:0/1845567120 shutdown_connections 2024-08-02T17:52:15.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f877c10beb0 0x7f877c19c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87680a2cf0 0x7f87680a51b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f877c1033a0 0x7f877c1984e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 --2- 172.21.15.69:0/1845567120 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f877c102a80 0x7f877c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:15.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 -- 172.21.15.69:0/1845567120 >> 172.21.15.69:0/1845567120 conn(0x7f877c0fc500 msgr2=0x7f877c10e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:15.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 -- 172.21.15.69:0/1845567120 shutdown_connections 2024-08-02T17:52:15.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:15.011+0000 7f87828f8640 1 -- 172.21.15.69:0/1845567120 wait complete. 2024-08-02T17:52:15.773 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4e57b63c', quiesce took 6.1 seconds. Will release after: 22.799999999999997 2024-08-02T17:52:16.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:16 smithi077 ceph-mon[37108]: pgmap v178: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 899 op/s 2024-08-02T17:52:16.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:16 smithi069 ceph-mon[32746]: pgmap v178: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 899 op/s 2024-08-02T17:52:16.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:16 smithi083 ceph-mon[37031]: pgmap v178: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 14 MiB/s wr, 899 op/s 2024-08-02T17:52:18.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:18 smithi077 ceph-mon[37108]: pgmap v179: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2024-08-02T17:52:18.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:18 smithi069 ceph-mon[32746]: pgmap v179: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2024-08-02T17:52:18.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:18 smithi083 ceph-mon[37031]: pgmap v179: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 20 MiB/s wr, 1.07k op/s 2024-08-02T17:52:19.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 310.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:52:20.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:20 smithi077 ceph-mon[37108]: pgmap v180: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 17 MiB/s wr, 900 op/s 2024-08-02T17:52:20.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:20 smithi069 ceph-mon[32746]: pgmap v180: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 17 MiB/s wr, 900 op/s 2024-08-02T17:52:20.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:20 smithi083 ceph-mon[37031]: pgmap v180: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 17 MiB/s wr, 900 op/s 2024-08-02T17:52:22.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:22 smithi077 ceph-mon[37108]: pgmap v181: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 16 MiB/s wr, 869 op/s 2024-08-02T17:52:22.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:22 smithi069 ceph-mon[32746]: pgmap v181: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 16 MiB/s wr, 869 op/s 2024-08-02T17:52:22.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:22 smithi083 ceph-mon[37031]: pgmap v181: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 16 MiB/s wr, 869 op/s 2024-08-02T17:52:24.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:24 smithi077 ceph-mon[37108]: pgmap v182: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 16 MiB/s wr, 818 op/s 2024-08-02T17:52:24.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:24 smithi069 ceph-mon[32746]: pgmap v182: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 16 MiB/s wr, 818 op/s 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 -- 172.21.15.69:0/3786995582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c100f80 msgr2=0x7fb54c1076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 --2- 172.21.15.69:0/3786995582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c100f80 0x7fb54c1076c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009900 tx=0x7fb53c02a410 comp rx=0 tx=0).stop 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 -- 172.21.15.69:0/3786995582 shutdown_connections 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 --2- 172.21.15.69:0/3786995582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c100f80 0x7fb54c1076c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 --2- 172.21.15.69:0/3786995582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c107e20 0x7fb54c10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 --2- 172.21.15.69:0/3786995582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb54c1005d0 0x7fb54c1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 -- 172.21.15.69:0/3786995582 >> 172.21.15.69:0/3786995582 conn(0x7fb54c075b50 msgr2=0x7fb54c075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 -- 172.21.15.69:0/3786995582 shutdown_connections 2024-08-02T17:52:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 -- 172.21.15.69:0/3786995582 wait complete. 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 Processor -- start 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 -- start start 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.678+0000 7fb5535cb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb54c1005d0 0x7fb54c101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5535cb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c107e20 0x7fb54c101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5535cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c101fc0 0x7fb54c1a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5535cb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c111c30 con 0x7fb54c107e20 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5535cb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c111ab0 con 0x7fb54c1005d0 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5535cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c111db0 con 0x7fb54c101fc0 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c107e20 0x7fb54c101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c107e20 0x7fb54c101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35428/0 (socket says 172.21.15.69:35428) 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 -- 172.21.15.69:0/1537444192 learned_addr learned my addr 172.21.15.69:0/1537444192 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb551367640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb54c1005d0 0x7fb54c101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb551b68640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c101fc0 0x7fb54c1a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 -- 172.21.15.69:0/1537444192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c101fc0 msgr2=0x7fb54c1a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c101fc0 0x7fb54c1a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 -- 172.21.15.69:0/1537444192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb54c1005d0 msgr2=0x7fb54c101540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb54c1005d0 0x7fb54c101540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 -- 172.21.15.69:0/1537444192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c1a6a00 con 0x7fb54c107e20 2024-08-02T17:52:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb551367640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb54c1005d0 0x7fb54c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb550b66640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c107e20 0x7fb54c101a80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb53c005940 tx=0x7fb53c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5427fc640 1 -- 172.21.15.69:0/1537444192 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c03f030 con 0x7fb54c107e20 2024-08-02T17:52:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5427fc640 1 -- 172.21.15.69:0/1537444192 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c0044c0 con 0x7fb54c107e20 2024-08-02T17:52:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5535cb640 1 -- 172.21.15.69:0/1537444192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c1a6c90 con 0x7fb54c107e20 2024-08-02T17:52:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5427fc640 1 -- 172.21.15.69:0/1537444192 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c035070 con 0x7fb54c107e20 2024-08-02T17:52:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.679+0000 7fb5535cb640 1 -- 172.21.15.69:0/1537444192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1a7180 con 0x7fb54c107e20 2024-08-02T17:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.682+0000 7fb5427fc640 1 -- 172.21.15.69:0/1537444192 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb53c042030 con 0x7fb54c107e20 2024-08-02T17:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.682+0000 7fb5563f4740 1 Processor -- start 2024-08-02T17:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.682+0000 7fb5563f4740 1 -- start start 2024-08-02T17:52:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e22692bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692c0d0 0x55e226936430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e226936970 0x55e226938d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5563f4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e22688f9b0 con 0x55e22692b7b0 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5563f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e226837800 con 0x55e22692c0d0 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5563f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e22688b970 con 0x55e226936970 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e22692bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e22692bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35440/0 (socket says 172.21.15.69:35440) 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 -- 172.21.15.69:0/2305126533 learned_addr learned my addr 172.21.15.69:0/2305126533 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551367640 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692c0d0 0x55e226936430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5427fc640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb5380a2d20 0x7fb5380a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb550b66640 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e226936970 0x55e226938d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 -- 172.21.15.69:0/2305126533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e226936970 msgr2=0x55e226938d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e226936970 0x55e226938d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 -- 172.21.15.69:0/2305126533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692c0d0 msgr2=0x55e226936430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692c0d0 0x55e226936430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 -- 172.21.15.69:0/2305126533 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2265a3210 con 0x55e22692b7b0 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb5380a2d20 0x7fb5380a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5427fc640 1 -- 172.21.15.69:0/1537444192 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb53c0efc70 con 0x7fb54c107e20 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb551b68640 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e22692bb90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb544009fb0 tx=0x7fb54400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a30b5c4ff30d17 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.683+0000 7fb5377fe640 1 -- 172.21.15.69:0/2305126533 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544021070 con 0x55e22692b7b0 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5377fe640 1 -- 172.21.15.69:0/2305126533 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb54401cde0 con 0x55e22692b7b0 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5377fe640 1 -- 172.21.15.69:0/2305126533 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54401bb80 con 0x55e22692b7b0 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 -- 172.21.15.69:0/2305126533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 msgr2=0x55e22692bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.694 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e22692bb90 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb544009fb0 tx=0x7fb54400f7f0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb551b68640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb5380a2d20 0x7fb5380a51e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb54401e000 tx=0x7fb54401c000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 -- 172.21.15.69:0/2305126533 shutdown_connections 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e226936970 0x55e226938d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692c0d0 0x55e226936430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2305126533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e22692bb90 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 -- 172.21.15.69:0/2305126533 >> 172.21.15.69:0/2305126533 conn(0x55e226927d20 msgr2=0x55e226928100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 -- 172.21.15.69:0/2305126533 shutdown_connections 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.684+0000 7fb5563f4740 1 -- 172.21.15.69:0/2305126533 wait complete. 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 1 -- start start 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 1 Processor -- start 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e226933890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 0x55e226933dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 0x55e226934310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e226939540 con 0x55e22692b7b0 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e22684b500 con 0x55e226936970 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e2269396c0 con 0x55e22692c0d0 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb5563f4740 10 client.0 _ll_register_callbacks cb 0x55e2265a2a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb551367640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e226933890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb551367640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e226933890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35446/0 (socket says 172.21.15.69:35446) 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb551367640 1 -- 172.21.15.69:0/2435574940 learned_addr learned my addr 172.21.15.69:0/2435574940 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.685+0000 7fb551b68640 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 0x55e226934310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb550b66640 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 0x55e226933dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb551367640 1 -- 172.21.15.69:0/2435574940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 msgr2=0x55e226933dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb551367640 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 0x55e226933dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb551367640 1 -- 172.21.15.69:0/2435574940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 msgr2=0x55e226934310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb551367640 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 0x55e226934310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb551367640 1 -- 172.21.15.69:0/2435574940 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e22666f620 con 0x55e22692b7b0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb551367640 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e226933890 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb548009d40 tx=0x7fb5480039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff114a739442eef server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb534ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb534ff9640 1 -- 172.21.15.69:0/2435574940 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548013930 con 0x55e22692b7b0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb534ff9640 1 -- 172.21.15.69:0/2435574940 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb548013ad0 con 0x55e22692b7b0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 1 -- 172.21.15.69:0/2435574940 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb54401cde0 con 0x55e22692b7b0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb534ff9640 1 -- 172.21.15.69:0/2435574940 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb548013cd0 con 0x55e22692b7b0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 10 client.16965 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 1 -- 172.21.15.69:0/2435574940 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e226945220 con 0x55e22692b7b0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb534ff9640 1 -- 172.21.15.69:0/2435574940 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb54401cde0 con 0x55e22692b7b0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb534ff9640 1 -- 172.21.15.69:0/2435574940 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb54801a070 con 0x55e22692b7b0 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 10 client.16965 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 20 client.16965 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2269466e0 0x55e226948b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 4 client.16965 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.686+0000 7fb5563f4740 1 -- 172.21.15.69:0/2435574940 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e22689d0e0 con 0x55e2269466e0 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.687+0000 7fb550b66640 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2269466e0 0x55e226948b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.687+0000 7fb550b66640 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2269466e0 0x55e226948b30 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62231c4055c6642 server_cookie=8af48500cac00db8 in_seq=0 out_seq=0 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.687+0000 7fb534ff9640 10 client.16965 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb534ff9640 1 -- 172.21.15.69:0/2435574940 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e22689d0e0 con 0x55e2269466e0 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb534ff9640 10 client.16965 handle_command_reply: tid=0 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 client.16965 shutdown 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 -- 172.21.15.69:0/2435574940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 msgr2=0x55e226933890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e226933890 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb548009d40 tx=0x7fb5480039c0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 -- 172.21.15.69:0/2435574940 shutdown_connections 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2269466e0 0x55e226948b30 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 0x55e226934310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 0x55e226933dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 --2- 172.21.15.69:0/2435574940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692b7b0 0x55e226933890 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb534ff9640 0 client.16965 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 -- 172.21.15.69:0/2435574940 >> 172.21.15.69:0/2435574940 conn(0x55e226927d20 msgr2=0x55e22692da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 -- 172.21.15.69:0/2435574940 shutdown_connections 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 1 -- 172.21.15.69:0/2435574940 wait complete. 2024-08-02T17:52:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.688+0000 7fb5563f4740 20 client.16965 trim_cache size 0 max 16384 2024-08-02T17:52:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb5563f4740 1 Processor -- start 2024-08-02T17:52:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb5563f4740 1 -- start start 2024-08-02T17:52:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692b7b0 0x55e22692e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692c0d0 0x55e2269356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 0x55e22694e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb5563f4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1b3850 con 0x55e22692c0d0 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb5563f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1b1120 con 0x55e226936970 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb5563f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1aef70 con 0x55e22692b7b0 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692c0d0 0x55e2269356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692c0d0 0x55e2269356f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35458/0 (socket says 172.21.15.69:35458) 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 -- 172.21.15.69:0/3726349687 learned_addr learned my addr 172.21.15.69:0/3726349687 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb551b68640 1 --2- 172.21.15.69:0/3726349687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 0x55e22694e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 -- 172.21.15.69:0/3726349687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692b7b0 msgr2=0x55e22692e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 --2- 172.21.15.69:0/3726349687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692b7b0 0x55e22692e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 -- 172.21.15.69:0/3726349687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 msgr2=0x55e22694e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 --2- 172.21.15.69:0/3726349687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 0x55e22694e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 -- 172.21.15.69:0/3726349687 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e226928dd0 con 0x55e22692c0d0 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.703+0000 7fb550b66640 1 --2- 172.21.15.69:0/3726349687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692c0d0 0x55e2269356f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0ed600 tx=0x7fb53c03b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582b94b149b130d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb536ffd640 1 -- 172.21.15.69:0/3726349687 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c049040 con 0x55e22692c0d0 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb536ffd640 1 -- 172.21.15.69:0/3726349687 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c0c0180 con 0x55e22692c0d0 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb536ffd640 1 -- 172.21.15.69:0/3726349687 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0b85d0 con 0x55e22692c0d0 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 -- 172.21.15.69:0/3726349687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692c0d0 msgr2=0x55e2269356f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 --2- 172.21.15.69:0/3726349687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692c0d0 0x55e2269356f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0ed600 tx=0x7fb53c03b640 comp rx=0 tx=0).stop 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 -- 172.21.15.69:0/3726349687 shutdown_connections 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 --2- 172.21.15.69:0/3726349687 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e22692c0d0 0x55e2269356f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 --2- 172.21.15.69:0/3726349687 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e226936970 0x55e22694e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 --2- 172.21.15.69:0/3726349687 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692b7b0 0x55e22692e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 -- 172.21.15.69:0/3726349687 >> 172.21.15.69:0/3726349687 conn(0x55e226927d20 msgr2=0x55e226934fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 -- 172.21.15.69:0/3726349687 shutdown_connections 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 -- 172.21.15.69:0/3726349687 wait complete. 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.704+0000 7fb5563f4740 1 -- start start 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb5563f4740 1 Processor -- start 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692b7b0 0x55e22693a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 0x55e22693ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb5563f4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e226936970 0x55e22693b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb5563f4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1af5e0 con 0x55e226936970 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb5563f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1b0a50 con 0x55e22692b7b0 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb5563f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1afcb0 con 0x55e22692c0d0 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb5563f4740 10 client.0 _ll_register_callbacks cb 0x7fb548004160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551b68640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e226936970 0x55e22693b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551b68640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e226936970 0x55e22693b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35464/0 (socket says 172.21.15.69:35464) 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551b68640 1 -- 172.21.15.69:0/4255205822 learned_addr learned my addr 172.21.15.69:0/4255205822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb550b66640 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 0x55e22693ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551367640 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692b7b0 0x55e22693a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551b68640 1 -- 172.21.15.69:0/4255205822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 msgr2=0x55e22693ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551b68640 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 0x55e22693ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551b68640 1 -- 172.21.15.69:0/4255205822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692b7b0 msgr2=0x55e22693a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551b68640 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692b7b0 0x55e22693a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.705+0000 7fb551b68640 1 -- 172.21.15.69:0/4255205822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e22692b5e0 con 0x55e226936970 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb551b68640 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e226936970 0x55e22693b060 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb54401f040 tx=0x7fb5440092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178077ed88818ce9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb534ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb534ff9640 1 -- 172.21.15.69:0/4255205822 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb544021040 con 0x55e226936970 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb534ff9640 1 -- 172.21.15.69:0/4255205822 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fb54401b9a0 con 0x55e226936970 2024-08-02T17:52:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 1 -- 172.21.15.69:0/4255205822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb53c0c0180 con 0x55e226936970 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb534ff9640 1 -- 172.21.15.69:0/4255205822 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb54403f050 con 0x55e226936970 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 10 client.16971 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 1 -- 172.21.15.69:0/4255205822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e2269b14e0 con 0x55e226936970 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb534ff9640 1 -- 172.21.15.69:0/4255205822 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c0c0180 con 0x55e226936970 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb534ff9640 1 -- 172.21.15.69:0/4255205822 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb54401ca40 con 0x55e226936970 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 10 client.16971 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 20 client.16971 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2269b1a60 0x55e2269b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 4 client.16971 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb5563f4740 1 -- 172.21.15.69:0/4255205822 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e226897430 con 0x55e2269b1a60 2024-08-02T17:52:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.706+0000 7fb551367640 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2269b1a60 0x55e2269b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.707+0000 7fb534ff9640 10 client.16971 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:24.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.707+0000 7fb551367640 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2269b1a60 0x55e2269b3e50 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=486bb0b38a8e3f45 server_cookie=fb9ad4e6f5b793fe in_seq=0 out_seq=0 2024-08-02T17:52:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.730+0000 7fb534ff9640 1 -- 172.21.15.69:0/4255205822 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e226897430 con 0x55e2269b1a60 2024-08-02T17:52:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.730+0000 7fb534ff9640 10 client.16971 handle_command_reply: tid=0 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.730+0000 7fb5563f4740 1 client.16971 shutdown 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.730+0000 7fb5563f4740 1 -- 172.21.15.69:0/4255205822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e226936970 msgr2=0x55e22693b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.730+0000 7fb5563f4740 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e226936970 0x55e22693b060 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb54401f040 tx=0x7fb5440092a0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 1 -- 172.21.15.69:0/4255205822 shutdown_connections 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e226936970 0x55e22693b060 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2269b1a60 0x55e2269b3e50 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e22692c0d0 0x55e22693ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 1 --2- 172.21.15.69:0/4255205822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e22692b7b0 0x55e22693a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb534ff9640 0 client.16971 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 1 -- 172.21.15.69:0/4255205822 >> 172.21.15.69:0/4255205822 conn(0x55e226927d20 msgr2=0x55e226926660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 1 -- 172.21.15.69:0/4255205822 shutdown_connections 2024-08-02T17:52:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 1 -- 172.21.15.69:0/4255205822 wait complete. 2024-08-02T17:52:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5563f4740 20 client.16971 trim_cache size 0 max 16384 2024-08-02T17:52:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:52:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:52:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:52:24.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:52:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 -- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb5380a2d20 msgr2=0x7fb5380a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb5380a2d20 0x7fb5380a51e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb54401e000 tx=0x7fb54401c000 comp rx=0 tx=0).stop 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 -- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c107e20 msgr2=0x7fb54c101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c107e20 0x7fb54c101a80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb53c005940 tx=0x7fb53c0040c0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 -- 172.21.15.69:0/1537444192 shutdown_connections 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb54c107e20 0x7fb54c101a80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb5380a2d20 0x7fb5380a51e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb54c101fc0 0x7fb54c1a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 --2- 172.21.15.69:0/1537444192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb54c1005d0 0x7fb54c101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 -- 172.21.15.69:0/1537444192 >> 172.21.15.69:0/1537444192 conn(0x7fb54c075b50 msgr2=0x7fb54c1058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 -- 172.21.15.69:0/1537444192 shutdown_connections 2024-08-02T17:52:24.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:24.731+0000 7fb5535cb640 1 -- 172.21.15.69:0/1537444192 wait complete. 2024-08-02T17:52:24.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:24 smithi083 ceph-mon[37031]: pgmap v182: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 16 MiB/s wr, 818 op/s 2024-08-02T17:52:24.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 315.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:52:26.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:26 smithi077 ceph-mon[37108]: pgmap v183: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.5 MiB/s wr, 187 op/s 2024-08-02T17:52:26.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:26 smithi069 ceph-mon[32746]: pgmap v183: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.5 MiB/s wr, 187 op/s 2024-08-02T17:52:26.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:26 smithi083 ceph-mon[37031]: pgmap v183: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.5 MiB/s wr, 187 op/s 2024-08-02T17:52:28.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:28 smithi077 ceph-mon[37108]: pgmap v184: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.3 MiB/s wr, 287 op/s 2024-08-02T17:52:28.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:28 smithi069 ceph-mon[32746]: pgmap v184: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.3 MiB/s wr, 287 op/s 2024-08-02T17:52:28.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:28 smithi083 ceph-mon[37031]: pgmap v184: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.3 MiB/s wr, 287 op/s 2024-08-02T17:52:29.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 320.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:52:30.244 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:52:30.513 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:30.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:30 smithi077 ceph-mon[37108]: pgmap v185: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-02T17:52:30.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:30 smithi069 ceph-mon[32746]: pgmap v185: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-02T17:52:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:30 smithi083 ceph-mon[37031]: pgmap v185: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.3 MiB/s wr, 130 op/s 2024-08-02T17:52:30.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 -- 172.21.15.69:0/2118370081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 msgr2=0x7f982810b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:30.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 --2- 172.21.15.69:0/2118370081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 0x7f982810b880 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f981c009900 tx=0x7f981c02a480 comp rx=0 tx=0).stop 2024-08-02T17:52:30.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 -- 172.21.15.69:0/2118370081 shutdown_connections 2024-08-02T17:52:30.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 --2- 172.21.15.69:0/2118370081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 0x7f982810b880 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 --2- 172.21.15.69:0/2118370081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f982810bea0 0x7f982810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 --2- 172.21.15.69:0/2118370081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9828102a70 0x7f9828102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 -- 172.21.15.69:0/2118370081 >> 172.21.15.69:0/2118370081 conn(0x7f98280fc4d0 msgr2=0x7f98280fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 -- 172.21.15.69:0/2118370081 shutdown_connections 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.955+0000 7f982e398640 1 -- 172.21.15.69:0/2118370081 wait complete. 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.956+0000 7f982e398640 1 Processor -- start 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.956+0000 7f982e398640 1 -- start start 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.956+0000 7f982e398640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828102a70 0x7f98281982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.956+0000 7f982e398640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 0x7f9828198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.956+0000 7f982e398640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f982810bea0 0x7f982819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f982e398640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9828111ce0 con 0x7f9828103390 2024-08-02T17:52:30.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f982e398640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9828111b60 con 0x7f982810bea0 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f982e398640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9828111e60 con 0x7f9828102a70 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 0x7f9828198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 0x7f9828198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60602/0 (socket says 172.21.15.69:60602) 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 -- 172.21.15.69:0/2300197635 learned_addr learned my addr 172.21.15.69:0/2300197635 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f982c935640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f982810bea0 0x7f982819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f9827fff640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828102a70 0x7f98281982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 -- 172.21.15.69:0/2300197635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828102a70 msgr2=0x7f98281982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828102a70 0x7f98281982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 -- 172.21.15.69:0/2300197635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f982810bea0 msgr2=0x7f982819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:30.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f982810bea0 0x7f982819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 -- 172.21.15.69:0/2300197635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982819d270 con 0x7f9828103390 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f9827fff640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828102a70 0x7f98281982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f982c935640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f982810bea0 0x7f982819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98277fe640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 0x7f9828198800 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f981c005ab0 tx=0x7f981c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98257fa640 1 -- 172.21.15.69:0/2300197635 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c03f030 con 0x7f9828103390 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98257fa640 1 -- 172.21.15.69:0/2300197635 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f981c003ce0 con 0x7f9828103390 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f982e398640 1 -- 172.21.15.69:0/2300197635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982819d500 con 0x7f9828103390 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.957+0000 7f98257fa640 1 -- 172.21.15.69:0/2300197635 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c035070 con 0x7f9828103390 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.958+0000 7f982e398640 1 -- 172.21.15.69:0/2300197635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9828100600 con 0x7f9828103390 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f98257fa640 1 -- 172.21.15.69:0/2300197635 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f981c042030 con 0x7f9828103390 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9830796740 1 Processor -- start 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9830796740 1 -- start start 2024-08-02T17:52:30.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9830796740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e3275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9830796740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e331eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9830796740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e3347e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9830796740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56374e28c980 con 0x56374e327b30 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9830796740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56374e224a70 con 0x56374e3323f0 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9830796740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56374e288940 con 0x56374e327210 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9827fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e331eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f98257fa640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f98100a2cf0 0x7f98100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9827fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e331eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60604/0 (socket says 172.21.15.69:60604) 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f9827fff640 1 -- 172.21.15.69:0/2448567802 learned_addr learned my addr 172.21.15.69:0/2448567802 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f982c935640 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e3275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.960+0000 7f98277fe640 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e3347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f982c935640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f98100a2cf0 0x7f98100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f9827fff640 1 -- 172.21.15.69:0/2448567802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 msgr2=0x56374e3275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:30.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f9827fff640 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e3275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f9827fff640 1 -- 172.21.15.69:0/2448567802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 msgr2=0x56374e3347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f9827fff640 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e3347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f9827fff640 1 -- 172.21.15.69:0/2448567802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374e2876e0 con 0x56374e327b30 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f982c935640 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e3275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f98257fa640 1 -- 172.21.15.69:0/2300197635 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f981c0efd50 con 0x7f9828103390 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f9827fff640 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e331eb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f981800cca0 tx=0x7f981800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6fe7c7aff5cfb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f980e7fc640 1 -- 172.21.15.69:0/2448567802 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818005220 con 0x56374e327b30 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f980e7fc640 1 -- 172.21.15.69:0/2448567802 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9818004c80 con 0x56374e327b30 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.961+0000 7f980e7fc640 1 -- 172.21.15.69:0/2448567802 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818010890 con 0x56374e327b30 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 -- 172.21.15.69:0/2448567802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 msgr2=0x56374e331eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e331eb0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f981800cca0 tx=0x7f981800e720 comp rx=0 tx=0).stop 2024-08-02T17:52:30.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f982c935640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f98100a2cf0 0x7f98100a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9820009fd0 tx=0x7f982000c040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 -- 172.21.15.69:0/2448567802 shutdown_connections 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e331eb0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e3347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 --2- 172.21.15.69:0/2448567802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e3275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 -- 172.21.15.69:0/2448567802 >> 172.21.15.69:0/2448567802 conn(0x56374e3237a0 msgr2=0x56374e323b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 -- 172.21.15.69:0/2448567802 shutdown_connections 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 -- 172.21.15.69:0/2448567802 wait complete. 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 -- start start 2024-08-02T17:52:30.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.962+0000 7f9830796740 1 Processor -- start 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9830796740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327210 0x56374e32f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9830796740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 0x56374e32f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9830796740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e32fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9830796740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56374e334fc0 con 0x56374e327210 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9830796740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56374e247aa0 con 0x56374e3323f0 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9830796740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56374e335140 con 0x56374e327b30 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9830796740 10 client.0 _ll_register_callbacks cb 0x56374e236e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327210 0x56374e32f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327210 0x56374e32f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60606/0 (socket says 172.21.15.69:60606) 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 -- 172.21.15.69:0/1459812914 learned_addr learned my addr 172.21.15.69:0/1459812914 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f982c935640 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e32fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f98277fe640 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 0x56374e32f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 -- 172.21.15.69:0/1459812914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 msgr2=0x56374e32f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 0x56374e32f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 -- 172.21.15.69:0/1459812914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 msgr2=0x56374e32fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e32fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 -- 172.21.15.69:0/1459812914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374e22b650 con 0x56374e327210 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9827fff640 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327210 0x56374e32f290 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f981800c9e0 tx=0x7f981800cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b97e82cb5d57c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f98077fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f98077fe640 1 -- 172.21.15.69:0/1459812914 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981802a060 con 0x56374e327210 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f98077fe640 1 -- 172.21.15.69:0/1459812914 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f9818005210 con 0x56374e327210 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.963+0000 7f9830796740 1 -- 172.21.15.69:0/1459812914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9818004c80 con 0x56374e327210 2024-08-02T17:52:30.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f98077fe640 1 -- 172.21.15.69:0/1459812914 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9818031050 con 0x56374e327210 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f9830796740 10 client.16983 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f9830796740 1 -- 172.21.15.69:0/1459812914 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56374dffe3c0 con 0x56374e327210 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f98077fe640 1 -- 172.21.15.69:0/1459812914 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9818004c80 con 0x56374e327210 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f98077fe640 1 -- 172.21.15.69:0/1459812914 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9818025470 con 0x56374e327210 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f9830796740 10 client.16983 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f9830796740 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f9830796740 20 client.16983 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f9830796740 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56374e341e80 0x56374e344270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f9830796740 4 client.16983 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f9830796740 1 -- 172.21.15.69:0/1459812914 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56374e299ef0 con 0x56374e341e80 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.964+0000 7f98277fe640 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56374e341e80 0x56374e344270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.965+0000 7f98277fe640 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56374e341e80 0x56374e344270 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aaf4209c5754673 server_cookie=707a0d46f814e724 in_seq=0 out_seq=0 2024-08-02T17:52:31.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.965+0000 7f98077fe640 10 client.16983 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.965+0000 7f98077fe640 1 -- 172.21.15.69:0/1459812914 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56374e299ef0 con 0x56374e341e80 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.965+0000 7f98077fe640 10 client.16983 handle_command_reply: tid=0 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.965+0000 7f9830796740 1 client.16983 shutdown 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 -- 172.21.15.69:0/1459812914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327210 msgr2=0x56374e32f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327210 0x56374e32f290 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f981800c9e0 tx=0x7f981800cfa0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 -- 172.21.15.69:0/1459812914 shutdown_connections 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56374e341e80 0x56374e344270 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e32fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 0x56374e32f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 --2- 172.21.15.69:0/1459812914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327210 0x56374e32f290 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f98077fe640 0 client.16983 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:31.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 -- 172.21.15.69:0/1459812914 >> 172.21.15.69:0/1459812914 conn(0x56374e3237a0 msgr2=0x56374e329480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 -- 172.21.15.69:0/1459812914 shutdown_connections 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 1 -- 172.21.15.69:0/1459812914 wait complete. 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.966+0000 7f9830796740 20 client.16983 trim_cache size 0 max 16384 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9830796740 1 Processor -- start 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9830796740 1 -- start start 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9830796740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e327210 0x56374e329c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9830796740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 0x56374e349870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9830796740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e3323f0 0x56374e349db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9830796740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98281b37a0 con 0x56374e3323f0 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9830796740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98281b1070 con 0x56374e327210 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9830796740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98281aeec0 con 0x56374e327b30 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f982c935640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e3323f0 0x56374e349db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:31.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f982c935640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e3323f0 0x56374e349db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60608/0 (socket says 172.21.15.69:60608) 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f982c935640 1 -- 172.21.15.69:0/4162259426 learned_addr learned my addr 172.21.15.69:0/4162259426 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9827fff640 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e327210 0x56374e329c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f982c935640 1 -- 172.21.15.69:0/4162259426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 msgr2=0x56374e349870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f982c935640 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 0x56374e349870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f982c935640 1 -- 172.21.15.69:0/4162259426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e327210 msgr2=0x56374e329c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f982c935640 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e327210 0x56374e329c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f982c935640 1 -- 172.21.15.69:0/4162259426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374e323240 con 0x56374e3323f0 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.984+0000 7f9827fff640 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e327210 0x56374e329c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f982c935640 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e3323f0 0x56374e349db0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9820017720 tx=0x7f9820004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b492c67b4e226d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f980dffb640 1 -- 172.21.15.69:0/4162259426 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820013040 con 0x56374e3323f0 2024-08-02T17:52:31.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f980dffb640 1 -- 172.21.15.69:0/4162259426 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9820028070 con 0x56374e3323f0 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f980dffb640 1 -- 172.21.15.69:0/4162259426 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820023a50 con 0x56374e3323f0 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f9830796740 1 -- 172.21.15.69:0/4162259426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e3323f0 msgr2=0x56374e349db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f9830796740 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e3323f0 0x56374e349db0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9820017720 tx=0x7f9820004a40 comp rx=0 tx=0).stop 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f9830796740 1 -- 172.21.15.69:0/4162259426 shutdown_connections 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f9830796740 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e3323f0 0x56374e349db0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f9830796740 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327b30 0x56374e349870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f9830796740 1 --2- 172.21.15.69:0/4162259426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e327210 0x56374e329c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.985+0000 7f9830796740 1 -- 172.21.15.69:0/4162259426 >> 172.21.15.69:0/4162259426 conn(0x56374e3237a0 msgr2=0x56374e331200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 -- 172.21.15.69:0/4162259426 shutdown_connections 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 -- 172.21.15.69:0/4162259426 wait complete. 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 -- start start 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 Processor -- start 2024-08-02T17:52:31.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e335430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e335970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e3aa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98281af530 con 0x56374e327b30 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98281b09a0 con 0x56374e3323f0 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98281afc00 con 0x56374e327210 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f9830796740 10 client.0 _ll_register_callbacks cb 0x7f9818014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f98277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e335970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f98277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e335970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60620/0 (socket says 172.21.15.69:60620) 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.986+0000 7f98277fe640 1 -- 172.21.15.69:0/2808831795 learned_addr learned my addr 172.21.15.69:0/2808831795 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f9827fff640 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e335430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f982c935640 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e3aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98277fe640 1 -- 172.21.15.69:0/2808831795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 msgr2=0x56374e335430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98277fe640 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e335430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98277fe640 1 -- 172.21.15.69:0/2808831795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 msgr2=0x56374e3aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:31.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98277fe640 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e3aa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98277fe640 1 -- 172.21.15.69:0/2808831795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374e3aad70 con 0x56374e327b30 2024-08-02T17:52:31.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98277fe640 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e335970 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f981c006010 tx=0x7f981c086ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6732d2a6d67dc9ba server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:31.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98077fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:31.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98077fe640 1 -- 172.21.15.69:0/2808831795 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981c049040 con 0x56374e327b30 2024-08-02T17:52:31.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98077fe640 1 -- 172.21.15.69:0/2808831795 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f981c03b460 con 0x56374e327b30 2024-08-02T17:52:31.007 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:52:31.008 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:52:31.008 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:52:31.008 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:52:31.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f9830796740 1 -- 172.21.15.69:0/2808831795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9820028070 con 0x56374e327b30 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98077fe640 1 -- 172.21.15.69:0/2808831795 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f981c086f40 con 0x56374e327b30 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f9830796740 10 client.16989 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f9830796740 1 -- 172.21.15.69:0/2808831795 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56374e3abed0 con 0x56374e327b30 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98077fe640 1 -- 172.21.15.69:0/2808831795 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9820028070 con 0x56374e327b30 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f98077fe640 1 -- 172.21.15.69:0/2808831795 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f981c0ed360 con 0x56374e327b30 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f9830796740 10 client.16989 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f9830796740 10 client.16989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.987+0000 7f9830796740 20 client.16989 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56374e3ac6d0 0x56374e3aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 4 client.16989 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 -- 172.21.15.69:0/2808831795 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56374e294400 con 0x56374e3ac6d0 2024-08-02T17:52:31.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9827fff640 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56374e3ac6d0 0x56374e3aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9827fff640 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56374e3ac6d0 0x56374e3aeac0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e9ac0d20967d565 server_cookie=3cea15a564beb41f in_seq=0 out_seq=0 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f98077fe640 10 client.16989 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f98077fe640 1 -- 172.21.15.69:0/2808831795 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56374e294400 con 0x56374e3ac6d0 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f98077fe640 10 client.16989 handle_command_reply: tid=0 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 client.16989 shutdown 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 -- 172.21.15.69:0/2808831795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 msgr2=0x56374e335970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e335970 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f981c006010 tx=0x7f981c086ed0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 -- 172.21.15.69:0/2808831795 shutdown_connections 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56374e327b30 0x56374e335970 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56374e3ac6d0 0x56374e3aeac0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56374e3323f0 0x56374e3aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.988+0000 7f9830796740 1 --2- 172.21.15.69:0/2808831795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56374e327210 0x56374e335430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f9830796740 1 -- 172.21.15.69:0/2808831795 >> 172.21.15.69:0/2808831795 conn(0x56374e3237a0 msgr2=0x56374e3220e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f98077fe640 0 client.16989 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f9830796740 1 -- 172.21.15.69:0/2808831795 shutdown_connections 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f9830796740 1 -- 172.21.15.69:0/2808831795 wait complete. 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f9830796740 20 client.16989 trim_cache size 0 max 16384 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 -- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f98100a2cf0 msgr2=0x7f98100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f98100a2cf0 0x7f98100a51b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9820009fd0 tx=0x7f982000c040 comp rx=0 tx=0).stop 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 -- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 msgr2=0x7f9828198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 0x7f9828198800 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f981c005ab0 tx=0x7f981c0042f0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 -- 172.21.15.69:0/2300197635 shutdown_connections 2024-08-02T17:52:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9828103390 0x7f9828198800 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f98100a2cf0 0x7f98100a51b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f982810bea0 0x7f982819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 --2- 172.21.15.69:0/2300197635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9828102a70 0x7f98281982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.989+0000 7f982e398640 1 -- 172.21.15.69:0/2300197635 >> 172.21.15.69:0/2300197635 conn(0x7f98280fc4d0 msgr2=0x7f98280fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.990+0000 7f982e398640 1 -- 172.21.15.69:0/2300197635 shutdown_connections 2024-08-02T17:52:31.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:30.990+0000 7f982e398640 1 -- 172.21.15.69:0/2300197635 wait complete. 2024-08-02T17:52:32.004 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:52:32.005 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:52:32.267 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:32.510 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:32 smithi069 ceph-mon[32746]: pgmap v186: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 893 KiB/s wr, 117 op/s 2024-08-02T17:52:32.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:32 smithi077 ceph-mon[37108]: pgmap v186: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 893 KiB/s wr, 117 op/s 2024-08-02T17:52:32.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 -- 172.21.15.69:0/2441010482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501078b0 msgr2=0x7f8c50107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 --2- 172.21.15.69:0/2441010482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501078b0 0x7f8c50107c90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009930 tx=0x7f8c4402a420 comp rx=0 tx=0).stop 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 -- 172.21.15.69:0/2441010482 shutdown_connections 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 --2- 172.21.15.69:0/2441010482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c50102250 0x7f8c5010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 --2- 172.21.15.69:0/2441010482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c501018b0 0x7f8c50101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 --2- 172.21.15.69:0/2441010482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501078b0 0x7f8c50107c90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 -- 172.21.15.69:0/2441010482 >> 172.21.15.69:0/2441010482 conn(0x7f8c500fd5f0 msgr2=0x7f8c500ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 -- 172.21.15.69:0/2441010482 shutdown_connections 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.721+0000 7f8c580d2640 1 -- 172.21.15.69:0/2441010482 wait complete. 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c580d2640 1 Processor -- start 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c580d2640 1 -- start start 2024-08-02T17:52:32.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c580d2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501018b0 0x7f8c50198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c580d2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50102250 0x7f8c50198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c580d2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c501078b0 0x7f8c5019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c580d2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50111c70 con 0x7f8c501018b0 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c580d2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50111af0 con 0x7f8c501078b0 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c580d2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8c50111df0 con 0x7f8c50102250 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c55e6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501018b0 0x7f8c50198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c55e6e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501018b0 0x7f8c50198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60636/0 (socket says 172.21.15.69:60636) 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c55e6e640 1 -- 172.21.15.69:0/3148742299 learned_addr learned my addr 172.21.15.69:0/3148742299 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c5666f640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c501078b0 0x7f8c5019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c5566d640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50102250 0x7f8c50198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c55e6e640 1 -- 172.21.15.69:0/3148742299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50102250 msgr2=0x7f8c50198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:32.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c55e6e640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50102250 0x7f8c50198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c55e6e640 1 -- 172.21.15.69:0/3148742299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c501078b0 msgr2=0x7f8c5019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c55e6e640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c501078b0 0x7f8c5019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c55e6e640 1 -- 172.21.15.69:0/3148742299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5019d100 con 0x7f8c501018b0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.722+0000 7f8c5666f640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c501078b0 0x7f8c5019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.723+0000 7f8c55e6e640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501018b0 0x7f8c50198150 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009a50 tx=0x7f8c44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.723+0000 7f8c42ffd640 1 -- 172.21.15.69:0/3148742299 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4403f030 con 0x7f8c501018b0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.723+0000 7f8c42ffd640 1 -- 172.21.15.69:0/3148742299 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c44003ce0 con 0x7f8c501018b0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.723+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5019d390 con 0x7f8c501018b0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.723+0000 7f8c42ffd640 1 -- 172.21.15.69:0/3148742299 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44035070 con 0x7f8c501018b0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.723+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c501a4cd0 con 0x7f8c501018b0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.724+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c50102ff0 con 0x7f8c501018b0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.725+0000 7f8c42ffd640 1 -- 172.21.15.69:0/3148742299 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8c44042030 con 0x7f8c501018b0 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.726+0000 7f8c42ffd640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c3c0a2cf0 0x7f8c3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:32.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.726+0000 7f8c5566d640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c3c0a2cf0 0x7f8c3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:32.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.726+0000 7f8c42ffd640 1 -- 172.21.15.69:0/3148742299 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8c440efe80 con 0x7f8c501018b0 2024-08-02T17:52:32.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.726+0000 7f8c5566d640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c3c0a2cf0 0x7f8c3c0a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005e10 tx=0x7f8c4c005da0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:32.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.730+0000 7f8c42ffd640 1 -- 172.21.15.69:0/3148742299 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8c440bc6e0 con 0x7f8c501018b0 2024-08-02T17:52:32.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:32 smithi083 ceph-mon[37031]: pgmap v186: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 893 KiB/s wr, 117 op/s 2024-08-02T17:52:32.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.932+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c50109e20 con 0x7f8c501018b0 2024-08-02T17:52:32.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.933+0000 7f8c42ffd640 1 -- 172.21.15.69:0/3148742299 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f8c440bbfe0 con 0x7f8c501018b0 2024-08-02T17:52:32.936 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:52:32.936 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c3c0a2cf0 msgr2=0x7f8c3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c3c0a2cf0 0x7f8c3c0a51b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c005e10 tx=0x7f8c4c005da0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501018b0 msgr2=0x7f8c50198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501018b0 0x7f8c50198150 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009a50 tx=0x7f8c44004270 comp rx=0 tx=0).stop 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 shutdown_connections 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8c3c0a2cf0 0x7f8c3c0a51b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8c501078b0 0x7f8c5019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8c50102250 0x7f8c50198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 --2- 172.21.15.69:0/3148742299 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8c501018b0 0x7f8c50198150 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 >> 172.21.15.69:0/3148742299 conn(0x7f8c500fd5f0 msgr2=0x7f8c500fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:32.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 shutdown_connections 2024-08-02T17:52:32.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:32.935+0000 7f8c580d2640 1 -- 172.21.15.69:0/3148742299 wait complete. 2024-08-02T17:52:33.587 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:33 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3148742299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:33.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:33 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3148742299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:33.617 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:52:33.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:33 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3148742299' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:33.873 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:34.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 -- 172.21.15.69:0/1673248975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 msgr2=0x7fd620106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1673248975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 0x7fd620106b90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd61c009930 tx=0x7fd61c02a420 comp rx=0 tx=0).stop 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 -- 172.21.15.69:0/1673248975 shutdown_connections 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1673248975 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd620101150 0x7fd62010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1673248975 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6201007b0 0x7fd620100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1673248975 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 0x7fd620106b90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 -- 172.21.15.69:0/1673248975 >> 172.21.15.69:0/1673248975 conn(0x7fd6200fc4d0 msgr2=0x7fd6200fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 -- 172.21.15.69:0/1673248975 shutdown_connections 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.338+0000 7fd6287cb640 1 -- 172.21.15.69:0/1673248975 wait complete. 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd6287cb640 1 Processor -- start 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd6287cb640 1 -- start start 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd6287cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6201007b0 0x7fd6200fcac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd6287cb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd620101150 0x7fd6200fd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd6287cb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 0x7fd6200fd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd6287cb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd620110bf0 con 0x7fd6201067b0 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd6287cb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd620110a70 con 0x7fd620101150 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd6287cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd620110d70 con 0x7fd6201007b0 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd626d68640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 0x7fd6200fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd626d68640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 0x7fd6200fd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60662/0 (socket says 172.21.15.69:60662) 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.339+0000 7fd626d68640 1 -- 172.21.15.69:0/1193411548 learned_addr learned my addr 172.21.15.69:0/1193411548 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd626567640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6201007b0 0x7fd6200fcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd625d66640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd620101150 0x7fd6200fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd626d68640 1 -- 172.21.15.69:0/1193411548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6201007b0 msgr2=0x7fd6200fcac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd626d68640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6201007b0 0x7fd6200fcac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd626d68640 1 -- 172.21.15.69:0/1193411548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd620101150 msgr2=0x7fd6200fd000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd626d68640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd620101150 0x7fd6200fd000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd626d68640 1 -- 172.21.15.69:0/1193411548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6201a66d0 con 0x7fd6201067b0 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd626d68640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 0x7fd6200fd540 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd61800eaa0 tx=0x7fd61800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd6177fe640 1 -- 172.21.15.69:0/1193411548 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800cdb0 con 0x7fd6201067b0 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd6177fe640 1 -- 172.21.15.69:0/1193411548 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd618005020 con 0x7fd6201067b0 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd6287cb640 1 -- 172.21.15.69:0/1193411548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6201a69c0 con 0x7fd6201067b0 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd6177fe640 1 -- 172.21.15.69:0/1193411548 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180116b0 con 0x7fd6201067b0 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.340+0000 7fd6287cb640 1 -- 172.21.15.69:0/1193411548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6201a6f50 con 0x7fd6201067b0 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd6177fe640 1 -- 172.21.15.69:0/1193411548 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd618005560 con 0x7fd6201067b0 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd62abc9740 1 Processor -- start 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd62abc9740 1 -- start start 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 0x555df43dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 0x555df43e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd62abc9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555df4341a40 con 0x555df43e71d0 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd62abc9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555df42d99d0 con 0x555df43dc910 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd62abc9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555df433da00 con 0x555df43dbff0 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd625d66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd625d66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60674/0 (socket says 172.21.15.69:60674) 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd626567640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 0x555df43e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd625d66640 1 -- 172.21.15.69:0/3010456934 learned_addr learned my addr 172.21.15.69:0/3010456934 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd6177fe640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd60c0a2d70 0x7fd60c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.343+0000 7fd625d66640 1 --2- 172.21.15.69:0/3010456934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 0x555df43dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd626d68640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd60c0a2d70 0x7fd60c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd6177fe640 1 -- 172.21.15.69:0/1193411548 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd6180cf9e0 con 0x7fd6201067b0 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd625d66640 1 -- 172.21.15.69:0/3010456934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 msgr2=0x555df43dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd625d66640 1 --2- 172.21.15.69:0/3010456934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 0x555df43dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd625d66640 1 -- 172.21.15.69:0/3010456934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 msgr2=0x555df43e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd625d66640 1 --2- 172.21.15.69:0/3010456934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 0x555df43e6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd625d66640 1 -- 172.21.15.69:0/3010456934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555df42e0650 con 0x555df43e71d0 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd625d66640 1 --2- 172.21.15.69:0/3010456934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e95c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd610016530 tx=0x7fd61000a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec10e22ce0bb8f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.344+0000 7fd626d68640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd60c0a2d70 0x7fd60c0a5230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6180cce20 tx=0x7fd6180054d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.347+0000 7fd614ff9640 1 -- 172.21.15.69:0/3010456934 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610016df0 con 0x555df43e71d0 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.347+0000 7fd614ff9640 1 -- 172.21.15.69:0/3010456934 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd610018910 con 0x555df43e71d0 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.347+0000 7fd614ff9640 1 -- 172.21.15.69:0/3010456934 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6100178a0 con 0x555df43e71d0 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.347+0000 7fd62abc9740 1 -- 172.21.15.69:0/3010456934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 msgr2=0x555df43e95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 --2- 172.21.15.69:0/3010456934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e95c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd610016530 tx=0x7fd61000a2b0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 -- 172.21.15.69:0/3010456934 shutdown_connections 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 --2- 172.21.15.69:0/3010456934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e95c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 --2- 172.21.15.69:0/3010456934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 0x555df43e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 --2- 172.21.15.69:0/3010456934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 0x555df43dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 -- 172.21.15.69:0/3010456934 >> 172.21.15.69:0/3010456934 conn(0x555df43d8580 msgr2=0x555df43d8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 -- 172.21.15.69:0/3010456934 shutdown_connections 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 -- 172.21.15.69:0/3010456934 wait complete. 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 -- start start 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 Processor -- start 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 0x555df43e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dc910 0x555df43e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555df43e9d40 con 0x555df43e71d0 2024-08-02T17:52:34.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555df42fcb20 con 0x555df43dbff0 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.348+0000 7fd62abc9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555df43e9ec0 con 0x555df43dc910 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd62abc9740 10 client.0 _ll_register_callbacks cb 0x555df42fabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60686/0 (socket says 172.21.15.69:60686) 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 -- 172.21.15.69:0/2609298802 learned_addr learned my addr 172.21.15.69:0/2609298802 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd626567640 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dc910 0x555df43e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 0x555df43e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 -- 172.21.15.69:0/2609298802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dc910 msgr2=0x555df43e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dc910 0x555df43e45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 -- 172.21.15.69:0/2609298802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 msgr2=0x555df43e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 0x555df43e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 -- 172.21.15.69:0/2609298802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555df40b33c0 con 0x555df43e71d0 2024-08-02T17:52:34.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd625d66640 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e4b20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd610016e30 tx=0x7fd610017480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41b86f2c606733e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd609ffb640 1 -- 172.21.15.69:0/2609298802 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61002a060 con 0x555df43e71d0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd62abc9740 1 -- 172.21.15.69:0/2609298802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd610018910 con 0x555df43e71d0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd609ffb640 1 -- 172.21.15.69:0/2609298802 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd610018d80 con 0x555df43e71d0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd609ffb640 1 -- 172.21.15.69:0/2609298802 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd610031050 con 0x555df43e71d0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.349+0000 7fd62abc9740 10 client.17007 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd62abc9740 1 -- 172.21.15.69:0/2609298802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555df43f5940 con 0x555df43e71d0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd609ffb640 1 -- 172.21.15.69:0/2609298802 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd610018910 con 0x555df43e71d0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd609ffb640 1 -- 172.21.15.69:0/2609298802 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd610012050 con 0x555df43e71d0 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd62abc9740 10 client.17007 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd62abc9740 10 client.17007 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd62abc9740 20 client.17007 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd62abc9740 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x555df43f6c90 0x555df43f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd62abc9740 4 client.17007 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd62abc9740 1 -- 172.21.15.69:0/2609298802 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555df434efb0 con 0x555df43f6c90 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.350+0000 7fd626567640 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x555df43f6c90 0x555df43f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd626567640 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x555df43f6c90 0x555df43f9080 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a800ed5689ba9806 server_cookie=4bdda49159495afa in_seq=0 out_seq=0 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd609ffb640 10 client.17007 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd609ffb640 1 -- 172.21.15.69:0/2609298802 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555df434efb0 con 0x555df43f6c90 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd609ffb640 10 client.17007 handle_command_reply: tid=0 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 client.17007 shutdown 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 -- 172.21.15.69:0/2609298802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 msgr2=0x555df43e4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e4b20 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd610016e30 tx=0x7fd610017480 comp rx=0 tx=0).stop 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 -- 172.21.15.69:0/2609298802 shutdown_connections 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x555df43f6c90 0x555df43f9080 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43e4b20 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dc910 0x555df43e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 --2- 172.21.15.69:0/2609298802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 0x555df43e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd62abc9740 1 -- 172.21.15.69:0/2609298802 >> 172.21.15.69:0/2609298802 conn(0x555df43d8580 msgr2=0x555df43de260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:34.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.351+0000 7fd609ffb640 0 client.17007 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:52:34.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.352+0000 7fd62abc9740 1 -- 172.21.15.69:0/2609298802 shutdown_connections 2024-08-02T17:52:34.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.352+0000 7fd62abc9740 1 -- 172.21.15.69:0/2609298802 wait complete. 2024-08-02T17:52:34.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.352+0000 7fd62abc9740 20 client.17007 trim_cache size 0 max 16384 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.364+0000 7fd62abc9740 1 Processor -- start 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.364+0000 7fd62abc9740 1 -- start start 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 0x555df43e3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43dc910 0x555df43f2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43e71d0 0x555df43e1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd62abc9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201b39a0 con 0x555df43dc910 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd62abc9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201b1270 con 0x555df43dbff0 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd62abc9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201af0c0 con 0x555df43e71d0 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43dc910 0x555df43f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43dc910 0x555df43f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60698/0 (socket says 172.21.15.69:60698) 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 -- 172.21.15.69:0/3600612106 learned_addr learned my addr 172.21.15.69:0/3600612106 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd625d66640 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 0x555df43e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd625d66640 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43e71d0 0x555df43e1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 -- 172.21.15.69:0/3600612106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43e71d0 msgr2=0x555df43e1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43e71d0 0x555df43e1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 -- 172.21.15.69:0/3600612106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 msgr2=0x555df43e3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 0x555df43e3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 -- 172.21.15.69:0/3600612106 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555df43d83b0 con 0x555df43dc910 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd625d66640 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43e71d0 0x555df43e1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd626567640 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43dc910 0x555df43f2820 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000c00 tx=0x7fd61c031b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c0b2d20535d0ba server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd60bfff640 1 -- 172.21.15.69:0/3600612106 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c035040 con 0x555df43dc910 2024-08-02T17:52:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd60bfff640 1 -- 172.21.15.69:0/3600612106 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c005500 con 0x555df43dc910 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.365+0000 7fd60bfff640 1 -- 172.21.15.69:0/3600612106 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61c004d90 con 0x555df43dc910 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 -- 172.21.15.69:0/3600612106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43dc910 msgr2=0x555df43f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43dc910 0x555df43f2820 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd61c000c00 tx=0x7fd61c031b70 comp rx=0 tx=0).stop 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 -- 172.21.15.69:0/3600612106 shutdown_connections 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43dc910 0x555df43f2820 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43e71d0 0x555df43e1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 --2- 172.21.15.69:0/3600612106 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dbff0 0x555df43e3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 -- 172.21.15.69:0/3600612106 >> 172.21.15.69:0/3600612106 conn(0x555df43d8580 msgr2=0x555df43f2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 -- 172.21.15.69:0/3600612106 shutdown_connections 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 -- 172.21.15.69:0/3600612106 wait complete. 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 -- start start 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.366+0000 7fd62abc9740 1 Processor -- start 2024-08-02T17:52:34.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 0x555df43f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 0x555df43f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd62abc9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43f9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd62abc9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201af730 con 0x555df43e71d0 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd62abc9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201b0ba0 con 0x555df43dc910 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd62abc9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6201afe00 con 0x555df43dbff0 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd62abc9740 10 client.0 _ll_register_callbacks cb 0x7fd61000ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60706/0 (socket says 172.21.15.69:60706) 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 -- 172.21.15.69:0/432983234 learned_addr learned my addr 172.21.15.69:0/432983234 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd626567640 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 0x555df43f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 0x555df43f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 -- 172.21.15.69:0/432983234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 msgr2=0x555df43f8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:34.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 0x555df43f8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 -- 172.21.15.69:0/432983234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 msgr2=0x555df43f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 0x555df43f91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 -- 172.21.15.69:0/432983234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555df445f5c0 con 0x555df43e71d0 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd625d66640 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43f9720 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd610004700 tx=0x7fd6100173e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946a1a85b240a084 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.367+0000 7fd609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd609ffb640 1 -- 172.21.15.69:0/432983234 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61002a040 con 0x555df43e71d0 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 1 -- 172.21.15.69:0/432983234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd61c005500 con 0x555df43e71d0 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd609ffb640 1 -- 172.21.15.69:0/432983234 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fd61000a8c0 con 0x555df43e71d0 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd609ffb640 1 -- 172.21.15.69:0/432983234 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd61000a6c0 con 0x555df43e71d0 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 10 client.17013 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 1 -- 172.21.15.69:0/432983234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555df44607f0 con 0x555df43e71d0 2024-08-02T17:52:34.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd609ffb640 1 -- 172.21.15.69:0/432983234 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd61c005500 con 0x555df43e71d0 2024-08-02T17:52:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd609ffb640 1 -- 172.21.15.69:0/432983234 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd610012050 con 0x555df43e71d0 2024-08-02T17:52:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 10 client.17013 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 10 client.17013 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:52:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 20 client.17013 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x555df4460f40 0x555df4463360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 4 client.17013 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:52:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.368+0000 7fd62abc9740 1 -- 172.21.15.69:0/432983234 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555df43494c0 con 0x555df4460f40 2024-08-02T17:52:34.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd626567640 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x555df4460f40 0x555df4463360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd609ffb640 10 client.17013 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:52:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd626567640 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x555df4460f40 0x555df4463360 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24098f84cf34f210 server_cookie=dd9ba71eb588ad0 in_seq=0 out_seq=0 2024-08-02T17:52:34.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd609ffb640 1 -- 172.21.15.69:0/432983234 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555df43494c0 con 0x555df4460f40 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd609ffb640 10 client.17013 handle_command_reply: tid=0 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 client.17013 shutdown 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 -- 172.21.15.69:0/432983234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 msgr2=0x555df43f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43f9720 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd610004700 tx=0x7fd6100173e0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 -- 172.21.15.69:0/432983234 shutdown_connections 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x555df4460f40 0x555df4463360 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555df43e71d0 0x555df43f9720 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555df43dc910 0x555df43f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 --2- 172.21.15.69:0/432983234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555df43dbff0 0x555df43f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 -- 172.21.15.69:0/432983234 >> 172.21.15.69:0/432983234 conn(0x555df43d8580 msgr2=0x555df43de230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd609ffb640 0 client.17013 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 -- 172.21.15.69:0/432983234 shutdown_connections 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 1 -- 172.21.15.69:0/432983234 wait complete. 2024-08-02T17:52:34.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.369+0000 7fd62abc9740 20 client.17013 trim_cache size 0 max 16384 2024-08-02T17:52:34.378 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:52:34.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 -- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd60c0a2d70 msgr2=0x7fd60c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd60c0a2d70 0x7fd60c0a5230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd6180cce20 tx=0x7fd6180054d0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 -- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 msgr2=0x7fd6200fd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 0x7fd6200fd540 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd61800eaa0 tx=0x7fd61800ef60 comp rx=0 tx=0).stop 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 -- 172.21.15.69:0/1193411548 shutdown_connections 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6201067b0 0x7fd6200fd540 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd60c0a2d70 0x7fd60c0a5230 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd620101150 0x7fd6200fd000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 --2- 172.21.15.69:0/1193411548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6201007b0 0x7fd6200fcac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 -- 172.21.15.69:0/1193411548 >> 172.21.15.69:0/1193411548 conn(0x7fd6200fc4d0 msgr2=0x7fd62006a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 -- 172.21.15.69:0/1193411548 shutdown_connections 2024-08-02T17:52:34.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:34.370+0000 7fd6287cb640 1 -- 172.21.15.69:0/1193411548 wait complete. 2024-08-02T17:52:34.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:34 smithi077 ceph-mon[37108]: pgmap v187: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 979 KiB/s wr, 125 op/s 2024-08-02T17:52:34.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:34 smithi069 ceph-mon[32746]: pgmap v187: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 979 KiB/s wr, 125 op/s 2024-08-02T17:52:34.775 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:34 smithi083 ceph-mon[37031]: pgmap v187: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 979 KiB/s wr, 125 op/s 2024-08-02T17:52:34.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 325.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:52:35.075 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:52:35.330 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:35.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 -- 172.21.15.69:0/394017493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d8101150 msgr2=0x7f28d810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 --2- 172.21.15.69:0/394017493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d8101150 0x7f28d810d680 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009930 tx=0x7f28c802a420 comp rx=0 tx=0).stop 2024-08-02T17:52:35.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 -- 172.21.15.69:0/394017493 shutdown_connections 2024-08-02T17:52:35.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 --2- 172.21.15.69:0/394017493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d8101150 0x7f28d810d680 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 --2- 172.21.15.69:0/394017493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d81007b0 0x7f28d8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 --2- 172.21.15.69:0/394017493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28d81067b0 0x7f28d8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 -- 172.21.15.69:0/394017493 >> 172.21.15.69:0/394017493 conn(0x7f28d80fc4d0 msgr2=0x7f28d80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 -- 172.21.15.69:0/394017493 shutdown_connections 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.768+0000 7f28dd691640 1 -- 172.21.15.69:0/394017493 wait complete. 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28dd691640 1 Processor -- start 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28dd691640 1 -- start start 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28dd691640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28d81007b0 0x7f28d819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28dd691640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d8101150 0x7f28d819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28dd691640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d81067b0 0x7f28d81a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28dd691640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8110bb0 con 0x7f28d81067b0 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28dd691640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8110a30 con 0x7f28d8101150 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28dd691640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28d8110d30 con 0x7f28d81007b0 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28d77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d81067b0 0x7f28d81a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28d77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d81067b0 0x7f28d81a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56236/0 (socket says 172.21.15.69:56236) 2024-08-02T17:52:35.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.769+0000 7f28d77fe640 1 -- 172.21.15.69:0/2561284487 learned_addr learned my addr 172.21.15.69:0/2561284487 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28d81007b0 0x7f28d819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28d67fc640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d8101150 0x7f28d819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28d77fe640 1 -- 172.21.15.69:0/2561284487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28d81007b0 msgr2=0x7f28d819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28d77fe640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28d81007b0 0x7f28d819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28d77fe640 1 -- 172.21.15.69:0/2561284487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d8101150 msgr2=0x7f28d819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28d77fe640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d8101150 0x7f28d819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28d77fe640 1 -- 172.21.15.69:0/2561284487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d81a3780 con 0x7f28d81067b0 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28d77fe640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d81067b0 0x7f28d81a3050 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f28c8005ab0 tx=0x7f28c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28cffff640 1 -- 172.21.15.69:0/2561284487 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c803f030 con 0x7f28d81067b0 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28cffff640 1 -- 172.21.15.69:0/2561284487 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c8003ce0 con 0x7f28d81067b0 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28dd691640 1 -- 172.21.15.69:0/2561284487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d81a3a10 con 0x7f28d81067b0 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28cffff640 1 -- 172.21.15.69:0/2561284487 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8035070 con 0x7f28d81067b0 2024-08-02T17:52:35.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.770+0000 7f28dd691640 1 -- 172.21.15.69:0/2561284487 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d81ab300 con 0x7f28d81067b0 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28cffff640 1 -- 172.21.15.69:0/2561284487 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f28c8042030 con 0x7f28d81067b0 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28dfa8f740 1 Processor -- start 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28dfa8f740 1 -- start start 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1ee4910 0x5648f1eeec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1eef1d0 0x5648f1ef15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648f1e49a40 con 0x5648f1ee4910 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5648f1de19d0 con 0x5648f1ee3ff0 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5648f1e45a00 con 0x5648f1eef1d0 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28cffff640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f28c40a2cf0 0x7f28c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28d6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1ee4910 0x5648f1eeec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28d6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1ee4910 0x5648f1eeec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56242/0 (socket says 172.21.15.69:56242) 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.773+0000 7f28d6ffd640 1 -- 172.21.15.69:0/3046683480 learned_addr learned my addr 172.21.15.69:0/3046683480 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:35.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d77fe640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f28c40a2cf0 0x7f28c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d67fc640 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1eef1d0 0x5648f1ef15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d67fc640 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28cffff640 1 -- 172.21.15.69:0/2561284487 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f28c80f0080 con 0x7f28d81067b0 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d6ffd640 1 -- 172.21.15.69:0/3046683480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1eef1d0 msgr2=0x5648f1ef15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1eef1d0 0x5648f1ef15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d6ffd640 1 -- 172.21.15.69:0/3046683480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 msgr2=0x5648f1ee43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d6ffd640 1 -- 172.21.15.69:0/3046683480 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f1de8650 con 0x5648f1ee4910 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d67fc640 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1ee4910 0x5648f1eeec90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f28d000ae80 tx=0x7f28d000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a8340533404239 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28d77fe640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f28c40a2cf0 0x7f28c40a51b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f28c80eda10 tx=0x7f28c808f250 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28cd7fa640 1 -- 172.21.15.69:0/3046683480 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0004d20 con 0x5648f1ee4910 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28cd7fa640 1 -- 172.21.15.69:0/3046683480 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d0010b20 con 0x5648f1ee4910 2024-08-02T17:52:35.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.774+0000 7f28cd7fa640 1 -- 172.21.15.69:0/3046683480 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d00057e0 con 0x5648f1ee4910 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 -- 172.21.15.69:0/3046683480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1ee4910 msgr2=0x5648f1eeec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1ee4910 0x5648f1eeec90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f28d000ae80 tx=0x7f28d000f790 comp rx=0 tx=0).stop 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 -- 172.21.15.69:0/3046683480 shutdown_connections 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1ee4910 0x5648f1eeec90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1eef1d0 0x5648f1ef15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/3046683480 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 -- 172.21.15.69:0/3046683480 >> 172.21.15.69:0/3046683480 conn(0x5648f1ee0580 msgr2=0x5648f1ee0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 -- 172.21.15.69:0/3046683480 shutdown_connections 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 -- 172.21.15.69:0/3046683480 wait complete. 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 -- start start 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.775+0000 7f28dfa8f740 1 Processor -- start 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1eec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1eec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1eecac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.800 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5648f1ef1da0 con 0x5648f1eef1d0 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5648f1e04b20 con 0x5648f1ee3ff0 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5648f1ef1f20 con 0x5648f1ee4910 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28dfa8f740 10 client.0 _ll_register_callbacks cb 0x5648f1e02bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1eecac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1eecac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56244/0 (socket says 172.21.15.69:56244) 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 -- 172.21.15.69:0/2047004093 learned_addr learned my addr 172.21.15.69:0/2047004093 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1eec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1eec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 -- 172.21.15.69:0/2047004093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 msgr2=0x5648f1eec580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1eec580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 -- 172.21.15.69:0/2047004093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 msgr2=0x5648f1eec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1eec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 -- 172.21.15.69:0/2047004093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f1bbb3c0 con 0x5648f1eef1d0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28d67fc640 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1eecac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f28c000a5c0 tx=0x7f28c000a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f5e00100b3ec80 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28be7fc640 1 -- 172.21.15.69:0/2047004093 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c001a040 con 0x5648f1eef1d0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28dfa8f740 1 -- 172.21.15.69:0/2047004093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28d0010b20 con 0x5648f1eef1d0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.776+0000 7f28be7fc640 1 -- 172.21.15.69:0/2047004093 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f28c0005220 con 0x5648f1eef1d0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28be7fc640 1 -- 172.21.15.69:0/2047004093 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28c0027050 con 0x5648f1eef1d0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28dfa8f740 10 client.17025 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28dfa8f740 1 -- 172.21.15.69:0/2047004093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5648f1efd9b0 con 0x5648f1eef1d0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28be7fc640 1 -- 172.21.15.69:0/2047004093 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d0010b20 con 0x5648f1eef1d0 2024-08-02T17:52:35.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28be7fc640 1 -- 172.21.15.69:0/2047004093 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f28c001f3b0 con 0x5648f1eef1d0 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28dfa8f740 10 client.17025 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28dfa8f740 10 client.17025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28dfa8f740 20 client.17025 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5648f1efedc0 0x5648f1f011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28dfa8f740 4 client.17025 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28dfa8f740 1 -- 172.21.15.69:0/2047004093 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648f1e56fb0 con 0x5648f1efedc0 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.777+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5648f1efedc0 0x5648f1f011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.778+0000 7f28be7fc640 10 client.17025 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.778+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5648f1efedc0 0x5648f1f011b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2daa8275740c889b server_cookie=572118ff1f60add in_seq=0 out_seq=0 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.778+0000 7f28be7fc640 1 -- 172.21.15.69:0/2047004093 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5648f1e56fb0 con 0x5648f1efedc0 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.778+0000 7f28be7fc640 10 client.17025 handle_command_reply: tid=0 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.778+0000 7f28dfa8f740 1 client.17025 shutdown 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 -- 172.21.15.69:0/2047004093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 msgr2=0x5648f1eecac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.803 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1eecac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f28c000a5c0 tx=0x7f28c000a530 comp rx=0 tx=0).stop 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 -- 172.21.15.69:0/2047004093 shutdown_connections 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5648f1efedc0 0x5648f1f011b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1eecac0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1eec580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/2047004093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1eec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28be7fc640 0 client.17025 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 -- 172.21.15.69:0/2047004093 >> 172.21.15.69:0/2047004093 conn(0x5648f1ee0580 msgr2=0x5648f1ee62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 -- 172.21.15.69:0/2047004093 shutdown_connections 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 1 -- 172.21.15.69:0/2047004093 wait complete. 2024-08-02T17:52:35.804 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.779+0000 7f28dfa8f740 20 client.17025 trim_cache size 0 max 16384 2024-08-02T17:52:35.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28dfa8f740 1 Processor -- start 2024-08-02T17:52:35.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28dfa8f740 1 -- start start 2024-08-02T17:52:35.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1f06a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1f06f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b7bf0 con 0x5648f1eef1d0 2024-08-02T17:52:35.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b54c0 con 0x5648f1ee3ff0 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b3310 con 0x5648f1ee4910 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28d67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1f06f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28d67fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1f06f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56250/0 (socket says 172.21.15.69:56250) 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.792+0000 7f28d67fc640 1 -- 172.21.15.69:0/1534604954 learned_addr learned my addr 172.21.15.69:0/1534604954 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d67fc640 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1f06a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d67fc640 1 -- 172.21.15.69:0/1534604954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 msgr2=0x5648f1f06a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d67fc640 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1f06a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d67fc640 1 -- 172.21.15.69:0/1534604954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 msgr2=0x5648f1ee6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d67fc640 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d67fc640 1 -- 172.21.15.69:0/1534604954 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648f1f01810 con 0x5648f1eef1d0 2024-08-02T17:52:35.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1f06a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28d67fc640 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1f06f80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f28c000fe60 tx=0x7f28c0016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc79437b089907 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28ccff9640 1 -- 172.21.15.69:0/1534604954 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0005d60 con 0x5648f1eef1d0 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28ccff9640 1 -- 172.21.15.69:0/1534604954 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c001fe10 con 0x5648f1eef1d0 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.793+0000 7f28ccff9640 1 -- 172.21.15.69:0/1534604954 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c001fbd0 con 0x5648f1eef1d0 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 -- 172.21.15.69:0/1534604954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 msgr2=0x5648f1f06f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1f06f80 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f28c000fe60 tx=0x7f28c0016040 comp rx=0 tx=0).stop 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 -- 172.21.15.69:0/1534604954 shutdown_connections 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1f06f80 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1f06a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/1534604954 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ee6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 -- 172.21.15.69:0/1534604954 >> 172.21.15.69:0/1534604954 conn(0x5648f1ee0580 msgr2=0x5648f1edb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 -- 172.21.15.69:0/1534604954 shutdown_connections 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 -- 172.21.15.69:0/1534604954 wait complete. 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 -- start start 2024-08-02T17:52:35.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.794+0000 7f28dfa8f740 1 Processor -- start 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ef7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1ef2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28dfa8f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1ef2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b3980 con 0x5648f1eef1d0 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b4df0 con 0x5648f1ee3ff0 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28dfa8f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f28d81b4050 con 0x5648f1ee4910 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28dfa8f740 10 client.0 _ll_register_callbacks cb 0x7f28c000ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ef7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ef7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43092/0 (socket says 172.21.15.69:43092) 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 -- 172.21.15.69:0/377167317 learned_addr learned my addr 172.21.15.69:0/377167317 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1ef2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1ef2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 -- 172.21.15.69:0/377167317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 msgr2=0x5648f1ef2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:35.812 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1ef2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 -- 172.21.15.69:0/377167317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 msgr2=0x5648f1ef7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ef7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 -- 172.21.15.69:0/377167317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648f1ef3510 con 0x5648f1eef1d0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28d67fc640 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1ef2f50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f28c001f8f0 tx=0x7f28c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5595e70ef292d5b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.795+0000 7f28be7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28be7fc640 1 -- 172.21.15.69:0/377167317 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c001fb70 con 0x5648f1eef1d0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28be7fc640 1 -- 172.21.15.69:0/377167317 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f28c00046c0 con 0x5648f1eef1d0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 1 -- 172.21.15.69:0/377167317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c001fe10 con 0x5648f1eef1d0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28be7fc640 1 -- 172.21.15.69:0/377167317 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f28c0044050 con 0x5648f1eef1d0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 10 client.17031 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 1 -- 172.21.15.69:0/377167317 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5648f1f69710 con 0x5648f1eef1d0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28be7fc640 1 -- 172.21.15.69:0/377167317 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c001fe10 con 0x5648f1eef1d0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28be7fc640 1 -- 172.21.15.69:0/377167317 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f28c0020440 con 0x5648f1eef1d0 2024-08-02T17:52:35.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 10 client.17031 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 10 client.17031 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 20 client.17031 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5648f1f69e20 0x5648f1f6c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 4 client.17031 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.796+0000 7f28dfa8f740 1 -- 172.21.15.69:0/377167317 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5648f1e514c0 con 0x5648f1f69e20 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5648f1f69e20 0x5648f1f6c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28d6ffd640 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5648f1f69e20 0x5648f1f6c240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af159424a70e40d5 server_cookie=c3e6d71240cbbaa3 in_seq=0 out_seq=0 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28be7fc640 10 client.17031 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28be7fc640 1 -- 172.21.15.69:0/377167317 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5648f1e514c0 con 0x5648f1f69e20 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28be7fc640 10 client.17031 handle_command_reply: tid=0 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 client.17031 shutdown 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 -- 172.21.15.69:0/377167317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 msgr2=0x5648f1ef2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1ef2f50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f28c001f8f0 tx=0x7f28c0000bc0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 -- 172.21.15.69:0/377167317 shutdown_connections 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5648f1f69e20 0x5648f1f6c240 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5648f1eef1d0 0x5648f1ef2f50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28be7fc640 0 client.17031 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5648f1ee4910 0x5648f1ef2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 --2- 172.21.15.69:0/377167317 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5648f1ee3ff0 0x5648f1ef7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.797+0000 7f28dfa8f740 1 -- 172.21.15.69:0/377167317 >> 172.21.15.69:0/377167317 conn(0x5648f1ee0580 msgr2=0x5648f1eded90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dfa8f740 1 -- 172.21.15.69:0/377167317 shutdown_connections 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dfa8f740 1 -- 172.21.15.69:0/377167317 wait complete. 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dfa8f740 20 client.17031 trim_cache size 0 max 16384 2024-08-02T17:52:35.815 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:52:35.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 -- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f28c40a2cf0 msgr2=0x7f28c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f28c40a2cf0 0x7f28c40a51b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f28c80eda10 tx=0x7f28c808f250 comp rx=0 tx=0).stop 2024-08-02T17:52:35.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 -- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d81067b0 msgr2=0x7f28d81a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:35.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d81067b0 0x7f28d81a3050 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f28c8005ab0 tx=0x7f28c80042f0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 -- 172.21.15.69:0/2561284487 shutdown_connections 2024-08-02T17:52:35.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f28d81067b0 0x7f28d81a3050 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f28c40a2cf0 0x7f28c40a51b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f28d8101150 0x7f28d819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 --2- 172.21.15.69:0/2561284487 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f28d81007b0 0x7f28d819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:35.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 -- 172.21.15.69:0/2561284487 >> 172.21.15.69:0/2561284487 conn(0x7f28d80fc4d0 msgr2=0x7f28d806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:35.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 -- 172.21.15.69:0/2561284487 shutdown_connections 2024-08-02T17:52:35.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:35.798+0000 7f28dd691640 1 -- 172.21.15.69:0/2561284487 wait complete. 2024-08-02T17:52:36.586 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:36 smithi069 ceph-mon[32746]: pgmap v188: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 974 KiB/s wr, 123 op/s 2024-08-02T17:52:36.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:36 smithi077 ceph-mon[37108]: pgmap v188: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 974 KiB/s wr, 123 op/s 2024-08-02T17:52:36.615 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:52:36.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:36 smithi083 ceph-mon[37031]: pgmap v188: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 974 KiB/s wr, 123 op/s 2024-08-02T17:52:36.871 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 -- 172.21.15.69:0/600419126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 msgr2=0x7f41cc100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 --2- 172.21.15.69:0/600419126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 0x7f41cc100c10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009930 tx=0x7f41c002a420 comp rx=0 tx=0).stop 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 -- 172.21.15.69:0/600419126 shutdown_connections 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 --2- 172.21.15.69:0/600419126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 0x7f41cc100c10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 --2- 172.21.15.69:0/600419126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41cc101150 0x7f41cc10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 --2- 172.21.15.69:0/600419126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41cc1067b0 0x7f41cc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 -- 172.21.15.69:0/600419126 >> 172.21.15.69:0/600419126 conn(0x7f41cc0fc4d0 msgr2=0x7f41cc0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 -- 172.21.15.69:0/600419126 shutdown_connections 2024-08-02T17:52:37.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.328+0000 7f41d1037640 1 -- 172.21.15.69:0/600419126 wait complete. 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41d1037640 1 Processor -- start 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41d1037640 1 -- start start 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41d1037640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 0x7f41cc1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41d1037640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41cc101150 0x7f41cc198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41d1037640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41cc1067b0 0x7f41cc19c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41d1037640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc110bb0 con 0x7f41cc1007b0 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41d1037640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc110a30 con 0x7f41cc101150 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41d1037640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc110d30 con 0x7f41cc1067b0 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41ca59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41cc101150 0x7f41cc198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41cad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 0x7f41cc1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41cb59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41cc1067b0 0x7f41cc19c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.329+0000 7f41ca59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41cc101150 0x7f41cc198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43112/0 (socket says 172.21.15.69:43112) 2024-08-02T17:52:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41ca59c640 1 -- 172.21.15.69:0/2973433695 learned_addr learned my addr 172.21.15.69:0/2973433695 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41cad9d640 1 -- 172.21.15.69:0/2973433695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41cc1067b0 msgr2=0x7f41cc19c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41cad9d640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41cc1067b0 0x7f41cc19c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41cad9d640 1 -- 172.21.15.69:0/2973433695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41cc101150 msgr2=0x7f41cc198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41cad9d640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41cc101150 0x7f41cc198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41cad9d640 1 -- 172.21.15.69:0/2973433695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41cc19d0a0 con 0x7f41cc1007b0 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41ca59c640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41cc101150 0x7f41cc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41cad9d640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 0x7f41cc1980f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e9f0 tx=0x7f41bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41b3fff640 1 -- 172.21.15.69:0/2973433695 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc00ccc0 con 0x7f41cc1007b0 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41d1037640 1 -- 172.21.15.69:0/2973433695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41cc19d390 con 0x7f41cc1007b0 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41b3fff640 1 -- 172.21.15.69:0/2973433695 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41bc010910 con 0x7f41cc1007b0 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41b3fff640 1 -- 172.21.15.69:0/2973433695 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc018960 con 0x7f41cc1007b0 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.330+0000 7f41d1037640 1 -- 172.21.15.69:0/2973433695 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41cc1a4c70 con 0x7f41cc1007b0 2024-08-02T17:52:37.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41b3fff640 1 -- 172.21.15.69:0/2973433695 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f41bc010ac0 con 0x7f41cc1007b0 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41d3435740 1 Processor -- start 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41d3435740 1 -- start start 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41d3435740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab3c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41d3435740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3c910 0x558b6ab46c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41d3435740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab495c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41b3fff640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41b40a2cf0 0x7f41b40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41d3435740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558b6aaa1a40 con 0x558b6ab3bff0 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41d3435740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558b6aa399d0 con 0x558b6ab471d0 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41d3435740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b6aa9da00 con 0x558b6ab3c910 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41cb59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab3c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41cb59e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab3c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56290/0 (socket says 172.21.15.69:56290) 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41cb59e640 1 -- 172.21.15.69:0/2387910227 learned_addr learned my addr 172.21.15.69:0/2387910227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41cb59e640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41b40a2cf0 0x7f41b40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.333+0000 7f41cad9d640 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3c910 0x558b6ab46c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41ca59c640 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41b3fff640 1 -- 172.21.15.69:0/2973433695 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f41bc0cfb30 con 0x7f41cc1007b0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41cb59e640 1 -- 172.21.15.69:0/2387910227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3c910 msgr2=0x558b6ab46c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41cb59e640 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3c910 0x558b6ab46c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41cb59e640 1 -- 172.21.15.69:0/2387910227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 msgr2=0x558b6ab495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41cb59e640 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41cb59e640 1 -- 172.21.15.69:0/2387910227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b6aa40650 con 0x558b6ab3bff0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41ca59c640 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41cb59e640 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab3c3d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f41c4014f10 tx=0x7f41c40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d791f9d1830bd8ab server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41cb59e640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41b40a2cf0 0x7f41b40a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41c4005c40 tx=0x7f41c4016000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41b17fa640 1 -- 172.21.15.69:0/2387910227 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4024070 con 0x558b6ab3bff0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41b17fa640 1 -- 172.21.15.69:0/2387910227 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41c4007ce0 con 0x558b6ab3bff0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.334+0000 7f41b17fa640 1 -- 172.21.15.69:0/2387910227 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c4005760 con 0x558b6ab3bff0 2024-08-02T17:52:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 -- 172.21.15.69:0/2387910227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 msgr2=0x558b6ab3c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab3c3d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f41c4014f10 tx=0x7f41c40117b0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 -- 172.21.15.69:0/2387910227 shutdown_connections 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3c910 0x558b6ab46c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 --2- 172.21.15.69:0/2387910227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab3c3d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 -- 172.21.15.69:0/2387910227 >> 172.21.15.69:0/2387910227 conn(0x558b6ab38580 msgr2=0x558b6ab38960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 -- 172.21.15.69:0/2387910227 shutdown_connections 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 -- 172.21.15.69:0/2387910227 wait complete. 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 -- start start 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 Processor -- start 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3bff0 0x558b6ab440e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.335+0000 7f41d3435740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab44620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41d3435740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab44b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41d3435740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558b6ab49da0 con 0x558b6ab3c910 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab44620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41d3435740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558b6aa5cb20 con 0x558b6ab471d0 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41d3435740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558b6ab49f20 con 0x558b6ab3bff0 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab44620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56294/0 (socket says 172.21.15.69:56294) 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 -- 172.21.15.69:0/4056804302 learned_addr learned my addr 172.21.15.69:0/4056804302 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41d3435740 10 client.0 _ll_register_callbacks cb 0x558b6aa5abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41cb59e640 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab44b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41cad9d640 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3bff0 0x558b6ab440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 -- 172.21.15.69:0/4056804302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3bff0 msgr2=0x558b6ab440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3bff0 0x558b6ab440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 -- 172.21.15.69:0/4056804302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 msgr2=0x558b6ab44b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab44b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 -- 172.21.15.69:0/4056804302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b6a8133c0 con 0x558b6ab3c910 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41ca59c640 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab44620 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009a50 tx=0x7f41c0031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d9374aa018cca1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41aeff5640 1 -- 172.21.15.69:0/4056804302 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0040070 con 0x558b6ab3c910 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41aeff5640 1 -- 172.21.15.69:0/4056804302 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f41c0004460 con 0x558b6ab3c910 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41d3435740 1 -- 172.21.15.69:0/4056804302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558b6aaaefb0 con 0x558b6ab3c910 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41aeff5640 1 -- 172.21.15.69:0/4056804302 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f41c0035040 con 0x558b6ab3c910 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41d3435740 10 client.17043 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.336+0000 7f41d3435740 1 -- 172.21.15.69:0/4056804302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b6ab55920 con 0x558b6ab3c910 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41aeff5640 1 -- 172.21.15.69:0/4056804302 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558b6aaaefb0 con 0x558b6ab3c910 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41aeff5640 1 -- 172.21.15.69:0/4056804302 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f41c00035f0 con 0x558b6ab3c910 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41d3435740 10 client.17043 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41d3435740 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41d3435740 20 client.17043 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41d3435740 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558b6ab56de0 0x558b6ab59250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41d3435740 4 client.17043 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41d3435740 1 -- 172.21.15.69:0/4056804302 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b6aaa94c0 con 0x558b6ab56de0 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.337+0000 7f41cad9d640 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558b6ab56de0 0x558b6ab59250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.338+0000 7f41aeff5640 10 client.17043 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.338+0000 7f41cad9d640 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558b6ab56de0 0x558b6ab59250 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db71b4ca9e65955 server_cookie=c104b146321566b6 in_seq=0 out_seq=0 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.338+0000 7f41aeff5640 1 -- 172.21.15.69:0/4056804302 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558b6aaa94c0 con 0x558b6ab56de0 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.338+0000 7f41aeff5640 10 client.17043 handle_command_reply: tid=0 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.338+0000 7f41d3435740 1 client.17043 shutdown 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 -- 172.21.15.69:0/4056804302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 msgr2=0x558b6ab44620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab44620 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009a50 tx=0x7f41c0031940 comp rx=0 tx=0).stop 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 -- 172.21.15.69:0/4056804302 shutdown_connections 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab44620 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558b6ab56de0 0x558b6ab59250 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab471d0 0x558b6ab44b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 --2- 172.21.15.69:0/4056804302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab3bff0 0x558b6ab440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 -- 172.21.15.69:0/4056804302 >> 172.21.15.69:0/4056804302 conn(0x558b6ab38580 msgr2=0x558b6ab3e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41aeff5640 0 client.17043 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:37.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 -- 172.21.15.69:0/4056804302 shutdown_connections 2024-08-02T17:52:37.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 1 -- 172.21.15.69:0/4056804302 wait complete. 2024-08-02T17:52:37.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.339+0000 7f41d3435740 20 client.17043 trim_cache size 0 max 16384 2024-08-02T17:52:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41d3435740 1 Processor -- start 2024-08-02T17:52:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41d3435740 1 -- start start 2024-08-02T17:52:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41d3435740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41d3435740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3c910 0x558b6ab5eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41d3435740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 0x558b6ab5f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41d3435740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1b1680 con 0x558b6ab3bff0 2024-08-02T17:52:37.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41d3435740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1aef50 con 0x558b6ab3c910 2024-08-02T17:52:37.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41d3435740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1acda0 con 0x558b6ab471d0 2024-08-02T17:52:37.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.352+0000 7f41cad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab374c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56300/0 (socket says 172.21.15.69:56300) 2024-08-02T17:52:37.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cad9d640 1 -- 172.21.15.69:0/2092551259 learned_addr learned my addr 172.21.15.69:0/2092551259 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cb59e640 1 --2- 172.21.15.69:0/2092551259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 0x558b6ab5f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cad9d640 1 -- 172.21.15.69:0/2092551259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 msgr2=0x558b6ab5f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cad9d640 1 --2- 172.21.15.69:0/2092551259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 0x558b6ab5f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cad9d640 1 -- 172.21.15.69:0/2092551259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3c910 msgr2=0x558b6ab5eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cad9d640 1 --2- 172.21.15.69:0/2092551259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3c910 0x558b6ab5eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cad9d640 1 -- 172.21.15.69:0/2092551259 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b6ab595c0 con 0x558b6ab3bff0 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41cad9d640 1 --2- 172.21.15.69:0/2092551259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab374c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f41bc010f50 tx=0x7f41bc025360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771bb024192518d3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41b0ff9640 1 -- 172.21.15.69:0/2092551259 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc09bd40 con 0x558b6ab3bff0 2024-08-02T17:52:37.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41b0ff9640 1 -- 172.21.15.69:0/2092551259 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41bc0a0040 con 0x558b6ab3bff0 2024-08-02T17:52:37.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.353+0000 7f41b0ff9640 1 -- 172.21.15.69:0/2092551259 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41bc0983b0 con 0x558b6ab3bff0 2024-08-02T17:52:37.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 -- 172.21.15.69:0/2092551259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 msgr2=0x558b6ab374c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 --2- 172.21.15.69:0/2092551259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab374c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f41bc010f50 tx=0x7f41bc025360 comp rx=0 tx=0).stop 2024-08-02T17:52:37.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 -- 172.21.15.69:0/2092551259 shutdown_connections 2024-08-02T17:52:37.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 --2- 172.21.15.69:0/2092551259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 0x558b6ab5f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 --2- 172.21.15.69:0/2092551259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3c910 0x558b6ab5eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 --2- 172.21.15.69:0/2092551259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3bff0 0x558b6ab374c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 -- 172.21.15.69:0/2092551259 >> 172.21.15.69:0/2092551259 conn(0x558b6ab38580 msgr2=0x558b6ab36db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 -- 172.21.15.69:0/2092551259 shutdown_connections 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 -- 172.21.15.69:0/2092551259 wait complete. 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 -- start start 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 Processor -- start 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3bff0 0x558b6ab4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab4ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.354+0000 7f41d3435740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 0x558b6abbfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41d3435740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1ad410 con 0x558b6ab3c910 2024-08-02T17:52:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41d3435740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1ae880 con 0x558b6ab3bff0 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41d3435740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f41cc1adae0 con 0x558b6ab471d0 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41d3435740 10 client.0 _ll_register_callbacks cb 0x7f41c0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab4ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab4ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56302/0 (socket says 172.21.15.69:56302) 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 -- 172.21.15.69:0/4197597981 learned_addr learned my addr 172.21.15.69:0/4197597981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41cad9d640 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3bff0 0x558b6ab4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41cb59e640 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 0x558b6abbfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 -- 172.21.15.69:0/4197597981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 msgr2=0x558b6abbfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 0x558b6abbfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 -- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3bff0 msgr2=0x558b6ab4a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3bff0 0x558b6ab4a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 -- 172.21.15.69:0/4197597981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b6abc0060 con 0x558b6ab3c910 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41cad9d640 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3bff0 0x558b6ab4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:37.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41ca59c640 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab4ac10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f41c0003b80 tx=0x7f41c0003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e660851b7c074ae5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41aeff5640 1 -- 172.21.15.69:0/4197597981 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0040070 con 0x558b6ab3c910 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41d3435740 1 -- 172.21.15.69:0/4197597981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41bc0a0040 con 0x558b6ab3c910 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41aeff5640 1 -- 172.21.15.69:0/4197597981 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f41c0004770 con 0x558b6ab3c910 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41aeff5640 1 -- 172.21.15.69:0/4197597981 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f41c005e050 con 0x558b6ab3c910 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41d3435740 10 client.17049 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.355+0000 7f41d3435740 1 -- 172.21.15.69:0/4197597981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558b6abc12a0 con 0x558b6ab3c910 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41aeff5640 1 -- 172.21.15.69:0/4197597981 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41bc0a0040 con 0x558b6ab3c910 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41aeff5640 1 -- 172.21.15.69:0/4197597981 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f41c0059030 con 0x558b6ab3c910 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41d3435740 10 client.17049 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41d3435740 10 client.17049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:37.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41d3435740 20 client.17049 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41d3435740 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558b6abc1aa0 0x558b6abc3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41d3435740 4 client.17049 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41d3435740 1 -- 172.21.15.69:0/4197597981 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b6aa26170 con 0x558b6abc1aa0 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.356+0000 7f41cad9d640 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558b6abc1aa0 0x558b6abc3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41aeff5640 10 client.17049 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41cad9d640 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558b6abc1aa0 0x558b6abc3ec0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86575d2889eefc8d server_cookie=328e2ae5dd0ccfa4 in_seq=0 out_seq=0 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41aeff5640 1 -- 172.21.15.69:0/4197597981 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b6aa26170 con 0x558b6abc1aa0 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41aeff5640 10 client.17049 handle_command_reply: tid=0 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 client.17049 shutdown 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 -- 172.21.15.69:0/4197597981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 msgr2=0x558b6ab4ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab4ac10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f41c0003b80 tx=0x7f41c0003eb0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 -- 172.21.15.69:0/4197597981 shutdown_connections 2024-08-02T17:52:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558b6ab3c910 0x558b6ab4ac10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558b6abc1aa0 0x558b6abc3ec0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558b6ab471d0 0x558b6abbfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 --2- 172.21.15.69:0/4197597981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558b6ab3bff0 0x558b6ab4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41aeff5640 0 client.17049 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 -- 172.21.15.69:0/4197597981 >> 172.21.15.69:0/4197597981 conn(0x558b6ab38580 msgr2=0x558b6ab36db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 -- 172.21.15.69:0/4197597981 shutdown_connections 2024-08-02T17:52:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 1 -- 172.21.15.69:0/4197597981 wait complete. 2024-08-02T17:52:37.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.357+0000 7f41d3435740 20 client.17049 trim_cache size 0 max 16384 2024-08-02T17:52:37.368 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 -- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41b40a2cf0 msgr2=0x7f41b40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41b40a2cf0 0x7f41b40a51b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41c4005c40 tx=0x7f41c4016000 comp rx=0 tx=0).stop 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 -- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 msgr2=0x7f41cc1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 0x7f41cc1980f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00e9f0 tx=0x7f41bc00eeb0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 -- 172.21.15.69:0/2973433695 shutdown_connections 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41b40a2cf0 0x7f41b40a51b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f41cc1067b0 0x7f41cc19c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f41cc101150 0x7f41cc198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 --2- 172.21.15.69:0/2973433695 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f41cc1007b0 0x7f41cc1980f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 -- 172.21.15.69:0/2973433695 >> 172.21.15.69:0/2973433695 conn(0x7f41cc0fc4d0 msgr2=0x7f41cc06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 -- 172.21.15.69:0/2973433695 shutdown_connections 2024-08-02T17:52:37.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:37.358+0000 7f41d1037640 1 -- 172.21.15.69:0/2973433695 wait complete. 2024-08-02T17:52:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 -- 172.21.15.69:0/1974079937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88100700 msgr2=0x7f4a88100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 --2- 172.21.15.69:0/1974079937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88100700 0x7f4a88100b60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009930 tx=0x7f4a7002a420 comp rx=0 tx=0).stop 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 -- 172.21.15.69:0/1974079937 shutdown_connections 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 --2- 172.21.15.69:0/1974079937 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88100700 0x7f4a88100b60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 --2- 172.21.15.69:0/1974079937 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a881010a0 0x7f4a8810d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 --2- 172.21.15.69:0/1974079937 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a88106710 0x7f4a88106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 -- 172.21.15.69:0/1974079937 >> 172.21.15.69:0/1974079937 conn(0x7f4a880fc3e0 msgr2=0x7f4a880fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 -- 172.21.15.69:0/1974079937 shutdown_connections 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.820+0000 7f4a8d965640 1 -- 172.21.15.69:0/1974079937 wait complete. 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8d965640 1 Processor -- start 2024-08-02T17:52:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8d965640 1 -- start start 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8d965640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a88100700 0x7f4a88198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8d965640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a881010a0 0x7f4a881986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8d965640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88106710 0x7f4a8819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8d965640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88111c40 con 0x7f4a88106710 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8d965640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88111ac0 con 0x7f4a88100700 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8d965640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a88111dc0 con 0x7f4a881010a0 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88106710 0x7f4a8819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88106710 0x7f4a8819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56326/0 (socket says 172.21.15.69:56326) 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 -- 172.21.15.69:0/3369015156 learned_addr learned my addr 172.21.15.69:0/3369015156 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a881010a0 0x7f4a881986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a88100700 0x7f4a88198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 -- 172.21.15.69:0/3369015156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a881010a0 msgr2=0x7f4a881986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a881010a0 0x7f4a881986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 -- 172.21.15.69:0/3369015156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a88100700 msgr2=0x7f4a88198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a88100700 0x7f4a88198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 -- 172.21.15.69:0/3369015156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8819d160 con 0x7f4a88106710 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a877fe640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88106710 0x7f4a8819ca30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000eaa0 tx=0x7f4a8000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.821+0000 7f4a8c963640 1 -- 172.21.15.69:0/3369015156 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8000cdb0 con 0x7f4a88106710 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.822+0000 7f4a8c963640 1 -- 172.21.15.69:0/3369015156 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a80005020 con 0x7f4a88106710 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.822+0000 7f4a8d965640 1 -- 172.21.15.69:0/3369015156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8819d450 con 0x7f4a88106710 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.822+0000 7f4a8c963640 1 -- 172.21.15.69:0/3369015156 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a800116b0 con 0x7f4a88106710 2024-08-02T17:52:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.822+0000 7f4a8d965640 1 -- 172.21.15.69:0/3369015156 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a881a4ce0 con 0x7f4a88106710 2024-08-02T17:52:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.824+0000 7f4a8c963640 1 -- 172.21.15.69:0/3369015156 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4a80005560 con 0x7f4a88106710 2024-08-02T17:52:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.824+0000 7f4a90799740 1 Processor -- start 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.824+0000 7f4a90799740 1 -- start start 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a90799740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c6928db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a90799740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69298430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a90799740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c6929ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a90799740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560c691f19b0 con 0x560c69298970 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a90799740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560c69199800 con 0x560c6928e0d0 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a90799740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560c691ed970 con 0x560c6928d7b0 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c6929ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a86ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69298430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c6929ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56340/0 (socket says 172.21.15.69:56340) 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 -- 172.21.15.69:0/1309489385 learned_addr learned my addr 172.21.15.69:0/1309489385 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a8c963640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a740a2d70 0x7f4a740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c6928db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a877fe640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a740a2d70 0x7f4a740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 -- 172.21.15.69:0/1309489385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 msgr2=0x560c6928db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c6928db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 -- 172.21.15.69:0/1309489385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 msgr2=0x560c69298430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69298430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 -- 172.21.15.69:0/1309489385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c68f05210 con 0x560c69298970 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69298430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a8c963640 1 -- 172.21.15.69:0/3369015156 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4a800cfa20 con 0x7f4a88106710 2024-08-02T17:52:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c6929ad60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4a70006010 tx=0x7f4a70009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e525d2ae99636b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a877fe640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a740a2d70 0x7f4a740a5230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4a8009bab0 tx=0x7f4a8006dae0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7dffb640 1 -- 172.21.15.69:0/1309489385 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70035070 con 0x560c69298970 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.825+0000 7f4a7dffb640 1 -- 172.21.15.69:0/1309489385 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a70004030 con 0x560c69298970 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a7dffb640 1 -- 172.21.15.69:0/1309489385 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70005620 con 0x560c69298970 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 -- 172.21.15.69:0/1309489385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 msgr2=0x560c6929ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c6929ad60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4a70006010 tx=0x7f4a70009560 comp rx=0 tx=0).stop 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 -- 172.21.15.69:0/1309489385 shutdown_connections 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c6929ad60 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69298430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 --2- 172.21.15.69:0/1309489385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c6928db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 -- 172.21.15.69:0/1309489385 >> 172.21.15.69:0/1309489385 conn(0x560c69289d20 msgr2=0x560c6928a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 -- 172.21.15.69:0/1309489385 shutdown_connections 2024-08-02T17:52:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 -- 172.21.15.69:0/1309489385 wait complete. 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.826+0000 7f4a90799740 1 -- start start 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a90799740 1 Processor -- start 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a90799740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c69295860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a90799740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69295da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a90799740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a90799740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560c6929b540 con 0x560c69298970 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a90799740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560c691ad500 con 0x560c6928e0d0 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a90799740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560c6929b6c0 con 0x560c6928d7b0 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a90799740 10 client.0 _ll_register_callbacks cb 0x560c68f04a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56346/0 (socket says 172.21.15.69:56346) 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a7ffff640 1 -- 172.21.15.69:0/2952067728 learned_addr learned my addr 172.21.15.69:0/2952067728 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c69295860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a7ffff640 1 -- 172.21.15.69:0/2952067728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 msgr2=0x560c69295860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69295da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c69295860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.827+0000 7f4a7ffff640 1 -- 172.21.15.69:0/2952067728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 msgr2=0x560c69295da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69295da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a7ffff640 1 -- 172.21.15.69:0/2952067728 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c68fd1620 con 0x560c69298970 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692962e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4a70000c00 tx=0x7f4a700051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52b2a11d504dac3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a66ffd640 1 -- 172.21.15.69:0/2952067728 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70035070 con 0x560c69298970 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a66ffd640 1 -- 172.21.15.69:0/2952067728 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4a700043f0 con 0x560c69298970 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 1 -- 172.21.15.69:0/2952067728 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a70004030 con 0x560c69298970 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a66ffd640 1 -- 172.21.15.69:0/2952067728 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a700316b0 con 0x560c69298970 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 10 client.17061 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 1 -- 172.21.15.69:0/2952067728 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c692a71e0 con 0x560c69298970 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a66ffd640 1 -- 172.21.15.69:0/2952067728 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a70004030 con 0x560c69298970 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a66ffd640 1 -- 172.21.15.69:0/2952067728 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a70003680 con 0x560c69298970 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 10 client.17061 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 20 client.17061 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560c692a8530 0x560c692aa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 4 client.17061 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.828+0000 7f4a90799740 1 -- 172.21.15.69:0/2952067728 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c691ff0e0 con 0x560c692a8530 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.829+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560c692a8530 0x560c692aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.829+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560c692a8530 0x560c692aa920 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8768dae4cc5489fd server_cookie=90a9bd7e2d0aa05a in_seq=0 out_seq=0 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.829+0000 7f4a66ffd640 10 client.17061 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a66ffd640 1 -- 172.21.15.69:0/2952067728 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560c691ff0e0 con 0x560c692a8530 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a66ffd640 10 client.17061 handle_command_reply: tid=0 2024-08-02T17:52:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 client.17061 shutdown 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 -- 172.21.15.69:0/2952067728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 msgr2=0x560c692962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692962e0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4a70000c00 tx=0x7f4a700051a0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 -- 172.21.15.69:0/2952067728 shutdown_connections 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692962e0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560c692a8530 0x560c692aa920 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c69295da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 --2- 172.21.15.69:0/2952067728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c69295860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a66ffd640 0 client.17061 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 -- 172.21.15.69:0/2952067728 >> 172.21.15.69:0/2952067728 conn(0x560c69289d20 msgr2=0x560c6928fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 -- 172.21.15.69:0/2952067728 shutdown_connections 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 1 -- 172.21.15.69:0/2952067728 wait complete. 2024-08-02T17:52:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.830+0000 7f4a90799740 20 client.17061 trim_cache size 0 max 16384 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a90799740 1 Processor -- start 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a90799740 1 -- start start 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a90799740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c6928d7b0 0x560c69290220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a90799740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ad940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a90799740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c69298970 0x560c692ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a90799740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881b16c0 con 0x560c6928d7b0 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a90799740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881aef90 con 0x560c6928e0d0 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a90799740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881acde0 con 0x560c69298970 2024-08-02T17:52:37.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c6928d7b0 0x560c69290220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c6928d7b0 0x560c69290220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56362/0 (socket says 172.21.15.69:56362) 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a7ffff640 1 -- 172.21.15.69:0/3676829458 learned_addr learned my addr 172.21.15.69:0/3676829458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a7ffff640 1 -- 172.21.15.69:0/3676829458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c69298970 msgr2=0x560c692ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c69298970 0x560c692ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a7ffff640 1 -- 172.21.15.69:0/3676829458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 msgr2=0x560c692ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a7ffff640 1 -- 172.21.15.69:0/3676829458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c6929b390 con 0x560c6928d7b0 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.845+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c6928d7b0 0x560c69290220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4a70003cc0 tx=0x7f4a700314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be482ee170ddc20e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a7d7fa640 1 -- 172.21.15.69:0/3676829458 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70035040 con 0x560c6928d7b0 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a7d7fa640 1 -- 172.21.15.69:0/3676829458 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a70005850 con 0x560c6928d7b0 2024-08-02T17:52:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a7d7fa640 1 -- 172.21.15.69:0/3676829458 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70003d40 con 0x560c6928d7b0 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 -- 172.21.15.69:0/3676829458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c6928d7b0 msgr2=0x560c69290220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c6928d7b0 0x560c69290220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4a70003cc0 tx=0x7f4a700314a0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 -- 172.21.15.69:0/3676829458 shutdown_connections 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c69298970 0x560c692ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 --2- 172.21.15.69:0/3676829458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c6928d7b0 0x560c69290220 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 -- 172.21.15.69:0/3676829458 >> 172.21.15.69:0/3676829458 conn(0x560c69289d20 msgr2=0x560c69288e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 -- 172.21.15.69:0/3676829458 shutdown_connections 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.846+0000 7f4a90799740 1 -- 172.21.15.69:0/3676829458 wait complete. 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 1 -- start start 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 1 Processor -- start 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c692adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692ac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881ad450 con 0x560c69298970 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881ae8c0 con 0x560c6928e0d0 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a881adb20 con 0x560c6928d7b0 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a90799740 10 client.0 _ll_register_callbacks cb 0x7f4a7800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56374/0 (socket says 172.21.15.69:56374) 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a7ffff640 1 -- 172.21.15.69:0/4201811287 learned_addr learned my addr 172.21.15.69:0/4201811287 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.847+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c692adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a7ffff640 1 -- 172.21.15.69:0/4201811287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 msgr2=0x560c692adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c692adda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a7ffff640 1 -- 172.21.15.69:0/4201811287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 msgr2=0x560c692ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ac140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a7ffff640 1 -- 172.21.15.69:0/4201811287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c69289b50 con 0x560c69298970 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a7ffff640 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692ac680 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4a700095a0 tx=0x7f4a7003f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9aeb5961399196 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a66ffd640 1 -- 172.21.15.69:0/4201811287 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70035030 con 0x560c69298970 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a66ffd640 1 -- 172.21.15.69:0/4201811287 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f4a70003f70 con 0x560c69298970 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 1 -- 172.21.15.69:0/4201811287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a70005850 con 0x560c69298970 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a66ffd640 1 -- 172.21.15.69:0/4201811287 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a70031970 con 0x560c69298970 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 10 client.17067 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 1 -- 172.21.15.69:0/4201811287 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c69317df0 con 0x560c69298970 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a66ffd640 1 -- 172.21.15.69:0/4201811287 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a70005850 con 0x560c69298970 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a66ffd640 1 -- 172.21.15.69:0/4201811287 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a7005e030 con 0x560c69298970 2024-08-02T17:52:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 10 client.17067 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 20 client.17067 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560c693182f0 0x560c6931a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 4 client.17067 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:52:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.848+0000 7f4a90799740 1 -- 172.21.15.69:0/4201811287 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c691f9430 con 0x560c693182f0 2024-08-02T17:52:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.849+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560c693182f0 0x560c6931a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.849+0000 7f4a86ffd640 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560c693182f0 0x560c6931a710 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1e619976ba2264 server_cookie=585ab4fff4dbc71 in_seq=0 out_seq=0 2024-08-02T17:52:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.849+0000 7f4a66ffd640 10 client.17067 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a66ffd640 1 -- 172.21.15.69:0/4201811287 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c691f9430 con 0x560c693182f0 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a66ffd640 10 client.17067 handle_command_reply: tid=0 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 client.17067 shutdown 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 -- 172.21.15.69:0/4201811287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 msgr2=0x560c692ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692ac680 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4a700095a0 tx=0x7f4a7003f040 comp rx=0 tx=0).stop 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 -- 172.21.15.69:0/4201811287 shutdown_connections 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560c69298970 0x560c692ac680 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560c693182f0 0x560c6931a710 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560c6928e0d0 0x560c692ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 --2- 172.21.15.69:0/4201811287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560c6928d7b0 0x560c692adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a66ffd640 0 client.17067 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 -- 172.21.15.69:0/4201811287 >> 172.21.15.69:0/4201811287 conn(0x560c69289d20 msgr2=0x560c692a3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 -- 172.21.15.69:0/4201811287 shutdown_connections 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 1 -- 172.21.15.69:0/4201811287 wait complete. 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.862+0000 7f4a90799740 20 client.17067 trim_cache size 0 max 16384 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:52:37.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:52:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 -- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a740a2d70 msgr2=0x7f4a740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a740a2d70 0x7f4a740a5230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4a8009bab0 tx=0x7f4a8006dae0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 -- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88106710 msgr2=0x7f4a8819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88106710 0x7f4a8819ca30 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4a8000eaa0 tx=0x7f4a8000ef60 comp rx=0 tx=0).stop 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 -- 172.21.15.69:0/3369015156 shutdown_connections 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a88106710 0x7f4a8819ca30 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a740a2d70 0x7f4a740a5230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a881010a0 0x7f4a881986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 --2- 172.21.15.69:0/3369015156 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a88100700 0x7f4a88198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 -- 172.21.15.69:0/3369015156 >> 172.21.15.69:0/3369015156 conn(0x7f4a880fc3e0 msgr2=0x7f4a880fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 -- 172.21.15.69:0/3369015156 shutdown_connections 2024-08-02T17:52:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:37.863+0000 7f4a8d965640 1 -- 172.21.15.69:0/3369015156 wait complete. 2024-08-02T17:52:38.574 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:52:38.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:38 smithi069 ceph-mon[32746]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 974 KiB/s wr, 125 op/s 2024-08-02T17:52:38.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:38 smithi083 ceph-mon[37031]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 974 KiB/s wr, 125 op/s 2024-08-02T17:52:38.827 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:38.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:38 smithi077 ceph-mon[37108]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 974 KiB/s wr, 125 op/s 2024-08-02T17:52:39.223 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:52:39.223 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:52:39.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.297+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2813842890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef581022a0 msgr2=0x7fef581026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:39.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.297+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2813842890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef581022a0 0x7fef581026e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009900 tx=0x7fef4c02a480 comp rx=0 tx=0).stop 2024-08-02T17:52:39.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.297+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2813842890 shutdown_connections 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.297+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2813842890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef581022a0 0x7fef581026e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.297+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2813842890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef58102c20 0x7fef5810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.298+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2813842890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef58100b50 0x7fef58100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.298+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2813842890 >> 172.21.15.69:0/2813842890 conn(0x7fef580fc500 msgr2=0x7fef580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.298+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2813842890 shutdown_connections 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.298+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2813842890 wait complete. 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5cbf5640 1 Processor -- start 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5cbf5640 1 -- start start 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5cbf5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef58100b50 0x7fef581970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5cbf5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef581022a0 0x7fef581975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5cbf5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef58102c20 0x7fef5819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5cbf5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef58110c70 con 0x7fef58102c20 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5cbf5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef58110af0 con 0x7fef58100b50 2024-08-02T17:52:39.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5cbf5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef58110df0 con 0x7fef581022a0 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef56d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef58102c20 0x7fef5819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef56d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef58102c20 0x7fef5819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56386/0 (socket says 172.21.15.69:56386) 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef56d9d640 1 -- 172.21.15.69:0/2610086392 learned_addr learned my addr 172.21.15.69:0/2610086392 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef5659c640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef58100b50 0x7fef581970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef55d9b640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef581022a0 0x7fef581975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef56d9d640 1 -- 172.21.15.69:0/2610086392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef581022a0 msgr2=0x7fef581975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef56d9d640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef581022a0 0x7fef581975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef56d9d640 1 -- 172.21.15.69:0/2610086392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef58100b50 msgr2=0x7fef581970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef56d9d640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef58100b50 0x7fef581970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef56d9d640 1 -- 172.21.15.69:0/2610086392 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5819c060 con 0x7fef58102c20 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.299+0000 7fef55d9b640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef581022a0 0x7fef581975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.300+0000 7fef56d9d640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef58102c20 0x7fef5819b980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fef5000eaa0 tx=0x7fef5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.300+0000 7fef477fe640 1 -- 172.21.15.69:0/2610086392 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5000cdb0 con 0x7fef58102c20 2024-08-02T17:52:39.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.300+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5819c350 con 0x7fef58102c20 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.300+0000 7fef477fe640 1 -- 172.21.15.69:0/2610086392 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef50005020 con 0x7fef58102c20 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.300+0000 7fef477fe640 1 -- 172.21.15.69:0/2610086392 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef500116b0 con 0x7fef58102c20 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.300+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581a3c30 con 0x7fef58102c20 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.301+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef58100f30 con 0x7fef58102c20 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.303+0000 7fef477fe640 1 -- 172.21.15.69:0/2610086392 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef50005560 con 0x7fef58102c20 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.303+0000 7fef477fe640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef400a2cf0 0x7fef400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.304+0000 7fef5659c640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef400a2cf0 0x7fef400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.304+0000 7fef477fe640 1 -- 172.21.15.69:0/2610086392 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fef500cfef0 con 0x7fef58102c20 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.304+0000 7fef5659c640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef400a2cf0 0x7fef400a51b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fef4800ba40 tx=0x7fef48005cc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:39.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.307+0000 7fef477fe640 1 -- 172.21.15.69:0/2610086392 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fef50016030 con 0x7fef58102c20 2024-08-02T17:52:39.484 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:39.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.522+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef58198410 con 0x7fef58102c20 2024-08-02T17:52:39.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.523+0000 7fef477fe640 1 -- 172.21.15.69:0/2610086392 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fef58198410 con 0x7fef58102c20 2024-08-02T17:52:39.525 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:52:39.525 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:52:39.525 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef400a2cf0 msgr2=0x7fef400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef400a2cf0 0x7fef400a51b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fef4800ba40 tx=0x7fef48005cc0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef58102c20 msgr2=0x7fef5819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef58102c20 0x7fef5819b980 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fef5000eaa0 tx=0x7fef5000ef60 comp rx=0 tx=0).stop 2024-08-02T17:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 shutdown_connections 2024-08-02T17:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef58102c20 0x7fef5819b980 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef400a2cf0 0x7fef400a51b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef581022a0 0x7fef581975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 --2- 172.21.15.69:0/2610086392 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef58100b50 0x7fef581970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:39.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 >> 172.21.15.69:0/2610086392 conn(0x7fef580fc500 msgr2=0x7fef581065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:39.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 shutdown_connections 2024-08-02T17:52:39.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:39.525+0000 7fef5cbf5640 1 -- 172.21.15.69:0/2610086392 wait complete. 2024-08-02T17:52:39.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 330.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:52:40.450 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 4e57b63c --release --await' 2024-08-02T17:52:40.450 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 4e57b63c --release --await 2024-08-02T17:52:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:40 smithi069 ceph-mon[32746]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 198 KiB/s wr, 25 op/s 2024-08-02T17:52:40.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:40 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2610086392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:40.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 -- 172.21.15.69:0/600424542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541007c0 msgr2=0x7fbd54100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 --2- 172.21.15.69:0/600424542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541007c0 0x7fbd54100c20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009930 tx=0x7fbd3c02a420 comp rx=0 tx=0).stop 2024-08-02T17:52:40.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 -- 172.21.15.69:0/600424542 shutdown_connections 2024-08-02T17:52:40.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 --2- 172.21.15.69:0/600424542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541007c0 0x7fbd54100c20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 --2- 172.21.15.69:0/600424542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd54101160 0x7fbd5410d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 --2- 172.21.15.69:0/600424542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd541067c0 0x7fbd54106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 -- 172.21.15.69:0/600424542 >> 172.21.15.69:0/600424542 conn(0x7fbd540fc500 msgr2=0x7fbd540fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 -- 172.21.15.69:0/600424542 shutdown_connections 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 -- 172.21.15.69:0/600424542 wait complete. 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.650+0000 7fbd58aac640 1 Processor -- start 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd58aac640 1 -- start start 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd58aac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd541007c0 0x7fbd540fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd58aac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54101160 0x7fbd540fd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd58aac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541067c0 0x7fbd540fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd58aac640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54110b80 con 0x7fbd541067c0 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd58aac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54110a00 con 0x7fbd541007c0 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd58aac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54110d00 con 0x7fbd54101160 2024-08-02T17:52:40.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541067c0 0x7fbd540fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541067c0 0x7fbd540fd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56406/0 (socket says 172.21.15.69:56406) 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 -- 172.21.15.69:0/2100417960 learned_addr learned my addr 172.21.15.69:0/2100417960 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd5259c640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd541007c0 0x7fbd540fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54101160 0x7fbd540fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 -- 172.21.15.69:0/2100417960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54101160 msgr2=0x7fbd540fd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54101160 0x7fbd540fd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 -- 172.21.15.69:0/2100417960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd541007c0 msgr2=0x7fbd540fcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd541007c0 0x7fbd540fcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 -- 172.21.15.69:0/2100417960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd541a65c0 con 0x7fbd541067c0 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd52d9d640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541067c0 0x7fbd540fd5a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00eaa0 tx=0x7fbd4c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.651+0000 7fbd4b7fe640 1 -- 172.21.15.69:0/2100417960 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00cdb0 con 0x7fbd541067c0 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.652+0000 7fbd58aac640 1 -- 172.21.15.69:0/2100417960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd541a68b0 con 0x7fbd541067c0 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.652+0000 7fbd4b7fe640 1 -- 172.21.15.69:0/2100417960 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd4c005020 con 0x7fbd541067c0 2024-08-02T17:52:40.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.652+0000 7fbd58aac640 1 -- 172.21.15.69:0/2100417960 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd541a6dc0 con 0x7fbd541067c0 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.652+0000 7fbd4b7fe640 1 -- 172.21.15.69:0/2100417960 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c01f060 con 0x7fbd541067c0 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd5aeaa740 1 Processor -- start 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd4b7fe640 1 -- 172.21.15.69:0/2100417960 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbd4c005560 con 0x7fbd541067c0 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd5aeaa740 1 -- start start 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d59845a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 0x55d598464ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 0x55d5984677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d59841de40 con 0x55d59845a200 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5983bf7f0 con 0x55d5984653e0 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.654+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d598366fb0 con 0x55d59845ab20 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d59845a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d59845a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56414/0 (socket says 172.21.15.69:56414) 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 -- 172.21.15.69:0/631658764 learned_addr learned my addr 172.21.15.69:0/631658764 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:40.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 0x55d5984677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd5259c640 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 0x55d598464ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4b7fe640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd400a2cf0 0x7fbd400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 -- 172.21.15.69:0/631658764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 msgr2=0x55d598464ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 0x55d598464ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 -- 172.21.15.69:0/631658764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 msgr2=0x55d5984677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 0x55d5984677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 -- 172.21.15.69:0/631658764 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5981316a0 con 0x55d59845a200 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd52d9d640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd400a2cf0 0x7fbd400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d59845a5e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55d598457b10 tx=0x7fbd3c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6d75cfd8f04fac server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd4b7fe640 1 -- 172.21.15.69:0/2100417960 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbd4c0cfe90 con 0x7fbd541067c0 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.655+0000 7fbd48ff9640 1 -- 172.21.15.69:0/631658764 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c035030 con 0x55d59845a200 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd48ff9640 1 -- 172.21.15.69:0/631658764 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c005050 con 0x55d59845a200 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd48ff9640 1 -- 172.21.15.69:0/631658764 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c004920 con 0x55d59845a200 2024-08-02T17:52:40.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/631658764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 msgr2=0x55d59845a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd52d9d640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd400a2cf0 0x7fbd400a51b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0cd2d0 tx=0x7fbd4c06af20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d59845a5e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55d598457b10 tx=0x7fbd3c0058b0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/631658764 shutdown_connections 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 0x55d5984677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 0x55d598464ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/631658764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d59845a5e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/631658764 >> 172.21.15.69:0/631658764 conn(0x55d598456790 msgr2=0x55d598456b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/631658764 shutdown_connections 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.656+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/631658764 wait complete. 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 1 -- start start 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 1 Processor -- start 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d598462340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 0x55d598462880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 0x55d598462dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d59837acb0 con 0x55d59845a200 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d5983bb7b0 con 0x55d5984653e0 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d598467fb0 con 0x55d59845ab20 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd5aeaa740 10 client.0 _ll_register_callbacks cb 0x55d5980ceba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d598462340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d598462340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56420/0 (socket says 172.21.15.69:56420) 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd4bfff640 1 -- 172.21.15.69:0/716149797 learned_addr learned my addr 172.21.15.69:0/716149797 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd4bfff640 1 -- 172.21.15.69:0/716149797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 msgr2=0x55d598462880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 0x55d598462880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd4bfff640 1 -- 172.21.15.69:0/716149797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 msgr2=0x55d598462dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 0x55d598462dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.657+0000 7fbd4bfff640 1 -- 172.21.15.69:0/716149797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d598379bc0 con 0x55d59845a200 2024-08-02T17:52:40.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d598462340 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0096c0 tx=0x7fbd3c02aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2857704867bc3b23 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd39ffb640 1 -- 172.21.15.69:0/716149797 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c035030 con 0x55d59845a200 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd39ffb640 1 -- 172.21.15.69:0/716149797 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbd3c0048c0 con 0x55d59845a200 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/716149797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3c005050 con 0x55d59845a200 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd39ffb640 1 -- 172.21.15.69:0/716149797 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd3c004ac0 con 0x55d59845a200 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 10 client.17085 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/716149797 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d598473ef0 con 0x55d59845a200 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd39ffb640 1 -- 172.21.15.69:0/716149797 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c005050 con 0x55d59845a200 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd39ffb640 1 -- 172.21.15.69:0/716149797 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbd3c049480 con 0x55d59845a200 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 10 client.17085 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 10 client.17085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 20 client.17085 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d598475240 0x55d598477630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 4 client.17085 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.658+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/716149797 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5983ccd60 con 0x55d598475240 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.659+0000 7fbd5259c640 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d598475240 0x55d598477630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.659+0000 7fbd39ffb640 10 client.17085 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.659+0000 7fbd5259c640 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d598475240 0x55d598477630 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3865564ca8751668 server_cookie=ab13279c40d103d in_seq=0 out_seq=0 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd39ffb640 1 -- 172.21.15.69:0/716149797 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d5983ccd60 con 0x55d598475240 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd39ffb640 10 client.17085 handle_command_reply: tid=0 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 client.17085 shutdown 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/716149797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 msgr2=0x55d598462340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d598462340 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0096c0 tx=0x7fbd3c02aaa0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/716149797 shutdown_connections 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d598475240 0x55d598477630 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d5984653e0 0x55d598462dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd39ffb640 0 client.17085 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d59845ab20 0x55d598462880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/716149797 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845a200 0x55d598462340 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/716149797 >> 172.21.15.69:0/716149797 conn(0x55d598456790 msgr2=0x55d59845c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:40.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/716149797 shutdown_connections 2024-08-02T17:52:40.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/716149797 wait complete. 2024-08-02T17:52:40.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.660+0000 7fbd5aeaa740 20 client.17085 trim_cache size 0 max 16384 2024-08-02T17:52:40.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5aeaa740 1 Processor -- start 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5aeaa740 1 -- start start 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d59845ab20 0x55d59845cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 0x55d5984611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d598461730 0x55d59847a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541b37a0 con 0x55d598461730 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541b1070 con 0x55d59845ab20 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541aeec0 con 0x55d5984653e0 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d598461730 0x55d59847a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d598461730 0x55d59847a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56432/0 (socket says 172.21.15.69:56432) 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 -- 172.21.15.69:0/1745593653 learned_addr learned my addr 172.21.15.69:0/1745593653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d59845ab20 0x55d59845cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd5259c640 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 0x55d5984611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 -- 172.21.15.69:0/1745593653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 msgr2=0x55d5984611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 0x55d5984611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 -- 172.21.15.69:0/1745593653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d59845ab20 msgr2=0x55d59845cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d59845ab20 0x55d59845cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.678+0000 7fbd4bfff640 1 -- 172.21.15.69:0/1745593653 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5984560e0 con 0x55d598461730 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.679+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d598461730 0x55d59847a650 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c036040 tx=0x7fbd3c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7c9a0632d7514f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.679+0000 7fbd3bfff640 1 -- 172.21.15.69:0/1745593653 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c049ac0 con 0x55d598461730 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.679+0000 7fbd3bfff640 1 -- 172.21.15.69:0/1745593653 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c04e070 con 0x55d598461730 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.679+0000 7fbd3bfff640 1 -- 172.21.15.69:0/1745593653 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c004c60 con 0x55d598461730 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/1745593653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d598461730 msgr2=0x55d59847a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d598461730 0x55d59847a650 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c036040 tx=0x7fbd3c000bc0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/1745593653 shutdown_connections 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d598461730 0x55d59847a650 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 0x55d5984611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/1745593653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d59845ab20 0x55d59845cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/1745593653 >> 172.21.15.69:0/1745593653 conn(0x55d598456790 msgr2=0x55d598460ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/1745593653 shutdown_connections 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/1745593653 wait complete. 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- start start 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 Processor -- start 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845ab20 0x55d598478400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d598461730 0x55d598478940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 0x55d598478e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541af530 con 0x55d59845ab20 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541b09a0 con 0x55d598461730 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.680+0000 7fbd5aeaa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd541afc00 con 0x55d5984653e0 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd5aeaa740 10 client.0 _ll_register_callbacks cb 0x7fbd3c032eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845ab20 0x55d598478400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845ab20 0x55d598478400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56448/0 (socket says 172.21.15.69:56448) 2024-08-02T17:52:40.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 -- 172.21.15.69:0/2807690505 learned_addr learned my addr 172.21.15.69:0/2807690505 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd5259c640 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d598461730 0x55d598478940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 0x55d598478e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 -- 172.21.15.69:0/2807690505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 msgr2=0x55d598478e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 0x55d598478e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 -- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d598461730 msgr2=0x55d598478940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d598461730 0x55d598478940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 -- 172.21.15.69:0/2807690505 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5984deb80 con 0x55d59845ab20 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd5259c640 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d598461730 0x55d598478940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd4bfff640 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845ab20 0x55d598478400 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0059d0 tx=0x7fbd3c032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee7c0016640a310 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd39ffb640 1 -- 172.21.15.69:0/2807690505 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c06d060 con 0x55d59845ab20 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd39ffb640 1 -- 172.21.15.69:0/2807690505 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7fbd3c031d10 con 0x55d59845ab20 2024-08-02T17:52:40.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.681+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/2807690505 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3c04e070 con 0x55d59845ab20 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd39ffb640 1 -- 172.21.15.69:0/2807690505 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd3c035070 con 0x55d59845ab20 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5aeaa740 10 client.17091 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/2807690505 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5984dfe70 con 0x55d59845ab20 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd39ffb640 1 -- 172.21.15.69:0/2807690505 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd3c04e070 con 0x55d59845ab20 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd39ffb640 1 -- 172.21.15.69:0/2807690505 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbd3c03db20 con 0x55d59845ab20 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5aeaa740 10 client.17091 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5aeaa740 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5aeaa740 20 client.17091 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d5984e02a0 0x55d5984e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5aeaa740 4 client.17091 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/2807690505 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d5983c7270 con 0x55d5984e02a0 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.682+0000 7fbd5259c640 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d5984e02a0 0x55d5984e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.683+0000 7fbd5259c640 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d5984e02a0 0x55d5984e2690 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d708ee4fbe6db9 server_cookie=3b67ff0a6392d8c5 in_seq=0 out_seq=0 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.683+0000 7fbd39ffb640 10 client.17091 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:40.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd39ffb640 1 -- 172.21.15.69:0/2807690505 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d5983c7270 con 0x55d5984e02a0 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd39ffb640 10 client.17091 handle_command_reply: tid=0 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 client.17091 shutdown 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/2807690505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845ab20 msgr2=0x55d598478400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845ab20 0x55d598478400 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0059d0 tx=0x7fbd3c032e20 comp rx=0 tx=0).stop 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/2807690505 shutdown_connections 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d5984e02a0 0x55d5984e2690 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d5984653e0 0x55d598478e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d598461730 0x55d598478940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd39ffb640 0 client.17091 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 --2- 172.21.15.69:0/2807690505 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d59845ab20 0x55d598478400 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/2807690505 >> 172.21.15.69:0/2807690505 conn(0x55d598456790 msgr2=0x55d598470200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/2807690505 shutdown_connections 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 1 -- 172.21.15.69:0/2807690505 wait complete. 2024-08-02T17:52:40.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.684+0000 7fbd5aeaa740 20 client.17091 trim_cache size 0 max 16384 2024-08-02T17:52:40.693 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:52:40.693 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:52:40.693 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "0c79fe6f-e8a5-498f-9395-8cd814e28e2f", 2024-08-02T17:52:40.693 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:52:40.693 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:52:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 -- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd400a2cf0 msgr2=0x7fbd400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd400a2cf0 0x7fbd400a51b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0cd2d0 tx=0x7fbd4c06af20 comp rx=0 tx=0).stop 2024-08-02T17:52:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 -- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541067c0 msgr2=0x7fbd540fd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541067c0 0x7fbd540fd5a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00eaa0 tx=0x7fbd4c00ef60 comp rx=0 tx=0).stop 2024-08-02T17:52:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 -- 172.21.15.69:0/2100417960 shutdown_connections 2024-08-02T17:52:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd541067c0 0x7fbd540fd5a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd400a2cf0 0x7fbd400a51b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd54101160 0x7fbd540fd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 --2- 172.21.15.69:0/2100417960 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd541007c0 0x7fbd540fcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:40.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 -- 172.21.15.69:0/2100417960 >> 172.21.15.69:0/2100417960 conn(0x7fbd540fc500 msgr2=0x7fbd5406a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:40.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 -- 172.21.15.69:0/2100417960 shutdown_connections 2024-08-02T17:52:40.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:40.685+0000 7fbd58aac640 1 -- 172.21.15.69:0/2100417960 wait complete. 2024-08-02T17:52:40.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:40 smithi083 ceph-mon[37031]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 198 KiB/s wr, 25 op/s 2024-08-02T17:52:40.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:40 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2610086392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:40.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:40 smithi077 ceph-mon[37108]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 198 KiB/s wr, 25 op/s 2024-08-02T17:52:40.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:40 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2610086392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:52:41.430 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:52:41.477 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 -- 172.21.15.69:0/1287244474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c0102c20 msgr2=0x7f11c010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 --2- 172.21.15.69:0/1287244474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c0102c20 0x7f11c010d770 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009ca0 tx=0x7f11b802a950 comp rx=0 tx=0).stop 2024-08-02T17:52:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 -- 172.21.15.69:0/1287244474 shutdown_connections 2024-08-02T17:52:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 --2- 172.21.15.69:0/1287244474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c0102c20 0x7f11c010d770 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 --2- 172.21.15.69:0/1287244474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c01022a0 0x7f11c01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 --2- 172.21.15.69:0/1287244474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11c0100b50 0x7f11c0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 -- 172.21.15.69:0/1287244474 >> 172.21.15.69:0/1287244474 conn(0x7f11c00fc500 msgr2=0x7f11c00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 -- 172.21.15.69:0/1287244474 shutdown_connections 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.962+0000 7f11c69ce640 1 -- 172.21.15.69:0/1287244474 wait complete. 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c69ce640 1 Processor -- start 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c69ce640 1 -- start start 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c69ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11c0100b50 0x7f11c01970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c69ce640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c01022a0 0x7f11c01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c69ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c0102c20 0x7f11c019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c69ce640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11c0110cb0 con 0x7f11c01022a0 2024-08-02T17:52:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c69ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11c0110b30 con 0x7f11c0102c20 2024-08-02T17:52:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c69ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11c0110e30 con 0x7f11c0100b50 2024-08-02T17:52:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c01022a0 0x7f11c01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c01022a0 0x7f11c01975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56456/0 (socket says 172.21.15.69:56456) 2024-08-02T17:52:41.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11bf7fe640 1 -- 172.21.15.69:0/544881259 learned_addr learned my addr 172.21.15.69:0/544881259 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11bffff640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11c0100b50 0x7f11c01970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.963+0000 7f11c4f6b640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c0102c20 0x7f11c019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bffff640 1 -- 172.21.15.69:0/544881259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c0102c20 msgr2=0x7f11c019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:41.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bffff640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c0102c20 0x7f11c019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bffff640 1 -- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c01022a0 msgr2=0x7f11c01975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bffff640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c01022a0 0x7f11c01975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bffff640 1 -- 172.21.15.69:0/544881259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c019c060 con 0x7f11c0100b50 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c01022a0 0x7f11c01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11c4f6b640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c0102c20 0x7f11c019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bffff640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11c0100b50 0x7f11c01970b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f11b000e9f0 tx=0x7f11b000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bd7fa640 1 -- 172.21.15.69:0/544881259 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b000ccc0 con 0x7f11c0100b50 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11c69ce640 1 -- 172.21.15.69:0/544881259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c019c350 con 0x7f11c0100b50 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bd7fa640 1 -- 172.21.15.69:0/544881259 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b0005020 con 0x7f11c0100b50 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11bd7fa640 1 -- 172.21.15.69:0/544881259 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b00049e0 con 0x7f11c0100b50 2024-08-02T17:52:41.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.964+0000 7f11c69ce640 1 -- 172.21.15.69:0/544881259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c01a3c30 con 0x7f11c0100b50 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.966+0000 7f11bd7fa640 1 -- 172.21.15.69:0/544881259 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f11b0005560 con 0x7f11c0100b50 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c8dcc740 1 Processor -- start 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c8dcc740 1 -- start start 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bd7fa640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11a80a2cf0 0x7f11a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d88a908c60 con 0x55d88a9afd80 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d88a8a0b90 con 0x55d88a9a2830 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d88a904c20 con 0x55d88a9a3150 2024-08-02T17:52:41.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11a80a2cf0 0x7f11a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bd7fa640 1 -- 172.21.15.69:0/544881259 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f11b00cf7e0 con 0x7f11c0100b50 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c4f6b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37356/0 (socket says 172.21.15.69:37356) 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bffff640 1 -- 172.21.15.69:0/2769751074 learned_addr learned my addr 172.21.15.69:0/2769751074 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11c4f6b640 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bffff640 1 -- 172.21.15.69:0/2769751074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 msgr2=0x55d88a9a2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bffff640 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9a2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bffff640 1 -- 172.21.15.69:0/2769751074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 msgr2=0x55d88a9b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.967+0000 7f11bffff640 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.968+0000 7f11bffff640 1 -- 172.21.15.69:0/2769751074 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88a618580 con 0x55d88a9a3150 2024-08-02T17:52:41.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.968+0000 7f11c4f6b640 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.968+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11a80a2cf0 0x7f11a80a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f11c00ffa60 tx=0x7f11b4005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.968+0000 7f11bffff640 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9af840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f11b00a4020 tx=0x7f11b009b300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bec6fc1ef8aef26 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.968+0000 7f11a67fc640 1 -- 172.21.15.69:0/2769751074 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b001c070 con 0x55d88a9a3150 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.968+0000 7f11a67fc640 1 -- 172.21.15.69:0/2769751074 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b009fc90 con 0x55d88a9a3150 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.968+0000 7f11a67fc640 1 -- 172.21.15.69:0/2769751074 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b00860a0 con 0x55d88a9a3150 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2769751074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 msgr2=0x55d88a9af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9af840 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f11b00a4020 tx=0x7f11b009b300 comp rx=0 tx=0).stop 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2769751074 shutdown_connections 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9af840 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/2769751074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2769751074 >> 172.21.15.69:0/2769751074 conn(0x55d88a99ef20 msgr2=0x55d88a99f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2769751074 shutdown_connections 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2769751074 wait complete. 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- start start 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 Processor -- start 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a2830 0x55d88a9ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9afd80 0x55d88a9b9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d88a9a5810 con 0x55d88a9a3150 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d88a8a67c0 con 0x55d88a9afd80 2024-08-02T17:52:41.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.969+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d88a9a5990 con 0x55d88a9a2830 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11c8dcc740 10 client.0 _ll_register_callbacks cb 0x55d88a8aff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a2830 0x55d88a9ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9ab8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56472/0 (socket says 172.21.15.69:56472) 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bf7fe640 1 -- 172.21.15.69:0/3340684314 learned_addr learned my addr 172.21.15.69:0/3340684314 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11c4f6b640 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9afd80 0x55d88a9b9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bffff640 1 -- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9afd80 msgr2=0x55d88a9b9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bffff640 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9afd80 0x55d88a9b9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bffff640 1 -- 172.21.15.69:0/3340684314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 msgr2=0x55d88a9ab8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bffff640 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9ab8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bffff640 1 -- 172.21.15.69:0/3340684314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88a6e2960 con 0x55d88a9a2830 2024-08-02T17:52:41.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11c4f6b640 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9afd80 0x55d88a9b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11bffff640 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a2830 0x55d88a9ab3b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55d88a9a37a0 tx=0x7f11b00054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=182d9279cf211fb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f119f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f119f7fe640 1 -- 172.21.15.69:0/3340684314 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b001c040 con 0x55d88a9a2830 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f119f7fe640 1 -- 172.21.15.69:0/3340684314 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f11b0099be0 con 0x55d88a9a2830 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11c8dcc740 1 -- 172.21.15.69:0/3340684314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b009fc90 con 0x55d88a9a2830 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f119f7fe640 1 -- 172.21.15.69:0/3340684314 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11b00862d0 con 0x55d88a9a2830 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11c8dcc740 10 client.26500 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.970+0000 7f11c8dcc740 1 -- 172.21.15.69:0/3340684314 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d88a9a5810 con 0x55d88a9a2830 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f119f7fe640 1 -- 172.21.15.69:0/3340684314 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b009fc90 con 0x55d88a9a2830 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f119f7fe640 1 -- 172.21.15.69:0/3340684314 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f11b00cf350 con 0x55d88a9a2830 2024-08-02T17:52:41.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f11c8dcc740 10 client.26500 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f11c8dcc740 10 client.26500 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f11c8dcc740 20 client.26500 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d88a9bda10 0x55d88a9bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f11c8dcc740 4 client.26500 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f11c8dcc740 1 -- 172.21.15.69:0/3340684314 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d88a9161d0 con 0x55d88a9bda10 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d88a9bda10 0x55d88a9bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f119f7fe640 10 client.26500 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.971+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d88a9bda10 0x55d88a9bfe00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fbb131c46dc741f server_cookie=c89ea8e92e4f6ca0 in_seq=0 out_seq=0 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.972+0000 7f119f7fe640 1 -- 172.21.15.69:0/3340684314 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d88a9161d0 con 0x55d88a9bda10 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.972+0000 7f119f7fe640 10 client.26500 handle_command_reply: tid=0 2024-08-02T17:52:41.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.972+0000 7f11c8dcc740 1 client.26500 shutdown 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 -- 172.21.15.69:0/3340684314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a2830 msgr2=0x55d88a9ab3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a2830 0x55d88a9ab3b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55d88a9a37a0 tx=0x7f11b00054d0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 -- 172.21.15.69:0/3340684314 shutdown_connections 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9ab8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d88a9bda10 0x55d88a9bfe00 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9afd80 0x55d88a9b9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/3340684314 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a2830 0x55d88a9ab3b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f119f7fe640 0 client.26500 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 -- 172.21.15.69:0/3340684314 >> 172.21.15.69:0/3340684314 conn(0x55d88a99ef20 msgr2=0x55d88a9a4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 -- 172.21.15.69:0/3340684314 shutdown_connections 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 1 -- 172.21.15.69:0/3340684314 wait complete. 2024-08-02T17:52:41.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.973+0000 7f11c8dcc740 20 client.26500 trim_cache size 0 max 16384 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11c8dcc740 1 Processor -- start 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11c8dcc740 1 -- start start 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9ab190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11c01af380 con 0x55d88a9afd80 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11c01acc50 con 0x55d88a9a2830 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11c01aaaa0 con 0x55d88a9a3150 2024-08-02T17:52:41.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.993+0000 7f11bf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9ab190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37382/0 (socket says 172.21.15.69:37382) 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bf7fe640 1 -- 172.21.15.69:0/2453771287 learned_addr learned my addr 172.21.15.69:0/2453771287 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bffff640 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11c4f6b640 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bf7fe640 1 -- 172.21.15.69:0/2453771287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 msgr2=0x55d88a9af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9af0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bf7fe640 1 -- 172.21.15.69:0/2453771287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 msgr2=0x55d88a9c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bf7fe640 1 -- 172.21.15.69:0/2453771287 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88a9af890 con 0x55d88a9a3150 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9ab190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55d88a9aaa60 tx=0x7f11b4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b29b1a097f2b27a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:41.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11a5ffb640 1 -- 172.21.15.69:0/2453771287 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4005220 con 0x55d88a9a3150 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11a5ffb640 1 -- 172.21.15.69:0/2453771287 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b401ccc0 con 0x55d88a9a3150 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.994+0000 7f11a5ffb640 1 -- 172.21.15.69:0/2453771287 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b4004ba0 con 0x55d88a9a3150 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2453771287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 msgr2=0x55d88a9ab190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9ab190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55d88a9aaa60 tx=0x7f11b4014000 comp rx=0 tx=0).stop 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2453771287 shutdown_connections 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9afd80 0x55d88a9c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9a3150 0x55d88a9ab190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/2453771287 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2453771287 >> 172.21.15.69:0/2453771287 conn(0x55d88a99ef20 msgr2=0x55d88a99e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2453771287 shutdown_connections 2024-08-02T17:52:41.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.995+0000 7f11c8dcc740 1 -- 172.21.15.69:0/2453771287 wait complete. 2024-08-02T17:52:42.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.996+0000 7f11c8dcc740 1 -- start start 2024-08-02T17:52:42.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.996+0000 7f11c8dcc740 1 Processor -- start 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.996+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.996+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.996+0000 7f11c8dcc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9afd80 0x55d88a9b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.996+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f11c01ab110 con 0x55d88a9a3150 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.996+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f11c01ac580 con 0x55d88a9a2830 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.996+0000 7f11c8dcc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f11c01ab7e0 con 0x55d88a9afd80 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11c4f6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9afd80 0x55d88a9b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43334/0 (socket says 172.21.15.69:43334) 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11c4f6b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9afd80 0x55d88a9b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37396/0 (socket says 172.21.15.69:37396) 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 -- 172.21.15.69:0/468132765 learned_addr learned my addr 172.21.15.69:0/468132765 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11c8dcc740 10 client.0 _ll_register_callbacks cb 0x7f11b00cca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:42.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 -- 172.21.15.69:0/468132765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9afd80 msgr2=0x55d88a9b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9afd80 0x55d88a9b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 -- 172.21.15.69:0/468132765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 msgr2=0x55d88a9b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9b5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 -- 172.21.15.69:0/468132765 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88aa27420 con 0x55d88a9a2830 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11c4f6b640 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9afd80 0x55d88a9b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11bffff640 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9b4d30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f11b00982b0 tx=0x7f11b009c120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd739022862cb4b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f119f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f119f7fe640 1 -- 172.21.15.69:0/468132765 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b00862d0 con 0x55d88a9a2830 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f119f7fe640 1 -- 172.21.15.69:0/468132765 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f11b00b27a0 con 0x55d88a9a2830 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.997+0000 7f11c8dcc740 1 -- 172.21.15.69:0/468132765 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b401ccc0 con 0x55d88a9a2830 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f119f7fe640 1 -- 172.21.15.69:0/468132765 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f11b009fc90 con 0x55d88a9a2830 2024-08-02T17:52:42.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11c8dcc740 10 client.26627 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11c8dcc740 1 -- 172.21.15.69:0/468132765 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d88aa28720 con 0x55d88a9a2830 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f119f7fe640 1 -- 172.21.15.69:0/468132765 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b401ccc0 con 0x55d88a9a2830 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f119f7fe640 1 -- 172.21.15.69:0/468132765 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f11b009bb40 con 0x55d88a9a2830 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11c8dcc740 10 client.26627 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11c8dcc740 10 client.26627 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11c8dcc740 20 client.26627 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d88aa28ba0 0x55d88aa2af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11c8dcc740 4 client.26627 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4e57b63c", "release": true, "await": true}] 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11c8dcc740 1 -- 172.21.15.69:0/468132765 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4e57b63c", "release": true, "await": true}) -- 0x55d88a9106e0 con 0x55d88aa28ba0 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.998+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d88aa28ba0 0x55d88aa2af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.999+0000 7f119f7fe640 10 client.26627 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:42.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:41.999+0000 7f11bf7fe640 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d88aa28ba0 0x55d88aa2af90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8420269da5101519 server_cookie=1b5fa301da12548 in_seq=0 out_seq=0 2024-08-02T17:52:42.022 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:52:42.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.041+0000 7f119f7fe640 1 -- 172.21.15.69:0/468132765 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d88a9106e0 con 0x55d88aa28ba0 2024-08-02T17:52:42.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.041+0000 7f119f7fe640 10 client.26627 handle_command_reply: tid=0 2024-08-02T17:52:42.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.041+0000 7f11c8dcc740 1 client.26627 shutdown 2024-08-02T17:52:42.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.044+0000 7f11c8dcc740 1 -- 172.21.15.69:0/468132765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 msgr2=0x55d88a9b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:42.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.044+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9b4d30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f11b00982b0 tx=0x7f11b009c120 comp rx=0 tx=0).stop 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.047+0000 7f11c8dcc740 1 -- 172.21.15.69:0/468132765 shutdown_connections 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.047+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d88a9a3150 0x55d88a9b5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.047+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d88aa28ba0 0x55d88aa2af90 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.047+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d88a9afd80 0x55d88a9b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.047+0000 7f119f7fe640 0 client.26627 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.047+0000 7f11c8dcc740 1 --2- 172.21.15.69:0/468132765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d88a9a2830 0x55d88a9b4d30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.047+0000 7f11c8dcc740 1 -- 172.21.15.69:0/468132765 >> 172.21.15.69:0/468132765 conn(0x55d88a99ef20 msgr2=0x55d88a9a9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.050+0000 7f11c8dcc740 1 -- 172.21.15.69:0/468132765 shutdown_connections 2024-08-02T17:52:42.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.050+0000 7f11c8dcc740 1 -- 172.21.15.69:0/468132765 wait complete. 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 24, 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "4e57b63c": { 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 24, 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:52:42.060 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 148.9, 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:42.061 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:42.062 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:42.062 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:52:42.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.052+0000 7f11c8dcc740 20 client.26627 trim_cache size 0 max 16384 2024-08-02T17:52:42.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.056+0000 7f11c69ce640 1 -- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11a80a2cf0 msgr2=0x7f11a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:42.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.056+0000 7f11c69ce640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11a80a2cf0 0x7f11a80a51b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f11c00ffa60 tx=0x7f11b4005f90 comp rx=0 tx=0).stop 2024-08-02T17:52:42.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.057+0000 7f11c69ce640 1 -- 172.21.15.69:0/544881259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11c0100b50 msgr2=0x7f11c01970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:42.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.057+0000 7f11c69ce640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11c0100b50 0x7f11c01970b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f11b000e9f0 tx=0x7f11b000eeb0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.058+0000 7f11c69ce640 1 -- 172.21.15.69:0/544881259 shutdown_connections 2024-08-02T17:52:42.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.058+0000 7f11c69ce640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f11c01022a0 0x7f11c01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.058+0000 7f11c69ce640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f11a80a2cf0 0x7f11a80a51b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.058+0000 7f11c69ce640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f11c0102c20 0x7f11c019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.058+0000 7f11c69ce640 1 --2- 172.21.15.69:0/544881259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f11c0100b50 0x7f11c01970b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:42.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.058+0000 7f11c69ce640 1 -- 172.21.15.69:0/544881259 >> 172.21.15.69:0/544881259 conn(0x7f11c00fc500 msgr2=0x7f11c00fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:42.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.061+0000 7f11c69ce640 1 -- 172.21.15.69:0/544881259 shutdown_connections 2024-08-02T17:52:42.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:42.061+0000 7f11c69ce640 1 -- 172.21.15.69:0/544881259 wait complete. 2024-08-02T17:52:42.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:42 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:52:42.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:42 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:52:42.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:42 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:52:42.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:42 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:52:42.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:42 smithi069 ceph-mon[32746]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 96 KiB/s wr, 17 op/s 2024-08-02T17:52:42.609 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:42 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:52:42.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:42 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:52:42.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:42 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:52:42.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:42 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:52:42.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:42 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:52:42.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:42 smithi083 ceph-mon[37031]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 96 KiB/s wr, 17 op/s 2024-08-02T17:52:42.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:42 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:52:42.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:42 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:52:42.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:42 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:52:42.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:42 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:52:42.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:42 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:52:42.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:42 smithi077 ceph-mon[37108]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 96 KiB/s wr, 17 op/s 2024-08-02T17:52:42.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:42 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:52:43.195 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4e57b63c', total seconds elapsed: 33.5 2024-08-02T17:52:43.397 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:43 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:52:43.397 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:43 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:52:43.398 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:43 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:52:43.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.679+0000 7f8fc94e4640 1 -- 172.21.15.69:0/1908651368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109560 msgr2=0x7f8fc41099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.679+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/1908651368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109560 0x7f8fc41099c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8fb801a910 tx=0x7f8fb803b470 comp rx=0 tx=0).stop 2024-08-02T17:52:43.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.681+0000 7f8fc94e4640 1 -- 172.21.15.69:0/1908651368 shutdown_connections 2024-08-02T17:52:43.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.681+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/1908651368 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc411ae70 0x7f8fc411d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.681+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/1908651368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109560 0x7f8fc41099c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.681+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/1908651368 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc4109180 0x7f8fc410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.681+0000 7f8fc94e4640 1 -- 172.21.15.69:0/1908651368 >> 172.21.15.69:0/1908651368 conn(0x7f8fc406de70 msgr2=0x7f8fc406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:43.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.688+0000 7f8fc94e4640 1 -- 172.21.15.69:0/1908651368 shutdown_connections 2024-08-02T17:52:43.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.688+0000 7f8fc94e4640 1 -- 172.21.15.69:0/1908651368 wait complete. 2024-08-02T17:52:43.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.689+0000 7f8fc94e4640 1 Processor -- start 2024-08-02T17:52:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.689+0000 7f8fc94e4640 1 -- start start 2024-08-02T17:52:43.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.690+0000 7f8fc94e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109180 0x7f8fc411a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.690+0000 7f8fc94e4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc4109560 0x7f8fc4116780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.690+0000 7f8fc2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109180 0x7f8fc411a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.690+0000 7f8fc2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109180 0x7f8fc411a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37402/0 (socket says 172.21.15.69:37402) 2024-08-02T17:52:43.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.690+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/2833771555 learned_addr learned my addr 172.21.15.69:0/2833771555 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:43.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.691+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc411ae70 0x7f8fc4116cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.691+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41130f0 con 0x7f8fc4109560 2024-08-02T17:52:43.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.691+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4112f70 con 0x7f8fc411ae70 2024-08-02T17:52:43.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.691+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc411ae70 0x7f8fc4116cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.691+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4113270 con 0x7f8fc4109180 2024-08-02T17:52:43.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.695+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc4109560 0x7f8fc4116780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.695+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/2833771555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc411ae70 msgr2=0x7f8fc4116cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.695+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc411ae70 0x7f8fc4116cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.695+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc4109560 msgr2=0x7f8fc4116780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.695+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc4109560 0x7f8fc4116780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.695+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/2833771555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc4117560 con 0x7f8fc4109180 2024-08-02T17:52:43.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.695+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109180 0x7f8fc411a6e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00ba50 tx=0x7f8fbc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:43.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.695+0000 7f8fabfff640 1 -- 172.21.15.69:0/2833771555 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc019070 con 0x7f8fc4109180 2024-08-02T17:52:43.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.696+0000 7f8fabfff640 1 -- 172.21.15.69:0/2833771555 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc00cbe0 con 0x7f8fc4109180 2024-08-02T17:52:43.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.696+0000 7f8fabfff640 1 -- 172.21.15.69:0/2833771555 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0076e0 con 0x7f8fc4109180 2024-08-02T17:52:43.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.696+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc41aaea0 con 0x7f8fc4109180 2024-08-02T17:52:43.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.696+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41ab2a0 con 0x7f8fc4109180 2024-08-02T17:52:43.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.699+0000 7f8fcb8e2740 1 Processor -- start 2024-08-02T17:52:43.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.700+0000 7f8fcb8e2740 1 -- start start 2024-08-02T17:52:43.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.701+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447627c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.701+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x564476286eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.701+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x5644762897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.701+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5644761e1980 con 0x56447627cb30 2024-08-02T17:52:43.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.701+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564476179a70 con 0x56447627c210 2024-08-02T17:52:43.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.701+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5644761dd940 con 0x5644762873f0 2024-08-02T17:52:43.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.701+0000 7f8fabfff640 1 -- 172.21.15.69:0/2833771555 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8fbc01c070 con 0x7f8fc4109180 2024-08-02T17:52:43.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.702+0000 7f8fabfff640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fac0a2e40 0x7f8fac0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.702+0000 7f8fabfff640 1 -- 172.21.15.69:0/2833771555 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8fbc0cec40 con 0x7f8fc4109180 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.703+0000 7f8fc27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x5644762897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.703+0000 7f8fc27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x5644762897e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37416/0 (socket says 172.21.15.69:37416) 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.703+0000 7f8fc27fc640 1 -- 172.21.15.69:0/1021557670 learned_addr learned my addr 172.21.15.69:0/1021557670 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.703+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/1021557670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447627c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.704+0000 7f8fc37fe640 1 -- 172.21.15.69:0/1021557670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 msgr2=0x5644762897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.704+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/1021557670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x5644762897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.704+0000 7f8fc37fe640 1 -- 172.21.15.69:0/1021557670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 msgr2=0x564476286eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.704+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/1021557670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x564476286eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.704+0000 7f8fc37fe640 1 -- 172.21.15.69:0/1021557670 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644761dc6e0 con 0x56447627c210 2024-08-02T17:52:43.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.704+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fac0a2e40 0x7f8fac0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.707+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/1021557670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447627c5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009a00 tx=0x7f8fb4009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c041aae930dad7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:43.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.713+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fac0a2e40 0x7f8fac0a5300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8fb401f000 tx=0x7f8fb4020000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:43.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.715+0000 7f8fa97fa640 1 -- 172.21.15.69:0/1021557670 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4018070 con 0x56447627c210 2024-08-02T17:52:43.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.716+0000 7f8fa97fa640 1 -- 172.21.15.69:0/1021557670 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb401ec40 con 0x56447627c210 2024-08-02T17:52:43.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.716+0000 7f8fa97fa640 1 -- 172.21.15.69:0/1021557670 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400f040 con 0x56447627c210 2024-08-02T17:52:43.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1021557670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 msgr2=0x56447627c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1021557670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447627c5f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009a00 tx=0x7f8fb4009ec0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1021557670 shutdown_connections 2024-08-02T17:52:43.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1021557670 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x564476286eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1021557670 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x5644762897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1021557670 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447627c5f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1021557670 >> 172.21.15.69:0/1021557670 conn(0x5644762787a0 msgr2=0x564476278b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:43.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1021557670 shutdown_connections 2024-08-02T17:52:43.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.717+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1021557670 wait complete. 2024-08-02T17:52:43.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.718+0000 7f8fcb8e2740 1 -- start start 2024-08-02T17:52:43.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.718+0000 7f8fcb8e2740 1 Processor -- start 2024-08-02T17:52:43.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.718+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627c210 0x564476284390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.718+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x5644762848d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x564476284e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564476289fc0 con 0x56447627c210 2024-08-02T17:52:43.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56447619caa0 con 0x56447627cb30 2024-08-02T17:52:43.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56447628a140 con 0x5644762873f0 2024-08-02T17:52:43.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fcb8e2740 10 client.0 _ll_register_callbacks cb 0x56447618be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:43.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x5644762848d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x5644762848d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43354/0 (socket says 172.21.15.69:43354) 2024-08-02T17:52:43.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc27fc640 1 -- 172.21.15.69:0/1369188218 learned_addr learned my addr 172.21.15.69:0/1369188218 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:43.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x564476284e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc27fc640 1 -- 172.21.15.69:0/1369188218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 msgr2=0x564476284e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x564476284e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc27fc640 1 -- 172.21.15.69:0/1369188218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627c210 msgr2=0x564476284390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627c210 0x564476284390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.719+0000 7f8fc27fc640 1 -- 172.21.15.69:0/1369188218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564476180650 con 0x56447627cb30 2024-08-02T17:52:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.720+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x5644762848d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8004d80 tx=0x7f8fb8004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3dd0afd9a87ab76 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.721+0000 7f8fa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:52:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.721+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/1369188218 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8046030 con 0x56447627cb30 2024-08-02T17:52:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.721+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/1369188218 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8fb8005140 con 0x56447627cb30 2024-08-02T17:52:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.721+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1369188218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb401ec40 con 0x56447627cb30 2024-08-02T17:52:43.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.723+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/1369188218 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8fb8005360 con 0x56447627cb30 2024-08-02T17:52:43.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.724+0000 7f8fcb8e2740 10 client.26636 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:43.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.724+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1369188218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564475f533c0 con 0x56447627cb30 2024-08-02T17:52:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.724+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/1369188218 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb401ec40 con 0x56447627cb30 2024-08-02T17:52:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.724+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/1369188218 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8fb804b040 con 0x56447627cb30 2024-08-02T17:52:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.725+0000 7f8fcb8e2740 10 client.26636 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.725+0000 7f8fcb8e2740 10 client.26636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.725+0000 7f8fcb8e2740 20 client.26636 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.725+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564476297420 0x5644762998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.725+0000 7f8fcb8e2740 4 client.26636 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.725+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1369188218 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644761eeef0 con 0x564476297420 2024-08-02T17:52:43.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.727+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564476297420 0x5644762998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.727+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564476297420 0x5644762998c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3cddfe3932ddb7 server_cookie=150193bde1100b2 in_seq=0 out_seq=0 2024-08-02T17:52:43.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.727+0000 7f8fa6ff5640 10 client.26636 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:43.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.728+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/1369188218 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5644761eeef0 con 0x564476297420 2024-08-02T17:52:43.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.728+0000 7f8fa6ff5640 10 client.26636 handle_command_reply: tid=0 2024-08-02T17:52:43.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.728+0000 7f8fcb8e2740 1 client.26636 shutdown 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.728+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1369188218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 msgr2=0x5644762848d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.728+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x5644762848d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8004d80 tx=0x7f8fb8004db0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1369188218 shutdown_connections 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564476297420 0x5644762998c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x564476284e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x5644762848d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/1369188218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627c210 0x564476284390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1369188218 >> 172.21.15.69:0/1369188218 conn(0x5644762787a0 msgr2=0x56447627e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fa6ff5640 0 client.26636 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1369188218 shutdown_connections 2024-08-02T17:52:43.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/1369188218 wait complete. 2024-08-02T17:52:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.729+0000 7f8fcb8e2740 20 client.26636 trim_cache size 0 max 16384 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.754+0000 7f8fcb8e2740 1 Processor -- start 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.754+0000 7f8fcb8e2740 1 -- start start 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.754+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56447627c210 0x56447627ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.754+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x564476278b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.754+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5644762873f0 0x5644762861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb00128d0 con 0x5644762873f0 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb00101a0 con 0x56447627cb30 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fcb8e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb000dff0 con 0x56447627c210 2024-08-02T17:52:43.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56447627c210 0x56447627ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5644762873f0 0x5644762861f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56447627c210 0x56447627ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:37440/0 (socket says 172.21.15.69:37440) 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/110552827 learned_addr learned my addr 172.21.15.69:0/110552827 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x564476278b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/110552827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 msgr2=0x564476278b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x564476278b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/110552827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5644762873f0 msgr2=0x5644762861f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5644762873f0 0x5644762861f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/110552827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564476283340 con 0x56447627c210 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x564476278b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56447627c210 0x56447627ecc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc097360 tx=0x7f8fbc015fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=468807827f2b422b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.755+0000 7f8fa8ff9640 1 -- 172.21.15.69:0/110552827 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0a3ac0 con 0x56447627c210 2024-08-02T17:52:43.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fa8ff9640 1 -- 172.21.15.69:0/110552827 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc09d030 con 0x56447627c210 2024-08-02T17:52:43.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fa8ff9640 1 -- 172.21.15.69:0/110552827 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0a3630 con 0x56447627c210 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/110552827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56447627c210 msgr2=0x56447627ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56447627c210 0x56447627ecc0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc097360 tx=0x7f8fbc015fc0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/110552827 shutdown_connections 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5644762873f0 0x5644762861f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627cb30 0x564476278b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/110552827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56447627c210 0x56447627ecc0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/110552827 >> 172.21.15.69:0/110552827 conn(0x5644762787a0 msgr2=0x564476285ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.756+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/110552827 shutdown_connections 2024-08-02T17:52:43.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/110552827 wait complete. 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 -- start start 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 Processor -- start 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447628a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x56447628af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x564476300190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fc27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x56447628af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fc27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x56447628af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56534/0 (socket says 172.21.15.69:56534) 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fc27fc640 1 -- 172.21.15.69:0/2048046841 learned_addr learned my addr 172.21.15.69:0/2048046841 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb000e660 con 0x56447627cb30 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb000fad0 con 0x56447627c210 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb000ed30 con 0x5644762873f0 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.757+0000 7f8fcb8e2740 10 client.0 _ll_register_callbacks cb 0x7f8fb801a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447628a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc37fe640 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x564476300190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/2048046841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 msgr2=0x564476300190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x564476300190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/2048046841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 msgr2=0x56447628af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x56447628af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc2ffd640 1 -- 172.21.15.69:0/2048046841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56447627c960 con 0x56447627c210 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x56447628af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fc2ffd640 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447628a9d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc09ae90 tx=0x7f8fbc097570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b393d1448ce9a02 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.758+0000 7f8fa6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.759+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/2048046841 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc07d070 con 0x56447627c210 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.759+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/2048046841 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f8fbc00cd90 con 0x56447627c210 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.759+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fbc09d030 con 0x56447627c210 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.759+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/2048046841 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8fbc0840d0 con 0x56447627c210 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.759+0000 7f8fcb8e2740 10 client.26642 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:43.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.759+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564476301800 con 0x56447627c210 2024-08-02T17:52:43.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.759+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/2048046841 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc09d030 con 0x56447627c210 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.760+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/2048046841 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8fbc09a350 con 0x56447627c210 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.760+0000 7f8fcb8e2740 10 client.26642 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.760+0000 7f8fcb8e2740 10 client.26642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.760+0000 7f8fcb8e2740 20 client.26642 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.760+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564476301e20 0x564476304240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.760+0000 7f8fcb8e2740 4 client.26642 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.760+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644761e9400 con 0x564476301e20 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.760+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564476301e20 0x564476304240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:43.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.761+0000 7f8fc27fc640 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564476301e20 0x564476304240 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce25d3dc725c772f server_cookie=f1e5c463ffedbb2b in_seq=0 out_seq=0 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.761+0000 7f8fa6ff5640 10 client.26642 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.761+0000 7f8fa6ff5640 1 -- 172.21.15.69:0/2048046841 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5644761e9400 con 0x564476301e20 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.761+0000 7f8fa6ff5640 10 client.26642 handle_command_reply: tid=0 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 client.26642 shutdown 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 msgr2=0x56447628a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447628a9d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc09ae90 tx=0x7f8fbc097570 comp rx=0 tx=0).stop 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 shutdown_connections 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56447627cb30 0x56447628af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564476301e20 0x564476304240 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5644762873f0 0x564476300190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 --2- 172.21.15.69:0/2048046841 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56447627c210 0x56447628a9d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 >> 172.21.15.69:0/2048046841 conn(0x5644762787a0 msgr2=0x564476277900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fa6ff5640 0 client.26642 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:43.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 shutdown_connections 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 1 -- 172.21.15.69:0/2048046841 wait complete. 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.762+0000 7f8fcb8e2740 20 client.26642 trim_cache size 0 max 16384 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout: "0c79fe6f-e8a5-498f-9395-8cd814e28e2f": { 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "0c79fe6f-e8a5-498f-9395-8cd814e28e2f", 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:43.768 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:52:43.769 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:52:43.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fac0a2e40 msgr2=0x7f8fac0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fac0a2e40 0x7f8fac0a5300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8fb401f000 tx=0x7f8fb4020000 comp rx=0 tx=0).stop 2024-08-02T17:52:43.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109180 msgr2=0x7f8fc411a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:43.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109180 0x7f8fc411a6e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00ba50 tx=0x7f8fbc00bf10 comp rx=0 tx=0).stop 2024-08-02T17:52:43.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 shutdown_connections 2024-08-02T17:52:43.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fc4109560 0x7f8fc4116780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fac0a2e40 0x7f8fac0a5300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fc411ae70 0x7f8fc4116cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:43.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 --2- 172.21.15.69:0/2833771555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fc4109180 0x7f8fc411a6e0 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00ba50 tx=0x7f8fbc00bf10 comp rx=0 tx=0).stop 2024-08-02T17:52:43.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.764+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 >> 172.21.15.69:0/2833771555 conn(0x7f8fc406de70 msgr2=0x7f8fc40709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:43.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.765+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 shutdown_connections 2024-08-02T17:52:43.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:52:43.765+0000 7f8fc94e4640 1 -- 172.21.15.69:0/2833771555 wait complete. 2024-08-02T17:52:43.770 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:43 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:52:43.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:43 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:52:43.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:43 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:52:43.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:43 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:52:43.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:43 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:52:43.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:43 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:52:44.542 INFO:tasks.cephfs.filesystem:scrub status for tag:0c79fe6f-e8a5-498f-9395-8cd814e28e2f - {'path': '/', 'tag': '0c79fe6f-e8a5-498f-9395-8cd814e28e2f', 'options': 'recursive,force'} 2024-08-02T17:52:44.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:44 smithi069 ceph-mon[32746]: pgmap v192: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 132 KiB/s wr, 109 op/s 2024-08-02T17:52:44.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:44 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:52:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 335.0 s, 421.6 tps, lat 1436.948 ms stddev 6046.556 2024-08-02T17:52:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:44 smithi083 ceph-mon[37031]: pgmap v192: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 132 KiB/s wr, 109 op/s 2024-08-02T17:52:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:44 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:52:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:44 smithi077 ceph-mon[37108]: pgmap v192: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 132 KiB/s wr, 109 op/s 2024-08-02T17:52:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:44 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:52:47.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:46 smithi083 ceph-mon[37031]: pgmap v193: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 47 KiB/s wr, 101 op/s 2024-08-02T17:52:47.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:46 smithi077 ceph-mon[37108]: pgmap v193: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 47 KiB/s wr, 101 op/s 2024-08-02T17:52:47.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:46 smithi069 ceph-mon[32746]: pgmap v193: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 47 KiB/s wr, 101 op/s 2024-08-02T17:52:48.909 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:48 smithi069 ceph-mon[32746]: pgmap v194: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.2 MiB/s wr, 332 op/s 2024-08-02T17:52:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:48 smithi083 ceph-mon[37031]: pgmap v194: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.2 MiB/s wr, 332 op/s 2024-08-02T17:52:49.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:48 smithi077 ceph-mon[37108]: pgmap v194: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 6.2 MiB/s wr, 332 op/s 2024-08-02T17:52:49.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 340.0 s, 1274.0 tps, lat 78.986 ms stddev 105.125 2024-08-02T17:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.990+0000 7ff36e028640 1 -- 172.21.15.69:0/2229487326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3681006d0 msgr2=0x7ff368100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:50.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.990+0000 7ff36e028640 1 --2- 172.21.15.69:0/2229487326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3681006d0 0x7ff368100b30 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff35c009930 tx=0x7ff35c02a420 comp rx=0 tx=0).stop 2024-08-02T17:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.991+0000 7ff36e028640 1 -- 172.21.15.69:0/2229487326 shutdown_connections 2024-08-02T17:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.991+0000 7ff36e028640 1 --2- 172.21.15.69:0/2229487326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3681006d0 0x7ff368100b30 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.991+0000 7ff36e028640 1 --2- 172.21.15.69:0/2229487326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff368101070 0x7ff36810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.991+0000 7ff36e028640 1 --2- 172.21.15.69:0/2229487326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3681066e0 0x7ff368106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.991+0000 7ff36e028640 1 -- 172.21.15.69:0/2229487326 >> 172.21.15.69:0/2229487326 conn(0x7ff3680776d0 msgr2=0x7ff3680fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.991+0000 7ff36e028640 1 -- 172.21.15.69:0/2229487326 shutdown_connections 2024-08-02T17:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.992+0000 7ff36e028640 1 -- 172.21.15.69:0/2229487326 wait complete. 2024-08-02T17:52:50.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.992+0000 7ff36e028640 1 Processor -- start 2024-08-02T17:52:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.992+0000 7ff36e028640 1 -- start start 2024-08-02T17:52:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff36e028640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3681006d0 0x7ff36810d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:50.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff36e028640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff368101070 0x7ff368109240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff36e028640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3681066e0 0x7ff368109780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff36e028640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff36810fa10 con 0x7ff3681006d0 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff36e028640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff36810f890 con 0x7ff3681066e0 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff36e028640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff36810fb90 con 0x7ff368101070 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff367fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3681066e0 0x7ff368109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff3677fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3681006d0 0x7ff36810d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff366ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff368101070 0x7ff368109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff367fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3681066e0 0x7ff368109780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44086/0 (socket says 172.21.15.69:44086) 2024-08-02T17:52:50.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff367fff640 1 -- 172.21.15.69:0/2339125396 learned_addr learned my addr 172.21.15.69:0/2339125396 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff367fff640 1 -- 172.21.15.69:0/2339125396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff368101070 msgr2=0x7ff368109240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff367fff640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff368101070 0x7ff368109240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff367fff640 1 -- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3681006d0 msgr2=0x7ff36810d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff367fff640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3681006d0 0x7ff36810d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff367fff640 1 -- 172.21.15.69:0/2339125396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff368109e80 con 0x7ff3681066e0 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.993+0000 7ff366ffd640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff368101070 0x7ff368109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.994+0000 7ff367fff640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3681066e0 0x7ff368109780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff36000eb10 tx=0x7ff36000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.994+0000 7ff364ff9640 1 -- 172.21.15.69:0/2339125396 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360004b20 con 0x7ff3681066e0 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.994+0000 7ff364ff9640 1 -- 172.21.15.69:0/2339125396 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff360004cc0 con 0x7ff3681066e0 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.994+0000 7ff36e028640 1 -- 172.21.15.69:0/2339125396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3681aadb0 con 0x7ff3681066e0 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.994+0000 7ff364ff9640 1 -- 172.21.15.69:0/2339125396 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360010760 con 0x7ff3681066e0 2024-08-02T17:52:50.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.994+0000 7ff36e028640 1 -- 172.21.15.69:0/2339125396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3681ab160 con 0x7ff3681066e0 2024-08-02T17:52:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.998+0000 7ff370d80740 1 Processor -- start 2024-08-02T17:52:50.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.998+0000 7ff364ff9640 1 -- 172.21.15.69:0/2339125396 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff360005020 con 0x7ff3681066e0 2024-08-02T17:52:51.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.998+0000 7ff370d80740 1 -- start start 2024-08-02T17:52:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff370d80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff370d80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff370d80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4d4970 0x55b72a4d6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff370d80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b72a42d9b0 con 0x55b72a4d4970 2024-08-02T17:52:51.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff370d80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b72a3d5800 con 0x55b72a4c97b0 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff370d80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b72a429970 con 0x55b72a4ca0d0 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff3677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4d4970 0x55b72a4d6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff3677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:52758/0 (socket says 172.21.15.69:52758) 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4d4970 0x55b72a4d6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33440/0 (socket says 172.21.15.69:33440) 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 -- 172.21.15.69:0/1084027112 learned_addr learned my addr 172.21.15.69:0/1084027112 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 -- 172.21.15.69:0/1084027112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4ca0d0 msgr2=0x55b72a4d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 --2- 172.21.15.69:0/1084027112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 -- 172.21.15.69:0/1084027112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 msgr2=0x55b72a4c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 --2- 172.21.15.69:0/1084027112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff366ffd640 1 -- 172.21.15.69:0/1084027112 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72a141210 con 0x55b72a4d4970 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff364ff9640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3500a2cf0 0x7ff3500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:50.999+0000 7ff3677fe640 1 --2- 172.21.15.69:0/1084027112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.000+0000 7ff366ffd640 1 --2- 172.21.15.69:0/1084027112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4d4970 0x55b72a4d6d60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff35c02a940 tx=0x7ff35c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122781db76df3039 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.000+0000 7ff364ff9640 1 -- 172.21.15.69:0/2339125396 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff3600cfd30 con 0x7ff3681066e0 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.000+0000 7ff3677fe640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3500a2cf0 0x7ff3500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.000+0000 7ff34dffb640 1 -- 172.21.15.69:0/1084027112 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c041020 con 0x55b72a4d4970 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.000+0000 7ff34dffb640 1 -- 172.21.15.69:0/1084027112 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c0317a0 con 0x55b72a4d4970 2024-08-02T17:52:51.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.000+0000 7ff34dffb640 1 -- 172.21.15.69:0/1084027112 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c035020 con 0x55b72a4d4970 2024-08-02T17:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff3677fe640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3500a2cf0 0x7ff3500a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55b72a4c6b80 tx=0x7ff358005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 -- 172.21.15.69:0/1084027112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4d4970 msgr2=0x55b72a4d6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 --2- 172.21.15.69:0/1084027112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4d4970 0x55b72a4d6d60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff35c02a940 tx=0x7ff35c005fd0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 -- 172.21.15.69:0/1084027112 shutdown_connections 2024-08-02T17:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 --2- 172.21.15.69:0/1084027112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4d4970 0x55b72a4d6d60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 --2- 172.21.15.69:0/1084027112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 --2- 172.21.15.69:0/1084027112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 -- 172.21.15.69:0/1084027112 >> 172.21.15.69:0/1084027112 conn(0x55b72a4c5d20 msgr2=0x55b72a4c6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 -- 172.21.15.69:0/1084027112 shutdown_connections 2024-08-02T17:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.001+0000 7ff370d80740 1 -- 172.21.15.69:0/1084027112 wait complete. 2024-08-02T17:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.002+0000 7ff370d80740 1 -- start start 2024-08-02T17:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.002+0000 7ff370d80740 1 Processor -- start 2024-08-02T17:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff370d80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4d18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff370d80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff370d80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4d4970 0x55b72a4d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff370d80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b72a4d7540 con 0x55b72a4ca0d0 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff370d80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b72a3e9500 con 0x55b72a4c97b0 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff370d80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b72a4d76c0 con 0x55b72a4d4970 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff3677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4d18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff3677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4d18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44092/0 (socket says 172.21.15.69:44092) 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff3677fe640 1 -- 172.21.15.69:0/2070379123 learned_addr learned my addr 172.21.15.69:0/2070379123 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff370d80740 10 client.0 _ll_register_callbacks cb 0x55b72a140a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff3677fe640 1 -- 172.21.15.69:0/2070379123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4d4970 msgr2=0x55b72a4d2360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:52:51.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff3677fe640 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4d4970 0x55b72a4d2360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff3677fe640 1 -- 172.21.15.69:0/2070379123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 msgr2=0x55b72a4d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff3677fe640 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.003+0000 7ff3677fe640 1 -- 172.21.15.69:0/2070379123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72a20d620 con 0x55b72a4c97b0 2024-08-02T17:52:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff3677fe640 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4d18e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff358004610 tx=0x7ff358004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6ef081f1266939 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff346ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:52:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff346ffd640 1 -- 172.21.15.69:0/2070379123 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35801e680 con 0x55b72a4c97b0 2024-08-02T17:52:51.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff370d80740 1 -- 172.21.15.69:0/2070379123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff35c0317a0 con 0x55b72a4c97b0 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff346ffd640 1 -- 172.21.15.69:0/2070379123 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff35801e820 con 0x55b72a4c97b0 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff346ffd640 1 -- 172.21.15.69:0/2070379123 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff35801ea20 con 0x55b72a4c97b0 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff370d80740 10 client.26654 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff370d80740 1 -- 172.21.15.69:0/2070379123 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b72a4e32f0 con 0x55b72a4c97b0 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.004+0000 7ff346ffd640 1 -- 172.21.15.69:0/2070379123 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff35c0317a0 con 0x55b72a4c97b0 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff346ffd640 1 -- 172.21.15.69:0/2070379123 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff358027070 con 0x55b72a4c97b0 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff370d80740 10 client.26654 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff370d80740 10 client.26654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff370d80740 20 client.26654 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff370d80740 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b72a4e4850 0x55b72a4e6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff370d80740 4 client.26654 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:52:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff370d80740 1 -- 172.21.15.69:0/2070379123 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b72a43b0e0 con 0x55b72a4e4850 2024-08-02T17:52:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff366ffd640 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b72a4e4850 0x55b72a4e6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff366ffd640 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b72a4e4850 0x55b72a4e6d10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b891dbc8001d96 server_cookie=d9ec4fdceb309ce8 in_seq=0 out_seq=0 2024-08-02T17:52:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.005+0000 7ff346ffd640 10 client.26654 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff346ffd640 1 -- 172.21.15.69:0/2070379123 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b72a43b0e0 con 0x55b72a4e4850 2024-08-02T17:52:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff346ffd640 10 client.26654 handle_command_reply: tid=0 2024-08-02T17:52:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 client.26654 shutdown 2024-08-02T17:52:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 -- 172.21.15.69:0/2070379123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 msgr2=0x55b72a4d18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4d18e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff358004610 tx=0x7ff358004730 comp rx=0 tx=0).stop 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 -- 172.21.15.69:0/2070379123 shutdown_connections 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b72a4e4850 0x55b72a4e6d10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4d4970 0x55b72a4d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 --2- 172.21.15.69:0/2070379123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4d18e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 -- 172.21.15.69:0/2070379123 >> 172.21.15.69:0/2070379123 conn(0x55b72a4c5d20 msgr2=0x55b72a4cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff346ffd640 0 client.26654 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 -- 172.21.15.69:0/2070379123 shutdown_connections 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.006+0000 7ff370d80740 1 -- 172.21.15.69:0/2070379123 wait complete. 2024-08-02T17:52:51.015 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.007+0000 7ff370d80740 20 client.26654 trim_cache size 0 max 16384 2024-08-02T17:52:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:50 smithi083 ceph-mon[37031]: pgmap v195: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.5 MiB/s wr, 368 op/s 2024-08-02T17:52:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.035+0000 7ff370d80740 1 Processor -- start 2024-08-02T17:52:51.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.035+0000 7ff370d80740 1 -- start start 2024-08-02T17:52:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.036+0000 7ff370d80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4c97b0 0x55b72a4ec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.038 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.036+0000 7ff370d80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4ecb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.036+0000 7ff3677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4c97b0 0x55b72a4ec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.036+0000 7ff3677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4c97b0 0x55b72a4ec620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:52788/0 (socket says 172.21.15.69:52788) 2024-08-02T17:52:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.036+0000 7ff3677fe640 1 -- 172.21.15.69:0/3745709939 learned_addr learned my addr 172.21.15.69:0/3745709939 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.036+0000 7ff370d80740 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4d4970 0x55b72a4d3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.037+0000 7ff370d80740 1 -- 172.21.15.69:0/3745709939 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3681b7ca0 con 0x55b72a4ca0d0 2024-08-02T17:52:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.037+0000 7ff370d80740 1 -- 172.21.15.69:0/3745709939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3681b5570 con 0x55b72a4d4970 2024-08-02T17:52:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.037+0000 7ff367fff640 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4d4970 0x55b72a4d3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.037+0000 7ff370d80740 1 -- 172.21.15.69:0/3745709939 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3681b33c0 con 0x55b72a4c97b0 2024-08-02T17:52:51.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.039+0000 7ff367fff640 1 -- 172.21.15.69:0/3745709939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4c97b0 msgr2=0x55b72a4ec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.039+0000 7ff367fff640 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4c97b0 0x55b72a4ec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.039+0000 7ff367fff640 1 -- 172.21.15.69:0/3745709939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 msgr2=0x55b72a4ecb60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:52:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.039+0000 7ff367fff640 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4ecb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.039+0000 7ff367fff640 1 -- 172.21.15.69:0/3745709939 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72a4d4390 con 0x55b72a4d4970 2024-08-02T17:52:51.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.040+0000 7ff367fff640 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4d4970 0x55b72a4d3030 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3600cd0c0 tx=0x7ff36009bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d431e4c2dea0269 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.044+0000 7ff34d7fa640 1 -- 172.21.15.69:0/3745709939 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36009be90 con 0x55b72a4d4970 2024-08-02T17:52:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.044+0000 7ff34d7fa640 1 -- 172.21.15.69:0/3745709939 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3600a0240 con 0x55b72a4d4970 2024-08-02T17:52:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.045+0000 7ff370d80740 1 -- 172.21.15.69:0/3745709939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4d4970 msgr2=0x55b72a4d3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:51.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.045+0000 7ff370d80740 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4d4970 0x55b72a4d3030 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3600cd0c0 tx=0x7ff36009bd20 comp rx=0 tx=0).stop 2024-08-02T17:52:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.045+0000 7ff370d80740 1 -- 172.21.15.69:0/3745709939 shutdown_connections 2024-08-02T17:52:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.045+0000 7ff370d80740 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4ecb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.045+0000 7ff370d80740 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4d4970 0x55b72a4d3030 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.045+0000 7ff370d80740 1 --2- 172.21.15.69:0/3745709939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4c97b0 0x55b72a4ec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.045+0000 7ff370d80740 1 -- 172.21.15.69:0/3745709939 >> 172.21.15.69:0/3745709939 conn(0x55b72a4c5d20 msgr2=0x55b72a4ccbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.052+0000 7ff370d80740 1 -- 172.21.15.69:0/3745709939 shutdown_connections 2024-08-02T17:52:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.054+0000 7ff370d80740 1 -- 172.21.15.69:0/3745709939 wait complete. 2024-08-02T17:52:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.055+0000 7ff370d80740 1 -- start start 2024-08-02T17:52:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.055+0000 7ff370d80740 1 Processor -- start 2024-08-02T17:52:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.055+0000 7ff370d80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.055+0000 7ff370d80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.055+0000 7ff370d80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4d4970 0x55b72a4d8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.055+0000 7ff370d80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3681b3a30 con 0x55b72a4ca0d0 2024-08-02T17:52:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.055+0000 7ff370d80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3681b4ea0 con 0x55b72a4c97b0 2024-08-02T17:52:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.055+0000 7ff370d80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3681b4100 con 0x55b72a4d4970 2024-08-02T17:52:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.056+0000 7ff370d80740 10 client.0 _ll_register_callbacks cb 0x7ff3580044a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:52:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.056+0000 7ff3677fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.057+0000 7ff366ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.057+0000 7ff366ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33452/0 (socket says 172.21.15.69:33452) 2024-08-02T17:52:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.057+0000 7ff366ffd640 1 -- 172.21.15.69:0/4237314712 learned_addr learned my addr 172.21.15.69:0/4237314712 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:52:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.058+0000 7ff366ffd640 1 -- 172.21.15.69:0/4237314712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4d4970 msgr2=0x55b72a4d8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.058+0000 7ff366ffd640 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4d4970 0x55b72a4d8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.058+0000 7ff366ffd640 1 -- 172.21.15.69:0/4237314712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 msgr2=0x55b72a4dcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:52:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.058+0000 7ff366ffd640 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4dcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.058+0000 7ff366ffd640 1 -- 172.21.15.69:0/4237314712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72a4d8d80 con 0x55b72a4ca0d0 2024-08-02T17:52:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.058+0000 7ff3677fe640 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4dcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:52:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.059+0000 7ff366ffd640 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d8200 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff35c048040 tx=0x7ff35c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39576986ce455424 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:52:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.059+0000 7ff346ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:52:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.060+0000 7ff346ffd640 1 -- 172.21.15.69:0/4237314712 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c041030 con 0x55b72a4ca0d0 2024-08-02T17:52:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.060+0000 7ff346ffd640 1 -- 172.21.15.69:0/4237314712 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7ff35c042070 con 0x55b72a4ca0d0 2024-08-02T17:52:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.061+0000 7ff370d80740 1 -- 172.21.15.69:0/4237314712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3600a0240 con 0x55b72a4ca0d0 2024-08-02T17:52:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.062+0000 7ff346ffd640 1 -- 172.21.15.69:0/4237314712 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff35c035040 con 0x55b72a4ca0d0 2024-08-02T17:52:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.065+0000 7ff370d80740 10 client.17127 fetch_fsmap learned FSMap version 26 2024-08-02T17:52:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.065+0000 7ff370d80740 1 -- 172.21.15.69:0/4237314712 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b72a560420 con 0x55b72a4ca0d0 2024-08-02T17:52:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.068+0000 7ff346ffd640 1 -- 172.21.15.69:0/4237314712 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3600a0240 con 0x55b72a4ca0d0 2024-08-02T17:52:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.068+0000 7ff346ffd640 1 -- 172.21.15.69:0/4237314712 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff35c03a040 con 0x55b72a4ca0d0 2024-08-02T17:52:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.069+0000 7ff370d80740 10 client.17127 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:52:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.069+0000 7ff370d80740 10 client.17127 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:52:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.069+0000 7ff370d80740 20 client.17127 populate_metadata read hostname 'smithi069' 2024-08-02T17:52:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.069+0000 7ff370d80740 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b72a560a90 0x55b72a562e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:52:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.069+0000 7ff370d80740 4 client.17127 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:52:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.069+0000 7ff370d80740 1 -- 172.21.15.69:0/4237314712 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b72a435430 con 0x55b72a560a90 2024-08-02T17:52:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.071+0000 7ff3677fe640 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b72a560a90 0x55b72a562e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:52:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.079+0000 7ff3677fe640 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b72a560a90 0x55b72a562e80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59a43d2574c53fe3 server_cookie=dd774e1191733add in_seq=0 out_seq=0 2024-08-02T17:52:51.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.080+0000 7ff346ffd640 10 client.17127 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:51.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:50 smithi077 ceph-mon[37108]: pgmap v195: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.5 MiB/s wr, 368 op/s 2024-08-02T17:52:51.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:50 smithi069 ceph-mon[32746]: pgmap v195: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 7.5 MiB/s wr, 368 op/s 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.169+0000 7ff346ffd640 1 -- 172.21.15.69:0/4237314712 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b72a435430 con 0x55b72a560a90 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.169+0000 7ff346ffd640 10 client.17127 handle_command_reply: tid=0 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.169+0000 7ff370d80740 1 client.17127 shutdown 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff370d80740 1 -- 172.21.15.69:0/4237314712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 msgr2=0x55b72a4d8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff370d80740 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d8200 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff35c048040 tx=0x7ff35c02a420 comp rx=0 tx=0).stop 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff370d80740 1 -- 172.21.15.69:0/4237314712 shutdown_connections 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff370d80740 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b72a4ca0d0 0x55b72a4d8200 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff370d80740 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b72a560a90 0x55b72a562e80 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff370d80740 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b72a4d4970 0x55b72a4d8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff370d80740 1 --2- 172.21.15.69:0/4237314712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b72a4c97b0 0x55b72a4dcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff370d80740 1 -- 172.21.15.69:0/4237314712 >> 172.21.15.69:0/4237314712 conn(0x55b72a4c5d20 msgr2=0x55b72a4c4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.170+0000 7ff346ffd640 0 client.17127 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:52:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.171+0000 7ff370d80740 1 -- 172.21.15.69:0/4237314712 shutdown_connections 2024-08-02T17:52:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.171+0000 7ff370d80740 1 -- 172.21.15.69:0/4237314712 wait complete. 2024-08-02T17:52:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.171+0000 7ff370d80740 20 client.17127 trim_cache size 0 max 16384 2024-08-02T17:52:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:52:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:52:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:52:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 -- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3500a2cf0 msgr2=0x7ff3500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3500a2cf0 0x7ff3500a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55b72a4c6b80 tx=0x7ff358005f90 comp rx=0 tx=0).stop 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 -- 172.21.15.69:0/2339125396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3681066e0 msgr2=0x7ff368109780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3681066e0 0x7ff368109780 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff36000eb10 tx=0x7ff36000c2d0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 -- 172.21.15.69:0/2339125396 shutdown_connections 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3500a2cf0 0x7ff3500a51b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3681066e0 0x7ff368109780 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff368101070 0x7ff368109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 --2- 172.21.15.69:0/2339125396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3681006d0 0x7ff36810d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:52:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 -- 172.21.15.69:0/2339125396 >> 172.21.15.69:0/2339125396 conn(0x7ff3680776d0 msgr2=0x7ff3680fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:52:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 -- 172.21.15.69:0/2339125396 shutdown_connections 2024-08-02T17:52:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:52:51.172+0000 7ff36e028640 1 -- 172.21.15.69:0/2339125396 wait complete. 2024-08-02T17:52:53.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:52 smithi083 ceph-mon[37031]: pgmap v196: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 7.5 MiB/s wr, 376 op/s 2024-08-02T17:52:53.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:52 smithi077 ceph-mon[37108]: pgmap v196: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 7.5 MiB/s wr, 376 op/s 2024-08-02T17:52:53.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:52 smithi069 ceph-mon[32746]: pgmap v196: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 7.5 MiB/s wr, 376 op/s 2024-08-02T17:52:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 345.0 s, 1327.2 tps, lat 75.221 ms stddev 96.322 2024-08-02T17:52:55.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:54 smithi083 ceph-mon[37031]: pgmap v197: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 16 MiB/s wr, 616 op/s 2024-08-02T17:52:55.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:54 smithi077 ceph-mon[37108]: pgmap v197: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 16 MiB/s wr, 616 op/s 2024-08-02T17:52:55.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:54 smithi069 ceph-mon[32746]: pgmap v197: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 16 MiB/s wr, 616 op/s 2024-08-02T17:52:57.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:56 smithi083 ceph-mon[37031]: pgmap v198: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 16 MiB/s wr, 523 op/s 2024-08-02T17:52:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:56 smithi077 ceph-mon[37108]: pgmap v198: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 16 MiB/s wr, 523 op/s 2024-08-02T17:52:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:56 smithi069 ceph-mon[32746]: pgmap v198: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 16 MiB/s wr, 523 op/s 2024-08-02T17:52:59.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:52:58 smithi083 ceph-mon[37031]: pgmap v199: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 25 MiB/s wr, 751 op/s 2024-08-02T17:52:59.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:52:58 smithi077 ceph-mon[37108]: pgmap v199: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 25 MiB/s wr, 751 op/s 2024-08-02T17:52:59.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:52:58 smithi069 ceph-mon[32746]: pgmap v199: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 25 MiB/s wr, 751 op/s 2024-08-02T17:52:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 350.0 s, 1335.8 tps, lat 75.099 ms stddev 88.924 2024-08-02T17:53:01.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:00 smithi083 ceph-mon[37031]: pgmap v200: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 19 MiB/s wr, 545 op/s 2024-08-02T17:53:01.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:00 smithi077 ceph-mon[37108]: pgmap v200: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 19 MiB/s wr, 545 op/s 2024-08-02T17:53:01.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:00 smithi069 ceph-mon[32746]: pgmap v200: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 19 MiB/s wr, 545 op/s 2024-08-02T17:53:03.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:02 smithi083 ceph-mon[37031]: pgmap v201: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 19 MiB/s wr, 522 op/s 2024-08-02T17:53:03.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:02 smithi077 ceph-mon[37108]: pgmap v201: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 19 MiB/s wr, 522 op/s 2024-08-02T17:53:03.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:02 smithi069 ceph-mon[32746]: pgmap v201: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 19 MiB/s wr, 522 op/s 2024-08-02T17:53:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.299+0000 7f2b45846640 1 -- 172.21.15.69:0/251719210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b401007d0 msgr2=0x7f2b40100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.299+0000 7f2b45846640 1 --2- 172.21.15.69:0/251719210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b401007d0 0x7f2b40100bb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009930 tx=0x7f2b3802a510 comp rx=0 tx=0).stop 2024-08-02T17:53:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.300+0000 7f2b45846640 1 -- 172.21.15.69:0/251719210 shutdown_connections 2024-08-02T17:53:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.300+0000 7f2b45846640 1 --2- 172.21.15.69:0/251719210 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b401010f0 0x7f2b401055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.300+0000 7f2b45846640 1 --2- 172.21.15.69:0/251719210 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b40105c00 0x7f2b401060b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.300+0000 7f2b45846640 1 --2- 172.21.15.69:0/251719210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b401007d0 0x7f2b40100bb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.300+0000 7f2b45846640 1 -- 172.21.15.69:0/251719210 >> 172.21.15.69:0/251719210 conn(0x7f2b400fc450 msgr2=0x7f2b400fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.301+0000 7f2b45846640 1 -- 172.21.15.69:0/251719210 shutdown_connections 2024-08-02T17:53:04.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.303+0000 7f2b45846640 1 -- 172.21.15.69:0/251719210 wait complete. 2024-08-02T17:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.303+0000 7f2b45846640 1 Processor -- start 2024-08-02T17:53:04.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.304+0000 7f2b45846640 1 -- start start 2024-08-02T17:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.304+0000 7f2b45846640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b401007d0 0x7f2b401148c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.304+0000 7f2b45846640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b401010f0 0x7f2b40114e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.304+0000 7f2b45846640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b40105c00 0x7f2b4010db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.304+0000 7f2b45846640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401182c0 con 0x7f2b401010f0 2024-08-02T17:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.304+0000 7f2b45846640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40118140 con 0x7f2b40105c00 2024-08-02T17:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.304+0000 7f2b45846640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b40118440 con 0x7f2b401007d0 2024-08-02T17:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.305+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b401010f0 0x7f2b40114e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.305+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b401010f0 0x7f2b40114e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53150/0 (socket says 172.21.15.69:53150) 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.305+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/4191587532 learned_addr learned my addr 172.21.15.69:0/4191587532 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.305+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/4191587532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b401007d0 msgr2=0x7f2b401148c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.305+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b401007d0 0x7f2b401148c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.305+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/4191587532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b40105c00 msgr2=0x7f2b4010db20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.305+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b40105c00 0x7f2b4010db20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.305+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/4191587532 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4010e3e0 con 0x7f2b401010f0 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.306+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b401010f0 0x7f2b40114e00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000ba50 tx=0x7f2b3000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.306+0000 7f2b44844640 1 -- 172.21.15.69:0/4191587532 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30019070 con 0x7f2b401010f0 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.306+0000 7f2b44844640 1 -- 172.21.15.69:0/4191587532 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3000cbe0 con 0x7f2b401010f0 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.306+0000 7f2b44844640 1 -- 172.21.15.69:0/4191587532 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b300076f0 con 0x7f2b401010f0 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.308+0000 7f2b45846640 1 -- 172.21.15.69:0/4191587532 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4010e670 con 0x7f2b401010f0 2024-08-02T17:53:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.308+0000 7f2b45846640 1 -- 172.21.15.69:0/4191587532 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b401125c0 con 0x7f2b401010f0 2024-08-02T17:53:04.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.316+0000 7f2b44844640 1 -- 172.21.15.69:0/4191587532 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2b30014a40 con 0x7f2b401010f0 2024-08-02T17:53:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.319+0000 7f2b44844640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b2c0c4f10 0x7f2b2c0c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.319+0000 7f2b44844640 1 -- 172.21.15.69:0/4191587532 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2b300cec70 con 0x7f2b401010f0 2024-08-02T17:53:04.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.320+0000 7f2b3effd640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b2c0c4f10 0x7f2b2c0c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.320+0000 7f2b485a5740 1 Processor -- start 2024-08-02T17:53:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.320+0000 7f2b485a5740 1 -- start start 2024-08-02T17:53:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.321+0000 7f2b3effd640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b2c0c4f10 0x7f2b2c0c73d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b380096c0 tx=0x7f2b38000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.321+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1aab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.321+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.321+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1aab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.321+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1aab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39862/0 (socket says 172.21.15.69:39862) 2024-08-02T17:53:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.322+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.322+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:53166/0 (socket says 172.21.15.69:53166) 2024-08-02T17:53:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.322+0000 7f2b3effd640 1 -- 172.21.15.69:0/3936120529 learned_addr learned my addr 172.21.15.69:0/3936120529 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.322+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.322+0000 7f2b485a5740 1 -- 172.21.15.69:0/3936120529 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b5ae10e9b0 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.322+0000 7f2b485a5740 1 -- 172.21.15.69:0/3936120529 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5ae0b6800 con 0x55b5ae1b5970 2024-08-02T17:53:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.322+0000 7f2b3f7fe640 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.322+0000 7f2b485a5740 1 -- 172.21.15.69:0/3936120529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5ae10a970 con 0x55b5ae1aa7b0 2024-08-02T17:53:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.324+0000 7f2b3effd640 1 -- 172.21.15.69:0/3936120529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 msgr2=0x55b5ae1b7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.324+0000 7f2b3effd640 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.324+0000 7f2b3effd640 1 -- 172.21.15.69:0/3936120529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 msgr2=0x55b5ae1b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.324+0000 7f2b3effd640 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.324+0000 7f2b3effd640 1 -- 172.21.15.69:0/3936120529 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ade22210 con 0x55b5ae1aa7b0 2024-08-02T17:53:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.325+0000 7f2b3effd640 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1aab90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b38032000 tx=0x7f2b3803c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33d609f502f7f84a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:04.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.325+0000 7f2b3f7fe640 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:53:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.327+0000 7f2b35ffb640 1 -- 172.21.15.69:0/3936120529 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b38042070 con 0x55b5ae1aa7b0 2024-08-02T17:53:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.327+0000 7f2b35ffb640 1 -- 172.21.15.69:0/3936120529 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3804e070 con 0x55b5ae1aa7b0 2024-08-02T17:53:04.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.327+0000 7f2b35ffb640 1 -- 172.21.15.69:0/3936120529 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b380497f0 con 0x55b5ae1aa7b0 2024-08-02T17:53:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.328+0000 7f2b485a5740 1 -- 172.21.15.69:0/3936120529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 msgr2=0x55b5ae1aab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.328+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1aab90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b38032000 tx=0x7f2b3803c000 comp rx=0 tx=0).stop 2024-08-02T17:53:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.329+0000 7f2b485a5740 1 -- 172.21.15.69:0/3936120529 shutdown_connections 2024-08-02T17:53:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.329+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.329+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.330+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3936120529 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1aab90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.330+0000 7f2b485a5740 1 -- 172.21.15.69:0/3936120529 >> 172.21.15.69:0/3936120529 conn(0x55b5ae1a6d20 msgr2=0x55b5ae1a7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:04.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.336+0000 7f2b485a5740 1 -- 172.21.15.69:0/3936120529 shutdown_connections 2024-08-02T17:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.336+0000 7f2b485a5740 1 -- 172.21.15.69:0/3936120529 wait complete. 2024-08-02T17:53:04.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 1 -- start start 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 1 Processor -- start 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1b2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b5ae1b8540 con 0x55b5ae1aa7b0 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b5ae0ca500 con 0x55b5ae1b5970 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b5ae1b86c0 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.337+0000 7f2b485a5740 10 client.0 _ll_register_callbacks cb 0x55b5ade21a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.338+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.338+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39872/0 (socket says 172.21.15.69:39872) 2024-08-02T17:53:04.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.338+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/3057682727 learned_addr learned my addr 172.21.15.69:0/3057682727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.339+0000 7f2b3effd640 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.339+0000 7f2b3f7fe640 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.339+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/3057682727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 msgr2=0x55b5ae1b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.339+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.339+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/3057682727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1aa7b0 msgr2=0x55b5ae1b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.339+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.339+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/3057682727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5adeee620 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.339+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b2ec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2b300972e0 tx=0x7f2b30004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c8b3eace180575 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.341+0000 7f2b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:04.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.341+0000 7f2b1effd640 1 -- 172.21.15.69:0/3057682727 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b300cc430 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.341+0000 7f2b1effd640 1 -- 172.21.15.69:0/3057682727 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2b300a3860 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.341+0000 7f2b485a5740 1 -- 172.21.15.69:0/3057682727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b3804e070 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.342+0000 7f2b1effd640 1 -- 172.21.15.69:0/3057682727 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b300b7070 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.342+0000 7f2b485a5740 10 client.26545 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:04.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.342+0000 7f2b485a5740 1 -- 172.21.15.69:0/3057682727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5ae1c4280 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.344+0000 7f2b1effd640 1 -- 172.21.15.69:0/3057682727 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b3804e070 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.345+0000 7f2b1effd640 1 -- 172.21.15.69:0/3057682727 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2b300b9b90 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.345+0000 7f2b485a5740 10 client.26545 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.345+0000 7f2b485a5740 10 client.26545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.345+0000 7f2b485a5740 20 client.26545 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.345+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5ae1c5740 0x55b5ae1c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.345+0000 7f2b485a5740 4 client.26545 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.345+0000 7f2b3effd640 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5ae1c5740 0x55b5ae1c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.345+0000 7f2b485a5740 1 -- 172.21.15.69:0/3057682727 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5ae11c0e0 con 0x55b5ae1c5740 2024-08-02T17:53:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.347+0000 7f2b1effd640 10 client.26545 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:04.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.347+0000 7f2b3effd640 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5ae1c5740 0x55b5ae1c7b30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=524a3003c0064ed server_cookie=7c60ebaf8ca0b70e in_seq=0 out_seq=0 2024-08-02T17:53:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.348+0000 7f2b1effd640 1 -- 172.21.15.69:0/3057682727 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b5ae11c0e0 con 0x55b5ae1c5740 2024-08-02T17:53:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.348+0000 7f2b1effd640 10 client.26545 handle_command_reply: tid=0 2024-08-02T17:53:04.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.348+0000 7f2b485a5740 1 client.26545 shutdown 2024-08-02T17:53:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.351+0000 7f2b485a5740 1 -- 172.21.15.69:0/3057682727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 msgr2=0x55b5ae1b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.351+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b2ec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2b300972e0 tx=0x7f2b30004b50 comp rx=0 tx=0).stop 2024-08-02T17:53:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.352+0000 7f2b485a5740 1 -- 172.21.15.69:0/3057682727 shutdown_connections 2024-08-02T17:53:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.352+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5ae1c5740 0x55b5ae1c7b30 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.355 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.353+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.353+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1b2ec0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.353+0000 7f2b485a5740 1 --2- 172.21.15.69:0/3057682727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.353+0000 7f2b485a5740 1 -- 172.21.15.69:0/3057682727 >> 172.21.15.69:0/3057682727 conn(0x55b5ae1a6d20 msgr2=0x55b5ae1aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:04.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.353+0000 7f2b1effd640 0 client.26545 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.355+0000 7f2b485a5740 1 -- 172.21.15.69:0/3057682727 shutdown_connections 2024-08-02T17:53:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.355+0000 7f2b485a5740 1 -- 172.21.15.69:0/3057682727 wait complete. 2024-08-02T17:53:04.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.355+0000 7f2b485a5740 20 client.26545 trim_cache size 0 max 16384 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.377+0000 7f2b485a5740 1 Processor -- start 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.377+0000 7f2b485a5740 1 -- start start 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.377+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.377+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.377+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1b5970 0x55b5ae1ac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.377+0000 7f2b485a5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401a93c0 con 0x55b5ae1b5970 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.377+0000 7f2b485a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401a6c90 con 0x55b5ae1aa7b0 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.377+0000 7f2b485a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401a4ae0 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39880/0 (socket says 172.21.15.69:39880) 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/1751886174 learned_addr learned my addr 172.21.15.69:0/1751886174 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3effd640 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3f7fe640 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1b5970 0x55b5ae1ac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/1751886174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1aa7b0 msgr2=0x55b5ae1cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/1751886174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1b5970 msgr2=0x55b5ae1ac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1b5970 0x55b5ae1ac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/1751886174 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ae1b3d00 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1cd8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2b3009ab80 tx=0x7f2b300bb0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85577a6a4ef13d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b357fa640 1 -- 172.21.15.69:0/1751886174 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b300cc430 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b357fa640 1 -- 172.21.15.69:0/1751886174 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b30099240 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.378+0000 7f2b357fa640 1 -- 172.21.15.69:0/1751886174 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b300b9ac0 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 -- 172.21.15.69:0/1751886174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 msgr2=0x55b5ae1cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1cd8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2b3009ab80 tx=0x7f2b300bb0b0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 -- 172.21.15.69:0/1751886174 shutdown_connections 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1b5970 0x55b5ae1ac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1cd8f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 --2- 172.21.15.69:0/1751886174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 -- 172.21.15.69:0/1751886174 >> 172.21.15.69:0/1751886174 conn(0x55b5ae1a6d20 msgr2=0x55b5ae1b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 -- 172.21.15.69:0/1751886174 shutdown_connections 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.379+0000 7f2b485a5740 1 -- 172.21.15.69:0/1751886174 wait complete. 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b485a5740 1 -- start start 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b485a5740 1 Processor -- start 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1be080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b485a5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b485a5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401a5150 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b485a5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401a65c0 con 0x55b5ae1b5970 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b485a5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b401a5820 con 0x55b5ae1aa7b0 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.380+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b485a5740 10 client.0 _ll_register_callbacks cb 0x7f2b3009b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1bdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39892/0 (socket says 172.21.15.69:39892) 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3effd640 1 -- 172.21.15.69:0/903365010 learned_addr learned my addr 172.21.15.69:0/903365010 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1be080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3f7fe640 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/903365010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 msgr2=0x55b5ae1bdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1bdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/903365010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 msgr2=0x55b5ae1af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3e7fc640 1 -- 172.21.15.69:0/903365010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ae1af9e0 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.381+0000 7f2b3e7fc640 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1be080 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2b300a3cb0 tx=0x7f2b3009aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e66d4dfe8aa96c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.382+0000 7f2b1effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.382+0000 7f2b1effd640 1 -- 172.21.15.69:0/903365010 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3007d020 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.382+0000 7f2b1effd640 1 -- 172.21.15.69:0/903365010 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9480+0+0 (secure 0 0 0) 0x7f2b3009a4e0 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.382+0000 7f2b485a5740 1 -- 172.21.15.69:0/903365010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5ae116430 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.382+0000 7f2b1effd640 1 -- 172.21.15.69:0/903365010 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b300cc430 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.384+0000 7f2b485a5740 10 client.17142 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.384+0000 7f2b485a5740 1 -- 172.21.15.69:0/903365010 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b5ae22feb0 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b1effd640 1 -- 172.21.15.69:0/903365010 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b30099de0 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b1effd640 1 -- 172.21.15.69:0/903365010 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2b300aa070 con 0x55b5ae1ab0d0 2024-08-02T17:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b485a5740 10 client.17142 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b485a5740 10 client.17142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b485a5740 20 client.17142 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b485a5740 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5ae230330 0x55b5ae232720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b485a5740 4 client.17142 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:53:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b485a5740 1 -- 172.21.15.69:0/903365010 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5ae116430 con 0x55b5ae230330 2024-08-02T17:53:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.385+0000 7f2b3effd640 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5ae230330 0x55b5ae232720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.386+0000 7f2b3effd640 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5ae230330 0x55b5ae232720 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7ba393ac74d142 server_cookie=26bc8d2d2ebf810a in_seq=0 out_seq=0 2024-08-02T17:53:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.386+0000 7f2b1effd640 10 client.17142 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.418+0000 7f2b1effd640 1 -- 172.21.15.69:0/903365010 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b5ae116430 con 0x55b5ae230330 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.418+0000 7f2b1effd640 10 client.17142 handle_command_reply: tid=0 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.418+0000 7f2b485a5740 1 client.17142 shutdown 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.418+0000 7f2b485a5740 1 -- 172.21.15.69:0/903365010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 msgr2=0x55b5ae1be080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.418+0000 7f2b485a5740 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1be080 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2b300a3cb0 tx=0x7f2b3009aaf0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.418+0000 7f2b485a5740 1 -- 172.21.15.69:0/903365010 shutdown_connections 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.418+0000 7f2b485a5740 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b5ae1ab0d0 0x55b5ae1be080 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.419+0000 7f2b485a5740 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b5ae230330 0x55b5ae232720 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.419+0000 7f2b1effd640 0 client.17142 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:04.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.419+0000 7f2b485a5740 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b5ae1b5970 0x55b5ae1af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.419+0000 7f2b485a5740 1 --2- 172.21.15.69:0/903365010 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b5ae1aa7b0 0x55b5ae1bdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.419+0000 7f2b485a5740 1 -- 172.21.15.69:0/903365010 >> 172.21.15.69:0/903365010 conn(0x55b5ae1a6d20 msgr2=0x55b5ae1ac210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.419+0000 7f2b485a5740 1 -- 172.21.15.69:0/903365010 shutdown_connections 2024-08-02T17:53:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.419+0000 7f2b485a5740 1 -- 172.21.15.69:0/903365010 wait complete. 2024-08-02T17:53:04.422 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.419+0000 7f2b485a5740 20 client.17142 trim_cache size 0 max 16384 2024-08-02T17:53:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:53:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:53:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:53:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:53:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 -- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b2c0c4f10 msgr2=0x7f2b2c0c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b2c0c4f10 0x7f2b2c0c73d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2b380096c0 tx=0x7f2b38000bc0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 -- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b401010f0 msgr2=0x7f2b40114e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b401010f0 0x7f2b40114e00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000ba50 tx=0x7f2b3000bf10 comp rx=0 tx=0).stop 2024-08-02T17:53:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 -- 172.21.15.69:0/4191587532 shutdown_connections 2024-08-02T17:53:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2b401010f0 0x7f2b40114e00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b2c0c4f10 0x7f2b2c0c73d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2b40105c00 0x7f2b4010db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 --2- 172.21.15.69:0/4191587532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b401007d0 0x7f2b401148c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 -- 172.21.15.69:0/4191587532 >> 172.21.15.69:0/4191587532 conn(0x7f2b400fc450 msgr2=0x7f2b40103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 -- 172.21.15.69:0/4191587532 shutdown_connections 2024-08-02T17:53:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:04.420+0000 7f2b45846640 1 -- 172.21.15.69:0/4191587532 wait complete. 2024-08-02T17:53:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 355.0 s, 645.2 tps, lat 148.763 ms stddev 263.302 2024-08-02T17:53:05.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:04 smithi083 ceph-mon[37031]: pgmap v202: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 29 MiB/s wr, 1.15k op/s 2024-08-02T17:53:05.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:04 smithi077 ceph-mon[37108]: pgmap v202: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 29 MiB/s wr, 1.15k op/s 2024-08-02T17:53:05.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:04 smithi069 ceph-mon[32746]: pgmap v202: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 29 MiB/s wr, 1.15k op/s 2024-08-02T17:53:06.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:05 smithi083 ceph-mon[37031]: osdmap e96: 12 total, 12 up, 12 in 2024-08-02T17:53:06.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:05 smithi077 ceph-mon[37108]: osdmap e96: 12 total, 12 up, 12 in 2024-08-02T17:53:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:05 smithi069 ceph-mon[32746]: osdmap e96: 12 total, 12 up, 12 in 2024-08-02T17:53:07.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:06 smithi083 ceph-mon[37031]: pgmap v204: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-08-02T17:53:07.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:06 smithi077 ceph-mon[37108]: pgmap v204: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-08-02T17:53:07.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:06 smithi069 ceph-mon[32746]: pgmap v204: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 25 MiB/s wr, 1.09k op/s 2024-08-02T17:53:09.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:08 smithi083 ceph-mon[37031]: pgmap v205: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 21 MiB/s wr, 1.28k op/s 2024-08-02T17:53:09.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:08 smithi077 ceph-mon[37108]: pgmap v205: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 21 MiB/s wr, 1.28k op/s 2024-08-02T17:53:09.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:08 smithi069 ceph-mon[32746]: pgmap v205: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 21 MiB/s wr, 1.28k op/s 2024-08-02T17:53:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 360.0 s, 1350.4 tps, lat 77.125 ms stddev 110.883 2024-08-02T17:53:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:10 smithi083 ceph-mon[37031]: pgmap v206: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 21 MiB/s wr, 1.29k op/s 2024-08-02T17:53:11.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:10 smithi077 ceph-mon[37108]: pgmap v206: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 21 MiB/s wr, 1.29k op/s 2024-08-02T17:53:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:10 smithi069 ceph-mon[32746]: pgmap v206: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 21 MiB/s wr, 1.29k op/s 2024-08-02T17:53:13.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:12 smithi083 ceph-mon[37031]: pgmap v207: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-08-02T17:53:13.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:12 smithi077 ceph-mon[37108]: pgmap v207: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-08-02T17:53:13.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:12 smithi069 ceph-mon[32746]: pgmap v207: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-08-02T17:53:14.544 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:53:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 365.0 s, 724.4 tps, lat 134.252 ms stddev 219.129 2024-08-02T17:53:15.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:14 smithi083 ceph-mon[37031]: pgmap v208: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-02T17:53:15.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:14 smithi083 ceph-mon[37031]: osdmap e97: 12 total, 12 up, 12 in 2024-08-02T17:53:15.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:14 smithi077 ceph-mon[37108]: pgmap v208: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-02T17:53:15.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:14 smithi077 ceph-mon[37108]: osdmap e97: 12 total, 12 up, 12 in 2024-08-02T17:53:15.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:14 smithi069 ceph-mon[32746]: pgmap v208: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-02T17:53:15.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:14 smithi069 ceph-mon[32746]: osdmap e97: 12 total, 12 up, 12 in 2024-08-02T17:53:16.077 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:53:16.802 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:16 smithi069 ceph-mon[32746]: pgmap v210: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-02T17:53:16.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.840+0000 7fc51bdee640 1 -- 172.21.15.69:0/1615416315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc514109180 msgr2=0x7fc51410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:16.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.840+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1615416315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc514109180 0x7fc51410ff80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc51001a910 tx=0x7fc51003b420 comp rx=0 tx=0).stop 2024-08-02T17:53:16.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.840+0000 7fc51bdee640 1 -- 172.21.15.69:0/1615416315 shutdown_connections 2024-08-02T17:53:16.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.840+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1615416315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc51411ae70 0x7fc51411d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.840+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1615416315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514109560 0x7fc5141099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.840+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1615416315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc514109180 0x7fc51410ff80 secure :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc51001a910 tx=0x7fc51003b420 comp rx=0 tx=0).stop 2024-08-02T17:53:16.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.840+0000 7fc51bdee640 1 -- 172.21.15.69:0/1615416315 >> 172.21.15.69:0/1615416315 conn(0x7fc51406de70 msgr2=0x7fc51406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:16.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.844+0000 7fc51bdee640 1 -- 172.21.15.69:0/1615416315 shutdown_connections 2024-08-02T17:53:16.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.848+0000 7fc51bdee640 1 -- 172.21.15.69:0/1615416315 wait complete. 2024-08-02T17:53:16.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.848+0000 7fc51bdee640 1 Processor -- start 2024-08-02T17:53:16.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.848+0000 7fc51bdee640 1 -- start start 2024-08-02T17:53:16.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc51bdee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc514109560 0x7fc51411a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc51bdee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc51411ae70 0x7fc514116850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc514109560 0x7fc51411a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc514109560 0x7fc51411a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39758/0 (socket says 172.21.15.69:39758) 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc519b8a640 1 -- 172.21.15.69:0/1468290150 learned_addr learned my addr 172.21.15.69:0/1468290150 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514116d90 0x7fc514117220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc514113030 con 0x7fc51411ae70 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc514112eb0 con 0x7fc514109560 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.849+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141131b0 con 0x7fc514116d90 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.850+0000 7fc519b8a640 1 -- 172.21.15.69:0/1468290150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514116d90 msgr2=0x7fc514117220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.850+0000 7fc519b8a640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514116d90 0x7fc514117220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.850+0000 7fc519b8a640 1 -- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc51411ae70 msgr2=0x7fc514116850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:16.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.850+0000 7fc519b8a640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc51411ae70 0x7fc514116850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.850+0000 7fc519b8a640 1 -- 172.21.15.69:0/1468290150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5141ab090 con 0x7fc514109560 2024-08-02T17:53:16.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.850+0000 7fc519b8a640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc514109560 0x7fc51411a7b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc510000c00 tx=0x7fc510004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:16.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.851+0000 7fc50affd640 1 -- 172.21.15.69:0/1468290150 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510046070 con 0x7fc514109560 2024-08-02T17:53:16.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.851+0000 7fc50affd640 1 -- 172.21.15.69:0/1468290150 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc510004230 con 0x7fc514109560 2024-08-02T17:53:16.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.851+0000 7fc50affd640 1 -- 172.21.15.69:0/1468290150 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510003850 con 0x7fc514109560 2024-08-02T17:53:16.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.855+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5141ab320 con 0x7fc514109560 2024-08-02T17:53:16.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.855+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5141ab6c0 con 0x7fc514109560 2024-08-02T17:53:16.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.858+0000 7fc50affd640 1 -- 172.21.15.69:0/1468290150 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc510005500 con 0x7fc514109560 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.860+0000 7fc51e1ec740 1 Processor -- start 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.860+0000 7fc51e1ec740 1 -- start start 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.860+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 0x556eaffa45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.860+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4b30 0x556eaffaeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffaf3f0 0x556eaffb17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556eaff09980 con 0x556eaffa4210 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556eafea1a70 con 0x556eaffa4b30 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556eaff05940 con 0x556eaffaf3f0 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4b30 0x556eaffaeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4b30 0x556eaffaeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39766/0 (socket says 172.21.15.69:39766) 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 -- 172.21.15.69:0/2175791 learned_addr learned my addr 172.21.15.69:0/2175791 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:16.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 -- 172.21.15.69:0/2175791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffaf3f0 msgr2=0x556eaffb17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:16.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 --2- 172.21.15.69:0/2175791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffaf3f0 0x556eaffb17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 -- 172.21.15.69:0/2175791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 msgr2=0x556eaffa45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:16.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 --2- 172.21.15.69:0/2175791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 0x556eaffa45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 -- 172.21.15.69:0/2175791 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaff046e0 con 0x556eaffa4b30 2024-08-02T17:53:16.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.861+0000 7fc519b8a640 1 --2- 172.21.15.69:0/2175791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4b30 0x556eaffaeeb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc5100fde50 tx=0x7fc510104c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c1d0add9cf633ab server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:16.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.862+0000 7fc50affd640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc5000a2f00 0x7fc5000a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.862+0000 7fc50affd640 1 -- 172.21.15.69:0/1468290150 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7fc510100c10 con 0x7fc514109560 2024-08-02T17:53:16.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.863+0000 7fc4fffff640 1 -- 172.21.15.69:0/2175791 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510110020 con 0x556eaffa4b30 2024-08-02T17:53:16.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.863+0000 7fc4fffff640 1 -- 172.21.15.69:0/2175791 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51010c400 con 0x556eaffa4b30 2024-08-02T17:53:16.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.863+0000 7fc51a38b640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc5000a2f00 0x7fc5000a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.864+0000 7fc4fffff640 1 -- 172.21.15.69:0/2175791 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510105040 con 0x556eaffa4b30 2024-08-02T17:53:16.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.864+0000 7fc51a38b640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc5000a2f00 0x7fc5000a53c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005fd0 tx=0x7fc50c009880 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:16.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.866+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2175791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4b30 msgr2=0x556eaffaeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:16.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.866+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/2175791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4b30 0x556eaffaeeb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc5100fde50 tx=0x7fc510104c30 comp rx=0 tx=0).stop 2024-08-02T17:53:16.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.867+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2175791 shutdown_connections 2024-08-02T17:53:16.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.867+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/2175791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffaf3f0 0x556eaffb17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.867+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/2175791 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4b30 0x556eaffaeeb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.867+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/2175791 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 0x556eaffa45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.867+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2175791 >> 172.21.15.69:0/2175791 conn(0x556eaffa07a0 msgr2=0x556eaffa0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:16.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.867+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2175791 shutdown_connections 2024-08-02T17:53:16.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.867+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2175791 wait complete. 2024-08-02T17:53:16.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.868+0000 7fc51e1ec740 1 -- start start 2024-08-02T17:53:16.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.868+0000 7fc51e1ec740 1 Processor -- start 2024-08-02T17:53:16.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.869+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.869+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.869+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffacdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.869+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556eaffb1d80 con 0x556eaffaf3f0 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.869+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556eafec4aa0 con 0x556eaffa4210 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.869+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556eaffb1f00 con 0x556eaffa4b30 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.869+0000 7fc51e1ec740 10 client.0 _ll_register_callbacks cb 0x556eafeb3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.870+0000 7fc519389640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.870+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.870+0000 7fc519389640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:60060/0 (socket says 172.21.15.69:60060) 2024-08-02T17:53:16.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.870+0000 7fc519389640 1 -- 172.21.15.69:0/1462671611 learned_addr learned my addr 172.21.15.69:0/1462671611 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:16.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.871+0000 7fc51a38b640 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffacdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.871+0000 7fc519389640 1 -- 172.21.15.69:0/1462671611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 msgr2=0x556eaffac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:16.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.871+0000 7fc519389640 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.871+0000 7fc519389640 1 -- 172.21.15.69:0/1462671611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 msgr2=0x556eaffacdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:16.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.871+0000 7fc519389640 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffacdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.871+0000 7fc519389640 1 -- 172.21.15.69:0/1462671611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eafea8650 con 0x556eaffa4b30 2024-08-02T17:53:16.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.871+0000 7fc51a38b640 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffacdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:53:16.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.874+0000 7fc519389640 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffac870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc50400e5d0 tx=0x7fc50400ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63da440d8dec432 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:16.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.874+0000 7fc4fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:16.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.874+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/1462671611 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504005220 con 0x556eaffa4b30 2024-08-02T17:53:16.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.874+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/1462671611 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7fc5040053c0 con 0x556eaffa4b30 2024-08-02T17:53:16.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.874+0000 7fc51e1ec740 1 -- 172.21.15.69:0/1462671611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc51010c400 con 0x556eaffa4b30 2024-08-02T17:53:16.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.875+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/1462671611 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5040055c0 con 0x556eaffa4b30 2024-08-02T17:53:16.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.877+0000 7fc51e1ec740 10 client.26557 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:16.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.877+0000 7fc51e1ec740 1 -- 172.21.15.69:0/1462671611 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556eafc7b3c0 con 0x556eaffa4b30 2024-08-02T17:53:16.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.879+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/1462671611 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc51010c400 con 0x556eaffa4b30 2024-08-02T17:53:16.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.879+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/1462671611 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc504010ce0 con 0x556eaffa4b30 2024-08-02T17:53:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.881+0000 7fc51e1ec740 10 client.26557 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.881+0000 7fc51e1ec740 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.881+0000 7fc51e1ec740 20 client.26557 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.881+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556eaffbec40 0x556eaffc1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.881+0000 7fc51e1ec740 4 client.26557 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:16.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.881+0000 7fc51e1ec740 1 -- 172.21.15.69:0/1462671611 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eaff16ef0 con 0x556eaffbec40 2024-08-02T17:53:16.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.883+0000 7fc519b8a640 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556eaffbec40 0x556eaffc1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.883+0000 7fc519b8a640 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556eaffbec40 0x556eaffc1030 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4df7a1f2c1753e2b server_cookie=4dd593beca58c8a6 in_seq=0 out_seq=0 2024-08-02T17:53:16.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.883+0000 7fc4fd7fa640 10 client.26557 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:16.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.884+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/1462671611 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556eaff16ef0 con 0x556eaffbec40 2024-08-02T17:53:16.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.884+0000 7fc4fd7fa640 10 client.26557 handle_command_reply: tid=0 2024-08-02T17:53:16.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.884+0000 7fc51e1ec740 1 client.26557 shutdown 2024-08-02T17:53:16.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc51e1ec740 1 -- 172.21.15.69:0/1462671611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 msgr2=0x556eaffac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:16.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffac870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc50400e5d0 tx=0x7fc50400ea90 comp rx=0 tx=0).stop 2024-08-02T17:53:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc51e1ec740 1 -- 172.21.15.69:0/1462671611 shutdown_connections 2024-08-02T17:53:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffacdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556eaffbec40 0x556eaffc1030 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffac870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/1462671611 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc51e1ec740 1 -- 172.21.15.69:0/1462671611 >> 172.21.15.69:0/1462671611 conn(0x556eaffa07a0 msgr2=0x556eaffa64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.885+0000 7fc4fd7fa640 0 client.26557 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:16.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.886+0000 7fc51e1ec740 1 -- 172.21.15.69:0/1462671611 shutdown_connections 2024-08-02T17:53:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.887+0000 7fc51e1ec740 1 -- 172.21.15.69:0/1462671611 wait complete. 2024-08-02T17:53:16.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.887+0000 7fc51e1ec740 20 client.26557 trim_cache size 0 max 16384 2024-08-02T17:53:16.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc51e1ec740 1 Processor -- start 2024-08-02T17:53:16.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc51e1ec740 1 -- start start 2024-08-02T17:53:16.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 0x556eaffa7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffa6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffaf3f0 0x556eaffc6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141b8460 con 0x556eaffa4210 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141b5d30 con 0x556eaffaf3f0 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc51e1ec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141b3b80 con 0x556eaffa4b30 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 0x556eaffa7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 0x556eaffa7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47866/0 (socket says 172.21.15.69:47866) 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.918+0000 7fc519b8a640 1 -- 172.21.15.69:0/2923909323 learned_addr learned my addr 172.21.15.69:0/2923909323 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc519389640 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffa6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc51a38b640 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffaf3f0 0x556eaffc6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc519389640 1 -- 172.21.15.69:0/2923909323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffaf3f0 msgr2=0x556eaffc6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc519389640 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffaf3f0 0x556eaffc6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc519389640 1 -- 172.21.15.69:0/2923909323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 msgr2=0x556eaffa7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc519389640 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 0x556eaffa7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc519389640 1 -- 172.21.15.69:0/2923909323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eaffbd890 con 0x556eaffa4b30 2024-08-02T17:53:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc519389640 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffa6370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x556eaffad8e0 tx=0x7fc504022990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e450d6975d3108fa server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc4ff7fe640 1 -- 172.21.15.69:0/2923909323 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504022d30 con 0x556eaffa4b30 2024-08-02T17:53:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc4ff7fe640 1 -- 172.21.15.69:0/2923909323 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc504004550 con 0x556eaffa4b30 2024-08-02T17:53:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.919+0000 7fc4ff7fe640 1 -- 172.21.15.69:0/2923909323 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504010c00 con 0x556eaffa4b30 2024-08-02T17:53:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2923909323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 msgr2=0x556eaffa6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffa6370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x556eaffad8e0 tx=0x7fc504022990 comp rx=0 tx=0).stop 2024-08-02T17:53:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2923909323 shutdown_connections 2024-08-02T17:53:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffaf3f0 0x556eaffc6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffa6370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/2923909323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffa4210 0x556eaffa7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2923909323 >> 172.21.15.69:0/2923909323 conn(0x556eaffa07a0 msgr2=0x556eaffa5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2923909323 shutdown_connections 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.920+0000 7fc51e1ec740 1 -- 172.21.15.69:0/2923909323 wait complete. 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.921+0000 7fc51e1ec740 1 -- start start 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.921+0000 7fc51e1ec740 1 Processor -- start 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.921+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffb7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.921+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffb2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.921+0000 7fc51e1ec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519389640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffb2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffb7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffb7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39804/0 (socket says 172.21.15.69:39804) 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc51a38b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc51a38b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffb3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47870/0 (socket says 172.21.15.69:47870) 2024-08-02T17:53:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519389640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffb2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:60070/0 (socket says 172.21.15.69:60070) 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 -- 172.21.15.69:0/4294225676 learned_addr learned my addr 172.21.15.69:0/4294225676 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141b41f0 con 0x556eaffaf3f0 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141b5660 con 0x556eaffa4210 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc5141b48c0 con 0x556eaffa4b30 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 -- 172.21.15.69:0/4294225676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 msgr2=0x556eaffb2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffb2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 -- 172.21.15.69:0/4294225676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 msgr2=0x556eaffb3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffb3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 -- 172.21.15.69:0/4294225676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eaffb3970 con 0x556eaffa4210 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc51e1ec740 10 client.0 _ll_register_callbacks cb 0x7fc5040095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc519b8a640 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffb7870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc510005ab0 tx=0x7fc51001a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4808ad03430bdc5a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc4fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc504004550 con 0x556eaffa4210 2024-08-02T17:53:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.922+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/4294225676 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510110040 con 0x556eaffa4210 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/4294225676 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7fc5100043e0 con 0x556eaffa4210 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/4294225676 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc5100c9010 con 0x556eaffa4210 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc51e1ec740 10 client.26687 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x556eb0029720 con 0x556eaffa4210 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/4294225676 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc51010b040 con 0x556eaffa4210 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc51e1ec740 10 client.26687 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc51e1ec740 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc51e1ec740 20 client.26687 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556eb0029be0 0x556eb002c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc51e1ec740 4 client.26687 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.923+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc504004550 con 0x556eb0029be0 2024-08-02T17:53:16.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.924+0000 7fc519389640 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556eb0029be0 0x556eb002c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.924+0000 7fc4fd7fa640 10 client.26687 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.924+0000 7fc519389640 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556eb0029be0 0x556eb002c0a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d91d83a8dd0cd2b server_cookie=3316c9dcede32896 in_seq=0 out_seq=0 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.924+0000 7fc4fd7fa640 1 -- 172.21.15.69:0/4294225676 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc504004550 con 0x556eb0029be0 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.924+0000 7fc4fd7fa640 10 client.26687 handle_command_reply: tid=0 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.924+0000 7fc51e1ec740 1 client.26687 shutdown 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.924+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 msgr2=0x556eaffb7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.924+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffb7870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc510005ab0 tx=0x7fc51001a270 comp rx=0 tx=0).stop 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 shutdown_connections 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556eaffaf3f0 0x556eaffb3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556eb0029be0 0x556eb002c0a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc4fd7fa640 0 client.26687 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556eaffa4b30 0x556eaffb2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 1 --2- 172.21.15.69:0/4294225676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556eaffa4210 0x556eaffb7870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 >> 172.21.15.69:0/4294225676 conn(0x556eaffa07a0 msgr2=0x556eaffa6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 shutdown_connections 2024-08-02T17:53:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 1 -- 172.21.15.69:0/4294225676 wait complete. 2024-08-02T17:53:16.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.925+0000 7fc51e1ec740 20 client.26687 trim_cache size 0 max 16384 2024-08-02T17:53:16.932 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:53:16.932 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:53:16.932 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:53:16.932 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:53:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc5000a2f00 msgr2=0x7fc5000a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc5000a2f00 0x7fc5000a53c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc50c005fd0 tx=0x7fc50c009880 comp rx=0 tx=0).stop 2024-08-02T17:53:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc514109560 msgr2=0x7fc51411a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc514109560 0x7fc51411a7b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc510000c00 tx=0x7fc510004c80 comp rx=0 tx=0).stop 2024-08-02T17:53:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 shutdown_connections 2024-08-02T17:53:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc51411ae70 0x7fc514116850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc5000a2f00 0x7fc5000a53c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc514116d90 0x7fc514117220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 --2- 172.21.15.69:0/1468290150 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc514109560 0x7fc51411a7b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 >> 172.21.15.69:0/1468290150 conn(0x7fc51406de70 msgr2=0x7fc514070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 shutdown_connections 2024-08-02T17:53:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:16.926+0000 7fc51bdee640 1 -- 172.21.15.69:0/1468290150 wait complete. 2024-08-02T17:53:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:16 smithi083 ceph-mon[37031]: pgmap v210: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-02T17:53:17.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:16 smithi077 ceph-mon[37108]: pgmap v210: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2024-08-02T17:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.017+0000 7f9d4e0da640 1 -- 172.21.15.69:0/2568973946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4811ae70 msgr2=0x7f9d4811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.017+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/2568973946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4811ae70 0x7f9d4811d260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009cd0 tx=0x7f9d4002a930 comp rx=0 tx=0).stop 2024-08-02T17:53:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.019+0000 7f9d4e0da640 1 -- 172.21.15.69:0/2568973946 shutdown_connections 2024-08-02T17:53:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.019+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/2568973946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4811ae70 0x7f9d4811d260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.019+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/2568973946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d48109300 0x7f9d48109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.019+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/2568973946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d48108f20 0x7f9d4810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.019+0000 7f9d4e0da640 1 -- 172.21.15.69:0/2568973946 >> 172.21.15.69:0/2568973946 conn(0x7f9d4806cb20 msgr2=0x7f9d4806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:18.030 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.027+0000 7f9d4e0da640 1 -- 172.21.15.69:0/2568973946 shutdown_connections 2024-08-02T17:53:18.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.029+0000 7f9d4e0da640 1 -- 172.21.15.69:0/2568973946 wait complete. 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.030+0000 7f9d4e0da640 1 Processor -- start 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.030+0000 7f9d4e0da640 1 -- start start 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.030+0000 7f9d4e0da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d48108f20 0x7f9d481182e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.033+0000 7f9d4e0da640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d48109300 0x7f9d48118820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.034+0000 7f9d4e0da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4811ae70 0x7f9d48111480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.034+0000 7f9d4e0da640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48120b20 con 0x7f9d48108f20 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.034+0000 7f9d4e0da640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d481209a0 con 0x7f9d48109300 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.034+0000 7f9d4e0da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48120ca0 con 0x7f9d4811ae70 2024-08-02T17:53:18.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.035+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d48109300 0x7f9d48118820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.035+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d48109300 0x7f9d48118820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39820/0 (socket says 172.21.15.69:39820) 2024-08-02T17:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.035+0000 7f9d3ffff640 1 -- 172.21.15.69:0/1383718116 learned_addr learned my addr 172.21.15.69:0/1383718116 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.036+0000 7f9d477fe640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d48108f20 0x7f9d481182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.038+0000 7f9d3ffff640 1 -- 172.21.15.69:0/1383718116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4811ae70 msgr2=0x7f9d48111480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.038+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4811ae70 0x7f9d48111480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.038+0000 7f9d3ffff640 1 -- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d48108f20 msgr2=0x7f9d481182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.038+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d48108f20 0x7f9d481182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.040 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.038+0000 7f9d3ffff640 1 -- 172.21.15.69:0/1383718116 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d48111d40 con 0x7f9d48109300 2024-08-02T17:53:18.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.041+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d48109300 0x7f9d48118820 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b450 tx=0x7f9d3000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.046+0000 7f9d457fa640 1 -- 172.21.15.69:0/1383718116 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d30015020 con 0x7f9d48109300 2024-08-02T17:53:18.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.046+0000 7f9d457fa640 1 -- 172.21.15.69:0/1383718116 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d30011d70 con 0x7f9d48109300 2024-08-02T17:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.046+0000 7f9d457fa640 1 -- 172.21.15.69:0/1383718116 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3000f340 con 0x7f9d48109300 2024-08-02T17:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.047+0000 7f9d4e0da640 1 -- 172.21.15.69:0/1383718116 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d48112030 con 0x7f9d48109300 2024-08-02T17:53:18.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.047+0000 7f9d4e0da640 1 -- 172.21.15.69:0/1383718116 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d48115fe0 con 0x7f9d48109300 2024-08-02T17:53:18.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.052+0000 7f9d457fa640 1 -- 172.21.15.69:0/1383718116 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9d30018020 con 0x7f9d48109300 2024-08-02T17:53:18.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.054+0000 7f9d457fa640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d340a2f00 0x7f9d340a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.056+0000 7f9d457fa640 1 -- 172.21.15.69:0/1383718116 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f9d300ceee0 con 0x7f9d48109300 2024-08-02T17:53:18.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.059+0000 7f9d477fe640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d340a2f00 0x7f9d340a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.066 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.065+0000 7f9d50e39740 1 Processor -- start 2024-08-02T17:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.066+0000 7f9d50e39740 1 -- start start 2024-08-02T17:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.066+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.066+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.066+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 0x55e7ae9dfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.066+0000 7f9d50e39740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e7ae9369b0 con 0x55e7ae9d27b0 2024-08-02T17:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.066+0000 7f9d50e39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e7ae8de800 con 0x55e7ae9d30d0 2024-08-02T17:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.066+0000 7f9d50e39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7ae932970 con 0x55e7ae9dd970 2024-08-02T17:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.067+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.067+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39834/0 (socket says 172.21.15.69:39834) 2024-08-02T17:53:18.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.067+0000 7f9d3ffff640 1 -- 172.21.15.69:0/957982793 learned_addr learned my addr 172.21.15.69:0/957982793 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:18.069 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.068+0000 7f9d47fff640 1 --2- 172.21.15.69:0/957982793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 0x55e7ae9dfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.068+0000 7f9d3ffff640 1 -- 172.21.15.69:0/957982793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 msgr2=0x55e7ae9dfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.068+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/957982793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 0x55e7ae9dfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.068+0000 7f9d3ffff640 1 -- 172.21.15.69:0/957982793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d27b0 msgr2=0x55e7ae9d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.068+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/957982793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.068+0000 7f9d3ffff640 1 -- 172.21.15.69:0/957982793 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ae64a210 con 0x55e7ae9d30d0 2024-08-02T17:53:18.070 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.069+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/957982793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dd430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d30097a90 tx=0x7f9d3009b0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b00562dfa38335 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:18.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.072+0000 7f9d477fe640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d340a2f00 0x7f9d340a53c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006fd0 tx=0x7f9d380090e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.073+0000 7f9d3dffb640 1 -- 172.21.15.69:0/957982793 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d300104e0 con 0x55e7ae9d30d0 2024-08-02T17:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.073+0000 7f9d3dffb640 1 -- 172.21.15.69:0/957982793 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d300993c0 con 0x55e7ae9d30d0 2024-08-02T17:53:18.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.073+0000 7f9d3dffb640 1 -- 172.21.15.69:0/957982793 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d300cc460 con 0x55e7ae9d30d0 2024-08-02T17:53:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.074+0000 7f9d50e39740 1 -- 172.21.15.69:0/957982793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d30d0 msgr2=0x55e7ae9dd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.074+0000 7f9d50e39740 1 --2- 172.21.15.69:0/957982793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dd430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9d30097a90 tx=0x7f9d3009b0c0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.079+0000 7f9d50e39740 1 -- 172.21.15.69:0/957982793 shutdown_connections 2024-08-02T17:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.079+0000 7f9d50e39740 1 --2- 172.21.15.69:0/957982793 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 0x55e7ae9dfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.079+0000 7f9d50e39740 1 --2- 172.21.15.69:0/957982793 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dd430 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.079+0000 7f9d50e39740 1 --2- 172.21.15.69:0/957982793 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.079+0000 7f9d50e39740 1 -- 172.21.15.69:0/957982793 >> 172.21.15.69:0/957982793 conn(0x55e7ae9ced20 msgr2=0x55e7ae9cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:18.086 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.085+0000 7f9d50e39740 1 -- 172.21.15.69:0/957982793 shutdown_connections 2024-08-02T17:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.089+0000 7f9d50e39740 1 -- 172.21.15.69:0/957982793 wait complete. 2024-08-02T17:53:18.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.089+0000 7f9d50e39740 1 -- start start 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.089+0000 7f9d50e39740 1 Processor -- start 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 0x55e7ae9db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d50e39740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e7ae9e0540 con 0x55e7ae9d30d0 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d50e39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e7ae8f2500 con 0x55e7ae9d27b0 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d50e39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e7ae9e06c0 con 0x55e7ae9dd970 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d50e39740 10 client.0 _ll_register_callbacks cb 0x55e7ae649a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d477fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d477fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9da950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39846/0 (socket says 172.21.15.69:39846) 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d477fe640 1 -- 172.21.15.69:0/2971233283 learned_addr learned my addr 172.21.15.69:0/2971233283 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.090+0000 7f9d47fff640 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 0x55e7ae9db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.091+0000 7f9d477fe640 1 -- 172.21.15.69:0/2971233283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 msgr2=0x55e7ae9db3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.091+0000 7f9d477fe640 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 0x55e7ae9db3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.091+0000 7f9d477fe640 1 -- 172.21.15.69:0/2971233283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 msgr2=0x55e7ae9dae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.091+0000 7f9d477fe640 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.091+0000 7f9d477fe640 1 -- 172.21.15.69:0/2971233283 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ae716620 con 0x55e7ae9d27b0 2024-08-02T17:53:18.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.091+0000 7f9d477fe640 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9da950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9d38015700 tx=0x7f9d38015b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ecfd32fe1254f31 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.092+0000 7f9d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.092+0000 7f9d26ffd640 1 -- 172.21.15.69:0/2971233283 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38023020 con 0x55e7ae9d27b0 2024-08-02T17:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.092+0000 7f9d26ffd640 1 -- 172.21.15.69:0/2971233283 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f9d38013780 con 0x55e7ae9d27b0 2024-08-02T17:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.092+0000 7f9d50e39740 1 -- 172.21.15.69:0/2971233283 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d300993c0 con 0x55e7ae9d27b0 2024-08-02T17:53:18.095 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.093+0000 7f9d26ffd640 1 -- 172.21.15.69:0/2971233283 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d3802e050 con 0x55e7ae9d27b0 2024-08-02T17:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.095+0000 7f9d50e39740 10 client.26696 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:18.096 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.095+0000 7f9d50e39740 1 -- 172.21.15.69:0/2971233283 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7aea0e3a0 con 0x55e7ae9d27b0 2024-08-02T17:53:18.098 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.097+0000 7f9d26ffd640 1 -- 172.21.15.69:0/2971233283 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d300993c0 con 0x55e7ae9d27b0 2024-08-02T17:53:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.098+0000 7f9d26ffd640 1 -- 172.21.15.69:0/2971233283 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d3801e030 con 0x55e7ae9d27b0 2024-08-02T17:53:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.098+0000 7f9d50e39740 10 client.26696 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.098+0000 7f9d50e39740 10 client.26696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.098+0000 7f9d50e39740 20 client.26696 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.098+0000 7f9d50e39740 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7aea0fcd0 0x55e7aea12150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.098+0000 7f9d50e39740 4 client.26696 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:18.099 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.098+0000 7f9d50e39740 1 -- 172.21.15.69:0/2971233283 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ae9440e0 con 0x55e7aea0fcd0 2024-08-02T17:53:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.102+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7aea0fcd0 0x55e7aea12150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.103+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7aea0fcd0 0x55e7aea12150 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b15d8e4224f13fe server_cookie=d23eac1fd8358966 in_seq=0 out_seq=0 2024-08-02T17:53:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.104+0000 7f9d26ffd640 10 client.26696 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.104+0000 7f9d26ffd640 1 -- 172.21.15.69:0/2971233283 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e7ae9440e0 con 0x55e7aea0fcd0 2024-08-02T17:53:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.104+0000 7f9d26ffd640 10 client.26696 handle_command_reply: tid=0 2024-08-02T17:53:18.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.104+0000 7f9d50e39740 1 client.26696 shutdown 2024-08-02T17:53:18.108 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.105+0000 7f9d50e39740 1 -- 172.21.15.69:0/2971233283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d27b0 msgr2=0x55e7ae9da950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.105+0000 7f9d50e39740 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9da950 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9d38015700 tx=0x7f9d38015b00 comp rx=0 tx=0).stop 2024-08-02T17:53:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.106+0000 7f9d50e39740 1 -- 172.21.15.69:0/2971233283 shutdown_connections 2024-08-02T17:53:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.106+0000 7f9d50e39740 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9dae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.106+0000 7f9d50e39740 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7aea0fcd0 0x55e7aea12150 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.106+0000 7f9d50e39740 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9dd970 0x55e7ae9db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.106+0000 7f9d50e39740 1 --2- 172.21.15.69:0/2971233283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9da950 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.106+0000 7f9d50e39740 1 -- 172.21.15.69:0/2971233283 >> 172.21.15.69:0/2971233283 conn(0x55e7ae9ced20 msgr2=0x55e7ae9d4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:18.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.107+0000 7f9d26ffd640 0 client.26696 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:18.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.109+0000 7f9d50e39740 1 -- 172.21.15.69:0/2971233283 shutdown_connections 2024-08-02T17:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.113+0000 7f9d50e39740 1 -- 172.21.15.69:0/2971233283 wait complete. 2024-08-02T17:53:18.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.114+0000 7f9d50e39740 20 client.26696 trim_cache size 0 max 16384 2024-08-02T17:53:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.152+0000 7f9d50e39740 1 Processor -- start 2024-08-02T17:53:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.152+0000 7f9d50e39740 1 -- start start 2024-08-02T17:53:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.152+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7aea17a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.152+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7aea17fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.152+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 0x55e7ae9cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.152+0000 7f9d50e39740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4807fe40 con 0x55e7ae9d30d0 2024-08-02T17:53:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.152+0000 7f9d50e39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4807d750 con 0x55e7ae9dd970 2024-08-02T17:53:18.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.152+0000 7f9d50e39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48073740 con 0x55e7ae9d27b0 2024-08-02T17:53:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7aea17a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d477fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7aea17a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:60102/0 (socket says 172.21.15.69:60102) 2024-08-02T17:53:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d477fe640 1 -- 172.21.15.69:0/3878705079 learned_addr learned my addr 172.21.15.69:0/3878705079 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d47fff640 1 --2- 172.21.15.69:0/3878705079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 0x55e7ae9cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d477fe640 1 -- 172.21.15.69:0/3878705079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 msgr2=0x55e7ae9cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d477fe640 1 --2- 172.21.15.69:0/3878705079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 0x55e7ae9cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d477fe640 1 -- 172.21.15.69:0/3878705079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 msgr2=0x55e7aea17fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:18.156 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d477fe640 1 --2- 172.21.15.69:0/3878705079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7aea17fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.153+0000 7f9d477fe640 1 -- 172.21.15.69:0/3878705079 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ae9cf9a0 con 0x55e7ae9d27b0 2024-08-02T17:53:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.154+0000 7f9d477fe640 1 --2- 172.21.15.69:0/3878705079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7aea17a60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9d38017010 tx=0x7f9d3800d0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59ab03686e58a50 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.154+0000 7f9d3d7fa640 1 -- 172.21.15.69:0/3878705079 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38036070 con 0x55e7ae9d27b0 2024-08-02T17:53:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.155+0000 7f9d3d7fa640 1 -- 172.21.15.69:0/3878705079 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d38006cb0 con 0x55e7ae9d27b0 2024-08-02T17:53:18.157 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.155+0000 7f9d3d7fa640 1 -- 172.21.15.69:0/3878705079 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38023040 con 0x55e7ae9d27b0 2024-08-02T17:53:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.156+0000 7f9d50e39740 1 -- 172.21.15.69:0/3878705079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 msgr2=0x55e7aea17a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.156+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3878705079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7aea17a60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9d38017010 tx=0x7f9d3800d0c0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.156+0000 7f9d50e39740 1 -- 172.21.15.69:0/3878705079 shutdown_connections 2024-08-02T17:53:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.156+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3878705079 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7aea17fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.156+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3878705079 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 0x55e7ae9cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.156+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3878705079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7aea17a60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.156+0000 7f9d50e39740 1 -- 172.21.15.69:0/3878705079 >> 172.21.15.69:0/3878705079 conn(0x55e7ae9ced20 msgr2=0x55e7ae9d5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:18.159 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.157+0000 7f9d50e39740 1 -- 172.21.15.69:0/3878705079 shutdown_connections 2024-08-02T17:53:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.157+0000 7f9d50e39740 1 -- 172.21.15.69:0/3878705079 wait complete. 2024-08-02T17:53:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.157+0000 7f9d50e39740 1 -- start start 2024-08-02T17:53:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.157+0000 7f9d50e39740 1 Processor -- start 2024-08-02T17:53:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d50e39740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 0x55e7ae9e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d50e39740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48116b90 con 0x55e7ae9d30d0 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d50e39740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4807d080 con 0x55e7ae9dd970 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d50e39740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4807c2e0 con 0x55e7ae9d27b0 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d50e39740 10 client.0 _ll_register_callbacks cb 0x7f9d380095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d3ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9e1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47914/0 (socket says 172.21.15.69:47914) 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d3ffff640 1 -- 172.21.15.69:0/3496252746 learned_addr learned my addr 172.21.15.69:0/3496252746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d477fe640 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.158+0000 7f9d47fff640 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 0x55e7ae9e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d477fe640 1 -- 172.21.15.69:0/3496252746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 msgr2=0x55e7ae9e16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d477fe640 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 0x55e7ae9e16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d477fe640 1 -- 172.21.15.69:0/3496252746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 msgr2=0x55e7ae9e1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d477fe640 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9e1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d477fe640 1 -- 172.21.15.69:0/3496252746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ae9e1c60 con 0x55e7ae9d27b0 2024-08-02T17:53:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d477fe640 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9e5e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9d38015700 tx=0x7f9d3801f470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d338f26335cb5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:18.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d26ffd640 1 -- 172.21.15.69:0/3496252746 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3803e020 con 0x55e7ae9d27b0 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d26ffd640 1 -- 172.21.15.69:0/3496252746 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f9d38004430 con 0x55e7ae9d27b0 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.159+0000 7f9d50e39740 1 -- 172.21.15.69:0/3496252746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e7ae93e430 con 0x55e7ae9d27b0 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.160+0000 7f9d26ffd640 1 -- 172.21.15.69:0/3496252746 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d38049050 con 0x55e7ae9d27b0 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.160+0000 7f9d50e39740 10 client.26575 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.160+0000 7f9d50e39740 1 -- 172.21.15.69:0/3496252746 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e7aea7a7d0 con 0x55e7ae9d27b0 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.160+0000 7f9d26ffd640 1 -- 172.21.15.69:0/3496252746 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e7ae93e430 con 0x55e7ae9d27b0 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.160+0000 7f9d26ffd640 1 -- 172.21.15.69:0/3496252746 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d3801e040 con 0x55e7ae9d27b0 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.160+0000 7f9d50e39740 10 client.26575 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.160+0000 7f9d50e39740 10 client.26575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.161+0000 7f9d50e39740 20 client.26575 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:18.164 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.161+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7aea7ae80 0x55e7aea7d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.161+0000 7f9d50e39740 4 client.26575 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:53:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.161+0000 7f9d50e39740 1 -- 172.21.15.69:0/3496252746 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7ae8bb8e0 con 0x55e7aea7ae80 2024-08-02T17:53:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.161+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7aea7ae80 0x55e7aea7d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.162+0000 7f9d3ffff640 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7aea7ae80 0x55e7aea7d270 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82301a5c3f6703ad server_cookie=e01879f5cd7f9977 in_seq=0 out_seq=0 2024-08-02T17:53:18.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.162+0000 7f9d26ffd640 10 client.26575 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d26ffd640 1 -- 172.21.15.69:0/3496252746 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7ae8bb8e0 con 0x55e7aea7ae80 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d26ffd640 10 client.26575 handle_command_reply: tid=0 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 client.26575 shutdown 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 -- 172.21.15.69:0/3496252746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 msgr2=0x55e7ae9e5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9e5e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9d38015700 tx=0x7f9d3801f470 comp rx=0 tx=0).stop 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 -- 172.21.15.69:0/3496252746 shutdown_connections 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e7ae9d30d0 0x55e7ae9e1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e7aea7ae80 0x55e7aea7d270 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e7ae9dd970 0x55e7ae9e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 --2- 172.21.15.69:0/3496252746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e7ae9d27b0 0x55e7ae9e5e40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 -- 172.21.15.69:0/3496252746 >> 172.21.15.69:0/3496252746 conn(0x55e7ae9ced20 msgr2=0x55e7ae9cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d26ffd640 0 client.26575 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.190+0000 7f9d50e39740 1 -- 172.21.15.69:0/3496252746 shutdown_connections 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.191+0000 7f9d50e39740 1 -- 172.21.15.69:0/3496252746 wait complete. 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.191+0000 7f9d50e39740 20 client.26575 trim_cache size 0 max 16384 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:53:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:53:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.191+0000 7f9d4e0da640 1 -- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d340a2f00 msgr2=0x7f9d340a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.191+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d340a2f00 0x7f9d340a53c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9d38006fd0 tx=0x7f9d380090e0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.191+0000 7f9d4e0da640 1 -- 172.21.15.69:0/1383718116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d48109300 msgr2=0x7f9d48118820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.191+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d48109300 0x7f9d48118820 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b450 tx=0x7f9d3000b910 comp rx=0 tx=0).stop 2024-08-02T17:53:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.192+0000 7f9d4e0da640 1 -- 172.21.15.69:0/1383718116 shutdown_connections 2024-08-02T17:53:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.192+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d340a2f00 0x7f9d340a53c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.192+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d4811ae70 0x7f9d48111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.192+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d48109300 0x7f9d48118820 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.192+0000 7f9d4e0da640 1 --2- 172.21.15.69:0/1383718116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d48108f20 0x7f9d481182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.192+0000 7f9d4e0da640 1 -- 172.21.15.69:0/1383718116 >> 172.21.15.69:0/1383718116 conn(0x7f9d4806cb20 msgr2=0x7f9d4811b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.192+0000 7f9d4e0da640 1 -- 172.21.15.69:0/1383718116 shutdown_connections 2024-08-02T17:53:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:18.192+0000 7f9d4e0da640 1 -- 172.21.15.69:0/1383718116 wait complete. 2024-08-02T17:53:19.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:18 smithi083 ceph-mon[37031]: pgmap v211: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.11k op/s 2024-08-02T17:53:19.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:18 smithi077 ceph-mon[37108]: pgmap v211: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.11k op/s 2024-08-02T17:53:19.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:18 smithi069 ceph-mon[32746]: pgmap v211: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.11k op/s 2024-08-02T17:53:19.559 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:53:19.560 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:53:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 370.0 s, 678.0 tps, lat 150.558 ms stddev 218.459 2024-08-02T17:53:19.970 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:53:20.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.543+0000 7f9418bb2640 1 -- 172.21.15.69:0/529676387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9414109180 msgr2=0x7f9414109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:20.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.543+0000 7f9418bb2640 1 --2- 172.21.15.69:0/529676387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9414109180 0x7f9414109560 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f940400b880 tx=0x7f9404005f90 comp rx=0 tx=0).stop 2024-08-02T17:53:20.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.549+0000 7f9418bb2640 1 -- 172.21.15.69:0/529676387 shutdown_connections 2024-08-02T17:53:20.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.549+0000 7f9418bb2640 1 --2- 172.21.15.69:0/529676387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9414109b30 0x7f9414074000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.549+0000 7f9418bb2640 1 --2- 172.21.15.69:0/529676387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94140746c0 0x7f941407b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.549+0000 7f9418bb2640 1 --2- 172.21.15.69:0/529676387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9414109180 0x7f9414109560 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.549+0000 7f9418bb2640 1 -- 172.21.15.69:0/529676387 >> 172.21.15.69:0/529676387 conn(0x7f941406de70 msgr2=0x7f941406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:20.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.550+0000 7f9418bb2640 1 -- 172.21.15.69:0/529676387 shutdown_connections 2024-08-02T17:53:20.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.550+0000 7f9418bb2640 1 -- 172.21.15.69:0/529676387 wait complete. 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.551+0000 7f9418bb2640 1 Processor -- start 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.551+0000 7f9418bb2640 1 -- start start 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.551+0000 7f9418bb2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94140746c0 0x7f941407ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.551+0000 7f9418bb2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9414109180 0x7f9414075ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.551+0000 7f9418bb2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9414109b30 0x7f9414076400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.551+0000 7f9418bb2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f941407e4b0 con 0x7f94140746c0 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.551+0000 7f9418bb2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f941407e330 con 0x7f9414109180 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.551+0000 7f9418bb2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f941407e630 con 0x7f9414109b30 2024-08-02T17:53:20.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.552+0000 7f9412d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9414109b30 0x7f9414076400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:20.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.552+0000 7f9412d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9414109b30 0x7f9414076400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:60124/0 (socket says 172.21.15.69:60124) 2024-08-02T17:53:20.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.552+0000 7f9412d9d640 1 -- 172.21.15.69:0/3225913187 learned_addr learned my addr 172.21.15.69:0/3225913187 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:20.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.552+0000 7f9412d9d640 1 -- 172.21.15.69:0/3225913187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9414109180 msgr2=0x7f9414075ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:53:20.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.552+0000 7f9412d9d640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9414109180 0x7f9414075ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.553+0000 7f9412d9d640 1 -- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94140746c0 msgr2=0x7f941407ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:20.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.553+0000 7f9412d9d640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94140746c0 0x7f941407ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.553+0000 7f9412d9d640 1 -- 172.21.15.69:0/3225913187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9414076b80 con 0x7f9414109b30 2024-08-02T17:53:20.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.553+0000 7f9412d9d640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9414109b30 0x7f9414076400 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f940c00bcc0 tx=0x7f940c008600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:20.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.555+0000 7f94037fe640 1 -- 172.21.15.69:0/3225913187 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c004b20 con 0x7f9414109b30 2024-08-02T17:53:20.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.555+0000 7f94037fe640 1 -- 172.21.15.69:0/3225913187 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c004cc0 con 0x7f9414109b30 2024-08-02T17:53:20.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.555+0000 7f94037fe640 1 -- 172.21.15.69:0/3225913187 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c015420 con 0x7f9414109b30 2024-08-02T17:53:20.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.563+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94141bbcd0 con 0x7f9414109b30 2024-08-02T17:53:20.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.563+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94141bc130 con 0x7f9414109b30 2024-08-02T17:53:20.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.565+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9414073830 con 0x7f9414109b30 2024-08-02T17:53:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.567+0000 7f94037fe640 1 -- 172.21.15.69:0/3225913187 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f940c01c030 con 0x7f9414109b30 2024-08-02T17:53:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.568+0000 7f94037fe640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:20.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.569+0000 7f94037fe640 1 -- 172.21.15.69:0/3225913187 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f940c0cf6a0 con 0x7f9414109b30 2024-08-02T17:53:20.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.569+0000 7f941259c640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:20.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.571+0000 7f941259c640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f940400b880 tx=0x7f94040056d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:20.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.577+0000 7f94037fe640 1 -- 172.21.15.69:0/3225913187 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f940c023070 con 0x7f9414109b30 2024-08-02T17:53:20.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.948+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9414109590 con 0x7f9414109b30 2024-08-02T17:53:20.951 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:53:20.951 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:53:20.951 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:53:20.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.949+0000 7f94037fe640 1 -- 172.21.15.69:0/3225913187 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f9414109590 con 0x7f9414109b30 2024-08-02T17:53:20.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 msgr2=0x7f93fc0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:20.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f940400b880 tx=0x7f94040056d0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9414109b30 msgr2=0x7f9414076400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:20.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9414109b30 0x7f9414076400 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f940c00bcc0 tx=0x7f940c008600 comp rx=0 tx=0).stop 2024-08-02T17:53:20.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 shutdown_connections 2024-08-02T17:53:20.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9414109b30 0x7f9414076400 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9414109180 0x7f9414075ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 --2- 172.21.15.69:0/3225913187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f94140746c0 0x7f941407ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:20.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.952+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 >> 172.21.15.69:0/3225913187 conn(0x7f941406de70 msgr2=0x7f9414072120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:20.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.953+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 shutdown_connections 2024-08-02T17:53:20.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:20.953+0000 7f9418bb2640 1 -- 172.21.15.69:0/3225913187 wait complete. 2024-08-02T17:53:20.970 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:20 smithi069 ceph-mon[32746]: pgmap v212: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-08-02T17:53:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:20 smithi083 ceph-mon[37031]: pgmap v212: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-08-02T17:53:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:20 smithi077 ceph-mon[37108]: pgmap v212: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 18 MiB/s wr, 1.12k op/s 2024-08-02T17:53:22.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:21 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3225913187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:53:22.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:21 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3225913187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:53:22.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:21 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3225913187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:53:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:22 smithi083 ceph-mon[37031]: pgmap v213: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-08-02T17:53:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:22 smithi077 ceph-mon[37108]: pgmap v213: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-08-02T17:53:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:22 smithi069 ceph-mon[32746]: pgmap v213: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-08-02T17:53:23.483 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:53:23.927 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:53:24.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.502+0000 7f96f129a640 1 -- 172.21.15.69:0/3745642198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec102c20 msgr2=0x7f96ec10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:24.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.502+0000 7f96f129a640 1 --2- 172.21.15.69:0/3745642198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec102c20 0x7f96ec10d770 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009ca0 tx=0x7f96e402a980 comp rx=0 tx=0).stop 2024-08-02T17:53:24.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.505+0000 7f96f129a640 1 -- 172.21.15.69:0/3745642198 shutdown_connections 2024-08-02T17:53:24.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.505+0000 7f96f129a640 1 --2- 172.21.15.69:0/3745642198 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec102c20 0x7f96ec10d770 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.505+0000 7f96f129a640 1 --2- 172.21.15.69:0/3745642198 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96ec1022a0 0x7f96ec1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.505+0000 7f96f129a640 1 --2- 172.21.15.69:0/3745642198 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96ec100b50 0x7f96ec100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.505+0000 7f96f129a640 1 -- 172.21.15.69:0/3745642198 >> 172.21.15.69:0/3745642198 conn(0x7f96ec0fc500 msgr2=0x7f96ec0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:24.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.505+0000 7f96f129a640 1 -- 172.21.15.69:0/3745642198 shutdown_connections 2024-08-02T17:53:24.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.509+0000 7f96f129a640 1 -- 172.21.15.69:0/3745642198 wait complete. 2024-08-02T17:53:24.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.509+0000 7f96f129a640 1 Processor -- start 2024-08-02T17:53:24.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.510+0000 7f96f129a640 1 -- start start 2024-08-02T17:53:24.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.510+0000 7f96f129a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96ec1022a0 0x7f96ec1161c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.510+0000 7f96f129a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96ec102c20 0x7f96ec116700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.510+0000 7f96f129a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec1112e0 0x7f96ec111790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.510+0000 7f96f129a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec10fc80 con 0x7f96ec1112e0 2024-08-02T17:53:24.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.510+0000 7f96f129a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec10fb00 con 0x7f96ec102c20 2024-08-02T17:53:24.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.510+0000 7f96f129a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec10fe00 con 0x7f96ec1022a0 2024-08-02T17:53:24.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.511+0000 7f96eb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec1112e0 0x7f96ec111790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.511+0000 7f96eb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec1112e0 0x7f96ec111790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47940/0 (socket says 172.21.15.69:47940) 2024-08-02T17:53:24.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.511+0000 7f96eb7fe640 1 -- 172.21.15.69:0/1149857024 learned_addr learned my addr 172.21.15.69:0/1149857024 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:24.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.512+0000 7f96eaffd640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96ec1022a0 0x7f96ec1161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.513+0000 7f96eb7fe640 1 -- 172.21.15.69:0/1149857024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96ec1022a0 msgr2=0x7f96ec1161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:24.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.513+0000 7f96eb7fe640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96ec1022a0 0x7f96ec1161c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.513+0000 7f96eb7fe640 1 -- 172.21.15.69:0/1149857024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96ec102c20 msgr2=0x7f96ec116700 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:53:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.513+0000 7f96eb7fe640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96ec102c20 0x7f96ec116700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.513+0000 7f96eb7fe640 1 -- 172.21.15.69:0/1149857024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec112050 con 0x7f96ec1112e0 2024-08-02T17:53:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.517+0000 7f96eb7fe640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec1112e0 0x7f96ec111790 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009dc0 tx=0x7f96e40308b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.517+0000 7f96d3fff640 1 -- 172.21.15.69:0/1149857024 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4034050 con 0x7f96ec1112e0 2024-08-02T17:53:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.517+0000 7f96f129a640 1 -- 172.21.15.69:0/1149857024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec11def0 con 0x7f96ec1112e0 2024-08-02T17:53:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.517+0000 7f96f129a640 1 -- 172.21.15.69:0/1149857024 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec11e430 con 0x7f96ec1112e0 2024-08-02T17:53:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.517+0000 7f96d3fff640 1 -- 172.21.15.69:0/1149857024 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e4004040 con 0x7f96ec1112e0 2024-08-02T17:53:24.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.517+0000 7f96d3fff640 1 -- 172.21.15.69:0/1149857024 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4039680 con 0x7f96ec1112e0 2024-08-02T17:53:24.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.522+0000 7f96d3fff640 1 -- 172.21.15.69:0/1149857024 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f96e4037020 con 0x7f96ec1112e0 2024-08-02T17:53:24.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.522+0000 7f96d3fff640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f96d40a2f00 0x7f96d40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.523+0000 7f96d3fff640 1 -- 172.21.15.69:0/1149857024 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f96e40f1860 con 0x7f96ec1112e0 2024-08-02T17:53:24.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96eaffd640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f96d40a2f00 0x7f96d40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96f3698740 1 Processor -- start 2024-08-02T17:53:24.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96f3698740 1 -- start start 2024-08-02T17:53:24.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96f3698740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96f3698740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455dc910 0x5636455e6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96f3698740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96f3698740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563645541a40 con 0x5636455dbff0 2024-08-02T17:53:24.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96f3698740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636454d99d0 con 0x5636455e71d0 2024-08-02T17:53:24.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.526+0000 7f96f3698740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56364553da00 con 0x5636455dc910 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.528+0000 7f96eb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.528+0000 7f96eb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39896/0 (socket says 172.21.15.69:39896) 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.528+0000 7f96eb7fe640 1 -- 172.21.15.69:0/2911868496 learned_addr learned my addr 172.21.15.69:0/2911868496 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96ea7fc640 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455dc910 0x5636455e6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96eaffd640 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96eb7fe640 1 -- 172.21.15.69:0/2911868496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455dc910 msgr2=0x5636455e6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96eb7fe640 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455dc910 0x5636455e6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96eb7fe640 1 -- 172.21.15.69:0/2911868496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 msgr2=0x5636455dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96eb7fe640 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96eb7fe640 1 -- 172.21.15.69:0/2911868496 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636454e0650 con 0x5636455e71d0 2024-08-02T17:53:24.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96eaffd640 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:53:24.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96eb7fe640 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e95c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f96e4030f50 tx=0x7f96e4009990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73bd98eefa12410d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:24.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.529+0000 7f96d17fa640 1 -- 172.21.15.69:0/2911868496 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e40d5020 con 0x5636455e71d0 2024-08-02T17:53:24.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.530+0000 7f96d17fa640 1 -- 172.21.15.69:0/2911868496 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e40d1e50 con 0x5636455e71d0 2024-08-02T17:53:24.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.530+0000 7f96f3698740 1 -- 172.21.15.69:0/2911868496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 msgr2=0x5636455e95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:24.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.530+0000 7f96f3698740 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e95c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f96e4030f50 tx=0x7f96e4009990 comp rx=0 tx=0).stop 2024-08-02T17:53:24.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.531+0000 7f96f3698740 1 -- 172.21.15.69:0/2911868496 shutdown_connections 2024-08-02T17:53:24.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.531+0000 7f96f3698740 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e95c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.531+0000 7f96f3698740 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455dc910 0x5636455e6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.531+0000 7f96f3698740 1 --2- 172.21.15.69:0/2911868496 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.531+0000 7f96f3698740 1 -- 172.21.15.69:0/2911868496 >> 172.21.15.69:0/2911868496 conn(0x5636455d8580 msgr2=0x5636455d8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:24.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.531+0000 7f96eaffd640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f96d40a2f00 0x7f96d40a53c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f96ec101890 tx=0x7f96e00094a0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:24.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.534+0000 7f96f3698740 1 -- 172.21.15.69:0/2911868496 shutdown_connections 2024-08-02T17:53:24.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.537+0000 7f96f3698740 1 -- 172.21.15.69:0/2911868496 wait complete. 2024-08-02T17:53:24.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.537+0000 7f96f3698740 1 -- start start 2024-08-02T17:53:24.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.537+0000 7f96f3698740 1 Processor -- start 2024-08-02T17:53:24.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96f3698740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455e4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96f3698740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96f3698740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e4b10 0x563645603eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96ea7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96ea7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39902/0 (socket says 172.21.15.69:39902) 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96ea7fc640 1 -- 172.21.15.69:0/1761816138 learned_addr learned my addr 172.21.15.69:0/1761816138 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96eb7fe640 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e4b10 0x563645603eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636454d99d0 con 0x5636455dbff0 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636454fcb20 con 0x5636455e71d0 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636455fab70 con 0x5636455e4b10 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96f3698740 10 client.0 _ll_register_callbacks cb 0x5636454fabb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96ea7fc640 1 -- 172.21.15.69:0/1761816138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e4b10 msgr2=0x563645603eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96ea7fc640 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e4b10 0x563645603eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96ea7fc640 1 -- 172.21.15.69:0/1761816138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 msgr2=0x5636455e4090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.538+0000 7f96ea7fc640 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.539+0000 7f96ea7fc640 1 -- 172.21.15.69:0/1761816138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636452b33c0 con 0x5636455e71d0 2024-08-02T17:53:24.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.539+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e40d1e50 con 0x5636455e71d0 2024-08-02T17:53:24.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.540+0000 7f96ea7fc640 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e45d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f96ec112b30 tx=0x7f96dc02e530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a7717c337bf5e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:24.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.540+0000 7f96ceff5640 10 client.26711 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:53:24.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.543+0000 7f96ceff5640 1 -- 172.21.15.69:0/1761816138 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc041050 con 0x5636455e71d0 2024-08-02T17:53:24.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.543+0000 7f96ceff5640 1 -- 172.21.15.69:0/1761816138 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f96dc03c440 con 0x5636455e71d0 2024-08-02T17:53:24.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.543+0000 7f96ceff5640 1 -- 172.21.15.69:0/1761816138 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96dc037070 con 0x5636455e71d0 2024-08-02T17:53:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.548+0000 7f96f3698740 10 client.26711 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.548+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5636455ec0d0 con 0x5636455e71d0 2024-08-02T17:53:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.548+0000 7f96ceff5640 1 -- 172.21.15.69:0/1761816138 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f96dc03c6a0 con 0x5636455e71d0 2024-08-02T17:53:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.549+0000 7f96f3698740 10 client.26711 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.549+0000 7f96f3698740 10 client.26711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:53:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.549+0000 7f96f3698740 20 client.26711 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:24.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.549+0000 7f96f3698740 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5636455ed480 0x5636455ef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.549+0000 7f96f3698740 4 client.26711 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:24.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.549+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56364554efb0 con 0x5636455ed480 2024-08-02T17:53:24.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.550+0000 7f96eaffd640 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5636455ed480 0x5636455ef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.554+0000 7f96eaffd640 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5636455ed480 0x5636455ef940 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b968b3322bf96f79 server_cookie=8924a4dc64f6d575 in_seq=0 out_seq=0 2024-08-02T17:53:24.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.554+0000 7f96ceff5640 10 client.26711 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:53:24.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.555+0000 7f96ceff5640 1 -- 172.21.15.69:0/1761816138 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56364554efb0 con 0x5636455ed480 2024-08-02T17:53:24.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.555+0000 7f96ceff5640 10 client.26711 handle_command_reply: tid=0 2024-08-02T17:53:24.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.555+0000 7f96f3698740 1 client.26711 shutdown 2024-08-02T17:53:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.556+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 msgr2=0x5636455e45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.556+0000 7f96f3698740 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e45d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f96ec112b30 tx=0x7f96dc02e530 comp rx=0 tx=0).stop 2024-08-02T17:53:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.557+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 shutdown_connections 2024-08-02T17:53:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.557+0000 7f96f3698740 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5636455ed480 0x5636455ef940 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.557+0000 7f96f3698740 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e4b10 0x563645603eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.557+0000 7f96f3698740 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e71d0 0x5636455e45d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.557+0000 7f96f3698740 1 --2- 172.21.15.69:0/1761816138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.557+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 >> 172.21.15.69:0/1761816138 conn(0x5636455d8580 msgr2=0x5636455de2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:24.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.559+0000 7f96ceff5640 0 client.26711 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:53:24.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.559+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 shutdown_connections 2024-08-02T17:53:24.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.559+0000 7f96f3698740 1 -- 172.21.15.69:0/1761816138 wait complete. 2024-08-02T17:53:24.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.560+0000 7f96f3698740 20 client.26711 trim_cache size 0 max 16384 2024-08-02T17:53:24.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.588+0000 7f96f3698740 1 Processor -- start 2024-08-02T17:53:24.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.588+0000 7f96f3698740 1 -- start start 2024-08-02T17:53:24.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.588+0000 7f96f3698740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.588+0000 7f96f3698740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e4b10 0x563645603ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.588+0000 7f96f3698740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e71d0 0x5636455d74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.588+0000 7f96f3698740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8012a50 con 0x5636455dbff0 2024-08-02T17:53:24.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.588+0000 7f96f3698740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96d8010320 con 0x5636455e4b10 2024-08-02T17:53:24.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.588+0000 7f96f3698740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96d800e170 con 0x5636455e71d0 2024-08-02T17:53:24.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.589+0000 7f96ea7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e4b10 0x563645603ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.589+0000 7f96eaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.589+0000 7f96eaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455ef490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47970/0 (socket says 172.21.15.69:47970) 2024-08-02T17:53:24.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.589+0000 7f96eaffd640 1 -- 172.21.15.69:0/2245936981 learned_addr learned my addr 172.21.15.69:0/2245936981 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.590+0000 7f96eaffd640 1 -- 172.21.15.69:0/2245936981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e71d0 msgr2=0x5636455d74a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:53:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.590+0000 7f96eaffd640 1 --2- 172.21.15.69:0/2245936981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e71d0 0x5636455d74a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.590+0000 7f96eaffd640 1 -- 172.21.15.69:0/2245936981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e4b10 msgr2=0x563645603ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.590+0000 7f96eaffd640 1 --2- 172.21.15.69:0/2245936981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e4b10 0x563645603ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.590+0000 7f96eaffd640 1 -- 172.21.15.69:0/2245936981 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636455d96f0 con 0x5636455dbff0 2024-08-02T17:53:24.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.591+0000 7f96eaffd640 1 --2- 172.21.15.69:0/2245936981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455ef490 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009760 tx=0x7f96e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4484091d5f0ca71 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.595+0000 7f96d0ff9640 1 -- 172.21.15.69:0/2245936981 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e001d040 con 0x5636455dbff0 2024-08-02T17:53:24.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.595+0000 7f96d0ff9640 1 -- 172.21.15.69:0/2245936981 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e0030070 con 0x5636455dbff0 2024-08-02T17:53:24.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.596+0000 7f96f3698740 1 -- 172.21.15.69:0/2245936981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 msgr2=0x5636455ef490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:24.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.596+0000 7f96f3698740 1 --2- 172.21.15.69:0/2245936981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455ef490 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009760 tx=0x7f96e00046b0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.603+0000 7f96f3698740 1 -- 172.21.15.69:0/2245936981 shutdown_connections 2024-08-02T17:53:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.603+0000 7f96f3698740 1 --2- 172.21.15.69:0/2245936981 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e71d0 0x5636455d74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.603+0000 7f96f3698740 1 --2- 172.21.15.69:0/2245936981 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455e4b10 0x563645603ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.603+0000 7f96f3698740 1 --2- 172.21.15.69:0/2245936981 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455dbff0 0x5636455ef490 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.603+0000 7f96f3698740 1 -- 172.21.15.69:0/2245936981 >> 172.21.15.69:0/2245936981 conn(0x5636455d8580 msgr2=0x5636455d6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:24.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.606+0000 7f96f3698740 1 -- 172.21.15.69:0/2245936981 shutdown_connections 2024-08-02T17:53:24.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.608+0000 7f96f3698740 1 -- 172.21.15.69:0/2245936981 wait complete. 2024-08-02T17:53:24.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.608+0000 7f96f3698740 1 -- start start 2024-08-02T17:53:24.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.608+0000 7f96f3698740 1 Processor -- start 2024-08-02T17:53:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.612+0000 7f96f3698740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455dbff0 0x563645600300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.612+0000 7f96f3698740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455e4b10 0x5636455fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.612+0000 7f96f3698740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e71d0 0x5636455fd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.612+0000 7f96f3698740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f96d800e7e0 con 0x5636455e4b10 2024-08-02T17:53:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.612+0000 7f96f3698740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f96d800fc50 con 0x5636455dbff0 2024-08-02T17:53:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.612+0000 7f96f3698740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96d800eeb0 con 0x5636455e71d0 2024-08-02T17:53:24.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.612+0000 7f96f3698740 10 client.0 _ll_register_callbacks cb 0x7f96dc006e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.613+0000 7f96eaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455dbff0 0x563645600300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.613+0000 7f96eaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455dbff0 0x563645600300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39934/0 (socket says 172.21.15.69:39934) 2024-08-02T17:53:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.613+0000 7f96eaffd640 1 -- 172.21.15.69:0/3965514704 learned_addr learned my addr 172.21.15.69:0/3965514704 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.615+0000 7f96ea7fc640 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455e4b10 0x5636455fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.616+0000 7f96eaffd640 1 -- 172.21.15.69:0/3965514704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e71d0 msgr2=0x5636455fd320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:53:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.616+0000 7f96eaffd640 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e71d0 0x5636455fd320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.616+0000 7f96eaffd640 1 -- 172.21.15.69:0/3965514704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455e4b10 msgr2=0x5636455fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:24.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.616+0000 7f96eaffd640 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455e4b10 0x5636455fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.616+0000 7f96eaffd640 1 -- 172.21.15.69:0/3965514704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636455fd8e0 con 0x5636455dbff0 2024-08-02T17:53:24.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.621+0000 7f96eaffd640 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455dbff0 0x563645600300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009120 tx=0x7f96e0024d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9578384e3992d70e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.622+0000 7f96ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:53:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.622+0000 7f96ceff5640 1 -- 172.21.15.69:0/3965514704 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0038020 con 0x5636455dbff0 2024-08-02T17:53:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.622+0000 7f96f3698740 1 -- 172.21.15.69:0/3965514704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e0030070 con 0x5636455dbff0 2024-08-02T17:53:24.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.622+0000 7f96ceff5640 1 -- 172.21.15.69:0/3965514704 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f96e0017910 con 0x5636455dbff0 2024-08-02T17:53:24.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.622+0000 7f96ceff5640 1 -- 172.21.15.69:0/3965514704 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96e0043050 con 0x5636455dbff0 2024-08-02T17:53:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.625+0000 7f96f3698740 10 client.26714 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:24.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.625+0000 7f96f3698740 1 -- 172.21.15.69:0/3965514704 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563645661300 con 0x5636455dbff0 2024-08-02T17:53:24.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.628+0000 7f96ceff5640 1 -- 172.21.15.69:0/3965514704 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e0030070 con 0x5636455dbff0 2024-08-02T17:53:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.629+0000 7f96ceff5640 1 -- 172.21.15.69:0/3965514704 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f96e0016050 con 0x5636455dbff0 2024-08-02T17:53:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.630+0000 7f96f3698740 10 client.26714 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:24.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.630+0000 7f96f3698740 10 client.26714 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:53:24.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.630+0000 7f96f3698740 20 client.26714 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:24.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.630+0000 7f96f3698740 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x563645661a80 0x563645663ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:24.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.630+0000 7f96f3698740 4 client.26714 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:53:24.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.630+0000 7f96f3698740 1 -- 172.21.15.69:0/3965514704 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636455494c0 con 0x563645661a80 2024-08-02T17:53:24.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.631+0000 7f96ea7fc640 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x563645661a80 0x563645663ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:24.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.639+0000 7f96ea7fc640 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x563645661a80 0x563645663ea0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d9831a82f6856b0 server_cookie=6296891b07ae202d in_seq=0 out_seq=0 2024-08-02T17:53:24.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.641+0000 7f96ceff5640 10 client.26714 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:53:24.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.641+0000 7f96ceff5640 1 -- 172.21.15.69:0/3965514704 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636455494c0 con 0x563645661a80 2024-08-02T17:53:24.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.641+0000 7f96ceff5640 10 client.26714 handle_command_reply: tid=0 2024-08-02T17:53:24.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.641+0000 7f96f3698740 1 client.26714 shutdown 2024-08-02T17:53:24.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.644+0000 7f96f3698740 1 -- 172.21.15.69:0/3965514704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455dbff0 msgr2=0x563645600300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:24.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.644+0000 7f96f3698740 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455dbff0 0x563645600300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009120 tx=0x7f96e0024d80 comp rx=0 tx=0).stop 2024-08-02T17:53:24.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.646+0000 7f96f3698740 1 -- 172.21.15.69:0/3965514704 shutdown_connections 2024-08-02T17:53:24.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.646+0000 7f96f3698740 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x563645661a80 0x563645663ea0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.646+0000 7f96f3698740 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636455e4b10 0x5636455fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.646+0000 7f96f3698740 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636455e71d0 0x5636455fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.646+0000 7f96f3698740 1 --2- 172.21.15.69:0/3965514704 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636455dbff0 0x563645600300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.646+0000 7f96f3698740 1 -- 172.21.15.69:0/3965514704 >> 172.21.15.69:0/3965514704 conn(0x5636455d8580 msgr2=0x5636455d75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:24.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.646+0000 7f96ceff5640 0 client.26714 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:53:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.650+0000 7f96f3698740 1 -- 172.21.15.69:0/3965514704 shutdown_connections 2024-08-02T17:53:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.650+0000 7f96f3698740 1 -- 172.21.15.69:0/3965514704 wait complete. 2024-08-02T17:53:24.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.650+0000 7f96f3698740 20 client.26714 trim_cache size 0 max 16384 2024-08-02T17:53:24.651 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:53:24.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 -- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f96d40a2f00 msgr2=0x7f96d40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:24.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f96d40a2f00 0x7f96d40a53c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f96ec101890 tx=0x7f96e00094a0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 -- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec1112e0 msgr2=0x7f96ec111790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec1112e0 0x7f96ec111790 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009dc0 tx=0x7f96e40308b0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 -- 172.21.15.69:0/1149857024 shutdown_connections 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f96ec1112e0 0x7f96ec111790 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f96d40a2f00 0x7f96d40a53c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f96ec102c20 0x7f96ec116700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 --2- 172.21.15.69:0/1149857024 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96ec1022a0 0x7f96ec1161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 -- 172.21.15.69:0/1149857024 >> 172.21.15.69:0/1149857024 conn(0x7f96ec0fc500 msgr2=0x7f96ec0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 -- 172.21.15.69:0/1149857024 shutdown_connections 2024-08-02T17:53:24.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:24.654+0000 7f96f129a640 1 -- 172.21.15.69:0/1149857024 wait complete. 2024-08-02T17:53:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 375.0 s, 808.6 tps, lat 122.383 ms stddev 165.739 2024-08-02T17:53:24.996 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:24 smithi069 ceph-mon[32746]: pgmap v214: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 675 op/s 2024-08-02T17:53:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:24 smithi083 ceph-mon[37031]: pgmap v214: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 675 op/s 2024-08-02T17:53:25.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:24 smithi077 ceph-mon[37108]: pgmap v214: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 12 MiB/s wr, 675 op/s 2024-08-02T17:53:25.492 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:53:25.956 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:53:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:26 smithi083 ceph-mon[37031]: pgmap v215: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 12 MiB/s wr, 677 op/s 2024-08-02T17:53:27.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:26 smithi077 ceph-mon[37108]: pgmap v215: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 12 MiB/s wr, 677 op/s 2024-08-02T17:53:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:26 smithi069 ceph-mon[32746]: pgmap v215: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 12 MiB/s wr, 677 op/s 2024-08-02T17:53:28.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2324287138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c0108f20 msgr2=0x7f05c0109380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2324287138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c0108f20 0x7f05c0109380 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009900 tx=0x7f05a802a410 comp rx=0 tx=0).stop 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2324287138 shutdown_connections 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2324287138 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c0108f20 0x7f05c0109380 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2324287138 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c01098c0 0x7f05c01155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2324287138 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c010e8d0 0x7f05c010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2324287138 >> 172.21.15.69:0/2324287138 conn(0x7f05c006e000 msgr2=0x7f05c006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2324287138 shutdown_connections 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.705+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2324287138 wait complete. 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05c5a5c640 1 Processor -- start 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05c5a5c640 1 -- start start 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05c5a5c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108f20 0x7f05c01a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05c5a5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c01098c0 0x7f05c01a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05c5a5c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c010e8d0 0x7f05c01a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05c5a5c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01182b0 con 0x7f05c010e8d0 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05c5a5c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0118130 con 0x7f05c0108f20 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05c5a5c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05c0118430 con 0x7f05c01098c0 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c010e8d0 0x7f05c01a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05bf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c010e8d0 0x7f05c01a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60704/0 (socket says 172.21.15.69:60704) 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05bf7fe640 1 -- 172.21.15.69:0/2729561279 learned_addr learned my addr 172.21.15.69:0/2729561279 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05beffd640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108f20 0x7f05c01a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.706+0000 7f05b7fff640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c01098c0 0x7f05c01a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05bf7fe640 1 -- 172.21.15.69:0/2729561279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c01098c0 msgr2=0x7f05c01a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05bf7fe640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c01098c0 0x7f05c01a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05bf7fe640 1 -- 172.21.15.69:0/2729561279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108f20 msgr2=0x7f05c01a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05bf7fe640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108f20 0x7f05c01a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05bf7fe640 1 -- 172.21.15.69:0/2729561279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c01a0f70 con 0x7f05c010e8d0 2024-08-02T17:53:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05bf7fe640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c010e8d0 0x7f05c01a07d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f05b800b970 tx=0x7f05b800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05c4a5a640 1 -- 172.21.15.69:0/2729561279 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8019070 con 0x7f05c010e8d0 2024-08-02T17:53:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05c4a5a640 1 -- 172.21.15.69:0/2729561279 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b800cb40 con 0x7f05c010e8d0 2024-08-02T17:53:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2729561279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c01a1260 con 0x7f05c010e8d0 2024-08-02T17:53:28.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05c4a5a640 1 -- 172.21.15.69:0/2729561279 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8007650 con 0x7f05c010e8d0 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.707+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2729561279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c01ad4b0 con 0x7f05c010e8d0 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.709+0000 7f05c4a5a640 1 -- 172.21.15.69:0/2729561279 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f05b801c070 con 0x7f05c010e8d0 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.709+0000 7f05c7e5a740 1 Processor -- start 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c7e5a740 1 -- start start 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 0x559ddd0583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd062c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 0x559ddd0655c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c7e5a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559ddcfbda40 con 0x559ddd058910 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c7e5a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ddcf559d0 con 0x559ddd0631d0 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c7e5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ddcfb9a00 con 0x559ddd057ff0 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd062c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd062c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60710/0 (socket says 172.21.15.69:60710) 2024-08-02T17:53:28.532 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 -- 172.21.15.69:0/1612295495 learned_addr learned my addr 172.21.15.69:0/1612295495 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05b7fff640 1 --2- 172.21.15.69:0/1612295495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 0x559ddd0583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 -- 172.21.15.69:0/1612295495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 msgr2=0x559ddd0583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 --2- 172.21.15.69:0/1612295495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 0x559ddd0583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 -- 172.21.15.69:0/1612295495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 msgr2=0x559ddd0655c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 --2- 172.21.15.69:0/1612295495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 0x559ddd0655c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 -- 172.21.15.69:0/1612295495 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ddcf5c650 con 0x559ddd058910 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c4a5a640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05ac0a2cf0 0x7f05ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05beffd640 1 --2- 172.21.15.69:0/1612295495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd062c90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f05b000ad60 tx=0x7f05b0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d9d61e138ba4f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05b7fff640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05ac0a2cf0 0x7f05ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.710+0000 7f05c4a5a640 1 -- 172.21.15.69:0/2729561279 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f05b80cecb0 con 0x7f05c010e8d0 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05b5ffb640 1 -- 172.21.15.69:0/1612295495 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b001c070 con 0x559ddd058910 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05b5ffb640 1 -- 172.21.15.69:0/1612295495 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b000ae50 con 0x559ddd058910 2024-08-02T17:53:28.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05b5ffb640 1 -- 172.21.15.69:0/1612295495 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0017750 con 0x559ddd058910 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 -- 172.21.15.69:0/1612295495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 msgr2=0x559ddd062c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05b7fff640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05ac0a2cf0 0x7f05ac0a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009230 tx=0x7f05a8000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/1612295495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd062c90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f05b000ad60 tx=0x7f05b0005f90 comp rx=0 tx=0).stop 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 -- 172.21.15.69:0/1612295495 shutdown_connections 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/1612295495 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd062c90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/1612295495 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 0x559ddd0655c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/1612295495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 0x559ddd0583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 -- 172.21.15.69:0/1612295495 >> 172.21.15.69:0/1612295495 conn(0x559ddd054580 msgr2=0x559ddd054960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 -- 172.21.15.69:0/1612295495 shutdown_connections 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.711+0000 7f05c7e5a740 1 -- 172.21.15.69:0/1612295495 wait complete. 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 1 -- start start 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 1 Processor -- start 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd057ff0 0x559ddd060010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd060550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd0631d0 0x559ddd060a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559ddd065da0 con 0x559ddd058910 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ddcf78b20 con 0x559ddd057ff0 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ddd065f20 con 0x559ddd0631d0 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05c7e5a740 10 client.0 _ll_register_callbacks cb 0x559ddcf76bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd060550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.712+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd060550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60712/0 (socket says 172.21.15.69:60712) 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.713+0000 7f05b7fff640 1 -- 172.21.15.69:0/3567865938 learned_addr learned my addr 172.21.15.69:0/3567865938 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.716+0000 7f05bf7fe640 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd0631d0 0x559ddd060a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.718+0000 7f05beffd640 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd057ff0 0x559ddd060010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.721+0000 7f05b7fff640 1 -- 172.21.15.69:0/3567865938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd0631d0 msgr2=0x559ddd060a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.721+0000 7f05b7fff640 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd0631d0 0x559ddd060a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.721+0000 7f05b7fff640 1 -- 172.21.15.69:0/3567865938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd057ff0 msgr2=0x559ddd060010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.721+0000 7f05b7fff640 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd057ff0 0x559ddd060010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.721+0000 7f05b7fff640 1 -- 172.21.15.69:0/3567865938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ddcd2f3c0 con 0x559ddd058910 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.722+0000 7f05b7fff640 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd060550 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f05a8033000 tx=0x7f05a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd653e4eae29915d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.724+0000 7f059effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.725+0000 7f059effd640 1 -- 172.21.15.69:0/3567865938 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8035040 con 0x559ddd058910 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.725+0000 7f059effd640 1 -- 172.21.15.69:0/3567865938 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f05a80037f0 con 0x559ddd058910 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.725+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3567865938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b000ae50 con 0x559ddd058910 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.729+0000 7f059effd640 1 -- 172.21.15.69:0/3567865938 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05a80039f0 con 0x559ddd058910 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.730+0000 7f05c7e5a740 10 client.17184 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.730+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3567865938 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ddd071910 con 0x559ddd058910 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f059effd640 1 -- 172.21.15.69:0/3567865938 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b000ae50 con 0x559ddd058910 2024-08-02T17:53:28.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f059effd640 1 -- 172.21.15.69:0/3567865938 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05a805c420 con 0x559ddd058910 2024-08-02T17:53:28.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f05c7e5a740 10 client.17184 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:28.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f05c7e5a740 10 client.17184 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:53:28.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f05c7e5a740 20 client.17184 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:28.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x559ddd072dd0 0x559ddd0751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f05c7e5a740 4 client.17184 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:28.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3567865938 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ddcfcafb0 con 0x559ddd072dd0 2024-08-02T17:53:28.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.746+0000 7f05beffd640 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x559ddd072dd0 0x559ddd0751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.750+0000 7f05beffd640 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x559ddd072dd0 0x559ddd0751c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e735c23b57f1d3c server_cookie=5ca704603f860b08 in_seq=0 out_seq=0 2024-08-02T17:53:28.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.750+0000 7f059effd640 10 client.17184 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:53:28.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.751+0000 7f059effd640 1 -- 172.21.15.69:0/3567865938 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559ddcfcafb0 con 0x559ddd072dd0 2024-08-02T17:53:28.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.751+0000 7f059effd640 10 client.17184 handle_command_reply: tid=0 2024-08-02T17:53:28.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.755+0000 7f05c7e5a740 1 client.17184 shutdown 2024-08-02T17:53:28.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.758+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3567865938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 msgr2=0x559ddd060550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.758+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd060550 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f05a8033000 tx=0x7f05a8009260 comp rx=0 tx=0).stop 2024-08-02T17:53:28.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3567865938 shutdown_connections 2024-08-02T17:53:28.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x559ddd072dd0 0x559ddd0751c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd058910 0x559ddd060550 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd0631d0 0x559ddd060a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3567865938 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd057ff0 0x559ddd060010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f059effd640 0 client.17184 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:53:28.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3567865938 >> 172.21.15.69:0/3567865938 conn(0x559ddd054580 msgr2=0x559ddd05a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:28.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3567865938 shutdown_connections 2024-08-02T17:53:28.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3567865938 wait complete. 2024-08-02T17:53:28.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.759+0000 7f05c7e5a740 20 client.17184 trim_cache size 0 max 16384 2024-08-02T17:53:28.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.785+0000 7f05c7e5a740 1 Processor -- start 2024-08-02T17:53:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.786+0000 7f05c7e5a740 1 -- start start 2024-08-02T17:53:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.786+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd057ff0 0x559ddd07aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.786+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd058910 0x559ddd07afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.786+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 0x559ddd052d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.787+0000 7f05c7e5a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01b7ec0 con 0x559ddd057ff0 2024-08-02T17:53:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.787+0000 7f05bf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 0x559ddd052d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.787+0000 7f05bf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 0x559ddd052d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:34244/0 (socket says 172.21.15.69:34244) 2024-08-02T17:53:28.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.787+0000 7f05bf7fe640 1 -- 172.21.15.69:0/2785489473 learned_addr learned my addr 172.21.15.69:0/2785489473 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:28.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.787+0000 7f05c7e5a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01b5790 con 0x559ddd0631d0 2024-08-02T17:53:28.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.787+0000 7f05c7e5a740 1 -- 172.21.15.69:0/2785489473 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01b35e0 con 0x559ddd058910 2024-08-02T17:53:28.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.788+0000 7f05beffd640 1 --2- 172.21.15.69:0/2785489473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd057ff0 0x559ddd07aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.789+0000 7f05beffd640 1 -- 172.21.15.69:0/2785489473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd058910 msgr2=0x559ddd07afd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:53:28.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.789+0000 7f05beffd640 1 --2- 172.21.15.69:0/2785489473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd058910 0x559ddd07afd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.789+0000 7f05beffd640 1 -- 172.21.15.69:0/2785489473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 msgr2=0x559ddd052d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.789+0000 7f05beffd640 1 --2- 172.21.15.69:0/2785489473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 0x559ddd052d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.789+0000 7f05beffd640 1 -- 172.21.15.69:0/2785489473 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ddd075530 con 0x559ddd057ff0 2024-08-02T17:53:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.789+0000 7f05beffd640 1 --2- 172.21.15.69:0/2785489473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd057ff0 0x559ddd07aa90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f05b001a040 tx=0x7f05b000acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2298a6621d44e51d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.790+0000 7f05b57fa640 1 -- 172.21.15.69:0/2785489473 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b001c040 con 0x559ddd057ff0 2024-08-02T17:53:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.790+0000 7f05b57fa640 1 -- 172.21.15.69:0/2785489473 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b0005070 con 0x559ddd057ff0 2024-08-02T17:53:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.790+0000 7f05b57fa640 1 -- 172.21.15.69:0/2785489473 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b002c980 con 0x559ddd057ff0 2024-08-02T17:53:28.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.793+0000 7f05c7e5a740 1 -- 172.21.15.69:0/2785489473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd057ff0 msgr2=0x559ddd07aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.793+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/2785489473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd057ff0 0x559ddd07aa90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f05b001a040 tx=0x7f05b000acf0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.798+0000 7f05c7e5a740 1 -- 172.21.15.69:0/2785489473 shutdown_connections 2024-08-02T17:53:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.798+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/2785489473 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd0631d0 0x559ddd052d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.798+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/2785489473 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd058910 0x559ddd07afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.798+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/2785489473 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd057ff0 0x559ddd07aa90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.798+0000 7f05c7e5a740 1 -- 172.21.15.69:0/2785489473 >> 172.21.15.69:0/2785489473 conn(0x559ddd054580 msgr2=0x559ddd05e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.800+0000 7f05c7e5a740 1 -- 172.21.15.69:0/2785489473 shutdown_connections 2024-08-02T17:53:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.802+0000 7f05c7e5a740 1 -- 172.21.15.69:0/2785489473 wait complete. 2024-08-02T17:53:28.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.802+0000 7f05c7e5a740 1 -- start start 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05c7e5a740 1 Processor -- start 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 0x559ddd06b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd058910 0x559ddd066a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05c7e5a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd0631d0 0x559ddd066f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd058910 0x559ddd066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05beffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 0x559ddd06b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05b7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd058910 0x559ddd066a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:34248/0 (socket says 172.21.15.69:34248) 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05b7fff640 1 -- 172.21.15.69:0/3525134065 learned_addr learned my addr 172.21.15.69:0/3525134065 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01b3c50 con 0x559ddd0631d0 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.803+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01b50c0 con 0x559ddd058910 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.804+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05c01b4320 con 0x559ddd057ff0 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.804+0000 7f05c7e5a740 10 client.0 _ll_register_callbacks cb 0x7f05a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.804+0000 7f05bf7fe640 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd0631d0 0x559ddd066f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.806+0000 7f05b7fff640 1 -- 172.21.15.69:0/3525134065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 msgr2=0x559ddd06b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.806+0000 7f05b7fff640 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 0x559ddd06b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.806+0000 7f05b7fff640 1 -- 172.21.15.69:0/3525134065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd0631d0 msgr2=0x559ddd066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.806+0000 7f05b7fff640 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd0631d0 0x559ddd066f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.806+0000 7f05b7fff640 1 -- 172.21.15.69:0/3525134065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ddd067580 con 0x559ddd058910 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.807+0000 7f05b7fff640 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd058910 0x559ddd066a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f05a805c9e0 tx=0x7f05a805cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2c5d35ce85b7b0b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.807+0000 7f059effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.811+0000 7f059effd640 1 -- 172.21.15.69:0/3525134065 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05a8035070 con 0x559ddd058910 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.811+0000 7f059effd640 1 -- 172.21.15.69:0/3525134065 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f05a8003d90 con 0x559ddd058910 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.811+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05b0005070 con 0x559ddd058910 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.811+0000 7f059effd640 1 -- 172.21.15.69:0/3525134065 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05a8003800 con 0x559ddd058910 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.811+0000 7f05c7e5a740 10 client.26729 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.811+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ddd0e6060 con 0x559ddd058910 2024-08-02T17:53:28.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.811+0000 7f059effd640 1 -- 172.21.15.69:0/3525134065 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a8004c50 con 0x559ddd058910 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.812+0000 7f059effd640 1 -- 172.21.15.69:0/3525134065 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05a8005520 con 0x559ddd058910 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.812+0000 7f05c7e5a740 10 client.26729 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.812+0000 7f05c7e5a740 10 client.26729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.812+0000 7f05c7e5a740 20 client.26729 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.812+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x559ddd0e67b0 0x559ddd0e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.812+0000 7f05c7e5a740 4 client.26729 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.812+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f05b0005070 con 0x559ddd0e67b0 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.814+0000 7f05beffd640 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x559ddd0e67b0 0x559ddd0e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.819+0000 7f059effd640 10 client.26729 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.819+0000 7f05beffd640 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x559ddd0e67b0 0x559ddd0e8bd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce18afa4b50756f1 server_cookie=d8f09ca7d5baaf94 in_seq=0 out_seq=0 2024-08-02T17:53:28.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.819+0000 7f059effd640 1 -- 172.21.15.69:0/3525134065 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f05b0005070 con 0x559ddd0e67b0 2024-08-02T17:53:28.550 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:53:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.819+0000 7f059effd640 10 client.26729 handle_command_reply: tid=0 2024-08-02T17:53:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.819+0000 7f05c7e5a740 1 client.26729 shutdown 2024-08-02T17:53:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd058910 msgr2=0x559ddd066a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd058910 0x559ddd066a00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f05a805c9e0 tx=0x7f05a805cc90 comp rx=0 tx=0).stop 2024-08-02T17:53:28.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 shutdown_connections 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x559ddd0e67b0 0x559ddd0e8bd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f059effd640 0 client.26729 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ddd0631d0 0x559ddd066f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ddd058910 0x559ddd066a00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 --2- 172.21.15.69:0/3525134065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ddd057ff0 0x559ddd06b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 >> 172.21.15.69:0/3525134065 conn(0x559ddd054580 msgr2=0x559ddd052d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 shutdown_connections 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 1 -- 172.21.15.69:0/3525134065 wait complete. 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.820+0000 7f05c7e5a740 20 client.26729 trim_cache size 0 max 16384 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05ac0a2cf0 msgr2=0x7f05ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05ac0a2cf0 0x7f05ac0a51b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009230 tx=0x7f05a8000bc0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c010e8d0 msgr2=0x7f05c01a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c010e8d0 0x7f05c01a07d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f05b800b970 tx=0x7f05b800be30 comp rx=0 tx=0).stop 2024-08-02T17:53:28.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2729561279 shutdown_connections 2024-08-02T17:53:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05c010e8d0 0x7f05c01a07d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05ac0a2cf0 0x7f05ac0a51b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05c01098c0 0x7f05c01a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 --2- 172.21.15.69:0/2729561279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05c0108f20 0x7f05c01a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.821+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2729561279 >> 172.21.15.69:0/2729561279 conn(0x7f05c006e000 msgr2=0x7f05c0072130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.822+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2729561279 shutdown_connections 2024-08-02T17:53:28.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:27.822+0000 7f05c5a5c640 1 -- 172.21.15.69:0/2729561279 wait complete. 2024-08-02T17:53:29.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:28 smithi083 ceph-mon[37031]: pgmap v216: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 16 MiB/s wr, 912 op/s 2024-08-02T17:53:29.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:28 smithi077 ceph-mon[37108]: pgmap v216: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 16 MiB/s wr, 912 op/s 2024-08-02T17:53:29.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:28 smithi069 ceph-mon[32746]: pgmap v216: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 16 MiB/s wr, 912 op/s 2024-08-02T17:53:29.411 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:53:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:53:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 380.0 s, 1009.4 tps, lat 99.954 ms stddev 137.495 2024-08-02T17:53:30.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.466+0000 7f3ad7480640 1 -- 172.21.15.69:0/400080113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 msgr2=0x7f3ad010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:30.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.466+0000 7f3ad7480640 1 --2- 172.21.15.69:0/400080113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 0x7f3ad010ff80 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009900 tx=0x7f3acc02a440 comp rx=0 tx=0).stop 2024-08-02T17:53:30.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.471+0000 7f3ad7480640 1 -- 172.21.15.69:0/400080113 shutdown_connections 2024-08-02T17:53:30.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.471+0000 7f3ad7480640 1 --2- 172.21.15.69:0/400080113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad011ae70 0x7f3ad011d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.471+0000 7f3ad7480640 1 --2- 172.21.15.69:0/400080113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad0109560 0x7f3ad01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.471+0000 7f3ad7480640 1 --2- 172.21.15.69:0/400080113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 0x7f3ad010ff80 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.471+0000 7f3ad7480640 1 -- 172.21.15.69:0/400080113 >> 172.21.15.69:0/400080113 conn(0x7f3ad006de70 msgr2=0x7f3ad006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:30.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.472+0000 7f3ad7480640 1 -- 172.21.15.69:0/400080113 shutdown_connections 2024-08-02T17:53:30.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.476+0000 7f3ad7480640 1 -- 172.21.15.69:0/400080113 wait complete. 2024-08-02T17:53:30.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.477+0000 7f3ad7480640 1 Processor -- start 2024-08-02T17:53:30.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.477+0000 7f3ad7480640 1 -- start start 2024-08-02T17:53:30.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad7480640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 0x7f3ad011a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad7480640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad0109560 0x7f3ad011a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad7480640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad011ae70 0x7f3ad0113620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad7480640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0120b40 con 0x7f3ad0109180 2024-08-02T17:53:30.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad7480640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01209c0 con 0x7f3ad011ae70 2024-08-02T17:53:30.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad7480640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0120cc0 con 0x7f3ad0109560 2024-08-02T17:53:30.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad521c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 0x7f3ad011a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad521c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 0x7f3ad011a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60738/0 (socket says 172.21.15.69:60738) 2024-08-02T17:53:30.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.478+0000 7f3ad521c640 1 -- 172.21.15.69:0/281614739 learned_addr learned my addr 172.21.15.69:0/281614739 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ad521c640 1 -- 172.21.15.69:0/281614739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad0109560 msgr2=0x7f3ad011a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ad521c640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad0109560 0x7f3ad011a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ad521c640 1 -- 172.21.15.69:0/281614739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad011ae70 msgr2=0x7f3ad0113620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ad521c640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad011ae70 0x7f3ad0113620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ad521c640 1 -- 172.21.15.69:0/281614739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad0113ee0 con 0x7f3ad0109180 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ad521c640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 0x7f3ad011a3b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3acc005ab0 tx=0x7f3acc005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ac67fc640 1 -- 172.21.15.69:0/281614739 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc040030 con 0x7f3ad0109180 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ac67fc640 1 -- 172.21.15.69:0/281614739 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3acc004a30 con 0x7f3ad0109180 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.479+0000 7f3ac67fc640 1 -- 172.21.15.69:0/281614739 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc03c860 con 0x7f3ad0109180 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.482+0000 7f3ad7480640 1 -- 172.21.15.69:0/281614739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad0114170 con 0x7f3ad0109180 2024-08-02T17:53:30.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.482+0000 7f3ad7480640 1 -- 172.21.15.69:0/281614739 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01180b0 con 0x7f3ad0109180 2024-08-02T17:53:30.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.487+0000 7f3ac67fc640 1 -- 172.21.15.69:0/281614739 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3acc043030 con 0x7f3ad0109180 2024-08-02T17:53:30.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.488+0000 7f3ac67fc640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3abc0a2e80 0x7f3abc0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.491+0000 7f3ad987e740 1 Processor -- start 2024-08-02T17:53:30.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.491+0000 7f3ac67fc640 1 -- 172.21.15.69:0/281614739 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f3acc0094e0 con 0x7f3ad0109180 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad4a1b640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3abc0a2e80 0x7f3abc0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad987e740 1 -- start start 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494952ff0 0x55d4949533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 0x55d49495dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d4949605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad521c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494952ff0 0x55d4949533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad521c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494952ff0 0x55d4949533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:34262/0 (socket says 172.21.15.69:34262) 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad521c640 1 -- 172.21.15.69:0/452210546 learned_addr learned my addr 172.21.15.69:0/452210546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad987e740 1 -- 172.21.15.69:0/452210546 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d4948b8a40 con 0x55d494953910 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.495+0000 7f3ad987e740 1 -- 172.21.15.69:0/452210546 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d4948509d0 con 0x55d494952ff0 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.496+0000 7f3ad987e740 1 -- 172.21.15.69:0/452210546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d4948b4a00 con 0x55d49495e1d0 2024-08-02T17:53:30.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.498+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d4949605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.499+0000 7f3ad5a1d640 1 -- 172.21.15.69:0/452210546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494952ff0 msgr2=0x55d4949533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.499+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494952ff0 0x55d4949533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.499+0000 7f3ad5a1d640 1 -- 172.21.15.69:0/452210546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 msgr2=0x55d49495dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.499+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 0x55d49495dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.499+0000 7f3ad5a1d640 1 -- 172.21.15.69:0/452210546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d494857650 con 0x55d49495e1d0 2024-08-02T17:53:30.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.501+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d4949605c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009d60 tx=0x7f3ac800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7262f3633327c131 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:30.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.501+0000 7f3abb7fe640 1 -- 172.21.15.69:0/452210546 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8004b20 con 0x55d49495e1d0 2024-08-02T17:53:30.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.501+0000 7f3abb7fe640 1 -- 172.21.15.69:0/452210546 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8003e50 con 0x55d49495e1d0 2024-08-02T17:53:30.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.501+0000 7f3abb7fe640 1 -- 172.21.15.69:0/452210546 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8004840 con 0x55d49495e1d0 2024-08-02T17:53:30.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.501+0000 7f3ad521c640 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494952ff0 0x55d4949533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:53:30.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.502+0000 7f3ad987e740 1 -- 172.21.15.69:0/452210546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 msgr2=0x55d4949605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:30.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.502+0000 7f3ad987e740 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d4949605c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8009d60 tx=0x7f3ac800e720 comp rx=0 tx=0).stop 2024-08-02T17:53:30.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.504+0000 7f3ad987e740 1 -- 172.21.15.69:0/452210546 shutdown_connections 2024-08-02T17:53:30.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.504+0000 7f3ad987e740 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 0x55d49495dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.504+0000 7f3ad987e740 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d4949605c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.504+0000 7f3ad987e740 1 --2- 172.21.15.69:0/452210546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494952ff0 0x55d4949533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.504+0000 7f3ad987e740 1 -- 172.21.15.69:0/452210546 >> 172.21.15.69:0/452210546 conn(0x55d49494f580 msgr2=0x55d49494f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:30.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.506+0000 7f3ad4a1b640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3abc0a2e80 0x7f3abc0a5340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0114c00 tx=0x7f3ac00091b0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:30.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.508+0000 7f3ad987e740 1 -- 172.21.15.69:0/452210546 shutdown_connections 2024-08-02T17:53:30.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.508+0000 7f3ad987e740 1 -- 172.21.15.69:0/452210546 wait complete. 2024-08-02T17:53:30.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.508+0000 7f3ad987e740 1 -- start start 2024-08-02T17:53:30.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.508+0000 7f3ad987e740 1 Processor -- start 2024-08-02T17:53:30.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494953910 0x55d49495b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d49495bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad987e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d494960da0 con 0x55d494952ff0 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad987e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d494873b20 con 0x55d494953910 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad987e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d494960f20 con 0x55d49495e1d0 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad987e740 10 client.0 _ll_register_callbacks cb 0x55d494871bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad5a1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d49495bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad5a1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d49495bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47730/0 (socket says 172.21.15.69:47730) 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.509+0000 7f3ad5a1d640 1 -- 172.21.15.69:0/4280785963 learned_addr learned my addr 172.21.15.69:0/4280785963 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:30.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.510+0000 7f3ad4a1b640 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494953910 0x55d49495b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.512+0000 7f3ad5a1d640 1 -- 172.21.15.69:0/4280785963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494953910 msgr2=0x55d49495b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:30.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.512+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494953910 0x55d49495b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.512+0000 7f3ad5a1d640 1 -- 172.21.15.69:0/4280785963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 msgr2=0x55d49495b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:30.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.512+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.512+0000 7f3ad5a1d640 1 -- 172.21.15.69:0/4280785963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d49462a3c0 con 0x55d49495e1d0 2024-08-02T17:53:30.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.517+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d49495bb80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80046f0 tx=0x7f3ac8004720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=592eb9b030886099 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:30.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.519+0000 7f3ab8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:30.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.519+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/4280785963 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8028060 con 0x55d49495e1d0 2024-08-02T17:53:30.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.519+0000 7f3ad987e740 1 -- 172.21.15.69:0/4280785963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac8003e50 con 0x55d49495e1d0 2024-08-02T17:53:30.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.519+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/4280785963 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f3ac8004c50 con 0x55d49495e1d0 2024-08-02T17:53:30.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.521+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/4280785963 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ac802f050 con 0x55d49495e1d0 2024-08-02T17:53:30.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.522+0000 7f3ad987e740 10 client.26605 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:30.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.522+0000 7f3ad987e740 1 -- 172.21.15.69:0/4280785963 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d49496c950 con 0x55d49495e1d0 2024-08-02T17:53:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.527+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/4280785963 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac8003e50 con 0x55d49495e1d0 2024-08-02T17:53:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.527+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/4280785963 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ac8018470 con 0x55d49495e1d0 2024-08-02T17:53:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.528+0000 7f3ad987e740 10 client.26605 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:30.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.528+0000 7f3ad987e740 10 client.26605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:30.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.528+0000 7f3ad987e740 20 client.26605 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:30.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.528+0000 7f3ad987e740 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d49496dd00 0x55d4949700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.528+0000 7f3ad987e740 4 client.26605 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:30.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.528+0000 7f3ad987e740 1 -- 172.21.15.69:0/4280785963 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4948c5fb0 con 0x55d49496dd00 2024-08-02T17:53:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.528+0000 7f3ad521c640 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d49496dd00 0x55d4949700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.531+0000 7f3ad521c640 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d49496dd00 0x55d4949700f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a87edbd31f2b3e04 server_cookie=2274f6dad021d14d in_seq=0 out_seq=0 2024-08-02T17:53:30.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.531+0000 7f3ab8ff9640 10 client.26605 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.534+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/4280785963 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d4948c5fb0 con 0x55d49496dd00 2024-08-02T17:53:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.534+0000 7f3ab8ff9640 10 client.26605 handle_command_reply: tid=0 2024-08-02T17:53:30.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.534+0000 7f3ad987e740 1 client.26605 shutdown 2024-08-02T17:53:30.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.537+0000 7f3ad987e740 1 -- 172.21.15.69:0/4280785963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 msgr2=0x55d49495bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:30.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.537+0000 7f3ad987e740 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d49495bb80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80046f0 tx=0x7f3ac8004720 comp rx=0 tx=0).stop 2024-08-02T17:53:30.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.542+0000 7f3ad987e740 1 -- 172.21.15.69:0/4280785963 shutdown_connections 2024-08-02T17:53:30.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.542+0000 7f3ad987e740 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d49496dd00 0x55d4949700f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.542+0000 7f3ad987e740 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d49495e1d0 0x55d49495bb80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.542+0000 7f3ad987e740 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d494953910 0x55d49495b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.542+0000 7f3ad987e740 1 --2- 172.21.15.69:0/4280785963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.542+0000 7f3ad987e740 1 -- 172.21.15.69:0/4280785963 >> 172.21.15.69:0/4280785963 conn(0x55d49494f580 msgr2=0x55d4949552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:30.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.542+0000 7f3ab8ff9640 0 client.26605 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.545+0000 7f3ad987e740 1 -- 172.21.15.69:0/4280785963 shutdown_connections 2024-08-02T17:53:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.546+0000 7f3ad987e740 1 -- 172.21.15.69:0/4280785963 wait complete. 2024-08-02T17:53:30.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.546+0000 7f3ad987e740 20 client.26605 trim_cache size 0 max 16384 2024-08-02T17:53:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.571+0000 7f3ad987e740 1 Processor -- start 2024-08-02T17:53:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.571+0000 7f3ad987e740 1 -- start start 2024-08-02T17:53:30.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.571+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494953910 0x55d49495a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad521c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad521c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60766/0 (socket says 172.21.15.69:60766) 2024-08-02T17:53:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad521c640 1 -- 172.21.15.69:0/1044773903 learned_addr learned my addr 172.21.15.69:0/1044773903 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad987e740 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d494958330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad987e740 1 -- 172.21.15.69:0/1044773903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01c8f50 con 0x55d494952ff0 2024-08-02T17:53:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad987e740 1 -- 172.21.15.69:0/1044773903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01c6820 con 0x55d49495e1d0 2024-08-02T17:53:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad987e740 1 -- 172.21.15.69:0/1044773903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01c4670 con 0x55d494953910 2024-08-02T17:53:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.572+0000 7f3ad4a1b640 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494953910 0x55d49495a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.573+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d494958330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.573+0000 7f3ad521c640 1 -- 172.21.15.69:0/1044773903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494953910 msgr2=0x55d49495a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.573+0000 7f3ad521c640 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494953910 0x55d49495a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.573+0000 7f3ad521c640 1 -- 172.21.15.69:0/1044773903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 msgr2=0x55d494958330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.573+0000 7f3ad521c640 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d494958330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.573+0000 7f3ad521c640 1 -- 172.21.15.69:0/1044773903 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d494954500 con 0x55d494952ff0 2024-08-02T17:53:30.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.573+0000 7f3ad521c640 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495a9c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0bb680 tx=0x7f3acc0f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3f9bff63379567 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.574+0000 7f3abaffd640 1 -- 172.21.15.69:0/1044773903 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0c4360 con 0x55d494952ff0 2024-08-02T17:53:30.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.574+0000 7f3abaffd640 1 -- 172.21.15.69:0/1044773903 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3acc005630 con 0x55d494952ff0 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.574+0000 7f3abaffd640 1 -- 172.21.15.69:0/1044773903 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc003f10 con 0x55d494952ff0 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.575+0000 7f3ad987e740 1 -- 172.21.15.69:0/1044773903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 msgr2=0x55d49495a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.575+0000 7f3ad987e740 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495a9c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0bb680 tx=0x7f3acc0f2000 comp rx=0 tx=0).stop 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.575+0000 7f3ad987e740 1 -- 172.21.15.69:0/1044773903 shutdown_connections 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.575+0000 7f3ad987e740 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d494958330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.575+0000 7f3ad987e740 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494953910 0x55d49495a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.575+0000 7f3ad987e740 1 --2- 172.21.15.69:0/1044773903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494952ff0 0x55d49495a9c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.575+0000 7f3ad987e740 1 -- 172.21.15.69:0/1044773903 >> 172.21.15.69:0/1044773903 conn(0x55d49494f580 msgr2=0x55d4949599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.575+0000 7f3ad987e740 1 -- 172.21.15.69:0/1044773903 shutdown_connections 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.576+0000 7f3ad987e740 1 -- 172.21.15.69:0/1044773903 wait complete. 2024-08-02T17:53:30.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.576+0000 7f3ad987e740 1 -- start start 2024-08-02T17:53:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.577+0000 7f3ad987e740 1 Processor -- start 2024-08-02T17:53:30.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494952ff0 0x55d4949701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 0x55d494970720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad987e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d4949d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad987e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01c4ce0 con 0x55d494953910 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad987e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01c6150 con 0x55d49495e1d0 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad987e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01c53b0 con 0x55d494952ff0 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad987e740 10 client.0 _ll_register_callbacks cb 0x7f3ac8009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad521c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494952ff0 0x55d4949701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad521c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494952ff0 0x55d4949701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47734/0 (socket says 172.21.15.69:47734) 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad521c640 1 -- 172.21.15.69:0/40378887 learned_addr learned my addr 172.21.15.69:0/40378887 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:30.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.578+0000 7f3ad4a1b640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 0x55d494970720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d4949d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad521c640 1 -- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 msgr2=0x55d4949d6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad521c640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d4949d6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad521c640 1 -- 172.21.15.69:0/40378887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 msgr2=0x55d494970720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad521c640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 0x55d494970720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad521c640 1 -- 172.21.15.69:0/40378887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4949d6b70 con 0x55d494952ff0 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad4a1b640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 0x55d494970720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad5a1d640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d4949d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ad521c640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494952ff0 0x55d4949701e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0c3e30 tx=0x7f3acc02a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65d5b4a14b364a3d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:30.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.579+0000 7f3ab8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.580+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/40378887 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0b7d60 con 0x55d494952ff0 2024-08-02T17:53:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.581+0000 7f3ad987e740 1 -- 172.21.15.69:0/40378887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3acc005630 con 0x55d494952ff0 2024-08-02T17:53:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.581+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/40378887 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f3acc003f10 con 0x55d494952ff0 2024-08-02T17:53:30.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.582+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/40378887 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3acc0b7f00 con 0x55d494952ff0 2024-08-02T17:53:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.582+0000 7f3ad987e740 10 client.26611 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:30.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.582+0000 7f3ad987e740 1 -- 172.21.15.69:0/40378887 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4949d7e70 con 0x55d494952ff0 2024-08-02T17:53:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.584+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/40378887 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3acc0ecba0 con 0x55d494952ff0 2024-08-02T17:53:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.584+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/40378887 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3acc0c3ed0 con 0x55d494952ff0 2024-08-02T17:53:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.584+0000 7f3ad987e740 10 client.26611 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.584+0000 7f3ad987e740 10 client.26611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.584+0000 7f3ad987e740 20 client.26611 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.584+0000 7f3ad987e740 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4949d84a0 0x55d4949da8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:30.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.584+0000 7f3ad987e740 4 client.26611 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:53:30.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.584+0000 7f3ad987e740 1 -- 172.21.15.69:0/40378887 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f3acc005630 con 0x55d4949d84a0 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.585+0000 7f3ad4a1b640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4949d84a0 0x55d4949da8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad4a1b640 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4949d84a0 0x55d4949da8c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6741022a1d22068 server_cookie=2bf264f3bdba7943 in_seq=0 out_seq=0 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ab8ff9640 10 client.26611 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ab8ff9640 1 -- 172.21.15.69:0/40378887 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f3acc005630 con 0x55d4949d84a0 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ab8ff9640 10 client.26611 handle_command_reply: tid=0 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 client.26611 shutdown 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 -- 172.21.15.69:0/40378887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494952ff0 msgr2=0x55d4949701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494952ff0 0x55d4949701e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0c3e30 tx=0x7f3acc02a440 comp rx=0 tx=0).stop 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 -- 172.21.15.69:0/40378887 shutdown_connections 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d494953910 0x55d494970720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d4949d84a0 0x55d4949da8c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d49495e1d0 0x55d4949d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 --2- 172.21.15.69:0/40378887 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d494952ff0 0x55d4949701e0 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0c3e30 tx=0x7f3acc02a440 comp rx=0 tx=0).stop 2024-08-02T17:53:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.586+0000 7f3ad987e740 1 -- 172.21.15.69:0/40378887 >> 172.21.15.69:0/40378887 conn(0x55d49494f580 msgr2=0x55d49495a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.587+0000 7f3ab8ff9640 0 client.26611 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.587+0000 7f3ad987e740 1 -- 172.21.15.69:0/40378887 shutdown_connections 2024-08-02T17:53:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.587+0000 7f3ad987e740 1 -- 172.21.15.69:0/40378887 wait complete. 2024-08-02T17:53:30.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.587+0000 7f3ad987e740 20 client.26611 trim_cache size 0 max 16384 2024-08-02T17:53:30.590 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 -- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3abc0a2e80 msgr2=0x7f3abc0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3abc0a2e80 0x7f3abc0a5340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0114c00 tx=0x7f3ac00091b0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 -- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 msgr2=0x7f3ad011a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 0x7f3ad011a3b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3acc005ab0 tx=0x7f3acc005230 comp rx=0 tx=0).stop 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 -- 172.21.15.69:0/281614739 shutdown_connections 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3abc0a2e80 0x7f3abc0a5340 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ad011ae70 0x7f3ad0113620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ad0109560 0x7f3ad011a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 --2- 172.21.15.69:0/281614739 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ad0109180 0x7f3ad011a3b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 -- 172.21.15.69:0/281614739 >> 172.21.15.69:0/281614739 conn(0x7f3ad006de70 msgr2=0x7f3ad0072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 -- 172.21.15.69:0/281614739 shutdown_connections 2024-08-02T17:53:30.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:30.589+0000 7f3ad7480640 1 -- 172.21.15.69:0/281614739 wait complete. 2024-08-02T17:53:30.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:30 smithi069 ceph-mon[32746]: pgmap v217: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.9 MiB/s wr, 650 op/s 2024-08-02T17:53:31.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:30 smithi083 ceph-mon[37031]: pgmap v217: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.9 MiB/s wr, 650 op/s 2024-08-02T17:53:31.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:30 smithi077 ceph-mon[37108]: pgmap v217: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 9.9 MiB/s wr, 650 op/s 2024-08-02T17:53:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.369+0000 7f04d392b640 1 -- 172.21.15.69:0/2243850829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 msgr2=0x7f04cc106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.369+0000 7f04d392b640 1 --2- 172.21.15.69:0/2243850829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 0x7f04cc106060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009900 tx=0x7f04c002a4b0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.374+0000 7f04d392b640 1 -- 172.21.15.69:0/2243850829 shutdown_connections 2024-08-02T17:53:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.374+0000 7f04d392b640 1 --2- 172.21.15.69:0/2243850829 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc1010e0 0x7f04cc1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.374+0000 7f04d392b640 1 --2- 172.21.15.69:0/2243850829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 0x7f04cc106060 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.374+0000 7f04d392b640 1 --2- 172.21.15.69:0/2243850829 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04cc1007c0 0x7f04cc100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.374+0000 7f04d392b640 1 -- 172.21.15.69:0/2243850829 >> 172.21.15.69:0/2243850829 conn(0x7f04cc0fc3e0 msgr2=0x7f04cc0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.376+0000 7f04d392b640 1 -- 172.21.15.69:0/2243850829 shutdown_connections 2024-08-02T17:53:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.377+0000 7f04d392b640 1 -- 172.21.15.69:0/2243850829 wait complete. 2024-08-02T17:53:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.378+0000 7f04d392b640 1 Processor -- start 2024-08-02T17:53:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.378+0000 7f04d392b640 1 -- start start 2024-08-02T17:53:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.378+0000 7f04d392b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc1007c0 0x7f04cc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.378+0000 7f04d392b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04cc1010e0 0x7f04cc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.378+0000 7f04d392b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 0x7f04cc1a2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.378+0000 7f04d392b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc1181e0 con 0x7f04cc1007c0 2024-08-02T17:53:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.378+0000 7f04d392b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc118060 con 0x7f04cc1010e0 2024-08-02T17:53:31.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.378+0000 7f04d392b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc118360 con 0x7f04cc105bf0 2024-08-02T17:53:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.379+0000 7f04d16c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc1007c0 0x7f04cc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.379+0000 7f04d16c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc1007c0 0x7f04cc19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60794/0 (socket says 172.21.15.69:60794) 2024-08-02T17:53:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.379+0000 7f04d16c7640 1 -- 172.21.15.69:0/2775577237 learned_addr learned my addr 172.21.15.69:0/2775577237 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:31.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.379+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 0x7f04cc1a2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.383 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.379+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04cc1010e0 0x7f04cc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.383+0000 7f04d1ec8640 1 -- 172.21.15.69:0/2775577237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04cc1010e0 msgr2=0x7f04cc19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.383+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04cc1010e0 0x7f04cc19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.383+0000 7f04d1ec8640 1 -- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc1007c0 msgr2=0x7f04cc19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.383+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc1007c0 0x7f04cc19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.385 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.383+0000 7f04d1ec8640 1 -- 172.21.15.69:0/2775577237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc1a36e0 con 0x7f04cc105bf0 2024-08-02T17:53:31.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.384+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 0x7f04cc1a2fb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009850 tx=0x7f04c0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.385+0000 7f04ca7fc640 1 -- 172.21.15.69:0/2775577237 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c003f070 con 0x7f04cc105bf0 2024-08-02T17:53:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.386+0000 7f04ca7fc640 1 -- 172.21.15.69:0/2775577237 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c0031c20 con 0x7f04cc105bf0 2024-08-02T17:53:31.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.386+0000 7f04ca7fc640 1 -- 172.21.15.69:0/2775577237 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0035030 con 0x7f04cc105bf0 2024-08-02T17:53:31.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.387+0000 7f04d392b640 1 -- 172.21.15.69:0/2775577237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04cc1a3970 con 0x7f04cc105bf0 2024-08-02T17:53:31.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.387+0000 7f04d392b640 1 -- 172.21.15.69:0/2775577237 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04cc1a3de0 con 0x7f04cc105bf0 2024-08-02T17:53:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.392+0000 7f04ca7fc640 1 -- 172.21.15.69:0/2775577237 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f04c0042070 con 0x7f04cc105bf0 2024-08-02T17:53:31.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.393+0000 7f04ca7fc640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f04b40a2e80 0x7f04b40a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.393+0000 7f04ca7fc640 1 -- 172.21.15.69:0/2775577237 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f04c00bace0 con 0x7f04cc105bf0 2024-08-02T17:53:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.394+0000 7f04d16c7640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f04b40a2e80 0x7f04b40a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.400+0000 7f04d667e740 1 Processor -- start 2024-08-02T17:53:31.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.401+0000 7f04d667e740 1 -- start start 2024-08-02T17:53:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.401+0000 7f04d667e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba8cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.401+0000 7f04d667e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba97430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.401+0000 7f04d16c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba8cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.401+0000 7f04d16c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba8cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47768/0 (socket says 172.21.15.69:47768) 2024-08-02T17:53:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.401+0000 7f04d16c7640 1 -- 172.21.15.69:0/4156020833 learned_addr learned my addr 172.21.15.69:0/4156020833 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:31.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.401+0000 7f04d667e740 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 0x557adba99d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d667e740 1 -- 172.21.15.69:0/4156020833 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557adb9f09b0 con 0x557adba97970 2024-08-02T17:53:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d667e740 1 -- 172.21.15.69:0/4156020833 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557adb998800 con 0x557adba8d0d0 2024-08-02T17:53:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d667e740 1 -- 172.21.15.69:0/4156020833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557adb9ec970 con 0x557adba8c7b0 2024-08-02T17:53:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba97430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d16c7640 1 -- 172.21.15.69:0/4156020833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 msgr2=0x557adba97430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d16c7640 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba97430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d16c7640 1 -- 172.21.15.69:0/4156020833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 msgr2=0x557adba99d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:31.404 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d16c7640 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 0x557adba99d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.402+0000 7f04d16c7640 1 -- 172.21.15.69:0/4156020833 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557adb704210 con 0x557adba8c7b0 2024-08-02T17:53:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.403+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba97430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:53:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.403+0000 7f04d16c7640 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba8cb90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04bc010010 tx=0x7f04bc004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7a99b3f617b2870 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:31.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.403+0000 7f04d16c7640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f04b40a2e80 0x7f04b40a5340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04bc005c40 tx=0x7f04bc007620 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.406+0000 7f04b37fe640 1 -- 172.21.15.69:0/4156020833 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc020070 con 0x557adba8c7b0 2024-08-02T17:53:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.406+0000 7f04b37fe640 1 -- 172.21.15.69:0/4156020833 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04bc00cd60 con 0x557adba8c7b0 2024-08-02T17:53:31.407 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.406+0000 7f04b37fe640 1 -- 172.21.15.69:0/4156020833 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc01c440 con 0x557adba8c7b0 2024-08-02T17:53:31.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.407+0000 7f04d667e740 1 -- 172.21.15.69:0/4156020833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 msgr2=0x557adba8cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.407+0000 7f04d667e740 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba8cb90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04bc010010 tx=0x7f04bc004ab0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.408+0000 7f04d667e740 1 -- 172.21.15.69:0/4156020833 shutdown_connections 2024-08-02T17:53:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.408+0000 7f04d667e740 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 0x557adba99d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.408+0000 7f04d667e740 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba97430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.408+0000 7f04d667e740 1 --2- 172.21.15.69:0/4156020833 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba8cb90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.408+0000 7f04d667e740 1 -- 172.21.15.69:0/4156020833 >> 172.21.15.69:0/4156020833 conn(0x557adba88d20 msgr2=0x557adba89100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.409+0000 7f04d667e740 1 -- 172.21.15.69:0/4156020833 shutdown_connections 2024-08-02T17:53:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.409+0000 7f04d667e740 1 -- 172.21.15.69:0/4156020833 wait complete. 2024-08-02T17:53:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.409+0000 7f04d667e740 1 -- start start 2024-08-02T17:53:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.409+0000 7f04d667e740 1 Processor -- start 2024-08-02T17:53:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d667e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8c7b0 0x557adba948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d667e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba94e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d667e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba97970 0x557adba95370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d667e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557adba9a540 con 0x557adba8c7b0 2024-08-02T17:53:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d667e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557adb9ac500 con 0x557adba8d0d0 2024-08-02T17:53:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d667e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557adba9a6c0 con 0x557adba97970 2024-08-02T17:53:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d667e740 10 client.0 _ll_register_callbacks cb 0x557adb703a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d1ec8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba97970 0x557adba95370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d1ec8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba97970 0x557adba95370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47770/0 (socket says 172.21.15.69:47770) 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d1ec8640 1 -- 172.21.15.69:0/2173644645 learned_addr learned my addr 172.21.15.69:0/2173644645 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba94e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.410+0000 7f04d16c7640 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8c7b0 0x557adba948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04d1ec8640 1 -- 172.21.15.69:0/2173644645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 msgr2=0x557adba94e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba94e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04d1ec8640 1 -- 172.21.15.69:0/2173644645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8c7b0 msgr2=0x557adba948f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8c7b0 0x557adba948f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04d1ec8640 1 -- 172.21.15.69:0/2173644645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557adb7d0620 con 0x557adba97970 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba97970 0x557adba95370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f04c00bc940 tx=0x7f04c00b7740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82318f5bfeb1d5af server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04b0ff9640 1 -- 172.21.15.69:0/2173644645 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00b9670 con 0x557adba97970 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04b0ff9640 1 -- 172.21.15.69:0/2173644645 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f04c00bc5a0 con 0x557adba97970 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04d667e740 1 -- 172.21.15.69:0/2173644645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04bc00cd60 con 0x557adba97970 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04b0ff9640 1 -- 172.21.15.69:0/2173644645 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04c00b9810 con 0x557adba97970 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.411+0000 7f04d667e740 10 client.26623 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04d667e740 1 -- 172.21.15.69:0/2173644645 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557adba9a570 con 0x557adba97970 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04b0ff9640 1 -- 172.21.15.69:0/2173644645 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c00c5800 con 0x557adba97970 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04b0ff9640 1 -- 172.21.15.69:0/2173644645 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f04c004c060 con 0x557adba97970 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04d667e740 10 client.26623 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04d667e740 10 client.26623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04d667e740 20 client.26623 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04d667e740 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557adbaa7770 0x557adbaa9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04d667e740 4 client.26623 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.412+0000 7f04d667e740 1 -- 172.21.15.69:0/2173644645 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f04bc00cd60 con 0x557adbaa7770 2024-08-02T17:53:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.413+0000 7f04d16c7640 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557adbaa7770 0x557adbaa9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.413+0000 7f04d16c7640 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557adbaa7770 0x557adbaa9b90 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1d372309acacd5 server_cookie=f378630f7b815cdb in_seq=0 out_seq=0 2024-08-02T17:53:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.413+0000 7f04b0ff9640 10 client.26623 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.414+0000 7f04b0ff9640 1 -- 172.21.15.69:0/2173644645 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f04bc00cd60 con 0x557adbaa7770 2024-08-02T17:53:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.414+0000 7f04b0ff9640 10 client.26623 handle_command_reply: tid=0 2024-08-02T17:53:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.416+0000 7f04d667e740 1 client.26623 shutdown 2024-08-02T17:53:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 -- 172.21.15.69:0/2173644645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba97970 msgr2=0x557adba95370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba97970 0x557adba95370 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f04c00bc940 tx=0x7f04c00b7740 comp rx=0 tx=0).stop 2024-08-02T17:53:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 -- 172.21.15.69:0/2173644645 shutdown_connections 2024-08-02T17:53:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557adbaa7770 0x557adbaa9b90 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba97970 0x557adba95370 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8d0d0 0x557adba94e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 --2- 172.21.15.69:0/2173644645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8c7b0 0x557adba948f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04b0ff9640 0 client.26623 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 -- 172.21.15.69:0/2173644645 >> 172.21.15.69:0/2173644645 conn(0x557adba88d20 msgr2=0x557adba87e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 -- 172.21.15.69:0/2173644645 shutdown_connections 2024-08-02T17:53:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.417+0000 7f04d667e740 1 -- 172.21.15.69:0/2173644645 wait complete. 2024-08-02T17:53:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.418+0000 7f04d667e740 20 client.26623 trim_cache size 0 max 16384 2024-08-02T17:53:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.447+0000 7f04d667e740 1 Processor -- start 2024-08-02T17:53:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.447+0000 7f04d667e740 1 -- start start 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.447+0000 7f04d667e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba88670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.447+0000 7f04d667e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8d0d0 0x557adba8f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d667e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba97970 0x557adba931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d16c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba88670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d0ec6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8d0d0 0x557adba8f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d16c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba88670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47774/0 (socket says 172.21.15.69:47774) 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d0ec6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8d0d0 0x557adba8f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60818/0 (socket says 172.21.15.69:60818) 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d16c7640 1 -- 172.21.15.69:0/3793917275 learned_addr learned my addr 172.21.15.69:0/3793917275 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d667e740 1 -- 172.21.15.69:0/3793917275 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04b80129c0 con 0x557adba8d0d0 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d667e740 1 -- 172.21.15.69:0/3793917275 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f04b8010290 con 0x557adba97970 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d667e740 1 -- 172.21.15.69:0/3793917275 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04b800e0e0 con 0x557adba8c7b0 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/3793917275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba97970 0x557adba931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d0ec6640 1 -- 172.21.15.69:0/3793917275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 msgr2=0x557adba88670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/3793917275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba88670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d0ec6640 1 -- 172.21.15.69:0/3793917275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba97970 msgr2=0x557adba931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/3793917275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba97970 0x557adba931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.448+0000 7f04d0ec6640 1 -- 172.21.15.69:0/3793917275 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557adba899b0 con 0x557adba8d0d0 2024-08-02T17:53:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.449+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/3793917275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8d0d0 0x557adba8f220 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x557adba95f30 tx=0x7f04c400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c810a2aa5bb762 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.449+0000 7f04b2ffd640 1 -- 172.21.15.69:0/3793917275 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c400be20 con 0x557adba8d0d0 2024-08-02T17:53:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.449+0000 7f04b2ffd640 1 -- 172.21.15.69:0/3793917275 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c4005020 con 0x557adba8d0d0 2024-08-02T17:53:31.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.449+0000 7f04b2ffd640 1 -- 172.21.15.69:0/3793917275 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4004b90 con 0x557adba8d0d0 2024-08-02T17:53:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 -- 172.21.15.69:0/3793917275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8d0d0 msgr2=0x557adba8f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 --2- 172.21.15.69:0/3793917275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8d0d0 0x557adba8f220 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x557adba95f30 tx=0x7f04c400bb10 comp rx=0 tx=0).stop 2024-08-02T17:53:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 -- 172.21.15.69:0/3793917275 shutdown_connections 2024-08-02T17:53:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 --2- 172.21.15.69:0/3793917275 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba8d0d0 0x557adba8f220 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 --2- 172.21.15.69:0/3793917275 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba97970 0x557adba931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 --2- 172.21.15.69:0/3793917275 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8c7b0 0x557adba88670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 -- 172.21.15.69:0/3793917275 >> 172.21.15.69:0/3793917275 conn(0x557adba88d20 msgr2=0x557adba96860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 -- 172.21.15.69:0/3793917275 shutdown_connections 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.450+0000 7f04d667e740 1 -- 172.21.15.69:0/3793917275 wait complete. 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.451+0000 7f04d667e740 1 -- start start 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.451+0000 7f04d667e740 1 Processor -- start 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d667e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8c7b0 0x557adba9f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d667e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8d0d0 0x557adba9fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d667e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 0x557adba9c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d0ec6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8d0d0 0x557adba9fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d0ec6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8d0d0 0x557adba9fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47776/0 (socket says 172.21.15.69:47776) 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d16c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8c7b0 0x557adba9f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d16c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8c7b0 0x557adba9f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:34316/0 (socket says 172.21.15.69:34316) 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d0ec6640 1 -- 172.21.15.69:0/3534798757 learned_addr learned my addr 172.21.15.69:0/3534798757 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f04b800e750 con 0x557adba97970 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f04b800fbc0 con 0x557adba8c7b0 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.452+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f04b800ee20 con 0x557adba8d0d0 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04d667e740 10 client.0 _ll_register_callbacks cb 0x7f04c0047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04d1ec8640 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 0x557adba9c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04d0ec6640 1 -- 172.21.15.69:0/3534798757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8c7b0 msgr2=0x557adba9f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8c7b0 0x557adba9f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04d0ec6640 1 -- 172.21.15.69:0/3534798757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 msgr2=0x557adba9c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 0x557adba9c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04d0ec6640 1 -- 172.21.15.69:0/3534798757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557adba9ce90 con 0x557adba8d0d0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04d0ec6640 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8d0d0 0x557adba9fec0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04c4003e10 tx=0x7f04c4003e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31644e251afc3ba8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.453+0000 7f04b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.454+0000 7f04b0ff9640 1 -- 172.21.15.69:0/3534798757 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c4020030 con 0x557adba8d0d0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.454+0000 7f04b0ff9640 1 -- 172.21.15.69:0/3534798757 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f04c4005460 con 0x557adba8d0d0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.454+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c4005020 con 0x557adba8d0d0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.454+0000 7f04b0ff9640 1 -- 172.21.15.69:0/3534798757 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04c4030050 con 0x557adba8d0d0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.454+0000 7f04d667e740 10 client.26629 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.454+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557adbb12a40 con 0x557adba8d0d0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.454+0000 7f04b0ff9640 1 -- 172.21.15.69:0/3534798757 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c40056a0 con 0x557adba8d0d0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.455+0000 7f04b0ff9640 1 -- 172.21.15.69:0/3534798757 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f04c401ca40 con 0x557adba8d0d0 2024-08-02T17:53:31.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.455+0000 7f04d667e740 10 client.26629 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.455+0000 7f04d667e740 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.455+0000 7f04d667e740 20 client.26629 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.455+0000 7f04d667e740 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557adbb130c0 0x557adbb154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.455+0000 7f04d667e740 4 client.26629 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:53:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.455+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f04c4005020 con 0x557adbb130c0 2024-08-02T17:53:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.455+0000 7f04d16c7640 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557adbb130c0 0x557adbb154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.456+0000 7f04d16c7640 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557adbb130c0 0x557adbb154e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50614f8a07d5f4f0 server_cookie=7eb1d96da1adee00 in_seq=0 out_seq=0 2024-08-02T17:53:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.456+0000 7f04b0ff9640 10 client.26629 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:31.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04b0ff9640 1 -- 172.21.15.69:0/3534798757 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f04c4005020 con 0x557adbb130c0 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04b0ff9640 10 client.26629 handle_command_reply: tid=0 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 client.26629 shutdown 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8d0d0 msgr2=0x557adba9fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8d0d0 0x557adba9fec0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04c4003e10 tx=0x7f04c4003e40 comp rx=0 tx=0).stop 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 shutdown_connections 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557adba97970 0x557adba9c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557adbb130c0 0x557adbb154e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557adba8d0d0 0x557adba9fec0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 --2- 172.21.15.69:0/3534798757 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557adba8c7b0 0x557adba9f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 >> 172.21.15.69:0/3534798757 conn(0x557adba88d20 msgr2=0x557adba95f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04b0ff9640 0 client.26629 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:31.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.550+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 shutdown_connections 2024-08-02T17:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.551+0000 7f04d667e740 1 -- 172.21.15.69:0/3534798757 wait complete. 2024-08-02T17:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.551+0000 7f04d667e740 20 client.26629 trim_cache size 0 max 16384 2024-08-02T17:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:53:31.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.553+0000 7f04d392b640 1 -- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f04b40a2e80 msgr2=0x7f04b40a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.553+0000 7f04d392b640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f04b40a2e80 0x7f04b40a5340 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04bc005c40 tx=0x7f04bc007620 comp rx=0 tx=0).stop 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.553+0000 7f04d392b640 1 -- 172.21.15.69:0/2775577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 msgr2=0x7f04cc1a2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.553+0000 7f04d392b640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 0x7f04cc1a2fb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009850 tx=0x7f04c0003b90 comp rx=0 tx=0).stop 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.554+0000 7f04d392b640 1 -- 172.21.15.69:0/2775577237 shutdown_connections 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.554+0000 7f04d392b640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f04b40a2e80 0x7f04b40a5340 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.554+0000 7f04d392b640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f04cc105bf0 0x7f04cc1a2fb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.554+0000 7f04d392b640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f04cc1010e0 0x7f04cc19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.554+0000 7f04d392b640 1 --2- 172.21.15.69:0/2775577237 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f04cc1007c0 0x7f04cc19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.554+0000 7f04d392b640 1 -- 172.21.15.69:0/2775577237 >> 172.21.15.69:0/2775577237 conn(0x7f04cc0fc3e0 msgr2=0x7f04cc10f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.554+0000 7f04d392b640 1 -- 172.21.15.69:0/2775577237 shutdown_connections 2024-08-02T17:53:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:31.554+0000 7f04d392b640 1 -- 172.21.15.69:0/2775577237 wait complete. 2024-08-02T17:53:32.295 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:53:32.295 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:53:32.708 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:53:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:32 smithi083 ceph-mon[37031]: pgmap v218: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.4 MiB/s wr, 626 op/s 2024-08-02T17:53:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:32 smithi077 ceph-mon[37108]: pgmap v218: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.4 MiB/s wr, 626 op/s 2024-08-02T17:53:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:32 smithi069 ceph-mon[32746]: pgmap v218: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 9.4 MiB/s wr, 626 op/s 2024-08-02T17:53:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 385.0 s, 1130.0 tps, lat 89.247 ms stddev 108.680 2024-08-02T17:53:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:34 smithi083 ceph-mon[37031]: pgmap v219: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 830 op/s 2024-08-02T17:53:35.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:34 smithi077 ceph-mon[37108]: pgmap v219: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 830 op/s 2024-08-02T17:53:35.114 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:34 smithi069 ceph-mon[32746]: pgmap v219: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 15 MiB/s wr, 830 op/s 2024-08-02T17:53:35.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 -- 172.21.15.69:0/1739002898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 msgr2=0x7f6e30100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1739002898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 0x7f6e30100a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c009900 tx=0x7f6e2c02a410 comp rx=0 tx=0).stop 2024-08-02T17:53:35.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 -- 172.21.15.69:0/1739002898 shutdown_connections 2024-08-02T17:53:35.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1739002898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e3010e220 0x7f6e30110610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1739002898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e30101020 0x7f6e3010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1739002898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 0x7f6e30100a50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 -- 172.21.15.69:0/1739002898 >> 172.21.15.69:0/1739002898 conn(0x7f6e300fc500 msgr2=0x7f6e300fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:35.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 -- 172.21.15.69:0/1739002898 shutdown_connections 2024-08-02T17:53:35.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.394+0000 7f6e3848e640 1 -- 172.21.15.69:0/1739002898 wait complete. 2024-08-02T17:53:35.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3848e640 1 Processor -- start 2024-08-02T17:53:35.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3848e640 1 -- start start 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3848e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 0x7f6e3019a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3848e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e30101020 0x7f6e3019a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3848e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e3010e220 0x7f6e3019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3848e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30113ed0 con 0x7f6e30101020 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3848e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30113d50 con 0x7f6e3010e220 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3848e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e30114050 con 0x7f6e30100670 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e35a29640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e30101020 0x7f6e3019a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3622a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 0x7f6e3019a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e35a29640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e30101020 0x7f6e3019a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41468/0 (socket says 172.21.15.69:41468) 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e3622a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 0x7f6e3019a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42756/0 (socket says 172.21.15.69:42756) 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e35a29640 1 -- 172.21.15.69:0/1138544002 learned_addr learned my addr 172.21.15.69:0/1138544002 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.396+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e3010e220 0x7f6e3019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.397+0000 7f6e35a29640 1 -- 172.21.15.69:0/1138544002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 msgr2=0x7f6e3019a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.397+0000 7f6e35a29640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 0x7f6e3019a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.397+0000 7f6e35a29640 1 -- 172.21.15.69:0/1138544002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e3010e220 msgr2=0x7f6e3019ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.397+0000 7f6e35a29640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e3010e220 0x7f6e3019ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.397+0000 7f6e35a29640 1 -- 172.21.15.69:0/1138544002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3019f300 con 0x7f6e30101020 2024-08-02T17:53:35.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.397+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e3010e220 0x7f6e3019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:53:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.397+0000 7f6e35a29640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e30101020 0x7f6e3019a890 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000d950 tx=0x7f6e2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.398+0000 7f6e277fe640 1 -- 172.21.15.69:0/1138544002 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20017070 con 0x7f6e30101020 2024-08-02T17:53:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.398+0000 7f6e3848e640 1 -- 172.21.15.69:0/1138544002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3019f5f0 con 0x7f6e30101020 2024-08-02T17:53:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.398+0000 7f6e3848e640 1 -- 172.21.15.69:0/1138544002 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e301a6ed0 con 0x7f6e30101020 2024-08-02T17:53:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.399+0000 7f6e277fe640 1 -- 172.21.15.69:0/1138544002 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e20009c30 con 0x7f6e30101020 2024-08-02T17:53:35.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.399+0000 7f6e277fe640 1 -- 172.21.15.69:0/1138544002 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e20020060 con 0x7f6e30101020 2024-08-02T17:53:35.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.402+0000 7f6e3a88c740 1 Processor -- start 2024-08-02T17:53:35.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.403+0000 7f6e3a88c740 1 -- start start 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.403+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 0x558edae0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.403+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0eb20 0x558edae18ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.403+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 0x558edae1b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.403+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558edadd1e40 con 0x558edae193e0 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.403+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558edad737f0 con 0x558edae0e200 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.403+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558edad1afb0 con 0x558edae0eb20 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0eb20 0x558edae18ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0eb20 0x558edae18ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42768/0 (socket says 172.21.15.69:42768) 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 -- 172.21.15.69:0/2022025789 learned_addr learned my addr 172.21.15.69:0/2022025789 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 -- 172.21.15.69:0/2022025789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 msgr2=0x558edae0e5e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/2022025789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 0x558edae0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 --2- 172.21.15.69:0/2022025789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 0x558edae0e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 -- 172.21.15.69:0/2022025789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 msgr2=0x558edae1b7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:53:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 --2- 172.21.15.69:0/2022025789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 0x558edae1b7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 -- 172.21.15.69:0/2022025789 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edaae56a0 con 0x558edae0eb20 2024-08-02T17:53:35.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.404+0000 7f6e3622a640 1 --2- 172.21.15.69:0/2022025789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0eb20 0x558edae18ea0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c02a930 tx=0x7f6e2c0315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66b369d87f0813ac server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.405+0000 7f6e277fe640 1 -- 172.21.15.69:0/1138544002 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6e2000f6b0 con 0x7f6e30101020 2024-08-02T17:53:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.405+0000 7f6e24ff9640 1 -- 172.21.15.69:0/2022025789 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c035040 con 0x558edae0eb20 2024-08-02T17:53:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.405+0000 7f6e24ff9640 1 -- 172.21.15.69:0/2022025789 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c005040 con 0x558edae0eb20 2024-08-02T17:53:35.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.405+0000 7f6e24ff9640 1 -- 172.21.15.69:0/2022025789 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c0319b0 con 0x558edae0eb20 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.406+0000 7f6e277fe640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e1c0a2d70 0x7f6e1c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.406+0000 7f6e277fe640 1 -- 172.21.15.69:0/1138544002 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f6e200cf2d0 con 0x7f6e30101020 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.406+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e1c0a2d70 0x7f6e1c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.406+0000 7f6e3a88c740 1 -- 172.21.15.69:0/2022025789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0eb20 msgr2=0x558edae18ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.406+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/2022025789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0eb20 0x558edae18ea0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c02a930 tx=0x7f6e2c0315b0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.407+0000 7f6e3a88c740 1 -- 172.21.15.69:0/2022025789 shutdown_connections 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.407+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e1c0a2d70 0x7f6e1c0a5230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e30102850 tx=0x7f6e28009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.407+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/2022025789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 0x558edae1b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.407+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/2022025789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0eb20 0x558edae18ea0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.407+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/2022025789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 0x558edae0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.407+0000 7f6e3a88c740 1 -- 172.21.15.69:0/2022025789 >> 172.21.15.69:0/2022025789 conn(0x558edae0a790 msgr2=0x558edae0ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:35.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.408+0000 7f6e3a88c740 1 -- 172.21.15.69:0/2022025789 shutdown_connections 2024-08-02T17:53:35.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.408+0000 7f6e3a88c740 1 -- 172.21.15.69:0/2022025789 wait complete. 2024-08-02T17:53:35.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.408+0000 7f6e3a88c740 1 -- start start 2024-08-02T17:53:35.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.408+0000 7f6e3a88c740 1 Processor -- start 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 0x558edae162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae16820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae193e0 0x558edae16d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e35a29640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae16820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e35a29640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae16820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41472/0 (socket says 172.21.15.69:41472) 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e35a29640 1 -- 172.21.15.69:0/441386850 learned_addr learned my addr 172.21.15.69:0/441386850 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558edad2ecb0 con 0x558edae0eb20 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558edad6f7b0 con 0x558edae0e200 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558edae1bfb0 con 0x558edae193e0 2024-08-02T17:53:35.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.409+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae193e0 0x558edae16d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e3a88c740 10 client.0 _ll_register_callbacks cb 0x558edaa82ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e3622a640 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 0x558edae162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e35a29640 1 -- 172.21.15.69:0/441386850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae193e0 msgr2=0x558edae16d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e35a29640 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae193e0 0x558edae16d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e35a29640 1 -- 172.21.15.69:0/441386850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 msgr2=0x558edae162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e35a29640 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 0x558edae162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e35a29640 1 -- 172.21.15.69:0/441386850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edad2dbc0 con 0x558edae0eb20 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae193e0 0x558edae16d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e35a29640 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae16820 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6e2009b370 tx=0x7f6e20009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386ad5b77a3f6ab6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.410+0000 7f6e19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.411+0000 7f6e19ffb640 1 -- 172.21.15.69:0/441386850 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2009b3e0 con 0x558edae0eb20 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.411+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e2c005040 con 0x558edae0eb20 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.411+0000 7f6e19ffb640 1 -- 172.21.15.69:0/441386850 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f6e2009a830 con 0x558edae0eb20 2024-08-02T17:53:35.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.411+0000 7f6e19ffb640 1 -- 172.21.15.69:0/441386850 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6e2009f790 con 0x558edae0eb20 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.411+0000 7f6e3a88c740 10 client.17223 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.411+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558edae27ef0 con 0x558edae0eb20 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.411+0000 7f6e19ffb640 1 -- 172.21.15.69:0/441386850 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e2c005040 con 0x558edae0eb20 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.412+0000 7f6e19ffb640 1 -- 172.21.15.69:0/441386850 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6e2009a2a0 con 0x558edae0eb20 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.412+0000 7f6e3a88c740 10 client.17223 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.412+0000 7f6e3a88c740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.412+0000 7f6e3a88c740 20 client.17223 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.412+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558edae297a0 0x558edae2bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.412+0000 7f6e3a88c740 4 client.17223 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.412+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558edad80d60 con 0x558edae297a0 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.413+0000 7f6e3622a640 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558edae297a0 0x558edae2bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.413+0000 7f6e3622a640 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558edae297a0 0x558edae2bbc0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=725acd3505432ddd server_cookie=e1ff96c51e0c1de6 in_seq=0 out_seq=0 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.413+0000 7f6e19ffb640 10 client.17223 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:35.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e19ffb640 1 -- 172.21.15.69:0/441386850 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558edad80d60 con 0x558edae297a0 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e19ffb640 10 client.17223 handle_command_reply: tid=0 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 client.17223 shutdown 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 msgr2=0x558edae16820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae16820 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f6e2009b370 tx=0x7f6e20009de0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 shutdown_connections 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae16820 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558edae297a0 0x558edae2bbc0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae193e0 0x558edae16d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/441386850 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0e200 0x558edae162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.414+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 >> 172.21.15.69:0/441386850 conn(0x558edae0a790 msgr2=0x558edae10470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.415+0000 7f6e19ffb640 0 client.17223 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.415+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 shutdown_connections 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.415+0000 7f6e3a88c740 1 -- 172.21.15.69:0/441386850 wait complete. 2024-08-02T17:53:35.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.415+0000 7f6e3a88c740 20 client.17223 trim_cache size 0 max 16384 2024-08-02T17:53:35.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.448+0000 7f6e3a88c740 1 Processor -- start 2024-08-02T17:53:35.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.449+0000 7f6e3a88c740 1 -- start start 2024-08-02T17:53:35.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.449+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 0x558edae10360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.449+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0eb20 0x558edae31470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.450+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 0x558edae319b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.450+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e140127e0 con 0x558edae193e0 2024-08-02T17:53:35.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.450+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e140100b0 con 0x558edae0eb20 2024-08-02T17:53:35.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.450+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400dfb0 con 0x558edae0e200 2024-08-02T17:53:35.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.456+0000 7f6e35a29640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0eb20 0x558edae31470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.457+0000 7f6e36a2b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 0x558edae319b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.457+0000 7f6e36a2b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 0x558edae319b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41480/0 (socket says 172.21.15.69:41480) 2024-08-02T17:53:35.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.457+0000 7f6e36a2b640 1 -- 172.21.15.69:0/3067522103 learned_addr learned my addr 172.21.15.69:0/3067522103 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:35.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.457+0000 7f6e3622a640 1 --2- 172.21.15.69:0/3067522103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 0x558edae10360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.460+0000 7f6e35a29640 1 -- 172.21.15.69:0/3067522103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 msgr2=0x558edae10360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:35.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.460+0000 7f6e35a29640 1 --2- 172.21.15.69:0/3067522103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 0x558edae10360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.460+0000 7f6e35a29640 1 -- 172.21.15.69:0/3067522103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 msgr2=0x558edae319b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:35.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.460+0000 7f6e35a29640 1 --2- 172.21.15.69:0/3067522103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 0x558edae319b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.460+0000 7f6e35a29640 1 -- 172.21.15.69:0/3067522103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558edae15290 con 0x558edae0eb20 2024-08-02T17:53:35.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.463+0000 7f6e35a29640 1 --2- 172.21.15.69:0/3067522103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0eb20 0x558edae31470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e20010ef0 tx=0x7f6e20000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a23795215575a59 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.469+0000 7f6e1bfff640 1 -- 172.21.15.69:0/3067522103 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200a4050 con 0x558edae0eb20 2024-08-02T17:53:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.469+0000 7f6e1bfff640 1 -- 172.21.15.69:0/3067522103 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e20097a50 con 0x558edae0eb20 2024-08-02T17:53:35.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.469+0000 7f6e1bfff640 1 -- 172.21.15.69:0/3067522103 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e200a0ce0 con 0x558edae0eb20 2024-08-02T17:53:35.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.470+0000 7f6e3a88c740 1 -- 172.21.15.69:0/3067522103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0eb20 msgr2=0x558edae31470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.470+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/3067522103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0eb20 0x558edae31470 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6e20010ef0 tx=0x7f6e20000bc0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.476+0000 7f6e3a88c740 1 -- 172.21.15.69:0/3067522103 shutdown_connections 2024-08-02T17:53:35.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.476+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/3067522103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae193e0 0x558edae319b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.476+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/3067522103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae0eb20 0x558edae31470 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.476+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/3067522103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 0x558edae10360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.476+0000 7f6e3a88c740 1 -- 172.21.15.69:0/3067522103 >> 172.21.15.69:0/3067522103 conn(0x558edae0a790 msgr2=0x558edae0fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:35.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.480+0000 7f6e3a88c740 1 -- 172.21.15.69:0/3067522103 shutdown_connections 2024-08-02T17:53:35.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.483+0000 7f6e3a88c740 1 -- 172.21.15.69:0/3067522103 wait complete. 2024-08-02T17:53:35.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.483+0000 7f6e3a88c740 1 -- start start 2024-08-02T17:53:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.483+0000 7f6e3a88c740 1 Processor -- start 2024-08-02T17:53:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.484+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 0x558edae1c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.484+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae1cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.484+0000 7f6e3a88c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae193e0 0x558edae9ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.484+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400e570 con 0x558edae0eb20 2024-08-02T17:53:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.484+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400f9e0 con 0x558edae193e0 2024-08-02T17:53:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.484+0000 7f6e3a88c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1400ec40 con 0x558edae0e200 2024-08-02T17:53:35.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.484+0000 7f6e3a88c740 10 client.0 _ll_register_callbacks cb 0x7f6e20095f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:35.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.485+0000 7f6e36a2b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae193e0 0x558edae9ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.485+0000 7f6e36a2b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae193e0 0x558edae9ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60742/0 (socket says 172.21.15.69:60742) 2024-08-02T17:53:35.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.485+0000 7f6e36a2b640 1 -- 172.21.15.69:0/1001780205 learned_addr learned my addr 172.21.15.69:0/1001780205 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:35.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.487+0000 7f6e35a29640 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae1cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.487+0000 7f6e3622a640 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 0x558edae1c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.488+0000 7f6e36a2b640 1 -- 172.21.15.69:0/1001780205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 msgr2=0x558edae1c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:35.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.488+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 0x558edae1c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.488+0000 7f6e36a2b640 1 -- 172.21.15.69:0/1001780205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 msgr2=0x558edae1cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:35.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.488+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae1cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.488+0000 7f6e36a2b640 1 -- 172.21.15.69:0/1001780205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558edaeac190 con 0x558edae193e0 2024-08-02T17:53:35.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.490+0000 7f6e36a2b640 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae193e0 0x558edae9ac40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6e280206b0 tx=0x7f6e28004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14fbc4ace66bb82f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:35.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.490+0000 7f6e19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:53:35.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.492+0000 7f6e19ffb640 1 -- 172.21.15.69:0/1001780205 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2802e060 con 0x558edae193e0 2024-08-02T17:53:35.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.493+0000 7f6e3a88c740 1 -- 172.21.15.69:0/1001780205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e20097a50 con 0x558edae193e0 2024-08-02T17:53:35.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.493+0000 7f6e19ffb640 1 -- 172.21.15.69:0/1001780205 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f6e2801e5c0 con 0x558edae193e0 2024-08-02T17:53:35.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.493+0000 7f6e19ffb640 1 -- 172.21.15.69:0/1001780205 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6e28035050 con 0x558edae193e0 2024-08-02T17:53:35.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.495+0000 7f6e3a88c740 10 client.26768 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:35.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.495+0000 7f6e3a88c740 1 -- 172.21.15.69:0/1001780205 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558edaead3e0 con 0x558edae193e0 2024-08-02T17:53:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.497+0000 7f6e19ffb640 1 -- 172.21.15.69:0/1001780205 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e20097a50 con 0x558edae193e0 2024-08-02T17:53:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.497+0000 7f6e19ffb640 1 -- 172.21.15.69:0/1001780205 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6e2801ba80 con 0x558edae193e0 2024-08-02T17:53:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.497+0000 7f6e3a88c740 10 client.26768 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.497+0000 7f6e3a88c740 10 client.26768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.497+0000 7f6e3a88c740 20 client.26768 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.497+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558edaeadc90 0x558edaeb00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.498+0000 7f6e3a88c740 4 client.26768 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:53:35.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.498+0000 7f6e3a88c740 1 -- 172.21.15.69:0/1001780205 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558edad7b270 con 0x558edaeadc90 2024-08-02T17:53:35.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.505+0000 7f6e3622a640 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558edaeadc90 0x558edaeb00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:35.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.513+0000 7f6e19ffb640 10 client.26768 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:35.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.513+0000 7f6e3622a640 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558edaeadc90 0x558edaeb00b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49edf030a0b191f3 server_cookie=70e1d91b33634ed6 in_seq=0 out_seq=0 2024-08-02T17:53:35.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.515+0000 7f6e19ffb640 1 -- 172.21.15.69:0/1001780205 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558edad7b270 con 0x558edaeadc90 2024-08-02T17:53:35.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.515+0000 7f6e19ffb640 10 client.26768 handle_command_reply: tid=0 2024-08-02T17:53:35.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.515+0000 7f6e3a88c740 1 client.26768 shutdown 2024-08-02T17:53:35.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.515+0000 7f6e3a88c740 1 -- 172.21.15.69:0/1001780205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae193e0 msgr2=0x558edae9ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.515+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae193e0 0x558edae9ac40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6e280206b0 tx=0x7f6e28004a40 comp rx=0 tx=0).stop 2024-08-02T17:53:35.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.516+0000 7f6e3a88c740 1 -- 172.21.15.69:0/1001780205 shutdown_connections 2024-08-02T17:53:35.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.516+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558edae0eb20 0x558edae1cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.516+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558edaeadc90 0x558edaeb00b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.516+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558edae193e0 0x558edae9ac40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.516+0000 7f6e3a88c740 1 --2- 172.21.15.69:0/1001780205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558edae0e200 0x558edae1c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.516+0000 7f6e3a88c740 1 -- 172.21.15.69:0/1001780205 >> 172.21.15.69:0/1001780205 conn(0x558edae0a790 msgr2=0x558edae098f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:35.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.516+0000 7f6e19ffb640 0 client.26768 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:35.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.519+0000 7f6e3a88c740 1 -- 172.21.15.69:0/1001780205 shutdown_connections 2024-08-02T17:53:35.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.520+0000 7f6e3a88c740 1 -- 172.21.15.69:0/1001780205 wait complete. 2024-08-02T17:53:35.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.520+0000 7f6e3a88c740 20 client.26768 trim_cache size 0 max 16384 2024-08-02T17:53:35.522 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:53:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:53:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "953e4372-8445-4b00-90bd-6c4c09507980", 2024-08-02T17:53:35.522 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:53:35.522 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:53:35.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 -- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e1c0a2d70 msgr2=0x7f6e1c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e1c0a2d70 0x7f6e1c0a5230 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e30102850 tx=0x7f6e28009500 comp rx=0 tx=0).stop 2024-08-02T17:53:35.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 -- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e30101020 msgr2=0x7f6e3019a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:35.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e30101020 0x7f6e3019a890 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000d950 tx=0x7f6e2000de10 comp rx=0 tx=0).stop 2024-08-02T17:53:35.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 -- 172.21.15.69:0/1138544002 shutdown_connections 2024-08-02T17:53:35.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e30101020 0x7f6e3019a890 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e1c0a2d70 0x7f6e1c0a5230 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e3010e220 0x7f6e3019ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 --2- 172.21.15.69:0/1138544002 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e30100670 0x7f6e3019a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:35.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.521+0000 7f6e3848e640 1 -- 172.21.15.69:0/1138544002 >> 172.21.15.69:0/1138544002 conn(0x7f6e300fc500 msgr2=0x7f6e300fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:35.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.522+0000 7f6e3848e640 1 -- 172.21.15.69:0/1138544002 shutdown_connections 2024-08-02T17:53:35.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:35.522+0000 7f6e3848e640 1 -- 172.21.15.69:0/1138544002 wait complete. 2024-08-02T17:53:36.298 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:53:36.735 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:53:37.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:36 smithi069 ceph-mon[32746]: pgmap v220: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 639 op/s 2024-08-02T17:53:37.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:36 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:53:37.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:36 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:53:37.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:36 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:53:37.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:36 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:53:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:36 smithi077 ceph-mon[37108]: pgmap v220: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 639 op/s 2024-08-02T17:53:37.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:36 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:53:37.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:36 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:53:37.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:36 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:53:37.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:36 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:53:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:36 smithi083 ceph-mon[37031]: pgmap v220: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 12 MiB/s wr, 639 op/s 2024-08-02T17:53:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:36 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:53:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:36 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:53:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:36 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:53:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:36 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:53:37.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.310+0000 7f379aae8640 1 -- 172.21.15.69:0/3379594626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3794109180 msgr2=0x7f379410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.310+0000 7f379aae8640 1 --2- 172.21.15.69:0/3379594626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3794109180 0x7f379410ff80 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f378c01acb0 tx=0x7f378c03b900 comp rx=0 tx=0).stop 2024-08-02T17:53:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.314+0000 7f379aae8640 1 -- 172.21.15.69:0/3379594626 shutdown_connections 2024-08-02T17:53:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.314+0000 7f379aae8640 1 --2- 172.21.15.69:0/3379594626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379411ae70 0x7f379411d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.314+0000 7f379aae8640 1 --2- 172.21.15.69:0/3379594626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109560 0x7f37941099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.314+0000 7f379aae8640 1 --2- 172.21.15.69:0/3379594626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3794109180 0x7f379410ff80 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.314+0000 7f379aae8640 1 -- 172.21.15.69:0/3379594626 >> 172.21.15.69:0/3379594626 conn(0x7f379406de70 msgr2=0x7f379406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:37.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.314+0000 7f379aae8640 1 -- 172.21.15.69:0/3379594626 shutdown_connections 2024-08-02T17:53:37.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.315+0000 7f379aae8640 1 -- 172.21.15.69:0/3379594626 wait complete. 2024-08-02T17:53:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.316+0000 7f379aae8640 1 Processor -- start 2024-08-02T17:53:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.316+0000 7f379aae8640 1 -- start start 2024-08-02T17:53:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.316+0000 7f379aae8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109180 0x7f379411a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.316+0000 7f379aae8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3794109560 0x7f3794116630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.316+0000 7f379aae8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379411ae70 0x7f3794116b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.316+0000 7f379aae8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3794113050 con 0x7f3794109560 2024-08-02T17:53:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.316+0000 7f379aae8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3794112ed0 con 0x7f3794109180 2024-08-02T17:53:37.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.316+0000 7f379aae8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37941131d0 con 0x7f379411ae70 2024-08-02T17:53:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.317+0000 7f3798884640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109180 0x7f379411a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.317+0000 7f3798884640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109180 0x7f379411a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60762/0 (socket says 172.21.15.69:60762) 2024-08-02T17:53:37.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.317+0000 7f3798884640 1 -- 172.21.15.69:0/2004145536 learned_addr learned my addr 172.21.15.69:0/2004145536 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.318+0000 7f3798884640 1 -- 172.21.15.69:0/2004145536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379411ae70 msgr2=0x7f3794116b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:53:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.318+0000 7f3798884640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379411ae70 0x7f3794116b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.318+0000 7f3798884640 1 -- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3794109560 msgr2=0x7f3794116630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.318+0000 7f3798884640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3794109560 0x7f3794116630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.318+0000 7f3798884640 1 -- 172.21.15.69:0/2004145536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3794117270 con 0x7f3794109180 2024-08-02T17:53:37.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.318+0000 7f3798884640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109180 0x7f379411a590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f378c03be70 tx=0x7f378c0406c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:37.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.322+0000 7f3791ffb640 1 -- 172.21.15.69:0/2004145536 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c045070 con 0x7f3794109180 2024-08-02T17:53:37.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.322+0000 7f3791ffb640 1 -- 172.21.15.69:0/2004145536 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f378c003e50 con 0x7f3794109180 2024-08-02T17:53:37.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.322+0000 7f379aae8640 1 -- 172.21.15.69:0/2004145536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37941aaea0 con 0x7f3794109180 2024-08-02T17:53:37.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.322+0000 7f3791ffb640 1 -- 172.21.15.69:0/2004145536 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c004430 con 0x7f3794109180 2024-08-02T17:53:37.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.322+0000 7f379aae8640 1 -- 172.21.15.69:0/2004145536 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37941ab240 con 0x7f3794109180 2024-08-02T17:53:37.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.326+0000 7f379cee6740 1 Processor -- start 2024-08-02T17:53:37.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.326+0000 7f379cee6740 1 -- start start 2024-08-02T17:53:37.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f379cee6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f379cee6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4b30 0x55588e5aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f379cee6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5af3f0 0x55588e5b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f379cee6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55588e509980 con 0x55588e5a4210 2024-08-02T17:53:37.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f379cee6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55588e4a1a70 con 0x55588e5a4b30 2024-08-02T17:53:37.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f379cee6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55588e505940 con 0x55588e5af3f0 2024-08-02T17:53:37.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f3799085640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f3799085640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5a45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41530/0 (socket says 172.21.15.69:41530) 2024-08-02T17:53:37.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.327+0000 7f3799085640 1 -- 172.21.15.69:0/103846418 learned_addr learned my addr 172.21.15.69:0/103846418 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:37.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.329+0000 7f3798884640 1 --2- 172.21.15.69:0/103846418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4b30 0x55588e5aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.330+0000 7f3799085640 1 -- 172.21.15.69:0/103846418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5af3f0 msgr2=0x55588e5b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:37.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.330+0000 7f3799085640 1 --2- 172.21.15.69:0/103846418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5af3f0 0x55588e5b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.330+0000 7f3799085640 1 -- 172.21.15.69:0/103846418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4b30 msgr2=0x55588e5aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:37.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.330+0000 7f3799085640 1 --2- 172.21.15.69:0/103846418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4b30 0x55588e5aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.330+0000 7f3799085640 1 -- 172.21.15.69:0/103846418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55588e5046e0 con 0x55588e5a4210 2024-08-02T17:53:37.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.332+0000 7f3791ffb640 1 -- 172.21.15.69:0/2004145536 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f378c048020 con 0x7f3794109180 2024-08-02T17:53:37.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.338+0000 7f3791ffb640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f377c0a2f00 0x7f377c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.338+0000 7f3791ffb640 1 -- 172.21.15.69:0/2004145536 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f378c009040 con 0x7f3794109180 2024-08-02T17:53:37.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.340+0000 7f3798884640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f377c0a2f00 0x7f377c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.343+0000 7f3798884640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f377c0a2f00 0x7f377c0a53c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55588e5a1600 tx=0x7f378c03b900 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.346+0000 7f3799085640 1 --2- 172.21.15.69:0/103846418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5a45f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f378402da00 tx=0x7f378402dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b195f1660fe1bcd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.347+0000 7f377affd640 1 -- 172.21.15.69:0/103846418 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784038030 con 0x55588e5a4210 2024-08-02T17:53:37.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.347+0000 7f377affd640 1 -- 172.21.15.69:0/103846418 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3784005020 con 0x55588e5a4210 2024-08-02T17:53:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.348+0000 7f379cee6740 1 -- 172.21.15.69:0/103846418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 msgr2=0x55588e5a45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:37.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.348+0000 7f379cee6740 1 --2- 172.21.15.69:0/103846418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5a45f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f378402da00 tx=0x7f378402dec0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.350+0000 7f379cee6740 1 -- 172.21.15.69:0/103846418 shutdown_connections 2024-08-02T17:53:37.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.350+0000 7f379cee6740 1 --2- 172.21.15.69:0/103846418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5af3f0 0x55588e5b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.350+0000 7f379cee6740 1 --2- 172.21.15.69:0/103846418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4b30 0x55588e5aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.350+0000 7f379cee6740 1 --2- 172.21.15.69:0/103846418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5a45f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.350+0000 7f379cee6740 1 -- 172.21.15.69:0/103846418 >> 172.21.15.69:0/103846418 conn(0x55588e5a07a0 msgr2=0x55588e5a0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:37.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.350+0000 7f379cee6740 1 -- 172.21.15.69:0/103846418 shutdown_connections 2024-08-02T17:53:37.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.350+0000 7f379cee6740 1 -- 172.21.15.69:0/103846418 wait complete. 2024-08-02T17:53:37.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.351+0000 7f379cee6740 1 -- start start 2024-08-02T17:53:37.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.351+0000 7f379cee6740 1 Processor -- start 2024-08-02T17:53:37.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.351+0000 7f379cee6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5ac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f379cee6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f379cee6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 0x55588e5acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f379cee6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55588e5b2060 con 0x55588e5af3f0 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f379cee6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55588e4c4aa0 con 0x55588e5a4210 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f379cee6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55588e5b21e0 con 0x55588e5a4b30 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f379cee6740 10 client.0 _ll_register_callbacks cb 0x55588e4b3e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f3793fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f3793fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42840/0 (socket says 172.21.15.69:42840) 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.352+0000 7f3793fff640 1 -- 172.21.15.69:0/1809153718 learned_addr learned my addr 172.21.15.69:0/1809153718 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.353+0000 7f3793fff640 1 -- 172.21.15.69:0/1809153718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 msgr2=0x55588e5ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.353+0000 7f3793fff640 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.353+0000 7f3793fff640 1 -- 172.21.15.69:0/1809153718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 msgr2=0x55588e5acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.353+0000 7f3793fff640 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 0x55588e5acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.353+0000 7f3793fff640 1 -- 172.21.15.69:0/1809153718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55588e4a8650 con 0x55588e5a4b30 2024-08-02T17:53:37.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.353+0000 7f3793fff640 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5ac7d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3788009e60 tx=0x7f378800b4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6e88e67e01c3177 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:37.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.354+0000 7f3773fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:37.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.354+0000 7f3773fff640 1 -- 172.21.15.69:0/1809153718 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788015020 con 0x55588e5a4b30 2024-08-02T17:53:37.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.354+0000 7f379cee6740 1 -- 172.21.15.69:0/1809153718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55588e516ef0 con 0x55588e5a4b30 2024-08-02T17:53:37.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.354+0000 7f3773fff640 1 -- 172.21.15.69:0/1809153718 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f3788011620 con 0x55588e5a4b30 2024-08-02T17:53:37.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.354+0000 7f3773fff640 1 -- 172.21.15.69:0/1809153718 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3788020050 con 0x55588e5a4b30 2024-08-02T17:53:37.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.355+0000 7f379cee6740 10 client.26650 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:37.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.355+0000 7f379cee6740 1 -- 172.21.15.69:0/1809153718 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55588e27b3c0 con 0x55588e5a4b30 2024-08-02T17:53:37.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.356+0000 7f3773fff640 1 -- 172.21.15.69:0/1809153718 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55588e516ef0 con 0x55588e5a4b30 2024-08-02T17:53:37.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.356+0000 7f3773fff640 1 -- 172.21.15.69:0/1809153718 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3788019a10 con 0x55588e5a4b30 2024-08-02T17:53:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.357+0000 7f379cee6740 10 client.26650 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.357+0000 7f379cee6740 10 client.26650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.357+0000 7f379cee6740 20 client.26650 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.357+0000 7f379cee6740 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55588e5bf5c0 0x55588e5c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.357+0000 7f379cee6740 4 client.26650 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:37.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.357+0000 7f379cee6740 1 -- 172.21.15.69:0/1809153718 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55588e511400 con 0x55588e5bf5c0 2024-08-02T17:53:37.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.361+0000 7f3799085640 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55588e5bf5c0 0x55588e5c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.374+0000 7f3773fff640 10 client.26650 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:37.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.374+0000 7f3799085640 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55588e5bf5c0 0x55588e5c1a00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df4224116196d70e server_cookie=46b2ba2964ee19f7 in_seq=0 out_seq=0 2024-08-02T17:53:37.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.374+0000 7f3773fff640 1 -- 172.21.15.69:0/1809153718 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55588e511400 con 0x55588e5bf5c0 2024-08-02T17:53:37.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.374+0000 7f3773fff640 10 client.26650 handle_command_reply: tid=0 2024-08-02T17:53:37.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.375+0000 7f379cee6740 1 client.26650 shutdown 2024-08-02T17:53:37.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.375+0000 7f379cee6740 1 -- 172.21.15.69:0/1809153718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 msgr2=0x55588e5ac7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:37.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.375+0000 7f379cee6740 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5ac7d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3788009e60 tx=0x7f378800b4b0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.377+0000 7f379cee6740 1 -- 172.21.15.69:0/1809153718 shutdown_connections 2024-08-02T17:53:37.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.377+0000 7f379cee6740 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 0x55588e5acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.377+0000 7f379cee6740 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55588e5bf5c0 0x55588e5c1a00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.377+0000 7f379cee6740 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5ac7d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.377+0000 7f379cee6740 1 --2- 172.21.15.69:0/1809153718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.377+0000 7f379cee6740 1 -- 172.21.15.69:0/1809153718 >> 172.21.15.69:0/1809153718 conn(0x55588e5a07a0 msgr2=0x55588e5a6500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:37.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.377+0000 7f3773fff640 0 client.26650 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:37.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.378+0000 7f379cee6740 1 -- 172.21.15.69:0/1809153718 shutdown_connections 2024-08-02T17:53:37.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.378+0000 7f379cee6740 1 -- 172.21.15.69:0/1809153718 wait complete. 2024-08-02T17:53:37.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.378+0000 7f379cee6740 20 client.26650 trim_cache size 0 max 16384 2024-08-02T17:53:37.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.406+0000 7f379cee6740 1 Processor -- start 2024-08-02T17:53:37.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.406+0000 7f379cee6740 1 -- start start 2024-08-02T17:53:37.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f379cee6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5ab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f379cee6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f379cee6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5af3f0 0x55588e5c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f379cee6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3780012aa0 con 0x55588e5a4210 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f379cee6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3780010370 con 0x55588e5af3f0 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f379cee6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f378000e1c0 con 0x55588e5a4b30 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3799085640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3799085640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5ab270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41554/0 (socket says 172.21.15.69:41554) 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3799085640 1 -- 172.21.15.69:0/1408002927 learned_addr learned my addr 172.21.15.69:0/1408002927 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3793fff640 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3799085640 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5af3f0 0x55588e5c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3793fff640 1 -- 172.21.15.69:0/1408002927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5af3f0 msgr2=0x55588e5c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3793fff640 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5af3f0 0x55588e5c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3793fff640 1 -- 172.21.15.69:0/1408002927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 msgr2=0x55588e5ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:37.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3793fff640 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.407+0000 7f3793fff640 1 -- 172.21.15.69:0/1408002927 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55588e5a05d0 con 0x55588e5a4b30 2024-08-02T17:53:37.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.408+0000 7f3793fff640 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5c7310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3788001eb0 tx=0x7f37880047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34e2837632c05d4d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:37.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.408+0000 7f377a7fc640 1 -- 172.21.15.69:0/1408002927 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788029070 con 0x55588e5a4b30 2024-08-02T17:53:37.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.408+0000 7f377a7fc640 1 -- 172.21.15.69:0/1408002927 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3788006c20 con 0x55588e5a4b30 2024-08-02T17:53:37.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.408+0000 7f377a7fc640 1 -- 172.21.15.69:0/1408002927 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788015040 con 0x55588e5a4b30 2024-08-02T17:53:37.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 -- 172.21.15.69:0/1408002927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 msgr2=0x55588e5c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:37.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5c7310 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3788001eb0 tx=0x7f37880047d0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 -- 172.21.15.69:0/1408002927 shutdown_connections 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5af3f0 0x55588e5c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5c7310 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 --2- 172.21.15.69:0/1408002927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5a4210 0x55588e5ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 -- 172.21.15.69:0/1408002927 >> 172.21.15.69:0/1408002927 conn(0x55588e5a07a0 msgr2=0x55588e5aab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 -- 172.21.15.69:0/1408002927 shutdown_connections 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 -- 172.21.15.69:0/1408002927 wait complete. 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.409+0000 7f379cee6740 1 -- start start 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.410+0000 7f379cee6740 1 Processor -- start 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.410+0000 7f379cee6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.410+0000 7f379cee6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f379cee6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 0x55588e5b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60794/0 (socket says 172.21.15.69:60794) 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3793fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3793fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42856/0 (socket says 172.21.15.69:42856) 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 -- 172.21.15.69:0/3760956040 learned_addr learned my addr 172.21.15.69:0/3760956040 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f378000e830 con 0x55588e5af3f0 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f378000fca0 con 0x55588e5a4210 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f378000ef00 con 0x55588e5a4b30 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 0x55588e5b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 -- 172.21.15.69:0/3760956040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 msgr2=0x55588e5b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 -- 172.21.15.69:0/3760956040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 msgr2=0x55588e5b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:37.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 0x55588e5b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f3799085640 1 -- 172.21.15.69:0/3760956040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55588e6284a0 con 0x55588e5a4210 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f3799085640 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5b24a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f378402da00 tx=0x7f3784000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af3ada1bc4ad699 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.411+0000 7f379cee6740 10 client.0 _ll_register_callbacks cb 0x7f3788006f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3788006c20 con 0x55588e5a4210 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f3773fff640 10 client.26780 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f3773fff640 1 -- 172.21.15.69:0/3760956040 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784004f90 con 0x55588e5a4210 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f3773fff640 1 -- 172.21.15.69:0/3760956040 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7f3784005130 con 0x55588e5a4210 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f3773fff640 1 -- 172.21.15.69:0/3760956040 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3784005390 con 0x55588e5a4210 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f379cee6740 10 client.26780 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:37.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55588e629570 con 0x55588e5a4210 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.412+0000 7f3773fff640 1 -- 172.21.15.69:0/3760956040 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3784030040 con 0x55588e5a4210 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 10 client.26780 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 10 client.26780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 20 client.26780 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55588e629b10 0x55588e62bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 4 client.26780 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3788006c20 con 0x55588e629b10 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f3793fff640 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55588e629b10 0x55588e62bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f3793fff640 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55588e629b10 0x55588e62bf70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9229f5c0b8e52e43 server_cookie=acbd310cbecc517d in_seq=0 out_seq=0 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f3773fff640 10 client.26780 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f3773fff640 1 -- 172.21.15.69:0/3760956040 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f3788006c20 con 0x55588e629b10 2024-08-02T17:53:37.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f3773fff640 10 client.26780 handle_command_reply: tid=0 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 1 client.26780 shutdown 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 msgr2=0x55588e5b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.413+0000 7f379cee6740 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5b24a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f378402da00 tx=0x7f3784000bc0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 shutdown_connections 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55588e5af3f0 0x55588e5b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55588e629b10 0x55588e62bf70 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55588e5a4b30 0x55588e5b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 1 --2- 172.21.15.69:0/3760956040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55588e5a4210 0x55588e5b24a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 >> 172.21.15.69:0/3760956040 conn(0x55588e5a07a0 msgr2=0x55588e5ab380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f3773fff640 0 client.26780 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 shutdown_connections 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 1 -- 172.21.15.69:0/3760956040 wait complete. 2024-08-02T17:53:37.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.414+0000 7f379cee6740 20 client.26780 trim_cache size 0 max 16384 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "953e4372-8445-4b00-90bd-6c4c09507980": { 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "953e4372-8445-4b00-90bd-6c4c09507980", 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:53:37.421 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:53:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 -- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f377c0a2f00 msgr2=0x7f377c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f377c0a2f00 0x7f377c0a53c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55588e5a1600 tx=0x7f378c03b900 comp rx=0 tx=0).stop 2024-08-02T17:53:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 -- 172.21.15.69:0/2004145536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109180 msgr2=0x7f379411a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109180 0x7f379411a590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f378c03be70 tx=0x7f378c0406c0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 -- 172.21.15.69:0/2004145536 shutdown_connections 2024-08-02T17:53:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3794109560 0x7f3794116630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f377c0a2f00 0x7f377c0a53c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f379411ae70 0x7f3794116b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 --2- 172.21.15.69:0/2004145536 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3794109180 0x7f379411a590 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:37.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 -- 172.21.15.69:0/2004145536 >> 172.21.15.69:0/2004145536 conn(0x7f379406de70 msgr2=0x7f3794072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:37.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 -- 172.21.15.69:0/2004145536 shutdown_connections 2024-08-02T17:53:37.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:53:37.415+0000 7f379aae8640 1 -- 172.21.15.69:0/2004145536 wait complete. 2024-08-02T17:53:38.084 INFO:tasks.cephfs.filesystem:scrub status for tag:953e4372-8445-4b00-90bd-6c4c09507980 - {'path': '/', 'tag': '953e4372-8445-4b00-90bd-6c4c09507980', 'options': 'recursive,force'} 2024-08-02T17:53:39.072 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:38 smithi083 ceph-mon[37031]: pgmap v221: 129 pgs: 129 active+clean; 3.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 14 MiB/s wr, 893 op/s 2024-08-02T17:53:39.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:38 smithi077 ceph-mon[37108]: pgmap v221: 129 pgs: 129 active+clean; 3.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 14 MiB/s wr, 893 op/s 2024-08-02T17:53:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:38 smithi069 ceph-mon[32746]: pgmap v221: 129 pgs: 129 active+clean; 3.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 14 MiB/s wr, 893 op/s 2024-08-02T17:53:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 390.0 s, 709.6 tps, lat 140.463 ms stddev 177.668 2024-08-02T17:53:40.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:39 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:53:40.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:39 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:53:40.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:39 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:53:41.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:40 smithi077 ceph-mon[37108]: pgmap v222: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 623 op/s 2024-08-02T17:53:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:40 smithi069 ceph-mon[32746]: pgmap v222: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 623 op/s 2024-08-02T17:53:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:40 smithi083 ceph-mon[37031]: pgmap v222: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 623 op/s 2024-08-02T17:53:43.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:42 smithi077 ceph-mon[37108]: pgmap v223: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 10 MiB/s wr, 589 op/s 2024-08-02T17:53:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:42 smithi069 ceph-mon[32746]: pgmap v223: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 10 MiB/s wr, 589 op/s 2024-08-02T17:53:43.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:42 smithi083 ceph-mon[37031]: pgmap v223: 129 pgs: 129 active+clean; 3.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 10 MiB/s wr, 589 op/s 2024-08-02T17:53:44.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:43 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:53:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:43 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:53:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:43 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:53:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:43 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:53:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:43 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:53:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:43 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:53:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:43 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:53:44.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:43 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:53:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:43 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:53:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:43 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:53:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:43 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:53:44.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:43 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:53:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 395.0 s, 1164.2 tps, lat 84.904 ms stddev 101.219 2024-08-02T17:53:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:44 smithi077 ceph-mon[37108]: pgmap v224: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 14 MiB/s wr, 786 op/s 2024-08-02T17:53:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:44 smithi069 ceph-mon[32746]: pgmap v224: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 14 MiB/s wr, 786 op/s 2024-08-02T17:53:45.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.139+0000 7ff39a2bf640 1 -- 172.21.15.69:0/3142900609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff394105230 msgr2=0x7ff39410cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.139+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/3142900609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff394105230 0x7ff39410cd80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00adb0 tx=0x7ff38c02b7e0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.139+0000 7ff39a2bf640 1 -- 172.21.15.69:0/3142900609 shutdown_connections 2024-08-02T17:53:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.139+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/3142900609 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff394105230 0x7ff39410cd80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.139+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/3142900609 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff394100ef0 0x7ff394104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.139+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/3142900609 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394100540 0x7ff394100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.139+0000 7ff39a2bf640 1 -- 172.21.15.69:0/3142900609 >> 172.21.15.69:0/3142900609 conn(0x7ff394075be0 msgr2=0x7ff394075ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.142+0000 7ff39a2bf640 1 -- 172.21.15.69:0/3142900609 shutdown_connections 2024-08-02T17:53:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.142+0000 7ff39a2bf640 1 -- 172.21.15.69:0/3142900609 wait complete. 2024-08-02T17:53:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.142+0000 7ff39a2bf640 1 Processor -- start 2024-08-02T17:53:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.143+0000 7ff39a2bf640 1 -- start start 2024-08-02T17:53:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.143+0000 7ff39a2bf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394100540 0x7ff3940737c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.143+0000 7ff39a2bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff394105230 0x7ff39406c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.143+0000 7ff39a2bf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff39406cd80 0x7ff39406d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.143+0000 7ff39a2bf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410fad0 con 0x7ff394100540 2024-08-02T17:53:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.143+0000 7ff39a2bf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410f950 con 0x7ff39406cd80 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.143+0000 7ff39a2bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410fc50 con 0x7ff394105230 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.144+0000 7ff39885c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff39406cd80 0x7ff39406d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.144+0000 7ff39885c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff39406cd80 0x7ff39406d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:46706/0 (socket says 172.21.15.69:46706) 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.144+0000 7ff39885c640 1 -- 172.21.15.69:0/2520926 learned_addr learned my addr 172.21.15.69:0/2520926 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.144+0000 7ff39885c640 1 -- 172.21.15.69:0/2520926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff394105230 msgr2=0x7ff39406c840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.144+0000 7ff39885c640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff394105230 0x7ff39406c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.144+0000 7ff39885c640 1 -- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394100540 msgr2=0x7ff3940737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.144+0000 7ff39885c640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394100540 0x7ff3940737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.144+0000 7ff39885c640 1 -- 172.21.15.69:0/2520926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3940712e0 con 0x7ff39406cd80 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.145+0000 7ff39885c640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff39406cd80 0x7ff39406d230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00a9e0 tx=0x7ff38c032630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.145+0000 7ff3917fa640 1 -- 172.21.15.69:0/2520926 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c040070 con 0x7ff39406cd80 2024-08-02T17:53:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.145+0000 7ff39a2bf640 1 -- 172.21.15.69:0/2520926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff394071570 con 0x7ff39406cd80 2024-08-02T17:53:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.145+0000 7ff39a2bf640 1 -- 172.21.15.69:0/2520926 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff394071ab0 con 0x7ff39406cd80 2024-08-02T17:53:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.145+0000 7ff3917fa640 1 -- 172.21.15.69:0/2520926 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c0309f0 con 0x7ff39406cd80 2024-08-02T17:53:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.145+0000 7ff3917fa640 1 -- 172.21.15.69:0/2520926 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c045460 con 0x7ff39406cd80 2024-08-02T17:53:45.160 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.158+0000 7ff3917fa640 1 -- 172.21.15.69:0/2520926 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff38c03c410 con 0x7ff39406cd80 2024-08-02T17:53:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.159+0000 7ff3917fa640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff37c0a2e40 0x7ff37c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.159+0000 7ff3917fa640 1 -- 172.21.15.69:0/2520926 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7ff38c0f1dc0 con 0x7ff39406cd80 2024-08-02T17:53:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.160+0000 7ff393fff640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff37c0a2e40 0x7ff37c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.161 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.160+0000 7ff39d015740 1 Processor -- start 2024-08-02T17:53:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.160+0000 7ff39d015740 1 -- start start 2024-08-02T17:53:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39d015740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 0x559895e88b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39d015740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e93430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39d015740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e95d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39d015740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559895dec9b0 con 0x559895e93970 2024-08-02T17:53:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39d015740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559895d94800 con 0x559895e887b0 2024-08-02T17:53:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39d015740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559895de8970 con 0x559895e890d0 2024-08-02T17:53:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39885c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e95d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39885c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e95d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43914/0 (socket says 172.21.15.69:43914) 2024-08-02T17:53:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.161+0000 7ff39885c640 1 -- 172.21.15.69:0/2526540603 learned_addr learned my addr 172.21.15.69:0/2526540603 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.164+0000 7ff3937fe640 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e93430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.164+0000 7ff393fff640 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 0x559895e88b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.164+0000 7ff393fff640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff37c0a2e40 0x7ff37c0a5300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff384009710 tx=0x7ff384006c90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.166+0000 7ff39885c640 1 -- 172.21.15.69:0/2526540603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 msgr2=0x559895e93430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.166+0000 7ff39885c640 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e93430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.166+0000 7ff39885c640 1 -- 172.21.15.69:0/2526540603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 msgr2=0x559895e88b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.166+0000 7ff39885c640 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 0x559895e88b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.166+0000 7ff39885c640 1 -- 172.21.15.69:0/2526540603 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559895b00210 con 0x559895e93970 2024-08-02T17:53:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.170+0000 7ff39885c640 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e95d60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff38c003fd0 tx=0x7ff38c0be590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc3bc9d8419e0ae server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.176+0000 7ff37a7fc640 1 -- 172.21.15.69:0/2526540603 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0c2240 con 0x559895e93970 2024-08-02T17:53:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.176+0000 7ff37a7fc640 1 -- 172.21.15.69:0/2526540603 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c0c6b00 con 0x559895e93970 2024-08-02T17:53:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.176+0000 7ff37a7fc640 1 -- 172.21.15.69:0/2526540603 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c004040 con 0x559895e93970 2024-08-02T17:53:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.177+0000 7ff39d015740 1 -- 172.21.15.69:0/2526540603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 msgr2=0x559895e95d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.177+0000 7ff39d015740 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e95d60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff38c003fd0 tx=0x7ff38c0be590 comp rx=0 tx=0).stop 2024-08-02T17:53:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.178+0000 7ff39d015740 1 -- 172.21.15.69:0/2526540603 shutdown_connections 2024-08-02T17:53:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.178+0000 7ff39d015740 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e95d60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.178+0000 7ff39d015740 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e93430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.178+0000 7ff39d015740 1 --2- 172.21.15.69:0/2526540603 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 0x559895e88b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.178+0000 7ff39d015740 1 -- 172.21.15.69:0/2526540603 >> 172.21.15.69:0/2526540603 conn(0x559895e84d20 msgr2=0x559895e85100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.178+0000 7ff39d015740 1 -- 172.21.15.69:0/2526540603 shutdown_connections 2024-08-02T17:53:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.178+0000 7ff39d015740 1 -- 172.21.15.69:0/2526540603 wait complete. 2024-08-02T17:53:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 1 -- start start 2024-08-02T17:53:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 1 Processor -- start 2024-08-02T17:53:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e887b0 0x559895e90890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e890d0 0x559895e90dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e91310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559895e96300 con 0x559895e93970 2024-08-02T17:53:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559895da8500 con 0x559895e890d0 2024-08-02T17:53:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559895e96480 con 0x559895e887b0 2024-08-02T17:53:45.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.179+0000 7ff39d015740 10 client.0 _ll_register_callbacks cb 0x559895affa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff393fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e887b0 0x559895e90890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff393fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e887b0 0x559895e90890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45758/0 (socket says 172.21.15.69:45758) 2024-08-02T17:53:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff393fff640 1 -- 172.21.15.69:0/170534431 learned_addr learned my addr 172.21.15.69:0/170534431 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff3937fe640 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e890d0 0x559895e90dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff3937fe640 1 -- 172.21.15.69:0/170534431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e887b0 msgr2=0x559895e90890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:45.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff3937fe640 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e887b0 0x559895e90890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff3937fe640 1 -- 172.21.15.69:0/170534431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 msgr2=0x559895e91310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff3937fe640 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e91310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff3937fe640 1 -- 172.21.15.69:0/170534431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559895bcc620 con 0x559895e890d0 2024-08-02T17:53:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.180+0000 7ff3937fe640 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e890d0 0x559895e90dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff388006fd0 tx=0x7ff38800e660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660806cb99ced0e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.181+0000 7ff3737fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:53:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.181+0000 7ff3737fe640 1 -- 172.21.15.69:0/170534431 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388017070 con 0x559895e890d0 2024-08-02T17:53:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.181+0000 7ff3737fe640 1 -- 172.21.15.69:0/170534431 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7ff388013470 con 0x559895e890d0 2024-08-02T17:53:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.181+0000 7ff39d015740 1 -- 172.21.15.69:0/170534431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38c0c6b00 con 0x559895e890d0 2024-08-02T17:53:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff3737fe640 1 -- 172.21.15.69:0/170534431 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff388021050 con 0x559895e890d0 2024-08-02T17:53:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff39d015740 10 client.26789 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff39d015740 1 -- 172.21.15.69:0/170534431 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559895ea1f90 con 0x559895e890d0 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff3737fe640 1 -- 172.21.15.69:0/170534431 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c0c6b00 con 0x559895e890d0 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff3737fe640 1 -- 172.21.15.69:0/170534431 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff3880128c0 con 0x559895e890d0 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff39d015740 10 client.26789 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff39d015740 10 client.26789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff39d015740 20 client.26789 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff39d015740 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559895ea37b0 0x559895ea5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff39d015740 4 client.26789 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.182+0000 7ff39d015740 1 -- 172.21.15.69:0/170534431 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559895dfa0e0 con 0x559895ea37b0 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.183+0000 7ff393fff640 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559895ea37b0 0x559895ea5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.183+0000 7ff3737fe640 10 client.26789 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.184+0000 7ff393fff640 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559895ea37b0 0x559895ea5c30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db5900d2f329a77a server_cookie=8ea6f184a9dafbab in_seq=0 out_seq=0 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff3737fe640 1 -- 172.21.15.69:0/170534431 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559895dfa0e0 con 0x559895ea37b0 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff3737fe640 10 client.26789 handle_command_reply: tid=0 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 client.26789 shutdown 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 -- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e890d0 msgr2=0x559895e90dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e890d0 0x559895e90dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff388006fd0 tx=0x7ff38800e660 comp rx=0 tx=0).stop 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 -- 172.21.15.69:0/170534431 shutdown_connections 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e91310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559895ea37b0 0x559895ea5c30 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e890d0 0x559895e90dd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 --2- 172.21.15.69:0/170534431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e887b0 0x559895e90890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 -- 172.21.15.69:0/170534431 >> 172.21.15.69:0/170534431 conn(0x559895e84d20 msgr2=0x559895e8aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff3737fe640 0 client.26789 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 -- 172.21.15.69:0/170534431 shutdown_connections 2024-08-02T17:53:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.185+0000 7ff39d015740 1 -- 172.21.15.69:0/170534431 wait complete. 2024-08-02T17:53:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.186+0000 7ff39d015740 20 client.26789 trim_cache size 0 max 16384 2024-08-02T17:53:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.216+0000 7ff39d015740 1 Processor -- start 2024-08-02T17:53:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.216+0000 7ff39d015740 1 -- start start 2024-08-02T17:53:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff39d015740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e887b0 0x559895e8bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff39d015740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895eab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff393fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e887b0 0x559895e8bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff393fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e887b0 0x559895e8bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43926/0 (socket says 172.21.15.69:43926) 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff393fff640 1 -- 172.21.15.69:0/268539494 learned_addr learned my addr 172.21.15.69:0/268539494 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff39d015740 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e93970 0x559895eaba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff39d015740 1 -- 172.21.15.69:0/268539494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff380012a10 con 0x559895e887b0 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff3937fe640 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895eab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff39d015740 1 -- 172.21.15.69:0/268539494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3800102e0 con 0x559895e93970 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.217+0000 7ff39d015740 1 -- 172.21.15.69:0/268539494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff38000e130 con 0x559895e890d0 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff39885c640 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e93970 0x559895eaba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff3937fe640 1 -- 172.21.15.69:0/268539494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e93970 msgr2=0x559895eaba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff3937fe640 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e93970 0x559895eaba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff3937fe640 1 -- 172.21.15.69:0/268539494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e887b0 msgr2=0x559895e8bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff3937fe640 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e887b0 0x559895e8bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff3937fe640 1 -- 172.21.15.69:0/268539494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559895e88f00 con 0x559895e890d0 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff39885c640 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e93970 0x559895eaba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff393fff640 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e887b0 0x559895e8bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff3937fe640 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895eab500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff388012e80 tx=0x7ff38800ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b49aab8f5ad311 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff379ffb640 1 -- 172.21.15.69:0/268539494 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388017070 con 0x559895e890d0 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff379ffb640 1 -- 172.21.15.69:0/268539494 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff388012eb0 con 0x559895e890d0 2024-08-02T17:53:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.218+0000 7ff379ffb640 1 -- 172.21.15.69:0/268539494 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3880129e0 con 0x559895e890d0 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 -- 172.21.15.69:0/268539494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 msgr2=0x559895eab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895eab500 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff388012e80 tx=0x7ff38800ebe0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 -- 172.21.15.69:0/268539494 shutdown_connections 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e93970 0x559895eaba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895eab500 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 --2- 172.21.15.69:0/268539494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e887b0 0x559895e8bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 -- 172.21.15.69:0/268539494 >> 172.21.15.69:0/268539494 conn(0x559895e84d20 msgr2=0x559895e8f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 -- 172.21.15.69:0/268539494 shutdown_connections 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.219+0000 7ff39d015740 1 -- 172.21.15.69:0/268539494 wait complete. 2024-08-02T17:53:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.220+0000 7ff39d015740 1 -- start start 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.220+0000 7ff39d015740 1 Processor -- start 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.220+0000 7ff39d015740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 0x559895e9b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.220+0000 7ff39d015740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e9bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.220+0000 7ff39d015740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e96c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff39d015740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff38000e7a0 con 0x559895e93970 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e9bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff39885c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e96c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff393fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 0x559895e9b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e9bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45776/0 (socket says 172.21.15.69:45776) 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 -- 172.21.15.69:0/135489051 learned_addr learned my addr 172.21.15.69:0/135489051 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff38000fc10 con 0x559895e887b0 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff38000ee70 con 0x559895e890d0 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff39d015740 10 client.0 _ll_register_callbacks cb 0x7ff3880092f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 -- 172.21.15.69:0/135489051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 msgr2=0x559895e9b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 0x559895e9b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 -- 172.21.15.69:0/135489051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 msgr2=0x559895e96c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e96c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 -- 172.21.15.69:0/135489051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559895e97480 con 0x559895e890d0 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559895df4430 con 0x559895e890d0 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3937fe640 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e9bdc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff388006b40 tx=0x7ff38801b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51430f9b4e606f67 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3737fe640 10 client.26671 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3737fe640 1 -- 172.21.15.69:0/135489051 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388035060 con 0x559895e890d0 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3737fe640 1 -- 172.21.15.69:0/135489051 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7ff38801ba50 con 0x559895e890d0 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.221+0000 7ff3737fe640 1 -- 172.21.15.69:0/135489051 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff388017040 con 0x559895e890d0 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff39d015740 10 client.26671 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x559895f0de90 con 0x559895e890d0 2024-08-02T17:53:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff3737fe640 1 -- 172.21.15.69:0/135489051 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff38801bc90 con 0x559895e890d0 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff39d015740 10 client.26671 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff39d015740 10 client.26671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff39d015740 20 client.26671 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff39d015740 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559895f0e420 0x559895f10880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff39d015740 4 client.26671 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559895d718e0 con 0x559895f0e420 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.222+0000 7ff393fff640 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559895f0e420 0x559895f10880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.223+0000 7ff393fff640 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559895f0e420 0x559895f10880 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94078b59267f361c server_cookie=69cc04bafb66c3d2 in_seq=0 out_seq=0 2024-08-02T17:53:45.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.223+0000 7ff3737fe640 10 client.26671 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:45.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:44 smithi083 ceph-mon[37031]: pgmap v224: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 14 MiB/s wr, 786 op/s 2024-08-02T17:53:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff3737fe640 1 -- 172.21.15.69:0/135489051 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559895d718e0 con 0x559895f0e420 2024-08-02T17:53:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff3737fe640 10 client.26671 handle_command_reply: tid=0 2024-08-02T17:53:45.292 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff39d015740 1 client.26671 shutdown 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 msgr2=0x559895e9bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff39d015740 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e9bdc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff388006b40 tx=0x7ff38801b860 comp rx=0 tx=0).stop 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 shutdown_connections 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff39d015740 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559895e93970 0x559895e96c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff39d015740 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559895f0e420 0x559895f10880 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff39d015740 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559895e890d0 0x559895e9bdc0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff39d015740 1 --2- 172.21.15.69:0/135489051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559895e887b0 0x559895e9b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.291+0000 7ff3737fe640 0 client.26671 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:45.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.292+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 >> 172.21.15.69:0/135489051 conn(0x559895e84d20 msgr2=0x559895e90830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.293+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 shutdown_connections 2024-08-02T17:53:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.293+0000 7ff39d015740 1 -- 172.21.15.69:0/135489051 wait complete. 2024-08-02T17:53:45.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.293+0000 7ff39d015740 20 client.26671 trim_cache size 0 max 16384 2024-08-02T17:53:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:53:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:53:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:53:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:53:45.295 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.294+0000 7ff39a2bf640 1 -- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff37c0a2e40 msgr2=0x7ff37c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.294+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff37c0a2e40 0x7ff37c0a5300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff384009710 tx=0x7ff384006c90 comp rx=0 tx=0).stop 2024-08-02T17:53:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.294+0000 7ff39a2bf640 1 -- 172.21.15.69:0/2520926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff39406cd80 msgr2=0x7ff39406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:45.296 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.294+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff39406cd80 0x7ff39406d230 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00a9e0 tx=0x7ff38c032630 comp rx=0 tx=0).stop 2024-08-02T17:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.295+0000 7ff39a2bf640 1 -- 172.21.15.69:0/2520926 shutdown_connections 2024-08-02T17:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.295+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff37c0a2e40 0x7ff37c0a5300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.295+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff39406cd80 0x7ff39406d230 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.295+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff394105230 0x7ff39406c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.295+0000 7ff39a2bf640 1 --2- 172.21.15.69:0/2520926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff394100540 0x7ff3940737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.295+0000 7ff39a2bf640 1 -- 172.21.15.69:0/2520926 >> 172.21.15.69:0/2520926 conn(0x7ff394075be0 msgr2=0x7ff3940fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:45.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.295+0000 7ff39a2bf640 1 -- 172.21.15.69:0/2520926 shutdown_connections 2024-08-02T17:53:45.298 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:45.296+0000 7ff39a2bf640 1 -- 172.21.15.69:0/2520926 wait complete. 2024-08-02T17:53:47.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:46 smithi077 ceph-mon[37108]: pgmap v225: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.8 MiB/s wr, 594 op/s 2024-08-02T17:53:47.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:46 smithi069 ceph-mon[32746]: pgmap v225: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.8 MiB/s wr, 594 op/s 2024-08-02T17:53:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:46 smithi083 ceph-mon[37031]: pgmap v225: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.8 MiB/s wr, 594 op/s 2024-08-02T17:53:49.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:48 smithi077 ceph-mon[37108]: pgmap v226: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2024-08-02T17:53:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:48 smithi069 ceph-mon[32746]: pgmap v226: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2024-08-02T17:53:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:48 smithi083 ceph-mon[37031]: pgmap v226: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 15 MiB/s wr, 1.22k op/s 2024-08-02T17:53:49.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 400.0 s, 675.2 tps, lat 149.011 ms stddev 267.884 2024-08-02T17:53:51.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:50 smithi077 ceph-mon[37108]: pgmap v227: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-08-02T17:53:51.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:50 smithi069 ceph-mon[32746]: pgmap v227: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-08-02T17:53:51.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:50 smithi083 ceph-mon[37031]: pgmap v227: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-08-02T17:53:53.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:52 smithi077 ceph-mon[37108]: pgmap v228: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 12 MiB/s wr, 996 op/s 2024-08-02T17:53:53.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:52 smithi069 ceph-mon[32746]: pgmap v228: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 12 MiB/s wr, 996 op/s 2024-08-02T17:53:53.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:52 smithi083 ceph-mon[37031]: pgmap v228: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 12 MiB/s wr, 996 op/s 2024-08-02T17:53:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 405.0 s, 611.6 tps, lat 161.056 ms stddev 197.290 2024-08-02T17:53:55.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:54 smithi077 ceph-mon[37108]: pgmap v229: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-08-02T17:53:55.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:54 smithi069 ceph-mon[32746]: pgmap v229: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-08-02T17:53:55.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:54 smithi083 ceph-mon[37031]: pgmap v229: 129 pgs: 129 active+clean; 3.5 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 16 MiB/s wr, 1.24k op/s 2024-08-02T17:53:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:56 smithi069 ceph-mon[32746]: pgmap v230: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-08-02T17:53:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:56 smithi083 ceph-mon[37031]: pgmap v230: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-08-02T17:53:57.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:56 smithi077 ceph-mon[37108]: pgmap v230: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-08-02T17:53:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.441+0000 7fb78d228640 1 -- 172.21.15.69:0/3474737395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788109300 msgr2=0x7fb788109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.441+0000 7fb78d228640 1 --2- 172.21.15.69:0/3474737395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788109300 0x7fb788109740 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb78001a910 tx=0x7fb78003b470 comp rx=0 tx=0).stop 2024-08-02T17:53:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.446+0000 7fb78d228640 1 -- 172.21.15.69:0/3474737395 shutdown_connections 2024-08-02T17:53:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.446+0000 7fb78d228640 1 --2- 172.21.15.69:0/3474737395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788109300 0x7fb788109740 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.446+0000 7fb78d228640 1 --2- 172.21.15.69:0/3474737395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb78811ae70 0x7fb78811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.446+0000 7fb78d228640 1 --2- 172.21.15.69:0/3474737395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb788108f20 0x7fb78810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.446+0000 7fb78d228640 1 -- 172.21.15.69:0/3474737395 >> 172.21.15.69:0/3474737395 conn(0x7fb78806cb20 msgr2=0x7fb78806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:58.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.453+0000 7fb78d228640 1 -- 172.21.15.69:0/3474737395 shutdown_connections 2024-08-02T17:53:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.458+0000 7fb78d228640 1 -- 172.21.15.69:0/3474737395 wait complete. 2024-08-02T17:53:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.459+0000 7fb78d228640 1 Processor -- start 2024-08-02T17:53:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.460+0000 7fb78d228640 1 -- start start 2024-08-02T17:53:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.460+0000 7fb78d228640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788108f20 0x7fb78811a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.460+0000 7fb78d228640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb788109300 0x7fb788116780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.460+0000 7fb78d228640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb78811ae70 0x7fb788116cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.460+0000 7fb78d228640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb788113130 con 0x7fb788108f20 2024-08-02T17:53:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.460+0000 7fb78d228640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb788112fb0 con 0x7fb78811ae70 2024-08-02T17:53:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.460+0000 7fb78d228640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb7881132b0 con 0x7fb788109300 2024-08-02T17:53:58.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788108f20 0x7fb78811a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788108f20 0x7fb78811a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57050/0 (socket says 172.21.15.69:57050) 2024-08-02T17:53:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 -- 172.21.15.69:0/673517289 learned_addr learned my addr 172.21.15.69:0/673517289 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 -- 172.21.15.69:0/673517289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb788109300 msgr2=0x7fb788116780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb788109300 0x7fb788116780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 -- 172.21.15.69:0/673517289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb78811ae70 msgr2=0x7fb788116cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:53:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb78811ae70 0x7fb788116cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 -- 172.21.15.69:0/673517289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb788117580 con 0x7fb788108f20 2024-08-02T17:53:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.461+0000 7fb786d9d640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788108f20 0x7fb78811a6f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb77000b450 tx=0x7fb77000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.462+0000 7fb77ffff640 1 -- 172.21.15.69:0/673517289 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770015020 con 0x7fb788108f20 2024-08-02T17:53:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.462+0000 7fb78d228640 1 -- 172.21.15.69:0/673517289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7881aae00 con 0x7fb788108f20 2024-08-02T17:53:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.462+0000 7fb78d228640 1 -- 172.21.15.69:0/673517289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7881ab280 con 0x7fb788108f20 2024-08-02T17:53:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.462+0000 7fb77ffff640 1 -- 172.21.15.69:0/673517289 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb770011d70 con 0x7fb788108f20 2024-08-02T17:53:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.462+0000 7fb77ffff640 1 -- 172.21.15.69:0/673517289 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77000f340 con 0x7fb788108f20 2024-08-02T17:53:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.466+0000 7fb77ffff640 1 -- 172.21.15.69:0/673517289 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb770018020 con 0x7fb788108f20 2024-08-02T17:53:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.467+0000 7fb77ffff640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb7740a2f00 0x7fb7740a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.467+0000 7fb78659c640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb7740a2f00 0x7fb7740a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.467+0000 7fb77ffff640 1 -- 172.21.15.69:0/673517289 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7fb7700d02f0 con 0x7fb788108f20 2024-08-02T17:53:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.469+0000 7fb78ff80740 1 Processor -- start 2024-08-02T17:53:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.469+0000 7fb78659c640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb7740a2f00 0x7fb7740a53c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb78001a6d0 tx=0x7fb780000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.474+0000 7fb78ff80740 1 -- start start 2024-08-02T17:53:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.475+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x562792464b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.475+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.475+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 0x562792471d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.475+0000 7fb78ff80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5627923c89b0 con 0x5627924650d0 2024-08-02T17:53:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.475+0000 7fb78ff80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562792370800 con 0x5627924647b0 2024-08-02T17:53:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.475+0000 7fb78ff80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627923c4970 con 0x56279246f970 2024-08-02T17:53:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.477+0000 7fb78659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.477+0000 7fb78659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:57064/0 (socket says 172.21.15.69:57064) 2024-08-02T17:53:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.477+0000 7fb78659c640 1 -- 172.21.15.69:0/841684852 learned_addr learned my addr 172.21.15.69:0/841684852 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.478+0000 7fb78659c640 1 -- 172.21.15.69:0/841684852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 msgr2=0x562792471d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.477+0000 7fb78759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 0x562792471d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.479+0000 7fb78659c640 1 --2- 172.21.15.69:0/841684852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 0x562792471d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.479+0000 7fb78659c640 1 -- 172.21.15.69:0/841684852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 msgr2=0x562792464b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:53:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.479+0000 7fb78659c640 1 --2- 172.21.15.69:0/841684852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x562792464b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.479+0000 7fb78659c640 1 -- 172.21.15.69:0/841684852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627920dc210 con 0x5627924650d0 2024-08-02T17:53:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.479+0000 7fb78759e640 1 --2- 172.21.15.69:0/841684852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 0x562792471d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:53:58.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.500+0000 7fb78659c640 1 --2- 172.21.15.69:0/841684852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246f430 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb780044000 tx=0x7fb78004b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf527776a0fb90e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.501+0000 7fb77d7fa640 1 -- 172.21.15.69:0/841684852 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7800054c0 con 0x5627924650d0 2024-08-02T17:53:58.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.501+0000 7fb77d7fa640 1 -- 172.21.15.69:0/841684852 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb78005c070 con 0x5627924650d0 2024-08-02T17:53:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.501+0000 7fb77d7fa640 1 -- 172.21.15.69:0/841684852 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7800576c0 con 0x5627924650d0 2024-08-02T17:53:58.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.502+0000 7fb78ff80740 1 -- 172.21.15.69:0/841684852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 msgr2=0x56279246f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.502+0000 7fb78ff80740 1 --2- 172.21.15.69:0/841684852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246f430 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb780044000 tx=0x7fb78004b000 comp rx=0 tx=0).stop 2024-08-02T17:53:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.503+0000 7fb78ff80740 1 -- 172.21.15.69:0/841684852 shutdown_connections 2024-08-02T17:53:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.503+0000 7fb78ff80740 1 --2- 172.21.15.69:0/841684852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246f430 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.503+0000 7fb78ff80740 1 --2- 172.21.15.69:0/841684852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 0x562792471d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.503+0000 7fb78ff80740 1 --2- 172.21.15.69:0/841684852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x562792464b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.503+0000 7fb78ff80740 1 -- 172.21.15.69:0/841684852 >> 172.21.15.69:0/841684852 conn(0x562792460d20 msgr2=0x562792461100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:58.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.505+0000 7fb78ff80740 1 -- 172.21.15.69:0/841684852 shutdown_connections 2024-08-02T17:53:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.508+0000 7fb78ff80740 1 -- 172.21.15.69:0/841684852 wait complete. 2024-08-02T17:53:58.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.508+0000 7fb78ff80740 1 -- start start 2024-08-02T17:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.509+0000 7fb78ff80740 1 Processor -- start 2024-08-02T17:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.509+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627924647b0 0x56279246c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.509+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.509+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 0x56279246d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.509+0000 7fb78ff80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562792472540 con 0x5627924650d0 2024-08-02T17:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.509+0000 7fb78ff80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562792384500 con 0x56279246f970 2024-08-02T17:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.509+0000 7fb78ff80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627924726c0 con 0x5627924647b0 2024-08-02T17:53:58.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.509+0000 7fb78ff80740 10 client.0 _ll_register_callbacks cb 0x5627920dba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb786d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627924647b0 0x56279246c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb786d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627924647b0 0x56279246c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:52876/0 (socket says 172.21.15.69:52876) 2024-08-02T17:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb786d9d640 1 -- 172.21.15.69:0/784777270 learned_addr learned my addr 172.21.15.69:0/784777270 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb78759e640 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 0x56279246d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb786d9d640 1 -- 172.21.15.69:0/784777270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 msgr2=0x56279246d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb786d9d640 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 0x56279246d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb786d9d640 1 -- 172.21.15.69:0/784777270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 msgr2=0x56279246ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb786d9d640 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.510+0000 7fb786d9d640 1 -- 172.21.15.69:0/784777270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627921a8620 con 0x5627924647b0 2024-08-02T17:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.511+0000 7fb786d9d640 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627924647b0 0x56279246c910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb770098ea0 tx=0x7fb77009cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e1fad774aeea85f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.511+0000 7fb76e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.511+0000 7fb76e7fc640 1 -- 172.21.15.69:0/784777270 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7700b5060 con 0x5627924647b0 2024-08-02T17:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.511+0000 7fb76e7fc640 1 -- 172.21.15.69:0/784777270 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7fb7700b5400 con 0x5627924647b0 2024-08-02T17:53:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.511+0000 7fb78ff80740 1 -- 172.21.15.69:0/784777270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb78005c070 con 0x5627924647b0 2024-08-02T17:53:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.511+0000 7fb76e7fc640 1 -- 172.21.15.69:0/784777270 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb7700cda80 con 0x5627924647b0 2024-08-02T17:53:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.512+0000 7fb78ff80740 10 client.26674 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.512+0000 7fb78ff80740 1 -- 172.21.15.69:0/784777270 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56279247e260 con 0x5627924647b0 2024-08-02T17:53:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.517+0000 7fb76e7fc640 1 -- 172.21.15.69:0/784777270 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb78005c070 con 0x5627924647b0 2024-08-02T17:53:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.517+0000 7fb76e7fc640 1 -- 172.21.15.69:0/784777270 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb77009b390 con 0x5627924647b0 2024-08-02T17:53:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.517+0000 7fb78ff80740 10 client.26674 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.517+0000 7fb78ff80740 10 client.26674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.517+0000 7fb78ff80740 20 client.26674 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.517+0000 7fb78ff80740 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56279247fc10 0x562792482070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.517+0000 7fb78ff80740 4 client.26674 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:53:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.517+0000 7fb78ff80740 1 -- 172.21.15.69:0/784777270 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627923d60e0 con 0x56279247fc10 2024-08-02T17:53:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.521+0000 7fb78659c640 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56279247fc10 0x562792482070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.531+0000 7fb78659c640 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56279247fc10 0x562792482070 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1e1b26fe9a94ce server_cookie=c486aab6384bd4af in_seq=0 out_seq=0 2024-08-02T17:53:58.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.531+0000 7fb76e7fc640 10 client.26674 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.533+0000 7fb76e7fc640 1 -- 172.21.15.69:0/784777270 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5627923d60e0 con 0x56279247fc10 2024-08-02T17:53:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.533+0000 7fb76e7fc640 10 client.26674 handle_command_reply: tid=0 2024-08-02T17:53:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.534+0000 7fb78ff80740 1 client.26674 shutdown 2024-08-02T17:53:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.534+0000 7fb78ff80740 1 -- 172.21.15.69:0/784777270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627924647b0 msgr2=0x56279246c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.534+0000 7fb78ff80740 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627924647b0 0x56279246c910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb770098ea0 tx=0x7fb77009cd10 comp rx=0 tx=0).stop 2024-08-02T17:53:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.536+0000 7fb78ff80740 1 -- 172.21.15.69:0/784777270 shutdown_connections 2024-08-02T17:53:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.536+0000 7fb78ff80740 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x56279246ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.536+0000 7fb78ff80740 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56279247fc10 0x562792482070 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.536+0000 7fb78ff80740 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 0x56279246d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.536+0000 7fb78ff80740 1 --2- 172.21.15.69:0/784777270 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627924647b0 0x56279246c910 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.536+0000 7fb78ff80740 1 -- 172.21.15.69:0/784777270 >> 172.21.15.69:0/784777270 conn(0x562792460d20 msgr2=0x562792466a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.536+0000 7fb76e7fc640 0 client.26674 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.538+0000 7fb78ff80740 1 -- 172.21.15.69:0/784777270 shutdown_connections 2024-08-02T17:53:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.538+0000 7fb78ff80740 1 -- 172.21.15.69:0/784777270 wait complete. 2024-08-02T17:53:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.538+0000 7fb78ff80740 20 client.26674 trim_cache size 0 max 16384 2024-08-02T17:53:58.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.575+0000 7fb78ff80740 1 Processor -- start 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb78ff80740 1 -- start start 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x56279246d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x562792487940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 0x562792487e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb78ff80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb768012960 con 0x5627924650d0 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb78ff80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb768010230 con 0x5627924647b0 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb78ff80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb76800e080 con 0x56279246f970 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb786d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x56279246d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb786d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x56279246d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60876/0 (socket says 172.21.15.69:60876) 2024-08-02T17:53:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.579+0000 7fb786d9d640 1 -- 172.21.15.69:0/3089107863 learned_addr learned my addr 172.21.15.69:0/3089107863 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.580+0000 7fb786d9d640 1 -- 172.21.15.69:0/3089107863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 msgr2=0x562792487e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:53:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.580+0000 7fb786d9d640 1 --2- 172.21.15.69:0/3089107863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 0x562792487e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.580+0000 7fb786d9d640 1 -- 172.21.15.69:0/3089107863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 msgr2=0x562792487940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.580+0000 7fb786d9d640 1 --2- 172.21.15.69:0/3089107863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x562792487940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.580+0000 7fb786d9d640 1 -- 172.21.15.69:0/3089107863 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562792464530 con 0x5627924647b0 2024-08-02T17:53:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.580+0000 7fb786d9d640 1 --2- 172.21.15.69:0/3089107863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x56279246d1c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb77009c1d0 tx=0x7fb77009be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a0b5dbab86545 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.582+0000 7fb77cff9640 1 -- 172.21.15.69:0/3089107863 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7700ab020 con 0x5627924647b0 2024-08-02T17:53:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.582+0000 7fb77cff9640 1 -- 172.21.15.69:0/3089107863 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7700a1b80 con 0x5627924647b0 2024-08-02T17:53:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.583+0000 7fb78ff80740 1 -- 172.21.15.69:0/3089107863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 msgr2=0x56279246d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:53:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.583+0000 7fb78ff80740 1 --2- 172.21.15.69:0/3089107863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x56279246d1c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb77009c1d0 tx=0x7fb77009be30 comp rx=0 tx=0).stop 2024-08-02T17:53:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.584+0000 7fb78ff80740 1 -- 172.21.15.69:0/3089107863 shutdown_connections 2024-08-02T17:53:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.584+0000 7fb78ff80740 1 --2- 172.21.15.69:0/3089107863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x562792487940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.584+0000 7fb78ff80740 1 --2- 172.21.15.69:0/3089107863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56279246f970 0x562792487e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.584+0000 7fb78ff80740 1 --2- 172.21.15.69:0/3089107863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627924647b0 0x56279246d1c0 secure :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb77009c1d0 tx=0x7fb77009be30 comp rx=0 tx=0).stop 2024-08-02T17:53:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.584+0000 7fb78ff80740 1 -- 172.21.15.69:0/3089107863 >> 172.21.15.69:0/3089107863 conn(0x562792460d20 msgr2=0x562792467bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:53:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.588+0000 7fb78ff80740 1 -- 172.21.15.69:0/3089107863 shutdown_connections 2024-08-02T17:53:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 -- 172.21.15.69:0/3089107863 wait complete. 2024-08-02T17:53:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 -- start start 2024-08-02T17:53:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 Processor -- start 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x562792477d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 0x5627924782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562792469390 0x562792469810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb76800e6f0 con 0x5627924650d0 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb76800fb60 con 0x56279246f970 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb76800edc0 con 0x562792469390 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.592+0000 7fb78ff80740 10 client.0 _ll_register_callbacks cb 0x7fb7700987f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562792469390 0x562792469810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562792469390 0x562792469810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:52894/0 (socket says 172.21.15.69:52894) 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78759e640 1 -- 172.21.15.69:0/107506686 learned_addr learned my addr 172.21.15.69:0/107506686 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78659c640 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 0x5627924782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78759e640 1 -- 172.21.15.69:0/107506686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 msgr2=0x5627924782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:53:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78759e640 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 0x5627924782a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78759e640 1 -- 172.21.15.69:0/107506686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 msgr2=0x562792477d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:53:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78759e640 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x562792477d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:53:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.593+0000 7fb78759e640 1 -- 172.21.15.69:0/107506686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56279246a010 con 0x562792469390 2024-08-02T17:53:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.594+0000 7fb78759e640 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562792469390 0x562792469810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb77800e5b0 tx=0x7fb77800ea70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f747e3eec46effdf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:53:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.594+0000 7fb76e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:53:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.594+0000 7fb76e7fc640 1 -- 172.21.15.69:0/107506686 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778017070 con 0x562792469390 2024-08-02T17:53:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.594+0000 7fb78ff80740 1 -- 172.21.15.69:0/107506686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7700a1b80 con 0x562792469390 2024-08-02T17:53:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.594+0000 7fb76e7fc640 1 -- 172.21.15.69:0/107506686 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9568+0+0 (secure 0 0 0) 0x7fb7780108b0 con 0x562792469390 2024-08-02T17:53:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.595+0000 7fb76e7fc640 1 -- 172.21.15.69:0/107506686 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb778023050 con 0x562792469390 2024-08-02T17:53:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb78ff80740 10 client.26677 fetch_fsmap learned FSMap version 26 2024-08-02T17:53:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb78ff80740 1 -- 172.21.15.69:0/107506686 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627924eaa50 con 0x562792469390 2024-08-02T17:53:58.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb76e7fc640 1 -- 172.21.15.69:0/107506686 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7700a1b80 con 0x562792469390 2024-08-02T17:53:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb76e7fc640 1 -- 172.21.15.69:0/107506686 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb77801b3b0 con 0x562792469390 2024-08-02T17:53:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb78ff80740 10 client.26677 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:53:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb78ff80740 10 client.26677 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:53:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb78ff80740 20 client.26677 populate_metadata read hostname 'smithi069' 2024-08-02T17:53:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb78ff80740 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627924eb200 0x5627924ed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:53:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb78ff80740 4 client.26677 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:53:58.600 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.597+0000 7fb78ff80740 1 -- 172.21.15.69:0/107506686 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627923d0430 con 0x5627924eb200 2024-08-02T17:53:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.602+0000 7fb786d9d640 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627924eb200 0x5627924ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:53:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.608+0000 7fb786d9d640 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627924eb200 0x5627924ed620 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be3404ed99a68b0 server_cookie=11085e7a34c41063 in_seq=0 out_seq=0 2024-08-02T17:53:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:53:58.609+0000 7fb76e7fc640 10 client.26677 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:53:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:53:58 smithi083 ceph-mon[37031]: pgmap v231: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-08-02T17:53:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:53:58 smithi077 ceph-mon[37108]: pgmap v231: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-08-02T17:53:59.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:53:58 smithi069 ceph-mon[32746]: pgmap v231: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-08-02T17:53:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 410.0 s, 856.1 tps, lat 99.275 ms stddev 139.391 2024-08-02T17:54:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:00 smithi083 ceph-mon[37031]: pgmap v232: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 8.5 MiB/s wr, 662 op/s 2024-08-02T17:54:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:00 smithi077 ceph-mon[37108]: pgmap v232: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 8.5 MiB/s wr, 662 op/s 2024-08-02T17:54:01.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:00 smithi069 ceph-mon[32746]: pgmap v232: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 8.5 MiB/s wr, 662 op/s 2024-08-02T17:54:03.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:02 smithi083 ceph-mon[37031]: pgmap v233: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 7.9 MiB/s wr, 595 op/s 2024-08-02T17:54:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:02 smithi077 ceph-mon[37108]: pgmap v233: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 7.9 MiB/s wr, 595 op/s 2024-08-02T17:54:03.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:02 smithi069 ceph-mon[32746]: pgmap v233: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 7.9 MiB/s wr, 595 op/s 2024-08-02T17:54:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb76e7fc640 1 -- 172.21.15.69:0/107506686 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627923d0430 con 0x5627924eb200 2024-08-02T17:54:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb76e7fc640 10 client.26677 handle_command_reply: tid=0 2024-08-02T17:54:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 client.26677 shutdown 2024-08-02T17:54:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 -- 172.21.15.69:0/107506686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562792469390 msgr2=0x562792469810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562792469390 0x562792469810 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb77800e5b0 tx=0x7fb77800ea70 comp rx=0 tx=0).stop 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 -- 172.21.15.69:0/107506686 shutdown_connections 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627924eb200 0x5627924ed620 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562792469390 0x562792469810 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56279246f970 0x5627924782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 --2- 172.21.15.69:0/107506686 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627924650d0 0x562792477d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb76e7fc640 0 client.26677 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 -- 172.21.15.69:0/107506686 >> 172.21.15.69:0/107506686 conn(0x562792460d20 msgr2=0x56279245f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 -- 172.21.15.69:0/107506686 shutdown_connections 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 1 -- 172.21.15.69:0/107506686 wait complete. 2024-08-02T17:54:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.422+0000 7fb78ff80740 20 client.26677 trim_cache size 0 max 16384 2024-08-02T17:54:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:54:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:54:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:54:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:54:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 -- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb7740a2f00 msgr2=0x7fb7740a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb7740a2f00 0x7fb7740a53c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb78001a6d0 tx=0x7fb780000bc0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 -- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788108f20 msgr2=0x7fb78811a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788108f20 0x7fb78811a6f0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb77000b450 tx=0x7fb77000b910 comp rx=0 tx=0).stop 2024-08-02T17:54:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 -- 172.21.15.69:0/673517289 shutdown_connections 2024-08-02T17:54:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb7740a2f00 0x7fb7740a53c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb78811ae70 0x7fb788116cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb788109300 0x7fb788116780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 --2- 172.21.15.69:0/673517289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb788108f20 0x7fb78811a6f0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 -- 172.21.15.69:0/673517289 >> 172.21.15.69:0/673517289 conn(0x7fb78806cb20 msgr2=0x7fb788071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 -- 172.21.15.69:0/673517289 shutdown_connections 2024-08-02T17:54:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:04.423+0000 7fb78d228640 1 -- 172.21.15.69:0/673517289 wait complete. 2024-08-02T17:54:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 415.0 s, 150.6 tps, lat 769.501 ms stddev 1839.636 2024-08-02T17:54:05.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:04 smithi083 ceph-mon[37031]: pgmap v234: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 10 MiB/s wr, 833 op/s 2024-08-02T17:54:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:04 smithi077 ceph-mon[37108]: pgmap v234: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 10 MiB/s wr, 833 op/s 2024-08-02T17:54:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:04 smithi069 ceph-mon[32746]: pgmap v234: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 10 MiB/s wr, 833 op/s 2024-08-02T17:54:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:05 smithi083 ceph-mon[37031]: osdmap e98: 12 total, 12 up, 12 in 2024-08-02T17:54:06.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:05 smithi077 ceph-mon[37108]: osdmap e98: 12 total, 12 up, 12 in 2024-08-02T17:54:06.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:05 smithi069 ceph-mon[32746]: osdmap e98: 12 total, 12 up, 12 in 2024-08-02T17:54:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:06 smithi083 ceph-mon[37031]: pgmap v236: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-08-02T17:54:07.272 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:06 smithi069 ceph-mon[32746]: pgmap v236: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-08-02T17:54:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:06 smithi077 ceph-mon[37108]: pgmap v236: 129 pgs: 129 active+clean; 3.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 7.1 MiB/s wr, 722 op/s 2024-08-02T17:54:08.086 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:54:08.602 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:09.068 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:08 smithi069 ceph-mon[32746]: pgmap v237: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-08-02T17:54:09.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.218+0000 7ff4ada82640 1 -- 172.21.15.69:0/1463914093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a8113670 msgr2=0x7ff4a8115a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.218+0000 7ff4ada82640 1 --2- 172.21.15.69:0/1463914093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a8113670 0x7ff4a8115a60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009be0 tx=0x7ff4a002a6f0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.222+0000 7ff4ada82640 1 -- 172.21.15.69:0/1463914093 shutdown_connections 2024-08-02T17:54:09.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.222+0000 7ff4ada82640 1 --2- 172.21.15.69:0/1463914093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4a810abb0 0x7ff4a81130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.222+0000 7ff4ada82640 1 --2- 172.21.15.69:0/1463914093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a8113670 0x7ff4a8115a60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.222+0000 7ff4ada82640 1 --2- 172.21.15.69:0/1463914093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a810a290 0x7ff4a810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.222+0000 7ff4ada82640 1 -- 172.21.15.69:0/1463914093 >> 172.21.15.69:0/1463914093 conn(0x7ff4a806de70 msgr2=0x7ff4a806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:09.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.223+0000 7ff4ada82640 1 -- 172.21.15.69:0/1463914093 shutdown_connections 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.223+0000 7ff4ada82640 1 -- 172.21.15.69:0/1463914093 wait complete. 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.224+0000 7ff4ada82640 1 Processor -- start 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.224+0000 7ff4ada82640 1 -- start start 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4ada82640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a810a290 0x7ff4a819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4ada82640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4a810abb0 0x7ff4a819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4ada82640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8113670 0x7ff4a81aa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4ada82640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8119360 con 0x7ff4a810abb0 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4ada82640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81191e0 con 0x7ff4a810a290 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4ada82640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81194e0 con 0x7ff4a8113670 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4a77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a810a290 0x7ff4a819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4a77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a810a290 0x7ff4a819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:41848/0 (socket says 172.21.15.69:41848) 2024-08-02T17:54:09.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.225+0000 7ff4a77fe640 1 -- 172.21.15.69:0/850197779 learned_addr learned my addr 172.21.15.69:0/850197779 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:09.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.226+0000 7ff4a77fe640 1 -- 172.21.15.69:0/850197779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8113670 msgr2=0x7ff4a81aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.226+0000 7ff4a77fe640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8113670 0x7ff4a81aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.226+0000 7ff4a77fe640 1 -- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4a810abb0 msgr2=0x7ff4a819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.226+0000 7ff4a77fe640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4a810abb0 0x7ff4a819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.230 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.226+0000 7ff4a77fe640 1 -- 172.21.15.69:0/850197779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81aab80 con 0x7ff4a810a290 2024-08-02T17:54:09.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.229+0000 7ff4a77fe640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a810a290 0x7ff4a819e830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00d950 tx=0x7ff49c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:09.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.230+0000 7ff4a4ff9640 1 -- 172.21.15.69:0/850197779 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c017070 con 0x7ff4a810a290 2024-08-02T17:54:09.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.230+0000 7ff4ada82640 1 -- 172.21.15.69:0/850197779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a81aae70 con 0x7ff4a810a290 2024-08-02T17:54:09.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.230+0000 7ff4ada82640 1 -- 172.21.15.69:0/850197779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81ab3b0 con 0x7ff4a810a290 2024-08-02T17:54:09.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.231+0000 7ff4a4ff9640 1 -- 172.21.15.69:0/850197779 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff49c009c30 con 0x7ff4a810a290 2024-08-02T17:54:09.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.232+0000 7ff4a4ff9640 1 -- 172.21.15.69:0/850197779 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00f620 con 0x7ff4a810a290 2024-08-02T17:54:09.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.234+0000 7ff4a4ff9640 1 -- 172.21.15.69:0/850197779 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff49c010460 con 0x7ff4a810a290 2024-08-02T17:54:09.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.242+0000 7ff4afe80740 1 Processor -- start 2024-08-02T17:54:09.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4afe80740 1 -- start start 2024-08-02T17:54:09.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523210 0x556be35235f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 0x556be35307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4afe80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556be3488980 con 0x556be3523b30 2024-08-02T17:54:09.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4afe80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556be3420a70 con 0x556be352e3f0 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4afe80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556be3484940 con 0x556be3523210 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523210 0x556be35235f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523210 0x556be35235f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42064/0 (socket says 172.21.15.69:42064) 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4a7fff640 1 -- 172.21.15.69:0/3409400345 learned_addr learned my addr 172.21.15.69:0/3409400345 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.243+0000 7ff4a6ffd640 1 --2- 172.21.15.69:0/3409400345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 0x556be35307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.244+0000 7ff4a7fff640 1 -- 172.21.15.69:0/3409400345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 msgr2=0x556be35307e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.244+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/3409400345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 0x556be35307e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.244+0000 7ff4a7fff640 1 -- 172.21.15.69:0/3409400345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 msgr2=0x556be352deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.244+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/3409400345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.244+0000 7ff4a7fff640 1 -- 172.21.15.69:0/3409400345 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556be34836e0 con 0x556be3523210 2024-08-02T17:54:09.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.244+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/3409400345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523210 0x556be35235f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002ac60 tx=0x7ff4a0030430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16d0226419c375f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:09.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.245+0000 7ff48dffb640 1 -- 172.21.15.69:0/3409400345 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0040020 con 0x556be3523210 2024-08-02T17:54:09.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.245+0000 7ff48dffb640 1 -- 172.21.15.69:0/3409400345 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0003e50 con 0x556be3523210 2024-08-02T17:54:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.245+0000 7ff4a4ff9640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4900a2d60 0x7ff4900a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4a4ff9640 1 -- 172.21.15.69:0/850197779 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9656+0+0 (secure 0 0 0) 0x7ff49c0cf530 con 0x7ff4a810a290 2024-08-02T17:54:09.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4afe80740 1 -- 172.21.15.69:0/3409400345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523210 msgr2=0x556be35235f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3409400345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523210 0x556be35235f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff4a002ac60 tx=0x7ff4a0030430 comp rx=0 tx=0).stop 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4a77fe640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4900a2d60 0x7ff4900a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4afe80740 1 -- 172.21.15.69:0/3409400345 shutdown_connections 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3409400345 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3409400345 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 0x556be35307e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3409400345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523210 0x556be35235f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.246+0000 7ff4afe80740 1 -- 172.21.15.69:0/3409400345 >> 172.21.15.69:0/3409400345 conn(0x556be351f7a0 msgr2=0x556be351fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.247+0000 7ff4afe80740 1 -- 172.21.15.69:0/3409400345 shutdown_connections 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.247+0000 7ff4afe80740 1 -- 172.21.15.69:0/3409400345 wait complete. 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.247+0000 7ff4a77fe640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4900a2d60 0x7ff4900a5220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x556be3520600 tx=0x7ff49c097940 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.247+0000 7ff4afe80740 1 -- start start 2024-08-02T17:54:09.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.247+0000 7ff4afe80740 1 Processor -- start 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be352e3f0 0x556be352afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 0x556be352b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4afe80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556be3531060 con 0x556be3523b30 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4afe80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556be3443aa0 con 0x556be352b4f0 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4afe80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556be35311e0 con 0x556be352e3f0 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4afe80740 10 client.0 _ll_register_callbacks cb 0x556be3432e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42560/0 (socket says 172.21.15.69:42560) 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 -- 172.21.15.69:0/3091184471 learned_addr learned my addr 172.21.15.69:0/3091184471 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a6ffd640 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be352e3f0 0x556be352afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 0x556be352b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 -- 172.21.15.69:0/3091184471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be352e3f0 msgr2=0x556be352afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be352e3f0 0x556be352afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 -- 172.21.15.69:0/3091184471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 msgr2=0x556be352b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 0x556be352b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.248+0000 7ff4a7fff640 1 -- 172.21.15.69:0/3091184471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556be3427650 con 0x556be3523b30 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.249+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352dd00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0005f00 tx=0x7ff4a0009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ae2275c9f8c170 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.249+0000 7ff486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.249+0000 7ff486ffd640 1 -- 172.21.15.69:0/3091184471 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0049020 con 0x556be3523b30 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.249+0000 7ff486ffd640 1 -- 172.21.15.69:0/3091184471 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9656+0+0 (secure 0 0 0) 0x7ff4a0004720 con 0x556be3523b30 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.249+0000 7ff4afe80740 1 -- 172.21.15.69:0/3091184471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556be3495ef0 con 0x556be3523b30 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.249+0000 7ff486ffd640 1 -- 172.21.15.69:0/3091184471 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4a005a050 con 0x556be3523b30 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.249+0000 7ff4afe80740 10 client.17265 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:09.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.249+0000 7ff4afe80740 1 -- 172.21.15.69:0/3091184471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556be31fa3c0 con 0x556be3523b30 2024-08-02T17:54:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff486ffd640 1 -- 172.21.15.69:0/3091184471 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0003e50 con 0x556be3523b30 2024-08-02T17:54:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff486ffd640 1 -- 172.21.15.69:0/3091184471 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff4a003c470 con 0x556be3523b30 2024-08-02T17:54:09.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff4afe80740 10 client.17265 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff4afe80740 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff4afe80740 20 client.17265 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556be353e0a0 0x556be3540490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff4afe80740 4 client.17265 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff4afe80740 1 -- 172.21.15.69:0/3091184471 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556be3495ef0 con 0x556be353e0a0 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.250+0000 7ff4a6ffd640 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556be353e0a0 0x556be3540490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.251+0000 7ff4a6ffd640 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556be353e0a0 0x556be3540490 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b961792c7736771 server_cookie=56029ba6b268d219 in_seq=0 out_seq=0 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.251+0000 7ff486ffd640 10 client.17265 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.251+0000 7ff486ffd640 1 -- 172.21.15.69:0/3091184471 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556be3495ef0 con 0x556be353e0a0 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.251+0000 7ff486ffd640 10 client.17265 handle_command_reply: tid=0 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.251+0000 7ff4afe80740 1 client.17265 shutdown 2024-08-02T17:54:09.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.251+0000 7ff4afe80740 1 -- 172.21.15.69:0/3091184471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 msgr2=0x556be352dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.251+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352dd00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0005f00 tx=0x7ff4a0009480 comp rx=0 tx=0).stop 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 1 -- 172.21.15.69:0/3091184471 shutdown_connections 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556be353e0a0 0x556be3540490 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 0x556be352b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff486ffd640 0 client.17265 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be352e3f0 0x556be352afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 1 --2- 172.21.15.69:0/3091184471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be3523b30 0x556be352dd00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 1 -- 172.21.15.69:0/3091184471 >> 172.21.15.69:0/3091184471 conn(0x556be351f7a0 msgr2=0x556be351e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 1 -- 172.21.15.69:0/3091184471 shutdown_connections 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 1 -- 172.21.15.69:0/3091184471 wait complete. 2024-08-02T17:54:09.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.252+0000 7ff4afe80740 20 client.17265 trim_cache size 0 max 16384 2024-08-02T17:54:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:08 smithi083 ceph-mon[37031]: pgmap v237: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-08-02T17:54:09.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.276+0000 7ff4afe80740 1 Processor -- start 2024-08-02T17:54:09.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4afe80740 1 -- start start 2024-08-02T17:54:09.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be352d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 0x556be3529880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352e3f0 0x556be3529dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4afe80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff494012a80 con 0x556be352e3f0 2024-08-02T17:54:09.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4afe80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff494010350 con 0x556be352b4f0 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be352d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be352d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42084/0 (socket says 172.21.15.69:42084) 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4a7fff640 1 -- 172.21.15.69:0/1852376028 learned_addr learned my addr 172.21.15.69:0/1852376028 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.277+0000 7ff4afe80740 1 -- 172.21.15.69:0/1852376028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff49400e1c0 con 0x556be3523b30 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.278+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352e3f0 0x556be3529dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.278+0000 7ff4a6ffd640 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 0x556be3529880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.278+0000 7ff4a7fff640 1 -- 172.21.15.69:0/1852376028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 msgr2=0x556be3529880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.278+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 0x556be3529880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.278+0000 7ff4a7fff640 1 -- 172.21.15.69:0/1852376028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352e3f0 msgr2=0x556be3529dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.278+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352e3f0 0x556be3529dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.278+0000 7ff4a7fff640 1 -- 172.21.15.69:0/1852376028 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556be3529500 con 0x556be3523b30 2024-08-02T17:54:09.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.279+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be352d820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009960 tx=0x7ff4a0000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf5f4f91b747d9c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:09.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.279+0000 7ff48d7fa640 1 -- 172.21.15.69:0/1852376028 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0049070 con 0x556be3523b30 2024-08-02T17:54:09.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.279+0000 7ff48d7fa640 1 -- 172.21.15.69:0/1852376028 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0030ea0 con 0x556be3523b30 2024-08-02T17:54:09.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.279+0000 7ff48d7fa640 1 -- 172.21.15.69:0/1852376028 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0040040 con 0x556be3523b30 2024-08-02T17:54:09.281 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 -- 172.21.15.69:0/1852376028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 msgr2=0x556be352d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be352d820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009960 tx=0x7ff4a0000f40 comp rx=0 tx=0).stop 2024-08-02T17:54:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 -- 172.21.15.69:0/1852376028 shutdown_connections 2024-08-02T17:54:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352e3f0 0x556be3529dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352b4f0 0x556be3529880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 --2- 172.21.15.69:0/1852376028 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be352d820 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 -- 172.21.15.69:0/1852376028 >> 172.21.15.69:0/1852376028 conn(0x556be351f7a0 msgr2=0x556be3522f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:09.282 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 -- 172.21.15.69:0/1852376028 shutdown_connections 2024-08-02T17:54:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 -- 172.21.15.69:0/1852376028 wait complete. 2024-08-02T17:54:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.280+0000 7ff4afe80740 1 -- start start 2024-08-02T17:54:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.281+0000 7ff4afe80740 1 Processor -- start 2024-08-02T17:54:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.281+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be3531880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.283 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.281+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352b4f0 0x556be3531dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.281+0000 7ff4afe80740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 0x556be35a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.281+0000 7ff4afe80740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff49400e810 con 0x556be352b4f0 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.281+0000 7ff4afe80740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff49400fc80 con 0x556be352e3f0 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.281+0000 7ff4afe80740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff49400eee0 con 0x556be3523b30 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.281+0000 7ff4afe80740 10 client.0 _ll_register_callbacks cb 0x7ff498009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352b4f0 0x556be3531dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be3531880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be3531880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42100/0 (socket says 172.21.15.69:42100) 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 -- 172.21.15.69:0/2473122376 learned_addr learned my addr 172.21.15.69:0/2473122376 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 0x556be35a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 -- 172.21.15.69:0/2473122376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 msgr2=0x556be35a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 0x556be35a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.284 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 -- 172.21.15.69:0/2473122376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352b4f0 msgr2=0x556be3531dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352b4f0 0x556be3531dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.285 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 -- 172.21.15.69:0/2473122376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556be35a6f30 con 0x556be3523b30 2024-08-02T17:54:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff4a7fff640 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be3531880 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0005d70 tx=0x7ff4a003cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed47fad8ec2f5341 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:09.287 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff486ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff486ffd640 1 -- 172.21.15.69:0/2473122376 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0040070 con 0x556be3523b30 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.282+0000 7ff486ffd640 1 -- 172.21.15.69:0/2473122376 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9656+0+0 (secure 0 0 0) 0x7ff4a0030bc0 con 0x556be3523b30 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 1 -- 172.21.15.69:0/2473122376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4a0030ea0 con 0x556be3523b30 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff486ffd640 1 -- 172.21.15.69:0/2473122376 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff4a0049040 con 0x556be3523b30 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 10 client.26692 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 1 -- 172.21.15.69:0/2473122376 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556be35a80b0 con 0x556be3523b30 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff486ffd640 1 -- 172.21.15.69:0/2473122376 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0030ea0 con 0x556be3523b30 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff486ffd640 1 -- 172.21.15.69:0/2473122376 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff4a00314d0 con 0x556be3523b30 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 10 client.26692 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 10 client.26692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 20 client.26692 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556be353e0a0 0x556be35aa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:09.288 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 4 client.26692 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.283+0000 7ff4afe80740 1 -- 172.21.15.69:0/2473122376 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556be3490400 con 0x556be353e0a0 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.284+0000 7ff4a6ffd640 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556be353e0a0 0x556be35aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.284+0000 7ff486ffd640 10 client.26692 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.284+0000 7ff4a6ffd640 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556be353e0a0 0x556be35aa670 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875c010db5248cca server_cookie=f97dedfb8f31cc0 in_seq=0 out_seq=0 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.284+0000 7ff486ffd640 1 -- 172.21.15.69:0/2473122376 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556be3490400 con 0x556be353e0a0 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.284+0000 7ff486ffd640 10 client.26692 handle_command_reply: tid=0 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 client.26692 shutdown 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 -- 172.21.15.69:0/2473122376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 msgr2=0x556be3531880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be3531880 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0005d70 tx=0x7ff4a003cab0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 -- 172.21.15.69:0/2473122376 shutdown_connections 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556be352b4f0 0x556be3531dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.289 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556be353e0a0 0x556be35aa670 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556be352e3f0 0x556be35a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 --2- 172.21.15.69:0/2473122376 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556be3523b30 0x556be3531880 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 -- 172.21.15.69:0/2473122376 >> 172.21.15.69:0/2473122376 conn(0x556be351f7a0 msgr2=0x556be351e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff486ffd640 0 client.26692 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 -- 172.21.15.69:0/2473122376 shutdown_connections 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 1 -- 172.21.15.69:0/2473122376 wait complete. 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.285+0000 7ff4afe80740 20 client.26692 trim_cache size 0 max 16384 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:54:09.290 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:54:09.291 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:54:09.291 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:54:09.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.286+0000 7ff4ada82640 1 -- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4900a2d60 msgr2=0x7ff4900a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.286+0000 7ff4ada82640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4900a2d60 0x7ff4900a5220 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x556be3520600 tx=0x7ff49c097940 comp rx=0 tx=0).stop 2024-08-02T17:54:09.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.286+0000 7ff4ada82640 1 -- 172.21.15.69:0/850197779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a810a290 msgr2=0x7ff4a819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:09.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.286+0000 7ff4ada82640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a810a290 0x7ff4a819e830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00d950 tx=0x7ff49c00de10 comp rx=0 tx=0).stop 2024-08-02T17:54:09.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.286+0000 7ff4ada82640 1 -- 172.21.15.69:0/850197779 shutdown_connections 2024-08-02T17:54:09.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.286+0000 7ff4ada82640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4a810abb0 0x7ff4a819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.287+0000 7ff4ada82640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4900a2d60 0x7ff4900a5220 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.287+0000 7ff4ada82640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4a8113670 0x7ff4a81aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.287+0000 7ff4ada82640 1 --2- 172.21.15.69:0/850197779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4a810a290 0x7ff4a819e830 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:09.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.287+0000 7ff4ada82640 1 -- 172.21.15.69:0/850197779 >> 172.21.15.69:0/850197779 conn(0x7ff4a806de70 msgr2=0x7ff4a80714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:09.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.287+0000 7ff4ada82640 1 -- 172.21.15.69:0/850197779 shutdown_connections 2024-08-02T17:54:09.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:09.287+0000 7ff4ada82640 1 -- 172.21.15.69:0/850197779 wait complete. 2024-08-02T17:54:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:08 smithi077 ceph-mon[37108]: pgmap v237: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.3 MiB/s wr, 566 op/s 2024-08-02T17:54:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 420.0 s, 727.0 tps, lat 136.567 ms stddev 203.033 2024-08-02T17:54:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:10 smithi069 ceph-mon[32746]: pgmap v238: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-08-02T17:54:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:10 smithi083 ceph-mon[37031]: pgmap v238: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-08-02T17:54:11.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:10 smithi077 ceph-mon[37108]: pgmap v238: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.5 MiB/s wr, 570 op/s 2024-08-02T17:54:13.228 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:12 smithi069 ceph-mon[32746]: pgmap v239: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-08-02T17:54:13.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:12 smithi083 ceph-mon[37031]: pgmap v239: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-08-02T17:54:13.286 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:54:13.287 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:54:13.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:12 smithi077 ceph-mon[37108]: pgmap v239: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 5.5 MiB/s wr, 564 op/s 2024-08-02T17:54:13.776 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:14.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.753+0000 7f5f9d49c640 1 -- 172.21.15.69:0/379333694 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90040c50 con 0x7f5f98109560 2024-08-02T17:54:14.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 -- 172.21.15.69:0/379333694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f98109560 msgr2=0x7f5f981099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:14.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 --2- 172.21.15.69:0/379333694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f98109560 0x7f5f981099c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5f9001acb0 tx=0x7f5f9003b920 comp rx=0 tx=0).stop 2024-08-02T17:54:14.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 -- 172.21.15.69:0/379333694 shutdown_connections 2024-08-02T17:54:14.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 --2- 172.21.15.69:0/379333694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9811ae70 0x7f5f9811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:14.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 --2- 172.21.15.69:0/379333694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f98109560 0x7f5f981099c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:14.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 --2- 172.21.15.69:0/379333694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f98109180 0x7f5f9810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:14.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 -- 172.21.15.69:0/379333694 >> 172.21.15.69:0/379333694 conn(0x7f5f9806de70 msgr2=0x7f5f9806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 -- 172.21.15.69:0/379333694 shutdown_connections 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.754+0000 7f5fa0702640 1 -- 172.21.15.69:0/379333694 wait complete. 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.755+0000 7f5fa0702640 1 Processor -- start 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.755+0000 7f5fa0702640 1 -- start start 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.755+0000 7f5fa0702640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f98109180 0x7f5f9811a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.755+0000 7f5fa0702640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f98109560 0x7f5f98116670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.755+0000 7f5fa0702640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9811ae70 0x7f5f98116bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.755+0000 7f5fa0702640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98113070 con 0x7f5f98109180 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.755+0000 7f5fa0702640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98112ef0 con 0x7f5f98109560 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.755+0000 7f5fa0702640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f981131f0 con 0x7f5f9811ae70 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.756+0000 7f5f9ec9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9811ae70 0x7f5f98116bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:14.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.756+0000 7f5f9ec9f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9811ae70 0x7f5f98116bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54418/0 (socket says 172.21.15.69:54418) 2024-08-02T17:54:14.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.756+0000 7f5f9ec9f640 1 -- 172.21.15.69:0/1161080435 learned_addr learned my addr 172.21.15.69:0/1161080435 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:14.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.756+0000 7f5f9dc9d640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f98109560 0x7f5f98116670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:14.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.757+0000 7f5f9e49e640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f98109180 0x7f5f9811a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:14.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.757+0000 7f5f9dc9d640 1 -- 172.21.15.69:0/1161080435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9811ae70 msgr2=0x7f5f98116bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:14.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.757+0000 7f5f9dc9d640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9811ae70 0x7f5f98116bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:14.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.757+0000 7f5f9dc9d640 1 -- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f98109180 msgr2=0x7f5f9811a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:14.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.757+0000 7f5f9dc9d640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f98109180 0x7f5f9811a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:14.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.757+0000 7f5f9dc9d640 1 -- 172.21.15.69:0/1161080435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f981172e0 con 0x7f5f98109560 2024-08-02T17:54:14.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.757+0000 7f5f9dc9d640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f98109560 0x7f5f98116670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005e30 tx=0x7f5f90040b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:14.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.758+0000 7f5f8f7fe640 1 -- 172.21.15.69:0/1161080435 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f90045070 con 0x7f5f98109560 2024-08-02T17:54:14.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.758+0000 7f5f8f7fe640 1 -- 172.21.15.69:0/1161080435 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f90040d10 con 0x7f5f98109560 2024-08-02T17:54:14.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.768+0000 7f5f8f7fe640 1 -- 172.21.15.69:0/1161080435 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9004a720 con 0x7f5f98109560 2024-08-02T17:54:14.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.768+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f981aaea0 con 0x7f5f98109560 2024-08-02T17:54:14.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.769+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f981ab240 con 0x7f5f98109560 2024-08-02T17:54:14.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.771+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f98073830 con 0x7f5f98109560 2024-08-02T17:54:14.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.773+0000 7f5f8f7fe640 1 -- 172.21.15.69:0/1161080435 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f90048020 con 0x7f5f98109560 2024-08-02T17:54:14.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.774+0000 7f5f8f7fe640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f840a2ca0 0x7f5f840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:14.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.775+0000 7f5f8f7fe640 1 -- 172.21.15.69:0/1161080435 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f5f901015b0 con 0x7f5f98109560 2024-08-02T17:54:14.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.775+0000 7f5f9e49e640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f840a2ca0 0x7f5f840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:14.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.775+0000 7f5f9e49e640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f840a2ca0 0x7f5f840a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009bd0 tx=0x7f5f94005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:14.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:14.778+0000 7f5f8f7fe640 1 -- 172.21.15.69:0/1161080435 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5f9003e070 con 0x7f5f98109560 2024-08-02T17:54:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 425.0 s, 568.4 tps, lat 176.549 ms stddev 304.177 2024-08-02T17:54:15.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.140+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f981099c0 con 0x7f5f98109560 2024-08-02T17:54:15.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.143+0000 7f5f8f7fe640 1 -- 172.21.15.69:0/1161080435 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f5f900cdf10 con 0x7f5f98109560 2024-08-02T17:54:15.145 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:54:15.145 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:54:15.145 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:54:15.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.148+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f840a2ca0 msgr2=0x7f5f840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:15.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.149+0000 7f5fa0702640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f840a2ca0 0x7f5f840a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5f94009bd0 tx=0x7f5f94005f90 comp rx=0 tx=0).stop 2024-08-02T17:54:15.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.149+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f98109560 msgr2=0x7f5f98116670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.149+0000 7f5fa0702640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f98109560 0x7f5f98116670 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5f90005e30 tx=0x7f5f90040b20 comp rx=0 tx=0).stop 2024-08-02T17:54:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.150+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 shutdown_connections 2024-08-02T17:54:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.150+0000 7f5fa0702640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f840a2ca0 0x7f5f840a5160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.151+0000 7f5fa0702640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f9811ae70 0x7f5f98116bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.151+0000 7f5fa0702640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f98109560 0x7f5f98116670 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.151+0000 7f5fa0702640 1 --2- 172.21.15.69:0/1161080435 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f98109180 0x7f5f9811a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:15.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.151+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 >> 172.21.15.69:0/1161080435 conn(0x7f5f9806de70 msgr2=0x7f5f980709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:15.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.157+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 shutdown_connections 2024-08-02T17:54:15.158 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:15.157+0000 7f5fa0702640 1 -- 172.21.15.69:0/1161080435 wait complete. 2024-08-02T17:54:15.173 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:14 smithi069 ceph-mon[32746]: pgmap v240: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.2 MiB/s wr, 484 op/s 2024-08-02T17:54:15.174 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:14 smithi069 ceph-mon[32746]: osdmap e99: 12 total, 12 up, 12 in 2024-08-02T17:54:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:14 smithi083 ceph-mon[37031]: pgmap v240: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.2 MiB/s wr, 484 op/s 2024-08-02T17:54:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:14 smithi083 ceph-mon[37031]: osdmap e99: 12 total, 12 up, 12 in 2024-08-02T17:54:15.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:14 smithi077 ceph-mon[37108]: pgmap v240: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 5.2 MiB/s wr, 484 op/s 2024-08-02T17:54:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:14 smithi077 ceph-mon[37108]: osdmap e99: 12 total, 12 up, 12 in 2024-08-02T17:54:16.001 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:54:16.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:15 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1161080435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:16.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:15 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1161080435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:16.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:15 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1161080435' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:16.405 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 -- 172.21.15.69:0/1491967901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981078a0 msgr2=0x7ff398107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/1491967901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981078a0 0x7ff398107c80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009930 tx=0x7ff38c02a420 comp rx=0 tx=0).stop 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 -- 172.21.15.69:0/1491967901 shutdown_connections 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/1491967901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff398102240 0x7ff39810e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/1491967901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3981018a0 0x7ff398101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/1491967901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981078a0 0x7ff398107c80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 -- 172.21.15.69:0/1491967901 >> 172.21.15.69:0/1491967901 conn(0x7ff3980fd5e0 msgr2=0x7ff3980ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 -- 172.21.15.69:0/1491967901 shutdown_connections 2024-08-02T17:54:16.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.916+0000 7ff39f0b7640 1 -- 172.21.15.69:0/1491967901 wait complete. 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39f0b7640 1 Processor -- start 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39f0b7640 1 -- start start 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39f0b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981018a0 0x7ff398074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39f0b7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398102240 0x7ff398074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39f0b7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3981078a0 0x7ff398074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39f0b7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff398111cf0 con 0x7ff3981018a0 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39f0b7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff398111b70 con 0x7ff3981078a0 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39f0b7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff398111e70 con 0x7ff398102240 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39ce53640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981018a0 0x7ff398074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39ce53640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981018a0 0x7ff398074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48584/0 (socket says 172.21.15.69:48584) 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39ce53640 1 -- 172.21.15.69:0/3708273401 learned_addr learned my addr 172.21.15.69:0/3708273401 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff397fff640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398102240 0x7ff398074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.917+0000 7ff39d654640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3981078a0 0x7ff398074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff39ce53640 1 -- 172.21.15.69:0/3708273401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398102240 msgr2=0x7ff398074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff39ce53640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398102240 0x7ff398074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff39ce53640 1 -- 172.21.15.69:0/3708273401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3981078a0 msgr2=0x7ff398074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff39ce53640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3981078a0 0x7ff398074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff39ce53640 1 -- 172.21.15.69:0/3708273401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3981a66b0 con 0x7ff3981018a0 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff39ce53640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981018a0 0x7ff398074220 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff38c02a940 tx=0x7ff38c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff395ffb640 1 -- 172.21.15.69:0/3708273401 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c03f070 con 0x7ff3981018a0 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff395ffb640 1 -- 172.21.15.69:0/3708273401 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff38c031c90 con 0x7ff3981018a0 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff395ffb640 1 -- 172.21.15.69:0/3708273401 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c035030 con 0x7ff3981018a0 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff39f0b7640 1 -- 172.21.15.69:0/3708273401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3981a6940 con 0x7ff3981018a0 2024-08-02T17:54:16.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.918+0000 7ff39f0b7640 1 -- 172.21.15.69:0/3708273401 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3981a6dd0 con 0x7ff3981018a0 2024-08-02T17:54:16.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff395ffb640 1 -- 172.21.15.69:0/3708273401 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff38c042070 con 0x7ff3981018a0 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff3a14b5740 1 Processor -- start 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff3a14b5740 1 -- start start 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac433d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac4dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac4e1d0 0x56552ac505c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56552aba8a40 con 0x56552ac42ff0 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56552ab409d0 con 0x56552ac43910 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56552aba4a00 con 0x56552ac4e1d0 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff395ffb640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3880a2cf0 0x7ff3880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.921+0000 7ff39d654640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff39d654640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac433d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48588/0 (socket says 172.21.15.69:48588) 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff39d654640 1 -- 172.21.15.69:0/414430567 learned_addr learned my addr 172.21.15.69:0/414430567 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff397fff640 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac4e1d0 0x56552ac505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff39d654640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3880a2cf0 0x7ff3880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff39ce53640 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac4dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff395ffb640 1 -- 172.21.15.69:0/3708273401 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff38c0eff30 con 0x7ff3981018a0 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff397fff640 1 -- 172.21.15.69:0/414430567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 msgr2=0x56552ac4dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff397fff640 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac4dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff397fff640 1 -- 172.21.15.69:0/414430567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 msgr2=0x56552ac433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff397fff640 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff397fff640 1 -- 172.21.15.69:0/414430567 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56552ab47650 con 0x56552ac4e1d0 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff39d654640 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:54:16.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.922+0000 7ff397fff640 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac4e1d0 0x56552ac505c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff384005fd0 tx=0x7ff38400bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471f950638647549 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.923+0000 7ff39d654640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3880a2cf0 0x7ff3880a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3900099e0 tx=0x7ff390009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:16.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.923+0000 7ff382ffd640 1 -- 172.21.15.69:0/414430567 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384003bd0 con 0x56552ac4e1d0 2024-08-02T17:54:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.926+0000 7ff382ffd640 1 -- 172.21.15.69:0/414430567 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384015070 con 0x56552ac4e1d0 2024-08-02T17:54:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.926+0000 7ff382ffd640 1 -- 172.21.15.69:0/414430567 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384004d20 con 0x56552ac4e1d0 2024-08-02T17:54:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 -- 172.21.15.69:0/414430567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac4e1d0 msgr2=0x56552ac505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac4e1d0 0x56552ac505c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff384005fd0 tx=0x7ff38400bb20 comp rx=0 tx=0).stop 2024-08-02T17:54:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 -- 172.21.15.69:0/414430567 shutdown_connections 2024-08-02T17:54:16.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac4e1d0 0x56552ac505c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac4dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/414430567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 -- 172.21.15.69:0/414430567 >> 172.21.15.69:0/414430567 conn(0x56552ac3f580 msgr2=0x56552ac3f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 -- 172.21.15.69:0/414430567 shutdown_connections 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 -- 172.21.15.69:0/414430567 wait complete. 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.927+0000 7ff3a14b5740 1 -- start start 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff3a14b5740 1 Processor -- start 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac4b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac43910 0x56552ac4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac4e1d0 0x56552ac4bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56552ac50da0 con 0x56552ac42ff0 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56552ab63b20 con 0x56552ac4e1d0 2024-08-02T17:54:16.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56552ac50f20 con 0x56552ac43910 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff3a14b5740 10 client.0 _ll_register_callbacks cb 0x56552ab61bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39ce53640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac4b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff397fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac43910 0x56552ac4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39ce53640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac4b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48596/0 (socket says 172.21.15.69:48596) 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39ce53640 1 -- 172.21.15.69:0/1704772948 learned_addr learned my addr 172.21.15.69:0/1704772948 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39d654640 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac4e1d0 0x56552ac4bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39ce53640 1 -- 172.21.15.69:0/1704772948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac43910 msgr2=0x56552ac4b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39ce53640 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac43910 0x56552ac4b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39ce53640 1 -- 172.21.15.69:0/1704772948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac4e1d0 msgr2=0x56552ac4bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39ce53640 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac4e1d0 0x56552ac4bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.928+0000 7ff39ce53640 1 -- 172.21.15.69:0/1704772948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56552a91a3c0 con 0x56552ac42ff0 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff39ce53640 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac4b0a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff38c02a9e0 tx=0x7ff38c031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2390e9e56f9472f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff377fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:16.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff377fff640 1 -- 172.21.15.69:0/1704772948 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c049040 con 0x56552ac42ff0 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff377fff640 1 -- 172.21.15.69:0/1704772948 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff38c0b8760 con 0x56552ac42ff0 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 1 -- 172.21.15.69:0/1704772948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff384015070 con 0x56552ac42ff0 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff377fff640 1 -- 172.21.15.69:0/1704772948 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff38c0bc0f0 con 0x56552ac42ff0 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 10 client.17289 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 1 -- 172.21.15.69:0/1704772948 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56552ac5c9a0 con 0x56552ac42ff0 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff377fff640 1 -- 172.21.15.69:0/1704772948 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384015070 con 0x56552ac42ff0 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff377fff640 1 -- 172.21.15.69:0/1704772948 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff38c03bce0 con 0x56552ac42ff0 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 10 client.17289 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 10 client.17289 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 20 client.17289 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56552ac5dd90 0x56552ac601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 4 client.17289 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:16.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.929+0000 7ff3a14b5740 1 -- 172.21.15.69:0/1704772948 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56552abb5fb0 con 0x56552ac5dd90 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.930+0000 7ff397fff640 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56552ac5dd90 0x56552ac601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.930+0000 7ff397fff640 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56552ac5dd90 0x56552ac601e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=166e72d000adf8c9 server_cookie=d44eeb2ac12a65b2 in_seq=0 out_seq=0 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.930+0000 7ff377fff640 10 client.17289 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff377fff640 1 -- 172.21.15.69:0/1704772948 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56552abb5fb0 con 0x56552ac5dd90 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff377fff640 10 client.17289 handle_command_reply: tid=0 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 client.17289 shutdown 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 -- 172.21.15.69:0/1704772948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 msgr2=0x56552ac4b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac4b0a0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff38c02a9e0 tx=0x7ff38c031490 comp rx=0 tx=0).stop 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 -- 172.21.15.69:0/1704772948 shutdown_connections 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56552ac5dd90 0x56552ac601e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac4e1d0 0x56552ac4bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff377fff640 0 client.17289 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:54:16.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac43910 0x56552ac4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/1704772948 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac42ff0 0x56552ac4b0a0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 -- 172.21.15.69:0/1704772948 >> 172.21.15.69:0/1704772948 conn(0x56552ac3f580 msgr2=0x56552ac452b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 -- 172.21.15.69:0/1704772948 shutdown_connections 2024-08-02T17:54:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 1 -- 172.21.15.69:0/1704772948 wait complete. 2024-08-02T17:54:16.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.931+0000 7ff3a14b5740 20 client.17289 trim_cache size 0 max 16384 2024-08-02T17:54:16.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.954+0000 7ff3a14b5740 1 Processor -- start 2024-08-02T17:54:16.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.954+0000 7ff3a14b5740 1 -- start start 2024-08-02T17:54:16.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.955+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac45aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.955+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac65ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.956+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 0x56552ac66000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.956+0000 7ff397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac65ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.956+0000 7ff39ce53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac45aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.956+0000 7ff397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac65ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:51156/0 (socket says 172.21.15.69:51156) 2024-08-02T17:54:16.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.956+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981b38c0 con 0x56552ac4e1d0 2024-08-02T17:54:16.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.957+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981b1190 con 0x56552ac43910 2024-08-02T17:54:16.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.957+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981aefe0 con 0x56552ac42ff0 2024-08-02T17:54:16.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.958+0000 7ff39ce53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac45aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54476/0 (socket says 172.21.15.69:54476) 2024-08-02T17:54:16.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.958+0000 7ff39ce53640 1 -- 172.21.15.69:0/129965716 learned_addr learned my addr 172.21.15.69:0/129965716 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:16.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.959+0000 7ff397fff640 1 -- 172.21.15.69:0/129965716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 msgr2=0x56552ac45aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:16.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.959+0000 7ff397fff640 1 --2- 172.21.15.69:0/129965716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac45aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.959+0000 7ff397fff640 1 -- 172.21.15.69:0/129965716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 msgr2=0x56552ac66000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:16.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.959+0000 7ff397fff640 1 --2- 172.21.15.69:0/129965716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 0x56552ac66000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.959+0000 7ff397fff640 1 -- 172.21.15.69:0/129965716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56552ac4db50 con 0x56552ac43910 2024-08-02T17:54:16.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.962+0000 7ff397fff640 1 --2- 172.21.15.69:0/129965716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac65ac0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff384004880 tx=0x7ff38400b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17db9c854693e937 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:16.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.971+0000 7ff3827fc640 1 -- 172.21.15.69:0/129965716 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384003dd0 con 0x56552ac43910 2024-08-02T17:54:16.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.971+0000 7ff3827fc640 1 -- 172.21.15.69:0/129965716 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384029030 con 0x56552ac43910 2024-08-02T17:54:16.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.972+0000 7ff3a14b5740 1 -- 172.21.15.69:0/129965716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 msgr2=0x56552ac65ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:16.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.972+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/129965716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac65ac0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff384004880 tx=0x7ff38400b420 comp rx=0 tx=0).stop 2024-08-02T17:54:16.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.974+0000 7ff3a14b5740 1 -- 172.21.15.69:0/129965716 shutdown_connections 2024-08-02T17:54:16.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.974+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/129965716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 0x56552ac66000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.974+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/129965716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac65ac0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.974+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/129965716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac45aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.974+0000 7ff3a14b5740 1 -- 172.21.15.69:0/129965716 >> 172.21.15.69:0/129965716 conn(0x56552ac3f580 msgr2=0x56552ac49970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:16.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.980+0000 7ff3a14b5740 1 -- 172.21.15.69:0/129965716 shutdown_connections 2024-08-02T17:54:16.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.980+0000 7ff3a14b5740 1 -- 172.21.15.69:0/129965716 wait complete. 2024-08-02T17:54:16.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.981+0000 7ff3a14b5740 1 -- start start 2024-08-02T17:54:16.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.981+0000 7ff3a14b5740 1 Processor -- start 2024-08-02T17:54:16.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.981+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac51650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.982+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac51b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.982+0000 7ff3a14b5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 0x56552accf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.982+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981af650 con 0x56552ac4e1d0 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.982+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981b0ac0 con 0x56552ac43910 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.982+0000 7ff3a14b5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981afd20 con 0x56552ac42ff0 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.982+0000 7ff3a14b5740 10 client.0 _ll_register_callbacks cb 0x7ff38c009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.982+0000 7ff39ce53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac51650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.982+0000 7ff397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac51b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.983+0000 7ff397fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac51b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:51170/0 (socket says 172.21.15.69:51170) 2024-08-02T17:54:16.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.983+0000 7ff397fff640 1 -- 172.21.15.69:0/872050351 learned_addr learned my addr 172.21.15.69:0/872050351 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:16.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.984+0000 7ff39d654640 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 0x56552accf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:16.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.989+0000 7ff39d654640 1 -- 172.21.15.69:0/872050351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 msgr2=0x56552ac51650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:16.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.989+0000 7ff39d654640 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac51650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.989+0000 7ff39d654640 1 -- 172.21.15.69:0/872050351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 msgr2=0x56552ac51b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:16.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.989+0000 7ff39d654640 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac51b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:16.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.989+0000 7ff39d654640 1 -- 172.21.15.69:0/872050351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56552acc0a50 con 0x56552ac4e1d0 2024-08-02T17:54:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.996+0000 7ff39d654640 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 0x56552accf290 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff3900099b0 tx=0x7ff390009c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49f3973307082f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:16.998+0000 7ff377fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.002+0000 7ff377fff640 1 -- 172.21.15.69:0/872050351 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390004b20 con 0x56552ac4e1d0 2024-08-02T17:54:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.002+0000 7ff3a14b5740 1 -- 172.21.15.69:0/872050351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff384029030 con 0x56552ac4e1d0 2024-08-02T17:54:17.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.002+0000 7ff377fff640 1 -- 172.21.15.69:0/872050351 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff390004cc0 con 0x56552ac4e1d0 2024-08-02T17:54:17.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.009+0000 7ff377fff640 1 -- 172.21.15.69:0/872050351 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff390035050 con 0x56552ac4e1d0 2024-08-02T17:54:17.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.013+0000 7ff3a14b5740 10 client.17292 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:17.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.013+0000 7ff3a14b5740 1 -- 172.21.15.69:0/872050351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56552acc1d50 con 0x56552ac4e1d0 2024-08-02T17:54:17.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.013+0000 7ff377fff640 1 -- 172.21.15.69:0/872050351 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384029030 con 0x56552ac4e1d0 2024-08-02T17:54:17.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.018+0000 7ff377fff640 1 -- 172.21.15.69:0/872050351 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff39001d040 con 0x56552ac4e1d0 2024-08-02T17:54:17.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.018+0000 7ff3a14b5740 10 client.17292 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:17.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.018+0000 7ff3a14b5740 10 client.17292 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:54:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.018+0000 7ff3a14b5740 20 client.17292 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.018+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56552ac5dd90 0x56552acc4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.018+0000 7ff3a14b5740 4 client.17292 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:54:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.018+0000 7ff3a14b5740 1 -- 172.21.15.69:0/872050351 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56552abb04c0 con 0x56552ac5dd90 2024-08-02T17:54:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.022+0000 7ff39ce53640 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56552ac5dd90 0x56552acc4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.023+0000 7ff39ce53640 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56552ac5dd90 0x56552acc4510 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6c2fd7e5c186ed9 server_cookie=6f7f21e54919f774 in_seq=0 out_seq=0 2024-08-02T17:54:17.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.023+0000 7ff377fff640 10 client.17292 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:54:17.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.024+0000 7ff377fff640 1 -- 172.21.15.69:0/872050351 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56552abb04c0 con 0x56552ac5dd90 2024-08-02T17:54:17.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.024+0000 7ff377fff640 10 client.17292 handle_command_reply: tid=0 2024-08-02T17:54:17.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.024+0000 7ff3a14b5740 1 client.17292 shutdown 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff3a14b5740 1 -- 172.21.15.69:0/872050351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 msgr2=0x56552accf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 0x56552accf290 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff3900099b0 tx=0x7ff390009c10 comp rx=0 tx=0).stop 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff3a14b5740 1 -- 172.21.15.69:0/872050351 shutdown_connections 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56552ac5dd90 0x56552acc4510 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56552ac4e1d0 0x56552accf290 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56552ac43910 0x56552ac51b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff3a14b5740 1 --2- 172.21.15.69:0/872050351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56552ac42ff0 0x56552ac51650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff3a14b5740 1 -- 172.21.15.69:0/872050351 >> 172.21.15.69:0/872050351 conn(0x56552ac3f580 msgr2=0x56552ac3dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:17.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.029+0000 7ff377fff640 0 client.17292 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:54:17.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.032+0000 7ff3a14b5740 1 -- 172.21.15.69:0/872050351 shutdown_connections 2024-08-02T17:54:17.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.032+0000 7ff3a14b5740 1 -- 172.21.15.69:0/872050351 wait complete. 2024-08-02T17:54:17.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.033+0000 7ff3a14b5740 20 client.17292 trim_cache size 0 max 16384 2024-08-02T17:54:17.037 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 -- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3880a2cf0 msgr2=0x7ff3880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3880a2cf0 0x7ff3880a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff3900099e0 tx=0x7ff390009500 comp rx=0 tx=0).stop 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 -- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981018a0 msgr2=0x7ff398074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981018a0 0x7ff398074220 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff38c02a940 tx=0x7ff38c003b90 comp rx=0 tx=0).stop 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 -- 172.21.15.69:0/3708273401 shutdown_connections 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff3880a2cf0 0x7ff3880a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff3981078a0 0x7ff398074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff398102240 0x7ff398074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 --2- 172.21.15.69:0/3708273401 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff3981018a0 0x7ff398074220 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 -- 172.21.15.69:0/3708273401 >> 172.21.15.69:0/3708273401 conn(0x7ff3980fd5e0 msgr2=0x7ff3980fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:17.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 -- 172.21.15.69:0/3708273401 shutdown_connections 2024-08-02T17:54:17.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:17.037+0000 7ff39f0b7640 1 -- 172.21.15.69:0/3708273401 wait complete. 2024-08-02T17:54:17.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:16 smithi069 ceph-mon[32746]: pgmap v242: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 5.5 MiB/s wr, 477 op/s 2024-08-02T17:54:17.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:16 smithi083 ceph-mon[37031]: pgmap v242: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 5.5 MiB/s wr, 477 op/s 2024-08-02T17:54:17.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:16 smithi077 ceph-mon[37108]: pgmap v242: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 5.5 MiB/s wr, 477 op/s 2024-08-02T17:54:17.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 -- 172.21.15.69:0/4263853556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5288112550 msgr2=0x7f5288114940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 --2- 172.21.15.69:0/4263853556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5288112550 0x7f5288114940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5280009ca0 tx=0x7f528002a950 comp rx=0 tx=0).stop 2024-08-02T17:54:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 -- 172.21.15.69:0/4263853556 shutdown_connections 2024-08-02T17:54:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 --2- 172.21.15.69:0/4263853556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52881001e0 0x7f5288112010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 --2- 172.21.15.69:0/4263853556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5288112550 0x7f5288114940 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 --2- 172.21.15.69:0/4263853556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52880ff8c0 0x7f52880ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 -- 172.21.15.69:0/4263853556 >> 172.21.15.69:0/4263853556 conn(0x7f52880fb760 msgr2=0x7f52880fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 -- 172.21.15.69:0/4263853556 shutdown_connections 2024-08-02T17:54:17.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.559+0000 7f528e5d7640 1 -- 172.21.15.69:0/4263853556 wait complete. 2024-08-02T17:54:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528e5d7640 1 Processor -- start 2024-08-02T17:54:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528e5d7640 1 -- start start 2024-08-02T17:54:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528e5d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52880ff8c0 0x7f52881a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528e5d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52881001e0 0x7f52881a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528e5d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5288112550 0x7f528819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528e5d7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5288118270 con 0x7f5288112550 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528e5d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52881180f0 con 0x7f52880ff8c0 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528e5d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52881183f0 con 0x7f52881001e0 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f5287fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52880ff8c0 0x7f52881a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f528cb74640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5288112550 0x7f528819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f5287fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52880ff8c0 0x7f52881a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:51186/0 (socket says 172.21.15.69:51186) 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f5287fff640 1 -- 172.21.15.69:0/1350992743 learned_addr learned my addr 172.21.15.69:0/1350992743 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.560+0000 7f52877fe640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52881001e0 0x7f52881a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.561+0000 7f528cb74640 1 -- 172.21.15.69:0/1350992743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52881001e0 msgr2=0x7f52881a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.561+0000 7f528cb74640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52881001e0 0x7f52881a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.561+0000 7f528cb74640 1 -- 172.21.15.69:0/1350992743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52880ff8c0 msgr2=0x7f52881a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.561+0000 7f528cb74640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52880ff8c0 0x7f52881a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.561+0000 7f528cb74640 1 -- 172.21.15.69:0/1350992743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528819ed20 con 0x7f5288112550 2024-08-02T17:54:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.561+0000 7f528cb74640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5288112550 0x7f528819e5a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5280005cc0 tx=0x7f528002f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:17.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.562+0000 7f52857fa640 1 -- 172.21.15.69:0/1350992743 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280034040 con 0x7f5288112550 2024-08-02T17:54:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.562+0000 7f52857fa640 1 -- 172.21.15.69:0/1350992743 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5280030920 con 0x7f5288112550 2024-08-02T17:54:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.562+0000 7f52857fa640 1 -- 172.21.15.69:0/1350992743 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280004610 con 0x7f5288112550 2024-08-02T17:54:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.562+0000 7f528e5d7640 1 -- 172.21.15.69:0/1350992743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528819efb0 con 0x7f5288112550 2024-08-02T17:54:17.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.562+0000 7f528e5d7640 1 -- 172.21.15.69:0/1350992743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52881ab2b0 con 0x7f5288112550 2024-08-02T17:54:17.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.565+0000 7f52857fa640 1 -- 172.21.15.69:0/1350992743 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5280037020 con 0x7f5288112550 2024-08-02T17:54:17.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.566+0000 7f52857fa640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52700a2d70 0x7f52700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.566+0000 7f52857fa640 1 -- 172.21.15.69:0/1350992743 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f52800f0510 con 0x7f5288112550 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.566+0000 7f5287fff640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52700a2d70 0x7f52700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.566+0000 7f5291406740 1 Processor -- start 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.566+0000 7f5291406740 1 -- start start 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.566+0000 7f5291406740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e32c7b0 0x55b45e32cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.566+0000 7f5291406740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e337430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5291406740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e337970 0x55b45e339d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5291406740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b45e2909b0 con 0x55b45e32c7b0 2024-08-02T17:54:17.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f528cb74640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e337970 0x55b45e339d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f52877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e337430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5287fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e32c7b0 0x55b45e32cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f52877fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e337430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54516/0 (socket says 172.21.15.69:54516) 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f52877fe640 1 -- 172.21.15.69:0/1584686385 learned_addr learned my addr 172.21.15.69:0/1584686385 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5291406740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b45e238800 con 0x55b45e337970 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5291406740 1 -- 172.21.15.69:0/1584686385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b45e28c970 con 0x55b45e32d0d0 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5287fff640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52700a2d70 0x7f52700a5230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f527c009a40 tx=0x7f527c0091c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5287fff640 1 -- 172.21.15.69:0/1584686385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 msgr2=0x55b45e337430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5287fff640 1 --2- 172.21.15.69:0/1584686385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e337430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5287fff640 1 -- 172.21.15.69:0/1584686385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e337970 msgr2=0x55b45e339d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5287fff640 1 --2- 172.21.15.69:0/1584686385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e337970 0x55b45e339d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5287fff640 1 -- 172.21.15.69:0/1584686385 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b45dfa4210 con 0x55b45e32c7b0 2024-08-02T17:54:17.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.567+0000 7f5287fff640 1 --2- 172.21.15.69:0/1584686385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e32c7b0 0x55b45e32cb90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f527c00eba0 tx=0x7f527c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bc1f0d42484d1b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.568+0000 7f526e7fc640 1 -- 172.21.15.69:0/1584686385 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c01b040 con 0x55b45e32c7b0 2024-08-02T17:54:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.568+0000 7f526e7fc640 1 -- 172.21.15.69:0/1584686385 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527c024070 con 0x55b45e32c7b0 2024-08-02T17:54:17.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.568+0000 7f526e7fc640 1 -- 172.21.15.69:0/1584686385 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c01f650 con 0x55b45e32c7b0 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.568+0000 7f5291406740 1 -- 172.21.15.69:0/1584686385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e32c7b0 msgr2=0x55b45e32cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.568+0000 7f5291406740 1 --2- 172.21.15.69:0/1584686385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e32c7b0 0x55b45e32cb90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f527c00eba0 tx=0x7f527c011000 comp rx=0 tx=0).stop 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 -- 172.21.15.69:0/1584686385 shutdown_connections 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 --2- 172.21.15.69:0/1584686385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e337970 0x55b45e339d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 --2- 172.21.15.69:0/1584686385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e337430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 --2- 172.21.15.69:0/1584686385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e32c7b0 0x55b45e32cb90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 -- 172.21.15.69:0/1584686385 >> 172.21.15.69:0/1584686385 conn(0x55b45e328d20 msgr2=0x55b45e329100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 -- 172.21.15.69:0/1584686385 shutdown_connections 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 -- 172.21.15.69:0/1584686385 wait complete. 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 -- start start 2024-08-02T17:54:17.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.569+0000 7f5291406740 1 Processor -- start 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.570+0000 7f5291406740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e334810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.570+0000 7f5291406740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 0x55b45e334d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.570+0000 7f5291406740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e335290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.570+0000 7f5291406740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b45e33a540 con 0x55b45e337970 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.570+0000 7f5291406740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b45e24c500 con 0x55b45e32d0d0 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.570+0000 7f5291406740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b45e33a6c0 con 0x55b45e32c7b0 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.570+0000 7f5291406740 10 client.0 _ll_register_callbacks cb 0x55b45dfa3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.571+0000 7f5287fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e334810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.572+0000 7f5287fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e334810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54524/0 (socket says 172.21.15.69:54524) 2024-08-02T17:54:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.572+0000 7f5287fff640 1 -- 172.21.15.69:0/535134692 learned_addr learned my addr 172.21.15.69:0/535134692 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.572+0000 7f52877fe640 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 0x55b45e334d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.572+0000 7f5287fff640 1 -- 172.21.15.69:0/535134692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 msgr2=0x55b45e334d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.572+0000 7f5287fff640 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 0x55b45e334d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.573+0000 7f5287fff640 1 -- 172.21.15.69:0/535134692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 msgr2=0x55b45e335290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.573+0000 7f5287fff640 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e335290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.573+0000 7f5287fff640 1 -- 172.21.15.69:0/535134692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b45e070620 con 0x55b45e32c7b0 2024-08-02T17:54:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.576+0000 7f5287fff640 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e334810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f527c015910 tx=0x7f527c00e990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547192ac06e3c6ef server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.576+0000 7f52677fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:54:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.577+0000 7f52677fe640 1 -- 172.21.15.69:0/535134692 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c02d070 con 0x55b45e32c7b0 2024-08-02T17:54:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.577+0000 7f5291406740 1 -- 172.21.15.69:0/535134692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527c024070 con 0x55b45e32c7b0 2024-08-02T17:54:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.577+0000 7f52677fe640 1 -- 172.21.15.69:0/535134692 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f527c013860 con 0x55b45e32c7b0 2024-08-02T17:54:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.578+0000 7f52677fe640 1 -- 172.21.15.69:0/535134692 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f527c03b050 con 0x55b45e32c7b0 2024-08-02T17:54:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.579+0000 7f5291406740 10 client.26722 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.579+0000 7f5291406740 1 -- 172.21.15.69:0/535134692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b45e346150 con 0x55b45e32c7b0 2024-08-02T17:54:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.581+0000 7f52677fe640 1 -- 172.21.15.69:0/535134692 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527c024070 con 0x55b45e32c7b0 2024-08-02T17:54:17.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.581+0000 7f52677fe640 1 -- 172.21.15.69:0/535134692 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f527c01e030 con 0x55b45e32c7b0 2024-08-02T17:54:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.585+0000 7f5291406740 10 client.26722 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.585+0000 7f5291406740 10 client.26722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.585+0000 7f5291406740 20 client.26722 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.585+0000 7f5291406740 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b45e3475b0 0x55b45e3499a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.585+0000 7f5291406740 4 client.26722 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.585+0000 7f5291406740 1 -- 172.21.15.69:0/535134692 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b45e29e0e0 con 0x55b45e3475b0 2024-08-02T17:54:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.586+0000 7f52877fe640 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b45e3475b0 0x55b45e3499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.598+0000 7f52877fe640 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b45e3475b0 0x55b45e3499a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cbc12688bcb3805 server_cookie=df62b330143d1326 in_seq=0 out_seq=0 2024-08-02T17:54:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.598+0000 7f52677fe640 10 client.26722 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.599+0000 7f52677fe640 1 -- 172.21.15.69:0/535134692 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b45e29e0e0 con 0x55b45e3475b0 2024-08-02T17:54:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.599+0000 7f52677fe640 10 client.26722 handle_command_reply: tid=0 2024-08-02T17:54:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.600+0000 7f5291406740 1 client.26722 shutdown 2024-08-02T17:54:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.609+0000 7f5291406740 1 -- 172.21.15.69:0/535134692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 msgr2=0x55b45e334810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.609+0000 7f5291406740 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e334810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f527c015910 tx=0x7f527c00e990 comp rx=0 tx=0).stop 2024-08-02T17:54:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.611+0000 7f5291406740 1 -- 172.21.15.69:0/535134692 shutdown_connections 2024-08-02T17:54:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.611+0000 7f5291406740 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e335290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.611+0000 7f5291406740 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b45e3475b0 0x55b45e3499a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.611+0000 7f5291406740 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 0x55b45e334d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.611+0000 7f5291406740 1 --2- 172.21.15.69:0/535134692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e334810 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.611+0000 7f5291406740 1 -- 172.21.15.69:0/535134692 >> 172.21.15.69:0/535134692 conn(0x55b45e328d20 msgr2=0x55b45e327e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.611+0000 7f52677fe640 0 client.26722 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.614+0000 7f5291406740 1 -- 172.21.15.69:0/535134692 shutdown_connections 2024-08-02T17:54:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.614+0000 7f5291406740 1 -- 172.21.15.69:0/535134692 wait complete. 2024-08-02T17:54:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.615+0000 7f5291406740 20 client.26722 trim_cache size 0 max 16384 2024-08-02T17:54:17.650 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.649+0000 7f5291406740 1 Processor -- start 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5291406740 1 -- start start 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5291406740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32c7b0 0x55b45e32f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5291406740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e336670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5291406740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e34efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5291406740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52881b7ea0 con 0x55b45e337970 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5291406740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52881b5770 con 0x55b45e32c7b0 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5291406740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52881b35c0 con 0x55b45e32d0d0 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5287fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32c7b0 0x55b45e32f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5287fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32c7b0 0x55b45e32f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:51206/0 (socket says 172.21.15.69:51206) 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f5287fff640 1 -- 172.21.15.69:0/1692056826 learned_addr learned my addr 172.21.15.69:0/1692056826 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:17.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f528cb74640 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e34efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.650+0000 7f52877fe640 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e336670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f5287fff640 1 -- 172.21.15.69:0/1692056826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 msgr2=0x55b45e336670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f5287fff640 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e336670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f5287fff640 1 -- 172.21.15.69:0/1692056826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 msgr2=0x55b45e34efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f5287fff640 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e34efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f5287fff640 1 -- 172.21.15.69:0/1692056826 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b45e328b50 con 0x55b45e32c7b0 2024-08-02T17:54:17.653 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f5287fff640 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32c7b0 0x55b45e32f220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f527c00e5a0 tx=0x7f527c009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a3421ad004e0202 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f526dffb640 1 -- 172.21.15.69:0/1692056826 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c044070 con 0x55b45e32c7b0 2024-08-02T17:54:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f526dffb640 1 -- 172.21.15.69:0/1692056826 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527c024050 con 0x55b45e32c7b0 2024-08-02T17:54:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.651+0000 7f526dffb640 1 -- 172.21.15.69:0/1692056826 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c02d070 con 0x55b45e32c7b0 2024-08-02T17:54:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.652+0000 7f5291406740 1 -- 172.21.15.69:0/1692056826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32c7b0 msgr2=0x55b45e32f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.652+0000 7f5291406740 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32c7b0 0x55b45e32f220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f527c00e5a0 tx=0x7f527c009230 comp rx=0 tx=0).stop 2024-08-02T17:54:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.655+0000 7f5291406740 1 -- 172.21.15.69:0/1692056826 shutdown_connections 2024-08-02T17:54:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.655+0000 7f5291406740 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e34efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.655+0000 7f5291406740 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32d0d0 0x55b45e336670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.655+0000 7f5291406740 1 --2- 172.21.15.69:0/1692056826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32c7b0 0x55b45e32f220 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.655+0000 7f5291406740 1 -- 172.21.15.69:0/1692056826 >> 172.21.15.69:0/1692056826 conn(0x55b45e328d20 msgr2=0x55b45e335f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.656+0000 7f5291406740 1 -- 172.21.15.69:0/1692056826 shutdown_connections 2024-08-02T17:54:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.656+0000 7f5291406740 1 -- 172.21.15.69:0/1692056826 wait complete. 2024-08-02T17:54:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.656+0000 7f5291406740 1 -- start start 2024-08-02T17:54:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.657+0000 7f5291406740 1 Processor -- start 2024-08-02T17:54:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.657+0000 7f5291406740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e3ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.657+0000 7f5291406740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 0x55b45e3ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.657+0000 7f5291406740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e33f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.657+0000 7f5291406740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52881b3c30 con 0x55b45e337970 2024-08-02T17:54:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.657+0000 7f5291406740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52881b50a0 con 0x55b45e32d0d0 2024-08-02T17:54:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.657+0000 7f5291406740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52881b4300 con 0x55b45e32c7b0 2024-08-02T17:54:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.657+0000 7f5291406740 10 client.0 _ll_register_callbacks cb 0x7f52800099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f5287fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e3ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f5287fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e3ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54530/0 (socket says 172.21.15.69:54530) 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f5287fff640 1 -- 172.21.15.69:0/3588068574 learned_addr learned my addr 172.21.15.69:0/3588068574 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f52877fe640 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 0x55b45e3ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f5287fff640 1 -- 172.21.15.69:0/3588068574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 msgr2=0x55b45e3ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f5287fff640 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 0x55b45e3ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f5287fff640 1 -- 172.21.15.69:0/3588068574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 msgr2=0x55b45e33f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f5287fff640 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e33f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.658+0000 7f5287fff640 1 -- 172.21.15.69:0/3588068574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b45e340180 con 0x55b45e32c7b0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.659+0000 7f5287fff640 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e3ad260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f527c010010 tx=0x7f527c015980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3b01359aafc9e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.659+0000 7f52677fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.659+0000 7f52677fe640 1 -- 172.21.15.69:0/3588068574 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c044030 con 0x55b45e32c7b0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.659+0000 7f5291406740 1 -- 172.21.15.69:0/3588068574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527c024050 con 0x55b45e32c7b0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.659+0000 7f52677fe640 1 -- 172.21.15.69:0/3588068574 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f527c013d80 con 0x55b45e32c7b0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f52677fe640 1 -- 172.21.15.69:0/3588068574 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f527c04e050 con 0x55b45e32c7b0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f5291406740 10 client.26728 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f5291406740 1 -- 172.21.15.69:0/3588068574 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b45e331d40 con 0x55b45e32c7b0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f52677fe640 1 -- 172.21.15.69:0/3588068574 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f527c024050 con 0x55b45e32c7b0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f52677fe640 1 -- 172.21.15.69:0/3588068574 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f527c01e070 con 0x55b45e32c7b0 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f5291406740 10 client.26728 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:17.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f5291406740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f5291406740 20 client.26728 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f5291406740 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b45e3c31e0 0x55b45e3c5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f5291406740 4 client.26728 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:54:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.660+0000 7f5291406740 1 -- 172.21.15.69:0/3588068574 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b45e3298d0 con 0x55b45e3c31e0 2024-08-02T17:54:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.661+0000 7f52877fe640 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b45e3c31e0 0x55b45e3c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.661+0000 7f52877fe640 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b45e3c31e0 0x55b45e3c5600 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996e23e1b6356e41 server_cookie=ff1aed7e85547b86 in_seq=0 out_seq=0 2024-08-02T17:54:17.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.661+0000 7f52677fe640 10 client.26728 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.738+0000 7f52677fe640 1 -- 172.21.15.69:0/3588068574 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b45e3298d0 con 0x55b45e3c31e0 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.738+0000 7f52677fe640 10 client.26728 handle_command_reply: tid=0 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.738+0000 7f5291406740 1 client.26728 shutdown 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 -- 172.21.15.69:0/3588068574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 msgr2=0x55b45e3ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e3ad260 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f527c010010 tx=0x7f527c015980 comp rx=0 tx=0).stop 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 -- 172.21.15.69:0/3588068574 shutdown_connections 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b45e337970 0x55b45e33f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b45e3c31e0 0x55b45e3c5600 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b45e32d0d0 0x55b45e3ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 --2- 172.21.15.69:0/3588068574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b45e32c7b0 0x55b45e3ad260 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f52677fe640 0 client.26728 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:17.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 -- 172.21.15.69:0/3588068574 >> 172.21.15.69:0/3588068574 conn(0x55b45e328d20 msgr2=0x55b45e327e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 -- 172.21.15.69:0/3588068574 shutdown_connections 2024-08-02T17:54:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 1 -- 172.21.15.69:0/3588068574 wait complete. 2024-08-02T17:54:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f5291406740 20 client.26728 trim_cache size 0 max 16384 2024-08-02T17:54:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:54:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:54:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:54:17.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:54:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f528e5d7640 1 -- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52700a2d70 msgr2=0x7f52700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.739+0000 7f528e5d7640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52700a2d70 0x7f52700a5230 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f527c009a40 tx=0x7f527c0091c0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 -- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5288112550 msgr2=0x7f528819e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:17.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5288112550 0x7f528819e5a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5280005cc0 tx=0x7f528002f420 comp rx=0 tx=0).stop 2024-08-02T17:54:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 -- 172.21.15.69:0/1350992743 shutdown_connections 2024-08-02T17:54:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5288112550 0x7f528819e5a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52700a2d70 0x7f52700a5230 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52881001e0 0x7f52881a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 --2- 172.21.15.69:0/1350992743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52880ff8c0 0x7f52881a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 -- 172.21.15.69:0/1350992743 >> 172.21.15.69:0/1350992743 conn(0x7f52880fb760 msgr2=0x7f5288113ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 -- 172.21.15.69:0/1350992743 shutdown_connections 2024-08-02T17:54:17.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:17.740+0000 7f528e5d7640 1 -- 172.21.15.69:0/1350992743 wait complete. 2024-08-02T17:54:17.785 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:54:18.237 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:18 smithi083 ceph-mon[37031]: pgmap v243: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:54:19.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:18 smithi077 ceph-mon[37108]: pgmap v243: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:54:19.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:18 smithi069 ceph-mon[32746]: pgmap v243: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:54:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 430.0 s, 765.6 tps, lat 131.023 ms stddev 192.759 2024-08-02T17:54:21.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:20 smithi083 ceph-mon[37031]: pgmap v244: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-08-02T17:54:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:20 smithi077 ceph-mon[37108]: pgmap v244: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-08-02T17:54:21.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:20 smithi069 ceph-mon[32746]: pgmap v244: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.2 MiB/s wr, 536 op/s 2024-08-02T17:54:22.168 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:54:22.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 -- 172.21.15.69:0/3210200535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 msgr2=0x7f63b8100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 --2- 172.21.15.69:0/3210200535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 0x7f63b8100f30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009900 tx=0x7f63b402a480 comp rx=0 tx=0).stop 2024-08-02T17:54:22.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 -- 172.21.15.69:0/3210200535 shutdown_connections 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 --2- 172.21.15.69:0/3210200535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b8102c20 0x7f63b810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 --2- 172.21.15.69:0/3210200535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b81022a0 0x7f63b81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 --2- 172.21.15.69:0/3210200535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 0x7f63b8100f30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 -- 172.21.15.69:0/3210200535 >> 172.21.15.69:0/3210200535 conn(0x7f63b80fc500 msgr2=0x7f63b80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 -- 172.21.15.69:0/3210200535 shutdown_connections 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.616+0000 7f63c004f640 1 -- 172.21.15.69:0/3210200535 wait complete. 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63c004f640 1 Processor -- start 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63c004f640 1 -- start start 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63c004f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 0x7f63b81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63c004f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b81022a0 0x7f63b8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63c004f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b8102c20 0x7f63b819cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63c004f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8111dd0 con 0x7f63b8100b50 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63c004f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8111c50 con 0x7f63b81022a0 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63c004f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63b8111f50 con 0x7f63b8102c20 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63bddeb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 0x7f63b81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63bddeb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 0x7f63b81981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48682/0 (socket says 172.21.15.69:48682) 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63bddeb640 1 -- 172.21.15.69:0/4236375227 learned_addr learned my addr 172.21.15.69:0/4236375227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b81022a0 0x7f63b8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63be5ec640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b8102c20 0x7f63b819cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63bddeb640 1 -- 172.21.15.69:0/4236375227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b8102c20 msgr2=0x7f63b819cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.617+0000 7f63bddeb640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b8102c20 0x7f63b819cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63bddeb640 1 -- 172.21.15.69:0/4236375227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b81022a0 msgr2=0x7f63b8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63bddeb640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b81022a0 0x7f63b8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63bddeb640 1 -- 172.21.15.69:0/4236375227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b819d190 con 0x7f63b8100b50 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63bddeb640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 0x7f63b81981e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006010 tx=0x7f63b4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63aeffd640 1 -- 172.21.15.69:0/4236375227 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b403f070 con 0x7f63b8100b50 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63aeffd640 1 -- 172.21.15.69:0/4236375227 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b4031c30 con 0x7f63b8100b50 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63c004f640 1 -- 172.21.15.69:0/4236375227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b819d420 con 0x7f63b8100b50 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63aeffd640 1 -- 172.21.15.69:0/4236375227 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4035030 con 0x7f63b8100b50 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.618+0000 7f63c004f640 1 -- 172.21.15.69:0/4236375227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b81a4d60 con 0x7f63b8100b50 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.620+0000 7f63aeffd640 1 -- 172.21.15.69:0/4236375227 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f63b4042070 con 0x7f63b8100b50 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.620+0000 7f63c244d740 1 Processor -- start 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63c244d740 1 -- start start 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63aeffd640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63a40a2cf0 0x7f63a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63c244d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a45d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63c244d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45d7f0 0x562b6a46c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63c244d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a46e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63c244d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b6a3c2a40 con 0x562b6a45d7f0 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63c244d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b6a35a9d0 con 0x562b6a45ced0 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63c244d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b6a3bea00 con 0x562b6a46c5e0 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63be5ec640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63a40a2cf0 0x7f63a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63aeffd640 1 -- 172.21.15.69:0/4236375227 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f63b40f0010 con 0x7f63b8100b50 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bddeb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a45d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bd5ea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45d7f0 0x562b6a46c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bddeb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a45d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:51232/0 (socket says 172.21.15.69:51232) 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bddeb640 1 -- 172.21.15.69:0/1547178592 learned_addr learned my addr 172.21.15.69:0/1547178592 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:22.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63be5ec640 1 --2- 172.21.15.69:0/1547178592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a46e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bddeb640 1 -- 172.21.15.69:0/1547178592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 msgr2=0x562b6a46e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bddeb640 1 --2- 172.21.15.69:0/1547178592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a46e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bddeb640 1 -- 172.21.15.69:0/1547178592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45d7f0 msgr2=0x562b6a46c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bddeb640 1 --2- 172.21.15.69:0/1547178592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45d7f0 0x562b6a46c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.621+0000 7f63bddeb640 1 -- 172.21.15.69:0/1547178592 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6a361650 con 0x562b6a45ced0 2024-08-02T17:54:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.622+0000 7f63bddeb640 1 --2- 172.21.15.69:0/1547178592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a45d2b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63b40b8ba0 tx=0x7f63b40bbdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf586ebd103dccbe server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.622+0000 7f63be5ec640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63a40a2cf0 0x7f63a40a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005fd0 tx=0x7f63b0004410 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:22.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.622+0000 7f63a3fff640 1 -- 172.21.15.69:0/1547178592 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4049050 con 0x562b6a45ced0 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.622+0000 7f63a3fff640 1 -- 172.21.15.69:0/1547178592 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b40bc1d0 con 0x562b6a45ced0 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.622+0000 7f63a3fff640 1 -- 172.21.15.69:0/1547178592 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40dc210 con 0x562b6a45ced0 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 -- 172.21.15.69:0/1547178592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 msgr2=0x562b6a45d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 --2- 172.21.15.69:0/1547178592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a45d2b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f63b40b8ba0 tx=0x7f63b40bbdd0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 -- 172.21.15.69:0/1547178592 shutdown_connections 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 --2- 172.21.15.69:0/1547178592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45d7f0 0x562b6a46c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 --2- 172.21.15.69:0/1547178592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a46e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 --2- 172.21.15.69:0/1547178592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a45d2b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 -- 172.21.15.69:0/1547178592 >> 172.21.15.69:0/1547178592 conn(0x562b6a459580 msgr2=0x562b6a459960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 -- 172.21.15.69:0/1547178592 shutdown_connections 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 -- 172.21.15.69:0/1547178592 wait complete. 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 -- start start 2024-08-02T17:54:22.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.623+0000 7f63c244d740 1 Processor -- start 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63c244d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a469500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63c244d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45d7f0 0x562b6a469a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63c244d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a469f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63c244d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b6a46ef70 con 0x562b6a45ced0 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63c244d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b6a37db20 con 0x562b6a45d7f0 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63c244d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b6a46f0f0 con 0x562b6a46c5e0 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63c244d740 10 client.0 _ll_register_callbacks cb 0x562b6a37bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a469500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a469500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48706/0 (socket says 172.21.15.69:48706) 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 -- 172.21.15.69:0/2969218466 learned_addr learned my addr 172.21.15.69:0/2969218466 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45d7f0 0x562b6a469a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63be5ec640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a469f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 -- 172.21.15.69:0/2969218466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 msgr2=0x562b6a469f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a469f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 -- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45d7f0 msgr2=0x562b6a469a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45d7f0 0x562b6a469a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 -- 172.21.15.69:0/2969218466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b6a1343c0 con 0x562b6a45ced0 2024-08-02T17:54:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45d7f0 0x562b6a469a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63be5ec640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a469f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:22.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a469500 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f63b40b8630 tx=0x7f63b4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d5e5941851e28a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.624+0000 7f63a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.625+0000 7f63a17fa640 1 -- 172.21.15.69:0/2969218466 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4049070 con 0x562b6a45ced0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.625+0000 7f63c244d740 1 -- 172.21.15.69:0/2969218466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b40bc1d0 con 0x562b6a45ced0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.625+0000 7f63a17fa640 1 -- 172.21.15.69:0/2969218466 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f63b40b8840 con 0x562b6a45ced0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.625+0000 7f63a17fa640 1 -- 172.21.15.69:0/2969218466 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63b40d5120 con 0x562b6a45ced0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.625+0000 7f63c244d740 10 client.17313 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.625+0000 7f63c244d740 1 -- 172.21.15.69:0/2969218466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b6a46ef70 con 0x562b6a45ced0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.625+0000 7f63a17fa640 1 -- 172.21.15.69:0/2969218466 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b40bc1d0 con 0x562b6a45ced0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.626+0000 7f63a17fa640 1 -- 172.21.15.69:0/2969218466 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f63b40c0080 con 0x562b6a45ced0 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.631+0000 7f63c244d740 10 client.17313 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.631+0000 7f63c244d740 10 client.17313 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:54:22.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.631+0000 7f63c244d740 20 client.17313 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.631+0000 7f63c244d740 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562b6a47bd90 0x562b6a47e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.631+0000 7f63c244d740 4 client.17313 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.631+0000 7f63c244d740 1 -- 172.21.15.69:0/2969218466 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b6a3cffb0 con 0x562b6a47bd90 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.632+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562b6a47bd90 0x562b6a47e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.633+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562b6a47bd90 0x562b6a47e1f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=75871adc9bce5a09 server_cookie=90cfcd16b2062c16 in_seq=0 out_seq=0 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.633+0000 7f63a17fa640 10 client.17313 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63a17fa640 1 -- 172.21.15.69:0/2969218466 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b6a3cffb0 con 0x562b6a47bd90 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63a17fa640 10 client.17313 handle_command_reply: tid=0 2024-08-02T17:54:22.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 client.17313 shutdown 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 -- 172.21.15.69:0/2969218466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 msgr2=0x562b6a469500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a469500 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f63b40b8630 tx=0x7f63b4005b50 comp rx=0 tx=0).stop 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 -- 172.21.15.69:0/2969218466 shutdown_connections 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562b6a47bd90 0x562b6a47e1f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63a17fa640 0 client.17313 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a46c5e0 0x562b6a469f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45d7f0 0x562b6a469a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 --2- 172.21.15.69:0/2969218466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a469500 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 -- 172.21.15.69:0/2969218466 >> 172.21.15.69:0/2969218466 conn(0x562b6a459580 msgr2=0x562b6a45f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 -- 172.21.15.69:0/2969218466 shutdown_connections 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.634+0000 7f63c244d740 1 -- 172.21.15.69:0/2969218466 wait complete. 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.635+0000 7f63c244d740 20 client.17313 trim_cache size 0 max 16384 2024-08-02T17:54:22.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63c244d740 1 Processor -- start 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63c244d740 1 -- start start 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63c244d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a483ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63c244d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a484000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63c244d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a46c5e0 0x562b6a46ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63c244d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81af550 con 0x562b6a45ced0 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63c244d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81ace20 con 0x562b6a46c5e0 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63c244d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81aac70 con 0x562b6a45d7f0 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bddeb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a483ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bddeb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a483ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48710/0 (socket says 172.21.15.69:48710) 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bddeb640 1 -- 172.21.15.69:0/2867518773 learned_addr learned my addr 172.21.15.69:0/2867518773 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:22.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63be5ec640 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a46c5e0 0x562b6a46ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a484000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bddeb640 1 -- 172.21.15.69:0/2867518773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 msgr2=0x562b6a484000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a484000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bddeb640 1 -- 172.21.15.69:0/2867518773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a46c5e0 msgr2=0x562b6a46ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a46c5e0 0x562b6a46ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bddeb640 1 -- 172.21.15.69:0/2867518773 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6a459020 con 0x562b6a45ced0 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.648+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a484000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.649+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a483ac0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f63b4000c00 tx=0x7f63b4005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5910ce16d852b99b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.649+0000 7f63a37fe640 1 -- 172.21.15.69:0/2867518773 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b4049070 con 0x562b6a45ced0 2024-08-02T17:54:22.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.649+0000 7f63a37fe640 1 -- 172.21.15.69:0/2867518773 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b40ed5d0 con 0x562b6a45ced0 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.649+0000 7f63a37fe640 1 -- 172.21.15.69:0/2867518773 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40b8840 con 0x562b6a45ced0 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.649+0000 7f63c244d740 1 -- 172.21.15.69:0/2867518773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 msgr2=0x562b6a483ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a483ac0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f63b4000c00 tx=0x7f63b4005890 comp rx=0 tx=0).stop 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 -- 172.21.15.69:0/2867518773 shutdown_connections 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a46c5e0 0x562b6a46ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a484000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 --2- 172.21.15.69:0/2867518773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a45ced0 0x562b6a483ac0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 -- 172.21.15.69:0/2867518773 >> 172.21.15.69:0/2867518773 conn(0x562b6a459580 msgr2=0x562b6a4541b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 -- 172.21.15.69:0/2867518773 shutdown_connections 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 -- 172.21.15.69:0/2867518773 wait complete. 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 -- start start 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.650+0000 7f63c244d740 1 Processor -- start 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63c244d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a474c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63c244d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a46ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63c244d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a46c5e0 0x562b6a4706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63c244d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81ab2e0 con 0x562b6a46c5e0 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63c244d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81ac750 con 0x562b6a45ced0 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63c244d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63b81ab9b0 con 0x562b6a45d7f0 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63c244d740 10 client.0 _ll_register_callbacks cb 0x7f63a800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a46c5e0 0x562b6a4706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a46c5e0 0x562b6a4706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:48720/0 (socket says 172.21.15.69:48720) 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 -- 172.21.15.69:0/2279151897 learned_addr learned my addr 172.21.15.69:0/2279151897 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a474c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a46ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 -- 172.21.15.69:0/2279151897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 msgr2=0x562b6a46ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:22.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a46ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 -- 172.21.15.69:0/2279151897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 msgr2=0x562b6a474c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a474c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 -- 172.21.15.69:0/2279151897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b6a464270 con 0x562b6a46c5e0 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63bd5ea640 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a46ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63be5ec640 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a46c5e0 0x562b6a4706a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f63b00129d0 tx=0x7f63b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efa790fe05bf91e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63a17fa640 1 -- 172.21.15.69:0/2279151897 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b0021da0 con 0x562b6a46c5e0 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63c244d740 1 -- 172.21.15.69:0/2279151897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b6a3ca4c0 con 0x562b6a46c5e0 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.651+0000 7f63a17fa640 1 -- 172.21.15.69:0/2279151897 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f63b00293e0 con 0x562b6a46c5e0 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63a17fa640 1 -- 172.21.15.69:0/2279151897 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63b00295e0 con 0x562b6a46c5e0 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63c244d740 10 client.17319 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63c244d740 1 -- 172.21.15.69:0/2279151897 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b6a4e6ab0 con 0x562b6a46c5e0 2024-08-02T17:54:22.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63a17fa640 1 -- 172.21.15.69:0/2279151897 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562b6a3ca4c0 con 0x562b6a46c5e0 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63a17fa640 1 -- 172.21.15.69:0/2279151897 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f63b0028070 con 0x562b6a46c5e0 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63c244d740 10 client.17319 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63c244d740 10 client.17319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63c244d740 20 client.17319 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63c244d740 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562b6a47bd90 0x562b6a4e9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63c244d740 4 client.17319 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63c244d740 1 -- 172.21.15.69:0/2279151897 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b6a347170 con 0x562b6a47bd90 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.652+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562b6a47bd90 0x562b6a4e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63bddeb640 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562b6a47bd90 0x562b6a4e9270 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8b8bfa28f77d712 server_cookie=68f3f97f2f33cf9c in_seq=0 out_seq=0 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63a17fa640 10 client.17319 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63a17fa640 1 -- 172.21.15.69:0/2279151897 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b6a347170 con 0x562b6a47bd90 2024-08-02T17:54:22.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63a17fa640 10 client.17319 handle_command_reply: tid=0 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 client.17319 shutdown 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 -- 172.21.15.69:0/2279151897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a46c5e0 msgr2=0x562b6a4706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a46c5e0 0x562b6a4706a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f63b00129d0 tx=0x7f63b0004ab0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 -- 172.21.15.69:0/2279151897 shutdown_connections 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562b6a47bd90 0x562b6a4e9270 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63a17fa640 0 client.17319 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b6a46c5e0 0x562b6a4706a0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b6a45d7f0 0x562b6a46ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 --2- 172.21.15.69:0/2279151897 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b6a45ced0 0x562b6a474c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 -- 172.21.15.69:0/2279151897 >> 172.21.15.69:0/2279151897 conn(0x562b6a459580 msgr2=0x562b6a46ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 -- 172.21.15.69:0/2279151897 shutdown_connections 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 1 -- 172.21.15.69:0/2279151897 wait complete. 2024-08-02T17:54:22.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.653+0000 7f63c244d740 20 client.17319 trim_cache size 0 max 16384 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 -- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63a40a2cf0 msgr2=0x7f63a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63a40a2cf0 0x7f63a40a51b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63b0005fd0 tx=0x7f63b0004410 comp rx=0 tx=0).stop 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 -- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 msgr2=0x7f63b81981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 0x7f63b81981e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006010 tx=0x7f63b4003c10 comp rx=0 tx=0).stop 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 -- 172.21.15.69:0/4236375227 shutdown_connections 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63a40a2cf0 0x7f63a40a51b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63b8102c20 0x7f63b819cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63b81022a0 0x7f63b8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 --2- 172.21.15.69:0/4236375227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63b8100b50 0x7f63b81981e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 -- 172.21.15.69:0/4236375227 >> 172.21.15.69:0/4236375227 conn(0x7f63b80fc500 msgr2=0x7f63b80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 -- 172.21.15.69:0/4236375227 shutdown_connections 2024-08-02T17:54:22.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:20.654+0000 7f63c004f640 1 -- 172.21.15.69:0/4236375227 wait complete. 2024-08-02T17:54:23.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:22 smithi083 ceph-mon[37031]: pgmap v245: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 579 op/s 2024-08-02T17:54:23.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:22 smithi077 ceph-mon[37108]: pgmap v245: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 579 op/s 2024-08-02T17:54:23.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:22 smithi069 ceph-mon[32746]: pgmap v245: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.4 MiB/s wr, 579 op/s 2024-08-02T17:54:24.569 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:54:24.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 435.0 s, 709.4 tps, lat 141.279 ms stddev 177.424 2024-08-02T17:54:25.103 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:25.243 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:24 smithi069 ceph-mon[32746]: pgmap v246: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 688 op/s 2024-08-02T17:54:25.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:24 smithi083 ceph-mon[37031]: pgmap v246: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 688 op/s 2024-08-02T17:54:25.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:24 smithi077 ceph-mon[37108]: pgmap v246: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 7.0 MiB/s wr, 688 op/s 2024-08-02T17:54:26.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.392+0000 7fd16406b640 1 -- 172.21.15.69:0/3457558309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c10d250 msgr2=0x7fd15c114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:26.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.392+0000 7fd16406b640 1 --2- 172.21.15.69:0/3457558309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c10d250 0x7fd15c114dd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd15801a910 tx=0x7fd15803b430 comp rx=0 tx=0).stop 2024-08-02T17:54:26.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.395+0000 7fd16406b640 1 -- 172.21.15.69:0/3457558309 shutdown_connections 2024-08-02T17:54:26.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.396+0000 7fd16406b640 1 --2- 172.21.15.69:0/3457558309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c10d250 0x7fd15c114dd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.396+0000 7fd16406b640 1 --2- 172.21.15.69:0/3457558309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd15c108ad0 0x7fd15c10cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.396+0000 7fd16406b640 1 --2- 172.21.15.69:0/3457558309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c108120 0x7fd15c108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.396+0000 7fd16406b640 1 -- 172.21.15.69:0/3457558309 >> 172.21.15.69:0/3457558309 conn(0x7fd15c077820 msgr2=0x7fd15c079c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:26.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.398+0000 7fd16406b640 1 -- 172.21.15.69:0/3457558309 shutdown_connections 2024-08-02T17:54:26.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.399+0000 7fd16406b640 1 -- 172.21.15.69:0/3457558309 wait complete. 2024-08-02T17:54:26.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.400+0000 7fd16406b640 1 Processor -- start 2024-08-02T17:54:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.407+0000 7fd16406b640 1 -- start start 2024-08-02T17:54:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.407+0000 7fd16406b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c108120 0x7fd15c114940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.408+0000 7fd16406b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c108ad0 0x7fd15c1109e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.408+0000 7fd16406b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd15c10d250 0x7fd15c0827c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.408+0000 7fd16406b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c06e2d0 con 0x7fd15c108120 2024-08-02T17:54:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.408+0000 7fd16406b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c06e150 con 0x7fd15c10d250 2024-08-02T17:54:26.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.408+0000 7fd16406b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c06e450 con 0x7fd15c108ad0 2024-08-02T17:54:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.409+0000 7fd162608640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd15c10d250 0x7fd15c0827c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.410+0000 7fd161606640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c108ad0 0x7fd15c1109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.410+0000 7fd161606640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c108ad0 0x7fd15c1109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53460/0 (socket says 172.21.15.69:53460) 2024-08-02T17:54:26.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.410+0000 7fd161606640 1 -- 172.21.15.69:0/1698277311 learned_addr learned my addr 172.21.15.69:0/1698277311 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:26.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.411+0000 7fd161606640 1 -- 172.21.15.69:0/1698277311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd15c10d250 msgr2=0x7fd15c0827c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:26.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.411+0000 7fd161606640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd15c10d250 0x7fd15c0827c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.411+0000 7fd161606640 1 -- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c108120 msgr2=0x7fd15c114940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:54:26.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.411+0000 7fd161606640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c108120 0x7fd15c114940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.411+0000 7fd161606640 1 -- 172.21.15.69:0/1698277311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c082d80 con 0x7fd15c108ad0 2024-08-02T17:54:26.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.411+0000 7fd161606640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c108ad0 0x7fd15c1109e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00e9f0 tx=0x7fd14c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:26.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.416+0000 7fd152ffd640 1 -- 172.21.15.69:0/1698277311 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00cd80 con 0x7fd15c108ad0 2024-08-02T17:54:26.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.416+0000 7fd152ffd640 1 -- 172.21.15.69:0/1698277311 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd14c005500 con 0x7fd15c108ad0 2024-08-02T17:54:26.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.416+0000 7fd152ffd640 1 -- 172.21.15.69:0/1698277311 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c010690 con 0x7fd15c108ad0 2024-08-02T17:54:26.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.417+0000 7fd16406b640 1 -- 172.21.15.69:0/1698277311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c111040 con 0x7fd15c108ad0 2024-08-02T17:54:26.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.417+0000 7fd16406b640 1 -- 172.21.15.69:0/1698277311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c111550 con 0x7fd15c108ad0 2024-08-02T17:54:26.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.420+0000 7fd152ffd640 1 -- 172.21.15.69:0/1698277311 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd14c005020 con 0x7fd15c108ad0 2024-08-02T17:54:26.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.425+0000 7fd152ffd640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1480a2f30 0x7fd1480a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.425+0000 7fd152ffd640 1 -- 172.21.15.69:0/1698277311 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fd14c0cfb80 con 0x7fd15c108ad0 2024-08-02T17:54:26.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.426+0000 7fd166469740 1 Processor -- start 2024-08-02T17:54:26.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.426+0000 7fd166469740 1 -- start start 2024-08-02T17:54:26.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd166469740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee7a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd166469740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee84c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd161e07640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee7a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd161e07640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee7a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59548/0 (socket says 172.21.15.69:59548) 2024-08-02T17:54:26.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd166469740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd161e07640 1 -- 172.21.15.69:0/2615560607 learned_addr learned my addr 172.21.15.69:0/2615560607 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:26.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd166469740 1 -- 172.21.15.69:0/2615560607 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56491eddfa40 con 0x56491ee7a910 2024-08-02T17:54:26.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd166469740 1 -- 172.21.15.69:0/2615560607 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56491ed779d0 con 0x56491ee79ff0 2024-08-02T17:54:26.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.427+0000 7fd166469740 1 -- 172.21.15.69:0/2615560607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56491eddba00 con 0x56491ee851d0 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.428+0000 7fd162608640 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.428+0000 7fd161606640 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee84c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.428+0000 7fd161e07640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1480a2f30 0x7fd1480a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.428+0000 7fd161e07640 1 -- 172.21.15.69:0/2615560607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 msgr2=0x56491ee875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.428+0000 7fd161e07640 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.428+0000 7fd161e07640 1 -- 172.21.15.69:0/2615560607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 msgr2=0x56491ee84c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.428+0000 7fd161e07640 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee84c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.428+0000 7fd161e07640 1 -- 172.21.15.69:0/2615560607 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56491ed7e650 con 0x56491ee79ff0 2024-08-02T17:54:26.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.429+0000 7fd161e07640 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee7a3d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd154014960 tx=0x7fd154014e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=816ade7398bbb82e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:26.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.429+0000 7fd161e07640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1480a2f30 0x7fd1480a53f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd154009270 tx=0x7fd15401a000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:26.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.431+0000 7fd147fff640 1 -- 172.21.15.69:0/2615560607 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154014310 con 0x56491ee79ff0 2024-08-02T17:54:26.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.431+0000 7fd147fff640 1 -- 172.21.15.69:0/2615560607 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15400dd40 con 0x56491ee79ff0 2024-08-02T17:54:26.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.432+0000 7fd166469740 1 -- 172.21.15.69:0/2615560607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 msgr2=0x56491ee7a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:26.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.432+0000 7fd166469740 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee7a3d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd154014960 tx=0x7fd154014e20 comp rx=0 tx=0).stop 2024-08-02T17:54:26.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.434+0000 7fd166469740 1 -- 172.21.15.69:0/2615560607 shutdown_connections 2024-08-02T17:54:26.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.434+0000 7fd166469740 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee84c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.434+0000 7fd166469740 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.434+0000 7fd166469740 1 --2- 172.21.15.69:0/2615560607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee7a3d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.434+0000 7fd166469740 1 -- 172.21.15.69:0/2615560607 >> 172.21.15.69:0/2615560607 conn(0x56491ee76580 msgr2=0x56491ee76960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:26.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.436+0000 7fd166469740 1 -- 172.21.15.69:0/2615560607 shutdown_connections 2024-08-02T17:54:26.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.436+0000 7fd166469740 1 -- 172.21.15.69:0/2615560607 wait complete. 2024-08-02T17:54:26.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.436+0000 7fd166469740 1 -- start start 2024-08-02T17:54:26.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.437+0000 7fd166469740 1 Processor -- start 2024-08-02T17:54:26.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.437+0000 7fd166469740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee79ff0 0x56491ee84be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.437+0000 7fd166469740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee81f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.437+0000 7fd166469740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee851d0 0x56491ee82540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.437+0000 7fd166469740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56491ee87e40 con 0x56491ee7a910 2024-08-02T17:54:26.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.437+0000 7fd166469740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56491ed9ab20 con 0x56491ee851d0 2024-08-02T17:54:26.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.437+0000 7fd166469740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56491ee87fc0 con 0x56491ee79ff0 2024-08-02T17:54:26.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.437+0000 7fd166469740 10 client.0 _ll_register_callbacks cb 0x56491ed98bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:26.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.438+0000 7fd162608640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee851d0 0x56491ee82540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.438+0000 7fd162608640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee851d0 0x56491ee82540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59560/0 (socket says 172.21.15.69:59560) 2024-08-02T17:54:26.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.438+0000 7fd162608640 1 -- 172.21.15.69:0/2470573418 learned_addr learned my addr 172.21.15.69:0/2470573418 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:26.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.438+0000 7fd161e07640 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee79ff0 0x56491ee84be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.439+0000 7fd161606640 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee81f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.440+0000 7fd162608640 1 -- 172.21.15.69:0/2470573418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee79ff0 msgr2=0x56491ee84be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:26.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.440+0000 7fd162608640 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee79ff0 0x56491ee84be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.440+0000 7fd162608640 1 -- 172.21.15.69:0/2470573418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 msgr2=0x56491ee81f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:26.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.440+0000 7fd162608640 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee81f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.440+0000 7fd162608640 1 -- 172.21.15.69:0/2470573418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56491eb513c0 con 0x56491ee851d0 2024-08-02T17:54:26.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.440+0000 7fd161606640 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee81f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:26.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.440+0000 7fd162608640 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee851d0 0x56491ee82540 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd15801a910 tx=0x7fd158005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23aad2f9b9575081 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:26.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.448+0000 7fd1457fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:54:26.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.448+0000 7fd1457fa640 1 -- 172.21.15.69:0/2470573418 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158046040 con 0x56491ee851d0 2024-08-02T17:54:26.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.448+0000 7fd1457fa640 1 -- 172.21.15.69:0/2470573418 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fd158004260 con 0x56491ee851d0 2024-08-02T17:54:26.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.449+0000 7fd166469740 1 -- 172.21.15.69:0/2470573418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15400dd40 con 0x56491ee851d0 2024-08-02T17:54:26.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.457+0000 7fd1457fa640 1 -- 172.21.15.69:0/2470573418 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd158004460 con 0x56491ee851d0 2024-08-02T17:54:26.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.459+0000 7fd166469740 10 client.26888 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:26.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.459+0000 7fd166469740 1 -- 172.21.15.69:0/2470573418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56491ee93970 con 0x56491ee851d0 2024-08-02T17:54:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.465+0000 7fd1457fa640 1 -- 172.21.15.69:0/2470573418 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd158005430 con 0x56491ee851d0 2024-08-02T17:54:26.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.465+0000 7fd1457fa640 1 -- 172.21.15.69:0/2470573418 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd158040510 con 0x56491ee851d0 2024-08-02T17:54:26.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.466+0000 7fd166469740 10 client.26888 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:26.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.466+0000 7fd166469740 10 client.26888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:26.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.466+0000 7fd166469740 20 client.26888 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.467+0000 7fd166469740 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56491ee94d80 0x56491ee97170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.467+0000 7fd166469740 4 client.26888 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.467+0000 7fd166469740 1 -- 172.21.15.69:0/2470573418 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd15400dd40 con 0x56491ee94d80 2024-08-02T17:54:26.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.467+0000 7fd161e07640 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56491ee94d80 0x56491ee97170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.470+0000 7fd161e07640 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56491ee94d80 0x56491ee97170 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740f71d573b8e782 server_cookie=3bbc39011594ff82 in_seq=0 out_seq=0 2024-08-02T17:54:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.473+0000 7fd1457fa640 10 client.26888 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.473+0000 7fd1457fa640 1 -- 172.21.15.69:0/2470573418 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fd15400dd40 con 0x56491ee94d80 2024-08-02T17:54:26.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.473+0000 7fd1457fa640 10 client.26888 handle_command_reply: tid=0 2024-08-02T17:54:26.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.474+0000 7fd166469740 1 client.26888 shutdown 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.476+0000 7fd166469740 1 -- 172.21.15.69:0/2470573418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee851d0 msgr2=0x56491ee82540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.476+0000 7fd166469740 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee851d0 0x56491ee82540 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd15801a910 tx=0x7fd158005bd0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.476+0000 7fd166469740 1 -- 172.21.15.69:0/2470573418 shutdown_connections 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.476+0000 7fd166469740 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee81f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.476+0000 7fd166469740 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56491ee94d80 0x56491ee97170 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.476+0000 7fd166469740 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee851d0 0x56491ee82540 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.476+0000 7fd166469740 1 --2- 172.21.15.69:0/2470573418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee79ff0 0x56491ee84be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.476+0000 7fd166469740 1 -- 172.21.15.69:0/2470573418 >> 172.21.15.69:0/2470573418 conn(0x56491ee76580 msgr2=0x56491ee7c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:26.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.478+0000 7fd1457fa640 0 client.26888 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:26.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.479+0000 7fd166469740 1 -- 172.21.15.69:0/2470573418 shutdown_connections 2024-08-02T17:54:26.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.480+0000 7fd166469740 1 -- 172.21.15.69:0/2470573418 wait complete. 2024-08-02T17:54:26.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.480+0000 7fd166469740 20 client.26888 trim_cache size 0 max 16384 2024-08-02T17:54:26.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.505+0000 7fd166469740 1 Processor -- start 2024-08-02T17:54:26.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.505+0000 7fd166469740 1 -- start start 2024-08-02T17:54:26.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.506+0000 7fd166469740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee910f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.506+0000 7fd166469740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.506+0000 7fd166469740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee81d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.506+0000 7fd166469740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b7e90 con 0x56491ee7a910 2024-08-02T17:54:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.506+0000 7fd162608640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee81d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.507+0000 7fd162608640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee81d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53480/0 (socket says 172.21.15.69:53480) 2024-08-02T17:54:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.507+0000 7fd162608640 1 -- 172.21.15.69:0/3059497090 learned_addr learned my addr 172.21.15.69:0/3059497090 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.507+0000 7fd161e07640 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee910f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.507+0000 7fd166469740 1 -- 172.21.15.69:0/3059497090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b5760 con 0x56491ee79ff0 2024-08-02T17:54:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.507+0000 7fd166469740 1 -- 172.21.15.69:0/3059497090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b35b0 con 0x56491ee851d0 2024-08-02T17:54:26.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.507+0000 7fd161606640 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.508+0000 7fd161e07640 1 -- 172.21.15.69:0/3059497090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 msgr2=0x56491ee81d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.508+0000 7fd161e07640 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee81d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.508+0000 7fd161e07640 1 -- 172.21.15.69:0/3059497090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 msgr2=0x56491ee817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.508+0000 7fd161e07640 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee817f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.508+0000 7fd161e07640 1 -- 172.21.15.69:0/3059497090 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56491ee97750 con 0x56491ee79ff0 2024-08-02T17:54:26.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.509+0000 7fd161e07640 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee910f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd15401e040 tx=0x7fd154013a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6755997bac4a4270 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.509+0000 7fd1477fe640 1 -- 172.21.15.69:0/3059497090 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd154013660 con 0x56491ee79ff0 2024-08-02T17:54:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.509+0000 7fd1477fe640 1 -- 172.21.15.69:0/3059497090 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd154013800 con 0x56491ee79ff0 2024-08-02T17:54:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.509+0000 7fd1477fe640 1 -- 172.21.15.69:0/3059497090 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15400dac0 con 0x56491ee79ff0 2024-08-02T17:54:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.513+0000 7fd166469740 1 -- 172.21.15.69:0/3059497090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 msgr2=0x56491ee910f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:26.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.513+0000 7fd166469740 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee910f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd15401e040 tx=0x7fd154013a60 comp rx=0 tx=0).stop 2024-08-02T17:54:26.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.515+0000 7fd166469740 1 -- 172.21.15.69:0/3059497090 shutdown_connections 2024-08-02T17:54:26.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.515+0000 7fd166469740 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.515+0000 7fd166469740 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee81d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.515+0000 7fd166469740 1 --2- 172.21.15.69:0/3059497090 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee910f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.515+0000 7fd166469740 1 -- 172.21.15.69:0/3059497090 >> 172.21.15.69:0/3059497090 conn(0x56491ee76580 msgr2=0x56491ee76960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:26.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.519+0000 7fd166469740 1 -- 172.21.15.69:0/3059497090 shutdown_connections 2024-08-02T17:54:26.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.522+0000 7fd166469740 1 -- 172.21.15.69:0/3059497090 wait complete. 2024-08-02T17:54:26.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.522+0000 7fd166469740 1 -- start start 2024-08-02T17:54:26.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.522+0000 7fd166469740 1 Processor -- start 2024-08-02T17:54:26.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.528+0000 7fd166469740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee8d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.528+0000 7fd166469740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee85f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.530 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.528+0000 7fd166469740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee86460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.528+0000 7fd166469740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b3c20 con 0x56491ee7a910 2024-08-02T17:54:26.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.528+0000 7fd166469740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b5090 con 0x56491ee79ff0 2024-08-02T17:54:26.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.528+0000 7fd166469740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c1b42f0 con 0x56491ee851d0 2024-08-02T17:54:26.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.528+0000 7fd166469740 10 client.0 _ll_register_callbacks cb 0x7fd14c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:26.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.530+0000 7fd161e07640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee8d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.530+0000 7fd161e07640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee8d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59576/0 (socket says 172.21.15.69:59576) 2024-08-02T17:54:26.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.530+0000 7fd161e07640 1 -- 172.21.15.69:0/1859166919 learned_addr learned my addr 172.21.15.69:0/1859166919 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:26.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.530+0000 7fd161606640 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee85f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.531+0000 7fd161606640 1 -- 172.21.15.69:0/1859166919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 msgr2=0x56491ee86460 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:54:26.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.531+0000 7fd161606640 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee86460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.531+0000 7fd161606640 1 -- 172.21.15.69:0/1859166919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 msgr2=0x56491ee8d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:26.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.531+0000 7fd161606640 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee8d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.531+0000 7fd161606640 1 -- 172.21.15.69:0/1859166919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56491ee763b0 con 0x56491ee7a910 2024-08-02T17:54:26.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.531+0000 7fd161606640 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee85f20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0cce80 tx=0x7fd14c0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e57c2d68e136c76 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:26.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.531+0000 7fd1457fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:26.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.536+0000 7fd1457fa640 1 -- 172.21.15.69:0/1859166919 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c0b9070 con 0x56491ee7a910 2024-08-02T17:54:26.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.536+0000 7fd166469740 1 -- 172.21.15.69:0/1859166919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56491edecfb0 con 0x56491ee7a910 2024-08-02T17:54:26.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.536+0000 7fd1457fa640 1 -- 172.21.15.69:0/1859166919 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fd14c004830 con 0x56491ee7a910 2024-08-02T17:54:26.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.543+0000 7fd1457fa640 1 -- 172.21.15.69:0/1859166919 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd14c0b1040 con 0x56491ee7a910 2024-08-02T17:54:26.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.545+0000 7fd166469740 10 client.17334 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:26.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.545+0000 7fd166469740 1 -- 172.21.15.69:0/1859166919 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56491ee8c0a0 con 0x56491ee7a910 2024-08-02T17:54:26.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.547+0000 7fd1457fa640 1 -- 172.21.15.69:0/1859166919 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56491edecfb0 con 0x56491ee7a910 2024-08-02T17:54:26.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.547+0000 7fd1457fa640 1 -- 172.21.15.69:0/1859166919 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd14c0cd100 con 0x56491ee7a910 2024-08-02T17:54:26.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.547+0000 7fd166469740 10 client.17334 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:26.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.548+0000 7fd166469740 10 client.17334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:26.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.548+0000 7fd166469740 20 client.17334 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:26.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.548+0000 7fd166469740 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56491eeffe10 0x56491ef02230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:26.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.548+0000 7fd166469740 4 client.17334 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:54:26.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.548+0000 7fd166469740 1 -- 172.21.15.69:0/1859166919 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56491ede74c0 con 0x56491eeffe10 2024-08-02T17:54:26.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.548+0000 7fd161e07640 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56491eeffe10 0x56491ef02230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:26.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.549+0000 7fd161e07640 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56491eeffe10 0x56491ef02230 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1d63ed1ea1716d8 server_cookie=95a36ebb62ea59bc in_seq=0 out_seq=0 2024-08-02T17:54:26.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.549+0000 7fd1457fa640 10 client.17334 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:26.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.549+0000 7fd1457fa640 1 -- 172.21.15.69:0/1859166919 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56491ede74c0 con 0x56491eeffe10 2024-08-02T17:54:26.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.549+0000 7fd1457fa640 10 client.17334 handle_command_reply: tid=0 2024-08-02T17:54:26.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.549+0000 7fd166469740 1 client.17334 shutdown 2024-08-02T17:54:26.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 -- 172.21.15.69:0/1859166919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 msgr2=0x56491ee85f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:26.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee85f20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0cce80 tx=0x7fd14c0cf040 comp rx=0 tx=0).stop 2024-08-02T17:54:26.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 -- 172.21.15.69:0/1859166919 shutdown_connections 2024-08-02T17:54:26.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56491ee7a910 0x56491ee85f20 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56491eeffe10 0x56491ef02230 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56491ee851d0 0x56491ee86460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 --2- 172.21.15.69:0/1859166919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56491ee79ff0 0x56491ee8d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 -- 172.21.15.69:0/1859166919 >> 172.21.15.69:0/1859166919 conn(0x56491ee76580 msgr2=0x56491ee81780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:26.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd1457fa640 0 client.17334 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:26.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 -- 172.21.15.69:0/1859166919 shutdown_connections 2024-08-02T17:54:26.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 1 -- 172.21.15.69:0/1859166919 wait complete. 2024-08-02T17:54:26.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.557+0000 7fd166469740 20 client.17334 trim_cache size 0 max 16384 2024-08-02T17:54:26.560 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:54:26.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.560+0000 7fd16406b640 1 -- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1480a2f30 msgr2=0x7fd1480a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:26.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.560+0000 7fd16406b640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1480a2f30 0x7fd1480a53f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd154009270 tx=0x7fd15401a000 comp rx=0 tx=0).stop 2024-08-02T17:54:26.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.560+0000 7fd16406b640 1 -- 172.21.15.69:0/1698277311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c108ad0 msgr2=0x7fd15c1109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.560+0000 7fd16406b640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c108ad0 0x7fd15c1109e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00e9f0 tx=0x7fd14c00eeb0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.561+0000 7fd16406b640 1 -- 172.21.15.69:0/1698277311 shutdown_connections 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.561+0000 7fd16406b640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd1480a2f30 0x7fd1480a53f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.561+0000 7fd16406b640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd15c10d250 0x7fd15c0827c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.561+0000 7fd16406b640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd15c108ad0 0x7fd15c1109e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.561+0000 7fd16406b640 1 --2- 172.21.15.69:0/1698277311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd15c108120 0x7fd15c114940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.561+0000 7fd16406b640 1 -- 172.21.15.69:0/1698277311 >> 172.21.15.69:0/1698277311 conn(0x7fd15c077820 msgr2=0x7fd15c078ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.561+0000 7fd16406b640 1 -- 172.21.15.69:0/1698277311 shutdown_connections 2024-08-02T17:54:26.564 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:26.561+0000 7fd16406b640 1 -- 172.21.15.69:0/1698277311 wait complete. 2024-08-02T17:54:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:26 smithi083 ceph-mon[37031]: pgmap v247: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 643 op/s 2024-08-02T17:54:27.298 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:26 smithi069 ceph-mon[32746]: pgmap v247: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 643 op/s 2024-08-02T17:54:27.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:26 smithi077 ceph-mon[37108]: pgmap v247: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.4 MiB/s wr, 643 op/s 2024-08-02T17:54:28.748 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:54:28.748 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:54:29.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:28 smithi069 ceph-mon[32746]: pgmap v248: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 825 op/s 2024-08-02T17:54:29.178 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:28 smithi083 ceph-mon[37031]: pgmap v248: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 825 op/s 2024-08-02T17:54:29.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:28 smithi077 ceph-mon[37108]: pgmap v248: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 8.4 MiB/s wr, 825 op/s 2024-08-02T17:54:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 440.0 s, 882.7 tps, lat 107.022 ms stddev 135.528 2024-08-02T17:54:30.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.304+0000 7ff312eb9640 1 -- 172.21.15.69:0/1574931839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c109840 msgr2=0x7ff30c1142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.304+0000 7ff312eb9640 1 --2- 172.21.15.69:0/1574931839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c109840 0x7ff30c1142a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff300009860 tx=0x7ff30002e820 comp rx=0 tx=0).stop 2024-08-02T17:54:30.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.311+0000 7ff312eb9640 1 -- 172.21.15.69:0/1574931839 shutdown_connections 2024-08-02T17:54:30.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.311+0000 7ff312eb9640 1 --2- 172.21.15.69:0/1574931839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c109840 0x7ff30c1142a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.311+0000 7ff312eb9640 1 --2- 172.21.15.69:0/1574931839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c1147e0 0x7ff30c116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.311+0000 7ff312eb9640 1 --2- 172.21.15.69:0/1574931839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c108f20 0x7ff30c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.311+0000 7ff312eb9640 1 -- 172.21.15.69:0/1574931839 >> 172.21.15.69:0/1574931839 conn(0x7ff30c06de70 msgr2=0x7ff30c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.315+0000 7ff312eb9640 1 -- 172.21.15.69:0/1574931839 shutdown_connections 2024-08-02T17:54:30.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.315+0000 7ff312eb9640 1 -- 172.21.15.69:0/1574931839 wait complete. 2024-08-02T17:54:30.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.316+0000 7ff312eb9640 1 Processor -- start 2024-08-02T17:54:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.317+0000 7ff312eb9640 1 -- start start 2024-08-02T17:54:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.318+0000 7ff312eb9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c108f20 0x7ff30c1b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.318+0000 7ff312eb9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c109840 0x7ff30c1b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.318+0000 7ff312eb9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c1147e0 0x7ff30c1af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.318+0000 7ff312eb9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c1292e0 con 0x7ff30c109840 2024-08-02T17:54:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.318+0000 7ff312eb9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c129160 con 0x7ff30c1147e0 2024-08-02T17:54:30.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.318+0000 7ff312eb9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c129460 con 0x7ff30c108f20 2024-08-02T17:54:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.321+0000 7ff311456640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c1147e0 0x7ff30c1af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.321+0000 7ff311456640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c1147e0 0x7ff30c1af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59588/0 (socket says 172.21.15.69:59588) 2024-08-02T17:54:30.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.321+0000 7ff311456640 1 -- 172.21.15.69:0/481166500 learned_addr learned my addr 172.21.15.69:0/481166500 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.322+0000 7ff311456640 1 -- 172.21.15.69:0/481166500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c108f20 msgr2=0x7ff30c1b5510 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:54:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.322+0000 7ff311456640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c108f20 0x7ff30c1b5510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.322+0000 7ff311456640 1 -- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c109840 msgr2=0x7ff30c1b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.322+0000 7ff311456640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c109840 0x7ff30c1b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.322+0000 7ff311456640 1 -- 172.21.15.69:0/481166500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c1afea0 con 0x7ff30c1147e0 2024-08-02T17:54:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.322+0000 7ff311456640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c1147e0 0x7ff30c1af670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff30400ef30 tx=0x7ff30400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.323+0000 7ff309ffb640 1 -- 172.21.15.69:0/481166500 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff304004b20 con 0x7ff30c1147e0 2024-08-02T17:54:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.323+0000 7ff312eb9640 1 -- 172.21.15.69:0/481166500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c1b0190 con 0x7ff30c1147e0 2024-08-02T17:54:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.323+0000 7ff312eb9640 1 -- 172.21.15.69:0/481166500 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c1bc500 con 0x7ff30c1147e0 2024-08-02T17:54:30.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.326+0000 7ff3152b7740 1 Processor -- start 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff3152b7740 1 -- start start 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110d95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9b20 0x55e0110e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff3152b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e01109ce40 con 0x55e0110d9b20 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff3152b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e01103e7f0 con 0x55e0110e43e0 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff3152b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e010fe5fb0 con 0x55e0110d9200 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff309ffb640 1 -- 172.21.15.69:0/481166500 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff304003e50 con 0x7ff30c1147e0 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.328+0000 7ff309ffb640 1 -- 172.21.15.69:0/481166500 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30400b570 con 0x7ff30c1147e0 2024-08-02T17:54:30.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.329+0000 7ff309ffb640 1 -- 172.21.15.69:0/481166500 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff30401d020 con 0x7ff30c1147e0 2024-08-02T17:54:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.330+0000 7ff309ffb640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff2f40a2e10 0x7ff2f40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.330+0000 7ff309ffb640 1 -- 172.21.15.69:0/481166500 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff3040d0880 con 0x7ff30c1147e0 2024-08-02T17:54:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.333+0000 7ff310c55640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9b20 0x55e0110e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.333+0000 7ff310c55640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9b20 0x55e0110e3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39176/0 (socket says 172.21.15.69:39176) 2024-08-02T17:54:30.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.333+0000 7ff310c55640 1 -- 172.21.15.69:0/2629130234 learned_addr learned my addr 172.21.15.69:0/2629130234 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.335+0000 7ff311456640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff2f40a2e10 0x7ff2f40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.335+0000 7ff310c55640 1 -- 172.21.15.69:0/2629130234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 msgr2=0x55e0110d95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.335+0000 7ff310c55640 1 --2- 172.21.15.69:0/2629130234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110d95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.335+0000 7ff310c55640 1 -- 172.21.15.69:0/2629130234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 msgr2=0x55e0110e67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.335+0000 7ff310c55640 1 --2- 172.21.15.69:0/2629130234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.335+0000 7ff310c55640 1 -- 172.21.15.69:0/2629130234 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e010db06a0 con 0x55e0110d9b20 2024-08-02T17:54:30.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.339+0000 7ff310c55640 1 --2- 172.21.15.69:0/2629130234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9b20 0x55e0110e3ea0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009bd0 tx=0x7ff2fc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aa066b72714deb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.341+0000 7ff311456640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff2f40a2e10 0x7ff2f40a52d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3040cdc60 tx=0x7ff3040099a0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.343+0000 7ff2f2ffd640 1 -- 172.21.15.69:0/2629130234 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc005020 con 0x55e0110d9b20 2024-08-02T17:54:30.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.343+0000 7ff2f2ffd640 1 -- 172.21.15.69:0/2629130234 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc0047a0 con 0x55e0110d9b20 2024-08-02T17:54:30.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.343+0000 7ff2f2ffd640 1 -- 172.21.15.69:0/2629130234 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0186d0 con 0x55e0110d9b20 2024-08-02T17:54:30.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.346+0000 7ff3152b7740 1 -- 172.21.15.69:0/2629130234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9b20 msgr2=0x55e0110e3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.346+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2629130234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9b20 0x55e0110e3ea0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009bd0 tx=0x7ff2fc00e720 comp rx=0 tx=0).stop 2024-08-02T17:54:30.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.351+0000 7ff3152b7740 1 -- 172.21.15.69:0/2629130234 shutdown_connections 2024-08-02T17:54:30.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.351+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2629130234 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9b20 0x55e0110e3ea0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.351+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2629130234 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.351+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2629130234 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110d95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.351+0000 7ff3152b7740 1 -- 172.21.15.69:0/2629130234 >> 172.21.15.69:0/2629130234 conn(0x55e0110d5790 msgr2=0x55e0110d5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.355+0000 7ff3152b7740 1 -- 172.21.15.69:0/2629130234 shutdown_connections 2024-08-02T17:54:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.356+0000 7ff3152b7740 1 -- 172.21.15.69:0/2629130234 wait complete. 2024-08-02T17:54:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.356+0000 7ff3152b7740 1 -- start start 2024-08-02T17:54:30.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.357+0000 7ff3152b7740 1 Processor -- start 2024-08-02T17:54:30.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.357+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110e1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.357+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.357+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110e1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.357+0000 7ff3152b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e010ff9cb0 con 0x55e0110e43e0 2024-08-02T17:54:30.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.357+0000 7ff3152b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e01103a7b0 con 0x55e0110d9b20 2024-08-02T17:54:30.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.357+0000 7ff3152b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e0110e6fb0 con 0x55e0110d9200 2024-08-02T17:54:30.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.357+0000 7ff3152b7740 10 client.0 _ll_register_callbacks cb 0x55e010d4dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:30.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.358+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110e1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.358+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110e1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39186/0 (socket says 172.21.15.69:39186) 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.358+0000 7ff30bfff640 1 -- 172.21.15.69:0/2191810227 learned_addr learned my addr 172.21.15.69:0/2191810227 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.358+0000 7ff30bfff640 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110e1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.358+0000 7ff310c55640 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.359+0000 7ff30bfff640 1 -- 172.21.15.69:0/2191810227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 msgr2=0x55e0110e1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.359+0000 7ff30bfff640 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110e1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.359+0000 7ff30bfff640 1 -- 172.21.15.69:0/2191810227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 msgr2=0x55e0110e1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.359+0000 7ff30bfff640 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.359+0000 7ff30bfff640 1 -- 172.21.15.69:0/2191810227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e010ff8bc0 con 0x55e0110e43e0 2024-08-02T17:54:30.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.359+0000 7ff30bfff640 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110e1dd0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff30002ed40 tx=0x7ff300005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac07df0d416cde68 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.359+0000 7ff2ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:30.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.360+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2191810227 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300046060 con 0x55e0110e43e0 2024-08-02T17:54:30.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.360+0000 7ff3152b7740 1 -- 172.21.15.69:0/2191810227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2fc0047a0 con 0x55e0110e43e0 2024-08-02T17:54:30.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.360+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2191810227 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff300004050 con 0x55e0110e43e0 2024-08-02T17:54:30.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.361+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2191810227 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff30004d050 con 0x55e0110e43e0 2024-08-02T17:54:30.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.363+0000 7ff3152b7740 10 client.17343 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:30.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.363+0000 7ff3152b7740 1 -- 172.21.15.69:0/2191810227 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e0110f2f80 con 0x55e0110e43e0 2024-08-02T17:54:30.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.364+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2191810227 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc0047a0 con 0x55e0110e43e0 2024-08-02T17:54:30.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.364+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2191810227 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff3000356b0 con 0x55e0110e43e0 2024-08-02T17:54:30.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.364+0000 7ff3152b7740 10 client.17343 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:30.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.364+0000 7ff3152b7740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:30.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.364+0000 7ff3152b7740 20 client.17343 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:30.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.364+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0110f4890 0x55e0110f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.364+0000 7ff3152b7740 4 client.17343 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:30.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.364+0000 7ff3152b7740 1 -- 172.21.15.69:0/2191810227 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e01104bd60 con 0x55e0110f4890 2024-08-02T17:54:30.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.366+0000 7ff310c55640 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0110f4890 0x55e0110f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.368+0000 7ff310c55640 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0110f4890 0x55e0110f6cf0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3118ee23f6b5df server_cookie=4f70b7e9a4b2fe14 in_seq=0 out_seq=0 2024-08-02T17:54:30.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.368+0000 7ff2ebfff640 10 client.17343 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:30.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.369+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2191810227 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e01104bd60 con 0x55e0110f4890 2024-08-02T17:54:30.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.369+0000 7ff2ebfff640 10 client.17343 handle_command_reply: tid=0 2024-08-02T17:54:30.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.369+0000 7ff3152b7740 1 client.17343 shutdown 2024-08-02T17:54:30.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.370+0000 7ff3152b7740 1 -- 172.21.15.69:0/2191810227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 msgr2=0x55e0110e1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.370+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110e1dd0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff30002ed40 tx=0x7ff300005c70 comp rx=0 tx=0).stop 2024-08-02T17:54:30.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.370+0000 7ff3152b7740 1 -- 172.21.15.69:0/2191810227 shutdown_connections 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.370+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110e1dd0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.370+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0110f4890 0x55e0110f6cf0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.370+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.370+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2191810227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110e1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.370+0000 7ff3152b7740 1 -- 172.21.15.69:0/2191810227 >> 172.21.15.69:0/2191810227 conn(0x55e0110d5790 msgr2=0x55e0110db4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.371+0000 7ff2ebfff640 0 client.17343 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.371+0000 7ff3152b7740 1 -- 172.21.15.69:0/2191810227 shutdown_connections 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.371+0000 7ff3152b7740 1 -- 172.21.15.69:0/2191810227 wait complete. 2024-08-02T17:54:30.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.372+0000 7ff3152b7740 20 client.17343 trim_cache size 0 max 16384 2024-08-02T17:54:30.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.408+0000 7ff3152b7740 1 Processor -- start 2024-08-02T17:54:30.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.408+0000 7ff3152b7740 1 -- start start 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.408+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110dbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.408+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.409+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110dbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.409+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110dbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53528/0 (socket says 172.21.15.69:53528) 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.409+0000 7ff30bfff640 1 -- 172.21.15.69:0/3164245569 learned_addr learned my addr 172.21.15.69:0/3164245569 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.409+0000 7ff3152b7740 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.409+0000 7ff310c55640 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.409+0000 7ff3152b7740 1 -- 172.21.15.69:0/3164245569 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8012a60 con 0x55e0110e43e0 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.409+0000 7ff3152b7740 1 -- 172.21.15.69:0/3164245569 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f8010330 con 0x55e0110d9b20 2024-08-02T17:54:30.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.409+0000 7ff3152b7740 1 -- 172.21.15.69:0/3164245569 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f800e180 con 0x55e0110d9200 2024-08-02T17:54:30.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.410+0000 7ff310c55640 1 -- 172.21.15.69:0/3164245569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 msgr2=0x55e0110dbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.410+0000 7ff310c55640 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110dbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.410+0000 7ff310c55640 1 -- 172.21.15.69:0/3164245569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 msgr2=0x55e0110fc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.410+0000 7ff310c55640 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.410+0000 7ff310c55640 1 -- 172.21.15.69:0/3164245569 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0110e0320 con 0x55e0110d9b20 2024-08-02T17:54:30.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.411+0000 7ff310c55640 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00ec20 tx=0x7ff2fc009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962e8a763a510f59 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.413+0000 7ff2f27fc640 1 -- 172.21.15.69:0/3164245569 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc009870 con 0x55e0110d9b20 2024-08-02T17:54:30.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.413+0000 7ff2f27fc640 1 -- 172.21.15.69:0/3164245569 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc005520 con 0x55e0110d9b20 2024-08-02T17:54:30.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.413+0000 7ff2f27fc640 1 -- 172.21.15.69:0/3164245569 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc025470 con 0x55e0110d9b20 2024-08-02T17:54:30.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.414+0000 7ff3152b7740 1 -- 172.21.15.69:0/3164245569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 msgr2=0x55e0110e1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.414+0000 7ff3152b7740 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1c20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00ec20 tx=0x7ff2fc009d60 comp rx=0 tx=0).stop 2024-08-02T17:54:30.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.414+0000 7ff3152b7740 1 -- 172.21.15.69:0/3164245569 shutdown_connections 2024-08-02T17:54:30.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.414+0000 7ff3152b7740 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110e43e0 0x55e0110fc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.414+0000 7ff3152b7740 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110d9b20 0x55e0110e1c20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.414+0000 7ff3152b7740 1 --2- 172.21.15.69:0/3164245569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9200 0x55e0110dbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.414+0000 7ff3152b7740 1 -- 172.21.15.69:0/3164245569 >> 172.21.15.69:0/3164245569 conn(0x55e0110d5790 msgr2=0x55e0110e33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.418+0000 7ff3152b7740 1 -- 172.21.15.69:0/3164245569 shutdown_connections 2024-08-02T17:54:30.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.418+0000 7ff3152b7740 1 -- 172.21.15.69:0/3164245569 wait complete. 2024-08-02T17:54:30.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.419+0000 7ff3152b7740 1 -- start start 2024-08-02T17:54:30.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.419+0000 7ff3152b7740 1 Processor -- start 2024-08-02T17:54:30.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9200 0x55e0110de3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9b20 0x55e0110de930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff3152b7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff3152b7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f800e7f0 con 0x55e0110d9200 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff3152b7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f800fc60 con 0x55e0110e43e0 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff3152b7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f800eec0 con 0x55e0110d9b20 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff3152b7740 10 client.0 _ll_register_callbacks cb 0x7ff300039070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff310c55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9b20 0x55e0110de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff30bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110dee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59636/0 (socket says 172.21.15.69:59636) 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff310c55640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9b20 0x55e0110de930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53538/0 (socket says 172.21.15.69:53538) 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.420+0000 7ff30bfff640 1 -- 172.21.15.69:0/2913690139 learned_addr learned my addr 172.21.15.69:0/2913690139 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.421+0000 7ff30bfff640 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9200 0x55e0110de3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.421+0000 7ff30bfff640 1 -- 172.21.15.69:0/2913690139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9b20 msgr2=0x55e0110de930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.421+0000 7ff30bfff640 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9b20 0x55e0110de930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.421+0000 7ff30bfff640 1 -- 172.21.15.69:0/2913690139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9200 msgr2=0x55e0110de3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.421+0000 7ff30bfff640 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9200 0x55e0110de3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.421+0000 7ff30bfff640 1 -- 172.21.15.69:0/2913690139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e01115e620 con 0x55e0110e43e0 2024-08-02T17:54:30.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.426+0000 7ff30bfff640 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110dee70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff300005c40 tx=0x7ff300005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e002a0c29909b7a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.426+0000 7ff2ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:54:30.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.426+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2913690139 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300046070 con 0x55e0110e43e0 2024-08-02T17:54:30.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.426+0000 7ff3152b7740 1 -- 172.21.15.69:0/2913690139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2fc005520 con 0x55e0110e43e0 2024-08-02T17:54:30.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.426+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2913690139 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff3000356b0 con 0x55e0110e43e0 2024-08-02T17:54:30.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.426+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2913690139 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff30005e050 con 0x55e0110e43e0 2024-08-02T17:54:30.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.428+0000 7ff3152b7740 10 client.26906 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:30.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.428+0000 7ff3152b7740 1 -- 172.21.15.69:0/2913690139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e01115f960 con 0x55e0110e43e0 2024-08-02T17:54:30.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.434+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2913690139 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc005520 con 0x55e0110e43e0 2024-08-02T17:54:30.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.434+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2913690139 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff300004a60 con 0x55e0110e43e0 2024-08-02T17:54:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.434+0000 7ff3152b7740 10 client.26906 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.434+0000 7ff3152b7740 10 client.26906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.434+0000 7ff3152b7740 20 client.26906 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.434+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e011160090 0x55e0111624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.434+0000 7ff3152b7740 4 client.26906 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:54:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.434+0000 7ff3152b7740 1 -- 172.21.15.69:0/2913690139 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e011046270 con 0x55e011160090 2024-08-02T17:54:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.435+0000 7ff310c55640 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e011160090 0x55e0111624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.435+0000 7ff2ebfff640 10 client.26906 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:30.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.436+0000 7ff310c55640 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e011160090 0x55e0111624b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e064c7a06f53bc6 server_cookie=577d28fad627a9e5 in_seq=0 out_seq=0 2024-08-02T17:54:30.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.437+0000 7ff2ebfff640 1 -- 172.21.15.69:0/2913690139 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e011046270 con 0x55e011160090 2024-08-02T17:54:30.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.437+0000 7ff2ebfff640 10 client.26906 handle_command_reply: tid=0 2024-08-02T17:54:30.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.437+0000 7ff3152b7740 1 client.26906 shutdown 2024-08-02T17:54:30.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.440+0000 7ff3152b7740 1 -- 172.21.15.69:0/2913690139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 msgr2=0x55e0110dee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.440+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110dee70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff300005c40 tx=0x7ff300005bd0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.440+0000 7ff3152b7740 1 -- 172.21.15.69:0/2913690139 shutdown_connections 2024-08-02T17:54:30.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.440+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e011160090 0x55e0111624b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.440+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0110e43e0 0x55e0110dee70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.440+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0110d9b20 0x55e0110de930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.440+0000 7ff3152b7740 1 --2- 172.21.15.69:0/2913690139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0110d9200 0x55e0110de3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.440+0000 7ff3152b7740 1 -- 172.21.15.69:0/2913690139 >> 172.21.15.69:0/2913690139 conn(0x55e0110d5790 msgr2=0x55e0110d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.441+0000 7ff2ebfff640 0 client.26906 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.441+0000 7ff3152b7740 1 -- 172.21.15.69:0/2913690139 shutdown_connections 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.441+0000 7ff3152b7740 1 -- 172.21.15.69:0/2913690139 wait complete. 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.441+0000 7ff3152b7740 20 client.26906 trim_cache size 0 max 16384 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:54:30.443 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "a9438187-9e45-44c6-a937-47726ca17d1b", 2024-08-02T17:54:30.444 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:54:30.444 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:54:30.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 -- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff2f40a2e10 msgr2=0x7ff2f40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff2f40a2e10 0x7ff2f40a52d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3040cdc60 tx=0x7ff3040099a0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 -- 172.21.15.69:0/481166500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c1147e0 msgr2=0x7ff30c1af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c1147e0 0x7ff30c1af670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff30400ef30 tx=0x7ff30400c800 comp rx=0 tx=0).stop 2024-08-02T17:54:30.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 -- 172.21.15.69:0/481166500 shutdown_connections 2024-08-02T17:54:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff30c109840 0x7ff30c1b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff2f40a2e10 0x7ff2f40a52d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff30c1147e0 0x7ff30c1af670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 --2- 172.21.15.69:0/481166500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff30c108f20 0x7ff30c1b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 -- 172.21.15.69:0/481166500 >> 172.21.15.69:0/481166500 conn(0x7ff30c06de70 msgr2=0x7ff30c070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 -- 172.21.15.69:0/481166500 shutdown_connections 2024-08-02T17:54:30.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:30.442+0000 7ff312eb9640 1 -- 172.21.15.69:0/481166500 wait complete. 2024-08-02T17:54:30.539 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-02T17:54:30.539 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.4 2024-08-02T17:54:30.540 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:54:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 -- 172.21.15.69:0/317983013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c101070 msgr2=0x7fba8c10d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 --2- 172.21.15.69:0/317983013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c101070 0x7fba8c10d5a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fba84009cd0 tx=0x7fba8402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 -- 172.21.15.69:0/317983013 shutdown_connections 2024-08-02T17:54:30.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 --2- 172.21.15.69:0/317983013 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c101070 0x7fba8c10d5a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 --2- 172.21.15.69:0/317983013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba8c1006d0 0x7fba8c100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 --2- 172.21.15.69:0/317983013 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba8c1066e0 0x7fba8c106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 -- 172.21.15.69:0/317983013 >> 172.21.15.69:0/317983013 conn(0x7fba8c0776d0 msgr2=0x7fba8c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 -- 172.21.15.69:0/317983013 shutdown_connections 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.882+0000 7fba91f36640 1 -- 172.21.15.69:0/317983013 wait complete. 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba91f36640 1 Processor -- start 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba91f36640 1 -- start start 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba91f36640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1006d0 0x7fba8c196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba91f36640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba8c101070 0x7fba8c197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba91f36640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba8c1066e0 0x7fba8c19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba91f36640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c110b20 con 0x7fba8c1006d0 2024-08-02T17:54:30.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba91f36640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c1109a0 con 0x7fba8c101070 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba91f36640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c110ca0 con 0x7fba8c1066e0 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1006d0 0x7fba8c196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1006d0 0x7fba8c196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39216/0 (socket says 172.21.15.69:39216) 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba8b7fe640 1 -- 172.21.15.69:0/932779329 learned_addr learned my addr 172.21.15.69:0/932779329 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba83fff640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba8c101070 0x7fba8c197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba8b7fe640 1 -- 172.21.15.69:0/932779329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba8c1066e0 msgr2=0x7fba8c19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba8c1066e0 0x7fba8c19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba8b7fe640 1 -- 172.21.15.69:0/932779329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba8c101070 msgr2=0x7fba8c197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba8c101070 0x7fba8c197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.884+0000 7fba8b7fe640 1 -- 172.21.15.69:0/932779329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8c19bfc0 con 0x7fba8c1006d0 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.885+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1006d0 0x7fba8c196fd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009970 tx=0x7fba7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.885+0000 7fba897fa640 1 -- 172.21.15.69:0/932779329 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00d020 con 0x7fba8c1006d0 2024-08-02T17:54:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.885+0000 7fba897fa640 1 -- 172.21.15.69:0/932779329 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba7c007d40 con 0x7fba8c1006d0 2024-08-02T17:54:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.885+0000 7fba91f36640 1 -- 172.21.15.69:0/932779329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba8c19c2b0 con 0x7fba8c1006d0 2024-08-02T17:54:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.885+0000 7fba897fa640 1 -- 172.21.15.69:0/932779329 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00ccc0 con 0x7fba8c1006d0 2024-08-02T17:54:30.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.885+0000 7fba91f36640 1 -- 172.21.15.69:0/932779329 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8c069740 con 0x7fba8c1006d0 2024-08-02T17:54:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.887+0000 7fba94d65740 1 Processor -- start 2024-08-02T17:54:30.897 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.887+0000 7fba94d65740 1 -- start start 2024-08-02T17:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.887+0000 7fba94d65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 0x55bdb471fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.887+0000 7fba94d65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.887+0000 7fba94d65740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4731170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.887+0000 7fba94d65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bdb46839b0 con 0x55bdb471ff90 2024-08-02T17:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.887+0000 7fba94d65740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bdb462b800 con 0x55bdb472ed80 2024-08-02T17:54:30.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.887+0000 7fba94d65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bdb467f970 con 0x55bdb471f670 2024-08-02T17:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba897fa640 1 -- 172.21.15.69:0/932779329 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fba7c00c460 con 0x7fba8c1006d0 2024-08-02T17:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4731170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba8bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 0x55bdb471fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4731170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59668/0 (socket says 172.21.15.69:59668) 2024-08-02T17:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 -- 172.21.15.69:0/1768634612 learned_addr learned my addr 172.21.15.69:0/1768634612 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 -- 172.21.15.69:0/1768634612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 msgr2=0x55bdb471fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 0x55bdb471fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 -- 172.21.15.69:0/1768634612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 msgr2=0x55bdb472e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 -- 172.21.15.69:0/1768634612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb4397210 con 0x55bdb472ed80 2024-08-02T17:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba83fff640 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4731170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fba7400b690 tx=0x7fba7400bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b69372ac3a55133 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba81ffb640 1 -- 172.21.15.69:0/1768634612 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74003bd0 con 0x55bdb472ed80 2024-08-02T17:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba81ffb640 1 -- 172.21.15.69:0/1768634612 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba74015070 con 0x55bdb472ed80 2024-08-02T17:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba81ffb640 1 -- 172.21.15.69:0/1768634612 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74004de0 con 0x55bdb472ed80 2024-08-02T17:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.888+0000 7fba897fa640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba780a2cf0 0x7fba780a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba8bfff640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba780a2cf0 0x7fba780a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba897fa640 1 -- 172.21.15.69:0/932779329 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fba7c0cea10 con 0x7fba8c1006d0 2024-08-02T17:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 -- 172.21.15.69:0/1768634612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 msgr2=0x55bdb4731170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba8bfff640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba780a2cf0 0x7fba780a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fba84009a60 tx=0x7fba84009360 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4731170 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fba7400b690 tx=0x7fba7400bb50 comp rx=0 tx=0).stop 2024-08-02T17:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 -- 172.21.15.69:0/1768634612 shutdown_connections 2024-08-02T17:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4731170 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 --2- 172.21.15.69:0/1768634612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 0x55bdb471fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 -- 172.21.15.69:0/1768634612 >> 172.21.15.69:0/1768634612 conn(0x55bdb471bd20 msgr2=0x55bdb471c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 -- 172.21.15.69:0/1768634612 shutdown_connections 2024-08-02T17:54:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.889+0000 7fba94d65740 1 -- 172.21.15.69:0/1768634612 wait complete. 2024-08-02T17:54:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 1 -- start start 2024-08-02T17:54:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 1 Processor -- start 2024-08-02T17:54:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb471f670 0x55bdb472bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb472ed80 0x55bdb472c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bdb4731950 con 0x55bdb471ff90 2024-08-02T17:54:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bdb463f500 con 0x55bdb471f670 2024-08-02T17:54:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bdb4731ad0 con 0x55bdb472ed80 2024-08-02T17:54:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba94d65740 10 client.0 _ll_register_callbacks cb 0x55bdb4396a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba8b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb471f670 0x55bdb472bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39234/0 (socket says 172.21.15.69:39234) 2024-08-02T17:54:30.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba8b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb471f670 0x55bdb472bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59674/0 (socket says 172.21.15.69:59674) 2024-08-02T17:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba83fff640 1 -- 172.21.15.69:0/3245852282 learned_addr learned my addr 172.21.15.69:0/3245852282 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba8bfff640 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb472ed80 0x55bdb472c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba8b7fe640 1 -- 172.21.15.69:0/3245852282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb472ed80 msgr2=0x55bdb472c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb472ed80 0x55bdb472c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba8b7fe640 1 -- 172.21.15.69:0/3245852282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 msgr2=0x55bdb472c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.890+0000 7fba8b7fe640 1 -- 172.21.15.69:0/3245852282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb4463620 con 0x55bdb471f670 2024-08-02T17:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb471f670 0x55bdb472bcc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0cbd20 tx=0x7fba7c09cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=893390e90c1176a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba6affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba6affd640 1 -- 172.21.15.69:0/3245852282 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0cc1e0 con 0x55bdb471f670 2024-08-02T17:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba6affd640 1 -- 172.21.15.69:0/3245852282 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fba7c09c180 con 0x55bdb471f670 2024-08-02T17:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba94d65740 1 -- 172.21.15.69:0/3245852282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba74015070 con 0x55bdb471f670 2024-08-02T17:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba6affd640 1 -- 172.21.15.69:0/3245852282 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba7c09d430 con 0x55bdb471f670 2024-08-02T17:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba94d65740 10 client.26918 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:30.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba94d65740 1 -- 172.21.15.69:0/3245852282 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bdb473d6d0 con 0x55bdb471f670 2024-08-02T17:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba6affd640 1 -- 172.21.15.69:0/3245852282 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba74015070 con 0x55bdb471f670 2024-08-02T17:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.891+0000 7fba6affd640 1 -- 172.21.15.69:0/3245852282 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fba7c098a80 con 0x55bdb471f670 2024-08-02T17:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba94d65740 10 client.26918 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba94d65740 10 client.26918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba94d65740 20 client.26918 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba94d65740 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bdb473eb90 0x55bdb4740f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba94d65740 4 client.26918 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba94d65740 1 -- 172.21.15.69:0/3245852282 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdb46910e0 con 0x55bdb473eb90 2024-08-02T17:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba83fff640 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bdb473eb90 0x55bdb4740f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba83fff640 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bdb473eb90 0x55bdb4740f80 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eda82175bbd5b3f server_cookie=6f9824451c8b85a8 in_seq=0 out_seq=0 2024-08-02T17:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.892+0000 7fba6affd640 10 client.26918 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba6affd640 1 -- 172.21.15.69:0/3245852282 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bdb46910e0 con 0x55bdb473eb90 2024-08-02T17:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba6affd640 10 client.26918 handle_command_reply: tid=0 2024-08-02T17:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 client.26918 shutdown 2024-08-02T17:54:30.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 -- 172.21.15.69:0/3245852282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb471f670 msgr2=0x55bdb472bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb471f670 0x55bdb472bcc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba7c0cbd20 tx=0x7fba7c09cbc0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 -- 172.21.15.69:0/3245852282 shutdown_connections 2024-08-02T17:54:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb472c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bdb473eb90 0x55bdb4740f80 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb472ed80 0x55bdb472c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 --2- 172.21.15.69:0/3245852282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb471f670 0x55bdb472bcc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba6affd640 0 client.26918 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:30.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 -- 172.21.15.69:0/3245852282 >> 172.21.15.69:0/3245852282 conn(0x55bdb471bd20 msgr2=0x55bdb47219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 -- 172.21.15.69:0/3245852282 shutdown_connections 2024-08-02T17:54:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 1 -- 172.21.15.69:0/3245852282 wait complete. 2024-08-02T17:54:30.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.893+0000 7fba94d65740 20 client.26918 trim_cache size 0 max 16384 2024-08-02T17:54:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba94d65740 1 Processor -- start 2024-08-02T17:54:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba94d65740 1 -- start start 2024-08-02T17:54:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba94d65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471f670 0x55bdb472db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba94d65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471ff90 0x55bdb4746890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba94d65740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4746dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba94d65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c1af5d0 con 0x55bdb471f670 2024-08-02T17:54:30.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba94d65740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c1acea0 con 0x55bdb472ed80 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba94d65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c1aacf0 con 0x55bdb471ff90 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471f670 0x55bdb472db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba8bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4746dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba83fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471ff90 0x55bdb4746890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba8b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471f670 0x55bdb472db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39246/0 (socket says 172.21.15.69:39246) 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba8bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4746dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59676/0 (socket says 172.21.15.69:59676) 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba8b7fe640 1 -- 172.21.15.69:0/2009869447 learned_addr learned my addr 172.21.15.69:0/2009869447 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.909+0000 7fba83fff640 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471ff90 0x55bdb4746890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53572/0 (socket says 172.21.15.69:53572) 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba8b7fe640 1 -- 172.21.15.69:0/2009869447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471ff90 msgr2=0x55bdb4746890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471ff90 0x55bdb4746890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba8b7fe640 1 -- 172.21.15.69:0/2009869447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 msgr2=0x55bdb4746dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4746dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba8b7fe640 1 -- 172.21.15.69:0/2009869447 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb47314b0 con 0x55bdb471f670 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba8bfff640 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4746dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471f670 0x55bdb472db20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fba7c000c00 tx=0x7fba7c09cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb483e5f1acfb367 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba817fa640 1 -- 172.21.15.69:0/2009869447 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c098fd0 con 0x55bdb471f670 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba817fa640 1 -- 172.21.15.69:0/2009869447 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba7c09cd30 con 0x55bdb471f670 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.910+0000 7fba817fa640 1 -- 172.21.15.69:0/2009869447 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c09c690 con 0x55bdb471f670 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 -- 172.21.15.69:0/2009869447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471f670 msgr2=0x55bdb472db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471f670 0x55bdb472db20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fba7c000c00 tx=0x7fba7c09cc30 comp rx=0 tx=0).stop 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 -- 172.21.15.69:0/2009869447 shutdown_connections 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4746dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.918 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471ff90 0x55bdb4746890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 --2- 172.21.15.69:0/2009869447 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471f670 0x55bdb472db20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 -- 172.21.15.69:0/2009869447 >> 172.21.15.69:0/2009869447 conn(0x55bdb471bd20 msgr2=0x55bdb472d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 -- 172.21.15.69:0/2009869447 shutdown_connections 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 -- 172.21.15.69:0/2009869447 wait complete. 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 -- start start 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.911+0000 7fba94d65740 1 Processor -- start 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba94d65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 0x55bdb4737580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba94d65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb47328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.919 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba94d65740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4732e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba94d65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c1ab360 con 0x55bdb471ff90 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba94d65740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c1ac7d0 con 0x55bdb472ed80 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba94d65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba8c1aba30 con 0x55bdb471f670 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba94d65740 10 client.0 _ll_register_callbacks cb 0x7fba7c00d3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb47328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba8bfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4732e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb47328a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39250/0 (socket says 172.21.15.69:39250) 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 -- 172.21.15.69:0/2238556565 learned_addr learned my addr 172.21.15.69:0/2238556565 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba8b7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 0x55bdb4737580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 -- 172.21.15.69:0/2238556565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 msgr2=0x55bdb4737580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 0x55bdb4737580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 -- 172.21.15.69:0/2238556565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 msgr2=0x55bdb4732e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4732e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 -- 172.21.15.69:0/2238556565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb4733630 con 0x55bdb471ff90 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba8bfff640 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4732e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba83fff640 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb47328a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fba74004ac0 tx=0x7fba74005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670e3df613d47d10 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba6affd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba6affd640 1 -- 172.21.15.69:0/2238556565 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba74003dd0 con 0x55bdb471ff90 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba6affd640 1 -- 172.21.15.69:0/2238556565 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fba74011440 con 0x55bdb471ff90 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.912+0000 7fba94d65740 1 -- 172.21.15.69:0/2238556565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba7c09cd30 con 0x55bdb471ff90 2024-08-02T17:54:30.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba6affd640 1 -- 172.21.15.69:0/2238556565 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fba74011640 con 0x55bdb471ff90 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba94d65740 10 client.17364 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba94d65740 1 -- 172.21.15.69:0/2238556565 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bdb47a9900 con 0x55bdb471ff90 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba6affd640 1 -- 172.21.15.69:0/2238556565 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba7c09cd30 con 0x55bdb471ff90 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba6affd640 1 -- 172.21.15.69:0/2238556565 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fba740197c0 con 0x55bdb471ff90 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba94d65740 10 client.17364 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba94d65740 10 client.17364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba94d65740 20 client.17364 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba94d65740 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bdb47a9f10 0x55bdb47ac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba94d65740 4 client.17364 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba94d65740 1 -- 172.21.15.69:0/2238556565 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdb468b430 con 0x55bdb47a9f10 2024-08-02T17:54:30.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.913+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bdb47a9f10 0x55bdb47ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.914+0000 7fba6affd640 10 client.17364 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:30.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:30.914+0000 7fba8b7fe640 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bdb47a9f10 0x55bdb47ac330 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16e84ddfff007be6 server_cookie=a3c588026bb50bc1 in_seq=0 out_seq=0 2024-08-02T17:54:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:30 smithi083 ceph-mon[37031]: pgmap v249: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-08-02T17:54:31.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:30 smithi077 ceph-mon[37108]: pgmap v249: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-08-02T17:54:31.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:30 smithi069 ceph-mon[32746]: pgmap v249: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.5 MiB/s wr, 670 op/s 2024-08-02T17:54:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:31 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:54:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:31 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:31 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:54:32.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:31 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:31 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:54:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:31 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:31 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:54:32.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:31 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:31 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:54:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:31 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:31 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:54:32.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:31 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:32 smithi083 ceph-mon[37031]: pgmap v250: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-08-02T17:54:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:32 smithi077 ceph-mon[37108]: pgmap v250: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-08-02T17:54:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:32 smithi069 ceph-mon[32746]: pgmap v250: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.6 MiB/s wr, 686 op/s 2024-08-02T17:54:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba6affd640 1 -- 172.21.15.69:0/2238556565 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdb468b430 con 0x55bdb47a9f10 2024-08-02T17:54:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba6affd640 10 client.17364 handle_command_reply: tid=0 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 client.17364 shutdown 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 -- 172.21.15.69:0/2238556565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 msgr2=0x55bdb47328a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb47328a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fba74004ac0 tx=0x7fba74005e50 comp rx=0 tx=0).stop 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 -- 172.21.15.69:0/2238556565 shutdown_connections 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bdb471ff90 0x55bdb47328a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bdb47a9f10 0x55bdb47ac330 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bdb472ed80 0x55bdb4732e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 --2- 172.21.15.69:0/2238556565 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bdb471f670 0x55bdb4737580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba6affd640 0 client.17364 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 -- 172.21.15.69:0/2238556565 >> 172.21.15.69:0/2238556565 conn(0x55bdb471bd20 msgr2=0x55bdb471ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 -- 172.21.15.69:0/2238556565 shutdown_connections 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.868+0000 7fba94d65740 1 -- 172.21.15.69:0/2238556565 wait complete. 2024-08-02T17:54:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba94d65740 20 client.17364 trim_cache size 0 max 16384 2024-08-02T17:54:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:54:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:54:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:54:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:54:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 -- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba780a2cf0 msgr2=0x7fba780a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba780a2cf0 0x7fba780a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fba84009a60 tx=0x7fba84009360 comp rx=0 tx=0).stop 2024-08-02T17:54:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 -- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1006d0 msgr2=0x7fba8c196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1006d0 0x7fba8c196fd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fba7c009970 tx=0x7fba7c009e30 comp rx=0 tx=0).stop 2024-08-02T17:54:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 -- 172.21.15.69:0/932779329 shutdown_connections 2024-08-02T17:54:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fba780a2cf0 0x7fba780a51b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba8c1066e0 0x7fba8c19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:33.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba8c101070 0x7fba8c197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 --2- 172.21.15.69:0/932779329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba8c1006d0 0x7fba8c196fd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 -- 172.21.15.69:0/932779329 >> 172.21.15.69:0/932779329 conn(0x7fba8c0776d0 msgr2=0x7fba8c103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 -- 172.21.15.69:0/932779329 shutdown_connections 2024-08-02T17:54:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:33.869+0000 7fba91f36640 1 -- 172.21.15.69:0/932779329 wait complete. 2024-08-02T17:54:34.275 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:54:34.293 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 445.0 s, 839.6 tps, lat 127.918 ms stddev 197.644 2024-08-02T17:54:34.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.986+0000 7f6981a4e640 1 -- 172.21.15.69:0/1437916736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074c70 msgr2=0x7f697c10dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.986+0000 7f6981a4e640 1 --2- 172.21.15.69:0/1437916736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074c70 0x7f697c10dbc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6964009860 tx=0x7f696402a3d0 comp rx=0 tx=0).stop 2024-08-02T17:54:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.987+0000 7f6981a4e640 1 -- 172.21.15.69:0/1437916736 shutdown_connections 2024-08-02T17:54:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.987+0000 7f6981a4e640 1 --2- 172.21.15.69:0/1437916736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c10e100 0x7f697c110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.987+0000 7f6981a4e640 1 --2- 172.21.15.69:0/1437916736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074c70 0x7f697c10dbc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.987+0000 7f6981a4e640 1 --2- 172.21.15.69:0/1437916736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f697c074350 0x7f697c074730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.987+0000 7f6981a4e640 1 -- 172.21.15.69:0/1437916736 >> 172.21.15.69:0/1437916736 conn(0x7f697c0fd620 msgr2=0x7f697c0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:34.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.988+0000 7f6981a4e640 1 -- 172.21.15.69:0/1437916736 shutdown_connections 2024-08-02T17:54:34.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.988+0000 7f6981a4e640 1 -- 172.21.15.69:0/1437916736 wait complete. 2024-08-02T17:54:34.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.988+0000 7f6981a4e640 1 Processor -- start 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.988+0000 7f6981a4e640 1 -- start start 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f6981a4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074350 0x7f697c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f6981a4e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f697c074c70 0x7f697c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f6981a4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c10e100 0x7f697c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f6981a4e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f697c113f00 con 0x7f697c074c70 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f6981a4e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f697c113d80 con 0x7f697c10e100 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f6981a4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f697c114080 con 0x7f697c074350 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074350 0x7f697c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c10e100 0x7f697c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697b7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c10e100 0x7f697c19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59400/0 (socket says 172.21.15.69:59400) 2024-08-02T17:54:34.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074350 0x7f697c19a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45926/0 (socket says 172.21.15.69:45926) 2024-08-02T17:54:34.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697b7fe640 1 -- 172.21.15.69:0/2230929094 learned_addr learned my addr 172.21.15.69:0/2230929094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:34.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f6973fff640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f697c074c70 0x7f697c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697b7fe640 1 -- 172.21.15.69:0/2230929094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074350 msgr2=0x7f697c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697b7fe640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074350 0x7f697c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697b7fe640 1 -- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f697c074c70 msgr2=0x7f697c19a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697b7fe640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f697c074c70 0x7f697c19a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.989+0000 7f697b7fe640 1 -- 172.21.15.69:0/2230929094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697c19f410 con 0x7f697c10e100 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.990+0000 7f697b7fe640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c10e100 0x7f697c19ed30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f697400efd0 tx=0x7f697400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.990+0000 7f6980a4c640 1 -- 172.21.15.69:0/2230929094 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6974003b70 con 0x7f697c10e100 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.990+0000 7f6980a4c640 1 -- 172.21.15.69:0/2230929094 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6974003d10 con 0x7f697c10e100 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.990+0000 7f6980a4c640 1 -- 172.21.15.69:0/2230929094 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69740108d0 con 0x7f697c10e100 2024-08-02T17:54:34.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.990+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f697c19f700 con 0x7f697c10e100 2024-08-02T17:54:34.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.990+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697c1a6fe0 con 0x7f697c10e100 2024-08-02T17:54:34.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.991+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697c074730 con 0x7f697c10e100 2024-08-02T17:54:34.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.993+0000 7f6980a4c640 1 -- 172.21.15.69:0/2230929094 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6974004b40 con 0x7f697c10e100 2024-08-02T17:54:34.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.994+0000 7f6980a4c640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f695c0a2d70 0x7f695c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:34.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.995+0000 7f6980a4c640 1 -- 172.21.15.69:0/2230929094 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f69740cfca0 con 0x7f697c10e100 2024-08-02T17:54:34.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.995+0000 7f697affd640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f695c0a2d70 0x7f695c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:34.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.995+0000 7f697affd640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f695c0a2d70 0x7f695c0a5230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f696c005fd0 tx=0x7f696c009260 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:35.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:34.998+0000 7f6980a4c640 1 -- 172.21.15.69:0/2230929094 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f697409c4b0 con 0x7f697c10e100 2024-08-02T17:54:35.012 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:35.162 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:34 smithi069 ceph-mon[32746]: pgmap v251: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.8 MiB/s wr, 991 op/s 2024-08-02T17:54:35.162 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:34 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:54:35.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:34 smithi083 ceph-mon[37031]: pgmap v251: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.8 MiB/s wr, 991 op/s 2024-08-02T17:54:35.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:34 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:54:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:34 smithi077 ceph-mon[37108]: pgmap v251: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 8.8 MiB/s wr, 991 op/s 2024-08-02T17:54:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:34 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:54:35.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.395+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f697c1092e0 con 0x7f697c10e100 2024-08-02T17:54:35.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.396+0000 7f6980a4c640 1 -- 172.21.15.69:0/2230929094 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f697409bdb0 con 0x7f697c10e100 2024-08-02T17:54:35.397 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:54:35.422 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:54:35.423 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:54:35.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f695c0a2d70 msgr2=0x7f695c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:35.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f695c0a2d70 0x7f695c0a5230 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f696c005fd0 tx=0x7f696c009260 comp rx=0 tx=0).stop 2024-08-02T17:54:35.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c10e100 msgr2=0x7f697c19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:35.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c10e100 0x7f697c19ed30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f697400efd0 tx=0x7f697400c6a0 comp rx=0 tx=0).stop 2024-08-02T17:54:35.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 shutdown_connections 2024-08-02T17:54:35.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f697c074c70 0x7f697c19a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:35.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f695c0a2d70 0x7f695c0a5230 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:35.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f697c10e100 0x7f697c19ed30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:35.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 --2- 172.21.15.69:0/2230929094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f697c074350 0x7f697c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:35.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 >> 172.21.15.69:0/2230929094 conn(0x7f697c0fd620 msgr2=0x7f697c0fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:35.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 shutdown_connections 2024-08-02T17:54:35.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.398+0000 7f6981a4e640 1 -- 172.21.15.69:0/2230929094 wait complete. 2024-08-02T17:54:36.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:35 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2230929094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:36.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:35 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2230929094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:36.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:35 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2230929094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:36.674 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:54:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:54:36.674 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:54:36.675 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:54:36.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.649+0000 7f97c97d4640 1 -- 172.21.15.69:0/1110635533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 msgr2=0x7f97c4114b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.649+0000 7f97c97d4640 1 --2- 172.21.15.69:0/1110635533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 0x7f97c4114b00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009860 tx=0x7f97b402a3f0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.650+0000 7f97c97d4640 1 -- 172.21.15.69:0/1110635533 shutdown_connections 2024-08-02T17:54:36.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.650+0000 7f97c97d4640 1 --2- 172.21.15.69:0/1110635533 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 0x7f97c4114b00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.650+0000 7f97c97d4640 1 --2- 172.21.15.69:0/1110635533 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c4074be0 0x7f97c41121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.650+0000 7f97c97d4640 1 --2- 172.21.15.69:0/1110635533 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c40742c0 0x7f97c40746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.650+0000 7f97c97d4640 1 -- 172.21.15.69:0/1110635533 >> 172.21.15.69:0/1110635533 conn(0x7f97c40fd440 msgr2=0x7f97c40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:36.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.650+0000 7f97c97d4640 1 -- 172.21.15.69:0/1110635533 shutdown_connections 2024-08-02T17:54:36.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.652+0000 7f97c97d4640 1 -- 172.21.15.69:0/1110635533 wait complete. 2024-08-02T17:54:36.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.653+0000 7f97c97d4640 1 Processor -- start 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.653+0000 7f97c97d4640 1 -- start start 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.653+0000 7f97c97d4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c40742c0 0x7f97c4073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.653+0000 7f97c97d4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c4074be0 0x7f97c4070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.653+0000 7f97c97d4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 0x7f97c40713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.653+0000 7f97c97d4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41083c0 con 0x7f97c40742c0 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.653+0000 7f97c97d4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4108240 con 0x7f97c4112710 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.653+0000 7f97c97d4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97c4108540 con 0x7f97c4074be0 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 0x7f97c40713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 0x7f97c40713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59408/0 (socket says 172.21.15.69:59408) 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 -- 172.21.15.69:0/2668263066 learned_addr learned my addr 172.21.15.69:0/2668263066 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c37fe640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c4074be0 0x7f97c4070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c3fff640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c40742c0 0x7f97c4073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 -- 172.21.15.69:0/2668263066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c4074be0 msgr2=0x7f97c4070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c4074be0 0x7f97c4070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 -- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c40742c0 msgr2=0x7f97c4073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c40742c0 0x7f97c4073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 -- 172.21.15.69:0/2668263066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c4111be0 con 0x7f97c4112710 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c3fff640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c40742c0 0x7f97c4073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 0x7f97c40713a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f97b40308d0 tx=0x7f97b4030900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.654+0000 7f97c17fa640 1 -- 172.21.15.69:0/2668263066 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4040070 con 0x7f97c4112710 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.655+0000 7f97c97d4640 1 -- 172.21.15.69:0/2668263066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c4111e70 con 0x7f97c4112710 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.655+0000 7f97c17fa640 1 -- 172.21.15.69:0/2668263066 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b4005620 con 0x7f97c4112710 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.655+0000 7f97c17fa640 1 -- 172.21.15.69:0/2668263066 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4004920 con 0x7f97c4112710 2024-08-02T17:54:36.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.655+0000 7f97c97d4640 1 -- 172.21.15.69:0/2668263066 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c4112370 con 0x7f97c4112710 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.658+0000 7f97cce34740 1 Processor -- start 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.659+0000 7f97c17fa640 1 -- 172.21.15.69:0/2668263066 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f97b4043070 con 0x7f97c4112710 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.659+0000 7f97cce34740 1 -- start start 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97cce34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf153c0 0x5636fbf157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97cce34740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf20060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97cce34740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf205a0 0x5636fbf22990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97cce34740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636fbe7a980 con 0x5636fbf205a0 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97cce34740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636fbe12a70 con 0x5636fbf15ce0 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97cce34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636fbe76940 con 0x5636fbf153c0 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf153c0 0x5636fbf157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c37fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf153c0 0x5636fbf157a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45948/0 (socket says 172.21.15.69:45948) 2024-08-02T17:54:36.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c37fe640 1 -- 172.21.15.69:0/1336761356 learned_addr learned my addr 172.21.15.69:0/1336761356 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c3fff640 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf20060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c37fe640 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf205a0 0x5636fbf22990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c3fff640 1 -- 172.21.15.69:0/1336761356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf153c0 msgr2=0x5636fbf157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c3fff640 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf153c0 0x5636fbf157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c3fff640 1 -- 172.21.15.69:0/1336761356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf205a0 msgr2=0x5636fbf22990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c3fff640 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf205a0 0x5636fbf22990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.660+0000 7f97c3fff640 1 -- 172.21.15.69:0/1336761356 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636fbe756e0 con 0x5636fbf15ce0 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.661+0000 7f97c3fff640 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf20060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97b801ac40 tx=0x7f97b80227c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df65ad2c9b058ba6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.661+0000 7f97c17fa640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f97ac0a2d70 0x7f97ac0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.661+0000 7f97aa7fc640 1 -- 172.21.15.69:0/1336761356 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b802f070 con 0x5636fbf15ce0 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.661+0000 7f97aa7fc640 1 -- 172.21.15.69:0/1336761356 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b802a8f0 con 0x5636fbf15ce0 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.661+0000 7f97aa7fc640 1 -- 172.21.15.69:0/1336761356 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8020900 con 0x5636fbf15ce0 2024-08-02T17:54:36.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.661+0000 7f97c17fa640 1 -- 172.21.15.69:0/2668263066 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f97b40f0280 con 0x7f97c4112710 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.662+0000 7f97cce34740 1 -- 172.21.15.69:0/1336761356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 msgr2=0x5636fbf20060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.662+0000 7f97cce34740 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf20060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97b801ac40 tx=0x7f97b80227c0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.662+0000 7f97c37fe640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f97ac0a2d70 0x7f97ac0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.663+0000 7f97c37fe640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f97ac0a2d70 0x7f97ac0a5230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5636fbf12cd0 tx=0x7f97bc00d040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.663+0000 7f97cce34740 1 -- 172.21.15.69:0/1336761356 shutdown_connections 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.663+0000 7f97cce34740 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf205a0 0x5636fbf22990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.663+0000 7f97cce34740 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf20060 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.663+0000 7f97cce34740 1 --2- 172.21.15.69:0/1336761356 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf153c0 0x5636fbf157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.663+0000 7f97cce34740 1 -- 172.21.15.69:0/1336761356 >> 172.21.15.69:0/1336761356 conn(0x5636fbf11950 msgr2=0x5636fbf11d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.663+0000 7f97cce34740 1 -- 172.21.15.69:0/1336761356 shutdown_connections 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.663+0000 7f97cce34740 1 -- 172.21.15.69:0/1336761356 wait complete. 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 1 -- start start 2024-08-02T17:54:36.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 1 Processor -- start 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf1da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5636fbf23170 con 0x5636fbf153c0 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5636fbe35aa0 con 0x5636fbf15ce0 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5636fbf232f0 con 0x5636fbf205a0 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.664+0000 7f97cce34740 10 client.0 _ll_register_callbacks cb 0x5636fbe24e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58082/0 (socket says 172.21.15.69:58082) 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 -- 172.21.15.69:0/1260656384 learned_addr learned my addr 172.21.15.69:0/1260656384 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:36.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c37fe640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf1da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 -- 172.21.15.69:0/1260656384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 msgr2=0x5636fbf1dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 -- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 msgr2=0x5636fbf1da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf1da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 -- 172.21.15.69:0/1260656384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636fbe19650 con 0x5636fbf153c0 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c37fe640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf1da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97c3fff640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1d530 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f97b802d040 tx=0x7f97b802acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1502178d24a84cf5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.665+0000 7f97a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:36.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.666+0000 7f97a37fe640 1 -- 172.21.15.69:0/1260656384 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b802f040 con 0x5636fbf153c0 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.666+0000 7f97cce34740 1 -- 172.21.15.69:0/1260656384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b802a8f0 con 0x5636fbf153c0 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.666+0000 7f97a37fe640 1 -- 172.21.15.69:0/1260656384 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f97b8027040 con 0x5636fbf153c0 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.666+0000 7f97a37fe640 1 -- 172.21.15.69:0/1260656384 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f97b8043050 con 0x5636fbf153c0 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.666+0000 7f97cce34740 10 client.17382 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.666+0000 7f97cce34740 1 -- 172.21.15.69:0/1260656384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636fbbec3c0 con 0x5636fbf153c0 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.667+0000 7f97a37fe640 1 -- 172.21.15.69:0/1260656384 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b802a8f0 con 0x5636fbf153c0 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.667+0000 7f97a37fe640 1 -- 172.21.15.69:0/1260656384 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f97b8020880 con 0x5636fbf153c0 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.667+0000 7f97cce34740 10 client.17382 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.667+0000 7f97cce34740 10 client.17382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.667+0000 7f97cce34740 20 client.17382 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.667+0000 7f97cce34740 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5636fbf30060 0x5636fbf32450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.667+0000 7f97cce34740 4 client.17382 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:36.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.667+0000 7f97cce34740 1 -- 172.21.15.69:0/1260656384 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636fbe87ef0 con 0x5636fbf30060 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.668+0000 7f97c37fe640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5636fbf30060 0x5636fbf32450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.668+0000 7f97c37fe640 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5636fbf30060 0x5636fbf32450 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3a1523ab9df53a server_cookie=41dd7ca8e0d18c5d in_seq=0 out_seq=0 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.668+0000 7f97a37fe640 10 client.17382 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97a37fe640 1 -- 172.21.15.69:0/1260656384 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5636fbe87ef0 con 0x5636fbf30060 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97a37fe640 10 client.17382 handle_command_reply: tid=0 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 client.17382 shutdown 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 -- 172.21.15.69:0/1260656384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 msgr2=0x5636fbf1d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1d530 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f97b802d040 tx=0x7f97b802acd0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 -- 172.21.15.69:0/1260656384 shutdown_connections 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5636fbf30060 0x5636fbf32450 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf15ce0 0x5636fbf1da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 --2- 172.21.15.69:0/1260656384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1d530 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97cce34740 1 -- 172.21.15.69:0/1260656384 >> 172.21.15.69:0/1260656384 conn(0x5636fbf11950 msgr2=0x5636fbf10ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.669+0000 7f97a37fe640 0 client.17382 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.670+0000 7f97cce34740 1 -- 172.21.15.69:0/1260656384 shutdown_connections 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.670+0000 7f97cce34740 1 -- 172.21.15.69:0/1260656384 wait complete. 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.670+0000 7f97cce34740 20 client.17382 trim_cache size 0 max 16384 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.708+0000 7f97cce34740 1 Processor -- start 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.708+0000 7f97cce34740 1 -- start start 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97cce34740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97cce34740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf15ce0 0x5636fbf17e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97cce34740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf205a0 0x5636fbf37be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97cce34740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b7ca0 con 0x5636fbf153c0 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97cce34740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b5570 con 0x5636fbf205a0 2024-08-02T17:54:36.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97cce34740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b33c0 con 0x5636fbf15ce0 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97c8fd3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf205a0 0x5636fbf37be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97c8fd3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf205a0 0x5636fbf37be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59436/0 (socket says 172.21.15.69:59436) 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.709+0000 7f97c8fd3640 1 -- 172.21.15.69:0/830381132 learned_addr learned my addr 172.21.15.69:0/830381132 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.714+0000 7f97c8fd3640 1 -- 172.21.15.69:0/830381132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf15ce0 msgr2=0x5636fbf17e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.714+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/830381132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf15ce0 0x5636fbf17e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.714+0000 7f97c8fd3640 1 -- 172.21.15.69:0/830381132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 msgr2=0x5636fbf1be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.714+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/830381132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.714+0000 7f97c8fd3640 1 -- 172.21.15.69:0/830381132 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636fbf1c4e0 con 0x5636fbf205a0 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.714+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/830381132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf205a0 0x5636fbf37be0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97b4030970 tx=0x7f97b4031890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b448f9d274f1b477 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.716+0000 7f97a9ffb640 1 -- 172.21.15.69:0/830381132 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40bca90 con 0x5636fbf205a0 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.716+0000 7f97a9ffb640 1 -- 172.21.15.69:0/830381132 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b40bc390 con 0x5636fbf205a0 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.716+0000 7f97a9ffb640 1 -- 172.21.15.69:0/830381132 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4031400 con 0x5636fbf205a0 2024-08-02T17:54:36.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.717+0000 7f97cce34740 1 -- 172.21.15.69:0/830381132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf205a0 msgr2=0x5636fbf37be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.717+0000 7f97cce34740 1 --2- 172.21.15.69:0/830381132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf205a0 0x5636fbf37be0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97b4030970 tx=0x7f97b4031890 comp rx=0 tx=0).stop 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.720+0000 7f97cce34740 1 -- 172.21.15.69:0/830381132 shutdown_connections 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.720+0000 7f97cce34740 1 --2- 172.21.15.69:0/830381132 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf205a0 0x5636fbf37be0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.720+0000 7f97cce34740 1 --2- 172.21.15.69:0/830381132 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf15ce0 0x5636fbf17e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.720+0000 7f97cce34740 1 --2- 172.21.15.69:0/830381132 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf153c0 0x5636fbf1be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.720+0000 7f97cce34740 1 -- 172.21.15.69:0/830381132 >> 172.21.15.69:0/830381132 conn(0x5636fbf11950 msgr2=0x5636fbf1f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.729+0000 7f97cce34740 1 -- 172.21.15.69:0/830381132 shutdown_connections 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.731+0000 7f97cce34740 1 -- 172.21.15.69:0/830381132 wait complete. 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.731+0000 7f97cce34740 1 -- start start 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.737+0000 7f97cce34740 1 Processor -- start 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.740+0000 7f97cce34740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf153c0 0x5636fbf28930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97cce34740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf15ce0 0x5636fbf28e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf153c0 0x5636fbf28930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf153c0 0x5636fbf28930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59438/0 (socket says 172.21.15.69:59438) 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c3fff640 1 -- 172.21.15.69:0/501401350 learned_addr learned my addr 172.21.15.69:0/501401350 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97cce34740 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b3a30 con 0x5636fbf15ce0 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b4ea0 con 0x5636fbf153c0 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41b4100 con 0x5636fbf205a0 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97cce34740 10 client.0 _ll_register_callbacks cb 0x7f97b802d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c3fff640 1 -- 172.21.15.69:0/501401350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 msgr2=0x5636fbf1a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c3fff640 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c3fff640 1 -- 172.21.15.69:0/501401350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf15ce0 msgr2=0x5636fbf28e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:36.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c3fff640 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf15ce0 0x5636fbf28e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c3fff640 1 -- 172.21.15.69:0/501401350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636fbf1a9c0 con 0x5636fbf153c0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.741+0000 7f97c8fd3640 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.742+0000 7f97c3fff640 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf153c0 0x5636fbf28930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97b802ad10 tx=0x7f97b80209f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d9b7ab0debbd340 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.743+0000 7f97a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.743+0000 7f97a37fe640 1 -- 172.21.15.69:0/501401350 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b802f070 con 0x5636fbf153c0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.743+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97b40bc390 con 0x5636fbf153c0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.743+0000 7f97a37fe640 1 -- 172.21.15.69:0/501401350 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f97b802a5f0 con 0x5636fbf153c0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.743+0000 7f97a37fe640 1 -- 172.21.15.69:0/501401350 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f97b801a270 con 0x5636fbf153c0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.743+0000 7f97cce34740 10 client.26948 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.743+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5636fbf9a8d0 con 0x5636fbf153c0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.750+0000 7f97a37fe640 1 -- 172.21.15.69:0/501401350 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b40bc390 con 0x5636fbf153c0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.750+0000 7f97a37fe640 1 -- 172.21.15.69:0/501401350 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f97b80035f0 con 0x5636fbf153c0 2024-08-02T17:54:36.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.750+0000 7f97cce34740 10 client.26948 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.750+0000 7f97cce34740 10 client.26948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.750+0000 7f97cce34740 20 client.26948 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.750+0000 7f97cce34740 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5636fbf9af30 0x5636fbf9d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.750+0000 7f97cce34740 4 client.26948 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.750+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636fbe82400 con 0x5636fbf9af30 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.751+0000 7f97c37fe640 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5636fbf9af30 0x5636fbf9d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.753+0000 7f97c37fe640 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5636fbf9af30 0x5636fbf9d350 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27c25ad9f2e7784 server_cookie=64275b9d1743815c in_seq=0 out_seq=0 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.755+0000 7f97a37fe640 10 client.26948 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.755+0000 7f97a37fe640 1 -- 172.21.15.69:0/501401350 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5636fbe82400 con 0x5636fbf9af30 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.755+0000 7f97a37fe640 10 client.26948 handle_command_reply: tid=0 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.755+0000 7f97cce34740 1 client.26948 shutdown 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.755+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf153c0 msgr2=0x5636fbf28930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.755+0000 7f97cce34740 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf153c0 0x5636fbf28930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f97b802ad10 tx=0x7f97b80209f0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.758+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 shutdown_connections 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.758+0000 7f97cce34740 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5636fbf15ce0 0x5636fbf28e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.758+0000 7f97cce34740 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5636fbf9af30 0x5636fbf9d350 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.758+0000 7f97cce34740 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5636fbf205a0 0x5636fbf1a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.758+0000 7f97cce34740 1 --2- 172.21.15.69:0/501401350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5636fbf153c0 0x5636fbf28930 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.758+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 >> 172.21.15.69:0/501401350 conn(0x5636fbf11950 msgr2=0x5636fbf10ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.758+0000 7f97a37fe640 0 client.26948 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.761+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 shutdown_connections 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.761+0000 7f97cce34740 1 -- 172.21.15.69:0/501401350 wait complete. 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.766+0000 7f97cce34740 20 client.26948 trim_cache size 0 max 16384 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 -- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f97ac0a2d70 msgr2=0x7f97ac0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f97ac0a2d70 0x7f97ac0a5230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x5636fbf12cd0 tx=0x7f97bc00d040 comp rx=0 tx=0).stop 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 -- 172.21.15.69:0/2668263066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 msgr2=0x7f97c40713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:36.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 0x7f97c40713a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f97b40308d0 tx=0x7f97b4030900 comp rx=0 tx=0).stop 2024-08-02T17:54:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 -- 172.21.15.69:0/2668263066 shutdown_connections 2024-08-02T17:54:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f97ac0a2d70 0x7f97ac0a5230 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f97c4112710 0x7f97c40713a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f97c4074be0 0x7f97c4070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 --2- 172.21.15.69:0/2668263066 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f97c40742c0 0x7f97c4073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.773+0000 7f97c97d4640 1 -- 172.21.15.69:0/2668263066 >> 172.21.15.69:0/2668263066 conn(0x7f97c40fd440 msgr2=0x7f97c40feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.777+0000 7f97c97d4640 1 -- 172.21.15.69:0/2668263066 shutdown_connections 2024-08-02T17:54:36.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:35.777+0000 7f97c97d4640 1 -- 172.21.15.69:0/2668263066 wait complete. 2024-08-02T17:54:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:36 smithi083 ceph-mon[37031]: pgmap v252: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.3 MiB/s wr, 742 op/s 2024-08-02T17:54:37.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:36 smithi077 ceph-mon[37108]: pgmap v252: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.3 MiB/s wr, 742 op/s 2024-08-02T17:54:37.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:36 smithi069 ceph-mon[32746]: pgmap v252: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 6.3 MiB/s wr, 742 op/s 2024-08-02T17:54:37.450 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:54:37.450 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:54:39.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:38 smithi083 ceph-mon[37031]: pgmap v253: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 9.0 MiB/s wr, 1.00k op/s 2024-08-02T17:54:39.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:38 smithi077 ceph-mon[37108]: pgmap v253: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 9.0 MiB/s wr, 1.00k op/s 2024-08-02T17:54:39.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:38 smithi069 ceph-mon[32746]: pgmap v253: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 9.0 MiB/s wr, 1.00k op/s 2024-08-02T17:54:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 450.0 s, 488.2 tps, lat 194.317 ms stddev 318.488 2024-08-02T17:54:40.369 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0' 2024-08-02T17:54:40.370 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0 2024-08-02T17:54:40.627 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:41 smithi069 ceph-mon[32746]: pgmap v254: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.4 MiB/s wr, 776 op/s 2024-08-02T17:54:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:41 smithi083 ceph-mon[37031]: pgmap v254: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.4 MiB/s wr, 776 op/s 2024-08-02T17:54:41.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:41 smithi077 ceph-mon[37108]: pgmap v254: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.4 MiB/s wr, 776 op/s 2024-08-02T17:54:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.173+0000 7f59f8e6b640 1 -- 172.21.15.69:0/1044078452 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec040ca0 con 0x7f59f4105bf0 2024-08-02T17:54:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.174+0000 7f59fc0d1640 1 -- 172.21.15.69:0/1044078452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 msgr2=0x7f59f4106070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.174+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/1044078452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 0x7f59f4106070 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f59ec01acb0 tx=0x7f59ec03b900 comp rx=0 tx=0).stop 2024-08-02T17:54:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.174+0000 7f59fc0d1640 1 -- 172.21.15.69:0/1044078452 shutdown_connections 2024-08-02T17:54:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.174+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/1044078452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 0x7f59f4106070 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.174+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/1044078452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f4100f10 0x7f59f41056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.174+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/1044078452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f41005f0 0x7f59f41009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.174+0000 7f59fc0d1640 1 -- 172.21.15.69:0/1044078452 >> 172.21.15.69:0/1044078452 conn(0x7f59f40fc500 msgr2=0x7f59f40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:42.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.174+0000 7f59fc0d1640 1 -- 172.21.15.69:0/1044078452 shutdown_connections 2024-08-02T17:54:42.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.175+0000 7f59fc0d1640 1 -- 172.21.15.69:0/1044078452 wait complete. 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.175+0000 7f59fc0d1640 1 Processor -- start 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.175+0000 7f59fc0d1640 1 -- start start 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fc0d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f41005f0 0x7f59f4114b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fc0d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f4100f10 0x7f59f4110bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fc0d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 0x7f59f4111130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fc0d1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4109610 con 0x7f59f4105bf0 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fc0d1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4109490 con 0x7f59f41005f0 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fc0d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59f4109790 con 0x7f59f4100f10 2024-08-02T17:54:42.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fa66e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 0x7f59f4111130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:42.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fa66e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 0x7f59f4111130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58106/0 (socket says 172.21.15.69:58106) 2024-08-02T17:54:42.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59fa66e640 1 -- 172.21.15.69:0/3980437334 learned_addr learned my addr 172.21.15.69:0/3980437334 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:42.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.176+0000 7f59f9e6d640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f41005f0 0x7f59f4114b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:42.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.177+0000 7f59fa66e640 1 -- 172.21.15.69:0/3980437334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f4100f10 msgr2=0x7f59f4110bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:42.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.177+0000 7f59fa66e640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f4100f10 0x7f59f4110bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.177+0000 7f59fa66e640 1 -- 172.21.15.69:0/3980437334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f41005f0 msgr2=0x7f59f4114b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:42.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.177+0000 7f59fa66e640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f41005f0 0x7f59f4114b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.177+0000 7f59fa66e640 1 -- 172.21.15.69:0/3980437334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f4111860 con 0x7f59f4105bf0 2024-08-02T17:54:42.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.177+0000 7f59fa66e640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 0x7f59f4111130 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f59ec004af0 tx=0x7f59ec0406c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:42.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.178+0000 7f59eaffd640 1 -- 172.21.15.69:0/3980437334 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec045070 con 0x7f59f4105bf0 2024-08-02T17:54:42.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.178+0000 7f59eaffd640 1 -- 172.21.15.69:0/3980437334 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59ec0408a0 con 0x7f59f4105bf0 2024-08-02T17:54:42.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.178+0000 7f59eaffd640 1 -- 172.21.15.69:0/3980437334 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec004500 con 0x7f59f4105bf0 2024-08-02T17:54:42.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.180+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f41a2790 con 0x7f59f4105bf0 2024-08-02T17:54:42.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.180+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f41a2b30 con 0x7f59f4105bf0 2024-08-02T17:54:42.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.183+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f406a580 con 0x7f59f4105bf0 2024-08-02T17:54:42.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.184+0000 7f59eaffd640 1 -- 172.21.15.69:0/3980437334 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f59ec048020 con 0x7f59f4105bf0 2024-08-02T17:54:42.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.184+0000 7f59eaffd640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59e00a2ec0 0x7f59e00a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:42.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.185+0000 7f59eaffd640 1 -- 172.21.15.69:0/3980437334 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f59ec101220 con 0x7f59f4105bf0 2024-08-02T17:54:42.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.185+0000 7f59f9e6d640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59e00a2ec0 0x7f59e00a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:42.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.186+0000 7f59f9e6d640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59e00a2ec0 0x7f59e00a5380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f59f0006fd0 tx=0x7f59f00090e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:42.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.193+0000 7f59eaffd640 1 -- 172.21.15.69:0/3980437334 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f59ec059060 con 0x7f59f4105bf0 2024-08-02T17:54:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:42.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.575+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59f41a2d60 con 0x7f59f4105bf0 2024-08-02T17:54:42.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.578+0000 7f59eaffd640 1 -- 172.21.15.69:0/3980437334 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f59ec0cdb00 con 0x7f59f4105bf0 2024-08-02T17:54:42.580 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:54:42.580 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:54:42.580 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59e00a2ec0 msgr2=0x7f59e00a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59e00a2ec0 0x7f59e00a5380 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f59f0006fd0 tx=0x7f59f00090e0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 msgr2=0x7f59f4111130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 0x7f59f4111130 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f59ec004af0 tx=0x7f59ec0406c0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 shutdown_connections 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59f4105bf0 0x7f59f4111130 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59e00a2ec0 0x7f59e00a5380 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59f4100f10 0x7f59f4110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 --2- 172.21.15.69:0/3980437334 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59f41005f0 0x7f59f4114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 >> 172.21.15.69:0/3980437334 conn(0x7f59f40fc500 msgr2=0x7f59f4103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:42.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 shutdown_connections 2024-08-02T17:54:42.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:42.582+0000 7f59fc0d1640 1 -- 172.21.15.69:0/3980437334 wait complete. 2024-08-02T17:54:43.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.129+0000 7f8bc0016640 1 -- 172.21.15.69:0/2700002362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 msgr2=0x7f8bb810d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:43.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.129+0000 7f8bc0016640 1 --2- 172.21.15.69:0/2700002362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 0x7f8bb810d690 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009cd0 tx=0x7f8bb002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.135+0000 7f8bc0016640 1 -- 172.21.15.69:0/2700002362 shutdown_connections 2024-08-02T17:54:43.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.135+0000 7f8bc0016640 1 --2- 172.21.15.69:0/2700002362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 0x7f8bb810d690 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.135+0000 7f8bc0016640 1 --2- 172.21.15.69:0/2700002362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb81007c0 0x7f8bb8100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.135+0000 7f8bc0016640 1 --2- 172.21.15.69:0/2700002362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb81067c0 0x7f8bb8106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.135+0000 7f8bc0016640 1 -- 172.21.15.69:0/2700002362 >> 172.21.15.69:0/2700002362 conn(0x7f8bb80fc500 msgr2=0x7f8bb80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:43.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.143+0000 7f8bc0016640 1 -- 172.21.15.69:0/2700002362 shutdown_connections 2024-08-02T17:54:43.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.146+0000 7f8bc0016640 1 -- 172.21.15.69:0/2700002362 wait complete. 2024-08-02T17:54:43.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.147+0000 7f8bc0016640 1 Processor -- start 2024-08-02T17:54:43.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.147+0000 7f8bc0016640 1 -- start start 2024-08-02T17:54:43.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.148+0000 7f8bc0016640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb81007c0 0x7f8bb81161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.150+0000 7f8bc0016640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 0x7f8bb8110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bc0016640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb81067c0 0x7f8bb8110790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.153 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bc0016640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810ec90 con 0x7f8bb8101160 2024-08-02T17:54:43.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bc0016640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810eb10 con 0x7f8bb81067c0 2024-08-02T17:54:43.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bc0016640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb810ee10 con 0x7f8bb81007c0 2024-08-02T17:54:43.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bbe5b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb81067c0 0x7f8bb8110790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bbe5b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb81067c0 0x7f8bb8110790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59484/0 (socket says 172.21.15.69:59484) 2024-08-02T17:54:43.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/809866513 learned_addr learned my addr 172.21.15.69:0/809866513 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:43.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bbd5b1640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 0x7f8bb8110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bbd5b1640 1 -- 172.21.15.69:0/809866513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb81007c0 msgr2=0x7f8bb81161d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:54:43.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bbd5b1640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb81007c0 0x7f8bb81161d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bbd5b1640 1 -- 172.21.15.69:0/809866513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb81067c0 msgr2=0x7f8bb8110790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:43.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.152+0000 7f8bbd5b1640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb81067c0 0x7f8bb8110790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.153+0000 7f8bbd5b1640 1 -- 172.21.15.69:0/809866513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb8110ec0 con 0x7f8bb8101160 2024-08-02T17:54:43.155 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.153+0000 7f8bbd5b1640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 0x7f8bb8110250 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009990 tx=0x7f8ba8009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:43.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:43 smithi069 ceph-mon[32746]: pgmap v255: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.8 MiB/s wr, 789 op/s 2024-08-02T17:54:43.161 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:43 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3980437334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:43.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.159+0000 7f8baeffd640 1 -- 172.21.15.69:0/809866513 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8004020 con 0x7f8bb8101160 2024-08-02T17:54:43.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.159+0000 7f8baeffd640 1 -- 172.21.15.69:0/809866513 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba80041c0 con 0x7f8bb8101160 2024-08-02T17:54:43.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.159+0000 7f8baeffd640 1 -- 172.21.15.69:0/809866513 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8016410 con 0x7f8bb8101160 2024-08-02T17:54:43.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.159+0000 7f8bc0016640 1 -- 172.21.15.69:0/809866513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb81b3600 con 0x7f8bb8101160 2024-08-02T17:54:43.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.159+0000 7f8bc0016640 1 -- 172.21.15.69:0/809866513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81b3a00 con 0x7f8bb8101160 2024-08-02T17:54:43.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.163+0000 7f8bc2414740 1 Processor -- start 2024-08-02T17:54:43.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.171+0000 7f8bc2414740 1 -- start start 2024-08-02T17:54:43.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.171+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37130 0x55d3cab37510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.171+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab41dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.171+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab42310 0x55d3cab44700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.171+0000 7f8bc2414740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d3caa9ca70 con 0x55d3cab37a50 2024-08-02T17:54:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.171+0000 7f8bc2414740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3caa34780 con 0x55d3cab42310 2024-08-02T17:54:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.171+0000 7f8bc2414740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d3caa98a30 con 0x55d3cab37130 2024-08-02T17:54:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab41dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab41dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58132/0 (socket says 172.21.15.69:58132) 2024-08-02T17:54:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 -- 172.21.15.69:0/1492715482 learned_addr learned my addr 172.21.15.69:0/1492715482 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 -- 172.21.15.69:0/1492715482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37130 msgr2=0x55d3cab37510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:54:43.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/1492715482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37130 0x55d3cab37510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 -- 172.21.15.69:0/1492715482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab42310 msgr2=0x55d3cab44700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:54:43.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/1492715482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab42310 0x55d3cab44700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 -- 172.21.15.69:0/1492715482 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3ca80eb90 con 0x55d3cab37a50 2024-08-02T17:54:43.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.172+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/1492715482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab41dd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009b50 tx=0x7f8bb400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2bebb20a50a123 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:43.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.175+0000 7f8ba3fff640 1 -- 172.21.15.69:0/1492715482 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4009e40 con 0x55d3cab37a50 2024-08-02T17:54:43.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.176+0000 7f8baeffd640 1 -- 172.21.15.69:0/809866513 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8ba801d020 con 0x7f8bb8101160 2024-08-02T17:54:43.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.176+0000 7f8ba3fff640 1 -- 172.21.15.69:0/1492715482 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4005500 con 0x55d3cab37a50 2024-08-02T17:54:43.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.177+0000 7f8baeffd640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ba40a2f20 0x7f8ba40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.177+0000 7f8bc2414740 1 -- 172.21.15.69:0/1492715482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 msgr2=0x55d3cab41dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:43.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.177+0000 7f8bc2414740 1 --2- 172.21.15.69:0/1492715482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab41dd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009b50 tx=0x7f8bb400e720 comp rx=0 tx=0).stop 2024-08-02T17:54:43.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.177+0000 7f8baeffd640 1 -- 172.21.15.69:0/809866513 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f8ba80cf010 con 0x7f8bb8101160 2024-08-02T17:54:43.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ba40a2f20 0x7f8ba40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bc2414740 1 -- 172.21.15.69:0/1492715482 shutdown_connections 2024-08-02T17:54:43.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bc2414740 1 --2- 172.21.15.69:0/1492715482 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab41dd0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bc2414740 1 --2- 172.21.15.69:0/1492715482 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab42310 0x55d3cab44700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bc2414740 1 --2- 172.21.15.69:0/1492715482 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37130 0x55d3cab37510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bc2414740 1 -- 172.21.15.69:0/1492715482 >> 172.21.15.69:0/1492715482 conn(0x55d3cab336c0 msgr2=0x55d3cab33aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bc2414740 1 -- 172.21.15.69:0/1492715482 shutdown_connections 2024-08-02T17:54:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bc2414740 1 -- 172.21.15.69:0/1492715482 wait complete. 2024-08-02T17:54:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.180+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ba40a2f20 0x7f8ba40a53e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009a60 tx=0x7f8bb0005f80 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.181+0000 7f8bc2414740 1 -- start start 2024-08-02T17:54:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.181+0000 7f8bc2414740 1 Processor -- start 2024-08-02T17:54:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.182+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab3f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.182+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37a50 0x55d3cab3f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.182+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab42310 0x55d3cab3fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.182+0000 7f8bc2414740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d3cab44f30 con 0x55d3cab42310 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.182+0000 7f8bc2414740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3caa57990 con 0x55d3cab37130 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.182+0000 7f8bc2414740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d3cab450b0 con 0x55d3cab37a50 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.182+0000 7f8bc2414740 10 client.0 _ll_register_callbacks cb 0x55d3ca7ac090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.183+0000 7f8bbe5b3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab42310 0x55d3cab3fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.183+0000 7f8bbe5b3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab42310 0x55d3cab3fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58144/0 (socket says 172.21.15.69:58144) 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.183+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/2986959029 learned_addr learned my addr 172.21.15.69:0/2986959029 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.183+0000 7f8bbd5b1640 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37a50 0x55d3cab3f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.183+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab3f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.184+0000 7f8bbd5b1640 1 -- 172.21.15.69:0/2986959029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 msgr2=0x55d3cab3f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.184+0000 7f8bbd5b1640 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab3f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.184+0000 7f8bbd5b1640 1 -- 172.21.15.69:0/2986959029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab42310 msgr2=0x55d3cab3fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.184+0000 7f8bbd5b1640 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab42310 0x55d3cab3fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.184+0000 7f8bbd5b1640 1 -- 172.21.15.69:0/2986959029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3caa566c0 con 0x55d3cab37a50 2024-08-02T17:54:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.185+0000 7f8bbd5b1640 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37a50 0x55d3cab3f6b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80cc310 tx=0x7f8ba800d570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4bf2d62938b9bb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.186+0000 7f8ba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:54:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.186+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2986959029 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba809b0e0 con 0x55d3cab37a50 2024-08-02T17:54:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.186+0000 7f8bc2414740 1 -- 172.21.15.69:0/2986959029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb4005500 con 0x55d3cab37a50 2024-08-02T17:54:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.186+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2986959029 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f8ba809f120 con 0x55d3cab37a50 2024-08-02T17:54:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.187+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2986959029 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ba809f490 con 0x55d3cab37a50 2024-08-02T17:54:43.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.187+0000 7f8bc2414740 10 client.26806 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:43.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.187+0000 7f8bc2414740 1 -- 172.21.15.69:0/2986959029 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3cab50c40 con 0x55d3cab37a50 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.189+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2986959029 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4005500 con 0x55d3cab37a50 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.189+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2986959029 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8ba80a38c0 con 0x55d3cab37a50 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.190+0000 7f8bc2414740 10 client.26806 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.190+0000 7f8bc2414740 10 client.26806 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.190+0000 7f8bc2414740 20 client.26806 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.190+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3cab525f0 0x55d3cab54a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.190+0000 7f8bc2414740 4 client.26806 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.190+0000 7f8bc2414740 1 -- 172.21.15.69:0/2986959029 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3caaaa1a0 con 0x55d3cab525f0 2024-08-02T17:54:43.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.191+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3cab525f0 0x55d3cab54a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.199+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3cab525f0 0x55d3cab54a30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0b91ebbd055635 server_cookie=719d906bf874c001 in_seq=0 out_seq=0 2024-08-02T17:54:43.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.199+0000 7f8ba17fa640 10 client.26806 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:43.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.200+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2986959029 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d3caaaa1a0 con 0x55d3cab525f0 2024-08-02T17:54:43.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.200+0000 7f8ba17fa640 10 client.26806 handle_command_reply: tid=0 2024-08-02T17:54:43.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.202+0000 7f8bc2414740 1 client.26806 shutdown 2024-08-02T17:54:43.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.203+0000 7f8bc2414740 1 -- 172.21.15.69:0/2986959029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37a50 msgr2=0x55d3cab3f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:43.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.203+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37a50 0x55d3cab3f6b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80cc310 tx=0x7f8ba800d570 comp rx=0 tx=0).stop 2024-08-02T17:54:43.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.206+0000 7f8bc2414740 1 -- 172.21.15.69:0/2986959029 shutdown_connections 2024-08-02T17:54:43.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.206+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab42310 0x55d3cab3fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.206+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3cab525f0 0x55d3cab54a30 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.206+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab37a50 0x55d3cab3f6b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.206+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2986959029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab3f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.206+0000 7f8bc2414740 1 -- 172.21.15.69:0/2986959029 >> 172.21.15.69:0/2986959029 conn(0x55d3cab336c0 msgr2=0x55d3cab393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:43.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.207+0000 7f8ba17fa640 0 client.26806 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:43.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.209+0000 7f8bc2414740 1 -- 172.21.15.69:0/2986959029 shutdown_connections 2024-08-02T17:54:43.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.209+0000 7f8bc2414740 1 -- 172.21.15.69:0/2986959029 wait complete. 2024-08-02T17:54:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.210+0000 7f8bc2414740 20 client.26806 trim_cache size 0 max 16384 2024-08-02T17:54:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.250+0000 7f8bc2414740 1 Processor -- start 2024-08-02T17:54:43.251 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.250+0000 7f8bc2414740 1 -- start start 2024-08-02T17:54:43.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.250+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab3fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.251+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab5a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.251+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab42310 0x55d3cab5a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.251+0000 7f8bc2414740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c0129f0 con 0x55d3cab37a50 2024-08-02T17:54:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.251+0000 7f8bc2414740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c0102c0 con 0x55d3cab37130 2024-08-02T17:54:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.251+0000 7f8bbe5b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab42310 0x55d3cab5a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.251+0000 7f8bbe5b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab42310 0x55d3cab5a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:46036/0 (socket says 172.21.15.69:46036) 2024-08-02T17:54:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.251+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/4251368983 learned_addr learned my addr 172.21.15.69:0/4251368983 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.251+0000 7f8bc2414740 1 -- 172.21.15.69:0/4251368983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c00e110 con 0x55d3cab42310 2024-08-02T17:54:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.252+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/4251368983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 msgr2=0x55d3cab3fa40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:54:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.252+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/4251368983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab3fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.252+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/4251368983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 msgr2=0x55d3cab5a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.252+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/4251368983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab5a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.252+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/4251368983 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3cab37880 con 0x55d3cab42310 2024-08-02T17:54:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.252+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/4251368983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab42310 0x55d3cab5a880 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00318f0 tx=0x7f8bb003d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c009d0c708af83ff server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.253+0000 7f8ba37fe640 1 -- 172.21.15.69:0/4251368983 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0041070 con 0x55d3cab42310 2024-08-02T17:54:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.253+0000 7f8ba37fe640 1 -- 172.21.15.69:0/4251368983 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb0036040 con 0x55d3cab42310 2024-08-02T17:54:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.254+0000 7f8bc2414740 1 -- 172.21.15.69:0/4251368983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab42310 msgr2=0x55d3cab5a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.254+0000 7f8bc2414740 1 --2- 172.21.15.69:0/4251368983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab42310 0x55d3cab5a880 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00318f0 tx=0x7f8bb003d000 comp rx=0 tx=0).stop 2024-08-02T17:54:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.255+0000 7f8bc2414740 1 -- 172.21.15.69:0/4251368983 shutdown_connections 2024-08-02T17:54:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.255+0000 7f8bc2414740 1 --2- 172.21.15.69:0/4251368983 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cab37a50 0x55d3cab5a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.255+0000 7f8bc2414740 1 --2- 172.21.15.69:0/4251368983 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab42310 0x55d3cab5a880 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.255+0000 7f8bc2414740 1 --2- 172.21.15.69:0/4251368983 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab3fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.255+0000 7f8bc2414740 1 -- 172.21.15.69:0/4251368983 >> 172.21.15.69:0/4251368983 conn(0x55d3cab336c0 msgr2=0x55d3cab411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:43.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.260+0000 7f8bc2414740 1 -- 172.21.15.69:0/4251368983 shutdown_connections 2024-08-02T17:54:43.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.261+0000 7f8bc2414740 1 -- 172.21.15.69:0/4251368983 wait complete. 2024-08-02T17:54:43.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.261+0000 7f8bc2414740 1 -- start start 2024-08-02T17:54:43.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.261+0000 7f8bc2414740 1 Processor -- start 2024-08-02T17:54:43.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.262+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab46910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.262+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab46e50 0x55d3cabb62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.262+0000 7f8bc2414740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cabb6820 0x55d3cabb6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.262+0000 7f8bc2414740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c00e780 con 0x55d3cabb6820 2024-08-02T17:54:43.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.262+0000 7f8bc2414740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c00fbf0 con 0x55d3cab37130 2024-08-02T17:54:43.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.262+0000 7f8bc2414740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c00ee50 con 0x55d3cab46e50 2024-08-02T17:54:43.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.262+0000 7f8bc2414740 10 client.0 _ll_register_callbacks cb 0x7f8ba8096c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:43.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.262+0000 7f8bbd5b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab46e50 0x55d3cabb62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.263+0000 7f8bbd5b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab46e50 0x55d3cabb62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:46046/0 (socket says 172.21.15.69:46046) 2024-08-02T17:54:43.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.263+0000 7f8bbd5b1640 1 -- 172.21.15.69:0/2463731652 learned_addr learned my addr 172.21.15.69:0/2463731652 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:43.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.263+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab46910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.263+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cabb6820 0x55d3cabb6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.264+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/2463731652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab46e50 msgr2=0x55d3cabb62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:43.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.264+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab46e50 0x55d3cabb62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.264+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/2463731652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 msgr2=0x55d3cab46910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:43.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.264+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab46910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.264+0000 7f8bbe5b3640 1 -- 172.21.15.69:0/2463731652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3cab474e0 con 0x55d3cabb6820 2024-08-02T17:54:43.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.265+0000 7f8bbe5b3640 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cabb6820 0x55d3cabb6ca0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0006010 tx=0x7f8bb002ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424d45bd35072a3d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:43.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.265+0000 7f8ba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:43.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.265+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2463731652 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00307b0 con 0x55d3cabb6820 2024-08-02T17:54:43.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.266+0000 7f8bc2414740 1 -- 172.21.15.69:0/2463731652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d3caaa44f0 con 0x55d3cabb6820 2024-08-02T17:54:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.266+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2463731652 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f8bb0003fb0 con 0x55d3cabb6820 2024-08-02T17:54:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.266+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2463731652 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8bb0030950 con 0x55d3cabb6820 2024-08-02T17:54:43.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.267+0000 7f8bc2414740 10 client.17403 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.267+0000 7f8bc2414740 1 -- 172.21.15.69:0/2463731652 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3cabbe480 con 0x55d3cabb6820 2024-08-02T17:54:43.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.268+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2463731652 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d3caaa44f0 con 0x55d3cabb6820 2024-08-02T17:54:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.268+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2463731652 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8bb0061030 con 0x55d3cabb6820 2024-08-02T17:54:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.269+0000 7f8bc2414740 10 client.17403 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.269+0000 7f8bc2414740 10 client.17403 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:54:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.269+0000 7f8bc2414740 20 client.17403 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.269+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3cabbec60 0x55d3cabc1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:43.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.269+0000 7f8bc2414740 4 client.17403 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}] 2024-08-02T17:54:43.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.269+0000 7f8bc2414740 1 -- 172.21.15.69:0/2463731652 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}) -- 0x55d3caa20ec0 con 0x55d3cabbec60 2024-08-02T17:54:43.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:43 smithi083 ceph-mon[37031]: pgmap v255: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.8 MiB/s wr, 789 op/s 2024-08-02T17:54:43.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:43 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3980437334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:43.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.272+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3cabbec60 0x55d3cabc1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:43.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.275+0000 7f8bbddb2640 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3cabbec60 0x55d3cabc1080 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c454081dc074cae6 server_cookie=b07060a981f56fab in_seq=0 out_seq=0 2024-08-02T17:54:43.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.275+0000 7f8ba17fa640 10 client.17403 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:43.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:43 smithi077 ceph-mon[37108]: pgmap v255: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.8 MiB/s wr, 789 op/s 2024-08-02T17:54:43.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:43 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3980437334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:43.527 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:54:43.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8ba17fa640 1 -- 172.21.15.69:0/2463731652 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d3caa20ec0 con 0x55d3cabbec60 2024-08-02T17:54:43.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8ba17fa640 10 client.17403 handle_command_reply: tid=0 2024-08-02T17:54:43.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 client.17403 shutdown 2024-08-02T17:54:43.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 -- 172.21.15.69:0/2463731652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cabb6820 msgr2=0x55d3cabb6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:43.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cabb6820 0x55d3cabb6ca0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0006010 tx=0x7f8bb002ade0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 -- 172.21.15.69:0/2463731652 shutdown_connections 2024-08-02T17:54:43.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3cabb6820 0x55d3cabb6ca0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3cabbec60 0x55d3cabc1080 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3cab46e50 0x55d3cabb62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 --2- 172.21.15.69:0/2463731652 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3cab37130 0x55d3cab46910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 -- 172.21.15.69:0/2463731652 >> 172.21.15.69:0/2463731652 conn(0x55d3cab336c0 msgr2=0x55d3cab411f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8ba17fa640 0 client.17403 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 -- 172.21.15.69:0/2463731652 shutdown_connections 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 1 -- 172.21.15.69:0/2463731652 wait complete. 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.622+0000 7f8bc2414740 20 client.17403 trim_cache size 0 max 16384 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 26, 2024-08-02T17:54:43.627 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "81a0063a": { 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 26, 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.4, 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:54:43.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:54:43.629 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:54:43.629 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:43.629 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:43.629 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:43.629 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:43.629 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:43.629 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:54:43.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 -- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ba40a2f20 msgr2=0x7f8ba40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:43.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ba40a2f20 0x7f8ba40a53e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009a60 tx=0x7f8bb0005f80 comp rx=0 tx=0).stop 2024-08-02T17:54:43.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 -- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 msgr2=0x7f8bb8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:43.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 0x7f8bb8110250 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009990 tx=0x7f8ba8009e50 comp rx=0 tx=0).stop 2024-08-02T17:54:43.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 -- 172.21.15.69:0/809866513 shutdown_connections 2024-08-02T17:54:43.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8bb8101160 0x7f8bb8110250 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ba40a2f20 0x7f8ba40a53e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8bb81067c0 0x7f8bb8110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 --2- 172.21.15.69:0/809866513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8bb81007c0 0x7f8bb81161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:43.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 -- 172.21.15.69:0/809866513 >> 172.21.15.69:0/809866513 conn(0x7f8bb80fc500 msgr2=0x7f8bb80fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:43.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 -- 172.21.15.69:0/809866513 shutdown_connections 2024-08-02T17:54:43.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:43.623+0000 7f8bc0016640 1 -- 172.21.15.69:0/809866513 wait complete. 2024-08-02T17:54:44.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:44 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:54:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:44 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:54:44.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:44 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:54:44.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 455.0 s, 478.7 tps, lat 156.289 ms stddev 191.656 2024-08-02T17:54:45.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:45 smithi077 ceph-mon[37108]: pgmap v256: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.8 MiB/s wr, 908 op/s 2024-08-02T17:54:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:45 smithi069 ceph-mon[32746]: pgmap v256: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.8 MiB/s wr, 908 op/s 2024-08-02T17:54:45.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:45 smithi083 ceph-mon[37031]: pgmap v256: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.8 MiB/s wr, 908 op/s 2024-08-02T17:54:45.896 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '81a0063a', quiesce took 15.4 seconds. Will release after: 4.999999999999998 2024-08-02T17:54:45.948 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:46.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 -- 172.21.15.69:0/3812893740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701007b0 msgr2=0x7f0f70100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 --2- 172.21.15.69:0/3812893740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701007b0 0x7f0f70100c10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009930 tx=0x7f0f6002a420 comp rx=0 tx=0).stop 2024-08-02T17:54:46.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 -- 172.21.15.69:0/3812893740 shutdown_connections 2024-08-02T17:54:46.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 --2- 172.21.15.69:0/3812893740 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701007b0 0x7f0f70100c10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 --2- 172.21.15.69:0/3812893740 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f70101150 0x7f0f7010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 --2- 172.21.15.69:0/3812893740 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f701067b0 0x7f0f70106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 -- 172.21.15.69:0/3812893740 >> 172.21.15.69:0/3812893740 conn(0x7f0f700fc4d0 msgr2=0x7f0f700fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 -- 172.21.15.69:0/3812893740 shutdown_connections 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.470+0000 7f0f77720640 1 -- 172.21.15.69:0/3812893740 wait complete. 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f77720640 1 Processor -- start 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f77720640 1 -- start start 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f77720640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f701007b0 0x7f0f70198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f77720640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f70101150 0x7f0f701986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f77720640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701067b0 0x7f0f7019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f77720640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70111cd0 con 0x7f0f701067b0 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f77720640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70111b50 con 0x7f0f701007b0 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f77720640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70111e50 con 0x7f0f70101150 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f75cbd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701067b0 0x7f0f7019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f75cbd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701067b0 0x7f0f7019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50206/0 (socket says 172.21.15.69:50206) 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f75cbd640 1 -- 172.21.15.69:0/4001369083 learned_addr learned my addr 172.21.15.69:0/4001369083 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f70101150 0x7f0f701986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f754bc640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f701007b0 0x7f0f70198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f75cbd640 1 -- 172.21.15.69:0/4001369083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f70101150 msgr2=0x7f0f701986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f75cbd640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f70101150 0x7f0f701986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f75cbd640 1 -- 172.21.15.69:0/4001369083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f701007b0 msgr2=0x7f0f70198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f75cbd640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f701007b0 0x7f0f70198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f75cbd640 1 -- 172.21.15.69:0/4001369083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7019d140 con 0x7f0f701067b0 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.471+0000 7f0f754bc640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f701007b0 0x7f0f70198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.472+0000 7f0f75cbd640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701067b0 0x7f0f7019ca60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800cce0 tx=0x7f0f6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.472+0000 7f0f667fc640 1 -- 172.21.15.69:0/4001369083 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800f450 con 0x7f0f701067b0 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.472+0000 7f0f77720640 1 -- 172.21.15.69:0/4001369083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7019d430 con 0x7f0f701067b0 2024-08-02T17:54:46.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.472+0000 7f0f667fc640 1 -- 172.21.15.69:0/4001369083 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6800fad0 con 0x7f0f701067b0 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.472+0000 7f0f77720640 1 -- 172.21.15.69:0/4001369083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f701a4d10 con 0x7f0f701067b0 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.472+0000 7f0f667fc640 1 -- 172.21.15.69:0/4001369083 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f68018bb0 con 0x7f0f701067b0 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.474+0000 7f0f667fc640 1 -- 172.21.15.69:0/4001369083 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0f6800f5f0 con 0x7f0f701067b0 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f79b1e740 1 Processor -- start 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f79b1e740 1 -- start start 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 0x557236f3f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f49c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f4c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557236ea4a40 con 0x557236f3f910 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557236e3c9d0 con 0x557236f4a1d0 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557236ea0a00 con 0x557236f3eff0 2024-08-02T17:54:46.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f754bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f49c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f754bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f49c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50222/0 (socket says 172.21.15.69:50222) 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f754bc640 1 -- 172.21.15.69:0/1866039657 learned_addr learned my addr 172.21.15.69:0/1866039657 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f667fc640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f5c0a2cf0 0x7f0f5c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 0x557236f3f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.475+0000 7f0f75cbd640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f5c0a2cf0 0x7f0f5c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f4c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f754bc640 1 -- 172.21.15.69:0/1866039657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 msgr2=0x557236f3f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 0x557236f3f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f754bc640 1 -- 172.21.15.69:0/1866039657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 msgr2=0x557236f4c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f4c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f754bc640 1 -- 172.21.15.69:0/1866039657 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557236e43650 con 0x557236f3f910 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f667fc640 1 -- 172.21.15.69:0/4001369083 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f0f680cfd70 con 0x7f0f701067b0 2024-08-02T17:54:46.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f4c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f49c90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00aa10 tx=0x7f0f6c00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2482e3530829d7be server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f75cbd640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f5c0a2cf0 0x7f0f5c0a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f68098810 tx=0x7f0f6806eeb0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f5b7fe640 1 -- 172.21.15.69:0/1866039657 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c015070 con 0x557236f3f910 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f5b7fe640 1 -- 172.21.15.69:0/1866039657 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6c00cc60 con 0x557236f3f910 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.476+0000 7f0f5b7fe640 1 -- 172.21.15.69:0/1866039657 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c0106c0 con 0x557236f3f910 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1866039657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 msgr2=0x557236f49c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f49c90 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00aa10 tx=0x7f0f6c00aed0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1866039657 shutdown_connections 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f49c90 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f4c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1866039657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 0x557236f3f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1866039657 >> 172.21.15.69:0/1866039657 conn(0x557236f3b580 msgr2=0x557236f3b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1866039657 shutdown_connections 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1866039657 wait complete. 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 -- start start 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.477+0000 7f0f79b1e740 1 Processor -- start 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f49c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3f910 0x557236f46fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f47770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557236f4cda0 con 0x557236f3eff0 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557236e5fb20 con 0x557236f4a1d0 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557236f4cf20 con 0x557236f3f910 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f79b1e740 10 client.0 _ll_register_callbacks cb 0x557236e5dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f49c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f49c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50232/0 (socket says 172.21.15.69:50232) 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 -- 172.21.15.69:0/1551690853 learned_addr learned my addr 172.21.15.69:0/1551690853 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:46.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f47770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3f910 0x557236f46fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 -- 172.21.15.69:0/1551690853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3f910 msgr2=0x557236f46fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3f910 0x557236f46fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 -- 172.21.15.69:0/1551690853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 msgr2=0x557236f47770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f47770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 -- 172.21.15.69:0/1551690853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557236c163c0 con 0x557236f3eff0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f49c90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005940 tx=0x7f0f60005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15034f7ec4a390a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f58ff9640 1 -- 172.21.15.69:0/1551690853 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60035040 con 0x557236f3eff0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f58ff9640 1 -- 172.21.15.69:0/1551690853 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f0f60005220 con 0x557236f3eff0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.478+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1551690853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6c00cc60 con 0x557236f3eff0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f58ff9640 1 -- 172.21.15.69:0/1551690853 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f60050050 con 0x557236f3eff0 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f79b1e740 10 client.17415 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:46.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1551690853 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557236f58840 con 0x557236f3eff0 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f58ff9640 1 -- 172.21.15.69:0/1551690853 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6c00cc60 con 0x557236f3eff0 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f58ff9640 1 -- 172.21.15.69:0/1551690853 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f60048020 con 0x557236f3eff0 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f79b1e740 10 client.17415 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f79b1e740 10 client.17415 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f79b1e740 20 client.17415 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557236f59d00 0x557236f5c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f79b1e740 4 client.17415 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1551690853 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557236eb1fb0 con 0x557236f59d00 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.479+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557236f59d00 0x557236f5c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.480+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557236f59d00 0x557236f5c0f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4051b11fe5419a2d server_cookie=fea2f5617eafaa03 in_seq=0 out_seq=0 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.480+0000 7f0f58ff9640 10 client.17415 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:54:46.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.480+0000 7f0f58ff9640 1 -- 172.21.15.69:0/1551690853 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557236eb1fb0 con 0x557236f59d00 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.480+0000 7f0f58ff9640 10 client.17415 handle_command_reply: tid=0 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.480+0000 7f0f79b1e740 1 client.17415 shutdown 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1551690853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 msgr2=0x557236f49c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f49c90 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005940 tx=0x7f0f60005a20 comp rx=0 tx=0).stop 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1551690853 shutdown_connections 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557236f59d00 0x557236f5c0f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f47770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3f910 0x557236f46fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f58ff9640 0 client.17415 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1551690853 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f49c90 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1551690853 >> 172.21.15.69:0/1551690853 conn(0x557236f3b580 msgr2=0x557236f41260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1551690853 shutdown_connections 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1551690853 wait complete. 2024-08-02T17:54:46.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.481+0000 7f0f79b1e740 20 client.17415 trim_cache size 0 max 16384 2024-08-02T17:54:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f79b1e740 1 Processor -- start 2024-08-02T17:54:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f79b1e740 1 -- start start 2024-08-02T17:54:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 0x557236f42270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f427b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.504 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701b16d0 con 0x557236f3f910 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701aefa0 con 0x557236f4a1d0 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701acdf0 con 0x557236f3eff0 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f427b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50238/0 (socket says 172.21.15.69:50238) 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 -- 172.21.15.69:0/1372056694 learned_addr learned my addr 172.21.15.69:0/1372056694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 -- 172.21.15.69:0/1372056694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 msgr2=0x557236f42270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1372056694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 0x557236f42270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 -- 172.21.15.69:0/1372056694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 msgr2=0x557236f458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1372056694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 -- 172.21.15.69:0/1372056694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557236f47ae0 con 0x557236f3f910 2024-08-02T17:54:46.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.494+0000 7f0f754bc640 1 --2- 172.21.15.69:0/1372056694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f427b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00c520 tx=0x7f0f6c00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe966e989229490 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f5affd640 1 -- 172.21.15.69:0/1372056694 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c015040 con 0x557236f3f910 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f5affd640 1 -- 172.21.15.69:0/1372056694 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6c00a5e0 con 0x557236f3f910 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f5affd640 1 -- 172.21.15.69:0/1372056694 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c01a7b0 con 0x557236f3f910 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1372056694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 msgr2=0x557236f427b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1372056694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f427b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00c520 tx=0x7f0f6c00a2d0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1372056694 shutdown_connections 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1372056694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3f910 0x557236f427b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1372056694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f4a1d0 0x557236f458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/1372056694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f3eff0 0x557236f42270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1372056694 >> 172.21.15.69:0/1372056694 conn(0x557236f3b580 msgr2=0x557236f361b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1372056694 shutdown_connections 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.495+0000 7f0f79b1e740 1 -- 172.21.15.69:0/1372056694 wait complete. 2024-08-02T17:54:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 1 -- start start 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 1 Processor -- start 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f4a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f3f910 0x557236f4aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f4a1d0 0x557236f51170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701ad460 con 0x557236f3eff0 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701ae8d0 con 0x557236f3f910 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701adb30 con 0x557236f4a1d0 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f79b1e740 10 client.0 _ll_register_callbacks cb 0x7f0f60009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f74cbb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f4a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f74cbb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f4a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50250/0 (socket says 172.21.15.69:50250) 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f74cbb640 1 -- 172.21.15.69:0/2304048414 learned_addr learned my addr 172.21.15.69:0/2304048414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.496+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f4a1d0 0x557236f51170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f754bc640 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f3f910 0x557236f4aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f74cbb640 1 -- 172.21.15.69:0/2304048414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f4a1d0 msgr2=0x557236f51170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f4a1d0 0x557236f51170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f74cbb640 1 -- 172.21.15.69:0/2304048414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f3f910 msgr2=0x557236f4aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f3f910 0x557236f4aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f74cbb640 1 -- 172.21.15.69:0/2304048414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557236f516b0 con 0x557236f3eff0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f74cbb640 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f4a970 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005ab0 tx=0x7f0f60005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31de2f561c73674f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f58ff9640 1 -- 172.21.15.69:0/2304048414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f60035040 con 0x557236f3eff0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f58ff9640 1 -- 172.21.15.69:0/2304048414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f0f600053c0 con 0x557236f3eff0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 1 -- 172.21.15.69:0/2304048414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f6c00a5e0 con 0x557236f3eff0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f58ff9640 1 -- 172.21.15.69:0/2304048414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f60004420 con 0x557236f3eff0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 10 client.17421 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 1 -- 172.21.15.69:0/2304048414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557236f52970 con 0x557236f3eff0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f58ff9640 1 -- 172.21.15.69:0/2304048414 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6c00a5e0 con 0x557236f3eff0 2024-08-02T17:54:46.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f58ff9640 1 -- 172.21.15.69:0/2304048414 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f60048050 con 0x557236f3eff0 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 10 client.17421 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 10 client.17421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 20 client.17421 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557236fc4970 0x557236fc6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 4 client.17421 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.497+0000 7f0f79b1e740 1 -- 172.21.15.69:0/2304048414 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557236eac4c0 con 0x557236fc4970 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.498+0000 7f0f754bc640 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557236fc4970 0x557236fc6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.498+0000 7f0f58ff9640 10 client.17421 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.498+0000 7f0f754bc640 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557236fc4970 0x557236fc6d60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ecb8b0636fa9ac37 server_cookie=620bd5306584dcb0 in_seq=0 out_seq=0 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f58ff9640 1 -- 172.21.15.69:0/2304048414 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557236eac4c0 con 0x557236fc4970 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f58ff9640 10 client.17421 handle_command_reply: tid=0 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 client.17421 shutdown 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 -- 172.21.15.69:0/2304048414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 msgr2=0x557236f4a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f4a970 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0f60005ab0 tx=0x7f0f60005a20 comp rx=0 tx=0).stop 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 -- 172.21.15.69:0/2304048414 shutdown_connections 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557236fc4970 0x557236fc6d60 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557236f4a1d0 0x557236f51170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557236f3f910 0x557236f4aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f58ff9640 0 client.17421 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 --2- 172.21.15.69:0/2304048414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557236f3eff0 0x557236f4a970 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 -- 172.21.15.69:0/2304048414 >> 172.21.15.69:0/2304048414 conn(0x557236f3b580 msgr2=0x557236f42270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 -- 172.21.15.69:0/2304048414 shutdown_connections 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 1 -- 172.21.15.69:0/2304048414 wait complete. 2024-08-02T17:54:46.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f79b1e740 20 client.17421 trim_cache size 0 max 16384 2024-08-02T17:54:46.511 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:54:46.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 -- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f5c0a2cf0 msgr2=0x7f0f5c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f5c0a2cf0 0x7f0f5c0a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f68098810 tx=0x7f0f6806eeb0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 -- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701067b0 msgr2=0x7f0f7019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701067b0 0x7f0f7019ca60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800cce0 tx=0x7f0f6800a620 comp rx=0 tx=0).stop 2024-08-02T17:54:46.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 -- 172.21.15.69:0/4001369083 shutdown_connections 2024-08-02T17:54:46.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0f701067b0 0x7f0f7019ca60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f5c0a2cf0 0x7f0f5c0a51b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0f70101150 0x7f0f701986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 --2- 172.21.15.69:0/4001369083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0f701007b0 0x7f0f70198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.499+0000 7f0f77720640 1 -- 172.21.15.69:0/4001369083 >> 172.21.15.69:0/4001369083 conn(0x7f0f700fc4d0 msgr2=0x7f0f700fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.500+0000 7f0f77720640 1 -- 172.21.15.69:0/4001369083 shutdown_connections 2024-08-02T17:54:46.512 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:46.500+0000 7f0f77720640 1 -- 172.21.15.69:0/4001369083 wait complete. 2024-08-02T17:54:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 -- 172.21.15.69:0/1359942626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 msgr2=0x7fd9d0106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 --2- 172.21.15.69:0/1359942626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 0x7fd9d0106ac0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009930 tx=0x7fd9c002a420 comp rx=0 tx=0).stop 2024-08-02T17:54:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 -- 172.21.15.69:0/1359942626 shutdown_connections 2024-08-02T17:54:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 --2- 172.21.15.69:0/1359942626 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d0101070 0x7fd9d010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 --2- 172.21.15.69:0/1359942626 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d01006d0 0x7fd9d0100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 --2- 172.21.15.69:0/1359942626 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 0x7fd9d0106ac0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 -- 172.21.15.69:0/1359942626 >> 172.21.15.69:0/1359942626 conn(0x7fd9d00776d0 msgr2=0x7fd9d00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 -- 172.21.15.69:0/1359942626 shutdown_connections 2024-08-02T17:54:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.973+0000 7fd9d5729640 1 -- 172.21.15.69:0/1359942626 wait complete. 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.974+0000 7fd9d5729640 1 Processor -- start 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9d5729640 1 -- start start 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9d5729640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d01006d0 0x7fd9d0198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9d5729640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d0101070 0x7fd9d01986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9d5729640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 0x7fd9d019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9d5729640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0111c10 con 0x7fd9d01066e0 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9d5729640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0111a90 con 0x7fd9d01006d0 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9d5729640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0111d90 con 0x7fd9d0101070 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 0x7fd9d019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9cf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 0x7fd9d019ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50274/0 (socket says 172.21.15.69:50274) 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9cf7fe640 1 -- 172.21.15.69:0/3905571837 learned_addr learned my addr 172.21.15.69:0/3905571837 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d01006d0 0x7fd9d0198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d0101070 0x7fd9d01986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9cf7fe640 1 -- 172.21.15.69:0/3905571837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d0101070 msgr2=0x7fd9d01986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9cf7fe640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d0101070 0x7fd9d01986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9cf7fe640 1 -- 172.21.15.69:0/3905571837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d01006d0 msgr2=0x7fd9d0198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9cf7fe640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d01006d0 0x7fd9d0198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9cf7fe640 1 -- 172.21.15.69:0/3905571837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d019d150 con 0x7fd9d01066e0 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.975+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d0101070 0x7fd9d01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.976+0000 7fd9cf7fe640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 0x7fd9d019ca20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800eaa0 tx=0x7fd9c800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.976+0000 7fd9b7fff640 1 -- 172.21.15.69:0/3905571837 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800cdb0 con 0x7fd9d01066e0 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.976+0000 7fd9b7fff640 1 -- 172.21.15.69:0/3905571837 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c8005020 con 0x7fd9d01066e0 2024-08-02T17:54:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.976+0000 7fd9b7fff640 1 -- 172.21.15.69:0/3905571837 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80116b0 con 0x7fd9d01066e0 2024-08-02T17:54:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.976+0000 7fd9d5729640 1 -- 172.21.15.69:0/3905571837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d019d440 con 0x7fd9d01066e0 2024-08-02T17:54:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.976+0000 7fd9d5729640 1 -- 172.21.15.69:0/3905571837 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d01a4cd0 con 0x7fd9d01066e0 2024-08-02T17:54:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.978+0000 7fd9b7fff640 1 -- 172.21.15.69:0/3905571837 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd9c8005560 con 0x7fd9d01066e0 2024-08-02T17:54:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9d8482740 1 Processor -- start 2024-08-02T17:54:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9d8482740 1 -- start start 2024-08-02T17:54:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4bad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9d8482740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e0ec4119b0 con 0x55e0ec4b8970 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9d8482740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e0ec3b9800 con 0x55e0ec4ae0d0 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9d8482740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e0ec40d970 con 0x55e0ec4ad7b0 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9b7fff640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd9b80a2cf0 0x7fd9b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4bad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4bad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50286/0 (socket says 172.21.15.69:50286) 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/2854679097 learned_addr learned my addr 172.21.15.69:0/2854679097 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9cf7fe640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd9b80a2cf0 0x7fd9b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9b7fff640 1 -- 172.21.15.69:0/3905571837 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fd9c80cfa70 con 0x7fd9d01066e0 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/2854679097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 msgr2=0x55e0ec4adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/2854679097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 msgr2=0x55e0ec4b8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/2854679097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0ec125210 con 0x55e0ec4b8970 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.979+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9cf7fe640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd9b80a2cf0 0x7fd9b80a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8098550 tx=0x7fd9c80054d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4bad60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4012470 tx=0x7fd9c400b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdc8c92c9b1f4da server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9b57fa640 1 -- 172.21.15.69:0/2854679097 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4018070 con 0x55e0ec4b8970 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9b57fa640 1 -- 172.21.15.69:0/2854679097 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c40047a0 con 0x55e0ec4b8970 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9b57fa640 1 -- 172.21.15.69:0/2854679097 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4010670 con 0x55e0ec4b8970 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9d8482740 1 -- 172.21.15.69:0/2854679097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 msgr2=0x55e0ec4bad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9d8482740 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4bad60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4012470 tx=0x7fd9c400b600 comp rx=0 tx=0).stop 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9d8482740 1 -- 172.21.15.69:0/2854679097 shutdown_connections 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9d8482740 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4bad60 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9d8482740 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9d8482740 1 --2- 172.21.15.69:0/2854679097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.980+0000 7fd9d8482740 1 -- 172.21.15.69:0/2854679097 >> 172.21.15.69:0/2854679097 conn(0x55e0ec4a9d20 msgr2=0x55e0ec4aa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 -- 172.21.15.69:0/2854679097 shutdown_connections 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 -- 172.21.15.69:0/2854679097 wait complete. 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 -- start start 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 Processor -- start 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e0ec4bb540 con 0x55e0ec4b8970 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e0ec3cd500 con 0x55e0ec4ae0d0 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e0ec4bb6c0 con 0x55e0ec4ad7b0 2024-08-02T17:54:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9d8482740 10 client.0 _ll_register_callbacks cb 0x55e0ec124a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50298/0 (socket says 172.21.15.69:50298) 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/683929690 learned_addr learned my addr 172.21.15.69:0/683929690 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.981+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/683929690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 msgr2=0x55e0ec4b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4b58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 msgr2=0x55e0ec4b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/683929690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0ec1f1620 con 0x55e0ec4b8970 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6320 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40058d0 tx=0x7fd9c4012f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae534c37043a507 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/683929690 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4018040 con 0x55e0ec4b8970 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 1 -- 172.21.15.69:0/683929690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c40047a0 con 0x55e0ec4b8970 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/683929690 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fd9c4004020 con 0x55e0ec4b8970 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/683929690 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9c4004220 con 0x55e0ec4b8970 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 10 client.17433 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 1 -- 172.21.15.69:0/683929690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e0ec4c71e0 con 0x55e0ec4b8970 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/683929690 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c40047a0 con 0x55e0ec4b8970 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/683929690 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd9c400f040 con 0x55e0ec4b8970 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 10 client.17433 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 10 client.17433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 20 client.17433 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:46.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0ec4c8650 0x55e0ec4caa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 4 client.17433 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.982+0000 7fd9d8482740 1 -- 172.21.15.69:0/683929690 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0ec41f0e0 con 0x55e0ec4c8650 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.983+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0ec4c8650 0x55e0ec4caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.983+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0ec4c8650 0x55e0ec4caa40 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8633c461c876fde2 server_cookie=418e7c7edce601e8 in_seq=0 out_seq=0 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.983+0000 7fd9b2ff5640 10 client.17433 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/683929690 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e0ec41f0e0 con 0x55e0ec4c8650 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9b2ff5640 10 client.17433 handle_command_reply: tid=0 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 client.17433 shutdown 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 -- 172.21.15.69:0/683929690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 msgr2=0x55e0ec4b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6320 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40058d0 tx=0x7fd9c4012f70 comp rx=0 tx=0).stop 2024-08-02T17:54:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 -- 172.21.15.69:0/683929690 shutdown_connections 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6320 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0ec4c8650 0x55e0ec4caa40 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 --2- 172.21.15.69:0/683929690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9b2ff5640 0 client.17433 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 -- 172.21.15.69:0/683929690 >> 172.21.15.69:0/683929690 conn(0x55e0ec4a9d20 msgr2=0x55e0ec4afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 -- 172.21.15.69:0/683929690 shutdown_connections 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 1 -- 172.21.15.69:0/683929690 wait complete. 2024-08-02T17:54:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.984+0000 7fd9d8482740 20 client.17433 trim_cache size 0 max 16384 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9d8482740 1 Processor -- start 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9d8482740 1 -- start start 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4c0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4c1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9d8482740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d01b1890 con 0x55e0ec4b8970 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9d8482740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d01af160 con 0x55e0ec4ad7b0 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9d8482740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d01acfb0 con 0x55e0ec4ae0d0 2024-08-02T17:54:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50302/0 (socket says 172.21.15.69:50302) 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:46.999+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/2132821889 learned_addr learned my addr 172.21.15.69:0/2132821889 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4c0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/2132821889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 msgr2=0x55e0ec4c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4c1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/2132821889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 msgr2=0x55e0ec4c0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4c0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/2132821889 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0ec4b5670 con 0x55e0ec4b8970 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6ff0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd9c401c620 tx=0x7fd9c4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91164524d5d44c10 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9b4ff9640 1 -- 172.21.15.69:0/2132821889 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4018070 con 0x55e0ec4b8970 2024-08-02T17:54:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9b4ff9640 1 -- 172.21.15.69:0/2132821889 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c40043c0 con 0x55e0ec4b8970 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.000+0000 7fd9b4ff9640 1 -- 172.21.15.69:0/2132821889 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4004780 con 0x55e0ec4b8970 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- 172.21.15.69:0/2132821889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 msgr2=0x55e0ec4b6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6ff0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd9c401c620 tx=0x7fd9c4014040 comp rx=0 tx=0).stop 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- 172.21.15.69:0/2132821889 shutdown_connections 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b6ff0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 --2- 172.21.15.69:0/2132821889 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4c0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- 172.21.15.69:0/2132821889 >> 172.21.15.69:0/2132821889 conn(0x55e0ec4a9d20 msgr2=0x55e0ec4b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- 172.21.15.69:0/2132821889 shutdown_connections 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- 172.21.15.69:0/2132821889 wait complete. 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- start start 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 Processor -- start 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d01ad620 con 0x55e0ec4b8970 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d01aea90 con 0x55e0ec4ad7b0 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d01adcf0 con 0x55e0ec4ae0d0 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.001+0000 7fd9d8482740 10 client.0 _ll_register_callbacks cb 0x7fd9c400bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50308/0 (socket says 172.21.15.69:50308) 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/4064215921 learned_addr learned my addr 172.21.15.69:0/4064215921 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/4064215921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 msgr2=0x55e0ec4b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/4064215921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 msgr2=0x55e0ec4b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 -- 172.21.15.69:0/4064215921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0ec4b27c0 con 0x55e0ec4b8970 2024-08-02T17:54:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9ce7fc640 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b21d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x55e0ec4d0ec0 tx=0x7fd9c4004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7286a56781918423 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/4064215921 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4018030 con 0x55e0ec4b8970 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/4064215921 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fd9c4005470 con 0x55e0ec4b8970 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9d8482740 1 -- 172.21.15.69:0/4064215921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9c40043c0 con 0x55e0ec4b8970 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/4064215921 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9c404b050 con 0x55e0ec4b8970 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9d8482740 10 client.17439 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9d8482740 1 -- 172.21.15.69:0/4064215921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e0ec536e00 con 0x55e0ec4b8970 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.002+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/4064215921 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9c40043c0 con 0x55e0ec4b8970 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/4064215921 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd9c4042720 con 0x55e0ec4b8970 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9d8482740 10 client.17439 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9d8482740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9d8482740 20 client.17439 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:47.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9d8482740 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0ec5373c0 0x55e0ec5397b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9d8482740 4 client.17439 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:54:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9d8482740 1 -- 172.21.15.69:0/4064215921 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0ec419430 con 0x55e0ec5373c0 2024-08-02T17:54:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0ec5373c0 0x55e0ec5397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9b2ff5640 10 client.17439 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.003+0000 7fd9ceffd640 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0ec5373c0 0x55e0ec5397b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36fbc200b93867f server_cookie=aed24e84a6fb4050 in_seq=0 out_seq=0 2024-08-02T17:54:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9b2ff5640 1 -- 172.21.15.69:0/4064215921 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0ec419430 con 0x55e0ec5373c0 2024-08-02T17:54:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9b2ff5640 10 client.17439 handle_command_reply: tid=0 2024-08-02T17:54:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 client.17439 shutdown 2024-08-02T17:54:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 -- 172.21.15.69:0/4064215921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 msgr2=0x55e0ec4b21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b21d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x55e0ec4d0ec0 tx=0x7fd9c4004680 comp rx=0 tx=0).stop 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 -- 172.21.15.69:0/4064215921 shutdown_connections 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e0ec4b8970 0x55e0ec4b21d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e0ec5373c0 0x55e0ec5397b0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e0ec4ae0d0 0x55e0ec4b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9b2ff5640 0 client.17439 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 --2- 172.21.15.69:0/4064215921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e0ec4ad7b0 0x55e0ec4b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.023+0000 7fd9d8482740 1 -- 172.21.15.69:0/4064215921 >> 172.21.15.69:0/4064215921 conn(0x55e0ec4a9d20 msgr2=0x55e0ec4b5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d8482740 1 -- 172.21.15.69:0/4064215921 shutdown_connections 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d8482740 1 -- 172.21.15.69:0/4064215921 wait complete. 2024-08-02T17:54:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d8482740 20 client.17439 trim_cache size 0 max 16384 2024-08-02T17:54:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:54:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:54:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:54:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 -- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd9b80a2cf0 msgr2=0x7fd9b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd9b80a2cf0 0x7fd9b80a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8098550 tx=0x7fd9c80054d0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 -- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 msgr2=0x7fd9d019ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 0x7fd9d019ca20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800eaa0 tx=0x7fd9c800ef60 comp rx=0 tx=0).stop 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 -- 172.21.15.69:0/3905571837 shutdown_connections 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9d01066e0 0x7fd9d019ca20 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd9b80a2cf0 0x7fd9b80a51b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd9d0101070 0x7fd9d01986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 --2- 172.21.15.69:0/3905571837 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd9d01006d0 0x7fd9d0198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.024+0000 7fd9d5729640 1 -- 172.21.15.69:0/3905571837 >> 172.21.15.69:0/3905571837 conn(0x7fd9d00776d0 msgr2=0x7fd9d00feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.025+0000 7fd9d5729640 1 -- 172.21.15.69:0/3905571837 shutdown_connections 2024-08-02T17:54:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:54:47.025+0000 7fd9d5729640 1 -- 172.21.15.69:0/3905571837 wait complete. 2024-08-02T17:54:47.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:47 smithi069 ceph-mon[32746]: pgmap v257: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-08-02T17:54:47.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:47 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:54:47.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:47 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:54:47.312 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:47 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:54:47.338 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:54:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:47 smithi077 ceph-mon[37108]: pgmap v257: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-08-02T17:54:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:47 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:54:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:47 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:54:47.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:47 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:54:47.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:47 smithi083 ceph-mon[37031]: pgmap v257: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.5 MiB/s wr, 582 op/s 2024-08-02T17:54:47.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:47 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:54:47.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:47 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:54:47.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:47 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:54:47.593 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:48.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/4281291649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941018b0 msgr2=0x7f9d94101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/4281291649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941018b0 0x7f9d94101d10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9d84009930 tx=0x7f9d8402a420 comp rx=0 tx=0).stop 2024-08-02T17:54:48.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/4281291649 shutdown_connections 2024-08-02T17:54:48.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/4281291649 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941018b0 0x7f9d94101d10 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/4281291649 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d94102250 0x7f9d9410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/4281291649 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d941078b0 0x7f9d94107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/4281291649 >> 172.21.15.69:0/4281291649 conn(0x7f9d940fd5f0 msgr2=0x7f9d940ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/4281291649 shutdown_connections 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.071+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/4281291649 wait complete. 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9b2d0640 1 Processor -- start 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9b2d0640 1 -- start start 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9b2d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d941018b0 0x7f9d941981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9b2d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d94102250 0x7f9d94198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9b2d0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941078b0 0x7f9d9419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9b2d0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111c70 con 0x7f9d941078b0 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9b2d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111af0 con 0x7f9d941018b0 2024-08-02T17:54:48.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9b2d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111df0 con 0x7f9d94102250 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9986d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941078b0 0x7f9d9419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9986d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941078b0 0x7f9d9419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50328/0 (socket says 172.21.15.69:50328) 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9986d640 1 -- 172.21.15.69:0/3311186920 learned_addr learned my addr 172.21.15.69:0/3311186920 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9906c640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d941018b0 0x7f9d941981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9986d640 1 -- 172.21.15.69:0/3311186920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d94102250 msgr2=0x7f9d94198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9986d640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d94102250 0x7f9d94198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9986d640 1 -- 172.21.15.69:0/3311186920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d941018b0 msgr2=0x7f9d941981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9986d640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d941018b0 0x7f9d941981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9986d640 1 -- 172.21.15.69:0/3311186920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9419d180 con 0x7f9d941078b0 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.072+0000 7f9d9906c640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d941018b0 0x7f9d941981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.073+0000 7f9d9986d640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941078b0 0x7f9d9419caa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00eb10 tx=0x7f9d8c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.073+0000 7f9d8a7fc640 1 -- 172.21.15.69:0/3311186920 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c004b20 con 0x7f9d941078b0 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.073+0000 7f9d8a7fc640 1 -- 172.21.15.69:0/3311186920 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d8c004cc0 con 0x7f9d941078b0 2024-08-02T17:54:48.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.073+0000 7f9d8a7fc640 1 -- 172.21.15.69:0/3311186920 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c010760 con 0x7f9d941078b0 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.073+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/3311186920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9419d470 con 0x7f9d941078b0 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.073+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/3311186920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941a4d50 con 0x7f9d941078b0 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.075+0000 7f9d8a7fc640 1 -- 172.21.15.69:0/3311186920 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9d8c005020 con 0x7f9d941078b0 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.075+0000 7f9d9d6ce740 1 Processor -- start 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9d6ce740 1 -- start start 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0d20 0x55fbde5ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fbde501a40 con 0x55fbde5a0400 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fbde4999d0 con 0x55fbde5a0d20 2024-08-02T17:54:48.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbde4fda00 con 0x55fbde5ab5e0 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50340/0 (socket says 172.21.15.69:50340) 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 -- 172.21.15.69:0/3878057651 learned_addr learned my addr 172.21.15.69:0/3878057651 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d8a7fc640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d800a2d70 0x7f9d800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9986d640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d800a2d70 0x7f9d800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9906c640 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0d20 0x55fbde5ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 -- 172.21.15.69:0/3878057651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 msgr2=0x55fbde5ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d8a7fc640 1 -- 172.21.15.69:0/3311186920 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f9d8c0cff00 con 0x7f9d941078b0 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 -- 172.21.15.69:0/3878057651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0d20 msgr2=0x55fbde5ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0d20 0x55fbde5ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 -- 172.21.15.69:0/3878057651 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbde4a0650 con 0x55fbde5a0400 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.076+0000 7f9d9886b640 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a07e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9d84031550 tx=0x7f9d840314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae7a73431874476 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d7f7fe640 1 -- 172.21.15.69:0/3878057651 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84035040 con 0x55fbde5a0400 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d9986d640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d800a2d70 0x7f9d800a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0a4750 tx=0x7f9d8c00efd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d7f7fe640 1 -- 172.21.15.69:0/3878057651 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d84044070 con 0x55fbde5a0400 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d7f7fe640 1 -- 172.21.15.69:0/3878057651 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8403f410 con 0x55fbde5a0400 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3878057651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 msgr2=0x55fbde5a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a07e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f9d84031550 tx=0x7f9d840314a0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3878057651 shutdown_connections 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0d20 0x55fbde5ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3878057651 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a07e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.077+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3878057651 >> 172.21.15.69:0/3878057651 conn(0x55fbde59c990 msgr2=0x55fbde59cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:48.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3878057651 shutdown_connections 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3878057651 wait complete. 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 -- start start 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 Processor -- start 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0d20 0x55fbde5a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fbde5ae1b0 con 0x55fbde5a0400 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fbde4bcb20 con 0x55fbde5ab5e0 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fbde5ae330 con 0x55fbde5a0d20 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9d6ce740 10 client.0 _ll_register_callbacks cb 0x55fbde4babb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9886b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9886b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50344/0 (socket says 172.21.15.69:50344) 2024-08-02T17:54:48.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.078+0000 7f9d9886b640 1 -- 172.21.15.69:0/3983988991 learned_addr learned my addr 172.21.15.69:0/3983988991 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9906c640 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0d20 0x55fbde5a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9886b640 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9886b640 1 -- 172.21.15.69:0/3983988991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0d20 msgr2=0x55fbde5a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9886b640 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0d20 0x55fbde5a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9886b640 1 -- 172.21.15.69:0/3983988991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 msgr2=0x55fbde5a8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9886b640 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9886b640 1 -- 172.21.15.69:0/3983988991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbde2733c0 con 0x55fbde5a0400 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9886b640 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a84a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9d84006010 tx=0x7f9d8403dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad094da2476d1582 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d7cff9640 1 -- 172.21.15.69:0/3983988991 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84035030 con 0x55fbde5a0400 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3983988991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d84044070 con 0x55fbde5a0400 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d7cff9640 1 -- 172.21.15.69:0/3983988991 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f9d840043d0 con 0x55fbde5a0400 2024-08-02T17:54:48.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d7cff9640 1 -- 172.21.15.69:0/3983988991 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d84059050 con 0x55fbde5a0400 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9d6ce740 10 client.17451 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3983988991 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fbde5b9d90 con 0x55fbde5a0400 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.079+0000 7f9d7cff9640 1 -- 172.21.15.69:0/3983988991 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d84044070 con 0x55fbde5a0400 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d7cff9640 1 -- 172.21.15.69:0/3983988991 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d84003b10 con 0x55fbde5a0400 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d9d6ce740 10 client.17451 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d9d6ce740 10 client.17451 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d9d6ce740 20 client.17451 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55fbde5bb0e0 0x55fbde5bd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d9d6ce740 4 client.17451 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3983988991 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbde50efb0 con 0x55fbde5bb0e0 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d9906c640 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55fbde5bb0e0 0x55fbde5bd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d7cff9640 10 client.17451 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.080+0000 7f9d9906c640 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55fbde5bb0e0 0x55fbde5bd4d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69e5ad96a439e5df server_cookie=1d51966eb990a8d0 in_seq=0 out_seq=0 2024-08-02T17:54:48.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d7cff9640 1 -- 172.21.15.69:0/3983988991 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fbde50efb0 con 0x55fbde5bb0e0 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d7cff9640 10 client.17451 handle_command_reply: tid=0 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 client.17451 shutdown 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3983988991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 msgr2=0x55fbde5a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a84a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f9d84006010 tx=0x7f9d8403dbf0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3983988991 shutdown_connections 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55fbde5bb0e0 0x55fbde5bd4d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0d20 0x55fbde5a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/3983988991 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0400 0x55fbde5a84a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3983988991 >> 172.21.15.69:0/3983988991 conn(0x55fbde59c990 msgr2=0x55fbde5a26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d7cff9640 0 client.17451 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3983988991 shutdown_connections 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/3983988991 wait complete. 2024-08-02T17:54:48.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.081+0000 7f9d9d6ce740 20 client.17451 trim_cache size 0 max 16384 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.094+0000 7f9d9d6ce740 1 Processor -- start 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.094+0000 7f9d9d6ce740 1 -- start start 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.094+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0400 0x55fbde5a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.094+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5a5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.094+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.094+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941af520 con 0x55fbde5a0d20 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.094+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941acdf0 con 0x55fbde5a0400 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941aac40 con 0x55fbde5ab5e0 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5a5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50348/0 (socket says 172.21.15.69:50348) 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 -- 172.21.15.69:0/2359561548 learned_addr learned my addr 172.21.15.69:0/2359561548 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9886b640 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 -- 172.21.15.69:0/2359561548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 msgr2=0x55fbde5a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 -- 172.21.15.69:0/2359561548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0400 msgr2=0x55fbde5a7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0400 0x55fbde5a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 -- 172.21.15.69:0/2359561548 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbde59c410 con 0x55fbde5a0d20 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9886b640 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d9906c640 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5a5720 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9d90005d20 tx=0x7f9d900039a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0fb5b4b088f471 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d7effd640 1 -- 172.21.15.69:0/2359561548 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90014b80 con 0x55fbde5a0d20 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d7effd640 1 -- 172.21.15.69:0/2359561548 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90014d20 con 0x55fbde5a0d20 2024-08-02T17:54:48.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.095+0000 7f9d7effd640 1 -- 172.21.15.69:0/2359561548 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90012450 con 0x55fbde5a0d20 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/2359561548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 msgr2=0x55fbde5a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5a5720 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f9d90005d20 tx=0x7f9d900039a0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/2359561548 shutdown_connections 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5a5720 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5ab5e0 0x55fbde5a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/2359561548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5a0400 0x55fbde5a7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/2359561548 >> 172.21.15.69:0/2359561548 conn(0x55fbde59c990 msgr2=0x55fbde5a6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/2359561548 shutdown_connections 2024-08-02T17:54:48.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/2359561548 wait complete. 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 -- start start 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.096+0000 7f9d9d6ce740 1 Processor -- start 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0400 0x55fbde5c06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5bd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9d6ce740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5bd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941ab2b0 con 0x55fbde5a0d20 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941ac720 con 0x55fbde5ab5e0 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9d6ce740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941ab980 con 0x55fbde5a0400 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9d6ce740 10 client.0 _ll_register_callbacks cb 0x7f9d84009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5bd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5bd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50364/0 (socket says 172.21.15.69:50364) 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 -- 172.21.15.69:0/390668995 learned_addr learned my addr 172.21.15.69:0/390668995 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9886b640 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9886b640 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0400 0x55fbde5c06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 -- 172.21.15.69:0/390668995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0400 msgr2=0x55fbde5c06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0400 0x55fbde5c06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 -- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 msgr2=0x55fbde5bd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5bd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 -- 172.21.15.69:0/390668995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbde5bdbf0 con 0x55fbde5a0d20 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9886b640 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5bd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d9906c640 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5bd0f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9d90000c00 tx=0x7f9d900141d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc11227dc1b7fc6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.097+0000 7f9d7cff9640 1 -- 172.21.15.69:0/390668995 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90004700 con 0x55fbde5a0d20 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d7cff9640 1 -- 172.21.15.69:0/390668995 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f9d90004ee0 con 0x55fbde5a0d20 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/390668995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d90014d20 con 0x55fbde5a0d20 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d7cff9640 1 -- 172.21.15.69:0/390668995 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9d90039050 con 0x55fbde5a0d20 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 10 client.17457 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/390668995 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fbde624750 con 0x55fbde5a0d20 2024-08-02T17:54:48.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d7cff9640 1 -- 172.21.15.69:0/390668995 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d90014d20 con 0x55fbde5a0d20 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d7cff9640 1 -- 172.21.15.69:0/390668995 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9d90012b70 con 0x55fbde5a0d20 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 10 client.17457 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 10 client.17457 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 20 client.17457 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55fbde624ed0 0x55fbde6272f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 4 client.17457 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/390668995 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbde5094c0 con 0x55fbde624ed0 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.098+0000 7f9d9886b640 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55fbde624ed0 0x55fbde6272f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9886b640 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55fbde624ed0 0x55fbde6272f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b177a0e503ec746c server_cookie=5b173190b312eaf3 in_seq=0 out_seq=0 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d7cff9640 10 client.17457 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d7cff9640 1 -- 172.21.15.69:0/390668995 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbde5094c0 con 0x55fbde624ed0 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d7cff9640 10 client.17457 handle_command_reply: tid=0 2024-08-02T17:54:48.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 client.17457 shutdown 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/390668995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 msgr2=0x55fbde5bd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5bd0f0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f9d90000c00 tx=0x7f9d900141d0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/390668995 shutdown_connections 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55fbde624ed0 0x55fbde6272f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fbde5a0d20 0x55fbde5bd0f0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fbde5ab5e0 0x55fbde5bd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 --2- 172.21.15.69:0/390668995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fbde5a0400 0x55fbde5c06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d7cff9640 0 client.17457 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/390668995 >> 172.21.15.69:0/390668995 conn(0x55fbde59c990 msgr2=0x55fbde5a7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/390668995 shutdown_connections 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 1 -- 172.21.15.69:0/390668995 wait complete. 2024-08-02T17:54:48.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.099+0000 7f9d9d6ce740 20 client.17457 trim_cache size 0 max 16384 2024-08-02T17:54:48.107 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:54:48.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d800a2d70 msgr2=0x7f9d800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d800a2d70 0x7f9d800a5230 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0a4750 tx=0x7f9d8c00efd0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941078b0 msgr2=0x7f9d9419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941078b0 0x7f9d9419caa0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00eb10 tx=0x7f9d8c00c2d0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/3311186920 shutdown_connections 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9d941078b0 0x7f9d9419caa0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9d800a2d70 0x7f9d800a5230 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9d94102250 0x7f9d94198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 --2- 172.21.15.69:0/3311186920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9d941018b0 0x7f9d941981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/3311186920 >> 172.21.15.69:0/3311186920 conn(0x7f9d940fd5f0 msgr2=0x7f9d940fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/3311186920 shutdown_connections 2024-08-02T17:54:48.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:48.100+0000 7f9d9b2d0640 1 -- 172.21.15.69:0/3311186920 wait complete. 2024-08-02T17:54:48.807 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:54:49.060 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:49.325 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:49 smithi069 ceph-mon[32746]: pgmap v258: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-08-02T17:54:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:49 smithi077 ceph-mon[37108]: pgmap v258: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-08-02T17:54:49.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:49 smithi083 ceph-mon[37031]: pgmap v258: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 6.1 MiB/s wr, 630 op/s 2024-08-02T17:54:49.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.587+0000 7f5f9b298640 1 -- 172.21.15.69:0/14331051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94102a70 msgr2=0x7f5f94102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.587+0000 7f5f9b298640 1 --2- 172.21.15.69:0/14331051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94102a70 0x7f5f94102e50 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009900 tx=0x7f5f8802a480 comp rx=0 tx=0).stop 2024-08-02T17:54:49.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 -- 172.21.15.69:0/14331051 shutdown_connections 2024-08-02T17:54:49.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 --2- 172.21.15.69:0/14331051 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410bea0 0x7f5f9410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 --2- 172.21.15.69:0/14331051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94103390 0x7f5f9410b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 --2- 172.21.15.69:0/14331051 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94102a70 0x7f5f94102e50 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 -- 172.21.15.69:0/14331051 >> 172.21.15.69:0/14331051 conn(0x7f5f940fc4d0 msgr2=0x7f5f940fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:49.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 -- 172.21.15.69:0/14331051 shutdown_connections 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 -- 172.21.15.69:0/14331051 wait complete. 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 Processor -- start 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 -- start start 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94102a70 0x7f5f941006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94103390 0x7f5f94100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410bea0 0x7f5f94101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111ca0 con 0x7f5f94103390 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111b20 con 0x7f5f9410bea0 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.588+0000 7f5f9b298640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111e20 con 0x7f5f94102a70 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94103390 0x7f5f94100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94103390 0x7f5f94100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50390/0 (socket says 172.21.15.69:50390) 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 -- 172.21.15.69:0/1401540466 learned_addr learned my addr 172.21.15.69:0/1401540466 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:49.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f99034640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94102a70 0x7f5f941006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f99835640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410bea0 0x7f5f94101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 -- 172.21.15.69:0/1401540466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94102a70 msgr2=0x7f5f941006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94102a70 0x7f5f941006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 -- 172.21.15.69:0/1401540466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410bea0 msgr2=0x7f5f94101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410bea0 0x7f5f94101150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 -- 172.21.15.69:0/1401540466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f941a65e0 con 0x7f5f94103390 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f98833640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94103390 0x7f5f94100c10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c970 tx=0x7f5f8400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f927fc640 1 -- 172.21.15.69:0/1401540466 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84007bb0 con 0x7f5f94103390 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f9b298640 1 -- 172.21.15.69:0/1401540466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f941a68d0 con 0x7f5f94103390 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f927fc640 1 -- 172.21.15.69:0/1401540466 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f84007d50 con 0x7f5f94103390 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f927fc640 1 -- 172.21.15.69:0/1401540466 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8400f450 con 0x7f5f94103390 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.589+0000 7f5f9b298640 1 -- 172.21.15.69:0/1401540466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f941a6e10 con 0x7f5f94103390 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f9d696740 1 Processor -- start 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f9d696740 1 -- start start 2024-08-02T17:54:49.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f927fc640 1 -- 172.21.15.69:0/1401540466 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f84004f60 con 0x7f5f94103390 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b2ff0 0x55d3088b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b3910 0x55d3088bdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088be1d0 0x55d3088c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f9d696740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d308818a40 con 0x55d3088b2ff0 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f9d696740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3087b09d0 con 0x55d3088be1d0 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f9d696740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d308814a00 con 0x55d3088b3910 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f99835640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b2ff0 0x55d3088b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f99835640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b2ff0 0x55d3088b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50392/0 (socket says 172.21.15.69:50392) 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f99835640 1 -- 172.21.15.69:0/2249410217 learned_addr learned my addr 172.21.15.69:0/2249410217 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.592+0000 7f5f99034640 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b3910 0x55d3088bdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f98833640 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088be1d0 0x55d3088c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f99835640 1 -- 172.21.15.69:0/2249410217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b3910 msgr2=0x55d3088bdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f99835640 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b3910 0x55d3088bdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f99835640 1 -- 172.21.15.69:0/2249410217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088be1d0 msgr2=0x55d3088c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:49.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f99835640 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088be1d0 0x55d3088c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f99835640 1 -- 172.21.15.69:0/2249410217 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3087b7650 con 0x55d3088b2ff0 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f98833640 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088be1d0 0x55d3088c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f927fc640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2cf0 0x7f5f7c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f99835640 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b2ff0 0x55d3088b33d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c007dd0 tx=0x7f5f8c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672a6227827c47fc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f99034640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2cf0 0x7f5f7c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f7b7fe640 1 -- 172.21.15.69:0/2249410217 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c014070 con 0x55d3088b2ff0 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f927fc640 1 -- 172.21.15.69:0/1401540466 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f5f840cf4d0 con 0x7f5f94103390 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f7b7fe640 1 -- 172.21.15.69:0/2249410217 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f8c004d20 con 0x55d3088b2ff0 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.593+0000 7f5f7b7fe640 1 -- 172.21.15.69:0/2249410217 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c010850 con 0x55d3088b2ff0 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 -- 172.21.15.69:0/2249410217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b2ff0 msgr2=0x55d3088b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f99034640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2cf0 0x7f5f7c0a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f88000c00 tx=0x7f5f880058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b2ff0 0x55d3088b33d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c007dd0 tx=0x7f5f8c00e720 comp rx=0 tx=0).stop 2024-08-02T17:54:49.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 -- 172.21.15.69:0/2249410217 shutdown_connections 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088be1d0 0x55d3088c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b3910 0x55d3088bdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 --2- 172.21.15.69:0/2249410217 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b2ff0 0x55d3088b33d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 -- 172.21.15.69:0/2249410217 >> 172.21.15.69:0/2249410217 conn(0x55d3088af580 msgr2=0x55d3088af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 -- 172.21.15.69:0/2249410217 shutdown_connections 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 -- 172.21.15.69:0/2249410217 wait complete. 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.594+0000 7f5f9d696740 1 -- start start 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f9d696740 1 Processor -- start 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088bb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f9d696740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d3088c0da0 con 0x55d3088b3910 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f9d696740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d3087d3b20 con 0x55d3088b2ff0 2024-08-02T17:54:49.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f9d696740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d3088c0f20 con 0x55d3088be1d0 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f9d696740 10 client.0 _ll_register_callbacks cb 0x55d3087d1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f98833640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088bb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f98833640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088bb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50408/0 (socket says 172.21.15.69:50408) 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f98833640 1 -- 172.21.15.69:0/925575711 learned_addr learned my addr 172.21.15.69:0/925575711 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f99835640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f98833640 1 -- 172.21.15.69:0/925575711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 msgr2=0x55d3088bbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f98833640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088bbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f99034640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f98833640 1 -- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 msgr2=0x55d3088bb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f98833640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088bb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f98833640 1 -- 172.21.15.69:0/925575711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d30858a3c0 con 0x55d3088b3910 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f99835640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.595+0000 7f5f99034640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088bb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:49.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f98833640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088bb590 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5f84007590 tx=0x7f5f84097a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebb0ca39989f495 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f78ff9640 1 -- 172.21.15.69:0/925575711 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8401d040 con 0x55d3088b3910 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f78ff9640 1 -- 172.21.15.69:0/925575711 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f5f840d2c40 con 0x55d3088b3910 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 1 -- 172.21.15.69:0/925575711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8c004d20 con 0x55d3088b3910 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f78ff9640 1 -- 172.21.15.69:0/925575711 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f840b4070 con 0x55d3088b3910 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 10 client.17469 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 1 -- 172.21.15.69:0/925575711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d3088cc920 con 0x55d3088b3910 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f78ff9640 1 -- 172.21.15.69:0/925575711 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f8c004d20 con 0x55d3088b3910 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f78ff9640 1 -- 172.21.15.69:0/925575711 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5f84084bc0 con 0x55d3088b3910 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 10 client.17469 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 20 client.17469 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3088cdd80 0x55d3088d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 4 client.17469 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.596+0000 7f5f9d696740 1 -- 172.21.15.69:0/925575711 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d308825fb0 con 0x55d3088cdd80 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.597+0000 7f5f99034640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3088cdd80 0x55d3088d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.597+0000 7f5f99034640 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3088cdd80 0x55d3088d01f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e14298dab17bb7db server_cookie=443395ac1c819446 in_seq=0 out_seq=0 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.597+0000 7f5f78ff9640 10 client.17469 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f78ff9640 1 -- 172.21.15.69:0/925575711 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d308825fb0 con 0x55d3088cdd80 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f78ff9640 10 client.17469 handle_command_reply: tid=0 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 client.17469 shutdown 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 -- 172.21.15.69:0/925575711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 msgr2=0x55d3088bb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088bb590 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f5f84007590 tx=0x7f5f84097a10 comp rx=0 tx=0).stop 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 -- 172.21.15.69:0/925575711 shutdown_connections 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088bb590 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3088cdd80 0x55d3088d01f0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 --2- 172.21.15.69:0/925575711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088bb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f78ff9640 0 client.17469 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 -- 172.21.15.69:0/925575711 >> 172.21.15.69:0/925575711 conn(0x55d3088af580 msgr2=0x55d3088b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 -- 172.21.15.69:0/925575711 shutdown_connections 2024-08-02T17:54:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 1 -- 172.21.15.69:0/925575711 wait complete. 2024-08-02T17:54:49.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.598+0000 7f5f9d696740 20 client.17469 trim_cache size 0 max 16384 2024-08-02T17:54:49.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f9d696740 1 Processor -- start 2024-08-02T17:54:49.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f9d696740 1 -- start start 2024-08-02T17:54:49.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088d5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088d6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f9d696740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b39a0 con 0x55d3088b3910 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f9d696740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b1270 con 0x55d3088b2ff0 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f9d696740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941af0c0 con 0x55d3088be1d0 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f98833640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088d5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f98833640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088d5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50414/0 (socket says 172.21.15.69:50414) 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f98833640 1 -- 172.21.15.69:0/1176875489 learned_addr learned my addr 172.21.15.69:0/1176875489 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f99034640 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f98833640 1 -- 172.21.15.69:0/1176875489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 msgr2=0x55d3088d6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f99835640 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088d6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f98833640 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088d6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f98833640 1 -- 172.21.15.69:0/1176875489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 msgr2=0x55d3088b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.613+0000 7f5f98833640 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.614+0000 7f5f98833640 1 -- 172.21.15.69:0/1176875489 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3088af020 con 0x55d3088b3910 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.614+0000 7f5f99835640 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088d6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.614+0000 7f5f98833640 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088d5ac0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5f8409b720 tx=0x7f5f840a4400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380b7d3f146a2dff server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.614+0000 7f5f7affd640 1 -- 172.21.15.69:0/1176875489 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8401d040 con 0x55d3088b3910 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.614+0000 7f5f7affd640 1 -- 172.21.15.69:0/1176875489 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f84086070 con 0x55d3088b3910 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.614+0000 7f5f7affd640 1 -- 172.21.15.69:0/1176875489 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f840afba0 con 0x55d3088b3910 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 -- 172.21.15.69:0/1176875489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 msgr2=0x55d3088d5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088d5ac0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5f8409b720 tx=0x7f5f840a4400 comp rx=0 tx=0).stop 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 -- 172.21.15.69:0/1176875489 shutdown_connections 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088b3910 0x55d3088d5ac0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088be1d0 0x55d3088d6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1176875489 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b2ff0 0x55d3088b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 -- 172.21.15.69:0/1176875489 >> 172.21.15.69:0/1176875489 conn(0x55d3088af580 msgr2=0x55d3088ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 -- 172.21.15.69:0/1176875489 shutdown_connections 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 -- 172.21.15.69:0/1176875489 wait complete. 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 -- start start 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.615+0000 7f5f9d696740 1 Processor -- start 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b2ff0 0x55d3088c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b3910 0x55d3088c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f9d696740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088be1d0 0x55d308936a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f9d696740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941af730 con 0x55d3088be1d0 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f9d696740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941b0ba0 con 0x55d3088b3910 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f9d696740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f941afe00 con 0x55d3088b2ff0 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f9d696740 10 client.0 _ll_register_callbacks cb 0x7f5f840058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:49.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088be1d0 0x55d308936a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088be1d0 0x55d308936a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50426/0 (socket says 172.21.15.69:50426) 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 -- 172.21.15.69:0/1275357091 learned_addr learned my addr 172.21.15.69:0/1275357091 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99034640 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b2ff0 0x55d3088c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 -- 172.21.15.69:0/1275357091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b2ff0 msgr2=0x55d3088c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b2ff0 0x55d3088c16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 -- 172.21.15.69:0/1275357091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b3910 msgr2=0x55d3088c1c10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b3910 0x55d3088c1c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 -- 172.21.15.69:0/1275357091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d308937090 con 0x55d3088be1d0 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f99835640 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088be1d0 0x55d308936a80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c005c00 tx=0x7f5f8c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9dc7a72ec832f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f78ff9640 1 -- 172.21.15.69:0/1275357091 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c014030 con 0x55d3088be1d0 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f78ff9640 1 -- 172.21.15.69:0/1275357091 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f5f8c0050c0 con 0x55d3088be1d0 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.616+0000 7f5f9d696740 1 -- 172.21.15.69:0/1275357091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f84086070 con 0x55d3088be1d0 2024-08-02T17:54:49.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f78ff9640 1 -- 172.21.15.69:0/1275357091 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f8c003e50 con 0x55d3088be1d0 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f9d696740 10 client.17475 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f9d696740 1 -- 172.21.15.69:0/1275357091 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d308938370 con 0x55d3088be1d0 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f78ff9640 1 -- 172.21.15.69:0/1275357091 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f84086070 con 0x55d3088be1d0 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f78ff9640 1 -- 172.21.15.69:0/1275357091 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5f8c010920 con 0x55d3088be1d0 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f9d696740 10 client.17475 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f9d696740 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f9d696740 20 client.17475 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3089389e0 0x55d30893ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f9d696740 4 client.17475 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f9d696740 1 -- 172.21.15.69:0/1275357091 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3088204c0 con 0x55d3089389e0 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.617+0000 7f5f99034640 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3089389e0 0x55d30893ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f99034640 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3089389e0 0x55d30893ae00 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30bf5fb1854a331c server_cookie=eb355692c8bbc13f in_seq=0 out_seq=0 2024-08-02T17:54:49.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f78ff9640 10 client.17475 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f78ff9640 1 -- 172.21.15.69:0/1275357091 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3088204c0 con 0x55d3089389e0 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f78ff9640 10 client.17475 handle_command_reply: tid=0 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 client.17475 shutdown 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 -- 172.21.15.69:0/1275357091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088be1d0 msgr2=0x55d308936a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088be1d0 0x55d308936a80 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c005c00 tx=0x7f5f8c005c30 comp rx=0 tx=0).stop 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 -- 172.21.15.69:0/1275357091 shutdown_connections 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d3088be1d0 0x55d308936a80 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d3089389e0 0x55d30893ae00 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f78ff9640 0 client.17475 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d3088b3910 0x55d3088c1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 --2- 172.21.15.69:0/1275357091 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d3088b2ff0 0x55d3088c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.618+0000 7f5f9d696740 1 -- 172.21.15.69:0/1275357091 >> 172.21.15.69:0/1275357091 conn(0x55d3088af580 msgr2=0x55d3088bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9d696740 1 -- 172.21.15.69:0/1275357091 shutdown_connections 2024-08-02T17:54:49.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9d696740 1 -- 172.21.15.69:0/1275357091 wait complete. 2024-08-02T17:54:49.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9d696740 20 client.17475 trim_cache size 0 max 16384 2024-08-02T17:54:49.630 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:54:49.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 -- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2cf0 msgr2=0x7f5f7c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2cf0 0x7f5f7c0a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5f88000c00 tx=0x7f5f880058d0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 -- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94103390 msgr2=0x7f5f94100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94103390 0x7f5f94100c10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c970 tx=0x7f5f8400ce30 comp rx=0 tx=0).stop 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 -- 172.21.15.69:0/1401540466 shutdown_connections 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94103390 0x7f5f94100c10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2cf0 0x7f5f7c0a51b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f9410bea0 0x7f5f94101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 --2- 172.21.15.69:0/1401540466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94102a70 0x7f5f941006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 -- 172.21.15.69:0/1401540466 >> 172.21.15.69:0/1401540466 conn(0x7f5f940fc4d0 msgr2=0x7f5f940fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 -- 172.21.15.69:0/1401540466 shutdown_connections 2024-08-02T17:54:49.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:49.619+0000 7f5f9b298640 1 -- 172.21.15.69:0/1401540466 wait complete. 2024-08-02T17:54:49.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 460.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:54:50.897 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:54:51.174 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:51.297 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:51 smithi069 ceph-mon[32746]: pgmap v259: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-08-02T17:54:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:51 smithi077 ceph-mon[37108]: pgmap v259: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-08-02T17:54:51.417 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:54:51.417 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:54:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:51 smithi083 ceph-mon[37031]: pgmap v259: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 3.3 MiB/s wr, 348 op/s 2024-08-02T17:54:51.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 -- 172.21.15.69:0/2919909608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81007b0 msgr2=0x7f50b8100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 --2- 172.21.15.69:0/2919909608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81007b0 0x7f50b8100c10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f50a0009930 tx=0x7f50a002a420 comp rx=0 tx=0).stop 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 -- 172.21.15.69:0/2919909608 shutdown_connections 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 --2- 172.21.15.69:0/2919909608 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81007b0 0x7f50b8100c10 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 --2- 172.21.15.69:0/2919909608 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b8101150 0x7f50b810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 --2- 172.21.15.69:0/2919909608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50b81067b0 0x7f50b8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 -- 172.21.15.69:0/2919909608 >> 172.21.15.69:0/2919909608 conn(0x7f50b80fc4d0 msgr2=0x7f50b80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 -- 172.21.15.69:0/2919909608 shutdown_connections 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 -- 172.21.15.69:0/2919909608 wait complete. 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.720+0000 7f50bde49640 1 Processor -- start 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50bde49640 1 -- start start 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50bde49640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b81007b0 0x7f50b819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:51.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50bde49640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50b8101150 0x7f50b819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50bde49640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81067b0 0x7f50b819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50bde49640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810fae0 con 0x7f50b81067b0 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50bde49640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810f960 con 0x7f50b81007b0 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50bde49640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f50b810fc60 con 0x7f50b8101150 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81067b0 0x7f50b819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81067b0 0x7f50b819ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50438/0 (socket says 172.21.15.69:50438) 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 -- 172.21.15.69:0/2240959357 learned_addr learned my addr 172.21.15.69:0/2240959357 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b77fe640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b81007b0 0x7f50b819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50affff640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50b8101150 0x7f50b819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 -- 172.21.15.69:0/2240959357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50b8101150 msgr2=0x7f50b819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50b8101150 0x7f50b819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 -- 172.21.15.69:0/2240959357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b81007b0 msgr2=0x7f50b819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b81007b0 0x7f50b819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 -- 172.21.15.69:0/2240959357 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b819f350 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50affff640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50b8101150 0x7f50b819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.721+0000 7f50b7fff640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81067b0 0x7f50b819ec70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f50b000cce0 tx=0x7f50b000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.722+0000 7f50b57fa640 1 -- 172.21.15.69:0/2240959357 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b000f450 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.722+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b819f640 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.722+0000 7f50b57fa640 1 -- 172.21.15.69:0/2240959357 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50b000fad0 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.722+0000 7f50b57fa640 1 -- 172.21.15.69:0/2240959357 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0018b60 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.722+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b81a6f20 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.723+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b8100c10 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.724+0000 7f50b57fa640 1 -- 172.21.15.69:0/2240959357 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f50b000fc80 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.724+0000 7f50b57fa640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50a40a2cf0 0x7f50a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.724+0000 7f50b77fe640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50a40a2cf0 0x7f50a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.724+0000 7f50b57fa640 1 -- 172.21.15.69:0/2240959357 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f50b00cfd90 con 0x7f50b81067b0 2024-08-02T17:54:51.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.725+0000 7f50b77fe640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50a40a2cf0 0x7f50a40a51b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009990 tx=0x7f50a8009110 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:51.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.729+0000 7f50b57fa640 1 -- 172.21.15.69:0/2240959357 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f50b0014030 con 0x7f50b81067b0 2024-08-02T17:54:51.834 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:51.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.946+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50b81a7180 con 0x7f50b81067b0 2024-08-02T17:54:51.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.947+0000 7f50b57fa640 1 -- 172.21.15.69:0/2240959357 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f50b009c5a0 con 0x7f50b81067b0 2024-08-02T17:54:51.949 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:54:51.949 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:54:51.949 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50a40a2cf0 msgr2=0x7f50a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50a40a2cf0 0x7f50a40a51b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009990 tx=0x7f50a8009110 comp rx=0 tx=0).stop 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81067b0 msgr2=0x7f50b819ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81067b0 0x7f50b819ec70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f50b000cce0 tx=0x7f50b000a620 comp rx=0 tx=0).stop 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 shutdown_connections 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f50b81067b0 0x7f50b819ec70 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50a40a2cf0 0x7f50a40a51b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f50b8101150 0x7f50b819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 --2- 172.21.15.69:0/2240959357 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f50b81007b0 0x7f50b819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 >> 172.21.15.69:0/2240959357 conn(0x7f50b80fc4d0 msgr2=0x7f50b80fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 shutdown_connections 2024-08-02T17:54:51.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:51.949+0000 7f50bde49640 1 -- 172.21.15.69:0/2240959357 wait complete. 2024-08-02T17:54:52.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:52 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2240959357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:52.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:52 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2240959357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:52.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:52 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2240959357' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:52.793 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 81a0063a --release --await' 2024-08-02T17:54:52.793 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 81a0063a --release --await 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 -- 172.21.15.69:0/2242966135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 msgr2=0x7f5a141026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/2242966135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 0x7f5a141026e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f59fc009900 tx=0x7f59fc02a480 comp rx=0 tx=0).stop 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 -- 172.21.15.69:0/2242966135 shutdown_connections 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/2242966135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 0x7f5a141026e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/2242966135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14102c20 0x7f5a1410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/2242966135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a14100b50 0x7f5a14100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 -- 172.21.15.69:0/2242966135 >> 172.21.15.69:0/2242966135 conn(0x7f5a140fc500 msgr2=0x7f5a140fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 -- 172.21.15.69:0/2242966135 shutdown_connections 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.026+0000 7f5a18b5d640 1 -- 172.21.15.69:0/2242966135 wait complete. 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.027+0000 7f5a18b5d640 1 Processor -- start 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.027+0000 7f5a18b5d640 1 -- start start 2024-08-02T17:54:53.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a18b5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14100b50 0x7f5a141990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a18b5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 0x7f5a141995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a18b5d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a14102c20 0x7f5a141a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a18b5d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14110c70 con 0x7f5a141022a0 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a18b5d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14110af0 con 0x7f5a14102c20 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a18b5d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a14110df0 con 0x7f5a14100b50 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 0x7f5a141995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 0x7f5a141995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50452/0 (socket says 172.21.15.69:50452) 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 -- 172.21.15.69:0/3287258197 learned_addr learned my addr 172.21.15.69:0/3287258197 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a1259c640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14100b50 0x7f5a141990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a14102c20 0x7f5a141a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 -- 172.21.15.69:0/3287258197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14100b50 msgr2=0x7f5a141990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14100b50 0x7f5a141990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 -- 172.21.15.69:0/3287258197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a14102c20 msgr2=0x7f5a141a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a14102c20 0x7f5a141a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 -- 172.21.15.69:0/3287258197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a141a4660 con 0x7f5a141022a0 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a14102c20 0x7f5a141a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.028+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 0x7f5a141995e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f59fc006010 tx=0x7f59fc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.029+0000 7f5a0bfff640 1 -- 172.21.15.69:0/3287258197 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc03f070 con 0x7f5a141022a0 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.029+0000 7f5a18b5d640 1 -- 172.21.15.69:0/3287258197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a141a4830 con 0x7f5a141022a0 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.029+0000 7f5a0bfff640 1 -- 172.21.15.69:0/3287258197 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59fc031e30 con 0x7f5a141022a0 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.029+0000 7f5a0bfff640 1 -- 172.21.15.69:0/3287258197 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc035030 con 0x7f5a141022a0 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.029+0000 7f5a18b5d640 1 -- 172.21.15.69:0/3287258197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a141a4d10 con 0x7f5a141022a0 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.031+0000 7f5a0bfff640 1 -- 172.21.15.69:0/3287258197 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f59fc042070 con 0x7f5a141022a0 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.031+0000 7f5a1af5b740 1 Processor -- start 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.031+0000 7f5a1af5b740 1 -- start start 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec34a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 0x5619ec354ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec3577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5619ec30de40 con 0x5619ec34a200 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5619ec2af7f0 con 0x5619ec34ab20 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5619ec256fb0 con 0x5619ec3553e0 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec34a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec34a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50466/0 (socket says 172.21.15.69:50466) 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 -- 172.21.15.69:0/2944249915 learned_addr learned my addr 172.21.15.69:0/2944249915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a0bfff640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59f40a7100 0x7f59f40a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec3577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a1259c640 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 0x5619ec354ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59f40a7100 0x7f59f40a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 -- 172.21.15.69:0/2944249915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 msgr2=0x5619ec3577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec3577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 -- 172.21.15.69:0/2944249915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 msgr2=0x5619ec354ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 0x5619ec354ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a12d9d640 1 -- 172.21.15.69:0/2944249915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ec0216a0 con 0x5619ec34a200 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec3577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.032+0000 7f5a0bfff640 1 -- 172.21.15.69:0/3287258197 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f59fc0f01b0 con 0x7f5a141022a0 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.033+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec34a5e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c012920 tx=0x7f5a0c012de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58105a82d30c93cc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.033+0000 7f5a08d99640 1 -- 172.21.15.69:0/2944249915 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c022070 con 0x5619ec34a200 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.033+0000 7f5a08d99640 1 -- 172.21.15.69:0/2944249915 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c003b70 con 0x5619ec34a200 2024-08-02T17:54:53.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.033+0000 7f5a08d99640 1 -- 172.21.15.69:0/2944249915 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c01e690 con 0x5619ec34a200 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2944249915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 msgr2=0x5619ec34a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec34a5e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c012920 tx=0x7f5a0c012de0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59f40a7100 0x7f59f40a95c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c020000 tx=0x7f5a0c01f000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2944249915 shutdown_connections 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec3577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 0x5619ec354ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2944249915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec34a5e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2944249915 >> 172.21.15.69:0/2944249915 conn(0x5619ec346790 msgr2=0x5619ec346b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2944249915 shutdown_connections 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2944249915 wait complete. 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- start start 2024-08-02T17:54:53.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 Processor -- start 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec352260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 0x5619ec3527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec352ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5619ec26acb0 con 0x5619ec34a200 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5619ec2ab7b0 con 0x5619ec34ab20 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.034+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5619ec357fb0 con 0x5619ec3553e0 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1af5b740 10 client.0 _ll_register_callbacks cb 0x5619ebfbeba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec352260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec352260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50474/0 (socket says 172.21.15.69:50474) 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 -- 172.21.15.69:0/2866065282 learned_addr learned my addr 172.21.15.69:0/2866065282 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec352ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 0x5619ec3527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 -- 172.21.15.69:0/2866065282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 msgr2=0x5619ec352ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec352ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 -- 172.21.15.69:0/2866065282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 msgr2=0x5619ec3527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 0x5619ec3527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 -- 172.21.15.69:0/2866065282 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ec269bc0 con 0x5619ec34a200 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1259c640 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec352260 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5a04010850 tx=0x7f5a04010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cade6ba8757e98b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a01ffb640 1 -- 172.21.15.69:0/2866065282 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0400fb90 con 0x5619ec34a200 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a01ffb640 1 -- 172.21.15.69:0/2866065282 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f5a0400fd30 con 0x5619ec34a200 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.035+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2866065282 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a0c003b70 con 0x5619ec34a200 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a01ffb640 1 -- 172.21.15.69:0/2866065282 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a04024050 con 0x5619ec34a200 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a1af5b740 10 client.17493 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2866065282 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5619ec363e00 con 0x5619ec34a200 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a01ffb640 1 -- 172.21.15.69:0/2866065282 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a0c003b70 con 0x5619ec34a200 2024-08-02T17:54:53.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a01ffb640 1 -- 172.21.15.69:0/2866065282 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a04019410 con 0x5619ec34a200 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a1af5b740 10 client.17493 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a1af5b740 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a1af5b740 20 client.17493 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5619ec365310 0x5619ec3677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a1af5b740 4 client.17493 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.036+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2866065282 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619ec2bcd60 con 0x5619ec365310 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.037+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5619ec365310 0x5619ec3677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.037+0000 7f5a01ffb640 10 client.17493 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.037+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5619ec365310 0x5619ec3677b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84433a96a2aded server_cookie=ec60bab1d38313c in_seq=0 out_seq=0 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a01ffb640 1 -- 172.21.15.69:0/2866065282 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5619ec2bcd60 con 0x5619ec365310 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a01ffb640 10 client.17493 handle_command_reply: tid=0 2024-08-02T17:54:53.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 client.17493 shutdown 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2866065282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 msgr2=0x5619ec352260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec352260 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5a04010850 tx=0x7f5a04010d10 comp rx=0 tx=0).stop 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2866065282 shutdown_connections 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5619ec365310 0x5619ec3677b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec3553e0 0x5619ec352ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec34ab20 0x5619ec3527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/2866065282 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34a200 0x5619ec352260 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2866065282 >> 172.21.15.69:0/2866065282 conn(0x5619ec346790 msgr2=0x5619ec34c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a01ffb640 0 client.17493 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.038+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2866065282 shutdown_connections 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.039+0000 7f5a1af5b740 1 -- 172.21.15.69:0/2866065282 wait complete. 2024-08-02T17:54:53.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.039+0000 7f5a1af5b740 20 client.17493 trim_cache size 0 max 16384 2024-08-02T17:54:53.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.057+0000 7f5a1af5b740 1 Processor -- start 2024-08-02T17:54:53.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.057+0000 7f5a1af5b740 1 -- start start 2024-08-02T17:54:53.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec3541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34ab20 0x5619ec36d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec36d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141af660 con 0x5619ec34ab20 2024-08-02T17:54:53.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141acf30 con 0x5619ec3553e0 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141aad80 con 0x5619ec34a200 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34ab20 0x5619ec36d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34ab20 0x5619ec36d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50482/0 (socket says 172.21.15.69:50482) 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 -- 172.21.15.69:0/307656503 learned_addr learned my addr 172.21.15.69:0/307656503 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec36d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a1259c640 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec3541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 -- 172.21.15.69:0/307656503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 msgr2=0x5619ec3541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec3541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 -- 172.21.15.69:0/307656503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 msgr2=0x5619ec36d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec36d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 -- 172.21.15.69:0/307656503 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ec34a950 con 0x5619ec34ab20 2024-08-02T17:54:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec36d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34ab20 0x5619ec36d080 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0b8720 tx=0x7f59fc08f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4577bd045cad02 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.058+0000 7f5a03fff640 1 -- 172.21.15.69:0/307656503 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc049040 con 0x5619ec34ab20 2024-08-02T17:54:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a03fff640 1 -- 172.21.15.69:0/307656503 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59fc0bc9c0 con 0x5619ec34ab20 2024-08-02T17:54:53.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a03fff640 1 -- 172.21.15.69:0/307656503 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59fc03b5f0 con 0x5619ec34ab20 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 -- 172.21.15.69:0/307656503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34ab20 msgr2=0x5619ec36d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34ab20 0x5619ec36d080 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0b8720 tx=0x7f59fc08f330 comp rx=0 tx=0).stop 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 -- 172.21.15.69:0/307656503 shutdown_connections 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34ab20 0x5619ec36d080 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec36d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/307656503 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec3541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 -- 172.21.15.69:0/307656503 >> 172.21.15.69:0/307656503 conn(0x5619ec346790 msgr2=0x5619ec353ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 -- 172.21.15.69:0/307656503 shutdown_connections 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.059+0000 7f5a1af5b740 1 -- 172.21.15.69:0/307656503 wait complete. 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 1 -- start start 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 1 Processor -- start 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec35d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec35d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34e850 0x5619ec34ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141ab3f0 con 0x5619ec34e850 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141ac860 con 0x5619ec3553e0 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a141abac0 con 0x5619ec34a200 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1af5b740 10 client.0 _ll_register_callbacks cb 0x7f5a0400a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec35d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec35d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59168/0 (socket says 172.21.15.69:59168) 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a1259c640 1 -- 172.21.15.69:0/1113811698 learned_addr learned my addr 172.21.15.69:0/1113811698 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.060+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34e850 0x5619ec34ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec35d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a12d9d640 1 -- 172.21.15.69:0/1113811698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 msgr2=0x5619ec35d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec35d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a12d9d640 1 -- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 msgr2=0x5619ec35d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:53.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec35d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a12d9d640 1 -- 172.21.15.69:0/1113811698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ec34f310 con 0x5619ec34e850 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a09d9b640 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec35d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a12d9d640 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34e850 0x5619ec34ecd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c010740 tx=0x7f5a0c01ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da88565cefc2ebf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a01ffb640 1 -- 172.21.15.69:0/1113811698 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c022040 con 0x5619ec34e850 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a1af5b740 1 -- 172.21.15.69:0/1113811698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59fc0bc9c0 con 0x5619ec34e850 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a01ffb640 1 -- 172.21.15.69:0/1113811698 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f5a0c026430 con 0x5619ec34e850 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a01ffb640 1 -- 172.21.15.69:0/1113811698 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5a0c040050 con 0x5619ec34e850 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a1af5b740 10 client.17499 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.061+0000 7f5a1af5b740 1 -- 172.21.15.69:0/1113811698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5619ec3cfe30 con 0x5619ec34e850 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a01ffb640 1 -- 172.21.15.69:0/1113811698 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59fc0bc9c0 con 0x5619ec34e850 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a01ffb640 1 -- 172.21.15.69:0/1113811698 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5a0c033410 con 0x5619ec34e850 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a1af5b740 10 client.17499 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a1af5b740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:53.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a1af5b740 20 client.17499 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5619ec3d0440 0x5619ec3d2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a1af5b740 4 client.17499 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a1af5b740 1 -- 172.21.15.69:0/1113811698 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5619ec2b7270 con 0x5619ec3d0440 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a1259c640 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5619ec3d0440 0x5619ec3d2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a01ffb640 10 client.17499 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.062+0000 7f5a1259c640 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5619ec3d0440 0x5619ec3d2860 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e13ed62043c76e4 server_cookie=8795a378c3ec4e3 in_seq=0 out_seq=0 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a01ffb640 1 -- 172.21.15.69:0/1113811698 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5619ec2b7270 con 0x5619ec3d0440 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a01ffb640 10 client.17499 handle_command_reply: tid=0 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 client.17499 shutdown 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 -- 172.21.15.69:0/1113811698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34e850 msgr2=0x5619ec34ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34e850 0x5619ec34ecd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c010740 tx=0x7f5a0c01ee00 comp rx=0 tx=0).stop 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 -- 172.21.15.69:0/1113811698 shutdown_connections 2024-08-02T17:54:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5619ec34e850 0x5619ec34ecd0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5619ec3d0440 0x5619ec3d2860 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a01ffb640 0 client.17499 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5619ec3553e0 0x5619ec35d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 --2- 172.21.15.69:0/1113811698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5619ec34a200 0x5619ec35d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 -- 172.21.15.69:0/1113811698 >> 172.21.15.69:0/1113811698 conn(0x5619ec346790 msgr2=0x5619ec3458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 -- 172.21.15.69:0/1113811698 shutdown_connections 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 1 -- 172.21.15.69:0/1113811698 wait complete. 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.064+0000 7f5a1af5b740 20 client.17499 trim_cache size 0 max 16384 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:54:53.071 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:54:53.072 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "8963ec58-326e-465c-9932-e4587cec2864", 2024-08-02T17:54:53.072 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:54:53.072 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 -- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59f40a7100 msgr2=0x7f59f40a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59f40a7100 0x7f59f40a95c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c020000 tx=0x7f5a0c01f000 comp rx=0 tx=0).stop 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 -- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 msgr2=0x7f5a141995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 0x7f5a141995e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f59fc006010 tx=0x7f59fc003c80 comp rx=0 tx=0).stop 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 -- 172.21.15.69:0/3287258197 shutdown_connections 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a141022a0 0x7f5a141995e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59f40a7100 0x7f59f40a95c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a14102c20 0x7f5a141a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 --2- 172.21.15.69:0/3287258197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a14100b50 0x7f5a141990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 -- 172.21.15.69:0/3287258197 >> 172.21.15.69:0/3287258197 conn(0x7f5a140fc500 msgr2=0x7f5a140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 -- 172.21.15.69:0/3287258197 shutdown_connections 2024-08-02T17:54:53.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:53.065+0000 7f5a18b5d640 1 -- 172.21.15.69:0/3287258197 wait complete. 2024-08-02T17:54:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:53 smithi077 ceph-mon[37108]: pgmap v260: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-08-02T17:54:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:53 smithi069 ceph-mon[32746]: pgmap v260: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-08-02T17:54:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:53 smithi083 ceph-mon[37031]: pgmap v260: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 3.0 MiB/s wr, 318 op/s 2024-08-02T17:54:53.712 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:53.738 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:54:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3302783111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 msgr2=0x7f2d2c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3302783111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 0x7f2d2c10e790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009cd0 tx=0x7f2d2402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3302783111 shutdown_connections 2024-08-02T17:54:54.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3302783111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 0x7f2d2c10e790 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3302783111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c1018c0 0x7f2d2c101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3302783111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d2c1078c0 0x7f2d2c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3302783111 >> 172.21.15.69:0/3302783111 conn(0x7f2d2c0fd5e0 msgr2=0x7f2d2c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3302783111 shutdown_connections 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.202+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3302783111 wait complete. 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d33f1a640 1 Processor -- start 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d33f1a640 1 -- start start 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d33f1a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c1018c0 0x7f2d2c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d33f1a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 0x7f2d2c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d33f1a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d2c1078c0 0x7f2d2c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d33f1a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c111cc0 con 0x7f2d2c102260 2024-08-02T17:54:54.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d33f1a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c111b40 con 0x7f2d2c1018c0 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d33f1a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c111e40 con 0x7f2d2c1078c0 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d314b5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 0x7f2d2c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d314b5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 0x7f2d2c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50498/0 (socket says 172.21.15.69:50498) 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.203+0000 7f2d314b5640 1 -- 172.21.15.69:0/3871037497 learned_addr learned my addr 172.21.15.69:0/3871037497 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c1018c0 0x7f2d2c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d324b7640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d2c1078c0 0x7f2d2c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d314b5640 1 -- 172.21.15.69:0/3871037497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d2c1078c0 msgr2=0x7f2d2c19c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d314b5640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d2c1078c0 0x7f2d2c19c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d314b5640 1 -- 172.21.15.69:0/3871037497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c1018c0 msgr2=0x7f2d2c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d314b5640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c1018c0 0x7f2d2c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d314b5640 1 -- 172.21.15.69:0/3871037497 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c19d050 con 0x7f2d2c102260 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d324b7640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d2c1078c0 0x7f2d2c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d314b5640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 0x7f2d2c1985e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00c920 tx=0x7f2d1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d22ffd640 1 -- 172.21.15.69:0/3871037497 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c007c70 con 0x7f2d2c102260 2024-08-02T17:54:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d22ffd640 1 -- 172.21.15.69:0/3871037497 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c007e10 con 0x7f2d2c102260 2024-08-02T17:54:54.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d22ffd640 1 -- 172.21.15.69:0/3871037497 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c018890 con 0x7f2d2c102260 2024-08-02T17:54:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3871037497 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c19d340 con 0x7f2d2c102260 2024-08-02T17:54:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.204+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3871037497 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c1a4c20 con 0x7f2d2c102260 2024-08-02T17:54:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.207+0000 7f2d22ffd640 1 -- 172.21.15.69:0/3871037497 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2d1c018a50 con 0x7f2d2c102260 2024-08-02T17:54:54.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.207+0000 7f2d22ffd640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d180a2cf0 0x7f2d180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.207+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d180a2cf0 0x7f2d180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.207+0000 7f2d22ffd640 1 -- 172.21.15.69:0/3871037497 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f2d1c0cf9d0 con 0x7f2d2c102260 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d180a2cf0 0x7f2d180a51b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2d280058c0 tx=0x7f2d2800a250 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d36318740 1 Processor -- start 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d36318740 1 -- start start 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d36318740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d36318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d36318740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376b0b70 0x5612376b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d36318740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56123760bc60 con 0x5612376a5990 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d36318740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5612375a3b90 con 0x5612376b0b70 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d36318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561237607c20 con 0x5612376a62b0 2024-08-02T17:54:54.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.208+0000 7f2d31cb6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d31cb6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376a5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50502/0 (socket says 172.21.15.69:50502) 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d31cb6640 1 -- 172.21.15.69:0/3628795070 learned_addr learned my addr 172.21.15.69:0/3628795070 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d314b5640 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d324b7640 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376b0b70 0x5612376b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d31cb6640 1 -- 172.21.15.69:0/3628795070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 msgr2=0x5612376b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d31cb6640 1 -- 172.21.15.69:0/3628795070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376b0b70 msgr2=0x5612376b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376b0b70 0x5612376b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d31cb6640 1 -- 172.21.15.69:0/3628795070 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56123731b580 con 0x5612376a5990 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d324b7640 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376b0b70 0x5612376b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376a5d70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800a500 tx=0x7f2d28012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09c7df8e1b63b73 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:54.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d17fff640 1 -- 172.21.15.69:0/3628795070 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28016070 con 0x5612376a5990 2024-08-02T17:54:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d17fff640 1 -- 172.21.15.69:0/3628795070 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d28010e40 con 0x5612376a5990 2024-08-02T17:54:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.209+0000 7f2d17fff640 1 -- 172.21.15.69:0/3628795070 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2801c8c0 con 0x5612376a5990 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.210+0000 7f2d36318740 1 -- 172.21.15.69:0/3628795070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 msgr2=0x5612376a5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.210+0000 7f2d36318740 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376a5d70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800a500 tx=0x7f2d28012000 comp rx=0 tx=0).stop 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.210+0000 7f2d36318740 1 -- 172.21.15.69:0/3628795070 shutdown_connections 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.210+0000 7f2d36318740 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376b0b70 0x5612376b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.210+0000 7f2d36318740 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.210+0000 7f2d36318740 1 --2- 172.21.15.69:0/3628795070 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376a5d70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.210+0000 7f2d36318740 1 -- 172.21.15.69:0/3628795070 >> 172.21.15.69:0/3628795070 conn(0x5612376a1f20 msgr2=0x5612376a2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.211+0000 7f2d36318740 1 -- 172.21.15.69:0/3628795070 shutdown_connections 2024-08-02T17:54:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.211+0000 7f2d36318740 1 -- 172.21.15.69:0/3628795070 wait complete. 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.211+0000 7f2d36318740 1 -- start start 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.211+0000 7f2d36318740 1 Processor -- start 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d36318740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d36318740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a62b0 0x5612376adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d36318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 0x5612376ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d36318740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5612376b7b50 con 0x5612376a62b0 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d36318740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5612375a97c0 con 0x5612376a5990 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d36318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5612376b7cd0 con 0x5612376b0b70 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a62b0 0x5612376adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a62b0 0x5612376adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50508/0 (socket says 172.21.15.69:50508) 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 -- 172.21.15.69:0/1627493892 learned_addr learned my addr 172.21.15.69:0/1627493892 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:54.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d36318740 10 client.0 _ll_register_callbacks cb 0x5612375b2f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d324b7640 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 0x5612376ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 -- 172.21.15.69:0/1627493892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 msgr2=0x5612376ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 0x5612376ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 -- 172.21.15.69:0/1627493892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 msgr2=0x5612376ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376ada20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 -- 172.21.15.69:0/1627493892 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612373e5960 con 0x5612376a62b0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d36318740 1 -- 172.21.15.69:0/1627493892 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612376191d0 con 0x5612376a62b0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d314b5640 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a62b0 0x5612376adf60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c09b7e0 tx=0x7f2d1c09bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00938f4fad34c1d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d157fa640 10 client.17511 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d157fa640 1 -- 172.21.15.69:0/1627493892 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c00fb30 con 0x5612376a62b0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.212+0000 7f2d157fa640 1 -- 172.21.15.69:0/1627493892 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f2d1c098150 con 0x5612376a62b0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d157fa640 1 -- 172.21.15.69:0/1627493892 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d1c0ba280 con 0x5612376a62b0 2024-08-02T17:54:54.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d36318740 10 client.17511 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d36318740 1 -- 172.21.15.69:0/1627493892 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5612376c3b10 con 0x5612376a62b0 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d157fa640 1 -- 172.21.15.69:0/1627493892 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d1c0b4ce0 con 0x5612376a62b0 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d36318740 10 client.17511 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d36318740 10 client.17511 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d36318740 20 client.17511 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d36318740 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5612376c4ec0 0x5612376c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d36318740 4 client.17511 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.213+0000 7f2d36318740 1 -- 172.21.15.69:0/1627493892 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612376136e0 con 0x5612376c4ec0 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.214+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5612376c4ec0 0x5612376c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.214+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5612376c4ec0 0x5612376c72b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=672e21e2eeaea6b3 server_cookie=bbea460cb393b4b4 in_seq=0 out_seq=0 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.214+0000 7f2d157fa640 10 client.17511 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d157fa640 1 -- 172.21.15.69:0/1627493892 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5612376136e0 con 0x5612376c4ec0 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d157fa640 10 client.17511 handle_command_reply: tid=0 2024-08-02T17:54:54.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 client.17511 shutdown 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 -- 172.21.15.69:0/1627493892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a62b0 msgr2=0x5612376adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a62b0 0x5612376adf60 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c09b7e0 tx=0x7f2d1c09bae0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 -- 172.21.15.69:0/1627493892 shutdown_connections 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a62b0 0x5612376adf60 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5612376c4ec0 0x5612376c72b0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 0x5612376ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 --2- 172.21.15.69:0/1627493892 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 -- 172.21.15.69:0/1627493892 >> 172.21.15.69:0/1627493892 conn(0x5612376a1f20 msgr2=0x5612376a7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d157fa640 0 client.17511 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 -- 172.21.15.69:0/1627493892 shutdown_connections 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 1 -- 172.21.15.69:0/1627493892 wait complete. 2024-08-02T17:54:54.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.215+0000 7f2d36318740 20 client.17511 trim_cache size 0 max 16384 2024-08-02T17:54:54.235 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:54.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.236+0000 7f2d36318740 1 Processor -- start 2024-08-02T17:54:54.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.236+0000 7f2d36318740 1 -- start start 2024-08-02T17:54:54.238 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:54 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:54:54.238 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:54 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:54.238 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:54 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:54:54.238 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:54 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:54.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.236+0000 7f2d36318740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.236+0000 7f2d36318740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a62b0 0x5612376ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.236+0000 7f2d36318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 0x5612376cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.236+0000 7f2d36318740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1af540 con 0x5612376a5990 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.236+0000 7f2d36318740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1ace10 con 0x5612376a62b0 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.236+0000 7f2d36318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1aac60 con 0x5612376b0b70 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d31cb6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d31cb6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50524/0 (socket says 172.21.15.69:50524) 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d31cb6640 1 -- 172.21.15.69:0/2601135326 learned_addr learned my addr 172.21.15.69:0/2601135326 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d314b5640 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a62b0 0x5612376ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d324b7640 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 0x5612376cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d314b5640 1 -- 172.21.15.69:0/2601135326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 msgr2=0x5612376cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:54.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d314b5640 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 0x5612376cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d314b5640 1 -- 172.21.15.69:0/2601135326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 msgr2=0x5612376af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d314b5640 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d314b5640 1 -- 172.21.15.69:0/2601135326 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612376ac9d0 con 0x5612376a62b0 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d314b5640 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a62b0 0x5612376ccb60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00cfd0 tx=0x7f2d1c09d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80a36b1f4f102374 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d177fe640 1 -- 172.21.15.69:0/2601135326 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c065060 con 0x5612376a62b0 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d177fe640 1 -- 172.21.15.69:0/2601135326 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612376191d0 con 0x5612376a62b0 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.237+0000 7f2d177fe640 1 -- 172.21.15.69:0/2601135326 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c0a4500 con 0x5612376a62b0 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.238+0000 7f2d36318740 1 -- 172.21.15.69:0/2601135326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a62b0 msgr2=0x5612376ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.238+0000 7f2d36318740 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a62b0 0x5612376ccb60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00cfd0 tx=0x7f2d1c09d000 comp rx=0 tx=0).stop 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 -- 172.21.15.69:0/2601135326 shutdown_connections 2024-08-02T17:54:54.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376b0b70 0x5612376cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a62b0 0x5612376ccb60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 --2- 172.21.15.69:0/2601135326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376a5990 0x5612376af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 -- 172.21.15.69:0/2601135326 >> 172.21.15.69:0/2601135326 conn(0x5612376a1f20 msgr2=0x5612376af280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:54.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 -- 172.21.15.69:0/2601135326 shutdown_connections 2024-08-02T17:54:54.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 -- 172.21.15.69:0/2601135326 wait complete. 2024-08-02T17:54:54.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 -- start start 2024-08-02T17:54:54.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 Processor -- start 2024-08-02T17:54:54.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.239+0000 7f2d36318740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d36318740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d36318740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376b0b70 0x5612376b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d36318740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1ab2d0 con 0x5612376b0b70 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d36318740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1ac740 con 0x5612376a5990 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d36318740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c1ab9a0 con 0x5612376a62b0 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d36318740 10 client.0 _ll_register_callbacks cb 0x7f2d1c09b510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376b0b70 0x5612376b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d31cb6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376b0b70 0x5612376b98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:50528/0 (socket says 172.21.15.69:50528) 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 -- 172.21.15.69:0/1066522584 learned_addr learned my addr 172.21.15.69:0/1066522584 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d314b5640 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 -- 172.21.15.69:0/1066522584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 msgr2=0x5612376b93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 -- 172.21.15.69:0/1066522584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 msgr2=0x5612376b8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376b8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 -- 172.21.15.69:0/1066522584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56123772ed30 con 0x5612376b0b70 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376b8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d314b5640 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d324b7640 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376b0b70 0x5612376b98f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009cd0 tx=0x7f2d24031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c19e74f6107058 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.240+0000 7f2d157fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d157fa640 1 -- 172.21.15.69:0/1066522584 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24048020 con 0x5612376b0b70 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d157fa640 1 -- 172.21.15.69:0/1066522584 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f2d24003d80 con 0x5612376b0b70 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d36318740 1 -- 172.21.15.69:0/1066522584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612376191d0 con 0x5612376b0b70 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d157fa640 1 -- 172.21.15.69:0/1066522584 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d24053050 con 0x5612376b0b70 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d36318740 10 client.17517 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d36318740 1 -- 172.21.15.69:0/1066522584 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561237730030 con 0x5612376b0b70 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d157fa640 1 -- 172.21.15.69:0/1066522584 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612376191d0 con 0x5612376b0b70 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d157fa640 1 -- 172.21.15.69:0/1066522584 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d24030070 con 0x5612376b0b70 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d36318740 10 client.17517 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d36318740 10 client.17517 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:54:54.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d36318740 20 client.17517 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:54.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.241+0000 7f2d36318740 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561237730800 0x561237732c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:54.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.242+0000 7f2d36318740 4 client.17517 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "81a0063a", "release": true, "await": true}] 2024-08-02T17:54:54.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.242+0000 7f2d36318740 1 -- 172.21.15.69:0/1066522584 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "81a0063a", "release": true, "await": true}) -- 0x5612375905a0 con 0x561237730800 2024-08-02T17:54:54.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.242+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561237730800 0x561237732c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:54.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.242+0000 7f2d157fa640 10 client.17517 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:54.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.242+0000 7f2d31cb6640 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561237730800 0x561237732c20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c53b87fdf9cfa439 server_cookie=366c2df116c855c9 in_seq=0 out_seq=0 2024-08-02T17:54:54.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d157fa640 1 -- 172.21.15.69:0/1066522584 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5612375905a0 con 0x561237730800 2024-08-02T17:54:54.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d157fa640 10 client.17517 handle_command_reply: tid=0 2024-08-02T17:54:54.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 client.17517 shutdown 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 -- 172.21.15.69:0/1066522584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376b0b70 msgr2=0x5612376b98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376b0b70 0x5612376b98f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009cd0 tx=0x7f2d24031c30 comp rx=0 tx=0).stop 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 -- 172.21.15.69:0/1066522584 shutdown_connections 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5612376b0b70 0x5612376b98f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561237730800 0x561237732c20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5612376a62b0 0x5612376b93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 --2- 172.21.15.69:0/1066522584 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5612376a5990 0x5612376b8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d157fa640 0 client.17517 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 -- 172.21.15.69:0/1066522584 >> 172.21.15.69:0/1066522584 conn(0x5612376a1f20 msgr2=0x5612376a1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.339+0000 7f2d36318740 1 -- 172.21.15.69:0/1066522584 shutdown_connections 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.340+0000 7f2d36318740 1 -- 172.21.15.69:0/1066522584 wait complete. 2024-08-02T17:54:54.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.340+0000 7f2d36318740 20 client.17517 trim_cache size 0 max 16384 2024-08-02T17:54:54.343 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:54:54.343 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:54:54.343 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:54:54.343 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 28, 2024-08-02T17:54:54.343 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:54:54.343 INFO:teuthology.orchestra.run.smithi069.stdout: "81a0063a": { 2024-08-02T17:54:54.343 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 28, 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.4, 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:54:54.344 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.340+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d180a2cf0 msgr2=0x7f2d180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.340+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d180a2cf0 0x7f2d180a51b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2d280058c0 tx=0x7f2d2800a250 comp rx=0 tx=0).stop 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.340+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 msgr2=0x7f2d2c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 0x7f2d2c1985e0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00c920 tx=0x7f2d1c00cde0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3871037497 shutdown_connections 2024-08-02T17:54:54.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d2c102260 0x7f2d2c1985e0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d180a2cf0 0x7f2d180a51b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d2c1078c0 0x7f2d2c19c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 --2- 172.21.15.69:0/3871037497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d2c1018c0 0x7f2d2c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:54.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3871037497 >> 172.21.15.69:0/3871037497 conn(0x7f2d2c0fd5e0 msgr2=0x7f2d2c0fdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:54.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3871037497 shutdown_connections 2024-08-02T17:54:54.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:54.341+0000 7f2d33f1a640 1 -- 172.21.15.69:0/3871037497 wait complete. 2024-08-02T17:54:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:54 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:54:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:54 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:54 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:54:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:54 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:54 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:54:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:54 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:54 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:54:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:54 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:54:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 465.0 s, 148.7 tps, lat 1515.250 ms stddev 3729.660 2024-08-02T17:54:55.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:55 smithi069 ceph-mon[32746]: pgmap v261: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-08-02T17:54:55.309 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:55 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:54:55.342 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '81a0063a', total seconds elapsed: 24.8 2024-08-02T17:54:55.342 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.4 2024-08-02T17:54:55.343 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:54:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:55 smithi077 ceph-mon[37108]: pgmap v261: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-08-02T17:54:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:55 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:54:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:55 smithi083 ceph-mon[37031]: pgmap v261: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-08-02T17:54:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:55 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:54:55.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.610+0000 7ff72bed2640 1 -- 172.21.15.69:0/126695450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff72411ae70 msgr2=0x7ff72411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:55.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.610+0000 7ff72bed2640 1 --2- 172.21.15.69:0/126695450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff72411ae70 0x7ff72411d260 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7ff71c01acb0 tx=0x7ff71c03b920 comp rx=0 tx=0).stop 2024-08-02T17:54:55.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.610+0000 7ff72bed2640 1 -- 172.21.15.69:0/126695450 shutdown_connections 2024-08-02T17:54:55.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.610+0000 7ff72bed2640 1 --2- 172.21.15.69:0/126695450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff72411ae70 0x7ff72411d260 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.610+0000 7ff72bed2640 1 --2- 172.21.15.69:0/126695450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff724109560 0x7ff7241099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.610+0000 7ff72bed2640 1 --2- 172.21.15.69:0/126695450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff724109180 0x7ff72410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.610+0000 7ff72bed2640 1 -- 172.21.15.69:0/126695450 >> 172.21.15.69:0/126695450 conn(0x7ff72406de70 msgr2=0x7ff72406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:55.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.612+0000 7ff72bed2640 1 -- 172.21.15.69:0/126695450 shutdown_connections 2024-08-02T17:54:55.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.612+0000 7ff72bed2640 1 -- 172.21.15.69:0/126695450 wait complete. 2024-08-02T17:54:55.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.613+0000 7ff72bed2640 1 Processor -- start 2024-08-02T17:54:55.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.617+0000 7ff72bed2640 1 -- start start 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.617+0000 7ff72bed2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff724109180 0x7ff72411ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.617+0000 7ff72bed2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff724109560 0x7ff724118ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.617+0000 7ff72bed2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff724119400 0x7ff7241198b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.617+0000 7ff72bed2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff724113110 con 0x7ff724109560 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.617+0000 7ff72bed2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff724112f90 con 0x7ff724119400 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.617+0000 7ff72bed2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff724113290 con 0x7ff724109180 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.618+0000 7ff72a46f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff724119400 0x7ff7241198b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.618+0000 7ff72a46f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff724119400 0x7ff7241198b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45870/0 (socket says 172.21.15.69:45870) 2024-08-02T17:54:55.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.618+0000 7ff72a46f640 1 -- 172.21.15.69:0/1636042170 learned_addr learned my addr 172.21.15.69:0/1636042170 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:55.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.620+0000 7ff72a46f640 1 -- 172.21.15.69:0/1636042170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff724109180 msgr2=0x7ff72411ce20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:54:55.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.620+0000 7ff72a46f640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff724109180 0x7ff72411ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.620+0000 7ff72a46f640 1 -- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff724109560 msgr2=0x7ff724118ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:55.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.620+0000 7ff72a46f640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff724109560 0x7ff724118ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.620+0000 7ff72a46f640 1 -- 172.21.15.69:0/1636042170 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7241aaea0 con 0x7ff724119400 2024-08-02T17:54:55.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.621+0000 7ff72a46f640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff724119400 0x7ff7241198b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff71c006010 tx=0x7ff71c003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:55.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.623+0000 7ff71affd640 1 -- 172.21.15.69:0/1636042170 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c045050 con 0x7ff724119400 2024-08-02T17:54:55.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.624+0000 7ff71affd640 1 -- 172.21.15.69:0/1636042170 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c0406c0 con 0x7ff724119400 2024-08-02T17:54:55.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.624+0000 7ff71affd640 1 -- 172.21.15.69:0/1636042170 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c041a90 con 0x7ff724119400 2024-08-02T17:54:55.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.625+0000 7ff72bed2640 1 -- 172.21.15.69:0/1636042170 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7241ab0d0 con 0x7ff724119400 2024-08-02T17:54:55.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.625+0000 7ff72bed2640 1 -- 172.21.15.69:0/1636042170 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7241ab530 con 0x7ff724119400 2024-08-02T17:54:55.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.628+0000 7ff71affd640 1 -- 172.21.15.69:0/1636042170 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff71c048020 con 0x7ff724119400 2024-08-02T17:54:55.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.631+0000 7ff72e2d0740 1 Processor -- start 2024-08-02T17:54:55.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.633+0000 7ff72e2d0740 1 -- start start 2024-08-02T17:54:55.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.634+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.634+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.634+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.634+0000 7ff72e2d0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e20f511980 con 0x55e20f5acb30 2024-08-02T17:54:55.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.634+0000 7ff72e2d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e20f4a9a70 con 0x55e20f5ac210 2024-08-02T17:54:55.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.634+0000 7ff72e2d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e20f50d940 con 0x55e20f5b73f0 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.634+0000 7ff72946d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.635+0000 7ff72946d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53248/0 (socket says 172.21.15.69:53248) 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.635+0000 7ff72946d640 1 -- 172.21.15.69:0/2026501512 learned_addr learned my addr 172.21.15.69:0/2026501512 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.635+0000 7ff72946d640 1 --2- 172.21.15.69:0/2026501512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.636+0000 7ff72946d640 1 -- 172.21.15.69:0/2026501512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 msgr2=0x55e20f5ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.636+0000 7ff72946d640 1 --2- 172.21.15.69:0/2026501512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.636+0000 7ff72946d640 1 -- 172.21.15.69:0/2026501512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 msgr2=0x55e20f5b6eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.636+0000 7ff72946d640 1 --2- 172.21.15.69:0/2026501512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5b6eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.636+0000 7ff72946d640 1 -- 172.21.15.69:0/2026501512 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20f50c6e0 con 0x55e20f5b73f0 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.636+0000 7ff71affd640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7100a2f00 0x7ff7100a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.636+0000 7ff71affd640 1 -- 172.21.15.69:0/1636042170 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff71c101020 con 0x7ff724119400 2024-08-02T17:54:55.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.636+0000 7ff72946d640 1 --2- 172.21.15.69:0/2026501512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b97e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff720001d10 tx=0x7ff720009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ec78b9f0223275f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:55.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.637+0000 7ff70ffff640 1 -- 172.21.15.69:0/2026501512 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720012040 con 0x55e20f5b73f0 2024-08-02T17:54:55.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.637+0000 7ff70ffff640 1 -- 172.21.15.69:0/2026501512 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff720008900 con 0x55e20f5b73f0 2024-08-02T17:54:55.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.637+0000 7ff70ffff640 1 -- 172.21.15.69:0/2026501512 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720004b30 con 0x55e20f5b73f0 2024-08-02T17:54:55.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.638+0000 7ff72e2d0740 1 -- 172.21.15.69:0/2026501512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 msgr2=0x55e20f5b97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:55.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.638+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/2026501512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b97e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff720001d10 tx=0x7ff720009d80 comp rx=0 tx=0).stop 2024-08-02T17:54:55.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.642+0000 7ff729c6e640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7100a2f00 0x7ff7100a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.645+0000 7ff72e2d0740 1 -- 172.21.15.69:0/2026501512 shutdown_connections 2024-08-02T17:54:55.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.645+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/2026501512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.645+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/2026501512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b97e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.645+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/2026501512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.645+0000 7ff72e2d0740 1 -- 172.21.15.69:0/2026501512 >> 172.21.15.69:0/2026501512 conn(0x55e20f5a87a0 msgr2=0x55e20f5a8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:55.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.646+0000 7ff72e2d0740 1 -- 172.21.15.69:0/2026501512 shutdown_connections 2024-08-02T17:54:55.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.646+0000 7ff72e2d0740 1 -- 172.21.15.69:0/2026501512 wait complete. 2024-08-02T17:54:55.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.646+0000 7ff72e2d0740 1 -- start start 2024-08-02T17:54:55.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.646+0000 7ff72e2d0740 1 Processor -- start 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.647+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5ac210 0x55e20f5b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.647+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5acb30 0x55e20f5b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.647+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5b73f0 0x55e20f5b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.647+0000 7ff72e2d0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e20f5ba060 con 0x55e20f5ac210 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.647+0000 7ff72e2d0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e20f4ccaa0 con 0x55e20f5b73f0 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.647+0000 7ff72e2d0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e20f5ba1e0 con 0x55e20f5acb30 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.647+0000 7ff72e2d0740 10 client.0 _ll_register_callbacks cb 0x55e20f4bbe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5b73f0 0x55e20f5b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5b73f0 0x55e20f5b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45892/0 (socket says 172.21.15.69:45892) 2024-08-02T17:54:55.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 -- 172.21.15.69:0/3466564185 learned_addr learned my addr 172.21.15.69:0/3466564185 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:55.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 -- 172.21.15.69:0/3466564185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5acb30 msgr2=0x55e20f5b47f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5acb30 0x55e20f5b47f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 -- 172.21.15.69:0/3466564185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5ac210 msgr2=0x55e20f5b42b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5ac210 0x55e20f5b42b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 -- 172.21.15.69:0/3466564185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20f4b0650 con 0x55e20f5b73f0 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.648+0000 7ff72a46f640 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5b73f0 0x55e20f5b4d30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0c9550 tx=0x7ff71c01a9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcea7186d3e0591e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.649+0000 7ff70d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.649+0000 7ff70d7fa640 1 -- 172.21.15.69:0/3466564185 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c0d5d60 con 0x55e20f5b73f0 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.649+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3466564185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff720008900 con 0x55e20f5b73f0 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.649+0000 7ff70d7fa640 1 -- 172.21.15.69:0/3466564185 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff71c0d11b0 con 0x55e20f5b73f0 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.649+0000 7ff70d7fa640 1 -- 172.21.15.69:0/3466564185 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff71c0ed0f0 con 0x55e20f5b73f0 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.649+0000 7ff72e2d0740 10 client.27041 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:55.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.649+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3466564185 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e20f2833c0 con 0x55e20f5b73f0 2024-08-02T17:54:55.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.651+0000 7ff729c6e640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7100a2f00 0x7ff7100a53c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55e20f5a9600 tx=0x7ff714006bb0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:55.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.652+0000 7ff70d7fa640 1 -- 172.21.15.69:0/3466564185 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff720008900 con 0x55e20f5b73f0 2024-08-02T17:54:55.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.652+0000 7ff70d7fa640 1 -- 172.21.15.69:0/3466564185 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff71c0e7070 con 0x55e20f5b73f0 2024-08-02T17:54:55.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.652+0000 7ff72e2d0740 10 client.27041 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:55.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.652+0000 7ff72e2d0740 10 client.27041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:55.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.652+0000 7ff72e2d0740 20 client.27041 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:55.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.652+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e20f5c71f0 0x55e20f5c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.652+0000 7ff72e2d0740 4 client.27041 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:54:55.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.653+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3466564185 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e20f51eef0 con 0x55e20f5c71f0 2024-08-02T17:54:55.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.653+0000 7ff729c6e640 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e20f5c71f0 0x55e20f5c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.654+0000 7ff70d7fa640 10 client.27041 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:55.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.654+0000 7ff729c6e640 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e20f5c71f0 0x55e20f5c96b0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1710564407111d1 server_cookie=51aec4bc8b32ae5f in_seq=0 out_seq=0 2024-08-02T17:54:55.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.659+0000 7ff70d7fa640 1 -- 172.21.15.69:0/3466564185 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ff714013430 con 0x55e20f5c71f0 2024-08-02T17:54:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.659+0000 7ff70d7fa640 10 client.27041 handle_command_reply: tid=0 2024-08-02T17:54:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.659+0000 7ff72e2d0740 1 client.27041 shutdown 2024-08-02T17:54:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.659+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3466564185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5b73f0 msgr2=0x55e20f5b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.659+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5b73f0 0x55e20f5b4d30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff71c0c9550 tx=0x7ff71c01a9a0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.659+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3466564185 shutdown_connections 2024-08-02T17:54:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.659+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e20f5c71f0 0x55e20f5c96b0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.659+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5b73f0 0x55e20f5b4d30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.660+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5acb30 0x55e20f5b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.660+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3466564185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5ac210 0x55e20f5b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.660+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3466564185 >> 172.21.15.69:0/3466564185 conn(0x55e20f5a87a0 msgr2=0x55e20f5ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:55.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.660+0000 7ff70d7fa640 0 client.27041 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:55.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.662+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3466564185 shutdown_connections 2024-08-02T17:54:55.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.664+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3466564185 wait complete. 2024-08-02T17:54:55.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.664+0000 7ff72e2d0740 20 client.27041 trim_cache size 0 max 16384 2024-08-02T17:54:55.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.698+0000 7ff72e2d0740 1 Processor -- start 2024-08-02T17:54:55.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.698+0000 7ff72e2d0740 1 -- start start 2024-08-02T17:54:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.698+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff729c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff729c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45904/0 (socket says 172.21.15.69:45904) 2024-08-02T17:54:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff729c6e640 1 -- 172.21.15.69:0/3494923378 learned_addr learned my addr 172.21.15.69:0/3494923378 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff72946d640 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3494923378 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b8120 con 0x55e20f5acb30 2024-08-02T17:54:55.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3494923378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b59f0 con 0x55e20f5ac210 2024-08-02T17:54:55.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff72a46f640 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.699+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3494923378 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b3840 con 0x55e20f5b73f0 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.700+0000 7ff729c6e640 1 -- 172.21.15.69:0/3494923378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 msgr2=0x55e20f5b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.700+0000 7ff729c6e640 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.700+0000 7ff729c6e640 1 -- 172.21.15.69:0/3494923378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 msgr2=0x55e20f5cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.700+0000 7ff729c6e640 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.700+0000 7ff729c6e640 1 -- 172.21.15.69:0/3494923378 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20f5ac960 con 0x55e20f5ac210 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.700+0000 7ff729c6e640 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5cefc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55e20f5c9f30 tx=0x7ff714012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5b8b920ee241db8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.701+0000 7ff70f7fe640 1 -- 172.21.15.69:0/3494923378 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71402c070 con 0x55e20f5ac210 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.701+0000 7ff70f7fe640 1 -- 172.21.15.69:0/3494923378 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e20f51eef0 con 0x55e20f5ac210 2024-08-02T17:54:55.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.701+0000 7ff70f7fe640 1 -- 172.21.15.69:0/3494923378 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714027d30 con 0x55e20f5ac210 2024-08-02T17:54:55.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3494923378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 msgr2=0x55e20f5cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:55.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5cefc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55e20f5c9f30 tx=0x7ff714012000 comp rx=0 tx=0).stop 2024-08-02T17:54:55.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3494923378 shutdown_connections 2024-08-02T17:54:55.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/3494923378 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5cefc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3494923378 >> 172.21.15.69:0/3494923378 conn(0x55e20f5a87a0 msgr2=0x55e20f5ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:55.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3494923378 shutdown_connections 2024-08-02T17:54:55.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.702+0000 7ff72e2d0740 1 -- 172.21.15.69:0/3494923378 wait complete. 2024-08-02T17:54:55.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.703+0000 7ff72e2d0740 1 -- start start 2024-08-02T17:54:55.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.703+0000 7ff72e2d0740 1 Processor -- start 2024-08-02T17:54:55.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.704+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.704+0000 7ff72e2d0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.704+0000 7ff729c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.704+0000 7ff729c6e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45908/0 (socket says 172.21.15.69:45908) 2024-08-02T17:54:55.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.704+0000 7ff729c6e640 1 -- 172.21.15.69:0/1651383494 learned_addr learned my addr 172.21.15.69:0/1651383494 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:55.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.704+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.704+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b3eb0 con 0x55e20f5acb30 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff72a46f640 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b5320 con 0x55e20f5ac210 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff7241b4580 con 0x55e20f5b73f0 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff72946d640 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff72e2d0740 10 client.0 _ll_register_callbacks cb 0x7ff71c0c8d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff729c6e640 1 -- 172.21.15.69:0/1651383494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 msgr2=0x55e20f5b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff729c6e640 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff729c6e640 1 -- 172.21.15.69:0/1651383494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 msgr2=0x55e20f5b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:54:55.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff729c6e640 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff729c6e640 1 -- 172.21.15.69:0/1651383494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20f630fe0 con 0x55e20f5ac210 2024-08-02T17:54:55.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.705+0000 7ff72946d640 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:54:55.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.706+0000 7ff729c6e640 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5b13a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff71402a040 tx=0x7ff714028e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d216f4fe7a172d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:55.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.706+0000 7ff70d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:54:55.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.706+0000 7ff70d7fa640 1 -- 172.21.15.69:0/1651383494 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714037060 con 0x55e20f5ac210 2024-08-02T17:54:55.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.706+0000 7ff70d7fa640 1 -- 172.21.15.69:0/1651383494 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7ff714012860 con 0x55e20f5ac210 2024-08-02T17:54:55.709 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.706+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e20f51eef0 con 0x55e20f5ac210 2024-08-02T17:54:55.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.706+0000 7ff70d7fa640 1 -- 172.21.15.69:0/1651383494 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff71403e050 con 0x55e20f5ac210 2024-08-02T17:54:55.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.706+0000 7ff72e2d0740 10 client.27047 fetch_fsmap learned FSMap version 26 2024-08-02T17:54:55.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.706+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e20f632290 con 0x55e20f5ac210 2024-08-02T17:54:55.710 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.707+0000 7ff70d7fa640 1 -- 172.21.15.69:0/1651383494 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e20f51eef0 con 0x55e20f5ac210 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.707+0000 7ff70d7fa640 1 -- 172.21.15.69:0/1651383494 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff714026030 con 0x55e20f5ac210 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.707+0000 7ff72e2d0740 10 client.27047 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.707+0000 7ff72e2d0740 10 client.27047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.707+0000 7ff72e2d0740 20 client.27047 populate_metadata read hostname 'smithi069' 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.707+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e20f6328d0 0x55e20f634cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.707+0000 7ff72e2d0740 4 client.27047 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.707+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e20f519400 con 0x55e20f6328d0 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.708+0000 7ff72946d640 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e20f6328d0 0x55e20f634cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.708+0000 7ff72946d640 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e20f6328d0 0x55e20f634cf0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=572a73186b2c2e97 server_cookie=fb3a8d2c952ea9d8 in_seq=0 out_seq=0 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.708+0000 7ff70d7fa640 10 client.27047 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.708+0000 7ff70d7fa640 1 -- 172.21.15.69:0/1651383494 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e20f519400 con 0x55e20f6328d0 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.708+0000 7ff70d7fa640 10 client.27047 handle_command_reply: tid=0 2024-08-02T17:54:55.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.708+0000 7ff72e2d0740 1 client.27047 shutdown 2024-08-02T17:54:55.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.709+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 msgr2=0x55e20f5b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:55.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.709+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5b13a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff71402a040 tx=0x7ff714028e70 comp rx=0 tx=0).stop 2024-08-02T17:54:55.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.709+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 shutdown_connections 2024-08-02T17:54:55.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.709+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e20f5acb30 0x55e20f5b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.709+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e20f6328d0 0x55e20f634cf0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.709+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20f5b73f0 0x55e20f5b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.709+0000 7ff72e2d0740 1 --2- 172.21.15.69:0/1651383494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e20f5ac210 0x55e20f5b13a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.709+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 >> 172.21.15.69:0/1651383494 conn(0x55e20f5a87a0 msgr2=0x55e20f5a70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.710+0000 7ff70d7fa640 0 client.27047 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.710+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 shutdown_connections 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:54:55.713 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:54:55.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.710+0000 7ff72e2d0740 1 -- 172.21.15.69:0/1651383494 wait complete. 2024-08-02T17:54:55.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.710+0000 7ff72e2d0740 20 client.27047 trim_cache size 0 max 16384 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.711+0000 7ff72bed2640 1 -- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7100a2f00 msgr2=0x7ff7100a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.711+0000 7ff72bed2640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7100a2f00 0x7ff7100a53c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55e20f5a9600 tx=0x7ff714006bb0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 -- 172.21.15.69:0/1636042170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff724119400 msgr2=0x7ff7241198b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff724119400 0x7ff7241198b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff71c006010 tx=0x7ff71c003f10 comp rx=0 tx=0).stop 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 -- 172.21.15.69:0/1636042170 shutdown_connections 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff724109560 0x7ff724118ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff7100a2f00 0x7ff7100a53c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff724119400 0x7ff7241198b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 --2- 172.21.15.69:0/1636042170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff724109180 0x7ff72411ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:55.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 -- 172.21.15.69:0/1636042170 >> 172.21.15.69:0/1636042170 conn(0x7ff72406de70 msgr2=0x7ff724109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:55.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 -- 172.21.15.69:0/1636042170 shutdown_connections 2024-08-02T17:54:55.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:55.712+0000 7ff72bed2640 1 -- 172.21.15.69:0/1636042170 wait complete. 2024-08-02T17:54:56.406 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:54:56.407 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:54:56.458 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:57.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.061+0000 7fe430fee640 1 -- 172.21.15.69:0/938337511 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe424030e30 con 0x7fe42c0742c0 2024-08-02T17:54:57.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.061+0000 7fe432ff2640 1 -- 172.21.15.69:0/938337511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0742c0 msgr2=0x7fe42c0746a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:57.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.061+0000 7fe432ff2640 1 --2- 172.21.15.69:0/938337511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0742c0 0x7fe42c0746a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe424009be0 tx=0x7fe42402a770 comp rx=0 tx=0).stop 2024-08-02T17:54:57.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.061+0000 7fe432ff2640 1 -- 172.21.15.69:0/938337511 shutdown_connections 2024-08-02T17:54:57.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.062+0000 7fe432ff2640 1 --2- 172.21.15.69:0/938337511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c112700 0x7fe42c114af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.062+0000 7fe432ff2640 1 --2- 172.21.15.69:0/938337511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe42c074be0 0x7fe42c1121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.062+0000 7fe432ff2640 1 --2- 172.21.15.69:0/938337511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c0742c0 0x7fe42c0746a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.062+0000 7fe432ff2640 1 -- 172.21.15.69:0/938337511 >> 172.21.15.69:0/938337511 conn(0x7fe42c0fd470 msgr2=0x7fe42c0ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:57.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.062+0000 7fe432ff2640 1 -- 172.21.15.69:0/938337511 shutdown_connections 2024-08-02T17:54:57.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.062+0000 7fe432ff2640 1 -- 172.21.15.69:0/938337511 wait complete. 2024-08-02T17:54:57.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.062+0000 7fe432ff2640 1 Processor -- start 2024-08-02T17:54:57.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.062+0000 7fe432ff2640 1 -- start start 2024-08-02T17:54:57.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe432ff2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe42c0742c0 0x7fe42c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:57.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe432ff2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c074be0 0x7fe42c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:57.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe432ff2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c112700 0x7fe42c1a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:57.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe432ff2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c1181a0 con 0x7fe42c074be0 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe432ff2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c118020 con 0x7fe42c0742c0 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe432ff2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe42c118320 con 0x7fe42c112700 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe4317ef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c074be0 0x7fe42c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe431ff0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe42c0742c0 0x7fe42c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe4327f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c112700 0x7fe42c1a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe431ff0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe42c0742c0 0x7fe42c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45922/0 (socket says 172.21.15.69:45922) 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe431ff0640 1 -- 172.21.15.69:0/3062368653 learned_addr learned my addr 172.21.15.69:0/3062368653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe4327f1640 1 -- 172.21.15.69:0/3062368653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe42c0742c0 msgr2=0x7fe42c19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe4327f1640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe42c0742c0 0x7fe42c19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe4327f1640 1 -- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c074be0 msgr2=0x7fe42c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe4327f1640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c074be0 0x7fe42c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe4327f1640 1 -- 172.21.15.69:0/3062368653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42c1a3650 con 0x7fe42c112700 2024-08-02T17:54:57.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.063+0000 7fe4317ef640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c074be0 0x7fe42c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:54:57.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.064+0000 7fe4327f1640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c112700 0x7fe42c1a2f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00d650 tx=0x7fe41c00db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:57.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.064+0000 7fe422ffd640 1 -- 172.21.15.69:0/3062368653 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c017070 con 0x7fe42c112700 2024-08-02T17:54:57.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.064+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe42c1a3940 con 0x7fe42c112700 2024-08-02T17:54:57.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.064+0000 7fe422ffd640 1 -- 172.21.15.69:0/3062368653 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe41c009820 con 0x7fe42c112700 2024-08-02T17:54:57.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.064+0000 7fe422ffd640 1 -- 172.21.15.69:0/3062368653 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41c004c60 con 0x7fe42c112700 2024-08-02T17:54:57.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.064+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe42c1ab220 con 0x7fe42c112700 2024-08-02T17:54:57.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.067+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe42c0746a0 con 0x7fe42c112700 2024-08-02T17:54:57.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.068+0000 7fe422ffd640 1 -- 172.21.15.69:0/3062368653 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe41c0099d0 con 0x7fe42c112700 2024-08-02T17:54:57.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.068+0000 7fe422ffd640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4180a2d70 0x7fe4180a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:57.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.069+0000 7fe422ffd640 1 -- 172.21.15.69:0/3062368653 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fe41c0ce990 con 0x7fe42c112700 2024-08-02T17:54:57.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.069+0000 7fe431ff0640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4180a2d70 0x7fe4180a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:57.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.069+0000 7fe431ff0640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4180a2d70 0x7fe4180a5230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4240099a0 tx=0x7fe42402af90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:57.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.075+0000 7fe422ffd640 1 -- 172.21.15.69:0/3062368653 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe41c09ab50 con 0x7fe42c112700 2024-08-02T17:54:57.128 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:57.276 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:57 smithi069 ceph-mon[32746]: pgmap v262: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-08-02T17:54:57.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:57 smithi077 ceph-mon[37108]: pgmap v262: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-08-02T17:54:57.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.441+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe42c104be0 con 0x7fe42c112700 2024-08-02T17:54:57.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.442+0000 7fe422ffd640 1 -- 172.21.15.69:0/3062368653 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fe41c09ee50 con 0x7fe42c112700 2024-08-02T17:54:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:54:57.444 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:54:57.444 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:54:57.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4180a2d70 msgr2=0x7fe4180a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:57.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4180a2d70 0x7fe4180a5230 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe4240099a0 tx=0x7fe42402af90 comp rx=0 tx=0).stop 2024-08-02T17:54:57.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c112700 msgr2=0x7fe42c1a2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:57.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c112700 0x7fe42c1a2f70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe41c00d650 tx=0x7fe41c00db10 comp rx=0 tx=0).stop 2024-08-02T17:54:57.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 shutdown_connections 2024-08-02T17:54:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe42c074be0 0x7fe42c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4180a2d70 0x7fe4180a5230 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe42c112700 0x7fe42c1a2f70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 --2- 172.21.15.69:0/3062368653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe42c0742c0 0x7fe42c19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 >> 172.21.15.69:0/3062368653 conn(0x7fe42c0fd470 msgr2=0x7fe42c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.445+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 shutdown_connections 2024-08-02T17:54:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.446+0000 7fe432ff2640 1 -- 172.21.15.69:0/3062368653 wait complete. 2024-08-02T17:54:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:57 smithi083 ceph-mon[37031]: pgmap v262: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 1.2 MiB/s wr, 141 op/s 2024-08-02T17:54:57.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.974+0000 7f2bb5f21640 1 -- 172.21.15.69:0/950914953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bb0109560 msgr2=0x7f2bb01099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:57.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.974+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/950914953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bb0109560 0x7f2bb01099c0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009860 tx=0x7f2ba402a380 comp rx=0 tx=0).stop 2024-08-02T17:54:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.979+0000 7f2bb5f21640 1 -- 172.21.15.69:0/950914953 shutdown_connections 2024-08-02T17:54:57.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.979+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/950914953 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bb0109560 0x7f2bb01099c0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.979+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/950914953 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bb011ae70 0x7f2bb011d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.979+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/950914953 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bb0109180 0x7f2bb010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:57.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.979+0000 7f2bb5f21640 1 -- 172.21.15.69:0/950914953 >> 172.21.15.69:0/950914953 conn(0x7f2bb006de70 msgr2=0x7f2bb006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.984+0000 7f2bb5f21640 1 -- 172.21.15.69:0/950914953 shutdown_connections 2024-08-02T17:54:57.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.985+0000 7f2bb5f21640 1 -- 172.21.15.69:0/950914953 wait complete. 2024-08-02T17:54:57.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.986+0000 7f2bb5f21640 1 Processor -- start 2024-08-02T17:54:57.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.993+0000 7f2bb5f21640 1 -- start start 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.993+0000 7f2bb5f21640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bb0109180 0x7f2bb01a7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.993+0000 7f2bb5f21640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bb0109560 0x7f2bb01a7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.993+0000 7f2bb5f21640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bb011ae70 0x7f2bb01ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.993+0000 7f2bb5f21640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0120b60 con 0x7f2bb011ae70 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.993+0000 7f2bb5f21640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01209e0 con 0x7f2bb0109180 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.993+0000 7f2bb5f21640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0120ce0 con 0x7f2bb0109560 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.994+0000 7f2baf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bb0109180 0x7f2bb01a7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.994+0000 7f2baf7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bb0109180 0x7f2bb01a7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45940/0 (socket says 172.21.15.69:45940) 2024-08-02T17:54:57.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.994+0000 7f2baf7fe640 1 -- 172.21.15.69:0/139567997 learned_addr learned my addr 172.21.15.69:0/139567997 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:54:58.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.995+0000 7f2baffff640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bb011ae70 0x7f2bb01ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:58.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.998+0000 7f2baeffd640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bb0109560 0x7f2bb01a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:58.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.998+0000 7f2baf7fe640 1 -- 172.21.15.69:0/139567997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bb0109560 msgr2=0x7f2bb01a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:58.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.998+0000 7f2baf7fe640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bb0109560 0x7f2bb01a7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:58.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.998+0000 7f2baf7fe640 1 -- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bb011ae70 msgr2=0x7f2bb01ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:58.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.998+0000 7f2baf7fe640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bb011ae70 0x7f2bb01ab9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:58.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.998+0000 7f2baf7fe640 1 -- 172.21.15.69:0/139567997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb01ac0d0 con 0x7f2bb0109180 2024-08-02T17:54:58.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:57.998+0000 7f2baf7fe640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bb0109180 0x7f2bb01a7120 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000d950 tx=0x7f2ba000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.002+0000 7f2bacff9640 1 -- 172.21.15.69:0/139567997 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0017070 con 0x7f2bb0109180 2024-08-02T17:54:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.002+0000 7f2bacff9640 1 -- 172.21.15.69:0/139567997 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba0009c30 con 0x7f2bb0109180 2024-08-02T17:54:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.002+0000 7f2bacff9640 1 -- 172.21.15.69:0/139567997 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000f6b0 con 0x7f2bb0109180 2024-08-02T17:54:58.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.002+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb01ac3c0 con 0x7f2bb0109180 2024-08-02T17:54:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.002+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb01b3ca0 con 0x7f2bb0109180 2024-08-02T17:54:58.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.006+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b9c005290 con 0x7f2bb0109180 2024-08-02T17:54:58.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.018+0000 7f2bacff9640 1 -- 172.21.15.69:0/139567997 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2ba0004020 con 0x7f2bb0109180 2024-08-02T17:54:58.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.018+0000 7f2bacff9640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b980a2f00 0x7f2b980a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:54:58.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.019+0000 7f2bacff9640 1 -- 172.21.15.69:0/139567997 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7f2ba00cf290 con 0x7f2bb0109180 2024-08-02T17:54:58.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.019+0000 7f2bacff9640 1 -- 172.21.15.69:0/139567997 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2ba0020020 con 0x7f2bb0109180 2024-08-02T17:54:58.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.022+0000 7f2baeffd640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b980a2f00 0x7f2b980a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:54:58.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.031+0000 7f2baeffd640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b980a2f00 0x7f2b980a53c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40095f0 tx=0x7f2ba4000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:54:58.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:58 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3062368653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:58.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:58 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3062368653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:58.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.377+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b9c005580 con 0x7f2bb0109180 2024-08-02T17:54:58.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.383+0000 7f2bacff9640 1 -- 172.21.15.69:0/139567997 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f2ba009bb60 con 0x7f2bb0109180 2024-08-02T17:54:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:54:58.385 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:54:58.386 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:54:58.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b980a2f00 msgr2=0x7f2b980a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:58.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b980a2f00 0x7f2b980a53c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40095f0 tx=0x7f2ba4000bc0 comp rx=0 tx=0).stop 2024-08-02T17:54:58.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bb0109180 msgr2=0x7f2bb01a7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bb0109180 0x7f2bb01a7120 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000d950 tx=0x7f2ba000de10 comp rx=0 tx=0).stop 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 shutdown_connections 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2bb011ae70 0x7f2bb01ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2b980a2f00 0x7f2b980a53c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2bb0109560 0x7f2bb01a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 --2- 172.21.15.69:0/139567997 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2bb0109180 0x7f2bb01a7120 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 >> 172.21.15.69:0/139567997 conn(0x7f2bb006de70 msgr2=0x7f2bb011bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 shutdown_connections 2024-08-02T17:54:58.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:54:58.387+0000 7f2bb5f21640 1 -- 172.21.15.69:0/139567997 wait complete. 2024-08-02T17:54:58.435 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0' 2024-08-02T17:54:58.435 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 140.4 --await-for 45.0 2024-08-02T17:54:58.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:58 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3062368653' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:59 smithi077 ceph-mon[37108]: pgmap v263: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-08-02T17:54:59.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:54:59 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/139567997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:59.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:59 smithi069 ceph-mon[32746]: pgmap v263: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-08-02T17:54:59.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:54:59 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/139567997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:59 smithi083 ceph-mon[37031]: pgmap v263: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-08-02T17:54:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:54:59 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/139567997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:54:59.751 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:54:59.848 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:54:59.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 470.0 s, 867.0 tps, lat 102.899 ms stddev 282.444 2024-08-02T17:55:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.198+0000 7fbc1713d640 1 -- 172.21.15.69:0/379781541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc080a5170 msgr2=0x7fbc080b7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.198+0000 7fbc1713d640 1 --2- 172.21.15.69:0/379781541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc080a5170 0x7fbc080b7030 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbc0001a910 tx=0x7fbc0003b500 comp rx=0 tx=0).stop 2024-08-02T17:55:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.199+0000 7fbc1713d640 1 -- 172.21.15.69:0/379781541 shutdown_connections 2024-08-02T17:55:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.200+0000 7fbc1713d640 1 --2- 172.21.15.69:0/379781541 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc080a5170 0x7fbc080b7030 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.200+0000 7fbc1713d640 1 --2- 172.21.15.69:0/379781541 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc080b7570 0x7fbc080b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.200+0000 7fbc1713d640 1 --2- 172.21.15.69:0/379781541 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc080a4850 0x7fbc080a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.200+0000 7fbc1713d640 1 -- 172.21.15.69:0/379781541 >> 172.21.15.69:0/379781541 conn(0x7fbc0801a140 msgr2=0x7fbc0801a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:00.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.208+0000 7fbc1713d640 1 -- 172.21.15.69:0/379781541 shutdown_connections 2024-08-02T17:55:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.212+0000 7fbc1713d640 1 -- 172.21.15.69:0/379781541 wait complete. 2024-08-02T17:55:00.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.212+0000 7fbc1713d640 1 Processor -- start 2024-08-02T17:55:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc1713d640 1 -- start start 2024-08-02T17:55:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc1713d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc080a4850 0x7fbc080b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc1713d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc080a5170 0x7fbc080b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc1713d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc080b7570 0x7fbc080b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc1713d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080af250 con 0x7fbc080a4850 2024-08-02T17:55:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc1713d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080af0d0 con 0x7fbc080b7570 2024-08-02T17:55:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc1713d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080af3d0 con 0x7fbc080a5170 2024-08-02T17:55:00.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc156da640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc080b7570 0x7fbc080b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc156da640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc080b7570 0x7fbc080b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45958/0 (socket says 172.21.15.69:45958) 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.213+0000 7fbc156da640 1 -- 172.21.15.69:0/476962691 learned_addr learned my addr 172.21.15.69:0/476962691 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.214+0000 7fbc156da640 1 -- 172.21.15.69:0/476962691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc080a5170 msgr2=0x7fbc080b2db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.214+0000 7fbc156da640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc080a5170 0x7fbc080b2db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.214+0000 7fbc156da640 1 -- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc080a4850 msgr2=0x7fbc080b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.214+0000 7fbc156da640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc080a4850 0x7fbc080b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.214+0000 7fbc156da640 1 -- 172.21.15.69:0/476962691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc080b3a70 con 0x7fbc080b7570 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.214+0000 7fbc156da640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc080b7570 0x7fbc080b32f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc100514c0 tx=0x7fbc10073a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:00.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.215+0000 7fbc0dffb640 1 -- 172.21.15.69:0/476962691 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100759c0 con 0x7fbc080b7570 2024-08-02T17:55:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.215+0000 7fbc1713d640 1 -- 172.21.15.69:0/476962691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc081473c0 con 0x7fbc080b7570 2024-08-02T17:55:00.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.215+0000 7fbc0dffb640 1 -- 172.21.15.69:0/476962691 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc10075b60 con 0x7fbc080b7570 2024-08-02T17:55:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.215+0000 7fbc0dffb640 1 -- 172.21.15.69:0/476962691 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10074620 con 0x7fbc080b7570 2024-08-02T17:55:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.219+0000 7fbc1713d640 1 -- 172.21.15.69:0/476962691 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc08147830 con 0x7fbc080b7570 2024-08-02T17:55:00.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.245+0000 7fbc19f6a740 1 Processor -- start 2024-08-02T17:55:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.245+0000 7fbc19f6a740 1 -- start start 2024-08-02T17:55:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.245+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee885ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.245+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885f260 0x55bee88695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.245+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8869b00 0x55bee886bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.245+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bee87c29b0 con 0x55bee885e940 2024-08-02T17:55:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.245+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bee876a800 con 0x55bee8869b00 2024-08-02T17:55:00.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.245+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bee87be970 con 0x55bee885f260 2024-08-02T17:55:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.248+0000 7fbc0dffb640 1 -- 172.21.15.69:0/476962691 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc100747c0 con 0x7fbc080b7570 2024-08-02T17:55:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.249+0000 7fbc0dffb640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbf80a2f00 0x7fbbf80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.249+0000 7fbc0dffb640 1 -- 172.21.15.69:0/476962691 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fbc10133670 con 0x7fbc080b7570 2024-08-02T17:55:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.251+0000 7fbc14ed9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885f260 0x55bee88695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.251+0000 7fbc14ed9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885f260 0x55bee88695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53346/0 (socket says 172.21.15.69:53346) 2024-08-02T17:55:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.251+0000 7fbc14ed9640 1 -- 172.21.15.69:0/2682994108 learned_addr learned my addr 172.21.15.69:0/2682994108 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.251+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8869b00 0x55bee886bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.251+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee885ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.253+0000 7fbc156da640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbf80a2f00 0x7fbbf80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.253+0000 7fbc0ffff640 1 -- 172.21.15.69:0/2682994108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885f260 msgr2=0x55bee88695c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.253+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885f260 0x55bee88695c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.253+0000 7fbc0ffff640 1 -- 172.21.15.69:0/2682994108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 msgr2=0x55bee885ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.253+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee885ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.253+0000 7fbc0ffff640 1 -- 172.21.15.69:0/2682994108 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee84d6210 con 0x55bee8869b00 2024-08-02T17:55:00.255 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.253+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885f260 0x55bee88695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.254+0000 7fbc156da640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbf80a2f00 0x7fbbf80a53c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbc10050a90 tx=0x7fbc100681e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.255+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8869b00 0x55bee886bef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc0003baf0 tx=0x7fbc00000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b03a8dcce7854a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.258+0000 7fbbf6ffd640 1 -- 172.21.15.69:0/2682994108 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00056070 con 0x55bee8869b00 2024-08-02T17:55:00.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.258+0000 7fbbf6ffd640 1 -- 172.21.15.69:0/2682994108 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc000048c0 con 0x55bee8869b00 2024-08-02T17:55:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.259+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2682994108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8869b00 msgr2=0x55bee886bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.259+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8869b00 0x55bee886bef0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc0003baf0 tx=0x7fbc00000bc0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.262+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2682994108 shutdown_connections 2024-08-02T17:55:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.262+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8869b00 0x55bee886bef0 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbc0003baf0 tx=0x7fbc00000bc0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.262+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885f260 0x55bee88695c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.262+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2682994108 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee885ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.262+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2682994108 >> 172.21.15.69:0/2682994108 conn(0x55bee885aeb0 msgr2=0x55bee885b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.268+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2682994108 shutdown_connections 2024-08-02T17:55:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.271+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2682994108 wait complete. 2024-08-02T17:55:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.271+0000 7fbc19f6a740 1 -- start start 2024-08-02T17:55:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.271+0000 7fbc19f6a740 1 Processor -- start 2024-08-02T17:55:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.272+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8866af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.272+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8867030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.272+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee8867570 0x55bee8875cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.272+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bee886c770 con 0x55bee885e940 2024-08-02T17:55:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.272+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bee877e500 con 0x55bee885f260 2024-08-02T17:55:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.272+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bee886c8f0 con 0x55bee8867570 2024-08-02T17:55:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.272+0000 7fbc19f6a740 10 client.0 _ll_register_callbacks cb 0x55bee84d5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.278+0000 7fbc14ed9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8867030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.279+0000 7fbc14ed9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8867030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45980/0 (socket says 172.21.15.69:45980) 2024-08-02T17:55:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.279+0000 7fbc14ed9640 1 -- 172.21.15.69:0/238493594 learned_addr learned my addr 172.21.15.69:0/238493594 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.280+0000 7fbc14ed9640 1 -- 172.21.15.69:0/238493594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee8867570 msgr2=0x55bee8875cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:55:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.280+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee8867570 0x55bee8875cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.280+0000 7fbc14ed9640 1 -- 172.21.15.69:0/238493594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 msgr2=0x55bee8866af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.280+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8866af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.280+0000 7fbc14ed9640 1 -- 172.21.15.69:0/238493594 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee85a2620 con 0x55bee885f260 2024-08-02T17:55:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.280+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8867030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400d230 tx=0x7fbc0400d6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804cf13875afb155 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.282+0000 7fbbeffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:55:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.282+0000 7fbbeffff640 1 -- 172.21.15.69:0/238493594 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc040125f0 con 0x55bee885f260 2024-08-02T17:55:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.282+0000 7fbbeffff640 1 -- 172.21.15.69:0/238493594 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fbc04012790 con 0x55bee885f260 2024-08-02T17:55:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.284+0000 7fbc19f6a740 1 -- 172.21.15.69:0/238493594 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc000048c0 con 0x55bee885f260 2024-08-02T17:55:00.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.285+0000 7fbbeffff640 1 -- 172.21.15.69:0/238493594 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc04012990 con 0x55bee885f260 2024-08-02T17:55:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.287+0000 7fbc19f6a740 10 client.27065 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:00.289 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.287+0000 7fbc19f6a740 1 -- 172.21.15.69:0/238493594 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bee8878a90 con 0x55bee885f260 2024-08-02T17:55:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.290+0000 7fbbeffff640 1 -- 172.21.15.69:0/238493594 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc000048c0 con 0x55bee885f260 2024-08-02T17:55:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.290+0000 7fbbeffff640 1 -- 172.21.15.69:0/238493594 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc0401d450 con 0x55bee885f260 2024-08-02T17:55:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.291+0000 7fbc19f6a740 10 client.27065 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.291+0000 7fbc19f6a740 10 client.27065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.291+0000 7fbc19f6a740 20 client.27065 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.292+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bee8879fa0 0x55bee887c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.292+0000 7fbc19f6a740 4 client.27065 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:00.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.292+0000 7fbc19f6a740 1 -- 172.21.15.69:0/238493594 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bee87d00e0 con 0x55bee8879fa0 2024-08-02T17:55:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.295+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bee8879fa0 0x55bee887c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.299+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bee8879fa0 0x55bee887c3f0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5265edb67a1c755f server_cookie=686d88a93c99af36 in_seq=0 out_seq=0 2024-08-02T17:55:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.300+0000 7fbbeffff640 10 client.27065 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.302+0000 7fbbeffff640 1 -- 172.21.15.69:0/238493594 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bee87d00e0 con 0x55bee8879fa0 2024-08-02T17:55:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.302+0000 7fbbeffff640 10 client.27065 handle_command_reply: tid=0 2024-08-02T17:55:00.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.302+0000 7fbc19f6a740 1 client.27065 shutdown 2024-08-02T17:55:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.308+0000 7fbc19f6a740 1 -- 172.21.15.69:0/238493594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 msgr2=0x55bee8867030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.308+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8867030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400d230 tx=0x7fbc0400d6f0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 1 -- 172.21.15.69:0/238493594 shutdown_connections 2024-08-02T17:55:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bee8879fa0 0x55bee887c3f0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee8867570 0x55bee8875cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8867030 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/238493594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8866af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbbeffff640 0 client.27065 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 1 -- 172.21.15.69:0/238493594 >> 172.21.15.69:0/238493594 conn(0x55bee885aeb0 msgr2=0x55bee8860c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 1 -- 172.21.15.69:0/238493594 shutdown_connections 2024-08-02T17:55:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 1 -- 172.21.15.69:0/238493594 wait complete. 2024-08-02T17:55:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.309+0000 7fbc19f6a740 20 client.27065 trim_cache size 0 max 16384 2024-08-02T17:55:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.332+0000 7fbc19f6a740 1 Processor -- start 2024-08-02T17:55:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.332+0000 7fbc19f6a740 1 -- start start 2024-08-02T17:55:00.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8875ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8881d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee8867570 0x55bee8882240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc012a90 con 0x55bee885e940 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc010360 con 0x55bee885f260 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc0ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8875ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc00e1b0 con 0x55bee8867570 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc0ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8875ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37880/0 (socket says 172.21.15.69:37880) 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc0ffff640 1 -- 172.21.15.69:0/863695397 learned_addr learned my addr 172.21.15.69:0/863695397 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8881d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc0ffff640 1 -- 172.21.15.69:0/863695397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee8867570 msgr2=0x55bee8882240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee8867570 0x55bee8882240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc0ffff640 1 -- 172.21.15.69:0/863695397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 msgr2=0x55bee8881d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8881d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc0ffff640 1 -- 172.21.15.69:0/863695397 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bee8862690 con 0x55bee885e940 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.333+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8881d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.334+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8875ac0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbc000058c0 tx=0x7fbc0003ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f77c61002fa249d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.334+0000 7fbbf67fc640 1 -- 172.21.15.69:0/863695397 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00060040 con 0x55bee885e940 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.334+0000 7fbbf67fc640 1 -- 172.21.15.69:0/863695397 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00004780 con 0x55bee885e940 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.334+0000 7fbbf67fc640 1 -- 172.21.15.69:0/863695397 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00056040 con 0x55bee885e940 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 -- 172.21.15.69:0/863695397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 msgr2=0x55bee8875ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8875ac0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbc000058c0 tx=0x7fbc0003ba00 comp rx=0 tx=0).stop 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 -- 172.21.15.69:0/863695397 shutdown_connections 2024-08-02T17:55:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee8867570 0x55bee8882240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee885f260 0x55bee8881d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/863695397 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885e940 0x55bee8875ac0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 -- 172.21.15.69:0/863695397 >> 172.21.15.69:0/863695397 conn(0x55bee885aeb0 msgr2=0x55bee88653a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 -- 172.21.15.69:0/863695397 shutdown_connections 2024-08-02T17:55:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 -- 172.21.15.69:0/863695397 wait complete. 2024-08-02T17:55:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 -- start start 2024-08-02T17:55:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 Processor -- start 2024-08-02T17:55:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885e940 0x55bee886d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885f260 0x55bee886d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.335+0000 7fbc19f6a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8867570 0x55bee88dcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc00e820 con 0x55bee885f260 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc00fc90 con 0x55bee8867570 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc19f6a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbbfc00eef0 con 0x55bee885e940 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc19f6a740 10 client.0 _ll_register_callbacks cb 0x7fbc0400b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885f260 0x55bee886d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885f260 0x55bee886d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37894/0 (socket says 172.21.15.69:37894) 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 -- 172.21.15.69:0/2089387384 learned_addr learned my addr 172.21.15.69:0/2089387384 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885e940 0x55bee886d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8867570 0x55bee88dcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 -- 172.21.15.69:0/2089387384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885e940 msgr2=0x55bee886d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885e940 0x55bee886d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 -- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8867570 msgr2=0x55bee88dcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8867570 0x55bee88dcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 -- 172.21.15.69:0/2089387384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bee88dd370 con 0x55bee885f260 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8867570 0x55bee88dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc14ed9640 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885f260 0x55bee886d8e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbc04000c00 tx=0x7fbc0400bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8705d52383c31cce server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbbeffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbbeffff640 1 -- 172.21.15.69:0/2089387384 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0402c020 con 0x55bee885f260 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2089387384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc00004780 con 0x55bee885f260 2024-08-02T17:55:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbbeffff640 1 -- 172.21.15.69:0/2089387384 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fbc0401da90 con 0x55bee885f260 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbbeffff640 1 -- 172.21.15.69:0/2089387384 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc04037050 con 0x55bee885f260 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc19f6a740 10 client.17550 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.336+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2089387384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bee88e47b0 con 0x55bee885f260 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbbeffff640 1 -- 172.21.15.69:0/2089387384 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc00004780 con 0x55bee885f260 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbbeffff640 1 -- 172.21.15.69:0/2089387384 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc04025070 con 0x55bee885f260 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbc19f6a740 10 client.17550 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbc19f6a740 10 client.17550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbc19f6a740 20 client.17550 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bee88e4f20 0x55bee88e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbc19f6a740 4 client.17550 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2089387384 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bee87ca430 con 0x55bee88e4f20 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.337+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bee88e4f20 0x55bee88e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.338+0000 7fbc0ffff640 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bee88e4f20 0x55bee88e7340 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff3ccbcbe40b7b8 server_cookie=ae2c9379285c4bd3 in_seq=0 out_seq=0 2024-08-02T17:55:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.338+0000 7fbbeffff640 10 client.17550 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbbeffff640 1 -- 172.21.15.69:0/2089387384 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bee87ca430 con 0x55bee88e4f20 2024-08-02T17:55:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbbeffff640 10 client.17550 handle_command_reply: tid=0 2024-08-02T17:55:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbc19f6a740 1 client.17550 shutdown 2024-08-02T17:55:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2089387384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885f260 msgr2=0x55bee886d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885f260 0x55bee886d8e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fbc04000c00 tx=0x7fbc0400bd00 comp rx=0 tx=0).stop 2024-08-02T17:55:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2089387384 shutdown_connections 2024-08-02T17:55:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bee885f260 0x55bee886d8e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bee88e4f20 0x55bee88e7340 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bee8867570 0x55bee88dcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.420+0000 7fbbeffff640 0 client.17550 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.421+0000 7fbc19f6a740 1 --2- 172.21.15.69:0/2089387384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bee885e940 0x55bee886d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.421+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2089387384 >> 172.21.15.69:0/2089387384 conn(0x55bee885aeb0 msgr2=0x55bee8859ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.421+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2089387384 shutdown_connections 2024-08-02T17:55:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.421+0000 7fbc19f6a740 1 -- 172.21.15.69:0/2089387384 wait complete. 2024-08-02T17:55:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.421+0000 7fbc19f6a740 20 client.17550 trim_cache size 0 max 16384 2024-08-02T17:55:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:55:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:55:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:55:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.422+0000 7fbc1713d640 1 -- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbf80a2f00 msgr2=0x7fbbf80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.422+0000 7fbc1713d640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbf80a2f00 0x7fbbf80a53c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbc10050a90 tx=0x7fbc100681e0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.422+0000 7fbc1713d640 1 -- 172.21.15.69:0/476962691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc080b7570 msgr2=0x7fbc080b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.422+0000 7fbc1713d640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc080b7570 0x7fbc080b32f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc100514c0 tx=0x7fbc10073a90 comp rx=0 tx=0).stop 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.423+0000 7fbc1713d640 1 -- 172.21.15.69:0/476962691 shutdown_connections 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.423+0000 7fbc1713d640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbbf80a2f00 0x7fbbf80a53c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.423+0000 7fbc1713d640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc080b7570 0x7fbc080b32f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.423+0000 7fbc1713d640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc080a5170 0x7fbc080b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.423+0000 7fbc1713d640 1 --2- 172.21.15.69:0/476962691 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc080a4850 0x7fbc080b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:00.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.423+0000 7fbc1713d640 1 -- 172.21.15.69:0/476962691 >> 172.21.15.69:0/476962691 conn(0x7fbc0801a140 msgr2=0x7fbc080b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.426+0000 7fbc1713d640 1 -- 172.21.15.69:0/476962691 shutdown_connections 2024-08-02T17:55:00.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:00.427+0000 7fbc1713d640 1 -- 172.21.15.69:0/476962691 wait complete. 2024-08-02T17:55:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:01 smithi077 ceph-mon[37108]: pgmap v264: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.4 MiB/s wr, 217 op/s 2024-08-02T17:55:01.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:01 smithi069 ceph-mon[32746]: pgmap v264: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.4 MiB/s wr, 217 op/s 2024-08-02T17:55:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:01 smithi083 ceph-mon[37031]: pgmap v264: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.4 MiB/s wr, 217 op/s 2024-08-02T17:55:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:03 smithi077 ceph-mon[37108]: pgmap v265: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-08-02T17:55:03.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:03 smithi069 ceph-mon[32746]: pgmap v265: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-08-02T17:55:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:03 smithi083 ceph-mon[37031]: pgmap v265: 129 pgs: 129 active+clean; 3.3 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-08-02T17:55:04.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.703+0000 7fe92bceb640 1 -- 172.21.15.69:0/2296887045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe92411ae70 msgr2=0x7fe92411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.703+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2296887045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe92411ae70 0x7fe92411d260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe91c01acb0 tx=0x7fe91c03b920 comp rx=0 tx=0).stop 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.707+0000 7fe92bceb640 1 -- 172.21.15.69:0/2296887045 shutdown_connections 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.707+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2296887045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe924109560 0x7fe9241099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.707+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2296887045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe92411ae70 0x7fe92411d260 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.707+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2296887045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe924109180 0x7fe92410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.707+0000 7fe92bceb640 1 -- 172.21.15.69:0/2296887045 >> 172.21.15.69:0/2296887045 conn(0x7fe92406de70 msgr2=0x7fe92406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.715+0000 7fe92bceb640 1 -- 172.21.15.69:0/2296887045 shutdown_connections 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.717+0000 7fe92bceb640 1 -- 172.21.15.69:0/2296887045 wait complete. 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.718+0000 7fe92bceb640 1 Processor -- start 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.720+0000 7fe92bceb640 1 -- start start 2024-08-02T17:55:04.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.720+0000 7fe92bceb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe924109180 0x7fe92411a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.720+0000 7fe92bceb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe924109560 0x7fe924116600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.720+0000 7fe92bceb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe92411ae70 0x7fe924116b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.720+0000 7fe92bceb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe924113110 con 0x7fe924109180 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.720+0000 7fe92bceb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe924112f90 con 0x7fe924109560 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.720+0000 7fe92bceb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe924113290 con 0x7fe92411ae70 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.721+0000 7fe929a87640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe924109180 0x7fe92411a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.721+0000 7fe929a87640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe924109180 0x7fe92411a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37920/0 (socket says 172.21.15.69:37920) 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.721+0000 7fe929a87640 1 -- 172.21.15.69:0/2467868563 learned_addr learned my addr 172.21.15.69:0/2467868563 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:04.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.722+0000 7fe929286640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe924109560 0x7fe924116600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.723+0000 7fe929a87640 1 -- 172.21.15.69:0/2467868563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe92411ae70 msgr2=0x7fe924116b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.723+0000 7fe929a87640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe92411ae70 0x7fe924116b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.724+0000 7fe929a87640 1 -- 172.21.15.69:0/2467868563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe924109560 msgr2=0x7fe924116600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.724+0000 7fe929a87640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe924109560 0x7fe924116600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.724+0000 7fe929a87640 1 -- 172.21.15.69:0/2467868563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe924117270 con 0x7fe924109180 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.730+0000 7fe929a87640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe924109180 0x7fe92411a560 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe91400b450 tx=0x7fe91400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.734+0000 7fe91affd640 1 -- 172.21.15.69:0/2467868563 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe914015020 con 0x7fe924109180 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.735+0000 7fe92bceb640 1 -- 172.21.15.69:0/2467868563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9241aaea0 con 0x7fe924109180 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.735+0000 7fe92bceb640 1 -- 172.21.15.69:0/2467868563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9241ab240 con 0x7fe924109180 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.735+0000 7fe91affd640 1 -- 172.21.15.69:0/2467868563 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe914011d70 con 0x7fe924109180 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.736+0000 7fe91affd640 1 -- 172.21.15.69:0/2467868563 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91400f340 con 0x7fe924109180 2024-08-02T17:55:04.720 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.743+0000 7fe92e0e9740 1 Processor -- start 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.746+0000 7fe91affd640 1 -- 172.21.15.69:0/2467868563 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe914018020 con 0x7fe924109180 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.747+0000 7fe91affd640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9100a2de0 0x7fe9100a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.747+0000 7fe91affd640 1 -- 172.21.15.69:0/2467868563 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fe9140cf0d0 con 0x7fe924109180 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.753+0000 7fe92e0e9740 1 -- start start 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.754+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d37873510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.754+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.754+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 0x558d37880700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.754+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558d377d8a70 con 0x558d37873a50 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.754+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d37770780 con 0x558d37873130 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.754+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d377d4a30 con 0x558d3787e310 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.757+0000 7fe929a87640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d37873510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.757+0000 7fe929286640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.757+0000 7fe929286640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37934/0 (socket says 172.21.15.69:37934) 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.757+0000 7fe929286640 1 -- 172.21.15.69:0/4231252556 learned_addr learned my addr 172.21.15.69:0/4231252556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.758+0000 7fe92a288640 1 --2- 172.21.15.69:0/4231252556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 0x558d37880700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.758+0000 7fe929286640 1 -- 172.21.15.69:0/4231252556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 msgr2=0x558d37880700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.758+0000 7fe929286640 1 --2- 172.21.15.69:0/4231252556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 0x558d37880700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.758+0000 7fe929286640 1 -- 172.21.15.69:0/4231252556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 msgr2=0x558d37873510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.758+0000 7fe929286640 1 --2- 172.21.15.69:0/4231252556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d37873510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.758+0000 7fe929286640 1 -- 172.21.15.69:0/4231252556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d3754ab90 con 0x558d37873a50 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.761+0000 7fe92a288640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9100a2de0 0x7fe9100a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.763+0000 7fe929286640 1 --2- 172.21.15.69:0/4231252556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ddd0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe92000b7c0 tx=0x7fe92000bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef09615665892dd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:04.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.763+0000 7fe90ffff640 1 -- 172.21.15.69:0/4231252556 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92000be10 con 0x558d37873a50 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.764+0000 7fe90ffff640 1 -- 172.21.15.69:0/4231252556 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe920018070 con 0x558d37873a50 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.764+0000 7fe90ffff640 1 -- 172.21.15.69:0/4231252556 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9200123e0 con 0x558d37873a50 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.765+0000 7fe92e0e9740 1 -- 172.21.15.69:0/4231252556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 msgr2=0x558d3787ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.765+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/4231252556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ddd0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fe92000b7c0 tx=0x7fe92000bbc0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.768+0000 7fe92e0e9740 1 -- 172.21.15.69:0/4231252556 shutdown_connections 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.768+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/4231252556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ddd0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.768+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/4231252556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 0x558d37880700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.768+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/4231252556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d37873510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.768+0000 7fe92e0e9740 1 -- 172.21.15.69:0/4231252556 >> 172.21.15.69:0/4231252556 conn(0x558d3786f6c0 msgr2=0x558d3786faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.773+0000 7fe92a288640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9100a2de0 0x7fe9100a52a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe91c01ac00 tx=0x7fe91c01a5d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:04.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.773+0000 7fe92e0e9740 1 -- 172.21.15.69:0/4231252556 shutdown_connections 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.773+0000 7fe92e0e9740 1 -- 172.21.15.69:0/4231252556 wait complete. 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.773+0000 7fe92e0e9740 1 -- start start 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.773+0000 7fe92e0e9740 1 Processor -- start 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d3787b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe929a87640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d3787b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe929a87640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d3787b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:46030/0 (socket says 172.21.15.69:46030) 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe929a87640 1 -- 172.21.15.69:0/3287492623 learned_addr learned my addr 172.21.15.69:0/3287492623 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 0x558d3787bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558d37880ee0 con 0x558d37873a50 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558d37793990 con 0x558d37873130 2024-08-02T17:55:04.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558d37881060 con 0x558d3787e310 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.774+0000 7fe92e0e9740 10 client.0 _ll_register_callbacks cb 0x558d374e8090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.775+0000 7fe929286640 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.780+0000 7fe929286640 1 -- 172.21.15.69:0/3287492623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 msgr2=0x558d3787bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.780+0000 7fe929286640 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 0x558d3787bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.780+0000 7fe929286640 1 -- 172.21.15.69:0/3287492623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 msgr2=0x558d3787b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.780+0000 7fe929286640 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d3787b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.780+0000 7fe929286640 1 -- 172.21.15.69:0/3287492623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d377926c0 con 0x558d37873a50 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.780+0000 7fe929286640 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787b690 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x558d37870520 tx=0x7fe9200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bb422858fef5f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.780+0000 7fe90d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.780+0000 7fe929a87640 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d3787b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:04.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.781+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3287492623 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92000dd60 con 0x558d37873a50 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.781+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3287492623 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fe920024420 con 0x558d37873a50 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.786+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe920018070 con 0x558d37873a50 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.790+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3287492623 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe92000be10 con 0x558d37873a50 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.795+0000 7fe92e0e9740 10 client.17559 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.795+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d3788cb70 con 0x558d37873a50 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.799+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3287492623 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe920024b40 con 0x558d37873a50 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.799+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3287492623 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe92001c7f0 con 0x558d37873a50 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.799+0000 7fe92e0e9740 10 client.17559 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.799+0000 7fe92e0e9740 10 client.17559 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.799+0000 7fe92e0e9740 20 client.17559 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:04.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.799+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558d3788dec0 0x558d378902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.799+0000 7fe92e0e9740 4 client.17559 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.799+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe920018070 con 0x558d3788dec0 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.801+0000 7fe929a87640 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558d3788dec0 0x558d378902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.803+0000 7fe929a87640 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558d3788dec0 0x558d378902b0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0e28910eda1c27c server_cookie=28d75e80839dbfd6 in_seq=0 out_seq=0 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.804+0000 7fe90d7fa640 10 client.17559 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.806+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3287492623 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe920018070 con 0x558d3788dec0 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.807+0000 7fe90d7fa640 10 client.17559 handle_command_reply: tid=0 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.807+0000 7fe92e0e9740 1 client.17559 shutdown 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 msgr2=0x558d3787b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787b690 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x558d37870520 tx=0x7fe9200046c0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 shutdown_connections 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787b690 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558d3788dec0 0x558d378902b0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d3787e310 0x558d3787bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe90d7fa640 0 client.17559 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3287492623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d37873130 0x558d3787b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 >> 172.21.15.69:0/3287492623 conn(0x558d3786f6c0 msgr2=0x558d378753f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 shutdown_connections 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.810+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3287492623 wait complete. 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.811+0000 7fe92e0e9740 20 client.17559 trim_cache size 0 max 16384 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92e0e9740 1 Processor -- start 2024-08-02T17:55:04.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92e0e9740 1 -- start start 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37875b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37895940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe9080129c0 con 0x558d37873a50 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe908010290 con 0x558d3787e310 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe90800e0e0 con 0x558d37873130 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe929286640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe929286640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37958/0 (socket says 172.21.15.69:37958) 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe929286640 1 -- 172.21.15.69:0/1129629555 learned_addr learned my addr 172.21.15.69:0/1129629555 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe92a288640 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37895940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.842+0000 7fe929a87640 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37875b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe929286640 1 -- 172.21.15.69:0/1129629555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 msgr2=0x558d37875b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe929286640 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37875b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe929286640 1 -- 172.21.15.69:0/1129629555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 msgr2=0x558d37895940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe929286640 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37895940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe929286640 1 -- 172.21.15.69:0/1129629555 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d378796a0 con 0x558d37873a50 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe929a87640 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37875b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe92a288640 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37895940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe929286640 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ba20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe92000bfd0 tx=0x7fe92000bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da7bb7e876fa6ca server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe90f7fe640 1 -- 172.21.15.69:0/1129629555 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92000a030 con 0x558d37873a50 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe90f7fe640 1 -- 172.21.15.69:0/1129629555 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe920018040 con 0x558d37873a50 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.843+0000 7fe90f7fe640 1 -- 172.21.15.69:0/1129629555 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92002d6a0 con 0x558d37873a50 2024-08-02T17:55:04.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 -- 172.21.15.69:0/1129629555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 msgr2=0x558d3787ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ba20 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe92000bfd0 tx=0x7fe92000bbc0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 -- 172.21.15.69:0/1129629555 shutdown_connections 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d3787ba20 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37895940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/1129629555 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37875b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 -- 172.21.15.69:0/1129629555 >> 172.21.15.69:0/1129629555 conn(0x558d3786f6c0 msgr2=0x558d3787d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 -- 172.21.15.69:0/1129629555 shutdown_connections 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 -- 172.21.15.69:0/1129629555 wait complete. 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 -- start start 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.844+0000 7fe92e0e9740 1 Processor -- start 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37882230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d37882770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92e0e9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37882cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe90800e750 con 0x558d37873a50 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe90800fbc0 con 0x558d3787e310 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92e0e9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe90800ee20 con 0x558d37873130 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92e0e9740 10 client.0 _ll_register_callbacks cb 0x7fe92000b610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929286640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d37882770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929286640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d37882770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37962/0 (socket says 172.21.15.69:37962) 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929286640 1 -- 172.21.15.69:0/3989480766 learned_addr learned my addr 172.21.15.69:0/3989480766 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929a87640 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37882230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92a288640 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37882cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929286640 1 -- 172.21.15.69:0/3989480766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 msgr2=0x558d37882230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929286640 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37882230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929286640 1 -- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 msgr2=0x558d37882cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929286640 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37882cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe929286640 1 -- 172.21.15.69:0/3989480766 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d378f7ab0 con 0x558d37873a50 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.845+0000 7fe92a288640 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37882cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe929286640 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d37882770 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe92000a000 tx=0x7fe92000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd97b9d811fa2dc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe90d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3989480766 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe920014df0 con 0x558d37873a50 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3989480766 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9568+0+0 (secure 0 0 0) 0x7fe92002d3d0 con 0x558d37873a50 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3989480766 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe920018040 con 0x558d37873a50 2024-08-02T17:55:04.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3989480766 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe920047050 con 0x558d37873a50 2024-08-02T17:55:04.734 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:55:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:55:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:55:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 30, 2024-08-02T17:55:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:55:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "65e61135": { 2024-08-02T17:55:04.734 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 30, 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.4, 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:55:04.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 10 client.17565 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:04.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3989480766 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558d378f8d70 con 0x558d37873a50 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3989480766 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe920018040 con 0x558d37873a50 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3989480766 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe92002dab0 con 0x558d37873a50 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 10 client.17565 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 10 client.17565 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 20 client.17565 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558d3788dec0 0x558d378fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 4 client.17565 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}] 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.846+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3989480766 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.4, "await_for": 45.0}) -- 0x558d377e61a0 con 0x558d3788dec0 2024-08-02T17:55:04.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.847+0000 7fe929a87640 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558d3788dec0 0x558d378fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:04.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.847+0000 7fe90d7fa640 10 client.17565 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:04.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:03.847+0000 7fe929a87640 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558d3788dec0 0x558d378fb4e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9dbaf5c243fd213 server_cookie=8f7cbfb1d029de2e in_seq=0 out_seq=0 2024-08-02T17:55:04.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.143+0000 7fe90d7fa640 1 -- 172.21.15.69:0/3989480766 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558d377e61a0 con 0x558d3788dec0 2024-08-02T17:55:04.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.143+0000 7fe90d7fa640 10 client.17565 handle_command_reply: tid=0 2024-08-02T17:55:04.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.143+0000 7fe92e0e9740 1 client.17565 shutdown 2024-08-02T17:55:04.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.143+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3989480766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 msgr2=0x558d37882770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:04.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.143+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d37882770 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fe92000a000 tx=0x7fe92000de80 comp rx=0 tx=0).stop 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3989480766 shutdown_connections 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558d37873a50 0x558d37882770 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558d3788dec0 0x558d378fb4e0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe90d7fa640 0 client.17565 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558d3787e310 0x558d37882cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 1 --2- 172.21.15.69:0/3989480766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558d37873130 0x558d37882230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3989480766 >> 172.21.15.69:0/3989480766 conn(0x558d3786f6c0 msgr2=0x558d378891c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3989480766 shutdown_connections 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 1 -- 172.21.15.69:0/3989480766 wait complete. 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.144+0000 7fe92e0e9740 20 client.17565 trim_cache size 0 max 16384 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 -- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9100a2de0 msgr2=0x7fe9100a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:04.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9100a2de0 0x7fe9100a52a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe91c01ac00 tx=0x7fe91c01a5d0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 -- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe924109180 msgr2=0x7fe92411a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe924109180 0x7fe92411a560 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe91400b450 tx=0x7fe91400b910 comp rx=0 tx=0).stop 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 -- 172.21.15.69:0/2467868563 shutdown_connections 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9100a2de0 0x7fe9100a52a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe92411ae70 0x7fe924116b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe924109560 0x7fe924116600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 --2- 172.21.15.69:0/2467868563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe924109180 0x7fe92411a560 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 -- 172.21.15.69:0/2467868563 >> 172.21.15.69:0/2467868563 conn(0x7fe92406de70 msgr2=0x7fe924070910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 -- 172.21.15.69:0/2467868563 shutdown_connections 2024-08-02T17:55:04.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:04.145+0000 7fe92bceb640 1 -- 172.21.15.69:0/2467868563 wait complete. 2024-08-02T17:55:04.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 475.0 s, 344.3 tps, lat 249.797 ms stddev 351.512 2024-08-02T17:55:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:05 smithi077 ceph-mon[37108]: pgmap v266: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.6 MiB/s wr, 572 op/s 2024-08-02T17:55:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:05 smithi069 ceph-mon[32746]: pgmap v266: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.6 MiB/s wr, 572 op/s 2024-08-02T17:55:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:05 smithi083 ceph-mon[37031]: pgmap v266: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 4.6 MiB/s wr, 572 op/s 2024-08-02T17:55:05.762 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:06.213 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:06 smithi069 ceph-mon[32746]: osdmap e100: 12 total, 12 up, 12 in 2024-08-02T17:55:06.213 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:06 smithi069 ceph-mon[32746]: pgmap v268: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 5.8 MiB/s wr, 710 op/s 2024-08-02T17:55:06.242 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '65e61135', quiesce took 10.9 seconds. Will release after: 9.499999999999998 2024-08-02T17:55:06.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:06 smithi077 ceph-mon[37108]: osdmap e100: 12 total, 12 up, 12 in 2024-08-02T17:55:06.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:06 smithi077 ceph-mon[37108]: pgmap v268: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 5.8 MiB/s wr, 710 op/s 2024-08-02T17:55:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:06 smithi083 ceph-mon[37031]: osdmap e100: 12 total, 12 up, 12 in 2024-08-02T17:55:06.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:06 smithi083 ceph-mon[37031]: pgmap v268: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 5.8 MiB/s wr, 710 op/s 2024-08-02T17:55:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 -- 172.21.15.69:0/2258778642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 msgr2=0x7fc76c107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 --2- 172.21.15.69:0/2258778642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 0x7fc76c107c90 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fc768009930 tx=0x7fc76802a420 comp rx=0 tx=0).stop 2024-08-02T17:55:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 -- 172.21.15.69:0/2258778642 shutdown_connections 2024-08-02T17:55:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 --2- 172.21.15.69:0/2258778642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76c102250 0x7fc76c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 --2- 172.21.15.69:0/2258778642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc76c1018b0 0x7fc76c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 --2- 172.21.15.69:0/2258778642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 0x7fc76c107c90 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 -- 172.21.15.69:0/2258778642 >> 172.21.15.69:0/2258778642 conn(0x7fc76c0fd5f0 msgr2=0x7fc76c0ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:06.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 -- 172.21.15.69:0/2258778642 shutdown_connections 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 -- 172.21.15.69:0/2258778642 wait complete. 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.654+0000 7fc773fa9640 1 Processor -- start 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc773fa9640 1 -- start start 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc773fa9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc76c1018b0 0x7fc76c074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc773fa9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76c102250 0x7fc76c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc773fa9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 0x7fc76c074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc773fa9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c111cb0 con 0x7fc76c1078b0 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc773fa9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c111b30 con 0x7fc76c1018b0 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc773fa9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c111e30 con 0x7fc76c102250 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 0x7fc76c074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 0x7fc76c074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51148/0 (socket says 172.21.15.69:51148) 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 -- 172.21.15.69:0/95474915 learned_addr learned my addr 172.21.15.69:0/95474915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:06.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc771d45640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc76c1018b0 0x7fc76c074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc771544640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76c102250 0x7fc76c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 -- 172.21.15.69:0/95474915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76c102250 msgr2=0x7fc76c074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76c102250 0x7fc76c074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 -- 172.21.15.69:0/95474915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc76c1018b0 msgr2=0x7fc76c074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc76c1018b0 0x7fc76c074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 -- 172.21.15.69:0/95474915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c1a6690 con 0x7fc76c1078b0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.655+0000 7fc772546640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 0x7fc76c074ca0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc76400cad0 tx=0x7fc76400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.656+0000 7fc762ffd640 1 -- 172.21.15.69:0/95474915 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400ae20 con 0x7fc76c1078b0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.656+0000 7fc762ffd640 1 -- 172.21.15.69:0/95474915 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc764005020 con 0x7fc76c1078b0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.656+0000 7fc762ffd640 1 -- 172.21.15.69:0/95474915 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400f6f0 con 0x7fc76c1078b0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.656+0000 7fc773fa9640 1 -- 172.21.15.69:0/95474915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c1a6980 con 0x7fc76c1078b0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.656+0000 7fc773fa9640 1 -- 172.21.15.69:0/95474915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c1a6f10 con 0x7fc76c1078b0 2024-08-02T17:55:06.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc7763a7740 1 Processor -- start 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc762ffd640 1 -- 172.21.15.69:0/95474915 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc7640051d0 con 0x7fc76c1078b0 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc7763a7740 1 -- start start 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x55804402f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x558044039c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc7763a7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558043f94a40 con 0x55804403a1d0 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc7763a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558043f2c9d0 con 0x55804402f910 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.658+0000 7fc7763a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558043f90a00 con 0x55804402eff0 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51158/0 (socket says 172.21.15.69:51158) 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 -- 172.21.15.69:0/117159151 learned_addr learned my addr 172.21.15.69:0/117159151 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:06.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771d45640 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x558044039c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x55804402f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc762ffd640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7580a2d70 0x7fc7580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 -- 172.21.15.69:0/117159151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 msgr2=0x55804402f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x55804402f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 -- 172.21.15.69:0/117159151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 msgr2=0x558044039c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x558044039c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 -- 172.21.15.69:0/117159151 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558043f33650 con 0x55804403a1d0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc772546640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7580a2d70 0x7fc7580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc771544640 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403c5c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc75c012530 tx=0x7fc75c00b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c8f378aaae0779 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc762ffd640 1 -- 172.21.15.69:0/95474915 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7fc7640cfa70 con 0x7fc76c1078b0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc757fff640 1 -- 172.21.15.69:0/117159151 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c004220 con 0x55804403a1d0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc757fff640 1 -- 172.21.15.69:0/117159151 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75c0043c0 con 0x55804403a1d0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.659+0000 7fc757fff640 1 -- 172.21.15.69:0/117159151 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c010910 con 0x55804403a1d0 2024-08-02T17:55:06.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 -- 172.21.15.69:0/117159151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 msgr2=0x55804403c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc772546640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7580a2d70 0x7fc7580a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc76409bb60 tx=0x7fc76404b040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403c5c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fc75c012530 tx=0x7fc75c00b650 comp rx=0 tx=0).stop 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 -- 172.21.15.69:0/117159151 shutdown_connections 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403c5c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x558044039c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 --2- 172.21.15.69:0/117159151 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x55804402f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 -- 172.21.15.69:0/117159151 >> 172.21.15.69:0/117159151 conn(0x55804402b580 msgr2=0x55804402b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 -- 172.21.15.69:0/117159151 shutdown_connections 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.660+0000 7fc7763a7740 1 -- 172.21.15.69:0/117159151 wait complete. 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 1 -- start start 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 1 Processor -- start 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402eff0 0x558044037070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402f910 0x5580440375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044037af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55804403cda0 con 0x55804403a1d0 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558043f4fb20 con 0x55804402eff0 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55804403cf20 con 0x55804402f910 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc7763a7740 10 client.0 _ll_register_callbacks cb 0x558043f4dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc771544640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044037af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc771544640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044037af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51160/0 (socket says 172.21.15.69:51160) 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc771544640 1 -- 172.21.15.69:0/3922542349 learned_addr learned my addr 172.21.15.69:0/3922542349 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc771d45640 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402f910 0x5580440375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.661+0000 7fc771544640 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402eff0 0x558044037070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc771544640 1 -- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402f910 msgr2=0x5580440375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc771544640 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402f910 0x5580440375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc771544640 1 -- 172.21.15.69:0/3922542349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402eff0 msgr2=0x558044037070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:06.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc771544640 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402eff0 0x558044037070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc771544640 1 -- 172.21.15.69:0/3922542349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558043d063c0 con 0x55804403a1d0 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc771d45640 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402f910 0x5580440375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc771544640 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044037af0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc75c005fd0 tx=0x7fc75c0104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452eff9f8c65d02b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7557fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7557fa640 1 -- 172.21.15.69:0/3922542349 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c021070 con 0x55804403a1d0 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7557fa640 1 -- 172.21.15.69:0/3922542349 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7fc75c003800 con 0x55804403a1d0 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 1 -- 172.21.15.69:0/3922542349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75c0043c0 con 0x55804403a1d0 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7557fa640 1 -- 172.21.15.69:0/3922542349 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc75c003a00 con 0x55804403a1d0 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 10 client.17577 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:06.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 1 -- 172.21.15.69:0/3922542349 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580440489a0 con 0x55804403a1d0 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7557fa640 1 -- 172.21.15.69:0/3922542349 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75c0043c0 con 0x55804403a1d0 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7557fa640 1 -- 172.21.15.69:0/3922542349 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc75c01c030 con 0x55804403a1d0 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 10 client.17577 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 10 client.17577 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 20 client.17577 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558044049cf0 0x55804404c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 4 client.17577 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.662+0000 7fc7763a7740 1 -- 172.21.15.69:0/3922542349 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558043fa1fb0 con 0x558044049cf0 2024-08-02T17:55:06.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.663+0000 7fc771d45640 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558044049cf0 0x55804404c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.663+0000 7fc771d45640 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558044049cf0 0x55804404c0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63246d5e4c80651c server_cookie=f074b6febb64e9d9 in_seq=0 out_seq=0 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.663+0000 7fc7557fa640 10 client.17577 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7557fa640 1 -- 172.21.15.69:0/3922542349 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558043fa1fb0 con 0x558044049cf0 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7557fa640 10 client.17577 handle_command_reply: tid=0 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 client.17577 shutdown 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 -- 172.21.15.69:0/3922542349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 msgr2=0x558044037af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044037af0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fc75c005fd0 tx=0x7fc75c0104d0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 -- 172.21.15.69:0/3922542349 shutdown_connections 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x558044049cf0 0x55804404c0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044037af0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402f910 0x5580440375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3922542349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402eff0 0x558044037070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7557fa640 0 client.17577 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:55:06.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 -- 172.21.15.69:0/3922542349 >> 172.21.15.69:0/3922542349 conn(0x55804402b580 msgr2=0x558044031260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:06.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 -- 172.21.15.69:0/3922542349 shutdown_connections 2024-08-02T17:55:06.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 1 -- 172.21.15.69:0/3922542349 wait complete. 2024-08-02T17:55:06.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.664+0000 7fc7763a7740 20 client.17577 trim_cache size 0 max 16384 2024-08-02T17:55:06.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc7763a7740 1 Processor -- start 2024-08-02T17:55:06.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc7763a7740 1 -- start start 2024-08-02T17:55:06.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x558044038ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x558044051850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044051d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc7763a7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1b3b60 con 0x55804403a1d0 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc7763a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1b1430 con 0x55804402f910 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc7763a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1af2a0 con 0x55804402eff0 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc771544640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044051d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc771544640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044051d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51170/0 (socket says 172.21.15.69:51170) 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc771544640 1 -- 172.21.15.69:0/2422681386 learned_addr learned my addr 172.21.15.69:0/2422681386 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc771544640 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x558044038ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.677+0000 7fc771d45640 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x558044051850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc771544640 1 -- 172.21.15.69:0/2422681386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 msgr2=0x558044038ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc771544640 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x558044038ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc771544640 1 -- 172.21.15.69:0/2422681386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 msgr2=0x558044051850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc771544640 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x558044051850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc771544640 1 -- 172.21.15.69:0/2422681386 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55804402f740 con 0x55804403a1d0 2024-08-02T17:55:06.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc771544640 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044051d90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc75c01d5c0 tx=0x7fc75c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c1c1c3e82ab987 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc7577fe640 1 -- 172.21.15.69:0/2422681386 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c021070 con 0x55804403a1d0 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc7577fe640 1 -- 172.21.15.69:0/2422681386 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75c01de20 con 0x55804403a1d0 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc7577fe640 1 -- 172.21.15.69:0/2422681386 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c0120e0 con 0x55804403a1d0 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc7763a7740 1 -- 172.21.15.69:0/2422681386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 msgr2=0x558044051d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.678+0000 7fc7763a7740 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044051d90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc75c01d5c0 tx=0x7fc75c00b410 comp rx=0 tx=0).stop 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 -- 172.21.15.69:0/2422681386 shutdown_connections 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x558044051d90 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x558044051850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 --2- 172.21.15.69:0/2422681386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x558044038ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 -- 172.21.15.69:0/2422681386 >> 172.21.15.69:0/2422681386 conn(0x55804402b580 msgr2=0x5580440387c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 -- 172.21.15.69:0/2422681386 shutdown_connections 2024-08-02T17:55:06.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 -- 172.21.15.69:0/2422681386 wait complete. 2024-08-02T17:55:06.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 -- start start 2024-08-02T17:55:06.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 Processor -- start 2024-08-02T17:55:06.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x55804403d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x55804403d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1af8f0 con 0x55804403a1d0 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1b0d60 con 0x55804402f910 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c1affc0 con 0x55804402eff0 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.679+0000 7fc7763a7740 10 client.0 _ll_register_callbacks cb 0x7fc75c012410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771544640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x55804403d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x55804403d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:54214/0 (socket says 172.21.15.69:54214) 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 -- 172.21.15.69:0/3577632920 learned_addr learned my addr 172.21.15.69:0/3577632920 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771544640 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x55804403d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 -- 172.21.15.69:0/3577632920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 msgr2=0x55804403d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x55804403d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 -- 172.21.15.69:0/3577632920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 msgr2=0x55804403dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 -- 172.21.15.69:0/3577632920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580440b2810 con 0x55804402f910 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc771d45640 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x55804403d6a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc76802a940 tx=0x7fc768031b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38facfacd0feb77 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc7557fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc7557fa640 1 -- 172.21.15.69:0/3577632920 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768035040 con 0x55804402f910 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc7763a7740 1 -- 172.21.15.69:0/3577632920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75c01de20 con 0x55804402f910 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.680+0000 7fc7557fa640 1 -- 172.21.15.69:0/3577632920 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7fc768004240 con 0x55804402f910 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7557fa640 1 -- 172.21.15.69:0/3577632920 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc768004440 con 0x55804402f910 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7763a7740 10 client.27101 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7763a7740 1 -- 172.21.15.69:0/3577632920 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580440b3ae0 con 0x55804402f910 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7557fa640 1 -- 172.21.15.69:0/3577632920 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc75c01de20 con 0x55804402f910 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7557fa640 1 -- 172.21.15.69:0/3577632920 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc76803d720 con 0x55804402f910 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7763a7740 10 client.27101 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:06.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7763a7740 10 client.27101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:55:06.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7763a7740 20 client.27101 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:06.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5580440b42c0 0x5580440b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:06.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7763a7740 4 client.27101 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:55:06.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc7763a7740 1 -- 172.21.15.69:0/3577632920 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558043f9c4c0 con 0x5580440b42c0 2024-08-02T17:55:06.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.681+0000 7fc771544640 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5580440b42c0 0x5580440b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:06.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc771544640 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5580440b42c0 0x5580440b66e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dda2f8eaf67fa8c3 server_cookie=c772673431e4ce1a in_seq=0 out_seq=0 2024-08-02T17:55:06.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7557fa640 10 client.27101 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:55:06.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7557fa640 1 -- 172.21.15.69:0/3577632920 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558043f9c4c0 con 0x5580440b42c0 2024-08-02T17:55:06.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7557fa640 10 client.27101 handle_command_reply: tid=0 2024-08-02T17:55:06.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 client.27101 shutdown 2024-08-02T17:55:06.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 -- 172.21.15.69:0/3577632920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 msgr2=0x55804403d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x55804403d6a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc76802a940 tx=0x7fc768031b40 comp rx=0 tx=0).stop 2024-08-02T17:55:06.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 -- 172.21.15.69:0/3577632920 shutdown_connections 2024-08-02T17:55:06.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5580440b42c0 0x5580440b66e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55804403a1d0 0x55804403dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55804402f910 0x55804403d6a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 --2- 172.21.15.69:0/3577632920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55804402eff0 0x55804403d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7557fa640 0 client.27101 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:55:06.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 -- 172.21.15.69:0/3577632920 >> 172.21.15.69:0/3577632920 conn(0x55804402b580 msgr2=0x558044037850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:06.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 -- 172.21.15.69:0/3577632920 shutdown_connections 2024-08-02T17:55:06.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 1 -- 172.21.15.69:0/3577632920 wait complete. 2024-08-02T17:55:06.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.682+0000 7fc7763a7740 20 client.27101 trim_cache size 0 max 16384 2024-08-02T17:55:06.693 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:55:06.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 -- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7580a2d70 msgr2=0x7fc7580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7580a2d70 0x7fc7580a5230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc76409bb60 tx=0x7fc76404b040 comp rx=0 tx=0).stop 2024-08-02T17:55:06.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 -- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 msgr2=0x7fc76c074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:06.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 0x7fc76c074ca0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc76400cad0 tx=0x7fc76400cf90 comp rx=0 tx=0).stop 2024-08-02T17:55:06.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 -- 172.21.15.69:0/95474915 shutdown_connections 2024-08-02T17:55:06.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc76c1078b0 0x7fc76c074ca0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc7580a2d70 0x7fc7580a5230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc76c102250 0x7fc76c074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 --2- 172.21.15.69:0/95474915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc76c1018b0 0x7fc76c074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:06.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 -- 172.21.15.69:0/95474915 >> 172.21.15.69:0/95474915 conn(0x7fc76c0fd5f0 msgr2=0x7fc76c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:06.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 -- 172.21.15.69:0/95474915 shutdown_connections 2024-08-02T17:55:06.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:06.683+0000 7fc773fa9640 1 -- 172.21.15.69:0/95474915 wait complete. 2024-08-02T17:55:07.340 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:55:07.593 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 -- 172.21.15.69:0/3865696056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d0102250 msgr2=0x7f54d010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 --2- 172.21.15.69:0/3865696056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d0102250 0x7f54d010e780 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009cd0 tx=0x7f54c802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 -- 172.21.15.69:0/3865696056 shutdown_connections 2024-08-02T17:55:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 --2- 172.21.15.69:0/3865696056 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d0102250 0x7f54d010e780 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 --2- 172.21.15.69:0/3865696056 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54d01018b0 0x7f54d0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 --2- 172.21.15.69:0/3865696056 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d01078b0 0x7f54d0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 -- 172.21.15.69:0/3865696056 >> 172.21.15.69:0/3865696056 conn(0x7f54d00fd5f0 msgr2=0x7f54d00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 -- 172.21.15.69:0/3865696056 shutdown_connections 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.113+0000 7f54d82ac640 1 -- 172.21.15.69:0/3865696056 wait complete. 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d82ac640 1 Processor -- start 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d82ac640 1 -- start start 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d82ac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54d01018b0 0x7f54d0074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d82ac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d0102250 0x7f54d0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d82ac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d01078b0 0x7f54d0074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d82ac640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0111cd0 con 0x7f54d01078b0 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d82ac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0111b50 con 0x7f54d01018b0 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d82ac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0111e50 con 0x7f54d0102250 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6849640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d01078b0 0x7f54d0074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6849640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d01078b0 0x7f54d0074ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51188/0 (socket says 172.21.15.69:51188) 2024-08-02T17:55:08.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6849640 1 -- 172.21.15.69:0/1219490205 learned_addr learned my addr 172.21.15.69:0/1219490205 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6048640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54d01018b0 0x7f54d0074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d5847640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d0102250 0x7f54d0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6849640 1 -- 172.21.15.69:0/1219490205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d0102250 msgr2=0x7f54d0074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6849640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d0102250 0x7f54d0074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6849640 1 -- 172.21.15.69:0/1219490205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54d01018b0 msgr2=0x7f54d0074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6849640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54d01018b0 0x7f54d0074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6849640 1 -- 172.21.15.69:0/1219490205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d01a6610 con 0x7f54d01078b0 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.114+0000 7f54d6048640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54d01018b0 0x7f54d0074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.115+0000 7f54d6849640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d01078b0 0x7f54d0074ca0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f54c8006010 tx=0x7f54c8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.115+0000 7f54c37fe640 1 -- 172.21.15.69:0/1219490205 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8004450 con 0x7f54d01078b0 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.115+0000 7f54c37fe640 1 -- 172.21.15.69:0/1219490205 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c8003e10 con 0x7f54d01078b0 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.115+0000 7f54d82ac640 1 -- 172.21.15.69:0/1219490205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d01a68a0 con 0x7f54d01078b0 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.115+0000 7f54c37fe640 1 -- 172.21.15.69:0/1219490205 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80466e0 con 0x7f54d01078b0 2024-08-02T17:55:08.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.115+0000 7f54d82ac640 1 -- 172.21.15.69:0/1219490205 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d01a6de0 con 0x7f54d01078b0 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.117+0000 7f54c37fe640 1 -- 172.21.15.69:0/1219490205 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f54c80305a0 con 0x7f54d01078b0 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.117+0000 7f54da6aa740 1 Processor -- start 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54da6aa740 1 -- start start 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6ccff0 0x5563ee6cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6cd910 0x5563ee6d7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6da5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54da6aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5563ee632a40 con 0x5563ee6ccff0 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54da6aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5563ee5ca9d0 con 0x5563ee6d81d0 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54da6aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563ee62ea00 con 0x5563ee6cd910 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54c37fe640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54bc0a2cf0 0x7f54bc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6ccff0 0x5563ee6cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6ccff0 0x5563ee6cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51198/0 (socket says 172.21.15.69:51198) 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 -- 172.21.15.69:0/2832639590 learned_addr learned my addr 172.21.15.69:0/2832639590 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:08.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d6849640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54bc0a2cf0 0x7f54bc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d6048640 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6cd910 0x5563ee6d7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54c37fe640 1 -- 172.21.15.69:0/1219490205 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f54c802e030 con 0x7f54d01078b0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6da5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 -- 172.21.15.69:0/2832639590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6cd910 msgr2=0x5563ee6d7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6cd910 0x5563ee6d7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 -- 172.21.15.69:0/2832639590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 msgr2=0x5563ee6da5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6da5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.118+0000 7f54d5847640 1 -- 172.21.15.69:0/2832639590 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ee5d1650 con 0x5563ee6ccff0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.119+0000 7f54d5847640 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6ccff0 0x5563ee6cd3d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006e10 tx=0x7f54c4009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9076a9abfb226f72 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.119+0000 7f54d6849640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54bc0a2cf0 0x7f54bc0a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f54c80b9830 tx=0x7f54c80b9670 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.119+0000 7f54c0dea640 1 -- 172.21.15.69:0/2832639590 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c401a070 con 0x5563ee6ccff0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.119+0000 7f54c0dea640 1 -- 172.21.15.69:0/2832639590 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c4010040 con 0x5563ee6ccff0 2024-08-02T17:55:08.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.119+0000 7f54c0dea640 1 -- 172.21.15.69:0/2832639590 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4014610 con 0x5563ee6ccff0 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.119+0000 7f54da6aa740 1 -- 172.21.15.69:0/2832639590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6ccff0 msgr2=0x5563ee6cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.119+0000 7f54da6aa740 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6ccff0 0x5563ee6cd3d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006e10 tx=0x7f54c4009420 comp rx=0 tx=0).stop 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 -- 172.21.15.69:0/2832639590 shutdown_connections 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6da5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6cd910 0x5563ee6d7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 --2- 172.21.15.69:0/2832639590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6ccff0 0x5563ee6cd3d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 -- 172.21.15.69:0/2832639590 >> 172.21.15.69:0/2832639590 conn(0x5563ee6c9580 msgr2=0x5563ee6c9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 -- 172.21.15.69:0/2832639590 shutdown_connections 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 -- 172.21.15.69:0/2832639590 wait complete. 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 -- start start 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 Processor -- start 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6d5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5563ee6dada0 con 0x5563ee6cd910 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5563ee5edb20 con 0x5563ee6d81d0 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5563ee6daf20 con 0x5563ee6ccff0 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54da6aa740 10 client.0 _ll_register_callbacks cb 0x5563ee5ebbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54d6048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54d6048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51204/0 (socket says 172.21.15.69:51204) 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54d6048640 1 -- 172.21.15.69:0/948578506 learned_addr learned my addr 172.21.15.69:0/948578506 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.120+0000 7f54d5847640 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54d5847640 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54d6048640 1 -- 172.21.15.69:0/948578506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 msgr2=0x5563ee6d5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54d6048640 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6d5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54d6048640 1 -- 172.21.15.69:0/948578506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 msgr2=0x5563ee6d5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54d6048640 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6d5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54d6048640 1 -- 172.21.15.69:0/948578506 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ee3a43c0 con 0x5563ee6cd910 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54d6048640 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6d55c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f54cc011a60 tx=0x7f54cc011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e29add96c1d890 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54b9ffb640 1 -- 172.21.15.69:0/948578506 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc004460 con 0x5563ee6cd910 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54b9ffb640 1 -- 172.21.15.69:0/948578506 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f54cc00f9a0 con 0x5563ee6cd910 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54da6aa740 1 -- 172.21.15.69:0/948578506 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c4010040 con 0x5563ee6cd910 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54b9ffb640 1 -- 172.21.15.69:0/948578506 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54cc00fba0 con 0x5563ee6cd910 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54da6aa740 10 client.17595 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54da6aa740 1 -- 172.21.15.69:0/948578506 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5563ee6e69d0 con 0x5563ee6cd910 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.121+0000 7f54b9ffb640 1 -- 172.21.15.69:0/948578506 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54c4010040 con 0x5563ee6cd910 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.122+0000 7f54b9ffb640 1 -- 172.21.15.69:0/948578506 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f54cc018070 con 0x5563ee6cd910 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.122+0000 7f54da6aa740 10 client.17595 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.122+0000 7f54da6aa740 10 client.17595 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:55:08.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.122+0000 7f54da6aa740 20 client.17595 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.122+0000 7f54da6aa740 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5563ee6e8090 0x5563ee6ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.122+0000 7f54da6aa740 4 client.17595 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.122+0000 7f54da6aa740 1 -- 172.21.15.69:0/948578506 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563ee63ffb0 con 0x5563ee6e8090 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.122+0000 7f54d5847640 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5563ee6e8090 0x5563ee6ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54b9ffb640 10 client.17595 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54d5847640 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5563ee6e8090 0x5563ee6ea4b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74f2b5b30124def2 server_cookie=e0e0e4ed162b2c50 in_seq=0 out_seq=0 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54b9ffb640 1 -- 172.21.15.69:0/948578506 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5563ee63ffb0 con 0x5563ee6e8090 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54b9ffb640 10 client.17595 handle_command_reply: tid=0 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54da6aa740 1 client.17595 shutdown 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54da6aa740 1 -- 172.21.15.69:0/948578506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 msgr2=0x5563ee6d55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54da6aa740 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6d55c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f54cc011a60 tx=0x7f54cc011f20 comp rx=0 tx=0).stop 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54da6aa740 1 -- 172.21.15.69:0/948578506 shutdown_connections 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.123+0000 7f54da6aa740 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5563ee6e8090 0x5563ee6ea4b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.124+0000 7f54b9ffb640 0 client.17595 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:55:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.124+0000 7f54da6aa740 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6d55c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.124+0000 7f54da6aa740 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6d5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.124+0000 7f54da6aa740 1 --2- 172.21.15.69:0/948578506 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6d5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.124+0000 7f54da6aa740 1 -- 172.21.15.69:0/948578506 >> 172.21.15.69:0/948578506 conn(0x5563ee6c9580 msgr2=0x5563ee6cf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.124+0000 7f54da6aa740 1 -- 172.21.15.69:0/948578506 shutdown_connections 2024-08-02T17:55:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.124+0000 7f54da6aa740 1 -- 172.21.15.69:0/948578506 wait complete. 2024-08-02T17:55:08.135 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.124+0000 7f54da6aa740 20 client.17595 trim_cache size 0 max 16384 2024-08-02T17:55:08.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.136+0000 7f54da6aa740 1 Processor -- start 2024-08-02T17:55:08.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.136+0000 7f54da6aa740 1 -- start start 2024-08-02T17:55:08.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6d5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6efd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54da6aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01b39b0 con 0x5563ee6cd910 2024-08-02T17:55:08.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54da6aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01b1280 con 0x5563ee6d81d0 2024-08-02T17:55:08.139 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54da6aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01af0f0 con 0x5563ee6ccff0 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6efd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51206/0 (socket says 172.21.15.69:51206) 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 -- 172.21.15.69:0/209079089 learned_addr learned my addr 172.21.15.69:0/209079089 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d5847640 1 --2- 172.21.15.69:0/209079089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 -- 172.21.15.69:0/209079089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 msgr2=0x5563ee6d5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 --2- 172.21.15.69:0/209079089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6d5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 -- 172.21.15.69:0/209079089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 msgr2=0x5563ee6f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 --2- 172.21.15.69:0/209079089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6f02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 -- 172.21.15.69:0/209079089 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ee6d7b30 con 0x5563ee6cd910 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54d6048640 1 --2- 172.21.15.69:0/209079089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6efd80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f54cc004d50 tx=0x7f54cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e35c6bee077470 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54bbfff640 1 -- 172.21.15.69:0/209079089 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc02b020 con 0x5563ee6cd910 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54bbfff640 1 -- 172.21.15.69:0/209079089 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54cc00fd40 con 0x5563ee6cd910 2024-08-02T17:55:08.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.137+0000 7f54bbfff640 1 -- 172.21.15.69:0/209079089 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc003c70 con 0x5563ee6cd910 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 -- 172.21.15.69:0/209079089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 msgr2=0x5563ee6efd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 --2- 172.21.15.69:0/209079089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6efd80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f54cc004d50 tx=0x7f54cc000bc0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 -- 172.21.15.69:0/209079089 shutdown_connections 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 --2- 172.21.15.69:0/209079089 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6efd80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 --2- 172.21.15.69:0/209079089 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 --2- 172.21.15.69:0/209079089 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6d5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 -- 172.21.15.69:0/209079089 >> 172.21.15.69:0/209079089 conn(0x5563ee6c9580 msgr2=0x5563ee6c85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 -- 172.21.15.69:0/209079089 shutdown_connections 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 -- 172.21.15.69:0/209079089 wait complete. 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.138+0000 7f54da6aa740 1 -- start start 2024-08-02T17:55:08.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54da6aa740 1 Processor -- start 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6e0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6e0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54da6aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54da6aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01af740 con 0x5563ee6cd910 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54da6aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01b0bb0 con 0x5563ee6d81d0 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54da6aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01afe10 con 0x5563ee6ccff0 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54da6aa740 10 client.0 _ll_register_callbacks cb 0x7f54cc011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6e0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6e0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51220/0 (socket says 172.21.15.69:51220) 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 -- 172.21.15.69:0/3548681748 learned_addr learned my addr 172.21.15.69:0/3548681748 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d5847640 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d5847640 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6e0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 -- 172.21.15.69:0/3548681748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 msgr2=0x5563ee6e0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:08.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6e0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 -- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 msgr2=0x5563ee6db870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6db870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 -- 172.21.15.69:0/3548681748 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ee6dc0a0 con 0x5563ee6cd910 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d5847640 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54d6048640 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6e0980 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f54cc004eb0 tx=0x7f54cc00fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ac54f03d4b0bb6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.139+0000 7f54b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54b9ffb640 1 -- 172.21.15.69:0/3548681748 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc039070 con 0x5563ee6cd910 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54b9ffb640 1 -- 172.21.15.69:0/3548681748 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f54cc005150 con 0x5563ee6cd910 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 1 -- 172.21.15.69:0/3548681748 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc00fd40 con 0x5563ee6cd910 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54b9ffb640 1 -- 172.21.15.69:0/3548681748 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f54cc042050 con 0x5563ee6cd910 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 10 client.17601 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 1 -- 172.21.15.69:0/3548681748 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5563ee7528d0 con 0x5563ee6cd910 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54b9ffb640 1 -- 172.21.15.69:0/3548681748 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54cc00fd40 con 0x5563ee6cd910 2024-08-02T17:55:08.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54b9ffb640 1 -- 172.21.15.69:0/3548681748 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f54cc03c020 con 0x5563ee6cd910 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 10 client.17601 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 10 client.17601 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 20 client.17601 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5563ee752f90 0x5563ee7553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 4 client.17601 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54da6aa740 1 -- 172.21.15.69:0/3548681748 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563ee63a4c0 con 0x5563ee752f90 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.140+0000 7f54d5847640 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5563ee752f90 0x5563ee7553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54d5847640 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5563ee752f90 0x5563ee7553b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80ee520a38e124af server_cookie=7bb9dc74df8b5ffa in_seq=0 out_seq=0 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54b9ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54b9ffb640 1 -- 172.21.15.69:0/3548681748 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563ee63a4c0 con 0x5563ee752f90 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54b9ffb640 10 client.17601 handle_command_reply: tid=0 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 client.17601 shutdown 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 -- 172.21.15.69:0/3548681748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 msgr2=0x5563ee6e0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6e0980 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f54cc004eb0 tx=0x7f54cc00fef0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 -- 172.21.15.69:0/3548681748 shutdown_connections 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5563ee752f90 0x5563ee7553b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5563ee6cd910 0x5563ee6e0980 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5563ee6d81d0 0x5563ee6db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 --2- 172.21.15.69:0/3548681748 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5563ee6ccff0 0x5563ee6e0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 -- 172.21.15.69:0/3548681748 >> 172.21.15.69:0/3548681748 conn(0x5563ee6c9580 msgr2=0x5563ee6d5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54b9ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 -- 172.21.15.69:0/3548681748 shutdown_connections 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 1 -- 172.21.15.69:0/3548681748 wait complete. 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.141+0000 7f54da6aa740 20 client.17601 trim_cache size 0 max 16384 2024-08-02T17:55:08.148 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:55:08.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 -- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54bc0a2cf0 msgr2=0x7f54bc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54bc0a2cf0 0x7f54bc0a51b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f54c80b9830 tx=0x7f54c80b9670 comp rx=0 tx=0).stop 2024-08-02T17:55:08.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 -- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d01078b0 msgr2=0x7f54d0074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:08.149 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d01078b0 0x7f54d0074ca0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f54c8006010 tx=0x7f54c8003c20 comp rx=0 tx=0).stop 2024-08-02T17:55:08.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 -- 172.21.15.69:0/1219490205 shutdown_connections 2024-08-02T17:55:08.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f54d01078b0 0x7f54d0074ca0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f54bc0a2cf0 0x7f54bc0a51b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54d0102250 0x7f54d0074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 --2- 172.21.15.69:0/1219490205 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f54d01018b0 0x7f54d0074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:08.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 -- 172.21.15.69:0/1219490205 >> 172.21.15.69:0/1219490205 conn(0x7f54d00fd5f0 msgr2=0x7f54d00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:08.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 -- 172.21.15.69:0/1219490205 shutdown_connections 2024-08-02T17:55:08.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:08.142+0000 7f54d82ac640 1 -- 172.21.15.69:0/1219490205 wait complete. 2024-08-02T17:55:08.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:08 smithi069 ceph-mon[32746]: pgmap v269: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.6 MiB/s wr, 635 op/s 2024-08-02T17:55:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:08 smithi083 ceph-mon[37031]: pgmap v269: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.6 MiB/s wr, 635 op/s 2024-08-02T17:55:08.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:08 smithi077 ceph-mon[37108]: pgmap v269: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.6 MiB/s wr, 635 op/s 2024-08-02T17:55:08.842 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:55:09.094 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:09.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 -- 172.21.15.69:0/1890639713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac81018b0 msgr2=0x7f4ac8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/1890639713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac81018b0 0x7f4ac8101d10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009930 tx=0x7f4ab802a420 comp rx=0 tx=0).stop 2024-08-02T17:55:09.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 -- 172.21.15.69:0/1890639713 shutdown_connections 2024-08-02T17:55:09.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/1890639713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac81018b0 0x7f4ac8101d10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/1890639713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac8102250 0x7f4ac810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/1890639713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac81078b0 0x7f4ac8107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 -- 172.21.15.69:0/1890639713 >> 172.21.15.69:0/1890639713 conn(0x7f4ac80fd5b0 msgr2=0x7f4ac80ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:09.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 -- 172.21.15.69:0/1890639713 shutdown_connections 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 -- 172.21.15.69:0/1890639713 wait complete. 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.580+0000 7f4acfe5d640 1 Processor -- start 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acfe5d640 1 -- start start 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acfe5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac81018b0 0x7f4ac8074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acfe5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8102250 0x7f4ac8074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acfe5d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac81078b0 0x7f4ac8074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acfe5d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8111d50 con 0x7f4ac8102250 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acfe5d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8111bd0 con 0x7f4ac81078b0 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acfe5d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac8111ed0 con 0x7f4ac81018b0 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8102250 0x7f4ac8074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8102250 0x7f4ac8074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51246/0 (socket says 172.21.15.69:51246) 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 -- 172.21.15.69:0/2018713935 learned_addr learned my addr 172.21.15.69:0/2018713935 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac81018b0 0x7f4ac8074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac81078b0 0x7f4ac8074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 -- 172.21.15.69:0/2018713935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac81018b0 msgr2=0x7f4ac8074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac81018b0 0x7f4ac8074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 -- 172.21.15.69:0/2018713935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac81078b0 msgr2=0x7f4ac8074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac81078b0 0x7f4ac8074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 -- 172.21.15.69:0/2018713935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac81a67d0 con 0x7f4ac8102250 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac81078b0 0x7f4ac8074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8102250 0x7f4ac8074760 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009a50 tx=0x7f4ab8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.581+0000 7f4abeffd640 1 -- 172.21.15.69:0/2018713935 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab803f030 con 0x7f4ac8102250 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.582+0000 7f4abeffd640 1 -- 172.21.15.69:0/2018713935 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab8003ce0 con 0x7f4ac8102250 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.582+0000 7f4acfe5d640 1 -- 172.21.15.69:0/2018713935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac81a6a60 con 0x7f4ac8102250 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.582+0000 7f4abeffd640 1 -- 172.21.15.69:0/2018713935 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8035070 con 0x7f4ac8102250 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.582+0000 7f4acfe5d640 1 -- 172.21.15.69:0/2018713935 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac81a6f90 con 0x7f4ac8102250 2024-08-02T17:55:09.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.584+0000 7f4abeffd640 1 -- 172.21.15.69:0/2018713935 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4ab8042030 con 0x7f4ac8102250 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.584+0000 7f4ad225b740 1 Processor -- start 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.584+0000 7f4ad225b740 1 -- start start 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d1715b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17165c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d171685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4ad225b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d170c0a40 con 0x560d171661d0 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4ad225b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560d170589d0 con 0x560d1715b910 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4ad225b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d170bca00 con 0x560d1715aff0 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acd3f8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d171685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acdbf9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4abeffd640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4ab40a2d70 0x7f4ab40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acd3f8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d171685c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51248/0 (socket says 172.21.15.69:51248) 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acd3f8640 1 -- 172.21.15.69:0/752583504 learned_addr learned my addr 172.21.15.69:0/752583504 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:09.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acdbf9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17165c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:54272/0 (socket says 172.21.15.69:54272) 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/752583504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d1715b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4ab40a2d70 0x7f4ab40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acdbf9640 1 -- 172.21.15.69:0/752583504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 msgr2=0x560d1715b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/752583504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d1715b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acdbf9640 1 -- 172.21.15.69:0/752583504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 msgr2=0x560d171685c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/752583504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d171685c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4abeffd640 1 -- 172.21.15.69:0/2018713935 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f4ab80eff10 con 0x7f4ac8102250 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.585+0000 7f4acdbf9640 1 -- 172.21.15.69:0/752583504 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1705f650 con 0x560d1715b910 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.586+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/752583504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17165c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400b760 tx=0x7f4ac400bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a211528db642f8d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.586+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4ab40a2d70 0x7f4ab40a5230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009fd0 tx=0x7f4ac0008220 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.586+0000 7f4ab3fff640 1 -- 172.21.15.69:0/752583504 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac400d950 con 0x560d1715b910 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.586+0000 7f4ab3fff640 1 -- 172.21.15.69:0/752583504 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac4019070 con 0x560d1715b910 2024-08-02T17:55:09.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.586+0000 7f4ab3fff640 1 -- 172.21.15.69:0/752583504 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4014e50 con 0x560d1715b910 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.586+0000 7f4ad225b740 1 -- 172.21.15.69:0/752583504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 msgr2=0x560d17165c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.586+0000 7f4ad225b740 1 --2- 172.21.15.69:0/752583504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17165c90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400b760 tx=0x7f4ac400bc20 comp rx=0 tx=0).stop 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 -- 172.21.15.69:0/752583504 shutdown_connections 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 --2- 172.21.15.69:0/752583504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d171685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 --2- 172.21.15.69:0/752583504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17165c90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 --2- 172.21.15.69:0/752583504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d1715b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 -- 172.21.15.69:0/752583504 >> 172.21.15.69:0/752583504 conn(0x560d17157580 msgr2=0x560d17157960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 -- 172.21.15.69:0/752583504 shutdown_connections 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 -- 172.21.15.69:0/752583504 wait complete. 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 -- start start 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 Processor -- start 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d17165c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d17163750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560d17168da0 con 0x560d171661d0 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560d1707bb20 con 0x560d1715b910 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.587+0000 7f4ad225b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560d17168f20 con 0x560d1715aff0 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ad225b740 10 client.0 _ll_register_callbacks cb 0x560d17079bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d17163750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d17163750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51262/0 (socket says 172.21.15.69:51262) 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 -- 172.21.15.69:0/3898290493 learned_addr learned my addr 172.21.15.69:0/3898290493 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d17165c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 -- 172.21.15.69:0/3898290493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 msgr2=0x560d17165c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d17165c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 -- 172.21.15.69:0/3898290493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 msgr2=0x560d17162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:09.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17162fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 -- 172.21.15.69:0/3898290493 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d16e323c0 con 0x560d171661d0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d17163750 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0015cf0 tx=0x7f4ac0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ba630f70944c45 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ab17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ab17fa640 1 -- 172.21.15.69:0/3898290493 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac00253f0 con 0x560d171661d0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ab17fa640 1 -- 172.21.15.69:0/3898290493 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f4ac0025590 con 0x560d171661d0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ad225b740 1 -- 172.21.15.69:0/3898290493 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac4019070 con 0x560d171661d0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ab17fa640 1 -- 172.21.15.69:0/3898290493 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ac00257d0 con 0x560d171661d0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ad225b740 10 client.17613 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.588+0000 7f4ad225b740 1 -- 172.21.15.69:0/3898290493 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d171748b0 con 0x560d171661d0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4ab17fa640 1 -- 172.21.15.69:0/3898290493 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac4019070 con 0x560d171661d0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4ab17fa640 1 -- 172.21.15.69:0/3898290493 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac001a950 con 0x560d171661d0 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4ad225b740 10 client.17613 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4ad225b740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:09.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4ad225b740 20 client.17613 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4ad225b740 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560d17175dc0 0x560d17178210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4ad225b740 4 client.17613 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4ad225b740 1 -- 172.21.15.69:0/3898290493 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d170cdfb0 con 0x560d17175dc0 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.589+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560d17175dc0 0x560d17178210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.590+0000 7f4ab17fa640 10 client.17613 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.590+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560d17175dc0 0x560d17178210 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4442cf53eba09b0 server_cookie=38b668bdbc543e65 in_seq=0 out_seq=0 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ab17fa640 1 -- 172.21.15.69:0/3898290493 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d170cdfb0 con 0x560d17175dc0 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ab17fa640 10 client.17613 handle_command_reply: tid=0 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 client.17613 shutdown 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 -- 172.21.15.69:0/3898290493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 msgr2=0x560d17163750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d17163750 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0015cf0 tx=0x7f4ac0004b50 comp rx=0 tx=0).stop 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 -- 172.21.15.69:0/3898290493 shutdown_connections 2024-08-02T17:55:09.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d17163750 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560d17175dc0 0x560d17178210 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d17162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 --2- 172.21.15.69:0/3898290493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d17165c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ab17fa640 0 client.17613 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 -- 172.21.15.69:0/3898290493 >> 172.21.15.69:0/3898290493 conn(0x560d17157580 msgr2=0x560d1715d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 -- 172.21.15.69:0/3898290493 shutdown_connections 2024-08-02T17:55:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 1 -- 172.21.15.69:0/3898290493 wait complete. 2024-08-02T17:55:09.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.591+0000 7f4ad225b740 20 client.17613 trim_cache size 0 max 16384 2024-08-02T17:55:09.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ad225b740 1 Processor -- start 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ad225b740 1 -- start start 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d1715e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d1715e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d1717dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ad225b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81b3c40 con 0x560d171661d0 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ad225b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81b1510 con 0x560d1715b910 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ad225b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81af380 con 0x560d1715aff0 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ace3fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d1717dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ace3fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d1717dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51274/0 (socket says 172.21.15.69:51274) 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ace3fa640 1 -- 172.21.15.69:0/1468583898 learned_addr learned my addr 172.21.15.69:0/1468583898 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d1715e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ace3fa640 1 -- 172.21.15.69:0/1468583898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 msgr2=0x560d1715e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:09.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d1715e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ace3fa640 1 -- 172.21.15.69:0/1468583898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 msgr2=0x560d1715e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d1715e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4ace3fa640 1 -- 172.21.15.69:0/1468583898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d17178890 con 0x560d171661d0 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.604+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d1715e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d1717dae0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0015a60 tx=0x7f4ac000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9adeb85ac08ea4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ab37fe640 1 -- 172.21.15.69:0/1468583898 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0009c40 con 0x560d171661d0 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ab37fe640 1 -- 172.21.15.69:0/1468583898 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0025e50 con 0x560d171661d0 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ab37fe640 1 -- 172.21.15.69:0/1468583898 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac001a3f0 con 0x560d171661d0 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 -- 172.21.15.69:0/1468583898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 msgr2=0x560d1717dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d1717dae0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0015a60 tx=0x7f4ac000efd0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 -- 172.21.15.69:0/1468583898 shutdown_connections 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d171661d0 0x560d1717dae0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d1715b910 0x560d1715e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1468583898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715aff0 0x560d1715e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 -- 172.21.15.69:0/1468583898 >> 172.21.15.69:0/1468583898 conn(0x560d17157580 msgr2=0x560d171521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 -- 172.21.15.69:0/1468583898 shutdown_connections 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.605+0000 7f4ad225b740 1 -- 172.21.15.69:0/1468583898 wait complete. 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 1 -- start start 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 1 Processor -- start 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d1715aff0 0x560d17166b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715b910 0x560d17167090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d171661d0 0x560d1716d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81af9d0 con 0x560d1715aff0 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81b0e40 con 0x560d171661d0 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac81b00a0 con 0x560d1715b910 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4ad225b740 10 client.0 _ll_register_callbacks cb 0x7f4ac000e7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4acdbf9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d1715aff0 0x560d17166b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4acdbf9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d1715aff0 0x560d17166b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51276/0 (socket says 172.21.15.69:51276) 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4acdbf9640 1 -- 172.21.15.69:0/1900728942 learned_addr learned my addr 172.21.15.69:0/1900728942 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:09.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.606+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715b910 0x560d17167090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d171661d0 0x560d1716d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4acdbf9640 1 -- 172.21.15.69:0/1900728942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715b910 msgr2=0x560d17167090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715b910 0x560d17167090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4acdbf9640 1 -- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d171661d0 msgr2=0x560d1716d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d171661d0 0x560d1716d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4acdbf9640 1 -- 172.21.15.69:0/1900728942 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1716d6b0 con 0x560d1715aff0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ace3fa640 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d171661d0 0x560d1716d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4acdbf9640 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d1715aff0 0x560d17166b50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4014ac0 tx=0x7f4ac4014990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e9bc88d10caddc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ab17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ab17fa640 1 -- 172.21.15.69:0/1900728942 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4029070 con 0x560d1715aff0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ab17fa640 1 -- 172.21.15.69:0/1900728942 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f4ac40146b0 con 0x560d1715aff0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ad225b740 1 -- 172.21.15.69:0/1900728942 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac0025e50 con 0x560d1715aff0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ab17fa640 1 -- 172.21.15.69:0/1900728942 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4ac400b260 con 0x560d1715aff0 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ad225b740 10 client.17619 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:09.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ad225b740 1 -- 172.21.15.69:0/1900728942 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560d1716e920 con 0x560d1715aff0 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ab17fa640 1 -- 172.21.15.69:0/1900728942 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ac0025e50 con 0x560d1715aff0 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.607+0000 7f4ab17fa640 1 -- 172.21.15.69:0/1900728942 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac402c020 con 0x560d1715aff0 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 10 client.17619 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 10 client.17619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 20 client.17619 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560d171e0ab0 0x560d171e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 4 client.17619 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 1 -- 172.21.15.69:0/1900728942 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560d170c84c0 con 0x560d171e0ab0 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560d171e0ab0 0x560d171e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4acd3f8640 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560d171e0ab0 0x560d171e2ea0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37517f1ab19a4ad9 server_cookie=1847a8c530143426 in_seq=0 out_seq=0 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ab17fa640 10 client.17619 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ab17fa640 1 -- 172.21.15.69:0/1900728942 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560d170c84c0 con 0x560d171e0ab0 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ab17fa640 10 client.17619 handle_command_reply: tid=0 2024-08-02T17:55:09.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 1 client.17619 shutdown 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 1 -- 172.21.15.69:0/1900728942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d1715aff0 msgr2=0x560d17166b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.608+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d1715aff0 0x560d17166b50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4014ac0 tx=0x7f4ac4014990 comp rx=0 tx=0).stop 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 1 -- 172.21.15.69:0/1900728942 shutdown_connections 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560d171e0ab0 0x560d171e2ea0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560d171661d0 0x560d1716d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560d1715b910 0x560d17167090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ab17fa640 0 client.17619 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 1 --2- 172.21.15.69:0/1900728942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560d1715aff0 0x560d17166b50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 1 -- 172.21.15.69:0/1900728942 >> 172.21.15.69:0/1900728942 conn(0x560d17157580 msgr2=0x560d1715e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 1 -- 172.21.15.69:0/1900728942 shutdown_connections 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 1 -- 172.21.15.69:0/1900728942 wait complete. 2024-08-02T17:55:09.619 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4ad225b740 20 client.17619 trim_cache size 0 max 16384 2024-08-02T17:55:09.620 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:55:09.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 -- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4ab40a2d70 msgr2=0x7f4ab40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4ab40a2d70 0x7f4ab40a5230 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009fd0 tx=0x7f4ac0008220 comp rx=0 tx=0).stop 2024-08-02T17:55:09.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 -- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8102250 msgr2=0x7f4ac8074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8102250 0x7f4ac8074760 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009a50 tx=0x7f4ab8004270 comp rx=0 tx=0).stop 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 -- 172.21.15.69:0/2018713935 shutdown_connections 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4ac8102250 0x7f4ac8074760 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4ab40a2d70 0x7f4ab40a5230 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4ac81078b0 0x7f4ac8074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 --2- 172.21.15.69:0/2018713935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4ac81018b0 0x7f4ac8074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.609+0000 7f4acfe5d640 1 -- 172.21.15.69:0/2018713935 >> 172.21.15.69:0/2018713935 conn(0x7f4ac80fd5b0 msgr2=0x7f4ac80fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.610+0000 7f4acfe5d640 1 -- 172.21.15.69:0/2018713935 shutdown_connections 2024-08-02T17:55:09.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:09.610+0000 7f4acfe5d640 1 -- 172.21.15.69:0/2018713935 wait complete. 2024-08-02T17:55:09.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 480.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:55:10.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:10 smithi069 ceph-mon[32746]: pgmap v270: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-08-02T17:55:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:10 smithi083 ceph-mon[37031]: pgmap v270: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-08-02T17:55:10.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:10 smithi077 ceph-mon[37108]: pgmap v270: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.1 MiB/s wr, 587 op/s 2024-08-02T17:55:11.614 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:55:11.615 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:55:11.885 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:12.293 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:12 smithi069 ceph-mon[32746]: pgmap v271: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-08-02T17:55:12.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.376+0000 7fc2cafa2640 1 -- 172.21.15.69:0/131224422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 msgr2=0x7fc2c4100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.376+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/131224422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 0x7fc2c4100f30 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4009900 tx=0x7fc2b402a480 comp rx=0 tx=0).stop 2024-08-02T17:55:12.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.376+0000 7fc2cafa2640 1 -- 172.21.15.69:0/131224422 shutdown_connections 2024-08-02T17:55:12.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.376+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/131224422 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2c4102c20 0x7fc2c410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.376+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/131224422 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c41022a0 0x7fc2c41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.376+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/131224422 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 0x7fc2c4100f30 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.376+0000 7fc2cafa2640 1 -- 172.21.15.69:0/131224422 >> 172.21.15.69:0/131224422 conn(0x7fc2c40fc500 msgr2=0x7fc2c40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.377+0000 7fc2cafa2640 1 -- 172.21.15.69:0/131224422 shutdown_connections 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.377+0000 7fc2cafa2640 1 -- 172.21.15.69:0/131224422 wait complete. 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.377+0000 7fc2cafa2640 1 Processor -- start 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.377+0000 7fc2cafa2640 1 -- start start 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.377+0000 7fc2cafa2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 0x7fc2c4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2cafa2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c41022a0 0x7fc2c41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2cafa2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2c4102c20 0x7fc2c419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2cafa2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4110cf0 con 0x7fc2c4100b50 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 0x7fc2c4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2cafa2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4110b70 con 0x7fc2c4102c20 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2cafa2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4110e70 con 0x7fc2c41022a0 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 0x7fc2c4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51290/0 (socket says 172.21.15.69:51290) 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 -- 172.21.15.69:0/2381803729 learned_addr learned my addr 172.21.15.69:0/2381803729 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:12.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c953f640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2c4102c20 0x7fc2c419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 -- 172.21.15.69:0/2381803729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c41022a0 msgr2=0x7fc2c41988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c41022a0 0x7fc2c41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 -- 172.21.15.69:0/2381803729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2c4102c20 msgr2=0x7fc2c419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2c4102c20 0x7fc2c419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 -- 172.21.15.69:0/2381803729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c419d320 con 0x7fc2c4100b50 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c953f640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2c4102c20 0x7fc2c419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c8d3e640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 0x7fc2c4198370 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4006010 tx=0x7fc2b4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c1ffb640 1 -- 172.21.15.69:0/2381803729 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b403f070 con 0x7fc2c4100b50 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2cafa2640 1 -- 172.21.15.69:0/2381803729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c419d5b0 con 0x7fc2c4100b50 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c1ffb640 1 -- 172.21.15.69:0/2381803729 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2b4031c30 con 0x7fc2c4100b50 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.378+0000 7fc2c1ffb640 1 -- 172.21.15.69:0/2381803729 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4035030 con 0x7fc2c4100b50 2024-08-02T17:55:12.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.379+0000 7fc2cafa2640 1 -- 172.21.15.69:0/2381803729 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c41a4ef0 con 0x7fc2c4100b50 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2c1ffb640 1 -- 172.21.15.69:0/2381803729 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc2b4042070 con 0x7fc2c4100b50 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2cd3a0740 1 Processor -- start 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2cd3a0740 1 -- start start 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 0x56092a6315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2c1ffb640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2ac0a2cf0 0x7fc2ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a63bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a63e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56092a5f4e40 con 0x56092a63c3e0 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56092a5967f0 con 0x56092a631b20 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.381+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56092a53dfb0 con 0x56092a631200 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a63e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c1ffb640 1 -- 172.21.15.69:0/2381803729 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7fc2b40f01d0 con 0x7fc2c4100b50 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a63e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51292/0 (socket says 172.21.15.69:51292) 2024-08-02T17:55:12.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c8d3e640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2ac0a2cf0 0x7fc2ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 -- 172.21.15.69:0/120861525 learned_addr learned my addr 172.21.15.69:0/120861525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 0x56092a6315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a63bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 -- 172.21.15.69:0/120861525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 msgr2=0x56092a6315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 0x56092a6315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 -- 172.21.15.69:0/120861525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 msgr2=0x56092a63bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a63bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 -- 172.21.15.69:0/120861525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092a3086a0 con 0x56092a63c3e0 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a63bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c8d3e640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2ac0a2cf0 0x7fc2ac0a51b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4005ab0 tx=0x7fc2b402a480 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2c953f640 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a63e7d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc014bc0 tx=0x7fc2bc00a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45a6a90a6ae2994 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2aaffd640 1 -- 172.21.15.69:0/120861525 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc014250 con 0x56092a63c3e0 2024-08-02T17:55:12.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2aaffd640 1 -- 172.21.15.69:0/120861525 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc011400 con 0x56092a63c3e0 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.382+0000 7fc2aaffd640 1 -- 172.21.15.69:0/120861525 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0135f0 con 0x56092a63c3e0 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/120861525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 msgr2=0x56092a63e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a63e7d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc014bc0 tx=0x7fc2bc00a6b0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/120861525 shutdown_connections 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a63e7d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a63bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/120861525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 0x56092a6315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/120861525 >> 172.21.15.69:0/120861525 conn(0x56092a62d790 msgr2=0x56092a62db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/120861525 shutdown_connections 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/120861525 wait complete. 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.383+0000 7fc2cd3a0740 1 -- start start 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2cd3a0740 1 Processor -- start 2024-08-02T17:55:12.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631200 0x56092a639340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631b20 0x56092a639880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 0x56092a639dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56092a551cb0 con 0x56092a631b20 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56092a5927b0 con 0x56092a631200 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56092a63efb0 con 0x56092a63c3e0 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2cd3a0740 10 client.0 _ll_register_callbacks cb 0x56092a2a5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631b20 0x56092a639880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631b20 0x56092a639880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51302/0 (socket says 172.21.15.69:51302) 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c3fff640 1 -- 172.21.15.69:0/1537037535 learned_addr learned my addr 172.21.15.69:0/1537037535 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c953f640 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 0x56092a639dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c953f640 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631200 0x56092a639340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c3fff640 1 -- 172.21.15.69:0/1537037535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 msgr2=0x56092a639dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:12.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 0x56092a639dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c3fff640 1 -- 172.21.15.69:0/1537037535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631200 msgr2=0x56092a639340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631200 0x56092a639340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.384+0000 7fc2c3fff640 1 -- 172.21.15.69:0/1537037535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092a550bc0 con 0x56092a631b20 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631b20 0x56092a639880 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800be70 tx=0x7fc2b800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144cb3ea1e3d161d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1537037535 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8015070 con 0x56092a631b20 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1537037535 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7fc2b8004230 con 0x56092a631b20 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1537037535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc011400 con 0x56092a631b20 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1537037535 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2b8021050 con 0x56092a631b20 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 10 client.17631 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1537037535 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56092a64afa0 con 0x56092a631b20 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1537037535 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc011400 con 0x56092a631b20 2024-08-02T17:55:12.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1537037535 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc2b8003850 con 0x56092a631b20 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 10 client.17631 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 10 client.17631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 20 client.17631 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56092a64c460 0x56092a64e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 4 client.17631 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.385+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1537037535 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56092a5a3d60 con 0x56092a64c460 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.386+0000 7fc2c953f640 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56092a64c460 0x56092a64e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.386+0000 7fc2a3fff640 10 client.17631 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.386+0000 7fc2c953f640 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56092a64c460 0x56092a64e8b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e193d2a9d98b5b04 server_cookie=7205ada3dd80da9c in_seq=0 out_seq=0 2024-08-02T17:55:12.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1537037535 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56092a5a3d60 con 0x56092a64c460 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2a3fff640 10 client.17631 handle_command_reply: tid=0 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 client.17631 shutdown 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1537037535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631b20 msgr2=0x56092a639880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631b20 0x56092a639880 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fc2b800be70 tx=0x7fc2b800f730 comp rx=0 tx=0).stop 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1537037535 shutdown_connections 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631b20 0x56092a639880 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56092a64c460 0x56092a64e8b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 0x56092a639dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1537037535 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631200 0x56092a639340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1537037535 >> 172.21.15.69:0/1537037535 conn(0x56092a62d790 msgr2=0x56092a633470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2a3fff640 0 client.17631 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1537037535 shutdown_connections 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1537037535 wait complete. 2024-08-02T17:55:12.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.387+0000 7fc2cd3a0740 20 client.17631 trim_cache size 0 max 16384 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.404+0000 7fc2cd3a0740 1 Processor -- start 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.404+0000 7fc2cd3a0740 1 -- start start 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 0x56092a6541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a654700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a632c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41b18a0 con 0x56092a63c3e0 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41af170 con 0x56092a631b20 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41acfe0 con 0x56092a631200 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a632c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a632c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51312/0 (socket says 172.21.15.69:51312) 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 -- 172.21.15.69:0/2847354431 learned_addr learned my addr 172.21.15.69:0/2847354431 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 0x56092a6541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a654700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 -- 172.21.15.69:0/2847354431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 msgr2=0x56092a6541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 0x56092a6541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 -- 172.21.15.69:0/2847354431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 msgr2=0x56092a654700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a654700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 -- 172.21.15.69:0/2847354431 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092a637890 con 0x56092a63c3e0 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2c953f640 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a632c50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc011ef0 tx=0x7fc2bc004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb72843668e89e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2aa7fc640 1 -- 172.21.15.69:0/2847354431 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00dad0 con 0x56092a63c3e0 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2aa7fc640 1 -- 172.21.15.69:0/2847354431 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc00dc70 con 0x56092a63c3e0 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.405+0000 7fc2aa7fc640 1 -- 172.21.15.69:0/2847354431 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0227a0 con 0x56092a63c3e0 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/2847354431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 msgr2=0x56092a632c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a632c50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc011ef0 tx=0x7fc2bc004b50 comp rx=0 tx=0).stop 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/2847354431 shutdown_connections 2024-08-02T17:55:12.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a63c3e0 0x56092a632c50 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a654700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/2847354431 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a631200 0x56092a6541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/2847354431 >> 172.21.15.69:0/2847354431 conn(0x56092a62d790 msgr2=0x56092a634660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/2847354431 shutdown_connections 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.406+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/2847354431 wait complete. 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 1 -- start start 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 1 Processor -- start 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631200 0x56092a636290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a6367d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 0x56092a636d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41ad630 con 0x56092a631200 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41aeaa0 con 0x56092a631b20 2024-08-02T17:55:12.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41add00 con 0x56092a63c3e0 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2cd3a0740 10 client.0 _ll_register_callbacks cb 0x7fc2b800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2c953f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631200 0x56092a636290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2c953f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631200 0x56092a636290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51322/0 (socket says 172.21.15.69:51322) 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2c953f640 1 -- 172.21.15.69:0/1515218452 learned_addr learned my addr 172.21.15.69:0/1515218452 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2c953f640 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 0x56092a636d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.407+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a6367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2c953f640 1 -- 172.21.15.69:0/1515218452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 msgr2=0x56092a636d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2c953f640 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 0x56092a636d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2c953f640 1 -- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 msgr2=0x56092a6367d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2c953f640 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a6367d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2c953f640 1 -- 172.21.15.69:0/1515218452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092a6b6190 con 0x56092a631200 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a6367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2c953f640 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631200 0x56092a636290 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0146e0 tx=0x7fc2bc00af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2e4f2a15e6b7f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:12.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1515218452 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00dad0 con 0x56092a631200 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1515218452 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7fc2bc011400 con 0x56092a631200 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1515218452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc00dc70 con 0x56092a631200 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1515218452 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc2bc011810 con 0x56092a631200 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 10 client.17637 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1515218452 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56092a6b73d0 con 0x56092a631200 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1515218452 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc00dc70 con 0x56092a631200 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1515218452 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fc2bc010040 con 0x56092a631200 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 10 client.17637 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 20 client.17637 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56092a6b78a0 0x56092a6b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:12.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 4 client.17637 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.408+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1515218452 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56092a59e270 con 0x56092a6b78a0 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.409+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56092a6b78a0 0x56092a6b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.409+0000 7fc2a3fff640 10 client.17637 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.409+0000 7fc2c3fff640 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56092a6b78a0 0x56092a6b9c90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf131dd3a3cd3e88 server_cookie=1fa970765b8787e2 in_seq=0 out_seq=0 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2a3fff640 1 -- 172.21.15.69:0/1515218452 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56092a59e270 con 0x56092a6b78a0 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2a3fff640 10 client.17637 handle_command_reply: tid=0 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 client.17637 shutdown 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1515218452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631200 msgr2=0x56092a636290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631200 0x56092a636290 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0146e0 tx=0x7fc2bc00af70 comp rx=0 tx=0).stop 2024-08-02T17:55:12.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1515218452 shutdown_connections 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "789553e5-6d8c-42ea-b7d0-02cc643b33c4", 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56092a6b78a0 0x56092a6b9c90 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56092a63c3e0 0x56092a636d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56092a631b20 0x56092a6367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 --2- 172.21.15.69:0/1515218452 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56092a631200 0x56092a636290 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.410+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1515218452 >> 172.21.15.69:0/1515218452 conn(0x56092a62d790 msgr2=0x56092a62c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2a3fff640 0 client.17637 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1515218452 shutdown_connections 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cd3a0740 1 -- 172.21.15.69:0/1515218452 wait complete. 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cd3a0740 20 client.17637 trim_cache size 0 max 16384 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 -- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2ac0a2cf0 msgr2=0x7fc2ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2ac0a2cf0 0x7fc2ac0a51b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4005ab0 tx=0x7fc2b402a480 comp rx=0 tx=0).stop 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 -- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 msgr2=0x7fc2c4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 0x7fc2c4198370 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4006010 tx=0x7fc2b4003c10 comp rx=0 tx=0).stop 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 -- 172.21.15.69:0/2381803729 shutdown_connections 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2ac0a2cf0 0x7fc2ac0a51b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc2c4102c20 0x7fc2c419cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc2c41022a0 0x7fc2c41988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 --2- 172.21.15.69:0/2381803729 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc2c4100b50 0x7fc2c4198370 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:12.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 -- 172.21.15.69:0/2381803729 >> 172.21.15.69:0/2381803729 conn(0x7fc2c40fc500 msgr2=0x7fc2c406a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:12.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 -- 172.21.15.69:0/2381803729 shutdown_connections 2024-08-02T17:55:12.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:12.411+0000 7fc2cafa2640 1 -- 172.21.15.69:0/2381803729 wait complete. 2024-08-02T17:55:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:12 smithi083 ceph-mon[37031]: pgmap v271: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-08-02T17:55:12.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:12 smithi077 ceph-mon[37108]: pgmap v271: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 4.8 MiB/s wr, 542 op/s 2024-08-02T17:55:13.378 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 -- 172.21.15.69:0/4156385347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862810bdd0 msgr2=0x7f862810e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 --2- 172.21.15.69:0/4156385347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862810bdd0 0x7f862810e270 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8620009ca0 tx=0x7f862002a950 comp rx=0 tx=0).stop 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 -- 172.21.15.69:0/4156385347 shutdown_connections 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 --2- 172.21.15.69:0/4156385347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86281033b0 0x7f862810b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 --2- 172.21.15.69:0/4156385347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f862810bdd0 0x7f862810e270 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 --2- 172.21.15.69:0/4156385347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8628102a90 0x7f8628102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 -- 172.21.15.69:0/4156385347 >> 172.21.15.69:0/4156385347 conn(0x7f86280776d0 msgr2=0x7f86280fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 -- 172.21.15.69:0/4156385347 shutdown_connections 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.527+0000 7f862fe51640 1 -- 172.21.15.69:0/4156385347 wait complete. 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862fe51640 1 Processor -- start 2024-08-02T17:55:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862fe51640 1 -- start start 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862fe51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8628102a90 0x7f86281006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862fe51640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86281033b0 0x7f8628100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862fe51640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f862810bdd0 0x7f8628101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862fe51640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8628111bd0 con 0x7f86281033b0 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862fe51640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8628111a50 con 0x7f862810bdd0 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862fe51640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8628111d50 con 0x7f8628102a90 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862d3ec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86281033b0 0x7f8628100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862d3ec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86281033b0 0x7f8628100c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51350/0 (socket says 172.21.15.69:51350) 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862d3ec640 1 -- 172.21.15.69:0/737249690 learned_addr learned my addr 172.21.15.69:0/737249690 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862d3ec640 1 -- 172.21.15.69:0/737249690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8628102a90 msgr2=0x7f86281006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862dbed640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8628102a90 0x7f86281006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862e3ee640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f862810bdd0 0x7f8628101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862d3ec640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8628102a90 0x7f86281006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862d3ec640 1 -- 172.21.15.69:0/737249690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f862810bdd0 msgr2=0x7f8628101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862d3ec640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f862810bdd0 0x7f8628101160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862d3ec640 1 -- 172.21.15.69:0/737249690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86281a6560 con 0x7f86281033b0 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.528+0000 7f862dbed640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8628102a90 0x7f86281006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.529+0000 7f862d3ec640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86281033b0 0x7f8628100c20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f861800ea60 tx=0x7f861800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.529+0000 7f861effd640 1 -- 172.21.15.69:0/737249690 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861800cde0 con 0x7f86281033b0 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.529+0000 7f862fe51640 1 -- 172.21.15.69:0/737249690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86281a6850 con 0x7f86281033b0 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.529+0000 7f861effd640 1 -- 172.21.15.69:0/737249690 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86180035f0 con 0x7f86281033b0 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.529+0000 7f862fe51640 1 -- 172.21.15.69:0/737249690 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86281a6df0 con 0x7f86281033b0 2024-08-02T17:55:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.529+0000 7f861effd640 1 -- 172.21.15.69:0/737249690 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618010670 con 0x7f86281033b0 2024-08-02T17:55:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.531+0000 7f861effd640 1 -- 172.21.15.69:0/737249690 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f86180037a0 con 0x7f86281033b0 2024-08-02T17:55:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.532+0000 7f861effd640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f86140a2cf0 0x7f86140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.532+0000 7f862dbed640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f86140a2cf0 0x7f86140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.532+0000 7f861effd640 1 -- 172.21.15.69:0/737249690 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f86180cfc50 con 0x7f86281033b0 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.532+0000 7f8632bab740 1 Processor -- start 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862dbed640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f86140a2cf0 0x7f86140a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8624009b40 tx=0x7f8624005e50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f8632bab740 1 -- start start 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f8632bab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1ba7b0 0x55df7c1bab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f8632bab740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f8632bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f8632bab740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df7c11e9b0 con 0x55df7c1bb0d0 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f8632bab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df7c0c6800 con 0x55df7c1ba7b0 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f8632bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df7c11a970 con 0x55df7c1c5970 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862d3ec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862d3ec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51358/0 (socket says 172.21.15.69:51358) 2024-08-02T17:55:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862d3ec640 1 -- 172.21.15.69:0/491633802 learned_addr learned my addr 172.21.15.69:0/491633802 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862e3ee640 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862dbed640 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1ba7b0 0x55df7c1bab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862d3ec640 1 -- 172.21.15.69:0/491633802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 msgr2=0x55df7c1c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862d3ec640 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862d3ec640 1 -- 172.21.15.69:0/491633802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1ba7b0 msgr2=0x55df7c1bab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862d3ec640 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1ba7b0 0x55df7c1bab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862d3ec640 1 -- 172.21.15.69:0/491633802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7be32210 con 0x55df7c1bb0d0 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862e3ee640 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.533+0000 7f862dbed640 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1ba7b0 0x55df7c1bab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.534+0000 7f862d3ec640 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c5430 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f861809bc90 tx=0x7f861809b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216bd3e9d69c6393 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.534+0000 7f8613fff640 1 -- 172.21.15.69:0/491633802 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86180a4970 con 0x55df7c1bb0d0 2024-08-02T17:55:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.534+0000 7f8613fff640 1 -- 172.21.15.69:0/491633802 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8618003b30 con 0x55df7c1bb0d0 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.534+0000 7f8613fff640 1 -- 172.21.15.69:0/491633802 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861809b150 con 0x55df7c1bb0d0 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 -- 172.21.15.69:0/491633802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 msgr2=0x55df7c1c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c5430 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f861809bc90 tx=0x7f861809b750 comp rx=0 tx=0).stop 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 -- 172.21.15.69:0/491633802 shutdown_connections 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c5430 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 --2- 172.21.15.69:0/491633802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1ba7b0 0x55df7c1bab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 -- 172.21.15.69:0/491633802 >> 172.21.15.69:0/491633802 conn(0x55df7c1b6d20 msgr2=0x55df7c1b7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 -- 172.21.15.69:0/491633802 shutdown_connections 2024-08-02T17:55:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 -- 172.21.15.69:0/491633802 wait complete. 2024-08-02T17:55:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 -- start start 2024-08-02T17:55:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.535+0000 7f8632bab740 1 Processor -- start 2024-08-02T17:55:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f8632bab740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f8632bab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f8632bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f8632bab740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df7c1c8540 con 0x55df7c1ba7b0 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f8632bab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df7c0da500 con 0x55df7c1bb0d0 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f8632bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df7c1c86c0 con 0x55df7c1c5970 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f8632bab740 10 client.0 _ll_register_callbacks cb 0x55df7be31a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51374/0 (socket says 172.21.15.69:51374) 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 -- 172.21.15.69:0/1827441009 learned_addr learned my addr 172.21.15.69:0/1827441009 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862e3ee640 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862d3ec640 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 -- 172.21.15.69:0/1827441009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 msgr2=0x55df7c1c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 -- 172.21.15.69:0/1827441009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1bb0d0 msgr2=0x55df7c1c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:13.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 -- 172.21.15.69:0/1827441009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7befe620 con 0x55df7c1ba7b0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f862dbed640 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c2830 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8624011890 tx=0x7f86240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518104c594bdcbca server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.536+0000 7f86117fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f86117fa640 1 -- 172.21.15.69:0/1827441009 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624022070 con 0x55df7c1ba7b0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f86117fa640 1 -- 172.21.15.69:0/1827441009 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f8624010d60 con 0x55df7c1ba7b0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 1 -- 172.21.15.69:0/1827441009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8618003b30 con 0x55df7c1ba7b0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f86117fa640 1 -- 172.21.15.69:0/1827441009 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f862402e050 con 0x55df7c1ba7b0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 10 client.17649 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 1 -- 172.21.15.69:0/1827441009 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df7c1d4150 con 0x55df7c1ba7b0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f86117fa640 1 -- 172.21.15.69:0/1827441009 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8618003b30 con 0x55df7c1ba7b0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f86117fa640 1 -- 172.21.15.69:0/1827441009 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f862400f040 con 0x55df7c1ba7b0 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 10 client.17649 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:13.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 10 client.17649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 20 client.17649 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55df7c1d55c0 0x55df7c1d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 4 client.17649 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f8632bab740 1 -- 172.21.15.69:0/1827441009 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df7c12c0e0 con 0x55df7c1d55c0 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.537+0000 7f862d3ec640 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55df7c1d55c0 0x55df7c1d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.538+0000 7f86117fa640 10 client.17649 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.538+0000 7f862d3ec640 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55df7c1d55c0 0x55df7c1d7a30 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ef0257434b3816 server_cookie=cb2dcc322abdfd5a in_seq=0 out_seq=0 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f86117fa640 1 -- 172.21.15.69:0/1827441009 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df7c12c0e0 con 0x55df7c1d55c0 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f86117fa640 10 client.17649 handle_command_reply: tid=0 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 client.17649 shutdown 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 -- 172.21.15.69:0/1827441009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 msgr2=0x55df7c1c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c2830 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8624011890 tx=0x7f86240046c0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 -- 172.21.15.69:0/1827441009 shutdown_connections 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55df7c1d55c0 0x55df7c1d7a30 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f86117fa640 0 client.17649 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1c5970 0x55df7c1c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 --2- 172.21.15.69:0/1827441009 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c2830 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 -- 172.21.15.69:0/1827441009 >> 172.21.15.69:0/1827441009 conn(0x55df7c1b6d20 msgr2=0x55df7c1bca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 -- 172.21.15.69:0/1827441009 shutdown_connections 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 1 -- 172.21.15.69:0/1827441009 wait complete. 2024-08-02T17:55:13.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.539+0000 7f8632bab740 20 client.17649 trim_cache size 0 max 16384 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.554+0000 7f8632bab740 1 Processor -- start 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.554+0000 7f8632bab740 1 -- start start 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.554+0000 7f8632bab740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1dd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.554+0000 7f8632bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1bb0d0 0x55df7c1dd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.554+0000 7f8632bab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c1bfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.554+0000 7f8632bab740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86281b3b00 con 0x55df7c1ba7b0 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.554+0000 7f8632bab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86281b13d0 con 0x55df7c1c5970 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.554+0000 7f8632bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86281af240 con 0x55df7c1bb0d0 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1dd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1dd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51386/0 (socket says 172.21.15.69:51386) 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 -- 172.21.15.69:0/1759620418 learned_addr learned my addr 172.21.15.69:0/1759620418 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862d3ec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1bb0d0 0x55df7c1dd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862e3ee640 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c1bfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 -- 172.21.15.69:0/1759620418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1bb0d0 msgr2=0x55df7c1dd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1bb0d0 0x55df7c1dd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 -- 172.21.15.69:0/1759620418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 msgr2=0x55df7c1bfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c1bfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 -- 172.21.15.69:0/1759620418 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df7c1b6b50 con 0x55df7c1ba7b0 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862e3ee640 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c1bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f862dbed640 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1dd390 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8624009310 tx=0x7f862400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f7c7d16ce2b298 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f86137fe640 1 -- 172.21.15.69:0/1759620418 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624022070 con 0x55df7c1ba7b0 2024-08-02T17:55:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f86137fe640 1 -- 172.21.15.69:0/1759620418 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8624017040 con 0x55df7c1ba7b0 2024-08-02T17:55:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.555+0000 7f86137fe640 1 -- 172.21.15.69:0/1759620418 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624004420 con 0x55df7c1ba7b0 2024-08-02T17:55:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 -- 172.21.15.69:0/1759620418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 msgr2=0x55df7c1dd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1dd390 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8624009310 tx=0x7f862400ebc0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 -- 172.21.15.69:0/1759620418 shutdown_connections 2024-08-02T17:55:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c1bfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1bb0d0 0x55df7c1dd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 --2- 172.21.15.69:0/1759620418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1ba7b0 0x55df7c1dd390 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 -- 172.21.15.69:0/1759620418 >> 172.21.15.69:0/1759620418 conn(0x55df7c1b6d20 msgr2=0x55df7c1c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 -- 172.21.15.69:0/1759620418 shutdown_connections 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 -- 172.21.15.69:0/1759620418 wait complete. 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.556+0000 7f8632bab740 1 -- start start 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f8632bab740 1 Processor -- start 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f8632bab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f8632bab740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f8632bab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c23f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f8632bab740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86281af890 con 0x55df7c1bb0d0 2024-08-02T17:55:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f8632bab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86281b0d00 con 0x55df7c1c5970 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f8632bab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86281aff60 con 0x55df7c1ba7b0 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f8632bab740 10 client.0 _ll_register_callbacks cb 0x7f8624009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862d3ec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862d3ec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51388/0 (socket says 172.21.15.69:51388) 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862d3ec640 1 -- 172.21.15.69:0/3817232711 learned_addr learned my addr 172.21.15.69:0/3817232711 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862e3ee640 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c23f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862dbed640 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862d3ec640 1 -- 172.21.15.69:0/3817232711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1ba7b0 msgr2=0x55df7c1c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862d3ec640 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862d3ec640 1 -- 172.21.15.69:0/3817232711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 msgr2=0x55df7c23f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862d3ec640 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c23f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.557+0000 7f862d3ec640 1 -- 172.21.15.69:0/3817232711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df7c1ca850 con 0x55df7c1bb0d0 2024-08-02T17:55:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f862d3ec640 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c9f90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f86180ccfe0 tx=0x7f8618009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51907b8602faa0ff server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f86117fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f86117fa640 1 -- 172.21.15.69:0/3817232711 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86180a4970 con 0x55df7c1bb0d0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f86117fa640 1 -- 172.21.15.69:0/3817232711 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f86180cd1b0 con 0x55df7c1bb0d0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 1 -- 172.21.15.69:0/3817232711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8624017040 con 0x55df7c1bb0d0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f86117fa640 1 -- 172.21.15.69:0/3817232711 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8618020390 con 0x55df7c1bb0d0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 10 client.17655 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 1 -- 172.21.15.69:0/3817232711 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df7c2409b0 con 0x55df7c1bb0d0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f86117fa640 1 -- 172.21.15.69:0/3817232711 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8624017040 con 0x55df7c1bb0d0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f86117fa640 1 -- 172.21.15.69:0/3817232711 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8618004090 con 0x55df7c1bb0d0 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 10 client.17655 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 20 client.17655 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55df7c1d55c0 0x55df7c242bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 4 client.17655 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:55:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.558+0000 7f8632bab740 1 -- 172.21.15.69:0/3817232711 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df7c126430 con 0x55df7c1d55c0 2024-08-02T17:55:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.559+0000 7f862dbed640 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55df7c1d55c0 0x55df7c242bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.559+0000 7f86117fa640 10 client.17655 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.559+0000 7f862dbed640 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55df7c1d55c0 0x55df7c242bf0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b758119a1079adc5 server_cookie=f3ddf59a12da98cc in_seq=0 out_seq=0 2024-08-02T17:55:13.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.582+0000 7f86117fa640 1 -- 172.21.15.69:0/3817232711 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df7c126430 con 0x55df7c1d55c0 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.582+0000 7f86117fa640 10 client.17655 handle_command_reply: tid=0 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.582+0000 7f8632bab740 1 client.17655 shutdown 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.582+0000 7f8632bab740 1 -- 172.21.15.69:0/3817232711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 msgr2=0x55df7c1c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.582+0000 7f8632bab740 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c9f90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f86180ccfe0 tx=0x7f8618009310 comp rx=0 tx=0).stop 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 1 -- 172.21.15.69:0/3817232711 shutdown_connections 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df7c1bb0d0 0x55df7c1c9f90 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55df7c1d55c0 0x55df7c242bf0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f86117fa640 0 client.17655 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df7c1c5970 0x55df7c23f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 1 --2- 172.21.15.69:0/3817232711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df7c1ba7b0 0x55df7c1c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 1 -- 172.21.15.69:0/3817232711 >> 172.21.15.69:0/3817232711 conn(0x55df7c1b6d20 msgr2=0x55df7c1bc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:13.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 1 -- 172.21.15.69:0/3817232711 shutdown_connections 2024-08-02T17:55:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 1 -- 172.21.15.69:0/3817232711 wait complete. 2024-08-02T17:55:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f8632bab740 20 client.17655 trim_cache size 0 max 16384 2024-08-02T17:55:13.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:55:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:55:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:55:13.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f862fe51640 1 -- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f86140a2cf0 msgr2=0x7f86140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f862fe51640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f86140a2cf0 0x7f86140a51b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8624009b40 tx=0x7f8624005e50 comp rx=0 tx=0).stop 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f862fe51640 1 -- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86281033b0 msgr2=0x7f8628100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.583+0000 7f862fe51640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86281033b0 0x7f8628100c20 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f861800ea60 tx=0x7f861800ef20 comp rx=0 tx=0).stop 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.584+0000 7f862fe51640 1 -- 172.21.15.69:0/737249690 shutdown_connections 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.584+0000 7f862fe51640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86281033b0 0x7f8628100c20 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.584+0000 7f862fe51640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f86140a2cf0 0x7f86140a51b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.584+0000 7f862fe51640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f862810bdd0 0x7f8628101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.584+0000 7f862fe51640 1 --2- 172.21.15.69:0/737249690 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8628102a90 0x7f86281006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.584+0000 7f862fe51640 1 -- 172.21.15.69:0/737249690 >> 172.21.15.69:0/737249690 conn(0x7f86280776d0 msgr2=0x7f86280fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:13.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.584+0000 7f862fe51640 1 -- 172.21.15.69:0/737249690 shutdown_connections 2024-08-02T17:55:13.589 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:13.584+0000 7f862fe51640 1 -- 172.21.15.69:0/737249690 wait complete. 2024-08-02T17:55:13.657 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:14.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 -- 172.21.15.69:0/3962085306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d841022a0 msgr2=0x7f1d841026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3962085306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d841022a0 0x7f1d841026e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1d78009900 tx=0x7f1d7802a480 comp rx=0 tx=0).stop 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 -- 172.21.15.69:0/3962085306 shutdown_connections 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3962085306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d841022a0 0x7f1d841026e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3962085306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d84102c20 0x7f1d8410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3962085306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d84100b50 0x7f1d84100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 -- 172.21.15.69:0/3962085306 >> 172.21.15.69:0/3962085306 conn(0x7f1d840fc500 msgr2=0x7f1d840fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 -- 172.21.15.69:0/3962085306 shutdown_connections 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.171+0000 7f1d891ce640 1 -- 172.21.15.69:0/3962085306 wait complete. 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d891ce640 1 Processor -- start 2024-08-02T17:55:14.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d891ce640 1 -- start start 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d891ce640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d84100b50 0x7f1d840fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d891ce640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d841022a0 0x7f1d840fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d891ce640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d84102c20 0x7f1d840fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d891ce640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84110cb0 con 0x7f1d84100b50 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d891ce640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84110b30 con 0x7f1d841022a0 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d891ce640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d84110e30 con 0x7f1d84102c20 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d82d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d84100b50 0x7f1d840fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d82d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d84100b50 0x7f1d840fc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51400/0 (socket says 172.21.15.69:51400) 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d82d9d640 1 -- 172.21.15.69:0/3732112235 learned_addr learned my addr 172.21.15.69:0/3732112235 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d8259c640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d841022a0 0x7f1d840fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d8359e640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d84102c20 0x7f1d840fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d82d9d640 1 -- 172.21.15.69:0/3732112235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d84102c20 msgr2=0x7f1d840fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d84102c20 0x7f1d840fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d82d9d640 1 -- 172.21.15.69:0/3732112235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d841022a0 msgr2=0x7f1d840fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d841022a0 0x7f1d840fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d82d9d640 1 -- 172.21.15.69:0/3732112235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d841a66c0 con 0x7f1d84100b50 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.172+0000 7f1d8359e640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d84102c20 0x7f1d840fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.173+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d84100b50 0x7f1d840fc9b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400d770 tx=0x7f1d7400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.173+0000 7f1d6bfff640 1 -- 172.21.15.69:0/3732112235 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74017070 con 0x7f1d84100b50 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.173+0000 7f1d891ce640 1 -- 172.21.15.69:0/3732112235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d841a69b0 con 0x7f1d84100b50 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.173+0000 7f1d6bfff640 1 -- 172.21.15.69:0/3732112235 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d74005500 con 0x7f1d84100b50 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.173+0000 7f1d6bfff640 1 -- 172.21.15.69:0/3732112235 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74004e00 con 0x7f1d84100b50 2024-08-02T17:55:14.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.173+0000 7f1d891ce640 1 -- 172.21.15.69:0/3732112235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d841a6ec0 con 0x7f1d84100b50 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d6bfff640 1 -- 172.21.15.69:0/3732112235 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1d74004fa0 con 0x7f1d84100b50 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8b5cc740 1 Processor -- start 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8b5cc740 1 -- start start 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 0x55c10c9d95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9e43f0 0x55c10c9e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c10c93e980 con 0x55c10c9d9b30 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c10c8d6a70 con 0x55c10c9e43f0 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c10c93a940 con 0x55c10c9d9210 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d82d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d82d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51412/0 (socket says 172.21.15.69:51412) 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d82d9d640 1 -- 172.21.15.69:0/2719942054 learned_addr learned my addr 172.21.15.69:0/2719942054 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d6bfff640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1d6c0a2cf0 0x7f1d6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.176+0000 7f1d8359e640 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 0x55c10c9d95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d8259c640 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9e43f0 0x55c10c9e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d8359e640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1d6c0a2cf0 0x7f1d6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d6bfff640 1 -- 172.21.15.69:0/3732112235 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f1d740cf2f0 con 0x7f1d84100b50 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d82d9d640 1 -- 172.21.15.69:0/2719942054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 msgr2=0x55c10c9d95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 0x55c10c9d95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d82d9d640 1 -- 172.21.15.69:0/2719942054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9e43f0 msgr2=0x55c10c9e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9e43f0 0x55c10c9e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d82d9d640 1 -- 172.21.15.69:0/2719942054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10c9396e0 con 0x55c10c9d9b30 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e3eb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1d7409b0a0 tx=0x7f1d74009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71adf352e00b592 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d8359e640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1d6c0a2cf0 0x7f1d6c0a51b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005c40 tx=0x7f1d7c009280 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d697fa640 1 -- 172.21.15.69:0/2719942054 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74020070 con 0x55c10c9d9b30 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d697fa640 1 -- 172.21.15.69:0/2719942054 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d740046c0 con 0x55c10c9d9b30 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.177+0000 7f1d697fa640 1 -- 172.21.15.69:0/2719942054 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74009cc0 con 0x55c10c9d9b30 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2719942054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 msgr2=0x55c10c9e3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e3eb0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1d7409b0a0 tx=0x7f1d74009c30 comp rx=0 tx=0).stop 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2719942054 shutdown_connections 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e3eb0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9e43f0 0x55c10c9e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2719942054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 0x55c10c9d95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2719942054 >> 172.21.15.69:0/2719942054 conn(0x55c10c9d57a0 msgr2=0x55c10c9d5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2719942054 shutdown_connections 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2719942054 wait complete. 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 -- start start 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.178+0000 7f1d8b5cc740 1 Processor -- start 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9210 0x55c10c9e1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9e43f0 0x55c10c9e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c10c9e6f60 con 0x55c10c9d9b30 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c10c8f9aa0 con 0x55c10c9d9210 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c10c9e70e0 con 0x55c10c9e43f0 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8b5cc740 10 client.0 _ll_register_callbacks cb 0x55c10c8e8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:14.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51424/0 (socket says 172.21.15.69:51424) 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 -- 172.21.15.69:0/2237826525 learned_addr learned my addr 172.21.15.69:0/2237826525 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8359e640 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9e43f0 0x55c10c9e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9210 0x55c10c9e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 -- 172.21.15.69:0/2237826525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9e43f0 msgr2=0x55c10c9e1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9e43f0 0x55c10c9e1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 -- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9210 msgr2=0x55c10c9e1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9210 0x55c10c9e1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 -- 172.21.15.69:0/2237826525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10c8dd650 con 0x55c10c9d9b30 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9210 0x55c10c9e1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8259c640 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e17c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1d78000c00 tx=0x7f1d78004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93f188f15d1b5aa server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:14.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d66ff5640 1 -- 172.21.15.69:0/2237826525 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d78035040 con 0x55c10c9d9b30 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d66ff5640 1 -- 172.21.15.69:0/2237826525 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f1d78003a00 con 0x55c10c9d9b30 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.179+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2237826525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c10c94bef0 con 0x55c10c9d9b30 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d66ff5640 1 -- 172.21.15.69:0/2237826525 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d7804e050 con 0x55c10c9d9b30 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d8b5cc740 10 client.17667 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2237826525 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c10c6b03c0 con 0x55c10c9d9b30 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d66ff5640 1 -- 172.21.15.69:0/2237826525 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c10c94bef0 con 0x55c10c9d9b30 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d66ff5640 1 -- 172.21.15.69:0/2237826525 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d7803c440 con 0x55c10c9d9b30 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d8b5cc740 10 client.17667 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d8b5cc740 10 client.17667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d8b5cc740 20 client.17667 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:14.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c10c9f3e50 0x55c10c9f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d8b5cc740 4 client.17667 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2237826525 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c10c946400 con 0x55c10c9f3e50 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.180+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c10c9f3e50 0x55c10c9f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d66ff5640 10 client.17667 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c10c9f3e50 0x55c10c9f62c0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1a69a859b3dbe81 server_cookie=3575470ff74474c2 in_seq=0 out_seq=0 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d66ff5640 1 -- 172.21.15.69:0/2237826525 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c10c946400 con 0x55c10c9f3e50 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d66ff5640 10 client.17667 handle_command_reply: tid=0 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 client.17667 shutdown 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2237826525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 msgr2=0x55c10c9e17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e17c0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f1d78000c00 tx=0x7f1d78004430 comp rx=0 tx=0).stop 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2237826525 shutdown_connections 2024-08-02T17:55:14.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9b30 0x55c10c9e17c0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c10c9f3e50 0x55c10c9f62c0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9e43f0 0x55c10c9e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/2237826525 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9210 0x55c10c9e1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2237826525 >> 172.21.15.69:0/2237826525 conn(0x55c10c9d57a0 msgr2=0x55c10c9db480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:14.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.181+0000 7f1d66ff5640 0 client.17667 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:14.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.182+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2237826525 shutdown_connections 2024-08-02T17:55:14.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.182+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/2237826525 wait complete. 2024-08-02T17:55:14.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.182+0000 7f1d8b5cc740 20 client.17667 trim_cache size 0 max 16384 2024-08-02T17:55:14.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8b5cc740 1 Processor -- start 2024-08-02T17:55:14.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8b5cc740 1 -- start start 2024-08-02T17:55:14.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:14 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:55:14.207 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:14 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:55:14.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:14 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:55:14.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:14 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:55:14.208 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:14 smithi069 ceph-mon[32746]: pgmap v272: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-08-02T17:55:14.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d57a0 0x55c10c9d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 0x55c10c9e30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 0x55c10c9fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841b3a40 con 0x55c10c9d57a0 2024-08-02T17:55:14.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841b1310 con 0x55c10c9d9b30 2024-08-02T17:55:14.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841af180 con 0x55c10c9d9210 2024-08-02T17:55:14.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d82d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d57a0 0x55c10c9d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d82d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d57a0 0x55c10c9d5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51430/0 (socket says 172.21.15.69:51430) 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d82d9d640 1 -- 172.21.15.69:0/3589486776 learned_addr learned my addr 172.21.15.69:0/3589486776 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8259c640 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 0x55c10c9e30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.206+0000 7f1d8359e640 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 0x55c10c9fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d82d9d640 1 -- 172.21.15.69:0/3589486776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 msgr2=0x55c10c9e30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 0x55c10c9e30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d82d9d640 1 -- 172.21.15.69:0/3589486776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 msgr2=0x55c10c9fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 0x55c10c9fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d82d9d640 1 -- 172.21.15.69:0/3589486776 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c10c9dc080 con 0x55c10c9d57a0 2024-08-02T17:55:14.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d57a0 0x55c10c9d5b80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1d740048e0 tx=0x7f1d74097520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e0c7ad41283f52 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:14.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d68ff9640 1 -- 172.21.15.69:0/3589486776 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74020040 con 0x55c10c9d57a0 2024-08-02T17:55:14.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d68ff9640 1 -- 172.21.15.69:0/3589486776 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d740a4010 con 0x55c10c9d57a0 2024-08-02T17:55:14.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.207+0000 7f1d68ff9640 1 -- 172.21.15.69:0/3589486776 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74097a10 con 0x55c10c9d57a0 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/3589486776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d57a0 msgr2=0x55c10c9d5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d57a0 0x55c10c9d5b80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f1d740048e0 tx=0x7f1d74097520 comp rx=0 tx=0).stop 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/3589486776 shutdown_connections 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 0x55c10c9fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d9210 0x55c10c9e30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/3589486776 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d57a0 0x55c10c9d5b80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/3589486776 >> 172.21.15.69:0/3589486776 conn(0x55c10c9dcc70 msgr2=0x55c10c9e29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/3589486776 shutdown_connections 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/3589486776 wait complete. 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.208+0000 7f1d8b5cc740 1 -- start start 2024-08-02T17:55:14.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8b5cc740 1 Processor -- start 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d57a0 0x55c10c9eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9210 0x55c10c9e7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8b5cc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 0x55c10c9e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841af7d0 con 0x55c10c9d9210 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841b0c40 con 0x55c10c9d9b30 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8b5cc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1d841afea0 con 0x55c10c9d57a0 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8b5cc740 10 client.0 _ll_register_callbacks cb 0x7f1d78009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d82d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d57a0 0x55c10c9eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d82d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d57a0 0x55c10c9eca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:54898/0 (socket says 172.21.15.69:54898) 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d82d9d640 1 -- 172.21.15.69:0/1776027385 learned_addr learned my addr 172.21.15.69:0/1776027385 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:14.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8259c640 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9210 0x55c10c9e7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d8359e640 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 0x55c10c9e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.209+0000 7f1d82d9d640 1 -- 172.21.15.69:0/1776027385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 msgr2=0x55c10c9e82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 0x55c10c9e82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d82d9d640 1 -- 172.21.15.69:0/1776027385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9210 msgr2=0x55c10c9e7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9210 0x55c10c9e7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d82d9d640 1 -- 172.21.15.69:0/1776027385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c10c9e8b10 con 0x55c10c9d57a0 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d8259c640 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9210 0x55c10c9e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d82d9d640 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d57a0 0x55c10c9eca70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d7409b330 tx=0x7f1d740cf5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3269de773f722c0e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d66ff5640 1 -- 172.21.15.69:0/1776027385 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d74020050 con 0x55c10c9d57a0 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/1776027385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d740a4010 con 0x55c10c9d57a0 2024-08-02T17:55:14.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d66ff5640 1 -- 172.21.15.69:0/1776027385 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9656+0+0 (secure 0 0 0) 0x7f1d740cc850 con 0x55c10c9d57a0 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d66ff5640 1 -- 172.21.15.69:0/1776027385 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1d7409f750 con 0x55c10c9d57a0 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d8b5cc740 10 client.26986 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/1776027385 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c10ca5eb60 con 0x55c10c9d57a0 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.210+0000 7f1d66ff5640 1 -- 172.21.15.69:0/1776027385 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d740a4010 con 0x55c10c9d57a0 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d66ff5640 1 -- 172.21.15.69:0/1776027385 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1d740970f0 con 0x55c10c9d57a0 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d8b5cc740 10 client.26986 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d8b5cc740 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d8b5cc740 20 client.26986 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c10c9e43f0 0x55c10ca60e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d8b5cc740 4 client.26986 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:55:14.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/1776027385 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c10c8c3210 con 0x55c10c9e43f0 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d8259c640 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c10c9e43f0 0x55c10ca60e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d66ff5640 10 client.26986 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.211+0000 7f1d8259c640 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c10c9e43f0 0x55c10ca60e00 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f816f916f81e9643 server_cookie=b1b4548ca5671995 in_seq=0 out_seq=0 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d66ff5640 1 -- 172.21.15.69:0/1776027385 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c10c8c3210 con 0x55c10c9e43f0 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d66ff5640 10 client.26986 handle_command_reply: tid=0 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 client.26986 shutdown 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/1776027385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d57a0 msgr2=0x55c10c9eca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d57a0 0x55c10c9eca70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1d7409b330 tx=0x7f1d740cf5f0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/1776027385 shutdown_connections 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c10c9d9210 0x55c10c9e7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c10c9e43f0 0x55c10ca60e00 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c10c9d9b30 0x55c10c9e82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 --2- 172.21.15.69:0/1776027385 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c10c9d57a0 0x55c10c9eca70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d66ff5640 0 client.26986 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/1776027385 >> 172.21.15.69:0/1776027385 conn(0x55c10c9dcc70 msgr2=0x55c10c9d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/1776027385 shutdown_connections 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 1 -- 172.21.15.69:0/1776027385 wait complete. 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.212+0000 7f1d8b5cc740 20 client.26986 trim_cache size 0 max 16384 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:55:14.220 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:55:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:55:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: "789553e5-6d8c-42ea-b7d0-02cc643b33c4": { 2024-08-02T17:55:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:55:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "789553e5-6d8c-42ea-b7d0-02cc643b33c4", 2024-08-02T17:55:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:55:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:14.221 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:14.221 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:55:14.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 -- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1d6c0a2cf0 msgr2=0x7f1d6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1d6c0a2cf0 0x7f1d6c0a51b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c005c40 tx=0x7f1d7c009280 comp rx=0 tx=0).stop 2024-08-02T17:55:14.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 -- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d84100b50 msgr2=0x7f1d840fc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:14.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d84100b50 0x7f1d840fc9b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400d770 tx=0x7f1d7400dc30 comp rx=0 tx=0).stop 2024-08-02T17:55:14.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 -- 172.21.15.69:0/3732112235 shutdown_connections 2024-08-02T17:55:14.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1d6c0a2cf0 0x7f1d6c0a51b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1d84102c20 0x7f1d840fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1d841022a0 0x7f1d840fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 --2- 172.21.15.69:0/3732112235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1d84100b50 0x7f1d840fc9b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:14.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 -- 172.21.15.69:0/3732112235 >> 172.21.15.69:0/3732112235 conn(0x7f1d840fc500 msgr2=0x7f1d84069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:14.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 -- 172.21.15.69:0/3732112235 shutdown_connections 2024-08-02T17:55:14.223 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:14.213+0000 7f1d891ce640 1 -- 172.21.15.69:0/3732112235 wait complete. 2024-08-02T17:55:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:14 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:55:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:14 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:55:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:14 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:55:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:14 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:55:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:14 smithi083 ceph-mon[37031]: pgmap v272: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-08-02T17:55:14.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:14 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:55:14.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:14 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:55:14.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:14 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:55:14.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:14 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:55:14.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:14 smithi077 ceph-mon[37108]: pgmap v272: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-08-02T17:55:14.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 485.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:55:15.027 INFO:tasks.cephfs.filesystem:scrub status for tag:789553e5-6d8c-42ea-b7d0-02cc643b33c4 - {'path': '/', 'tag': '789553e5-6d8c-42ea-b7d0-02cc643b33c4', 'options': 'recursive,force'} 2024-08-02T17:55:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:15 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:55:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:15 smithi083 ceph-mon[37031]: osdmap e101: 12 total, 12 up, 12 in 2024-08-02T17:55:15.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:15 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:55:15.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:15 smithi077 ceph-mon[37108]: osdmap e101: 12 total, 12 up, 12 in 2024-08-02T17:55:15.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:15 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:55:15.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:15 smithi069 ceph-mon[32746]: osdmap e101: 12 total, 12 up, 12 in 2024-08-02T17:55:15.743 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:55:16.002 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:16.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 -- 172.21.15.69:0/2637490992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd0100b60 msgr2=0x7f0bd0104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:16.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/2637490992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd0100b60 0x7f0bd0104bb0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009900 tx=0x7f0bb802a410 comp rx=0 tx=0).stop 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 -- 172.21.15.69:0/2637490992 shutdown_connections 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/2637490992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd0100b60 0x7f0bd0104bb0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/2637490992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd01052e0 0x7f0bd010ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/2637490992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd01001b0 0x7f0bd0100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 -- 172.21.15.69:0/2637490992 >> 172.21.15.69:0/2637490992 conn(0x7f0bd0076f40 msgr2=0x7f0bd00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 -- 172.21.15.69:0/2637490992 shutdown_connections 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 -- 172.21.15.69:0/2637490992 wait complete. 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 Processor -- start 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 -- start start 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd01001b0 0x7f0bd0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.531+0000 7f0bd61d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd0100b60 0x7f0bd01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bd61d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd01052e0 0x7f0bd019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:16.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bd61d0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0111cc0 con 0x7f0bd01001b0 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bd61d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0111b40 con 0x7f0bd0100b60 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bd61d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0111e40 con 0x7f0bd01052e0 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd01001b0 0x7f0bd0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd01001b0 0x7f0bd0198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39774/0 (socket says 172.21.15.69:39774) 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 -- 172.21.15.69:0/1566361559 learned_addr learned my addr 172.21.15.69:0/1566361559 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd0100b60 0x7f0bd01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcffff640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd01052e0 0x7f0bd019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 -- 172.21.15.69:0/1566361559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd01052e0 msgr2=0x7f0bd019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd01052e0 0x7f0bd019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 -- 172.21.15.69:0/1566361559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd0100b60 msgr2=0x7f0bd01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd0100b60 0x7f0bd01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.554 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 -- 172.21.15.69:0/1566361559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd019d130 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcf7fe640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd01001b0 0x7f0bd0198180 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000d950 tx=0x7f0bc000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcd7fa640 1 -- 172.21.15.69:0/1566361559 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0017070 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcd7fa640 1 -- 172.21.15.69:0/1566361559 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc0009c30 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd019d420 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bcd7fa640 1 -- 172.21.15.69:0/1566361559 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000f6b0 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.532+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd01a4d00 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.533+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd0100590 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.534+0000 7f0bcd7fa640 1 -- 172.21.15.69:0/1566361559 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0bc00109d0 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.535+0000 7f0bcd7fa640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bbc0a2d70 0x7f0bbc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.535+0000 7f0bcd7fa640 1 -- 172.21.15.69:0/1566361559 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f0bc00cf370 con 0x7f0bd01001b0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.535+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bbc0a2d70 0x7f0bbc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.536+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bbc0a2d70 0x7f0bbc0a5230 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009230 tx=0x7f0bb80058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:16.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.538+0000 7f0bcd7fa640 1 -- 172.21.15.69:0/1566361559 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f0bc0009de0 con 0x7f0bd01001b0 2024-08-02T17:55:16.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.741+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0bd01084f0 con 0x7f0bd01001b0 2024-08-02T17:55:16.743 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:16 smithi069 ceph-mon[32746]: pgmap v274: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 910 KiB/s wr, 122 op/s 2024-08-02T17:55:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.742+0000 7f0bcd7fa640 1 -- 172.21.15.69:0/1566361559 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f0bc009b5b0 con 0x7f0bd01001b0 2024-08-02T17:55:16.744 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:55:16.744 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:55:16.744 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bbc0a2d70 msgr2=0x7f0bbc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bbc0a2d70 0x7f0bbc0a5230 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8009230 tx=0x7f0bb80058d0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd01001b0 msgr2=0x7f0bd0198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd01001b0 0x7f0bd0198180 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000d950 tx=0x7f0bc000de10 comp rx=0 tx=0).stop 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 shutdown_connections 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bbc0a2d70 0x7f0bbc0a5230 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bd01052e0 0x7f0bd019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bd0100b60 0x7f0bd01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 --2- 172.21.15.69:0/1566361559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bd01001b0 0x7f0bd0198180 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:16.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 >> 172.21.15.69:0/1566361559 conn(0x7f0bd0076f40 msgr2=0x7f0bd01035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:16.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 shutdown_connections 2024-08-02T17:55:16.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:16.744+0000 7f0bd61d0640 1 -- 172.21.15.69:0/1566361559 wait complete. 2024-08-02T17:55:16.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:16 smithi077 ceph-mon[37108]: pgmap v274: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 910 KiB/s wr, 122 op/s 2024-08-02T17:55:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:16 smithi083 ceph-mon[37031]: pgmap v274: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 910 KiB/s wr, 122 op/s 2024-08-02T17:55:17.469 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 65e61135 --release --await' 2024-08-02T17:55:17.469 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 65e61135 --release --await 2024-08-02T17:55:17.724 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:17.846 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:17 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1566361559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:55:17.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:17 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1566361559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:55:18.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:17 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1566361559' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.236+0000 7fd87dbaa640 1 -- 172.21.15.69:0/242272845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878102c20 msgr2=0x7fd87810d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.236+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/242272845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878102c20 0x7fd87810d770 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd870009ca0 tx=0x7fd87002a950 comp rx=0 tx=0).stop 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.236+0000 7fd87dbaa640 1 -- 172.21.15.69:0/242272845 shutdown_connections 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.236+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/242272845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878102c20 0x7fd87810d770 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.236+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/242272845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8781022a0 0x7fd8781026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/242272845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd878100b50 0x7fd878100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 -- 172.21.15.69:0/242272845 >> 172.21.15.69:0/242272845 conn(0x7fd8780fc500 msgr2=0x7fd8780fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 -- 172.21.15.69:0/242272845 shutdown_connections 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 -- 172.21.15.69:0/242272845 wait complete. 2024-08-02T17:55:18.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 Processor -- start 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 -- start start 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878100b50 0x7fd878198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8781022a0 0x7fd878198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd878102c20 0x7fd87819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.237+0000 7fd87dbaa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd878110cf0 con 0x7fd878100b50 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd87dbaa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd878110b70 con 0x7fd8781022a0 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd87dbaa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd878110e70 con 0x7fd878102c20 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878100b50 0x7fd878198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878100b50 0x7fd878198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39802/0 (socket says 172.21.15.69:39802) 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 -- 172.21.15.69:0/3579426805 learned_addr learned my addr 172.21.15.69:0/3579426805 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd86effd640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8781022a0 0x7fd878198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd877fff640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd878102c20 0x7fd87819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 -- 172.21.15.69:0/3579426805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd878102c20 msgr2=0x7fd87819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd878102c20 0x7fd87819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 -- 172.21.15.69:0/3579426805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8781022a0 msgr2=0x7fd878198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8781022a0 0x7fd878198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 -- 172.21.15.69:0/3579426805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd87819d1f0 con 0x7fd878100b50 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8777fe640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878100b50 0x7fd878198240 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd86800b4a0 tx=0x7fd86800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8757fa640 1 -- 172.21.15.69:0/3579426805 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868014070 con 0x7fd878100b50 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd87dbaa640 1 -- 172.21.15.69:0/3579426805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd87819d4e0 con 0x7fd878100b50 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8757fa640 1 -- 172.21.15.69:0/3579426805 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd868010af0 con 0x7fd878100b50 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd8757fa640 1 -- 172.21.15.69:0/3579426805 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868017d10 con 0x7fd878100b50 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.238+0000 7fd87dbaa640 1 -- 172.21.15.69:0/3579426805 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8781a4dc0 con 0x7fd878100b50 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd8757fa640 1 -- 172.21.15.69:0/3579426805 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd868017070 con 0x7fd878100b50 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd87ffa8740 1 Processor -- start 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd87ffa8740 1 -- start start 2024-08-02T17:55:18.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c2fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c3a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 0x55a365c3cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a365b95c60 con 0x55a365c2f990 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a365b2db90 con 0x55a365c302b0 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a365b91c20 con 0x55a365c3ab70 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd8757fa640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd8640a2cf0 0x7fd8640a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd877fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c2fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd877fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c2fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39806/0 (socket says 172.21.15.69:39806) 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.241+0000 7fd877fff640 1 -- 172.21.15.69:0/3052649420 learned_addr learned my addr 172.21.15.69:0/3052649420 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd877fff640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd8640a2cf0 0x7fd8640a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd86effd640 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 0x55a365c3cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd8757fa640 1 -- 172.21.15.69:0/3579426805 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fd8680cea70 con 0x7fd878100b50 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd8777fe640 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c3a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd877fff640 1 -- 172.21.15.69:0/3052649420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 msgr2=0x55a365c3cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd877fff640 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 0x55a365c3cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd877fff640 1 -- 172.21.15.69:0/3052649420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 msgr2=0x55a365c3a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd877fff640 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c3a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd877fff640 1 -- 172.21.15.69:0/3052649420 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3658a5580 con 0x55a365c2f990 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd877fff640 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c2fd70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd870009a60 tx=0x7fd87002f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7882decb4dd84d2f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd877fff640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd8640a2cf0 0x7fd8640a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd870005fe0 tx=0x7fd870043000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd86dffb640 1 -- 172.21.15.69:0/3052649420 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870034040 con 0x55a365c2f990 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd86dffb640 1 -- 172.21.15.69:0/3052649420 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87004e070 con 0x55a365c2f990 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.242+0000 7fd86dffb640 1 -- 172.21.15.69:0/3052649420 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700052b0 con 0x55a365c2f990 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 -- 172.21.15.69:0/3052649420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 msgr2=0x55a365c2fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c2fd70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd870009a60 tx=0x7fd87002f760 comp rx=0 tx=0).stop 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 -- 172.21.15.69:0/3052649420 shutdown_connections 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 0x55a365c3cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c3a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/3052649420 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c2fd70 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 -- 172.21.15.69:0/3052649420 >> 172.21.15.69:0/3052649420 conn(0x55a365c2bf20 msgr2=0x55a365c2c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 -- 172.21.15.69:0/3052649420 shutdown_connections 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 -- 172.21.15.69:0/3052649420 wait complete. 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.243+0000 7fd87ffa8740 1 -- start start 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd87ffa8740 1 Processor -- start 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c37ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 0x55a365c38530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a365c3d740 con 0x55a365c2f990 2024-08-02T17:55:18.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a365b337c0 con 0x55a365c302b0 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a365c3d8c0 con 0x55a365c3ab70 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd87ffa8740 10 client.0 _ll_register_callbacks cb 0x55a365b3cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c37ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39808/0 (socket says 172.21.15.69:39808) 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 -- 172.21.15.69:0/2329283717 learned_addr learned my addr 172.21.15.69:0/2329283717 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd86effd640 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c37ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd877fff640 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 0x55a365c38530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 -- 172.21.15.69:0/2329283717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 msgr2=0x55a365c38530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 0x55a365c38530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 -- 172.21.15.69:0/2329283717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 msgr2=0x55a365c37ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c37ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 -- 172.21.15.69:0/2329283717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36596f960 con 0x55a365c2f990 2024-08-02T17:55:18.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd8777fe640 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c37ab0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd868097710 tx=0x7fd868009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7a6365f2414eb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.244+0000 7fd856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd856ffd640 1 -- 172.21.15.69:0/2329283717 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86801e040 con 0x55a365c2f990 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd856ffd640 1 -- 172.21.15.69:0/2329283717 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fd868010ca0 con 0x55a365c2f990 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 1 -- 172.21.15.69:0/2329283717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd87004e070 con 0x55a365c2f990 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd856ffd640 1 -- 172.21.15.69:0/2329283717 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd86809f040 con 0x55a365c2f990 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 10 client.17691 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 1 -- 172.21.15.69:0/2329283717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a365c49830 con 0x55a365c2f990 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd856ffd640 1 -- 172.21.15.69:0/2329283717 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87004e070 con 0x55a365c2f990 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd856ffd640 1 -- 172.21.15.69:0/2329283717 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd868061070 con 0x55a365c2f990 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 10 client.17691 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:18.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 10 client.17691 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 20 client.17691 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a365c4ac90 0x55a365c4d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 4 client.17691 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd87ffa8740 1 -- 172.21.15.69:0/2329283717 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a365ba31d0 con 0x55a365c4ac90 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.245+0000 7fd86effd640 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a365c4ac90 0x55a365c4d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.246+0000 7fd856ffd640 10 client.17691 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.246+0000 7fd86effd640 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a365c4ac90 0x55a365c4d0c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15c1115815e478b server_cookie=493e2c3b7c9c8d6d in_seq=0 out_seq=0 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd856ffd640 1 -- 172.21.15.69:0/2329283717 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a365ba31d0 con 0x55a365c4ac90 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd856ffd640 10 client.17691 handle_command_reply: tid=0 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 client.17691 shutdown 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 -- 172.21.15.69:0/2329283717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 msgr2=0x55a365c37ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c37ab0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd868097710 tx=0x7fd868009230 comp rx=0 tx=0).stop 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 -- 172.21.15.69:0/2329283717 shutdown_connections 2024-08-02T17:55:18.269 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a365c4ac90 0x55a365c4d0c0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c3ab70 0x55a365c38530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c37ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/2329283717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c2f990 0x55a365c37ab0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd856ffd640 0 client.17691 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:18.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 -- 172.21.15.69:0/2329283717 >> 172.21.15.69:0/2329283717 conn(0x55a365c2bf20 msgr2=0x55a365c2b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:18.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 -- 172.21.15.69:0/2329283717 shutdown_connections 2024-08-02T17:55:18.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 1 -- 172.21.15.69:0/2329283717 wait complete. 2024-08-02T17:55:18.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.247+0000 7fd87ffa8740 20 client.17691 trim_cache size 0 max 16384 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd87ffa8740 1 Processor -- start 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd87ffa8740 1 -- start start 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c2f990 0x55a365c31af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c42ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365c43430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781b18d0 con 0x55a365c3ab70 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781af1a0 con 0x55a365c302b0 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781acff0 con 0x55a365c2f990 2024-08-02T17:55:18.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd877fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365c43430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd877fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365c43430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39816/0 (socket says 172.21.15.69:39816) 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd877fff640 1 -- 172.21.15.69:0/286366438 learned_addr learned my addr 172.21.15.69:0/286366438 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd8777fe640 1 --2- 172.21.15.69:0/286366438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c2f990 0x55a365c31af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd877fff640 1 -- 172.21.15.69:0/286366438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c2f990 msgr2=0x55a365c31af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd877fff640 1 --2- 172.21.15.69:0/286366438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c2f990 0x55a365c31af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd877fff640 1 -- 172.21.15.69:0/286366438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 msgr2=0x55a365c42ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd877fff640 1 --2- 172.21.15.69:0/286366438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c42ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.264+0000 7fd877fff640 1 -- 172.21.15.69:0/286366438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a365c2cfd0 con 0x55a365c3ab70 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd877fff640 1 --2- 172.21.15.69:0/286366438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365c43430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd870009ca0 tx=0x7fd870005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33091c5a6a4f895 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd86d7fa640 1 -- 172.21.15.69:0/286366438 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870056070 con 0x55a365c3ab70 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd86d7fa640 1 -- 172.21.15.69:0/286366438 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87004e070 con 0x55a365c3ab70 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd86d7fa640 1 -- 172.21.15.69:0/286366438 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870030440 con 0x55a365c3ab70 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 -- 172.21.15.69:0/286366438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 msgr2=0x55a365c43430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/286366438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365c43430 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd870009ca0 tx=0x7fd870005d80 comp rx=0 tx=0).stop 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 -- 172.21.15.69:0/286366438 shutdown_connections 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/286366438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365c43430 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/286366438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c302b0 0x55a365c42ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/286366438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c2f990 0x55a365c31af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 -- 172.21.15.69:0/286366438 >> 172.21.15.69:0/286366438 conn(0x55a365c2bf20 msgr2=0x55a365c313e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 -- 172.21.15.69:0/286366438 shutdown_connections 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.265+0000 7fd87ffa8740 1 -- 172.21.15.69:0/286366438 wait complete. 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 1 -- start start 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 1 Processor -- start 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c2f990 0x55a365cb3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c302b0 0x55a365cb3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365cad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781ad660 con 0x55a365c3ab70 2024-08-02T17:55:18.277 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781aead0 con 0x55a365c2f990 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8781add30 con 0x55a365c302b0 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd87ffa8740 10 client.0 _ll_register_callbacks cb 0x7fd86809a880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd877fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365cad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd877fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365cad8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39824/0 (socket says 172.21.15.69:39824) 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd877fff640 1 -- 172.21.15.69:0/251689550 learned_addr learned my addr 172.21.15.69:0/251689550 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd8777fe640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c2f990 0x55a365cb3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.266+0000 7fd86effd640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c302b0 0x55a365cb3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd877fff640 1 -- 172.21.15.69:0/251689550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c302b0 msgr2=0x55a365cb3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd877fff640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c302b0 0x55a365cb3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd877fff640 1 -- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c2f990 msgr2=0x55a365cb3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd877fff640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c2f990 0x55a365cb3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.278 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd877fff640 1 -- 172.21.15.69:0/251689550 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a365c271b0 con 0x55a365c3ab70 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd8777fe640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c2f990 0x55a365cb3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd86effd640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c302b0 0x55a365cb3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd877fff640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365cad8d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd870005fb0 tx=0x7fd87002fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef8989489f17007 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd856ffd640 1 -- 172.21.15.69:0/251689550 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87005f070 con 0x55a365c3ab70 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd856ffd640 1 -- 172.21.15.69:0/251689550 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fd870004040 con 0x55a365c3ab70 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd87ffa8740 1 -- 172.21.15.69:0/251689550 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd87004e070 con 0x55a365c3ab70 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd856ffd640 1 -- 172.21.15.69:0/251689550 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd870056040 con 0x55a365c3ab70 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd87ffa8740 10 client.17697 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd87ffa8740 1 -- 172.21.15.69:0/251689550 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a365c2cbb0 con 0x55a365c3ab70 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.267+0000 7fd856ffd640 1 -- 172.21.15.69:0/251689550 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd87004e070 con 0x55a365c3ab70 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd856ffd640 1 -- 172.21.15.69:0/251689550 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd870030440 con 0x55a365c3ab70 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd87ffa8740 10 client.17697 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:18.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd87ffa8740 10 client.17697 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:55:18.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd87ffa8740 20 client.17697 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:18.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a365cba8b0 0x55a365cbcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:18.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd87ffa8740 4 client.17697 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "65e61135", "release": true, "await": true}] 2024-08-02T17:55:18.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd87ffa8740 1 -- 172.21.15.69:0/251689550 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "65e61135", "release": true, "await": true}) -- 0x55a365b9d6e0 con 0x55a365cba8b0 2024-08-02T17:55:18.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd8777fe640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a365cba8b0 0x55a365cbcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:18.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd8777fe640 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a365cba8b0 0x55a365cbcca0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55674d788cbca2c server_cookie=c2bb0d1481c69da9 in_seq=0 out_seq=0 2024-08-02T17:55:18.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.268+0000 7fd856ffd640 10 client.17697 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:18.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.317+0000 7fd856ffd640 1 -- 172.21.15.69:0/251689550 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a365b9d6e0 con 0x55a365cba8b0 2024-08-02T17:55:18.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.317+0000 7fd856ffd640 10 client.17697 handle_command_reply: tid=0 2024-08-02T17:55:18.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.317+0000 7fd87ffa8740 1 client.17697 shutdown 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.317+0000 7fd87ffa8740 1 -- 172.21.15.69:0/251689550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 msgr2=0x55a365cad8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.317+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365cad8d0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fd870005fb0 tx=0x7fd87002fd90 comp rx=0 tx=0).stop 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.317+0000 7fd87ffa8740 1 -- 172.21.15.69:0/251689550 shutdown_connections 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a365c3ab70 0x55a365cad8d0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a365cba8b0 0x55a365cbcca0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a365c302b0 0x55a365cb3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd87ffa8740 1 --2- 172.21.15.69:0/251689550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a365c2f990 0x55a365cb3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd87ffa8740 1 -- 172.21.15.69:0/251689550 >> 172.21.15.69:0/251689550 conn(0x55a365c2bf20 msgr2=0x55a365c2b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd856ffd640 0 client.17697 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd87ffa8740 1 -- 172.21.15.69:0/251689550 shutdown_connections 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd87ffa8740 1 -- 172.21.15.69:0/251689550 wait complete. 2024-08-02T17:55:18.320 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.318+0000 7fd87ffa8740 20 client.17697 trim_cache size 0 max 16384 2024-08-02T17:55:18.321 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:55:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:55:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:55:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 32, 2024-08-02T17:55:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:55:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "65e61135": { 2024-08-02T17:55:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 32, 2024-08-02T17:55:18.321 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 140.4, 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:55:18.322 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:18.323 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:55:18.323 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:55:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 -- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd8640a2cf0 msgr2=0x7fd8640a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd8640a2cf0 0x7fd8640a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd870005fe0 tx=0x7fd870043000 comp rx=0 tx=0).stop 2024-08-02T17:55:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 -- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878100b50 msgr2=0x7fd878198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878100b50 0x7fd878198240 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd86800b4a0 tx=0x7fd86800b960 comp rx=0 tx=0).stop 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 -- 172.21.15.69:0/3579426805 shutdown_connections 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd8640a2cf0 0x7fd8640a51b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd878102c20 0x7fd87819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd8781022a0 0x7fd878198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 --2- 172.21.15.69:0/3579426805 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd878100b50 0x7fd878198240 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 -- 172.21.15.69:0/3579426805 >> 172.21.15.69:0/3579426805 conn(0x7fd8780fc500 msgr2=0x7fd878069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 -- 172.21.15.69:0/3579426805 shutdown_connections 2024-08-02T17:55:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:18.319+0000 7fd87dbaa640 1 -- 172.21.15.69:0/3579426805 wait complete. 2024-08-02T17:55:18.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:18 smithi077 ceph-mon[37108]: pgmap v275: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 293 KiB/s wr, 123 op/s 2024-08-02T17:55:18.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:18 smithi069 ceph-mon[32746]: pgmap v275: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 293 KiB/s wr, 123 op/s 2024-08-02T17:55:19.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:18 smithi083 ceph-mon[37031]: pgmap v275: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 293 KiB/s wr, 123 op/s 2024-08-02T17:55:19.266 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '65e61135', total seconds elapsed: 23.9 2024-08-02T17:55:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 490.0 s, 231.2 tps, lat 1420.872 ms stddev 4192.407 2024-08-02T17:55:20.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:20 smithi077 ceph-mon[37108]: pgmap v276: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 28 KiB/s wr, 122 op/s 2024-08-02T17:55:20.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:20 smithi069 ceph-mon[32746]: pgmap v276: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 28 KiB/s wr, 122 op/s 2024-08-02T17:55:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:20 smithi083 ceph-mon[37031]: pgmap v276: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 28 KiB/s wr, 122 op/s 2024-08-02T17:55:22.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:22 smithi077 ceph-mon[37108]: pgmap v277: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 95 KiB/s wr, 131 op/s 2024-08-02T17:55:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:22 smithi069 ceph-mon[32746]: pgmap v277: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 95 KiB/s wr, 131 op/s 2024-08-02T17:55:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:22 smithi083 ceph-mon[37031]: pgmap v277: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 95 KiB/s wr, 131 op/s 2024-08-02T17:55:24.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:24 smithi077 ceph-mon[37108]: pgmap v278: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-08-02T17:55:24.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:24 smithi069 ceph-mon[32746]: pgmap v278: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-08-02T17:55:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 495.0 s, 577.8 tps, lat 168.981 ms stddev 197.849 2024-08-02T17:55:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:24 smithi083 ceph-mon[37031]: pgmap v278: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-08-02T17:55:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 -- 172.21.15.69:0/4291417350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 msgr2=0x7f05a0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 --2- 172.21.15.69:0/4291417350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 0x7f05a0100b30 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0594009930 tx=0x7f059402a420 comp rx=0 tx=0).stop 2024-08-02T17:55:26.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 -- 172.21.15.69:0/4291417350 shutdown_connections 2024-08-02T17:55:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 --2- 172.21.15.69:0/4291417350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 0x7f05a0100b30 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 --2- 172.21.15.69:0/4291417350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a0101070 0x7f05a010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 --2- 172.21.15.69:0/4291417350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a01066e0 0x7f05a0106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 -- 172.21.15.69:0/4291417350 >> 172.21.15.69:0/4291417350 conn(0x7f05a00776d0 msgr2=0x7f05a00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 -- 172.21.15.69:0/4291417350 shutdown_connections 2024-08-02T17:55:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.725+0000 7f05a6456640 1 -- 172.21.15.69:0/4291417350 wait complete. 2024-08-02T17:55:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f05a6456640 1 Processor -- start 2024-08-02T17:55:26.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f05a6456640 1 -- start start 2024-08-02T17:55:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f05a6456640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 0x7f05a0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f05a6456640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a0101070 0x7f05a0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f05a6456640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a01066e0 0x7f05a019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f05a6456640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05a0110b70 con 0x7f05a01006d0 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f05a6456640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01109f0 con 0x7f05a0101070 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f05a6456640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05a0110cf0 con 0x7f05a01066e0 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a0101070 0x7f05a0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f059ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 0x7f05a0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a0101070 0x7f05a0198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35024/0 (socket says 172.21.15.69:35024) 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f059ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 0x7f05a0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39908/0 (socket says 172.21.15.69:39908) 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.726+0000 7f059f7fe640 1 -- 172.21.15.69:0/232888450 learned_addr learned my addr 172.21.15.69:0/232888450 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.727+0000 7f05a49f3640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a01066e0 0x7f05a019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.727+0000 7f059ffff640 1 -- 172.21.15.69:0/232888450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a01066e0 msgr2=0x7f05a019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.727+0000 7f059ffff640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a01066e0 0x7f05a019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.727+0000 7f059ffff640 1 -- 172.21.15.69:0/232888450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a0101070 msgr2=0x7f05a0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.727+0000 7f059ffff640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a0101070 0x7f05a0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.727+0000 7f059ffff640 1 -- 172.21.15.69:0/232888450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a019cf60 con 0x7f05a01006d0 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.727+0000 7f059f7fe640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a0101070 0x7f05a0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.727+0000 7f059ffff640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 0x7f05a0198010 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f059000d6d0 tx=0x7f059000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.728+0000 7f059d7fa640 1 -- 172.21.15.69:0/232888450 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590017070 con 0x7f05a01006d0 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.728+0000 7f059d7fa640 1 -- 172.21.15.69:0/232888450 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0590009820 con 0x7f05a01006d0 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.728+0000 7f05a6456640 1 -- 172.21.15.69:0/232888450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a019d250 con 0x7f05a01006d0 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.728+0000 7f059d7fa640 1 -- 172.21.15.69:0/232888450 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590004d00 con 0x7f05a01006d0 2024-08-02T17:55:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.728+0000 7f05a6456640 1 -- 172.21.15.69:0/232888450 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a01a4ae0 con 0x7f05a01006d0 2024-08-02T17:55:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.730+0000 7f059d7fa640 1 -- 172.21.15.69:0/232888450 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f05900099d0 con 0x7f05a01006d0 2024-08-02T17:55:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.731+0000 7f059d7fa640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05880a2e40 0x7f05880a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.731+0000 7f059d7fa640 1 -- 172.21.15.69:0/232888450 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f05900cf200 con 0x7f05a01006d0 2024-08-02T17:55:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.731+0000 7f059f7fe640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05880a2e40 0x7f05880a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.732+0000 7f059f7fe640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05880a2e40 0x7f05880a5300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05940096c0 tx=0x7f0594000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.732+0000 7f05a91b4740 1 Processor -- start 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.732+0000 7f05a91b4740 1 -- start start 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.732+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee11b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.732+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee1c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.732+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee1ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f05a91b4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55914ed759b0 con 0x55914ee1c970 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f05a91b4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55914ed1d800 con 0x55914ee120d0 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f05a91b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55914ed71970 con 0x55914ee117b0 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f059ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee11b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f059ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee11b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:60424/0 (socket says 172.21.15.69:60424) 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f059ffff640 1 -- 172.21.15.69:0/803589758 learned_addr learned my addr 172.21.15.69:0/803589758 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f059f7fe640 1 --2- 172.21.15.69:0/803589758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee1c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f059ffff640 1 -- 172.21.15.69:0/803589758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 msgr2=0x55914ee1c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.733+0000 7f059ffff640 1 --2- 172.21.15.69:0/803589758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee1c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.734+0000 7f059ffff640 1 -- 172.21.15.69:0/803589758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 msgr2=0x55914ee1ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.734+0000 7f059ffff640 1 --2- 172.21.15.69:0/803589758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee1ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.734+0000 7f059ffff640 1 -- 172.21.15.69:0/803589758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914ea89210 con 0x55914ee117b0 2024-08-02T17:55:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.734+0000 7f059ffff640 1 --2- 172.21.15.69:0/803589758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee11b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0590097d90 tx=0x7f059009b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d42c049d5b195827 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.735+0000 7f05867fc640 1 -- 172.21.15.69:0/803589758 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590020070 con 0x55914ee117b0 2024-08-02T17:55:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.735+0000 7f05867fc640 1 -- 172.21.15.69:0/803589758 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0590009d40 con 0x55914ee117b0 2024-08-02T17:55:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.735+0000 7f05867fc640 1 -- 172.21.15.69:0/803589758 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590020070 con 0x55914ee117b0 2024-08-02T17:55:26.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 -- 172.21.15.69:0/803589758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 msgr2=0x55914ee11b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 --2- 172.21.15.69:0/803589758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee11b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0590097d90 tx=0x7f059009b3c0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 -- 172.21.15.69:0/803589758 shutdown_connections 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 --2- 172.21.15.69:0/803589758 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee1ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 --2- 172.21.15.69:0/803589758 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee1c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 --2- 172.21.15.69:0/803589758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee11b90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 -- 172.21.15.69:0/803589758 >> 172.21.15.69:0/803589758 conn(0x55914ee0dd20 msgr2=0x55914ee0e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 -- 172.21.15.69:0/803589758 shutdown_connections 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.736+0000 7f05a91b4740 1 -- 172.21.15.69:0/803589758 wait complete. 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.737+0000 7f05a91b4740 1 -- start start 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.737+0000 7f05a91b4740 1 Processor -- start 2024-08-02T17:55:26.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee19870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee120d0 0x55914ee19db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f059ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee19870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f059ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee19870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35046/0 (socket says 172.21.15.69:35046) 2024-08-02T17:55:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee1c970 0x55914ee1a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f05a91b4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55914ee1f4e0 con 0x55914ee120d0 2024-08-02T17:55:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f05a91b4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55914ed31500 con 0x55914ee117b0 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f05a91b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55914ee1f660 con 0x55914ee1c970 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee120d0 0x55914ee19db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f05a91b4740 10 client.0 _ll_register_callbacks cb 0x55914ea88a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee120d0 0x55914ee19db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39916/0 (socket says 172.21.15.69:39916) 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f059f7fe640 1 -- 172.21.15.69:0/1014872540 learned_addr learned my addr 172.21.15.69:0/1014872540 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.738+0000 7f05a49f3640 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee1c970 0x55914ee1a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f059ffff640 1 -- 172.21.15.69:0/1014872540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee1c970 msgr2=0x55914ee1a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f059ffff640 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee1c970 0x55914ee1a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f059ffff640 1 -- 172.21.15.69:0/1014872540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee120d0 msgr2=0x55914ee19db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f059ffff640 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee120d0 0x55914ee19db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f059ffff640 1 -- 172.21.15.69:0/1014872540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914eb55620 con 0x55914ee117b0 2024-08-02T17:55:26.745 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f05a49f3640 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee1c970 0x55914ee1a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f059ffff640 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee19870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0590009610 tx=0x7f0590009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c3c0a24e28e497f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f057f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f057f7fe640 1 -- 172.21.15.69:0/1014872540 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590020070 con 0x55914ee117b0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f05a91b4740 1 -- 172.21.15.69:0/1014872540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0590009d40 con 0x55914ee117b0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f057f7fe640 1 -- 172.21.15.69:0/1014872540 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f05900b9020 con 0x55914ee117b0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f057f7fe640 1 -- 172.21.15.69:0/1014872540 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05900ac050 con 0x55914ee117b0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f05a91b4740 10 client.27188 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.739+0000 7f05a91b4740 1 -- 172.21.15.69:0/1014872540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55914ee2b150 con 0x55914ee117b0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f057f7fe640 1 -- 172.21.15.69:0/1014872540 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0590009d40 con 0x55914ee117b0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f057f7fe640 1 -- 172.21.15.69:0/1014872540 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f059009a810 con 0x55914ee117b0 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f05a91b4740 10 client.27188 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:26.746 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f05a91b4740 10 client.27188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f05a91b4740 20 client.27188 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f05a91b4740 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55914ee2c4a0 0x55914ee2e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f05a91b4740 4 client.27188 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f05a91b4740 1 -- 172.21.15.69:0/1014872540 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914ed830e0 con 0x55914ee2c4a0 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f059f7fe640 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55914ee2c4a0 0x55914ee2e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f057f7fe640 10 client.27188 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.740+0000 7f059f7fe640 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55914ee2c4a0 0x55914ee2e890 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e3e58e1c581c86a server_cookie=7db974d03e643efa in_seq=0 out_seq=0 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f057f7fe640 1 -- 172.21.15.69:0/1014872540 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55914ed830e0 con 0x55914ee2c4a0 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f057f7fe640 10 client.27188 handle_command_reply: tid=0 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 client.27188 shutdown 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 -- 172.21.15.69:0/1014872540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 msgr2=0x55914ee19870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee19870 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0590009610 tx=0x7f0590009740 comp rx=0 tx=0).stop 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 -- 172.21.15.69:0/1014872540 shutdown_connections 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee120d0 0x55914ee19db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55914ee2c4a0 0x55914ee2e890 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee1c970 0x55914ee1a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 --2- 172.21.15.69:0/1014872540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee19870 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f05a91b4740 1 -- 172.21.15.69:0/1014872540 >> 172.21.15.69:0/1014872540 conn(0x55914ee0dd20 msgr2=0x55914ee13a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.741+0000 7f057f7fe640 0 client.27188 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.742+0000 7f05a91b4740 1 -- 172.21.15.69:0/1014872540 shutdown_connections 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.742+0000 7f05a91b4740 1 -- 172.21.15.69:0/1014872540 wait complete. 2024-08-02T17:55:26.748 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.742+0000 7f05a91b4740 20 client.27188 trim_cache size 0 max 16384 2024-08-02T17:55:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.761+0000 7f05a91b4740 1 Processor -- start 2024-08-02T17:55:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.761+0000 7f05a91b4740 1 -- start start 2024-08-02T17:55:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee33e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee120d0 0x55914ee343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee13210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a91b4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01b17a0 con 0x55914ee1c970 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a91b4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01af070 con 0x55914ee117b0 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a91b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01acec0 con 0x55914ee120d0 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a49f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee13210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee120d0 0x55914ee343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a49f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee13210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39930/0 (socket says 172.21.15.69:39930) 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a49f3640 1 -- 172.21.15.69:0/514347924 learned_addr learned my addr 172.21.15.69:0/514347924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:26.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f059ffff640 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee33e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f059f7fe640 1 -- 172.21.15.69:0/514347924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 msgr2=0x55914ee33e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f059f7fe640 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee33e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f059f7fe640 1 -- 172.21.15.69:0/514347924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 msgr2=0x55914ee13210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f059f7fe640 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee13210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f059f7fe640 1 -- 172.21.15.69:0/514347924 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914ee11f00 con 0x55914ee120d0 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.762+0000 7f05a49f3640 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee13210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.763+0000 7f059f7fe640 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee120d0 0x55914ee343d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55914ee2ee30 tx=0x7f0594040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dd56f5957ae6338 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.763+0000 7f0585ffb640 1 -- 172.21.15.69:0/514347924 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594035040 con 0x55914ee120d0 2024-08-02T17:55:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.763+0000 7f0585ffb640 1 -- 172.21.15.69:0/514347924 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0594003d90 con 0x55914ee120d0 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.763+0000 7f0585ffb640 1 -- 172.21.15.69:0/514347924 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594054b90 con 0x55914ee120d0 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 -- 172.21.15.69:0/514347924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee120d0 msgr2=0x55914ee343d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee120d0 0x55914ee343d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55914ee2ee30 tx=0x7f0594040000 comp rx=0 tx=0).stop 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 -- 172.21.15.69:0/514347924 shutdown_connections 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee13210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee120d0 0x55914ee343d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 --2- 172.21.15.69:0/514347924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee117b0 0x55914ee33e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 -- 172.21.15.69:0/514347924 >> 172.21.15.69:0/514347924 conn(0x55914ee0dd20 msgr2=0x55914ee1b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 -- 172.21.15.69:0/514347924 shutdown_connections 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.764+0000 7f05a91b4740 1 -- 172.21.15.69:0/514347924 wait complete. 2024-08-02T17:55:26.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.765+0000 7f05a91b4740 1 -- start start 2024-08-02T17:55:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.765+0000 7f05a91b4740 1 Processor -- start 2024-08-02T17:55:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.765+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee921c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.765+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee92700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.765+0000 7f05a91b4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee248f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f05a91b4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01ad530 con 0x55914ee1c970 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee921c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f05a91b4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01ae9a0 con 0x55914ee120d0 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f05a91b4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05a01adc00 con 0x55914ee117b0 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee921c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:60454/0 (socket says 172.21.15.69:60454) 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059ffff640 1 -- 172.21.15.69:0/2793306139 learned_addr learned my addr 172.21.15.69:0/2793306139 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059f7fe640 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee92700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f05a49f3640 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f05a91b4740 10 client.0 _ll_register_callbacks cb 0x7f05900b9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:26.771 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059f7fe640 1 -- 172.21.15.69:0/2793306139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 msgr2=0x55914ee921c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059f7fe640 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee921c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059f7fe640 1 -- 172.21.15.69:0/2793306139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 msgr2=0x55914ee248f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059f7fe640 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee248f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059f7fe640 1 -- 172.21.15.69:0/2793306139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914ee25120 con 0x55914ee120d0 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059ffff640 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f059f7fe640 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee92700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55914ee12880 tx=0x7f0594005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=243a13b430e1007 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f057f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f057f7fe640 1 -- 172.21.15.69:0/2793306139 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594059070 con 0x55914ee120d0 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.766+0000 7f057f7fe640 1 -- 172.21.15.69:0/2793306139 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f0594003ac0 con 0x55914ee120d0 2024-08-02T17:55:26.772 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 1 -- 172.21.15.69:0/2793306139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0594003d90 con 0x55914ee120d0 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f057f7fe640 1 -- 172.21.15.69:0/2793306139 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0594035040 con 0x55914ee120d0 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 10 client.27194 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 1 -- 172.21.15.69:0/2793306139 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55914ee16e50 con 0x55914ee120d0 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f057f7fe640 1 -- 172.21.15.69:0/2793306139 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0594003d90 con 0x55914ee120d0 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f057f7fe640 1 -- 172.21.15.69:0/2793306139 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05940543b0 con 0x55914ee120d0 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 10 client.27194 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 10 client.27194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 20 client.27194 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55914ee9b560 0x55914ee9d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 4 client.27194 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f05a91b4740 1 -- 172.21.15.69:0/2793306139 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55914ed7d430 con 0x55914ee9b560 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.767+0000 7f059ffff640 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55914ee9b560 0x55914ee9d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:26.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.768+0000 7f057f7fe640 10 client.27194 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:26.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.768+0000 7f059ffff640 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55914ee9b560 0x55914ee9d950 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4593a48029dc5af7 server_cookie=2e864b642b66a9ed in_seq=0 out_seq=0 2024-08-02T17:55:26.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:26 smithi077 ceph-mon[37108]: pgmap v279: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-08-02T17:55:26.861 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:26 smithi069 ceph-mon[32746]: pgmap v279: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-08-02T17:55:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.878+0000 7f057f7fe640 1 -- 172.21.15.69:0/2793306139 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55914ed7d430 con 0x55914ee9b560 2024-08-02T17:55:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.878+0000 7f057f7fe640 10 client.27194 handle_command_reply: tid=0 2024-08-02T17:55:26.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.878+0000 7f05a91b4740 1 client.27194 shutdown 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.878+0000 7f05a91b4740 1 -- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 msgr2=0x55914ee92700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.878+0000 7f05a91b4740 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee92700 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55914ee12880 tx=0x7f0594005d50 comp rx=0 tx=0).stop 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 1 -- 172.21.15.69:0/2793306139 shutdown_connections 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55914ee1c970 0x55914ee248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55914ee9b560 0x55914ee9d950 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f057f7fe640 0 client.27194 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55914ee120d0 0x55914ee92700 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 1 --2- 172.21.15.69:0/2793306139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55914ee117b0 0x55914ee921c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 1 -- 172.21.15.69:0/2793306139 >> 172.21.15.69:0/2793306139 conn(0x55914ee0dd20 msgr2=0x55914ee19810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 1 -- 172.21.15.69:0/2793306139 shutdown_connections 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 1 -- 172.21.15.69:0/2793306139 wait complete. 2024-08-02T17:55:26.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.879+0000 7f05a91b4740 20 client.27194 trim_cache size 0 max 16384 2024-08-02T17:55:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:55:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:55:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:55:26.882 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:55:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 -- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05880a2e40 msgr2=0x7f05880a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05880a2e40 0x7f05880a5300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05940096c0 tx=0x7f0594000bc0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 -- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 msgr2=0x7f05a0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 0x7f05a0198010 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f059000d6d0 tx=0x7f059000db90 comp rx=0 tx=0).stop 2024-08-02T17:55:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 -- 172.21.15.69:0/232888450 shutdown_connections 2024-08-02T17:55:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05880a2e40 0x7f05880a5300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05a01066e0 0x7f05a019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05a0101070 0x7f05a0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 --2- 172.21.15.69:0/232888450 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05a01006d0 0x7f05a0198010 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 -- 172.21.15.69:0/232888450 >> 172.21.15.69:0/232888450 conn(0x7f05a00776d0 msgr2=0x7f05a0069810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 -- 172.21.15.69:0/232888450 shutdown_connections 2024-08-02T17:55:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:26.880+0000 7f05a6456640 1 -- 172.21.15.69:0/232888450 wait complete. 2024-08-02T17:55:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:26 smithi083 ceph-mon[37031]: pgmap v279: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.7 MiB/s wr, 281 op/s 2024-08-02T17:55:28.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:28 smithi077 ceph-mon[37108]: pgmap v280: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 3.0 MiB/s wr, 418 op/s 2024-08-02T17:55:28.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:28 smithi069 ceph-mon[32746]: pgmap v280: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 3.0 MiB/s wr, 418 op/s 2024-08-02T17:55:29.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:28 smithi083 ceph-mon[37031]: pgmap v280: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 3.0 MiB/s wr, 418 op/s 2024-08-02T17:55:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 500.0 s, 659.8 tps, lat 153.846 ms stddev 191.293 2024-08-02T17:55:30.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:30 smithi077 ceph-mon[37108]: pgmap v281: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-08-02T17:55:30.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:30 smithi069 ceph-mon[32746]: pgmap v281: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-08-02T17:55:31.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:30 smithi083 ceph-mon[37031]: pgmap v281: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 3.6 MiB/s wr, 413 op/s 2024-08-02T17:55:32.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:32 smithi077 ceph-mon[37108]: pgmap v282: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-08-02T17:55:32.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:32 smithi069 ceph-mon[32746]: pgmap v282: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-08-02T17:55:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:32 smithi083 ceph-mon[37031]: pgmap v282: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 3.8 MiB/s wr, 418 op/s 2024-08-02T17:55:34.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:34 smithi069 ceph-mon[32746]: pgmap v283: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.0 MiB/s wr, 627 op/s 2024-08-02T17:55:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 505.0 s, 944.4 tps, lat 102.889 ms stddev 164.048 2024-08-02T17:55:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:34 smithi083 ceph-mon[37031]: pgmap v283: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.0 MiB/s wr, 627 op/s 2024-08-02T17:55:35.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:34 smithi077 ceph-mon[37108]: pgmap v283: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.0 MiB/s wr, 627 op/s 2024-08-02T17:55:36.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:36 smithi069 ceph-mon[32746]: pgmap v284: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-08-02T17:55:37.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:36 smithi083 ceph-mon[37031]: pgmap v284: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-08-02T17:55:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:36 smithi077 ceph-mon[37108]: pgmap v284: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 4.6 MiB/s wr, 483 op/s 2024-08-02T17:55:39.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:38 smithi083 ceph-mon[37031]: pgmap v285: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 826 op/s 2024-08-02T17:55:39.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:38 smithi077 ceph-mon[37108]: pgmap v285: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 826 op/s 2024-08-02T17:55:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:38 smithi069 ceph-mon[32746]: pgmap v285: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 7.7 MiB/s wr, 826 op/s 2024-08-02T17:55:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 510.0 s, 962.0 tps, lat 108.425 ms stddev 159.169 2024-08-02T17:55:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.032+0000 7fec7de95640 1 -- 172.21.15.69:0/4269873989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec7811ae70 msgr2=0x7fec7811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.032+0000 7fec7de95640 1 --2- 172.21.15.69:0/4269873989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec7811ae70 0x7fec7811d260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fec70009cd0 tx=0x7fec7002a910 comp rx=0 tx=0).stop 2024-08-02T17:55:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.033+0000 7fec7de95640 1 -- 172.21.15.69:0/4269873989 shutdown_connections 2024-08-02T17:55:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.033+0000 7fec7de95640 1 --2- 172.21.15.69:0/4269873989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec78109300 0x7fec78109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.033+0000 7fec7de95640 1 --2- 172.21.15.69:0/4269873989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec7811ae70 0x7fec7811d260 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.033+0000 7fec7de95640 1 --2- 172.21.15.69:0/4269873989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec78108f20 0x7fec7810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.033+0000 7fec7de95640 1 -- 172.21.15.69:0/4269873989 >> 172.21.15.69:0/4269873989 conn(0x7fec7806cb20 msgr2=0x7fec7806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:40.037 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.036+0000 7fec7de95640 1 -- 172.21.15.69:0/4269873989 shutdown_connections 2024-08-02T17:55:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.037+0000 7fec7de95640 1 -- 172.21.15.69:0/4269873989 wait complete. 2024-08-02T17:55:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.038+0000 7fec7de95640 1 Processor -- start 2024-08-02T17:55:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.039+0000 7fec7de95640 1 -- start start 2024-08-02T17:55:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec7de95640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec78108f20 0x7fec7811a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec7de95640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec78109300 0x7fec7811a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec7de95640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec7811ae70 0x7fec78114500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec7de95640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec78110d70 con 0x7fec78108f20 2024-08-02T17:55:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec7de95640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec78110bf0 con 0x7fec78109300 2024-08-02T17:55:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec7de95640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec78110ef0 con 0x7fec7811ae70 2024-08-02T17:55:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec777fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec78108f20 0x7fec7811a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec777fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec78108f20 0x7fec7811a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58112/0 (socket says 172.21.15.69:58112) 2024-08-02T17:55:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.040+0000 7fec777fe640 1 -- 172.21.15.69:0/264726540 learned_addr learned my addr 172.21.15.69:0/264726540 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.041+0000 7fec76ffd640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec78109300 0x7fec7811a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.043+0000 7fec76ffd640 1 -- 172.21.15.69:0/264726540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec7811ae70 msgr2=0x7fec78114500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.043+0000 7fec76ffd640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec7811ae70 0x7fec78114500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.043+0000 7fec76ffd640 1 -- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec78108f20 msgr2=0x7fec7811a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.043+0000 7fec76ffd640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec78108f20 0x7fec7811a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.043+0000 7fec76ffd640 1 -- 172.21.15.69:0/264726540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec78114d90 con 0x7fec78109300 2024-08-02T17:55:40.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.044+0000 7fec76ffd640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec78109300 0x7fec7811a840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec6800e9f0 tx=0x7fec6800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.046+0000 7fec74ff9640 1 -- 172.21.15.69:0/264726540 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6800ccc0 con 0x7fec78109300 2024-08-02T17:55:40.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.046+0000 7fec74ff9640 1 -- 172.21.15.69:0/264726540 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec68010910 con 0x7fec78109300 2024-08-02T17:55:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.046+0000 7fec74ff9640 1 -- 172.21.15.69:0/264726540 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680179d0 con 0x7fec78109300 2024-08-02T17:55:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.046+0000 7fec7de95640 1 -- 172.21.15.69:0/264726540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec78115080 con 0x7fec78109300 2024-08-02T17:55:40.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.046+0000 7fec7de95640 1 -- 172.21.15.69:0/264726540 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec781bc1e0 con 0x7fec78109300 2024-08-02T17:55:40.052 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.050+0000 7fec74ff9640 1 -- 172.21.15.69:0/264726540 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fec68017b70 con 0x7fec78109300 2024-08-02T17:55:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.052+0000 7fec74ff9640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec600a2f00 0x7fec600a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.052+0000 7fec74ff9640 1 -- 172.21.15.69:0/264726540 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fec680cfcc0 con 0x7fec78109300 2024-08-02T17:55:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.053+0000 7fec777fe640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec600a2f00 0x7fec600a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.055+0000 7fec777fe640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec600a2f00 0x7fec600a53c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00ac00 tx=0x7fec6c00a5e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.056+0000 7fec80bec740 1 Processor -- start 2024-08-02T17:55:40.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.056+0000 7fec80bec740 1 -- start start 2024-08-02T17:55:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec80bec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec80bec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 0x55a3a2505430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec80bec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a2505970 0x55a3a2507d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec80bec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3a245e9b0 con 0x55a3a24fb0d0 2024-08-02T17:55:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec80bec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3a2406800 con 0x55a3a24fa7b0 2024-08-02T17:55:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a2505970 0x55a3a2507d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec77fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a2505970 0x55a3a2507d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38008/0 (socket says 172.21.15.69:38008) 2024-08-02T17:55:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec77fff640 1 -- 172.21.15.69:0/3097351052 learned_addr learned my addr 172.21.15.69:0/3097351052 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.057+0000 7fec80bec740 1 -- 172.21.15.69:0/3097351052 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a3a245a970 con 0x55a3a2505970 2024-08-02T17:55:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.060+0000 7fec777fe640 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.060+0000 7fec76ffd640 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 0x55a3a2505430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.061+0000 7fec777fe640 1 -- 172.21.15.69:0/3097351052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a2505970 msgr2=0x55a3a2507d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.061+0000 7fec777fe640 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a2505970 0x55a3a2507d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.061+0000 7fec777fe640 1 -- 172.21.15.69:0/3097351052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 msgr2=0x55a3a2505430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.061+0000 7fec777fe640 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 0x55a3a2505430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.061+0000 7fec777fe640 1 -- 172.21.15.69:0/3097351052 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3a2172210 con 0x55a3a24fa7b0 2024-08-02T17:55:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.061+0000 7fec777fe640 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fab90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00ee20 tx=0x7fec6c011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a5cd072fd8173b6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.061+0000 7fec5dffb640 1 -- 172.21.15.69:0/3097351052 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c021070 con 0x55a3a24fa7b0 2024-08-02T17:55:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.062+0000 7fec5dffb640 1 -- 172.21.15.69:0/3097351052 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6c00ee50 con 0x55a3a24fa7b0 2024-08-02T17:55:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.062+0000 7fec5dffb640 1 -- 172.21.15.69:0/3097351052 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00fbb0 con 0x55a3a24fa7b0 2024-08-02T17:55:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.063+0000 7fec80bec740 1 -- 172.21.15.69:0/3097351052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 msgr2=0x55a3a24fab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.063+0000 7fec80bec740 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fab90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00ee20 tx=0x7fec6c011000 comp rx=0 tx=0).stop 2024-08-02T17:55:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.066+0000 7fec80bec740 1 -- 172.21.15.69:0/3097351052 shutdown_connections 2024-08-02T17:55:40.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.066+0000 7fec80bec740 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 0x55a3a2505430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.066+0000 7fec80bec740 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a2505970 0x55a3a2507d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.066+0000 7fec80bec740 1 --2- 172.21.15.69:0/3097351052 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fab90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.066+0000 7fec80bec740 1 -- 172.21.15.69:0/3097351052 >> 172.21.15.69:0/3097351052 conn(0x55a3a24f6d20 msgr2=0x55a3a24f7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:40.068 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.067+0000 7fec80bec740 1 -- 172.21.15.69:0/3097351052 shutdown_connections 2024-08-02T17:55:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.067+0000 7fec80bec740 1 -- 172.21.15.69:0/3097351052 wait complete. 2024-08-02T17:55:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.068+0000 7fec80bec740 1 -- start start 2024-08-02T17:55:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.068+0000 7fec80bec740 1 Processor -- start 2024-08-02T17:55:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec80bec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fa7b0 0x55a3a2502820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.072 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec80bec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a2502d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec80bec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 0x55a3a25032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec80bec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3a2508540 con 0x55a3a24fa7b0 2024-08-02T17:55:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec80bec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3a241a500 con 0x55a3a2505970 2024-08-02T17:55:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec80bec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a3a25086c0 con 0x55a3a24fb0d0 2024-08-02T17:55:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec80bec740 10 client.0 _ll_register_callbacks cb 0x55a3a2171a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec777fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fa7b0 0x55a3a2502820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec77fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 0x55a3a25032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec777fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fa7b0 0x55a3a2502820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58128/0 (socket says 172.21.15.69:58128) 2024-08-02T17:55:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.071+0000 7fec777fe640 1 -- 172.21.15.69:0/1090706864 learned_addr learned my addr 172.21.15.69:0/1090706864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.072+0000 7fec777fe640 1 -- 172.21.15.69:0/1090706864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 msgr2=0x55a3a2502d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:55:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.072+0000 7fec777fe640 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a2502d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.072+0000 7fec777fe640 1 -- 172.21.15.69:0/1090706864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 msgr2=0x55a3a25032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.072+0000 7fec777fe640 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 0x55a3a25032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.072+0000 7fec777fe640 1 -- 172.21.15.69:0/1090706864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3a223e620 con 0x55a3a24fa7b0 2024-08-02T17:55:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.072+0000 7fec777fe640 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fa7b0 0x55a3a2502820 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fec6c01d890 tx=0x7fec6c01d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191a7c054f1f057e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.072+0000 7fec56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.073+0000 7fec56ffd640 1 -- 172.21.15.69:0/1090706864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c021070 con 0x55a3a24fa7b0 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.073+0000 7fec80bec740 1 -- 172.21.15.69:0/1090706864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec6c00ee50 con 0x55a3a24fa7b0 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.073+0000 7fec56ffd640 1 -- 172.21.15.69:0/1090706864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fec6c00fe30 con 0x55a3a24fa7b0 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.073+0000 7fec56ffd640 1 -- 172.21.15.69:0/1090706864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec6c01dc70 con 0x55a3a24fa7b0 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.073+0000 7fec80bec740 10 client.17724 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.073+0000 7fec80bec740 1 -- 172.21.15.69:0/1090706864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a3a2514250 con 0x55a3a24fa7b0 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec56ffd640 1 -- 172.21.15.69:0/1090706864 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6c00ee50 con 0x55a3a24fa7b0 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec56ffd640 1 -- 172.21.15.69:0/1090706864 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec6c0046c0 con 0x55a3a24fa7b0 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec80bec740 10 client.17724 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec80bec740 10 client.17724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec80bec740 20 client.17724 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec80bec740 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3a2515ad0 0x55a3a2517f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec80bec740 4 client.17724 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec80bec740 1 -- 172.21.15.69:0/1090706864 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3a246c0e0 con 0x55a3a2515ad0 2024-08-02T17:55:40.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.074+0000 7fec76ffd640 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3a2515ad0 0x55a3a2517f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.075+0000 7fec56ffd640 10 client.17724 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.075+0000 7fec76ffd640 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3a2515ad0 0x55a3a2517f70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3df8fec0548b7a0 server_cookie=52d83ff99b408632 in_seq=0 out_seq=0 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.075+0000 7fec56ffd640 1 -- 172.21.15.69:0/1090706864 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a3a246c0e0 con 0x55a3a2515ad0 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.075+0000 7fec56ffd640 10 client.17724 handle_command_reply: tid=0 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.075+0000 7fec80bec740 1 client.17724 shutdown 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 -- 172.21.15.69:0/1090706864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fa7b0 msgr2=0x55a3a2502820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fa7b0 0x55a3a2502820 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fec6c01d890 tx=0x7fec6c01d8c0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 -- 172.21.15.69:0/1090706864 shutdown_connections 2024-08-02T17:55:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3a2515ad0 0x55a3a2517f70 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 0x55a3a25032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a2502d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 --2- 172.21.15.69:0/1090706864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fa7b0 0x55a3a2502820 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 -- 172.21.15.69:0/1090706864 >> 172.21.15.69:0/1090706864 conn(0x55a3a24f6d20 msgr2=0x55a3a24fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec56ffd640 0 client.17724 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 -- 172.21.15.69:0/1090706864 shutdown_connections 2024-08-02T17:55:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 1 -- 172.21.15.69:0/1090706864 wait complete. 2024-08-02T17:55:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.076+0000 7fec80bec740 20 client.17724 trim_cache size 0 max 16384 2024-08-02T17:55:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.099+0000 7fec80bec740 1 Processor -- start 2024-08-02T17:55:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.099+0000 7fec80bec740 1 -- start start 2024-08-02T17:55:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.099+0000 7fec80bec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec80bec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 0x55a3a251d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec80bec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 0x55a3a251dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec80bec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec64012940 con 0x55a3a24fb0d0 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec80bec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec64010210 con 0x55a3a2505970 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec80bec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec6400e060 con 0x55a3a24fa7b0 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38026/0 (socket says 172.21.15.69:38026) 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 -- 172.21.15.69:0/2353501171 learned_addr learned my addr 172.21.15.69:0/2353501171 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec76ffd640 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 0x55a3a251d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec77fff640 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 0x55a3a251dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 -- 172.21.15.69:0/2353501171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 msgr2=0x55a3a251dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 0x55a3a251dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 -- 172.21.15.69:0/2353501171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 msgr2=0x55a3a251d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 0x55a3a251d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 -- 172.21.15.69:0/2353501171 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3a24f6b50 con 0x55a3a24fa7b0 2024-08-02T17:55:40.103 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.100+0000 7fec777fe640 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fc970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00f800 tx=0x7fec6c011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ff123f3e11c0c03 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec5d7fa640 1 -- 172.21.15.69:0/2353501171 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c014040 con 0x55a3a24fa7b0 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec5d7fa640 1 -- 172.21.15.69:0/2353501171 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6c00e860 con 0x55a3a24fa7b0 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec5d7fa640 1 -- 172.21.15.69:0/2353501171 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c021040 con 0x55a3a24fa7b0 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 -- 172.21.15.69:0/2353501171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fa7b0 msgr2=0x55a3a24fc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fc970 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00f800 tx=0x7fec6c011f90 comp rx=0 tx=0).stop 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 -- 172.21.15.69:0/2353501171 shutdown_connections 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a24fb0d0 0x55a3a251d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a2505970 0x55a3a251dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.105 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 --2- 172.21.15.69:0/2353501171 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fa7b0 0x55a3a24fc970 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 -- 172.21.15.69:0/2353501171 >> 172.21.15.69:0/2353501171 conn(0x55a3a24f6d20 msgr2=0x55a3a24fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 -- 172.21.15.69:0/2353501171 shutdown_connections 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.101+0000 7fec80bec740 1 -- 172.21.15.69:0/2353501171 wait complete. 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.102+0000 7fec80bec740 1 -- start start 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.102+0000 7fec80bec740 1 Processor -- start 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec80bec740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 0x55a3a250da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec80bec740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a250df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec80bec740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a2505970 0x55a3a2508fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec80bec740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec6400e6d0 con 0x55a3a2505970 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec80bec740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec6400fb40 con 0x55a3a24fa7b0 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec80bec740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec6400eda0 con 0x55a3a24fb0d0 2024-08-02T17:55:40.106 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec80bec740 10 client.0 _ll_register_callbacks cb 0x7fec68097910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec76ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a250df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec76ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a250df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:38038/0 (socket says 172.21.15.69:38038) 2024-08-02T17:55:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec76ffd640 1 -- 172.21.15.69:0/45888471 learned_addr learned my addr 172.21.15.69:0/45888471 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec76ffd640 1 -- 172.21.15.69:0/45888471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 msgr2=0x55a3a250da40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:55:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec76ffd640 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 0x55a3a250da40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec76ffd640 1 -- 172.21.15.69:0/45888471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a2505970 msgr2=0x55a3a2508fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:55:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec76ffd640 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a2505970 0x55a3a2508fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.107 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.103+0000 7fec76ffd640 1 -- 172.21.15.69:0/45888471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3a24f7dd0 con 0x55a3a24fb0d0 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.104+0000 7fec76ffd640 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a250df80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec6809ba90 tx=0x7fec68017f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc142808ae8be9b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.104+0000 7fec56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.104+0000 7fec56ffd640 1 -- 172.21.15.69:0/45888471 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680a9060 con 0x55a3a24fb0d0 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.104+0000 7fec56ffd640 1 -- 172.21.15.69:0/45888471 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fec680cd240 con 0x55a3a24fb0d0 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.104+0000 7fec80bec740 1 -- 172.21.15.69:0/45888471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec6c00e860 con 0x55a3a24fb0d0 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.104+0000 7fec56ffd640 1 -- 172.21.15.69:0/45888471 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec680b0050 con 0x55a3a24fb0d0 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.104+0000 7fec80bec740 10 client.27031 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.105+0000 7fec80bec740 1 -- 172.21.15.69:0/45888471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a3a2580100 con 0x55a3a24fb0d0 2024-08-02T17:55:40.109 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.105+0000 7fec56ffd640 1 -- 172.21.15.69:0/45888471 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec6c00e860 con 0x55a3a24fb0d0 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec56ffd640 1 -- 172.21.15.69:0/45888471 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec6800bb50 con 0x55a3a24fb0d0 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec80bec740 10 client.27031 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec80bec740 10 client.27031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec80bec740 20 client.27031 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec80bec740 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3a2580860 0x55a3a2582c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec80bec740 4 client.27031 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec80bec740 1 -- 172.21.15.69:0/45888471 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a3a2466430 con 0x55a3a2580860 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec777fe640 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3a2580860 0x55a3a2582c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec777fe640 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3a2580860 0x55a3a2582c50 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd730c6ca2a02e07 server_cookie=60d3d5bfe39e736b in_seq=0 out_seq=0 2024-08-02T17:55:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.106+0000 7fec56ffd640 10 client.27031 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.168+0000 7fec56ffd640 1 -- 172.21.15.69:0/45888471 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3a2466430 con 0x55a3a2580860 2024-08-02T17:55:40.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.168+0000 7fec56ffd640 10 client.27031 handle_command_reply: tid=0 2024-08-02T17:55:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.170+0000 7fec80bec740 1 client.27031 shutdown 2024-08-02T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 -- 172.21.15.69:0/45888471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 msgr2=0x55a3a250df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a250df80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec6809ba90 tx=0x7fec68017f20 comp rx=0 tx=0).stop 2024-08-02T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 -- 172.21.15.69:0/45888471 shutdown_connections 2024-08-02T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3a2505970 0x55a3a2508fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3a2580860 0x55a3a2582c50 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3a24fb0d0 0x55a3a250df80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 --2- 172.21.15.69:0/45888471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3a24fa7b0 0x55a3a250da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 -- 172.21.15.69:0/45888471 >> 172.21.15.69:0/45888471 conn(0x55a3a24f6d20 msgr2=0x55a3a24f5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec56ffd640 0 client.27031 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 -- 172.21.15.69:0/45888471 shutdown_connections 2024-08-02T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.174+0000 7fec80bec740 1 -- 172.21.15.69:0/45888471 wait complete. 2024-08-02T17:55:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.175+0000 7fec80bec740 20 client.27031 trim_cache size 0 max 16384 2024-08-02T17:55:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:55:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:55:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:55:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:55:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 -- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec600a2f00 msgr2=0x7fec600a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec600a2f00 0x7fec600a53c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00ac00 tx=0x7fec6c00a5e0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 -- 172.21.15.69:0/264726540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec78109300 msgr2=0x7fec7811a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec78109300 0x7fec7811a840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fec6800e9f0 tx=0x7fec6800eeb0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 -- 172.21.15.69:0/264726540 shutdown_connections 2024-08-02T17:55:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec600a2f00 0x7fec600a53c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec7811ae70 0x7fec78114500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec78109300 0x7fec7811a840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 --2- 172.21.15.69:0/264726540 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec78108f20 0x7fec7811a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:40.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.181+0000 7fec7de95640 1 -- 172.21.15.69:0/264726540 >> 172.21.15.69:0/264726540 conn(0x7fec7806cb20 msgr2=0x7fec78071930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.185+0000 7fec7de95640 1 -- 172.21.15.69:0/264726540 shutdown_connections 2024-08-02T17:55:40.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:40.185+0000 7fec7de95640 1 -- 172.21.15.69:0/264726540 wait complete. 2024-08-02T17:55:40.899 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:40 smithi077 ceph-mon[37108]: pgmap v286: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 716 op/s 2024-08-02T17:55:41.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:40 smithi083 ceph-mon[37031]: pgmap v286: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 716 op/s 2024-08-02T17:55:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:40 smithi069 ceph-mon[32746]: pgmap v286: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.8 MiB/s wr, 716 op/s 2024-08-02T17:55:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:42 smithi083 ceph-mon[37031]: pgmap v287: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 689 op/s 2024-08-02T17:55:43.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:42 smithi077 ceph-mon[37108]: pgmap v287: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 689 op/s 2024-08-02T17:55:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:42 smithi069 ceph-mon[32746]: pgmap v287: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.3 MiB/s wr, 689 op/s 2024-08-02T17:55:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 515.0 s, 1067.6 tps, lat 93.352 ms stddev 117.104 2024-08-02T17:55:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:44 smithi083 ceph-mon[37031]: pgmap v288: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 943 op/s 2024-08-02T17:55:45.027 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:55:45.080 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:44 smithi069 ceph-mon[32746]: pgmap v288: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 943 op/s 2024-08-02T17:55:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:44 smithi077 ceph-mon[37108]: pgmap v288: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.9 MiB/s wr, 943 op/s 2024-08-02T17:55:45.800 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.566+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/3301754176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e48109b30 msgr2=0x7f3e4810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.566+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/3301754176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e48109b30 0x7f3e4810df70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b9f0 tx=0x7f3e4402c0c0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.566+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/3301754176 shutdown_connections 2024-08-02T17:55:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.566+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/3301754176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e4810e6a0 0x7f3e48115040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.566+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/3301754176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e48109b30 0x7f3e4810df70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.566+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/3301754176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48109180 0x7f3e48109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.566+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/3301754176 >> 172.21.15.69:0/3301754176 conn(0x7f3e48072eb0 msgr2=0x7f3e480732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:46.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.569+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/3301754176 shutdown_connections 2024-08-02T17:55:46.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.571+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/3301754176 wait complete. 2024-08-02T17:55:46.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.572+0000 7f3e4e7b1640 1 Processor -- start 2024-08-02T17:55:46.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.575+0000 7f3e4e7b1640 1 -- start start 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.575+0000 7f3e4e7b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e48109180 0x7f3e48113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.576+0000 7f3e4e7b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48109b30 0x7f3e48114510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.576+0000 7f3e4e7b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4810e6a0 0x7f3e481ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.576+0000 7f3e4e7b1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48117880 con 0x7f3e48109180 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.576+0000 7f3e4cfae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48109b30 0x7f3e48114510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.576+0000 7f3e4cfae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48109b30 0x7f3e48114510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39384/0 (socket says 172.21.15.69:39384) 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.576+0000 7f3e4cfae640 1 -- 172.21.15.69:0/1355484416 learned_addr learned my addr 172.21.15.69:0/1355484416 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.576+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48117700 con 0x7f3e4810e6a0 2024-08-02T17:55:46.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.576+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e48117a00 con 0x7f3e48109b30 2024-08-02T17:55:46.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.577+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e48109180 0x7f3e48113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.577+0000 7f3e4cfae640 1 -- 172.21.15.69:0/1355484416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4810e6a0 msgr2=0x7f3e481ab7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:55:46.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.577+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4810e6a0 0x7f3e481ab7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.577+0000 7f3e4cfae640 1 -- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e48109180 msgr2=0x7f3e48113fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:46.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.577+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e48109180 0x7f3e48113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.577+0000 7f3e4cfae640 1 -- 172.21.15.69:0/1355484416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e481abed0 con 0x7f3e48109b30 2024-08-02T17:55:46.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.577+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48109b30 0x7f3e48114510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b670 tx=0x7f3e44009b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:46.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.578+0000 7f3e3e7fc640 1 -- 172.21.15.69:0/1355484416 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e44039070 con 0x7f3e48109b30 2024-08-02T17:55:46.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.578+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e481ac160 con 0x7f3e48109b30 2024-08-02T17:55:46.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.578+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e481ac5d0 con 0x7f3e48109b30 2024-08-02T17:55:46.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.578+0000 7f3e3e7fc640 1 -- 172.21.15.69:0/1355484416 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e44005530 con 0x7f3e48109b30 2024-08-02T17:55:46.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.578+0000 7f3e3e7fc640 1 -- 172.21.15.69:0/1355484416 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e440035f0 con 0x7f3e48109b30 2024-08-02T17:55:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.585+0000 7f3e3e7fc640 1 -- 172.21.15.69:0/1355484416 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3e44044060 con 0x7f3e48109b30 2024-08-02T17:55:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.586+0000 7f3e3e7fc640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3e340a2dc0 0x7f3e340a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.588 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.587+0000 7f3e3e7fc640 1 -- 172.21.15.69:0/1355484416 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f3e440efba0 con 0x7f3e48109b30 2024-08-02T17:55:46.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.588+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3e340a2dc0 0x7f3e340a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.588+0000 7f3e51e11740 1 Processor -- start 2024-08-02T17:55:46.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.588+0000 7f3e51e11740 1 -- start start 2024-08-02T17:55:46.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9370 0x5610b3de9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3de9c90 0x5610b3df4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df4550 0x5610b3df6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e51e11740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5610b3d4e980 con 0x5610b3de9c90 2024-08-02T17:55:46.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e51e11740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610b3ce6a70 con 0x5610b3de9370 2024-08-02T17:55:46.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e51e11740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5610b3d4a940 con 0x5610b3df4550 2024-08-02T17:55:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e4d7af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9370 0x5610b3de9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e4d7af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9370 0x5610b3de9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42570/0 (socket says 172.21.15.69:42570) 2024-08-02T17:55:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e4d7af640 1 -- 172.21.15.69:0/1786287187 learned_addr learned my addr 172.21.15.69:0/1786287187 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.589+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3de9c90 0x5610b3df4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.590+0000 7f3e4dfb0640 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df4550 0x5610b3df6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.590+0000 7f3e4d7af640 1 -- 172.21.15.69:0/1786287187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df4550 msgr2=0x5610b3df6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.590+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df4550 0x5610b3df6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.590+0000 7f3e4d7af640 1 -- 172.21.15.69:0/1786287187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3de9c90 msgr2=0x5610b3df4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:46.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.590+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3de9c90 0x5610b3df4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.590+0000 7f3e4d7af640 1 -- 172.21.15.69:0/1786287187 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b3d496e0 con 0x5610b3de9370 2024-08-02T17:55:46.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.590+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9370 0x5610b3de9750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3e40004600 tx=0x7f3e40008830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e99cdc0ecbf8d6b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:46.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.590+0000 7f3e4dfb0640 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df4550 0x5610b3df6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:46.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.593+0000 7f3e337fe640 1 -- 172.21.15.69:0/1786287187 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e400047b0 con 0x5610b3de9370 2024-08-02T17:55:46.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.593+0000 7f3e337fe640 1 -- 172.21.15.69:0/1786287187 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e4000a530 con 0x5610b3de9370 2024-08-02T17:55:46.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.594+0000 7f3e51e11740 1 -- 172.21.15.69:0/1786287187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9370 msgr2=0x5610b3de9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:46.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.594+0000 7f3e51e11740 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9370 0x5610b3de9750 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3e40004600 tx=0x7f3e40008830 comp rx=0 tx=0).stop 2024-08-02T17:55:46.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.594+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3e340a2dc0 0x7f3e340a5280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e40003e80 tx=0x7f3e40019c90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:46.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.594+0000 7f3e51e11740 1 -- 172.21.15.69:0/1786287187 shutdown_connections 2024-08-02T17:55:46.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.595+0000 7f3e51e11740 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3de9c90 0x5610b3df4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.595+0000 7f3e51e11740 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df4550 0x5610b3df6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.595+0000 7f3e51e11740 1 --2- 172.21.15.69:0/1786287187 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9370 0x5610b3de9750 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.595+0000 7f3e51e11740 1 -- 172.21.15.69:0/1786287187 >> 172.21.15.69:0/1786287187 conn(0x5610b3de5900 msgr2=0x5610b3de5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:46.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.595+0000 7f3e51e11740 1 -- 172.21.15.69:0/1786287187 shutdown_connections 2024-08-02T17:55:46.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.595+0000 7f3e51e11740 1 -- 172.21.15.69:0/1786287187 wait complete. 2024-08-02T17:55:46.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.596+0000 7f3e51e11740 1 -- start start 2024-08-02T17:55:46.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.596+0000 7f3e51e11740 1 Processor -- start 2024-08-02T17:55:46.598 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.596+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3de9370 0x5610b3df3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3df12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e4d7af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3de9370 0x5610b3df3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e4d7af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3de9370 0x5610b3df3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39400/0 (socket says 172.21.15.69:39400) 2024-08-02T17:55:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e4d7af640 1 -- 172.21.15.69:0/2050382866 learned_addr learned my addr 172.21.15.69:0/2050382866 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3df12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5610b3df7170 con 0x5610b3df4550 2024-08-02T17:55:46.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5610b3d09aa0 con 0x5610b3de9c90 2024-08-02T17:55:46.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.597+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5610b3df72f0 con 0x5610b3de9370 2024-08-02T17:55:46.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.598+0000 7f3e51e11740 10 client.0 _ll_register_callbacks cb 0x5610b3cf8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:46.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.598+0000 7f3e4dfb0640 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.598+0000 7f3e4d7af640 1 -- 172.21.15.69:0/2050382866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 msgr2=0x5610b3df12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:46.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.598+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3df12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.598+0000 7f3e4d7af640 1 -- 172.21.15.69:0/2050382866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 msgr2=0x5610b3df1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:46.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.598+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.598+0000 7f3e4d7af640 1 -- 172.21.15.69:0/2050382866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b3ced650 con 0x5610b3de9370 2024-08-02T17:55:46.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.599+0000 7f3e4dfb0640 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:46.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.599+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3de9370 0x5610b3df3f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000afd0 tx=0x7f3e40004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e11af351dc50c318 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:46.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.599+0000 7f3e30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:55:46.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.600+0000 7f3e30ff9640 1 -- 172.21.15.69:0/2050382866 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40019e60 con 0x5610b3de9370 2024-08-02T17:55:46.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.600+0000 7f3e30ff9640 1 -- 172.21.15.69:0/2050382866 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f3e40003970 con 0x5610b3de9370 2024-08-02T17:55:46.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.600+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e4000a530 con 0x5610b3de9370 2024-08-02T17:55:46.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.600+0000 7f3e30ff9640 1 -- 172.21.15.69:0/2050382866 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e40004b20 con 0x5610b3de9370 2024-08-02T17:55:46.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.601+0000 7f3e51e11740 10 client.27040 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:46.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.601+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5610b3ac03c0 con 0x5610b3de9370 2024-08-02T17:55:46.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.601+0000 7f3e30ff9640 1 -- 172.21.15.69:0/2050382866 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e40029040 con 0x5610b3de9370 2024-08-02T17:55:46.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.601+0000 7f3e30ff9640 1 -- 172.21.15.69:0/2050382866 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3e4000a920 con 0x5610b3de9370 2024-08-02T17:55:46.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.602+0000 7f3e51e11740 10 client.27040 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:46.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.602+0000 7f3e51e11740 10 client.27040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:46.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.602+0000 7f3e51e11740 20 client.27040 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:46.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.602+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5610b3e03ec0 0x5610b3e062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.602+0000 7f3e51e11740 4 client.27040 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:46.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.602+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3e4000a530 con 0x5610b3e03ec0 2024-08-02T17:55:46.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.603+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5610b3e03ec0 0x5610b3e062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.607+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5610b3e03ec0 0x5610b3e062b0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1401b1e72686f7f2 server_cookie=3e065b5eae98c962 in_seq=0 out_seq=0 2024-08-02T17:55:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.607+0000 7f3e30ff9640 10 client.27040 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.608+0000 7f3e30ff9640 1 -- 172.21.15.69:0/2050382866 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f3e4000a530 con 0x5610b3e03ec0 2024-08-02T17:55:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.608+0000 7f3e30ff9640 10 client.27040 handle_command_reply: tid=0 2024-08-02T17:55:46.609 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.608+0000 7f3e51e11740 1 client.27040 shutdown 2024-08-02T17:55:46.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3de9370 msgr2=0x5610b3df3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:46.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3de9370 0x5610b3df3f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000afd0 tx=0x7f3e40004ab0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 shutdown_connections 2024-08-02T17:55:46.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5610b3e03ec0 0x5610b3e062b0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3df12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2050382866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3de9370 0x5610b3df3f80 secure :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000afd0 tx=0x7f3e40004ab0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 >> 172.21.15.69:0/2050382866 conn(0x5610b3de5900 msgr2=0x5610b3deb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:46.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.610+0000 7f3e30ff9640 0 client.27040 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:46.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.611+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 shutdown_connections 2024-08-02T17:55:46.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.611+0000 7f3e51e11740 1 -- 172.21.15.69:0/2050382866 wait complete. 2024-08-02T17:55:46.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.611+0000 7f3e51e11740 20 client.27040 trim_cache size 0 max 16384 2024-08-02T17:55:46.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.650+0000 7f3e51e11740 1 Processor -- start 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.650+0000 7f3e51e11740 1 -- start start 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3defba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3debe20 0x5610b3e0ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e51e11740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c012a50 con 0x5610b3df4550 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e51e11740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c010320 con 0x5610b3de9c90 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e51e11740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00e170 con 0x5610b3debe20 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e4dfb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3debe20 0x5610b3e0ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e4dfb0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3debe20 0x5610b3e0ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39416/0 (socket says 172.21.15.69:39416) 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e4dfb0640 1 -- 172.21.15.69:0/2341898523 learned_addr learned my addr 172.21.15.69:0/2341898523 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.651+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/2341898523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.652+0000 7f3e4dfb0640 1 -- 172.21.15.69:0/2341898523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 msgr2=0x5610b3defba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:55:46.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.652+0000 7f3e4dfb0640 1 --2- 172.21.15.69:0/2341898523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3defba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.652+0000 7f3e4dfb0640 1 -- 172.21.15.69:0/2341898523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 msgr2=0x5610b3df00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:46.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.652+0000 7f3e4dfb0640 1 --2- 172.21.15.69:0/2341898523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.652+0000 7f3e4dfb0640 1 -- 172.21.15.69:0/2341898523 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b3de5730 con 0x5610b3debe20 2024-08-02T17:55:46.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.652+0000 7f3e4dfb0640 1 --2- 172.21.15.69:0/2341898523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3debe20 0x5610b3e0ba70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e38004620 tx=0x7f3e38009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b388488ea288aa09 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:46.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.656+0000 7f3e32ffd640 1 -- 172.21.15.69:0/2341898523 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38003b20 con 0x5610b3debe20 2024-08-02T17:55:46.657 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.656+0000 7f3e32ffd640 1 -- 172.21.15.69:0/2341898523 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e38037070 con 0x5610b3debe20 2024-08-02T17:55:46.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.659+0000 7f3e32ffd640 1 -- 172.21.15.69:0/2341898523 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38004d80 con 0x5610b3debe20 2024-08-02T17:55:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.660+0000 7f3e51e11740 1 -- 172.21.15.69:0/2341898523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3debe20 msgr2=0x5610b3e0ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.660+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2341898523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3debe20 0x5610b3e0ba70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3e38004620 tx=0x7f3e38009b30 comp rx=0 tx=0).stop 2024-08-02T17:55:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.660+0000 7f3e51e11740 1 -- 172.21.15.69:0/2341898523 shutdown_connections 2024-08-02T17:55:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.660+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2341898523 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.660+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2341898523 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3debe20 0x5610b3e0ba70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.660+0000 7f3e51e11740 1 --2- 172.21.15.69:0/2341898523 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3defba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.660+0000 7f3e51e11740 1 -- 172.21.15.69:0/2341898523 >> 172.21.15.69:0/2341898523 conn(0x5610b3de5900 msgr2=0x5610b3dec550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.665+0000 7f3e51e11740 1 -- 172.21.15.69:0/2341898523 shutdown_connections 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.666+0000 7f3e51e11740 1 -- 172.21.15.69:0/2341898523 wait complete. 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.666+0000 7f3e51e11740 1 -- start start 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.666+0000 7f3e51e11740 1 Processor -- start 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.667+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3dfcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.667+0000 7f3e51e11740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.667+0000 7f3e4d7af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3dfcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.667+0000 7f3e4d7af640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3dfcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42600/0 (socket says 172.21.15.69:42600) 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.667+0000 7f3e4d7af640 1 -- 172.21.15.69:0/3094551696 learned_addr learned my addr 172.21.15.69:0/3094551696 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:46.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.667+0000 7f3e51e11740 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df5730 0x5610b3df5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00e7e0 con 0x5610b3df4550 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00fc50 con 0x5610b3de9c90 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e4dfb0640 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df5730 0x5610b3df5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c00eeb0 con 0x5610b3df5730 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e51e11740 10 client.0 _ll_register_callbacks cb 0x7f3e40004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e4d7af640 1 -- 172.21.15.69:0/3094551696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df5730 msgr2=0x5610b3df5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df5730 0x5610b3df5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e4d7af640 1 -- 172.21.15.69:0/3094551696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 msgr2=0x5610b3df51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:46.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.668+0000 7f3e4d7af640 1 -- 172.21.15.69:0/3094551696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b3dfa7d0 con 0x5610b3de9c90 2024-08-02T17:55:46.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.669+0000 7f3e4d7af640 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3dfcc70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e400045d0 tx=0x7f3e40019760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80c0d78fe642daa server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:46.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.669+0000 7f3e30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:55:46.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.669+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e38037070 con 0x5610b3de9c90 2024-08-02T17:55:46.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.670+0000 7f3e30ff9640 1 -- 172.21.15.69:0/3094551696 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40019e60 con 0x5610b3de9c90 2024-08-02T17:55:46.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.670+0000 7f3e30ff9640 1 -- 172.21.15.69:0/3094551696 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f3e40010040 con 0x5610b3de9c90 2024-08-02T17:55:46.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.671+0000 7f3e30ff9640 1 -- 172.21.15.69:0/3094551696 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3e40005470 con 0x5610b3de9c90 2024-08-02T17:55:46.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.675+0000 7f3e51e11740 10 client.27221 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:46.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.675+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5610b3e6ebe0 con 0x5610b3de9c90 2024-08-02T17:55:46.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.676+0000 7f3e30ff9640 1 -- 172.21.15.69:0/3094551696 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3e40005700 con 0x5610b3de9c90 2024-08-02T17:55:46.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.679+0000 7f3e51e11740 10 client.27221 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:46.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.679+0000 7f3e51e11740 10 client.27221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:46.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.679+0000 7f3e51e11740 20 client.27221 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:46.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.679+0000 7f3e51e11740 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5610b3e6f250 0x5610b3e71670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:46.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.679+0000 7f3e51e11740 4 client.27221 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:55:46.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.679+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610b3d5bef0 con 0x5610b3e6f250 2024-08-02T17:55:46.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.680+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5610b3e6f250 0x5610b3e71670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.693+0000 7f3e4cfae640 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5610b3e6f250 0x5610b3e71670 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c016b4b1251672d server_cookie=e8f77e9efd59b810 in_seq=0 out_seq=0 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.694+0000 7f3e30ff9640 10 client.27221 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.694+0000 7f3e30ff9640 1 -- 172.21.15.69:0/3094551696 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5610b3d5bef0 con 0x5610b3e6f250 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.694+0000 7f3e30ff9640 10 client.27221 handle_command_reply: tid=0 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.694+0000 7f3e51e11740 1 client.27221 shutdown 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.694+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 msgr2=0x5610b3dfcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.694+0000 7f3e51e11740 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3dfcc70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e400045d0 tx=0x7f3e40019760 comp rx=0 tx=0).stop 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.695+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 shutdown_connections 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.695+0000 7f3e51e11740 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5610b3df4550 0x5610b3df51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.695+0000 7f3e51e11740 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5610b3e6f250 0x5610b3e71670 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.695+0000 7f3e51e11740 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5610b3df5730 0x5610b3df5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.695+0000 7f3e51e11740 1 --2- 172.21.15.69:0/3094551696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5610b3de9c90 0x5610b3dfcc70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.695+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 >> 172.21.15.69:0/3094551696 conn(0x5610b3de5900 msgr2=0x5610b3defac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:46.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.695+0000 7f3e30ff9640 0 client.27221 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:46.701 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:55:46.701 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:55:46.701 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:55:46.701 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:55:46.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.698+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 shutdown_connections 2024-08-02T17:55:46.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.698+0000 7f3e51e11740 1 -- 172.21.15.69:0/3094551696 wait complete. 2024-08-02T17:55:46.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.698+0000 7f3e51e11740 20 client.27221 trim_cache size 0 max 16384 2024-08-02T17:55:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.699+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3e340a2dc0 msgr2=0x7f3e340a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.699+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3e340a2dc0 0x7f3e340a5280 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3e40003e80 tx=0x7f3e40019c90 comp rx=0 tx=0).stop 2024-08-02T17:55:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.699+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48109b30 msgr2=0x7f3e48114510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.699+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48109b30 0x7f3e48114510 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400b670 tx=0x7f3e44009b10 comp rx=0 tx=0).stop 2024-08-02T17:55:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.701+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 shutdown_connections 2024-08-02T17:55:46.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.701+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3e340a2dc0 0x7f3e340a5280 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.701+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3e4810e6a0 0x7f3e481ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.701+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e48109b30 0x7f3e48114510 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.701+0000 7f3e4e7b1640 1 --2- 172.21.15.69:0/1355484416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3e48109180 0x7f3e48113fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.701+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 >> 172.21.15.69:0/1355484416 conn(0x7f3e48072eb0 msgr2=0x7f3e4806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.703+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 shutdown_connections 2024-08-02T17:55:46.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:46.703+0000 7f3e4e7b1640 1 -- 172.21.15.69:0/1355484416 wait complete. 2024-08-02T17:55:46.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:46 smithi069 ceph-mon[32746]: pgmap v289: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.7 MiB/s wr, 727 op/s 2024-08-02T17:55:46.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:46 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:55:47.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:46 smithi083 ceph-mon[37031]: pgmap v289: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.7 MiB/s wr, 727 op/s 2024-08-02T17:55:47.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:46 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:55:47.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:46 smithi077 ceph-mon[37108]: pgmap v289: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.7 MiB/s wr, 727 op/s 2024-08-02T17:55:47.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:46 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:55:47.624 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:55:47.624 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:55:47.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:47 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:55:47.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:47 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:55:47.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:47 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:55:48.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:47 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:55:48.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:47 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:55:48.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:47 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:55:48.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:47 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:55:48.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:47 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:55:48.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:47 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:55:48.139 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:48.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.880+0000 7f8fc2134640 1 -- 172.21.15.69:0/2366263095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc06e760 msgr2=0x7f8fbc06eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:48.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.880+0000 7f8fc2134640 1 --2- 172.21.15.69:0/2366263095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc06e760 0x7f8fbc06eba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009900 tx=0x7f8fb802a3f0 comp rx=0 tx=0).stop 2024-08-02T17:55:48.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.881+0000 7f8fc2134640 1 -- 172.21.15.69:0/2366263095 shutdown_connections 2024-08-02T17:55:48.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.881+0000 7f8fc2134640 1 --2- 172.21.15.69:0/2366263095 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fbc11ae70 0x7f8fbc11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:48.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.881+0000 7f8fc2134640 1 --2- 172.21.15.69:0/2366263095 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc06e760 0x7f8fbc06eba0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:48.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.881+0000 7f8fc2134640 1 --2- 172.21.15.69:0/2366263095 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fbc10fec0 0x7f8fbc06e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:48.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.881+0000 7f8fc2134640 1 -- 172.21.15.69:0/2366263095 >> 172.21.15.69:0/2366263095 conn(0x7f8fbc072eb0 msgr2=0x7f8fbc0732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.883+0000 7f8fc2134640 1 -- 172.21.15.69:0/2366263095 shutdown_connections 2024-08-02T17:55:48.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.883+0000 7f8fc2134640 1 -- 172.21.15.69:0/2366263095 wait complete. 2024-08-02T17:55:48.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc2134640 1 Processor -- start 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc2134640 1 -- start start 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc2134640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fbc06e760 0x7f8fbc11a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc2134640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc10fec0 0x7f8fbc1134b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc2134640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fbc11ae70 0x7f8fbc1139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc2134640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc120980 con 0x7f8fbc06e760 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc2134640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc120800 con 0x7f8fbc11ae70 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc2134640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc120b00 con 0x7f8fbc10fec0 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.884+0000 7f8fc1933640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fbc11ae70 0x7f8fbc1139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1933640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fbc11ae70 0x7f8fbc1139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42612/0 (socket says 172.21.15.69:42612) 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1933640 1 -- 172.21.15.69:0/3687695445 learned_addr learned my addr 172.21.15.69:0/3687695445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1933640 1 -- 172.21.15.69:0/3687695445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc10fec0 msgr2=0x7f8fbc1134b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:55:48.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1933640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc10fec0 0x7f8fbc1134b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1933640 1 -- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fbc06e760 msgr2=0x7f8fbc11a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1132640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fbc06e760 0x7f8fbc11a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1933640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fbc06e760 0x7f8fbc11a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1933640 1 -- 172.21.15.69:0/3687695445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc113f60 con 0x7f8fbc11ae70 2024-08-02T17:55:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.885+0000 7f8fc1132640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fbc06e760 0x7f8fbc11a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:55:48.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.886+0000 7f8fc1933640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fbc11ae70 0x7f8fbc1139f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8fac030980 tx=0x7f8fac030e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:48.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.887+0000 7f8fb27fc640 1 -- 172.21.15.69:0/3687695445 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac02ecd0 con 0x7f8fbc11ae70 2024-08-02T17:55:48.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.887+0000 7f8fb27fc640 1 -- 172.21.15.69:0/3687695445 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fac005500 con 0x7f8fbc11ae70 2024-08-02T17:55:48.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.887+0000 7f8fb27fc640 1 -- 172.21.15.69:0/3687695445 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac032440 con 0x7f8fbc11ae70 2024-08-02T17:55:48.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.890+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc114250 con 0x7f8fbc11ae70 2024-08-02T17:55:48.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.890+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc1181c0 con 0x7f8fbc11ae70 2024-08-02T17:55:48.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.892+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc109420 con 0x7f8fbc11ae70 2024-08-02T17:55:48.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.897+0000 7f8fb27fc640 1 -- 172.21.15.69:0/3687695445 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8fac066bb0 con 0x7f8fbc11ae70 2024-08-02T17:55:48.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.898+0000 7f8fb27fc640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fa80a2ec0 0x7f8fa80a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:48.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.898+0000 7f8fb27fc640 1 -- 172.21.15.69:0/3687695445 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f8fac0f1d10 con 0x7f8fbc11ae70 2024-08-02T17:55:48.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.899+0000 7f8fc1132640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fa80a2ec0 0x7f8fa80a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:48.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.899+0000 7f8fb27fc640 1 -- 172.21.15.69:0/3687695445 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f8fac0ba4d0 con 0x7f8fbc11ae70 2024-08-02T17:55:48.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:48.903+0000 7f8fc1132640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fa80a2ec0 0x7f8fa80a5380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007eb0 tx=0x7f8fb40073c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:49.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.264+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fbc1149c0 con 0x7f8fbc11ae70 2024-08-02T17:55:49.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:49 smithi069 ceph-mon[32746]: pgmap v290: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 1.01k op/s 2024-08-02T17:55:49.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.265+0000 7f8fb27fc640 1 -- 172.21.15.69:0/3687695445 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f8fbc1149c0 con 0x7f8fbc11ae70 2024-08-02T17:55:49.269 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:55:49.270 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:55:49.270 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:55:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:49 smithi083 ceph-mon[37031]: pgmap v290: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 1.01k op/s 2024-08-02T17:55:49.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.272+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fa80a2ec0 msgr2=0x7f8fa80a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:49.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.272+0000 7f8fc2134640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fa80a2ec0 0x7f8fa80a5380 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007eb0 tx=0x7f8fb40073c0 comp rx=0 tx=0).stop 2024-08-02T17:55:49.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.272+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fbc11ae70 msgr2=0x7f8fbc1139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:49.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.272+0000 7f8fc2134640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fbc11ae70 0x7f8fbc1139f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8fac030980 tx=0x7f8fac030e40 comp rx=0 tx=0).stop 2024-08-02T17:55:49.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.273+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 shutdown_connections 2024-08-02T17:55:49.275 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.273+0000 7f8fc2134640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8fa80a2ec0 0x7f8fa80a5380 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:49.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.274+0000 7f8fc2134640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8fbc11ae70 0x7f8fbc1139f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:49.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.274+0000 7f8fc2134640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8fbc10fec0 0x7f8fbc1134b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:49.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.274+0000 7f8fc2134640 1 --2- 172.21.15.69:0/3687695445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8fbc06e760 0x7f8fbc11a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:49.276 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.274+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 >> 172.21.15.69:0/3687695445 conn(0x7f8fbc072eb0 msgr2=0x7f8fbc11b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:49.279 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.278+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 shutdown_connections 2024-08-02T17:55:49.280 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:49.279+0000 7f8fc2134640 1 -- 172.21.15.69:0/3687695445 wait complete. 2024-08-02T17:55:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:49 smithi077 ceph-mon[37108]: pgmap v290: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 9.2 MiB/s wr, 1.01k op/s 2024-08-02T17:55:49.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 520.0 s, 901.4 tps, lat 110.337 ms stddev 133.682 2024-08-02T17:55:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:50 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3687695445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:55:50.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:50 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3687695445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:55:50.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:50 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3687695445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:55:50.827 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:55:51.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:51 smithi069 ceph-mon[32746]: pgmap v291: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.5 MiB/s wr, 813 op/s 2024-08-02T17:55:51.267 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:51.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:51 smithi083 ceph-mon[37031]: pgmap v291: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.5 MiB/s wr, 813 op/s 2024-08-02T17:55:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:51 smithi077 ceph-mon[37108]: pgmap v291: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.5 MiB/s wr, 813 op/s 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 -- 172.21.15.69:0/1750917767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10d250 msgr2=0x7feebc114dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 --2- 172.21.15.69:0/1750917767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10d250 0x7feebc114dd0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009900 tx=0x7feeb402a410 comp rx=0 tx=0).stop 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 -- 172.21.15.69:0/1750917767 shutdown_connections 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 --2- 172.21.15.69:0/1750917767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc10d250 0x7feebc114dd0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 --2- 172.21.15.69:0/1750917767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc108ad0 0x7feebc10cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 --2- 172.21.15.69:0/1750917767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feebc108120 0x7feebc108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 -- 172.21.15.69:0/1750917767 >> 172.21.15.69:0/1750917767 conn(0x7feebc077830 msgr2=0x7feebc079c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 -- 172.21.15.69:0/1750917767 shutdown_connections 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.843+0000 7feec185b640 1 -- 172.21.15.69:0/1750917767 wait complete. 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.844+0000 7feec185b640 1 Processor -- start 2024-08-02T17:55:51.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.844+0000 7feec185b640 1 -- start start 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.844+0000 7feec185b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feebc108120 0x7feebc19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.844+0000 7feec185b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc108ad0 0x7feebc19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feec185b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc10d250 0x7feebc1a2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feec185b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feebc118260 con 0x7feebc108ad0 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feec185b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1180e0 con 0x7feebc108120 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feec185b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1183e0 con 0x7feebc10d250 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feebc108120 0x7feebc19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feebc108120 0x7feebc19e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42630/0 (socket says 172.21.15.69:42630) 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 -- 172.21.15.69:0/3865544834 learned_addr learned my addr 172.21.15.69:0/3865544834 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebb7fe640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc10d250 0x7feebc1a2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feeba7fc640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc108ad0 0x7feebc19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 -- 172.21.15.69:0/3865544834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc10d250 msgr2=0x7feebc1a2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc10d250 0x7feebc1a2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 -- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc108ad0 msgr2=0x7feebc19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc108ad0 0x7feebc19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 -- 172.21.15.69:0/3865544834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc1a2730 con 0x7feebc108120 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebb7fe640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc10d250 0x7feebc1a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.845+0000 7feebaffd640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feebc108120 0x7feebc19e7e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feeac00e9f0 tx=0x7feeac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.846+0000 7feec0859640 1 -- 172.21.15.69:0/3865544834 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac00cd80 con 0x7feebc108120 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.846+0000 7feec0859640 1 -- 172.21.15.69:0/3865544834 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac010910 con 0x7feebc108120 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.846+0000 7feec0859640 1 -- 172.21.15.69:0/3865544834 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac018990 con 0x7feebc108120 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.846+0000 7feec185b640 1 -- 172.21.15.69:0/3865544834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc1a2a20 con 0x7feebc108120 2024-08-02T17:55:51.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.846+0000 7feec185b640 1 -- 172.21.15.69:0/3865544834 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc1ab390 con 0x7feebc108120 2024-08-02T17:55:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.850+0000 7feec0859640 1 -- 172.21.15.69:0/3865544834 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feeac010ac0 con 0x7feebc108120 2024-08-02T17:55:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.850+0000 7feec3c59740 1 Processor -- start 2024-08-02T17:55:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.850+0000 7feec3c59740 1 -- start start 2024-08-02T17:55:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.850+0000 7feec3c59740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb29b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.850+0000 7feec3c59740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.850+0000 7feec0859640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feea40a2d70 0x7feea40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.850+0000 7feec3c59740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb2a61d0 0x55fecb2a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb29b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb29b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42646/0 (socket says 172.21.15.69:42646) 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 -- 172.21.15.69:0/2087556457 learned_addr learned my addr 172.21.15.69:0/2087556457 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feec3c59740 1 -- 172.21.15.69:0/2087556457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fecb200a40 con 0x55fecb2a61d0 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feec3c59740 1 -- 172.21.15.69:0/2087556457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fecb1989d0 con 0x55fecb29aff0 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feec3c59740 1 -- 172.21.15.69:0/2087556457 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fecb1fca00 con 0x55fecb29b910 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feeba7fc640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feea40a2d70 0x7feea40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feec0859640 1 -- 172.21.15.69:0/3865544834 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7feeac0cfbb0 con 0x7feebc108120 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebaffd640 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 -- 172.21.15.69:0/2087556457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 msgr2=0x55fecb2a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 -- 172.21.15.69:0/2087556457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb2a61d0 msgr2=0x55fecb2a85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb2a61d0 0x55fecb2a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 -- 172.21.15.69:0/2087556457 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fecb19f650 con 0x55fecb29aff0 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebaffd640 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feebb7fe640 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb29b3d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009900 tx=0x7feeb4005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e52b07414350d1f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.851+0000 7feeba7fc640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feea40a2d70 0x7feea40a5230 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feebc10eaf0 tx=0x7feeb0009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feea1ffb640 1 -- 172.21.15.69:0/2087556457 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035030 con 0x55fecb29aff0 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feea1ffb640 1 -- 172.21.15.69:0/2087556457 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb4041070 con 0x55fecb29aff0 2024-08-02T17:55:51.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feea1ffb640 1 -- 172.21.15.69:0/2087556457 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4003c90 con 0x55fecb29aff0 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feec3c59740 1 -- 172.21.15.69:0/2087556457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 msgr2=0x55fecb29b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feec3c59740 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb29b3d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009900 tx=0x7feeb4005d50 comp rx=0 tx=0).stop 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feec3c59740 1 -- 172.21.15.69:0/2087556457 shutdown_connections 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feec3c59740 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb2a61d0 0x55fecb2a85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feec3c59740 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feec3c59740 1 --2- 172.21.15.69:0/2087556457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb29b3d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.852+0000 7feec3c59740 1 -- 172.21.15.69:0/2087556457 >> 172.21.15.69:0/2087556457 conn(0x55fecb297580 msgr2=0x55fecb297960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.853+0000 7feec3c59740 1 -- 172.21.15.69:0/2087556457 shutdown_connections 2024-08-02T17:55:51.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.853+0000 7feec3c59740 1 -- 172.21.15.69:0/2087556457 wait complete. 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.853+0000 7feec3c59740 1 -- start start 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.853+0000 7feec3c59740 1 Processor -- start 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.853+0000 7feec3c59740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.853+0000 7feec3c59740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.853+0000 7feec3c59740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feec3c59740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fecb2a8da0 con 0x55fecb29aff0 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feec3c59740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fecb1bbb20 con 0x55fecb2a61d0 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feec3c59740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fecb2a8f20 con 0x55fecb29b910 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feebaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39474/0 (socket says 172.21.15.69:39474) 2024-08-02T17:55:51.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feebaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45896/0 (socket says 172.21.15.69:45896) 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 -- 172.21.15.69:0/2598784765 learned_addr learned my addr 172.21.15.69:0/2598784765 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feec3c59740 10 client.0 _ll_register_callbacks cb 0x55fecb1b9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 -- 172.21.15.69:0/2598784765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 msgr2=0x55fecb2a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 -- 172.21.15.69:0/2598784765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 msgr2=0x55fecb2a3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 -- 172.21.15.69:0/2598784765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fecaf723c0 con 0x55fecb29b910 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feebaffd640 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.854+0000 7feeba7fc640 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a3650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feeb00042c0 tx=0x7feeb0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad2301b0f607870 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.855+0000 7fee9affd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.855+0000 7fee9affd640 1 -- 172.21.15.69:0/2598784765 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb000ecc0 con 0x55fecb29b910 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.855+0000 7fee9affd640 1 -- 172.21.15.69:0/2598784765 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7feeb001e420 con 0x55fecb29b910 2024-08-02T17:55:51.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.855+0000 7feec3c59740 1 -- 172.21.15.69:0/2598784765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb4041070 con 0x55fecb29b910 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.855+0000 7fee9affd640 1 -- 172.21.15.69:0/2598784765 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feeb002f050 con 0x55fecb29b910 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.855+0000 7feec3c59740 10 client.27061 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.855+0000 7feec3c59740 1 -- 172.21.15.69:0/2598784765 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fecb2b4a20 con 0x55fecb29b910 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.855+0000 7fee9affd640 1 -- 172.21.15.69:0/2598784765 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb4041070 con 0x55fecb29b910 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7fee9affd640 1 -- 172.21.15.69:0/2598784765 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feeb0026520 con 0x55fecb29b910 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7feec3c59740 10 client.27061 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7feec3c59740 10 client.27061 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7feec3c59740 20 client.27061 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7feec3c59740 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55fecb2b5ee0 0x55fecb2b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7feec3c59740 4 client.27061 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7feec3c59740 1 -- 172.21.15.69:0/2598784765 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fecb20dfb0 con 0x55fecb2b5ee0 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7feebaffd640 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55fecb2b5ee0 0x55fecb2b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7fee9affd640 10 client.27061 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:55:51.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.856+0000 7feebaffd640 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55fecb2b5ee0 0x55fecb2b8370 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4b35665410cde406 server_cookie=adc6cb9af5a9eb3f in_seq=0 out_seq=0 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7fee9affd640 1 -- 172.21.15.69:0/2598784765 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fecb20dfb0 con 0x55fecb2b5ee0 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7fee9affd640 10 client.27061 handle_command_reply: tid=0 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 client.27061 shutdown 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 -- 172.21.15.69:0/2598784765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 msgr2=0x55fecb2a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a3650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7feeb00042c0 tx=0x7feeb0004ab0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 -- 172.21.15.69:0/2598784765 shutdown_connections 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55fecb2b5ee0 0x55fecb2b8370 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a3650 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 --2- 172.21.15.69:0/2598784765 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7feec3c59740 1 -- 172.21.15.69:0/2598784765 >> 172.21.15.69:0/2598784765 conn(0x55fecb297580 msgr2=0x55fecb29d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.857+0000 7fee9affd640 0 client.27061 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.858+0000 7feec3c59740 1 -- 172.21.15.69:0/2598784765 shutdown_connections 2024-08-02T17:55:51.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.858+0000 7feec3c59740 1 -- 172.21.15.69:0/2598784765 wait complete. 2024-08-02T17:55:51.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.858+0000 7feec3c59740 20 client.27061 trim_cache size 0 max 16384 2024-08-02T17:55:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.874+0000 7feec3c59740 1 Processor -- start 2024-08-02T17:55:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.874+0000 7feec3c59740 1 -- start start 2024-08-02T17:55:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.874+0000 7feec3c59740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb2a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feec3c59740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29b910 0x55fecb2bdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feec3c59740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb2a61d0 0x55fecb2be1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feec3c59740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b7ea0 con 0x55fecb29b910 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feec3c59740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b5770 con 0x55fecb29aff0 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feec3c59740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b35c0 con 0x55fecb2a61d0 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29b910 0x55fecb2bdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29b910 0x55fecb2bdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45900/0 (socket says 172.21.15.69:45900) 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 -- 172.21.15.69:0/2876100963 learned_addr learned my addr 172.21.15.69:0/2876100963 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feebaffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb2a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feebb7fe640 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb2a61d0 0x55fecb2be1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 -- 172.21.15.69:0/2876100963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb2a61d0 msgr2=0x55fecb2be1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb2a61d0 0x55fecb2be1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 -- 172.21.15.69:0/2876100963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 msgr2=0x55fecb2a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:51.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb2a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 -- 172.21.15.69:0/2876100963 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fecb2a8900 con 0x55fecb29b910 2024-08-02T17:55:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feebaffd640 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb2a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.875+0000 7feeba7fc640 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29b910 0x55fecb2bdc80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7feeb0016040 tx=0x7feeb00093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2e016fae0a3304 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:51.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.876+0000 7feea17fa640 1 -- 172.21.15.69:0/2876100963 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb003a020 con 0x55fecb29b910 2024-08-02T17:55:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.876+0000 7feea17fa640 1 -- 172.21.15.69:0/2876100963 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb0011cf0 con 0x55fecb29b910 2024-08-02T17:55:51.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.876+0000 7feea17fa640 1 -- 172.21.15.69:0/2876100963 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0026660 con 0x55fecb29b910 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.876+0000 7feec3c59740 1 -- 172.21.15.69:0/2876100963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29b910 msgr2=0x55fecb2bdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.876+0000 7feec3c59740 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29b910 0x55fecb2bdc80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7feeb0016040 tx=0x7feeb00093c0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 -- 172.21.15.69:0/2876100963 shutdown_connections 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29b910 0x55fecb2bdc80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb2a61d0 0x55fecb2be1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 --2- 172.21.15.69:0/2876100963 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb29aff0 0x55fecb2a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 -- 172.21.15.69:0/2876100963 >> 172.21.15.69:0/2876100963 conn(0x55fecb297580 msgr2=0x55fecb2a19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 -- 172.21.15.69:0/2876100963 shutdown_connections 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 -- 172.21.15.69:0/2876100963 wait complete. 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 -- start start 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 Processor -- start 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b3c30 con 0x55fecb29aff0 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b50a0 con 0x55fecb2a61d0 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b4300 con 0x55fecb29b910 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.877+0000 7feec3c59740 10 client.0 _ll_register_callbacks cb 0x7feeb0005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42664/0 (socket says 172.21.15.69:42664) 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feeba7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 -- 172.21.15.69:0/648217288 learned_addr learned my addr 172.21.15.69:0/648217288 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45902/0 (socket says 172.21.15.69:45902) 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 -- 172.21.15.69:0/648217288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 msgr2=0x55fecb2a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:51.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 -- 172.21.15.69:0/648217288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 msgr2=0x55fecb2a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 -- 172.21.15.69:0/648217288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fecb31ef30 con 0x55fecb2a61d0 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feeba7fc640 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7feebb7fe640 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a9bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feeb4033000 tx=0x7feeb4005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a4333aafafc513 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7fee9affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.878+0000 7fee9affd640 1 -- 172.21.15.69:0/648217288 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035070 con 0x55fecb2a61d0 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7fee9affd640 1 -- 172.21.15.69:0/648217288 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7feeb4005040 con 0x55fecb2a61d0 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7feec3c59740 1 -- 172.21.15.69:0/648217288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb0011cf0 con 0x55fecb2a61d0 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7fee9affd640 1 -- 172.21.15.69:0/648217288 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7feeb4005240 con 0x55fecb2a61d0 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7feec3c59740 10 client.27245 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7feec3c59740 1 -- 172.21.15.69:0/648217288 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fecb320190 con 0x55fecb2a61d0 2024-08-02T17:55:51.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7fee9affd640 1 -- 172.21.15.69:0/648217288 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb0011cf0 con 0x55fecb2a61d0 2024-08-02T17:55:51.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7fee9affd640 1 -- 172.21.15.69:0/648217288 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7feeb403d4e0 con 0x55fecb2a61d0 2024-08-02T17:55:51.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7feec3c59740 10 client.27245 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:51.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.879+0000 7feec3c59740 10 client.27245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:55:51.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.880+0000 7feec3c59740 20 client.27245 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.880+0000 7feec3c59740 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55fecb320480 0x55fecb322870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.880+0000 7feec3c59740 4 client.27245 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.880+0000 7feec3c59740 1 -- 172.21.15.69:0/648217288 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fecb2084c0 con 0x55fecb320480 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.880+0000 7feebaffd640 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55fecb320480 0x55fecb322870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feebaffd640 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55fecb320480 0x55fecb322870 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ab38b11d81128b7 server_cookie=9048e35b816bea7e in_seq=0 out_seq=0 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7fee9affd640 10 client.27245 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7fee9affd640 1 -- 172.21.15.69:0/648217288 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fecb2084c0 con 0x55fecb320480 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7fee9affd640 10 client.27245 handle_command_reply: tid=0 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 client.27245 shutdown 2024-08-02T17:55:51.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 -- 172.21.15.69:0/648217288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 msgr2=0x55fecb2a9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a9bf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feeb4033000 tx=0x7feeb4005850 comp rx=0 tx=0).stop 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 -- 172.21.15.69:0/648217288 shutdown_connections 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55fecb320480 0x55fecb322870 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fecb2a61d0 0x55fecb2a9bf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fecb29b910 0x55fecb2a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7fee9affd640 0 client.27245 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 --2- 172.21.15.69:0/648217288 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fecb29aff0 0x55fecb2a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.881+0000 7feec3c59740 1 -- 172.21.15.69:0/648217288 >> 172.21.15.69:0/648217288 conn(0x55fecb297580 msgr2=0x55fecb29ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec3c59740 1 -- 172.21.15.69:0/648217288 shutdown_connections 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec3c59740 1 -- 172.21.15.69:0/648217288 wait complete. 2024-08-02T17:55:51.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec3c59740 20 client.27245 trim_cache size 0 max 16384 2024-08-02T17:55:51.890 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 -- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feea40a2d70 msgr2=0x7feea40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feea40a2d70 0x7feea40a5230 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feebc10eaf0 tx=0x7feeb0009500 comp rx=0 tx=0).stop 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 -- 172.21.15.69:0/3865544834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feebc108120 msgr2=0x7feebc19e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feebc108120 0x7feebc19e7e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feeac00e9f0 tx=0x7feeac00eeb0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 -- 172.21.15.69:0/3865544834 shutdown_connections 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feebc108ad0 0x7feebc19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feea40a2d70 0x7feea40a5230 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc10d250 0x7feebc1a2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 --2- 172.21.15.69:0/3865544834 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feebc108120 0x7feebc19e7e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:51.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 -- 172.21.15.69:0/3865544834 >> 172.21.15.69:0/3865544834 conn(0x7feebc077830 msgr2=0x7feebc078b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:51.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 -- 172.21.15.69:0/3865544834 shutdown_connections 2024-08-02T17:55:51.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:51.882+0000 7feec185b640 1 -- 172.21.15.69:0/3865544834 wait complete. 2024-08-02T17:55:52.600 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:55:53.082 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.343+0000 7f90533d0640 1 -- 172.21.15.69:0/1682857035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c105bd0 msgr2=0x7f904c106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.343+0000 7f90533d0640 1 --2- 172.21.15.69:0/1682857035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c105bd0 0x7f904c106060 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9044009cd0 tx=0x7f904402a9a0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:53 smithi077 ceph-mon[37108]: pgmap v292: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.1 MiB/s wr, 766 op/s 2024-08-02T17:55:53.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.347+0000 7f90533d0640 1 -- 172.21.15.69:0/1682857035 shutdown_connections 2024-08-02T17:55:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.347+0000 7f90533d0640 1 --2- 172.21.15.69:0/1682857035 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c105bd0 0x7f904c106060 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.347+0000 7f90533d0640 1 --2- 172.21.15.69:0/1682857035 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c1010c0 0x7f904c105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.347+0000 7f90533d0640 1 --2- 172.21.15.69:0/1682857035 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f904c1007a0 0x7f904c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.347+0000 7f90533d0640 1 -- 172.21.15.69:0/1682857035 >> 172.21.15.69:0/1682857035 conn(0x7f904c0776d0 msgr2=0x7f904c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.354 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.353+0000 7f90533d0640 1 -- 172.21.15.69:0/1682857035 shutdown_connections 2024-08-02T17:55:53.356 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.354+0000 7f90533d0640 1 -- 172.21.15.69:0/1682857035 wait complete. 2024-08-02T17:55:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.355+0000 7f90533d0640 1 Processor -- start 2024-08-02T17:55:53.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.355+0000 7f90533d0640 1 -- start start 2024-08-02T17:55:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.356+0000 7f90533d0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c1007a0 0x7f904c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.356+0000 7f90533d0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f904c1010c0 0x7f904c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.356+0000 7f90533d0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c105bd0 0x7f904c1a2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.356+0000 7f90533d0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f904c118290 con 0x7f904c1007a0 2024-08-02T17:55:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.356+0000 7f90533d0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f904c118110 con 0x7f904c105bd0 2024-08-02T17:55:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.356+0000 7f90533d0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f904c118410 con 0x7f904c1010c0 2024-08-02T17:55:53.358 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.356+0000 7f905096b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f904c1010c0 0x7f904c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:53 smithi069 ceph-mon[32746]: pgmap v292: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.1 MiB/s wr, 766 op/s 2024-08-02T17:55:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905116c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c1007a0 0x7f904c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905116c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c1007a0 0x7f904c19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45924/0 (socket says 172.21.15.69:45924) 2024-08-02T17:55:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905116c640 1 -- 172.21.15.69:0/157099921 learned_addr learned my addr 172.21.15.69:0/157099921 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905196d640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c105bd0 0x7f904c1a2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905116c640 1 -- 172.21.15.69:0/157099921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f904c1010c0 msgr2=0x7f904c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905116c640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f904c1010c0 0x7f904c19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905116c640 1 -- 172.21.15.69:0/157099921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c105bd0 msgr2=0x7f904c1a2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905116c640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c105bd0 0x7f904c1a2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905116c640 1 -- 172.21.15.69:0/157099921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f904c1a3700 con 0x7f904c1007a0 2024-08-02T17:55:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905096b640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f904c1010c0 0x7f904c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.357+0000 7f905196d640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c105bd0 0x7f904c1a2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:55:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.358+0000 7f905116c640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c1007a0 0x7f904c19e710 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f903c00b450 tx=0x7f903c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.358+0000 7f90427fc640 1 -- 172.21.15.69:0/157099921 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c015020 con 0x7f904c1007a0 2024-08-02T17:55:53.361 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.358+0000 7f90427fc640 1 -- 172.21.15.69:0/157099921 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903c011d70 con 0x7f904c1007a0 2024-08-02T17:55:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.358+0000 7f90427fc640 1 -- 172.21.15.69:0/157099921 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c00f340 con 0x7f904c1007a0 2024-08-02T17:55:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.358+0000 7f90533d0640 1 -- 172.21.15.69:0/157099921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f904c1a39f0 con 0x7f904c1007a0 2024-08-02T17:55:53.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.359+0000 7f90533d0640 1 -- 172.21.15.69:0/157099921 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904c1ab280 con 0x7f904c1007a0 2024-08-02T17:55:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.362+0000 7f9056200740 1 Processor -- start 2024-08-02T17:55:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f9056200740 1 -- start start 2024-08-02T17:55:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f9056200740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29a7b0 0x5646aa29ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f9056200740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa2a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f9056200740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a5970 0x5646aa2a7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f9056200740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5646aa1fe9b0 con 0x5646aa29b0d0 2024-08-02T17:55:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f9056200740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5646aa1a6800 con 0x5646aa2a5970 2024-08-02T17:55:53.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f9056200740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646aa1fa970 con 0x5646aa29a7b0 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905196d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29a7b0 0x5646aa29ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905096b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a5970 0x5646aa2a7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905196d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29a7b0 0x5646aa29ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39502/0 (socket says 172.21.15.69:39502) 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905196d640 1 -- 172.21.15.69:0/1550663779 learned_addr learned my addr 172.21.15.69:0/1550663779 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f90427fc640 1 -- 172.21.15.69:0/157099921 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f903c018020 con 0x7f904c1007a0 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905096b640 1 -- 172.21.15.69:0/1550663779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29a7b0 msgr2=0x5646aa29ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905096b640 1 --2- 172.21.15.69:0/1550663779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29a7b0 0x5646aa29ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905096b640 1 -- 172.21.15.69:0/1550663779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 msgr2=0x5646aa2a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905096b640 1 --2- 172.21.15.69:0/1550663779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa2a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905096b640 1 -- 172.21.15.69:0/1550663779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646a9f12210 con 0x5646aa2a5970 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.363+0000 7f905196d640 1 --2- 172.21.15.69:0/1550663779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29a7b0 0x5646aa29ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:53.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.364+0000 7f905096b640 1 --2- 172.21.15.69:0/1550663779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a5970 0x5646aa2a7d60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9048009fa0 tx=0x7f90480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d819b00f9ca09dc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.364+0000 7f90377fe640 1 -- 172.21.15.69:0/1550663779 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048004030 con 0x5646aa2a5970 2024-08-02T17:55:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.364+0000 7f90377fe640 1 -- 172.21.15.69:0/1550663779 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9048005500 con 0x5646aa2a5970 2024-08-02T17:55:53.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.364+0000 7f90377fe640 1 -- 172.21.15.69:0/1550663779 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048004e10 con 0x5646aa2a5970 2024-08-02T17:55:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.364+0000 7f90427fc640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f90380a2e80 0x7f90380a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.364+0000 7f90427fc640 1 -- 172.21.15.69:0/157099921 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f903c0cf170 con 0x7f904c1007a0 2024-08-02T17:55:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.365+0000 7f905196d640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f90380a2e80 0x7f90380a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.365+0000 7f9056200740 1 -- 172.21.15.69:0/1550663779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a5970 msgr2=0x5646aa2a7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.365+0000 7f9056200740 1 --2- 172.21.15.69:0/1550663779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a5970 0x5646aa2a7d60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9048009fa0 tx=0x7f90480118d0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.365+0000 7f9056200740 1 -- 172.21.15.69:0/1550663779 shutdown_connections 2024-08-02T17:55:53.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.365+0000 7f9056200740 1 --2- 172.21.15.69:0/1550663779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa2a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.365+0000 7f9056200740 1 --2- 172.21.15.69:0/1550663779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a5970 0x5646aa2a7d60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.365+0000 7f9056200740 1 --2- 172.21.15.69:0/1550663779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29a7b0 0x5646aa29ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.365+0000 7f9056200740 1 -- 172.21.15.69:0/1550663779 >> 172.21.15.69:0/1550663779 conn(0x5646aa296d20 msgr2=0x5646aa297100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.366+0000 7f9056200740 1 -- 172.21.15.69:0/1550663779 shutdown_connections 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.366+0000 7f9056200740 1 -- 172.21.15.69:0/1550663779 wait complete. 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.366+0000 7f905196d640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f90380a2e80 0x7f90380a5340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9044005fe0 tx=0x7f904402ffa0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.366+0000 7f9056200740 1 -- start start 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.366+0000 7f9056200740 1 Processor -- start 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.367+0000 7f9056200740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29a7b0 0x5646aa2a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.367+0000 7f9056200740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29b0d0 0x5646aa2a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.367+0000 7f9056200740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa2a5970 0x5646aa2a3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.368+0000 7f9056200740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5646aa2a8540 con 0x5646aa2a5970 2024-08-02T17:55:53.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.368+0000 7f9056200740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5646aa1ba500 con 0x5646aa29a7b0 2024-08-02T17:55:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.368+0000 7f9056200740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646aa2a86c0 con 0x5646aa29b0d0 2024-08-02T17:55:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.368+0000 7f9056200740 10 client.0 _ll_register_callbacks cb 0x5646a9f11a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:53.371 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.370+0000 7f905096b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29b0d0 0x5646aa2a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.372+0000 7f905116c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29a7b0 0x5646aa2a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.373+0000 7f905096b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29b0d0 0x5646aa2a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39506/0 (socket says 172.21.15.69:39506) 2024-08-02T17:55:53.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.373+0000 7f905096b640 1 -- 172.21.15.69:0/1506322223 learned_addr learned my addr 172.21.15.69:0/1506322223 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.376+0000 7f905116c640 1 -- 172.21.15.69:0/1506322223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29b0d0 msgr2=0x5646aa2a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.376+0000 7f905116c640 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29b0d0 0x5646aa2a2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.376+0000 7f905116c640 1 -- 172.21.15.69:0/1506322223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa2a5970 msgr2=0x5646aa2a3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.376+0000 7f905116c640 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa2a5970 0x5646aa2a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.376+0000 7f905116c640 1 -- 172.21.15.69:0/1506322223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646a9fde620 con 0x5646aa29a7b0 2024-08-02T17:55:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.377+0000 7f905116c640 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29a7b0 0x5646aa2a28e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f903c0cc5c0 tx=0x7f903c09af90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5017c1014f5084 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.378+0000 7f9034ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:55:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.385+0000 7f9034ff9640 1 -- 172.21.15.69:0/1506322223 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c0cc930 con 0x5646aa29a7b0 2024-08-02T17:55:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.385+0000 7f9056200740 1 -- 172.21.15.69:0/1506322223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9048005500 con 0x5646aa29a7b0 2024-08-02T17:55:53.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.385+0000 7f9034ff9640 1 -- 172.21.15.69:0/1506322223 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f903c0d10f0 con 0x5646aa29a7b0 2024-08-02T17:55:53.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.386+0000 7f9034ff9640 1 -- 172.21.15.69:0/1506322223 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f903c0b9050 con 0x5646aa29a7b0 2024-08-02T17:55:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.387+0000 7f9056200740 10 client.27257 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:53.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.387+0000 7f9056200740 1 -- 172.21.15.69:0/1506322223 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646aa2b42b0 con 0x5646aa29a7b0 2024-08-02T17:55:53.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.389+0000 7f9034ff9640 1 -- 172.21.15.69:0/1506322223 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9048005500 con 0x5646aa29a7b0 2024-08-02T17:55:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.390+0000 7f9034ff9640 1 -- 172.21.15.69:0/1506322223 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f903c09a8f0 con 0x5646aa29a7b0 2024-08-02T17:55:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.390+0000 7f9056200740 10 client.27257 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.390+0000 7f9056200740 10 client.27257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.390+0000 7f9056200740 20 client.27257 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.390+0000 7f9056200740 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646aa2b5c00 0x5646aa2b8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.391+0000 7f9056200740 4 client.27257 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:53.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.391+0000 7f9056200740 1 -- 172.21.15.69:0/1506322223 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646aa20c0e0 con 0x5646aa2b5c00 2024-08-02T17:55:53.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.393+0000 7f905096b640 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646aa2b5c00 0x5646aa2b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.401+0000 7f905096b640 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646aa2b5c00 0x5646aa2b8080 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594197635d6f0f6a server_cookie=ea804ad46f64219a in_seq=0 out_seq=0 2024-08-02T17:55:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.401+0000 7f9034ff9640 10 client.27257 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.402+0000 7f9034ff9640 1 -- 172.21.15.69:0/1506322223 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5646aa20c0e0 con 0x5646aa2b5c00 2024-08-02T17:55:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.402+0000 7f9034ff9640 10 client.27257 handle_command_reply: tid=0 2024-08-02T17:55:53.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.402+0000 7f9056200740 1 client.27257 shutdown 2024-08-02T17:55:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.406+0000 7f9056200740 1 -- 172.21.15.69:0/1506322223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29a7b0 msgr2=0x5646aa2a28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.407+0000 7f9056200740 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29a7b0 0x5646aa2a28e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f903c0cc5c0 tx=0x7f903c09af90 comp rx=0 tx=0).stop 2024-08-02T17:55:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.407+0000 7f9056200740 1 -- 172.21.15.69:0/1506322223 shutdown_connections 2024-08-02T17:55:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.407+0000 7f9056200740 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa2a5970 0x5646aa2a3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.407+0000 7f9056200740 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646aa2b5c00 0x5646aa2b8080 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.407+0000 7f9056200740 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa29b0d0 0x5646aa2a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.407+0000 7f9056200740 1 --2- 172.21.15.69:0/1506322223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29a7b0 0x5646aa2a28e0 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f903c0cc5c0 tx=0x7f903c09af90 comp rx=0 tx=0).stop 2024-08-02T17:55:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.407+0000 7f9056200740 1 -- 172.21.15.69:0/1506322223 >> 172.21.15.69:0/1506322223 conn(0x5646aa296d20 msgr2=0x5646aa29ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.407+0000 7f9034ff9640 0 client.27257 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:53.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.409+0000 7f9056200740 1 -- 172.21.15.69:0/1506322223 shutdown_connections 2024-08-02T17:55:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.413+0000 7f9056200740 1 -- 172.21.15.69:0/1506322223 wait complete. 2024-08-02T17:55:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.413+0000 7f9056200740 20 client.27257 trim_cache size 0 max 16384 2024-08-02T17:55:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.449+0000 7f9056200740 1 Processor -- start 2024-08-02T17:55:53.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.449+0000 7f9056200740 1 -- start start 2024-08-02T17:55:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.450+0000 7f9056200740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa295d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.450+0000 7f9056200740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.450+0000 7f9056200740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29e280 0x5646aa2bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.450+0000 7f905096b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.450+0000 7f905096b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39520/0 (socket says 172.21.15.69:39520) 2024-08-02T17:55:53.452 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.450+0000 7f905096b640 1 -- 172.21.15.69:0/437365302 learned_addr learned my addr 172.21.15.69:0/437365302 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.450+0000 7f9056200740 1 -- 172.21.15.69:0/437365302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1b7c40 con 0x5646aa29b0d0 2024-08-02T17:55:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.450+0000 7f905196d640 1 --2- 172.21.15.69:0/437365302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29e280 0x5646aa2bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.451+0000 7f9056200740 1 -- 172.21.15.69:0/437365302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1b5510 con 0x5646aa29e280 2024-08-02T17:55:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.451+0000 7f9056200740 1 -- 172.21.15.69:0/437365302 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1b3410 con 0x5646aa2a5970 2024-08-02T17:55:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.452+0000 7f905196d640 1 -- 172.21.15.69:0/437365302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 msgr2=0x5646aa2bd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.453 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.452+0000 7f905196d640 1 --2- 172.21.15.69:0/437365302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2bd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.452+0000 7f905196d640 1 -- 172.21.15.69:0/437365302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 msgr2=0x5646aa295d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.452+0000 7f905196d640 1 --2- 172.21.15.69:0/437365302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa295d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.452+0000 7f905196d640 1 -- 172.21.15.69:0/437365302 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646aa296b50 con 0x5646aa29e280 2024-08-02T17:55:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.453+0000 7f905196d640 1 --2- 172.21.15.69:0/437365302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29e280 0x5646aa2bded0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9044006010 tx=0x7f904403b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56c3c4484209360e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.454+0000 7f9036ffd640 1 -- 172.21.15.69:0/437365302 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9044054070 con 0x5646aa29e280 2024-08-02T17:55:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.454+0000 7f9036ffd640 1 -- 172.21.15.69:0/437365302 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044003d40 con 0x5646aa29e280 2024-08-02T17:55:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.455+0000 7f9056200740 1 -- 172.21.15.69:0/437365302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29e280 msgr2=0x5646aa2bded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.456+0000 7f9056200740 1 --2- 172.21.15.69:0/437365302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29e280 0x5646aa2bded0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9044006010 tx=0x7f904403b050 comp rx=0 tx=0).stop 2024-08-02T17:55:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.458+0000 7f9056200740 1 -- 172.21.15.69:0/437365302 shutdown_connections 2024-08-02T17:55:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.458+0000 7f9056200740 1 --2- 172.21.15.69:0/437365302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa29e280 0x5646aa2bded0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.458+0000 7f9056200740 1 --2- 172.21.15.69:0/437365302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.458+0000 7f9056200740 1 --2- 172.21.15.69:0/437365302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa295d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.458+0000 7f9056200740 1 -- 172.21.15.69:0/437365302 >> 172.21.15.69:0/437365302 conn(0x5646aa296d20 msgr2=0x5646aa295660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.464+0000 7f9056200740 1 -- 172.21.15.69:0/437365302 shutdown_connections 2024-08-02T17:55:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.466+0000 7f9056200740 1 -- 172.21.15.69:0/437365302 wait complete. 2024-08-02T17:55:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.466+0000 7f9056200740 1 -- start start 2024-08-02T17:55:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.466+0000 7f9056200740 1 Processor -- start 2024-08-02T17:55:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.467+0000 7f9056200740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa2add80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.467+0000 7f9056200740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.467+0000 7f9056200740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a95e0 0x5646aa2a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.467+0000 7f9056200740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1b39d0 con 0x5646aa29b0d0 2024-08-02T17:55:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.467+0000 7f9056200740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1b4e40 con 0x5646aa2a95e0 2024-08-02T17:55:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.467+0000 7f9056200740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f904c1b40a0 con 0x5646aa2a5970 2024-08-02T17:55:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.467+0000 7f9056200740 10 client.0 _ll_register_callbacks cb 0x7f903c010890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.468+0000 7f905196d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a95e0 0x5646aa2a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.468+0000 7f905096b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.468+0000 7f905096b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2a90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39530/0 (socket says 172.21.15.69:39530) 2024-08-02T17:55:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.468+0000 7f905096b640 1 -- 172.21.15.69:0/489063139 learned_addr learned my addr 172.21.15.69:0/489063139 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.468+0000 7f905116c640 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa2add80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.469+0000 7f905096b640 1 -- 172.21.15.69:0/489063139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a95e0 msgr2=0x5646aa2a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.469+0000 7f905096b640 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a95e0 0x5646aa2a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.469+0000 7f905096b640 1 -- 172.21.15.69:0/489063139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 msgr2=0x5646aa2add80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.469+0000 7f905096b640 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa2add80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.469+0000 7f905096b640 1 -- 172.21.15.69:0/489063139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646aa2ae5d0 con 0x5646aa2a5970 2024-08-02T17:55:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.470+0000 7f905096b640 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2a90a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5646aa2a2150 tx=0x7f9048000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f0673df45555a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.471+0000 7f9034ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:55:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.471+0000 7f9034ff9640 1 -- 172.21.15.69:0/489063139 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9048032060 con 0x5646aa2a5970 2024-08-02T17:55:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.471+0000 7f9056200740 1 -- 172.21.15.69:0/489063139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9044003d40 con 0x5646aa2a5970 2024-08-02T17:55:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.475+0000 7f9034ff9640 1 -- 172.21.15.69:0/489063139 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f9048004030 con 0x5646aa2a5970 2024-08-02T17:55:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.475+0000 7f9034ff9640 1 -- 172.21.15.69:0/489063139 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9048039050 con 0x5646aa2a5970 2024-08-02T17:55:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.475+0000 7f9056200740 10 client.27082 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.475+0000 7f9056200740 1 -- 172.21.15.69:0/489063139 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646aa320a50 con 0x5646aa2a5970 2024-08-02T17:55:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.476+0000 7f9034ff9640 1 -- 172.21.15.69:0/489063139 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9044003d40 con 0x5646aa2a5970 2024-08-02T17:55:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.476+0000 7f9034ff9640 1 -- 172.21.15.69:0/489063139 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9048003700 con 0x5646aa2a5970 2024-08-02T17:55:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.476+0000 7f9056200740 10 client.27082 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.476+0000 7f9056200740 10 client.27082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.476+0000 7f9056200740 20 client.27082 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.476+0000 7f9056200740 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646aa321070 0x5646aa323490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.476+0000 7f9056200740 4 client.27082 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:55:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.476+0000 7f9056200740 1 -- 172.21.15.69:0/489063139 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646aa206430 con 0x5646aa321070 2024-08-02T17:55:53.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.477+0000 7f905116c640 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646aa321070 0x5646aa323490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.478+0000 7f905116c640 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646aa321070 0x5646aa323490 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e342b43bd38646f2 server_cookie=ac10a2fec03a16c0 in_seq=0 out_seq=0 2024-08-02T17:55:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.479+0000 7f9034ff9640 10 client.27082 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:53 smithi083 ceph-mon[37031]: pgmap v292: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.1 MiB/s wr, 766 op/s 2024-08-02T17:55:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.609+0000 7f9034ff9640 1 -- 172.21.15.69:0/489063139 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646aa206430 con 0x5646aa321070 2024-08-02T17:55:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.609+0000 7f9034ff9640 10 client.27082 handle_command_reply: tid=0 2024-08-02T17:55:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.609+0000 7f9056200740 1 client.27082 shutdown 2024-08-02T17:55:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 -- 172.21.15.69:0/489063139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 msgr2=0x5646aa2a90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2a90a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5646aa2a2150 tx=0x7f9048000bc0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 -- 172.21.15.69:0/489063139 shutdown_connections 2024-08-02T17:55:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646aa321070 0x5646aa323490 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646aa2a95e0 0x5646aa2a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5646aa2a5970 0x5646aa2a90a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 --2- 172.21.15.69:0/489063139 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646aa29b0d0 0x5646aa2add80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 -- 172.21.15.69:0/489063139 >> 172.21.15.69:0/489063139 conn(0x5646aa296d20 msgr2=0x5646aa2a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9034ff9640 0 client.27082 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 -- 172.21.15.69:0/489063139 shutdown_connections 2024-08-02T17:55:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.610+0000 7f9056200740 1 -- 172.21.15.69:0/489063139 wait complete. 2024-08-02T17:55:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.611+0000 7f9056200740 20 client.27082 trim_cache size 0 max 16384 2024-08-02T17:55:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:55:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:55:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:55:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 -- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f90380a2e80 msgr2=0x7f90380a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f90380a2e80 0x7f90380a5340 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9044005fe0 tx=0x7f904402ffa0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 -- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c1007a0 msgr2=0x7f904c19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c1007a0 0x7f904c19e710 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f903c00b450 tx=0x7f903c00b910 comp rx=0 tx=0).stop 2024-08-02T17:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 -- 172.21.15.69:0/157099921 shutdown_connections 2024-08-02T17:55:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f90380a2e80 0x7f90380a5340 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f904c105bd0 0x7f904c1a2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f904c1010c0 0x7f904c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 --2- 172.21.15.69:0/157099921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f904c1007a0 0x7f904c19e710 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 -- 172.21.15.69:0/157099921 >> 172.21.15.69:0/157099921 conn(0x7f904c0776d0 msgr2=0x7f904c103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 -- 172.21.15.69:0/157099921 shutdown_connections 2024-08-02T17:55:53.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:55:53.612+0000 7f90533d0640 1 -- 172.21.15.69:0/157099921 wait complete. 2024-08-02T17:55:53.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.669+0000 7f3036af8640 1 -- 172.21.15.69:0/883295068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 msgr2=0x7f3030109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.669+0000 7f3036af8640 1 --2- 172.21.15.69:0/883295068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 0x7f3030109560 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f30240093a0 tx=0x7f302402e990 comp rx=0 tx=0).stop 2024-08-02T17:55:53.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.674+0000 7f3036af8640 1 -- 172.21.15.69:0/883295068 shutdown_connections 2024-08-02T17:55:53.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.674+0000 7f3036af8640 1 --2- 172.21.15.69:0/883295068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f303010e7b0 0x7f3030115100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.674+0000 7f3036af8640 1 --2- 172.21.15.69:0/883295068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3030109b30 0x7f303010e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.674+0000 7f3036af8640 1 --2- 172.21.15.69:0/883295068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 0x7f3030109560 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.674+0000 7f3036af8640 1 -- 172.21.15.69:0/883295068 >> 172.21.15.69:0/883295068 conn(0x7f3030072f40 msgr2=0x7f3030073350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.675+0000 7f3036af8640 1 -- 172.21.15.69:0/883295068 shutdown_connections 2024-08-02T17:55:53.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.675+0000 7f3036af8640 1 -- 172.21.15.69:0/883295068 wait complete. 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.676+0000 7f3036af8640 1 Processor -- start 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.678+0000 7f3036af8640 1 -- start start 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3036af8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 0x7f303019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3036af8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3030109b30 0x7f303019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3034894640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 0x7f303019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3034894640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 0x7f303019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39562/0 (socket says 172.21.15.69:39562) 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3036af8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303010e7b0 0x7f30301a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3036af8640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3030118150 con 0x7f3030109b30 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3036af8640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3030117fd0 con 0x7f303010e7b0 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3036af8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30301182d0 con 0x7f3030109180 2024-08-02T17:55:53.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.679+0000 7f3034894640 1 -- 172.21.15.69:0/2014797381 learned_addr learned my addr 172.21.15.69:0/2014797381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.680+0000 7f302ffff640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3030109b30 0x7f303019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.680+0000 7f3034894640 1 -- 172.21.15.69:0/2014797381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303010e7b0 msgr2=0x7f30301a2f80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.680+0000 7f3034894640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303010e7b0 0x7f30301a2f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.680+0000 7f3034894640 1 -- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3030109b30 msgr2=0x7f303019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.680+0000 7f3034894640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3030109b30 0x7f303019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.680+0000 7f3034894640 1 -- 172.21.15.69:0/2014797381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30301a3660 con 0x7f3030109180 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.681+0000 7f3034894640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 0x7f303019e6b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3024005ab0 tx=0x7f3024005010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.681+0000 7f302dffb640 1 -- 172.21.15.69:0/2014797381 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024038040 con 0x7f3030109180 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.681+0000 7f302dffb640 1 -- 172.21.15.69:0/2014797381 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3024043070 con 0x7f3030109180 2024-08-02T17:55:53.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.681+0000 7f302dffb640 1 -- 172.21.15.69:0/2014797381 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302403e450 con 0x7f3030109180 2024-08-02T17:55:53.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.683+0000 7f3036af8640 1 -- 172.21.15.69:0/2014797381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30301a38f0 con 0x7f3030109180 2024-08-02T17:55:53.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.684+0000 7f3036af8640 1 -- 172.21.15.69:0/2014797381 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30301a3d50 con 0x7f3030109180 2024-08-02T17:55:53.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.687+0000 7f302dffb640 1 -- 172.21.15.69:0/2014797381 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3024005390 con 0x7f3030109180 2024-08-02T17:55:53.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.689+0000 7f302dffb640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30180a2f00 0x7f30180a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.689+0000 7f302ffff640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30180a2f00 0x7f30180a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.689+0000 7f3038ef6740 1 Processor -- start 2024-08-02T17:55:53.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3038ef6740 1 -- start start 2024-08-02T17:55:53.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311351d0 0x55df311355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df3113fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df311427a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3038ef6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df3109aa40 con 0x55df31135af0 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3038ef6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df310329d0 con 0x55df311351d0 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3038ef6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df31096a00 con 0x55df311403b0 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f302ffff640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30180a2f00 0x7f30180a53c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f303006dd60 tx=0x7f3020005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f302dffb640 1 -- 172.21.15.69:0/2014797381 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f30240ef6d0 con 0x7f3030109180 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3035095640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df311427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f3034894640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311351d0 0x55df311355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f302ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df3113fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.690+0000 7f302ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df3113fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46016/0 (socket says 172.21.15.69:46016) 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.691+0000 7f302ffff640 1 -- 172.21.15.69:0/1331538664 learned_addr learned my addr 172.21.15.69:0/1331538664 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.691+0000 7f3035095640 1 -- 172.21.15.69:0/1331538664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311351d0 msgr2=0x55df311355b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.691+0000 7f3035095640 1 --2- 172.21.15.69:0/1331538664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311351d0 0x55df311355b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.691+0000 7f3035095640 1 -- 172.21.15.69:0/1331538664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 msgr2=0x55df3113fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.691+0000 7f3035095640 1 --2- 172.21.15.69:0/1331538664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df3113fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.691+0000 7f3035095640 1 -- 172.21.15.69:0/1331538664 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df31039650 con 0x55df311403b0 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.692+0000 7f3035095640 1 --2- 172.21.15.69:0/1331538664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df311427a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3028009bb0 tx=0x7f302800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ffac6f7724f82ae server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.693+0000 7f3016ffd640 1 -- 172.21.15.69:0/1331538664 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028016070 con 0x55df311403b0 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.693+0000 7f3016ffd640 1 -- 172.21.15.69:0/1331538664 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3028003970 con 0x55df311403b0 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.693+0000 7f3016ffd640 1 -- 172.21.15.69:0/1331538664 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30280045c0 con 0x55df311403b0 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.694+0000 7f3038ef6740 1 -- 172.21.15.69:0/1331538664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 msgr2=0x55df311427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.694+0000 7f3038ef6740 1 --2- 172.21.15.69:0/1331538664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df311427a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3028009bb0 tx=0x7f302800e6c0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.694+0000 7f3038ef6740 1 -- 172.21.15.69:0/1331538664 shutdown_connections 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.694+0000 7f3038ef6740 1 --2- 172.21.15.69:0/1331538664 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df3113fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.694+0000 7f3038ef6740 1 --2- 172.21.15.69:0/1331538664 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df311427a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.694+0000 7f3038ef6740 1 --2- 172.21.15.69:0/1331538664 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311351d0 0x55df311355b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.694+0000 7f3038ef6740 1 -- 172.21.15.69:0/1331538664 >> 172.21.15.69:0/1331538664 conn(0x55df31131760 msgr2=0x55df31131b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.695+0000 7f3038ef6740 1 -- 172.21.15.69:0/1331538664 shutdown_connections 2024-08-02T17:55:53.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.695+0000 7f3038ef6740 1 -- 172.21.15.69:0/1331538664 wait complete. 2024-08-02T17:55:53.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.695+0000 7f3038ef6740 1 -- start start 2024-08-02T17:55:53.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.695+0000 7f3038ef6740 1 Processor -- start 2024-08-02T17:55:53.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df311351d0 0x55df3113d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df31135af0 0x55df3113d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df3113dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3034894640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df311351d0 0x55df3113d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3034894640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df311351d0 0x55df3113d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46032/0 (socket says 172.21.15.69:46032) 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3034894640 1 -- 172.21.15.69:0/990497093 learned_addr learned my addr 172.21.15.69:0/990497093 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f302ffff640 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df31135af0 0x55df3113d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55df31142f80 con 0x55df311351d0 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55df31055b20 con 0x55df31135af0 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.696+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55df31143100 con 0x55df311403b0 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f3035095640 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df3113dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f3038ef6740 10 client.0 _ll_register_callbacks cb 0x55df31053bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:53.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f3035095640 1 -- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df31135af0 msgr2=0x55df3113d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f3035095640 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df31135af0 0x55df3113d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f3035095640 1 -- 172.21.15.69:0/990497093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df311351d0 msgr2=0x55df3113d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f3035095640 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df311351d0 0x55df3113d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f3035095640 1 -- 172.21.15.69:0/990497093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df30e0c3c0 con 0x55df311403b0 2024-08-02T17:55:53.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f302ffff640 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df31135af0 0x55df3113d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:53.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f3035095640 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df3113dd70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3028004060 tx=0x7f3028009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceb80de17051cc37 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.697+0000 7f300ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:55:53.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.698+0000 7f300ffff640 1 -- 172.21.15.69:0/990497093 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028016070 con 0x55df311403b0 2024-08-02T17:55:53.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.698+0000 7f300ffff640 1 -- 172.21.15.69:0/990497093 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f3028004df0 con 0x55df311403b0 2024-08-02T17:55:53.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.698+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3028003970 con 0x55df311403b0 2024-08-02T17:55:53.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.699+0000 7f300ffff640 1 -- 172.21.15.69:0/990497093 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3028003e50 con 0x55df311403b0 2024-08-02T17:55:53.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.699+0000 7f3038ef6740 10 client.27094 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:53.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.699+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55df3114ebc0 con 0x55df311403b0 2024-08-02T17:55:53.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.700+0000 7f300ffff640 1 -- 172.21.15.69:0/990497093 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3028003970 con 0x55df311403b0 2024-08-02T17:55:53.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.700+0000 7f300ffff640 1 -- 172.21.15.69:0/990497093 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3028021070 con 0x55df311403b0 2024-08-02T17:55:53.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.700+0000 7f3038ef6740 10 client.27094 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:53.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.700+0000 7f3038ef6740 10 client.27094 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:55:53.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.700+0000 7f3038ef6740 20 client.27094 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:53.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.700+0000 7f3038ef6740 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55df31150080 0x55df31152470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.701+0000 7f3038ef6740 4 client.27094 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:53.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.701+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df310a7fb0 con 0x55df31150080 2024-08-02T17:55:53.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.701+0000 7f3034894640 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55df31150080 0x55df31152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.701+0000 7f3034894640 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55df31150080 0x55df31152470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43020119e525a933 server_cookie=c7959b6c3fca8e6e in_seq=0 out_seq=0 2024-08-02T17:55:53.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.701+0000 7f300ffff640 10 client.27094 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:55:53.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.702+0000 7f300ffff640 1 -- 172.21.15.69:0/990497093 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df310a7fb0 con 0x55df31150080 2024-08-02T17:55:53.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.702+0000 7f300ffff640 10 client.27094 handle_command_reply: tid=0 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.702+0000 7f3038ef6740 1 client.27094 shutdown 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.702+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 msgr2=0x55df3113dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.702+0000 7f3038ef6740 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df3113dd70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3028004060 tx=0x7f3028009760 comp rx=0 tx=0).stop 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.702+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 shutdown_connections 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.702+0000 7f3038ef6740 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55df31150080 0x55df31152470 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.702+0000 7f300ffff640 0 client.27094 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.703+0000 7f3038ef6740 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311403b0 0x55df3113dd70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.703+0000 7f3038ef6740 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df31135af0 0x55df3113d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.703+0000 7f3038ef6740 1 --2- 172.21.15.69:0/990497093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df311351d0 0x55df3113d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.703+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 >> 172.21.15.69:0/990497093 conn(0x55df31131760 msgr2=0x55df31137490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.703+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 shutdown_connections 2024-08-02T17:55:53.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.703+0000 7f3038ef6740 1 -- 172.21.15.69:0/990497093 wait complete. 2024-08-02T17:55:53.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.703+0000 7f3038ef6740 20 client.27094 trim_cache size 0 max 16384 2024-08-02T17:55:53.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.724+0000 7f3038ef6740 1 Processor -- start 2024-08-02T17:55:53.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.724+0000 7f3038ef6740 1 -- start start 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.724+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df3113dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31137380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 0x55df31157c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3034894640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df3113dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3034894640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df3113dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39580/0 (socket says 172.21.15.69:39580) 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3034894640 1 -- 172.21.15.69:0/1670420698 learned_addr learned my addr 172.21.15.69:0/1670420698 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f302ffff640 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31137380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3038ef6740 1 -- 172.21.15.69:0/1670420698 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30301c0490 con 0x55df31135af0 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3038ef6740 1 -- 172.21.15.69:0/1670420698 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30301bdd60 con 0x55df311403b0 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3035095640 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 0x55df31157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3038ef6740 1 -- 172.21.15.69:0/1670420698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30301bbbb0 con 0x55df311351d0 2024-08-02T17:55:53.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3034894640 1 -- 172.21.15.69:0/1670420698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 msgr2=0x55df31157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3034894640 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 0x55df31157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3034894640 1 -- 172.21.15.69:0/1670420698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 msgr2=0x55df31137380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3034894640 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31137380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.725+0000 7f3034894640 1 -- 172.21.15.69:0/1670420698 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df31135920 con 0x55df311351d0 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.726+0000 7f302ffff640 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31137380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.726+0000 7f3034894640 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df3113dbc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f30240b8200 tx=0x7f3024009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac5e7d1a2d573583 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.726+0000 7f30167fc640 1 -- 172.21.15.69:0/1670420698 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240d9070 con 0x55df311351d0 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.726+0000 7f30167fc640 1 -- 172.21.15.69:0/1670420698 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30240bfbe0 con 0x55df311351d0 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.726+0000 7f30167fc640 1 -- 172.21.15.69:0/1670420698 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024030040 con 0x55df311351d0 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 -- 172.21.15.69:0/1670420698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 msgr2=0x55df3113dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df3113dbc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f30240b8200 tx=0x7f3024009f10 comp rx=0 tx=0).stop 2024-08-02T17:55:53.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 -- 172.21.15.69:0/1670420698 shutdown_connections 2024-08-02T17:55:53.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31137380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 0x55df31157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 --2- 172.21.15.69:0/1670420698 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df3113dbc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 -- 172.21.15.69:0/1670420698 >> 172.21.15.69:0/1670420698 conn(0x55df31131760 msgr2=0x55df31136c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 -- 172.21.15.69:0/1670420698 shutdown_connections 2024-08-02T17:55:53.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 -- 172.21.15.69:0/1670420698 wait complete. 2024-08-02T17:55:53.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 -- start start 2024-08-02T17:55:53.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.727+0000 7f3038ef6740 1 Processor -- start 2024-08-02T17:55:53.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df31148530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3038ef6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31148a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3034894640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df31148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f302ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3034894640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df31148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39582/0 (socket says 172.21.15.69:39582) 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f302ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46048/0 (socket says 172.21.15.69:46048) 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3034894640 1 -- 172.21.15.69:0/290055865 learned_addr learned my addr 172.21.15.69:0/290055865 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3038ef6740 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 0x55df31139b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30301bc220 con 0x55df31135af0 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30301bd690 con 0x55df311403b0 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30301bc8f0 con 0x55df311351d0 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.728+0000 7f3035095640 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 0x55df31139b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f302ffff640 1 -- 172.21.15.69:0/290055865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 msgr2=0x55df31148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f302ffff640 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df31148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f302ffff640 1 -- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 msgr2=0x55df31139b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f302ffff640 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 0x55df31139b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f302ffff640 1 -- 172.21.15.69:0/290055865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3113a2b0 con 0x55df31135af0 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f302ffff640 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31148a70 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x55df3113e930 tx=0x7f3020004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82482d76985e976 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f300ffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f3038ef6740 10 client.0 _ll_register_callbacks cb 0x7f302000d830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.729+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30240bfbe0 con 0x55df31135af0 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f300ffff640 1 -- 172.21.15.69:0/290055865 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020036050 con 0x55df31135af0 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f300ffff640 1 -- 172.21.15.69:0/290055865 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7f3020018c80 con 0x55df31135af0 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f300ffff640 1 -- 172.21.15.69:0/290055865 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f302002c070 con 0x55df31135af0 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f3038ef6740 10 client.17781 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:53.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55df311ba530 con 0x55df31135af0 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f300ffff640 1 -- 172.21.15.69:0/290055865 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3020028430 con 0x55df31135af0 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f3038ef6740 10 client.17781 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f3038ef6740 10 client.17781 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f3038ef6740 20 client.17781 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f3038ef6740 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55df31150080 0x55df311bcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f3038ef6740 4 client.17781 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.730+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df310a24c0 con 0x55df31150080 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.731+0000 7f3034894640 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55df31150080 0x55df311bcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.731+0000 7f3034894640 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55df31150080 0x55df311bcbb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d001c54d087fe8f server_cookie=86d648c2212ea3ee in_seq=0 out_seq=0 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.731+0000 7f300ffff640 10 client.17781 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.731+0000 7f300ffff640 1 -- 172.21.15.69:0/290055865 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df310a24c0 con 0x55df31150080 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.731+0000 7f300ffff640 10 client.17781 handle_command_reply: tid=0 2024-08-02T17:55:53.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 client.17781 shutdown 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 msgr2=0x55df31148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31148a70 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x55df3113e930 tx=0x7f3020004730 comp rx=0 tx=0).stop 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 shutdown_connections 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55df31150080 0x55df311bcbb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55df31135af0 0x55df31148a70 secure :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0x55df3113e930 tx=0x7f3020004730 comp rx=0 tx=0).stop 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55df311403b0 0x55df31139b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 --2- 172.21.15.69:0/290055865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55df311351d0 0x55df31148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 >> 172.21.15.69:0/290055865 conn(0x55df31131760 msgr2=0x55df311308c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f300ffff640 0 client.17781 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.732+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 shutdown_connections 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.733+0000 7f3038ef6740 1 -- 172.21.15.69:0/290055865 wait complete. 2024-08-02T17:55:53.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.733+0000 7f3038ef6740 20 client.17781 trim_cache size 0 max 16384 2024-08-02T17:55:53.740 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:55:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 -- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30180a2f00 msgr2=0x7f30180a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30180a2f00 0x7f30180a53c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f303006dd60 tx=0x7f3020005f90 comp rx=0 tx=0).stop 2024-08-02T17:55:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 -- 172.21.15.69:0/2014797381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 msgr2=0x7f303019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 0x7f303019e6b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3024005ab0 tx=0x7f3024005010 comp rx=0 tx=0).stop 2024-08-02T17:55:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 -- 172.21.15.69:0/2014797381 shutdown_connections 2024-08-02T17:55:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3030109b30 0x7f303019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30180a2f00 0x7f30180a53c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f303010e7b0 0x7f30301a2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 --2- 172.21.15.69:0/2014797381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3030109180 0x7f303019e6b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:53.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 -- 172.21.15.69:0/2014797381 >> 172.21.15.69:0/2014797381 conn(0x7f3030072f40 msgr2=0x7f303010c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:53.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 -- 172.21.15.69:0/2014797381 shutdown_connections 2024-08-02T17:55:53.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:53.734+0000 7f3036af8640 1 -- 172.21.15.69:0/2014797381 wait complete. 2024-08-02T17:55:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 525.0 s, 609.8 tps, lat 147.044 ms stddev 177.853 2024-08-02T17:55:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:55 smithi077 ceph-mon[37108]: pgmap v293: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 939 op/s 2024-08-02T17:55:55.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:55 smithi069 ceph-mon[32746]: pgmap v293: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 939 op/s 2024-08-02T17:55:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:55 smithi083 ceph-mon[37031]: pgmap v293: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 8.9 MiB/s wr, 939 op/s 2024-08-02T17:55:56.238 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:55:56.751 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:55:57.199 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:57 smithi069 ceph-mon[32746]: pgmap v294: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.1 MiB/s wr, 660 op/s 2024-08-02T17:55:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.338+0000 7fdffec70640 1 -- 172.21.15.69:0/456477058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff81078c0 msgr2=0x7fdff8107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.339+0000 7fdffec70640 1 --2- 172.21.15.69:0/456477058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff81078c0 0x7fdff8107ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009cd0 tx=0x7fdff002a930 comp rx=0 tx=0).stop 2024-08-02T17:55:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.341+0000 7fdffec70640 1 -- 172.21.15.69:0/456477058 shutdown_connections 2024-08-02T17:55:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.341+0000 7fdffec70640 1 --2- 172.21.15.69:0/456477058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdff81018c0 0x7fdff8101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.341+0000 7fdffec70640 1 --2- 172.21.15.69:0/456477058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdff8102260 0x7fdff810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.341+0000 7fdffec70640 1 --2- 172.21.15.69:0/456477058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff81078c0 0x7fdff8107ca0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.341+0000 7fdffec70640 1 -- 172.21.15.69:0/456477058 >> 172.21.15.69:0/456477058 conn(0x7fdff80fd620 msgr2=0x7fdff80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.343+0000 7fdffec70640 1 -- 172.21.15.69:0/456477058 shutdown_connections 2024-08-02T17:55:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.343+0000 7fdffec70640 1 -- 172.21.15.69:0/456477058 wait complete. 2024-08-02T17:55:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.344+0000 7fdffec70640 1 Processor -- start 2024-08-02T17:55:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.344+0000 7fdffec70640 1 -- start start 2024-08-02T17:55:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.344+0000 7fdffec70640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdff81018c0 0x7fdff819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdffec70640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff8102260 0x7fdff819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdffec70640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdff81078c0 0x7fdff819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdff7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff8102260 0x7fdff819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdff7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff8102260 0x7fdff819a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35348/0 (socket says 172.21.15.69:35348) 2024-08-02T17:55:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdff7fff640 1 -- 172.21.15.69:0/2090618779 learned_addr learned my addr 172.21.15.69:0/2090618779 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdffd20d640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdff81078c0 0x7fdff819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdff8113fa0 con 0x7fdff81018c0 2024-08-02T17:55:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdff8113e20 con 0x7fdff81078c0 2024-08-02T17:55:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.345+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdff8114120 con 0x7fdff8102260 2024-08-02T17:55:57.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:57 smithi077 ceph-mon[37108]: pgmap v294: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.1 MiB/s wr, 660 op/s 2024-08-02T17:55:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.346+0000 7fdffd20d640 1 -- 172.21.15.69:0/2090618779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff8102260 msgr2=0x7fdff819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.346+0000 7fdffd20d640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff8102260 0x7fdff819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.346+0000 7fdffd20d640 1 -- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdff81018c0 msgr2=0x7fdff819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.346+0000 7fdffd20d640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdff81018c0 0x7fdff819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.346+0000 7fdffd20d640 1 -- 172.21.15.69:0/2090618779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff819f1f0 con 0x7fdff81078c0 2024-08-02T17:55:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.347+0000 7fdffd20d640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdff81078c0 0x7fdff819eb10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b3e0 tx=0x7fdfe800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.348+0000 7fdff5ffb640 1 -- 172.21.15.69:0/2090618779 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8015020 con 0x7fdff81078c0 2024-08-02T17:55:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.348+0000 7fdff5ffb640 1 -- 172.21.15.69:0/2090618779 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe8011d10 con 0x7fdff81078c0 2024-08-02T17:55:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.348+0000 7fdff5ffb640 1 -- 172.21.15.69:0/2090618779 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800f2e0 con 0x7fdff81078c0 2024-08-02T17:55:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.348+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff819f4e0 con 0x7fdff81078c0 2024-08-02T17:55:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.349+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff810efd0 con 0x7fdff81078c0 2024-08-02T17:55:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.352+0000 7fdff5ffb640 1 -- 172.21.15.69:0/2090618779 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdfe8018020 con 0x7fdff81078c0 2024-08-02T17:55:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.353+0000 7fdff5ffb640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfe00a2f00 0x7fdfe00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.354+0000 7fdff5ffb640 1 -- 172.21.15.69:0/2090618779 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fdfe80cff30 con 0x7fdff81078c0 2024-08-02T17:55:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.354+0000 7fdffca0c640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfe00a2f00 0x7fdfe00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.355+0000 7fdffca0c640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfe00a2f00 0x7fdfe00a53c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdff002af20 tx=0x7fdff002ae30 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.356+0000 7fe00106e740 1 Processor -- start 2024-08-02T17:55:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.356+0000 7fe00106e740 1 -- start start 2024-08-02T17:55:57.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fe00106e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 0x55fca79db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fe00106e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79db910 0x55fca79e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fe00106e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fe00106e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fca7940a40 con 0x55fca79daff0 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fe00106e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fca78d89d0 con 0x55fca79e61d0 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fe00106e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fca793ca00 con 0x55fca79db910 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fdffd20d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fdffd20d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:47580/0 (socket says 172.21.15.69:47580) 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fdffd20d640 1 -- 172.21.15.69:0/3312680645 learned_addr learned my addr 172.21.15.69:0/3312680645 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.357+0000 7fdffca0c640 1 --2- 172.21.15.69:0/3312680645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 0x55fca79db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.358+0000 7fdffd20d640 1 -- 172.21.15.69:0/3312680645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79db910 msgr2=0x55fca79e5c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.358+0000 7fdffd20d640 1 --2- 172.21.15.69:0/3312680645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79db910 0x55fca79e5c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.358+0000 7fdffd20d640 1 -- 172.21.15.69:0/3312680645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 msgr2=0x55fca79db3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.358+0000 7fdffd20d640 1 --2- 172.21.15.69:0/3312680645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 0x55fca79db3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.358+0000 7fdffd20d640 1 -- 172.21.15.69:0/3312680645 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca78df650 con 0x55fca79e61d0 2024-08-02T17:55:57.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.358+0000 7fdffd20d640 1 --2- 172.21.15.69:0/3312680645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e85c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800bda0 tx=0x7fdfe809c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de49536dc701047 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.358+0000 7fdfdeffd640 1 -- 172.21.15.69:0/3312680645 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80b5070 con 0x55fca79e61d0 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.359+0000 7fdfdeffd640 1 -- 172.21.15.69:0/3312680645 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe80a4d80 con 0x55fca79e61d0 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.359+0000 7fdfdeffd640 1 -- 172.21.15.69:0/3312680645 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe809b560 con 0x55fca79e61d0 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.360+0000 7fe00106e740 1 -- 172.21.15.69:0/3312680645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 msgr2=0x55fca79e85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.360+0000 7fe00106e740 1 --2- 172.21.15.69:0/3312680645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e85c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800bda0 tx=0x7fdfe809c810 comp rx=0 tx=0).stop 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.360+0000 7fe00106e740 1 -- 172.21.15.69:0/3312680645 shutdown_connections 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.360+0000 7fe00106e740 1 --2- 172.21.15.69:0/3312680645 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e85c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.360+0000 7fe00106e740 1 --2- 172.21.15.69:0/3312680645 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79db910 0x55fca79e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.360+0000 7fe00106e740 1 --2- 172.21.15.69:0/3312680645 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 0x55fca79db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.360+0000 7fe00106e740 1 -- 172.21.15.69:0/3312680645 >> 172.21.15.69:0/3312680645 conn(0x55fca79d7580 msgr2=0x55fca79d7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:57.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.365+0000 7fe00106e740 1 -- 172.21.15.69:0/3312680645 shutdown_connections 2024-08-02T17:55:57.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.367+0000 7fe00106e740 1 -- 172.21.15.69:0/3312680645 wait complete. 2024-08-02T17:55:57.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.367+0000 7fe00106e740 1 -- start start 2024-08-02T17:55:57.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.368+0000 7fe00106e740 1 Processor -- start 2024-08-02T17:55:57.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.368+0000 7fe00106e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 0x55fca79e3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.369+0000 7fe00106e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.369+0000 7fe00106e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79f2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.369+0000 7fe00106e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fca79e8da0 con 0x55fca79daff0 2024-08-02T17:55:57.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.369+0000 7fe00106e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fca78fbb20 con 0x55fca79e61d0 2024-08-02T17:55:57.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.369+0000 7fe00106e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fca79e8f20 con 0x55fca79e3bc0 2024-08-02T17:55:57.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.370+0000 7fe00106e740 10 client.0 _ll_register_callbacks cb 0x55fca78f9bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:57.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.372+0000 7fdffd20d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.372+0000 7fdff7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.372+0000 7fdffd20d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79f2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35364/0 (socket says 172.21.15.69:35364) 2024-08-02T17:55:57.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.372+0000 7fdffd20d640 1 -- 172.21.15.69:0/3924868940 learned_addr learned my addr 172.21.15.69:0/3924868940 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:57.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.373+0000 7fdffd20d640 1 -- 172.21.15.69:0/3924868940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 msgr2=0x55fca79e3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:57.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.373+0000 7fdffd20d640 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.373+0000 7fdffd20d640 1 -- 172.21.15.69:0/3924868940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 msgr2=0x55fca79e3140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:55:57.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.373+0000 7fdffd20d640 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 0x55fca79e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.373+0000 7fdffd20d640 1 -- 172.21.15.69:0/3924868940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca76b23c0 con 0x55fca79e3bc0 2024-08-02T17:55:57.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.375+0000 7fdffd20d640 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79f2270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdfe809be10 tx=0x7fdfe80989b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14f16d9698de0823 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:57.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.376+0000 7fdfd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:55:57.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.376+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3924868940 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80b5040 con 0x55fca79e3bc0 2024-08-02T17:55:57.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.376+0000 7fe00106e740 1 -- 172.21.15.69:0/3924868940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe80a4d80 con 0x55fca79e3bc0 2024-08-02T17:55:57.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.376+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3924868940 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fdfe80cd4b0 con 0x55fca79e3bc0 2024-08-02T17:55:57.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.377+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3924868940 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfe80ad050 con 0x55fca79e3bc0 2024-08-02T17:55:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.378+0000 7fe00106e740 10 client.27109 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.378+0000 7fe00106e740 1 -- 172.21.15.69:0/3924868940 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fca79e8da0 con 0x55fca79e3bc0 2024-08-02T17:55:57.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.379+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3924868940 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe80a4d80 con 0x55fca79e3bc0 2024-08-02T17:55:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.380+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3924868940 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdfe8097f70 con 0x55fca79e3bc0 2024-08-02T17:55:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.381+0000 7fe00106e740 10 client.27109 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.381+0000 7fe00106e740 10 client.27109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:57.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.381+0000 7fe00106e740 20 client.27109 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:57.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.381+0000 7fe00106e740 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fca79f63e0 0x55fca79f8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.381+0000 7fe00106e740 4 client.27109 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:55:57.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.381+0000 7fe00106e740 1 -- 172.21.15.69:0/3924868940 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fca794dfb0 con 0x55fca79f63e0 2024-08-02T17:55:57.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.382+0000 7fdffca0c640 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fca79f63e0 0x55fca79f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.384+0000 7fdffca0c640 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fca79f63e0 0x55fca79f8820 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1469a29df4fd7c4 server_cookie=6cbd844469963e in_seq=0 out_seq=0 2024-08-02T17:55:57.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.384+0000 7fdfd7fff640 10 client.27109 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:57.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.385+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3924868940 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fca794dfb0 con 0x55fca79f63e0 2024-08-02T17:55:57.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.385+0000 7fdfd7fff640 10 client.27109 handle_command_reply: tid=0 2024-08-02T17:55:57.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.385+0000 7fe00106e740 1 client.27109 shutdown 2024-08-02T17:55:57.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.390+0000 7fe00106e740 1 -- 172.21.15.69:0/3924868940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 msgr2=0x55fca79f2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.390+0000 7fe00106e740 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79f2270 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdfe809be10 tx=0x7fdfe80989b0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.397+0000 7fe00106e740 1 -- 172.21.15.69:0/3924868940 shutdown_connections 2024-08-02T17:55:57.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.397+0000 7fe00106e740 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fca79f63e0 0x55fca79f8820 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.397+0000 7fe00106e740 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79f2270 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.397+0000 7fe00106e740 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79e61d0 0x55fca79e3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.397+0000 7fe00106e740 1 --2- 172.21.15.69:0/3924868940 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79daff0 0x55fca79e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.397+0000 7fe00106e740 1 -- 172.21.15.69:0/3924868940 >> 172.21.15.69:0/3924868940 conn(0x55fca79d7580 msgr2=0x55fca79dd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.397+0000 7fdfd7fff640 0 client.27109 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:57.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.401+0000 7fe00106e740 1 -- 172.21.15.69:0/3924868940 shutdown_connections 2024-08-02T17:55:57.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.402+0000 7fe00106e740 1 -- 172.21.15.69:0/3924868940 wait complete. 2024-08-02T17:55:57.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.402+0000 7fe00106e740 20 client.27109 trim_cache size 0 max 16384 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.430+0000 7fe00106e740 1 Processor -- start 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fe00106e740 1 -- start start 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fe00106e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79f2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fe00106e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fe00106e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e61d0 0x55fca79de9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fe00106e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4012880 con 0x55fca79e61d0 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fe00106e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4010150 con 0x55fca79daff0 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fe00106e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe400e050 con 0x55fca79e3bc0 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fdffca0c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fdffca0c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79f2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:47588/0 (socket says 172.21.15.69:47588) 2024-08-02T17:55:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fdffca0c640 1 -- 172.21.15.69:0/4159967807 learned_addr learned my addr 172.21.15.69:0/4159967807 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.431+0000 7fdff7fff640 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.432+0000 7fdffd20d640 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e61d0 0x55fca79de9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.433+0000 7fdffd20d640 1 -- 172.21.15.69:0/4159967807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 msgr2=0x55fca79de4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.433+0000 7fdffd20d640 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79de4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.433+0000 7fdffd20d640 1 -- 172.21.15.69:0/4159967807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 msgr2=0x55fca79f2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.433+0000 7fdffd20d640 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79f2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.433+0000 7fdffd20d640 1 -- 172.21.15.69:0/4159967807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fca79db740 con 0x55fca79e61d0 2024-08-02T17:55:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.433+0000 7fdffd20d640 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e61d0 0x55fca79de9f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8006fd0 tx=0x7fdfe8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba39c51964b3ed6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:57.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.434+0000 7fdfde7fc640 1 -- 172.21.15.69:0/4159967807 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80b5040 con 0x55fca79e61d0 2024-08-02T17:55:57.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.434+0000 7fdfde7fc640 1 -- 172.21.15.69:0/4159967807 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe80b0390 con 0x55fca79e61d0 2024-08-02T17:55:57.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.434+0000 7fdfde7fc640 1 -- 172.21.15.69:0/4159967807 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80cd420 con 0x55fca79e61d0 2024-08-02T17:55:57.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.435+0000 7fe00106e740 1 -- 172.21.15.69:0/4159967807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e61d0 msgr2=0x55fca79de9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.435+0000 7fe00106e740 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e61d0 0x55fca79de9f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8006fd0 tx=0x7fdfe8000bc0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.436+0000 7fe00106e740 1 -- 172.21.15.69:0/4159967807 shutdown_connections 2024-08-02T17:55:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.436+0000 7fe00106e740 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e61d0 0x55fca79de9f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.436+0000 7fe00106e740 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e3bc0 0x55fca79de4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.436+0000 7fe00106e740 1 --2- 172.21.15.69:0/4159967807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79f2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.436+0000 7fe00106e740 1 -- 172.21.15.69:0/4159967807 >> 172.21.15.69:0/4159967807 conn(0x55fca79d7580 msgr2=0x55fca79d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.437+0000 7fe00106e740 1 -- 172.21.15.69:0/4159967807 shutdown_connections 2024-08-02T17:55:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.437+0000 7fe00106e740 1 -- 172.21.15.69:0/4159967807 wait complete. 2024-08-02T17:55:57.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.437+0000 7fe00106e740 1 -- start start 2024-08-02T17:55:57.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.437+0000 7fe00106e740 1 Processor -- start 2024-08-02T17:55:57.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.438+0000 7fe00106e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79ee1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.438+0000 7fe00106e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e3bc0 0x55fca79ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.438+0000 7fdffca0c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79ee1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.438+0000 7fdffca0c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79ee1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:47594/0 (socket says 172.21.15.69:47594) 2024-08-02T17:55:57.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.439+0000 7fdffca0c640 1 -- 172.21.15.69:0/3854992934 learned_addr learned my addr 172.21.15.69:0/3854992934 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:55:57.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.439+0000 7fe00106e740 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e61d0 0x55fca79df3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.439+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe400e610 con 0x55fca79e3bc0 2024-08-02T17:55:57.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.439+0000 7fdff7fff640 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e3bc0 0x55fca79ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.439+0000 7fdffd20d640 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e61d0 0x55fca79df3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.439+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe400fa80 con 0x55fca79daff0 2024-08-02T17:55:57.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.439+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe400ece0 con 0x55fca79e61d0 2024-08-02T17:55:57.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.439+0000 7fe00106e740 10 client.0 _ll_register_callbacks cb 0x7fdfe80b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:55:57.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.440+0000 7fdff7fff640 1 -- 172.21.15.69:0/3854992934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e61d0 msgr2=0x55fca79df3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:57.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.440+0000 7fdff7fff640 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e61d0 0x55fca79df3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.440+0000 7fdff7fff640 1 -- 172.21.15.69:0/3854992934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 msgr2=0x55fca79ee1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:55:57.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.440+0000 7fdff7fff640 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79ee1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.440+0000 7fdff7fff640 1 -- 172.21.15.69:0/3854992934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fca79df940 con 0x55fca79e3bc0 2024-08-02T17:55:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.446+0000 7fdff7fff640 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e3bc0 0x55fca79ee710 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55fca79e29d0 tx=0x7fdfec012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9d2c7467323e12 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:55:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.446+0000 7fdfd7fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:55:57.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.452+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3854992934 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec021060 con 0x55fca79e3bc0 2024-08-02T17:55:57.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.452+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe80b0390 con 0x55fca79e3bc0 2024-08-02T17:55:57.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.452+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3854992934 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9568+0+0 (secure 0 0 0) 0x7fdfec004230 con 0x55fca79e3bc0 2024-08-02T17:55:57.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.452+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3854992934 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfec028050 con 0x55fca79e3bc0 2024-08-02T17:55:57.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.453+0000 7fe00106e740 10 client.17793 fetch_fsmap learned FSMap version 26 2024-08-02T17:55:57.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.453+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fca79e0c60 con 0x55fca79e3bc0 2024-08-02T17:55:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.459+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3854992934 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfec012e50 con 0x55fca79e3bc0 2024-08-02T17:55:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.459+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3854992934 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdfec01b030 con 0x55fca79e3bc0 2024-08-02T17:55:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.460+0000 7fe00106e740 10 client.17793 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:55:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.460+0000 7fe00106e740 10 client.17793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:55:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.460+0000 7fe00106e740 20 client.17793 populate_metadata read hostname 'smithi069' 2024-08-02T17:55:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.460+0000 7fe00106e740 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fca7a60110 0x55fca7a62530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:55:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.460+0000 7fe00106e740 4 client.17793 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:55:57.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.460+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fdfe80b0390 con 0x55fca7a60110 2024-08-02T17:55:57.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.462+0000 7fdffca0c640 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fca7a60110 0x55fca7a62530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:55:57.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.462+0000 7fdfd7fff640 10 client.17793 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:57.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.462+0000 7fdffca0c640 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fca7a60110 0x55fca7a62530 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc567a2131a39d8 server_cookie=eb4863ac8df1d8d7 in_seq=0 out_seq=0 2024-08-02T17:55:57.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.462+0000 7fdfd7fff640 1 -- 172.21.15.69:0/3854992934 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fdfe80b0390 con 0x55fca7a60110 2024-08-02T17:55:57.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.462+0000 7fdfd7fff640 10 client.17793 handle_command_reply: tid=0 2024-08-02T17:55:57.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.463+0000 7fe00106e740 1 client.17793 shutdown 2024-08-02T17:55:57.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.463+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e3bc0 msgr2=0x55fca79ee710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.463+0000 7fe00106e740 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e3bc0 0x55fca79ee710 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x55fca79e29d0 tx=0x7fdfec012b10 comp rx=0 tx=0).stop 2024-08-02T17:55:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.463+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 shutdown_connections 2024-08-02T17:55:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.463+0000 7fe00106e740 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fca79e3bc0 0x55fca79ee710 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.463+0000 7fe00106e740 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fca7a60110 0x55fca7a62530 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.463+0000 7fe00106e740 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fca79e61d0 0x55fca79df3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.463+0000 7fdfd7fff640 0 client.17793 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:55:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.464+0000 7fe00106e740 1 --2- 172.21.15.69:0/3854992934 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fca79daff0 0x55fca79ee1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.464+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 >> 172.21.15.69:0/3854992934 conn(0x55fca79d7580 msgr2=0x55fca79de4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.464+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 shutdown_connections 2024-08-02T17:55:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.464+0000 7fe00106e740 1 -- 172.21.15.69:0/3854992934 wait complete. 2024-08-02T17:55:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.464+0000 7fe00106e740 20 client.17793 trim_cache size 0 max 16384 2024-08-02T17:55:57.467 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:55:57.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfe00a2f00 msgr2=0x7fdfe00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfe00a2f00 0x7fdfe00a53c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdff002af20 tx=0x7fdff002ae30 comp rx=0 tx=0).stop 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdff81078c0 msgr2=0x7fdff819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdff81078c0 0x7fdff819eb10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b3e0 tx=0x7fdfe800b8a0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 shutdown_connections 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfe00a2f00 0x7fdfe00a53c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdff81078c0 0x7fdff819eb10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdff8102260 0x7fdff819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 --2- 172.21.15.69:0/2090618779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdff81018c0 0x7fdff819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 >> 172.21.15.69:0/2090618779 conn(0x7fdff80fd620 msgr2=0x7fdff80fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 shutdown_connections 2024-08-02T17:55:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:55:57.469+0000 7fdffec70640 1 -- 172.21.15.69:0/2090618779 wait complete. 2024-08-02T17:55:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:57 smithi083 ceph-mon[37031]: pgmap v294: 129 pgs: 129 active+clean; 3.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.1 MiB/s wr, 660 op/s 2024-08-02T17:55:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:55:59 smithi077 ceph-mon[37108]: pgmap v295: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.3 MiB/s wr, 931 op/s 2024-08-02T17:55:59.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:55:59 smithi069 ceph-mon[32746]: pgmap v295: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.3 MiB/s wr, 931 op/s 2024-08-02T17:55:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:55:59 smithi083 ceph-mon[37031]: pgmap v295: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.3 MiB/s wr, 931 op/s 2024-08-02T17:55:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 530.0 s, 587.2 tps, lat 186.756 ms stddev 259.532 2024-08-02T17:56:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:01 smithi077 ceph-mon[37108]: pgmap v296: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 788 op/s 2024-08-02T17:56:01.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:01 smithi069 ceph-mon[32746]: pgmap v296: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 788 op/s 2024-08-02T17:56:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:01 smithi083 ceph-mon[37031]: pgmap v296: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 7.2 MiB/s wr, 788 op/s 2024-08-02T17:56:02.618 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:56:02.618 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:56:03.051 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:03 smithi069 ceph-mon[32746]: pgmap v297: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 671 op/s 2024-08-02T17:56:03.153 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:03 smithi077 ceph-mon[37108]: pgmap v297: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 671 op/s 2024-08-02T17:56:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:03 smithi083 ceph-mon[37031]: pgmap v297: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 5.9 MiB/s wr, 671 op/s 2024-08-02T17:56:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 535.0 s, 554.6 tps, lat 178.057 ms stddev 226.278 2024-08-02T17:56:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:05 smithi077 ceph-mon[37108]: pgmap v298: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 MiB/s wr, 848 op/s 2024-08-02T17:56:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:05 smithi069 ceph-mon[32746]: pgmap v298: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 MiB/s wr, 848 op/s 2024-08-02T17:56:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:05 smithi083 ceph-mon[37031]: pgmap v298: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 7.5 MiB/s wr, 848 op/s 2024-08-02T17:56:06.319 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:06 smithi069 ceph-mon[32746]: osdmap e102: 12 total, 12 up, 12 in 2024-08-02T17:56:06.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:06 smithi077 ceph-mon[37108]: osdmap e102: 12 total, 12 up, 12 in 2024-08-02T17:56:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:06 smithi083 ceph-mon[37031]: osdmap e102: 12 total, 12 up, 12 in 2024-08-02T17:56:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.749+0000 7f64460c7640 1 -- 172.21.15.69:0/712551046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6438095520 msgr2=0x7f6438099170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.749+0000 7f64460c7640 1 --2- 172.21.15.69:0/712551046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6438095520 0x7f6438099170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6430009900 tx=0x7f643002a410 comp rx=0 tx=0).stop 2024-08-02T17:56:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 -- 172.21.15.69:0/712551046 shutdown_connections 2024-08-02T17:56:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 --2- 172.21.15.69:0/712551046 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6438099880 0x7f64380a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 --2- 172.21.15.69:0/712551046 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6438095520 0x7f6438099170 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 --2- 172.21.15.69:0/712551046 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6438094b70 0x7f6438094f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 -- 172.21.15.69:0/712551046 >> 172.21.15.69:0/712551046 conn(0x7f643800a1e0 msgr2=0x7f643800a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 -- 172.21.15.69:0/712551046 shutdown_connections 2024-08-02T17:56:06.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 -- 172.21.15.69:0/712551046 wait complete. 2024-08-02T17:56:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 Processor -- start 2024-08-02T17:56:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.750+0000 7f64460c7640 1 -- start start 2024-08-02T17:56:06.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64460c7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6438094b70 0x7f643812e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64460c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6438099880 0x7f643812e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64460c7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6438132900 0x7f6438132db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64460c7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64380a3850 con 0x7f6438099880 2024-08-02T17:56:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64460c7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64380a36d0 con 0x7f6438094b70 2024-08-02T17:56:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64460c7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64380a39d0 con 0x7f6438132900 2024-08-02T17:56:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64448c4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6438099880 0x7f643812e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6438094b70 0x7f643812e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6438094b70 0x7f643812e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52242/0 (socket says 172.21.15.69:52242) 2024-08-02T17:56:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 -- 172.21.15.69:0/819435519 learned_addr learned my addr 172.21.15.69:0/819435519 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64458c6640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6438132900 0x7f6438132db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 -- 172.21.15.69:0/819435519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6438132900 msgr2=0x7f6438132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6438132900 0x7f6438132db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 -- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6438099880 msgr2=0x7f643812e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.837 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6438099880 0x7f643812e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 -- 172.21.15.69:0/819435519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64381333f0 con 0x7f6438094b70 2024-08-02T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.751+0000 7f64450c5640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6438094b70 0x7f643812e040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f643c00c890 tx=0x7f643c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.752+0000 7f64367fc640 1 -- 172.21.15.69:0/819435519 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c004220 con 0x7f6438094b70 2024-08-02T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.752+0000 7f64460c7640 1 -- 172.21.15.69:0/819435519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64381336e0 con 0x7f6438094b70 2024-08-02T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.752+0000 7f64367fc640 1 -- 172.21.15.69:0/819435519 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f643c0043c0 con 0x7f6438094b70 2024-08-02T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.752+0000 7f64460c7640 1 -- 172.21.15.69:0/819435519 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643809b730 con 0x7f6438094b70 2024-08-02T17:56:06.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.752+0000 7f64367fc640 1 -- 172.21.15.69:0/819435519 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0049e0 con 0x7f6438094b70 2024-08-02T17:56:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.755+0000 7f64367fc640 1 -- 172.21.15.69:0/819435519 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f643c0079b0 con 0x7f6438094b70 2024-08-02T17:56:06.843 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.757+0000 7f64367fc640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f642c0a2cf0 0x7f642c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.757+0000 7f64367fc640 1 -- 172.21.15.69:0/819435519 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7f643c0cf970 con 0x7f6438094b70 2024-08-02T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.757+0000 7f64448c4640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f642c0a2cf0 0x7f642c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.757+0000 7f6449d22740 1 Processor -- start 2024-08-02T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.757+0000 7f6449d22740 1 -- start start 2024-08-02T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f64448c4640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f642c0a2cf0 0x7f642c0a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430043040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f6449d22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f793fad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f6449d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fb2b0 0x559f79405610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.844 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f6449d22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f79405b50 0x559f79407f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f6449d22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f7935e9b0 con 0x559f79405b50 2024-08-02T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f6449d22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f79306800 con 0x559f793fa990 2024-08-02T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f6449d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f7935a970 con 0x559f793fb2b0 2024-08-02T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f64458c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f79405b50 0x559f79407f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f64458c6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f79405b50 0x559f79407f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59988/0 (socket says 172.21.15.69:59988) 2024-08-02T17:56:06.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f64458c6640 1 -- 172.21.15.69:0/1314328459 learned_addr learned my addr 172.21.15.69:0/1314328459 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f64448c4640 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fb2b0 0x559f79405610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.758+0000 7f64450c5640 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f793fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.762+0000 7f64458c6640 1 -- 172.21.15.69:0/1314328459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fb2b0 msgr2=0x559f79405610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.762+0000 7f64458c6640 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fb2b0 0x559f79405610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.762+0000 7f64458c6640 1 -- 172.21.15.69:0/1314328459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 msgr2=0x559f793fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.762+0000 7f64458c6640 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f793fad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.762+0000 7f64458c6640 1 -- 172.21.15.69:0/1314328459 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f79072210 con 0x559f79405b50 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.766+0000 7f64458c6640 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f79405b50 0x559f79407f40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f64400514c0 tx=0x7f6440068e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7379faa4cc86a164 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.768+0000 7f642b7fe640 1 -- 172.21.15.69:0/1314328459 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644007b450 con 0x559f79405b50 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.768+0000 7f642b7fe640 1 -- 172.21.15.69:0/1314328459 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644007bad0 con 0x559f79405b50 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.768+0000 7f642b7fe640 1 -- 172.21.15.69:0/1314328459 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644007aa60 con 0x559f79405b50 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.769+0000 7f6449d22740 1 -- 172.21.15.69:0/1314328459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f79405b50 msgr2=0x559f79407f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.769+0000 7f6449d22740 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f79405b50 0x559f79407f40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f64400514c0 tx=0x7f6440068e30 comp rx=0 tx=0).stop 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.778+0000 7f6449d22740 1 -- 172.21.15.69:0/1314328459 shutdown_connections 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.778+0000 7f6449d22740 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f79405b50 0x559f79407f40 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.778+0000 7f6449d22740 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fb2b0 0x559f79405610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.778+0000 7f6449d22740 1 --2- 172.21.15.69:0/1314328459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f793fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.778+0000 7f6449d22740 1 -- 172.21.15.69:0/1314328459 >> 172.21.15.69:0/1314328459 conn(0x559f793f6f00 msgr2=0x559f793f72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.789+0000 7f6449d22740 1 -- 172.21.15.69:0/1314328459 shutdown_connections 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.790+0000 7f6449d22740 1 -- 172.21.15.69:0/1314328459 wait complete. 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.790+0000 7f6449d22740 1 -- start start 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.790+0000 7f6449d22740 1 Processor -- start 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f6449d22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f79402a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f6449d22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f79402fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f6449d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f79405b50 0x559f794034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f6449d22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559f79408720 con 0x559f793fb2b0 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f6449d22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559f7931a500 con 0x559f793fa990 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f6449d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559f794088a0 con 0x559f79405b50 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f6449d22740 10 client.0 _ll_register_callbacks cb 0x559f79071a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f64450c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f79402a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f64450c5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f79402a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52262/0 (socket says 172.21.15.69:52262) 2024-08-02T17:56:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.793+0000 7f64450c5640 1 -- 172.21.15.69:0/856237572 learned_addr learned my addr 172.21.15.69:0/856237572 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.794+0000 7f64448c4640 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f79402fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.794+0000 7f64450c5640 1 -- 172.21.15.69:0/856237572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f79405b50 msgr2=0x559f794034e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:56:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.794+0000 7f64450c5640 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f79405b50 0x559f794034e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.794+0000 7f64450c5640 1 -- 172.21.15.69:0/856237572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 msgr2=0x559f79402fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.794+0000 7f64450c5640 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f79402fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.794+0000 7f64450c5640 1 -- 172.21.15.69:0/856237572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7913e620 con 0x559f793fa990 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.801+0000 7f64450c5640 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f79402a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f643c0ccc50 tx=0x7f643c09c200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554333ae08e0585a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.801+0000 7f6428ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.802+0000 7f6428ff9640 1 -- 172.21.15.69:0/856237572 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0b4060 con 0x559f793fa990 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.802+0000 7f6449d22740 1 -- 172.21.15.69:0/856237572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644007bad0 con 0x559f793fa990 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.802+0000 7f6428ff9640 1 -- 172.21.15.69:0/856237572 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7f643c0a41a0 con 0x559f793fa990 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.802+0000 7f6428ff9640 1 -- 172.21.15.69:0/856237572 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f643c01d070 con 0x559f793fa990 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.802+0000 7f6449d22740 10 client.27302 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.802+0000 7f6449d22740 1 -- 172.21.15.69:0/856237572 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f79411c90 con 0x559f793fa990 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.803+0000 7f6428ff9640 1 -- 172.21.15.69:0/856237572 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f644007bad0 con 0x559f793fa990 2024-08-02T17:56:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.805+0000 7f6428ff9640 1 -- 172.21.15.69:0/856237572 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f643c0860d0 con 0x559f793fa990 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.806+0000 7f6449d22740 10 client.27302 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.806+0000 7f6449d22740 10 client.27302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.806+0000 7f6449d22740 20 client.27302 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.806+0000 7f6449d22740 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f79413590 0x559f79415a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.806+0000 7f6449d22740 4 client.27302 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.806+0000 7f6449d22740 1 -- 172.21.15.69:0/856237572 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f7936c0e0 con 0x559f79413590 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.808+0000 7f64448c4640 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f79413590 0x559f79415a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.814+0000 7f64448c4640 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f79413590 0x559f79415a50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=651c4fb336f952b3 server_cookie=6c096adc8c69704d in_seq=0 out_seq=0 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.815+0000 7f6428ff9640 10 client.27302 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.823+0000 7f6428ff9640 1 -- 172.21.15.69:0/856237572 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559f7936c0e0 con 0x559f79413590 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.823+0000 7f6428ff9640 10 client.27302 handle_command_reply: tid=0 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.823+0000 7f6449d22740 1 client.27302 shutdown 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.827+0000 7f6449d22740 1 -- 172.21.15.69:0/856237572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 msgr2=0x559f79402a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.827+0000 7f6449d22740 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f79402a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f643c0ccc50 tx=0x7f643c09c200 comp rx=0 tx=0).stop 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.829+0000 7f6449d22740 1 -- 172.21.15.69:0/856237572 shutdown_connections 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.829+0000 7f6449d22740 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f79402fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.829+0000 7f6449d22740 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f79413590 0x559f79415a50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.829+0000 7f6449d22740 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f79405b50 0x559f794034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.829+0000 7f6449d22740 1 --2- 172.21.15.69:0/856237572 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fa990 0x559f79402a60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.829+0000 7f6449d22740 1 -- 172.21.15.69:0/856237572 >> 172.21.15.69:0/856237572 conn(0x559f793f6f00 msgr2=0x559f793fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.830+0000 7f6428ff9640 0 client.27302 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.836+0000 7f6449d22740 1 -- 172.21.15.69:0/856237572 shutdown_connections 2024-08-02T17:56:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.838+0000 7f6449d22740 1 -- 172.21.15.69:0/856237572 wait complete. 2024-08-02T17:56:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.840+0000 7f6449d22740 20 client.27302 trim_cache size 0 max 16384 2024-08-02T17:56:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.880+0000 7f6449d22740 1 Processor -- start 2024-08-02T17:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f6449d22740 1 -- start start 2024-08-02T17:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f6449d22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fa990 0x559f7941b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f6449d22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fb2b0 0x559f7941b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f6449d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f79405b50 0x559f794041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f6449d22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6424012a70 con 0x559f793fa990 2024-08-02T17:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f6449d22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6424010340 con 0x559f793fb2b0 2024-08-02T17:56:06.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f6449d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f642400e1b0 con 0x559f79405b50 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f64448c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fb2b0 0x559f7941b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f64448c4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fb2b0 0x559f7941b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52274/0 (socket says 172.21.15.69:52274) 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f64448c4640 1 -- 172.21.15.69:0/1430449728 learned_addr learned my addr 172.21.15.69:0/1430449728 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.884+0000 7f64450c5640 1 --2- 172.21.15.69:0/1430449728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fa990 0x559f7941b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.885+0000 7f64450c5640 1 -- 172.21.15.69:0/1430449728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f79405b50 msgr2=0x559f794041b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.885+0000 7f64450c5640 1 --2- 172.21.15.69:0/1430449728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f79405b50 0x559f794041b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.885+0000 7f64450c5640 1 -- 172.21.15.69:0/1430449728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fb2b0 msgr2=0x559f7941b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.885+0000 7f64450c5640 1 --2- 172.21.15.69:0/1430449728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fb2b0 0x559f7941b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.885+0000 7f64450c5640 1 -- 172.21.15.69:0/1430449728 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f79405510 con 0x559f793fa990 2024-08-02T17:56:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.885+0000 7f64450c5640 1 --2- 172.21.15.69:0/1430449728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fa990 0x559f7941b360 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f643c098240 tx=0x7f643c005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7706f256f7d327 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.885+0000 7f642affd640 1 -- 172.21.15.69:0/1430449728 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0860d0 con 0x559f793fa990 2024-08-02T17:56:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.886+0000 7f642affd640 1 -- 172.21.15.69:0/1430449728 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f643c007ca0 con 0x559f793fa990 2024-08-02T17:56:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.886+0000 7f642affd640 1 -- 172.21.15.69:0/1430449728 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643c0b4e20 con 0x559f793fa990 2024-08-02T17:56:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.888+0000 7f6449d22740 1 -- 172.21.15.69:0/1430449728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fa990 msgr2=0x559f7941b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.888+0000 7f6449d22740 1 --2- 172.21.15.69:0/1430449728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fa990 0x559f7941b360 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f643c098240 tx=0x7f643c005320 comp rx=0 tx=0).stop 2024-08-02T17:56:06.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.887+0000 7f4132d53640 1 -- 172.21.15.69:0/3368394824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f412c106730 msgr2=0x7f412c106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.887+0000 7f4132d53640 1 --2- 172.21.15.69:0/3368394824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f412c106730 0x7f412c106b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4124009900 tx=0x7f412402a3f0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.887+0000 7f4132d53640 1 -- 172.21.15.69:0/3368394824 shutdown_connections 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.887+0000 7f4132d53640 1 --2- 172.21.15.69:0/3368394824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c1005f0 0x7f412c1041a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.887+0000 7f4132d53640 1 --2- 172.21.15.69:0/3368394824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f412c0ffc70 0x7f412c1000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.887+0000 7f4132d53640 1 --2- 172.21.15.69:0/3368394824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f412c106730 0x7f412c106b10 secure :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4124009900 tx=0x7f412402a3f0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.887+0000 7f4132d53640 1 -- 172.21.15.69:0/3368394824 >> 172.21.15.69:0/3368394824 conn(0x7f412c076da0 msgr2=0x7f412c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.888+0000 7f4132d53640 1 -- 172.21.15.69:0/3368394824 shutdown_connections 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.888+0000 7f4132d53640 1 -- 172.21.15.69:0/3368394824 wait complete. 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132d53640 1 Processor -- start 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132d53640 1 -- start start 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132d53640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f412c0ffc70 0x7f412c0737d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132d53640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f412c1005f0 0x7f412c06c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132d53640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c06cda0 0x7f412c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132d53640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f412c118220 con 0x7f412c06cda0 2024-08-02T17:56:06.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132d53640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f412c1180a0 con 0x7f412c0ffc70 2024-08-02T17:56:06.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132d53640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f412c1183a0 con 0x7f412c1005f0 2024-08-02T17:56:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.892+0000 7f6449d22740 1 -- 172.21.15.69:0/1430449728 shutdown_connections 2024-08-02T17:56:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.892+0000 7f6449d22740 1 --2- 172.21.15.69:0/1430449728 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f79405b50 0x559f794041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.892+0000 7f6449d22740 1 --2- 172.21.15.69:0/1430449728 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f793fb2b0 0x559f7941b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.892+0000 7f6449d22740 1 --2- 172.21.15.69:0/1430449728 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fa990 0x559f7941b360 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.892+0000 7f6449d22740 1 -- 172.21.15.69:0/1430449728 >> 172.21.15.69:0/1430449728 conn(0x559f793f6f00 msgr2=0x559f79401330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132552640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c06cda0 0x7f412c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132552640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c06cda0 0x7f412c06d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60016/0 (socket says 172.21.15.69:60016) 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.889+0000 7f4132552640 1 -- 172.21.15.69:0/3629521665 learned_addr learned my addr 172.21.15.69:0/3629521665 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.890+0000 7f4132552640 1 -- 172.21.15.69:0/3629521665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f412c1005f0 msgr2=0x7f412c06c860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.890+0000 7f4132552640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f412c1005f0 0x7f412c06c860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.890+0000 7f4132552640 1 -- 172.21.15.69:0/3629521665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f412c0ffc70 msgr2=0x7f412c0737d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.890+0000 7f4132552640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f412c0ffc70 0x7f412c0737d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.890+0000 7f4132552640 1 -- 172.21.15.69:0/3629521665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c071300 con 0x7f412c06cda0 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.890+0000 7f4132552640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c06cda0 0x7f412c06d250 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f412800b9e0 tx=0x7f412800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.891+0000 7f4122ffd640 1 -- 172.21.15.69:0/3629521665 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128019070 con 0x7f412c06cda0 2024-08-02T17:56:06.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.891+0000 7f4132d53640 1 -- 172.21.15.69:0/3629521665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c0715f0 con 0x7f412c06cda0 2024-08-02T17:56:06.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.891+0000 7f4132d53640 1 -- 172.21.15.69:0/3629521665 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c071b30 con 0x7f412c06cda0 2024-08-02T17:56:06.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.891+0000 7f4122ffd640 1 -- 172.21.15.69:0/3629521665 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4128014440 con 0x7f412c06cda0 2024-08-02T17:56:06.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.891+0000 7f4122ffd640 1 -- 172.21.15.69:0/3629521665 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280126b0 con 0x7f412c06cda0 2024-08-02T17:56:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.895+0000 7f6449d22740 1 -- 172.21.15.69:0/1430449728 shutdown_connections 2024-08-02T17:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.897+0000 7f6449d22740 1 -- 172.21.15.69:0/1430449728 wait complete. 2024-08-02T17:56:06.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.895+0000 7f4122ffd640 1 -- 172.21.15.69:0/3629521665 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f412801c070 con 0x7f412c06cda0 2024-08-02T17:56:06.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.896+0000 7f4122ffd640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41180a2f00 0x7f41180a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.897+0000 7f6449d22740 1 -- start start 2024-08-02T17:56:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.897+0000 7f6449d22740 1 Processor -- start 2024-08-02T17:56:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.898+0000 7f6449d22740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fa990 0x559f7940db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.898+0000 7f6449d22740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f7940e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.898+0000 7f6449d22740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f79405b50 0x559f793ff0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.898+0000 7f6449d22740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f642400e800 con 0x559f793fb2b0 2024-08-02T17:56:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.898+0000 7f6449d22740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f642400fc70 con 0x559f79405b50 2024-08-02T17:56:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.898+0000 7f6449d22740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f642400eed0 con 0x559f793fa990 2024-08-02T17:56:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.898+0000 7f6449d22740 10 client.0 _ll_register_callbacks cb 0x7f643c09b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:06.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.897+0000 7f4122ffd640 1 -- 172.21.15.69:0/3629521665 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7f41280cfdb0 con 0x7f412c06cda0 2024-08-02T17:56:06.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.897+0000 7f4131d51640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41180a2f00 0x7f41180a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.900+0000 7f64450c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fa990 0x559f7940db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.900+0000 7f64450c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fa990 0x559f7940db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:58778/0 (socket says 172.21.15.69:58778) 2024-08-02T17:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.900+0000 7f64450c5640 1 -- 172.21.15.69:0/210511377 learned_addr learned my addr 172.21.15.69:0/210511377 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.900+0000 7f64448c4640 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f7940e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.901+0000 7f64448c4640 1 -- 172.21.15.69:0/210511377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fa990 msgr2=0x559f7940db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.901+0000 7f64448c4640 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fa990 0x559f7940db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.901+0000 7f64448c4640 1 -- 172.21.15.69:0/210511377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f79405b50 msgr2=0x559f793ff0a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.901+0000 7f64448c4640 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f79405b50 0x559f793ff0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.901+0000 7f64448c4640 1 -- 172.21.15.69:0/210511377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f793ff5e0 con 0x559f793fb2b0 2024-08-02T17:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.901+0000 7f64448c4640 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f7940e0d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x559f793fd500 tx=0x7f6430047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9002cd9f100a2c39 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.901+0000 7f6428ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.902+0000 7f6428ff9640 1 -- 172.21.15.69:0/210511377 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643005d060 con 0x559f793fb2b0 2024-08-02T17:56:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.902+0000 7f6449d22740 1 -- 172.21.15.69:0/210511377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f643c007ca0 con 0x559f793fb2b0 2024-08-02T17:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.902+0000 7f6428ff9640 1 -- 172.21.15.69:0/210511377 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7f64300049b0 con 0x559f793fb2b0 2024-08-02T17:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.902+0000 7f6428ff9640 1 -- 172.21.15.69:0/210511377 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6430064050 con 0x559f793fb2b0 2024-08-02T17:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.902+0000 7f6449d22740 10 client.17814 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6449d22740 1 -- 172.21.15.69:0/210511377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559f79491e60 con 0x559f793fb2b0 2024-08-02T17:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6428ff9640 1 -- 172.21.15.69:0/210511377 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f643c007ca0 con 0x559f793fb2b0 2024-08-02T17:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6428ff9640 1 -- 172.21.15.69:0/210511377 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f643004b070 con 0x559f793fb2b0 2024-08-02T17:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6449d22740 10 client.17814 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6449d22740 10 client.17814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6449d22740 20 client.17814 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6449d22740 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f79492590 0x559f794949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6449d22740 4 client.17814 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:56:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.903+0000 7f6449d22740 1 -- 172.21.15.69:0/210511377 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f79366430 con 0x559f79492590 2024-08-02T17:56:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.906+0000 7f64450c5640 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f79492590 0x559f794949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.908+0000 7f4131d51640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41180a2f00 0x7f41180a53c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f412402a960 tx=0x7f4124005fd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.908+0000 7f64450c5640 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f79492590 0x559f794949f0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76d8054de52a34b2 server_cookie=fed48281e83a171b in_seq=0 out_seq=0 2024-08-02T17:56:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.909+0000 7f6428ff9640 10 client.17814 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:06.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.912+0000 7f41363b3740 1 Processor -- start 2024-08-02T17:56:06.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.912+0000 7f41363b3740 1 -- start start 2024-08-02T17:56:06.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.912+0000 7f41363b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c43d0 0x558e050c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.912+0000 7f41363b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.912+0000 7f41363b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 0x558e050d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.912+0000 7f41363b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558e05087e40 con 0x558e050cf5b0 2024-08-02T17:56:06.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.912+0000 7f41363b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558e050297f0 con 0x558e050c4cf0 2024-08-02T17:56:06.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.912+0000 7f41363b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558e04fd0fb0 con 0x558e050c43d0 2024-08-02T17:56:06.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.914+0000 7f4131d51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c43d0 0x558e050c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.914+0000 7f4131d51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c43d0 0x558e050c47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:58780/0 (socket says 172.21.15.69:58780) 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.914+0000 7f4131d51640 1 -- 172.21.15.69:0/4270501366 learned_addr learned my addr 172.21.15.69:0/4270501366 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.914+0000 7f4132552640 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 0x558e050d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.915+0000 7f4131550640 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.915+0000 7f4131d51640 1 -- 172.21.15.69:0/4270501366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 msgr2=0x558e050cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.915+0000 7f4131d51640 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.915+0000 7f4131d51640 1 -- 172.21.15.69:0/4270501366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 msgr2=0x558e050d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.915+0000 7f4131d51640 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 0x558e050d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.915+0000 7f4131d51640 1 -- 172.21.15.69:0/4270501366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e04d9b6a0 con 0x558e050c43d0 2024-08-02T17:56:06.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.915+0000 7f4131d51640 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c43d0 0x558e050c47b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4124009900 tx=0x7f412403a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96a028b67f238e4d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.915+0000 7f4117fff640 1 -- 172.21.15.69:0/4270501366 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124040070 con 0x558e050c43d0 2024-08-02T17:56:06.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.916+0000 7f4117fff640 1 -- 172.21.15.69:0/4270501366 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412404c070 con 0x558e050c43d0 2024-08-02T17:56:06.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.917+0000 7f41363b3740 1 -- 172.21.15.69:0/4270501366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c43d0 msgr2=0x558e050c47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.917+0000 7f41363b3740 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c43d0 0x558e050c47b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4124009900 tx=0x7f412403a000 comp rx=0 tx=0).stop 2024-08-02T17:56:06.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.917+0000 7f41363b3740 1 -- 172.21.15.69:0/4270501366 shutdown_connections 2024-08-02T17:56:06.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.917+0000 7f41363b3740 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 0x558e050d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.917+0000 7f41363b3740 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.917+0000 7f41363b3740 1 --2- 172.21.15.69:0/4270501366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c43d0 0x558e050c47b0 secure :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4124009900 tx=0x7f412403a000 comp rx=0 tx=0).stop 2024-08-02T17:56:06.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.917+0000 7f41363b3740 1 -- 172.21.15.69:0/4270501366 >> 172.21.15.69:0/4270501366 conn(0x558e050c0960 msgr2=0x558e050c0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.917+0000 7f41363b3740 1 -- 172.21.15.69:0/4270501366 shutdown_connections 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.918+0000 7f41363b3740 1 -- 172.21.15.69:0/4270501366 wait complete. 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.918+0000 7f41363b3740 1 -- start start 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.918+0000 7f41363b3740 1 Processor -- start 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f41363b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c4cf0 0x558e050cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f41363b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 0x558e050cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f41363b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050cc800 0x558e050ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f41363b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558e04fe4cb0 con 0x558e050cf5b0 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f41363b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558e050257b0 con 0x558e050cc800 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f41363b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558e050d2220 con 0x558e050c4cf0 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f41363b3740 10 client.0 _ll_register_callbacks cb 0x558e04d38ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:06.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f4132552640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050cc800 0x558e050ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f4132552640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050cc800 0x558e050ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52304/0 (socket says 172.21.15.69:52304) 2024-08-02T17:56:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.919+0000 7f4132552640 1 -- 172.21.15.69:0/3367980700 learned_addr learned my addr 172.21.15.69:0/3367980700 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.920+0000 7f4132552640 1 -- 172.21.15.69:0/3367980700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c4cf0 msgr2=0x558e050cefc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:56:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.920+0000 7f4132552640 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c4cf0 0x558e050cefc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.920+0000 7f4132552640 1 -- 172.21.15.69:0/3367980700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 msgr2=0x558e050cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.920+0000 7f4132552640 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 0x558e050cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.920+0000 7f4132552640 1 -- 172.21.15.69:0/3367980700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e04fe3bc0 con 0x558e050cc800 2024-08-02T17:56:06.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.920+0000 7f4132552640 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050cc800 0x558e050ccc60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4128098370 tx=0x7f412809c050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da383e51dc296a08 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.924+0000 7f41157fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:56:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.924+0000 7f41157fa640 1 -- 172.21.15.69:0/3367980700 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41280b7070 con 0x558e050cc800 2024-08-02T17:56:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.924+0000 7f41363b3740 1 -- 172.21.15.69:0/3367980700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e05036d60 con 0x558e050cc800 2024-08-02T17:56:06.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.924+0000 7f41157fa640 1 -- 172.21.15.69:0/3367980700 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7f41280a46f0 con 0x558e050cc800 2024-08-02T17:56:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.924+0000 7f41157fa640 1 -- 172.21.15.69:0/3367980700 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f41280b4020 con 0x558e050cc800 2024-08-02T17:56:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.924+0000 7f41363b3740 10 client.27311 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:06.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.924+0000 7f41363b3740 1 -- 172.21.15.69:0/3367980700 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558e050de430 con 0x558e050cc800 2024-08-02T17:56:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.925+0000 7f41157fa640 1 -- 172.21.15.69:0/3367980700 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558e05036d60 con 0x558e050cc800 2024-08-02T17:56:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.925+0000 7f41157fa640 1 -- 172.21.15.69:0/3367980700 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f41280ba020 con 0x558e050cc800 2024-08-02T17:56:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.925+0000 7f41363b3740 10 client.27311 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.925+0000 7f41363b3740 10 client.27311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.925+0000 7f41363b3740 20 client.27311 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.925+0000 7f41363b3740 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558e050dfdd0 0x558e050e2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.925+0000 7f41363b3740 4 client.27311 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:06.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.925+0000 7f41363b3740 1 -- 172.21.15.69:0/3367980700 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e05031270 con 0x558e050dfdd0 2024-08-02T17:56:06.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.926+0000 7f4131d51640 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558e050dfdd0 0x558e050e2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.926+0000 7f4131d51640 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558e050dfdd0 0x558e050e2290 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57e5c09479f7092f server_cookie=293f3cb1c8570d6a in_seq=0 out_seq=0 2024-08-02T17:56:06.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.926+0000 7f41157fa640 10 client.27311 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:06.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.928+0000 7f41157fa640 1 -- 172.21.15.69:0/3367980700 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558e05031270 con 0x558e050dfdd0 2024-08-02T17:56:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.928+0000 7f41157fa640 10 client.27311 handle_command_reply: tid=0 2024-08-02T17:56:06.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.928+0000 7f41363b3740 1 client.27311 shutdown 2024-08-02T17:56:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.932+0000 7f41363b3740 1 -- 172.21.15.69:0/3367980700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050cc800 msgr2=0x558e050ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.932+0000 7f41363b3740 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050cc800 0x558e050ccc60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4128098370 tx=0x7f412809c050 comp rx=0 tx=0).stop 2024-08-02T17:56:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.932+0000 7f41363b3740 1 -- 172.21.15.69:0/3367980700 shutdown_connections 2024-08-02T17:56:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.932+0000 7f41363b3740 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050cf5b0 0x558e050cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.932+0000 7f41363b3740 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558e050dfdd0 0x558e050e2290 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.932+0000 7f41363b3740 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050cc800 0x558e050ccc60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.932+0000 7f41363b3740 1 --2- 172.21.15.69:0/3367980700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050c4cf0 0x558e050cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.932+0000 7f41363b3740 1 -- 172.21.15.69:0/3367980700 >> 172.21.15.69:0/3367980700 conn(0x558e050c0960 msgr2=0x558e050c5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.933+0000 7f41157fa640 0 client.27311 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:06.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.934+0000 7f41363b3740 1 -- 172.21.15.69:0/3367980700 shutdown_connections 2024-08-02T17:56:06.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.934+0000 7f41363b3740 1 -- 172.21.15.69:0/3367980700 wait complete. 2024-08-02T17:56:06.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.935+0000 7f41363b3740 20 client.27311 trim_cache size 0 max 16384 2024-08-02T17:56:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.958+0000 7f6428ff9640 1 -- 172.21.15.69:0/210511377 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f79366430 con 0x559f79492590 2024-08-02T17:56:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.958+0000 7f6428ff9640 10 client.17814 handle_command_reply: tid=0 2024-08-02T17:56:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.958+0000 7f6449d22740 1 client.17814 shutdown 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 -- 172.21.15.69:0/210511377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 msgr2=0x559f7940e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f7940e0d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x559f793fd500 tx=0x7f6430047000 comp rx=0 tx=0).stop 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 -- 172.21.15.69:0/210511377 shutdown_connections 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559f793fb2b0 0x559f7940e0d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559f79492590 0x559f794949f0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559f79405b50 0x559f793ff0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 --2- 172.21.15.69:0/210511377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559f793fa990 0x559f7940db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 -- 172.21.15.69:0/210511377 >> 172.21.15.69:0/210511377 conn(0x559f793f6f00 msgr2=0x559f793f5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6428ff9640 0 client.17814 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.959+0000 7f6449d22740 1 -- 172.21.15.69:0/210511377 shutdown_connections 2024-08-02T17:56:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.960+0000 7f6449d22740 1 -- 172.21.15.69:0/210511377 wait complete. 2024-08-02T17:56:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.960+0000 7f6449d22740 20 client.17814 trim_cache size 0 max 16384 2024-08-02T17:56:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:56:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:56:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:56:06.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:56:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.961+0000 7f64460c7640 1 -- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f642c0a2cf0 msgr2=0x7f642c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.961+0000 7f64460c7640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f642c0a2cf0 0x7f642c0a51b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430043040 comp rx=0 tx=0).stop 2024-08-02T17:56:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.962+0000 7f64460c7640 1 -- 172.21.15.69:0/819435519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6438094b70 msgr2=0x7f643812e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.962+0000 7f64460c7640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6438094b70 0x7f643812e040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f643c00c890 tx=0x7f643c00cd50 comp rx=0 tx=0).stop 2024-08-02T17:56:06.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.962+0000 7f64460c7640 1 -- 172.21.15.69:0/819435519 shutdown_connections 2024-08-02T17:56:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.962+0000 7f64460c7640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6438099880 0x7f643812e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.962+0000 7f64460c7640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f642c0a2cf0 0x7f642c0a51b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.962+0000 7f64460c7640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6438132900 0x7f6438132db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.962+0000 7f64460c7640 1 --2- 172.21.15.69:0/819435519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6438094b70 0x7f643812e040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.962+0000 7f64460c7640 1 -- 172.21.15.69:0/819435519 >> 172.21.15.69:0/819435519 conn(0x7f643800a1e0 msgr2=0x7f64380a0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.963+0000 7f64460c7640 1 -- 172.21.15.69:0/819435519 shutdown_connections 2024-08-02T17:56:06.964 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:06.963+0000 7f64460c7640 1 -- 172.21.15.69:0/819435519 wait complete. 2024-08-02T17:56:06.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.971+0000 7f41363b3740 1 Processor -- start 2024-08-02T17:56:06.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.971+0000 7f41363b3740 1 -- start start 2024-08-02T17:56:06.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.971+0000 7f41363b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.971+0000 7f41363b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.972+0000 7f41363b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 0x558e050c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.972+0000 7f41363b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4110012b50 con 0x558e050db3b0 2024-08-02T17:56:06.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.972+0000 7f41363b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4110010420 con 0x558e050c4cf0 2024-08-02T17:56:06.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.972+0000 7f41363b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f411000e290 con 0x558e050cf5b0 2024-08-02T17:56:06.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4132552640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 0x558e050c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4132552640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 0x558e050c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60046/0 (socket says 172.21.15.69:60046) 2024-08-02T17:56:06.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4132552640 1 -- 172.21.15.69:0/3084906677 learned_addr learned my addr 172.21.15.69:0/3084906677 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4131d51640 1 --2- 172.21.15.69:0/3084906677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4131d51640 1 -- 172.21.15.69:0/3084906677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 msgr2=0x558e050cb0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:56:06.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4131d51640 1 --2- 172.21.15.69:0/3084906677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050cb0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4131d51640 1 -- 172.21.15.69:0/3084906677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 msgr2=0x558e050c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4131d51640 1 --2- 172.21.15.69:0/3084906677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 0x558e050c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.973+0000 7f4131d51640 1 -- 172.21.15.69:0/3084906677 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e050c11a0 con 0x558e050c4cf0 2024-08-02T17:56:06.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.974+0000 7f4131d51640 1 --2- 172.21.15.69:0/3084906677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cab60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x558e050e2b10 tx=0x7f412402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db068518a601b562 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.974+0000 7f41177fe640 1 -- 172.21.15.69:0/3084906677 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124040040 con 0x558e050c4cf0 2024-08-02T17:56:06.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.974+0000 7f41177fe640 1 -- 172.21.15.69:0/3084906677 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412404c040 con 0x558e050c4cf0 2024-08-02T17:56:06.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.975+0000 7f41363b3740 1 -- 172.21.15.69:0/3084906677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 msgr2=0x558e050cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:06.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.975+0000 7f41363b3740 1 --2- 172.21.15.69:0/3084906677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cab60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x558e050e2b10 tx=0x7f412402a3f0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.975+0000 7f41363b3740 1 -- 172.21.15.69:0/3084906677 shutdown_connections 2024-08-02T17:56:06.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.975+0000 7f41363b3740 1 --2- 172.21.15.69:0/3084906677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 0x558e050c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.975+0000 7f41363b3740 1 --2- 172.21.15.69:0/3084906677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.975+0000 7f41363b3740 1 --2- 172.21.15.69:0/3084906677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050cab60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.975+0000 7f41363b3740 1 -- 172.21.15.69:0/3084906677 >> 172.21.15.69:0/3084906677 conn(0x558e050c0960 msgr2=0x558e050c4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:06.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.982+0000 7f41363b3740 1 -- 172.21.15.69:0/3084906677 shutdown_connections 2024-08-02T17:56:06.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.986+0000 7f41363b3740 1 -- 172.21.15.69:0/3084906677 wait complete. 2024-08-02T17:56:06.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.986+0000 7f41363b3740 1 -- start start 2024-08-02T17:56:06.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.986+0000 7f41363b3740 1 Processor -- start 2024-08-02T17:56:06.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f41363b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050d7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f41363b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f41363b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 0x558e050c9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:06.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f41363b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f411000e8e0 con 0x558e050db3b0 2024-08-02T17:56:06.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f41363b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f411000fd50 con 0x558e050c4cf0 2024-08-02T17:56:06.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f41363b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f411000efb0 con 0x558e050cf5b0 2024-08-02T17:56:06.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f41363b3740 10 client.0 _ll_register_callbacks cb 0x7f412809c6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:06.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f4131550640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:06.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f4131550640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050d7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:58796/0 (socket says 172.21.15.69:58796) 2024-08-02T17:56:06.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.987+0000 7f4131550640 1 -- 172.21.15.69:0/2849618566 learned_addr learned my addr 172.21.15.69:0/2849618566 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:06.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.989+0000 7f4131550640 1 -- 172.21.15.69:0/2849618566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 msgr2=0x558e050d7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.989+0000 7f4131550640 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050d7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.989+0000 7f4131550640 1 -- 172.21.15.69:0/2849618566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 msgr2=0x558e050c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:06.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.989+0000 7f4131550640 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 0x558e050c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:06.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.989+0000 7f4131550640 1 -- 172.21.15.69:0/2849618566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e050c9a50 con 0x558e050cf5b0 2024-08-02T17:56:06.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.992+0000 7f4131550640 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050d7a60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x558e050c17c0 tx=0x7f411c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdaaee0fcb3ed6f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:06.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.992+0000 7f41157fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:06.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.995+0000 7f41157fa640 1 -- 172.21.15.69:0/2849618566 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c004460 con 0x558e050cf5b0 2024-08-02T17:56:06.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.996+0000 7f41363b3740 1 -- 172.21.15.69:0/2849618566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e04fae110 con 0x558e050cf5b0 2024-08-02T17:56:06.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.996+0000 7f41157fa640 1 -- 172.21.15.69:0/2849618566 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7f411c005020 con 0x558e050cf5b0 2024-08-02T17:56:06.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.996+0000 7f41157fa640 1 -- 172.21.15.69:0/2849618566 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f411c022050 con 0x558e050cf5b0 2024-08-02T17:56:07.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.999+0000 7f41363b3740 10 client.27133 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:07.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:06.999+0000 7f41363b3740 1 -- 172.21.15.69:0/2849618566 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558e050d0350 con 0x558e050cf5b0 2024-08-02T17:56:07.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.003+0000 7f41157fa640 1 -- 172.21.15.69:0/2849618566 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x558e04fae110 con 0x558e050cf5b0 2024-08-02T17:56:07.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.003+0000 7f41157fa640 1 -- 172.21.15.69:0/2849618566 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f411c00f040 con 0x558e050cf5b0 2024-08-02T17:56:07.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.003+0000 7f41363b3740 10 client.27133 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:07.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.003+0000 7f41363b3740 10 client.27133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:07.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.003+0000 7f41363b3740 20 client.27133 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:07.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.003+0000 7f41363b3740 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558e0515bc90 0x558e0515e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:07.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.003+0000 7f41363b3740 4 client.27133 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:56:07.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.003+0000 7f41363b3740 1 -- 172.21.15.69:0/2849618566 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558e04ffaeb0 con 0x558e0515bc90 2024-08-02T17:56:07.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.008+0000 7f4131d51640 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558e0515bc90 0x558e0515e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:07.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.023+0000 7f4131d51640 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558e0515bc90 0x558e0515e0a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e82d553853492c server_cookie=66267eed3a15a888 in_seq=0 out_seq=0 2024-08-02T17:56:07.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.024+0000 7f41157fa640 10 client.27133 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:07.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.025+0000 7f41157fa640 1 -- 172.21.15.69:0/2849618566 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f412404c080 con 0x558e0515bc90 2024-08-02T17:56:07.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.025+0000 7f41157fa640 10 client.27133 handle_command_reply: tid=0 2024-08-02T17:56:07.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.025+0000 7f41363b3740 1 client.27133 shutdown 2024-08-02T17:56:07.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.026+0000 7f41363b3740 1 -- 172.21.15.69:0/2849618566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 msgr2=0x558e050d7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:07.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.026+0000 7f41363b3740 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050d7a60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x558e050c17c0 tx=0x7f411c010800 comp rx=0 tx=0).stop 2024-08-02T17:56:07.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.029+0000 7f41363b3740 1 -- 172.21.15.69:0/2849618566 shutdown_connections 2024-08-02T17:56:07.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.029+0000 7f41363b3740 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558e050db3b0 0x558e050c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.029+0000 7f41363b3740 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558e0515bc90 0x558e0515e0a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.029+0000 7f41363b3740 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558e050cf5b0 0x558e050d7a60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.029+0000 7f41363b3740 1 --2- 172.21.15.69:0/2849618566 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558e050c4cf0 0x558e050d7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.029+0000 7f41363b3740 1 -- 172.21.15.69:0/2849618566 >> 172.21.15.69:0/2849618566 conn(0x558e050c0960 msgr2=0x558e050c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:07.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.030+0000 7f41157fa640 0 client.27133 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:07.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.030+0000 7f41363b3740 1 -- 172.21.15.69:0/2849618566 shutdown_connections 2024-08-02T17:56:07.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.030+0000 7f41363b3740 1 -- 172.21.15.69:0/2849618566 wait complete. 2024-08-02T17:56:07.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.031+0000 7f41363b3740 20 client.27133 trim_cache size 0 max 16384 2024-08-02T17:56:07.032 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:56:07.032 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:56:07.032 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "a104d352-cfd7-4e2d-8bbc-57826b3186ce", 2024-08-02T17:56:07.032 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:56:07.032 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:56:07.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.035+0000 7f4132d53640 1 -- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41180a2f00 msgr2=0x7f41180a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:07.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.035+0000 7f4132d53640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41180a2f00 0x7f41180a53c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f412402a960 tx=0x7f4124005fd0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.035+0000 7f4132d53640 1 -- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c06cda0 msgr2=0x7f412c06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:07.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.035+0000 7f4132d53640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c06cda0 0x7f412c06d250 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f412800b9e0 tx=0x7f412800bea0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.037+0000 7f4132d53640 1 -- 172.21.15.69:0/3629521665 shutdown_connections 2024-08-02T17:56:07.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.037+0000 7f4132d53640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f412c06cda0 0x7f412c06d250 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.037+0000 7f4132d53640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f41180a2f00 0x7f41180a53c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.037+0000 7f4132d53640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f412c1005f0 0x7f412c06c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.037+0000 7f4132d53640 1 --2- 172.21.15.69:0/3629521665 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f412c0ffc70 0x7f412c0737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:07.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.037+0000 7f4132d53640 1 -- 172.21.15.69:0/3629521665 >> 172.21.15.69:0/3629521665 conn(0x7f412c076da0 msgr2=0x7f412c102260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:07.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.038+0000 7f4132d53640 1 -- 172.21.15.69:0/3629521665 shutdown_connections 2024-08-02T17:56:07.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:07.038+0000 7f4132d53640 1 -- 172.21.15.69:0/3629521665 wait complete. 2024-08-02T17:56:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:07 smithi077 ceph-mon[37108]: pgmap v300: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 768 op/s 2024-08-02T17:56:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:07 smithi069 ceph-mon[32746]: pgmap v300: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 768 op/s 2024-08-02T17:56:07.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:07 smithi083 ceph-mon[37031]: pgmap v300: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 6.6 MiB/s wr, 768 op/s 2024-08-02T17:56:07.764 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:56:08.193 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:08.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:08 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:56:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:08 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:56:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:08 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:56:08.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:08 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:56:08.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:08 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:56:08.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:08 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:56:08.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:08 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:56:08.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:08 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:56:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:08 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:56:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:08 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:56:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:08 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:56:08.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:08 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:56:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 -- 172.21.15.69:0/3056029969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca28112760 msgr2=0x7fca28114b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 --2- 172.21.15.69:0/3056029969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca28112760 0x7fca28114b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca20009ca0 tx=0x7fca2002a8c0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 -- 172.21.15.69:0/3056029969 shutdown_connections 2024-08-02T17:56:08.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 --2- 172.21.15.69:0/3056029969 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca28112760 0x7fca28114b50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 --2- 172.21.15.69:0/3056029969 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca28076d70 0x7fca28112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 --2- 172.21.15.69:0/3056029969 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca28076450 0x7fca28076830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 -- 172.21.15.69:0/3056029969 >> 172.21.15.69:0/3056029969 conn(0x7fca280ff7a0 msgr2=0x7fca28101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 -- 172.21.15.69:0/3056029969 shutdown_connections 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.813+0000 7fca2ff56640 1 -- 172.21.15.69:0/3056029969 wait complete. 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.814+0000 7fca2ff56640 1 Processor -- start 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.814+0000 7fca2ff56640 1 -- start start 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.814+0000 7fca2ff56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca28076450 0x7fca281a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.814+0000 7fca2ff56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca28076d70 0x7fca281a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.814+0000 7fca2ff56640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca28112760 0x7fca2819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.814+0000 7fca2ff56640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca281182f0 con 0x7fca28112760 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.814+0000 7fca2ff56640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca28118170 con 0x7fca28076450 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.814+0000 7fca2ff56640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca28118470 con 0x7fca28076d70 2024-08-02T17:56:08.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2e4f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca28112760 0x7fca2819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2d4f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca28076d70 0x7fca281a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2e4f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca28112760 0x7fca2819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60074/0 (socket says 172.21.15.69:60074) 2024-08-02T17:56:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2e4f3640 1 -- 172.21.15.69:0/2259719441 learned_addr learned my addr 172.21.15.69:0/2259719441 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:08.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2dcf2640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca28076450 0x7fca281a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2d4f1640 1 -- 172.21.15.69:0/2259719441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca28076450 msgr2=0x7fca281a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca28076450 0x7fca281a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2d4f1640 1 -- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca28112760 msgr2=0x7fca2819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca28112760 0x7fca2819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2d4f1640 1 -- 172.21.15.69:0/2259719441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2819ed10 con 0x7fca28076d70 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.815+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca28076d70 0x7fca281a4960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9f0 tx=0x7fca2400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.816+0000 7fca1effd640 1 -- 172.21.15.69:0/2259719441 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2400ccc0 con 0x7fca28076d70 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.816+0000 7fca1effd640 1 -- 172.21.15.69:0/2259719441 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca24010910 con 0x7fca28076d70 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.816+0000 7fca2ff56640 1 -- 172.21.15.69:0/2259719441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2819f000 con 0x7fca28076d70 2024-08-02T17:56:08.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.816+0000 7fca1effd640 1 -- 172.21.15.69:0/2259719441 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240189c0 con 0x7fca28076d70 2024-08-02T17:56:08.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.816+0000 7fca2ff56640 1 -- 172.21.15.69:0/2259719441 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281ab2f0 con 0x7fca28076d70 2024-08-02T17:56:08.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.818+0000 7fca32354740 1 Processor -- start 2024-08-02T17:56:08.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.819+0000 7fca1effd640 1 -- 172.21.15.69:0/2259719441 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fca24010ac0 con 0x7fca28076d70 2024-08-02T17:56:08.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.820+0000 7fca1effd640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fca140a2df0 0x7fca140a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.820+0000 7fca1effd640 1 -- 172.21.15.69:0/2259719441 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7fca240cfc70 con 0x7fca28076d70 2024-08-02T17:56:08.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca2e4f3640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fca140a2df0 0x7fca140a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca32354740 1 -- start start 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca32354740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 0x55606ac0f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca32354740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac19eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca32354740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac1c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca32354740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55606ab74980 con 0x55606ac1a3f0 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca32354740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55606ab0ca70 con 0x55606ac0f210 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca32354740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55606ab70940 con 0x55606ac0fb30 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca2e4f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac1c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca2e4f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac1c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60078/0 (socket says 172.21.15.69:60078) 2024-08-02T17:56:08.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.821+0000 7fca2e4f3640 1 -- 172.21.15.69:0/4175231556 learned_addr learned my addr 172.21.15.69:0/4175231556 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca2e4f3640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fca140a2df0 0x7fca140a52b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fca20006010 tx=0x7fca20009a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/4175231556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac19eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca2e4f3640 1 -- 172.21.15.69:0/4175231556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 msgr2=0x55606ac19eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca2e4f3640 1 --2- 172.21.15.69:0/4175231556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac19eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca2e4f3640 1 -- 172.21.15.69:0/4175231556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 msgr2=0x55606ac0f5f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca2e4f3640 1 --2- 172.21.15.69:0/4175231556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 0x55606ac0f5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca2e4f3640 1 -- 172.21.15.69:0/4175231556 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606ab6f6e0 con 0x55606ac1a3f0 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca2e4f3640 1 --2- 172.21.15.69:0/4175231556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac1c7e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fca20031e20 tx=0x7fca20047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347cc9f6076bfc3f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:08.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca13fff640 1 -- 172.21.15.69:0/4175231556 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20042040 con 0x55606ac1a3f0 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca13fff640 1 -- 172.21.15.69:0/4175231556 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2004d070 con 0x55606ac1a3f0 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.822+0000 7fca13fff640 1 -- 172.21.15.69:0/4175231556 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2002f870 con 0x55606ac1a3f0 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.823+0000 7fca32354740 1 -- 172.21.15.69:0/4175231556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 msgr2=0x55606ac1c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.823+0000 7fca32354740 1 --2- 172.21.15.69:0/4175231556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac1c7e0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fca20031e20 tx=0x7fca20047000 comp rx=0 tx=0).stop 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.823+0000 7fca32354740 1 -- 172.21.15.69:0/4175231556 shutdown_connections 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.823+0000 7fca32354740 1 --2- 172.21.15.69:0/4175231556 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac1c7e0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.823+0000 7fca32354740 1 --2- 172.21.15.69:0/4175231556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac19eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.823+0000 7fca32354740 1 --2- 172.21.15.69:0/4175231556 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 0x55606ac0f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.823+0000 7fca32354740 1 -- 172.21.15.69:0/4175231556 >> 172.21.15.69:0/4175231556 conn(0x55606ac0b7a0 msgr2=0x55606ac0bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:08.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.823+0000 7fca32354740 1 -- 172.21.15.69:0/4175231556 shutdown_connections 2024-08-02T17:56:08.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 -- 172.21.15.69:0/4175231556 wait complete. 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 -- start start 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 Processor -- start 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 0x55606ac172d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac17810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac17d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55606ac1cfc0 con 0x55606ac1a3f0 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55606ab2faa0 con 0x55606ac0f210 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55606ac1d140 con 0x55606ac0fb30 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.825+0000 7fca32354740 10 client.0 _ll_register_callbacks cb 0x55606ab1ee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:08.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2d4f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac17810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2d4f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac17810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:58820/0 (socket says 172.21.15.69:58820) 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2d4f1640 1 -- 172.21.15.69:0/92311846 learned_addr learned my addr 172.21.15.69:0/92311846 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2dcf2640 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 0x55606ac172d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2e4f3640 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac17d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2dcf2640 1 -- 172.21.15.69:0/92311846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 msgr2=0x55606ac17810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2dcf2640 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac17810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2dcf2640 1 -- 172.21.15.69:0/92311846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 msgr2=0x55606ac17d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2dcf2640 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac17d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2dcf2640 1 -- 172.21.15.69:0/92311846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606ab13650 con 0x55606ac0f210 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca2dcf2640 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 0x55606ac172d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fca1800bd30 tx=0x7fca1800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bafed3c7740b1b2b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca117fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca117fa640 1 -- 172.21.15.69:0/92311846 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18015070 con 0x55606ac0f210 2024-08-02T17:56:08.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca117fa640 1 -- 172.21.15.69:0/92311846 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7fca18011850 con 0x55606ac0f210 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.826+0000 7fca32354740 1 -- 172.21.15.69:0/92311846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca2004d070 con 0x55606ac0f210 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.827+0000 7fca117fa640 1 -- 172.21.15.69:0/92311846 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca18023050 con 0x55606ac0f210 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.827+0000 7fca32354740 10 client.27323 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.827+0000 7fca32354740 1 -- 172.21.15.69:0/92311846 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55606a8e63c0 con 0x55606ac0f210 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.827+0000 7fca117fa640 1 -- 172.21.15.69:0/92311846 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca2004d070 con 0x55606ac0f210 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.827+0000 7fca117fa640 1 -- 172.21.15.69:0/92311846 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca1801abd0 con 0x55606ac0f210 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.828+0000 7fca32354740 10 client.27323 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.828+0000 7fca32354740 10 client.27323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.828+0000 7fca32354740 20 client.27323 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.828+0000 7fca32354740 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55606ac29f60 0x55606ac2c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.828+0000 7fca32354740 4 client.27323 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.828+0000 7fca32354740 1 -- 172.21.15.69:0/92311846 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55606ab81ef0 con 0x55606ac29f60 2024-08-02T17:56:08.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.828+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55606ac29f60 0x55606ac2c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.829+0000 7fca117fa640 10 client.27323 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:08.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.829+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55606ac29f60 0x55606ac2c370 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3293b557347ea753 server_cookie=e4e773149ad1923 in_seq=0 out_seq=0 2024-08-02T17:56:08.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.829+0000 7fca117fa640 1 -- 172.21.15.69:0/92311846 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55606ab81ef0 con 0x55606ac29f60 2024-08-02T17:56:08.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.829+0000 7fca117fa640 10 client.27323 handle_command_reply: tid=0 2024-08-02T17:56:08.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.829+0000 7fca32354740 1 client.27323 shutdown 2024-08-02T17:56:08.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 -- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 msgr2=0x55606ac172d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 0x55606ac172d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fca1800bd30 tx=0x7fca1800f730 comp rx=0 tx=0).stop 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 -- 172.21.15.69:0/92311846 shutdown_connections 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac1a3f0 0x55606ac17d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55606ac29f60 0x55606ac2c370 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac17810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 --2- 172.21.15.69:0/92311846 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac0f210 0x55606ac172d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 -- 172.21.15.69:0/92311846 >> 172.21.15.69:0/92311846 conn(0x55606ac0b7a0 msgr2=0x55606ac11480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca117fa640 0 client.27323 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 -- 172.21.15.69:0/92311846 shutdown_connections 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 1 -- 172.21.15.69:0/92311846 wait complete. 2024-08-02T17:56:08.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.830+0000 7fca32354740 20 client.27323 trim_cache size 0 max 16384 2024-08-02T17:56:08.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.861+0000 7fca32354740 1 Processor -- start 2024-08-02T17:56:08.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.862+0000 7fca32354740 1 -- start start 2024-08-02T17:56:08.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.862+0000 7fca32354740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac17ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.862+0000 7fca32354740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac31c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.862+0000 7fca32354740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 0x55606ac32180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.862+0000 7fca32354740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca281b7ea0 con 0x55606ac0f210 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.862+0000 7fca32354740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca281b5770 con 0x55606ac1a3f0 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.862+0000 7fca32354740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca281b35e0 con 0x55606ac0fb30 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.863+0000 7fca2dcf2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac17ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.863+0000 7fca2e4f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 0x55606ac32180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.863+0000 7fca2dcf2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac17ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:60096/0 (socket says 172.21.15.69:60096) 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.863+0000 7fca2dcf2640 1 -- 172.21.15.69:0/3448034871 learned_addr learned my addr 172.21.15.69:0/3448034871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:08.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.863+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/3448034871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac31c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.864+0000 7fca2d4f1640 1 -- 172.21.15.69:0/3448034871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 msgr2=0x55606ac32180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:08.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.864+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/3448034871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 0x55606ac32180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.864+0000 7fca2d4f1640 1 -- 172.21.15.69:0/3448034871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 msgr2=0x55606ac17ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:08.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.864+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/3448034871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac17ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.864+0000 7fca2d4f1640 1 -- 172.21.15.69:0/3448034871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606ac19d80 con 0x55606ac0fb30 2024-08-02T17:56:08.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.869+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/3448034871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac31c40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca24098690 tx=0x7fca24097f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8e90cfd550696b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:08.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.875+0000 7fca137fe640 1 -- 172.21.15.69:0/3448034871 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240b9070 con 0x55606ac0fb30 2024-08-02T17:56:08.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.875+0000 7fca137fe640 1 -- 172.21.15.69:0/3448034871 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca240a4930 con 0x55606ac0fb30 2024-08-02T17:56:08.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.876+0000 7fca32354740 1 -- 172.21.15.69:0/3448034871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 msgr2=0x55606ac31c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.876+0000 7fca32354740 1 --2- 172.21.15.69:0/3448034871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac31c40 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca24098690 tx=0x7fca24097f60 comp rx=0 tx=0).stop 2024-08-02T17:56:08.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.878+0000 7fca32354740 1 -- 172.21.15.69:0/3448034871 shutdown_connections 2024-08-02T17:56:08.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.878+0000 7fca32354740 1 --2- 172.21.15.69:0/3448034871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 0x55606ac32180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.878+0000 7fca32354740 1 --2- 172.21.15.69:0/3448034871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac31c40 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.878+0000 7fca32354740 1 --2- 172.21.15.69:0/3448034871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac17ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.878+0000 7fca32354740 1 -- 172.21.15.69:0/3448034871 >> 172.21.15.69:0/3448034871 conn(0x55606ac0b7a0 msgr2=0x55606ac163c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:08.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.884+0000 7fca32354740 1 -- 172.21.15.69:0/3448034871 shutdown_connections 2024-08-02T17:56:08.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.888+0000 7fca32354740 1 -- 172.21.15.69:0/3448034871 wait complete. 2024-08-02T17:56:08.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.888+0000 7fca32354740 1 -- start start 2024-08-02T17:56:08.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.888+0000 7fca32354740 1 Processor -- start 2024-08-02T17:56:08.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.889+0000 7fca32354740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac8d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.889+0000 7fca32354740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.889+0000 7fca32354740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 0x55606ac21380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.889+0000 7fca32354740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fca281b3c30 con 0x55606ac0f210 2024-08-02T17:56:08.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.889+0000 7fca32354740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fca281b50a0 con 0x55606ac1a3f0 2024-08-02T17:56:08.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.889+0000 7fca32354740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fca281b4300 con 0x55606ac0fb30 2024-08-02T17:56:08.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.889+0000 7fca32354740 10 client.0 _ll_register_callbacks cb 0x7fca18005f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:08.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.890+0000 7fca2d4f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.890+0000 7fca2d4f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac8d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:58850/0 (socket says 172.21.15.69:58850) 2024-08-02T17:56:08.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.890+0000 7fca2d4f1640 1 -- 172.21.15.69:0/2952465454 learned_addr learned my addr 172.21.15.69:0/2952465454 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:08.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.891+0000 7fca2e4f3640 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 0x55606ac21380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.890+0000 7fca2dcf2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac8d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.891+0000 7fca2d4f1640 1 -- 172.21.15.69:0/2952465454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 msgr2=0x55606ac21380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:08.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.891+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 0x55606ac21380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.891+0000 7fca2d4f1640 1 -- 172.21.15.69:0/2952465454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 msgr2=0x55606ac8d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:08.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.891+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac8d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.891+0000 7fca2d4f1640 1 -- 172.21.15.69:0/2952465454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606ac218c0 con 0x55606ac0fb30 2024-08-02T17:56:08.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.892+0000 7fca2d4f1640 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac8d6b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca2409bc50 tx=0x7fca24018f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2f9ab94bf10c4e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:08.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.894+0000 7fca117fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:08.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.894+0000 7fca117fa640 1 -- 172.21.15.69:0/2952465454 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca240b9050 con 0x55606ac0fb30 2024-08-02T17:56:08.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.894+0000 7fca32354740 1 -- 172.21.15.69:0/2952465454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca240a4930 con 0x55606ac0fb30 2024-08-02T17:56:08.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.894+0000 7fca117fa640 1 -- 172.21.15.69:0/2952465454 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9656+0+0 (secure 0 0 0) 0x7fca240cd170 con 0x55606ac0fb30 2024-08-02T17:56:08.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.902+0000 7fca117fa640 1 -- 172.21.15.69:0/2952465454 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fca240cd370 con 0x55606ac0fb30 2024-08-02T17:56:08.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.902+0000 7fca32354740 10 client.27148 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:08.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.902+0000 7fca32354740 1 -- 172.21.15.69:0/2952465454 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55606ac22bd0 con 0x55606ac0fb30 2024-08-02T17:56:08.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca117fa640 1 -- 172.21.15.69:0/2952465454 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca240a4930 con 0x55606ac0fb30 2024-08-02T17:56:08.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca117fa640 1 -- 172.21.15.69:0/2952465454 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fca2407ac20 con 0x55606ac0fb30 2024-08-02T17:56:08.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca32354740 10 client.27148 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:08.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca32354740 10 client.27148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:08.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca32354740 20 client.27148 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:08.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca32354740 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55606aca5c70 0x55606aca8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:08.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca32354740 4 client.27148 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:56:08.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca32354740 1 -- 172.21.15.69:0/2952465454 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55606ab7c400 con 0x55606aca5c70 2024-08-02T17:56:08.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.903+0000 7fca2dcf2640 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55606aca5c70 0x55606aca8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.924+0000 7fca117fa640 10 client.27148 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:08.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.924+0000 7fca2dcf2640 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55606aca5c70 0x55606aca8090 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc123529ae7aaa3f server_cookie=18260246ee2757f2 in_seq=0 out_seq=0 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca117fa640 1 -- 172.21.15.69:0/2952465454 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55606ab7c400 con 0x55606aca5c70 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca117fa640 10 client.27148 handle_command_reply: tid=0 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 client.27148 shutdown 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 -- 172.21.15.69:0/2952465454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 msgr2=0x55606ac8d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac8d6b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca2409bc50 tx=0x7fca24018f60 comp rx=0 tx=0).stop 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 -- 172.21.15.69:0/2952465454 shutdown_connections 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55606aca5c70 0x55606aca8090 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55606ac1a3f0 0x55606ac21380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55606ac0fb30 0x55606ac8d6b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 --2- 172.21.15.69:0/2952465454 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55606ac0f210 0x55606ac8d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 -- 172.21.15.69:0/2952465454 >> 172.21.15.69:0/2952465454 conn(0x55606ac0b7a0 msgr2=0x55606ac10c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:08.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca117fa640 0 client.27148 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:08.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 -- 172.21.15.69:0/2952465454 shutdown_connections 2024-08-02T17:56:08.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.925+0000 7fca32354740 1 -- 172.21.15.69:0/2952465454 wait complete. 2024-08-02T17:56:08.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.926+0000 7fca32354740 20 client.27148 trim_cache size 0 max 16384 2024-08-02T17:56:08.928 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout: "a104d352-cfd7-4e2d-8bbc-57826b3186ce": { 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "a104d352-cfd7-4e2d-8bbc-57826b3186ce", 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:08.929 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:56:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.930+0000 7fca2ff56640 1 -- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fca140a2df0 msgr2=0x7fca140a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.930+0000 7fca2ff56640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fca140a2df0 0x7fca140a52b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fca20006010 tx=0x7fca20009a20 comp rx=0 tx=0).stop 2024-08-02T17:56:08.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.930+0000 7fca2ff56640 1 -- 172.21.15.69:0/2259719441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca28076d70 msgr2=0x7fca281a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:08.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.930+0000 7fca2ff56640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca28076d70 0x7fca281a4960 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fca2400e9f0 tx=0x7fca2400eeb0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.932+0000 7fca2ff56640 1 -- 172.21.15.69:0/2259719441 shutdown_connections 2024-08-02T17:56:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.932+0000 7fca2ff56640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fca28112760 0x7fca2819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.932+0000 7fca2ff56640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fca140a2df0 0x7fca140a52b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.932+0000 7fca2ff56640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fca28076d70 0x7fca281a4960 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.932+0000 7fca2ff56640 1 --2- 172.21.15.69:0/2259719441 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fca28076450 0x7fca281a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:08.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.932+0000 7fca2ff56640 1 -- 172.21.15.69:0/2259719441 >> 172.21.15.69:0/2259719441 conn(0x7fca280ff7a0 msgr2=0x7fca28101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:08.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.934+0000 7fca2ff56640 1 -- 172.21.15.69:0/2259719441 shutdown_connections 2024-08-02T17:56:08.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:08.934+0000 7fca2ff56640 1 -- 172.21.15.69:0/2259719441 wait complete. 2024-08-02T17:56:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:09 smithi077 ceph-mon[37108]: pgmap v301: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 627 op/s 2024-08-02T17:56:09.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:09 smithi069 ceph-mon[32746]: pgmap v301: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 627 op/s 2024-08-02T17:56:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:09 smithi083 ceph-mon[37031]: pgmap v301: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.4 MiB/s wr, 627 op/s 2024-08-02T17:56:09.943 INFO:tasks.cephfs.filesystem:scrub status for tag:a104d352-cfd7-4e2d-8bbc-57826b3186ce - {'path': '/', 'tag': 'a104d352-cfd7-4e2d-8bbc-57826b3186ce', 'options': 'recursive,force'} 2024-08-02T17:56:09.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 540.0 s, 656.8 tps, lat 155.622 ms stddev 243.910 2024-08-02T17:56:10.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:10 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:56:10.422 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:10 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:56:10.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:10 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:56:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:11 smithi069 ceph-mon[32746]: pgmap v302: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.6 MiB/s wr, 567 op/s 2024-08-02T17:56:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:11 smithi083 ceph-mon[37031]: pgmap v302: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.6 MiB/s wr, 567 op/s 2024-08-02T17:56:11.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:11 smithi077 ceph-mon[37108]: pgmap v302: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.6 MiB/s wr, 567 op/s 2024-08-02T17:56:12.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:12 smithi069 ceph-mon[32746]: pgmap v303: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.4 MiB/s wr, 536 op/s 2024-08-02T17:56:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:12 smithi083 ceph-mon[37031]: pgmap v303: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.4 MiB/s wr, 536 op/s 2024-08-02T17:56:12.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:12 smithi077 ceph-mon[37108]: pgmap v303: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 4.4 MiB/s wr, 536 op/s 2024-08-02T17:56:14.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:14 smithi083 ceph-mon[37031]: pgmap v304: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 5.9 MiB/s wr, 780 op/s 2024-08-02T17:56:14.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:14 smithi077 ceph-mon[37108]: pgmap v304: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 5.9 MiB/s wr, 780 op/s 2024-08-02T17:56:14.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:14 smithi069 ceph-mon[32746]: pgmap v304: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 5.9 MiB/s wr, 780 op/s 2024-08-02T17:56:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 545.0 s, 843.2 tps, lat 119.760 ms stddev 156.315 2024-08-02T17:56:15.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:15 smithi077 ceph-mon[37108]: osdmap e103: 12 total, 12 up, 12 in 2024-08-02T17:56:15.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:15 smithi069 ceph-mon[32746]: osdmap e103: 12 total, 12 up, 12 in 2024-08-02T17:56:16.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:15 smithi083 ceph-mon[37031]: osdmap e103: 12 total, 12 up, 12 in 2024-08-02T17:56:16.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:16 smithi077 ceph-mon[37108]: pgmap v306: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 5.9 MiB/s wr, 780 op/s 2024-08-02T17:56:16.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:16 smithi069 ceph-mon[32746]: pgmap v306: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 5.9 MiB/s wr, 780 op/s 2024-08-02T17:56:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:16 smithi083 ceph-mon[37031]: pgmap v306: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 5.9 MiB/s wr, 780 op/s 2024-08-02T17:56:18.850 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:18 smithi077 ceph-mon[37108]: pgmap v307: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.7 MiB/s wr, 847 op/s 2024-08-02T17:56:18.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:18 smithi069 ceph-mon[32746]: pgmap v307: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.7 MiB/s wr, 847 op/s 2024-08-02T17:56:19.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:18 smithi083 ceph-mon[37031]: pgmap v307: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 6.7 MiB/s wr, 847 op/s 2024-08-02T17:56:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 550.0 s, 662.8 tps, lat 148.252 ms stddev 192.061 2024-08-02T17:56:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 -- 172.21.15.69:0/2677498550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e40101070 msgr2=0x7f6e4010d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2677498550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e40101070 0x7f6e4010d5a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009c10 tx=0x7f6e3802a6f0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 -- 172.21.15.69:0/2677498550 shutdown_connections 2024-08-02T17:56:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2677498550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e401006d0 0x7f6e40100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2677498550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e40101070 0x7f6e4010d5a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2677498550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e401066e0 0x7f6e40106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 -- 172.21.15.69:0/2677498550 >> 172.21.15.69:0/2677498550 conn(0x7f6e400776d0 msgr2=0x7f6e400fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 -- 172.21.15.69:0/2677498550 shutdown_connections 2024-08-02T17:56:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.108+0000 7f6e467fb640 1 -- 172.21.15.69:0/2677498550 wait complete. 2024-08-02T17:56:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.109+0000 7f6e467fb640 1 Processor -- start 2024-08-02T17:56:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.109+0000 7f6e467fb640 1 -- start start 2024-08-02T17:56:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.109+0000 7f6e467fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e401006d0 0x7f6e40195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.109+0000 7f6e467fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e40101070 0x7f6e401963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.109+0000 7f6e467fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e401066e0 0x7f6e4019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.109+0000 7f6e467fb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4010fa20 con 0x7f6e401006d0 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.109+0000 7f6e467fb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4010f8a0 con 0x7f6e401066e0 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.109+0000 7f6e467fb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4010fba0 con 0x7f6e40101070 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e401066e0 0x7f6e4019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e40101070 0x7f6e401963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e401066e0 0x7f6e4019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39430/0 (socket says 172.21.15.69:39430) 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e401006d0 0x7f6e40195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 -- 172.21.15.69:0/2181526324 learned_addr learned my addr 172.21.15.69:0/2181526324 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 -- 172.21.15.69:0/2181526324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e40101070 msgr2=0x7f6e401963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e40101070 0x7f6e401963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 -- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e401006d0 msgr2=0x7f6e40195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e401006d0 0x7f6e40195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 -- 172.21.15.69:0/2181526324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4019ae60 con 0x7f6e401066e0 2024-08-02T17:56:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e44d98640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e401066e0 0x7f6e4019a730 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6e3802ac10 tx=0x7f6e380305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e3d7fa640 1 -- 172.21.15.69:0/2181526324 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38041020 con 0x7f6e401066e0 2024-08-02T17:56:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e467fb640 1 -- 172.21.15.69:0/2181526324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e4019b0f0 con 0x7f6e401066e0 2024-08-02T17:56:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e3d7fa640 1 -- 172.21.15.69:0/2181526324 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e38003e50 con 0x7f6e401066e0 2024-08-02T17:56:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e3d7fa640 1 -- 172.21.15.69:0/2181526324 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38036040 con 0x7f6e401066e0 2024-08-02T17:56:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.110+0000 7f6e467fb640 1 -- 172.21.15.69:0/2181526324 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e401a6df0 con 0x7f6e401066e0 2024-08-02T17:56:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.113+0000 7f6e3d7fa640 1 -- 172.21.15.69:0/2181526324 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6e38044020 con 0x7f6e401066e0 2024-08-02T17:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e3d7fa640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e280a2d70 0x7f6e280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e4962b740 1 Processor -- start 2024-08-02T17:56:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e3d7fa640 1 -- 172.21.15.69:0/2181526324 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f6e380f0570 con 0x7f6e401066e0 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e4962b740 1 -- start start 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e280a2d70 0x7f6e280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876f7eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876f89430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 0x557876f8bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e4962b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557876ee29b0 con 0x557876f89970 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e4962b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557876e8a800 con 0x557876f7f0d0 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.114+0000 7f6e4962b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557876ede970 con 0x557876f7e7b0 2024-08-02T17:56:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876f7eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876f7eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50814/0 (socket says 172.21.15.69:50814) 2024-08-02T17:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3ffff640 1 -- 172.21.15.69:0/3989675039 learned_addr learned my addr 172.21.15.69:0/3989675039 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/3989675039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876f89430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e280a2d70 0x7f6e280a5230 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000dac0 tx=0x7f6e3000f040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3f7fe640 1 -- 172.21.15.69:0/3989675039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 msgr2=0x557876f7eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/3989675039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876f7eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3f7fe640 1 -- 172.21.15.69:0/3989675039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 msgr2=0x557876f8bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/3989675039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 0x557876f8bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.115+0000 7f6e3f7fe640 1 -- 172.21.15.69:0/3989675039 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557876bf6210 con 0x557876f7f0d0 2024-08-02T17:56:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.116+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/3989675039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876f89430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009530 tx=0x7f6e34011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45dfc1a832c840d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.116+0000 7f6e267fc640 1 -- 172.21.15.69:0/3989675039 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34004020 con 0x557876f7f0d0 2024-08-02T17:56:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.117+0000 7f6e267fc640 1 -- 172.21.15.69:0/3989675039 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e340041c0 con 0x557876f7f0d0 2024-08-02T17:56:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.117+0000 7f6e267fc640 1 -- 172.21.15.69:0/3989675039 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e34019970 con 0x557876f7f0d0 2024-08-02T17:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.118+0000 7f6e4962b740 1 -- 172.21.15.69:0/3989675039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 msgr2=0x557876f89430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.118+0000 7f6e4962b740 1 --2- 172.21.15.69:0/3989675039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876f89430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6e34009530 tx=0x7f6e34011810 comp rx=0 tx=0).stop 2024-08-02T17:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.118+0000 7f6e4962b740 1 -- 172.21.15.69:0/3989675039 shutdown_connections 2024-08-02T17:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.118+0000 7f6e4962b740 1 --2- 172.21.15.69:0/3989675039 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 0x557876f8bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.118+0000 7f6e4962b740 1 --2- 172.21.15.69:0/3989675039 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876f89430 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.118+0000 7f6e4962b740 1 --2- 172.21.15.69:0/3989675039 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876f7eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.119 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.118+0000 7f6e4962b740 1 -- 172.21.15.69:0/3989675039 >> 172.21.15.69:0/3989675039 conn(0x557876f7ad20 msgr2=0x557876f7b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.118+0000 7f6e4962b740 1 -- 172.21.15.69:0/3989675039 shutdown_connections 2024-08-02T17:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.119+0000 7f6e4962b740 1 -- 172.21.15.69:0/3989675039 wait complete. 2024-08-02T17:56:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.119+0000 7f6e4962b740 1 -- start start 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.119+0000 7f6e4962b740 1 Processor -- start 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7e7b0 0x557876f86830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7f0d0 0x557876f86d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f89970 0x557876f872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e4962b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557876f9d5f0 con 0x557876f7f0d0 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e4962b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557876e9e500 con 0x557876f7e7b0 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e4962b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557876f9d770 con 0x557876f89970 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e4962b740 10 client.0 _ll_register_callbacks cb 0x557876bf5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7e7b0 0x557876f86830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7e7b0 0x557876f86830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39456/0 (socket says 172.21.15.69:39456) 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 -- 172.21.15.69:0/2525532463 learned_addr learned my addr 172.21.15.69:0/2525532463 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e44d98640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f89970 0x557876f872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7f0d0 0x557876f86d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 -- 172.21.15.69:0/2525532463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f89970 msgr2=0x557876f872b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f89970 0x557876f872b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 -- 172.21.15.69:0/2525532463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7f0d0 msgr2=0x557876f86d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7f0d0 0x557876f86d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 -- 172.21.15.69:0/2525532463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557876cc2620 con 0x557876f7e7b0 2024-08-02T17:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e44d98640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f89970 0x557876f872b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7f0d0 0x557876f86d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:56:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7e7b0 0x557876f86830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6e30019b70 tx=0x7f6e300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f6f39fb60d3cd47 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:56:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.120+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2525532463 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30017b50 con 0x557876f7e7b0 2024-08-02T17:56:20.125 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 1 -- 172.21.15.69:0/2525532463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e340041c0 con 0x557876f7e7b0 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2525532463 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f6e30022040 con 0x557876f7e7b0 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2525532463 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6e30032050 con 0x557876f7e7b0 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 10 client.27338 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 1 -- 172.21.15.69:0/2525532463 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557876fb1af0 con 0x557876f7e7b0 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2525532463 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e340041c0 con 0x557876f7e7b0 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2525532463 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6e30018370 con 0x557876f7e7b0 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 10 client.27338 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 10 client.27338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 20 client.27338 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:20.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557876fb3020 0x557876fb5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 4 client.27338 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e4962b740 1 -- 172.21.15.69:0/2525532463 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557876ef00e0 con 0x557876fb3020 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.121+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557876fb3020 0x557876fb5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.122+0000 7f6e1f7fe640 10 client.27338 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.122+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557876fb3020 0x557876fb5430 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=361b769a31a212f6 server_cookie=697b572d9b453018 in_seq=0 out_seq=0 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2525532463 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557876ef00e0 con 0x557876fb3020 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e1f7fe640 10 client.27338 handle_command_reply: tid=0 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 client.27338 shutdown 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 -- 172.21.15.69:0/2525532463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7e7b0 msgr2=0x557876f86830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7e7b0 0x557876f86830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6e30019b70 tx=0x7f6e300046c0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 -- 172.21.15.69:0/2525532463 shutdown_connections 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7f0d0 0x557876f86d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.127 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557876fb3020 0x557876fb5430 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f89970 0x557876f872b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2525532463 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7e7b0 0x557876f86830 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 -- 172.21.15.69:0/2525532463 >> 172.21.15.69:0/2525532463 conn(0x557876f7ad20 msgr2=0x557876f80a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e1f7fe640 0 client.27338 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 -- 172.21.15.69:0/2525532463 shutdown_connections 2024-08-02T17:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 1 -- 172.21.15.69:0/2525532463 wait complete. 2024-08-02T17:56:20.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.123+0000 7f6e4962b740 20 client.27338 trim_cache size 0 max 16384 2024-08-02T17:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.143+0000 7f6e4962b740 1 Processor -- start 2024-08-02T17:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.143+0000 7f6e4962b740 1 -- start start 2024-08-02T17:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.143+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7e7b0 0x557876f88690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.143+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7f0d0 0x557876f870e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.143+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f89970 0x557876fbad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.143+0000 7f6e4962b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40073e50 con 0x557876f7e7b0 2024-08-02T17:56:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.143+0000 7f6e4962b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40071720 con 0x557876f89970 2024-08-02T17:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.143+0000 7f6e4962b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4006f570 con 0x557876f7f0d0 2024-08-02T17:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7f0d0 0x557876f870e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7e7b0 0x557876f88690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7f0d0 0x557876f870e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50836/0 (socket says 172.21.15.69:50836) 2024-08-02T17:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3f7fe640 1 -- 172.21.15.69:0/237636362 learned_addr learned my addr 172.21.15.69:0/237636362 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:20.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e44d98640 1 --2- 172.21.15.69:0/237636362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f89970 0x557876fbad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3ffff640 1 -- 172.21.15.69:0/237636362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7f0d0 msgr2=0x557876f870e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/237636362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7f0d0 0x557876f870e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3ffff640 1 -- 172.21.15.69:0/237636362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f89970 msgr2=0x557876fbad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/237636362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f89970 0x557876fbad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3ffff640 1 -- 172.21.15.69:0/237636362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557876f7ab50 con 0x557876f7e7b0 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.144+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/237636362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7e7b0 0x557876f88690 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000f010 tx=0x7f6e30009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9cc2b066f45e63 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.145+0000 7f6e25ffb640 1 -- 172.21.15.69:0/237636362 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3003b070 con 0x557876f7e7b0 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.145+0000 7f6e25ffb640 1 -- 172.21.15.69:0/237636362 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e30017790 con 0x557876f7e7b0 2024-08-02T17:56:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.145+0000 7f6e25ffb640 1 -- 172.21.15.69:0/237636362 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30017e40 con 0x557876f7e7b0 2024-08-02T17:56:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.146+0000 7f6e4962b740 1 -- 172.21.15.69:0/237636362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7e7b0 msgr2=0x557876f88690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.146+0000 7f6e4962b740 1 --2- 172.21.15.69:0/237636362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7e7b0 0x557876f88690 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000f010 tx=0x7f6e30009260 comp rx=0 tx=0).stop 2024-08-02T17:56:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 -- 172.21.15.69:0/237636362 shutdown_connections 2024-08-02T17:56:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 --2- 172.21.15.69:0/237636362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f89970 0x557876fbad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 --2- 172.21.15.69:0/237636362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7f0d0 0x557876f870e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 --2- 172.21.15.69:0/237636362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f7e7b0 0x557876f88690 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 -- 172.21.15.69:0/237636362 >> 172.21.15.69:0/237636362 conn(0x557876f7ad20 msgr2=0x557876f87f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 -- 172.21.15.69:0/237636362 shutdown_connections 2024-08-02T17:56:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 -- 172.21.15.69:0/237636362 wait complete. 2024-08-02T17:56:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 -- start start 2024-08-02T17:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.147+0000 7f6e4962b740 1 Processor -- start 2024-08-02T17:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876fab290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876fab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e4962b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 0x557876f82c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e4962b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6e4006fbe0 con 0x557876f89970 2024-08-02T17:56:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876fab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876fab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876fab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39478/0 (socket says 172.21.15.69:39478) 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3f7fe640 1 -- 172.21.15.69:0/2927138124 learned_addr learned my addr 172.21.15.69:0/2927138124 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e44d98640 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 0x557876f82c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40071050 con 0x557876f7f0d0 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6e400702b0 con 0x557876f7e7b0 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e4962b740 10 client.0 _ll_register_callbacks cb 0x7f6e30009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3ffff640 1 -- 172.21.15.69:0/2927138124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 msgr2=0x557876fab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876fab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3ffff640 1 -- 172.21.15.69:0/2927138124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 msgr2=0x557876f82c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 0x557876f82c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3ffff640 1 -- 172.21.15.69:0/2927138124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557876f83180 con 0x557876f7e7b0 2024-08-02T17:56:20.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876fab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e3ffff640 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876fab290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e30019b40 tx=0x7f6e30017c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=767e1664c5e9e9fc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.148+0000 7f6e1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.149+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2927138124 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30044060 con 0x557876f7e7b0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.149+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e30017790 con 0x557876f7e7b0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.149+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2927138124 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f6e30004870 con 0x557876f7e7b0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.149+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2927138124 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6e3004b050 con 0x557876f7e7b0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.149+0000 7f6e4962b740 10 client.27166 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.149+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55787701d480 con 0x557876f7e7b0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.149+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2927138124 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e30017790 con 0x557876f7e7b0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.150+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2927138124 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6e30018980 con 0x557876f7e7b0 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.150+0000 7f6e4962b740 10 client.27166 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.150+0000 7f6e4962b740 10 client.27166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:20.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.150+0000 7f6e4962b740 20 client.27166 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.150+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55787701dab0 0x55787701fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.150+0000 7f6e4962b740 4 client.27166 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:56:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.150+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557876eea430 con 0x55787701dab0 2024-08-02T17:56:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.150+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55787701dab0 0x55787701fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.151+0000 7f6e1f7fe640 10 client.27166 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:20.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.151+0000 7f6e3f7fe640 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55787701dab0 0x55787701fed0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f857d0ed394fbe07 server_cookie=eb2505d46ab33043 in_seq=0 out_seq=0 2024-08-02T17:56:20.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e1f7fe640 1 -- 172.21.15.69:0/2927138124 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557876eea430 con 0x55787701dab0 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e1f7fe640 10 client.27166 handle_command_reply: tid=0 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e4962b740 1 client.27166 shutdown 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 msgr2=0x557876fab290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876fab290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6e30019b40 tx=0x7f6e30017c40 comp rx=0 tx=0).stop 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 shutdown_connections 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557876f89970 0x557876f82c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55787701dab0 0x55787701fed0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.545+0000 7f6e1f7fe640 0 client.27166 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.546+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557876f7f0d0 0x557876fab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.546+0000 7f6e4962b740 1 --2- 172.21.15.69:0/2927138124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557876f7e7b0 0x557876fab290 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.546+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 >> 172.21.15.69:0/2927138124 conn(0x557876f7ad20 msgr2=0x557876f79e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.546+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 shutdown_connections 2024-08-02T17:56:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.546+0000 7f6e4962b740 1 -- 172.21.15.69:0/2927138124 wait complete. 2024-08-02T17:56:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.546+0000 7f6e4962b740 20 client.27166 trim_cache size 0 max 16384 2024-08-02T17:56:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:56:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:56:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:56:20.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:56:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 -- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e280a2d70 msgr2=0x7f6e280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e280a2d70 0x7f6e280a5230 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000dac0 tx=0x7f6e3000f040 comp rx=0 tx=0).stop 2024-08-02T17:56:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 -- 172.21.15.69:0/2181526324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e401066e0 msgr2=0x7f6e4019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e401066e0 0x7f6e4019a730 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6e3802ac10 tx=0x7f6e380305d0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 -- 172.21.15.69:0/2181526324 shutdown_connections 2024-08-02T17:56:20.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6e280a2d70 0x7f6e280a5230 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6e401066e0 0x7f6e4019a730 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6e40101070 0x7f6e401963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 --2- 172.21.15.69:0/2181526324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6e401006d0 0x7f6e40195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 -- 172.21.15.69:0/2181526324 >> 172.21.15.69:0/2181526324 conn(0x7f6e400776d0 msgr2=0x7f6e400fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 -- 172.21.15.69:0/2181526324 shutdown_connections 2024-08-02T17:56:20.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:20.547+0000 7f6e467fb640 1 -- 172.21.15.69:0/2181526324 wait complete. 2024-08-02T17:56:20.750 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:20 smithi069 ceph-mon[32746]: pgmap v308: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.3 MiB/s wr, 884 op/s 2024-08-02T17:56:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:20 smithi083 ceph-mon[37031]: pgmap v308: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.3 MiB/s wr, 884 op/s 2024-08-02T17:56:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:20 smithi077 ceph-mon[37108]: pgmap v308: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.3 MiB/s wr, 884 op/s 2024-08-02T17:56:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:22 smithi069 ceph-mon[32746]: pgmap v309: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.3 MiB/s wr, 884 op/s 2024-08-02T17:56:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:22 smithi083 ceph-mon[37031]: pgmap v309: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.3 MiB/s wr, 884 op/s 2024-08-02T17:56:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:22 smithi077 ceph-mon[37108]: pgmap v309: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 7.3 MiB/s wr, 884 op/s 2024-08-02T17:56:24.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:24 smithi069 ceph-mon[32746]: pgmap v310: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 683 op/s 2024-08-02T17:56:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 555.0 s, 591.0 tps, lat 170.801 ms stddev 202.786 2024-08-02T17:56:25.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:24 smithi083 ceph-mon[37031]: pgmap v310: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 683 op/s 2024-08-02T17:56:25.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:24 smithi077 ceph-mon[37108]: pgmap v310: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 6.1 MiB/s wr, 683 op/s 2024-08-02T17:56:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:26 smithi083 ceph-mon[37031]: pgmap v311: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.7 MiB/s wr, 645 op/s 2024-08-02T17:56:27.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:26 smithi077 ceph-mon[37108]: pgmap v311: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.7 MiB/s wr, 645 op/s 2024-08-02T17:56:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:26 smithi069 ceph-mon[32746]: pgmap v311: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.7 MiB/s wr, 645 op/s 2024-08-02T17:56:29.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:28 smithi083 ceph-mon[37031]: pgmap v312: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.4 MiB/s wr, 720 op/s 2024-08-02T17:56:29.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:28 smithi077 ceph-mon[37108]: pgmap v312: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.4 MiB/s wr, 720 op/s 2024-08-02T17:56:29.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:28 smithi069 ceph-mon[32746]: pgmap v312: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 6.4 MiB/s wr, 720 op/s 2024-08-02T17:56:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 560.0 s, 850.2 tps, lat 116.801 ms stddev 133.264 2024-08-02T17:56:31.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:30 smithi083 ceph-mon[37031]: pgmap v313: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-08-02T17:56:31.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:30 smithi077 ceph-mon[37108]: pgmap v313: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-08-02T17:56:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:30 smithi069 ceph-mon[32746]: pgmap v313: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.4 MiB/s wr, 586 op/s 2024-08-02T17:56:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:32 smithi083 ceph-mon[37031]: pgmap v314: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-08-02T17:56:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:32 smithi077 ceph-mon[37108]: pgmap v314: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-08-02T17:56:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:32 smithi069 ceph-mon[32746]: pgmap v314: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 4.1 MiB/s wr, 462 op/s 2024-08-02T17:56:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.715+0000 7f705359e640 1 -- 172.21.15.69:0/2994658493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 msgr2=0x7f705411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.715+0000 7f705359e640 1 --2- 172.21.15.69:0/2994658493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 0x7f705411d260 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f704001a910 tx=0x7f704003b450 comp rx=0 tx=0).stop 2024-08-02T17:56:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.723+0000 7f705359e640 1 -- 172.21.15.69:0/2994658493 shutdown_connections 2024-08-02T17:56:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.723+0000 7f705359e640 1 --2- 172.21.15.69:0/2994658493 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 0x7f705411d260 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.723+0000 7f705359e640 1 --2- 172.21.15.69:0/2994658493 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054109300 0x7f7054109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.723+0000 7f705359e640 1 --2- 172.21.15.69:0/2994658493 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7054108f20 0x7f705410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.725 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.723+0000 7f705359e640 1 -- 172.21.15.69:0/2994658493 >> 172.21.15.69:0/2994658493 conn(0x7f705406c900 msgr2=0x7f705406cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.730+0000 7f705359e640 1 -- 172.21.15.69:0/2994658493 shutdown_connections 2024-08-02T17:56:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.734+0000 7f705359e640 1 -- 172.21.15.69:0/2994658493 wait complete. 2024-08-02T17:56:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.735+0000 7f705359e640 1 Processor -- start 2024-08-02T17:56:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.739+0000 7f705359e640 1 -- start start 2024-08-02T17:56:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.739+0000 7f705359e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7054108f20 0x7f705411a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.739+0000 7f705359e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054109300 0x7f7054116530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.739+0000 7f705359e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 0x7f7054116a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.739+0000 7f705359e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7054112fe0 con 0x7f705411ae70 2024-08-02T17:56:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.739+0000 7f705359e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7054112e60 con 0x7f7054108f20 2024-08-02T17:56:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.739+0000 7f705359e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7054113160 con 0x7f7054109300 2024-08-02T17:56:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.743+0000 7f705259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7054108f20 0x7f705411a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.743+0000 7f7051d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054109300 0x7f7054116530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.744+0000 7f7052d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 0x7f7054116a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.744+0000 7f7052d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 0x7f7054116a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46406/0 (socket says 172.21.15.69:46406) 2024-08-02T17:56:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.744+0000 7f7052d9d640 1 -- 172.21.15.69:0/382510658 learned_addr learned my addr 172.21.15.69:0/382510658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:33.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.748+0000 7f705259c640 1 -- 172.21.15.69:0/382510658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054109300 msgr2=0x7f7054116530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.748+0000 7f705259c640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054109300 0x7f7054116530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.748+0000 7f705259c640 1 -- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 msgr2=0x7f7054116a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.748+0000 7f705259c640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 0x7f7054116a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.748+0000 7f705259c640 1 -- 172.21.15.69:0/382510658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7054117030 con 0x7f7054108f20 2024-08-02T17:56:33.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.751+0000 7f705259c640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7054108f20 0x7f705411a4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f704c00eb00 tx=0x7f704c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:33.756 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.753+0000 7f704b7fe640 1 -- 172.21.15.69:0/382510658 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c010430 con 0x7f7054108f20 2024-08-02T17:56:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.753+0000 7f704b7fe640 1 -- 172.21.15.69:0/382510658 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f704c010ab0 con 0x7f7054108f20 2024-08-02T17:56:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.753+0000 7f704b7fe640 1 -- 172.21.15.69:0/382510658 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c017b30 con 0x7f7054108f20 2024-08-02T17:56:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.756+0000 7f705359e640 1 -- 172.21.15.69:0/382510658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7054117320 con 0x7f7054108f20 2024-08-02T17:56:33.757 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.756+0000 7f705359e640 1 -- 172.21.15.69:0/382510658 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70541aaf40 con 0x7f7054108f20 2024-08-02T17:56:33.761 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.760+0000 7f705b78c740 1 Processor -- start 2024-08-02T17:56:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f705b78c740 1 -- start start 2024-08-02T17:56:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f705b78c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a62940 0x55fdb5a62d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f705b78c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a63260 0x55fdb5a6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f705b78c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f705b78c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fdb59c69b0 con 0x55fdb5a63260 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f705b78c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fdb596e800 con 0x55fdb5a6db00 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f705b78c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fdb59c2970 con 0x55fdb5a62940 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f704b7fe640 1 -- 172.21.15.69:0/382510658 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f704c017cd0 con 0x7f7054108f20 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f7051d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f7051d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52792/0 (socket says 172.21.15.69:52792) 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f7051d9b640 1 -- 172.21.15.69:0/3286837325 learned_addr learned my addr 172.21.15.69:0/3286837325 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:33.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f705259c640 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a63260 0x55fdb5a6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f7051d9b640 1 -- 172.21.15.69:0/3286837325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a62940 msgr2=0x55fdb5a62d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f7051d9b640 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a62940 0x55fdb5a62d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f7051d9b640 1 -- 172.21.15.69:0/3286837325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a63260 msgr2=0x55fdb5a6d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f7051d9b640 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a63260 0x55fdb5a6d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.763+0000 7f7051d9b640 1 -- 172.21.15.69:0/3286837325 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb56da210 con 0x55fdb5a6db00 2024-08-02T17:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.764+0000 7f705259c640 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a63260 0x55fdb5a6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:56:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.764+0000 7f7051d9b640 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6fef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7044009b70 tx=0x7f70440117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee304a4a0ffd1dd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.765+0000 7f704b7fe640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f70380a2f40 0x7f70380a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.765+0000 7f704b7fe640 1 -- 172.21.15.69:0/382510658 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f704c0d0110 con 0x7f7054108f20 2024-08-02T17:56:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.765+0000 7f7048ff9640 1 -- 172.21.15.69:0/3286837325 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7044009e60 con 0x55fdb5a6db00 2024-08-02T17:56:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.765+0000 7f7048ff9640 1 -- 172.21.15.69:0/3286837325 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7044012930 con 0x55fdb5a6db00 2024-08-02T17:56:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.766+0000 7f705b78c740 1 -- 172.21.15.69:0/3286837325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 msgr2=0x55fdb5a6fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.766+0000 7f705b78c740 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6fef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7044009b70 tx=0x7f70440117b0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.772+0000 7f7052d9d640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f70380a2f40 0x7f70380a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.774+0000 7f705b78c740 1 -- 172.21.15.69:0/3286837325 shutdown_connections 2024-08-02T17:56:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.774+0000 7f705b78c740 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a63260 0x55fdb5a6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.774+0000 7f705b78c740 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6fef0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.774+0000 7f705b78c740 1 --2- 172.21.15.69:0/3286837325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a62940 0x55fdb5a62d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.774+0000 7f705b78c740 1 -- 172.21.15.69:0/3286837325 >> 172.21.15.69:0/3286837325 conn(0x55fdb5a5eeb0 msgr2=0x55fdb5a5f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.777+0000 7f705b78c740 1 -- 172.21.15.69:0/3286837325 shutdown_connections 2024-08-02T17:56:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.780+0000 7f705b78c740 1 -- 172.21.15.69:0/3286837325 wait complete. 2024-08-02T17:56:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.780+0000 7f705b78c740 1 -- start start 2024-08-02T17:56:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.780+0000 7f705b78c740 1 Processor -- start 2024-08-02T17:56:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.782+0000 7f7052d9d640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f70380a2f40 0x7f70380a5400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f704001a6d0 tx=0x7f7040005b50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.782+0000 7f705b78c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a62940 0x55fdb5a6a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.782+0000 7f705b78c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 0x55fdb5a6ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.782+0000 7f705b78c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.783+0000 7f705b78c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fdb5a70720 con 0x55fdb5a6db00 2024-08-02T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.783+0000 7f705b78c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fdb5982500 con 0x55fdb5a62940 2024-08-02T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.783+0000 7f705b78c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fdb5a708a0 con 0x55fdb5a63260 2024-08-02T17:56:33.784 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.783+0000 7f705b78c740 10 client.0 _ll_register_callbacks cb 0x55fdb56d9a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.783+0000 7f7052d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.783+0000 7f7052d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46426/0 (socket says 172.21.15.69:46426) 2024-08-02T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.783+0000 7f7052d9d640 1 -- 172.21.15.69:0/1963752600 learned_addr learned my addr 172.21.15.69:0/1963752600 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.784+0000 7f7052d9d640 1 -- 172.21.15.69:0/1963752600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 msgr2=0x55fdb5a6ae70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.784+0000 7f7052d9d640 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 0x55fdb5a6ae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.784+0000 7f7052d9d640 1 -- 172.21.15.69:0/1963752600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a62940 msgr2=0x55fdb5a6a930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.784+0000 7f7052d9d640 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a62940 0x55fdb5a6a930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.784+0000 7f7052d9d640 1 -- 172.21.15.69:0/1963752600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb57a6620 con 0x55fdb5a6db00 2024-08-02T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.784+0000 7f7052d9d640 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6b3b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7040004a90 tx=0x7f704001a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4db87533dcf8a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.784+0000 7f7035ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.785+0000 7f7035ffb640 1 -- 172.21.15.69:0/1963752600 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704005e070 con 0x55fdb5a6db00 2024-08-02T17:56:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.785+0000 7f705b78c740 1 -- 172.21.15.69:0/1963752600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7044012930 con 0x55fdb5a6db00 2024-08-02T17:56:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.785+0000 7f7035ffb640 1 -- 172.21.15.69:0/1963752600 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f7040040710 con 0x55fdb5a6db00 2024-08-02T17:56:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.785+0000 7f7035ffb640 1 -- 172.21.15.69:0/1963752600 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f704006a050 con 0x55fdb5a6db00 2024-08-02T17:56:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.786+0000 7f705b78c740 10 client.17859 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.786+0000 7f705b78c740 1 -- 172.21.15.69:0/1963752600 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fdb5a7c350 con 0x55fdb5a6db00 2024-08-02T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.787+0000 7f7035ffb640 1 -- 172.21.15.69:0/1963752600 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7044012930 con 0x55fdb5a6db00 2024-08-02T17:56:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.787+0000 7f7035ffb640 1 -- 172.21.15.69:0/1963752600 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f704005a760 con 0x55fdb5a6db00 2024-08-02T17:56:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.788+0000 7f705b78c740 10 client.17859 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.788+0000 7f705b78c740 10 client.17859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.788+0000 7f705b78c740 20 client.17859 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.788+0000 7f705b78c740 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fdb5a7d810 0x55fdb5a7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.788+0000 7f705b78c740 4 client.17859 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.788+0000 7f705b78c740 1 -- 172.21.15.69:0/1963752600 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdb59d40e0 con 0x55fdb5a7d810 2024-08-02T17:56:33.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.794+0000 7f705259c640 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fdb5a7d810 0x55fdb5a7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.795+0000 7f705259c640 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fdb5a7d810 0x55fdb5a7fc80 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281b507b921522a3 server_cookie=92d39ea0bb5a9fb1 in_seq=0 out_seq=0 2024-08-02T17:56:33.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.795+0000 7f7035ffb640 10 client.17859 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.796+0000 7f7035ffb640 1 -- 172.21.15.69:0/1963752600 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fdb59d40e0 con 0x55fdb5a7d810 2024-08-02T17:56:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.796+0000 7f7035ffb640 10 client.17859 handle_command_reply: tid=0 2024-08-02T17:56:33.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.797+0000 7f705b78c740 1 client.17859 shutdown 2024-08-02T17:56:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.803+0000 7f705b78c740 1 -- 172.21.15.69:0/1963752600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a6db00 msgr2=0x55fdb5a6b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.804 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.803+0000 7f705b78c740 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6b3b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f7040004a90 tx=0x7f704001a270 comp rx=0 tx=0).stop 2024-08-02T17:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.804+0000 7f705b78c740 1 -- 172.21.15.69:0/1963752600 shutdown_connections 2024-08-02T17:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.804+0000 7f705b78c740 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a6db00 0x55fdb5a6b3b0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.804+0000 7f705b78c740 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fdb5a7d810 0x55fdb5a7fc80 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.804+0000 7f705b78c740 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 0x55fdb5a6ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.804+0000 7f705b78c740 1 --2- 172.21.15.69:0/1963752600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a62940 0x55fdb5a6a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.804+0000 7f705b78c740 1 -- 172.21.15.69:0/1963752600 >> 172.21.15.69:0/1963752600 conn(0x55fdb5a5eeb0 msgr2=0x55fdb5a64c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.804+0000 7f7035ffb640 0 client.17859 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.805+0000 7f705b78c740 1 -- 172.21.15.69:0/1963752600 shutdown_connections 2024-08-02T17:56:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.807+0000 7f705b78c740 1 -- 172.21.15.69:0/1963752600 wait complete. 2024-08-02T17:56:33.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.808+0000 7f705b78c740 20 client.17859 trim_cache size 0 max 16384 2024-08-02T17:56:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.836+0000 7f705b78c740 1 Processor -- start 2024-08-02T17:56:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.837+0000 7f705b78c740 1 -- start start 2024-08-02T17:56:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.837+0000 7f705b78c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a85590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a85590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705b78c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a63260 0x55fdb5a85ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a85590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46432/0 (socket says 172.21.15.69:46432) 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705259c640 1 -- 172.21.15.69:0/3270882946 learned_addr learned my addr 172.21.15.69:0/3270882946 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705b78c740 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a6db00 0x55fdb5a5d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705b78c740 1 -- 172.21.15.69:0/3270882946 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b7c50 con 0x55fdb5a62940 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705b78c740 1 -- 172.21.15.69:0/3270882946 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b5520 con 0x55fdb5a63260 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705b78c740 1 -- 172.21.15.69:0/3270882946 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b3370 con 0x55fdb5a6db00 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f7051d9b640 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a63260 0x55fdb5a85ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f7052d9d640 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a6db00 0x55fdb5a5d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705259c640 1 -- 172.21.15.69:0/3270882946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a6db00 msgr2=0x55fdb5a5d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705259c640 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a6db00 0x55fdb5a5d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705259c640 1 -- 172.21.15.69:0/3270882946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a63260 msgr2=0x55fdb5a85ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.838+0000 7f705259c640 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a63260 0x55fdb5a85ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.839+0000 7f705259c640 1 -- 172.21.15.69:0/3270882946 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdb5a5d430 con 0x55fdb5a62940 2024-08-02T17:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.839+0000 7f705259c640 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a85590 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f704c00c7d0 tx=0x7f704c0d2560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5237f09be44cbc07 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.839+0000 7f7037fff640 1 -- 172.21.15.69:0/3270882946 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c09c8e0 con 0x55fdb5a62940 2024-08-02T17:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.839+0000 7f7037fff640 1 -- 172.21.15.69:0/3270882946 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f704c09c1e0 con 0x55fdb5a62940 2024-08-02T17:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.839+0000 7f7037fff640 1 -- 172.21.15.69:0/3270882946 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0a0170 con 0x55fdb5a62940 2024-08-02T17:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 -- 172.21.15.69:0/3270882946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 msgr2=0x55fdb5a85590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.845 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a85590 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f704c00c7d0 tx=0x7f704c0d2560 comp rx=0 tx=0).stop 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 -- 172.21.15.69:0/3270882946 shutdown_connections 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a6db00 0x55fdb5a5d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a63260 0x55fdb5a85ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 --2- 172.21.15.69:0/3270882946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a85590 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 -- 172.21.15.69:0/3270882946 >> 172.21.15.69:0/3270882946 conn(0x55fdb5a5eeb0 msgr2=0x55fdb5a69a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 -- 172.21.15.69:0/3270882946 shutdown_connections 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 -- 172.21.15.69:0/3270882946 wait complete. 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.840+0000 7f705b78c740 1 -- start start 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705b78c740 1 Processor -- start 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705b78c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a67350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.846 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705b78c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 0x55fdb5a67890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705b78c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a74bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705b78c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b39e0 con 0x55fdb5a62940 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705b78c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b4e50 con 0x55fdb5a6db00 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705b78c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b40b0 con 0x55fdb5a63260 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705b78c740 10 client.0 _ll_register_callbacks cb 0x7f7040051070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a67350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46444/0 (socket says 172.21.15.69:46444) 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705259c640 1 -- 172.21.15.69:0/2164134177 learned_addr learned my addr 172.21.15.69:0/2164134177 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f7051d9b640 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 0x55fdb5a67890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f7052d9d640 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a74bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.841+0000 7f705259c640 1 -- 172.21.15.69:0/2164134177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 msgr2=0x55fdb5a67890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f705259c640 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 0x55fdb5a67890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.847 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f705259c640 1 -- 172.21.15.69:0/2164134177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 msgr2=0x55fdb5a74bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f705259c640 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a74bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f705259c640 1 -- 172.21.15.69:0/2164134177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdb5a75170 con 0x55fdb5a62940 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f705259c640 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a67350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f704c0a4920 tx=0x7f704c0cd400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4f2b32b2010837 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f7035ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f7035ffb640 1 -- 172.21.15.69:0/2164134177 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0b4070 con 0x55fdb5a62940 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f7035ffb640 1 -- 172.21.15.69:0/2164134177 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f704c0a0170 con 0x55fdb5a62940 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f705b78c740 1 -- 172.21.15.69:0/2164134177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704c09c1e0 con 0x55fdb5a62940 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f7035ffb640 1 -- 172.21.15.69:0/2164134177 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f704c0a0590 con 0x55fdb5a62940 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f705b78c740 10 client.17865 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f705b78c740 1 -- 172.21.15.69:0/2164134177 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fdb5a76440 con 0x55fdb5a62940 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.842+0000 7f7035ffb640 1 -- 172.21.15.69:0/2164134177 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f704c0a4e50 con 0x55fdb5a62940 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f7035ffb640 1 -- 172.21.15.69:0/2164134177 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f704c0b01d0 con 0x55fdb5a62940 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f705b78c740 10 client.17865 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f705b78c740 10 client.17865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f705b78c740 20 client.17865 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f705b78c740 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fdb5ae86a0 0x55fdb5aeaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f705b78c740 4 client.17865 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:56:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f705b78c740 1 -- 172.21.15.69:0/2164134177 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f704c09c1e0 con 0x55fdb5ae86a0 2024-08-02T17:56:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f7051d9b640 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fdb5ae86a0 0x55fdb5aeaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f7035ffb640 10 client.17865 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.843+0000 7f7051d9b640 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fdb5ae86a0 0x55fdb5aeaac0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548b84d9d5515256 server_cookie=aa80e739bf97ff34 in_seq=0 out_seq=0 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f7035ffb640 1 -- 172.21.15.69:0/2164134177 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f704c09c1e0 con 0x55fdb5ae86a0 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f7035ffb640 10 client.17865 handle_command_reply: tid=0 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 client.17865 shutdown 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 -- 172.21.15.69:0/2164134177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 msgr2=0x55fdb5a67350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a67350 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f704c0a4920 tx=0x7f704c0cd400 comp rx=0 tx=0).stop 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 -- 172.21.15.69:0/2164134177 shutdown_connections 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fdb5ae86a0 0x55fdb5aeaac0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fdb5a6db00 0x55fdb5a74bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f7035ffb640 0 client.17865 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fdb5a63260 0x55fdb5a67890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 --2- 172.21.15.69:0/2164134177 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fdb5a62940 0x55fdb5a67350 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 -- 172.21.15.69:0/2164134177 >> 172.21.15.69:0/2164134177 conn(0x55fdb5a5eeb0 msgr2=0x55fdb5a5d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 -- 172.21.15.69:0/2164134177 shutdown_connections 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 1 -- 172.21.15.69:0/2164134177 wait complete. 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.865+0000 7f705b78c740 20 client.17865 trim_cache size 0 max 16384 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:56:33.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:56:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 -- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f70380a2f40 msgr2=0x7f70380a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f70380a2f40 0x7f70380a5400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f704001a6d0 tx=0x7f7040005b50 comp rx=0 tx=0).stop 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 -- 172.21.15.69:0/382510658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7054108f20 msgr2=0x7f705411a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7054108f20 0x7f705411a4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f704c00eb00 tx=0x7f704c00c2d0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 -- 172.21.15.69:0/382510658 shutdown_connections 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f705411ae70 0x7f7054116a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f70380a2f40 0x7f70380a5400 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7054109300 0x7f7054116530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 --2- 172.21.15.69:0/382510658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7054108f20 0x7f705411a4a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 -- 172.21.15.69:0/382510658 >> 172.21.15.69:0/382510658 conn(0x7f705406c900 msgr2=0x7f70540703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 -- 172.21.15.69:0/382510658 shutdown_connections 2024-08-02T17:56:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:33.866+0000 7f705359e640 1 -- 172.21.15.69:0/382510658 wait complete. 2024-08-02T17:56:34.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 565.0 s, 698.6 tps, lat 141.951 ms stddev 146.334 2024-08-02T17:56:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:34 smithi083 ceph-mon[37031]: pgmap v315: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 688 op/s 2024-08-02T17:56:35.038 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-02T17:56:35.038 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.9 2024-08-02T17:56:35.039 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:56:35.094 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:34 smithi069 ceph-mon[32746]: pgmap v315: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 688 op/s 2024-08-02T17:56:35.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:34 smithi077 ceph-mon[37108]: pgmap v315: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.1 MiB/s wr, 688 op/s 2024-08-02T17:56:35.745 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:36.966 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:36 smithi069 ceph-mon[32746]: pgmap v316: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-08-02T17:56:37.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:36 smithi083 ceph-mon[37031]: pgmap v316: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-08-02T17:56:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:36 smithi077 ceph-mon[37108]: pgmap v316: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.1 MiB/s wr, 453 op/s 2024-08-02T17:56:37.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.643+0000 7fc1379f9640 1 -- 172.21.15.69:0/3533786367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc130102c20 msgr2=0x7fc13010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:37.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.643+0000 7fc1379f9640 1 --2- 172.21.15.69:0/3533786367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc130102c20 0x7fc13010d770 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc128009ca0 tx=0x7fc12802a960 comp rx=0 tx=0).stop 2024-08-02T17:56:37.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.643+0000 7fc1379f9640 1 -- 172.21.15.69:0/3533786367 shutdown_connections 2024-08-02T17:56:37.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.643+0000 7fc1379f9640 1 --2- 172.21.15.69:0/3533786367 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc130102c20 0x7fc13010d770 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:37.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.643+0000 7fc1379f9640 1 --2- 172.21.15.69:0/3533786367 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301022a0 0x7fc1301026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:37.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.643+0000 7fc1379f9640 1 --2- 172.21.15.69:0/3533786367 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130100b50 0x7fc130100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.643+0000 7fc1379f9640 1 -- 172.21.15.69:0/3533786367 >> 172.21.15.69:0/3533786367 conn(0x7fc1300fc500 msgr2=0x7fc1300fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.644+0000 7fc1379f9640 1 -- 172.21.15.69:0/3533786367 shutdown_connections 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.645+0000 7fc1379f9640 1 -- 172.21.15.69:0/3533786367 wait complete. 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.646+0000 7fc1379f9640 1 Processor -- start 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.646+0000 7fc1379f9640 1 -- start start 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.646+0000 7fc1379f9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130100b50 0x7fc13010d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.646+0000 7fc1379f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301022a0 0x7fc13010a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.646+0000 7fc1379f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc130102c20 0x7fc13010ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.647+0000 7fc1379f9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc13006c350 con 0x7fc130102c20 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.647+0000 7fc134f94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301022a0 0x7fc13010a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.647+0000 7fc134f94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301022a0 0x7fc13010a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:44996/0 (socket says 172.21.15.69:44996) 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.647+0000 7fc134f94640 1 -- 172.21.15.69:0/4019700697 learned_addr learned my addr 172.21.15.69:0/4019700697 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:37.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.647+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc13006c1d0 con 0x7fc130100b50 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.647+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc13006c4d0 con 0x7fc1301022a0 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.649+0000 7fc135795640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130100b50 0x7fc13010d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.649+0000 7fc134f94640 1 -- 172.21.15.69:0/4019700697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130100b50 msgr2=0x7fc13010d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.649+0000 7fc134f94640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130100b50 0x7fc13010d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.650+0000 7fc134f94640 1 -- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc130102c20 msgr2=0x7fc13010ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.650+0000 7fc134f94640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc130102c20 0x7fc13010ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.650+0000 7fc134f94640 1 -- 172.21.15.69:0/4019700697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13010b510 con 0x7fc1301022a0 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.651+0000 7fc134f94640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301022a0 0x7fc13010a6f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc12000d950 tx=0x7fc12000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.655+0000 7fc1267fc640 1 -- 172.21.15.69:0/4019700697 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120017070 con 0x7fc1301022a0 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.655+0000 7fc1267fc640 1 -- 172.21.15.69:0/4019700697 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc120009c30 con 0x7fc1301022a0 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.655+0000 7fc1267fc640 1 -- 172.21.15.69:0/4019700697 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12000f6b0 con 0x7fc1301022a0 2024-08-02T17:56:37.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.655+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1301af140 con 0x7fc1301022a0 2024-08-02T17:56:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.655+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1301af630 con 0x7fc1301022a0 2024-08-02T17:56:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.657+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc130101ee0 con 0x7fc1301022a0 2024-08-02T17:56:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.659+0000 7fc1267fc640 1 -- 172.21.15.69:0/4019700697 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc1200109d0 con 0x7fc1301022a0 2024-08-02T17:56:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.660+0000 7fc1267fc640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc11c0a2ec0 0x7fc11c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.660+0000 7fc1267fc640 1 -- 172.21.15.69:0/4019700697 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7fc1200cf4b0 con 0x7fc1301022a0 2024-08-02T17:56:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.666+0000 7fc135795640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc11c0a2ec0 0x7fc11c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.666+0000 7fc1267fc640 1 -- 172.21.15.69:0/4019700697 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc120009de0 con 0x7fc1301022a0 2024-08-02T17:56:37.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:37.686+0000 7fc135795640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc11c0a2ec0 0x7fc11c0a5380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc12c005fd0 tx=0x7fc12c005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:38.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.089+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc13006e1c0 con 0x7fc1301022a0 2024-08-02T17:56:38.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.093+0000 7fc1267fc640 1 -- 172.21.15.69:0/4019700697 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fc12009bc80 con 0x7fc1301022a0 2024-08-02T17:56:38.095 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:56:38.095 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:56:38.096 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:56:38.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc11c0a2ec0 msgr2=0x7fc11c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc11c0a2ec0 0x7fc11c0a5380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc12c005fd0 tx=0x7fc12c005850 comp rx=0 tx=0).stop 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301022a0 msgr2=0x7fc13010a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301022a0 0x7fc13010a6f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc12000d950 tx=0x7fc12000de10 comp rx=0 tx=0).stop 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 shutdown_connections 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc130102c20 0x7fc13010ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc11c0a2ec0 0x7fc11c0a5380 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1301022a0 0x7fc13010a6f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 --2- 172.21.15.69:0/4019700697 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc130100b50 0x7fc13010d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 >> 172.21.15.69:0/4019700697 conn(0x7fc1300fc500 msgr2=0x7fc130105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 shutdown_connections 2024-08-02T17:56:38.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:38.098+0000 7fc1379f9640 1 -- 172.21.15.69:0/4019700697 wait complete. 2024-08-02T17:56:39.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:38 smithi083 ceph-mon[37031]: pgmap v317: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 615 op/s 2024-08-02T17:56:39.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:38 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4019700697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:39.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:38 smithi077 ceph-mon[37108]: pgmap v317: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 615 op/s 2024-08-02T17:56:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:38 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4019700697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:38 smithi069 ceph-mon[32746]: pgmap v317: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 615 op/s 2024-08-02T17:56:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:38 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4019700697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:39.944 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:56:39.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 570.0 s, 579.8 tps, lat 175.943 ms stddev 222.323 2024-08-02T17:56:41.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:40 smithi083 ceph-mon[37031]: pgmap v318: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 556 op/s 2024-08-02T17:56:41.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:40 smithi077 ceph-mon[37108]: pgmap v318: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 556 op/s 2024-08-02T17:56:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:40 smithi069 ceph-mon[32746]: pgmap v318: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.0 MiB/s wr, 556 op/s 2024-08-02T17:56:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:42 smithi083 ceph-mon[37031]: pgmap v319: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-08-02T17:56:43.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:42 smithi077 ceph-mon[37108]: pgmap v319: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-08-02T17:56:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:42 smithi069 ceph-mon[32746]: pgmap v319: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-08-02T17:56:44.875 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 139.9 --await-for 45.0' 2024-08-02T17:56:44.875 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 139.9 --await-for 45.0 2024-08-02T17:56:44.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 575.0 s, 711.6 tps, lat 132.618 ms stddev 154.686 2024-08-02T17:56:44.983 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:44 smithi069 ceph-mon[32746]: pgmap v320: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 844 op/s 2024-08-02T17:56:44.997 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:44 smithi083 ceph-mon[37031]: pgmap v320: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 844 op/s 2024-08-02T17:56:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:44 smithi077 ceph-mon[37108]: pgmap v320: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 7.4 MiB/s wr, 844 op/s 2024-08-02T17:56:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 -- 172.21.15.69:0/3007331353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31181066e0 msgr2=0x7f3118106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 --2- 172.21.15.69:0/3007331353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31181066e0 0x7f3118106ac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3108009890 tx=0x7f310802a380 comp rx=0 tx=0).stop 2024-08-02T17:56:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 -- 172.21.15.69:0/3007331353 shutdown_connections 2024-08-02T17:56:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 --2- 172.21.15.69:0/3007331353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31181006d0 0x7f3118100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 --2- 172.21.15.69:0/3007331353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3118101070 0x7f311810d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 --2- 172.21.15.69:0/3007331353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31181066e0 0x7f3118106ac0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 -- 172.21.15.69:0/3007331353 >> 172.21.15.69:0/3007331353 conn(0x7f31180776d0 msgr2=0x7f31180fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 -- 172.21.15.69:0/3007331353 shutdown_connections 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.984+0000 7f311e487640 1 -- 172.21.15.69:0/3007331353 wait complete. 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311e487640 1 Processor -- start 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311e487640 1 -- start start 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311e487640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31181006d0 0x7f3118197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311e487640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3118101070 0x7f3118197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311e487640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31181066e0 0x7f311819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311e487640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3118110ae0 con 0x7f31181006d0 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311e487640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3118110960 con 0x7f3118101070 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311e487640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3118110c60 con 0x7f31181066e0 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f3117fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31181006d0 0x7f3118197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f3117fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31181006d0 0x7f3118197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51668/0 (socket says 172.21.15.69:51668) 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f3117fff640 1 -- 172.21.15.69:0/3525208131 learned_addr learned my addr 172.21.15.69:0/3525208131 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f311ca24640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31181066e0 0x7f311819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f3117fff640 1 -- 172.21.15.69:0/3525208131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31181066e0 msgr2=0x7f311819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f3117fff640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31181066e0 0x7f311819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.985+0000 7f310ffff640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3118101070 0x7f3118197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f3117fff640 1 -- 172.21.15.69:0/3525208131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3118101070 msgr2=0x7f3118197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f3117fff640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3118101070 0x7f3118197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f3117fff640 1 -- 172.21.15.69:0/3525208131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311819c140 con 0x7f31181006d0 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f3117fff640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31181006d0 0x7f3118197150 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f3108033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f3115ffb640 1 -- 172.21.15.69:0/3525208131 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310803f020 con 0x7f31181006d0 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f311e487640 1 -- 172.21.15.69:0/3525208131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311819c3d0 con 0x7f31181006d0 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f3115ffb640 1 -- 172.21.15.69:0/3525208131 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3108032770 con 0x7f31181006d0 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f3115ffb640 1 -- 172.21.15.69:0/3525208131 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108037040 con 0x7f31181006d0 2024-08-02T17:56:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.986+0000 7f311e487640 1 -- 172.21.15.69:0/3525208131 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31181a3cc0 con 0x7f31181006d0 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.988+0000 7f3115ffb640 1 -- 172.21.15.69:0/3525208131 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3108042020 con 0x7f31181006d0 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.988+0000 7f31211d9740 1 Processor -- start 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.988+0000 7f31211d9740 1 -- start start 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f31211d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c961da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f31211d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c962a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f31211d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c962cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f31211d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5639c95819b0 con 0x5639c962ab80 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f31211d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5639c9529800 con 0x5639c961d630 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f31211d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5639c957d970 con 0x5639c961df50 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f3115ffb640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f31040a2cf0 0x7f31040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c962cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f311ca24640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c961da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f311ca24640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c961da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45686/0 (socket says 172.21.15.69:45686) 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f311ca24640 1 -- 172.21.15.69:0/2832111396 learned_addr learned my addr 172.21.15.69:0/2832111396 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c962cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51674/0 (socket says 172.21.15.69:51674) 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f3117fff640 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c962a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f311ca24640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f31040a2cf0 0x7f31040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310ffff640 1 -- 172.21.15.69:0/2832111396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 msgr2=0x5639c962a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f3115ffb640 1 -- 172.21.15.69:0/3525208131 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f31080ef8f0 con 0x7f31181006d0 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310ffff640 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c962a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310ffff640 1 -- 172.21.15.69:0/2832111396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 msgr2=0x5639c961da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310ffff640 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c961da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310ffff640 1 -- 172.21.15.69:0/2832111396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c9295210 con 0x5639c962ab80 2024-08-02T17:56:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f3117fff640 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c962a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310ffff640 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c962cf70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f3100009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e3214af9b3a5a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f311ca24640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f31040a2cf0 0x7f31040a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3110009fd0 tx=0x7f3110004ab0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310e7fc640 1 -- 172.21.15.69:0/2832111396 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100010d00 con 0x5639c962ab80 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310e7fc640 1 -- 172.21.15.69:0/2832111396 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3100005500 con 0x5639c962ab80 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.989+0000 7f310e7fc640 1 -- 172.21.15.69:0/2832111396 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31000126b0 con 0x5639c962ab80 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 -- 172.21.15.69:0/2832111396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 msgr2=0x5639c962cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c962cf70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f3100009650 comp rx=0 tx=0).stop 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 -- 172.21.15.69:0/2832111396 shutdown_connections 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c962cf70 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c962a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 --2- 172.21.15.69:0/2832111396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c961da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 -- 172.21.15.69:0/2832111396 >> 172.21.15.69:0/2832111396 conn(0x5639c9619d20 msgr2=0x5639c961a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 -- 172.21.15.69:0/2832111396 shutdown_connections 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 -- 172.21.15.69:0/2832111396 wait complete. 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.990+0000 7f31211d9740 1 -- start start 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f31211d9740 1 Processor -- start 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f31211d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c9627b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f31211d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9628090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f31211d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c96285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f31211d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5639c962d7b0 con 0x5639c962ab80 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f31211d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5639c953d500 con 0x5639c961d630 2024-08-02T17:56:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f31211d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5639c962d930 con 0x5639c961df50 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f31211d9740 10 client.0 _ll_register_callbacks cb 0x5639c9294a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c96285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c96285d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51690/0 (socket says 172.21.15.69:51690) 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 -- 172.21.15.69:0/3339652315 learned_addr learned my addr 172.21.15.69:0/3339652315 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f3117fff640 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c9627b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f310ffff640 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9628090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 -- 172.21.15.69:0/3339652315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 msgr2=0x5639c9628090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9628090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 -- 172.21.15.69:0/3339652315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 msgr2=0x5639c9627b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c9627b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 -- 172.21.15.69:0/3339652315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c9361620 con 0x5639c962ab80 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f310ffff640 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9628090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f311ca24640 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c96285d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f31100129d0 tx=0x7f3110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ea6b81feca599a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f30f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.991+0000 7f30f77fe640 1 -- 172.21.15.69:0/3339652315 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110025030 con 0x5639c962ab80 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f30f77fe640 1 -- 172.21.15.69:0/3339652315 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f311001b070 con 0x5639c962ab80 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 1 -- 172.21.15.69:0/3339652315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3100005500 con 0x5639c962ab80 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f30f77fe640 1 -- 172.21.15.69:0/3339652315 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3110031050 con 0x5639c962ab80 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 10 client.17880 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 1 -- 172.21.15.69:0/3339652315 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639c9639420 con 0x5639c962ab80 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f30f77fe640 1 -- 172.21.15.69:0/3339652315 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3100005500 con 0x5639c962ab80 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f30f77fe640 1 -- 172.21.15.69:0/3339652315 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f311001e050 con 0x5639c962ab80 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 10 client.17880 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 10 client.17880 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 20 client.17880 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5639c963a890 0x5639c963ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 4 client.17880 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f31211d9740 1 -- 172.21.15.69:0/3339652315 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639c958f0e0 con 0x5639c963a890 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.992+0000 7f3117fff640 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5639c963a890 0x5639c963ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f3117fff640 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5639c963a890 0x5639c963ccc0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a00398ca1842615 server_cookie=1a402b09ec92eb78 in_seq=0 out_seq=0 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f30f77fe640 10 client.17880 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f30f77fe640 1 -- 172.21.15.69:0/3339652315 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5639c958f0e0 con 0x5639c963a890 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f30f77fe640 10 client.17880 handle_command_reply: tid=0 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 client.17880 shutdown 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 -- 172.21.15.69:0/3339652315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 msgr2=0x5639c96285d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c96285d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f31100129d0 tx=0x7f3110004a40 comp rx=0 tx=0).stop 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 -- 172.21.15.69:0/3339652315 shutdown_connections 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c962ab80 0x5639c96285d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5639c963a890 0x5639c963ccc0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9628090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 --2- 172.21.15.69:0/3339652315 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c961d630 0x5639c9627b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f30f77fe640 0 client.17880 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 -- 172.21.15.69:0/3339652315 >> 172.21.15.69:0/3339652315 conn(0x5639c9619d20 msgr2=0x5639c961fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 -- 172.21.15.69:0/3339652315 shutdown_connections 2024-08-02T17:56:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.993+0000 7f31211d9740 1 -- 172.21.15.69:0/3339652315 wait complete. 2024-08-02T17:56:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:46.994+0000 7f31211d9740 20 client.17880 trim_cache size 0 max 16384 2024-08-02T17:56:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.016+0000 7f31211d9740 1 Processor -- start 2024-08-02T17:56:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.016+0000 7f31211d9740 1 -- start start 2024-08-02T17:56:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.016+0000 7f31211d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c96299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.016+0000 7f31211d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9642620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.016+0000 7f31211d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 0x5639c9642b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.016+0000 7f310ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9642620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.016+0000 7f310ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9642620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:34142/0 (socket says 172.21.15.69:34142) 2024-08-02T17:56:47.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f3117fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c96299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f310ffff640 1 -- 172.21.15.69:0/3114111190 learned_addr learned my addr 172.21.15.69:0/3114111190 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f31211d9740 1 -- 172.21.15.69:0/3114111190 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f31181af600 con 0x5639c961d630 2024-08-02T17:56:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f31211d9740 1 -- 172.21.15.69:0/3114111190 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31181aced0 con 0x5639c962ab80 2024-08-02T17:56:47.019 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f31211d9740 1 -- 172.21.15.69:0/3114111190 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31181aad20 con 0x5639c961df50 2024-08-02T17:56:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f311ca24640 1 --2- 172.21.15.69:0/3114111190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 0x5639c9642b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f3117fff640 1 -- 172.21.15.69:0/3114111190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 msgr2=0x5639c9642620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f3117fff640 1 --2- 172.21.15.69:0/3114111190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9642620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f3117fff640 1 -- 172.21.15.69:0/3114111190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 msgr2=0x5639c9642b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f3117fff640 1 --2- 172.21.15.69:0/3114111190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 0x5639c9642b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.017+0000 7f3117fff640 1 -- 172.21.15.69:0/3114111190 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c9620200 con 0x5639c961d630 2024-08-02T17:56:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.018+0000 7f3117fff640 1 --2- 172.21.15.69:0/3114111190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c96299b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f31080ecd40 tx=0x7f31080373d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247940fd974f2a7d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.018+0000 7f310dffb640 1 -- 172.21.15.69:0/3114111190 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080371e0 con 0x5639c961d630 2024-08-02T17:56:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.018+0000 7f310dffb640 1 -- 172.21.15.69:0/3114111190 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31080bbb10 con 0x5639c961d630 2024-08-02T17:56:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.018+0000 7f310dffb640 1 -- 172.21.15.69:0/3114111190 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31080b8180 con 0x5639c961d630 2024-08-02T17:56:47.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:46 smithi083 ceph-mon[37031]: pgmap v321: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 617 op/s 2024-08-02T17:56:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 -- 172.21.15.69:0/3114111190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 msgr2=0x5639c96299b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 --2- 172.21.15.69:0/3114111190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c96299b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f31080ecd40 tx=0x7f31080373d0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 -- 172.21.15.69:0/3114111190 shutdown_connections 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 --2- 172.21.15.69:0/3114111190 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 0x5639c9642b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 --2- 172.21.15.69:0/3114111190 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c9642620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 --2- 172.21.15.69:0/3114111190 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c96299b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 -- 172.21.15.69:0/3114111190 >> 172.21.15.69:0/3114111190 conn(0x5639c9619d20 msgr2=0x5639c96292a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 -- 172.21.15.69:0/3114111190 shutdown_connections 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 -- 172.21.15.69:0/3114111190 wait complete. 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 -- start start 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 Processor -- start 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c9632b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c96330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 0x5639c962db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.019+0000 7f31211d9740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f31181ab390 con 0x5639c961d630 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f31211d9740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f31181ac800 con 0x5639c962ab80 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f31211d9740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31181aba60 con 0x5639c961df50 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f31211d9740 10 client.0 _ll_register_callbacks cb 0x7f3110009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f3117fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c9632b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f310ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c96330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f3117fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c9632b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51694/0 (socket says 172.21.15.69:51694) 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f3117fff640 1 -- 172.21.15.69:0/3930288504 learned_addr learned my addr 172.21.15.69:0/3930288504 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f311ca24640 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 0x5639c962db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f310ffff640 1 -- 172.21.15.69:0/3930288504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 msgr2=0x5639c962db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f310ffff640 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 0x5639c962db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f310ffff640 1 -- 172.21.15.69:0/3930288504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 msgr2=0x5639c9632b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f310ffff640 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c9632b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f310ffff640 1 -- 172.21.15.69:0/3930288504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c962e080 con 0x5639c961df50 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f310ffff640 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c96330d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3100004c80 tx=0x7f3100010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17401cad07352a82 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f30f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f30f77fe640 1 -- 172.21.15.69:0/3930288504 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100004eb0 con 0x5639c961df50 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.020+0000 7f31211d9740 1 -- 172.21.15.69:0/3930288504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31080bbb10 con 0x5639c961df50 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f30f77fe640 1 -- 172.21.15.69:0/3930288504 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f3100005050 con 0x5639c961df50 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f30f77fe640 1 -- 172.21.15.69:0/3930288504 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3100036050 con 0x5639c961df50 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f31211d9740 10 client.27199 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f31211d9740 1 -- 172.21.15.69:0/3930288504 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639c96a3d50 con 0x5639c961df50 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f30f77fe640 1 -- 172.21.15.69:0/3930288504 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31080bbb10 con 0x5639c961df50 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f30f77fe640 1 -- 172.21.15.69:0/3930288504 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3100031030 con 0x5639c961df50 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f31211d9740 10 client.27199 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f31211d9740 10 client.27199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f31211d9740 20 client.27199 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f31211d9740 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5639c96a4140 0x5639c96a6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f31211d9740 4 client.27199 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:56:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.021+0000 7f31211d9740 1 -- 172.21.15.69:0/3930288504 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639c9589430 con 0x5639c96a4140 2024-08-02T17:56:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.022+0000 7f3117fff640 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5639c96a4140 0x5639c96a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.022+0000 7f3117fff640 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5639c96a4140 0x5639c96a6560 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6406fa0efbfba42f server_cookie=a5a6d559c1692f7a in_seq=0 out_seq=0 2024-08-02T17:56:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.022+0000 7f30f77fe640 10 client.27199 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f30f77fe640 1 -- 172.21.15.69:0/3930288504 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639c9589430 con 0x5639c96a4140 2024-08-02T17:56:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f30f77fe640 10 client.27199 handle_command_reply: tid=0 2024-08-02T17:56:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 client.27199 shutdown 2024-08-02T17:56:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 -- 172.21.15.69:0/3930288504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 msgr2=0x5639c96330d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c96330d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3100004c80 tx=0x7f3100010fd0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 -- 172.21.15.69:0/3930288504 shutdown_connections 2024-08-02T17:56:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5639c96a4140 0x5639c96a6560 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f30f77fe640 0 client.27199 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5639c962ab80 0x5639c962db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5639c961df50 0x5639c96330d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 --2- 172.21.15.69:0/3930288504 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5639c961d630 0x5639c9632b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 -- 172.21.15.69:0/3930288504 >> 172.21.15.69:0/3930288504 conn(0x5639c9619d20 msgr2=0x5639c961f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 -- 172.21.15.69:0/3930288504 shutdown_connections 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 1 -- 172.21.15.69:0/3930288504 wait complete. 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.040+0000 7f31211d9740 20 client.27199 trim_cache size 0 max 16384 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:56:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:56:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:56:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 -- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f31040a2cf0 msgr2=0x7f31040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f31040a2cf0 0x7f31040a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3110009fd0 tx=0x7f3110004ab0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 -- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31181006d0 msgr2=0x7f3118197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31181006d0 0x7f3118197150 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c00 tx=0x7f3108033d10 comp rx=0 tx=0).stop 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 -- 172.21.15.69:0/3525208131 shutdown_connections 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f31040a2cf0 0x7f31040a51b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f31181066e0 0x7f311819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3118101070 0x7f3118197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 --2- 172.21.15.69:0/3525208131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f31181006d0 0x7f3118197150 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 -- 172.21.15.69:0/3525208131 >> 172.21.15.69:0/3525208131 conn(0x7f31180776d0 msgr2=0x7f3118103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 -- 172.21.15.69:0/3525208131 shutdown_connections 2024-08-02T17:56:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:56:47.041+0000 7f311e487640 1 -- 172.21.15.69:0/3525208131 wait complete. 2024-08-02T17:56:47.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:46 smithi077 ceph-mon[37108]: pgmap v321: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 617 op/s 2024-08-02T17:56:47.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:46 smithi069 ceph-mon[32746]: pgmap v321: 129 pgs: 129 active+clean; 3.0 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.3 MiB/s wr, 617 op/s 2024-08-02T17:56:47.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.390+0000 7f123d4b2640 1 -- 172.21.15.69:0/3786393191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123806d8d0 msgr2=0x7f123806dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.390+0000 7f123d4b2640 1 --2- 172.21.15.69:0/3786393191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123806d8d0 0x7f123806dcb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1228009900 tx=0x7f122802a440 comp rx=0 tx=0).stop 2024-08-02T17:56:47.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.396+0000 7f123d4b2640 1 -- 172.21.15.69:0/3786393191 shutdown_connections 2024-08-02T17:56:47.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.396+0000 7f123d4b2640 1 --2- 172.21.15.69:0/3786393191 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123811af40 0x7f123811d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.396+0000 7f123d4b2640 1 --2- 172.21.15.69:0/3786393191 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123806e1f0 0x7f123811aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.396+0000 7f123d4b2640 1 --2- 172.21.15.69:0/3786393191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123806d8d0 0x7f123806dcb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.396+0000 7f123d4b2640 1 -- 172.21.15.69:0/3786393191 >> 172.21.15.69:0/3786393191 conn(0x7f123806caf0 msgr2=0x7f123806cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.402+0000 7f123d4b2640 1 -- 172.21.15.69:0/3786393191 shutdown_connections 2024-08-02T17:56:47.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.403+0000 7f123d4b2640 1 -- 172.21.15.69:0/3786393191 wait complete. 2024-08-02T17:56:47.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.404+0000 7f123d4b2640 1 Processor -- start 2024-08-02T17:56:47.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.405+0000 7f123d4b2640 1 -- start start 2024-08-02T17:56:47.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.405+0000 7f123d4b2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123806d8d0 0x7f12381a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.405+0000 7f123d4b2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123806e1f0 0x7f12381a7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.405+0000 7f123d4b2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123811af40 0x7f12381ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.406+0000 7f123d4b2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1238120b50 con 0x7f123806d8d0 2024-08-02T17:56:47.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.406+0000 7f12377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123811af40 0x7f12381ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.406+0000 7f12377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123811af40 0x7f12381ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:34152/0 (socket says 172.21.15.69:34152) 2024-08-02T17:56:47.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.406+0000 7f12377fe640 1 -- 172.21.15.69:0/4242077182 learned_addr learned my addr 172.21.15.69:0/4242077182 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:47.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.406+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12381209d0 con 0x7f123806e1f0 2024-08-02T17:56:47.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.406+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1238120cd0 con 0x7f123811af40 2024-08-02T17:56:47.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.411+0000 7f12367fc640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123806e1f0 0x7f12381a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.411+0000 7f1236ffd640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123806d8d0 0x7f12381a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.411+0000 7f12377fe640 1 -- 172.21.15.69:0/4242077182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123806e1f0 msgr2=0x7f12381a7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.411+0000 7f12377fe640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123806e1f0 0x7f12381a7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.411+0000 7f12377fe640 1 -- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123806d8d0 msgr2=0x7f12381a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.411+0000 7f12377fe640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123806d8d0 0x7f12381a6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.411+0000 7f12377fe640 1 -- 172.21.15.69:0/4242077182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12381abfa0 con 0x7f123811af40 2024-08-02T17:56:47.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.412+0000 7f12377fe640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123811af40 0x7f12381ab8c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f123000b9e0 tx=0x7f123000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.414+0000 7f121ffff640 1 -- 172.21.15.69:0/4242077182 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230019070 con 0x7f123811af40 2024-08-02T17:56:47.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.414+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12381ac290 con 0x7f123811af40 2024-08-02T17:56:47.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.414+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12381b3b70 con 0x7f123811af40 2024-08-02T17:56:47.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.414+0000 7f121ffff640 1 -- 172.21.15.69:0/4242077182 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1230014440 con 0x7f123811af40 2024-08-02T17:56:47.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.414+0000 7f121ffff640 1 -- 172.21.15.69:0/4242077182 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12300125a0 con 0x7f123811af40 2024-08-02T17:56:47.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.421+0000 7f121ffff640 1 -- 172.21.15.69:0/4242077182 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f123001c070 con 0x7f123811af40 2024-08-02T17:56:47.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.423+0000 7f121ffff640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12200a2f00 0x7f12200a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.423+0000 7f121ffff640 1 -- 172.21.15.69:0/4242077182 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f12300cfbc0 con 0x7f123811af40 2024-08-02T17:56:47.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.431+0000 7f1236ffd640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12200a2f00 0x7f12200a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f123f8b0740 1 Processor -- start 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f123f8b0740 1 -- start start 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8e99490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f123f8b0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8dfe980 con 0x55c2c8e999d0 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f123f8b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8d96a70 con 0x55c2c8eb5400 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f123f8b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8dfa940 con 0x55c2c8e990b0 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.434+0000 7f12377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f12377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45744/0 (socket says 172.21.15.69:45744) 2024-08-02T17:56:47.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f12377fe640 1 -- 172.21.15.69:0/2918910820 learned_addr learned my addr 172.21.15.69:0/2918910820 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:47.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f12367fc640 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f1236ffd640 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8e99490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f12377fe640 1 -- 172.21.15.69:0/2918910820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 msgr2=0x55c2c8e99490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f12377fe640 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8e99490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f12377fe640 1 -- 172.21.15.69:0/2918910820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 msgr2=0x55c2c8eb4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f12377fe640 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.435+0000 7f12377fe640 1 -- 172.21.15.69:0/2918910820 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c8df96e0 con 0x55c2c8eb5400 2024-08-02T17:56:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.436+0000 7f1236ffd640 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8e99490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:56:47.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.436+0000 7f12377fe640 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb77f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1230009fd0 tx=0x7f1230098220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a9ca6559a8e95d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.436+0000 7f121d7fa640 1 -- 172.21.15.69:0/2918910820 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123009f030 con 0x55c2c8eb5400 2024-08-02T17:56:47.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.437+0000 7f1236ffd640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12200a2f00 0x7f12200a53c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f12280096c0 tx=0x7f1228005fd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.437+0000 7f121d7fa640 1 -- 172.21.15.69:0/2918910820 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12300984c0 con 0x55c2c8eb5400 2024-08-02T17:56:47.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.438+0000 7f123f8b0740 1 -- 172.21.15.69:0/2918910820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 msgr2=0x55c2c8eb77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.438+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb77f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1230009fd0 tx=0x7f1230098220 comp rx=0 tx=0).stop 2024-08-02T17:56:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.441+0000 7f123f8b0740 1 -- 172.21.15.69:0/2918910820 shutdown_connections 2024-08-02T17:56:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.441+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.441+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb77f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.441+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2918910820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8e99490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.441+0000 7f123f8b0740 1 -- 172.21.15.69:0/2918910820 >> 172.21.15.69:0/2918910820 conn(0x55c2c8e957a0 msgr2=0x55c2c8e95b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.441+0000 7f123f8b0740 1 -- 172.21.15.69:0/2918910820 shutdown_connections 2024-08-02T17:56:47.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.445+0000 7f123f8b0740 1 -- 172.21.15.69:0/2918910820 wait complete. 2024-08-02T17:56:47.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.445+0000 7f123f8b0740 1 -- start start 2024-08-02T17:56:47.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.445+0000 7f123f8b0740 1 Processor -- start 2024-08-02T17:56:47.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.447+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.447+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.447+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.447+0000 7f123f8b0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8eb8070 con 0x55c2c8e999d0 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.447+0000 7f123f8b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8db9aa0 con 0x55c2c8e990b0 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.447+0000 7f123f8b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c2c8eb81f0 con 0x55c2c8eb5400 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.447+0000 7f123f8b0740 10 client.0 _ll_register_callbacks cb 0x55c2c8da8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.451+0000 7f12377fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.451+0000 7f12367fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.451+0000 7f12367fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51734/0 (socket says 172.21.15.69:51734) 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.451+0000 7f12367fc640 1 -- 172.21.15.69:0/104400822 learned_addr learned my addr 172.21.15.69:0/104400822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:47.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.451+0000 7f1236ffd640 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.452+0000 7f12377fe640 1 -- 172.21.15.69:0/104400822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 msgr2=0x55c2c8eb2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.452+0000 7f12377fe640 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.452+0000 7f12377fe640 1 -- 172.21.15.69:0/104400822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 msgr2=0x55c2c8eb2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.452+0000 7f12377fe640 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.452+0000 7f12377fe640 1 -- 172.21.15.69:0/104400822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c8d9d650 con 0x55c2c8eb5400 2024-08-02T17:56:47.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.454+0000 7f12377fe640 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb2ca0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f123009bde0 tx=0x7f123009a360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a539c0565034d57 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.454+0000 7f121aff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.461+0000 7f121aff5640 1 -- 172.21.15.69:0/104400822 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123009f070 con 0x55c2c8eb5400 2024-08-02T17:56:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.461+0000 7f123f8b0740 1 -- 172.21.15.69:0/104400822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12300984c0 con 0x55c2c8eb5400 2024-08-02T17:56:47.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.461+0000 7f121aff5640 1 -- 172.21.15.69:0/104400822 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f123000cd20 con 0x55c2c8eb5400 2024-08-02T17:56:47.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.462+0000 7f121aff5640 1 -- 172.21.15.69:0/104400822 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1230097ba0 con 0x55c2c8eb5400 2024-08-02T17:56:47.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.462+0000 7f123f8b0740 10 client.27208 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:47.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.462+0000 7f123f8b0740 1 -- 172.21.15.69:0/104400822 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2c8b703c0 con 0x55c2c8eb5400 2024-08-02T17:56:47.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.465+0000 7f121aff5640 1 -- 172.21.15.69:0/104400822 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12300984c0 con 0x55c2c8eb5400 2024-08-02T17:56:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.465+0000 7f121aff5640 1 -- 172.21.15.69:0/104400822 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f12300ba9e0 con 0x55c2c8eb5400 2024-08-02T17:56:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.465+0000 7f123f8b0740 10 client.27208 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.465+0000 7f123f8b0740 10 client.27208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.465+0000 7f123f8b0740 20 client.27208 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.465+0000 7f123f8b0740 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c8ec4da0 0x55c2c8ec7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.465+0000 7f123f8b0740 4 client.27208 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:47.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.465+0000 7f123f8b0740 1 -- 172.21.15.69:0/104400822 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c8e0bef0 con 0x55c2c8ec4da0 2024-08-02T17:56:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.466+0000 7f1236ffd640 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c8ec4da0 0x55c2c8ec7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.466+0000 7f121aff5640 10 client.27208 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:47.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.466+0000 7f1236ffd640 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c8ec4da0 0x55c2c8ec7190 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a489d8041965331f server_cookie=14c7027dd7cd8963 in_seq=0 out_seq=0 2024-08-02T17:56:47.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.471+0000 7f121aff5640 1 -- 172.21.15.69:0/104400822 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c2c8e0bef0 con 0x55c2c8ec4da0 2024-08-02T17:56:47.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.471+0000 7f121aff5640 10 client.27208 handle_command_reply: tid=0 2024-08-02T17:56:47.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.472+0000 7f123f8b0740 1 client.27208 shutdown 2024-08-02T17:56:47.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.475+0000 7f123f8b0740 1 -- 172.21.15.69:0/104400822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8eb5400 msgr2=0x55c2c8eb2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.475+0000 7f123f8b0740 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb2ca0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f123009bde0 tx=0x7f123009a360 comp rx=0 tx=0).stop 2024-08-02T17:56:47.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.475+0000 7f123f8b0740 1 -- 172.21.15.69:0/104400822 shutdown_connections 2024-08-02T17:56:47.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.475+0000 7f123f8b0740 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8eb2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.475+0000 7f123f8b0740 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c8ec4da0 0x55c2c8ec7190 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.475+0000 7f123f8b0740 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8eb5400 0x55c2c8eb2ca0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.475+0000 7f123f8b0740 1 --2- 172.21.15.69:0/104400822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.475+0000 7f123f8b0740 1 -- 172.21.15.69:0/104400822 >> 172.21.15.69:0/104400822 conn(0x55c2c8e957a0 msgr2=0x55c2c8e9b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.476+0000 7f121aff5640 0 client.27208 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:47.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.481+0000 7f123f8b0740 1 -- 172.21.15.69:0/104400822 shutdown_connections 2024-08-02T17:56:47.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.481+0000 7f123f8b0740 1 -- 172.21.15.69:0/104400822 wait complete. 2024-08-02T17:56:47.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.482+0000 7f123f8b0740 20 client.27208 trim_cache size 0 max 16384 2024-08-02T17:56:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.520+0000 7f123f8b0740 1 Processor -- start 2024-08-02T17:56:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f123f8b0740 1 -- start start 2024-08-02T17:56:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f123f8b0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1224012aa0 con 0x55c2c8e999d0 2024-08-02T17:56:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f123f8b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1224010370 con 0x55c2c8eb5400 2024-08-02T17:56:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f123f8b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f122400e1c0 con 0x55c2c8e990b0 2024-08-02T17:56:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f12377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f12377fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45760/0 (socket says 172.21.15.69:45760) 2024-08-02T17:56:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f12377fe640 1 -- 172.21.15.69:0/4127178229 learned_addr learned my addr 172.21.15.69:0/4127178229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.521+0000 7f1236ffd640 1 --2- 172.21.15.69:0/4127178229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.522+0000 7f1236ffd640 1 -- 172.21.15.69:0/4127178229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 msgr2=0x55c2c8eca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.522+0000 7f1236ffd640 1 --2- 172.21.15.69:0/4127178229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eca310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.522+0000 7f1236ffd640 1 -- 172.21.15.69:0/4127178229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 msgr2=0x55c2c8ec9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.522+0000 7f1236ffd640 1 --2- 172.21.15.69:0/4127178229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.522+0000 7f1236ffd640 1 -- 172.21.15.69:0/4127178229 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c8e99800 con 0x55c2c8e990b0 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.525+0000 7f1236ffd640 1 --2- 172.21.15.69:0/4127178229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2af0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f122802aa30 tx=0x7f1228042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1de8d2dec966f00 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.527+0000 7f121cff9640 1 -- 172.21.15.69:0/4127178229 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1228005200 con 0x55c2c8e990b0 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.527+0000 7f121cff9640 1 -- 172.21.15.69:0/4127178229 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1228058070 con 0x55c2c8e990b0 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.528+0000 7f123f8b0740 1 -- 172.21.15.69:0/4127178229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 msgr2=0x55c2c8eb2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.528+0000 7f123f8b0740 1 --2- 172.21.15.69:0/4127178229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2af0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f122802aa30 tx=0x7f1228042000 comp rx=0 tx=0).stop 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 -- 172.21.15.69:0/4127178229 shutdown_connections 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 --2- 172.21.15.69:0/4127178229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 --2- 172.21.15.69:0/4127178229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8eb5400 0x55c2c8eca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 --2- 172.21.15.69:0/4127178229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e990b0 0x55c2c8eb2af0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 -- 172.21.15.69:0/4127178229 >> 172.21.15.69:0/4127178229 conn(0x55c2c8e957a0 msgr2=0x55c2c8ec0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 -- 172.21.15.69:0/4127178229 shutdown_connections 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 -- 172.21.15.69:0/4127178229 wait complete. 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 -- start start 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.529+0000 7f123f8b0740 1 Processor -- start 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 0x55c2c8ec7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f123f8b0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8eb5400 0x55c2c8f26690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f123f8b0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f122400e830 con 0x55c2c8eb5400 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f123f8b0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f122400fca0 con 0x55c2c8e990b0 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f123f8b0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f122400ef00 con 0x55c2c8e999d0 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f123f8b0740 10 client.0 _ll_register_callbacks cb 0x7f123009b860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:47.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f12367fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f12367fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:34172/0 (socket says 172.21.15.69:34172) 2024-08-02T17:56:47.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f12367fc640 1 -- 172.21.15.69:0/2556867854 learned_addr learned my addr 172.21.15.69:0/2556867854 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:47.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.530+0000 7f1236ffd640 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 0x55c2c8ec7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.538+0000 7f12367fc640 1 -- 172.21.15.69:0/2556867854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 msgr2=0x55c2c8ec7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.538+0000 7f12367fc640 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 0x55c2c8ec7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.538+0000 7f12367fc640 1 -- 172.21.15.69:0/2556867854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8eb5400 msgr2=0x55c2c8f26690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.538+0000 7f12367fc640 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8eb5400 0x55c2c8f26690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.538+0000 7f12367fc640 1 -- 172.21.15.69:0/2556867854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c8f26cd0 con 0x55c2c8e999d0 2024-08-02T17:56:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.544+0000 7f12367fc640 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec7b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55c2c8eb3860 tx=0x7f122c010b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76b29736b3029ce6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.544+0000 7f121aff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:47.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.544+0000 7f121aff5640 1 -- 172.21.15.69:0/2556867854 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c00fcc0 con 0x55c2c8e999d0 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.545+0000 7f123f8b0740 1 -- 172.21.15.69:0/2556867854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1228058070 con 0x55c2c8e999d0 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.545+0000 7f121aff5640 1 -- 172.21.15.69:0/2556867854 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f122c005020 con 0x55c2c8e999d0 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.545+0000 7f121aff5640 1 -- 172.21.15.69:0/2556867854 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f122c00fe60 con 0x55c2c8e999d0 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.548+0000 7f123f8b0740 10 client.27214 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.548+0000 7f123f8b0740 1 -- 172.21.15.69:0/2556867854 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2c8f2f320 con 0x55c2c8e999d0 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.548+0000 7f121aff5640 1 -- 172.21.15.69:0/2556867854 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1228058070 con 0x55c2c8e999d0 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.548+0000 7f121aff5640 1 -- 172.21.15.69:0/2556867854 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f122c01b070 con 0x55c2c8e999d0 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.548+0000 7f123f8b0740 10 client.27214 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.548+0000 7f123f8b0740 10 client.27214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.548+0000 7f123f8b0740 20 client.27214 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:47.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.549+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c8f2fb50 0x55c2c8f31f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:47.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.549+0000 7f123f8b0740 4 client.27214 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:56:47.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.549+0000 7f123f8b0740 1 -- 172.21.15.69:0/2556867854 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2c8e06400 con 0x55c2c8f2fb50 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.551+0000 7f1236ffd640 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c8f2fb50 0x55c2c8f31f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.552+0000 7f1236ffd640 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c8f2fb50 0x55c2c8f31f70 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368fccc35b59f647 server_cookie=9a7eada8fb7e17de in_seq=0 out_seq=0 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.552+0000 7f121aff5640 10 client.27214 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.552+0000 7f121aff5640 1 -- 172.21.15.69:0/2556867854 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c2c8e06400 con 0x55c2c8f2fb50 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.552+0000 7f121aff5640 10 client.27214 handle_command_reply: tid=0 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.552+0000 7f123f8b0740 1 client.27214 shutdown 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.552+0000 7f123f8b0740 1 -- 172.21.15.69:0/2556867854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e999d0 msgr2=0x55c2c8ec7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.552+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec7b50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55c2c8eb3860 tx=0x7f122c010b40 comp rx=0 tx=0).stop 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.553+0000 7f123f8b0740 1 -- 172.21.15.69:0/2556867854 shutdown_connections 2024-08-02T17:56:47.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.553+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c8eb5400 0x55c2c8f26690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.553+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c8f2fb50 0x55c2c8f31f70 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.553+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c8e999d0 0x55c2c8ec7b50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.553+0000 7f123f8b0740 1 --2- 172.21.15.69:0/2556867854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c8e990b0 0x55c2c8ec7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.553+0000 7f123f8b0740 1 -- 172.21.15.69:0/2556867854 >> 172.21.15.69:0/2556867854 conn(0x55c2c8e957a0 msgr2=0x55c2c8ec0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.553+0000 7f121aff5640 0 client.27214 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:47.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.556+0000 7f123f8b0740 1 -- 172.21.15.69:0/2556867854 shutdown_connections 2024-08-02T17:56:47.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.556+0000 7f123f8b0740 1 -- 172.21.15.69:0/2556867854 wait complete. 2024-08-02T17:56:47.563 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.559+0000 7f123f8b0740 20 client.27214 trim_cache size 0 max 16384 2024-08-02T17:56:47.568 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:56:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:56:47.568 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:56:47.568 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.571+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12200a2f00 msgr2=0x7f12200a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.571+0000 7f123d4b2640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12200a2f00 0x7f12200a53c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f12280096c0 tx=0x7f1228005fd0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.571+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123811af40 msgr2=0x7f12381ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.571+0000 7f123d4b2640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123811af40 0x7f12381ab8c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f123000b9e0 tx=0x7f123000bea0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.574+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 shutdown_connections 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.574+0000 7f123d4b2640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12200a2f00 0x7f12200a53c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.574+0000 7f123d4b2640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f123811af40 0x7f12381ab8c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.574+0000 7f123d4b2640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f123806e1f0 0x7f12381a7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.574+0000 7f123d4b2640 1 --2- 172.21.15.69:0/4242077182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f123806d8d0 0x7f12381a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.574+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 >> 172.21.15.69:0/4242077182 conn(0x7f123806caf0 msgr2=0x7f123811b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:47.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.574+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 shutdown_connections 2024-08-02T17:56:47.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:47.574+0000 7f123d4b2640 1 -- 172.21.15.69:0/4242077182 wait complete. 2024-08-02T17:56:47.603 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:47.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:47 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:56:48.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:47 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:56:48.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:47 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:56:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:48 smithi083 ceph-mon[37031]: pgmap v322: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 841 op/s 2024-08-02T17:56:49.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:48 smithi077 ceph-mon[37108]: pgmap v322: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 841 op/s 2024-08-02T17:56:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:48 smithi069 ceph-mon[32746]: pgmap v322: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.3 MiB/s wr, 841 op/s 2024-08-02T17:56:49.862 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:56:49.862 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:56:49.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 580.0 s, 414.6 tps, lat 244.488 ms stddev 290.018 2024-08-02T17:56:50.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.106+0000 7f0ea99e0640 1 -- 172.21.15.69:0/988003942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103ac0 msgr2=0x7f0ea4103f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.106+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/988003942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103ac0 0x7f0ea4103f20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009930 tx=0x7f0e9802a440 comp rx=0 tx=0).stop 2024-08-02T17:56:50.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.107+0000 7f0ea99e0640 1 -- 172.21.15.69:0/988003942 shutdown_connections 2024-08-02T17:56:50.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.107+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/988003942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea4104460 0x7f0ea4110990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.108+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/988003942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103ac0 0x7f0ea4103f20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.108+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/988003942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4109ac0 0x7f0ea4109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.108+0000 7f0ea99e0640 1 -- 172.21.15.69:0/988003942 >> 172.21.15.69:0/988003942 conn(0x7f0ea40ff7e0 msgr2=0x7f0ea4101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:50.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.108+0000 7f0ea99e0640 1 -- 172.21.15.69:0/988003942 shutdown_connections 2024-08-02T17:56:50.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.108+0000 7f0ea99e0640 1 -- 172.21.15.69:0/988003942 wait complete. 2024-08-02T17:56:50.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.110+0000 7f0ea99e0640 1 Processor -- start 2024-08-02T17:56:50.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.110+0000 7f0ea99e0640 1 -- start start 2024-08-02T17:56:50.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.110+0000 7f0ea99e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103ac0 0x7f0ea4073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.110+0000 7f0ea99e0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4104460 0x7f0ea406ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.110+0000 7f0ea99e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea4109ac0 0x7f0ea406bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.110+0000 7f0ea99e0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4113ec0 con 0x7f0ea4109ac0 2024-08-02T17:56:50.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.110+0000 7f0ea99e0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4113d40 con 0x7f0ea4104460 2024-08-02T17:56:50.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.110+0000 7f0ea99e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea4114040 con 0x7f0ea4103ac0 2024-08-02T17:56:50.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.111+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea4109ac0 0x7f0ea406bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.111+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea4109ac0 0x7f0ea406bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51778/0 (socket says 172.21.15.69:51778) 2024-08-02T17:56:50.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.111+0000 7f0ea37fe640 1 -- 172.21.15.69:0/3464351501 learned_addr learned my addr 172.21.15.69:0/3464351501 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:50.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.111+0000 7f0ea27fc640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4104460 0x7f0ea406ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.111+0000 7f0ea2ffd640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103ac0 0x7f0ea4073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea27fc640 1 -- 172.21.15.69:0/3464351501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103ac0 msgr2=0x7f0ea4073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea27fc640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103ac0 0x7f0ea4073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea27fc640 1 -- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea4109ac0 msgr2=0x7f0ea406bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea27fc640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea4109ac0 0x7f0ea406bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea27fc640 1 -- 172.21.15.69:0/3464351501 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea406c770 con 0x7f0ea4104460 2024-08-02T17:56:50.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea27fc640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4104460 0x7f0ea406ba10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005940 tx=0x7f0e98031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:50.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea89de640 1 -- 172.21.15.69:0/3464351501 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9803f070 con 0x7f0ea4104460 2024-08-02T17:56:50.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea99e0640 1 -- 172.21.15.69:0/3464351501 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea41af110 con 0x7f0ea4104460 2024-08-02T17:56:50.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.112+0000 7f0ea89de640 1 -- 172.21.15.69:0/3464351501 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e98003ad0 con 0x7f0ea4104460 2024-08-02T17:56:50.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.113+0000 7f0ea89de640 1 -- 172.21.15.69:0/3464351501 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98035070 con 0x7f0ea4104460 2024-08-02T17:56:50.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.113+0000 7f0ea99e0640 1 -- 172.21.15.69:0/3464351501 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea41af640 con 0x7f0ea4104460 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.116+0000 7f0ea89de640 1 -- 172.21.15.69:0/3464351501 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0e98003c80 con 0x7f0ea4104460 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.117+0000 7f0eabdde740 1 Processor -- start 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0eabdde740 1 -- start start 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0ea89de640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0e8c0a2bd0 0x7f0e8c0a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9ce510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9db700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0eabdde740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556efe933a70 con 0x556efe9d9310 2024-08-02T17:56:50.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0eabdde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556efe8cb780 con 0x556efe9ce130 2024-08-02T17:56:50.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0eabdde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556efe92fa30 con 0x556efe9cea50 2024-08-02T17:56:50.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0ea89de640 1 -- 172.21.15.69:0/3464351501 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f0e980ef670 con 0x7f0ea4104460 2024-08-02T17:56:50.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0ea27fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0ea37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9db700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51792/0 (socket says 172.21.15.69:51792) 2024-08-02T17:56:50.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0ea37fe640 1 -- 172.21.15.69:0/2342521924 learned_addr learned my addr 172.21.15.69:0/2342521924 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:50.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.118+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0e8c0a2bd0 0x7f0e8c0a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.119+0000 7f0ea37fe640 1 -- 172.21.15.69:0/2342521924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 msgr2=0x556efe9d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:50.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.119+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/2342521924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.119+0000 7f0ea37fe640 1 -- 172.21.15.69:0/2342521924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 msgr2=0x556efe9ce510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:50.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.119+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/2342521924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.119+0000 7f0ea37fe640 1 -- 172.21.15.69:0/2342521924 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556efe6a5b90 con 0x556efe9d9310 2024-08-02T17:56:50.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.119+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/2342521924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9db700 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c012af0 tx=0x7f0e9c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de033168cf387792 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:50.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.119+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0e8c0a2bd0 0x7f0e8c0a5090 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0108d0 tx=0x7f0e9c008000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:50.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.120+0000 7f0e89ffb640 1 -- 172.21.15.69:0/2342521924 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c023070 con 0x556efe9d9310 2024-08-02T17:56:50.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.120+0000 7f0e89ffb640 1 -- 172.21.15.69:0/2342521924 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c0125f0 con 0x556efe9d9310 2024-08-02T17:56:50.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.120+0000 7f0e89ffb640 1 -- 172.21.15.69:0/2342521924 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c01f480 con 0x556efe9d9310 2024-08-02T17:56:50.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.121+0000 7f0eabdde740 1 -- 172.21.15.69:0/2342521924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 msgr2=0x556efe9db700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.121+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2342521924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9db700 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c012af0 tx=0x7f0e9c0103d0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.123+0000 7f0eabdde740 1 -- 172.21.15.69:0/2342521924 shutdown_connections 2024-08-02T17:56:50.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.123+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2342521924 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9db700 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.123+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2342521924 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.123+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2342521924 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.123+0000 7f0eabdde740 1 -- 172.21.15.69:0/2342521924 >> 172.21.15.69:0/2342521924 conn(0x556efe9ca6c0 msgr2=0x556efe9caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:50.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.123+0000 7f0eabdde740 1 -- 172.21.15.69:0/2342521924 shutdown_connections 2024-08-02T17:56:50.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.123+0000 7f0eabdde740 1 -- 172.21.15.69:0/2342521924 wait complete. 2024-08-02T17:56:50.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.123+0000 7f0eabdde740 1 -- start start 2024-08-02T17:56:50.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.124+0000 7f0eabdde740 1 Processor -- start 2024-08-02T17:56:50.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.124+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9d6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.124+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 0x556efe9d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9d9310 0x556efe9d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0eabdde740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556efe9dbee0 con 0x556efe9cea50 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0eabdde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556efe8ee990 con 0x556efe9ce130 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0eabdde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556efe9dc060 con 0x556efe9d9310 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 0x556efe9d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea27fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 0x556efe9d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51804/0 (socket says 172.21.15.69:51804) 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea27fc640 1 -- 172.21.15.69:0/1763708123 learned_addr learned my addr 172.21.15.69:0/1763708123 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9d9310 0x556efe9d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0eabdde740 10 client.0 _ll_register_callbacks cb 0x556efe643090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea37fe640 1 -- 172.21.15.69:0/1763708123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 msgr2=0x556efe9d6280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9d6280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea37fe640 1 -- 172.21.15.69:0/1763708123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 msgr2=0x556efe9d67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:50.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 0x556efe9d67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.125+0000 7f0ea37fe640 1 -- 172.21.15.69:0/1763708123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556efe8ed6c0 con 0x556efe9d9310 2024-08-02T17:56:50.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.126+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9d9310 0x556efe9d6d00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c012af0 tx=0x7f0e9c009c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbc6e058ac4d8498 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:50.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.126+0000 7f0e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:50.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.126+0000 7f0e82ffd640 1 -- 172.21.15.69:0/1763708123 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c023070 con 0x556efe9d9310 2024-08-02T17:56:50.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.126+0000 7f0e82ffd640 1 -- 172.21.15.69:0/1763708123 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f0e9c00dad0 con 0x556efe9d9310 2024-08-02T17:56:50.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.126+0000 7f0eabdde740 1 -- 172.21.15.69:0/1763708123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c0125f0 con 0x556efe9d9310 2024-08-02T17:56:50.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.127+0000 7f0e82ffd640 1 -- 172.21.15.69:0/1763708123 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e9c00dcd0 con 0x556efe9d9310 2024-08-02T17:56:50.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.127+0000 7f0eabdde740 10 client.27226 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:50.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.127+0000 7f0eabdde740 1 -- 172.21.15.69:0/1763708123 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x556efe9e7a90 con 0x556efe9d9310 2024-08-02T17:56:50.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.129+0000 7f0e82ffd640 1 -- 172.21.15.69:0/1763708123 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0e9c030030 con 0x556efe9d9310 2024-08-02T17:56:50.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.129+0000 7f0eabdde740 10 client.27226 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:50.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.129+0000 7f0eabdde740 10 client.27226 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:56:50.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.129+0000 7f0eabdde740 20 client.27226 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:50.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.129+0000 7f0eabdde740 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556efe9e8de0 0x556efe9eb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.129+0000 7f0eabdde740 4 client.27226 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:50.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.129+0000 7f0eabdde740 1 -- 172.21.15.69:0/1763708123 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0e9c0125f0 con 0x556efe9e8de0 2024-08-02T17:56:50.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.130+0000 7f0ea2ffd640 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556efe9e8de0 0x556efe9eb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.130+0000 7f0e82ffd640 10 client.27226 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:50.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.130+0000 7f0ea2ffd640 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556efe9e8de0 0x556efe9eb1d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6f45ac54d18cafb server_cookie=dd1d51306b08bcd1 in_seq=0 out_seq=0 2024-08-02T17:56:50.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.131+0000 7f0e82ffd640 1 -- 172.21.15.69:0/1763708123 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f0e9c0125f0 con 0x556efe9e8de0 2024-08-02T17:56:50.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.131+0000 7f0e82ffd640 10 client.27226 handle_command_reply: tid=0 2024-08-02T17:56:50.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.131+0000 7f0eabdde740 1 client.27226 shutdown 2024-08-02T17:56:50.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.131+0000 7f0eabdde740 1 -- 172.21.15.69:0/1763708123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9d9310 msgr2=0x556efe9d6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.131+0000 7f0eabdde740 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9d9310 0x556efe9d6d00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c012af0 tx=0x7f0e9c009c50 comp rx=0 tx=0).stop 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 1 -- 172.21.15.69:0/1763708123 shutdown_connections 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 0x556efe9d67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556efe9e8de0 0x556efe9eb1d0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9d9310 0x556efe9d6d00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 1 --2- 172.21.15.69:0/1763708123 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 1 -- 172.21.15.69:0/1763708123 >> 172.21.15.69:0/1763708123 conn(0x556efe9ca6c0 msgr2=0x556efe9d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0e82ffd640 0 client.27226 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 1 -- 172.21.15.69:0/1763708123 shutdown_connections 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 1 -- 172.21.15.69:0/1763708123 wait complete. 2024-08-02T17:56:50.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.132+0000 7f0eabdde740 20 client.27226 trim_cache size 0 max 16384 2024-08-02T17:56:50.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.168+0000 7f0eabdde740 1 Processor -- start 2024-08-02T17:56:50.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.168+0000 7f0eabdde740 1 -- start start 2024-08-02T17:56:50.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0ea2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0ea2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9f0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45818/0 (socket says 172.21.15.69:45818) 2024-08-02T17:56:50.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0ea2ffd640 1 -- 172.21.15.69:0/56267618 learned_addr learned my addr 172.21.15.69:0/56267618 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:50.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0ea27fc640 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0eabdde740 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9d7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0eabdde740 1 -- 172.21.15.69:0/56267618 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41bbe60 con 0x556efe9d9310 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0eabdde740 1 -- 172.21.15.69:0/56267618 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b9730 con 0x556efe9ce130 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0eabdde740 1 -- 172.21.15.69:0/56267618 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b7580 con 0x556efe9cea50 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.169+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9d7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.170+0000 7f0ea37fe640 1 -- 172.21.15.69:0/56267618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 msgr2=0x556efe9f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.170+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.170+0000 7f0ea37fe640 1 -- 172.21.15.69:0/56267618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 msgr2=0x556efe9f0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:50.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.170+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9f0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.170+0000 7f0ea37fe640 1 -- 172.21.15.69:0/56267618 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556efe9ce880 con 0x556efe9d9310 2024-08-02T17:56:50.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.170+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9d7ae0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009580 tx=0x7f0e9c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73e52badc5c6435 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:50.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.171+0000 7f0e897fa640 1 -- 172.21.15.69:0/56267618 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c00dad0 con 0x556efe9d9310 2024-08-02T17:56:50.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.171+0000 7f0e897fa640 1 -- 172.21.15.69:0/56267618 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c01fbe0 con 0x556efe9d9310 2024-08-02T17:56:50.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.171+0000 7f0e897fa640 1 -- 172.21.15.69:0/56267618 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c004680 con 0x556efe9d9310 2024-08-02T17:56:50.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.171+0000 7f0ea2ffd640 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9f0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:56:50.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.171+0000 7f0ea27fc640 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:56:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.172+0000 7f0eabdde740 1 -- 172.21.15.69:0/56267618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 msgr2=0x556efe9d7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.172+0000 7f0eabdde740 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9d7ae0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c009580 tx=0x7f0e9c023000 comp rx=0 tx=0).stop 2024-08-02T17:56:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 -- 172.21.15.69:0/56267618 shutdown_connections 2024-08-02T17:56:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9d9310 0x556efe9d7ae0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9cea50 0x556efe9f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 --2- 172.21.15.69:0/56267618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9ce130 0x556efe9f0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 -- 172.21.15.69:0/56267618 >> 172.21.15.69:0/56267618 conn(0x556efe9ca6c0 msgr2=0x556efe9eadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:50.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 -- 172.21.15.69:0/56267618 shutdown_connections 2024-08-02T17:56:50.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 -- 172.21.15.69:0/56267618 wait complete. 2024-08-02T17:56:50.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 -- start start 2024-08-02T17:56:50.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.173+0000 7f0eabdde740 1 Processor -- start 2024-08-02T17:56:50.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9ce130 0x556efe9e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0eabdde740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 0x556efe9d3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0ea2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9ce130 0x556efe9e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0ea2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9ce130 0x556efe9e1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:34244/0 (socket says 172.21.15.69:34244) 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0ea2ffd640 1 -- 172.21.15.69:0/2579170433 learned_addr learned my addr 172.21.15.69:0/2579170433 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9d9310 0x556efe9d3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b7bf0 con 0x556efe9cea50 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b9060 con 0x556efe9d9310 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea41b82c0 con 0x556efe9ce130 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.174+0000 7f0ea37fe640 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9d9310 0x556efe9d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.175+0000 7f0eabdde740 10 client.0 _ll_register_callbacks cb 0x7f0e98005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:50.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.175+0000 7f0ea2ffd640 1 -- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9d9310 msgr2=0x556efe9d3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.175+0000 7f0ea2ffd640 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9d9310 0x556efe9d3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.175+0000 7f0ea2ffd640 1 -- 172.21.15.69:0/2579170433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 msgr2=0x556efe9d3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.175+0000 7f0ea2ffd640 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 0x556efe9d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.175+0000 7f0ea2ffd640 1 -- 172.21.15.69:0/2579170433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556efe9d3f30 con 0x556efe9ce130 2024-08-02T17:56:50.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.175+0000 7f0ea2ffd640 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9ce130 0x556efe9e1b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e94005c40 tx=0x7f0e940118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac0396240ee0900a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:50.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.176+0000 7f0e82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:50.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.176+0000 7f0e82ffd640 1 -- 172.21.15.69:0/2579170433 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94003b70 con 0x556efe9ce130 2024-08-02T17:56:50.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.176+0000 7f0e82ffd640 1 -- 172.21.15.69:0/2579170433 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f0e94003d10 con 0x556efe9ce130 2024-08-02T17:56:50.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.176+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9c01fbe0 con 0x556efe9ce130 2024-08-02T17:56:50.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.177+0000 7f0e82ffd640 1 -- 172.21.15.69:0/2579170433 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0e9402a050 con 0x556efe9ce130 2024-08-02T17:56:50.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.177+0000 7f0eabdde740 10 client.27229 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:50.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.177+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556efea5c1c0 con 0x556efe9ce130 2024-08-02T17:56:50.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.177+0000 7f0e82ffd640 1 -- 172.21.15.69:0/2579170433 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e9c01fbe0 con 0x556efe9ce130 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.178+0000 7f0e82ffd640 1 -- 172.21.15.69:0/2579170433 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0e94004470 con 0x556efe9ce130 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.178+0000 7f0eabdde740 10 client.27229 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.178+0000 7f0eabdde740 10 client.27229 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.178+0000 7f0eabdde740 20 client.27229 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.178+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556efea5c950 0x556efea5ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.178+0000 7f0eabdde740 4 client.27229 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.9, "await_for": 45.0}] 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.178+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.9, "await_for": 45.0}) -- 0x556efe9411a0 con 0x556efea5c950 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.179+0000 7f0ea27fc640 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556efea5c950 0x556efea5ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.179+0000 7f0ea27fc640 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556efea5c950 0x556efea5ed60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b75fad14fae8ac server_cookie=a4bb8fbe18c87d16 in_seq=0 out_seq=0 2024-08-02T17:56:50.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.179+0000 7f0e82ffd640 10 client.27229 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:50.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0e82ffd640 1 -- 172.21.15.69:0/2579170433 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556efe9411a0 con 0x556efea5c950 2024-08-02T17:56:50.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0e82ffd640 10 client.27229 handle_command_reply: tid=0 2024-08-02T17:56:50.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 client.27229 shutdown 2024-08-02T17:56:50.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9ce130 msgr2=0x556efe9e1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9ce130 0x556efe9e1b20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0e94005c40 tx=0x7f0e940118d0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 shutdown_connections 2024-08-02T17:56:50.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556efe9cea50 0x556efe9d3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556efea5c950 0x556efea5ed60 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556efe9d9310 0x556efe9d3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 --2- 172.21.15.69:0/2579170433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556efe9ce130 0x556efe9e1b20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0e82ffd640 0 client.27229 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:50.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 >> 172.21.15.69:0/2579170433 conn(0x556efe9ca6c0 msgr2=0x556efe9d6240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:50.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 shutdown_connections 2024-08-02T17:56:50.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 1 -- 172.21.15.69:0/2579170433 wait complete. 2024-08-02T17:56:50.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.488+0000 7f0eabdde740 20 client.27229 trim_cache size 0 max 16384 2024-08-02T17:56:50.495 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:56:50.495 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:56:50.495 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:56:50.495 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 34, 2024-08-02T17:56:50.495 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:56:50.496 INFO:teuthology.orchestra.run.smithi069.stdout: "71426704": { 2024-08-02T17:56:50.496 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 34, 2024-08-02T17:56:50.496 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:56:50.496 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:56:50.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:56:50.496 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:56:50.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:56:50.497 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:56:50.497 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 139.9, 2024-08-02T17:56:50.497 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:56:50.497 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:56:50.497 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:56:50.497 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:56:50.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:56:50.498 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:56:50.498 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:50.498 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:50.498 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:50.498 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:50.498 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:50.499 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:56:50.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 -- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0e8c0a2bd0 msgr2=0x7f0e8c0a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0e8c0a2bd0 0x7f0e8c0a5090 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0108d0 tx=0x7f0e9c008000 comp rx=0 tx=0).stop 2024-08-02T17:56:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 -- 172.21.15.69:0/3464351501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4104460 msgr2=0x7f0ea406ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4104460 0x7f0ea406ba10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005940 tx=0x7f0e98031b50 comp rx=0 tx=0).stop 2024-08-02T17:56:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 -- 172.21.15.69:0/3464351501 shutdown_connections 2024-08-02T17:56:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ea4109ac0 0x7f0ea406bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0e8c0a2bd0 0x7f0e8c0a5090 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ea4104460 0x7f0ea406ba10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 --2- 172.21.15.69:0/3464351501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ea4103ac0 0x7f0ea4073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 -- 172.21.15.69:0/3464351501 >> 172.21.15.69:0/3464351501 conn(0x7f0ea40ff7e0 msgr2=0x7f0ea40fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 -- 172.21.15.69:0/3464351501 shutdown_connections 2024-08-02T17:56:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:50.489+0000 7f0ea99e0640 1 -- 172.21.15.69:0/3464351501 wait complete. 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: pgmap v323: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.4 MiB/s wr, 755 op/s 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:50.768 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:50 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: pgmap v323: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.4 MiB/s wr, 755 op/s 2024-08-02T17:56:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:50 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: pgmap v323: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.4 MiB/s wr, 755 op/s 2024-08-02T17:56:51.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:50 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:51.366 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:51.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:51 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-08-02T17:56:51.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:51 smithi069 ceph-mon[32746]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-08-02T17:56:51.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:51 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:51 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:51 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:51 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:51.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:51 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-08-02T17:56:51.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:51 smithi069 ceph-mon[32746]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-08-02T17:56:51.921 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '71426704', quiesce took 16.9 seconds. Will release after: 3.0 2024-08-02T17:56:52.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:51 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-08-02T17:56:52.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:51 smithi083 ceph-mon[37031]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-08-02T17:56:52.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:51 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:52.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:51 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:52.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:51 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:52.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:51 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:52.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:51 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-08-02T17:56:52.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:51 smithi083 ceph-mon[37031]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-08-02T17:56:52.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:51 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi083 to 176.2M 2024-08-02T17:56:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:51 smithi077 ceph-mon[37108]: Unable to set osd_memory_target on smithi083 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-08-02T17:56:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:51 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:51 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:51 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:51 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:51 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi077 to 176.2M 2024-08-02T17:56:52.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:51 smithi077 ceph-mon[37108]: Unable to set osd_memory_target on smithi077 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-08-02T17:56:52.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 -- 172.21.15.69:0/676730430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258102a70 msgr2=0x7f2258102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:52.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 --2- 172.21.15.69:0/676730430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258102a70 0x7f2258102e50 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f2254009900 tx=0x7f225402a480 comp rx=0 tx=0).stop 2024-08-02T17:56:52.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 -- 172.21.15.69:0/676730430 shutdown_connections 2024-08-02T17:56:52.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 --2- 172.21.15.69:0/676730430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f225810bea0 0x7f225810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 --2- 172.21.15.69:0/676730430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2258103390 0x7f225810b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 --2- 172.21.15.69:0/676730430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258102a70 0x7f2258102e50 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 -- 172.21.15.69:0/676730430 >> 172.21.15.69:0/676730430 conn(0x7f22580fc4d0 msgr2=0x7f22580fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 -- 172.21.15.69:0/676730430 shutdown_connections 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.353+0000 7f2260467640 1 -- 172.21.15.69:0/676730430 wait complete. 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f2260467640 1 Processor -- start 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f2260467640 1 -- start start 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f2260467640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2258102a70 0x7f2258198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f2260467640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258103390 0x7f22581987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f2260467640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f225810bea0 0x7f225819cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f2260467640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2258111d00 con 0x7f2258103390 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f2260467640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2258111b80 con 0x7f2258102a70 2024-08-02T17:56:52.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f2260467640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2258111e80 con 0x7f225810bea0 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225e203640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2258102a70 0x7f2258198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225e203640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2258102a70 0x7f2258198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:45838/0 (socket says 172.21.15.69:45838) 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225e203640 1 -- 172.21.15.69:0/92943433 learned_addr learned my addr 172.21.15.69:0/92943433 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225da02640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258103390 0x7f22581987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225ea04640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f225810bea0 0x7f225819cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225da02640 1 -- 172.21.15.69:0/92943433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f225810bea0 msgr2=0x7f225819cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225da02640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f225810bea0 0x7f225819cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225da02640 1 -- 172.21.15.69:0/92943433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2258102a70 msgr2=0x7f2258198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225da02640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2258102a70 0x7f2258198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.354+0000 7f225da02640 1 -- 172.21.15.69:0/92943433 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225819d210 con 0x7f2258103390 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.355+0000 7f225da02640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258103390 0x7f22581987a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f224800d650 tx=0x7f224800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.355+0000 7f224f7fe640 1 -- 172.21.15.69:0/92943433 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2248017070 con 0x7f2258103390 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.355+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225819d500 con 0x7f2258103390 2024-08-02T17:56:52.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.355+0000 7f224f7fe640 1 -- 172.21.15.69:0/92943433 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2248005500 con 0x7f2258103390 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.355+0000 7f224f7fe640 1 -- 172.21.15.69:0/92943433 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22480047a0 con 0x7f2258103390 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.355+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22581a4de0 con 0x7f2258103390 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.356+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2258102e50 con 0x7f2258103390 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.358+0000 7f224f7fe640 1 -- 172.21.15.69:0/92943433 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2248005020 con 0x7f2258103390 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.358+0000 7f224f7fe640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22440a2d70 0x7f22440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.359+0000 7f225e203640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22440a2d70 0x7f22440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.359+0000 7f224f7fe640 1 -- 172.21.15.69:0/92943433 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f22480cf750 con 0x7f2258103390 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.359+0000 7f225e203640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22440a2d70 0x7f22440a5230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22540096c0 tx=0x7f2254005a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:52.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.362+0000 7f224f7fe640 1 -- 172.21.15.69:0/92943433 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2248020020 con 0x7f2258103390 2024-08-02T17:56:52.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.582+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22581995c0 con 0x7f2258103390 2024-08-02T17:56:52.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.582+0000 7f224f7fe640 1 -- 172.21.15.69:0/92943433 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f224809bfe0 con 0x7f2258103390 2024-08-02T17:56:52.584 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:56:52.584 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:56:52.584 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:56:52.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.584+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22440a2d70 msgr2=0x7f22440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:52.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.584+0000 7f2260467640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22440a2d70 0x7f22440a5230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22540096c0 tx=0x7f2254005a20 comp rx=0 tx=0).stop 2024-08-02T17:56:52.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.584+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258103390 msgr2=0x7f22581987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:52.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.584+0000 7f2260467640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258103390 0x7f22581987a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f224800d650 tx=0x7f224800db10 comp rx=0 tx=0).stop 2024-08-02T17:56:52.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.585+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 shutdown_connections 2024-08-02T17:56:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.585+0000 7f2260467640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2258103390 0x7f22581987a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.585+0000 7f2260467640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f22440a2d70 0x7f22440a5230 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.585+0000 7f2260467640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f225810bea0 0x7f225819cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.585+0000 7f2260467640 1 --2- 172.21.15.69:0/92943433 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2258102a70 0x7f2258198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.585+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 >> 172.21.15.69:0/92943433 conn(0x7f22580fc4d0 msgr2=0x7f22580fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.585+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 shutdown_connections 2024-08-02T17:56:52.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:52.585+0000 7f2260467640 1 -- 172.21.15.69:0/92943433 wait complete. 2024-08-02T17:56:52.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:52 smithi069 ceph-mon[32746]: pgmap v324: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 661 op/s 2024-08-02T17:56:52.933 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:52 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/92943433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:53.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:52 smithi083 ceph-mon[37031]: pgmap v324: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 661 op/s 2024-08-02T17:56:53.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:52 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/92943433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:53.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:52 smithi077 ceph-mon[37108]: pgmap v324: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.7 MiB/s wr, 661 op/s 2024-08-02T17:56:53.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:52 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/92943433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:53.385 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:56:53.677 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:54.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:56:54.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:56:54.109 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:53 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:56:54.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:56:54.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:53 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-08-02T17:56:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:56:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:56:54.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:53 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:56:54.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 -- 172.21.15.69:0/268709590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 msgr2=0x7fb30c10d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 --2- 172.21.15.69:0/268709590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 0x7fb30c10d680 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb304009cd0 tx=0x7fb30402a8e0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 -- 172.21.15.69:0/268709590 shutdown_connections 2024-08-02T17:56:54.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 --2- 172.21.15.69:0/268709590 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 0x7fb30c10d680 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 --2- 172.21.15.69:0/268709590 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb30c1007b0 0x7fb30c100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 --2- 172.21.15.69:0/268709590 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c1067b0 0x7fb30c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 -- 172.21.15.69:0/268709590 >> 172.21.15.69:0/268709590 conn(0x7fb30c0fc4d0 msgr2=0x7fb30c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 -- 172.21.15.69:0/268709590 shutdown_connections 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.433+0000 7fb310cef640 1 -- 172.21.15.69:0/268709590 wait complete. 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb310cef640 1 Processor -- start 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb310cef640 1 -- start start 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb310cef640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c1007b0 0x7fb30c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb310cef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 0x7fb30c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb310cef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb30c1067b0 0x7fb30c19ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb310cef640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c110bb0 con 0x7fb30c101150 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb310cef640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c110a30 con 0x7fb30c1067b0 2024-08-02T17:56:54.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb310cef640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c110d30 con 0x7fb30c1007b0 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb309d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 0x7fb30c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb309d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 0x7fb30c198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51846/0 (socket says 172.21.15.69:51846) 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb309d9b640 1 -- 172.21.15.69:0/3855191310 learned_addr learned my addr 172.21.15.69:0/3855191310 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.434+0000 7fb30a59c640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c1007b0 0x7fb30c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb30c1067b0 0x7fb30c19ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb309d9b640 1 -- 172.21.15.69:0/3855191310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c1007b0 msgr2=0x7fb30c198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb309d9b640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c1007b0 0x7fb30c198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb309d9b640 1 -- 172.21.15.69:0/3855191310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb30c1067b0 msgr2=0x7fb30c19ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb309d9b640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb30c1067b0 0x7fb30c19ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb309d9b640 1 -- 172.21.15.69:0/3855191310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c19d100 con 0x7fb30c101150 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb30c1067b0 0x7fb30c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb309d9b640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 0x7fb30c198690 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c890 tx=0x7fb30000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb2fb7fe640 1 -- 172.21.15.69:0/3855191310 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300004220 con 0x7fb30c101150 2024-08-02T17:56:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb310cef640 1 -- 172.21.15.69:0/3855191310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c19d3f0 con 0x7fb30c101150 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb2fb7fe640 1 -- 172.21.15.69:0/3855191310 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3000043c0 con 0x7fb30c101150 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb2fb7fe640 1 -- 172.21.15.69:0/3855191310 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300004a00 con 0x7fb30c101150 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.435+0000 7fb310cef640 1 -- 172.21.15.69:0/3855191310 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c1a4cd0 con 0x7fb30c101150 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.437+0000 7fb2fb7fe640 1 -- 172.21.15.69:0/3855191310 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb3000079b0 con 0x7fb30c101150 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb3130ed740 1 Processor -- start 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb3130ed740 1 -- start start 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a737ff0 0x56343a7383d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a742c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a7455c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb3130ed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56343a69da40 con 0x56343a7431d0 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb3130ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56343a6359d0 con 0x56343a738910 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb3130ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56343a699a00 con 0x56343a737ff0 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb309d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a7455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb309d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a7455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51860/0 (socket says 172.21.15.69:51860) 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb309d9b640 1 -- 172.21.15.69:0/1947721872 learned_addr learned my addr 172.21.15.69:0/1947721872 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb2fb7fe640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2f40a2cf0 0x7fb2f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a737ff0 0x56343a7383d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb30a59c640 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a742c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb309d9b640 1 -- 172.21.15.69:0/1947721872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a737ff0 msgr2=0x56343a7383d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb309d9b640 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a737ff0 0x56343a7383d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb309d9b640 1 -- 172.21.15.69:0/1947721872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 msgr2=0x56343a742c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.438+0000 7fb309d9b640 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a742c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb2fb7fe640 1 -- 172.21.15.69:0/3855191310 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7fb3000cf5a0 con 0x7fb30c101150 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb309d9b640 1 -- 172.21.15.69:0/1947721872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343a63c650 con 0x56343a7431d0 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a737ff0 0x56343a7383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb30a59c640 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a742c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:54.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2f40a2cf0 0x7fb2f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb309d9b640 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a7455c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb3000a3db0 tx=0x7fb300005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115aff7c950ae2f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb2f8ff9640 1 -- 172.21.15.69:0/1947721872 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30001d070 con 0x56343a7431d0 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb2f8ff9640 1 -- 172.21.15.69:0/1947721872 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30009bd70 con 0x56343a7431d0 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.439+0000 7fb2f8ff9640 1 -- 172.21.15.69:0/1947721872 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300099970 con 0x56343a7431d0 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 -- 172.21.15.69:0/1947721872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 msgr2=0x56343a7455c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2f40a2cf0 0x7fb2f40a51b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb304009430 tx=0x7fb30403e040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a7455c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb3000a3db0 tx=0x7fb300005300 comp rx=0 tx=0).stop 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 -- 172.21.15.69:0/1947721872 shutdown_connections 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a7455c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a742c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 --2- 172.21.15.69:0/1947721872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a737ff0 0x56343a7383d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 -- 172.21.15.69:0/1947721872 >> 172.21.15.69:0/1947721872 conn(0x56343a734580 msgr2=0x56343a734960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 -- 172.21.15.69:0/1947721872 shutdown_connections 2024-08-02T17:56:54.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 -- 172.21.15.69:0/1947721872 wait complete. 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 -- start start 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.440+0000 7fb3130ed740 1 Processor -- start 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a737ff0 0x56343a740080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 0x56343a7405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a740b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb3130ed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56343a745d40 con 0x56343a7431d0 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb3130ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56343a658b20 con 0x56343a737ff0 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb3130ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56343a745ec0 con 0x56343a738910 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb3130ed740 10 client.0 _ll_register_callbacks cb 0x56343a656bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a740b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a740b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51862/0 (socket says 172.21.15.69:51862) 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 -- 172.21.15.69:0/177854701 learned_addr learned my addr 172.21.15.69:0/177854701 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:54.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30a59c640 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a737ff0 0x56343a740080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb309d9b640 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 0x56343a7405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 -- 172.21.15.69:0/177854701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 msgr2=0x56343a7405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 0x56343a7405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 -- 172.21.15.69:0/177854701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a737ff0 msgr2=0x56343a740080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a737ff0 0x56343a740080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 -- 172.21.15.69:0/177854701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343a40f3c0 con 0x56343a7431d0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a740b00 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb304003bf0 tx=0x7fb304009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c73465736528139 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb2f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/177854701 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304004450 con 0x56343a7431d0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/177854701 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7fb3040045f0 con 0x56343a7431d0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.441+0000 7fb3130ed740 1 -- 172.21.15.69:0/177854701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb30009bd70 con 0x56343a7431d0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/177854701 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb3040047f0 con 0x56343a7431d0 2024-08-02T17:56:54.729 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb3130ed740 10 client.17925 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb3130ed740 1 -- 172.21.15.69:0/177854701 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56343a751910 con 0x56343a7431d0 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/177854701 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb30009bd70 con 0x56343a7431d0 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/177854701 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb304030070 con 0x56343a7431d0 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb3130ed740 10 client.17925 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb3130ed740 10 client.17925 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb3130ed740 20 client.17925 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb3130ed740 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56343a752dd0 0x56343a7551c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb3130ed740 4 client.17925 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb3130ed740 1 -- 172.21.15.69:0/177854701 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56343a6aafb0 con 0x56343a752dd0 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.442+0000 7fb30a59c640 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56343a752dd0 0x56343a7551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.443+0000 7fb2f1ffb640 10 client.17925 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.443+0000 7fb30a59c640 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56343a752dd0 0x56343a7551c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=75eaa3162749278e server_cookie=dc7b4513cffea66e in_seq=0 out_seq=0 2024-08-02T17:56:54.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.443+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/177854701 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56343a6aafb0 con 0x56343a752dd0 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.443+0000 7fb2f1ffb640 10 client.17925 handle_command_reply: tid=0 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.443+0000 7fb3130ed740 1 client.17925 shutdown 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.443+0000 7fb3130ed740 1 -- 172.21.15.69:0/177854701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 msgr2=0x56343a740b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.443+0000 7fb3130ed740 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a740b00 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb304003bf0 tx=0x7fb304009360 comp rx=0 tx=0).stop 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 1 -- 172.21.15.69:0/177854701 shutdown_connections 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56343a752dd0 0x56343a7551c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a7431d0 0x56343a740b00 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 0x56343a7405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb2f1ffb640 0 client.17925 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:56:54.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 1 --2- 172.21.15.69:0/177854701 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a737ff0 0x56343a740080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 1 -- 172.21.15.69:0/177854701 >> 172.21.15.69:0/177854701 conn(0x56343a734580 msgr2=0x56343a73a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 1 -- 172.21.15.69:0/177854701 shutdown_connections 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 1 -- 172.21.15.69:0/177854701 wait complete. 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.444+0000 7fb3130ed740 20 client.17925 trim_cache size 0 max 16384 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb3130ed740 1 Processor -- start 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb3130ed740 1 -- start start 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a73f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 0x56343a75aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a7431d0 0x56343a75afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb3130ed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1b1880 con 0x56343a737ff0 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb3130ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1af150 con 0x56343a7431d0 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb3130ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1acfa0 con 0x56343a738910 2024-08-02T17:56:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb30a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a73f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.733 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:56:54.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb30a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a73f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51872/0 (socket says 172.21.15.69:51872) 2024-08-02T17:56:54.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb30a59c640 1 -- 172.21.15.69:0/826491300 learned_addr learned my addr 172.21.15.69:0/826491300 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:54.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb309d9b640 1 --2- 172.21.15.69:0/826491300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 0x56343a75aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb30a59c640 1 -- 172.21.15.69:0/826491300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 msgr2=0x56343a75aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb30a59c640 1 --2- 172.21.15.69:0/826491300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 0x56343a75aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb30a59c640 1 -- 172.21.15.69:0/826491300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a7431d0 msgr2=0x56343a75afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb30a59c640 1 --2- 172.21.15.69:0/826491300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a7431d0 0x56343a75afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.457+0000 7fb30a59c640 1 -- 172.21.15.69:0/826491300 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343a755530 con 0x56343a737ff0 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb30a59c640 1 --2- 172.21.15.69:0/826491300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a73f170 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00aa70 tx=0x7fb2fc0138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4896ec5ab693f2b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb2f3fff640 1 -- 172.21.15.69:0/826491300 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc01c460 con 0x56343a737ff0 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb2f3fff640 1 -- 172.21.15.69:0/826491300 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2fc01cae0 con 0x56343a737ff0 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb2f3fff640 1 -- 172.21.15.69:0/826491300 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc011ae0 con 0x56343a737ff0 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb3130ed740 1 -- 172.21.15.69:0/826491300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 msgr2=0x56343a73f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb3130ed740 1 --2- 172.21.15.69:0/826491300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a73f170 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00aa70 tx=0x7fb2fc0138f0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb3130ed740 1 -- 172.21.15.69:0/826491300 shutdown_connections 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb3130ed740 1 --2- 172.21.15.69:0/826491300 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a7431d0 0x56343a75afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb3130ed740 1 --2- 172.21.15.69:0/826491300 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a738910 0x56343a75aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb3130ed740 1 --2- 172.21.15.69:0/826491300 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a73f170 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.458+0000 7fb3130ed740 1 -- 172.21.15.69:0/826491300 >> 172.21.15.69:0/826491300 conn(0x56343a734580 msgr2=0x56343a72f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 -- 172.21.15.69:0/826491300 shutdown_connections 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 -- 172.21.15.69:0/826491300 wait complete. 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 -- start start 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 Processor -- start 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a74b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a74b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a7431d0 0x56343a73c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1ad610 con 0x56343a737ff0 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1aea80 con 0x56343a738910 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c1adce0 con 0x56343a7431d0 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb3130ed740 10 client.0 _ll_register_callbacks cb 0x7fb304009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a74b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a74b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51878/0 (socket says 172.21.15.69:51878) 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30a59c640 1 -- 172.21.15.69:0/965116426 learned_addr learned my addr 172.21.15.69:0/965116426 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb309d9b640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a74b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a7431d0 0x56343a73c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30a59c640 1 -- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a7431d0 msgr2=0x56343a73c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30a59c640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a7431d0 0x56343a73c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30a59c640 1 -- 172.21.15.69:0/965116426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 msgr2=0x56343a74b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30a59c640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a74b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30a59c640 1 -- 172.21.15.69:0/965116426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343a73d0c0 con 0x56343a737ff0 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb30ad9d640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a7431d0 0x56343a73c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.459+0000 7fb309d9b640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a74b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb30a59c640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a74b2f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0117c0 tx=0x7fb2fc013190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7138ec28db514a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:54.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb2f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/965116426 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc033060 con 0x56343a737ff0 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/965116426 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7fb2fc01c660 con 0x56343a737ff0 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 1 -- 172.21.15.69:0/965116426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2fc01cae0 con 0x56343a737ff0 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/965116426 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb2fc03a050 con 0x56343a737ff0 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 10 client.17931 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 1 -- 172.21.15.69:0/965116426 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56343a7bd500 con 0x56343a737ff0 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/965116426 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2fc01cae0 con 0x56343a737ff0 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/965116426 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb2fc011b60 con 0x56343a737ff0 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 10 client.17931 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 10 client.17931 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 20 client.17931 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56343a7bdbc0 0x56343a7bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 4 client.17931 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:56:54.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.460+0000 7fb3130ed740 1 -- 172.21.15.69:0/965116426 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56343a6a54c0 con 0x56343a7bdbc0 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb309d9b640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56343a7bdbc0 0x56343a7bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb309d9b640 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56343a7bdbc0 0x56343a7bffe0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3962e177aff652da server_cookie=e26233f630fda175 in_seq=0 out_seq=0 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb2f1ffb640 10 client.17931 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb2f1ffb640 1 -- 172.21.15.69:0/965116426 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56343a6a54c0 con 0x56343a7bdbc0 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb2f1ffb640 10 client.17931 handle_command_reply: tid=0 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 client.17931 shutdown 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 -- 172.21.15.69:0/965116426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 msgr2=0x56343a74b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a74b2f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc0117c0 tx=0x7fb2fc013190 comp rx=0 tx=0).stop 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 -- 172.21.15.69:0/965116426 shutdown_connections 2024-08-02T17:56:54.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56343a7bdbc0 0x56343a7bffe0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56343a7431d0 0x56343a73c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56343a738910 0x56343a74b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb2f1ffb640 0 client.17931 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 --2- 172.21.15.69:0/965116426 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56343a737ff0 0x56343a74b2f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 -- 172.21.15.69:0/965116426 >> 172.21.15.69:0/965116426 conn(0x56343a734580 msgr2=0x56343a73f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 -- 172.21.15.69:0/965116426 shutdown_connections 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 1 -- 172.21.15.69:0/965116426 wait complete. 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.461+0000 7fb3130ed740 20 client.17931 trim_cache size 0 max 16384 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 -- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2f40a2cf0 msgr2=0x7fb2f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2f40a2cf0 0x7fb2f40a51b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb304009430 tx=0x7fb30403e040 comp rx=0 tx=0).stop 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 -- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 msgr2=0x7fb30c198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 0x7fb30c198690 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c890 tx=0x7fb30000cd50 comp rx=0 tx=0).stop 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 -- 172.21.15.69:0/3855191310 shutdown_connections 2024-08-02T17:56:54.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb30c101150 0x7fb30c198690 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb2f40a2cf0 0x7fb2f40a51b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb30c1067b0 0x7fb30c19ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 --2- 172.21.15.69:0/3855191310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb30c1007b0 0x7fb30c198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:54.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.462+0000 7fb310cef640 1 -- 172.21.15.69:0/3855191310 >> 172.21.15.69:0/3855191310 conn(0x7fb30c0fc4d0 msgr2=0x7fb30c06a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:54.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.463+0000 7fb310cef640 1 -- 172.21.15.69:0/3855191310 shutdown_connections 2024-08-02T17:56:54.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:54.463+0000 7fb310cef640 1 -- 172.21.15.69:0/3855191310 wait complete. 2024-08-02T17:56:54.922 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:56:54.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 585.0 s, 43.6 tps, lat 210.608 ms stddev 262.018 2024-08-02T17:56:55.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:54 smithi069 ceph-mon[32746]: pgmap v325: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.8 MiB/s wr, 790 op/s 2024-08-02T17:56:55.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:54 smithi069 ceph-mon[32746]: Adjusting osd_memory_target on smithi069 to 175.9M 2024-08-02T17:56:55.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:54 smithi069 ceph-mon[32746]: Unable to set osd_memory_target on smithi069 to 184535859: error parsing value: Value '184535859' is below minimum 939524096 2024-08-02T17:56:55.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:54 smithi083 ceph-mon[37031]: pgmap v325: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.8 MiB/s wr, 790 op/s 2024-08-02T17:56:55.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:54 smithi083 ceph-mon[37031]: Adjusting osd_memory_target on smithi069 to 175.9M 2024-08-02T17:56:55.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:54 smithi083 ceph-mon[37031]: Unable to set osd_memory_target on smithi069 to 184535859: error parsing value: Value '184535859' is below minimum 939524096 2024-08-02T17:56:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:54 smithi077 ceph-mon[37108]: pgmap v325: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.8 MiB/s wr, 790 op/s 2024-08-02T17:56:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:54 smithi077 ceph-mon[37108]: Adjusting osd_memory_target on smithi069 to 175.9M 2024-08-02T17:56:55.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:54 smithi077 ceph-mon[37108]: Unable to set osd_memory_target on smithi069 to 184535859: error parsing value: Value '184535859' is below minimum 939524096 2024-08-02T17:56:55.518 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:56:55.614 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 -- 172.21.15.69:0/1765072439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1067b0 msgr2=0x7f03ac106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1765072439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1067b0 0x7f03ac106b90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009930 tx=0x7f03a002a420 comp rx=0 tx=0).stop 2024-08-02T17:56:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 -- 172.21.15.69:0/1765072439 shutdown_connections 2024-08-02T17:56:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1765072439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ac101150 0x7f03ac10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1765072439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ac1007b0 0x7f03ac100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1765072439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1067b0 0x7f03ac106b90 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 -- 172.21.15.69:0/1765072439 >> 172.21.15.69:0/1765072439 conn(0x7f03ac0fc4d0 msgr2=0x7f03ac0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 -- 172.21.15.69:0/1765072439 shutdown_connections 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.096+0000 7f03b37d3640 1 -- 172.21.15.69:0/1765072439 wait complete. 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b37d3640 1 Processor -- start 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b37d3640 1 -- start start 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b37d3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1007b0 0x7f03ac198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b37d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ac101150 0x7f03ac198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b37d3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ac1067b0 0x7f03ac19ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b37d3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac110bb0 con 0x7f03ac1007b0 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b37d3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac110a30 con 0x7f03ac1067b0 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b37d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac110d30 con 0x7f03ac101150 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b156f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1007b0 0x7f03ac198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b156f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1007b0 0x7f03ac198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42804/0 (socket says 172.21.15.69:42804) 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b156f640 1 -- 172.21.15.69:0/1457992961 learned_addr learned my addr 172.21.15.69:0/1457992961 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.097+0000 7f03b0d6e640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ac101150 0x7f03ac198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b1d70640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ac1067b0 0x7f03ac19ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b156f640 1 -- 172.21.15.69:0/1457992961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ac101150 msgr2=0x7f03ac198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b156f640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ac101150 0x7f03ac198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b156f640 1 -- 172.21.15.69:0/1457992961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ac1067b0 msgr2=0x7f03ac19ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b156f640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ac1067b0 0x7f03ac19ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b156f640 1 -- 172.21.15.69:0/1457992961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac19d0e0 con 0x7f03ac1007b0 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b1d70640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ac1067b0 0x7f03ac19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b156f640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1007b0 0x7f03ac198130 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009230 tx=0x7f03a0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f039e7fc640 1 -- 172.21.15.69:0/1457992961 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a003f030 con 0x7f03ac1007b0 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f039e7fc640 1 -- 172.21.15.69:0/1457992961 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03a0004440 con 0x7f03ac1007b0 2024-08-02T17:56:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f039e7fc640 1 -- 172.21.15.69:0/1457992961 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0035070 con 0x7f03ac1007b0 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac19d370 con 0x7f03ac1007b0 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.098+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1a4cb0 con 0x7f03ac1007b0 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.099+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ac101ef0 con 0x7f03ac1007b0 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.100+0000 7f039e7fc640 1 -- 172.21.15.69:0/1457992961 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f03a0042030 con 0x7f03ac1007b0 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.101+0000 7f039e7fc640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03980a2cf0 0x7f03980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.101+0000 7f03b0d6e640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03980a2cf0 0x7f03980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.101+0000 7f039e7fc640 1 -- 172.21.15.69:0/1457992961 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f03a00f0020 con 0x7f03ac1007b0 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.101+0000 7f03b0d6e640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03980a2cf0 0x7f03980a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f03ac104760 tx=0x7f03a8005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.104+0000 7f039e7fc640 1 -- 172.21.15.69:0/1457992961 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f03a00bc830 con 0x7f03ac1007b0 2024-08-02T17:56:56.157 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:56.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.323+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03ac100c10 con 0x7f03ac1007b0 2024-08-02T17:56:56.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.324+0000 7f039e7fc640 1 -- 172.21.15.69:0/1457992961 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f03ac100c10 con 0x7f03ac1007b0 2024-08-02T17:56:56.326 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:56:56.326 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:56:56.326 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:56:56.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03980a2cf0 msgr2=0x7f03980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:56.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03980a2cf0 0x7f03980a51b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f03ac104760 tx=0x7f03a8005f90 comp rx=0 tx=0).stop 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1007b0 msgr2=0x7f03ac198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1007b0 0x7f03ac198130 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f03a0009230 tx=0x7f03a0004040 comp rx=0 tx=0).stop 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 shutdown_connections 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03980a2cf0 0x7f03980a51b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f03ac1067b0 0x7f03ac19ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03ac101150 0x7f03ac198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 --2- 172.21.15.69:0/1457992961 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f03ac1007b0 0x7f03ac198130 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 >> 172.21.15.69:0/1457992961 conn(0x7f03ac0fc4d0 msgr2=0x7f03ac06a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 shutdown_connections 2024-08-02T17:56:56.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:56.326+0000 7f03b37d3640 1 -- 172.21.15.69:0/1457992961 wait complete. 2024-08-02T17:56:57.084 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 71426704 --release --await' 2024-08-02T17:56:57.084 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 71426704 --release --await 2024-08-02T17:56:57.219 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:56 smithi069 ceph-mon[32746]: pgmap v326: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.6 MiB/s wr, 427 op/s 2024-08-02T17:56:57.220 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:56 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1457992961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:56 smithi083 ceph-mon[37031]: pgmap v326: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.6 MiB/s wr, 427 op/s 2024-08-02T17:56:57.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:56 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1457992961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:57.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 -- 172.21.15.69:0/3803433115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1007b0 msgr2=0x7f210c100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 --2- 172.21.15.69:0/3803433115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1007b0 0x7f210c100c10 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2100009930 tx=0x7f210002a420 comp rx=0 tx=0).stop 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 -- 172.21.15.69:0/3803433115 shutdown_connections 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 --2- 172.21.15.69:0/3803433115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1007b0 0x7f210c100c10 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 --2- 172.21.15.69:0/3803433115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f210c101150 0x7f210c10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 --2- 172.21.15.69:0/3803433115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f210c1067b0 0x7f210c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 -- 172.21.15.69:0/3803433115 >> 172.21.15.69:0/3803433115 conn(0x7f210c0fc4d0 msgr2=0x7f210c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 -- 172.21.15.69:0/3803433115 shutdown_connections 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.308+0000 7f2110bad640 1 -- 172.21.15.69:0/3803433115 wait complete. 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2110bad640 1 Processor -- start 2024-08-02T17:56:57.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2110bad640 1 -- start start 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2110bad640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f210c1007b0 0x7f210c196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2110bad640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f210c101150 0x7f210c197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2110bad640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1067b0 0x7f210c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2110bad640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f210c110bf0 con 0x7f210c1067b0 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2110bad640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f210c110a70 con 0x7f210c1007b0 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2110bad640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f210c110d70 con 0x7f210c101150 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2109d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f210c101150 0x7f210c197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2109d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f210c101150 0x7f210c197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53326/0 (socket says 172.21.15.69:53326) 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f2109d9b640 1 -- 172.21.15.69:0/37788440 learned_addr learned my addr 172.21.15.69:0/37788440 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f210ad9d640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1067b0 0x7f210c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f210a59c640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f210c1007b0 0x7f210c196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f210ad9d640 1 -- 172.21.15.69:0/37788440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f210c101150 msgr2=0x7f210c197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f210ad9d640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f210c101150 0x7f210c197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f210ad9d640 1 -- 172.21.15.69:0/37788440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f210c1007b0 msgr2=0x7f210c196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.309+0000 7f210ad9d640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f210c1007b0 0x7f210c196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.310+0000 7f210ad9d640 1 -- 172.21.15.69:0/37788440 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c1a66a0 con 0x7f210c1067b0 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.310+0000 7f2109d9b640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f210c101150 0x7f210c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.310+0000 7f210ad9d640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1067b0 0x7f210c1978b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f210400e940 tx=0x7f210400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.310+0000 7f20fb7fe640 1 -- 172.21.15.69:0/37788440 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104004d20 con 0x7f210c1067b0 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.310+0000 7f20fb7fe640 1 -- 172.21.15.69:0/37788440 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2104005020 con 0x7f210c1067b0 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.310+0000 7f2110bad640 1 -- 172.21.15.69:0/37788440 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210c1a6990 con 0x7f210c1067b0 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.310+0000 7f20fb7fe640 1 -- 172.21.15.69:0/37788440 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21040116b0 con 0x7f210c1067b0 2024-08-02T17:56:57.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.310+0000 7f2110bad640 1 -- 172.21.15.69:0/37788440 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210c1a6ed0 con 0x7f210c1067b0 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.312+0000 7f20fb7fe640 1 -- 172.21.15.69:0/37788440 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f21040054d0 con 0x7f210c1067b0 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2112fab740 1 Processor -- start 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2112fab740 1 -- start start 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2112fab740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2112fab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2112fab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 0x5577b93005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2112fab740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577b9258a40 con 0x5577b92f2ff0 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2112fab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5577b91f09d0 con 0x5577b92f3910 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2112fab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5577b9254a00 con 0x5577b92fe1d0 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f20fb7fe640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f40a2d70 0x7f20f40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42850/0 (socket says 172.21.15.69:42850) 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 -- 172.21.15.69:0/3159694810 learned_addr learned my addr 172.21.15.69:0/3159694810 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f210ad9d640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f40a2d70 0x7f20f40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 0x5577b93005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f210a59c640 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 -- 172.21.15.69:0/3159694810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 msgr2=0x5577b93005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 0x5577b93005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 -- 172.21.15.69:0/3159694810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 msgr2=0x5577b92fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f20fb7fe640 1 -- 172.21.15.69:0/37788440 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f21040cfc50 con 0x7f210c1067b0 2024-08-02T17:56:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f2109d9b640 1 -- 172.21.15.69:0/3159694810 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b91f7650 con 0x5577b92f2ff0 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.313+0000 7f210a59c640 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.314+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92f33d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x5577b92f0900 tx=0x7f2100009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b8e8829a70b674 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.314+0000 7f210ad9d640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f40a2d70 0x7f20f40a5230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2104098180 tx=0x7f21040c68b0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.314+0000 7f20f8ff9640 1 -- 172.21.15.69:0/3159694810 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100044070 con 0x5577b92f2ff0 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.314+0000 7f20f8ff9640 1 -- 172.21.15.69:0/3159694810 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100005500 con 0x5577b92f2ff0 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.314+0000 7f20f8ff9640 1 -- 172.21.15.69:0/3159694810 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100035040 con 0x5577b92f2ff0 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.314+0000 7f2112fab740 1 -- 172.21.15.69:0/3159694810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 msgr2=0x5577b92f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.314+0000 7f2112fab740 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92f33d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x5577b92f0900 tx=0x7f2100009560 comp rx=0 tx=0).stop 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.314+0000 7f2112fab740 1 -- 172.21.15.69:0/3159694810 shutdown_connections 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 0x5577b93005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 --2- 172.21.15.69:0/3159694810 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92f33d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 -- 172.21.15.69:0/3159694810 >> 172.21.15.69:0/3159694810 conn(0x5577b92ef580 msgr2=0x5577b92ef960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 -- 172.21.15.69:0/3159694810 shutdown_connections 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 -- 172.21.15.69:0/3159694810 wait complete. 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 -- start start 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 Processor -- start 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92fb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 0x5577b92fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5577b9300da0 con 0x5577b92f2ff0 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5577b9213b20 con 0x5577b92f3910 2024-08-02T17:56:57.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5577b9300f20 con 0x5577b92fe1d0 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2112fab740 10 client.0 _ll_register_callbacks cb 0x5577b9211bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2109d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92fb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2109d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92fb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42852/0 (socket says 172.21.15.69:42852) 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2109d9b640 1 -- 172.21.15.69:0/1891418630 learned_addr learned my addr 172.21.15.69:0/1891418630 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f210a59c640 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.315+0000 7f2109d9b640 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 0x5577b92fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2109d9b640 1 -- 172.21.15.69:0/1891418630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 msgr2=0x5577b92fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2109d9b640 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 0x5577b92fbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2109d9b640 1 -- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 msgr2=0x5577b92fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:57.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2109d9b640 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2109d9b640 1 -- 172.21.15.69:0/1891418630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b8fca3c0 con 0x5577b92f2ff0 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f210a59c640 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2109d9b640 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92fb0a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f210002a460 tx=0x7f2100005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc58a042eba3fe77 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f20f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f20f1ffb640 1 -- 172.21.15.69:0/1891418630 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100044050 con 0x5577b92f2ff0 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2112fab740 1 -- 172.21.15.69:0/1891418630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2100005500 con 0x5577b92f2ff0 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f20f1ffb640 1 -- 172.21.15.69:0/1891418630 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f2100004bc0 con 0x5577b92f2ff0 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f20f1ffb640 1 -- 172.21.15.69:0/1891418630 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f210005b050 con 0x5577b92f2ff0 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2112fab740 10 client.17949 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2112fab740 1 -- 172.21.15.69:0/1891418630 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5577b930c940 con 0x5577b92f2ff0 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f20f1ffb640 1 -- 172.21.15.69:0/1891418630 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2100005500 con 0x5577b92f2ff0 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f20f1ffb640 1 -- 172.21.15.69:0/1891418630 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f210003a3f0 con 0x5577b92f2ff0 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2112fab740 10 client.17949 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2112fab740 10 client.17949 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.316+0000 7f2112fab740 20 client.17949 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.317+0000 7f2112fab740 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5577b930de40 0x5577b9310230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.317+0000 7f2112fab740 4 client.17949 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.317+0000 7f2112fab740 1 -- 172.21.15.69:0/1891418630 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577b9265fb0 con 0x5577b930de40 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.317+0000 7f210a59c640 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5577b930de40 0x5577b9310230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.317+0000 7f20f1ffb640 10 client.17949 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.317+0000 7f210a59c640 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5577b930de40 0x5577b9310230 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2ff36b706c585481 server_cookie=9279d8c73128e050 in_seq=0 out_seq=0 2024-08-02T17:56:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f20f1ffb640 1 -- 172.21.15.69:0/1891418630 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5577b9265fb0 con 0x5577b930de40 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f20f1ffb640 10 client.17949 handle_command_reply: tid=0 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 client.17949 shutdown 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 -- 172.21.15.69:0/1891418630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 msgr2=0x5577b92fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92fb0a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f210002a460 tx=0x7f2100005e20 comp rx=0 tx=0).stop 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 -- 172.21.15.69:0/1891418630 shutdown_connections 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5577b930de40 0x5577b9310230 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92fe1d0 0x5577b92fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f20f1ffb640 0 client.17949 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:56:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b92fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 --2- 172.21.15.69:0/1891418630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f2ff0 0x5577b92fb0a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 -- 172.21.15.69:0/1891418630 >> 172.21.15.69:0/1891418630 conn(0x5577b92ef580 msgr2=0x5577b92f52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 -- 172.21.15.69:0/1891418630 shutdown_connections 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 1 -- 172.21.15.69:0/1891418630 wait complete. 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.318+0000 7f2112fab740 20 client.17949 trim_cache size 0 max 16384 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2112fab740 1 Processor -- start 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2112fab740 1 -- start start 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2112fab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 0x5577b9315b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2112fab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b9316080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2112fab740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92fe1d0 0x5577b92fb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2112fab740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1b3930 con 0x5577b92fe1d0 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2112fab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1b1200 con 0x5577b92f3910 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2112fab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1af050 con 0x5577b92f2ff0 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92fe1d0 0x5577b92fb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92fe1d0 0x5577b92fb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42860/0 (socket says 172.21.15.69:42860) 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 -- 172.21.15.69:0/2180082020 learned_addr learned my addr 172.21.15.69:0/2180082020 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 -- 172.21.15.69:0/2180082020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 msgr2=0x5577b9315b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 --2- 172.21.15.69:0/2180082020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 0x5577b9315b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 -- 172.21.15.69:0/2180082020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 msgr2=0x5577b9316080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 --2- 172.21.15.69:0/2180082020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b9316080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 -- 172.21.15.69:0/2180082020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577b92f3740 con 0x5577b92fe1d0 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.331+0000 7f2109d9b640 1 --2- 172.21.15.69:0/2180082020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92fe1d0 0x5577b92fb060 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2100032c00 tx=0x7f210003aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac28dcb8acc4f49d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f20f3fff640 1 -- 172.21.15.69:0/2180082020 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100044040 con 0x5577b92fe1d0 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f20f3fff640 1 -- 172.21.15.69:0/2180082020 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21000046e0 con 0x5577b92fe1d0 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f20f3fff640 1 -- 172.21.15.69:0/2180082020 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100035040 con 0x5577b92fe1d0 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 -- 172.21.15.69:0/2180082020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92fe1d0 msgr2=0x5577b92fb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 --2- 172.21.15.69:0/2180082020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92fe1d0 0x5577b92fb060 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2100032c00 tx=0x7f210003aca0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 -- 172.21.15.69:0/2180082020 shutdown_connections 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 --2- 172.21.15.69:0/2180082020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92fe1d0 0x5577b92fb060 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 --2- 172.21.15.69:0/2180082020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92f3910 0x5577b9316080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 --2- 172.21.15.69:0/2180082020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 0x5577b9315b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 -- 172.21.15.69:0/2180082020 >> 172.21.15.69:0/2180082020 conn(0x5577b92ef580 msgr2=0x5577b92fa1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 -- 172.21.15.69:0/2180082020 shutdown_connections 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.332+0000 7f2112fab740 1 -- 172.21.15.69:0/2180082020 wait complete. 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 1 -- start start 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 1 Processor -- start 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 0x5577b93061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f3910 0x5577b93066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92fe1d0 0x5577b92f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1af6c0 con 0x5577b92f3910 2024-08-02T17:56:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1b0b30 con 0x5577b92fe1d0 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f210c1afd90 con 0x5577b92f2ff0 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2112fab740 10 client.0 _ll_register_callbacks cb 0x7f2100031930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f3910 0x5577b93066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f3910 0x5577b93066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42868/0 (socket says 172.21.15.69:42868) 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 -- 172.21.15.69:0/3912801812 learned_addr learned my addr 172.21.15.69:0/3912801812 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 0x5577b93061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92fe1d0 0x5577b92f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 -- 172.21.15.69:0/3912801812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 msgr2=0x5577b93061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:57.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 0x5577b93061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 -- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92fe1d0 msgr2=0x5577b92f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92fe1d0 0x5577b92f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 -- 172.21.15.69:0/3912801812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577b92f7cc0 con 0x5577b92f3910 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92fe1d0 0x5577b92f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f210a59c640 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f3910 0x5577b93066e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009330 tx=0x7f20fc017810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe3547f0bf40db8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.333+0000 7f20f1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f20f1ffb640 1 -- 172.21.15.69:0/3912801812 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00f040 con 0x5577b92f3910 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f20f1ffb640 1 -- 172.21.15.69:0/3912801812 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f20fc025630 con 0x5577b92f3910 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 1 -- 172.21.15.69:0/3912801812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21000046e0 con 0x5577b92f3910 2024-08-02T17:56:57.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f20f1ffb640 1 -- 172.21.15.69:0/3912801812 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20fc02e050 con 0x5577b92f3910 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 10 client.17955 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 1 -- 172.21.15.69:0/3912801812 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5577b9378350 con 0x5577b92f3910 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f20f1ffb640 1 -- 172.21.15.69:0/3912801812 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21000046e0 con 0x5577b92f3910 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f20f1ffb640 1 -- 172.21.15.69:0/3912801812 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f20fc015960 con 0x5577b92f3910 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 10 client.17955 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 10 client.17955 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 20 client.17955 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5577b9378b00 0x5577b937af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 4 client.17955 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.334+0000 7f2112fab740 1 -- 172.21.15.69:0/3912801812 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577b92604c0 con 0x5577b9378b00 2024-08-02T17:56:57.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.335+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5577b9378b00 0x5577b937af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.335+0000 7f20f1ffb640 10 client.17955 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.335+0000 7f2109d9b640 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5577b9378b00 0x5577b937af20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4861d1047a3d09a5 server_cookie=d2a4f81e4eaabce0 in_seq=0 out_seq=0 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.335+0000 7f20f1ffb640 1 -- 172.21.15.69:0/3912801812 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577b92604c0 con 0x5577b9378b00 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.335+0000 7f20f1ffb640 10 client.17955 handle_command_reply: tid=0 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.335+0000 7f2112fab740 1 client.17955 shutdown 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 -- 172.21.15.69:0/3912801812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f3910 msgr2=0x5577b93066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f3910 0x5577b93066e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f20fc009330 tx=0x7f20fc017810 comp rx=0 tx=0).stop 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 -- 172.21.15.69:0/3912801812 shutdown_connections 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5577b9378b00 0x5577b937af20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5577b92f3910 0x5577b93066e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5577b92fe1d0 0x5577b92f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 --2- 172.21.15.69:0/3912801812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5577b92f2ff0 0x5577b93061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 -- 172.21.15.69:0/3912801812 >> 172.21.15.69:0/3912801812 conn(0x5577b92ef580 msgr2=0x5577b92fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f20f1ffb640 0 client.17955 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:56:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 -- 172.21.15.69:0/3912801812 shutdown_connections 2024-08-02T17:56:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 1 -- 172.21.15.69:0/3912801812 wait complete. 2024-08-02T17:56:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2112fab740 20 client.17955 trim_cache size 0 max 16384 2024-08-02T17:56:57.357 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:56:57.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 -- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f40a2d70 msgr2=0x7f20f40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f40a2d70 0x7f20f40a5230 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2104098180 tx=0x7f21040c68b0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 -- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1067b0 msgr2=0x7f210c1978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:57.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1067b0 0x7f210c1978b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f210400e940 tx=0x7f210400ed40 comp rx=0 tx=0).stop 2024-08-02T17:56:57.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 -- 172.21.15.69:0/37788440 shutdown_connections 2024-08-02T17:56:57.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f210c1067b0 0x7f210c1978b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f20f40a2d70 0x7f20f40a5230 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f210c101150 0x7f210c197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 --2- 172.21.15.69:0/37788440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f210c1007b0 0x7f210c196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:57.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.336+0000 7f2110bad640 1 -- 172.21.15.69:0/37788440 >> 172.21.15.69:0/37788440 conn(0x7f210c0fc4d0 msgr2=0x7f210c0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:57.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.337+0000 7f2110bad640 1 -- 172.21.15.69:0/37788440 shutdown_connections 2024-08-02T17:56:57.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:57.337+0000 7f2110bad640 1 -- 172.21.15.69:0/37788440 wait complete. 2024-08-02T17:56:57.359 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:56 smithi077 ceph-mon[37108]: pgmap v326: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 3.6 MiB/s wr, 427 op/s 2024-08-02T17:56:57.359 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:56 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1457992961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:56:58.231 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:56:58.232 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:58.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 -- 172.21.15.69:0/3558996199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5210100b50 msgr2=0x7f5210100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 --2- 172.21.15.69:0/3558996199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5210100b50 0x7f5210100f30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009900 tx=0x7f51f802a480 comp rx=0 tx=0).stop 2024-08-02T17:56:58.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 -- 172.21.15.69:0/3558996199 shutdown_connections 2024-08-02T17:56:58.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 --2- 172.21.15.69:0/3558996199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5210102c20 0x7f521010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 --2- 172.21.15.69:0/3558996199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52101022a0 0x7f52101026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 --2- 172.21.15.69:0/3558996199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5210100b50 0x7f5210100f30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 -- 172.21.15.69:0/3558996199 >> 172.21.15.69:0/3558996199 conn(0x7f52100fc500 msgr2=0x7f52100fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:58.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 -- 172.21.15.69:0/3558996199 shutdown_connections 2024-08-02T17:56:58.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.739+0000 7f521658d640 1 -- 172.21.15.69:0/3558996199 wait complete. 2024-08-02T17:56:58.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f521658d640 1 Processor -- start 2024-08-02T17:56:58.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f521658d640 1 -- start start 2024-08-02T17:56:58.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f521658d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5210100b50 0x7f52101990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f521658d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52101022a0 0x7f52101995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f521658d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5210102c20 0x7f52101a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f521658d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5210110cf0 con 0x7f52101022a0 2024-08-02T17:56:58.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f521658d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5210110b70 con 0x7f5210100b50 2024-08-02T17:56:58.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f521658d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5210110e70 con 0x7f5210102c20 2024-08-02T17:56:58.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52101022a0 0x7f52101995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52101022a0 0x7f52101995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42878/0 (socket says 172.21.15.69:42878) 2024-08-02T17:56:58.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 -- 172.21.15.69:0/745297822 learned_addr learned my addr 172.21.15.69:0/745297822 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f5214b2a640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5210102c20 0x7f52101a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520ffff640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5210100b50 0x7f52101990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 -- 172.21.15.69:0/745297822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5210102c20 msgr2=0x7f52101a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5210102c20 0x7f52101a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 -- 172.21.15.69:0/745297822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5210100b50 msgr2=0x7f52101990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5210100b50 0x7f52101990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 -- 172.21.15.69:0/745297822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52101a4660 con 0x7f52101022a0 2024-08-02T17:56:58.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520ffff640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5210100b50 0x7f52101990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:58.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.740+0000 7f520f7fe640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52101022a0 0x7f52101995e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f520000b4a0 tx=0x7f520000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:58.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.741+0000 7f520d7fa640 1 -- 172.21.15.69:0/745297822 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200014070 con 0x7f52101022a0 2024-08-02T17:56:58.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.741+0000 7f520d7fa640 1 -- 172.21.15.69:0/745297822 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5200010af0 con 0x7f52101022a0 2024-08-02T17:56:58.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.741+0000 7f521658d640 1 -- 172.21.15.69:0/745297822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52101a4830 con 0x7f52101022a0 2024-08-02T17:56:58.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.741+0000 7f520d7fa640 1 -- 172.21.15.69:0/745297822 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200017d10 con 0x7f52101022a0 2024-08-02T17:56:58.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.741+0000 7f521658d640 1 -- 172.21.15.69:0/745297822 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52101a4d70 con 0x7f52101022a0 2024-08-02T17:56:58.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f520d7fa640 1 -- 172.21.15.69:0/745297822 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5200017070 con 0x7f52101022a0 2024-08-02T17:56:58.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f521898b740 1 Processor -- start 2024-08-02T17:56:58.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f521898b740 1 -- start start 2024-08-02T17:56:58.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f520d7fa640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51fc0a2cf0 0x7f51fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.759 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f521898b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d9999cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f521898b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f521898b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f521898b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556d99902c60 con 0x556d999a7b70 2024-08-02T17:56:58.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f521898b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556d9989ab90 con 0x556d9999d2b0 2024-08-02T17:56:58.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.743+0000 7f521898b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d998fec20 con 0x556d9999c990 2024-08-02T17:56:58.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51fc0a2cf0 0x7f51fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f520ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d9999cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f520d7fa640 1 -- 172.21.15.69:0/745297822 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f52000ceb20 con 0x7f52101022a0 2024-08-02T17:56:58.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f520ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d9999cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53386/0 (socket says 172.21.15.69:53386) 2024-08-02T17:56:58.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f520ffff640 1 -- 172.21.15.69:0/2077845053 learned_addr learned my addr 172.21.15.69:0/2077845053 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:58.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f520f7fe640 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 -- 172.21.15.69:0/2077845053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 msgr2=0x556d9999cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:58.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d9999cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 -- 172.21.15.69:0/2077845053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 msgr2=0x556d999a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:58.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 -- 172.21.15.69:0/2077845053 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d99612580 con 0x556d999a7b70 2024-08-02T17:56:58.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51fc0a2cf0 0x7f51fc0a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f52080098b0 tx=0x7f5208009e20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:58.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f5214b2a640 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a9f60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5208005fd0 tx=0x7f520801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea762ae6be201cc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f52067fc640 1 -- 172.21.15.69:0/2077845053 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208024070 con 0x556d999a7b70 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f52067fc640 1 -- 172.21.15.69:0/2077845053 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f520800e820 con 0x556d999a7b70 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.744+0000 7f52067fc640 1 -- 172.21.15.69:0/2077845053 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208005330 con 0x556d999a7b70 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 -- 172.21.15.69:0/2077845053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 msgr2=0x556d999a9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a9f60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f5208005fd0 tx=0x7f520801c000 comp rx=0 tx=0).stop 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 -- 172.21.15.69:0/2077845053 shutdown_connections 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a9f60 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 --2- 172.21.15.69:0/2077845053 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d9999cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 -- 172.21.15.69:0/2077845053 >> 172.21.15.69:0/2077845053 conn(0x556d99998f20 msgr2=0x556d99999300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:58.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 -- 172.21.15.69:0/2077845053 shutdown_connections 2024-08-02T17:56:58.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 -- 172.21.15.69:0/2077845053 wait complete. 2024-08-02T17:56:58.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 -- start start 2024-08-02T17:56:58.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.745+0000 7f521898b740 1 Processor -- start 2024-08-02T17:56:58.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f521898b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999c990 0x556d999a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f521898b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999d2b0 0x556d999a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f521898b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f521898b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556d999aa740 con 0x556d999a7b70 2024-08-02T17:56:58.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f521898b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556d998a07c0 con 0x556d9999c990 2024-08-02T17:56:58.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f521898b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d999aa8c0 con 0x556d9999d2b0 2024-08-02T17:56:58.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f521898b740 10 client.0 _ll_register_callbacks cb 0x556d998a9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:58.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42892/0 (socket says 172.21.15.69:42892) 2024-08-02T17:56:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 -- 172.21.15.69:0/3136934416 learned_addr learned my addr 172.21.15.69:0/3136934416 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f520ffff640 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999c990 0x556d999a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f520f7fe640 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999d2b0 0x556d999a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 -- 172.21.15.69:0/3136934416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999d2b0 msgr2=0x556d999a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999d2b0 0x556d999a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 -- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999c990 msgr2=0x556d999a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999c990 0x556d999a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 -- 172.21.15.69:0/3136934416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d996dc960 con 0x556d999a7b70 2024-08-02T17:56:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f520ffff640 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999c990 0x556d999a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f5214b2a640 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a54c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5208009880 tx=0x7f5208005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67640f7816d892fc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:58.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f51ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:56:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.746+0000 7f51ef7fe640 1 -- 172.21.15.69:0/3136934416 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208024050 con 0x556d999a7b70 2024-08-02T17:56:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 1 -- 172.21.15.69:0/3136934416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f520800e820 con 0x556d999a7b70 2024-08-02T17:56:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f51ef7fe640 1 -- 172.21.15.69:0/3136934416 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f520800ebe0 con 0x556d999a7b70 2024-08-02T17:56:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f51ef7fe640 1 -- 172.21.15.69:0/3136934416 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5208039050 con 0x556d999a7b70 2024-08-02T17:56:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 10 client.17967 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 1 -- 172.21.15.69:0/3136934416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556d999b67b0 con 0x556d999a7b70 2024-08-02T17:56:58.770 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f51ef7fe640 1 -- 172.21.15.69:0/3136934416 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f520800e820 con 0x556d999a7b70 2024-08-02T17:56:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f51ef7fe640 1 -- 172.21.15.69:0/3136934416 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f520801f070 con 0x556d999a7b70 2024-08-02T17:56:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 10 client.17967 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 10 client.17967 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:56:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 20 client.17967 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:58.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d999b7b00 0x556d999b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 4 client.17967 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f521898b740 1 -- 172.21.15.69:0/3136934416 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d999101d0 con 0x556d999b7b00 2024-08-02T17:56:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.747+0000 7f520ffff640 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d999b7b00 0x556d999b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.748+0000 7f51ef7fe640 10 client.17967 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.748+0000 7f520ffff640 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d999b7b00 0x556d999b9ef0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933d04b4490e1c40 server_cookie=aa88d991d2e714ac in_seq=0 out_seq=0 2024-08-02T17:56:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f51ef7fe640 1 -- 172.21.15.69:0/3136934416 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556d999101d0 con 0x556d999b7b00 2024-08-02T17:56:58.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f51ef7fe640 10 client.17967 handle_command_reply: tid=0 2024-08-02T17:56:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 client.17967 shutdown 2024-08-02T17:56:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 -- 172.21.15.69:0/3136934416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 msgr2=0x556d999a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a54c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5208009880 tx=0x7f5208005340 comp rx=0 tx=0).stop 2024-08-02T17:56:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 -- 172.21.15.69:0/3136934416 shutdown_connections 2024-08-02T17:56:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a54c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d999b7b00 0x556d999b9ef0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999d2b0 0x556d999a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 --2- 172.21.15.69:0/3136934416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999c990 0x556d999a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 -- 172.21.15.69:0/3136934416 >> 172.21.15.69:0/3136934416 conn(0x556d99998f20 msgr2=0x556d9999ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f51ef7fe640 0 client.17967 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 -- 172.21.15.69:0/3136934416 shutdown_connections 2024-08-02T17:56:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 1 -- 172.21.15.69:0/3136934416 wait complete. 2024-08-02T17:56:58.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.749+0000 7f521898b740 20 client.17967 trim_cache size 0 max 16384 2024-08-02T17:56:58.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.766+0000 7f521898b740 1 Processor -- start 2024-08-02T17:56:58.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.766+0000 7f521898b740 1 -- start start 2024-08-02T17:56:58.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.766+0000 7f521898b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999b9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.766+0000 7f521898b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.766+0000 7f521898b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.766+0000 7f521898b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52101b18e0 con 0x556d999a7b70 2024-08-02T17:56:58.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.766+0000 7f521898b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52101af1b0 con 0x556d9999d2b0 2024-08-02T17:56:58.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.766+0000 7f521898b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52101ad000 con 0x556d9999c990 2024-08-02T17:56:58.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999b9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999b9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53408/0 (socket says 172.21.15.69:53408) 2024-08-02T17:56:58.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 -- 172.21.15.69:0/3281127137 learned_addr learned my addr 172.21.15.69:0/3281127137 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f5214b2a640 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520f7fe640 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 -- 172.21.15.69:0/3281127137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 msgr2=0x556d999a39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 -- 172.21.15.69:0/3281127137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 msgr2=0x556d999bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999bf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 -- 172.21.15.69:0/3281127137 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d9999d8d0 con 0x556d9999c990 2024-08-02T17:56:58.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f5214b2a640 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f520ffff640 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999b9d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009900 tx=0x7f51f8005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28419c6435ff0d7f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f5205ffb640 1 -- 172.21.15.69:0/3281127137 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8035040 con 0x556d9999c990 2024-08-02T17:56:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f5205ffb640 1 -- 172.21.15.69:0/3281127137 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f80046c0 con 0x556d9999c990 2024-08-02T17:56:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.767+0000 7f5205ffb640 1 -- 172.21.15.69:0/3281127137 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8003850 con 0x556d9999c990 2024-08-02T17:56:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 -- 172.21.15.69:0/3281127137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 msgr2=0x556d999b9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999b9d40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009900 tx=0x7f51f8005b50 comp rx=0 tx=0).stop 2024-08-02T17:56:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 -- 172.21.15.69:0/3281127137 shutdown_connections 2024-08-02T17:56:58.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 --2- 172.21.15.69:0/3281127137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999b9d40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 -- 172.21.15.69:0/3281127137 >> 172.21.15.69:0/3281127137 conn(0x556d99998f20 msgr2=0x556d999a6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 -- 172.21.15.69:0/3281127137 shutdown_connections 2024-08-02T17:56:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.768+0000 7f521898b740 1 -- 172.21.15.69:0/3281127137 wait complete. 2024-08-02T17:56:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 1 -- start start 2024-08-02T17:56:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 1 Processor -- start 2024-08-02T17:56:58.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52101ad670 con 0x556d999a7b70 2024-08-02T17:56:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52101aeae0 con 0x556d9999d2b0 2024-08-02T17:56:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52101add40 con 0x556d9999c990 2024-08-02T17:56:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f521898b740 10 client.0 _ll_register_callbacks cb 0x7f5208009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f5214b2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f520ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f520ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999a1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53414/0 (socket says 172.21.15.69:53414) 2024-08-02T17:56:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f520ffff640 1 -- 172.21.15.69:0/2633920206 learned_addr learned my addr 172.21.15.69:0/2633920206 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f5214b2a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42908/0 (socket says 172.21.15.69:42908) 2024-08-02T17:56:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.769+0000 7f520f7fe640 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f520ffff640 1 -- 172.21.15.69:0/2633920206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 msgr2=0x556d999a2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:58.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f520ffff640 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f520ffff640 1 -- 172.21.15.69:0/2633920206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 msgr2=0x556d999a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f520ffff640 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f520ffff640 1 -- 172.21.15.69:0/2633920206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d99a215e0 con 0x556d9999c990 2024-08-02T17:56:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f5214b2a640 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:56:58.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f520ffff640 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999a1ad0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51f8004550 tx=0x7f51f8005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d19eb2f94d78d021 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:58.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f51ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:58.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f51ef7fe640 1 -- 172.21.15.69:0/2633920206 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8052070 con 0x556d9999c990 2024-08-02T17:56:58.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f521898b740 1 -- 172.21.15.69:0/2633920206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f80046c0 con 0x556d9999c990 2024-08-02T17:56:58.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f51ef7fe640 1 -- 172.21.15.69:0/2633920206 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f51f8004c80 con 0x556d9999c990 2024-08-02T17:56:58.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f51ef7fe640 1 -- 172.21.15.69:0/2633920206 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51f805e050 con 0x556d9999c990 2024-08-02T17:56:58.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f521898b740 10 client.27289 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.770+0000 7f521898b740 1 -- 172.21.15.69:0/2633920206 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556d99a22900 con 0x556d9999c990 2024-08-02T17:56:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f51ef7fe640 1 -- 172.21.15.69:0/2633920206 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f80046c0 con 0x556d9999c990 2024-08-02T17:56:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f51ef7fe640 1 -- 172.21.15.69:0/2633920206 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51f80315d0 con 0x556d9999c990 2024-08-02T17:56:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f521898b740 10 client.27289 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f521898b740 10 client.27289 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:56:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f521898b740 20 client.27289 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:58.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f521898b740 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d99a23060 0x556d99a25480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f521898b740 4 client.27289 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "71426704", "release": true, "await": true}] 2024-08-02T17:56:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f521898b740 1 -- 172.21.15.69:0/2633920206 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "71426704", "release": true, "await": true}) -- 0x556d9990a6e0 con 0x556d99a23060 2024-08-02T17:56:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.771+0000 7f520f7fe640 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d99a23060 0x556d99a25480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:58.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.772+0000 7f51ef7fe640 10 client.27289 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:58.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.772+0000 7f520f7fe640 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d99a23060 0x556d99a25480 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25409b04d400c64f server_cookie=9aeabad5e577ee4f in_seq=0 out_seq=0 2024-08-02T17:56:58.793 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:56:58.827 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:56:58.827 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 36, 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "71426704": { 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 36, 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 139.9, 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:56:58.828 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:56:58.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f51ef7fe640 1 -- 172.21.15.69:0/2633920206 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556d9990a6e0 con 0x556d99a23060 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f51ef7fe640 10 client.27289 handle_command_reply: tid=0 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f521898b740 1 client.27289 shutdown 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f521898b740 1 -- 172.21.15.69:0/2633920206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 msgr2=0x556d999a1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f521898b740 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999a1ad0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51f8004550 tx=0x7f51f8005450 comp rx=0 tx=0).stop 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f521898b740 1 -- 172.21.15.69:0/2633920206 shutdown_connections 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f521898b740 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d999a7b70 0x556d999a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f521898b740 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d99a23060 0x556d99a25480 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.822+0000 7f521898b740 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d9999d2b0 0x556d999a2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.823+0000 7f521898b740 1 --2- 172.21.15.69:0/2633920206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d9999c990 0x556d999a1ad0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.823+0000 7f51ef7fe640 0 client.27289 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.823+0000 7f521898b740 1 -- 172.21.15.69:0/2633920206 >> 172.21.15.69:0/2633920206 conn(0x556d99998f20 msgr2=0x556d9999e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.823+0000 7f521898b740 1 -- 172.21.15.69:0/2633920206 shutdown_connections 2024-08-02T17:56:58.830 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.823+0000 7f521898b740 1 -- 172.21.15.69:0/2633920206 wait complete. 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.823+0000 7f521898b740 20 client.27289 trim_cache size 0 max 16384 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 -- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51fc0a2cf0 msgr2=0x7f51fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51fc0a2cf0 0x7f51fc0a51b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f52080098b0 tx=0x7f5208009e20 comp rx=0 tx=0).stop 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 -- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52101022a0 msgr2=0x7f52101995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52101022a0 0x7f52101995e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f520000b4a0 tx=0x7f520000b960 comp rx=0 tx=0).stop 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 -- 172.21.15.69:0/745297822 shutdown_connections 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52101022a0 0x7f52101995e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51fc0a2cf0 0x7f51fc0a51b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5210102c20 0x7f52101a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 --2- 172.21.15.69:0/745297822 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5210100b50 0x7f52101990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:58.831 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.824+0000 7f521658d640 1 -- 172.21.15.69:0/745297822 >> 172.21.15.69:0/745297822 conn(0x7f52100fc500 msgr2=0x7f52100fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:58.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.828+0000 7f521658d640 1 -- 172.21.15.69:0/745297822 shutdown_connections 2024-08-02T17:56:58.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:58.831+0000 7f521658d640 1 -- 172.21.15.69:0/745297822 wait complete. 2024-08-02T17:56:59.036 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:56:58 smithi069 ceph-mon[32746]: pgmap v327: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-08-02T17:56:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:56:58 smithi083 ceph-mon[37031]: pgmap v327: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-08-02T17:56:59.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:56:58 smithi077 ceph-mon[37108]: pgmap v327: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 3.6 MiB/s wr, 430 op/s 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 -- 172.21.15.69:0/1701241919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ac0108010 msgr2=0x7f3ac010e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/1701241919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ac0108010 0x7f3ac010e980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8009c10 tx=0x7f3ab802a6f0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 -- 172.21.15.69:0/1701241919 shutdown_connections 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/1701241919 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0103490 0x7f3ac01078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/1701241919 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ac0108010 0x7f3ac010e980 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/1701241919 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac0102ae0 0x7f3ac0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 -- 172.21.15.69:0/1701241919 >> 172.21.15.69:0/1701241919 conn(0x7f3ac00fc500 msgr2=0x7f3ac00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 -- 172.21.15.69:0/1701241919 shutdown_connections 2024-08-02T17:56:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.734+0000 7f3ac6b60640 1 -- 172.21.15.69:0/1701241919 wait complete. 2024-08-02T17:56:59.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3ac6b60640 1 Processor -- start 2024-08-02T17:56:59.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3ac6b60640 1 -- start start 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3ac6b60640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac0102ae0 0x7f3ac0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3ac6b60640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0103490 0x7f3ac0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3ac6b60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ac0108010 0x7f3ac019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3ac6b60640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0111de0 con 0x7f3ac0103490 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3abffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0103490 0x7f3ac0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3abffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0103490 0x7f3ac0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42934/0 (socket says 172.21.15.69:42934) 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3abffff640 1 -- 172.21.15.69:0/341028858 learned_addr learned my addr 172.21.15.69:0/341028858 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ac0108010 0x7f3ac019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.735+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac0102ae0 0x7f3ac0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0111c60 con 0x7f3ac0102ae0 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac0111f60 con 0x7f3ac0108010 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3abffff640 1 -- 172.21.15.69:0/341028858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ac0108010 msgr2=0x7f3ac019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3abffff640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ac0108010 0x7f3ac019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3abffff640 1 -- 172.21.15.69:0/341028858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac0102ae0 msgr2=0x7f3ac0198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3abffff640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac0102ae0 0x7f3ac0198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3abffff640 1 -- 172.21.15.69:0/341028858 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac019d2d0 con 0x7f3ac0103490 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3abffff640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0103490 0x7f3ac0198860 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00e9c0 tx=0x7f3aac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3abdffb640 1 -- 172.21.15.69:0/341028858 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac00ccc0 con 0x7f3ac0103490 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.736+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac019d5c0 con 0x7f3ac0103490 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.737+0000 7f3abdffb640 1 -- 172.21.15.69:0/341028858 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3aac010910 con 0x7f3ac0103490 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.737+0000 7f3abdffb640 1 -- 172.21.15.69:0/341028858 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac018840 con 0x7f3ac0103490 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.737+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac0100670 con 0x7f3ac0103490 2024-08-02T17:56:59.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.739+0000 7f3abdffb640 1 -- 172.21.15.69:0/341028858 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3aac010ac0 con 0x7f3ac0103490 2024-08-02T17:56:59.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac8f5e740 1 Processor -- start 2024-08-02T17:56:59.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac8f5e740 1 -- start start 2024-08-02T17:56:59.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3abdffb640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ab00a2d70 0x7f3ab00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87dc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8e8734a40 con 0x55a8e87da1d0 2024-08-02T17:56:59.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a8e86cc9d0 con 0x55a8e87ceff0 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8e8730a00 con 0x55a8e87cf910 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac50fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87dc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac48fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac50fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87dc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42938/0 (socket says 172.21.15.69:42938) 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac48fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53436/0 (socket says 172.21.15.69:53436) 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac50fd640 1 -- 172.21.15.69:0/2351661167 learned_addr learned my addr 172.21.15.69:0/2351661167 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3abdffb640 1 -- 172.21.15.69:0/341028858 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f3aac0cfa10 con 0x7f3ac0103490 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.740+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/2351661167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3abffff640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ab00a2d70 0x7f3ab00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3ac50fd640 1 -- 172.21.15.69:0/2351661167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 msgr2=0x55a8e87d9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/2351661167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3ac50fd640 1 -- 172.21.15.69:0/2351661167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 msgr2=0x55a8e87cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:59.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/2351661167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3ac50fd640 1 -- 172.21.15.69:0/2351661167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e86d3650 con 0x55a8e87da1d0 2024-08-02T17:56:59.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/2351661167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87dc5c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8004af0 tx=0x7f3ab802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4c4da699cbfa7d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:59.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3abffff640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ab00a2d70 0x7f3ab00a5230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009230 tx=0x7f3aac00efb0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:59.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.741+0000 7f3aaaffd640 1 -- 172.21.15.69:0/2351661167 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8036040 con 0x55a8e87da1d0 2024-08-02T17:56:59.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.742+0000 7f3aaaffd640 1 -- 172.21.15.69:0/2351661167 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab8003e50 con 0x55a8e87da1d0 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.742+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/2351661167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 msgr2=0x55a8e87dc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.742+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/2351661167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87dc5c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8004af0 tx=0x7f3ab802a6f0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.742+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/2351661167 shutdown_connections 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.742+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/2351661167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87dc5c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.742+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/2351661167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.742+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/2351661167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.742+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/2351661167 >> 172.21.15.69:0/2351661167 conn(0x55a8e87cb580 msgr2=0x55a8e87cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.743+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/2351661167 shutdown_connections 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.743+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/2351661167 wait complete. 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.743+0000 7f3ac8f5e740 1 -- start start 2024-08-02T17:56:59.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.743+0000 7f3ac8f5e740 1 Processor -- start 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87ceff0 0x55a8e87d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87cf910 0x55a8e87d6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a8e87dce40 con 0x55a8e87da1d0 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a8e86efb20 con 0x55a8e87cf910 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a8e87dcfc0 con 0x55a8e87ceff0 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87ceff0 0x55a8e87d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac8f5e740 10 client.0 _ll_register_callbacks cb 0x55a8e86edbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87ceff0 0x55a8e87d9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53452/0 (socket says 172.21.15.69:53452) 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 -- 172.21.15.69:0/602496669 learned_addr learned my addr 172.21.15.69:0/602496669 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87cf910 0x55a8e87d6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 -- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87cf910 msgr2=0x55a8e87d6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87cf910 0x55a8e87d6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 -- 172.21.15.69:0/602496669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 msgr2=0x55a8e87d76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 -- 172.21.15.69:0/602496669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e84a63c0 con 0x55a8e87ceff0 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87cf910 0x55a8e87d6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87ceff0 0x55a8e87d9c40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8002230 tx=0x7f3ab80042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6665ef9ac3150370 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.744+0000 7f3a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3a9ffff640 1 -- 172.21.15.69:0/602496669 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8054060 con 0x55a8e87ceff0 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3a9ffff640 1 -- 172.21.15.69:0/602496669 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f3ab8003b80 con 0x55a8e87ceff0 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/602496669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab8003e50 con 0x55a8e87ceff0 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3a9ffff640 1 -- 172.21.15.69:0/602496669 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ab805b050 con 0x55a8e87ceff0 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3ac8f5e740 10 client.27301 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:59.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/602496669 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8e87dce40 con 0x55a8e87ceff0 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3a9ffff640 1 -- 172.21.15.69:0/602496669 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab8003e50 con 0x55a8e87ceff0 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3a9ffff640 1 -- 172.21.15.69:0/602496669 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ab803c040 con 0x55a8e87ceff0 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3ac8f5e740 10 client.27301 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3ac8f5e740 10 client.27301 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3ac8f5e740 20 client.27301 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.745+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8e87f22d0 0x55a8e87f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.746+0000 7f3ac8f5e740 4 client.27301 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.746+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/602496669 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8e8741fb0 con 0x55a8e87f22d0 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.746+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8e87f22d0 0x55a8e87f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.746+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8e87f22d0 0x55a8e87f46c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d743987fd097024c server_cookie=9d0b5d45b60634de in_seq=0 out_seq=0 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.746+0000 7f3a9ffff640 10 client.27301 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3a9ffff640 1 -- 172.21.15.69:0/602496669 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a8e8741fb0 con 0x55a8e87f22d0 2024-08-02T17:56:59.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3a9ffff640 10 client.27301 handle_command_reply: tid=0 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 client.27301 shutdown 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/602496669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87ceff0 msgr2=0x55a8e87d9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87ceff0 0x55a8e87d9c40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8002230 tx=0x7f3ab80042c0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/602496669 shutdown_connections 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8e87f22d0 0x55a8e87f46c0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3a9ffff640 0 client.27301 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87cf910 0x55a8e87d6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/602496669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87ceff0 0x55a8e87d9c40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/602496669 >> 172.21.15.69:0/602496669 conn(0x55a8e87cb580 msgr2=0x55a8e87d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/602496669 shutdown_connections 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.747+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/602496669 wait complete. 2024-08-02T17:56:59.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.748+0000 7f3ac8f5e740 20 client.27301 trim_cache size 0 max 16384 2024-08-02T17:56:59.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.769+0000 7f3ac8f5e740 1 Processor -- start 2024-08-02T17:56:59.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.769+0000 7f3ac8f5e740 1 -- start start 2024-08-02T17:56:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87ee980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01af5e0 con 0x55a8e87da1d0 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01aceb0 con 0x55a8e87ceff0 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01aad00 con 0x55a8e87cf910 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac50fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac50fd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42958/0 (socket says 172.21.15.69:42958) 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac50fd640 1 -- 172.21.15.69:0/263506507 learned_addr learned my addr 172.21.15.69:0/263506507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87ee980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac48fc640 1 -- 172.21.15.69:0/263506507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 msgr2=0x55a8e87ee980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87ee980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac48fc640 1 -- 172.21.15.69:0/263506507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 msgr2=0x55a8e87d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:59.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac48fc640 1 -- 172.21.15.69:0/263506507 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8e87d7a40 con 0x55a8e87cf910 2024-08-02T17:56:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.770+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:56:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.771+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d2210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55a8e87f4d10 tx=0x7f3ab4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8af0ba5b58f50b03 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.771+0000 7f3aaa7fc640 1 -- 172.21.15.69:0/263506507 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4005020 con 0x55a8e87cf910 2024-08-02T17:56:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.771+0000 7f3aaa7fc640 1 -- 172.21.15.69:0/263506507 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab40047a0 con 0x55a8e87cf910 2024-08-02T17:56:59.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.771+0000 7f3aaa7fc640 1 -- 172.21.15.69:0/263506507 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab40286b0 con 0x55a8e87cf910 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/263506507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 msgr2=0x55a8e87d2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d2210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55a8e87f4d10 tx=0x7f3ab4005f90 comp rx=0 tx=0).stop 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/263506507 shutdown_connections 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87da1d0 0x55a8e87d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87cf910 0x55a8e87d2210 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/263506507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87ee980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/263506507 >> 172.21.15.69:0/263506507 conn(0x55a8e87cb580 msgr2=0x55a8e87c61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/263506507 shutdown_connections 2024-08-02T17:56:59.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/263506507 wait complete. 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.772+0000 7f3ac8f5e740 1 -- start start 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac8f5e740 1 Processor -- start 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87da1d0 0x55a8e87daaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac8f5e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87f22d0 0x55a8e87db030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01ab370 con 0x55a8e87f22d0 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01ac7e0 con 0x55a8e87ceff0 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac8f5e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac01aba40 con 0x55a8e87da1d0 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac8f5e740 10 client.0 _ll_register_callbacks cb 0x7f3ab4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac50fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac50fd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87da5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35938/0 (socket says 172.21.15.69:35938) 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac50fd640 1 -- 172.21.15.69:0/237692612 learned_addr learned my addr 172.21.15.69:0/237692612 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:56:59.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.773+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87f22d0 0x55a8e87db030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.774+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87da1d0 0x55a8e87daaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.774+0000 7f3ac50fd640 1 -- 172.21.15.69:0/237692612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87da1d0 msgr2=0x55a8e87daaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.774+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87da1d0 0x55a8e87daaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.774+0000 7f3ac50fd640 1 -- 172.21.15.69:0/237692612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87f22d0 msgr2=0x55a8e87db030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.774+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87f22d0 0x55a8e87db030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.774+0000 7f3ac50fd640 1 -- 172.21.15.69:0/237692612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8e87e9af0 con 0x55a8e87ceff0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.775+0000 7f3ac50fd640 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87da5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8030c60 tx=0x7f3ab8004220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a33d4dad0e7aafa4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.775+0000 7f3a9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.775+0000 7f3a9ffff640 1 -- 172.21.15.69:0/237692612 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8054040 con 0x55a8e87ceff0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.775+0000 7f3a9ffff640 1 -- 172.21.15.69:0/237692612 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f3ab804d450 con 0x55a8e87ceff0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.775+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/237692612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab40047a0 con 0x55a8e87ceff0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.775+0000 7f3a9ffff640 1 -- 172.21.15.69:0/237692612 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3ab8036070 con 0x55a8e87ceff0 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.775+0000 7f3ac8f5e740 10 client.27470 fetch_fsmap learned FSMap version 26 2024-08-02T17:56:59.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.775+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/237692612 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a8e87eadd0 con 0x55a8e87ceff0 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3a9ffff640 1 -- 172.21.15.69:0/237692612 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab40047a0 con 0x55a8e87ceff0 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3a9ffff640 1 -- 172.21.15.69:0/237692612 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3ab803c040 con 0x55a8e87ceff0 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3ac8f5e740 10 client.27470 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3ac8f5e740 10 client.27470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3ac8f5e740 20 client.27470 populate_metadata read hostname 'smithi069' 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8e87eb520 0x55a8e885ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3ac8f5e740 4 client.27470 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/237692612 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8e873c4c0 con 0x55a8e87eb520 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.776+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8e87eb520 0x55a8e885ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3a9ffff640 10 client.27470 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac48fc640 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8e87eb520 0x55a8e885ed50 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b6c99bda0121fb6 server_cookie=7783d5930267b854 in_seq=0 out_seq=0 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3a9ffff640 1 -- 172.21.15.69:0/237692612 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8e873c4c0 con 0x55a8e87eb520 2024-08-02T17:56:59.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3a9ffff640 10 client.27470 handle_command_reply: tid=0 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 client.27470 shutdown 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/237692612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 msgr2=0x55a8e87da5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87da5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8030c60 tx=0x7f3ab8004220 comp rx=0 tx=0).stop 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/237692612 shutdown_connections 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a8e87f22d0 0x55a8e87db030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a8e87eb520 0x55a8e885ed50 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a8e87da1d0 0x55a8e87daaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3a9ffff640 0 client.27470 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 --2- 172.21.15.69:0/237692612 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a8e87ceff0 0x55a8e87da5b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.777+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/237692612 >> 172.21.15.69:0/237692612 conn(0x55a8e87cb580 msgr2=0x55a8e87d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.778+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/237692612 shutdown_connections 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.778+0000 7f3ac8f5e740 1 -- 172.21.15.69:0/237692612 wait complete. 2024-08-02T17:56:59.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.778+0000 7f3ac8f5e740 20 client.27470 trim_cache size 0 max 16384 2024-08-02T17:56:59.785 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:56:59.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.778+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ab00a2d70 msgr2=0x7f3ab00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.778+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ab00a2d70 0x7f3ab00a5230 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009230 tx=0x7f3aac00efb0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0103490 msgr2=0x7f3ac0198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:56:59.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0103490 0x7f3ac0198860 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00e9c0 tx=0x7f3aac00ee80 comp rx=0 tx=0).stop 2024-08-02T17:56:59.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 shutdown_connections 2024-08-02T17:56:59.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3ac0103490 0x7f3ac0198860 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ab00a2d70 0x7f3ab00a5230 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ac0108010 0x7f3ac019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 --2- 172.21.15.69:0/341028858 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3ac0102ae0 0x7f3ac0198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:56:59.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 >> 172.21.15.69:0/341028858 conn(0x7f3ac00fc500 msgr2=0x7f3ac0105ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:56:59.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 shutdown_connections 2024-08-02T17:56:59.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:56:59.779+0000 7f3ac6b60640 1 -- 172.21.15.69:0/341028858 wait complete. 2024-08-02T17:56:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 590.0 s, 56.0 tps, lat 3122.011 ms stddev 4180.676 2024-08-02T17:57:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 -- 172.21.15.69:0/995715468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0660101070 msgr2=0x7f066010d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 --2- 172.21.15.69:0/995715468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0660101070 0x7f066010d5a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0658009930 tx=0x7f065802a460 comp rx=0 tx=0).stop 2024-08-02T17:57:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 -- 172.21.15.69:0/995715468 shutdown_connections 2024-08-02T17:57:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 --2- 172.21.15.69:0/995715468 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0660101070 0x7f066010d5a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 --2- 172.21.15.69:0/995715468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06601006d0 0x7f0660100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 --2- 172.21.15.69:0/995715468 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06601066e0 0x7f0660106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 -- 172.21.15.69:0/995715468 >> 172.21.15.69:0/995715468 conn(0x7f06600776d0 msgr2=0x7f06600fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 -- 172.21.15.69:0/995715468 shutdown_connections 2024-08-02T17:57:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.180+0000 7f0665ae3640 1 -- 172.21.15.69:0/995715468 wait complete. 2024-08-02T17:57:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0665ae3640 1 Processor -- start 2024-08-02T17:57:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0665ae3640 1 -- start start 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0665ae3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06601006d0 0x7f06601980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0665ae3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0660101070 0x7f0660198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0665ae3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06601066e0 0x7f066019c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0665ae3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0660110b70 con 0x7f06601006d0 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0665ae3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f06601109f0 con 0x7f06601066e0 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0665ae3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0660110cf0 con 0x7f0660101070 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0656ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0660101070 0x7f0660198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0656ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0660101070 0x7f0660198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53494/0 (socket says 172.21.15.69:53494) 2024-08-02T17:57:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f0656ffd640 1 -- 172.21.15.69:0/932531445 learned_addr learned my addr 172.21.15.69:0/932531445 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f065f7fe640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06601006d0 0x7f06601980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.181+0000 7f065ffff640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06601066e0 0x7f066019c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.182+0000 7f065f7fe640 1 -- 172.21.15.69:0/932531445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0660101070 msgr2=0x7f0660198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.182+0000 7f065f7fe640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0660101070 0x7f0660198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.182+0000 7f065f7fe640 1 -- 172.21.15.69:0/932531445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06601066e0 msgr2=0x7f066019c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.182+0000 7f065f7fe640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06601066e0 0x7f066019c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.182+0000 7f065f7fe640 1 -- 172.21.15.69:0/932531445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066019d0c0 con 0x7f06601006d0 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.182+0000 7f065f7fe640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06601006d0 0x7f06601980d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f065000be20 tx=0x7f0650009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.183+0000 7f065d7fa640 1 -- 172.21.15.69:0/932531445 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650019070 con 0x7f06601006d0 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.183+0000 7f0665ae3640 1 -- 172.21.15.69:0/932531445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066019d3b0 con 0x7f06601006d0 2024-08-02T17:57:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.183+0000 7f065d7fa640 1 -- 172.21.15.69:0/932531445 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0650014920 con 0x7f06601006d0 2024-08-02T17:57:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.184+0000 7f065d7fa640 1 -- 172.21.15.69:0/932531445 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650019070 con 0x7f06601006d0 2024-08-02T17:57:00.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.184+0000 7f0665ae3640 1 -- 172.21.15.69:0/932531445 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066006b940 con 0x7f06601006d0 2024-08-02T17:57:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.187+0000 7f0668835740 1 Processor -- start 2024-08-02T17:57:00.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f0668835740 1 -- start start 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f0668835740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x563847148d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f0668835740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 0x563847144790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f0668835740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x563847150690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f0668835740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56384709e9b0 con 0x563847144330 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f0668835740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563847046800 con 0x563847148990 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f0668835740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56384709a970 con 0x563847144cd0 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f065ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x563847148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f065ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x563847148d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35978/0 (socket says 172.21.15.69:35978) 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f0656ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x563847150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.188+0000 7f065ffff640 1 -- 172.21.15.69:0/694230451 learned_addr learned my addr 172.21.15.69:0/694230451 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f065ffff640 1 -- 172.21.15.69:0/694230451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 msgr2=0x563847150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f065ffff640 1 --2- 172.21.15.69:0/694230451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x563847150690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.192 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f065ffff640 1 -- 172.21.15.69:0/694230451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 msgr2=0x563847144790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f065ffff640 1 --2- 172.21.15.69:0/694230451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 0x563847144790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f065ffff640 1 -- 172.21.15.69:0/694230451 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563846db2210 con 0x563847148990 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f065d7fa640 1 -- 172.21.15.69:0/932531445 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0650014440 con 0x7f06601006d0 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f065ffff640 1 --2- 172.21.15.69:0/694230451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x563847148d70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0658009930 tx=0x7f06580040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6b642e4f4ed3d64 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f0655ffb640 1 -- 172.21.15.69:0/694230451 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065803f030 con 0x563847148990 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f0655ffb640 1 -- 172.21.15.69:0/694230451 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0658005500 con 0x563847148990 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f0655ffb640 1 -- 172.21.15.69:0/694230451 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658035070 con 0x563847148990 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.189+0000 7f065d7fa640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f064c0c5180 0x7f064c0c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f065d7fa640 1 -- 172.21.15.69:0/932531445 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f065000ccf0 con 0x7f06601006d0 2024-08-02T17:57:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f065f7fe640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f064c0c5180 0x7f064c0c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f0668835740 1 -- 172.21.15.69:0/694230451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 msgr2=0x563847148d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f0668835740 1 --2- 172.21.15.69:0/694230451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x563847148d70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0658009930 tx=0x7f06580040f0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f0668835740 1 -- 172.21.15.69:0/694230451 shutdown_connections 2024-08-02T17:57:00.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f0668835740 1 --2- 172.21.15.69:0/694230451 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 0x563847144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f0668835740 1 --2- 172.21.15.69:0/694230451 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x563847150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f0668835740 1 --2- 172.21.15.69:0/694230451 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x563847148d70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.190+0000 7f0668835740 1 -- 172.21.15.69:0/694230451 >> 172.21.15.69:0/694230451 conn(0x5638471367e0 msgr2=0x563847136bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.191+0000 7f0668835740 1 -- 172.21.15.69:0/694230451 shutdown_connections 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.191+0000 7f0668835740 1 -- 172.21.15.69:0/694230451 wait complete. 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.191+0000 7f0668835740 1 -- start start 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.192+0000 7f0668835740 1 Processor -- start 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.192+0000 7f0668835740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 0x56384714daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.192+0000 7f0668835740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x56384714dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.192+0000 7f0668835740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x56384714e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.192+0000 7f0668835740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563847150d70 con 0x563847144330 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.192+0000 7f0668835740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56384705a500 con 0x563847148990 2024-08-02T17:57:00.195 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.192+0000 7f0668835740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563847150ef0 con 0x563847144cd0 2024-08-02T17:57:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.192+0000 7f0668835740 10 client.0 _ll_register_callbacks cb 0x563846db1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f0656ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x56384714dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f0656ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x56384714dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53520/0 (socket says 172.21.15.69:53520) 2024-08-02T17:57:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f0656ffd640 1 -- 172.21.15.69:0/1825045222 learned_addr learned my addr 172.21.15.69:0/1825045222 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f065ffff640 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 0x56384714daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f065ffff640 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x56384714e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f065ffff640 1 -- 172.21.15.69:0/1825045222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 msgr2=0x56384714dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f065ffff640 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x56384714dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f065ffff640 1 -- 172.21.15.69:0/1825045222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 msgr2=0x56384714daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f065ffff640 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 0x56384714daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f065ffff640 1 -- 172.21.15.69:0/1825045222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563846e7e620 con 0x563847148990 2024-08-02T17:57:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.193+0000 7f065f7fe640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f064c0c5180 0x7f064c0c7640 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x563847137f30 tx=0x7f0650009960 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.194+0000 7f065ffff640 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x56384714e520 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0658009850 tx=0x7f0658005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7460713b021e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.194+0000 7f063effd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:00.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.194+0000 7f063effd640 1 -- 172.21.15.69:0/1825045222 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658035070 con 0x563847148990 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.195+0000 7f063effd640 1 -- 172.21.15.69:0/1825045222 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f06580315b0 con 0x563847148990 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.195+0000 7f0668835740 1 -- 172.21.15.69:0/1825045222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0658005500 con 0x563847148990 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.195+0000 7f063effd640 1 -- 172.21.15.69:0/1825045222 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f065803f040 con 0x563847148990 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.195+0000 7f0668835740 10 client.27479 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.195+0000 7f0668835740 1 -- 172.21.15.69:0/1825045222 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563847143140 con 0x563847148990 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.196+0000 7f063effd640 1 -- 172.21.15.69:0/1825045222 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0658005500 con 0x563847148990 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.196+0000 7f063effd640 1 -- 172.21.15.69:0/1825045222 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0658003790 con 0x563847148990 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.196+0000 7f0668835740 10 client.27479 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.196+0000 7f0668835740 10 client.27479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.196+0000 7f0668835740 20 client.27479 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.196+0000 7f0668835740 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563847166540 0x563847168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.196+0000 7f0668835740 4 client.27479 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:00.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.196+0000 7f0668835740 1 -- 172.21.15.69:0/1825045222 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638470ac0e0 con 0x563847166540 2024-08-02T17:57:00.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.199+0000 7f0656ffd640 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563847166540 0x563847168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.200+0000 7f0656ffd640 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563847166540 0x563847168930 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e85daa1f99250c17 server_cookie=7a6d5dc5da9b330f in_seq=0 out_seq=0 2024-08-02T17:57:00.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.200+0000 7f063effd640 10 client.27479 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f063effd640 1 -- 172.21.15.69:0/1825045222 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5638470ac0e0 con 0x563847166540 2024-08-02T17:57:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f063effd640 10 client.27479 handle_command_reply: tid=0 2024-08-02T17:57:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 client.27479 shutdown 2024-08-02T17:57:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 -- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 msgr2=0x56384714e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x56384714e520 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0658009850 tx=0x7f0658005fd0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 -- 172.21.15.69:0/1825045222 shutdown_connections 2024-08-02T17:57:00.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563847166540 0x563847168930 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847148990 0x56384714e520 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144cd0 0x56384714dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 --2- 172.21.15.69:0/1825045222 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847144330 0x56384714daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f0668835740 1 -- 172.21.15.69:0/1825045222 >> 172.21.15.69:0/1825045222 conn(0x5638471367e0 msgr2=0x563847143b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.201+0000 7f063effd640 0 client.27479 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.202+0000 7f0668835740 1 -- 172.21.15.69:0/1825045222 shutdown_connections 2024-08-02T17:57:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.202+0000 7f0668835740 1 -- 172.21.15.69:0/1825045222 wait complete. 2024-08-02T17:57:00.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.202+0000 7f0668835740 20 client.27479 trim_cache size 0 max 16384 2024-08-02T17:57:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.237+0000 7f0668835740 1 Processor -- start 2024-08-02T17:57:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.238+0000 7f0668835740 1 -- start start 2024-08-02T17:57:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f0668835740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x56384714e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f0668835740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 0x56384714ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f065ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x56384714e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f065ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x56384714e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53532/0 (socket says 172.21.15.69:53532) 2024-08-02T17:57:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f065ffff640 1 -- 172.21.15.69:0/979647716 learned_addr learned my addr 172.21.15.69:0/979647716 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f0668835740 1 --2- 172.21.15.69:0/979647716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 0x56384714b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f0668835740 1 -- 172.21.15.69:0/979647716 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f06601a6c10 con 0x563847148990 2024-08-02T17:57:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f0668835740 1 -- 172.21.15.69:0/979647716 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0660073a40 con 0x563847144cd0 2024-08-02T17:57:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.239+0000 7f0668835740 1 -- 172.21.15.69:0/979647716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0660071890 con 0x563847144330 2024-08-02T17:57:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.244+0000 7f065ffff640 1 -- 172.21.15.69:0/979647716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 msgr2=0x56384714ace0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:57:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.244+0000 7f065ffff640 1 --2- 172.21.15.69:0/979647716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 0x56384714ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.244+0000 7f065ffff640 1 -- 172.21.15.69:0/979647716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 msgr2=0x56384714b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.244+0000 7f065ffff640 1 --2- 172.21.15.69:0/979647716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 0x56384714b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.245 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.244+0000 7f065ffff640 1 -- 172.21.15.69:0/979647716 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56384714bfd0 con 0x563847144330 2024-08-02T17:57:00.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.248+0000 7f065ffff640 1 --2- 172.21.15.69:0/979647716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x56384714e350 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0658005940 tx=0x7f0658031440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f51610f7232c7d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.251+0000 7f06557fa640 1 -- 172.21.15.69:0/979647716 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065805c030 con 0x563847144330 2024-08-02T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.251+0000 7f06557fa640 1 -- 172.21.15.69:0/979647716 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06580048a0 con 0x563847144330 2024-08-02T17:57:00.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.251+0000 7f06557fa640 1 -- 172.21.15.69:0/979647716 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658035070 con 0x563847144330 2024-08-02T17:57:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.255+0000 7f0668835740 1 -- 172.21.15.69:0/979647716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 msgr2=0x56384714e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.255+0000 7f0668835740 1 --2- 172.21.15.69:0/979647716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x56384714e350 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0658005940 tx=0x7f0658031440 comp rx=0 tx=0).stop 2024-08-02T17:57:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.259+0000 7f0668835740 1 -- 172.21.15.69:0/979647716 shutdown_connections 2024-08-02T17:57:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.259+0000 7f0668835740 1 --2- 172.21.15.69:0/979647716 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 0x56384714b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.259+0000 7f0668835740 1 --2- 172.21.15.69:0/979647716 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 0x56384714ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.259+0000 7f0668835740 1 --2- 172.21.15.69:0/979647716 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x56384714e350 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.259+0000 7f0668835740 1 -- 172.21.15.69:0/979647716 >> 172.21.15.69:0/979647716 conn(0x5638471367e0 msgr2=0x563847131b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:00.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.262+0000 7f0668835740 1 -- 172.21.15.69:0/979647716 shutdown_connections 2024-08-02T17:57:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.263+0000 7f0668835740 1 -- 172.21.15.69:0/979647716 wait complete. 2024-08-02T17:57:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.263+0000 7f0668835740 1 -- start start 2024-08-02T17:57:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.264+0000 7f0668835740 1 Processor -- start 2024-08-02T17:57:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.266+0000 7f0668835740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x5638471412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f0668835740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 0x56384713e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f0668835740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 0x56384713eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f0668835740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0660071f00 con 0x563847148990 2024-08-02T17:57:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f0668835740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0660073370 con 0x563847144cd0 2024-08-02T17:57:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f0668835740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06600725d0 con 0x563847144330 2024-08-02T17:57:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f0668835740 10 client.0 _ll_register_callbacks cb 0x7f064800a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f065ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x5638471412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f065ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x5638471412a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53546/0 (socket says 172.21.15.69:53546) 2024-08-02T17:57:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.267+0000 7f065ffff640 1 -- 172.21.15.69:0/1089491253 learned_addr learned my addr 172.21.15.69:0/1089491253 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.269+0000 7f065ffff640 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 0x56384713eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.269+0000 7f0656ffd640 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 0x56384713e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.269+0000 7f065ffff640 1 -- 172.21.15.69:0/1089491253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 msgr2=0x56384713e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.269+0000 7f065ffff640 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 0x56384713e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.269+0000 7f065ffff640 1 -- 172.21.15.69:0/1089491253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 msgr2=0x56384713eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.269+0000 7f065ffff640 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 0x56384713eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.269+0000 7f065ffff640 1 -- 172.21.15.69:0/1089491253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56384713f210 con 0x563847144330 2024-08-02T17:57:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.271+0000 7f065ffff640 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x5638471412a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06580059d0 tx=0x7f0658009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6b0709578a0ca7b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.272+0000 7f063effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:57:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.274+0000 7f063effd640 1 -- 172.21.15.69:0/1089491253 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658066020 con 0x563847144330 2024-08-02T17:57:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.275+0000 7f063effd640 1 -- 172.21.15.69:0/1089491253 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f06580037f0 con 0x563847144330 2024-08-02T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.276+0000 7f0668835740 1 -- 172.21.15.69:0/1089491253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06580048a0 con 0x563847144330 2024-08-02T17:57:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.277+0000 7f063effd640 1 -- 172.21.15.69:0/1089491253 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0658035020 con 0x563847144330 2024-08-02T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.278+0000 7f0668835740 10 client.27322 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.278+0000 7f0668835740 1 -- 172.21.15.69:0/1089491253 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638471d0d70 con 0x563847144330 2024-08-02T17:57:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.282+0000 7f063effd640 1 -- 172.21.15.69:0/1089491253 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06580048a0 con 0x563847144330 2024-08-02T17:57:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.282+0000 7f063effd640 1 -- 172.21.15.69:0/1089491253 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0658031b90 con 0x563847144330 2024-08-02T17:57:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.282+0000 7f0668835740 10 client.27322 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.282+0000 7f0668835740 10 client.27322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.282+0000 7f0668835740 20 client.27322 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.282+0000 7f0668835740 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638471d1420 0x5638471d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.282+0000 7f0668835740 4 client.27322 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:57:00.284 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.282+0000 7f0668835740 1 -- 172.21.15.69:0/1089491253 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638470a6430 con 0x5638471d1420 2024-08-02T17:57:00.285 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.283+0000 7f0656ffd640 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638471d1420 0x5638471d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.290+0000 7f0656ffd640 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638471d1420 0x5638471d3810 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95bcc4a6613470c4 server_cookie=188c68f9724e5f74 in_seq=0 out_seq=0 2024-08-02T17:57:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.292+0000 7f063effd640 10 client.27322 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:00.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.364+0000 7f063effd640 1 -- 172.21.15.69:0/1089491253 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638470a6430 con 0x5638471d1420 2024-08-02T17:57:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.364+0000 7f063effd640 10 client.27322 handle_command_reply: tid=0 2024-08-02T17:57:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.364+0000 7f0668835740 1 client.27322 shutdown 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 -- 172.21.15.69:0/1089491253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 msgr2=0x5638471412a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x5638471412a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f06580059d0 tx=0x7f0658009520 comp rx=0 tx=0).stop 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 -- 172.21.15.69:0/1089491253 shutdown_connections 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563847148990 0x56384713eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638471d1420 0x5638471d3810 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563847144cd0 0x56384713e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 --2- 172.21.15.69:0/1089491253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563847144330 0x5638471412a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 -- 172.21.15.69:0/1089491253 >> 172.21.15.69:0/1089491253 conn(0x5638471367e0 msgr2=0x563847143320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f063effd640 0 client.27322 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 -- 172.21.15.69:0/1089491253 shutdown_connections 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 1 -- 172.21.15.69:0/1089491253 wait complete. 2024-08-02T17:57:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.365+0000 7f0668835740 20 client.27322 trim_cache size 0 max 16384 2024-08-02T17:57:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:57:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:57:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:57:00.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 -- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f064c0c5180 msgr2=0x7f064c0c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f064c0c5180 0x7f064c0c7640 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x563847137f30 tx=0x7f0650009960 comp rx=0 tx=0).stop 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 -- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06601006d0 msgr2=0x7f06601980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06601006d0 0x7f06601980d0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f065000be20 tx=0x7f0650009f70 comp rx=0 tx=0).stop 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 -- 172.21.15.69:0/932531445 shutdown_connections 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f064c0c5180 0x7f064c0c7640 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f06601066e0 0x7f066019c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0660101070 0x7f0660198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 --2- 172.21.15.69:0/932531445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f06601006d0 0x7f06601980d0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 -- 172.21.15.69:0/932531445 >> 172.21.15.69:0/932531445 conn(0x7f06600776d0 msgr2=0x7f06601046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 -- 172.21.15.69:0/932531445 shutdown_connections 2024-08-02T17:57:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:00.366+0000 7f0665ae3640 1 -- 172.21.15.69:0/932531445 wait complete. 2024-08-02T17:57:00.388 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '71426704', total seconds elapsed: 25.3 2024-08-02T17:57:00.388 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.9 2024-08-02T17:57:00.388 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:57:01.102 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:00 smithi069 ceph-mon[32746]: pgmap v328: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-08-02T17:57:01.219 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:00 smithi083 ceph-mon[37031]: pgmap v328: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-08-02T17:57:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:00 smithi077 ceph-mon[37108]: pgmap v328: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-08-02T17:57:01.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.829+0000 7f150ce5c640 1 -- 172.21.15.69:0/316561025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 msgr2=0x7f150806eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:01.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.829+0000 7f150ce5c640 1 --2- 172.21.15.69:0/316561025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 0x7f150806eba0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f1500009ca0 tx=0x7f150002a8f0 comp rx=0 tx=0).stop 2024-08-02T17:57:01.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.831+0000 7f150ce5c640 1 -- 172.21.15.69:0/316561025 shutdown_connections 2024-08-02T17:57:01.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.831+0000 7f150ce5c640 1 --2- 172.21.15.69:0/316561025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f150811ae70 0x7f150811d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:01.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.831+0000 7f150ce5c640 1 --2- 172.21.15.69:0/316561025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 0x7f150806eba0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:01.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.831+0000 7f150ce5c640 1 --2- 172.21.15.69:0/316561025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f150810fec0 0x7f150806e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:01.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.831+0000 7f150ce5c640 1 -- 172.21.15.69:0/316561025 >> 172.21.15.69:0/316561025 conn(0x7f1508072eb0 msgr2=0x7f15080732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:01.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.831+0000 7f150ce5c640 1 -- 172.21.15.69:0/316561025 shutdown_connections 2024-08-02T17:57:01.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.832+0000 7f150ce5c640 1 -- 172.21.15.69:0/316561025 wait complete. 2024-08-02T17:57:01.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.832+0000 7f150ce5c640 1 Processor -- start 2024-08-02T17:57:01.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.833+0000 7f150ce5c640 1 -- start start 2024-08-02T17:57:01.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.833+0000 7f150ce5c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 0x7f15081a6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:01.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.833+0000 7f150ce5c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f150810fec0 0x7f15081a7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:01.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.833+0000 7f150ce5c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f150811ae70 0x7f15081ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:01.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.833+0000 7f150ce5c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1508120a10 con 0x7f150810fec0 2024-08-02T17:57:01.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.833+0000 7f150ce5c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1508120890 con 0x7f150806e760 2024-08-02T17:57:01.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.834+0000 7f150ce5c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1508120b90 con 0x7f150811ae70 2024-08-02T17:57:01.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.834+0000 7f15077fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 0x7f15081a6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:01.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.835+0000 7f15077fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 0x7f15081a6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:36020/0 (socket says 172.21.15.69:36020) 2024-08-02T17:57:01.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.834+0000 7f1507fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f150811ae70 0x7f15081ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:01.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.835+0000 7f1507fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f150811ae70 0x7f15081ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53548/0 (socket says 172.21.15.69:53548) 2024-08-02T17:57:01.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.835+0000 7f15077fe640 1 -- 172.21.15.69:0/1575097107 learned_addr learned my addr 172.21.15.69:0/1575097107 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:01.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.836+0000 7f1506ffd640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f150810fec0 0x7f15081a7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:01.837 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.836+0000 7f1507fff640 1 -- 172.21.15.69:0/1575097107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 msgr2=0x7f15081a6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.836+0000 7f1507fff640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 0x7f15081a6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.836+0000 7f1507fff640 1 -- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f150810fec0 msgr2=0x7f15081a7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.836+0000 7f1507fff640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f150810fec0 0x7f15081a7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.836+0000 7f1507fff640 1 -- 172.21.15.69:0/1575097107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15081abf80 con 0x7f150811ae70 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.837+0000 7f1507fff640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f150811ae70 0x7f15081ab8a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f150806b1d0 tx=0x7f14f800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.837+0000 7f1504ff9640 1 -- 172.21.15.69:0/1575097107 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8015020 con 0x7f150811ae70 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.837+0000 7f150ce5c640 1 -- 172.21.15.69:0/1575097107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15081ac270 con 0x7f150811ae70 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.837+0000 7f150ce5c640 1 -- 172.21.15.69:0/1575097107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15081b3b50 con 0x7f150811ae70 2024-08-02T17:57:01.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.837+0000 7f1506ffd640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f150810fec0 0x7f15081a7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.838+0000 7f15077fe640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 0x7f15081a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.838+0000 7f1504ff9640 1 -- 172.21.15.69:0/1575097107 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14f8011d10 con 0x7f150811ae70 2024-08-02T17:57:01.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.838+0000 7f1504ff9640 1 -- 172.21.15.69:0/1575097107 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f8015020 con 0x7f150811ae70 2024-08-02T17:57:01.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.841+0000 7f14ee7fc640 1 -- 172.21.15.69:0/1575097107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14f4005290 con 0x7f150811ae70 2024-08-02T17:57:01.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.843+0000 7f1504ff9640 1 -- 172.21.15.69:0/1575097107 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f14f8018020 con 0x7f150811ae70 2024-08-02T17:57:01.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.844+0000 7f1504ff9640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f14f00a3130 0x7f14f00a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:01.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.845+0000 7f1504ff9640 1 -- 172.21.15.69:0/1575097107 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7f14f8011630 con 0x7f150811ae70 2024-08-02T17:57:01.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.849+0000 7f15077fe640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f14f00a3130 0x7f14f00a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:01.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.850+0000 7f1504ff9640 1 -- 172.21.15.69:0/1575097107 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f14f809b330 con 0x7f150811ae70 2024-08-02T17:57:01.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:01.851+0000 7f15077fe640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f14f00a3130 0x7f14f00a55f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009e00 tx=0x7f14fc009190 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:02.146 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:57:02.146 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:57:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.256+0000 7f14ee7fc640 1 -- 172.21.15.69:0/1575097107 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14f4005580 con 0x7f150811ae70 2024-08-02T17:57:02.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.257+0000 7f1504ff9640 1 -- 172.21.15.69:0/1575097107 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f14f809b330 con 0x7f150811ae70 2024-08-02T17:57:02.259 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:57:02.260 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:57:02.260 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:57:02.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.262+0000 7f14ee7fc640 1 -- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f14f00a3130 msgr2=0x7f14f00a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:02.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.262+0000 7f14ee7fc640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f14f00a3130 0x7f14f00a55f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009e00 tx=0x7f14fc009190 comp rx=0 tx=0).stop 2024-08-02T17:57:02.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.262+0000 7f14ee7fc640 1 -- 172.21.15.69:0/1575097107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f150811ae70 msgr2=0x7f15081ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:02.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.262+0000 7f14ee7fc640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f150811ae70 0x7f15081ab8a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f150806b1d0 tx=0x7f14f800b870 comp rx=0 tx=0).stop 2024-08-02T17:57:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.264+0000 7f14ee7fc640 1 -- 172.21.15.69:0/1575097107 shutdown_connections 2024-08-02T17:57:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.264+0000 7f14ee7fc640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f150810fec0 0x7f15081a7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.264+0000 7f14ee7fc640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f14f00a3130 0x7f14f00a55f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.264+0000 7f14ee7fc640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f150811ae70 0x7f15081ab8a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.264+0000 7f14ee7fc640 1 --2- 172.21.15.69:0/1575097107 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f150806e760 0x7f15081a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:02.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.264+0000 7f14ee7fc640 1 -- 172.21.15.69:0/1575097107 >> 172.21.15.69:0/1575097107 conn(0x7f1508072eb0 msgr2=0x7f1508119210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:02.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.266+0000 7f14ee7fc640 1 -- 172.21.15.69:0/1575097107 shutdown_connections 2024-08-02T17:57:02.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:02.266+0000 7f14ee7fc640 1 -- 172.21.15.69:0/1575097107 wait complete. 2024-08-02T17:57:03.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:02 smithi069 ceph-mon[32746]: pgmap v329: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-08-02T17:57:03.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:02 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1575097107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:03.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:02 smithi083 ceph-mon[37031]: pgmap v329: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-08-02T17:57:03.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:02 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1575097107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:02 smithi077 ceph-mon[37108]: pgmap v329: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-08-02T17:57:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:02 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1575097107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:03.868 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 139.9 --await-for 45.0' 2024-08-02T17:57:03.869 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 139.9 --await-for 45.0 2024-08-02T17:57:04.082 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:04.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.771+0000 7fbd29b4c640 1 -- 172.21.15.69:0/2774488917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd241007c0 msgr2=0x7fbd24100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.771+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/2774488917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd241007c0 0x7fbd24100c20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800dc20 tx=0x7fbd1802e320 comp rx=0 tx=0).stop 2024-08-02T17:57:04.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.773+0000 7fbd29b4c640 1 -- 172.21.15.69:0/2774488917 shutdown_connections 2024-08-02T17:57:04.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.773+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/2774488917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd241007c0 0x7fbd24100c20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.773+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/2774488917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd24101160 0x7fbd2410d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.773+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/2774488917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd241067c0 0x7fbd24106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.773+0000 7fbd29b4c640 1 -- 172.21.15.69:0/2774488917 >> 172.21.15.69:0/2774488917 conn(0x7fbd240fc500 msgr2=0x7fbd240fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:04.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.774+0000 7fbd29b4c640 1 -- 172.21.15.69:0/2774488917 shutdown_connections 2024-08-02T17:57:04.775 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.774+0000 7fbd29b4c640 1 -- 172.21.15.69:0/2774488917 wait complete. 2024-08-02T17:57:04.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.774+0000 7fbd29b4c640 1 Processor -- start 2024-08-02T17:57:04.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.775+0000 7fbd29b4c640 1 -- start start 2024-08-02T17:57:04.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.775+0000 7fbd29b4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd241007c0 0x7fbd24073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.775+0000 7fbd29b4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd24101160 0x7fbd24073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.775+0000 7fbd29b4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd241067c0 0x7fbd2406e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.776+0000 7fbd29b4c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2406d240 con 0x7fbd24101160 2024-08-02T17:57:04.777 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.776+0000 7fbd29b4c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2406d0c0 con 0x7fbd241067c0 2024-08-02T17:57:04.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.776+0000 7fbd29b4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2406d3c0 con 0x7fbd241007c0 2024-08-02T17:57:04.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.777+0000 7fbd22ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd24101160 0x7fbd24073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.777+0000 7fbd22ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd24101160 0x7fbd24073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59354/0 (socket says 172.21.15.69:59354) 2024-08-02T17:57:04.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.777+0000 7fbd22ffd640 1 -- 172.21.15.69:0/1599177943 learned_addr learned my addr 172.21.15.69:0/1599177943 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:04.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.777+0000 7fbd237fe640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd241007c0 0x7fbd24073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd23fff640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd241067c0 0x7fbd2406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd22ffd640 1 -- 172.21.15.69:0/1599177943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd241007c0 msgr2=0x7fbd24073690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd241007c0 0x7fbd24073690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd22ffd640 1 -- 172.21.15.69:0/1599177943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd241067c0 msgr2=0x7fbd2406e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd241067c0 0x7fbd2406e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd22ffd640 1 -- 172.21.15.69:0/1599177943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2406f230 con 0x7fbd24101160 2024-08-02T17:57:04.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd24101160 0x7fbd24073bd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802e890 tx=0x7fbd18036800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:04.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd20ff9640 1 -- 172.21.15.69:0/1599177943 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18004250 con 0x7fbd24101160 2024-08-02T17:57:04.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.778+0000 7fbd20ff9640 1 -- 172.21.15.69:0/1599177943 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd180043f0 con 0x7fbd24101160 2024-08-02T17:57:04.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.779+0000 7fbd29b4c640 1 -- 172.21.15.69:0/1599177943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2406f4c0 con 0x7fbd24101160 2024-08-02T17:57:04.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.779+0000 7fbd29b4c640 1 -- 172.21.15.69:0/1599177943 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd241ab1b0 con 0x7fbd24101160 2024-08-02T17:57:04.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.779+0000 7fbd20ff9640 1 -- 172.21.15.69:0/1599177943 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18004250 con 0x7fbd24101160 2024-08-02T17:57:04.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.783+0000 7fbd20ff9640 1 -- 172.21.15.69:0/1599177943 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbd18045060 con 0x7fbd24101160 2024-08-02T17:57:04.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.784+0000 7fbd20ff9640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd0c0a2f30 0x7fbd0c0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.784+0000 7fbd20ff9640 1 -- 172.21.15.69:0/1599177943 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7fbd180eec40 con 0x7fbd24101160 2024-08-02T17:57:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.786+0000 7fbd237fe640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd0c0a2f30 0x7fbd0c0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.787+0000 7fbd2bf4a740 1 Processor -- start 2024-08-02T17:57:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd2bf4a740 1 -- start start 2024-08-02T17:57:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b4f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b4fb20 0x55fe87b59ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fe87b12e40 con 0x55fe87b4fb20 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe87ab47f0 con 0x55fe87b5a3e0 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe87a5bfb0 con 0x55fe87b4f200 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd22ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b4fb20 0x55fe87b59ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b4f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd237fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b4f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50174/0 (socket says 172.21.15.69:50174) 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd237fe640 1 -- 172.21.15.69:0/57784827 learned_addr learned my addr 172.21.15.69:0/57784827 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.788+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.789+0000 7fbd237fe640 1 -- 172.21.15.69:0/57784827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b5a3e0 msgr2=0x55fe87b5c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.789+0000 7fbd237fe640 1 --2- 172.21.15.69:0/57784827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.789+0000 7fbd237fe640 1 -- 172.21.15.69:0/57784827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b4fb20 msgr2=0x55fe87b59ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.789+0000 7fbd237fe640 1 --2- 172.21.15.69:0/57784827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b4fb20 0x55fe87b59ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.789+0000 7fbd237fe640 1 -- 172.21.15.69:0/57784827 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe878266a0 con 0x55fe87b4f200 2024-08-02T17:57:04.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.791+0000 7fbd237fe640 1 --2- 172.21.15.69:0/57784827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b4f5e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbd14022760 tx=0x7fbd14022c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bec38dcd3619f6c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:04.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.795+0000 7fbd09ffb640 1 -- 172.21.15.69:0/57784827 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd14022d50 con 0x55fe87b4f200 2024-08-02T17:57:04.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.795+0000 7fbd09ffb640 1 -- 172.21.15.69:0/57784827 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1401edb0 con 0x55fe87b4f200 2024-08-02T17:57:04.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.795+0000 7fbd09ffb640 1 -- 172.21.15.69:0/57784827 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400e3e0 con 0x55fe87b4f200 2024-08-02T17:57:04.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.796+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/57784827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 msgr2=0x55fe87b4f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.796+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/57784827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b4f5e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbd14022760 tx=0x7fbd14022c20 comp rx=0 tx=0).stop 2024-08-02T17:57:04.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.797+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/57784827 shutdown_connections 2024-08-02T17:57:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.797+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/57784827 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b4fb20 0x55fe87b59ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.797+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/57784827 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.797+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/57784827 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b4f5e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.797+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/57784827 >> 172.21.15.69:0/57784827 conn(0x55fe87b4b790 msgr2=0x55fe87b4bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.797+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/57784827 shutdown_connections 2024-08-02T17:57:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.797+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/57784827 wait complete. 2024-08-02T17:57:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 1 -- start start 2024-08-02T17:57:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 1 Processor -- start 2024-08-02T17:57:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b57340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b57880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b57dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fe87a6fcb0 con 0x55fe87b5a3e0 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe87ab07b0 con 0x55fe87b4fb20 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe87b5cfb0 con 0x55fe87b4f200 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.798+0000 7fbd2bf4a740 10 client.0 _ll_register_callbacks cb 0x55fe877c3ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b57dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b57dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59374/0 (socket says 172.21.15.69:59374) 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd23fff640 1 -- 172.21.15.69:0/145414573 learned_addr learned my addr 172.21.15.69:0/145414573 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b57880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd22ffd640 1 -- 172.21.15.69:0/145414573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 msgr2=0x55fe87b57340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b57340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd22ffd640 1 -- 172.21.15.69:0/145414573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 msgr2=0x55fe87b57dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b57dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.799+0000 7fbd22ffd640 1 -- 172.21.15.69:0/145414573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe87a6ebc0 con 0x55fe87b4fb20 2024-08-02T17:57:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.800+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b57880 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbd180ec060 tx=0x7fbd180baad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f80c285bec497c4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:04.806 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.805+0000 7fbd02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.806+0000 7fbd02ffd640 1 -- 172.21.15.69:0/145414573 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd180bb2c0 con 0x55fe87b4fb20 2024-08-02T17:57:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.806+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/145414573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd1401edb0 con 0x55fe87b4fb20 2024-08-02T17:57:04.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.806+0000 7fbd02ffd640 1 -- 172.21.15.69:0/145414573 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7fbd180c3640 con 0x55fe87b4fb20 2024-08-02T17:57:04.815 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.813+0000 7fbd02ffd640 1 -- 172.21.15.69:0/145414573 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd180c3ad0 con 0x55fe87b4fb20 2024-08-02T17:57:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.814+0000 7fbd2bf4a740 10 client.27497 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.814+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/145414573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe87b8af00 con 0x55fe87b4fb20 2024-08-02T17:57:04.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.816+0000 7fbd237fe640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd0c0a2f30 0x7fbd0c0a53f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbd14011040 tx=0x7fbd1401ab70 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.818+0000 7fbd02ffd640 1 -- 172.21.15.69:0/145414573 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1401edb0 con 0x55fe87b4fb20 2024-08-02T17:57:04.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.818+0000 7fbd02ffd640 1 -- 172.21.15.69:0/145414573 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbd180b9d20 con 0x55fe87b4fb20 2024-08-02T17:57:04.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.819+0000 7fbd2bf4a740 10 client.27497 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:04.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.819+0000 7fbd2bf4a740 10 client.27497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:04.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.819+0000 7fbd2bf4a740 20 client.27497 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:04.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.819+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe87b8c310 0x55fe87b8e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.819+0000 7fbd2bf4a740 4 client.27497 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:04.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.819+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/145414573 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe87ac1d60 con 0x55fe87b8c310 2024-08-02T17:57:04.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.823+0000 7fbd237fe640 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe87b8c310 0x55fe87b8e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.837+0000 7fbd237fe640 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe87b8c310 0x55fe87b8e730 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f2e01db4e68a14 server_cookie=42cf7c877cb4a0cc in_seq=0 out_seq=0 2024-08-02T17:57:04.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.837+0000 7fbd02ffd640 10 client.27497 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.839+0000 7fbd02ffd640 1 -- 172.21.15.69:0/145414573 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe87ac1d60 con 0x55fe87b8c310 2024-08-02T17:57:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.839+0000 7fbd02ffd640 10 client.27497 handle_command_reply: tid=0 2024-08-02T17:57:04.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.840+0000 7fbd2bf4a740 1 client.27497 shutdown 2024-08-02T17:57:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.840+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 msgr2=0x55fe87b57880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.840+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b57880 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbd180ec060 tx=0x7fbd180baad0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.847+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/145414573 shutdown_connections 2024-08-02T17:57:04.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.847+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b57dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.847+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe87b8c310 0x55fe87b8e730 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.847+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b57880 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.847+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/145414573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b57340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.847+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/145414573 >> 172.21.15.69:0/145414573 conn(0x55fe87b4b790 msgr2=0x55fe87b4a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:04.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.847+0000 7fbd02ffd640 0 client.27497 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:04.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.849+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/145414573 shutdown_connections 2024-08-02T17:57:04.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.849+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/145414573 wait complete. 2024-08-02T17:57:04.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.849+0000 7fbd2bf4a740 20 client.27497 trim_cache size 0 max 16384 2024-08-02T17:57:04.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.886+0000 7fbd2bf4a740 1 Processor -- start 2024-08-02T17:57:04.888 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.886+0000 7fbd2bf4a740 1 -- start start 2024-08-02T17:57:04.891 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.887+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4f200 0x55fe87b56340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.887+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4fb20 0x55fe87b57c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.887+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b94000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.887+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2411ac60 con 0x55fe87b5a3e0 2024-08-02T17:57:04.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.887+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24118570 con 0x55fe87b4f200 2024-08-02T17:57:04.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.887+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2406fca0 con 0x55fe87b4fb20 2024-08-02T17:57:04.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.888+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b94000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.888+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b94000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59376/0 (socket says 172.21.15.69:59376) 2024-08-02T17:57:04.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.888+0000 7fbd23fff640 1 -- 172.21.15.69:0/3184869185 learned_addr learned my addr 172.21.15.69:0/3184869185 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.890+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4fb20 0x55fe87b57c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.890+0000 7fbd237fe640 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4f200 0x55fe87b56340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.890+0000 7fbd23fff640 1 -- 172.21.15.69:0/3184869185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4fb20 msgr2=0x55fe87b57c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.890+0000 7fbd23fff640 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4fb20 0x55fe87b57c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.890+0000 7fbd23fff640 1 -- 172.21.15.69:0/3184869185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4f200 msgr2=0x55fe87b56340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.890+0000 7fbd23fff640 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4f200 0x55fe87b56340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.890+0000 7fbd23fff640 1 -- 172.21.15.69:0/3184869185 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe87b8edb0 con 0x55fe87b5a3e0 2024-08-02T17:57:04.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.890+0000 7fbd23fff640 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b94000 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00bf30 tx=0x7fbd1c009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b7097704c0fb60 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:04.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.891+0000 7fbd097fa640 1 -- 172.21.15.69:0/3184869185 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c01c020 con 0x55fe87b5a3e0 2024-08-02T17:57:04.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.892+0000 7fbd097fa640 1 -- 172.21.15.69:0/3184869185 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd1c009e80 con 0x55fe87b5a3e0 2024-08-02T17:57:04.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.892+0000 7fbd097fa640 1 -- 172.21.15.69:0/3184869185 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c017db0 con 0x55fe87b5a3e0 2024-08-02T17:57:04.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.893+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/3184869185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 msgr2=0x55fe87b94000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.893+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b94000 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c00bf30 tx=0x7fbd1c009730 comp rx=0 tx=0).stop 2024-08-02T17:57:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.895+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/3184869185 shutdown_connections 2024-08-02T17:57:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.895+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b94000 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.895+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4fb20 0x55fe87b57c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.895+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/3184869185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4f200 0x55fe87b56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.895+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/3184869185 >> 172.21.15.69:0/3184869185 conn(0x55fe87b4b790 msgr2=0x55fe87b55c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:04.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.897+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/3184869185 shutdown_connections 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.901+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/3184869185 wait complete. 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.901+0000 7fbd2bf4a740 1 -- start start 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.901+0000 7fbd2bf4a740 1 Processor -- start 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.902+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b62200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.902+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b62740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.902+0000 7fbd2bf4a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.902+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24116a30 con 0x55fe87b5a3e0 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.902+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24117ea0 con 0x55fe87b4fb20 2024-08-02T17:57:04.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.902+0000 7fbd2bf4a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24117100 con 0x55fe87b4f200 2024-08-02T17:57:04.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.902+0000 7fbd2bf4a740 10 client.0 _ll_register_callbacks cb 0x7fbd180db050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:04.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.903+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.903+0000 7fbd23fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59390/0 (socket says 172.21.15.69:59390) 2024-08-02T17:57:04.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.903+0000 7fbd23fff640 1 -- 172.21.15.69:0/4159961061 learned_addr learned my addr 172.21.15.69:0/4159961061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:04.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.904+0000 7fbd22ffd640 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b62740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.905+0000 7fbd237fe640 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b62200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.907+0000 7fbd23fff640 1 -- 172.21.15.69:0/4159961061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 msgr2=0x55fe87b62200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.907+0000 7fbd23fff640 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b62200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.907+0000 7fbd23fff640 1 -- 172.21.15.69:0/4159961061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 msgr2=0x55fe87b62740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.907+0000 7fbd23fff640 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b62740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.907+0000 7fbd23fff640 1 -- 172.21.15.69:0/4159961061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe87b62e00 con 0x55fe87b5a3e0 2024-08-02T17:57:04.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.910+0000 7fbd23fff640 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5f1d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c017930 tx=0x7fbd1c017750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ef81bcf29fdfcb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:04.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.917+0000 7fbd02ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:04.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.917+0000 7fbd02ffd640 1 -- 172.21.15.69:0/4159961061 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c01c030 con 0x55fe87b5a3e0 2024-08-02T17:57:04.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.917+0000 7fbd02ffd640 1 -- 172.21.15.69:0/4159961061 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9568+0+0 (secure 0 0 0) 0x7fbd1c017e40 con 0x55fe87b5a3e0 2024-08-02T17:57:04.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.918+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/4159961061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fe87abc270 con 0x55fe87b5a3e0 2024-08-02T17:57:04.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.920+0000 7fbd02ffd640 1 -- 172.21.15.69:0/4159961061 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbd1c031050 con 0x55fe87b5a3e0 2024-08-02T17:57:04.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.922+0000 7fbd2bf4a740 10 client.18021 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:04.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.922+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/4159961061 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe87bf75e0 con 0x55fe87b5a3e0 2024-08-02T17:57:04.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.933+0000 7fbd02ffd640 1 -- 172.21.15.69:0/4159961061 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fe87abc270 con 0x55fe87b5a3e0 2024-08-02T17:57:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.934+0000 7fbd02ffd640 1 -- 172.21.15.69:0/4159961061 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbd1c034020 con 0x55fe87b5a3e0 2024-08-02T17:57:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.936+0000 7fbd2bf4a740 10 client.18021 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.936+0000 7fbd2bf4a740 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.936+0000 7fbd2bf4a740 20 client.18021 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.936+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe87bf7bf0 0x55fe87bfa010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.936+0000 7fbd2bf4a740 4 client.18021 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:57:04.938 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.936+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/4159961061 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fe87a39110 con 0x55fe87bf7bf0 2024-08-02T17:57:04.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.940+0000 7fbd237fe640 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe87bf7bf0 0x55fe87bfa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:04.948 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.943+0000 7fbd237fe640 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe87bf7bf0 0x55fe87bfa010 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63930e67090de3c server_cookie=9252eecc3c2e0e40 in_seq=0 out_seq=0 2024-08-02T17:57:04.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.946+0000 7fbd02ffd640 10 client.18021 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:04.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.948+0000 7fbd02ffd640 1 -- 172.21.15.69:0/4159961061 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fe87a39110 con 0x55fe87bf7bf0 2024-08-02T17:57:04.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.948+0000 7fbd02ffd640 10 client.18021 handle_command_reply: tid=0 2024-08-02T17:57:04.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.949+0000 7fbd2bf4a740 1 client.18021 shutdown 2024-08-02T17:57:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 595.0 s, 588.2 tps, lat 199.409 ms stddev 604.234 2024-08-02T17:57:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.951+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/4159961061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 msgr2=0x55fe87b5f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.951+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5f1d0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c017930 tx=0x7fbd1c017750 comp rx=0 tx=0).stop 2024-08-02T17:57:04.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.957+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/4159961061 shutdown_connections 2024-08-02T17:57:04.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.957+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe87b5a3e0 0x55fe87b5f1d0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.957+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe87bf7bf0 0x55fe87bfa010 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.957+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe87b4fb20 0x55fe87b62740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.957+0000 7fbd2bf4a740 1 --2- 172.21.15.69:0/4159961061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe87b4f200 0x55fe87b62200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.957+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/4159961061 >> 172.21.15.69:0/4159961061 conn(0x55fe87b4b790 msgr2=0x55fe87b56450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:04.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.957+0000 7fbd02ffd640 0 client.18021 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:04.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.958+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/4159961061 shutdown_connections 2024-08-02T17:57:04.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.958+0000 7fbd2bf4a740 1 -- 172.21.15.69:0/4159961061 wait complete. 2024-08-02T17:57:04.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.959+0000 7fbd2bf4a740 20 client.18021 trim_cache size 0 max 16384 2024-08-02T17:57:04.960 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:57:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:57:04.960 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "212f758b-93aa-49f2-a332-c49556ac0dbf", 2024-08-02T17:57:04.961 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:57:04.961 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:57:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 -- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd0c0a2f30 msgr2=0x7fbd0c0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd0c0a2f30 0x7fbd0c0a53f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbd14011040 tx=0x7fbd1401ab70 comp rx=0 tx=0).stop 2024-08-02T17:57:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 -- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd24101160 msgr2=0x7fbd24073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:04.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd24101160 0x7fbd24073bd0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802e890 tx=0x7fbd18036800 comp rx=0 tx=0).stop 2024-08-02T17:57:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 -- 172.21.15.69:0/1599177943 shutdown_connections 2024-08-02T17:57:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbd24101160 0x7fbd24073bd0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbd0c0a2f30 0x7fbd0c0a53f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbd241067c0 0x7fbd2406e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 --2- 172.21.15.69:0/1599177943 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbd241007c0 0x7fbd24073690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 -- 172.21.15.69:0/1599177943 >> 172.21.15.69:0/1599177943 conn(0x7fbd240fc500 msgr2=0x7fbd240fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 -- 172.21.15.69:0/1599177943 shutdown_connections 2024-08-02T17:57:04.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:04.960+0000 7fbd29b4c640 1 -- 172.21.15.69:0/1599177943 wait complete. 2024-08-02T17:57:05.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:04 smithi069 ceph-mon[32746]: pgmap v330: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-08-02T17:57:05.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:04 smithi083 ceph-mon[37031]: pgmap v330: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-08-02T17:57:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:04 smithi077 ceph-mon[37108]: pgmap v330: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-08-02T17:57:05.740 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:57:06.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.174+0000 7f48526f3640 1 -- 172.21.15.69:0/3012843985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f484c1006b0 msgr2=0x7f484c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.174+0000 7f48526f3640 1 --2- 172.21.15.69:0/3012843985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f484c1006b0 0x7f484c100a90 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4838009860 tx=0x7f483802a3c0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.174+0000 7f48526f3640 1 -- 172.21.15.69:0/3012843985 shutdown_connections 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.174+0000 7f48526f3640 1 --2- 172.21.15.69:0/3012843985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f484c105ac0 0x7f484c105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.174+0000 7f48526f3640 1 --2- 172.21.15.69:0/3012843985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f484c100fd0 0x7f484c1054a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.174+0000 7f48526f3640 1 --2- 172.21.15.69:0/3012843985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f484c1006b0 0x7f484c100a90 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.174+0000 7f48526f3640 1 -- 172.21.15.69:0/3012843985 >> 172.21.15.69:0/3012843985 conn(0x7f484c0fc340 msgr2=0x7f484c0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.175+0000 7f48526f3640 1 -- 172.21.15.69:0/3012843985 shutdown_connections 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.175+0000 7f48526f3640 1 -- 172.21.15.69:0/3012843985 wait complete. 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f48526f3640 1 Processor -- start 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f48526f3640 1 -- start start 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f48526f3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f484c1006b0 0x7f484c073630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f48526f3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f484c100fd0 0x7f484c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f48526f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f484c105ac0 0x7f484c071950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f48526f3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f484c1074c0 con 0x7f484c100fd0 2024-08-02T17:57:06.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f48526f3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f484c107340 con 0x7f484c1006b0 2024-08-02T17:57:06.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f48526f3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f484c107640 con 0x7f484c105ac0 2024-08-02T17:57:06.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.176+0000 7f4851ef2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f484c105ac0 0x7f484c071950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f4851ef2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f484c105ac0 0x7f484c071950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50222/0 (socket says 172.21.15.69:50222) 2024-08-02T17:57:06.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f4851ef2640 1 -- 172.21.15.69:0/3168957711 learned_addr learned my addr 172.21.15.69:0/3168957711 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:06.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f48516f1640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f484c1006b0 0x7f484c073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f4850ef0640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f484c100fd0 0x7f484c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f48516f1640 1 -- 172.21.15.69:0/3168957711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f484c105ac0 msgr2=0x7f484c071950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f48516f1640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f484c105ac0 0x7f484c071950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f48516f1640 1 -- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f484c100fd0 msgr2=0x7f484c073b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f48516f1640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f484c100fd0 0x7f484c073b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.177+0000 7f48516f1640 1 -- 172.21.15.69:0/3168957711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c072210 con 0x7f484c1006b0 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.178+0000 7f48516f1640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f484c1006b0 0x7f484c073630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4838035040 tx=0x7f4838004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.178+0000 7f484a7fc640 1 -- 172.21.15.69:0/3168957711 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483803f020 con 0x7f484c1006b0 2024-08-02T17:57:06.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.178+0000 7f48526f3640 1 -- 172.21.15.69:0/3168957711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484c0724a0 con 0x7f484c1006b0 2024-08-02T17:57:06.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.178+0000 7f48526f3640 1 -- 172.21.15.69:0/3168957711 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484c112fb0 con 0x7f484c1006b0 2024-08-02T17:57:06.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.178+0000 7f484a7fc640 1 -- 172.21.15.69:0/3168957711 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4838003ad0 con 0x7f484c1006b0 2024-08-02T17:57:06.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.178+0000 7f484a7fc640 1 -- 172.21.15.69:0/3168957711 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838037040 con 0x7f484c1006b0 2024-08-02T17:57:06.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.184+0000 7f484a7fc640 1 -- 172.21.15.69:0/3168957711 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4838042020 con 0x7f484c1006b0 2024-08-02T17:57:06.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.186+0000 7f4855d53740 1 Processor -- start 2024-08-02T17:57:06.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.187+0000 7f4855d53740 1 -- start start 2024-08-02T17:57:06.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.187+0000 7f4855d53740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.187+0000 7f4855d53740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7bc00 0x55a924a85f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.187+0000 7f4855d53740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a864c0 0x55a924a888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.187+0000 7f4855d53740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a9249e0a70 con 0x55a924a7b2e0 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.187+0000 7f4855d53740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a924978780 con 0x55a924a864c0 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.187+0000 7f4855d53740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a9249dca30 con 0x55a924a7bc00 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.187+0000 7f4851ef2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4851ef2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59418/0 (socket says 172.21.15.69:59418) 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4851ef2640 1 -- 172.21.15.69:0/3374427277 learned_addr learned my addr 172.21.15.69:0/3374427277 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4850ef0640 1 --2- 172.21.15.69:0/3374427277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a864c0 0x55a924a888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4850ef0640 1 -- 172.21.15.69:0/3374427277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7bc00 msgr2=0x55a924a85f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4850ef0640 1 --2- 172.21.15.69:0/3374427277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7bc00 0x55a924a85f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4850ef0640 1 -- 172.21.15.69:0/3374427277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 msgr2=0x55a924a7b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4850ef0640 1 --2- 172.21.15.69:0/3374427277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4850ef0640 1 -- 172.21.15.69:0/3374427277 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a924752b90 con 0x55a924a864c0 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f4850ef0640 1 --2- 172.21.15.69:0/3374427277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a864c0 0x55a924a888b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f484c0ff9a0 tx=0x7f48400107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce4b6957b8b79de3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:06.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.188+0000 7f48377fe640 1 -- 172.21.15.69:0/3374427277 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484000c810 con 0x55a924a864c0 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.189+0000 7f48377fe640 1 -- 172.21.15.69:0/3374427277 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48400183e0 con 0x55a924a864c0 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.189+0000 7f48377fe640 1 -- 172.21.15.69:0/3374427277 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484000e640 con 0x55a924a864c0 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f4855d53740 1 -- 172.21.15.69:0/3374427277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a864c0 msgr2=0x55a924a888b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f4855d53740 1 --2- 172.21.15.69:0/3374427277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a864c0 0x55a924a888b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f484c0ff9a0 tx=0x7f48400107e0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f4855d53740 1 -- 172.21.15.69:0/3374427277 shutdown_connections 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f4855d53740 1 --2- 172.21.15.69:0/3374427277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a864c0 0x55a924a888b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f4855d53740 1 --2- 172.21.15.69:0/3374427277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7bc00 0x55a924a85f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f4855d53740 1 --2- 172.21.15.69:0/3374427277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f4855d53740 1 -- 172.21.15.69:0/3374427277 >> 172.21.15.69:0/3374427277 conn(0x55a924a77870 msgr2=0x55a924a77c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:06.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f484a7fc640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f483c0a2e40 0x7f483c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.190+0000 7f4850ef0640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f483c0a2e40 0x7f483c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.191+0000 7f484a7fc640 1 -- 172.21.15.69:0/3168957711 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f48380f0340 con 0x7f484c1006b0 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.191+0000 7f4850ef0640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f483c0a2e40 0x7f483c0a5300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4840009c70 tx=0x7f4840018dd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.191+0000 7f4855d53740 1 -- 172.21.15.69:0/3374427277 shutdown_connections 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.192+0000 7f4855d53740 1 -- 172.21.15.69:0/3374427277 wait complete. 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.192+0000 7f4855d53740 1 -- start start 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.192+0000 7f4855d53740 1 Processor -- start 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4855d53740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a83390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4855d53740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4855d53740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a864c0 0x55a924a83e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4855d53740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a924a89090 con 0x55a924a864c0 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4855d53740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a92499b990 con 0x55a924a7bc00 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4855d53740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a924a89210 con 0x55a924a7b2e0 2024-08-02T17:57:06.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4855d53740 10 client.0 _ll_register_callbacks cb 0x55a9246f0090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:06.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4850ef0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4850ef0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a838d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:51752/0 (socket says 172.21.15.69:51752) 2024-08-02T17:57:06.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.193+0000 7f4850ef0640 1 -- 172.21.15.69:0/1325549310 learned_addr learned my addr 172.21.15.69:0/1325549310 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:06.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4850ef0640 1 -- 172.21.15.69:0/1325549310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 msgr2=0x55a924a83390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4850ef0640 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a83390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4850ef0640 1 -- 172.21.15.69:0/1325549310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a864c0 msgr2=0x55a924a83e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4850ef0640 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a864c0 0x55a924a83e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4850ef0640 1 -- 172.21.15.69:0/1325549310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a92499a6c0 con 0x55a924a7bc00 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4850ef0640 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a838d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4840019000 tx=0x7f48400107e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f9a0713743e482 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4834ff9640 1 -- 172.21.15.69:0/1325549310 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484002a050 con 0x55a924a7bc00 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4834ff9640 1 -- 172.21.15.69:0/1325549310 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f484000c810 con 0x55a924a7bc00 2024-08-02T17:57:06.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.194+0000 7f4855d53740 1 -- 172.21.15.69:0/1325549310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48400183e0 con 0x55a924a7bc00 2024-08-02T17:57:06.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.195+0000 7f4834ff9640 1 -- 172.21.15.69:0/1325549310 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f484000ca10 con 0x55a924a7bc00 2024-08-02T17:57:06.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.195+0000 7f4855d53740 10 client.27509 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:06.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.195+0000 7f4855d53740 1 -- 172.21.15.69:0/1325549310 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a924a94de0 con 0x55a924a7bc00 2024-08-02T17:57:06.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.195+0000 7f4834ff9640 1 -- 172.21.15.69:0/1325549310 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48400183e0 con 0x55a924a7bc00 2024-08-02T17:57:06.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.196+0000 7f4834ff9640 1 -- 172.21.15.69:0/1325549310 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4840017070 con 0x55a924a7bc00 2024-08-02T17:57:06.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.196+0000 7f4855d53740 10 client.27509 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:06.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.196+0000 7f4855d53740 10 client.27509 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:57:06.198 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.197+0000 7f4855d53740 20 client.27509 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:06.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.197+0000 7f4855d53740 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a924a96240 0x55a924a98630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.197+0000 7f4855d53740 4 client.27509 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:06.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.197+0000 7f4855d53740 1 -- 172.21.15.69:0/1325549310 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9249ee1a0 con 0x55a924a96240 2024-08-02T17:57:06.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.197+0000 7f48516f1640 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a924a96240 0x55a924a98630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.199+0000 7f48516f1640 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a924a96240 0x55a924a98630 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3245d81ec5c90cd4 server_cookie=9a98da00e7bff169 in_seq=0 out_seq=0 2024-08-02T17:57:06.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.199+0000 7f4834ff9640 10 client.27509 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:06.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.199+0000 7f4834ff9640 1 -- 172.21.15.69:0/1325549310 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a9249ee1a0 con 0x55a924a96240 2024-08-02T17:57:06.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.199+0000 7f4834ff9640 10 client.27509 handle_command_reply: tid=0 2024-08-02T17:57:06.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.199+0000 7f4855d53740 1 client.27509 shutdown 2024-08-02T17:57:06.201 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4855d53740 1 -- 172.21.15.69:0/1325549310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 msgr2=0x55a924a838d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4855d53740 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a838d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4840019000 tx=0x7f48400107e0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4855d53740 1 -- 172.21.15.69:0/1325549310 shutdown_connections 2024-08-02T17:57:06.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4855d53740 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a864c0 0x55a924a83e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4855d53740 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a924a96240 0x55a924a98630 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4855d53740 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a838d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4855d53740 1 --2- 172.21.15.69:0/1325549310 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a83390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4855d53740 1 -- 172.21.15.69:0/1325549310 >> 172.21.15.69:0/1325549310 conn(0x55a924a77870 msgr2=0x55a924a769d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:06.202 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.200+0000 7f4834ff9640 0 client.27509 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:06.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.201+0000 7f4855d53740 1 -- 172.21.15.69:0/1325549310 shutdown_connections 2024-08-02T17:57:06.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.201+0000 7f4855d53740 1 -- 172.21.15.69:0/1325549310 wait complete. 2024-08-02T17:57:06.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.201+0000 7f4855d53740 20 client.27509 trim_cache size 0 max 16384 2024-08-02T17:57:06.231 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.230+0000 7f4855d53740 1 Processor -- start 2024-08-02T17:57:06.232 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.230+0000 7f4855d53740 1 -- start start 2024-08-02T17:57:06.232 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:05 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:57:06.232 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:05 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:06.232 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:05 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:57:06.232 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:05 smithi069 ceph-mon[32746]: osdmap e104: 12 total, 12 up, 12 in 2024-08-02T17:57:06.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.230+0000 7f4855d53740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.230+0000 7f4855d53740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a9dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.230+0000 7f4855d53740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a864c0 0x55a924a9e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4855d53740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4830012910 con 0x55a924a7b2e0 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4855d53740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48300101e0 con 0x55a924a7bc00 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f48516f1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4855d53740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f483000e0e0 con 0x55a924a864c0 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4850ef0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a9dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4850ef0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a9dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:51760/0 (socket says 172.21.15.69:51760) 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4850ef0640 1 -- 172.21.15.69:0/2054590673 learned_addr learned my addr 172.21.15.69:0/2054590673 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4851ef2640 1 --2- 172.21.15.69:0/2054590673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a864c0 0x55a924a9e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4850ef0640 1 -- 172.21.15.69:0/2054590673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a864c0 msgr2=0x55a924a9e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4850ef0640 1 --2- 172.21.15.69:0/2054590673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a864c0 0x55a924a9e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4850ef0640 1 -- 172.21.15.69:0/2054590673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 msgr2=0x55a924a7d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4850ef0640 1 --2- 172.21.15.69:0/2054590673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.234 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.231+0000 7f4850ef0640 1 -- 172.21.15.69:0/2054590673 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a924a82360 con 0x55a924a7bc00 2024-08-02T17:57:06.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.232+0000 7f4850ef0640 1 --2- 172.21.15.69:0/2054590673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a9dec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4840005900 tx=0x7f4840009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=516708f6c160567f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:06.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.233+0000 7f4836ffd640 1 -- 172.21.15.69:0/2054590673 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484002a040 con 0x55a924a7bc00 2024-08-02T17:57:06.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.234+0000 7f4836ffd640 1 -- 172.21.15.69:0/2054590673 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4840034070 con 0x55a924a7bc00 2024-08-02T17:57:06.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.234+0000 7f4836ffd640 1 -- 172.21.15.69:0/2054590673 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484000cbb0 con 0x55a924a7bc00 2024-08-02T17:57:06.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.237+0000 7f4855d53740 1 -- 172.21.15.69:0/2054590673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 msgr2=0x55a924a9dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.237+0000 7f4855d53740 1 --2- 172.21.15.69:0/2054590673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a9dec0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4840005900 tx=0x7f4840009fd0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.237+0000 7f4855d53740 1 -- 172.21.15.69:0/2054590673 shutdown_connections 2024-08-02T17:57:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.238+0000 7f4855d53740 1 --2- 172.21.15.69:0/2054590673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a864c0 0x55a924a9e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.238+0000 7f4855d53740 1 --2- 172.21.15.69:0/2054590673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a9dec0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.238+0000 7f4855d53740 1 --2- 172.21.15.69:0/2054590673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a7b2e0 0x55a924a7d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.238+0000 7f4855d53740 1 -- 172.21.15.69:0/2054590673 >> 172.21.15.69:0/2054590673 conn(0x55a924a77870 msgr2=0x55a924a7cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.238+0000 7f4855d53740 1 -- 172.21.15.69:0/2054590673 shutdown_connections 2024-08-02T17:57:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.238+0000 7f4855d53740 1 -- 172.21.15.69:0/2054590673 wait complete. 2024-08-02T17:57:06.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.238+0000 7f4855d53740 1 -- start start 2024-08-02T17:57:06.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.239+0000 7f4855d53740 1 Processor -- start 2024-08-02T17:57:06.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.239+0000 7f4855d53740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a8ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f4855d53740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a800e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f48516f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a8ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f48516f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a8ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50244/0 (socket says 172.21.15.69:50244) 2024-08-02T17:57:06.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f4850ef0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a800e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f4850ef0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a800e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:51768/0 (socket says 172.21.15.69:51768) 2024-08-02T17:57:06.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f48516f1640 1 -- 172.21.15.69:0/873768290 learned_addr learned my addr 172.21.15.69:0/873768290 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:06.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f4855d53740 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a864c0 0x55a924a80620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f483000e6a0 con 0x55a924a864c0 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f483000fb10 con 0x55a924a7bc00 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.240+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f483000ed70 con 0x55a924a7b2e0 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4850ef0640 1 -- 172.21.15.69:0/873768290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 msgr2=0x55a924a8ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4850ef0640 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a8ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4850ef0640 1 -- 172.21.15.69:0/873768290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a864c0 msgr2=0x55a924a80620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4850ef0640 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a864c0 0x55a924a80620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4850ef0640 1 -- 172.21.15.69:0/873768290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a924a80e50 con 0x55a924a7bc00 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f48516f1640 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a8ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4850ef0640 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a800e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4840015040 tx=0x7f4840005110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cbf274d20650ef server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4834ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4855d53740 10 client.0 _ll_register_callbacks cb 0x7f4840009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.241+0000 7f4834ff9640 1 -- 172.21.15.69:0/873768290 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48400183e0 con 0x55a924a7bc00 2024-08-02T17:57:06.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.242+0000 7f4834ff9640 1 -- 172.21.15.69:0/873768290 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f4840023730 con 0x55a924a7bc00 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.242+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4840034070 con 0x55a924a7bc00 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.242+0000 7f4834ff9640 1 -- 172.21.15.69:0/873768290 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f484000ee80 con 0x55a924a7bc00 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.242+0000 7f4855d53740 10 client.27515 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.242+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a924b00c80 con 0x55a924a7bc00 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.243+0000 7f4834ff9640 1 -- 172.21.15.69:0/873768290 <== mon.2 v2:172.21.15.77:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4840017070 con 0x55a924a7bc00 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.243+0000 7f4855d53740 10 client.27515 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.243+0000 7f4855d53740 10 client.27515 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.243+0000 7f4855d53740 20 client.27515 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.243+0000 7f4855d53740 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a924b01170 0x55a924b035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.243+0000 7f4855d53740 4 client.27515 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.9, "await_for": 45.0}] 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.243+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.9, "await_for": 45.0}) -- 0x7f4840034070 con 0x55a924b01170 2024-08-02T17:57:06.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.244+0000 7f48516f1640 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a924b01170 0x55a924b035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:06.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.249+0000 7f48516f1640 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a924b01170 0x55a924b035a0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37be0bde5d03480 server_cookie=3c13ce9e8075963d in_seq=0 out_seq=0 2024-08-02T17:57:06.252 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.249+0000 7f4834ff9640 10 client.27515 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:06.260 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:05 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:57:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:05 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:05 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:57:06.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:05 smithi083 ceph-mon[37031]: osdmap e104: 12 total, 12 up, 12 in 2024-08-02T17:57:06.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:05 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:57:06.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:05 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:06.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:05 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:57:06.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:05 smithi077 ceph-mon[37108]: osdmap e104: 12 total, 12 up, 12 in 2024-08-02T17:57:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4834ff9640 1 -- 172.21.15.69:0/873768290 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f4840034070 con 0x55a924b01170 2024-08-02T17:57:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4834ff9640 10 client.27515 handle_command_reply: tid=0 2024-08-02T17:57:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 client.27515 shutdown 2024-08-02T17:57:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 msgr2=0x55a924a800e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a800e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4840015040 tx=0x7f4840005110 comp rx=0 tx=0).stop 2024-08-02T17:57:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 shutdown_connections 2024-08-02T17:57:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a924a864c0 0x55a924a80620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a924b01170 0x55a924b035a0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a924a7bc00 0x55a924a800e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 --2- 172.21.15.69:0/873768290 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a924a7b2e0 0x55a924a8ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4834ff9640 0 client.27515 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 >> 172.21.15.69:0/873768290 conn(0x55a924a77870 msgr2=0x55a924a761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 shutdown_connections 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 1 -- 172.21.15.69:0/873768290 wait complete. 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.587+0000 7f4855d53740 20 client.27515 trim_cache size 0 max 16384 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:57:06.590 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 38, 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "82059677": { 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 38, 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 139.9, 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:57:06.591 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:06.592 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 -- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f483c0a2e40 msgr2=0x7f483c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f483c0a2e40 0x7f483c0a5300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4840009c70 tx=0x7f4840018dd0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 -- 172.21.15.69:0/3168957711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f484c1006b0 msgr2=0x7f484c073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f484c1006b0 0x7f484c073630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4838035040 tx=0x7f4838004270 comp rx=0 tx=0).stop 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 -- 172.21.15.69:0/3168957711 shutdown_connections 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f484c100fd0 0x7f484c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f483c0a2e40 0x7f483c0a5300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f484c105ac0 0x7f484c071950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 --2- 172.21.15.69:0/3168957711 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f484c1006b0 0x7f484c073630 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 -- 172.21.15.69:0/3168957711 >> 172.21.15.69:0/3168957711 conn(0x7f484c0fc340 msgr2=0x7f484c107c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 -- 172.21.15.69:0/3168957711 shutdown_connections 2024-08-02T17:57:06.594 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:06.588+0000 7f48526f3640 1 -- 172.21.15.69:0/3168957711 wait complete. 2024-08-02T17:57:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:06 smithi083 ceph-mon[37031]: pgmap v332: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.9 MiB/s wr, 183 op/s 2024-08-02T17:57:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:06 smithi077 ceph-mon[37108]: pgmap v332: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.9 MiB/s wr, 183 op/s 2024-08-02T17:57:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:06 smithi069 ceph-mon[32746]: pgmap v332: 129 pgs: 129 active+clean; 3.1 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 2.9 MiB/s wr, 183 op/s 2024-08-02T17:57:08.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:07 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:08.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:07 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:08.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:07 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:08 smithi083 ceph-mon[37031]: pgmap v333: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.3 MiB/s wr, 367 op/s 2024-08-02T17:57:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:08 smithi077 ceph-mon[37108]: pgmap v333: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.3 MiB/s wr, 367 op/s 2024-08-02T17:57:09.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:08 smithi069 ceph-mon[32746]: pgmap v333: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 5.3 MiB/s wr, 367 op/s 2024-08-02T17:57:09.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 600.0 s, 86.7 tps, lat 299.316 ms stddev 283.801 2024-08-02T17:57:10.249 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:09 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:57:10.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:09 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:57:10.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:09 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:57:11.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 -- 172.21.15.69:0/3909959677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4100020 msgr2=0x7f75f4100400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 --2- 172.21.15.69:0/3909959677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4100020 0x7f75f4100400 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009930 tx=0x7f75e402a420 comp rx=0 tx=0).stop 2024-08-02T17:57:11.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 -- 172.21.15.69:0/3909959677 shutdown_connections 2024-08-02T17:57:11.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 --2- 172.21.15.69:0/3909959677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75f4106c80 0x7f75f410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 --2- 172.21.15.69:0/3909959677 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75f41009d0 0x7f75f4106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 --2- 172.21.15.69:0/3909959677 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f4100020 0x7f75f4100400 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 -- 172.21.15.69:0/3909959677 >> 172.21.15.69:0/3909959677 conn(0x7f75f4076f40 msgr2=0x7f75f40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 -- 172.21.15.69:0/3909959677 shutdown_connections 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 -- 172.21.15.69:0/3909959677 wait complete. 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 Processor -- start 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.868+0000 7f75f882f640 1 -- start start 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f882f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75f4100020 0x7f75f4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f882f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f41009d0 0x7f75f4198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f882f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75f4106c80 0x7f75f419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f882f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75f4111cd0 con 0x7f75f41009d0 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f882f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f75f4111b50 con 0x7f75f4100020 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f882f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75f4111e50 con 0x7f75f4106c80 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f41009d0 0x7f75f4198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f41009d0 0x7f75f4198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59450/0 (socket says 172.21.15.69:59450) 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 -- 172.21.15.69:0/376009819 learned_addr learned my addr 172.21.15.69:0/376009819 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f259c640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75f4100020 0x7f75f4198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75f4106c80 0x7f75f419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 -- 172.21.15.69:0/376009819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75f4106c80 msgr2=0x7f75f419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75f4106c80 0x7f75f419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 -- 172.21.15.69:0/376009819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75f4100020 msgr2=0x7f75f4198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75f4100020 0x7f75f4198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 -- 172.21.15.69:0/376009819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f419d1c0 con 0x7f75f41009d0 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75f4106c80 0x7f75f419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:11.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75f1d9b640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f41009d0 0x7f75f4198750 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00d770 tx=0x7f75dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75eb7fe640 1 -- 172.21.15.69:0/376009819 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc017070 con 0x7f75f41009d0 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.869+0000 7f75eb7fe640 1 -- 172.21.15.69:0/376009819 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75dc009c30 con 0x7f75f41009d0 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.870+0000 7f75f882f640 1 -- 172.21.15.69:0/376009819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f419d4b0 con 0x7f75f41009d0 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.870+0000 7f75eb7fe640 1 -- 172.21.15.69:0/376009819 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00f620 con 0x7f75f41009d0 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.870+0000 7f75f882f640 1 -- 172.21.15.69:0/376009819 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f41a4d90 con 0x7f75f41009d0 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75eb7fe640 1 -- 172.21.15.69:0/376009819 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f75dc01a070 con 0x7f75f41009d0 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75fac2d740 1 Processor -- start 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75fac2d740 1 -- start start 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 0x557b862494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b862582c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b8625abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75eb7fe640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f75e00a2cf0 0x7f75e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75fac2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557b861ae980 con 0x557b862499f0 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75fac2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b86146a70 con 0x557b86258800 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.872+0000 7f75fac2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557b861aa940 con 0x557b862490d0 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b862582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b862582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59456/0 (socket says 172.21.15.69:59456) 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 -- 172.21.15.69:0/2990457394 learned_addr learned my addr 172.21.15.69:0/2990457394 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f1d9b640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f75e00a2cf0 0x7f75e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b8625abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75eb7fe640 1 -- 172.21.15.69:0/376009819 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f75dc0cf3d0 con 0x7f75f41009d0 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 0x557b862494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 -- 172.21.15.69:0/2990457394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 msgr2=0x557b862494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 0x557b862494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 -- 172.21.15.69:0/2990457394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 msgr2=0x557b8625abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b8625abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 -- 172.21.15.69:0/2990457394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b861a96e0 con 0x557b862499f0 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b8625abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f1d9b640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f75e00a2cf0 0x7f75e00a51b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0cc760 tx=0x7f75dc016b40 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75f259c640 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b862582c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f75e402a940 tx=0x7f75e4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881fb962950b5525 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75e8ff9640 1 -- 172.21.15.69:0/2990457394 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4040070 con 0x557b862499f0 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.873+0000 7f75e8ff9640 1 -- 172.21.15.69:0/2990457394 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e4031930 con 0x557b862499f0 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75e8ff9640 1 -- 172.21.15.69:0/2990457394 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4035040 con 0x557b862499f0 2024-08-02T17:57:11.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 -- 172.21.15.69:0/2990457394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 msgr2=0x557b862582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b862582c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f75e402a940 tx=0x7f75e4005c00 comp rx=0 tx=0).stop 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 -- 172.21.15.69:0/2990457394 shutdown_connections 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b862582c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b8625abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2990457394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 0x557b862494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 -- 172.21.15.69:0/2990457394 >> 172.21.15.69:0/2990457394 conn(0x557b862457a0 msgr2=0x557b86245b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 -- 172.21.15.69:0/2990457394 shutdown_connections 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.874+0000 7f75fac2d740 1 -- 172.21.15.69:0/2990457394 wait complete. 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 1 -- start start 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 1 Processor -- start 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86255660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86255ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b86258800 0x557b862560e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557b8625b370 con 0x557b862490d0 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557b86169aa0 con 0x557b862499f0 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557b8625b4f0 con 0x557b86258800 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75fac2d740 10 client.0 _ll_register_callbacks cb 0x557b86158e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75f2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86255660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75f2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86255660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59466/0 (socket says 172.21.15.69:59466) 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75f2d9d640 1 -- 172.21.15.69:0/2941643708 learned_addr learned my addr 172.21.15.69:0/2941643708 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75f259c640 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86255ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75f2d9d640 1 -- 172.21.15.69:0/2941643708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b86258800 msgr2=0x557b862560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b86258800 0x557b862560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75f2d9d640 1 -- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 msgr2=0x557b86255ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.875+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86255ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75f2d9d640 1 -- 172.21.15.69:0/2941643708 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b8614d650 con 0x557b862490d0 2024-08-02T17:57:11.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75f259c640 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86255ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86255660 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0149d0 tx=0x7f75ec014e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59325db40e83941f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75d9ffb640 1 -- 172.21.15.69:0/2941643708 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec011b30 con 0x557b862490d0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75d9ffb640 1 -- 172.21.15.69:0/2941643708 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f75ec011cd0 con 0x557b862490d0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 1 -- 172.21.15.69:0/2941643708 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e4031930 con 0x557b862490d0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75d9ffb640 1 -- 172.21.15.69:0/2941643708 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75ec026050 con 0x557b862490d0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 10 client.18045 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 1 -- 172.21.15.69:0/2941643708 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b85f203c0 con 0x557b862490d0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75d9ffb640 1 -- 172.21.15.69:0/2941643708 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e4031930 con 0x557b862490d0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75d9ffb640 1 -- 172.21.15.69:0/2941643708 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f75ec00c030 con 0x557b862490d0 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 10 client.18045 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 10 client.18045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 20 client.18045 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:11.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557b86268320 0x557b8626a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 4 client.18045 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.876+0000 7f75fac2d740 1 -- 172.21.15.69:0/2941643708 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b861bbef0 con 0x557b86268320 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.877+0000 7f75f259c640 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557b86268320 0x557b8626a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.877+0000 7f75f259c640 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557b86268320 0x557b8626a710 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2239cfae79fcfa59 server_cookie=2d06df9c086a2894 in_seq=0 out_seq=0 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.877+0000 7f75d9ffb640 10 client.18045 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75d9ffb640 1 -- 172.21.15.69:0/2941643708 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b861bbef0 con 0x557b86268320 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75d9ffb640 10 client.18045 handle_command_reply: tid=0 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 client.18045 shutdown 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 -- 172.21.15.69:0/2941643708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 msgr2=0x557b86255660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86255660 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0149d0 tx=0x7f75ec014e90 comp rx=0 tx=0).stop 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 -- 172.21.15.69:0/2941643708 shutdown_connections 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557b86268320 0x557b8626a710 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b86258800 0x557b862560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86255ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 --2- 172.21.15.69:0/2941643708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86255660 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75d9ffb640 0 client.18045 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 -- 172.21.15.69:0/2941643708 >> 172.21.15.69:0/2941643708 conn(0x557b862457a0 msgr2=0x557b8624b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 -- 172.21.15.69:0/2941643708 shutdown_connections 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 1 -- 172.21.15.69:0/2941643708 wait complete. 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.878+0000 7f75fac2d740 20 client.18045 trim_cache size 0 max 16384 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.895+0000 7f75fac2d740 1 Processor -- start 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.895+0000 7f75fac2d740 1 -- start start 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.895+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 0x557b86257dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.895+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b86254f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.059 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.895+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b86255e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.060 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:10 smithi069 ceph-mon[32746]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:57:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75fac2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41af730 con 0x557b862499f0 2024-08-02T17:57:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75fac2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41ad000 con 0x557b86258800 2024-08-02T17:57:11.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75fac2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41aae70 con 0x557b862490d0 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b86254f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b86254f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59470/0 (socket says 172.21.15.69:59470) 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 -- 172.21.15.69:0/3123887988 learned_addr learned my addr 172.21.15.69:0/3123887988 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/3123887988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b86255e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 -- 172.21.15.69:0/3123887988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 msgr2=0x557b86257dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 --2- 172.21.15.69:0/3123887988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 0x557b86257dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 -- 172.21.15.69:0/3123887988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 msgr2=0x557b86255e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 --2- 172.21.15.69:0/3123887988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b86255e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 -- 172.21.15.69:0/3123887988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b86266e80 con 0x557b862499f0 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75f259c640 1 --2- 172.21.15.69:0/3123887988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b86254f60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009930 tx=0x7f75e4005570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa12ebb7f83d392 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75dbfff640 1 -- 172.21.15.69:0/3123887988 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4040040 con 0x557b862499f0 2024-08-02T17:57:11.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75dbfff640 1 -- 172.21.15.69:0/3123887988 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e40035f0 con 0x557b862499f0 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.896+0000 7f75dbfff640 1 -- 172.21.15.69:0/3123887988 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e4035040 con 0x557b862499f0 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 -- 172.21.15.69:0/3123887988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 msgr2=0x557b86254f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 --2- 172.21.15.69:0/3123887988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b86254f60 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009930 tx=0x7f75e4005570 comp rx=0 tx=0).stop 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 -- 172.21.15.69:0/3123887988 shutdown_connections 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 --2- 172.21.15.69:0/3123887988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862499f0 0x557b86254f60 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 --2- 172.21.15.69:0/3123887988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b86258800 0x557b86255e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 --2- 172.21.15.69:0/3123887988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b862490d0 0x557b86257dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 -- 172.21.15.69:0/3123887988 >> 172.21.15.69:0/3123887988 conn(0x557b862457a0 msgr2=0x557b86244980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 -- 172.21.15.69:0/3123887988 shutdown_connections 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 -- 172.21.15.69:0/3123887988 wait complete. 2024-08-02T17:57:11.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 -- start start 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 Processor -- start 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86260960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.897+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86260ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75fac2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b86258800 0x557b86251e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75fac2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41ab4c0 con 0x557b862490d0 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75fac2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41ac930 con 0x557b862499f0 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75fac2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75f41abb90 con 0x557b86258800 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75fac2d740 10 client.0 _ll_register_callbacks cb 0x7f75ec009f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86260960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86260960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59472/0 (socket says 172.21.15.69:59472) 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 -- 172.21.15.69:0/777980820 learned_addr learned my addr 172.21.15.69:0/777980820 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f259c640 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86260ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 -- 172.21.15.69:0/777980820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b86258800 msgr2=0x557b86251e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b86258800 0x557b86251e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 -- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 msgr2=0x557b86260ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86260ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 -- 172.21.15.69:0/777980820 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b86250490 con 0x557b862490d0 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f259c640 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86260ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75f2d9d640 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86260960 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f75ec015fd0 tx=0x7f75ec01e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fd811aac8ce9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75d9ffb640 1 -- 172.21.15.69:0/777980820 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec009520 con 0x557b862490d0 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75d9ffb640 1 -- 172.21.15.69:0/777980820 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f75ec01e7a0 con 0x557b862490d0 2024-08-02T17:57:11.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75fac2d740 1 -- 172.21.15.69:0/777980820 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75e40035f0 con 0x557b862490d0 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75d9ffb640 1 -- 172.21.15.69:0/777980820 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f75ec038050 con 0x557b862490d0 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75fac2d740 10 client.18051 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.898+0000 7f75fac2d740 1 -- 172.21.15.69:0/777980820 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557b862d26f0 con 0x557b862490d0 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75d9ffb640 1 -- 172.21.15.69:0/777980820 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e40035f0 con 0x557b862490d0 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75d9ffb640 1 -- 172.21.15.69:0/777980820 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f75ec00c040 con 0x557b862490d0 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75fac2d740 10 client.18051 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75fac2d740 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75fac2d740 20 client.18051 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75fac2d740 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557b862d2e70 0x557b862d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75fac2d740 4 client.18051 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75fac2d740 1 -- 172.21.15.69:0/777980820 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557b861b6400 con 0x557b862d2e70 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75f259c640 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557b862d2e70 0x557b862d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:11.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75d9ffb640 10 client.18051 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.899+0000 7f75f259c640 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557b862d2e70 0x557b862d5290 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4218f159deb7dac4 server_cookie=6a7782fdee958ce0 in_seq=0 out_seq=0 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75d9ffb640 1 -- 172.21.15.69:0/777980820 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557b861b6400 con 0x557b862d2e70 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75d9ffb640 10 client.18051 handle_command_reply: tid=0 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 client.18051 shutdown 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 -- 172.21.15.69:0/777980820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 msgr2=0x557b86260960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86260960 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f75ec015fd0 tx=0x7f75ec01e620 comp rx=0 tx=0).stop 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 -- 172.21.15.69:0/777980820 shutdown_connections 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557b862d2e70 0x557b862d5290 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557b86258800 0x557b86251e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557b862499f0 0x557b86260ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 --2- 172.21.15.69:0/777980820 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557b862490d0 0x557b86260960 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 -- 172.21.15.69:0/777980820 >> 172.21.15.69:0/777980820 conn(0x557b862457a0 msgr2=0x557b86255e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:11.067 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:57:11.067 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:57:11.067 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:57:11.067 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:57:11.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75d9ffb640 0 client.18051 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:11.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 -- 172.21.15.69:0/777980820 shutdown_connections 2024-08-02T17:57:11.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 1 -- 172.21.15.69:0/777980820 wait complete. 2024-08-02T17:57:11.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.900+0000 7f75fac2d740 20 client.18051 trim_cache size 0 max 16384 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 -- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f75e00a2cf0 msgr2=0x7f75e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f75e00a2cf0 0x7f75e00a51b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0cc760 tx=0x7f75dc016b40 comp rx=0 tx=0).stop 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 -- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f41009d0 msgr2=0x7f75f4198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f41009d0 0x7f75f4198750 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00d770 tx=0x7f75dc00dc30 comp rx=0 tx=0).stop 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 -- 172.21.15.69:0/376009819 shutdown_connections 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f75f41009d0 0x7f75f4198750 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f75e00a2cf0 0x7f75e00a51b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75f4106c80 0x7f75f419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 --2- 172.21.15.69:0/376009819 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f75f4100020 0x7f75f4198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 -- 172.21.15.69:0/376009819 >> 172.21.15.69:0/376009819 conn(0x7f75f4076f40 msgr2=0x7f75f4105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 -- 172.21.15.69:0/376009819 shutdown_connections 2024-08-02T17:57:11.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:10.901+0000 7f75f882f640 1 -- 172.21.15.69:0/376009819 wait complete. 2024-08-02T17:57:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:10 smithi083 ceph-mon[37031]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:57:11.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:10 smithi077 ceph-mon[37108]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:57:12.115 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '82059677', quiesce took 11.7 seconds. Will release after: 8.2 2024-08-02T17:57:12.729 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:57:12.729 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:57:12.993 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:13.230 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:12 smithi069 ceph-mon[32746]: pgmap v335: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:57:13.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:12 smithi083 ceph-mon[37031]: pgmap v335: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:57:13.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:12 smithi077 ceph-mon[37108]: pgmap v335: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 6.4 MiB/s wr, 547 op/s 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3796244077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14100b50 msgr2=0x7f6b14100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3796244077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14100b50 0x7f6b14100f30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009900 tx=0x7f6b0802a480 comp rx=0 tx=0).stop 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3796244077 shutdown_connections 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3796244077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b14102c20 0x7f6b1410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3796244077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b141022a0 0x7f6b141026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3796244077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14100b50 0x7f6b14100f30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3796244077 >> 172.21.15.69:0/3796244077 conn(0x7f6b140fc500 msgr2=0x7f6b140fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3796244077 shutdown_connections 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.435+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3796244077 wait complete. 2024-08-02T17:57:13.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b1ae19640 1 Processor -- start 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b1ae19640 1 -- start start 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b1ae19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b14100b50 0x7f6b141982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b1ae19640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b141022a0 0x7f6b14198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b1ae19640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14102c20 0x7f6b1419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b1ae19640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14111dc0 con 0x7f6b14102c20 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b1ae19640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14111c40 con 0x7f6b14100b50 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b1ae19640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6b14111f40 con 0x7f6b141022a0 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b193b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14102c20 0x7f6b1419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b193b6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14102c20 0x7f6b1419cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59486/0 (socket says 172.21.15.69:59486) 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b193b6640 1 -- 172.21.15.69:0/3957568350 learned_addr learned my addr 172.21.15.69:0/3957568350 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b18bb5640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b14100b50 0x7f6b141982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b13fff640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b141022a0 0x7f6b14198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b193b6640 1 -- 172.21.15.69:0/3957568350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b141022a0 msgr2=0x7f6b14198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b193b6640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b141022a0 0x7f6b14198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b193b6640 1 -- 172.21.15.69:0/3957568350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b14100b50 msgr2=0x7f6b141982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b193b6640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b14100b50 0x7f6b141982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.436+0000 7f6b193b6640 1 -- 172.21.15.69:0/3957568350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1419d270 con 0x7f6b14102c20 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.437+0000 7f6b13fff640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b141022a0 0x7f6b14198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.437+0000 7f6b193b6640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14102c20 0x7f6b1419cb90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00eaa0 tx=0x7f6b0c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.437+0000 7f6b11ffb640 1 -- 172.21.15.69:0/3957568350 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c00cdb0 con 0x7f6b14102c20 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.437+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1419d560 con 0x7f6b14102c20 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.437+0000 7f6b11ffb640 1 -- 172.21.15.69:0/3957568350 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0c005020 con 0x7f6b14102c20 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.437+0000 7f6b11ffb640 1 -- 172.21.15.69:0/3957568350 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0116b0 con 0x7f6b14102c20 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.437+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141a4e40 con 0x7f6b14102c20 2024-08-02T17:57:13.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.438+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14100f30 con 0x7f6b14102c20 2024-08-02T17:57:13.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.439+0000 7f6b11ffb640 1 -- 172.21.15.69:0/3957568350 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6b0c005560 con 0x7f6b14102c20 2024-08-02T17:57:13.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.440+0000 7f6b11ffb640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b040a2cf0 0x7f6b040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.440+0000 7f6b18bb5640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b040a2cf0 0x7f6b040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.440+0000 7f6b11ffb640 1 -- 172.21.15.69:0/3957568350 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f6b0c0cfe90 con 0x7f6b14102c20 2024-08-02T17:57:13.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.441+0000 7f6b18bb5640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b040a2cf0 0x7f6b040a51b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009230 tx=0x7f6b080058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:13.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.447+0000 7f6b11ffb640 1 -- 172.21.15.69:0/3957568350 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6b0c016030 con 0x7f6b14102c20 2024-08-02T17:57:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/612383031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba01006d0 msgr2=0x7f0ba0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/612383031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba01006d0 0x7f0ba0100b30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009930 tx=0x7f0b9002a420 comp rx=0 tx=0).stop 2024-08-02T17:57:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/612383031 shutdown_connections 2024-08-02T17:57:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/612383031 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba01006d0 0x7f0ba0100b30 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/612383031 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ba0101070 0x7f0ba010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/612383031 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba01066e0 0x7f0ba0106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/612383031 >> 172.21.15.69:0/612383031 conn(0x7f0ba00776d0 msgr2=0x7f0ba00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/612383031 shutdown_connections 2024-08-02T17:57:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.467+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/612383031 wait complete. 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0ba4ed3640 1 Processor -- start 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0ba4ed3640 1 -- start start 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0ba4ed3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ba01006d0 0x7f0ba0199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0ba4ed3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba0101070 0x7f0ba01995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0ba4ed3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba01066e0 0x7f0ba0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0ba4ed3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0110b20 con 0x7f0ba0101070 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0ba4ed3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01109a0 con 0x7f0ba01006d0 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0ba4ed3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba0110ca0 con 0x7f0ba01066e0 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0b9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba0101070 0x7f0ba01995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0b9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba0101070 0x7f0ba01995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59500/0 (socket says 172.21.15.69:59500) 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/1958277647 learned_addr learned my addr 172.21.15.69:0/1958277647 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.468+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ba01006d0 0x7f0ba0199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba01066e0 0x7f0ba0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/1958277647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba01066e0 msgr2=0x7f0ba0199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba01066e0 0x7f0ba0199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/1958277647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ba01006d0 msgr2=0x7f0ba0199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ba01006d0 0x7f0ba0199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/1958277647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba01a4470 con 0x7f0ba0101070 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba01066e0 0x7f0ba0199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba0101070 0x7f0ba01995b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009a50 tx=0x7f0b90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b8f7fe640 1 -- 172.21.15.69:0/1958277647 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9003f030 con 0x7f0ba0101070 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/1958277647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba01a4700 con 0x7f0ba0101070 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b8f7fe640 1 -- 172.21.15.69:0/1958277647 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b90003ad0 con 0x7f0ba0101070 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0b8f7fe640 1 -- 172.21.15.69:0/1958277647 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90035070 con 0x7f0ba0101070 2024-08-02T17:57:13.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.469+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/1958277647 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba01a4c40 con 0x7f0ba0101070 2024-08-02T17:57:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.471+0000 7f0b8f7fe640 1 -- 172.21.15.69:0/1958277647 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0b90042030 con 0x7f0ba0101070 2024-08-02T17:57:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.471+0000 7f0ba7c2d740 1 Processor -- start 2024-08-02T17:57:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0ba7c2d740 1 -- start start 2024-08-02T17:57:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930b7b0 0x55f83930bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 0x55f839316430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f839318d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f83926f9b0 con 0x55f839316970 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f839217800 con 0x55f83930b7b0 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f83926b970 con 0x55f83930c0d0 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0b9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f839318d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0b8f7fe640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0b880a2cf0 0x7f0b880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0b9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f839318d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59516/0 (socket says 172.21.15.69:59516) 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/632951470 learned_addr learned my addr 172.21.15.69:0/632951470 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0b880a2cf0 0x7f0b880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 0x55f839316430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930b7b0 0x55f83930bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.472+0000 7f0b8f7fe640 1 -- 172.21.15.69:0/1958277647 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f0b900f00e0 con 0x7f0ba0101070 2024-08-02T17:57:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/632951470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 msgr2=0x55f839316430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 0x55f839316430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/632951470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930b7b0 msgr2=0x55f83930bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930b7b0 0x55f83930bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/632951470 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f838f83210 con 0x55f839316970 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0b880a2cf0 0x7f0b880a51b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b98013b90 tx=0x7f0b98007d70 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f839318d60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0b90031550 tx=0x7f0b90009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae685b6fd0f77e77 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b8cff9640 1 -- 172.21.15.69:0/632951470 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90049070 con 0x55f839316970 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b8cff9640 1 -- 172.21.15.69:0/632951470 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b900bc890 con 0x55f839316970 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.473+0000 7f0b8cff9640 1 -- 172.21.15.69:0/632951470 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b900dc110 con 0x55f839316970 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/632951470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 msgr2=0x55f839318d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f839318d60 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0b90031550 tx=0x7f0b90009260 comp rx=0 tx=0).stop 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/632951470 shutdown_connections 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f839318d60 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 0x55f839316430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/632951470 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930b7b0 0x55f83930bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/632951470 >> 172.21.15.69:0/632951470 conn(0x55f839307d20 msgr2=0x55f839308100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/632951470 shutdown_connections 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/632951470 wait complete. 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 -- start start 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.474+0000 7f0ba7c2d740 1 Processor -- start 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 0x55f8393137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930c0d0 0x55f839313d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 0x55f839314250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f839319300 con 0x55f83930c0d0 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f83922b500 con 0x55f839316970 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f839319480 con 0x55f83930b7b0 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0ba7c2d740 10 client.0 _ll_register_callbacks cb 0x55f838f82a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930c0d0 0x55f839313d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930c0d0 0x55f839313d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59532/0 (socket says 172.21.15.69:59532) 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/1958417614 learned_addr learned my addr 172.21.15.69:0/1958417614 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 0x55f8393137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 0x55f839314250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/1958417614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 msgr2=0x55f8393137d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:13.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 0x55f8393137d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/1958417614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 msgr2=0x55f839314250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 0x55f839314250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 -- 172.21.15.69:0/1958417614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83904f620 con 0x55f83930c0d0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930c0d0 0x55f839313d10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0b900ed470 tx=0x7f0b900ed390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a50659d3f3e5b44 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b85ffb640 1 -- 172.21.15.69:0/1958417614 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90049050 con 0x55f83930c0d0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b85ffb640 1 -- 172.21.15.69:0/1958417614 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f0b900dc110 con 0x55f83930c0d0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/1958417614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b900bc890 con 0x55f83930c0d0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.475+0000 7f0b85ffb640 1 -- 172.21.15.69:0/1958417614 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b900bc190 con 0x55f83930c0d0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0ba7c2d740 10 client.18069 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/1958417614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f839319300 con 0x55f83930c0d0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0b85ffb640 1 -- 172.21.15.69:0/1958417614 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b900bc890 con 0x55f83930c0d0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0b85ffb640 1 -- 172.21.15.69:0/1958417614 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b900c0170 con 0x55f83930c0d0 2024-08-02T17:57:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0ba7c2d740 10 client.18069 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0ba7c2d740 10 client.18069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0ba7c2d740 20 client.18069 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f8393260a0 0x55f839328490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0ba7c2d740 4 client.18069 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/1958417614 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83927d0e0 con 0x55f8393260a0 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.476+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f8393260a0 0x55f839328490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.477+0000 7f0b85ffb640 10 client.18069 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.477+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f8393260a0 0x55f839328490 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0ceb8aec44e9b7 server_cookie=f1204a48b95c5bed in_seq=0 out_seq=0 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.477+0000 7f0b85ffb640 1 -- 172.21.15.69:0/1958417614 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f83927d0e0 con 0x55f8393260a0 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.477+0000 7f0b85ffb640 10 client.18069 handle_command_reply: tid=0 2024-08-02T17:57:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.477+0000 7f0ba7c2d740 1 client.18069 shutdown 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/1958417614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930c0d0 msgr2=0x55f839313d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930c0d0 0x55f839313d10 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0b900ed470 tx=0x7f0b900ed390 comp rx=0 tx=0).stop 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/1958417614 shutdown_connections 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930c0d0 0x55f839313d10 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f8393260a0 0x55f839328490 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 0x55f839314250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/1958417614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 0x55f8393137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0b85ffb640 0 client.18069 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/1958417614 >> 172.21.15.69:0/1958417614 conn(0x55f839307d20 msgr2=0x55f83930da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/1958417614 shutdown_connections 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/1958417614 wait complete. 2024-08-02T17:57:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.478+0000 7f0ba7c2d740 20 client.18069 trim_cache size 0 max 16384 2024-08-02T17:57:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0ba7c2d740 1 Processor -- start 2024-08-02T17:57:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0ba7c2d740 1 -- start start 2024-08-02T17:57:13.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930b7b0 0x55f839314080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 0x55f83932dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 0x55f83932e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01b1750 con 0x55f83930b7b0 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01af020 con 0x55f839316970 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ace90 con 0x55f83930c0d0 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930b7b0 0x55f839314080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930b7b0 0x55f839314080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59544/0 (socket says 172.21.15.69:59544) 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9e59c640 1 -- 172.21.15.69:0/3046098864 learned_addr learned my addr 172.21.15.69:0/3046098864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9e59c640 1 -- 172.21.15.69:0/3046098864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 msgr2=0x55f83932dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/3046098864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 0x55f83932e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/3046098864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 0x55f83932dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9e59c640 1 -- 172.21.15.69:0/3046098864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 msgr2=0x55f83932e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:13.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/3046098864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 0x55f83932e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.493+0000 7f0b9e59c640 1 -- 172.21.15.69:0/3046098864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f839307b50 con 0x55f83930b7b0 2024-08-02T17:57:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/3046098864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930b7b0 0x55f839314080 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x55f839328cb0 tx=0x7f0b94009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c098b5d9d136336a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0b87fff640 1 -- 172.21.15.69:0/3046098864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94025070 con 0x55f83930b7b0 2024-08-02T17:57:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0b87fff640 1 -- 172.21.15.69:0/3046098864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b94015e10 con 0x55f83930b7b0 2024-08-02T17:57:13.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0b87fff640 1 -- 172.21.15.69:0/3046098864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94014670 con 0x55f83930b7b0 2024-08-02T17:57:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/3046098864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930b7b0 msgr2=0x55f839314080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/3046098864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930b7b0 0x55f839314080 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x55f839328cb0 tx=0x7f0b94009110 comp rx=0 tx=0).stop 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/3046098864 shutdown_connections 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/3046098864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f839316970 0x55f83932e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/3046098864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930c0d0 0x55f83932dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/3046098864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f83930b7b0 0x55f839314080 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.494+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/3046098864 >> 172.21.15.69:0/3046098864 conn(0x55f839307d20 msgr2=0x55f839315740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/3046098864 shutdown_connections 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/3046098864 wait complete. 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 -- start start 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 Processor -- start 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 0x55f83931eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930c0d0 0x55f83931a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f83931a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ad4e0 con 0x55f839316970 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01ae950 con 0x55f83930c0d0 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba01adbb0 con 0x55f83930b7b0 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0ba7c2d740 10 client.0 _ll_register_callbacks cb 0x7f0b900bbff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0b9ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f83931a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0b9ed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f83931a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:59546/0 (socket says 172.21.15.69:59546) 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0b9ed9d640 1 -- 172.21.15.69:0/604317886 learned_addr learned my addr 172.21.15.69:0/604317886 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 0x55f83931eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0b9dd9b640 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930c0d0 0x55f83931a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0b9ed9d640 1 -- 172.21.15.69:0/604317886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 msgr2=0x55f83931eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 0x55f83931eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.495+0000 7f0b9ed9d640 1 -- 172.21.15.69:0/604317886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930c0d0 msgr2=0x55f83931a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930c0d0 0x55f83931a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b9ed9d640 1 -- 172.21.15.69:0/604317886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83931afe0 con 0x55f839316970 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b9ed9d640 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f83931a7b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0b98010db0 tx=0x7f0b980072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7630b3b5179579 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b85ffb640 1 -- 172.21.15.69:0/604317886 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9802a680 con 0x55f839316970 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b85ffb640 1 -- 172.21.15.69:0/604317886 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9656+0+0 (secure 0 0 0) 0x7f0b9802a820 con 0x55f839316970 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/604317886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b94015e10 con 0x55f839316970 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b85ffb640 1 -- 172.21.15.69:0/604317886 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0b98033050 con 0x55f839316970 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0ba7c2d740 10 client.18075 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/604317886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f839390d90 con 0x55f839316970 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b85ffb640 1 -- 172.21.15.69:0/604317886 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b94015e10 con 0x55f839316970 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0b85ffb640 1 -- 172.21.15.69:0/604317886 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0b9801c040 con 0x55f839316970 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0ba7c2d740 10 client.18075 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0ba7c2d740 10 client.18075 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.496+0000 7f0ba7c2d740 20 client.18075 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.497+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f839391510 0x55f839393930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.497+0000 7f0ba7c2d740 4 client.18075 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:57:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.497+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/604317886 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f839277430 con 0x55f839391510 2024-08-02T17:57:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.497+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f839391510 0x55f839393930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.497+0000 7f0b9e59c640 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f839391510 0x55f839393930 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df380e63359488b8 server_cookie=ea3f9d99440c9ba4 in_seq=0 out_seq=0 2024-08-02T17:57:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.497+0000 7f0b85ffb640 10 client.18075 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0b85ffb640 1 -- 172.21.15.69:0/604317886 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f839277430 con 0x55f839391510 2024-08-02T17:57:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0b85ffb640 10 client.18075 handle_command_reply: tid=0 2024-08-02T17:57:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 client.18075 shutdown 2024-08-02T17:57:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/604317886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 msgr2=0x55f83931a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f83931a7b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f0b98010db0 tx=0x7f0b980072a0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/604317886 shutdown_connections 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f839316970 0x55f83931a7b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f839391510 0x55f839393930 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f83930c0d0 0x55f83931a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 --2- 172.21.15.69:0/604317886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f83930b7b0 0x55f83931eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/604317886 >> 172.21.15.69:0/604317886 conn(0x55f839307d20 msgr2=0x55f839314e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0b85ffb640 0 client.18075 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/604317886 shutdown_connections 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 1 -- 172.21.15.69:0/604317886 wait complete. 2024-08-02T17:57:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.525+0000 7f0ba7c2d740 20 client.18075 trim_cache size 0 max 16384 2024-08-02T17:57:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:57:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:57:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:57:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0b880a2cf0 msgr2=0x7f0b880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0b880a2cf0 0x7f0b880a51b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0b98013b90 tx=0x7f0b98007d70 comp rx=0 tx=0).stop 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba0101070 msgr2=0x7f0ba01995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba0101070 0x7f0ba01995b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009a50 tx=0x7f0b90004270 comp rx=0 tx=0).stop 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/1958277647 shutdown_connections 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0ba0101070 0x7f0ba01995b0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0b880a2cf0 0x7f0b880a51b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0ba01066e0 0x7f0ba0199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 --2- 172.21.15.69:0/1958277647 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0ba01006d0 0x7f0ba0199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/1958277647 >> 172.21.15.69:0/1958277647 conn(0x7f0ba00776d0 msgr2=0x7f0ba0103550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/1958277647 shutdown_connections 2024-08-02T17:57:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:13.526+0000 7f0ba4ed3640 1 -- 172.21.15.69:0/1958277647 wait complete. 2024-08-02T17:57:13.661 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.660+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b14108dc0 con 0x7f6b14102c20 2024-08-02T17:57:13.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.660+0000 7f6b11ffb640 1 -- 172.21.15.69:0/3957568350 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f6b0c09c640 con 0x7f6b14102c20 2024-08-02T17:57:13.662 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:57:13.662 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:57:13.662 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:57:13.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b040a2cf0 msgr2=0x7f6b040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b040a2cf0 0x7f6b040a51b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b08009230 tx=0x7f6b080058d0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14102c20 msgr2=0x7f6b1419cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14102c20 0x7f6b1419cb90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c00eaa0 tx=0x7f6b0c00ef60 comp rx=0 tx=0).stop 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 shutdown_connections 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6b14102c20 0x7f6b1419cb90 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b040a2cf0 0x7f6b040a51b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6b141022a0 0x7f6b14198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 --2- 172.21.15.69:0/3957568350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6b14100b50 0x7f6b141982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 >> 172.21.15.69:0/3957568350 conn(0x7f6b140fc500 msgr2=0x7f6b140fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 shutdown_connections 2024-08-02T17:57:13.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:13.663+0000 7f6b1ae19640 1 -- 172.21.15.69:0/3957568350 wait complete. 2024-08-02T17:57:14.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:13 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3957568350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:14.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:13 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3957568350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:14.354 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:13 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3957568350' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:14.397 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:57:14.651 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:14.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 605.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:57:15.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:14 smithi069 ceph-mon[32746]: pgmap v336: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-08-02T17:57:15.085 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:14 smithi069 ceph-mon[32746]: osdmap e105: 12 total, 12 up, 12 in 2024-08-02T17:57:15.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 -- 172.21.15.69:0/41179759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 msgr2=0x7f9ca010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 --2- 172.21.15.69:0/41179759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 0x7f9ca010e780 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009cd0 tx=0x7f9c9002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 -- 172.21.15.69:0/41179759 shutdown_connections 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 --2- 172.21.15.69:0/41179759 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 0x7f9ca010e780 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 --2- 172.21.15.69:0/41179759 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ca01018b0 0x7f9ca0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 --2- 172.21.15.69:0/41179759 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca01078b0 0x7f9ca0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 -- 172.21.15.69:0/41179759 >> 172.21.15.69:0/41179759 conn(0x7f9ca00fd5f0 msgr2=0x7f9ca00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 -- 172.21.15.69:0/41179759 shutdown_connections 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 -- 172.21.15.69:0/41179759 wait complete. 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 Processor -- start 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.084+0000 7f9ca8396640 1 -- start start 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca8396640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ca01018b0 0x7f9ca0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca8396640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 0x7f9ca0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca8396640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca01078b0 0x7f9ca019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca8396640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0111cb0 con 0x7f9ca0102250 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca8396640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0111b30 con 0x7f9ca01018b0 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca8396640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca0111e30 con 0x7f9ca01078b0 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 0x7f9ca0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 0x7f9ca0198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35288/0 (socket says 172.21.15.69:35288) 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 -- 172.21.15.69:0/522108023 learned_addr learned my addr 172.21.15.69:0/522108023 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca6132640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ca01018b0 0x7f9ca0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca6933640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca01078b0 0x7f9ca019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 -- 172.21.15.69:0/522108023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca01078b0 msgr2=0x7f9ca019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca01078b0 0x7f9ca019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 -- 172.21.15.69:0/522108023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ca01018b0 msgr2=0x7f9ca0198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ca01018b0 0x7f9ca0198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 -- 172.21.15.69:0/522108023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca019d0e0 con 0x7f9ca0102250 2024-08-02T17:57:15.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca6132640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ca01018b0 0x7f9ca0198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9ca5931640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 0x7f9ca0198670 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400d950 tx=0x7f9c9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.085+0000 7f9c9f7fe640 1 -- 172.21.15.69:0/522108023 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94017070 con 0x7f9ca0102250 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.086+0000 7f9c9f7fe640 1 -- 172.21.15.69:0/522108023 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c94009c30 con 0x7f9ca0102250 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.086+0000 7f9ca8396640 1 -- 172.21.15.69:0/522108023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca019d3d0 con 0x7f9ca0102250 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.086+0000 7f9c9f7fe640 1 -- 172.21.15.69:0/522108023 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9400f6b0 con 0x7f9ca0102250 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.086+0000 7f9ca8396640 1 -- 172.21.15.69:0/522108023 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca01a4cb0 con 0x7f9ca0102250 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9caa794740 1 Processor -- start 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9caa794740 1 -- start start 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9caa794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9bd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9caa794740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9caa794740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 0x5638cd9b8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9caa794740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5638cd913a40 con 0x5638cd9bdb50 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9caa794740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638cd8ab9d0 con 0x5638cd9b81f0 2024-08-02T17:57:15.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9caa794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5638cd90fa00 con 0x5638cd9bd230 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9ca6132640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9ca6132640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9b7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35304/0 (socket says 172.21.15.69:35304) 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.089+0000 7f9ca6132640 1 -- 172.21.15.69:0/796384658 learned_addr learned my addr 172.21.15.69:0/796384658 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9ca6933640 1 --2- 172.21.15.69:0/796384658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9bd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9ca6132640 1 -- 172.21.15.69:0/796384658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 msgr2=0x5638cd9bd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9ca6132640 1 --2- 172.21.15.69:0/796384658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9bd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9ca6132640 1 -- 172.21.15.69:0/796384658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 msgr2=0x5638cd9b8670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9ca6132640 1 --2- 172.21.15.69:0/796384658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 0x5638cd9b8670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9ca6132640 1 -- 172.21.15.69:0/796384658 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cd8b2650 con 0x5638cd9bdb50 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9c9f7fe640 1 -- 172.21.15.69:0/522108023 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9c940109d0 con 0x7f9ca0102250 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9ca6132640 1 --2- 172.21.15.69:0/796384658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9b7b80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800bcc0 tx=0x7f9c9800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282190594691f130 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9c9cdea640 1 -- 172.21.15.69:0/796384658 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98015070 con 0x5638cd9bdb50 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9c9cdea640 1 -- 172.21.15.69:0/796384658 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c98011920 con 0x5638cd9bdb50 2024-08-02T17:57:15.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.090+0000 7f9c9cdea640 1 -- 172.21.15.69:0/796384658 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98010a30 con 0x5638cd9bdb50 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9c9f7fe640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c880a2cf0 0x7f9c880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9ca6933640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c880a2cf0 0x7f9c880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9c9f7fe640 1 -- 172.21.15.69:0/522108023 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f9c940cf4e0 con 0x7f9ca0102250 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 -- 172.21.15.69:0/796384658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 msgr2=0x5638cd9b7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 --2- 172.21.15.69:0/796384658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9b7b80 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800bcc0 tx=0x7f9c9800f730 comp rx=0 tx=0).stop 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 -- 172.21.15.69:0/796384658 shutdown_connections 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 --2- 172.21.15.69:0/796384658 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9b7b80 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 --2- 172.21.15.69:0/796384658 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 0x5638cd9b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 --2- 172.21.15.69:0/796384658 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9bd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 -- 172.21.15.69:0/796384658 >> 172.21.15.69:0/796384658 conn(0x5638cd9aa030 msgr2=0x5638cd9aa410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 -- 172.21.15.69:0/796384658 shutdown_connections 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9ca6933640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c880a2cf0 0x7f9c880a51b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c90005e30 tx=0x7f9c90005c50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.091+0000 7f9caa794740 1 -- 172.21.15.69:0/796384658 wait complete. 2024-08-02T17:57:15.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 1 -- start start 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 1 Processor -- start 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9b81f0 0x5638cd9c1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bd230 0x5638cd9c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bdb50 0x5638cd9c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5638cd9c4570 con 0x5638cd9b81f0 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638cd8ceb20 con 0x5638cd9bd230 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5638cd9c46f0 con 0x5638cd9bdb50 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9caa794740 10 client.0 _ll_register_callbacks cb 0x5638cd8ccbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6132640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9b81f0 0x5638cd9c1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6132640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9b81f0 0x5638cd9c1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35310/0 (socket says 172.21.15.69:35310) 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6132640 1 -- 172.21.15.69:0/2789202126 learned_addr learned my addr 172.21.15.69:0/2789202126 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6933640 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bdb50 0x5638cd9c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca5931640 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bd230 0x5638cd9c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6132640 1 -- 172.21.15.69:0/2789202126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bdb50 msgr2=0x5638cd9c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6132640 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bdb50 0x5638cd9c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6132640 1 -- 172.21.15.69:0/2789202126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bd230 msgr2=0x5638cd9c1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6132640 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bd230 0x5638cd9c1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.092+0000 7f9ca6132640 1 -- 172.21.15.69:0/2789202126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cd6853c0 con 0x5638cd9b81f0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9ca6132640 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9b81f0 0x5638cd9c1140 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800bfe0 tx=0x7f9c980103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f720bf6f3171345c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9c85ffb640 1 -- 172.21.15.69:0/2789202126 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98028060 con 0x5638cd9b81f0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9c85ffb640 1 -- 172.21.15.69:0/2789202126 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f9c98011ce0 con 0x5638cd9b81f0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 1 -- 172.21.15.69:0/2789202126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c98011920 con 0x5638cd9b81f0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9c85ffb640 1 -- 172.21.15.69:0/2789202126 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c9802f050 con 0x5638cd9b81f0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 10 client.18087 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 1 -- 172.21.15.69:0/2789202126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638cd9d0320 con 0x5638cd9b81f0 2024-08-02T17:57:15.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9c85ffb640 1 -- 172.21.15.69:0/2789202126 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c98011920 con 0x5638cd9b81f0 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9c85ffb640 1 -- 172.21.15.69:0/2789202126 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c9801a460 con 0x5638cd9b81f0 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 10 client.18087 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 10 client.18087 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 20 client.18087 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5638cd9d1780 0x5638cd9d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 4 client.18087 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.093+0000 7f9caa794740 1 -- 172.21.15.69:0/2789202126 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638cd920fb0 con 0x5638cd9d1780 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.094+0000 7f9ca5931640 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5638cd9d1780 0x5638cd9d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.094+0000 7f9ca5931640 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5638cd9d1780 0x5638cd9d3b70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55cc28e4c5f25fcd server_cookie=c4020b41345cdd55 in_seq=0 out_seq=0 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.094+0000 7f9c85ffb640 10 client.18087 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9c85ffb640 1 -- 172.21.15.69:0/2789202126 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5638cd920fb0 con 0x5638cd9d1780 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9c85ffb640 10 client.18087 handle_command_reply: tid=0 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 client.18087 shutdown 2024-08-02T17:57:15.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 -- 172.21.15.69:0/2789202126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9b81f0 msgr2=0x5638cd9c1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9b81f0 0x5638cd9c1140 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800bfe0 tx=0x7f9c980103d0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 -- 172.21.15.69:0/2789202126 shutdown_connections 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5638cd9d1780 0x5638cd9d3b70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bdb50 0x5638cd9c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9c85ffb640 0 client.18087 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bd230 0x5638cd9c1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 --2- 172.21.15.69:0/2789202126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9b81f0 0x5638cd9c1140 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 -- 172.21.15.69:0/2789202126 >> 172.21.15.69:0/2789202126 conn(0x5638cd9aa030 msgr2=0x5638cd9a95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 -- 172.21.15.69:0/2789202126 shutdown_connections 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 1 -- 172.21.15.69:0/2789202126 wait complete. 2024-08-02T17:57:15.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.095+0000 7f9caa794740 20 client.18087 trim_cache size 0 max 16384 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.108+0000 7f9caa794740 1 Processor -- start 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.108+0000 7f9caa794740 1 -- start start 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.108+0000 7f9caa794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9b81f0 0x5638cd9b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.108+0000 7f9caa794740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bd230 0x5638cd9bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.108+0000 7f9caa794740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bdb50 0x5638cd9bc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.108+0000 7f9caa794740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01af6a0 con 0x5638cd9bd230 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.108+0000 7f9caa794740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01acf70 con 0x5638cd9bdb50 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.108+0000 7f9caa794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01aadc0 con 0x5638cd9b81f0 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bd230 0x5638cd9bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bd230 0x5638cd9bc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35322/0 (socket says 172.21.15.69:35322) 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 -- 172.21.15.69:0/1608941644 learned_addr learned my addr 172.21.15.69:0/1608941644 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:15.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca6132640 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9b81f0 0x5638cd9b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca6933640 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bdb50 0x5638cd9bc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 -- 172.21.15.69:0/1608941644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9b81f0 msgr2=0x5638cd9b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9b81f0 0x5638cd9b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 -- 172.21.15.69:0/1608941644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bdb50 msgr2=0x5638cd9bc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bdb50 0x5638cd9bc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 -- 172.21.15.69:0/1608941644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cd9ab690 con 0x5638cd9bd230 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca6132640 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9b81f0 0x5638cd9b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca6933640 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bdb50 0x5638cd9bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9ca5931640 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bd230 0x5638cd9bc2d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9c940058c0 tx=0x7f9c9400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae87eb9d00de3299 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9c87fff640 1 -- 172.21.15.69:0/1608941644 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94020040 con 0x5638cd9bd230 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9c87fff640 1 -- 172.21.15.69:0/1608941644 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c9409fa40 con 0x5638cd9bd230 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.109+0000 7f9c87fff640 1 -- 172.21.15.69:0/1608941644 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94084bd0 con 0x5638cd9bd230 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 -- 172.21.15.69:0/1608941644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bd230 msgr2=0x5638cd9bc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bd230 0x5638cd9bc2d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f9c940058c0 tx=0x7f9c9400bf60 comp rx=0 tx=0).stop 2024-08-02T17:57:15.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 -- 172.21.15.69:0/1608941644 shutdown_connections 2024-08-02T17:57:15.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bd230 0x5638cd9bc2d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9bdb50 0x5638cd9bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 --2- 172.21.15.69:0/1608941644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9b81f0 0x5638cd9b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 -- 172.21.15.69:0/1608941644 >> 172.21.15.69:0/1608941644 conn(0x5638cd9aa030 msgr2=0x5638cd9bfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:15.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 -- 172.21.15.69:0/1608941644 shutdown_connections 2024-08-02T17:57:15.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 -- 172.21.15.69:0/1608941644 wait complete. 2024-08-02T17:57:15.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 -- start start 2024-08-02T17:57:15.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.110+0000 7f9caa794740 1 Processor -- start 2024-08-02T17:57:15.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9caa794740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 0x5638cd9c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9caa794740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9caa794740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9d8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9caa794740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01ab430 con 0x5638cd9bdb50 2024-08-02T17:57:15.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9caa794740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01ac8a0 con 0x5638cd9b81f0 2024-08-02T17:57:15.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9caa794740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca01abb00 con 0x5638cd9bd230 2024-08-02T17:57:15.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9caa794740 10 client.0 _ll_register_callbacks cb 0x7f9c90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:15.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6933640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9d8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6132640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 0x5638cd9c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6933640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9d8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35334/0 (socket says 172.21.15.69:35334) 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6933640 1 -- 172.21.15.69:0/3713325514 learned_addr learned my addr 172.21.15.69:0/3713325514 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca5931640 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6132640 1 -- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 msgr2=0x5638cd9ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6132640 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6132640 1 -- 172.21.15.69:0/3713325514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 msgr2=0x5638cd9d8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6132640 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9d8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6132640 1 -- 172.21.15.69:0/3713325514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cd9d8c30 con 0x5638cd9b81f0 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6933640 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9d8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca5931640 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9ca6132640 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 0x5638cd9c9b60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c010 tx=0x7f9c9800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3799b339b94fcac server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:15.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9c85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.111+0000 7f9c85ffb640 1 -- 172.21.15.69:0/3713325514 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c98028050 con 0x5638cd9b81f0 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 1 -- 172.21.15.69:0/3713325514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c9409fa40 con 0x5638cd9b81f0 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9c85ffb640 1 -- 172.21.15.69:0/3713325514 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f9c9800b2c0 con 0x5638cd9b81f0 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9c85ffb640 1 -- 172.21.15.69:0/3713325514 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c98015050 con 0x5638cd9b81f0 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 10 client.27560 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 1 -- 172.21.15.69:0/3713325514 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5638cda3ba00 con 0x5638cd9b81f0 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9c85ffb640 1 -- 172.21.15.69:0/3713325514 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c9409fa40 con 0x5638cd9b81f0 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9c85ffb640 1 -- 172.21.15.69:0/3713325514 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f9c98011ce0 con 0x5638cd9b81f0 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 10 client.27560 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 10 client.27560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 20 client.27560 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5638cda3c0c0 0x5638cda3e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 4 client.27560 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:57:15.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9caa794740 1 -- 172.21.15.69:0/3713325514 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638cd91b4c0 con 0x5638cda3c0c0 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.112+0000 7f9ca5931640 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5638cda3c0c0 0x5638cda3e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9ca5931640 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5638cda3c0c0 0x5638cda3e4e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45a511d819d9fe79 server_cookie=a8736261bcb50ea9 in_seq=0 out_seq=0 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9c85ffb640 10 client.27560 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9c85ffb640 1 -- 172.21.15.69:0/3713325514 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638cd91b4c0 con 0x5638cda3c0c0 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9c85ffb640 10 client.27560 handle_command_reply: tid=0 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 client.27560 shutdown 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 -- 172.21.15.69:0/3713325514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 msgr2=0x5638cd9c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 0x5638cd9c9b60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800c010 tx=0x7f9c9800b8b0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 -- 172.21.15.69:0/3713325514 shutdown_connections 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5638cda3c0c0 0x5638cda3e4e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638cd9bdb50 0x5638cd9d8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9c85ffb640 0 client.27560 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638cd9bd230 0x5638cd9ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 --2- 172.21.15.69:0/3713325514 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638cd9b81f0 0x5638cd9c9b60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 -- 172.21.15.69:0/3713325514 >> 172.21.15.69:0/3713325514 conn(0x5638cd9aa030 msgr2=0x5638cd9bc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:15.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 -- 172.21.15.69:0/3713325514 shutdown_connections 2024-08-02T17:57:15.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 1 -- 172.21.15.69:0/3713325514 wait complete. 2024-08-02T17:57:15.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.113+0000 7f9caa794740 20 client.27560 trim_cache size 0 max 16384 2024-08-02T17:57:15.121 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 -- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c880a2cf0 msgr2=0x7f9c880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c880a2cf0 0x7f9c880a51b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9c90005e30 tx=0x7f9c90005c50 comp rx=0 tx=0).stop 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 -- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 msgr2=0x7f9ca0198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 0x7f9ca0198670 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400d950 tx=0x7f9c9400de10 comp rx=0 tx=0).stop 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 -- 172.21.15.69:0/522108023 shutdown_connections 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9ca0102250 0x7f9ca0198670 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c880a2cf0 0x7f9c880a51b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ca01078b0 0x7f9ca019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 --2- 172.21.15.69:0/522108023 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9ca01018b0 0x7f9ca0198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 -- 172.21.15.69:0/522108023 >> 172.21.15.69:0/522108023 conn(0x7f9ca00fd5f0 msgr2=0x7f9ca00fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:15.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 -- 172.21.15.69:0/522108023 shutdown_connections 2024-08-02T17:57:15.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:15.114+0000 7f9ca8396640 1 -- 172.21.15.69:0/522108023 wait complete. 2024-08-02T17:57:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:14 smithi083 ceph-mon[37031]: pgmap v336: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-08-02T17:57:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:14 smithi083 ceph-mon[37031]: osdmap e105: 12 total, 12 up, 12 in 2024-08-02T17:57:15.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:14 smithi077 ceph-mon[37108]: pgmap v336: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-08-02T17:57:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:14 smithi077 ceph-mon[37108]: osdmap e105: 12 total, 12 up, 12 in 2024-08-02T17:57:15.946 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:57:16.198 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:16.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 -- 172.21.15.69:0/1362091863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0106c40 msgr2=0x7f88d010e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1362091863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0106c40 0x7f88d010e7c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009ca0 tx=0x7f88c802a8c0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 -- 172.21.15.69:0/1362091863 shutdown_connections 2024-08-02T17:57:16.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1362091863 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0106c40 0x7f88d010e7c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.662 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1362091863 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88d01009d0 0x7f88d0106510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1362091863 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d0100020 0x7f88d0100400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 -- 172.21.15.69:0/1362091863 >> 172.21.15.69:0/1362091863 conn(0x7f88d00fbc10 msgr2=0x7f88d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 -- 172.21.15.69:0/1362091863 shutdown_connections 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 -- 172.21.15.69:0/1362091863 wait complete. 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.641+0000 7f88d55aa640 1 Processor -- start 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88d55aa640 1 -- start start 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88d55aa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0100020 0x7f88d0198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88d55aa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d01009d0 0x7f88d0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88d55aa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88d0106c40 0x7f88d019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88d55aa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0111cc0 con 0x7f88d0100020 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88d55aa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0111b40 con 0x7f88d0106c40 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88d55aa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0111e40 con 0x7f88d01009d0 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0100020 0x7f88d0198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0100020 0x7f88d0198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35350/0 (socket says 172.21.15.69:35350) 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ceffd640 1 -- 172.21.15.69:0/1432947615 learned_addr learned my addr 172.21.15.69:0/1432947615 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d01009d0 0x7f88d0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88d0106c40 0x7f88d019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ceffd640 1 -- 172.21.15.69:0/1432947615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d01009d0 msgr2=0x7f88d0198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ceffd640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d01009d0 0x7f88d0198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ceffd640 1 -- 172.21.15.69:0/1432947615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88d0106c40 msgr2=0x7f88d019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ceffd640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88d0106c40 0x7f88d019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88ceffd640 1 -- 172.21.15.69:0/1432947615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d019d0f0 con 0x7f88d0100020 2024-08-02T17:57:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.642+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88d0106c40 0x7f88d019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.643+0000 7f88ceffd640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0100020 0x7f88d0198140 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f88c000b4a0 tx=0x7f88c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.643+0000 7f88b7fff640 1 -- 172.21.15.69:0/1432947615 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0014070 con 0x7f88d0100020 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.643+0000 7f88d55aa640 1 -- 172.21.15.69:0/1432947615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d019d3e0 con 0x7f88d0100020 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.643+0000 7f88b7fff640 1 -- 172.21.15.69:0/1432947615 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c0010af0 con 0x7f88d0100020 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.643+0000 7f88b7fff640 1 -- 172.21.15.69:0/1432947615 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0017d10 con 0x7f88d0100020 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.643+0000 7f88d55aa640 1 -- 172.21.15.69:0/1432947615 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d01a4cc0 con 0x7f88d0100020 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.645+0000 7f88b7fff640 1 -- 172.21.15.69:0/1432947615 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f88c0017070 con 0x7f88d0100020 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.645+0000 7f88d79a8740 1 Processor -- start 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.645+0000 7f88d79a8740 1 -- start start 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.645+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557943122ff0 0x5579431233d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x55794312dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794312e1d0 0x5579431305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88d79a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557943088a40 con 0x55794312e1d0 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88d79a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5579430209d0 con 0x557943122ff0 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88d79a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557943084a00 con 0x557943123910 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794312e1d0 0x5579431305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794312e1d0 0x5579431305c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35354/0 (socket says 172.21.15.69:35354) 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88b7fff640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f88b80a2cf0 0x7f88b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 -- 172.21.15.69:0/1632184770 learned_addr learned my addr 172.21.15.69:0/1632184770 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557943122ff0 0x5579431233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ceffd640 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x55794312dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f88b80a2cf0 0x7f88b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 -- 172.21.15.69:0/1632184770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 msgr2=0x55794312dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x55794312dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 -- 172.21.15.69:0/1632184770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557943122ff0 msgr2=0x5579431233d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557943122ff0 0x5579431233d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 -- 172.21.15.69:0/1632184770 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557943027650 con 0x55794312e1d0 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88b7fff640 1 -- 172.21.15.69:0/1432947615 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f88c00cebd0 con 0x7f88d0100020 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557943122ff0 0x5579431233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794312e1d0 0x5579431305c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f88c400cc40 tx=0x7f88c400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae9e654787b35c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f88b80a2cf0 0x7f88b80a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009bf0 tx=0x7f88c80095c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.646+0000 7f88b57fa640 1 -- 172.21.15.69:0/1632184770 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4014070 con 0x55794312e1d0 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88b57fa640 1 -- 172.21.15.69:0/1632184770 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c40047a0 con 0x55794312e1d0 2024-08-02T17:57:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88b57fa640 1 -- 172.21.15.69:0/1632184770 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4010760 con 0x55794312e1d0 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 -- 172.21.15.69:0/1632184770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794312e1d0 msgr2=0x5579431305c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794312e1d0 0x5579431305c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f88c400cc40 tx=0x7f88c400e720 comp rx=0 tx=0).stop 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 -- 172.21.15.69:0/1632184770 shutdown_connections 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55794312e1d0 0x5579431305c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x55794312dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1632184770 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557943122ff0 0x5579431233d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 -- 172.21.15.69:0/1632184770 >> 172.21.15.69:0/1632184770 conn(0x55794311f580 msgr2=0x55794311f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 -- 172.21.15.69:0/1632184770 shutdown_connections 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.647+0000 7f88d79a8740 1 -- 172.21.15.69:0/1632184770 wait complete. 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 1 -- start start 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 1 Processor -- start 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 0x55794312b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x55794312b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x55794312baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557943130da0 con 0x557943123910 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557943043b20 con 0x55794312e1d0 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557943130f20 con 0x557943122ff0 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88d79a8740 10 client.0 _ll_register_callbacks cb 0x557943041bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x55794312b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x55794312b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35360/0 (socket says 172.21.15.69:35360) 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 -- 172.21.15.69:0/3235110167 learned_addr learned my addr 172.21.15.69:0/3235110167 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x55794312baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ceffd640 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 0x55794312b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 -- 172.21.15.69:0/3235110167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 msgr2=0x55794312b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 0x55794312b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 -- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 msgr2=0x55794312baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x55794312baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 -- 172.21.15.69:0/3235110167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557942dfa3c0 con 0x557943123910 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x55794312baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x55794312b560 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f88c4005da0 tx=0x7f88c4010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d0a54d68e8e581 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.648+0000 7f88b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88b2ff5640 1 -- 172.21.15.69:0/3235110167 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4027060 con 0x557943123910 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88b2ff5640 1 -- 172.21.15.69:0/3235110167 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f88c4018620 con 0x557943123910 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 1 -- 172.21.15.69:0/3235110167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c40047a0 con 0x557943123910 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88b2ff5640 1 -- 172.21.15.69:0/3235110167 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88c402e050 con 0x557943123910 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 10 client.18105 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 1 -- 172.21.15.69:0/3235110167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55794313c930 con 0x557943123910 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88b2ff5640 1 -- 172.21.15.69:0/3235110167 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c40047a0 con 0x557943123910 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88b2ff5640 1 -- 172.21.15.69:0/3235110167 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88c4022670 con 0x557943123910 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 10 client.18105 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 10 client.18105 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 20 client.18105 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55794313ddf0 0x5579431401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 4 client.18105 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.649+0000 7f88d79a8740 1 -- 172.21.15.69:0/3235110167 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557943095fb0 con 0x55794313ddf0 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.650+0000 7f88ceffd640 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55794313ddf0 0x5579431401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.650+0000 7f88ceffd640 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55794313ddf0 0x5579431401e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef24af7266661609 server_cookie=f518ae797a6e692a in_seq=0 out_seq=0 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.650+0000 7f88b2ff5640 10 client.18105 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88b2ff5640 1 -- 172.21.15.69:0/3235110167 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557943095fb0 con 0x55794313ddf0 2024-08-02T17:57:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88b2ff5640 10 client.18105 handle_command_reply: tid=0 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 client.18105 shutdown 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 -- 172.21.15.69:0/3235110167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 msgr2=0x55794312b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x55794312b560 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f88c4005da0 tx=0x7f88c4010cf0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 -- 172.21.15.69:0/3235110167 shutdown_connections 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55794313ddf0 0x5579431401e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x55794312b560 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x55794312baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 --2- 172.21.15.69:0/3235110167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 0x55794312b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 -- 172.21.15.69:0/3235110167 >> 172.21.15.69:0/3235110167 conn(0x55794311f580 msgr2=0x557943125260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88b2ff5640 0 client.18105 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 -- 172.21.15.69:0/3235110167 shutdown_connections 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 1 -- 172.21.15.69:0/3235110167 wait complete. 2024-08-02T17:57:16.672 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.651+0000 7f88d79a8740 20 client.18105 trim_cache size 0 max 16384 2024-08-02T17:57:16.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88d79a8740 1 Processor -- start 2024-08-02T17:57:16.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88d79a8740 1 -- start start 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 0x55794312ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x5579431457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x557943145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88d79a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01b1790 con 0x557943123910 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88d79a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01af060 con 0x55794312e1d0 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88d79a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01aceb0 con 0x557943122ff0 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x5579431457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ce7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x5579431457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35370/0 (socket says 172.21.15.69:35370) 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ce7fc640 1 -- 172.21.15.69:0/1585665243 learned_addr learned my addr 172.21.15.69:0/1585665243 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x557943145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ceffd640 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 0x55794312ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ce7fc640 1 -- 172.21.15.69:0/1585665243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 msgr2=0x55794312ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:16.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 0x55794312ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ce7fc640 1 -- 172.21.15.69:0/1585665243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 msgr2=0x557943145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x557943145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88ce7fc640 1 -- 172.21.15.69:0/1585665243 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794312bff0 con 0x557943123910 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.664+0000 7f88cf7fe640 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x557943145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x5579431457b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f88c4022be0 tx=0x7f88c400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc498ec291650a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88b4ff9640 1 -- 172.21.15.69:0/1585665243 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4014070 con 0x557943123910 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88b4ff9640 1 -- 172.21.15.69:0/1585665243 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c4004780 con 0x557943123910 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88b4ff9640 1 -- 172.21.15.69:0/1585665243 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c40052a0 con 0x557943123910 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 -- 172.21.15.69:0/1585665243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 msgr2=0x5579431457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x5579431457b0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f88c4022be0 tx=0x7f88c400e720 comp rx=0 tx=0).stop 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 -- 172.21.15.69:0/1585665243 shutdown_connections 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943123910 0x5579431457b0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x557943145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1585665243 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943122ff0 0x55794312ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 -- 172.21.15.69:0/1585665243 >> 172.21.15.69:0/1585665243 conn(0x55794311f580 msgr2=0x55794312c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 -- 172.21.15.69:0/1585665243 shutdown_connections 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.665+0000 7f88d79a8740 1 -- 172.21.15.69:0/1585665243 wait complete. 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 1 -- start start 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 1 Processor -- start 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943122ff0 0x557943136140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x557943136680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x557943127290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01ad520 con 0x557943122ff0 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01ae990 con 0x55794312e1d0 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01adbf0 con 0x557943123910 2024-08-02T17:57:16.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88d79a8740 10 client.0 _ll_register_callbacks cb 0x7f88c400ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943122ff0 0x557943136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943122ff0 0x557943136140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35382/0 (socket says 172.21.15.69:35382) 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ceffd640 1 -- 172.21.15.69:0/1210460717 learned_addr learned my addr 172.21.15.69:0/1210460717 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x557943136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ceffd640 1 -- 172.21.15.69:0/1210460717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 msgr2=0x557943136680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ceffd640 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x557943136680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ceffd640 1 -- 172.21.15.69:0/1210460717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 msgr2=0x557943127290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ceffd640 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x557943127290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ceffd640 1 -- 172.21.15.69:0/1210460717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557943128ea0 con 0x557943122ff0 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.666+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x557943136680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88ceffd640 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943122ff0 0x557943136140 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f88c000bfd0 tx=0x7f88c006ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f941e13b43505f83 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:16.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88b2ff5640 1 -- 172.21.15.69:0/1210460717 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c001e040 con 0x557943122ff0 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88b2ff5640 1 -- 172.21.15.69:0/1210460717 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f88c0097460 con 0x557943122ff0 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 1 -- 172.21.15.69:0/1210460717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c4004780 con 0x557943122ff0 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88b2ff5640 1 -- 172.21.15.69:0/1210460717 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f88c009f1a0 con 0x557943122ff0 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 10 client.18111 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 1 -- 172.21.15.69:0/1210460717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5579431a7bb0 con 0x557943122ff0 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88b2ff5640 1 -- 172.21.15.69:0/1210460717 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c4004780 con 0x557943122ff0 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88b2ff5640 1 -- 172.21.15.69:0/1210460717 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f88c00a3660 con 0x557943122ff0 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 10 client.18111 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 10 client.18111 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 20 client.18111 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5579431a8380 0x5579431aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:16.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 4 client.18111 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.667+0000 7f88d79a8740 1 -- 172.21.15.69:0/1210460717 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579430904c0 con 0x5579431a8380 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5579431a8380 0x5579431aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88b2ff5640 10 client.18111 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88ce7fc640 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5579431a8380 0x5579431aa7c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82cd7336f15a82f7 server_cookie=2d2474c8b44d0d0f in_seq=0 out_seq=0 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88b2ff5640 1 -- 172.21.15.69:0/1210460717 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579430904c0 con 0x5579431a8380 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88b2ff5640 10 client.18111 handle_command_reply: tid=0 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 client.18111 shutdown 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 -- 172.21.15.69:0/1210460717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943122ff0 msgr2=0x557943136140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943122ff0 0x557943136140 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f88c000bfd0 tx=0x7f88c006ac50 comp rx=0 tx=0).stop 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 -- 172.21.15.69:0/1210460717 shutdown_connections 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x5579431a8380 0x5579431aa7c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55794312e1d0 0x557943127290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557943123910 0x557943136680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88b2ff5640 0 client.18111 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:57:16.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 --2- 172.21.15.69:0/1210460717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557943122ff0 0x557943136140 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 -- 172.21.15.69:0/1210460717 >> 172.21.15.69:0/1210460717 conn(0x55794311f580 msgr2=0x557943129fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:16.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 -- 172.21.15.69:0/1210460717 shutdown_connections 2024-08-02T17:57:16.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.668+0000 7f88d79a8740 1 -- 172.21.15.69:0/1210460717 wait complete. 2024-08-02T17:57:16.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d79a8740 20 client.18111 trim_cache size 0 max 16384 2024-08-02T17:57:16.684 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:57:16.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 -- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f88b80a2cf0 msgr2=0x7f88b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f88b80a2cf0 0x7f88b80a51b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009bf0 tx=0x7f88c80095c0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 -- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0100020 msgr2=0x7f88d0198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:16.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0100020 0x7f88d0198140 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f88c000b4a0 tx=0x7f88c000b960 comp rx=0 tx=0).stop 2024-08-02T17:57:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 -- 172.21.15.69:0/1432947615 shutdown_connections 2024-08-02T17:57:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f88b80a2cf0 0x7f88b80a51b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f88d0106c40 0x7f88d019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f88d01009d0 0x7f88d0198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 --2- 172.21.15.69:0/1432947615 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f88d0100020 0x7f88d0198140 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 -- 172.21.15.69:0/1432947615 >> 172.21.15.69:0/1432947615 conn(0x7f88d00fbc10 msgr2=0x7f88d00fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 -- 172.21.15.69:0/1432947615 shutdown_connections 2024-08-02T17:57:16.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:16.669+0000 7f88d55aa640 1 -- 172.21.15.69:0/1432947615 wait complete. 2024-08-02T17:57:17.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:16 smithi083 ceph-mon[37031]: pgmap v338: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-08-02T17:57:17.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:16 smithi077 ceph-mon[37108]: pgmap v338: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-08-02T17:57:17.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:16 smithi069 ceph-mon[32746]: pgmap v338: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-08-02T17:57:17.471 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:57:17.729 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:18.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.301+0000 7f7fcbce0640 1 -- 172.21.15.69:0/2017547569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41067b0 msgr2=0x7f7fc4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.301+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/2017547569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41067b0 0x7f7fc4106b90 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009930 tx=0x7f7fb802a420 comp rx=0 tx=0).stop 2024-08-02T17:57:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 -- 172.21.15.69:0/2017547569 shutdown_connections 2024-08-02T17:57:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/2017547569 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc4101150 0x7f7fc410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/2017547569 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fc41007b0 0x7f7fc4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/2017547569 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41067b0 0x7f7fc4106b90 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.323 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 -- 172.21.15.69:0/2017547569 >> 172.21.15.69:0/2017547569 conn(0x7f7fc40fc4d0 msgr2=0x7f7fc40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 -- 172.21.15.69:0/2017547569 shutdown_connections 2024-08-02T17:57:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 -- 172.21.15.69:0/2017547569 wait complete. 2024-08-02T17:57:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 Processor -- start 2024-08-02T17:57:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 -- start start 2024-08-02T17:57:18.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41007b0 0x7f7fc410e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc4101150 0x7f7fc410e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fc41067b0 0x7f7fc410eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4111cd0 con 0x7f7fc41007b0 2024-08-02T17:57:18.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4111b50 con 0x7f7fc4101150 2024-08-02T17:57:18.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.302+0000 7f7fcbce0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4111e50 con 0x7f7fc41067b0 2024-08-02T17:57:18.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41007b0 0x7f7fc410e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41007b0 0x7f7fc410e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35392/0 (socket says 172.21.15.69:35392) 2024-08-02T17:57:18.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 -- 172.21.15.69:0/1302217964 learned_addr learned my addr 172.21.15.69:0/1302217964 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:18.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc927b640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc4101150 0x7f7fc410e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fca27d640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fc41067b0 0x7f7fc410eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 -- 172.21.15.69:0/1302217964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fc41067b0 msgr2=0x7f7fc410eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fc41067b0 0x7f7fc410eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 -- 172.21.15.69:0/1302217964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc4101150 msgr2=0x7f7fc410e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc4101150 0x7f7fc410e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 -- 172.21.15.69:0/1302217964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc41a6690 con 0x7f7fc41007b0 2024-08-02T17:57:18.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fca27d640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fc41067b0 0x7f7fc410eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:18.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41007b0 0x7f7fc410e110 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009230 tx=0x7f7fb8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:18.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fb6ffd640 1 -- 172.21.15.69:0/1302217964 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb803f030 con 0x7f7fc41007b0 2024-08-02T17:57:18.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fb6ffd640 1 -- 172.21.15.69:0/1302217964 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fb8003ad0 con 0x7f7fc41007b0 2024-08-02T17:57:18.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fcbce0640 1 -- 172.21.15.69:0/1302217964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc41a6920 con 0x7f7fc41007b0 2024-08-02T17:57:18.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fb6ffd640 1 -- 172.21.15.69:0/1302217964 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8035070 con 0x7f7fc41007b0 2024-08-02T17:57:18.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.303+0000 7f7fcbce0640 1 -- 172.21.15.69:0/1302217964 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a6e00 con 0x7f7fc41007b0 2024-08-02T17:57:18.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.306+0000 7f7fb6ffd640 1 -- 172.21.15.69:0/1302217964 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7fb8042030 con 0x7f7fc41007b0 2024-08-02T17:57:18.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.306+0000 7f7fce0de740 1 Processor -- start 2024-08-02T17:57:18.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fce0de740 1 -- start start 2024-08-02T17:57:18.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11581ff0 0x562b115823d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11582910 0x562b1158cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b1158f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fce0de740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b114e7a40 con 0x562b1158d1d0 2024-08-02T17:57:18.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fce0de740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b1147f9d0 con 0x562b11582910 2024-08-02T17:57:18.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fce0de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b114e3a00 con 0x562b11581ff0 2024-08-02T17:57:18.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc927b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b1158f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fb6ffd640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7fb00a2cf0 0x7f7fb00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc927b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b1158f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35396/0 (socket says 172.21.15.69:35396) 2024-08-02T17:57:18.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc927b640 1 -- 172.21.15.69:0/4286622323 learned_addr learned my addr 172.21.15.69:0/4286622323 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:18.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fca27d640 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11581ff0 0x562b115823d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fca27d640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7fb00a2cf0 0x7f7fb00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fb6ffd640 1 -- 172.21.15.69:0/1302217964 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f7fb80efd60 con 0x7f7fc41007b0 2024-08-02T17:57:18.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11582910 0x562b1158cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc927b640 1 -- 172.21.15.69:0/4286622323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11581ff0 msgr2=0x562b115823d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:18.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc927b640 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11581ff0 0x562b115823d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc927b640 1 -- 172.21.15.69:0/4286622323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11582910 msgr2=0x562b1158cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:18.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc927b640 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11582910 0x562b1158cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc927b640 1 -- 172.21.15.69:0/4286622323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b11486650 con 0x562b1158d1d0 2024-08-02T17:57:18.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.307+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11582910 0x562b1158cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:18.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.308+0000 7f7fca27d640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7fb00a2cf0 0x7f7fb00a51b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005fd0 tx=0x7f7fbc005c80 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:18.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.308+0000 7f7fc927b640 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b1158f5c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000bcb0 tx=0x7f7fc00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d10ecadd0033cd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:18.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.308+0000 7f7faffff640 1 -- 172.21.15.69:0/4286622323 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0004030 con 0x562b1158d1d0 2024-08-02T17:57:18.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.308+0000 7f7faffff640 1 -- 172.21.15.69:0/4286622323 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc00041d0 con 0x562b1158d1d0 2024-08-02T17:57:18.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.308+0000 7f7faffff640 1 -- 172.21.15.69:0/4286622323 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc00117b0 con 0x562b1158d1d0 2024-08-02T17:57:18.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.308+0000 7f7fce0de740 1 -- 172.21.15.69:0/4286622323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 msgr2=0x562b1158f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b1158f5c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000bcb0 tx=0x7f7fc00107a0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 -- 172.21.15.69:0/4286622323 shutdown_connections 2024-08-02T17:57:18.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b1158f5c0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11582910 0x562b1158cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 --2- 172.21.15.69:0/4286622323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11581ff0 0x562b115823d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 -- 172.21.15.69:0/4286622323 >> 172.21.15.69:0/4286622323 conn(0x562b1157e580 msgr2=0x562b1157e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:18.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 -- 172.21.15.69:0/4286622323 shutdown_connections 2024-08-02T17:57:18.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 -- 172.21.15.69:0/4286622323 wait complete. 2024-08-02T17:57:18.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 -- start start 2024-08-02T17:57:18.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 Processor -- start 2024-08-02T17:57:18.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11581ff0 0x562b1158a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b1158a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b1158d1d0 0x562b1158aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562b1158fda0 con 0x562b11581ff0 2024-08-02T17:57:18.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562b114a2b20 con 0x562b1158d1d0 2024-08-02T17:57:18.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562b1158ff20 con 0x562b11582910 2024-08-02T17:57:18.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fce0de740 10 client.0 _ll_register_callbacks cb 0x562b114a0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:18.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fc9a7c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11581ff0 0x562b1158a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fc9a7c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11581ff0 0x562b1158a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35410/0 (socket says 172.21.15.69:35410) 2024-08-02T17:57:18.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.309+0000 7f7fc9a7c640 1 -- 172.21.15.69:0/3074279457 learned_addr learned my addr 172.21.15.69:0/3074279457 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:18.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fc927b640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b1158a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fca27d640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b1158d1d0 0x562b1158aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fc9a7c640 1 -- 172.21.15.69:0/3074279457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 msgr2=0x562b1158a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:18.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b1158a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fc9a7c640 1 -- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b1158d1d0 msgr2=0x562b1158aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:18.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b1158d1d0 0x562b1158aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fc9a7c640 1 -- 172.21.15.69:0/3074279457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b112593c0 con 0x562b11581ff0 2024-08-02T17:57:18.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fca27d640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b1158d1d0 0x562b1158aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:18.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fc927b640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b1158a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:18.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11581ff0 0x562b1158a010 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80b8840 tx=0x7f7fb80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c4e5290e319120 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:18.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:18.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fad7fa640 1 -- 172.21.15.69:0/3074279457 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8049040 con 0x562b11581ff0 2024-08-02T17:57:18.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fad7fa640 1 -- 172.21.15.69:0/3074279457 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f7fb80b84e0 con 0x562b11581ff0 2024-08-02T17:57:18.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fce0de740 1 -- 172.21.15.69:0/3074279457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc00041d0 con 0x562b11581ff0 2024-08-02T17:57:18.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fad7fa640 1 -- 172.21.15.69:0/3074279457 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fb80c4ae0 con 0x562b11581ff0 2024-08-02T17:57:18.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fce0de740 10 client.18123 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:18.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fce0de740 1 -- 172.21.15.69:0/3074279457 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b1159b940 con 0x562b11581ff0 2024-08-02T17:57:18.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.310+0000 7f7fad7fa640 1 -- 172.21.15.69:0/3074279457 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc00041d0 con 0x562b11581ff0 2024-08-02T17:57:18.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fad7fa640 1 -- 172.21.15.69:0/3074279457 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7fb80c4650 con 0x562b11581ff0 2024-08-02T17:57:18.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fce0de740 10 client.18123 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:18.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fce0de740 10 client.18123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:18.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fce0de740 20 client.18123 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:18.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fce0de740 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562b1159cd50 0x562b1159f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fce0de740 4 client.18123 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:18.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fce0de740 1 -- 172.21.15.69:0/3074279457 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b114f4fb0 con 0x562b1159cd50 2024-08-02T17:57:18.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fc927b640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562b1159cd50 0x562b1159f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fc927b640 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562b1159cd50 0x562b1159f180 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230ae035b1e0d735 server_cookie=f01d9fe21164a5c3 in_seq=0 out_seq=0 2024-08-02T17:57:18.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.311+0000 7f7fad7fa640 10 client.18123 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:18.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fad7fa640 1 -- 172.21.15.69:0/3074279457 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b114f4fb0 con 0x562b1159cd50 2024-08-02T17:57:18.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fad7fa640 10 client.18123 handle_command_reply: tid=0 2024-08-02T17:57:18.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 client.18123 shutdown 2024-08-02T17:57:18.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 -- 172.21.15.69:0/3074279457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11581ff0 msgr2=0x562b1158a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11581ff0 0x562b1158a010 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7fb80b8840 tx=0x7f7fb80045f0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 -- 172.21.15.69:0/3074279457 shutdown_connections 2024-08-02T17:57:18.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562b1159cd50 0x562b1159f180 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b1158d1d0 0x562b1158aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b1158a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 --2- 172.21.15.69:0/3074279457 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11581ff0 0x562b1158a010 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fad7fa640 0 client.18123 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:18.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 -- 172.21.15.69:0/3074279457 >> 172.21.15.69:0/3074279457 conn(0x562b1157e580 msgr2=0x562b11584260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:18.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 -- 172.21.15.69:0/3074279457 shutdown_connections 2024-08-02T17:57:18.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 1 -- 172.21.15.69:0/3074279457 wait complete. 2024-08-02T17:57:18.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.312+0000 7f7fce0de740 20 client.18123 trim_cache size 0 max 16384 2024-08-02T17:57:18.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.325+0000 7f7fce0de740 1 Processor -- start 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.325+0000 7f7fce0de740 1 -- start start 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.325+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 0x562b11584a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.325+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11582910 0x562b115a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.325+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1158d1d0 0x562b115a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.325+0000 7f7fce0de740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41b3950 con 0x562b11582910 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.325+0000 7f7fce0de740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41b1220 con 0x562b11581ff0 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.325+0000 7f7fce0de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41af070 con 0x562b1158d1d0 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11582910 0x562b115a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11582910 0x562b115a4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35422/0 (socket says 172.21.15.69:35422) 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 -- 172.21.15.69:0/2818353796 learned_addr learned my addr 172.21.15.69:0/2818353796 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 0x562b11584a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fca27d640 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1158d1d0 0x562b115a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 -- 172.21.15.69:0/2818353796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1158d1d0 msgr2=0x562b115a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1158d1d0 0x562b115a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 -- 172.21.15.69:0/2818353796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 msgr2=0x562b11584a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 0x562b11584a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 -- 172.21.15.69:0/2818353796 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b11583500 con 0x562b11582910 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7fc927b640 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11582910 0x562b115a4a30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000bcb0 tx=0x7f7fc000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2260d5067d008df1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7faf7fe640 1 -- 172.21.15.69:0/2818353796 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0005240 con 0x562b11582910 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7faf7fe640 1 -- 172.21.15.69:0/2818353796 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0003ad0 con 0x562b11582910 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.326+0000 7f7faf7fe640 1 -- 172.21.15.69:0/2818353796 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0011730 con 0x562b11582910 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 -- 172.21.15.69:0/2818353796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11582910 msgr2=0x562b115a4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11582910 0x562b115a4a30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000bcb0 tx=0x7f7fc000b390 comp rx=0 tx=0).stop 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 -- 172.21.15.69:0/2818353796 shutdown_connections 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b11582910 0x562b115a4a30 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b1158d1d0 0x562b115a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2818353796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 0x562b11584a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 -- 172.21.15.69:0/2818353796 >> 172.21.15.69:0/2818353796 conn(0x562b1157e580 msgr2=0x562b115888e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 -- 172.21.15.69:0/2818353796 shutdown_connections 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 -- 172.21.15.69:0/2818353796 wait complete. 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 -- start start 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.327+0000 7f7fce0de740 1 Processor -- start 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 0x562b11586a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b11586f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fce0de740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b115941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fce0de740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41af6e0 con 0x562b1158d1d0 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fce0de740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41b0b50 con 0x562b11581ff0 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fce0de740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41afdb0 con 0x562b11582910 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fce0de740 10 client.0 _ll_register_callbacks cb 0x7f7fb8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:18.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b115941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b115941a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35436/0 (socket says 172.21.15.69:35436) 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 -- 172.21.15.69:0/2236244240 learned_addr learned my addr 172.21.15.69:0/2236244240 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fc9a7c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 0x562b11586a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fc927b640 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b11586f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 -- 172.21.15.69:0/2236244240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 msgr2=0x562b11586f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b11586f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 -- 172.21.15.69:0/2236244240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 msgr2=0x562b11586a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 0x562b11586a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 -- 172.21.15.69:0/2236244240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b115946e0 con 0x562b1158d1d0 2024-08-02T17:57:18.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fc927b640 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b11586f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fca27d640 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b115941a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00fb30 tx=0x7f7fbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e744866aea9b5b4a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fad7fa640 1 -- 172.21.15.69:0/2236244240 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00d400 con 0x562b1158d1d0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fad7fa640 1 -- 172.21.15.69:0/2236244240 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f7fbc00d5a0 con 0x562b1158d1d0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.328+0000 7f7fce0de740 1 -- 172.21.15.69:0/2236244240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fc0003ad0 con 0x562b1158d1d0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fad7fa640 1 -- 172.21.15.69:0/2236244240 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7fbc033050 con 0x562b1158d1d0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fce0de740 10 client.18129 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fce0de740 1 -- 172.21.15.69:0/2236244240 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562b115959b0 con 0x562b1158d1d0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fad7fa640 1 -- 172.21.15.69:0/2236244240 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc0003ad0 con 0x562b1158d1d0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fad7fa640 1 -- 172.21.15.69:0/2236244240 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7fbc01b040 con 0x562b1158d1d0 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fce0de740 10 client.18129 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:18.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fce0de740 10 client.18129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fce0de740 20 client.18129 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562b11607c10 0x562b1160a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fce0de740 4 client.18129 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fce0de740 1 -- 172.21.15.69:0/2236244240 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b114ef4c0 con 0x562b11607c10 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.329+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562b11607c10 0x562b1160a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fc9a7c640 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562b11607c10 0x562b1160a030 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a293c0b61a558a90 server_cookie=4775bc3de75d73e in_seq=0 out_seq=0 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fad7fa640 10 client.18129 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fad7fa640 1 -- 172.21.15.69:0/2236244240 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b114ef4c0 con 0x562b11607c10 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fad7fa640 10 client.18129 handle_command_reply: tid=0 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 client.18129 shutdown 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 -- 172.21.15.69:0/2236244240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 msgr2=0x562b115941a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b115941a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00fb30 tx=0x7f7fbc004a40 comp rx=0 tx=0).stop 2024-08-02T17:57:18.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 -- 172.21.15.69:0/2236244240 shutdown_connections 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562b1158d1d0 0x562b115941a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x562b11607c10 0x562b1160a030 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562b11582910 0x562b11586f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 --2- 172.21.15.69:0/2236244240 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562b11581ff0 0x562b11586a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 -- 172.21.15.69:0/2236244240 >> 172.21.15.69:0/2236244240 conn(0x562b1157e580 msgr2=0x562b1157cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fad7fa640 0 client.18129 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 -- 172.21.15.69:0/2236244240 shutdown_connections 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 1 -- 172.21.15.69:0/2236244240 wait complete. 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fce0de740 20 client.18129 trim_cache size 0 max 16384 2024-08-02T17:57:18.357 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:57:18.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fcbce0640 1 -- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7fb00a2cf0 msgr2=0x7f7fb00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.330+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7fb00a2cf0 0x7f7fb00a51b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc005fd0 tx=0x7f7fbc005c80 comp rx=0 tx=0).stop 2024-08-02T17:57:18.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 -- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41007b0 msgr2=0x7f7fc410e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:18.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41007b0 0x7f7fc410e110 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009230 tx=0x7f7fb8004270 comp rx=0 tx=0).stop 2024-08-02T17:57:18.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 -- 172.21.15.69:0/1302217964 shutdown_connections 2024-08-02T17:57:18.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7fb00a2cf0 0x7f7fb00a51b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7fc41067b0 0x7f7fc410eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7fc4101150 0x7f7fc410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 --2- 172.21.15.69:0/1302217964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7fc41007b0 0x7f7fc410e110 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:18.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 -- 172.21.15.69:0/1302217964 >> 172.21.15.69:0/1302217964 conn(0x7f7fc40fc4d0 msgr2=0x7f7fc40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:18.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 -- 172.21.15.69:0/1302217964 shutdown_connections 2024-08-02T17:57:18.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:18.331+0000 7f7fcbce0640 1 -- 172.21.15.69:0/1302217964 wait complete. 2024-08-02T17:57:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:18 smithi083 ceph-mon[37031]: pgmap v339: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-08-02T17:57:19.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:18 smithi077 ceph-mon[37108]: pgmap v339: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-08-02T17:57:19.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:18 smithi069 ceph-mon[32746]: pgmap v339: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-08-02T17:57:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 610.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:57:20.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:57:20.058 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:57:20.317 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:57:20.324 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:20.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 -- 172.21.15.69:0/3464229876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8100fc0 msgr2=0x7f83f810d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 --2- 172.21.15.69:0/3464229876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8100fc0 0x7f83f810d4d0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009cd0 tx=0x7f83f002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 -- 172.21.15.69:0/3464229876 shutdown_connections 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 --2- 172.21.15.69:0/3464229876 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8100fc0 0x7f83f810d4d0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 --2- 172.21.15.69:0/3464229876 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83f8100600 0x7f83f8100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 --2- 172.21.15.69:0/3464229876 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f8106690 0x7f83f8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 -- 172.21.15.69:0/3464229876 >> 172.21.15.69:0/3464229876 conn(0x7f83f80fc360 msgr2=0x7f83f80fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 -- 172.21.15.69:0/3464229876 shutdown_connections 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.820+0000 7f83fc99e640 1 -- 172.21.15.69:0/3464229876 wait complete. 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83fc99e640 1 Processor -- start 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83fc99e640 1 -- start start 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83fc99e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f8100600 0x7f83f8196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83fc99e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83f8100fc0 0x7f83f8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83fc99e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8106690 0x7f83f819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83fc99e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81109f0 con 0x7f83f8106690 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83fc99e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8110870 con 0x7f83f8100fc0 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83fc99e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8110b70 con 0x7f83f8100600 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8106690 0x7f83f819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8106690 0x7f83f819b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35450/0 (socket says 172.21.15.69:35450) 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.821+0000 7f83f7fff640 1 -- 172.21.15.69:0/598203415 learned_addr learned my addr 172.21.15.69:0/598203415 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83f8100fc0 0x7f83f8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f77fe640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f8100600 0x7f83f8196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f7fff640 1 -- 172.21.15.69:0/598203415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f8100600 msgr2=0x7f83f8196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f7fff640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f8100600 0x7f83f8196d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f7fff640 1 -- 172.21.15.69:0/598203415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83f8100fc0 msgr2=0x7f83f8197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f7fff640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83f8100fc0 0x7f83f8197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f7fff640 1 -- 172.21.15.69:0/598203415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f819bd00 con 0x7f83f8106690 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83f8100fc0 0x7f83f8197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f7fff640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8106690 0x7f83f819b620 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f83f0005cc0 tx=0x7f83f0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f4ff9640 1 -- 172.21.15.69:0/598203415 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0004040 con 0x7f83f8106690 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83fc99e640 1 -- 172.21.15.69:0/598203415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f819bf90 con 0x7f83f8106690 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f4ff9640 1 -- 172.21.15.69:0/598203415 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f00046c0 con 0x7f83f8106690 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83f4ff9640 1 -- 172.21.15.69:0/598203415 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0046480 con 0x7f83f8106690 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.822+0000 7f83fc99e640 1 -- 172.21.15.69:0/598203415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81a38d0 con 0x7f83f8106690 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.824+0000 7f83f4ff9640 1 -- 172.21.15.69:0/598203415 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83f00305a0 con 0x7f83f8106690 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83ffffe740 1 Processor -- start 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83ffffe740 1 -- start start 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab753d0 0x55eb2ab757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab80070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83ffffe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55eb2ab38e40 con 0x55eb2ab805b0 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83ffffe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb2aada7f0 con 0x55eb2ab753d0 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83ffffe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb2aa81fb0 con 0x55eb2ab75cf0 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab829a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35460/0 (socket says 172.21.15.69:35460) 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83f4ff9640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83e40a2cf0 0x7f83e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83f6ffd640 1 -- 172.21.15.69:0/4182592274 learned_addr learned my addr 172.21.15.69:0/4182592274 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab753d0 0x55eb2ab757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83f77fe640 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab80070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.825+0000 7f83f6ffd640 1 -- 172.21.15.69:0/4182592274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 msgr2=0x55eb2ab80070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab80070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83f6ffd640 1 -- 172.21.15.69:0/4182592274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab753d0 msgr2=0x55eb2ab757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab753d0 0x55eb2ab757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83f6ffd640 1 -- 172.21.15.69:0/4182592274 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2a84c6a0 con 0x55eb2ab805b0 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83f7fff640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83e40a2cf0 0x7f83e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83f4ff9640 1 -- 172.21.15.69:0/598203415 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f83f002e030 con 0x7f83f8106690 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab829a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f83e00125f0 tx=0x7f83e000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1772cd648688353 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83edffb640 1 -- 172.21.15.69:0/4182592274 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0018030 con 0x55eb2ab805b0 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83edffb640 1 -- 172.21.15.69:0/4182592274 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e0005500 con 0x55eb2ab805b0 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.826+0000 7f83edffb640 1 -- 172.21.15.69:0/4182592274 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0010920 con 0x55eb2ab805b0 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 -- 172.21.15.69:0/4182592274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 msgr2=0x55eb2ab829a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83f7fff640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83e40a2cf0 0x7f83e40a51b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83f00bcf40 tx=0x7f83f00dd040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab829a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f83e00125f0 tx=0x7f83e000b6e0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 -- 172.21.15.69:0/4182592274 shutdown_connections 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab829a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab80070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 --2- 172.21.15.69:0/4182592274 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab753d0 0x55eb2ab757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 -- 172.21.15.69:0/4182592274 >> 172.21.15.69:0/4182592274 conn(0x55eb2ab71960 msgr2=0x55eb2ab71d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:20.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 -- 172.21.15.69:0/4182592274 shutdown_connections 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 -- 172.21.15.69:0/4182592274 wait complete. 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 -- start start 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 Processor -- start 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.827+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab7da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 0x55eb2ab7dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83ffffe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55eb2aa95cb0 con 0x55eb2ab753d0 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83ffffe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55eb2aad67b0 con 0x55eb2ab805b0 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83ffffe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55eb2ab83180 con 0x55eb2ab75cf0 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83ffffe740 10 client.0 _ll_register_callbacks cb 0x55eb2a7e9ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35476/0 (socket says 172.21.15.69:35476) 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 -- 172.21.15.69:0/3782543596 learned_addr learned my addr 172.21.15.69:0/3782543596 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:20.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f77fe640 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab7da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 0x55eb2ab7dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 -- 172.21.15.69:0/3782543596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 msgr2=0x55eb2ab7da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab7da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 -- 172.21.15.69:0/3782543596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 msgr2=0x55eb2ab7dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 0x55eb2ab7dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 -- 172.21.15.69:0/3782543596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2aa94bc0 con 0x55eb2ab753d0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7d520 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f83e0006010 tx=0x7f83e0005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735563fb88da57bf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83d6ffd640 1 -- 172.21.15.69:0/3782543596 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0018040 con 0x55eb2ab753d0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83d6ffd640 1 -- 172.21.15.69:0/3782543596 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f83e0004040 con 0x55eb2ab753d0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83ffffe740 1 -- 172.21.15.69:0/3782543596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e0005500 con 0x55eb2ab753d0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83d6ffd640 1 -- 172.21.15.69:0/3782543596 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83e0032050 con 0x55eb2ab753d0 2024-08-02T17:57:20.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83ffffe740 10 client.18141 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83ffffe740 1 -- 172.21.15.69:0/3782543596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb2ab8f0c0 con 0x55eb2ab753d0 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.828+0000 7f83d6ffd640 1 -- 172.21.15.69:0/3782543596 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e0005500 con 0x55eb2ab753d0 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83d6ffd640 1 -- 172.21.15.69:0/3782543596 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f83e00035f0 con 0x55eb2ab753d0 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83ffffe740 10 client.18141 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83ffffe740 10 client.18141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83ffffe740 20 client.18141 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83ffffe740 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55eb2ab90470 0x55eb2ab92860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83ffffe740 4 client.18141 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83ffffe740 1 -- 172.21.15.69:0/3782543596 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb2aae7d60 con 0x55eb2ab90470 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83f77fe640 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55eb2ab90470 0x55eb2ab92860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83d6ffd640 10 client.18141 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.829+0000 7f83f77fe640 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55eb2ab90470 0x55eb2ab92860 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ae452927c422944 server_cookie=ba1ada5b09827b64 in_seq=0 out_seq=0 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83d6ffd640 1 -- 172.21.15.69:0/3782543596 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eb2aae7d60 con 0x55eb2ab90470 2024-08-02T17:57:20.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83d6ffd640 10 client.18141 handle_command_reply: tid=0 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 client.18141 shutdown 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 -- 172.21.15.69:0/3782543596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab753d0 msgr2=0x55eb2ab7d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7d520 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f83e0006010 tx=0x7f83e0005da0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 -- 172.21.15.69:0/3782543596 shutdown_connections 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55eb2ab90470 0x55eb2ab92860 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 0x55eb2ab7dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab7da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 --2- 172.21.15.69:0/3782543596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7d520 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83d6ffd640 0 client.18141 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 -- 172.21.15.69:0/3782543596 >> 172.21.15.69:0/3782543596 conn(0x55eb2ab71960 msgr2=0x55eb2ab77640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 -- 172.21.15.69:0/3782543596 shutdown_connections 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 1 -- 172.21.15.69:0/3782543596 wait complete. 2024-08-02T17:57:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.830+0000 7f83ffffe740 20 client.18141 trim_cache size 0 max 16384 2024-08-02T17:57:20.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83ffffe740 1 Processor -- start 2024-08-02T17:57:20.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83ffffe740 1 -- start start 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab98040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 0x55eb2ab98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83ffffe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81b0320 con 0x55eb2ab75cf0 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83ffffe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81adbf0 con 0x55eb2ab805b0 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83ffffe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81aba40 con 0x55eb2ab753d0 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab98040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab98040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35482/0 (socket says 172.21.15.69:35482) 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.848+0000 7f83f77fe640 1 -- 172.21.15.69:0/2803590657 learned_addr learned my addr 172.21.15.69:0/2803590657 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.851 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 0x55eb2ab98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f77fe640 1 -- 172.21.15.69:0/2803590657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 msgr2=0x55eb2ab7c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f77fe640 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f77fe640 1 -- 172.21.15.69:0/2803590657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 msgr2=0x55eb2ab98580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f77fe640 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 0x55eb2ab98580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f77fe640 1 -- 172.21.15.69:0/2803590657 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2ab92bd0 con 0x55eb2ab75cf0 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83f77fe640 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab98040 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f83e800fc60 tx=0x7f83e8005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee93edc329714a39 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83ed7fa640 1 -- 172.21.15.69:0/2803590657 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8005030 con 0x55eb2ab75cf0 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83ed7fa640 1 -- 172.21.15.69:0/2803590657 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8004730 con 0x55eb2ab75cf0 2024-08-02T17:57:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.849+0000 7f83ed7fa640 1 -- 172.21.15.69:0/2803590657 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e801b590 con 0x55eb2ab75cf0 2024-08-02T17:57:20.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 -- 172.21.15.69:0/2803590657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab75cf0 msgr2=0x55eb2ab98040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab98040 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f83e800fc60 tx=0x7f83e8005f30 comp rx=0 tx=0).stop 2024-08-02T17:57:20.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 -- 172.21.15.69:0/2803590657 shutdown_connections 2024-08-02T17:57:20.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab75cf0 0x55eb2ab98040 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab805b0 0x55eb2ab98580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 --2- 172.21.15.69:0/2803590657 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab7c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 -- 172.21.15.69:0/2803590657 >> 172.21.15.69:0/2803590657 conn(0x55eb2ab71960 msgr2=0x55eb2ab7bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:20.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 -- 172.21.15.69:0/2803590657 shutdown_connections 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.850+0000 7f83ffffe740 1 -- 172.21.15.69:0/2803590657 wait complete. 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 1 -- start start 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 1 Processor -- start 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab88b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab83dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab84310 0x55eb2ab84790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81ac0b0 con 0x55eb2ab805b0 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81ad520 con 0x55eb2ab84310 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81ac780 con 0x55eb2ab753d0 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83ffffe740 10 client.0 _ll_register_callbacks cb 0x7f83e0012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab83dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab83dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35494/0 (socket says 172.21.15.69:35494) 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83f77fe640 1 -- 172.21.15.69:0/802753762 learned_addr learned my addr 172.21.15.69:0/802753762 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:20.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab84310 0x55eb2ab84790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.851+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab88b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83f77fe640 1 -- 172.21.15.69:0/802753762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 msgr2=0x55eb2ab88b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83f77fe640 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab88b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83f77fe640 1 -- 172.21.15.69:0/802753762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab84310 msgr2=0x55eb2ab84790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83f77fe640 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab84310 0x55eb2ab84790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83f77fe640 1 -- 172.21.15.69:0/802753762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2abfe5d0 con 0x55eb2ab805b0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab88b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83f77fe640 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab83dd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000c00 tx=0x7f83e800fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fea459b7b8b53c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83d6ffd640 1 -- 172.21.15.69:0/802753762 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8005440 con 0x55eb2ab805b0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83d6ffd640 1 -- 172.21.15.69:0/802753762 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f83e80048e0 con 0x55eb2ab805b0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83ffffe740 1 -- 172.21.15.69:0/802753762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e8004730 con 0x55eb2ab805b0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83d6ffd640 1 -- 172.21.15.69:0/802753762 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f83e8004ae0 con 0x55eb2ab805b0 2024-08-02T17:57:20.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83ffffe740 10 client.18147 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:20.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.852+0000 7f83ffffe740 1 -- 172.21.15.69:0/802753762 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55eb2abff880 con 0x55eb2ab805b0 2024-08-02T17:57:20.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83d6ffd640 1 -- 172.21.15.69:0/802753762 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8004730 con 0x55eb2ab805b0 2024-08-02T17:57:20.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83d6ffd640 1 -- 172.21.15.69:0/802753762 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f83e802c020 con 0x55eb2ab805b0 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83ffffe740 10 client.18147 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83ffffe740 10 client.18147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83ffffe740 20 client.18147 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83ffffe740 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55eb2abfffd0 0x55eb2ac023f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83ffffe740 4 client.18147 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83ffffe740 1 -- 172.21.15.69:0/802753762 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eb2aae2270 con 0x55eb2abfffd0 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.853+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55eb2abfffd0 0x55eb2ac023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.854+0000 7f83d6ffd640 10 client.18147 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.854+0000 7f83f6ffd640 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55eb2abfffd0 0x55eb2ac023f0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50213b40aaee7aef server_cookie=92031ffe4e48dc62 in_seq=0 out_seq=0 2024-08-02T17:57:20.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83d6ffd640 1 -- 172.21.15.69:0/802753762 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eb2aae2270 con 0x55eb2abfffd0 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83d6ffd640 10 client.18147 handle_command_reply: tid=0 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 client.18147 shutdown 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 -- 172.21.15.69:0/802753762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 msgr2=0x55eb2ab83dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab83dd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000c00 tx=0x7f83e800fe80 comp rx=0 tx=0).stop 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 -- 172.21.15.69:0/802753762 shutdown_connections 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55eb2ab805b0 0x55eb2ab83dd0 secure :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000c00 tx=0x7f83e800fe80 comp rx=0 tx=0).stop 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55eb2abfffd0 0x55eb2ac023f0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83d6ffd640 0 client.18147 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55eb2ab84310 0x55eb2ab84790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 --2- 172.21.15.69:0/802753762 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55eb2ab753d0 0x55eb2ab88b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.855+0000 7f83ffffe740 1 -- 172.21.15.69:0/802753762 >> 172.21.15.69:0/802753762 conn(0x55eb2ab71960 msgr2=0x55eb2ab7f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.856+0000 7f83ffffe740 1 -- 172.21.15.69:0/802753762 shutdown_connections 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.856+0000 7f83ffffe740 1 -- 172.21.15.69:0/802753762 wait complete. 2024-08-02T17:57:20.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.856+0000 7f83ffffe740 20 client.18147 trim_cache size 0 max 16384 2024-08-02T17:57:20.860 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:57:20.860 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:57:20.860 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "9695cb0c-d98a-450a-8cfe-563e32d9d4f5", 2024-08-02T17:57:20.860 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:57:20.860 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.856+0000 7f83fc99e640 1 -- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83e40a2cf0 msgr2=0x7f83e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.856+0000 7f83fc99e640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83e40a2cf0 0x7f83e40a51b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f83f00bcf40 tx=0x7f83f00dd040 comp rx=0 tx=0).stop 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.856+0000 7f83fc99e640 1 -- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8106690 msgr2=0x7f83f819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.856+0000 7f83fc99e640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8106690 0x7f83f819b620 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f83f0005cc0 tx=0x7f83f0003970 comp rx=0 tx=0).stop 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.857+0000 7f83fc99e640 1 -- 172.21.15.69:0/598203415 shutdown_connections 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.857+0000 7f83fc99e640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83f8106690 0x7f83f819b620 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.857+0000 7f83fc99e640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83e40a2cf0 0x7f83e40a51b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.857+0000 7f83fc99e640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f83f8100fc0 0x7f83f8197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.857+0000 7f83fc99e640 1 --2- 172.21.15.69:0/598203415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f8100600 0x7f83f8196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:20.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.857+0000 7f83fc99e640 1 -- 172.21.15.69:0/598203415 >> 172.21.15.69:0/598203415 conn(0x7f83f80fc360 msgr2=0x7f83f80fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:20.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.857+0000 7f83fc99e640 1 -- 172.21.15.69:0/598203415 shutdown_connections 2024-08-02T17:57:20.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:20.857+0000 7f83fc99e640 1 -- 172.21.15.69:0/598203415 wait complete. 2024-08-02T17:57:21.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:20 smithi069 ceph-mon[32746]: pgmap v340: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-08-02T17:57:21.210 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:21.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:20 smithi083 ceph-mon[37031]: pgmap v340: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-08-02T17:57:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:20 smithi077 ceph-mon[37108]: pgmap v340: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-08-02T17:57:21.645 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:57:22.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:21 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:57:22.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:21 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:22.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:21 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:57:22.030 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:21 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:22.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 -- 172.21.15.69:0/906801921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1007b0 msgr2=0x7f35ac100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 --2- 172.21.15.69:0/906801921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1007b0 0x7f35ac100c10 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f3594009930 tx=0x7f359402a420 comp rx=0 tx=0).stop 2024-08-02T17:57:22.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 -- 172.21.15.69:0/906801921 shutdown_connections 2024-08-02T17:57:22.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 --2- 172.21.15.69:0/906801921 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1007b0 0x7f35ac100c10 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 --2- 172.21.15.69:0/906801921 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ac101150 0x7f35ac10d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 --2- 172.21.15.69:0/906801921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35ac1067b0 0x7f35ac106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 -- 172.21.15.69:0/906801921 >> 172.21.15.69:0/906801921 conn(0x7f35ac0fc4d0 msgr2=0x7f35ac0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 -- 172.21.15.69:0/906801921 shutdown_connections 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 -- 172.21.15.69:0/906801921 wait complete. 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 Processor -- start 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.107+0000 7f35b0bca640 1 -- start start 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35b0bca640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ac1007b0 0x7f35ac199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35b0bca640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35ac101150 0x7f35ac199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35b0bca640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1067b0 0x7f35ac1a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35b0bca640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac110bb0 con 0x7f35ac1067b0 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35b0bca640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac110a30 con 0x7f35ac1007b0 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35b0bca640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac110d30 con 0x7f35ac101150 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1067b0 0x7f35ac1a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1067b0 0x7f35ac1a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35512/0 (socket says 172.21.15.69:35512) 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 -- 172.21.15.69:0/2519147408 learned_addr learned my addr 172.21.15.69:0/2519147408 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aa59c640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ac1007b0 0x7f35ac199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35a3fff640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35ac101150 0x7f35ac199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 -- 172.21.15.69:0/2519147408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35ac101150 msgr2=0x7f35ac199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35ac101150 0x7f35ac199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 -- 172.21.15.69:0/2519147408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ac1007b0 msgr2=0x7f35ac199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ac1007b0 0x7f35ac199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 -- 172.21.15.69:0/2519147408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac1a45e0 con 0x7f35ac1067b0 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35aad9d640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1067b0 0x7f35ac1a40a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f35a400eaa0 tx=0x7f35a400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35a37fe640 1 -- 172.21.15.69:0/2519147408 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a400cdb0 con 0x7f35ac1067b0 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac1a47b0 con 0x7f35ac1067b0 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35a37fe640 1 -- 172.21.15.69:0/2519147408 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a4005020 con 0x7f35ac1067b0 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35a37fe640 1 -- 172.21.15.69:0/2519147408 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a40116b0 con 0x7f35ac1067b0 2024-08-02T17:57:22.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.108+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac1a4c70 con 0x7f35ac1067b0 2024-08-02T17:57:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.109+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac100c10 con 0x7f35ac1067b0 2024-08-02T17:57:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.110+0000 7f35a37fe640 1 -- 172.21.15.69:0/2519147408 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f35a4005560 con 0x7f35ac1067b0 2024-08-02T17:57:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.111+0000 7f35a37fe640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f35980a2cf0 0x7f35980a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.111+0000 7f35aa59c640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f35980a2cf0 0x7f35980a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.111+0000 7f35a37fe640 1 -- 172.21.15.69:0/2519147408 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f35a40cfdf0 con 0x7f35ac1067b0 2024-08-02T17:57:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.112+0000 7f35aa59c640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f35980a2cf0 0x7f35980a51b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f359c0058c0 tx=0x7f359c00a250 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:22.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.114+0000 7f35a37fe640 1 -- 172.21.15.69:0/2519147408 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f35a4016030 con 0x7f35ac1067b0 2024-08-02T17:57:22.176 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:22.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:21 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:57:22.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:21 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:22.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:21 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:57:22.272 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:21 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:22.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.337+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35ac02d5e0 con 0x7f35ac1067b0 2024-08-02T17:57:22.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.338+0000 7f35a37fe640 1 -- 172.21.15.69:0/2519147408 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f35a409bfb0 con 0x7f35ac1067b0 2024-08-02T17:57:22.340 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:57:22.341 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:57:22.342 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:57:22.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f35980a2cf0 msgr2=0x7f35980a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f35980a2cf0 0x7f35980a51b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f359c0058c0 tx=0x7f359c00a250 comp rx=0 tx=0).stop 2024-08-02T17:57:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1067b0 msgr2=0x7f35ac1a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1067b0 0x7f35ac1a40a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f35a400eaa0 tx=0x7f35a400ef60 comp rx=0 tx=0).stop 2024-08-02T17:57:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 shutdown_connections 2024-08-02T17:57:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f35ac1067b0 0x7f35ac1a40a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f35980a2cf0 0x7f35980a51b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35ac101150 0x7f35ac199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 --2- 172.21.15.69:0/2519147408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f35ac1007b0 0x7f35ac199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 >> 172.21.15.69:0/2519147408 conn(0x7f35ac0fc4d0 msgr2=0x7f35ac0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 shutdown_connections 2024-08-02T17:57:22.344 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.340+0000 7f35b0bca640 1 -- 172.21.15.69:0/2519147408 wait complete. 2024-08-02T17:57:22.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:21 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:57:22.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:21 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:22.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:21 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:57:22.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:21 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:57:22.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.847+0000 7fa85037e640 1 -- 172.21.15.69:0/3645241594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481018b0 msgr2=0x7fa848101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.847+0000 7fa85037e640 1 --2- 172.21.15.69:0/3645241594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481018b0 0x7fa848101d10 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa838009930 tx=0x7fa83802a420 comp rx=0 tx=0).stop 2024-08-02T17:57:22.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 -- 172.21.15.69:0/3645241594 shutdown_connections 2024-08-02T17:57:22.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 --2- 172.21.15.69:0/3645241594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481018b0 0x7fa848101d10 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 --2- 172.21.15.69:0/3645241594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa848102250 0x7fa84810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 --2- 172.21.15.69:0/3645241594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8481078b0 0x7fa848107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 -- 172.21.15.69:0/3645241594 >> 172.21.15.69:0/3645241594 conn(0x7fa8480fd5f0 msgr2=0x7fa8480ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 -- 172.21.15.69:0/3645241594 shutdown_connections 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 -- 172.21.15.69:0/3645241594 wait complete. 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 Processor -- start 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 -- start start 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8481018b0 0x7fa848198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa848102250 0x7fa848198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481078b0 0x7fa84819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa848111cb0 con 0x7fa8481078b0 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa848111b30 con 0x7fa8481018b0 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa85037e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa848111e30 con 0x7fa848102250 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.848+0000 7fa84e91b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481078b0 0x7fa84819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e91b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481078b0 0x7fa84819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35522/0 (socket says 172.21.15.69:35522) 2024-08-02T17:57:22.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e91b640 1 -- 172.21.15.69:0/3964878738 learned_addr learned my addr 172.21.15.69:0/3964878738 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84d919640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa848102250 0x7fa848198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e11a640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8481018b0 0x7fa848198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e91b640 1 -- 172.21.15.69:0/3964878738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa848102250 msgr2=0x7fa848198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e91b640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa848102250 0x7fa848198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e91b640 1 -- 172.21.15.69:0/3964878738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8481018b0 msgr2=0x7fa848198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e91b640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8481018b0 0x7fa848198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e91b640 1 -- 172.21.15.69:0/3964878738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84819d0e0 con 0x7fa8481078b0 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa84e91b640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481078b0 0x7fa84819ca00 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa84000eaa0 tx=0x7fa84000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa83f7fe640 1 -- 172.21.15.69:0/3964878738 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840004d20 con 0x7fa8481078b0 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa83f7fe640 1 -- 172.21.15.69:0/3964878738 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa840005020 con 0x7fa8481078b0 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa85037e640 1 -- 172.21.15.69:0/3964878738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84819d3d0 con 0x7fa8481078b0 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa83f7fe640 1 -- 172.21.15.69:0/3964878738 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400116d0 con 0x7fa8481078b0 2024-08-02T17:57:22.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.849+0000 7fa85037e640 1 -- 172.21.15.69:0/3964878738 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8481a4cb0 con 0x7fa8481078b0 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa83f7fe640 1 -- 172.21.15.69:0/3964878738 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa8400051d0 con 0x7fa8481078b0 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa85277c740 1 Processor -- start 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa85277c740 1 -- start start 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa85277c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 0x558406d8c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa85277c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406d96eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa85277c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 0x558406d997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa85277c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558406cf1980 con 0x558406d8cb30 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa85277c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558406c89a70 con 0x558406d973f0 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa85277c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558406ced940 con 0x558406d8c210 2024-08-02T17:57:22.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa83f7fe640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8340a2cf0 0x7fa8340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa84e11a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406d96eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa84e11a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406d96eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35528/0 (socket says 172.21.15.69:35528) 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.852+0000 7fa84e11a640 1 -- 172.21.15.69:0/3341171511 learned_addr learned my addr 172.21.15.69:0/3341171511 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84e91b640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8340a2cf0 0x7fa8340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84d919640 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 0x558406d8c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84d919640 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 0x558406d997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84e11a640 1 -- 172.21.15.69:0/3341171511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 msgr2=0x558406d8c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa83f7fe640 1 -- 172.21.15.69:0/3964878738 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7fa8400cfa40 con 0x7fa8481078b0 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84e11a640 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 0x558406d8c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84e11a640 1 -- 172.21.15.69:0/3341171511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 msgr2=0x558406d997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84e11a640 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 0x558406d997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84e11a640 1 -- 172.21.15.69:0/3341171511 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558406cec6e0 con 0x558406d8cb30 2024-08-02T17:57:22.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84d919640 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 0x558406d997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84e11a640 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406d96eb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa84400ac90 tx=0x7fa84400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c02f5c8dcc6956 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa84e91b640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8340a2cf0 0x7fa8340a51b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa84009bae0 tx=0x7fa840093040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa83cdea640 1 -- 172.21.15.69:0/3341171511 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84401d070 con 0x558406d8cb30 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa83cdea640 1 -- 172.21.15.69:0/3341171511 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844018b20 con 0x558406d8cb30 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.853+0000 7fa83cdea640 1 -- 172.21.15.69:0/3341171511 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844010ba0 con 0x558406d8cb30 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 -- 172.21.15.69:0/3341171511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 msgr2=0x558406d96eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406d96eb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa84400ac90 tx=0x7fa84400f790 comp rx=0 tx=0).stop 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 -- 172.21.15.69:0/3341171511 shutdown_connections 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406d96eb0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 0x558406d997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 --2- 172.21.15.69:0/3341171511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 0x558406d8c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 -- 172.21.15.69:0/3341171511 >> 172.21.15.69:0/3341171511 conn(0x558406d887a0 msgr2=0x558406d88b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:22.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 -- 172.21.15.69:0/3341171511 shutdown_connections 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 -- 172.21.15.69:0/3341171511 wait complete. 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 -- start start 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.854+0000 7fa85277c740 1 Processor -- start 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa85277c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8c210 0x558406d94350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa85277c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8cb30 0x558406d94890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa85277c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d973f0 0x558406d94dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa85277c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558406d99fc0 con 0x558406d8c210 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa85277c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558406cacaa0 con 0x558406d8cb30 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa85277c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558406d9a140 con 0x558406d973f0 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa85277c740 10 client.0 _ll_register_callbacks cb 0x558406c9be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8c210 0x558406d94350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8c210 0x558406d94350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35538/0 (socket says 172.21.15.69:35538) 2024-08-02T17:57:22.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 -- 172.21.15.69:0/942980996 learned_addr learned my addr 172.21.15.69:0/942980996 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84e11a640 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8cb30 0x558406d94890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d973f0 0x558406d94dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 -- 172.21.15.69:0/942980996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d973f0 msgr2=0x558406d94dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d973f0 0x558406d94dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 -- 172.21.15.69:0/942980996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8cb30 msgr2=0x558406d94890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8cb30 0x558406d94890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 -- 172.21.15.69:0/942980996 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558406c90650 con 0x558406d8c210 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa84d919640 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8c210 0x558406d94350 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa838009930 tx=0x7fa838005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0a395fb4521dd6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa831ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa831ffb640 1 -- 172.21.15.69:0/942980996 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838035040 con 0x558406d8c210 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa831ffb640 1 -- 172.21.15.69:0/942980996 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7fa838005020 con 0x558406d8c210 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.855+0000 7fa85277c740 1 -- 172.21.15.69:0/942980996 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa844018b20 con 0x558406d8c210 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa831ffb640 1 -- 172.21.15.69:0/942980996 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa838051050 con 0x558406d8c210 2024-08-02T17:57:22.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa85277c740 10 client.18165 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa85277c740 1 -- 172.21.15.69:0/942980996 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558406a633c0 con 0x558406d8c210 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa831ffb640 1 -- 172.21.15.69:0/942980996 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844018b20 con 0x558406d8c210 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa831ffb640 1 -- 172.21.15.69:0/942980996 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa838049020 con 0x558406d8c210 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa85277c740 10 client.18165 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa85277c740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa85277c740 20 client.18165 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa85277c740 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558406da6ea0 0x558406da9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa85277c740 4 client.18165 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa85277c740 1 -- 172.21.15.69:0/942980996 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558406cfeef0 con 0x558406da6ea0 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.856+0000 7fa84e11a640 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558406da6ea0 0x558406da9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.857+0000 7fa831ffb640 10 client.18165 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.857+0000 7fa84e11a640 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558406da6ea0 0x558406da9290 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1316b6e9242a6f95 server_cookie=ade95752ad2534ed in_seq=0 out_seq=0 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.857+0000 7fa831ffb640 1 -- 172.21.15.69:0/942980996 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558406cfeef0 con 0x558406da6ea0 2024-08-02T17:57:22.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.857+0000 7fa831ffb640 10 client.18165 handle_command_reply: tid=0 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.857+0000 7fa85277c740 1 client.18165 shutdown 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 -- 172.21.15.69:0/942980996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8c210 msgr2=0x558406d94350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8c210 0x558406d94350 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa838009930 tx=0x7fa838005c00 comp rx=0 tx=0).stop 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 -- 172.21.15.69:0/942980996 shutdown_connections 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558406da6ea0 0x558406da9290 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d973f0 0x558406d94dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8cb30 0x558406d94890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 --2- 172.21.15.69:0/942980996 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8c210 0x558406d94350 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 -- 172.21.15.69:0/942980996 >> 172.21.15.69:0/942980996 conn(0x558406d887a0 msgr2=0x558406d8e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa831ffb640 0 client.18165 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 -- 172.21.15.69:0/942980996 shutdown_connections 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 1 -- 172.21.15.69:0/942980996 wait complete. 2024-08-02T17:57:22.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.858+0000 7fa85277c740 20 client.18165 trim_cache size 0 max 16384 2024-08-02T17:57:22.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa85277c740 1 Processor -- start 2024-08-02T17:57:22.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa85277c740 1 -- start start 2024-08-02T17:57:22.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa85277c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 0x558406d8ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa85277c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406daeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa85277c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 0x558406daf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa85277c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8481b1800 con 0x558406d8cb30 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa85277c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8481af0d0 con 0x558406d973f0 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa85277c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8481acf20 con 0x558406d8c210 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa84e11a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406daeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa84e11a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406daeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35548/0 (socket says 172.21.15.69:35548) 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa84e11a640 1 -- 172.21.15.69:0/1811238008 learned_addr learned my addr 172.21.15.69:0/1811238008 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa84e11a640 1 -- 172.21.15.69:0/1811238008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 msgr2=0x558406d8ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa84e11a640 1 --2- 172.21.15.69:0/1811238008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 0x558406d8ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa84e11a640 1 -- 172.21.15.69:0/1811238008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 msgr2=0x558406daf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:22.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa84e11a640 1 --2- 172.21.15.69:0/1811238008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 0x558406daf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.875+0000 7fa84e11a640 1 -- 172.21.15.69:0/1811238008 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558406d8f1b0 con 0x558406d8cb30 2024-08-02T17:57:22.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa84e11a640 1 --2- 172.21.15.69:0/1811238008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406daeb40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa84400aa20 tx=0x7fa844010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1928bb1f1d879111 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:22.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa833fff640 1 -- 172.21.15.69:0/1811238008 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844031070 con 0x558406d8cb30 2024-08-02T17:57:22.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa833fff640 1 -- 172.21.15.69:0/1811238008 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844005d60 con 0x558406d8cb30 2024-08-02T17:57:22.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa833fff640 1 -- 172.21.15.69:0/1811238008 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84401d040 con 0x558406d8cb30 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa85277c740 1 -- 172.21.15.69:0/1811238008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 msgr2=0x558406daeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa85277c740 1 --2- 172.21.15.69:0/1811238008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406daeb40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa84400aa20 tx=0x7fa844010760 comp rx=0 tx=0).stop 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa85277c740 1 -- 172.21.15.69:0/1811238008 shutdown_connections 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa85277c740 1 --2- 172.21.15.69:0/1811238008 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d8cb30 0x558406daeb40 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa85277c740 1 --2- 172.21.15.69:0/1811238008 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d973f0 0x558406daf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa85277c740 1 --2- 172.21.15.69:0/1811238008 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8c210 0x558406d8ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.876+0000 7fa85277c740 1 -- 172.21.15.69:0/1811238008 >> 172.21.15.69:0/1811238008 conn(0x558406d887a0 msgr2=0x558406d962c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 -- 172.21.15.69:0/1811238008 shutdown_connections 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 -- 172.21.15.69:0/1811238008 wait complete. 2024-08-02T17:57:22.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 -- start start 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 Processor -- start 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8c210 0x558406d9fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8cb30 0x558406d911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d973f0 0x558406d917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8481ad590 con 0x558406d973f0 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa8481aea00 con 0x558406d8c210 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa8481adc60 con 0x558406d8cb30 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa85277c740 10 client.0 _ll_register_callbacks cb 0x7fa838005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa84d919640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d973f0 0x558406d917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa84d919640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d973f0 0x558406d917b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35556/0 (socket says 172.21.15.69:35556) 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa84d919640 1 -- 172.21.15.69:0/156977283 learned_addr learned my addr 172.21.15.69:0/156977283 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.877+0000 7fa84e11a640 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8cb30 0x558406d911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa84d919640 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8c210 0x558406d9fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa84d919640 1 -- 172.21.15.69:0/156977283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8cb30 msgr2=0x558406d911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa84d919640 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8cb30 0x558406d911a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa84d919640 1 -- 172.21.15.69:0/156977283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8c210 msgr2=0x558406d9fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa84d919640 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8c210 0x558406d9fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa84d919640 1 -- 172.21.15.69:0/156977283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558406d91fb0 con 0x558406d973f0 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa84d919640 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d973f0 0x558406d917b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x558406d90a30 tx=0x7fa838009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f59c092fd85387 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa831ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa831ffb640 1 -- 172.21.15.69:0/156977283 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838059070 con 0x558406d973f0 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa831ffb640 1 -- 172.21.15.69:0/156977283 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7fa838003bc0 con 0x558406d973f0 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa85277c740 1 -- 172.21.15.69:0/156977283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa844005d60 con 0x558406d973f0 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa831ffb640 1 -- 172.21.15.69:0/156977283 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa838064050 con 0x558406d973f0 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa85277c740 10 client.18171 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:22.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa85277c740 1 -- 172.21.15.69:0/156977283 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558406e11c80 con 0x558406d973f0 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.878+0000 7fa831ffb640 1 -- 172.21.15.69:0/156977283 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa844005d60 con 0x558406d973f0 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa831ffb640 1 -- 172.21.15.69:0/156977283 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa838049050 con 0x558406d973f0 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa85277c740 10 client.18171 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa85277c740 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa85277c740 20 client.18171 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa85277c740 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558406e123e0 0x558406e14800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa85277c740 4 client.18171 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa85277c740 1 -- 172.21.15.69:0/156977283 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558406cf9400 con 0x558406e123e0 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa84e11a640 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558406e123e0 0x558406e14800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa831ffb640 10 client.18171 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.879+0000 7fa84e11a640 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558406e123e0 0x558406e14800 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f6db2d6f83a7b82 server_cookie=53b73644f3bc5b6a in_seq=0 out_seq=0 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa831ffb640 1 -- 172.21.15.69:0/156977283 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558406cf9400 con 0x558406e123e0 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa831ffb640 10 client.18171 handle_command_reply: tid=0 2024-08-02T17:57:22.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 client.18171 shutdown 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 -- 172.21.15.69:0/156977283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d973f0 msgr2=0x558406d917b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d973f0 0x558406d917b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x558406d90a30 tx=0x7fa838009600 comp rx=0 tx=0).stop 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 -- 172.21.15.69:0/156977283 shutdown_connections 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558406d973f0 0x558406d917b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558406e123e0 0x558406e14800 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558406d8cb30 0x558406d911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 --2- 172.21.15.69:0/156977283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558406d8c210 0x558406d9fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 -- 172.21.15.69:0/156977283 >> 172.21.15.69:0/156977283 conn(0x558406d887a0 msgr2=0x558406d87900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa831ffb640 0 client.18171 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 -- 172.21.15.69:0/156977283 shutdown_connections 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 1 -- 172.21.15.69:0/156977283 wait complete. 2024-08-02T17:57:22.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.880+0000 7fa85277c740 20 client.18171 trim_cache size 0 max 16384 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout: "9695cb0c-d98a-450a-8cfe-563e32d9d4f5": { 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "9695cb0c-d98a-450a-8cfe-563e32d9d4f5", 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:22.888 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:57:22.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 -- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8340a2cf0 msgr2=0x7fa8340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8340a2cf0 0x7fa8340a51b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa84009bae0 tx=0x7fa840093040 comp rx=0 tx=0).stop 2024-08-02T17:57:22.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 -- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481078b0 msgr2=0x7fa84819ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:22.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481078b0 0x7fa84819ca00 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa84000eaa0 tx=0x7fa84000eea0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 -- 172.21.15.69:0/3964878738 shutdown_connections 2024-08-02T17:57:22.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa8481078b0 0x7fa84819ca00 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8340a2cf0 0x7fa8340a51b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa848102250 0x7fa848198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 --2- 172.21.15.69:0/3964878738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa8481018b0 0x7fa848198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:22.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 -- 172.21.15.69:0/3964878738 >> 172.21.15.69:0/3964878738 conn(0x7fa8480fd5f0 msgr2=0x7fa8480fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:22.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 -- 172.21.15.69:0/3964878738 shutdown_connections 2024-08-02T17:57:22.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:22.881+0000 7fa85037e640 1 -- 172.21.15.69:0/3964878738 wait complete. 2024-08-02T17:57:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:23 smithi069 ceph-mon[32746]: pgmap v341: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-08-02T17:57:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:23 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2519147408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:23.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:23 smithi083 ceph-mon[37031]: pgmap v341: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-08-02T17:57:23.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:23 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2519147408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:23.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:23 smithi077 ceph-mon[37108]: pgmap v341: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 49 KiB/s wr, 29 op/s 2024-08-02T17:57:23.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:23 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2519147408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:57:23.501 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 82059677 --release --await' 2024-08-02T17:57:23.655 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 82059677 --release --await 2024-08-02T17:57:24.194 INFO:tasks.cephfs.filesystem:scrub status for tag:9695cb0c-d98a-450a-8cfe-563e32d9d4f5 - {'path': '/', 'tag': '9695cb0c-d98a-450a-8cfe-563e32d9d4f5', 'options': 'recursive,force'} 2024-08-02T17:57:24.270 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.706+0000 7f48404b5640 1 -- 172.21.15.69:0/319065174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 msgr2=0x7f4838102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.706+0000 7f48404b5640 1 --2- 172.21.15.69:0/319065174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 0x7f4838102e40 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f4828009ca0 tx=0x7f482802a950 comp rx=0 tx=0).stop 2024-08-02T17:57:24.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 -- 172.21.15.69:0/319065174 shutdown_connections 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 --2- 172.21.15.69:0/319065174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483810be90 0x7f483810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 --2- 172.21.15.69:0/319065174 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4838103380 0x7f483810b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 --2- 172.21.15.69:0/319065174 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 0x7f4838102e40 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 -- 172.21.15.69:0/319065174 >> 172.21.15.69:0/319065174 conn(0x7f48380fc4c0 msgr2=0x7f48380fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 -- 172.21.15.69:0/319065174 shutdown_connections 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 -- 172.21.15.69:0/319065174 wait complete. 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 Processor -- start 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.707+0000 7f48404b5640 1 -- start start 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48404b5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 0x7f48381982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48404b5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4838103380 0x7f48381987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48404b5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483810be90 0x7f483819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48404b5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4838111d50 con 0x7f4838102a60 2024-08-02T17:57:24.731 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48404b5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4838111bd0 con 0x7f4838103380 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48404b5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4838111ed0 con 0x7f483810be90 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 0x7f48381982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 0x7f48381982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35572/0 (socket says 172.21.15.69:35572) 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 -- 172.21.15.69:0/3915845666 learned_addr learned my addr 172.21.15.69:0/3915845666 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483da50640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4838103380 0x7f48381987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483ea52640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483810be90 0x7f483819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 -- 172.21.15.69:0/3915845666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483810be90 msgr2=0x7f483819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483810be90 0x7f483819cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 -- 172.21.15.69:0/3915845666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4838103380 msgr2=0x7f48381987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4838103380 0x7f48381987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 -- 172.21.15.69:0/3915845666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f483819d260 con 0x7f4838102a60 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f483e251640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 0x7f48381982b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4828005cc0 tx=0x7f482802f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:24.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48377fe640 1 -- 172.21.15.69:0/3915845666 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828034040 con 0x7f4838102a60 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48377fe640 1 -- 172.21.15.69:0/3915845666 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4828003e40 con 0x7f4838102a60 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48404b5640 1 -- 172.21.15.69:0/3915845666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483819d4f0 con 0x7f4838102a60 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48377fe640 1 -- 172.21.15.69:0/3915845666 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828045480 con 0x7f4838102a60 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.708+0000 7f48404b5640 1 -- 172.21.15.69:0/3915845666 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48381a4e30 con 0x7f4838102a60 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.711+0000 7f48377fe640 1 -- 172.21.15.69:0/3915845666 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4828037020 con 0x7f4838102a60 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.711+0000 7f48428b3740 1 Processor -- start 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.711+0000 7f48428b3740 1 -- start start 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.711+0000 7f48428b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f654d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.711+0000 7f48428b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f65f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.711+0000 7f48428b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f65fb70 0x556d2f661f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.711+0000 7f48428b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556d2f5bac60 con 0x556d2f65fb70 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f48428b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556d2f552b90 con 0x556d2f654990 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f48428b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d2f5b6c20 con 0x556d2f6552b0 2024-08-02T17:57:24.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f48377fe640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48200a2cf0 0x7f48200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f65fb70 0x556d2f661f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f65fb70 0x556d2f661f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35588/0 (socket says 172.21.15.69:35588) 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 -- 172.21.15.69:0/2610723813 learned_addr learned my addr 172.21.15.69:0/2610723813 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483e251640 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f65f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483ea52640 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f654d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483ea52640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48200a2cf0 0x7f48200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f48377fe640 1 -- 172.21.15.69:0/3915845666 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f48280f0970 con 0x7f4838102a60 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 -- 172.21.15.69:0/2610723813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 msgr2=0x556d2f65f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f65f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 -- 172.21.15.69:0/2610723813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 msgr2=0x556d2f654d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f654d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 -- 172.21.15.69:0/2610723813 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2f2ca580 con 0x556d2f65fb70 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483ea52640 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f654d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483da50640 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f65fb70 0x556d2f661f60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f482c009a20 tx=0x7f482c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6696b45ec6140a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f483ea52640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48200a2cf0 0x7f48200a51b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f48300155f0 tx=0x7f4830006b20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f4834ff9640 1 -- 172.21.15.69:0/2610723813 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c017070 con 0x556d2f65fb70 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f4834ff9640 1 -- 172.21.15.69:0/2610723813 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482c013470 con 0x556d2f65fb70 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.712+0000 7f4834ff9640 1 -- 172.21.15.69:0/2610723813 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c012630 con 0x556d2f65fb70 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 -- 172.21.15.69:0/2610723813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f65fb70 msgr2=0x556d2f661f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f65fb70 0x556d2f661f60 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f482c009a20 tx=0x7f482c009ee0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 -- 172.21.15.69:0/2610723813 shutdown_connections 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f65fb70 0x556d2f661f60 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f65f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 --2- 172.21.15.69:0/2610723813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f654d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 -- 172.21.15.69:0/2610723813 >> 172.21.15.69:0/2610723813 conn(0x556d2f650f20 msgr2=0x556d2f651300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 -- 172.21.15.69:0/2610723813 shutdown_connections 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.713+0000 7f48428b3740 1 -- 172.21.15.69:0/2610723813 wait complete. 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 1 -- start start 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 1 Processor -- start 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f65c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556d2f662740 con 0x556d2f6552b0 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556d2f5587c0 con 0x556d2f654990 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556d2f6628c0 con 0x556d2f65fb70 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f48428b3740 10 client.0 _ll_register_callbacks cb 0x556d2f561f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35592/0 (socket says 172.21.15.69:35592) 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 -- 172.21.15.69:0/1369525633 learned_addr learned my addr 172.21.15.69:0/1369525633 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483ea52640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 -- 172.21.15.69:0/1369525633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 msgr2=0x556d2f65d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483e251640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f65c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 -- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 msgr2=0x556d2f65c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f65c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 -- 172.21.15.69:0/1369525633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2f394960 con 0x556d2f6552b0 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483ea52640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483e251640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f65c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:24.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f483da50640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65cf20 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f482c004700 tx=0x7f482c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0394d6ec33beac server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.714+0000 7f481dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f481dffb640 1 -- 172.21.15.69:0/1369525633 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c017050 con 0x556d2f6552b0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f481dffb640 1 -- 172.21.15.69:0/1369525633 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f482c01c450 con 0x556d2f6552b0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 1 -- 172.21.15.69:0/1369525633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f482c013470 con 0x556d2f6552b0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f481dffb640 1 -- 172.21.15.69:0/1369525633 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f482c02f050 con 0x556d2f6552b0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 10 client.18183 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 1 -- 172.21.15.69:0/1369525633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556d2f66e700 con 0x556d2f6552b0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f481dffb640 1 -- 172.21.15.69:0/1369525633 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f482c013470 con 0x556d2f6552b0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f481dffb640 1 -- 172.21.15.69:0/1369525633 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f482c026670 con 0x556d2f6552b0 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 10 client.18183 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 10 client.18183 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 20 client.18183 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d2f66fb00 0x556d2f671ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 4 client.18183 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.715+0000 7f48428b3740 1 -- 172.21.15.69:0/1369525633 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2f5c81d0 con 0x556d2f66fb00 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.716+0000 7f483e251640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d2f66fb00 0x556d2f671ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.716+0000 7f483e251640 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d2f66fb00 0x556d2f671ef0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8cef8d7d52e47ca server_cookie=fac73fd8f3485319 in_seq=0 out_seq=0 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.716+0000 7f481dffb640 10 client.18183 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f481dffb640 1 -- 172.21.15.69:0/1369525633 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556d2f5c81d0 con 0x556d2f66fb00 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f481dffb640 10 client.18183 handle_command_reply: tid=0 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 client.18183 shutdown 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 -- 172.21.15.69:0/1369525633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 msgr2=0x556d2f65cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65cf20 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f482c004700 tx=0x7f482c00ef20 comp rx=0 tx=0).stop 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 -- 172.21.15.69:0/1369525633 shutdown_connections 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65cf20 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d2f66fb00 0x556d2f671ef0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 --2- 172.21.15.69:0/1369525633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f65c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f481dffb640 0 client.18183 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:24.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 -- 172.21.15.69:0/1369525633 >> 172.21.15.69:0/1369525633 conn(0x556d2f650f20 msgr2=0x556d2f656c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:24.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 -- 172.21.15.69:0/1369525633 shutdown_connections 2024-08-02T17:57:24.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 1 -- 172.21.15.69:0/1369525633 wait complete. 2024-08-02T17:57:24.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.717+0000 7f48428b3740 20 client.18183 trim_cache size 0 max 16384 2024-08-02T17:57:24.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f48428b3740 1 Processor -- start 2024-08-02T17:57:24.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f48428b3740 1 -- start start 2024-08-02T17:57:24.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f48428b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f654990 0x556d2f65e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f48428b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f6774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f48428b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f65fb70 0x556d2f677a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f48428b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48381b1840 con 0x556d2f654990 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f48428b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48381af110 con 0x556d2f65fb70 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f48428b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f48381acf60 con 0x556d2f6552b0 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f483e251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f654990 0x556d2f65e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f483e251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f654990 0x556d2f65e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58372/0 (socket says 172.21.15.69:58372) 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f483e251640 1 -- 172.21.15.69:0/1430462888 learned_addr learned my addr 172.21.15.69:0/1430462888 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f483e251640 1 -- 172.21.15.69:0/1430462888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 msgr2=0x556d2f6774f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f483da50640 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f6774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.734+0000 7f483ea52640 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f65fb70 0x556d2f677a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f483e251640 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f6774f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f483e251640 1 -- 172.21.15.69:0/1430462888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f65fb70 msgr2=0x556d2f677a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:24.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f483e251640 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f65fb70 0x556d2f677a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f483e251640 1 -- 172.21.15.69:0/1430462888 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2f65b9d0 con 0x556d2f654990 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f483da50640 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f6774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f483ea52640 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f65fb70 0x556d2f677a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f483e251640 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f654990 0x556d2f65e950 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4828000f80 tx=0x7f482808fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274aa5748e3f1cc7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f481ffff640 1 -- 172.21.15.69:0/1430462888 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280bd250 con 0x556d2f654990 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f481ffff640 1 -- 172.21.15.69:0/1430462888 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48280bcb50 con 0x556d2f654990 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f481ffff640 1 -- 172.21.15.69:0/1430462888 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280edef0 con 0x556d2f654990 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f48428b3740 1 -- 172.21.15.69:0/1430462888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f654990 msgr2=0x556d2f65e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.735+0000 7f48428b3740 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f654990 0x556d2f65e950 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f4828000f80 tx=0x7f482808fbc0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 -- 172.21.15.69:0/1430462888 shutdown_connections 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f65fb70 0x556d2f677a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f6552b0 0x556d2f6774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 --2- 172.21.15.69:0/1430462888 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f654990 0x556d2f65e950 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 -- 172.21.15.69:0/1430462888 >> 172.21.15.69:0/1430462888 conn(0x556d2f650f20 msgr2=0x556d2f65e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 -- 172.21.15.69:0/1430462888 shutdown_connections 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 -- 172.21.15.69:0/1430462888 wait complete. 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 -- start start 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 Processor -- start 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f659af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ad5d0 con 0x556d2f6552b0 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f48381aea40 con 0x556d2f654990 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f48381adca0 con 0x556d2f65fb70 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f48428b3740 10 client.0 _ll_register_callbacks cb 0x7f482c009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f483da50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f483da50640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58384/0 (socket says 172.21.15.69:58384) 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f483da50640 1 -- 172.21.15.69:0/1759271244 learned_addr learned my addr 172.21.15.69:0/1759271244 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f483e251640 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f659af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.736+0000 7f483ea52640 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f483da50640 1 -- 172.21.15.69:0/1759271244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 msgr2=0x556d2f65a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f483da50640 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f483da50640 1 -- 172.21.15.69:0/1759271244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 msgr2=0x556d2f659af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f483da50640 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f659af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f483da50640 1 -- 172.21.15.69:0/1759271244 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2f6d9590 con 0x556d2f6552b0 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f483da50640 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65a030 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x556d2f65c270 tx=0x7f482c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee785d08869205e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f481dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f481dffb640 1 -- 172.21.15.69:0/1759271244 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482c017070 con 0x556d2f6552b0 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 1 -- 172.21.15.69:0/1759271244 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48280bcb50 con 0x556d2f6552b0 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f481dffb640 1 -- 172.21.15.69:0/1759271244 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f482c0263e0 con 0x556d2f6552b0 2024-08-02T17:57:24.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f481dffb640 1 -- 172.21.15.69:0/1759271244 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f482c013e00 con 0x556d2f6552b0 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 10 client.18189 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 1 -- 172.21.15.69:0/1759271244 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556d2f6da820 con 0x556d2f6552b0 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f481dffb640 1 -- 172.21.15.69:0/1759271244 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48280bcb50 con 0x556d2f6552b0 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f481dffb640 1 -- 172.21.15.69:0/1759271244 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f482c0124f0 con 0x556d2f6552b0 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 10 client.18189 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 10 client.18189 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 20 client.18189 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d2f6daf40 0x556d2f6dd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 4 client.18189 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "82059677", "release": true, "await": true}] 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.737+0000 7f48428b3740 1 -- 172.21.15.69:0/1759271244 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "82059677", "release": true, "await": true}) -- 0x556d2f64c1d0 con 0x556d2f6daf40 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.738+0000 7f483e251640 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d2f6daf40 0x556d2f6dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.738+0000 7f481dffb640 10 client.18189 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:24.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.738+0000 7f483e251640 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d2f6daf40 0x556d2f6dd360 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=692c1a57f83f2bc server_cookie=c7596cfe62a31a0a in_seq=0 out_seq=0 2024-08-02T17:57:24.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.790+0000 7f481dffb640 1 -- 172.21.15.69:0/1759271244 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556d2f64c1d0 con 0x556d2f6daf40 2024-08-02T17:57:24.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.790+0000 7f481dffb640 10 client.18189 handle_command_reply: tid=0 2024-08-02T17:57:24.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.790+0000 7f48428b3740 1 client.18189 shutdown 2024-08-02T17:57:24.792 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 -- 172.21.15.69:0/1759271244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 msgr2=0x556d2f65a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65a030 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x556d2f65c270 tx=0x7f482c00ef90 comp rx=0 tx=0).stop 2024-08-02T17:57:24.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 -- 172.21.15.69:0/1759271244 shutdown_connections 2024-08-02T17:57:24.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556d2f6552b0 0x556d2f65a030 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556d2f6daf40 0x556d2f6dd360 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556d2f65fb70 0x556d2f65a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 --2- 172.21.15.69:0/1759271244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556d2f654990 0x556d2f659af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.793 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 -- 172.21.15.69:0/1759271244 >> 172.21.15.69:0/1759271244 conn(0x556d2f650f20 msgr2=0x556d2f650080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f481dffb640 0 client.18189 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.791+0000 7f48428b3740 1 -- 172.21.15.69:0/1759271244 shutdown_connections 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.792+0000 7f48428b3740 1 -- 172.21.15.69:0/1759271244 wait complete. 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.792+0000 7f48428b3740 20 client.18189 trim_cache size 0 max 16384 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 40, 2024-08-02T17:57:24.794 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "82059677": { 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 40, 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 139.9, 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:57:24.795 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:57:24.796 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:57:24.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.792+0000 7f48404b5640 1 -- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48200a2cf0 msgr2=0x7f48200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.792+0000 7f48404b5640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48200a2cf0 0x7f48200a51b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f48300155f0 tx=0x7f4830006b20 comp rx=0 tx=0).stop 2024-08-02T17:57:24.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 -- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 msgr2=0x7f48381982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 0x7f48381982b0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f4828005cc0 tx=0x7f482802f770 comp rx=0 tx=0).stop 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 -- 172.21.15.69:0/3915845666 shutdown_connections 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f48200a2cf0 0x7f48200a51b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f483810be90 0x7f483819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4838103380 0x7f48381987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 --2- 172.21.15.69:0/3915845666 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4838102a60 0x7f48381982b0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 -- 172.21.15.69:0/3915845666 >> 172.21.15.69:0/3915845666 conn(0x7f48380fc4c0 msgr2=0x7f48380fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 -- 172.21.15.69:0/3915845666 shutdown_connections 2024-08-02T17:57:24.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:24.793+0000 7f48404b5640 1 -- 172.21.15.69:0/3915845666 wait complete. 2024-08-02T17:57:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 615.0 s, 8.0 tps, lat 16969.061 ms stddev 5636.275 2024-08-02T17:57:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:25 smithi069 ceph-mon[32746]: pgmap v342: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 40 KiB/s wr, 124 op/s 2024-08-02T17:57:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:25 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:57:25.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:25 smithi083 ceph-mon[37031]: pgmap v342: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 40 KiB/s wr, 124 op/s 2024-08-02T17:57:25.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:25 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:57:25.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:25 smithi077 ceph-mon[37108]: pgmap v342: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 40 KiB/s wr, 124 op/s 2024-08-02T17:57:25.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:25 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:57:25.503 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '82059677', total seconds elapsed: 25.1 2024-08-02T17:57:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.698+0000 7fb4daa28640 1 -- 172.21.15.69:0/2322986520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d4108f20 msgr2=0x7fb4d410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.698+0000 7fb4daa28640 1 --2- 172.21.15.69:0/2322986520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d4108f20 0x7fb4d410ff80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009890 tx=0x7fb4cc02a3b0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.702+0000 7fb4daa28640 1 -- 172.21.15.69:0/2322986520 shutdown_connections 2024-08-02T17:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.702+0000 7fb4daa28640 1 --2- 172.21.15.69:0/2322986520 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d411ae70 0x7fb4d411d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.702+0000 7fb4daa28640 1 --2- 172.21.15.69:0/2322986520 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4109300 0x7fb4d4109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.702+0000 7fb4daa28640 1 --2- 172.21.15.69:0/2322986520 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d4108f20 0x7fb4d410ff80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.702+0000 7fb4daa28640 1 -- 172.21.15.69:0/2322986520 >> 172.21.15.69:0/2322986520 conn(0x7fb4d406cb20 msgr2=0x7fb4d406cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:26.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.708+0000 7fb4daa28640 1 -- 172.21.15.69:0/2322986520 shutdown_connections 2024-08-02T17:57:26.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.712+0000 7fb4daa28640 1 -- 172.21.15.69:0/2322986520 wait complete. 2024-08-02T17:57:26.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.713+0000 7fb4daa28640 1 Processor -- start 2024-08-02T17:57:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.714+0000 7fb4daa28640 1 -- start start 2024-08-02T17:57:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.726+0000 7fb4daa28640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d4108f20 0x7fb4d411a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.726+0000 7fb4daa28640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4109300 0x7fb4d4114710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.726+0000 7fb4daa28640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d411ae70 0x7fb4d4114c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.726+0000 7fb4daa28640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4111060 con 0x7fb4d4108f20 2024-08-02T17:57:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.726+0000 7fb4daa28640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4110ee0 con 0x7fb4d411ae70 2024-08-02T17:57:26.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.726+0000 7fb4daa28640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41111e0 con 0x7fb4d4109300 2024-08-02T17:57:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.729+0000 7fb4d8fc5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d411ae70 0x7fb4d4114c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.729+0000 7fb4d8fc5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d411ae70 0x7fb4d4114c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52084/0 (socket says 172.21.15.69:52084) 2024-08-02T17:57:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.729+0000 7fb4d8fc5640 1 -- 172.21.15.69:0/785306994 learned_addr learned my addr 172.21.15.69:0/785306994 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:26.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.729+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4109300 0x7fb4d4114710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.732 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.730+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d4108f20 0x7fb4d411a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.731+0000 7fb4d8fc5640 1 -- 172.21.15.69:0/785306994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4109300 msgr2=0x7fb4d4114710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.731+0000 7fb4d8fc5640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4109300 0x7fb4d4114710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.731+0000 7fb4d8fc5640 1 -- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d4108f20 msgr2=0x7fb4d411a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.731+0000 7fb4d8fc5640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d4108f20 0x7fb4d411a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.733 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.731+0000 7fb4d8fc5640 1 -- 172.21.15.69:0/785306994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d4115450 con 0x7fb4d411ae70 2024-08-02T17:57:26.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.731+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d4108f20 0x7fb4d411a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.735+0000 7fb4d8fc5640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d411ae70 0x7fb4d4114c50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00efc0 tx=0x7fb4bc00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.736+0000 7fb4d17fa640 1 -- 172.21.15.69:0/785306994 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc003970 con 0x7fb4d411ae70 2024-08-02T17:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.736+0000 7fb4d17fa640 1 -- 172.21.15.69:0/785306994 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc004020 con 0x7fb4d411ae70 2024-08-02T17:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.736+0000 7fb4daa28640 1 -- 172.21.15.69:0/785306994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d41bbdc0 con 0x7fb4d411ae70 2024-08-02T17:57:26.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.737+0000 7fb4daa28640 1 -- 172.21.15.69:0/785306994 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41bc2e0 con 0x7fb4d411ae70 2024-08-02T17:57:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.739+0000 7fb4d17fa640 1 -- 172.21.15.69:0/785306994 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00b9f0 con 0x7fb4d411ae70 2024-08-02T17:57:26.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.740+0000 7fb4dd77f740 1 Processor -- start 2024-08-02T17:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.743+0000 7fb4d17fa640 1 -- 172.21.15.69:0/785306994 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb4bc01d020 con 0x7fb4d411ae70 2024-08-02T17:57:26.744 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.743+0000 7fb4dd77f740 1 -- start start 2024-08-02T17:57:26.749 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.744+0000 7fb4d17fa640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4c00a2f00 0x7fb4c00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.744+0000 7fb4d17fa640 1 -- 172.21.15.69:0/785306994 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7fb4bc0d06d0 con 0x7fb4d411ae70 2024-08-02T17:57:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.746+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4c00a2f00 0x7fb4c00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.749+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e205291ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.749+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e20529c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.749+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e20529ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.749+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e2051f59b0 con 0x55e2052920e0 2024-08-02T17:57:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.749+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e20519d800 con 0x55e2052917c0 2024-08-02T17:57:26.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.749+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e2051f1970 con 0x55e20529c980 2024-08-02T17:57:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.749+0000 7fb4d37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e20529c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.750+0000 7fb4d37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e20529c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58418/0 (socket says 172.21.15.69:58418) 2024-08-02T17:57:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.750+0000 7fb4d37fe640 1 -- 172.21.15.69:0/3108494915 learned_addr learned my addr 172.21.15.69:0/3108494915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:26.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.750+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4c00a2f00 0x7fb4c00a53c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40095f0 tx=0x7fb4c4005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.750+0000 7fb4d37fe640 1 -- 172.21.15.69:0/3108494915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 msgr2=0x55e20529ed70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:57:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.750+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e20529ed70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.750+0000 7fb4d37fe640 1 -- 172.21.15.69:0/3108494915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 msgr2=0x55e205291ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:57:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.750+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e205291ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.751+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e205291ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.751+0000 7fb4d37fe640 1 -- 172.21.15.69:0/3108494915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e204f09210 con 0x55e2052920e0 2024-08-02T17:57:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.751+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e205291ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:26.753 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.752+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e20529c440 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009460 tx=0x7fb4c401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923b6a39ac8a82a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.757+0000 7fb4ca7fc640 1 -- 172.21.15.69:0/3108494915 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4012e60 con 0x55e2052920e0 2024-08-02T17:57:26.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.757+0000 7fb4ca7fc640 1 -- 172.21.15.69:0/3108494915 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c4022070 con 0x55e2052920e0 2024-08-02T17:57:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.758+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3108494915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 msgr2=0x55e20529c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:26.759 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.758+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e20529c440 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009460 tx=0x7fb4c401c000 comp rx=0 tx=0).stop 2024-08-02T17:57:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.760+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3108494915 shutdown_connections 2024-08-02T17:57:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.760+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e20529c440 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.760+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e20529ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.760+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3108494915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e205291ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.760+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3108494915 >> 172.21.15.69:0/3108494915 conn(0x55e20528dd30 msgr2=0x55e20528e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:26.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.761+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3108494915 shutdown_connections 2024-08-02T17:57:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.764+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3108494915 wait complete. 2024-08-02T17:57:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.764+0000 7fb4dd77f740 1 -- start start 2024-08-02T17:57:26.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.764+0000 7fb4dd77f740 1 Processor -- start 2024-08-02T17:57:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.765+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e2052998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.765+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e205299de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.765+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e20529a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.765+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e20529f5a0 con 0x55e2052920e0 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.765+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e2051b1500 con 0x55e2052917c0 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.765+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e20529f720 con 0x55e20529c980 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.765+0000 7fb4dd77f740 10 client.0 _ll_register_callbacks cb 0x55e204f08a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.766+0000 7fb4d8fc5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e20529a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.767+0000 7fb4d37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e205299de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.767+0000 7fb4d37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e205299de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:58432/0 (socket says 172.21.15.69:58432) 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.767+0000 7fb4d37fe640 1 -- 172.21.15.69:0/3175138083 learned_addr learned my addr 172.21.15.69:0/3175138083 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.767+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e2052998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.768+0000 7fb4d37fe640 1 -- 172.21.15.69:0/3175138083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 msgr2=0x55e20529a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.768+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e20529a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.768+0000 7fb4d37fe640 1 -- 172.21.15.69:0/3175138083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 msgr2=0x55e2052998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.768+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e2052998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.770 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.768+0000 7fb4d37fe640 1 -- 172.21.15.69:0/3175138083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e204fd5620 con 0x55e2052920e0 2024-08-02T17:57:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.774+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e205299de0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb4c401e490 tx=0x7fb4c401fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6055c59d3cec2171 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.774+0000 7fb4b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:57:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.774+0000 7fb4b37fe640 1 -- 172.21.15.69:0/3175138083 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4012e60 con 0x55e2052920e0 2024-08-02T17:57:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.774+0000 7fb4b37fe640 1 -- 172.21.15.69:0/3175138083 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7fb4c40052c0 con 0x55e2052920e0 2024-08-02T17:57:26.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.775+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3175138083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c4022070 con 0x55e2052920e0 2024-08-02T17:57:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.775+0000 7fb4b37fe640 1 -- 172.21.15.69:0/3175138083 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4c401fe40 con 0x55e2052920e0 2024-08-02T17:57:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.776+0000 7fb4dd77f740 10 client.18201 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:26.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.776+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3175138083 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e2052ab240 con 0x55e2052920e0 2024-08-02T17:57:26.786 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.785+0000 7fb4b37fe640 1 -- 172.21.15.69:0/3175138083 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c4022070 con 0x55e2052920e0 2024-08-02T17:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.785+0000 7fb4b37fe640 1 -- 172.21.15.69:0/3175138083 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb4c403a020 con 0x55e2052920e0 2024-08-02T17:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.785+0000 7fb4dd77f740 10 client.18201 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.785+0000 7fb4dd77f740 10 client.18201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.785+0000 7fb4dd77f740 20 client.18201 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.785+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2052aca60 0x55e2052aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.785+0000 7fb4dd77f740 4 client.18201 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:26.787 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.785+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3175138083 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2052030e0 con 0x55e2052aca60 2024-08-02T17:57:26.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.797+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2052aca60 0x55e2052aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.813+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2052aca60 0x55e2052aee80 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d504e6809561cda3 server_cookie=a42d62c22eaa87ac in_seq=0 out_seq=0 2024-08-02T17:57:26.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.815+0000 7fb4b37fe640 10 client.18201 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.817+0000 7fb4b37fe640 1 -- 172.21.15.69:0/3175138083 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e2052030e0 con 0x55e2052aca60 2024-08-02T17:57:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.817+0000 7fb4b37fe640 10 client.18201 handle_command_reply: tid=0 2024-08-02T17:57:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.817+0000 7fb4dd77f740 1 client.18201 shutdown 2024-08-02T17:57:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.819+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3175138083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 msgr2=0x55e205299de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:26.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.819+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e205299de0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fb4c401e490 tx=0x7fb4c401fc50 comp rx=0 tx=0).stop 2024-08-02T17:57:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.824+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3175138083 shutdown_connections 2024-08-02T17:57:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.824+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052920e0 0x55e205299de0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.824+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e2052aca60 0x55e2052aee80 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.824+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e20529a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.824+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/3175138083 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052917c0 0x55e2052998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.824+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3175138083 >> 172.21.15.69:0/3175138083 conn(0x55e20528dd30 msgr2=0x55e205293a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:26.827 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.824+0000 7fb4b37fe640 0 client.18201 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.826+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3175138083 shutdown_connections 2024-08-02T17:57:26.828 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.826+0000 7fb4dd77f740 1 -- 172.21.15.69:0/3175138083 wait complete. 2024-08-02T17:57:26.832 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.828+0000 7fb4dd77f740 20 client.18201 trim_cache size 0 max 16384 2024-08-02T17:57:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.866+0000 7fb4dd77f740 1 Processor -- start 2024-08-02T17:57:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.868+0000 7fb4dd77f740 1 -- start start 2024-08-02T17:57:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.868+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 0x55e2052b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.868+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.868+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e205296ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.868+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4012a50 con 0x55e2052917c0 2024-08-02T17:57:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.868+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b4010320 con 0x55e2052920e0 2024-08-02T17:57:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.868+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b400e170 con 0x55e20529c980 2024-08-02T17:57:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.868+0000 7fb4d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.869+0000 7fb4d37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052b4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:52122/0 (socket says 172.21.15.69:52122) 2024-08-02T17:57:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.869+0000 7fb4d37fe640 1 -- 172.21.15.69:0/4047695984 learned_addr learned my addr 172.21.15.69:0/4047695984 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.871+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/4047695984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 0x55e2052b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.871+0000 7fb4d37fe640 1 -- 172.21.15.69:0/4047695984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 msgr2=0x55e205296ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:57:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.871+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/4047695984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e205296ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.871+0000 7fb4d37fe640 1 -- 172.21.15.69:0/4047695984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 msgr2=0x55e2052b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.871+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/4047695984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 0x55e2052b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.871+0000 7fb4d37fe640 1 -- 172.21.15.69:0/4047695984 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20528db60 con 0x55e2052920e0 2024-08-02T17:57:26.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.872+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/4047695984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052b4c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4026430 tx=0x7fb4c4004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b621237f9ac1495d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.875+0000 7fb4c9ffb640 1 -- 172.21.15.69:0/4047695984 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4004cd0 con 0x55e2052920e0 2024-08-02T17:57:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.875+0000 7fb4c9ffb640 1 -- 172.21.15.69:0/4047695984 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c4022070 con 0x55e2052920e0 2024-08-02T17:57:26.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.875+0000 7fb4c9ffb640 1 -- 172.21.15.69:0/4047695984 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4005320 con 0x55e2052920e0 2024-08-02T17:57:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.876+0000 7fb4dd77f740 1 -- 172.21.15.69:0/4047695984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 msgr2=0x55e2052b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.876+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/4047695984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052b4c90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4026430 tx=0x7fb4c4004990 comp rx=0 tx=0).stop 2024-08-02T17:57:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.876+0000 7fb4dd77f740 1 -- 172.21.15.69:0/4047695984 shutdown_connections 2024-08-02T17:57:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.876+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/4047695984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e205296ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.876+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/4047695984 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052b4c90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.876+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/4047695984 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 0x55e2052b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.876+0000 7fb4dd77f740 1 -- 172.21.15.69:0/4047695984 >> 172.21.15.69:0/4047695984 conn(0x55e20528dd30 msgr2=0x55e20529b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.883+0000 7fb4dd77f740 1 -- 172.21.15.69:0/4047695984 shutdown_connections 2024-08-02T17:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.883+0000 7fb4dd77f740 1 -- 172.21.15.69:0/4047695984 wait complete. 2024-08-02T17:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.883+0000 7fb4dd77f740 1 -- start start 2024-08-02T17:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.883+0000 7fb4dd77f740 1 Processor -- start 2024-08-02T17:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.884+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 0x55e2052a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.884+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.884+0000 7fb4dd77f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e2052a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.884+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b400e7e0 con 0x55e2052917c0 2024-08-02T17:57:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.884+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b400fc50 con 0x55e2052920e0 2024-08-02T17:57:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.884+0000 7fb4dd77f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b400eeb0 con 0x55e20529c980 2024-08-02T17:57:26.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.884+0000 7fb4dd77f740 10 client.0 _ll_register_callbacks cb 0x7fb4c4011e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.885+0000 7fb4d8fc5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e2052a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.885+0000 7fb4d8fc5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e2052a08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39926/0 (socket says 172.21.15.69:39926) 2024-08-02T17:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.885+0000 7fb4d8fc5640 1 -- 172.21.15.69:0/1822407898 learned_addr learned my addr 172.21.15.69:0/1822407898 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.885+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.886+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 0x55e2052a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.886+0000 7fb4d37fe640 1 -- 172.21.15.69:0/1822407898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 msgr2=0x55e2052a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.886+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e2052a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.886+0000 7fb4d37fe640 1 -- 172.21.15.69:0/1822407898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 msgr2=0x55e2052a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.886+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 0x55e2052a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:26.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.886+0000 7fb4d37fe640 1 -- 172.21.15.69:0/1822407898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2052a0eb0 con 0x55e2052920e0 2024-08-02T17:57:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.889+0000 7fb4d37fe640 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052a03b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55e2052996b0 tx=0x7fb4c4027ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6685530bde5e8d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:26.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.889+0000 7fb4b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.896+0000 7fb4b37fe640 1 -- 172.21.15.69:0/1822407898 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4048060 con 0x55e2052920e0 2024-08-02T17:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.896+0000 7fb4dd77f740 1 -- 172.21.15.69:0/1822407898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c4022070 con 0x55e2052920e0 2024-08-02T17:57:26.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.896+0000 7fb4b37fe640 1 -- 172.21.15.69:0/1822407898 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7fb4c4026ab0 con 0x55e2052920e0 2024-08-02T17:57:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.902+0000 7fb4b37fe640 1 -- 172.21.15.69:0/1822407898 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4c401e950 con 0x55e2052920e0 2024-08-02T17:57:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.905+0000 7fb4dd77f740 10 client.27629 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.905+0000 7fb4dd77f740 1 -- 172.21.15.69:0/1822407898 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e205317500 con 0x55e2052920e0 2024-08-02T17:57:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.908+0000 7fb4b37fe640 1 -- 172.21.15.69:0/1822407898 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c4022070 con 0x55e2052920e0 2024-08-02T17:57:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.908+0000 7fb4b37fe640 1 -- 172.21.15.69:0/1822407898 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb4c403a040 con 0x55e2052920e0 2024-08-02T17:57:26.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.908+0000 7fb4dd77f740 10 client.27629 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.908+0000 7fb4dd77f740 10 client.27629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.908+0000 7fb4dd77f740 20 client.27629 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.908+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e205317930 0x55e205319d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.908+0000 7fb4dd77f740 4 client.27629 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:57:26.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.908+0000 7fb4dd77f740 1 -- 172.21.15.69:0/1822407898 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e2051fd430 con 0x55e205317930 2024-08-02T17:57:26.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.910+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e205317930 0x55e205319d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:26.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.910+0000 7fb4b37fe640 10 client.27629 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:26.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:26.911+0000 7fb4d3fff640 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e205317930 0x55e205319d20 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566cf2b53bb26837 server_cookie=806d627d9907000e in_seq=0 out_seq=0 2024-08-02T17:57:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4b37fe640 1 -- 172.21.15.69:0/1822407898 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e2051fd430 con 0x55e205317930 2024-08-02T17:57:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4b37fe640 10 client.27629 handle_command_reply: tid=0 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 client.27629 shutdown 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 -- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 msgr2=0x55e2052a03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052a03b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x55e2052996b0 tx=0x7fb4c4027ed0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 -- 172.21.15.69:0/1822407898 shutdown_connections 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e205317930 0x55e205319d20 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e20529c980 0x55e2052a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e2052920e0 0x55e2052a03b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4b37fe640 0 client.27629 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 --2- 172.21.15.69:0/1822407898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e2052917c0 0x55e2052a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.010+0000 7fb4dd77f740 1 -- 172.21.15.69:0/1822407898 >> 172.21.15.69:0/1822407898 conn(0x55e20528dd30 msgr2=0x55e205293190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.011+0000 7fb4dd77f740 1 -- 172.21.15.69:0/1822407898 shutdown_connections 2024-08-02T17:57:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.011+0000 7fb4dd77f740 1 -- 172.21.15.69:0/1822407898 wait complete. 2024-08-02T17:57:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.011+0000 7fb4dd77f740 20 client.27629 trim_cache size 0 max 16384 2024-08-02T17:57:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:57:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:57:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:57:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:57:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 -- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4c00a2f00 msgr2=0x7fb4c00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4c00a2f00 0x7fb4c00a53c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40095f0 tx=0x7fb4c4005f90 comp rx=0 tx=0).stop 2024-08-02T17:57:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 -- 172.21.15.69:0/785306994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d411ae70 msgr2=0x7fb4d4114c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d411ae70 0x7fb4d4114c50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00efc0 tx=0x7fb4bc00c800 comp rx=0 tx=0).stop 2024-08-02T17:57:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 -- 172.21.15.69:0/785306994 shutdown_connections 2024-08-02T17:57:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4c00a2f00 0x7fb4c00a53c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4d411ae70 0x7fb4d4114c50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4d4109300 0x7fb4d4114710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 --2- 172.21.15.69:0/785306994 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb4d4108f20 0x7fb4d411a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 -- 172.21.15.69:0/785306994 >> 172.21.15.69:0/785306994 conn(0x7fb4d406cb20 msgr2=0x7fb4d40705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 -- 172.21.15.69:0/785306994 shutdown_connections 2024-08-02T17:57:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:27.012+0000 7fb4daa28640 1 -- 172.21.15.69:0/785306994 wait complete. 2024-08-02T17:57:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:27 smithi083 ceph-mon[37031]: pgmap v343: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 38 KiB/s wr, 117 op/s 2024-08-02T17:57:27.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:27 smithi077 ceph-mon[37108]: pgmap v343: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 38 KiB/s wr, 117 op/s 2024-08-02T17:57:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:27 smithi069 ceph-mon[32746]: pgmap v343: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 38 KiB/s wr, 117 op/s 2024-08-02T17:57:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:29 smithi083 ceph-mon[37031]: pgmap v344: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 711 KiB/s wr, 174 op/s 2024-08-02T17:57:29.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:29 smithi077 ceph-mon[37108]: pgmap v344: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 711 KiB/s wr, 174 op/s 2024-08-02T17:57:29.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:29 smithi069 ceph-mon[32746]: pgmap v344: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 711 KiB/s wr, 174 op/s 2024-08-02T17:57:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 620.0 s, 668.2 tps, lat 510.802 ms stddev 2627.682 2024-08-02T17:57:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:31 smithi083 ceph-mon[37031]: pgmap v345: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-08-02T17:57:31.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:31 smithi077 ceph-mon[37108]: pgmap v345: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-08-02T17:57:31.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:31 smithi069 ceph-mon[32746]: pgmap v345: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-08-02T17:57:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:33 smithi083 ceph-mon[37031]: pgmap v346: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-08-02T17:57:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:33 smithi077 ceph-mon[37108]: pgmap v346: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-08-02T17:57:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:33 smithi069 ceph-mon[32746]: pgmap v346: 129 pgs: 129 active+clean; 2.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-08-02T17:57:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 625.0 s, 719.4 tps, lat 137.422 ms stddev 147.174 2024-08-02T17:57:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:35 smithi077 ceph-mon[37108]: pgmap v347: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 6.2 MiB/s wr, 479 op/s 2024-08-02T17:57:35.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:35 smithi069 ceph-mon[32746]: pgmap v347: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 6.2 MiB/s wr, 479 op/s 2024-08-02T17:57:35.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:35 smithi083 ceph-mon[37031]: pgmap v347: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 6.2 MiB/s wr, 479 op/s 2024-08-02T17:57:37.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:37 smithi077 ceph-mon[37108]: pgmap v348: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 388 op/s 2024-08-02T17:57:37.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:37 smithi069 ceph-mon[32746]: pgmap v348: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 388 op/s 2024-08-02T17:57:37.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:37 smithi083 ceph-mon[37031]: pgmap v348: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 388 op/s 2024-08-02T17:57:39.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:39 smithi077 ceph-mon[37108]: pgmap v349: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.8 MiB/s wr, 480 op/s 2024-08-02T17:57:39.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:39 smithi069 ceph-mon[32746]: pgmap v349: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.8 MiB/s wr, 480 op/s 2024-08-02T17:57:39.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:39 smithi083 ceph-mon[37031]: pgmap v349: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 7.8 MiB/s wr, 480 op/s 2024-08-02T17:57:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 630.0 s, 606.6 tps, lat 161.478 ms stddev 193.601 2024-08-02T17:57:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.169+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/952331589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1abc109300 msgr2=0x7f1abc109740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.169+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/952331589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1abc109300 0x7f1abc109740 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009860 tx=0x7f1aa402a360 comp rx=0 tx=0).stop 2024-08-02T17:57:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.170+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/952331589 shutdown_connections 2024-08-02T17:57:40.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.170+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/952331589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1abc109300 0x7f1abc109740 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.170+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/952331589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1abc11ae70 0x7f1abc11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.170+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/952331589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1abc108f20 0x7f1abc10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.170+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/952331589 >> 172.21.15.69:0/952331589 conn(0x7f1abc06cb20 msgr2=0x7f1abc06cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.175+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/952331589 shutdown_connections 2024-08-02T17:57:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.177+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/952331589 wait complete. 2024-08-02T17:57:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1ac1e5d640 1 Processor -- start 2024-08-02T17:57:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1ac1e5d640 1 -- start start 2024-08-02T17:57:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1ac1e5d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1abc108f20 0x7f1abc118370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1ac1e5d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1abc109300 0x7f1abc1188b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1ac1e5d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1abc11ae70 0x7f1abc111620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1ac1e5d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc120a80 con 0x7f1abc108f20 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1ac1e5d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc120900 con 0x7f1abc11ae70 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1ac1e5d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc120c00 con 0x7f1abc109300 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1abbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1abc11ae70 0x7f1abc111620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1abbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1abc11ae70 0x7f1abc111620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:57454/0 (socket says 172.21.15.69:57454) 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.178+0000 7f1abbfff640 1 -- 172.21.15.69:0/687159930 learned_addr learned my addr 172.21.15.69:0/687159930 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.179+0000 7f1abbfff640 1 -- 172.21.15.69:0/687159930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1abc109300 msgr2=0x7f1abc1188b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.179+0000 7f1abbfff640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1abc109300 0x7f1abc1188b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.179+0000 7f1abbfff640 1 -- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1abc108f20 msgr2=0x7f1abc118370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.179+0000 7f1abbfff640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1abc108f20 0x7f1abc118370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.179+0000 7f1abbfff640 1 -- 172.21.15.69:0/687159930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc111ee0 con 0x7f1abc11ae70 2024-08-02T17:57:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.179+0000 7f1abbfff640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1abc11ae70 0x7f1abc111620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400e970 tx=0x7f1ab400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:40.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.180+0000 7f1ab97fa640 1 -- 172.21.15.69:0/687159930 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400ccc0 con 0x7f1abc11ae70 2024-08-02T17:57:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.180+0000 7f1ab97fa640 1 -- 172.21.15.69:0/687159930 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab4005500 con 0x7f1abc11ae70 2024-08-02T17:57:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.180+0000 7f1ab97fa640 1 -- 172.21.15.69:0/687159930 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4010690 con 0x7f1abc11ae70 2024-08-02T17:57:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.182+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/687159930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc1121d0 con 0x7f1abc11ae70 2024-08-02T17:57:40.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.182+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/687159930 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc1160d0 con 0x7f1abc11ae70 2024-08-02T17:57:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.202+0000 7f1ac4bbc740 1 Processor -- start 2024-08-02T17:57:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.204+0000 7f1ac4bbc740 1 -- start start 2024-08-02T17:57:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.204+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af969fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.204+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fc0d0 0x55af96a06430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.204+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a08d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.204+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55af9695f9b0 con 0x55af969fc0d0 2024-08-02T17:57:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.204+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af96907800 con 0x55af969fb7b0 2024-08-02T17:57:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.204+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af9695b970 con 0x55af96a06970 2024-08-02T17:57:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.205+0000 7f1ab97fa640 1 -- 172.21.15.69:0/687159930 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1ab4004e10 con 0x7f1abc11ae70 2024-08-02T17:57:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.205+0000 7f1abb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fc0d0 0x55af96a06430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.206+0000 7f1abb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fc0d0 0x55af96a06430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55268/0 (socket says 172.21.15.69:55268) 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.206+0000 7f1abb7fe640 1 -- 172.21.15.69:0/3370807285 learned_addr learned my addr 172.21.15.69:0/3370807285 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.206+0000 7f1ab97fa640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1aa80a2f00 0x7f1aa80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.206+0000 7f1ab97fa640 1 -- 172.21.15.69:0/687159930 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f1ab40cfc70 con 0x7f1abc11ae70 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.207+0000 7f1abbfff640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1aa80a2f00 0x7f1aa80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.207+0000 7f1abb7fe640 1 -- 172.21.15.69:0/3370807285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 msgr2=0x55af96a08d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.207+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a08d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.207+0000 7f1abb7fe640 1 -- 172.21.15.69:0/3370807285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 msgr2=0x55af969fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.207+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af969fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.207+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af969fbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.210 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.207+0000 7f1abb7fe640 1 -- 172.21.15.69:0/3370807285 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af96673210 con 0x55af969fc0d0 2024-08-02T17:57:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.207+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af969fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.208+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fc0d0 0x55af96a06430 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009da0 tx=0x7f1aac00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718b072df6de5a02 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.208+0000 7f1abbfff640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1aa80a2f00 0x7f1aa80a53c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ab409ba40 tx=0x7f1ab4004a90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.209+0000 7f1ab1ffb640 1 -- 172.21.15.69:0/3370807285 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00bcc0 con 0x55af969fc0d0 2024-08-02T17:57:40.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.209+0000 7f1ab1ffb640 1 -- 172.21.15.69:0/3370807285 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aac017070 con 0x55af969fc0d0 2024-08-02T17:57:40.214 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3370807285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fc0d0 msgr2=0x55af96a06430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fc0d0 0x55af96a06430 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009da0 tx=0x7f1aac00b420 comp rx=0 tx=0).stop 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3370807285 shutdown_connections 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fc0d0 0x55af96a06430 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a08d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3370807285 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af969fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3370807285 >> 172.21.15.69:0/3370807285 conn(0x55af969f7d20 msgr2=0x55af969f8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3370807285 shutdown_connections 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.210+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3370807285 wait complete. 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 1 -- start start 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 1 Processor -- start 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af969fb7b0 0x55af96a063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a06970 0x55af96a03680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55af96a09590 con 0x55af96a06970 2024-08-02T17:57:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55af9691b500 con 0x55af96a03d20 2024-08-02T17:57:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55af96a09710 con 0x55af969fb7b0 2024-08-02T17:57:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ac4bbc740 10 client.0 _ll_register_callbacks cb 0x55af96672a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.211+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:57476/0 (socket says 172.21.15.69:57476) 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 -- 172.21.15.69:0/186307082 learned_addr learned my addr 172.21.15.69:0/186307082 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af969fb7b0 0x55af96a063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a06970 0x55af96a03680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 -- 172.21.15.69:0/186307082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af969fb7b0 msgr2=0x55af96a063e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af969fb7b0 0x55af96a063e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 -- 172.21.15.69:0/186307082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a06970 msgr2=0x55af96a03680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a06970 0x55af96a03680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 -- 172.21.15.69:0/186307082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af9673f620 con 0x55af96a03d20 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.212+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12a90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4035040 tx=0x7f1aa4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a909887f3d9d2c4a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.213+0000 7f1a9affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.213+0000 7f1a9affd640 1 -- 172.21.15.69:0/186307082 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4037070 con 0x55af96a03d20 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.213+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/186307082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af9696d0e0 con 0x55af96a03d20 2024-08-02T17:57:40.217 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.213+0000 7f1a9affd640 1 -- 172.21.15.69:0/186307082 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f1aa4004460 con 0x55af96a03d20 2024-08-02T17:57:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.214+0000 7f1a9affd640 1 -- 172.21.15.69:0/186307082 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1aa4030940 con 0x55af96a03d20 2024-08-02T17:57:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.214+0000 7f1ac4bbc740 10 client.27638 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.214+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/186307082 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af96a15540 con 0x55af96a03d20 2024-08-02T17:57:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.214+0000 7f1a9affd640 1 -- 172.21.15.69:0/186307082 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55af9696d0e0 con 0x55af96a03d20 2024-08-02T17:57:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.216+0000 7f1a9affd640 1 -- 172.21.15.69:0/186307082 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1aa4004b80 con 0x55af96a03d20 2024-08-02T17:57:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.216+0000 7f1ac4bbc740 10 client.27638 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.216+0000 7f1ac4bbc740 10 client.27638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.216+0000 7f1ac4bbc740 20 client.27638 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.216+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55af96a16de0 0x55af96a191d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.216+0000 7f1ac4bbc740 4 client.27638 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.216+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/186307082 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af96967430 con 0x55af96a16de0 2024-08-02T17:57:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.217+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55af96a16de0 0x55af96a191d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.219+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55af96a16de0 0x55af96a191d0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c983c7edaf69b258 server_cookie=c6287f0380b6dd43 in_seq=0 out_seq=0 2024-08-02T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.219+0000 7f1a9affd640 10 client.27638 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.220+0000 7f1a9affd640 1 -- 172.21.15.69:0/186307082 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55af96967430 con 0x55af96a16de0 2024-08-02T17:57:40.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.220+0000 7f1a9affd640 10 client.27638 handle_command_reply: tid=0 2024-08-02T17:57:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.220+0000 7f1ac4bbc740 1 client.27638 shutdown 2024-08-02T17:57:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.221+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/186307082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 msgr2=0x55af96a12a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:40.222 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.221+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12a90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4035040 tx=0x7f1aa4000bc0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.222+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/186307082 shutdown_connections 2024-08-02T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.222+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a06970 0x55af96a03680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.222+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55af96a16de0 0x55af96a191d0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.222+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12a90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.222+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/186307082 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af969fb7b0 0x55af96a063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.222+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/186307082 >> 172.21.15.69:0/186307082 conn(0x55af969f7d20 msgr2=0x55af969f6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.223+0000 7f1a9affd640 0 client.27638 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.226+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/186307082 shutdown_connections 2024-08-02T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.226+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/186307082 wait complete. 2024-08-02T17:57:40.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.226+0000 7f1ac4bbc740 20 client.27638 trim_cache size 0 max 16384 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ac4bbc740 1 Processor -- start 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ac4bbc740 1 -- start start 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fb7b0 0x55af969fd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a12dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1d9cc0 con 0x55af969fb7b0 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1b5580 con 0x55af96a03d20 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1b33d0 con 0x55af96a06970 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fb7b0 0x55af969fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1abb7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fb7b0 0x55af969fd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55294/0 (socket says 172.21.15.69:55294) 2024-08-02T17:57:40.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1ab3fff640 1 -- 172.21.15.69:0/1835444064 learned_addr learned my addr 172.21.15.69:0/1835444064 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.249+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:57484/0 (socket says 172.21.15.69:57484) 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a12dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1abb7fe640 1 -- 172.21.15.69:0/1835444064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 msgr2=0x55af96a12dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a12dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1abb7fe640 1 -- 172.21.15.69:0/1835444064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fb7b0 msgr2=0x55af969fd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fb7b0 0x55af969fd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1abb7fe640 1 -- 172.21.15.69:0/1835444064 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af96a13830 con 0x55af96a03d20 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098a0 tx=0x7f1aac009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8188f0ba855a05 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1ab17fa640 1 -- 172.21.15.69:0/1835444064 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac011390 con 0x55af96a03d20 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1ab17fa640 1 -- 172.21.15.69:0/1835444064 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aac017040 con 0x55af96a03d20 2024-08-02T17:57:40.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.250+0000 7f1ab17fa640 1 -- 172.21.15.69:0/1835444064 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac011ec0 con 0x55af96a03d20 2024-08-02T17:57:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.251+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/1835444064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 msgr2=0x55af96a12880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.251+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12880 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0098a0 tx=0x7f1aac009710 comp rx=0 tx=0).stop 2024-08-02T17:57:40.256 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.251+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/1835444064 shutdown_connections 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.251+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a12dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.251+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af96a03d20 0x55af96a12880 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.251+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/1835444064 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af969fb7b0 0x55af969fd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.251+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/1835444064 >> 172.21.15.69:0/1835444064 conn(0x55af969f7d20 msgr2=0x55af969fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.252+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/1835444064 shutdown_connections 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.252+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/1835444064 wait complete. 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.252+0000 7f1ac4bbc740 1 -- start start 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.252+0000 7f1ac4bbc740 1 Processor -- start 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af96a06d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a03d20 0x55af96a07290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ac4bbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a077d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ac4bbc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1b3a40 con 0x55af96a03d20 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1abb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a03d20 0x55af96a07290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af96a06d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ab3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af96a06d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:57488/0 (socket says 172.21.15.69:57488) 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ab3fff640 1 -- 172.21.15.69:0/3851786727 learned_addr learned my addr 172.21.15.69:0/3851786727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1b4eb0 con 0x55af969fb7b0 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc1b4110 con 0x55af96a06970 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ac4bbc740 10 client.0 _ll_register_callbacks cb 0x7f1aa4035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ab3fff640 1 -- 172.21.15.69:0/3851786727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 msgr2=0x55af96a077d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.253+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a077d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1ab3fff640 1 -- 172.21.15.69:0/3851786727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a03d20 msgr2=0x55af96a07290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a03d20 0x55af96a07290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1ab3fff640 1 -- 172.21.15.69:0/3851786727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af96a0dba0 con 0x55af969fb7b0 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1ab3fff640 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af96a06d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40095f0 tx=0x7f1aa4005710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=596a02c998134d0f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:40.258 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1a9affd640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1a9affd640 1 -- 172.21.15.69:0/3851786727 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4037040 con 0x55af969fb7b0 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aac017040 con 0x55af969fb7b0 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1a9affd640 1 -- 172.21.15.69:0/3851786727 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f1aa40046c0 con 0x55af969fb7b0 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1a9affd640 1 -- 172.21.15.69:0/3851786727 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1aa40048c0 con 0x55af969fb7b0 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1ac4bbc740 10 client.27644 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.254+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55af96a0ee70 con 0x55af969fb7b0 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1a9affd640 1 -- 172.21.15.69:0/3851786727 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aac017040 con 0x55af969fb7b0 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1a9affd640 1 -- 172.21.15.69:0/3851786727 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1aa40604d0 con 0x55af969fb7b0 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1ac4bbc740 10 client.27644 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1ac4bbc740 10 client.27644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1ac4bbc740 20 client.27644 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55af96a81ab0 0x55af96a83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:40.259 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1ac4bbc740 4 client.27644 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af968e48e0 con 0x55af96a81ab0 2024-08-02T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.255+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55af96a81ab0 0x55af96a83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.256+0000 7f1a9affd640 10 client.27644 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:40.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.256+0000 7f1abb7fe640 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55af96a81ab0 0x55af96a83ea0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f01c7897fdafa9 server_cookie=27b9cf7790e4a3e in_seq=0 out_seq=0 2024-08-02T17:57:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.274+0000 7f1a9affd640 1 -- 172.21.15.69:0/3851786727 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af968e48e0 con 0x55af96a81ab0 2024-08-02T17:57:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.274+0000 7f1a9affd640 10 client.27644 handle_command_reply: tid=0 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.274+0000 7f1ac4bbc740 1 client.27644 shutdown 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 msgr2=0x55af96a06d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af96a06d50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40095f0 tx=0x7f1aa4005710 comp rx=0 tx=0).stop 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 shutdown_connections 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55af96a03d20 0x55af96a07290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55af96a81ab0 0x55af96a83ea0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55af96a06970 0x55af96a077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 --2- 172.21.15.69:0/3851786727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55af969fb7b0 0x55af96a06d50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1a9affd640 0 client.27644 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 >> 172.21.15.69:0/3851786727 conn(0x55af969f7d20 msgr2=0x55af96a05700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 shutdown_connections 2024-08-02T17:57:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 1 -- 172.21.15.69:0/3851786727 wait complete. 2024-08-02T17:57:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.275+0000 7f1ac4bbc740 20 client.27644 trim_cache size 0 max 16384 2024-08-02T17:57:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:57:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:57:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:57:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1aa80a2f00 msgr2=0x7f1aa80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1aa80a2f00 0x7f1aa80a53c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ab409ba40 tx=0x7f1ab4004a90 comp rx=0 tx=0).stop 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/687159930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1abc11ae70 msgr2=0x7f1abc111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1abc11ae70 0x7f1abc111620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400e970 tx=0x7f1ab400ee30 comp rx=0 tx=0).stop 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/687159930 shutdown_connections 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f1aa80a2f00 0x7f1aa80a53c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1abc11ae70 0x7f1abc111620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1abc109300 0x7f1abc1188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.281 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 --2- 172.21.15.69:0/687159930 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1abc108f20 0x7f1abc118370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/687159930 >> 172.21.15.69:0/687159930 conn(0x7f1abc06cb20 msgr2=0x7f1abc11b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/687159930 shutdown_connections 2024-08-02T17:57:40.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:40.276+0000 7f1ac1e5d640 1 -- 172.21.15.69:0/687159930 wait complete. 2024-08-02T17:57:41.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:41 smithi083 ceph-mon[37031]: pgmap v350: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.3 MiB/s wr, 553 op/s 2024-08-02T17:57:41.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:41 smithi077 ceph-mon[37108]: pgmap v350: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.3 MiB/s wr, 553 op/s 2024-08-02T17:57:41.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:41 smithi069 ceph-mon[32746]: pgmap v350: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.3 MiB/s wr, 553 op/s 2024-08-02T17:57:43.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:43 smithi077 ceph-mon[37108]: pgmap v351: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 484 op/s 2024-08-02T17:57:43.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:43 smithi069 ceph-mon[32746]: pgmap v351: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 484 op/s 2024-08-02T17:57:43.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:43 smithi083 ceph-mon[37031]: pgmap v351: 129 pgs: 129 active+clean; 2.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 8.2 MiB/s wr, 484 op/s 2024-08-02T17:57:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 635.0 s, 602.2 tps, lat 169.015 ms stddev 207.890 2024-08-02T17:57:45.350 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:45 smithi077 ceph-mon[37108]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 732 op/s 2024-08-02T17:57:45.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:45 smithi069 ceph-mon[32746]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 732 op/s 2024-08-02T17:57:45.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:45 smithi083 ceph-mon[37031]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 732 op/s 2024-08-02T17:57:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:47 smithi077 ceph-mon[37108]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.5 MiB/s wr, 482 op/s 2024-08-02T17:57:47.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:47 smithi069 ceph-mon[32746]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.5 MiB/s wr, 482 op/s 2024-08-02T17:57:47.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:47 smithi083 ceph-mon[37031]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 7.5 MiB/s wr, 482 op/s 2024-08-02T17:57:49.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:49 smithi077 ceph-mon[37108]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.3 MiB/s wr, 562 op/s 2024-08-02T17:57:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:49 smithi069 ceph-mon[32746]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.3 MiB/s wr, 562 op/s 2024-08-02T17:57:49.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:49 smithi083 ceph-mon[37031]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 8.3 MiB/s wr, 562 op/s 2024-08-02T17:57:49.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 640.0 s, 897.8 tps, lat 112.635 ms stddev 142.005 2024-08-02T17:57:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:51 smithi077 ceph-mon[37108]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-08-02T17:57:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:51 smithi069 ceph-mon[32746]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-08-02T17:57:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:51 smithi083 ceph-mon[37031]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 609 op/s 2024-08-02T17:57:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:53 smithi069 ceph-mon[32746]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 466 op/s 2024-08-02T17:57:53.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.429+0000 7f5f9159a640 1 -- 172.21.15.69:0/82025720 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84003650 con 0x7f5f94106710 2024-08-02T17:57:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.430+0000 7f5f98a41640 1 -- 172.21.15.69:0/82025720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94106710 msgr2=0x7f5f94106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:53.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.430+0000 7f5f98a41640 1 --2- 172.21.15.69:0/82025720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94106710 0x7f5f94106af0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009930 tx=0x7f5f8402a480 comp rx=0 tx=0).stop 2024-08-02T17:57:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.434+0000 7f5f98a41640 1 -- 172.21.15.69:0/82025720 shutdown_connections 2024-08-02T17:57:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.434+0000 7f5f98a41640 1 --2- 172.21.15.69:0/82025720 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941010a0 0x7f5f9410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.434+0000 7f5f98a41640 1 --2- 172.21.15.69:0/82025720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94100700 0x7f5f94100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.434+0000 7f5f98a41640 1 --2- 172.21.15.69:0/82025720 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94106710 0x7f5f94106af0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.434+0000 7f5f98a41640 1 -- 172.21.15.69:0/82025720 >> 172.21.15.69:0/82025720 conn(0x7f5f940fc3e0 msgr2=0x7f5f940fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:53.439 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.438+0000 7f5f98a41640 1 -- 172.21.15.69:0/82025720 shutdown_connections 2024-08-02T17:57:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.439+0000 7f5f98a41640 1 -- 172.21.15.69:0/82025720 wait complete. 2024-08-02T17:57:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.440+0000 7f5f98a41640 1 Processor -- start 2024-08-02T17:57:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.442+0000 7f5f98a41640 1 -- start start 2024-08-02T17:57:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.443+0000 7f5f98a41640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94100700 0x7f5f94107630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.443+0000 7f5f98a41640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941010a0 0x7f5f94107b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.443+0000 7f5f98a41640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94106710 0x7f5f941080b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.443+0000 7f5f98a41640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111cb0 con 0x7f5f94100700 2024-08-02T17:57:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.443+0000 7f5f98a41640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111b30 con 0x7f5f941010a0 2024-08-02T17:57:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.443+0000 7f5f98a41640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94111e30 con 0x7f5f94106710 2024-08-02T17:57:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.444+0000 7f5f91d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941010a0 0x7f5f94107b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.444+0000 7f5f91d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941010a0 0x7f5f94107b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:57202/0 (socket says 172.21.15.69:57202) 2024-08-02T17:57:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.444+0000 7f5f91d9b640 1 -- 172.21.15.69:0/3888777152 learned_addr learned my addr 172.21.15.69:0/3888777152 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.445+0000 7f5f9259c640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94100700 0x7f5f94107630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.445+0000 7f5f91d9b640 1 -- 172.21.15.69:0/3888777152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94106710 msgr2=0x7f5f941080b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:57:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.445+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94106710 0x7f5f941080b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.445+0000 7f5f91d9b640 1 -- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94100700 msgr2=0x7f5f94107630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.447+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94100700 0x7f5f94107630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.447+0000 7f5f91d9b640 1 -- 172.21.15.69:0/3888777152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9419a200 con 0x7f5f941010a0 2024-08-02T17:57:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.448+0000 7f5f9259c640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94100700 0x7f5f94107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.448+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941010a0 0x7f5f94107b70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00eab0 tx=0x7f5f8c00ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.448+0000 7f5f837fe640 1 -- 172.21.15.69:0/3888777152 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00ce40 con 0x7f5f941010a0 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.448+0000 7f5f837fe640 1 -- 172.21.15.69:0/3888777152 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f8c011950 con 0x7f5f941010a0 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.448+0000 7f5f98a41640 1 -- 172.21.15.69:0/3888777152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9419a4f0 con 0x7f5f941010a0 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.449+0000 7f5f837fe640 1 -- 172.21.15.69:0/3888777152 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c010b40 con 0x7f5f941010a0 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.449+0000 7f5f98a41640 1 -- 172.21.15.69:0/3888777152 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f9419aa40 con 0x7f5f941010a0 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.451+0000 7f5f837fe640 1 -- 172.21.15.69:0/3888777152 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f8c010d40 con 0x7f5f941010a0 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.452+0000 7f5f837fe640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2de0 0x7f5f7c0a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.452+0000 7f5f837fe640 1 -- 172.21.15.69:0/3888777152 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f5f8c0cf3a0 con 0x7f5f941010a0 2024-08-02T17:57:53.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.452+0000 7f5f9259c640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2de0 0x7f5f7c0a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9259c640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2de0 0x7f5f7c0a52a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5f840096c0 tx=0x7f5f84005fd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9b793740 1 Processor -- start 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9b793740 1 -- start start 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4216b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc4221430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc4221970 0x55bcc4223d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9b793740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bcc417a9b0 con 0x55bcc4221970 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9b793740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bcc4122800 con 0x55bcc42167b0 2024-08-02T17:57:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.453+0000 7f5f9b793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bcc4176970 con 0x55bcc42170d0 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc4221430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc4221430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40352/0 (socket says 172.21.15.69:40352) 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 -- 172.21.15.69:0/1976042624 learned_addr learned my addr 172.21.15.69:0/1976042624 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f9259c640 1 --2- 172.21.15.69:0/1976042624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4216b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 -- 172.21.15.69:0/1976042624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 msgr2=0x55bcc4216b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/1976042624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4216b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 -- 172.21.15.69:0/1976042624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc4221970 msgr2=0x55bcc4223d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/1976042624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc4221970 0x55bcc4223d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 -- 172.21.15.69:0/1976042624 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc3e8e210 con 0x55bcc42170d0 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/1976042624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc4221430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c097ec0 tx=0x7f5f8c09bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=979c5ce656bf3530 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.454+0000 7f5f80ff9640 1 -- 172.21.15.69:0/1976042624 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0b3070 con 0x55bcc42170d0 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.455+0000 7f5f80ff9640 1 -- 172.21.15.69:0/1976042624 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f8c0a4160 con 0x55bcc42170d0 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.455+0000 7f5f80ff9640 1 -- 172.21.15.69:0/1976042624 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0b4260 con 0x55bcc42170d0 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 -- 172.21.15.69:0/1976042624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 msgr2=0x55bcc4221430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 --2- 172.21.15.69:0/1976042624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc4221430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c097ec0 tx=0x7f5f8c09bbf0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 -- 172.21.15.69:0/1976042624 shutdown_connections 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 --2- 172.21.15.69:0/1976042624 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc4221970 0x55bcc4223d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 --2- 172.21.15.69:0/1976042624 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc4221430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 --2- 172.21.15.69:0/1976042624 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4216b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 -- 172.21.15.69:0/1976042624 >> 172.21.15.69:0/1976042624 conn(0x55bcc4212d20 msgr2=0x55bcc4213100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 -- 172.21.15.69:0/1976042624 shutdown_connections 2024-08-02T17:57:53.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 -- 172.21.15.69:0/1976042624 wait complete. 2024-08-02T17:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.456+0000 7f5f9b793740 1 -- start start 2024-08-02T17:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f9b793740 1 Processor -- start 2024-08-02T17:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42167b0 0x55bcc421e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc421ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc4221970 0x55bcc421f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f9b793740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55bcc4224540 con 0x55bcc42167b0 2024-08-02T17:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f9b793740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55bcc4136500 con 0x55bcc4221970 2024-08-02T17:57:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f9b793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55bcc42246c0 con 0x55bcc42170d0 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f91d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc421ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f91d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc421ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40354/0 (socket says 172.21.15.69:40354) 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f91d9b640 1 -- 172.21.15.69:0/353025 learned_addr learned my addr 172.21.15.69:0/353025 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f9259c640 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42167b0 0x55bcc421e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.457+0000 7f5f92d9d640 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc4221970 0x55bcc421f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f91d9b640 1 -- 172.21.15.69:0/353025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc4221970 msgr2=0x55bcc421f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc4221970 0x55bcc421f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f91d9b640 1 -- 172.21.15.69:0/353025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42167b0 msgr2=0x55bcc421e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42167b0 0x55bcc421e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f9b793740 10 client.0 _ll_register_callbacks cb 0x55bcc3e8da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f91d9b640 1 -- 172.21.15.69:0/353025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc3f5a620 con 0x55bcc42170d0 2024-08-02T17:57:53.464 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f9b793740 1 -- 172.21.15.69:0/353025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8c0a4160 con 0x55bcc42170d0 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc421ef20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00ea80 tx=0x7f5f8c0094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ff62d99767f7fc7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f79ffb640 10 client.27466 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f79ffb640 1 -- 172.21.15.69:0/353025 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0b9060 con 0x55bcc42170d0 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f79ffb640 1 -- 172.21.15.69:0/353025 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f5f8c0bb500 con 0x55bcc42170d0 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f79ffb640 1 -- 172.21.15.69:0/353025 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f8c0b3070 con 0x55bcc42170d0 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.458+0000 7f5f9b793740 10 client.27466 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f9b793740 1 -- 172.21.15.69:0/353025 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55bcc422f3f0 con 0x55bcc42170d0 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f79ffb640 1 -- 172.21.15.69:0/353025 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5f8c00f040 con 0x55bcc42170d0 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f9b793740 10 client.27466 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f9b793740 10 client.27466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f9b793740 20 client.27466 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f9b793740 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bcc4230ad0 0x55bcc4232f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f9b793740 4 client.27466 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f9b793740 1 -- 172.21.15.69:0/353025 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcc41880e0 con 0x55bcc4230ad0 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.459+0000 7f5f9259c640 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bcc4230ad0 0x55bcc4232f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.460+0000 7f5f9259c640 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bcc4230ad0 0x55bcc4232f20 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171093042607b01 server_cookie=5f66f403c2a1fd9d in_seq=0 out_seq=0 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.460+0000 7f5f79ffb640 10 client.27466 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.460+0000 7f5f79ffb640 1 -- 172.21.15.69:0/353025 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bcc41880e0 con 0x55bcc4230ad0 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.460+0000 7f5f79ffb640 10 client.27466 handle_command_reply: tid=0 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.460+0000 7f5f9b793740 1 client.27466 shutdown 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.460+0000 7f5f9b793740 1 -- 172.21.15.69:0/353025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 msgr2=0x55bcc421ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.460+0000 7f5f9b793740 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc421ef20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00ea80 tx=0x7f5f8c0094a0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 1 -- 172.21.15.69:0/353025 shutdown_connections 2024-08-02T17:57:53.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bcc4230ad0 0x55bcc4232f20 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc4221970 0x55bcc421f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42170d0 0x55bcc421ef20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 1 --2- 172.21.15.69:0/353025 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42167b0 0x55bcc421e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 1 -- 172.21.15.69:0/353025 >> 172.21.15.69:0/353025 conn(0x55bcc4212d20 msgr2=0x55bcc4218a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f79ffb640 0 client.27466 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 1 -- 172.21.15.69:0/353025 shutdown_connections 2024-08-02T17:57:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 1 -- 172.21.15.69:0/353025 wait complete. 2024-08-02T17:57:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.461+0000 7f5f9b793740 20 client.27466 trim_cache size 0 max 16384 2024-08-02T17:57:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.497+0000 7f5f9b793740 1 Processor -- start 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.502+0000 7f5f9b793740 1 -- start start 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.502+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42167b0 0x55bcc420d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.502+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42170d0 0x55bcc4218210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.502+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc4221970 0x55bcc4218750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.502+0000 7f5f9b793740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f940702d0 con 0x55bcc4221970 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.502+0000 7f5f9b793740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9406dbe0 con 0x55bcc42170d0 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.502+0000 7f5f9b793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94101e40 con 0x55bcc42167b0 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.502+0000 7f5f91d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42170d0 0x55bcc4218210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.503+0000 7f5f91d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42170d0 0x55bcc4218210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:57236/0 (socket says 172.21.15.69:57236) 2024-08-02T17:57:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.503+0000 7f5f91d9b640 1 -- 172.21.15.69:0/75668675 learned_addr learned my addr 172.21.15.69:0/75668675 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.503+0000 7f5f91d9b640 1 -- 172.21.15.69:0/75668675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42167b0 msgr2=0x55bcc420d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.503+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/75668675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42167b0 0x55bcc420d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.503+0000 7f5f91d9b640 1 -- 172.21.15.69:0/75668675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc4221970 msgr2=0x55bcc4218750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.503+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/75668675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc4221970 0x55bcc4218750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.503+0000 7f5f91d9b640 1 -- 172.21.15.69:0/75668675 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc42165e0 con 0x55bcc42170d0 2024-08-02T17:57:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.504+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/75668675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42170d0 0x55bcc4218210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c09b480 tx=0x7f5f8c0b0000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0c9788612ec1ada server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.506+0000 7f5f7bfff640 1 -- 172.21.15.69:0/75668675 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0aa060 con 0x55bcc42170d0 2024-08-02T17:57:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.506+0000 7f5f7bfff640 1 -- 172.21.15.69:0/75668675 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f8c0a4160 con 0x55bcc42170d0 2024-08-02T17:57:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.506+0000 7f5f7bfff640 1 -- 172.21.15.69:0/75668675 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c09a940 con 0x55bcc42170d0 2024-08-02T17:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.508+0000 7f5f9b793740 1 -- 172.21.15.69:0/75668675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42170d0 msgr2=0x55bcc4218210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.508+0000 7f5f9b793740 1 --2- 172.21.15.69:0/75668675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42170d0 0x55bcc4218210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c09b480 tx=0x7f5f8c0b0000 comp rx=0 tx=0).stop 2024-08-02T17:57:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.517+0000 7f5f9b793740 1 -- 172.21.15.69:0/75668675 shutdown_connections 2024-08-02T17:57:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.517+0000 7f5f9b793740 1 --2- 172.21.15.69:0/75668675 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc4221970 0x55bcc4218750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.517+0000 7f5f9b793740 1 --2- 172.21.15.69:0/75668675 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42170d0 0x55bcc4218210 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.517+0000 7f5f9b793740 1 --2- 172.21.15.69:0/75668675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc42167b0 0x55bcc420d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.517+0000 7f5f9b793740 1 -- 172.21.15.69:0/75668675 >> 172.21.15.69:0/75668675 conn(0x55bcc4212d20 msgr2=0x55bcc4220950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:53 smithi083 ceph-mon[37031]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 466 op/s 2024-08-02T17:57:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.522+0000 7f5f9b793740 1 -- 172.21.15.69:0/75668675 shutdown_connections 2024-08-02T17:57:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.526+0000 7f5f9b793740 1 -- 172.21.15.69:0/75668675 wait complete. 2024-08-02T17:57:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.526+0000 7f5f9b793740 1 -- start start 2024-08-02T17:57:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.526+0000 7f5f9b793740 1 Processor -- start 2024-08-02T17:57:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.526+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4229950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42170d0 0x55bcc4229e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9b793740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc4221970 0x55bcc4226920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9b793740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f94108bb0 con 0x55bcc42170d0 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9b793740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9406d510 con 0x55bcc42167b0 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9b793740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9406c770 con 0x55bcc4221970 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9b793740 10 client.0 _ll_register_callbacks cb 0x7f5f8c09b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4229950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4229950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:57242/0 (socket says 172.21.15.69:57242) 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.527+0000 7f5f9259c640 1 -- 172.21.15.69:0/3764535306 learned_addr learned my addr 172.21.15.69:0/3764535306 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.529+0000 7f5f92d9d640 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc4221970 0x55bcc4226920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.530+0000 7f5f9259c640 1 -- 172.21.15.69:0/3764535306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc4221970 msgr2=0x55bcc4226920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.530+0000 7f5f9259c640 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc4221970 0x55bcc4226920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.530+0000 7f5f9259c640 1 -- 172.21.15.69:0/3764535306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42170d0 msgr2=0x55bcc4229e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.530+0000 7f5f9259c640 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42170d0 0x55bcc4229e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.530+0000 7f5f9259c640 1 -- 172.21.15.69:0/3764535306 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc4226e60 con 0x55bcc42167b0 2024-08-02T17:57:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.530+0000 7f5f9259c640 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4229950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009930 tx=0x7f5f84000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79648a2d89fe628 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.532+0000 7f5f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.535+0000 7f5f79ffb640 1 -- 172.21.15.69:0/3764535306 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f840054d0 con 0x55bcc42167b0 2024-08-02T17:57:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.535+0000 7f5f79ffb640 1 -- 172.21.15.69:0/3764535306 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f5f840543b0 con 0x55bcc42167b0 2024-08-02T17:57:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.536+0000 7f5f9b793740 1 -- 172.21.15.69:0/3764535306 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8c0a4160 con 0x55bcc42167b0 2024-08-02T17:57:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.539+0000 7f5f79ffb640 1 -- 172.21.15.69:0/3764535306 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f84062050 con 0x55bcc42167b0 2024-08-02T17:57:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.540+0000 7f5f9b793740 10 client.27659 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.540+0000 7f5f9b793740 1 -- 172.21.15.69:0/3764535306 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcc429bf40 con 0x55bcc42167b0 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.549+0000 7f5f79ffb640 1 -- 172.21.15.69:0/3764535306 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f8c0a4160 con 0x55bcc42167b0 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.549+0000 7f5f79ffb640 1 -- 172.21.15.69:0/3764535306 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5f8404b890 con 0x55bcc42167b0 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.550+0000 7f5f9b793740 10 client.27659 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.550+0000 7f5f9b793740 10 client.27659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.550+0000 7f5f9b793740 20 client.27659 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.550+0000 7f5f9b793740 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bcc429c690 0x55bcc429eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.550+0000 7f5f9b793740 4 client.27659 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.550+0000 7f5f9b793740 1 -- 172.21.15.69:0/3764535306 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bcc4182430 con 0x55bcc429c690 2024-08-02T17:57:53.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.550+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bcc429c690 0x55bcc429eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.551+0000 7f5f91d9b640 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bcc429c690 0x55bcc429eab0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb8d57bc0ac35dea server_cookie=298fe1f8a43d7ec in_seq=0 out_seq=0 2024-08-02T17:57:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.552+0000 7f5f79ffb640 10 client.27659 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.573+0000 7f5f79ffb640 1 -- 172.21.15.69:0/3764535306 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bcc4182430 con 0x55bcc429c690 2024-08-02T17:57:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.574+0000 7f5f79ffb640 10 client.27659 handle_command_reply: tid=0 2024-08-02T17:57:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.574+0000 7f5f9b793740 1 client.27659 shutdown 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 -- 172.21.15.69:0/3764535306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 msgr2=0x55bcc4229950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4229950 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5f84009930 tx=0x7f5f84000bc0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 -- 172.21.15.69:0/3764535306 shutdown_connections 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55bcc42170d0 0x55bcc4229e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55bcc429c690 0x55bcc429eab0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55bcc4221970 0x55bcc4226920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 --2- 172.21.15.69:0/3764535306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55bcc42167b0 0x55bcc4229950 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 -- 172.21.15.69:0/3764535306 >> 172.21.15.69:0/3764535306 conn(0x55bcc4212d20 msgr2=0x55bcc4218210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f79ffb640 0 client.27659 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:53.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 -- 172.21.15.69:0/3764535306 shutdown_connections 2024-08-02T17:57:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.575+0000 7f5f9b793740 1 -- 172.21.15.69:0/3764535306 wait complete. 2024-08-02T17:57:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.576+0000 7f5f9b793740 20 client.27659 trim_cache size 0 max 16384 2024-08-02T17:57:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:57:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:57:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:57:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:57:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.578+0000 7f5f98a41640 1 -- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2de0 msgr2=0x7f5f7c0a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.578+0000 7f5f98a41640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2de0 0x7f5f7c0a52a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5f840096c0 tx=0x7f5f84005fd0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.578+0000 7f5f98a41640 1 -- 172.21.15.69:0/3888777152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941010a0 msgr2=0x7f5f94107b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.578+0000 7f5f98a41640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941010a0 0x7f5f94107b70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c00eab0 tx=0x7f5f8c00ef70 comp rx=0 tx=0).stop 2024-08-02T17:57:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.581+0000 7f5f98a41640 1 -- 172.21.15.69:0/3888777152 shutdown_connections 2024-08-02T17:57:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.581+0000 7f5f98a41640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f7c0a2de0 0x7f5f7c0a52a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.581+0000 7f5f98a41640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f94106710 0x7f5f941080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.581+0000 7f5f98a41640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f941010a0 0x7f5f94107b70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.581+0000 7f5f98a41640 1 --2- 172.21.15.69:0/3888777152 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f94100700 0x7f5f94107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.581+0000 7f5f98a41640 1 -- 172.21.15.69:0/3888777152 >> 172.21.15.69:0/3888777152 conn(0x7f5f940fc3e0 msgr2=0x7f5f940fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.581+0000 7f5f98a41640 1 -- 172.21.15.69:0/3888777152 shutdown_connections 2024-08-02T17:57:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:57:53.581+0000 7f5f98a41640 1 -- 172.21.15.69:0/3888777152 wait complete. 2024-08-02T17:57:53.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:53 smithi077 ceph-mon[37108]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.9 MiB/s wr, 466 op/s 2024-08-02T17:57:54.196 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:57:54.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:54 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:57:54.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:54 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:57:54.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:54 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:57:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 645.0 s, 654.6 tps, lat 149.422 ms stddev 174.608 2024-08-02T17:57:55.313 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:55 smithi069 ceph-mon[32746]: pgmap v357: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 703 op/s 2024-08-02T17:57:55.313 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:55 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:57:55.313 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:55 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:57:55.313 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:55 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.347+0000 7f5f47df6640 1 -- 172.21.15.69:0/973474831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 msgr2=0x7f5f4010d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.347+0000 7f5f47df6640 1 --2- 172.21.15.69:0/973474831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 0x7f5f4010d670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009c10 tx=0x7f5f3802a6f0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.348+0000 7f5f47df6640 1 -- 172.21.15.69:0/973474831 shutdown_connections 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.348+0000 7f5f47df6640 1 --2- 172.21.15.69:0/973474831 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f401007a0 0x7f5f40100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.348+0000 7f5f47df6640 1 --2- 172.21.15.69:0/973474831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 0x7f5f4010d670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.348+0000 7f5f47df6640 1 --2- 172.21.15.69:0/973474831 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f401067a0 0x7f5f40106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.348+0000 7f5f47df6640 1 -- 172.21.15.69:0/973474831 >> 172.21.15.69:0/973474831 conn(0x7f5f400fc500 msgr2=0x7f5f400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.348+0000 7f5f47df6640 1 -- 172.21.15.69:0/973474831 shutdown_connections 2024-08-02T17:57:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.348+0000 7f5f47df6640 1 -- 172.21.15.69:0/973474831 wait complete. 2024-08-02T17:57:55.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.350+0000 7f5f47df6640 1 Processor -- start 2024-08-02T17:57:55.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.350+0000 7f5f47df6640 1 -- start start 2024-08-02T17:57:55.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f47df6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f401007a0 0x7f5f401970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f47df6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 0x7f5f40197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f47df6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f401067a0 0x7f5f4019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f45391640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 0x7f5f40197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f45391640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 0x7f5f40197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55808/0 (socket says 172.21.15.69:55808) 2024-08-02T17:57:55.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f45b92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f401007a0 0x7f5f401970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f45b92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f401007a0 0x7f5f401970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:58166/0 (socket says 172.21.15.69:58166) 2024-08-02T17:57:55.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f45391640 1 -- 172.21.15.69:0/1177860232 learned_addr learned my addr 172.21.15.69:0/1177860232 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:55.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f47df6640 1 -- 172.21.15.69:0/1177860232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f40110ba0 con 0x7f5f401067a0 2024-08-02T17:57:55.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f46393640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f401067a0 0x7f5f4019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f47df6640 1 -- 172.21.15.69:0/1177860232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f40110a20 con 0x7f5f401007a0 2024-08-02T17:57:55.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.351+0000 7f5f47df6640 1 -- 172.21.15.69:0/1177860232 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f40110d20 con 0x7f5f40101140 2024-08-02T17:57:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.353+0000 7f5f45b92640 1 -- 172.21.15.69:0/1177860232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 msgr2=0x7f5f40197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.353+0000 7f5f45b92640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 0x7f5f40197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.353+0000 7f5f45b92640 1 -- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f401067a0 msgr2=0x7f5f4019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.353+0000 7f5f45b92640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f401067a0 0x7f5f4019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.353+0000 7f5f45b92640 1 -- 172.21.15.69:0/1177860232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4019c090 con 0x7f5f401007a0 2024-08-02T17:57:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.353+0000 7f5f45391640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 0x7f5f40197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.357+0000 7f5f45b92640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f401007a0 0x7f5f401970e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e790 tx=0x7f5f3c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.361+0000 7f5f36ffd640 1 -- 172.21.15.69:0/1177860232 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00eea0 con 0x7f5f401007a0 2024-08-02T17:57:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.361+0000 7f5f36ffd640 1 -- 172.21.15.69:0/1177860232 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c00cc40 con 0x7f5f401007a0 2024-08-02T17:57:55.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.362+0000 7f5f36ffd640 1 -- 172.21.15.69:0/1177860232 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c018730 con 0x7f5f401007a0 2024-08-02T17:57:55.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.366+0000 7f5f47df6640 1 -- 172.21.15.69:0/1177860232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4019c380 con 0x7f5f401007a0 2024-08-02T17:57:55.367 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.366+0000 7f5f47df6640 1 -- 172.21.15.69:0/1177860232 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f401a3c60 con 0x7f5f401007a0 2024-08-02T17:57:55.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.369+0000 7f5f4a1f4740 1 Processor -- start 2024-08-02T17:57:55.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.373+0000 7f5f4a1f4740 1 -- start start 2024-08-02T17:57:55.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.373+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca96a80 0x55fe0ca9e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.373+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca95170 0x55fe0ca955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.373+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caaafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.373+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c9fa980 con 0x55fe0ca95170 2024-08-02T17:57:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.373+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c992a70 con 0x55fe0ca96a80 2024-08-02T17:57:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.373+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c9f6940 con 0x55fe0ca95af0 2024-08-02T17:57:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.374+0000 7f5f45391640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caaafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.374+0000 7f5f45391640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caaafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55818/0 (socket says 172.21.15.69:55818) 2024-08-02T17:57:55.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.374+0000 7f5f45391640 1 -- 172.21.15.69:0/3470116630 learned_addr learned my addr 172.21.15.69:0/3470116630 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:55.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.375+0000 7f5f45391640 1 -- 172.21.15.69:0/3470116630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca96a80 msgr2=0x55fe0ca9e1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:57:55.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.375+0000 7f5f45391640 1 --2- 172.21.15.69:0/3470116630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca96a80 0x55fe0ca9e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.375+0000 7f5f45391640 1 -- 172.21.15.69:0/3470116630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca95170 msgr2=0x55fe0ca955b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:57:55.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.375+0000 7f5f45391640 1 --2- 172.21.15.69:0/3470116630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca95170 0x55fe0ca955b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.375+0000 7f5f45391640 1 -- 172.21.15.69:0/3470116630 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0c9f56e0 con 0x55fe0ca95af0 2024-08-02T17:57:55.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.377+0000 7f5f45391640 1 --2- 172.21.15.69:0/3470116630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caaafa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f300115f0 tx=0x7f5f30011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5796c6eaf961215f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.379+0000 7f5f47df6640 1 -- 172.21.15.69:0/3470116630 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3001e070 con 0x55fe0ca95af0 2024-08-02T17:57:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.380+0000 7f5f47df6640 1 -- 172.21.15.69:0/3470116630 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3000fba0 con 0x55fe0ca95af0 2024-08-02T17:57:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.381+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/3470116630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 msgr2=0x55fe0caaafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.381+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/3470116630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caaafa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f300115f0 tx=0x7f5f30011ab0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.384+0000 7f5f36ffd640 1 -- 172.21.15.69:0/1177860232 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5f3c01f020 con 0x7f5f401007a0 2024-08-02T17:57:55.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.385+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/3470116630 shutdown_connections 2024-08-02T17:57:55.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.385+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/3470116630 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca95170 0x55fe0ca955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.385+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/3470116630 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caaafa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.385+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/3470116630 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca96a80 0x55fe0ca9e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.385+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/3470116630 >> 172.21.15.69:0/3470116630 conn(0x55fe0ca91aa0 msgr2=0x55fe0ca91e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:55.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.385+0000 7f5f36ffd640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f2c0a2cb0 0x7f5f2c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.385+0000 7f5f36ffd640 1 -- 172.21.15.69:0/1177860232 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f5f3c0cf5b0 con 0x7f5f401007a0 2024-08-02T17:57:55.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.386+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/3470116630 shutdown_connections 2024-08-02T17:57:55.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.387+0000 7f5f45391640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f2c0a2cb0 0x7f5f2c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.387+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/3470116630 wait complete. 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.387+0000 7f5f4a1f4740 1 -- start start 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.387+0000 7f5f4a1f4740 1 Processor -- start 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca95170 0x55fe0caa5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caa5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0caaad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fe0ca9eb10 con 0x55fe0ca96a80 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe0c9b5aa0 con 0x55fe0ca95170 2024-08-02T17:57:55.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe0ca9ec90 con 0x55fe0ca95af0 2024-08-02T17:57:55.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f4a1f4740 10 client.0 _ll_register_callbacks cb 0x55fe0c9a4e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:55.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f46393640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0caaad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f45b92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca95170 0x55fe0caa5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f45b92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca95170 0x55fe0caa5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:58176/0 (socket says 172.21.15.69:58176) 2024-08-02T17:57:55.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f45b92640 1 -- 172.21.15.69:0/2669475971 learned_addr learned my addr 172.21.15.69:0/2669475971 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:55.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f45b92640 1 -- 172.21.15.69:0/2669475971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 msgr2=0x55fe0caa5580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:57:55.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f45b92640 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caa5580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f45b92640 1 -- 172.21.15.69:0/2669475971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 msgr2=0x55fe0caaad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:55.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f45b92640 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0caaad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.388+0000 7f5f45b92640 1 -- 172.21.15.69:0/2669475971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0c999650 con 0x55fe0ca95170 2024-08-02T17:57:55.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.389+0000 7f5f45391640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f2c0a2cb0 0x7f5f2c0a5170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5f30019850 tx=0x7f5f30005040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.393+0000 7f5f45b92640 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca95170 0x55fe0caa5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009230 tx=0x7f5f3c09b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d41d532e53f7cc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.395+0000 7f5f297fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:57:55.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.399+0000 7f5f297fa640 1 -- 172.21.15.69:0/2669475971 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0ccd30 con 0x55fe0ca95170 2024-08-02T17:57:55.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.399+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/2669475971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3000fba0 con 0x55fe0ca95170 2024-08-02T17:57:55.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.399+0000 7f5f297fa640 1 -- 172.21.15.69:0/2669475971 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f5f3c0cf160 con 0x55fe0ca95170 2024-08-02T17:57:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.402+0000 7f5f297fa640 1 -- 172.21.15.69:0/2669475971 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f3c0a42f0 con 0x55fe0ca95170 2024-08-02T17:57:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.402+0000 7f5f4a1f4740 10 client.27668 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:55.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.402+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/2669475971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe0c76c3c0 con 0x55fe0ca95170 2024-08-02T17:57:55.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.407+0000 7f5f297fa640 1 -- 172.21.15.69:0/2669475971 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3000fba0 con 0x55fe0ca95170 2024-08-02T17:57:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.408+0000 7f5f297fa640 1 -- 172.21.15.69:0/2669475971 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5f3c09aad0 con 0x55fe0ca95170 2024-08-02T17:57:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.408+0000 7f5f4a1f4740 10 client.27668 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.408+0000 7f5f4a1f4740 10 client.27668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.408+0000 7f5f4a1f4740 20 client.27668 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.408+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe0cabf370 0x55fe0cac1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.408+0000 7f5f4a1f4740 4 client.27668 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:57:55.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.408+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/2669475971 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe0ca07ef0 con 0x55fe0cabf370 2024-08-02T17:57:55.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.409+0000 7f5f45391640 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe0cabf370 0x55fe0cac1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.416+0000 7f5f45391640 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe0cabf370 0x55fe0cac1760 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220f9fedd75b6963 server_cookie=294361f3e89ada28 in_seq=0 out_seq=0 2024-08-02T17:57:55.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.416+0000 7f5f297fa640 10 client.27668 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:55.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.418+0000 7f5f297fa640 1 -- 172.21.15.69:0/2669475971 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe0ca07ef0 con 0x55fe0cabf370 2024-08-02T17:57:55.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.418+0000 7f5f297fa640 10 client.27668 handle_command_reply: tid=0 2024-08-02T17:57:55.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.418+0000 7f5f4a1f4740 1 client.27668 shutdown 2024-08-02T17:57:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/2669475971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca95170 msgr2=0x55fe0caa5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca95170 0x55fe0caa5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009230 tx=0x7f5f3c09b680 comp rx=0 tx=0).stop 2024-08-02T17:57:55.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/2669475971 shutdown_connections 2024-08-02T17:57:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0caaad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe0cabf370 0x55fe0cac1760 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caa5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/2669475971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca95170 0x55fe0caa5040 secure :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009230 tx=0x7f5f3c09b680 comp rx=0 tx=0).stop 2024-08-02T17:57:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f297fa640 0 client.27668 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.423+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/2669475971 >> 172.21.15.69:0/2669475971 conn(0x55fe0ca91aa0 msgr2=0x55fe0ca90da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:55.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.424+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/2669475971 shutdown_connections 2024-08-02T17:57:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.424+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/2669475971 wait complete. 2024-08-02T17:57:55.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.424+0000 7f5f4a1f4740 20 client.27668 trim_cache size 0 max 16384 2024-08-02T17:57:55.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.457+0000 7f5f4a1f4740 1 Processor -- start 2024-08-02T17:57:55.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.457+0000 7f5f4a1f4740 1 -- start start 2024-08-02T17:57:55.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0ca90c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f401a6ac0 con 0x55fe0ca96a80 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f401a4390 con 0x55fe0ca911d0 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f45b92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f45b92640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55840/0 (socket says 172.21.15.69:55840) 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f45b92640 1 -- 172.21.15.69:0/1916711223 learned_addr learned my addr 172.21.15.69:0/1916711223 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.458+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/1916711223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f400725b0 con 0x55fe0ca95af0 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.459+0000 7f5f46393640 1 --2- 172.21.15.69:0/1916711223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.459+0000 7f5f45b92640 1 -- 172.21.15.69:0/1916711223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 msgr2=0x55fe0cac7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.459+0000 7f5f45b92640 1 --2- 172.21.15.69:0/1916711223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.459+0000 7f5f45b92640 1 -- 172.21.15.69:0/1916711223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 msgr2=0x55fe0ca90c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.459+0000 7f5f45b92640 1 --2- 172.21.15.69:0/1916711223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0ca90c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.459+0000 7f5f45b92640 1 -- 172.21.15.69:0/1916711223 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0ca9ecc0 con 0x55fe0ca95af0 2024-08-02T17:57:55.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.459+0000 7f5f45b92640 1 --2- 172.21.15.69:0/1916711223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caceee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c010de0 tx=0x7f5f3c09b2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f2f8557b535281 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:55.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.461+0000 7f5f2b7fe640 1 -- 172.21.15.69:0/1916711223 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0ccb30 con 0x55fe0ca95af0 2024-08-02T17:57:55.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.461+0000 7f5f2b7fe640 1 -- 172.21.15.69:0/1916711223 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c0cccd0 con 0x55fe0ca95af0 2024-08-02T17:57:55.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.461+0000 7f5f2b7fe640 1 -- 172.21.15.69:0/1916711223 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0a40e0 con 0x55fe0ca95af0 2024-08-02T17:57:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.461+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/1916711223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 msgr2=0x55fe0caceee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.461+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/1916711223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caceee0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c010de0 tx=0x7f5f3c09b2b0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/1916711223 shutdown_connections 2024-08-02T17:57:55.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/1916711223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0ca90c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/1916711223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/1916711223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0caceee0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/1916711223 >> 172.21.15.69:0/1916711223 conn(0x55fe0ca91aa0 msgr2=0x55fe0ca90580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/1916711223 shutdown_connections 2024-08-02T17:57:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/1916711223 wait complete. 2024-08-02T17:57:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 -- start start 2024-08-02T17:57:55.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 Processor -- start 2024-08-02T17:57:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.462+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0cac3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f4a1f4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0cb4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5f40072b70 con 0x55fe0ca96a80 2024-08-02T17:57:55.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f4a1f4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5f401a3e30 con 0x55fe0ca911d0 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45391640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0cac3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45b92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45391640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0cac3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55846/0 (socket says 172.21.15.69:55846) 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45b92640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:58188/0 (socket says 172.21.15.69:58188) 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45391640 1 -- 172.21.15.69:0/436157497 learned_addr learned my addr 172.21.15.69:0/436157497 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5f40073240 con 0x55fe0ca95af0 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f4a1f4740 10 client.0 _ll_register_callbacks cb 0x7f5f3c097260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f46393640 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0cb4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45391640 1 -- 172.21.15.69:0/436157497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 msgr2=0x55fe0cac35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45391640 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45391640 1 -- 172.21.15.69:0/436157497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 msgr2=0x55fe0cb4c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45391640 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0cb4c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f45391640 1 -- 172.21.15.69:0/436157497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe0cb4ccb0 con 0x55fe0ca95af0 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.463+0000 7f5f46393640 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0cb4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.464+0000 7f5f45391640 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0cac3af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f30019920 tx=0x7f5f300198b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f530093274b1ded server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.464+0000 7f5f297fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.464+0000 7f5f297fa640 1 -- 172.21.15.69:0/436157497 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3001e040 con 0x55fe0ca95af0 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.464+0000 7f5f297fa640 1 -- 172.21.15.69:0/436157497 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f5f30005540 con 0x55fe0ca95af0 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.464+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3c0cccd0 con 0x55fe0ca95af0 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f297fa640 1 -- 172.21.15.69:0/436157497 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5f3000fa90 con 0x55fe0ca95af0 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f4a1f4740 10 client.27484 fetch_fsmap learned FSMap version 26 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe0cb4dfa0 con 0x55fe0ca95af0 2024-08-02T17:57:55.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f297fa640 1 -- 172.21.15.69:0/436157497 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3000fc30 con 0x55fe0ca95af0 2024-08-02T17:57:55.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f297fa640 1 -- 172.21.15.69:0/436157497 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5f300244e0 con 0x55fe0ca95af0 2024-08-02T17:57:55.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f4a1f4740 10 client.27484 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:57:55.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f4a1f4740 10 client.27484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f4a1f4740 20 client.27484 populate_metadata read hostname 'smithi069' 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe0cb4e540 0x55fe0cb50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f4a1f4740 4 client.27484 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.465+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5f3c0cccd0 con 0x55fe0cb4e540 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.466+0000 7f5f45b92640 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe0cb4e540 0x55fe0cb50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f45b92640 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe0cb4e540 0x55fe0cb50960 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=203392a532f37706 server_cookie=453f6430f49956cb in_seq=0 out_seq=0 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f297fa640 10 client.27484 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f297fa640 1 -- 172.21.15.69:0/436157497 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f5f3c0cccd0 con 0x55fe0cb4e540 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f297fa640 10 client.27484 handle_command_reply: tid=0 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 client.27484 shutdown 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 msgr2=0x55fe0cac3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0cac3af0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f30019920 tx=0x7f5f300198b0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 shutdown_connections 2024-08-02T17:57:55.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe0ca96a80 0x55fe0cb4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe0cb4e540 0x55fe0cb50960 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe0ca95af0 0x55fe0cac3af0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f297fa640 0 client.27484 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 --2- 172.21.15.69:0/436157497 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe0ca911d0 0x55fe0cac35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 >> 172.21.15.69:0/436157497 conn(0x55fe0ca91aa0 msgr2=0x55fe0ca96eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 shutdown_connections 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.467+0000 7f5f4a1f4740 1 -- 172.21.15.69:0/436157497 wait complete. 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.468+0000 7f5f4a1f4740 20 client.27484 trim_cache size 0 max 16384 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:57:55.472 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:57:55.473 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:57:55.473 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:57:55.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.468+0000 7f5f34ff9640 1 -- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f2c0a2cb0 msgr2=0x7f5f2c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.468+0000 7f5f34ff9640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f2c0a2cb0 0x7f5f2c0a5170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5f30019850 tx=0x7f5f30005040 comp rx=0 tx=0).stop 2024-08-02T17:57:55.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.468+0000 7f5f34ff9640 1 -- 172.21.15.69:0/1177860232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f401007a0 msgr2=0x7f5f401970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.468+0000 7f5f34ff9640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f401007a0 0x7f5f401970e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00e790 tx=0x7f5f3c00ec50 comp rx=0 tx=0).stop 2024-08-02T17:57:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.469+0000 7f5f34ff9640 1 -- 172.21.15.69:0/1177860232 shutdown_connections 2024-08-02T17:57:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.469+0000 7f5f34ff9640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5f401067a0 0x7f5f4019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.469+0000 7f5f34ff9640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5f2c0a2cb0 0x7f5f2c0a5170 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.469+0000 7f5f34ff9640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5f40101140 0x7f5f40197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.469+0000 7f5f34ff9640 1 --2- 172.21.15.69:0/1177860232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5f401007a0 0x7f5f401970e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:55.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.469+0000 7f5f34ff9640 1 -- 172.21.15.69:0/1177860232 >> 172.21.15.69:0/1177860232 conn(0x7f5f400fc500 msgr2=0x7f5f400fce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.473+0000 7f5f34ff9640 1 -- 172.21.15.69:0/1177860232 shutdown_connections 2024-08-02T17:57:55.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:55.473+0000 7f5f34ff9640 1 -- 172.21.15.69:0/1177860232 wait complete. 2024-08-02T17:57:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:55 smithi083 ceph-mon[37031]: pgmap v357: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 703 op/s 2024-08-02T17:57:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:55 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:57:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:55 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:57:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:55 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:57:55.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:55 smithi077 ceph-mon[37108]: pgmap v357: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 11 MiB/s wr, 703 op/s 2024-08-02T17:57:55.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:55 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:57:55.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:55 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:57:55.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:55 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:57:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:57 smithi083 ceph-mon[37031]: pgmap v358: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 456 op/s 2024-08-02T17:57:57.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:57 smithi077 ceph-mon[37108]: pgmap v358: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 456 op/s 2024-08-02T17:57:57.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:57 smithi069 ceph-mon[32746]: pgmap v358: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.2 MiB/s wr, 456 op/s 2024-08-02T17:57:57.697 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:57:57.698 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:57:58.209 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:57:58.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.839+0000 7f93e759e640 1 -- 172.21.15.69:0/4272720229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100600 msgr2=0x7f93e8100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:58.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.839+0000 7f93e759e640 1 --2- 172.21.15.69:0/4272720229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100600 0x7f93e8100a80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009900 tx=0x7f93dc02a410 comp rx=0 tx=0).stop 2024-08-02T17:57:58.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.839+0000 7f93e759e640 1 -- 172.21.15.69:0/4272720229 shutdown_connections 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.839+0000 7f93e759e640 1 --2- 172.21.15.69:0/4272720229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93e8100fc0 0x7f93e810d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.839+0000 7f93e759e640 1 --2- 172.21.15.69:0/4272720229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100600 0x7f93e8100a80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.839+0000 7f93e759e640 1 --2- 172.21.15.69:0/4272720229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8106690 0x7f93e8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.839+0000 7f93e759e640 1 -- 172.21.15.69:0/4272720229 >> 172.21.15.69:0/4272720229 conn(0x7f93e80fc340 msgr2=0x7f93e80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.840+0000 7f93e759e640 1 -- 172.21.15.69:0/4272720229 shutdown_connections 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.840+0000 7f93e759e640 1 -- 172.21.15.69:0/4272720229 wait complete. 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e759e640 1 Processor -- start 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e759e640 1 -- start start 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e759e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100fc0 0x7f93e8115a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93e8106690 0x7f93e8115fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8110bf0 0x7f93e81110a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e759e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810f8d0 con 0x7f93e8106690 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e759e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810f750 con 0x7f93e8100fc0 2024-08-02T17:57:58.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e759e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93e810fa50 con 0x7f93e8110bf0 2024-08-02T17:57:58.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8110bf0 0x7f93e81110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:58.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e5d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93e8106690 0x7f93e8115fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:58.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8110bf0 0x7f93e81110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55860/0 (socket says 172.21.15.69:55860) 2024-08-02T17:57:58.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e6d9d640 1 -- 172.21.15.69:0/4232953573 learned_addr learned my addr 172.21.15.69:0/4232953573 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:57:58.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e5d9b640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93e8106690 0x7f93e8115fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56076/0 (socket says 172.21.15.69:56076) 2024-08-02T17:57:58.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.841+0000 7f93e659c640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100fc0 0x7f93e8115a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:58.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.842+0000 7f93e659c640 1 -- 172.21.15.69:0/4232953573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8110bf0 msgr2=0x7f93e81110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.842+0000 7f93e659c640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8110bf0 0x7f93e81110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.842+0000 7f93e659c640 1 -- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93e8106690 msgr2=0x7f93e8115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.842+0000 7f93e659c640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93e8106690 0x7f93e8115fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.842+0000 7f93e659c640 1 -- 172.21.15.69:0/4232953573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e8111960 con 0x7f93e8100fc0 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.842+0000 7f93e659c640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100fc0 0x7f93e8115a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f93d800c970 tx=0x7f93d800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.842+0000 7f93d77fe640 1 -- 172.21.15.69:0/4232953573 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d8007bb0 con 0x7f93e8100fc0 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.843+0000 7f93e759e640 1 -- 172.21.15.69:0/4232953573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81197e0 con 0x7f93e8100fc0 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.843+0000 7f93e759e640 1 -- 172.21.15.69:0/4232953573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e8119cf0 con 0x7f93e8100fc0 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.843+0000 7f93d77fe640 1 -- 172.21.15.69:0/4232953573 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93d8007d50 con 0x7f93e8100fc0 2024-08-02T17:57:58.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.843+0000 7f93d77fe640 1 -- 172.21.15.69:0/4232953573 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d800f450 con 0x7f93e8100fc0 2024-08-02T17:57:58.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.844+0000 7f93e759e640 1 -- 172.21.15.69:0/4232953573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c8005290 con 0x7f93e8100fc0 2024-08-02T17:57:58.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.846+0000 7f93d77fe640 1 -- 172.21.15.69:0/4232953573 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f93d800f5f0 con 0x7f93e8100fc0 2024-08-02T17:57:58.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.847+0000 7f93d77fe640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93d00a2e80 0x7f93d00a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:57:58.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.847+0000 7f93d77fe640 1 -- 172.21.15.69:0/4232953573 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9568+0+0 (secure 0 0 0) 0x7f93d80cf210 con 0x7f93e8100fc0 2024-08-02T17:57:58.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.847+0000 7f93e5d9b640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93d00a2e80 0x7f93d00a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:57:58.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.847+0000 7f93e5d9b640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93d00a2e80 0x7f93d00a5340 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93dc006010 tx=0x7f93dc0094c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:57:58.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:58.851+0000 7f93d77fe640 1 -- 172.21.15.69:0/4232953573 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f93d8014030 con 0x7f93e8100fc0 2024-08-02T17:57:59.229 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.226+0000 7f93e759e640 1 -- 172.21.15.69:0/4232953573 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93c8005850 con 0x7f93e8100fc0 2024-08-02T17:57:59.233 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.232+0000 7f93d77fe640 1 -- 172.21.15.69:0/4232953573 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f93d809b390 con 0x7f93e8100fc0 2024-08-02T17:57:59.234 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:57:59.234 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:57:59.234 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:57:59.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.238+0000 7f93d57fa640 1 -- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93d00a2e80 msgr2=0x7f93d00a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:59.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.238+0000 7f93d57fa640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93d00a2e80 0x7f93d00a5340 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93dc006010 tx=0x7f93dc0094c0 comp rx=0 tx=0).stop 2024-08-02T17:57:59.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.238+0000 7f93d57fa640 1 -- 172.21.15.69:0/4232953573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100fc0 msgr2=0x7f93e8115a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:57:59.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.238+0000 7f93d57fa640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100fc0 0x7f93e8115a60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f93d800c970 tx=0x7f93d800ce30 comp rx=0 tx=0).stop 2024-08-02T17:57:59.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.239+0000 7f93d57fa640 1 -- 172.21.15.69:0/4232953573 shutdown_connections 2024-08-02T17:57:59.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.239+0000 7f93d57fa640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f93e8106690 0x7f93e8115fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:59.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.239+0000 7f93d57fa640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93d00a2e80 0x7f93d00a5340 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f93dc006010 tx=0x7f93dc0094c0 comp rx=0 tx=0).stop 2024-08-02T17:57:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.239+0000 7f93d57fa640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93e8110bf0 0x7f93e81110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:57:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.239+0000 7f93d57fa640 1 --2- 172.21.15.69:0/4232953573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f93e8100fc0 0x7f93e8115a60 secure :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f93d800c970 tx=0x7f93d800ce30 comp rx=0 tx=0).stop 2024-08-02T17:57:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.239+0000 7f93d57fa640 1 -- 172.21.15.69:0/4232953573 >> 172.21.15.69:0/4232953573 conn(0x7f93e80fc340 msgr2=0x7f93e80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:57:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.240+0000 7f93d57fa640 1 -- 172.21.15.69:0/4232953573 shutdown_connections 2024-08-02T17:57:59.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:57:59.240+0000 7f93d57fa640 1 -- 172.21.15.69:0/4232953573 wait complete. 2024-08-02T17:57:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:57:59 smithi083 ceph-mon[37031]: pgmap v359: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.9 MiB/s wr, 540 op/s 2024-08-02T17:57:59.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:57:59 smithi077 ceph-mon[37108]: pgmap v359: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.9 MiB/s wr, 540 op/s 2024-08-02T17:57:59.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:57:59 smithi069 ceph-mon[32746]: pgmap v359: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.9 MiB/s wr, 540 op/s 2024-08-02T17:57:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 650.0 s, 542.4 tps, lat 184.192 ms stddev 261.437 2024-08-02T17:58:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:00 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4232953573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:00.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:00 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4232953573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:00.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:00 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4232953573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:01.131 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:58:01.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:01 smithi069 ceph-mon[32746]: pgmap v360: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.3 MiB/s wr, 596 op/s 2024-08-02T17:58:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:01 smithi083 ceph-mon[37031]: pgmap v360: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.3 MiB/s wr, 596 op/s 2024-08-02T17:58:01.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:01 smithi077 ceph-mon[37108]: pgmap v360: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.3 MiB/s wr, 596 op/s 2024-08-02T17:58:01.617 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:02.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:02 smithi083 ceph-mon[37031]: pgmap v361: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 457 op/s 2024-08-02T17:58:02.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:02 smithi077 ceph-mon[37108]: pgmap v361: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 457 op/s 2024-08-02T17:58:02.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:02 smithi069 ceph-mon[32746]: pgmap v361: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.0 MiB/s wr, 457 op/s 2024-08-02T17:58:04.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:04 smithi069 ceph-mon[32746]: pgmap v362: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-08-02T17:58:04.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:04 smithi083 ceph-mon[37031]: pgmap v362: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-08-02T17:58:04.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:04 smithi077 ceph-mon[37108]: pgmap v362: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-08-02T17:58:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 655.0 s, 426.4 tps, lat 222.646 ms stddev 281.819 2024-08-02T17:58:06.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:06 smithi069 ceph-mon[32746]: pgmap v363: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-08-02T17:58:06.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:06 smithi069 ceph-mon[32746]: osdmap e106: 12 total, 12 up, 12 in 2024-08-02T17:58:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:06 smithi083 ceph-mon[37031]: pgmap v363: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-08-02T17:58:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:06 smithi083 ceph-mon[37031]: osdmap e106: 12 total, 12 up, 12 in 2024-08-02T17:58:06.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:06 smithi077 ceph-mon[37108]: pgmap v363: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.2 MiB/s wr, 605 op/s 2024-08-02T17:58:06.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:06 smithi077 ceph-mon[37108]: osdmap e106: 12 total, 12 up, 12 in 2024-08-02T17:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.693+0000 7f4597d5e640 1 -- 172.21.15.69:0/822961029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901006d0 msgr2=0x7f4590100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.693+0000 7f4597d5e640 1 --2- 172.21.15.69:0/822961029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901006d0 0x7f4590100b30 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4580009930 tx=0x7f458002a420 comp rx=0 tx=0).stop 2024-08-02T17:58:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.693+0000 7f4597d5e640 1 -- 172.21.15.69:0/822961029 shutdown_connections 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.693+0000 7f4597d5e640 1 --2- 172.21.15.69:0/822961029 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901006d0 0x7f4590100b30 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.693+0000 7f4597d5e640 1 --2- 172.21.15.69:0/822961029 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4590101070 0x7f459010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.693+0000 7f4597d5e640 1 --2- 172.21.15.69:0/822961029 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f45901066e0 0x7f4590106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.693+0000 7f4597d5e640 1 -- 172.21.15.69:0/822961029 >> 172.21.15.69:0/822961029 conn(0x7f45900776d0 msgr2=0x7f45900fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 -- 172.21.15.69:0/822961029 shutdown_connections 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 -- 172.21.15.69:0/822961029 wait complete. 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 Processor -- start 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 -- start start 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45901006d0 0x7f4590198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4590101070 0x7f4590198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901066e0 0x7f459019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4590111c10 con 0x7f45901066e0 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4590111a90 con 0x7f4590101070 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4597d5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4590111d90 con 0x7f45901006d0 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f45962fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901066e0 0x7f459019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.694+0000 7f4595afa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45901006d0 0x7f4590198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f4595afa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45901006d0 0x7f4590198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42902/0 (socket says 172.21.15.69:42902) 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f45962fb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901066e0 0x7f459019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55170/0 (socket says 172.21.15.69:55170) 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f4595afa640 1 -- 172.21.15.69:0/3978581988 learned_addr learned my addr 172.21.15.69:0/3978581988 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f45952f9640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4590101070 0x7f4590198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f45962fb640 1 -- 172.21.15.69:0/3978581988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45901006d0 msgr2=0x7f4590198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f45962fb640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45901006d0 0x7f4590198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f45962fb640 1 -- 172.21.15.69:0/3978581988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4590101070 msgr2=0x7f4590198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f45962fb640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4590101070 0x7f4590198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f45962fb640 1 -- 172.21.15.69:0/3978581988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f459019d0f0 con 0x7f45901066e0 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f45962fb640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901066e0 0x7f459019c9c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4588009fd0 tx=0x7f458800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f4586ffd640 1 -- 172.21.15.69:0/3978581988 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458800ce20 con 0x7f45901066e0 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f4586ffd640 1 -- 172.21.15.69:0/3978581988 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4588005020 con 0x7f45901066e0 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f4597d5e640 1 -- 172.21.15.69:0/3978581988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f459019d3e0 con 0x7f45901066e0 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f4586ffd640 1 -- 172.21.15.69:0/3978581988 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588010720 con 0x7f45901066e0 2024-08-02T17:58:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.695+0000 7f4597d5e640 1 -- 172.21.15.69:0/3978581988 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45901a4c70 con 0x7f45901066e0 2024-08-02T17:58:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.698+0000 7f4586ffd640 1 -- 172.21.15.69:0/3978581988 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4588005540 con 0x7f45901066e0 2024-08-02T17:58:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.698+0000 7f459ab92740 1 Processor -- start 2024-08-02T17:58:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.698+0000 7f459ab92740 1 -- start start 2024-08-02T17:58:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.698+0000 7f459ab92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 0x55d53ae14b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f459ab92740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f459ab92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae21d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f459ab92740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d53ad789b0 con 0x55d53ae150d0 2024-08-02T17:58:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f459ab92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d53ad20800 con 0x55d53ae147b0 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f459ab92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d53ad74970 con 0x55d53ae1f970 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4586ffd640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f457c0a2cf0 0x7f457c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4595afa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4595afa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55178/0 (socket says 172.21.15.69:55178) 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4595afa640 1 -- 172.21.15.69:0/1862984694 learned_addr learned my addr 172.21.15.69:0/1862984694 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4586ffd640 1 -- 172.21.15.69:0/3978581988 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9656+0+0 (secure 0 0 0) 0x7f45880cff80 con 0x7f45901066e0 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f45962fb640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f457c0a2cf0 0x7f457c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4595afa640 1 -- 172.21.15.69:0/1862984694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 msgr2=0x55d53ae21d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4595afa640 1 --2- 172.21.15.69:0/1862984694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae21d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4595afa640 1 -- 172.21.15.69:0/1862984694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 msgr2=0x55d53ae14b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4595afa640 1 --2- 172.21.15.69:0/1862984694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 0x55d53ae14b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.699+0000 7f4595afa640 1 -- 172.21.15.69:0/1862984694 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53aa8c210 con 0x55d53ae150d0 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.700+0000 7f4595afa640 1 --2- 172.21.15.69:0/1862984694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1f430 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f458c009cc0 tx=0x7f458c013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49745fee48bbd6bd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.700+0000 7f45962fb640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f457c0a2cf0 0x7f457c0a51b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f458809bd30 tx=0x7f45880bc040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.700+0000 7f457bfff640 1 -- 172.21.15.69:0/1862984694 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c004020 con 0x55d53ae150d0 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.700+0000 7f457bfff640 1 -- 172.21.15.69:0/1862984694 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458c0041c0 con 0x55d53ae150d0 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.700+0000 7f457bfff640 1 -- 172.21.15.69:0/1862984694 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c003c50 con 0x55d53ae150d0 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- 172.21.15.69:0/1862984694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 msgr2=0x55d53ae1f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 --2- 172.21.15.69:0/1862984694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1f430 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f458c009cc0 tx=0x7f458c013ba0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- 172.21.15.69:0/1862984694 shutdown_connections 2024-08-02T17:58:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 --2- 172.21.15.69:0/1862984694 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1f430 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 --2- 172.21.15.69:0/1862984694 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae21d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 --2- 172.21.15.69:0/1862984694 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 0x55d53ae14b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- 172.21.15.69:0/1862984694 >> 172.21.15.69:0/1862984694 conn(0x55d53ae10d20 msgr2=0x55d53ae11100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- 172.21.15.69:0/1862984694 shutdown_connections 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- 172.21.15.69:0/1862984694 wait complete. 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- start start 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 Processor -- start 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 0x55d53ae1c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae1d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d53ae22540 con 0x55d53ae150d0 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d53ad34500 con 0x55d53ae147b0 2024-08-02T17:58:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d53ae226c0 con 0x55d53ae1f970 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.701+0000 7f459ab92740 10 client.0 _ll_register_callbacks cb 0x55d53aa8ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f45952f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae1d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55192/0 (socket says 172.21.15.69:55192) 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 -- 172.21.15.69:0/1428179985 learned_addr learned my addr 172.21.15.69:0/1428179985 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f45952f9640 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 0x55d53ae1c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 -- 172.21.15.69:0/1428179985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 msgr2=0x55d53ae1d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae1d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 -- 172.21.15.69:0/1428179985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 msgr2=0x55d53ae1c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 0x55d53ae1c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 -- 172.21.15.69:0/1428179985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53ab58620 con 0x55d53ae150d0 2024-08-02T17:58:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f4595afa640 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1cce0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f458c00e780 tx=0x7f458c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52b77d63ce9112 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f45797fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f45797fa640 1 -- 172.21.15.69:0/1428179985 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c02a020 con 0x55d53ae150d0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f459ab92740 1 -- 172.21.15.69:0/1428179985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f458c0041c0 con 0x55d53ae150d0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f45797fa640 1 -- 172.21.15.69:0/1428179985 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9656+0+0 (secure 0 0 0) 0x7f458c003ba0 con 0x55d53ae150d0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f45797fa640 1 -- 172.21.15.69:0/1428179985 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f458c035050 con 0x55d53ae150d0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f459ab92740 10 client.18276 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.702+0000 7f459ab92740 1 -- 172.21.15.69:0/1428179985 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d53ae2e0f0 con 0x55d53ae150d0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f45797fa640 1 -- 172.21.15.69:0/1428179985 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458c0041c0 con 0x55d53ae150d0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f45797fa640 1 -- 172.21.15.69:0/1428179985 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f458c026c20 con 0x55d53ae150d0 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f459ab92740 10 client.18276 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f459ab92740 10 client.18276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f459ab92740 20 client.18276 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f459ab92740 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d53ae2f8e0 0x55d53ae31d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f459ab92740 4 client.18276 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f459ab92740 1 -- 172.21.15.69:0/1428179985 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d53ad860e0 con 0x55d53ae2f8e0 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.703+0000 7f45952f9640 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d53ae2f8e0 0x55d53ae31d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.704+0000 7f45797fa640 10 client.18276 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.704+0000 7f45952f9640 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d53ae2f8e0 0x55d53ae31d60 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb8e3587fc9555a server_cookie=3ed61c83ba7c8de9 in_seq=0 out_seq=0 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f45797fa640 1 -- 172.21.15.69:0/1428179985 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d53ad860e0 con 0x55d53ae2f8e0 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f45797fa640 10 client.18276 handle_command_reply: tid=0 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 client.18276 shutdown 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 -- 172.21.15.69:0/1428179985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 msgr2=0x55d53ae1cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1cce0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f458c00e780 tx=0x7f458c009780 comp rx=0 tx=0).stop 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 -- 172.21.15.69:0/1428179985 shutdown_connections 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae150d0 0x55d53ae1cce0 secure :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f458c00e780 tx=0x7f458c009780 comp rx=0 tx=0).stop 2024-08-02T17:58:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d53ae2f8e0 0x55d53ae31d60 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae1d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 --2- 172.21.15.69:0/1428179985 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae147b0 0x55d53ae1c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f459ab92740 1 -- 172.21.15.69:0/1428179985 >> 172.21.15.69:0/1428179985 conn(0x55d53ae10d20 msgr2=0x55d53ae16a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.705+0000 7f45797fa640 0 client.18276 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.706+0000 7f459ab92740 1 -- 172.21.15.69:0/1428179985 shutdown_connections 2024-08-02T17:58:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.706+0000 7f459ab92740 1 -- 172.21.15.69:0/1428179985 wait complete. 2024-08-02T17:58:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.707+0000 7f459ab92740 20 client.18276 trim_cache size 0 max 16384 2024-08-02T17:58:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.724+0000 7f459ab92740 1 Processor -- start 2024-08-02T17:58:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.724+0000 7f459ab92740 1 -- start start 2024-08-02T17:58:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f459ab92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae147b0 0x55d53ae17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f459ab92740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae1f970 0x55d53ae0fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f459ab92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f459ab92740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45901af640 con 0x55d53ae1f970 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f459ab92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45901acf10 con 0x55d53ae37670 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f459ab92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45901aad80 con 0x55d53ae147b0 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f4595afa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae1f970 0x55d53ae0fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f4595afa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae1f970 0x55d53ae0fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55204/0 (socket says 172.21.15.69:55204) 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f4595afa640 1 -- 172.21.15.69:0/2545726593 learned_addr learned my addr 172.21.15.69:0/2545726593 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f45952f9640 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae147b0 0x55d53ae17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f45952f9640 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f4595afa640 1 -- 172.21.15.69:0/2545726593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae147b0 msgr2=0x55d53ae17220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f4595afa640 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae147b0 0x55d53ae17220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f4595afa640 1 -- 172.21.15.69:0/2545726593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 msgr2=0x55d53ae37ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f4595afa640 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae37ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f4595afa640 1 -- 172.21.15.69:0/2545726593 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d53ae14530 con 0x55d53ae1f970 2024-08-02T17:58:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.725+0000 7f45952f9640 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae37ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.726+0000 7f4595afa640 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae1f970 0x55d53ae0fd70 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f458c009250 tx=0x7f458c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28876783011ca1bd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.726+0000 7f457b7fe640 1 -- 172.21.15.69:0/2545726593 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c02a040 con 0x55d53ae1f970 2024-08-02T17:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.726+0000 7f457b7fe640 1 -- 172.21.15.69:0/2545726593 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458c009940 con 0x55d53ae1f970 2024-08-02T17:58:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.726+0000 7f457b7fe640 1 -- 172.21.15.69:0/2545726593 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c01ce40 con 0x55d53ae1f970 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.727+0000 7f459ab92740 1 -- 172.21.15.69:0/2545726593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae1f970 msgr2=0x55d53ae0fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.727+0000 7f459ab92740 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae1f970 0x55d53ae0fd70 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f458c009250 tx=0x7f458c00eed0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.728+0000 7f459ab92740 1 -- 172.21.15.69:0/2545726593 shutdown_connections 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.728+0000 7f459ab92740 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae1f970 0x55d53ae0fd70 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.728+0000 7f459ab92740 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae37ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.728+0000 7f459ab92740 1 --2- 172.21.15.69:0/2545726593 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae147b0 0x55d53ae17220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.728+0000 7f459ab92740 1 -- 172.21.15.69:0/2545726593 >> 172.21.15.69:0/2545726593 conn(0x55d53ae10d20 msgr2=0x55d53ae0f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.729+0000 7f459ab92740 1 -- 172.21.15.69:0/2545726593 shutdown_connections 2024-08-02T17:58:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.729+0000 7f459ab92740 1 -- 172.21.15.69:0/2545726593 wait complete. 2024-08-02T17:58:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.729+0000 7f459ab92740 1 -- start start 2024-08-02T17:58:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.729+0000 7f459ab92740 1 Processor -- start 2024-08-02T17:58:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f459ab92740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae147b0 0x55d53ae27de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f459ab92740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae19230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f459ab92740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae19770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f459ab92740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f45901ab3d0 con 0x55d53ae147b0 2024-08-02T17:58:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f459ab92740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f45901ac840 con 0x55d53ae37670 2024-08-02T17:58:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f459ab92740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45901abaa0 con 0x55d53ae1f970 2024-08-02T17:58:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f459ab92740 10 client.0 _ll_register_callbacks cb 0x7f458c009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f45952f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae147b0 0x55d53ae27de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f4595afa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae19230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f45952f9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae147b0 0x55d53ae27de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55216/0 (socket says 172.21.15.69:55216) 2024-08-02T17:58:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f45952f9640 1 -- 172.21.15.69:0/504641408 learned_addr learned my addr 172.21.15.69:0/504641408 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f45952f9640 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae19770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f4595afa640 1 -- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 msgr2=0x55d53ae19770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f4595afa640 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae19770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f4595afa640 1 -- 172.21.15.69:0/504641408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae147b0 msgr2=0x55d53ae27de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f4595afa640 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae147b0 0x55d53ae27de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.730+0000 7f4595afa640 1 -- 172.21.15.69:0/504641408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53ae19ce0 con 0x55d53ae1f970 2024-08-02T17:58:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f45952f9640 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae19770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:06.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f4595afa640 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae19230 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f458c01c5d0 tx=0x7f458c01cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5de71160e2b10ff5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f45797fa640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f45797fa640 1 -- 172.21.15.69:0/504641408 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458c02a040 con 0x55d53ae1f970 2024-08-02T17:58:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f459ab92740 1 -- 172.21.15.69:0/504641408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f458c009940 con 0x55d53ae1f970 2024-08-02T17:58:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f45797fa640 1 -- 172.21.15.69:0/504641408 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9656+0+0 (secure 0 0 0) 0x7f458c013370 con 0x55d53ae1f970 2024-08-02T17:58:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f45797fa640 1 -- 172.21.15.69:0/504641408 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f458c01c6d0 con 0x55d53ae1f970 2024-08-02T17:58:06.739 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f459ab92740 10 client.27517 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f459ab92740 1 -- 172.21.15.69:0/504641408 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d53aea2b50 con 0x55d53ae1f970 2024-08-02T17:58:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.731+0000 7f45797fa640 1 -- 172.21.15.69:0/504641408 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458c009940 con 0x55d53ae1f970 2024-08-02T17:58:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f45797fa640 1 -- 172.21.15.69:0/504641408 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f458c005020 con 0x55d53ae1f970 2024-08-02T17:58:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f459ab92740 10 client.27517 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f459ab92740 10 client.27517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:06.740 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f459ab92740 20 client.27517 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f459ab92740 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d53aea3130 0x55d53aea5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f459ab92740 4 client.27517 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:58:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f459ab92740 1 -- 172.21.15.69:0/504641408 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d53ad80430 con 0x55d53aea3130 2024-08-02T17:58:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f45952f9640 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d53aea3130 0x55d53aea5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f45952f9640 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d53aea3130 0x55d53aea5550 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0342d658252c15c server_cookie=513549153dce7a9d in_seq=0 out_seq=0 2024-08-02T17:58:06.741 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.732+0000 7f45797fa640 10 client.27517 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f45797fa640 1 -- 172.21.15.69:0/504641408 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d53ad80430 con 0x55d53aea3130 2024-08-02T17:58:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f45797fa640 10 client.27517 handle_command_reply: tid=0 2024-08-02T17:58:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 client.27517 shutdown 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 -- 172.21.15.69:0/504641408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 msgr2=0x55d53ae19230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae19230 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f458c01c5d0 tx=0x7f458c01cd80 comp rx=0 tx=0).stop 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 -- 172.21.15.69:0/504641408 shutdown_connections 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d53aea3130 0x55d53aea5550 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d53ae37670 0x55d53ae19770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d53ae1f970 0x55d53ae19230 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 --2- 172.21.15.69:0/504641408 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d53ae147b0 0x55d53ae27de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f459ab92740 1 -- 172.21.15.69:0/504641408 >> 172.21.15.69:0/504641408 conn(0x55d53ae10d20 msgr2=0x55d53ae36660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.771+0000 7f45797fa640 0 client.27517 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.772+0000 7f459ab92740 1 -- 172.21.15.69:0/504641408 shutdown_connections 2024-08-02T17:58:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.772+0000 7f459ab92740 1 -- 172.21.15.69:0/504641408 wait complete. 2024-08-02T17:58:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.772+0000 7f459ab92740 20 client.27517 trim_cache size 0 max 16384 2024-08-02T17:58:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:58:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:58:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:58:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.772+0000 7f4597d5e640 1 -- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f457c0a2cf0 msgr2=0x7f457c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.772+0000 7f4597d5e640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f457c0a2cf0 0x7f457c0a51b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f458809bd30 tx=0x7f45880bc040 comp rx=0 tx=0).stop 2024-08-02T17:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.772+0000 7f4597d5e640 1 -- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901066e0 msgr2=0x7f459019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.772+0000 7f4597d5e640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901066e0 0x7f459019c9c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4588009fd0 tx=0x7f458800ef10 comp rx=0 tx=0).stop 2024-08-02T17:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.773+0000 7f4597d5e640 1 -- 172.21.15.69:0/3978581988 shutdown_connections 2024-08-02T17:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.773+0000 7f4597d5e640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f45901066e0 0x7f459019c9c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.773+0000 7f4597d5e640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f457c0a2cf0 0x7f457c0a51b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.773+0000 7f4597d5e640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4590101070 0x7f4590198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.773+0000 7f4597d5e640 1 --2- 172.21.15.69:0/3978581988 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45901006d0 0x7f4590198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.773+0000 7f4597d5e640 1 -- 172.21.15.69:0/3978581988 >> 172.21.15.69:0/3978581988 conn(0x7f45900776d0 msgr2=0x7f4590103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.773+0000 7f4597d5e640 1 -- 172.21.15.69:0/3978581988 shutdown_connections 2024-08-02T17:58:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:06.773+0000 7f4597d5e640 1 -- 172.21.15.69:0/3978581988 wait complete. 2024-08-02T17:58:07.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.158+0000 7f2d3b840640 1 -- 172.21.15.69:0/4019080808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d34102280 msgr2=0x7f2d3410e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.158+0000 7f2d3b840640 1 --2- 172.21.15.69:0/4019080808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d34102280 0x7f2d3410e7b0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009cd0 tx=0x7f2d2c02a8e0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 -- 172.21.15.69:0/4019080808 shutdown_connections 2024-08-02T17:58:07.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 --2- 172.21.15.69:0/4019080808 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d34102280 0x7f2d3410e7b0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 --2- 172.21.15.69:0/4019080808 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d341018e0 0x7f2d34101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 --2- 172.21.15.69:0/4019080808 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d341078e0 0x7f2d34107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 -- 172.21.15.69:0/4019080808 >> 172.21.15.69:0/4019080808 conn(0x7f2d340fd600 msgr2=0x7f2d340ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:07.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 -- 172.21.15.69:0/4019080808 shutdown_connections 2024-08-02T17:58:07.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 -- 172.21.15.69:0/4019080808 wait complete. 2024-08-02T17:58:07.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 Processor -- start 2024-08-02T17:58:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.159+0000 7f2d3b840640 1 -- start start 2024-08-02T17:58:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d3b840640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d341018e0 0x7f2d34198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d3b840640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d34102280 0x7f2d341986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d3b840640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d341078e0 0x7f2d3419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d3b840640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34111d30 con 0x7f2d341078e0 2024-08-02T17:58:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d3b840640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34111bb0 con 0x7f2d34102280 2024-08-02T17:58:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d3b840640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d34111eb0 con 0x7f2d341018e0 2024-08-02T17:58:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d39ddd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d341078e0 0x7f2d3419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d395dc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d341018e0 0x7f2d34198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d39ddd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d341078e0 0x7f2d3419ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55100/0 (socket says 172.21.15.69:55100) 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d39ddd640 1 -- 172.21.15.69:0/1695332277 learned_addr learned my addr 172.21.15.69:0/1695332277 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d34102280 0x7f2d341986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d39ddd640 1 -- 172.21.15.69:0/1695332277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d341018e0 msgr2=0x7f2d34198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d341018e0 0x7f2d34198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d39ddd640 1 -- 172.21.15.69:0/1695332277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d34102280 msgr2=0x7f2d341986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d34102280 0x7f2d341986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.160+0000 7f2d39ddd640 1 -- 172.21.15.69:0/1695332277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3419d140 con 0x7f2d341078e0 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.161+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d341078e0 0x7f2d3419ca60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c033040 tx=0x7f2d2c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.161+0000 7f2d2a7fc640 1 -- 172.21.15.69:0/1695332277 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c004040 con 0x7f2d341078e0 2024-08-02T17:58:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.161+0000 7f2d3b840640 1 -- 172.21.15.69:0/1695332277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3419d3d0 con 0x7f2d341078e0 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.161+0000 7f2d2a7fc640 1 -- 172.21.15.69:0/1695332277 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c0046c0 con 0x7f2d341078e0 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.161+0000 7f2d2a7fc640 1 -- 172.21.15.69:0/1695332277 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c046480 con 0x7f2d341078e0 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.161+0000 7f2d3b840640 1 -- 172.21.15.69:0/1695332277 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d341a4d10 con 0x7f2d341078e0 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.163+0000 7f2d2a7fc640 1 -- 172.21.15.69:0/1695332277 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2d2c0305a0 con 0x7f2d341078e0 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.163+0000 7f2d3dc3e740 1 Processor -- start 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.163+0000 7f2d3dc3e740 1 -- start start 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d2a7fc640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d200a2cf0 0x7f2d200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 0x55c029d602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d60810 0x55c029d6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c029cc5a40 con 0x55c029d60810 2024-08-02T17:58:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c029c5d9d0 con 0x55c029d6f5e0 2024-08-02T17:58:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c029cc1a00 con 0x55c029d5fef0 2024-08-02T17:58:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d200a2cf0 0x7f2d200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d2a7fc640 1 -- 172.21.15.69:0/1695332277 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9656+0+0 (secure 0 0 0) 0x7f2d2c02e030 con 0x7f2d341078e0 2024-08-02T17:58:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d38ddb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d60810 0x55c029d6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d39ddd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d719d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d38ddb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d60810 0x55c029d6f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55112/0 (socket says 172.21.15.69:55112) 2024-08-02T17:58:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d38ddb640 1 -- 172.21.15.69:0/3723737305 learned_addr learned my addr 172.21.15.69:0/3723737305 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d395dc640 1 --2- 172.21.15.69:0/3723737305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 0x55c029d602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d39ddd640 1 -- 172.21.15.69:0/3723737305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 msgr2=0x55c029d602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/3723737305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 0x55c029d602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d39ddd640 1 -- 172.21.15.69:0/3723737305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d60810 msgr2=0x55c029d6f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/3723737305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d60810 0x55c029d6f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d39ddd640 1 -- 172.21.15.69:0/3723737305 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c029c64650 con 0x55c029d6f5e0 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d200a2cf0 0x7f2d200a51b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2d24012960 tx=0x7f2d2400bb70 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/3723737305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d719d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0ee2a0 tx=0x7f2d2c003f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a62ae54efbc06d79 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.164+0000 7f2d1f7fe640 1 -- 172.21.15.69:0/3723737305 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c0bc9d0 con 0x55c029d6f5e0 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d1f7fe640 1 -- 172.21.15.69:0/3723737305 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c036070 con 0x55c029d6f5e0 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d1f7fe640 1 -- 172.21.15.69:0/3723737305 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c032570 con 0x55c029d6f5e0 2024-08-02T17:58:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3723737305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 msgr2=0x55c029d719d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3723737305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d719d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0ee2a0 tx=0x7f2d2c003f70 comp rx=0 tx=0).stop 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3723737305 shutdown_connections 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3723737305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d60810 0x55c029d6f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3723737305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d719d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3723737305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 0x55c029d602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3723737305 >> 172.21.15.69:0/3723737305 conn(0x55c029d5c580 msgr2=0x55c029d5c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3723737305 shutdown_connections 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3723737305 wait complete. 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.165+0000 7f2d3dc3e740 1 -- start start 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d3dc3e740 1 Processor -- start 2024-08-02T17:58:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d60810 0x55c029d6cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d6d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c029d71fd0 con 0x55c029d5fef0 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c029c80b20 con 0x55c029d6f5e0 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c029d72150 con 0x55c029d60810 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d3dc3e740 10 client.0 _ll_register_callbacks cb 0x55c029c7ebb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d395dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d395dc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:55118/0 (socket says 172.21.15.69:55118) 2024-08-02T17:58:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d395dc640 1 -- 172.21.15.69:0/3967204927 learned_addr learned my addr 172.21.15.69:0/3967204927 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d60810 0x55c029d6cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d6d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d395dc640 1 -- 172.21.15.69:0/3967204927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d60810 msgr2=0x55c029d6cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d395dc640 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d60810 0x55c029d6cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d395dc640 1 -- 172.21.15.69:0/3967204927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 msgr2=0x55c029d6d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d395dc640 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d6d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d395dc640 1 -- 172.21.15.69:0/3967204927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c029a373c0 con 0x55c029d5fef0 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.166+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d6d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d395dc640 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6c590 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000bf80 tx=0x7f2d300118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff99e1b921010471 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d1cff9640 1 -- 172.21.15.69:0/3967204927 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30004020 con 0x55c029d5fef0 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d1cff9640 1 -- 172.21.15.69:0/3967204927 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9656+0+0 (secure 0 0 0) 0x7f2d300041c0 con 0x55c029d5fef0 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3967204927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2c036070 con 0x55c029d5fef0 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d1cff9640 1 -- 172.21.15.69:0/3967204927 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d30025050 con 0x55c029d5fef0 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 10 client.18258 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3967204927 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c029d7dc30 con 0x55c029d5fef0 2024-08-02T17:58:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d1cff9640 1 -- 172.21.15.69:0/3967204927 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c036070 con 0x55c029d5fef0 2024-08-02T17:58:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d1cff9640 1 -- 172.21.15.69:0/3967204927 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d30003950 con 0x55c029d5fef0 2024-08-02T17:58:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 10 client.18258 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 10 client.18258 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:58:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 20 client.18258 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c029d7f140 0x55c029d815a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 4 client.18258 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:07.180 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.167+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3967204927 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c029cd2fb0 con 0x55c029d7f140 2024-08-02T17:58:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.168+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c029d7f140 0x55c029d815a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.168+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c029d7f140 0x55c029d815a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cec7ee4e14c03124 server_cookie=f5c15c3e7c1c73c9 in_seq=0 out_seq=0 2024-08-02T17:58:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.168+0000 7f2d1cff9640 10 client.18258 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:58:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d1cff9640 1 -- 172.21.15.69:0/3967204927 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c029cd2fb0 con 0x55c029d7f140 2024-08-02T17:58:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d1cff9640 10 client.18258 handle_command_reply: tid=0 2024-08-02T17:58:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 client.18258 shutdown 2024-08-02T17:58:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3967204927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 msgr2=0x55c029d6c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6c590 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000bf80 tx=0x7f2d300118d0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3967204927 shutdown_connections 2024-08-02T17:58:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c029d7f140 0x55c029d815a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d6f5e0 0x55c029d6d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d60810 0x55c029d6cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/3967204927 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6c590 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3967204927 >> 172.21.15.69:0/3967204927 conn(0x55c029d5c580 msgr2=0x55c029d622e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d1cff9640 0 client.18258 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:58:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3967204927 shutdown_connections 2024-08-02T17:58:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/3967204927 wait complete. 2024-08-02T17:58:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.169+0000 7f2d3dc3e740 20 client.18258 trim_cache size 0 max 16384 2024-08-02T17:58:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.183+0000 7f2d3dc3e740 1 Processor -- start 2024-08-02T17:58:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.186+0000 7f2d3dc3e740 1 -- start start 2024-08-02T17:58:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.186+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.186+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.186+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d6f5e0 0x55c029d86e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.186+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341af5c0 con 0x55c029d5fef0 2024-08-02T17:58:07.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.186+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341ace90 con 0x55c029d60810 2024-08-02T17:58:07.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.186+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341aad00 con 0x55c029d6f5e0 2024-08-02T17:58:07.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d38ddb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d38ddb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42932/0 (socket says 172.21.15.69:42932) 2024-08-02T17:58:07.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d38ddb640 1 -- 172.21.15.69:0/1600902775 learned_addr learned my addr 172.21.15.69:0/1600902775 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:07.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d395dc640 1 --2- 172.21.15.69:0/1600902775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d38ddb640 1 -- 172.21.15.69:0/1600902775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d6f5e0 msgr2=0x55c029d86e70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:58:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/1600902775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d6f5e0 0x55c029d86e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d38ddb640 1 -- 172.21.15.69:0/1600902775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 msgr2=0x55c029d6be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/1600902775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.187+0000 7f2d38ddb640 1 -- 172.21.15.69:0/1600902775 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c029d6b540 con 0x55c029d60810 2024-08-02T17:58:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.188+0000 7f2d38ddb640 1 --2- 172.21.15.69:0/1600902775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d621d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2d240058a0 tx=0x7f2d24004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d95f91ecc0b1a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:07.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.188+0000 7f2d1effd640 1 -- 172.21.15.69:0/1600902775 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24010e00 con 0x55c029d60810 2024-08-02T17:58:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.188+0000 7f2d1effd640 1 -- 172.21.15.69:0/1600902775 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2402f070 con 0x55c029d60810 2024-08-02T17:58:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.188+0000 7f2d1effd640 1 -- 172.21.15.69:0/1600902775 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2402aae0 con 0x55c029d60810 2024-08-02T17:58:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.189+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/1600902775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 msgr2=0x55c029d621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.189+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/1600902775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d621d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2d240058a0 tx=0x7f2d24004730 comp rx=0 tx=0).stop 2024-08-02T17:58:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.190+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/1600902775 shutdown_connections 2024-08-02T17:58:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.190+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/1600902775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d6f5e0 0x55c029d86e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.190+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/1600902775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d621d0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.190+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/1600902775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d5fef0 0x55c029d6be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.190+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/1600902775 >> 172.21.15.69:0/1600902775 conn(0x55c029d5c580 msgr2=0x55c029d61ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.190+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/1600902775 shutdown_connections 2024-08-02T17:58:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.190+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/1600902775 wait complete. 2024-08-02T17:58:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.190+0000 7f2d3dc3e740 1 -- start start 2024-08-02T17:58:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d3dc3e740 1 Processor -- start 2024-08-02T17:58:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 0x55c029d63330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d63870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d3dc3e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d6f5e0 0x55c029d76390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341ab350 con 0x55c029d6f5e0 2024-08-02T17:58:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d3dc3e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341ac7c0 con 0x55c029d60810 2024-08-02T17:58:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d38ddb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d63870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d38ddb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d63870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:42944/0 (socket says 172.21.15.69:42944) 2024-08-02T17:58:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d38ddb640 1 -- 172.21.15.69:0/2885110358 learned_addr learned my addr 172.21.15.69:0/2885110358 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d395dc640 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 0x55c029d63330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.191+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d6f5e0 0x55c029d76390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.192+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2d341aba20 con 0x55c029d5fef0 2024-08-02T17:58:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.192+0000 7f2d3dc3e740 10 client.0 _ll_register_callbacks cb 0x7f2d3000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.192+0000 7f2d39ddd640 1 -- 172.21.15.69:0/2885110358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 msgr2=0x55c029d63330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.192+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 0x55c029d63330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.192+0000 7f2d39ddd640 1 -- 172.21.15.69:0/2885110358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 msgr2=0x55c029d63870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.192+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d63870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.192+0000 7f2d39ddd640 1 -- 172.21.15.69:0/2885110358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c029d60640 con 0x55c029d6f5e0 2024-08-02T17:58:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.192+0000 7f2d39ddd640 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d6f5e0 0x55c029d76390 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0095c0 tx=0x7f2d2c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71201f06a32b07a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.193+0000 7f2d1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.193+0000 7f2d1cff9640 1 -- 172.21.15.69:0/2885110358 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c032570 con 0x55c029d6f5e0 2024-08-02T17:58:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.193+0000 7f2d1cff9640 1 -- 172.21.15.69:0/2885110358 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9656+0+0 (secure 0 0 0) 0x7f2d2c0b96e0 con 0x55c029d6f5e0 2024-08-02T17:58:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.193+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2402f070 con 0x55c029d6f5e0 2024-08-02T17:58:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.193+0000 7f2d1cff9640 1 -- 172.21.15.69:0/2885110358 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2d2c0bcbb0 con 0x55c029d6f5e0 2024-08-02T17:58:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.193+0000 7f2d3dc3e740 10 client.18264 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.193+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c029d77840 con 0x55c029d6f5e0 2024-08-02T17:58:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.194+0000 7f2d1cff9640 1 -- 172.21.15.69:0/2885110358 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c036050 con 0x55c029d6f5e0 2024-08-02T17:58:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.194+0000 7f2d1cff9640 1 -- 172.21.15.69:0/2885110358 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f2d2c04e070 con 0x55c029d6f5e0 2024-08-02T17:58:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.194+0000 7f2d3dc3e740 10 client.18264 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.194+0000 7f2d3dc3e740 10 client.18264 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:58:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.194+0000 7f2d3dc3e740 20 client.18264 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.194+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c029d77c40 0x55c029de7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.195+0000 7f2d3dc3e740 4 client.18264 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:58:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.195+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2d2402f070 con 0x55c029d77c40 2024-08-02T17:58:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.195+0000 7f2d395dc640 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c029d77c40 0x55c029de7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.196+0000 7f2d395dc640 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c029d77c40 0x55c029de7e30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=20e65cb164060335 server_cookie=e932911fc8e4d2f1 in_seq=0 out_seq=0 2024-08-02T17:58:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.196+0000 7f2d1cff9640 10 client.18264 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:58:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.196+0000 7f2d1cff9640 1 -- 172.21.15.69:0/2885110358 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2d2402f070 con 0x55c029d77c40 2024-08-02T17:58:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.196+0000 7f2d1cff9640 10 client.18264 handle_command_reply: tid=0 2024-08-02T17:58:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.196+0000 7f2d3dc3e740 1 client.18264 shutdown 2024-08-02T17:58:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.196+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d6f5e0 msgr2=0x55c029d76390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.196+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d6f5e0 0x55c029d76390 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c0095c0 tx=0x7f2d2c005f60 comp rx=0 tx=0).stop 2024-08-02T17:58:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.197+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 shutdown_connections 2024-08-02T17:58:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.197+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55c029d77c40 0x55c029de7e30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.197+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c029d6f5e0 0x55c029d76390 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.197+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c029d60810 0x55c029d63870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.197+0000 7f2d3dc3e740 1 --2- 172.21.15.69:0/2885110358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c029d5fef0 0x55c029d63330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.197+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 >> 172.21.15.69:0/2885110358 conn(0x55c029d5c580 msgr2=0x55c029d622e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.197+0000 7f2d1cff9640 0 client.18264 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:58:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.198+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 shutdown_connections 2024-08-02T17:58:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.198+0000 7f2d3dc3e740 1 -- 172.21.15.69:0/2885110358 wait complete. 2024-08-02T17:58:07.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.198+0000 7f2d3dc3e740 20 client.18264 trim_cache size 0 max 16384 2024-08-02T17:58:07.195 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:58:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 -- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d200a2cf0 msgr2=0x7f2d200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d200a2cf0 0x7f2d200a51b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2d24012960 tx=0x7f2d2400bb70 comp rx=0 tx=0).stop 2024-08-02T17:58:07.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 -- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d341078e0 msgr2=0x7f2d3419ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:07.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d341078e0 0x7f2d3419ca60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c033040 tx=0x7f2d2c003970 comp rx=0 tx=0).stop 2024-08-02T17:58:07.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 -- 172.21.15.69:0/1695332277 shutdown_connections 2024-08-02T17:58:07.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2d341078e0 0x7f2d3419ca60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2d200a2cf0 0x7f2d200a51b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2d34102280 0x7f2d341986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 --2- 172.21.15.69:0/1695332277 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2d341018e0 0x7f2d34198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:07.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.199+0000 7f2d3b840640 1 -- 172.21.15.69:0/1695332277 >> 172.21.15.69:0/1695332277 conn(0x7f2d340fd600 msgr2=0x7f2d340fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:07.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.200+0000 7f2d3b840640 1 -- 172.21.15.69:0/1695332277 shutdown_connections 2024-08-02T17:58:07.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:06.200+0000 7f2d3b840640 1 -- 172.21.15.69:0/1695332277 wait complete. 2024-08-02T17:58:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:08 smithi083 ceph-mon[37031]: pgmap v365: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.9 MiB/s wr, 720 op/s 2024-08-02T17:58:08.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:08 smithi077 ceph-mon[37108]: pgmap v365: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.9 MiB/s wr, 720 op/s 2024-08-02T17:58:08.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:08 smithi069 ceph-mon[32746]: pgmap v365: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.9 MiB/s wr, 720 op/s 2024-08-02T17:58:09.374 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:58:09.926 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 660.0 s, 416.4 tps, lat 248.056 ms stddev 380.516 2024-08-02T17:58:10.701 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:10 smithi069 ceph-mon[32746]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 708 op/s 2024-08-02T17:58:10.701 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:10 smithi069 ceph-mon[32746]: osdmap e107: 12 total, 12 up, 12 in 2024-08-02T17:58:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.730+0000 7ffa4526f640 1 -- 172.21.15.69:0/337428628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 msgr2=0x7ffa40105640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.730+0000 7ffa4526f640 1 --2- 172.21.15.69:0/337428628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 0x7ffa40105640 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009860 tx=0x7ffa3002a3f0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.737+0000 7ffa4526f640 1 -- 172.21.15.69:0/337428628 shutdown_connections 2024-08-02T17:58:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.737+0000 7ffa4526f640 1 --2- 172.21.15.69:0/337428628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40105c60 0x7ffa401060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.737+0000 7ffa4526f640 1 --2- 172.21.15.69:0/337428628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 0x7ffa40105640 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.737+0000 7ffa4526f640 1 --2- 172.21.15.69:0/337428628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa40100850 0x7ffa40100c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.737+0000 7ffa4526f640 1 -- 172.21.15.69:0/337428628 >> 172.21.15.69:0/337428628 conn(0x7ffa400fc500 msgr2=0x7ffa400fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:10.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.746+0000 7ffa4526f640 1 -- 172.21.15.69:0/337428628 shutdown_connections 2024-08-02T17:58:10.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.750+0000 7ffa4526f640 1 -- 172.21.15.69:0/337428628 wait complete. 2024-08-02T17:58:10.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.750+0000 7ffa4526f640 1 Processor -- start 2024-08-02T17:58:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.756+0000 7ffa4526f640 1 -- start start 2024-08-02T17:58:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.756+0000 7ffa4526f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40100850 0x7ffa40072970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.756+0000 7ffa4526f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 0x7ffa40072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.756+0000 7ffa4526f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa40105c60 0x7ffa4010ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.756+0000 7ffa4526f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40107220 con 0x7ffa40100850 2024-08-02T17:58:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.756+0000 7ffa4526f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401070a0 con 0x7ffa40101170 2024-08-02T17:58:10.758 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.756+0000 7ffa4526f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401073a0 con 0x7ffa40105c60 2024-08-02T17:58:10.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.759+0000 7ffa3e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 0x7ffa40072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.759+0000 7ffa3e7fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 0x7ffa40072eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43052/0 (socket says 172.21.15.69:43052) 2024-08-02T17:58:10.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.759+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/3434456653 learned_addr learned my addr 172.21.15.69:0/3434456653 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:10.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.762+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/3434456653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa40105c60 msgr2=0x7ffa4010ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.762+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa40105c60 0x7ffa4010ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.763+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40100850 msgr2=0x7ffa40072970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.763+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40100850 0x7ffa40072970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.763+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/3434456653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4010b200 con 0x7ffa40101170 2024-08-02T17:58:10.771 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.768+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 0x7ffa40072eb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffa300052a0 tx=0x7ffa300052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.771+0000 7ffa37fff640 1 -- 172.21.15.69:0/3434456653 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30031600 con 0x7ffa40101170 2024-08-02T17:58:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.771+0000 7ffa37fff640 1 -- 172.21.15.69:0/3434456653 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa300046c0 con 0x7ffa40101170 2024-08-02T17:58:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.771+0000 7ffa37fff640 1 -- 172.21.15.69:0/3434456653 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3002f7c0 con 0x7ffa40101170 2024-08-02T17:58:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.772+0000 7ffa4526f640 1 -- 172.21.15.69:0/3434456653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4010b490 con 0x7ffa40101170 2024-08-02T17:58:10.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.772+0000 7ffa4526f640 1 -- 172.21.15.69:0/3434456653 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa40112e10 con 0x7ffa40101170 2024-08-02T17:58:10.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.783+0000 7ffa4766d740 1 Processor -- start 2024-08-02T17:58:10.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.788+0000 7ffa4766d740 1 -- start start 2024-08-02T17:58:10.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.789+0000 7ffa37fff640 1 -- 172.21.15.69:0/3434456653 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa30039040 con 0x7ffa40101170 2024-08-02T17:58:10.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.789+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c52ff0 0x561a66c533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.789+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c53910 0x561a66c5dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.789+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.789+0000 7ffa4766d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561a66bb8a40 con 0x561a66c52ff0 2024-08-02T17:58:10.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.789+0000 7ffa4766d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a66b509d0 con 0x561a66c53910 2024-08-02T17:58:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.789+0000 7ffa4766d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561a66bb4a00 con 0x561a66c5e1d0 2024-08-02T17:58:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.791+0000 7ffa3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.791+0000 7ffa3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42962/0 (socket says 172.21.15.69:42962) 2024-08-02T17:58:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.791+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/2085831220 learned_addr learned my addr 172.21.15.69:0/2085831220 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:10.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.791+0000 7ffa3effd640 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c53910 0x561a66c5dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.795+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c52ff0 0x561a66c533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.799 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.798+0000 7ffa37fff640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffa2c0a2f00 0x7ffa2c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.798+0000 7ffa37fff640 1 -- 172.21.15.69:0/3434456653 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7ffa300f08f0 con 0x7ffa40101170 2024-08-02T17:58:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.799+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/2085831220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c53910 msgr2=0x561a66c5dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.799+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c53910 0x561a66c5dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.799+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/2085831220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c52ff0 msgr2=0x561a66c533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.799+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c52ff0 0x561a66c533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.799+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/2085831220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a66b57650 con 0x561a66c5e1d0 2024-08-02T17:58:10.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.800+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c605c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b9390 tx=0x7ffa3002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e769304fb8375f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:10.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.803+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffa2c0a2f00 0x7ffa2c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.803+0000 7ffa357fa640 1 -- 172.21.15.69:0/2085831220 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa300bc320 con 0x561a66c5e1d0 2024-08-02T17:58:10.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.803+0000 7ffa357fa640 1 -- 172.21.15.69:0/2085831220 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa300f3e50 con 0x561a66c5e1d0 2024-08-02T17:58:10.805 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.803+0000 7ffa357fa640 1 -- 172.21.15.69:0/2085831220 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3002f300 con 0x561a66c5e1d0 2024-08-02T17:58:10.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.804+0000 7ffa4766d740 1 -- 172.21.15.69:0/2085831220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 msgr2=0x561a66c605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.804+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c605c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b9390 tx=0x7ffa3002a3f0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.806+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffa2c0a2f00 0x7ffa2c0a53c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffa38005fd0 tx=0x7ffa380095c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:10.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.808+0000 7ffa4766d740 1 -- 172.21.15.69:0/2085831220 shutdown_connections 2024-08-02T17:58:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.808+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c605c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.808+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c53910 0x561a66c5dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.808+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2085831220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c52ff0 0x561a66c533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.808+0000 7ffa4766d740 1 -- 172.21.15.69:0/2085831220 >> 172.21.15.69:0/2085831220 conn(0x561a66c4f580 msgr2=0x561a66c4f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:10.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.808+0000 7ffa4766d740 1 -- 172.21.15.69:0/2085831220 shutdown_connections 2024-08-02T17:58:10.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.811+0000 7ffa4766d740 1 -- 172.21.15.69:0/2085831220 wait complete. 2024-08-02T17:58:10.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.811+0000 7ffa4766d740 1 -- start start 2024-08-02T17:58:10.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.811+0000 7ffa4766d740 1 Processor -- start 2024-08-02T17:58:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c52ff0 0x561a66c5b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 0x561a66c5b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c5e1d0 0x561a66c5bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa4766d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561a66c60b60 con 0x561a66c53910 2024-08-02T17:58:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa4766d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561a66b73b20 con 0x561a66c5e1d0 2024-08-02T17:58:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa4766d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561a66c60ce0 con 0x561a66c52ff0 2024-08-02T17:58:10.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa4766d740 10 client.0 _ll_register_callbacks cb 0x561a66b71bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c52ff0 0x561a66c5b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c5e1d0 0x561a66c5bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa3f7fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c5e1d0 0x561a66c5bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43076/0 (socket says 172.21.15.69:43076) 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa3f7fe640 1 -- 172.21.15.69:0/1023288872 learned_addr learned my addr 172.21.15.69:0/1023288872 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 0x561a66c5b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa3f7fe640 1 -- 172.21.15.69:0/1023288872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c52ff0 msgr2=0x561a66c5b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c52ff0 0x561a66c5b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.817+0000 7ffa3f7fe640 1 -- 172.21.15.69:0/1023288872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 msgr2=0x561a66c5b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.818+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 0x561a66c5b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.818+0000 7ffa3f7fe640 1 -- 172.21.15.69:0/1023288872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a6692a3c0 con 0x561a66c5e1d0 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.818+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c5e1d0 0x561a66c5bb20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ccb0 tx=0x7ffa38004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbfa57b311415476 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.819+0000 7ffa267fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:58:10.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.819+0000 7ffa267fc640 1 -- 172.21.15.69:0/1023288872 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38003b70 con 0x561a66c5e1d0 2024-08-02T17:58:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.819+0000 7ffa4766d740 1 -- 172.21.15.69:0/1023288872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa300f3e50 con 0x561a66c5e1d0 2024-08-02T17:58:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.819+0000 7ffa267fc640 1 -- 172.21.15.69:0/1023288872 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7ffa38003d10 con 0x561a66c5e1d0 2024-08-02T17:58:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.820+0000 7ffa267fc640 1 -- 172.21.15.69:0/1023288872 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa3802f050 con 0x561a66c5e1d0 2024-08-02T17:58:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.820+0000 7ffa4766d740 10 client.27722 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:10.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.820+0000 7ffa4766d740 1 -- 172.21.15.69:0/1023288872 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a66c6c780 con 0x561a66c5e1d0 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.824+0000 7ffa267fc640 1 -- 172.21.15.69:0/1023288872 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa300f3e50 con 0x561a66c5e1d0 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.824+0000 7ffa267fc640 1 -- 172.21.15.69:0/1023288872 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffa3800b040 con 0x561a66c5e1d0 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.824+0000 7ffa4766d740 10 client.27722 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.824+0000 7ffa4766d740 10 client.27722 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.824+0000 7ffa4766d740 20 client.27722 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.824+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561a66c6e020 0x561a66c704a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.824+0000 7ffa4766d740 4 client.27722 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.824+0000 7ffa4766d740 1 -- 172.21.15.69:0/1023288872 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a66bc5fb0 con 0x561a66c6e020 2024-08-02T17:58:10.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.825+0000 7ffa3effd640 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561a66c6e020 0x561a66c704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.828+0000 7ffa3effd640 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561a66c6e020 0x561a66c704a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=763dfb1c656bed45 server_cookie=3084ff78152cf5c1 in_seq=0 out_seq=0 2024-08-02T17:58:10.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.829+0000 7ffa267fc640 10 client.27722 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:58:10.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.831+0000 7ffa267fc640 1 -- 172.21.15.69:0/1023288872 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561a66bc5fb0 con 0x561a66c6e020 2024-08-02T17:58:10.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.831+0000 7ffa267fc640 10 client.27722 handle_command_reply: tid=0 2024-08-02T17:58:10.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.831+0000 7ffa4766d740 1 client.27722 shutdown 2024-08-02T17:58:10.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.836+0000 7ffa4766d740 1 -- 172.21.15.69:0/1023288872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c5e1d0 msgr2=0x561a66c5bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c5e1d0 0x561a66c5bb20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ccb0 tx=0x7ffa38004a40 comp rx=0 tx=0).stop 2024-08-02T17:58:10.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 -- 172.21.15.69:0/1023288872 shutdown_connections 2024-08-02T17:58:10.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561a66c6e020 0x561a66c704a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 0x561a66c5b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c5e1d0 0x561a66c5bb20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1023288872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c52ff0 0x561a66c5b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 -- 172.21.15.69:0/1023288872 >> 172.21.15.69:0/1023288872 conn(0x561a66c4f580 msgr2=0x561a66c552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa267fc640 0 client.27722 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:58:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 -- 172.21.15.69:0/1023288872 shutdown_connections 2024-08-02T17:58:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.837+0000 7ffa4766d740 1 -- 172.21.15.69:0/1023288872 wait complete. 2024-08-02T17:58:10.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.839+0000 7ffa4766d740 20 client.27722 trim_cache size 0 max 16384 2024-08-02T17:58:10.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.868+0000 7ffa4766d740 1 Processor -- start 2024-08-02T17:58:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.868+0000 7ffa4766d740 1 -- start start 2024-08-02T17:58:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.869+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c5b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.869+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 0x561a66c75db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.869+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.869+0000 7ffa3effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c5b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.869+0000 7ffa3effd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c5b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43078/0 (socket says 172.21.15.69:43078) 2024-08-02T17:58:10.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.869+0000 7ffa3effd640 1 -- 172.21.15.69:0/1201165235 learned_addr learned my addr 172.21.15.69:0/1201165235 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.869+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa4766d740 1 -- 172.21.15.69:0/1201165235 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40122160 con 0x561a66c53910 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa4766d740 1 -- 172.21.15.69:0/1201165235 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4011fa30 con 0x561a66c52ff0 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 0x561a66c75db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa4766d740 1 -- 172.21.15.69:0/1201165235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4011d880 con 0x561a66c5e1d0 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa3f7fe640 1 -- 172.21.15.69:0/1201165235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 msgr2=0x561a66c5b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c5b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa3f7fe640 1 -- 172.21.15.69:0/1201165235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 msgr2=0x561a66c75db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:10.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 0x561a66c75db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.870+0000 7ffa3f7fe640 1 -- 172.21.15.69:0/1201165235 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a66c4a750 con 0x561a66c5e1d0 2024-08-02T17:58:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.871+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c762f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800c390 tx=0x7ffa38009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a540e74c7b21b850 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.871+0000 7ffa34ff9640 1 -- 172.21.15.69:0/1201165235 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3802c030 con 0x561a66c5e1d0 2024-08-02T17:58:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.872+0000 7ffa34ff9640 1 -- 172.21.15.69:0/1201165235 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa3800eda0 con 0x561a66c5e1d0 2024-08-02T17:58:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.872+0000 7ffa34ff9640 1 -- 172.21.15.69:0/1201165235 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3801bd20 con 0x561a66c5e1d0 2024-08-02T17:58:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.872+0000 7ffa4766d740 1 -- 172.21.15.69:0/1201165235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 msgr2=0x561a66c762f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.872+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c762f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800c390 tx=0x7ffa38009630 comp rx=0 tx=0).stop 2024-08-02T17:58:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.873+0000 7ffa4766d740 1 -- 172.21.15.69:0/1201165235 shutdown_connections 2024-08-02T17:58:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.873+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c53910 0x561a66c75db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.873+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c5e1d0 0x561a66c762f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.873+0000 7ffa4766d740 1 --2- 172.21.15.69:0/1201165235 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c5b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.873+0000 7ffa4766d740 1 -- 172.21.15.69:0/1201165235 >> 172.21.15.69:0/1201165235 conn(0x561a66c4f580 msgr2=0x561a66c5a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.873+0000 7ffa4766d740 1 -- 172.21.15.69:0/1201165235 shutdown_connections 2024-08-02T17:58:10.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.873+0000 7ffa4766d740 1 -- 172.21.15.69:0/1201165235 wait complete. 2024-08-02T17:58:10.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.873+0000 7ffa4766d740 1 -- start start 2024-08-02T17:58:10.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.874+0000 7ffa4766d740 1 Processor -- start 2024-08-02T17:58:10.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.874+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c58cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.874+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c53910 0x561a66c65ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.874+0000 7ffa4766d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c5e1d0 0x561a66c661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.874+0000 7ffa4766d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4011def0 con 0x561a66c5e1d0 2024-08-02T17:58:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.874+0000 7ffa4766d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4011f360 con 0x561a66c52ff0 2024-08-02T17:58:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.874+0000 7ffa4766d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4011e5c0 con 0x561a66c53910 2024-08-02T17:58:10.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa4766d740 10 client.0 _ll_register_callbacks cb 0x7ffa38009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:10.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c53910 0x561a66c65ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3e7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c53910 0x561a66c65ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:42984/0 (socket says 172.21.15.69:42984) 2024-08-02T17:58:10.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/2852260568 learned_addr learned my addr 172.21.15.69:0/2852260568 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:10.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3effd640 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c58cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3f7fe640 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c5e1d0 0x561a66c661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/2852260568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 msgr2=0x561a66c58cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:10.879 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c58cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/2852260568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c5e1d0 msgr2=0x561a66c661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c5e1d0 0x561a66c661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.875+0000 7ffa3e7fc640 1 -- 172.21.15.69:0/2852260568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a66c57330 con 0x561a66c53910 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.876+0000 7ffa3e7fc640 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c53910 0x561a66c65ba0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa300bc950 tx=0x7ffa300c56b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d6e698fc1f9f64 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.876+0000 7ffa267fc640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.876+0000 7ffa267fc640 1 -- 172.21.15.69:0/2852260568 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa300d9070 con 0x561a66c53910 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.876+0000 7ffa267fc640 1 -- 172.21.15.69:0/2852260568 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7ffa300db2c0 con 0x561a66c53910 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 1 -- 172.21.15.69:0/2852260568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3800eda0 con 0x561a66c53910 2024-08-02T17:58:10.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa267fc640 1 -- 172.21.15.69:0/2852260568 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffa300f6460 con 0x561a66c53910 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 10 client.27529 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 1 -- 172.21.15.69:0/2852260568 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a66cd8000 con 0x561a66c53910 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa267fc640 1 -- 172.21.15.69:0/2852260568 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa3800eda0 con 0x561a66c53910 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa267fc640 1 -- 172.21.15.69:0/2852260568 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffa300b9030 con 0x561a66c53910 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 10 client.27529 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 10 client.27529 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 20 client.27529 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561a66cd8380 0x561a66cda7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:10.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 4 client.27529 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:58:10.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.877+0000 7ffa4766d740 1 -- 172.21.15.69:0/2852260568 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a66bc04c0 con 0x561a66cd8380 2024-08-02T17:58:10.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.878+0000 7ffa3effd640 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561a66cd8380 0x561a66cda7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:10.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.879+0000 7ffa3effd640 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561a66cd8380 0x561a66cda7a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88e27a2475fa5401 server_cookie=5a2bbaa0f125b8a1 in_seq=0 out_seq=0 2024-08-02T17:58:10.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.879+0000 7ffa267fc640 10 client.27529 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:58:10.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.879+0000 7ffa267fc640 1 -- 172.21.15.69:0/2852260568 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a66bc04c0 con 0x561a66cd8380 2024-08-02T17:58:10.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.879+0000 7ffa267fc640 10 client.27529 handle_command_reply: tid=0 2024-08-02T17:58:10.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 client.27529 shutdown 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 -- 172.21.15.69:0/2852260568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c53910 msgr2=0x561a66c65ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c53910 0x561a66c65ba0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffa300bc950 tx=0x7ffa300c56b0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 -- 172.21.15.69:0/2852260568 shutdown_connections 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x561a66cd8380 0x561a66cda7a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561a66c5e1d0 0x561a66c661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561a66c53910 0x561a66c65ba0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 --2- 172.21.15.69:0/2852260568 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561a66c52ff0 0x561a66c58cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 -- 172.21.15.69:0/2852260568 >> 172.21.15.69:0/2852260568 conn(0x561a66c4f580 msgr2=0x561a66c54a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa267fc640 0 client.27529 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 -- 172.21.15.69:0/2852260568 shutdown_connections 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 1 -- 172.21.15.69:0/2852260568 wait complete. 2024-08-02T17:58:10.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.880+0000 7ffa4766d740 20 client.27529 trim_cache size 0 max 16384 2024-08-02T17:58:10.885 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:58:10.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 -- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffa2c0a2f00 msgr2=0x7ffa2c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffa2c0a2f00 0x7ffa2c0a53c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ffa38005fd0 tx=0x7ffa380095c0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 -- 172.21.15.69:0/3434456653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 msgr2=0x7ffa40072eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 0x7ffa40072eb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ffa300052a0 tx=0x7ffa300052d0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 -- 172.21.15.69:0/3434456653 shutdown_connections 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffa2c0a2f00 0x7ffa2c0a53c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffa40105c60 0x7ffa4010ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffa40101170 0x7ffa40072eb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 --2- 172.21.15.69:0/3434456653 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffa40100850 0x7ffa40072970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 -- 172.21.15.69:0/3434456653 >> 172.21.15.69:0/3434456653 conn(0x7ffa400fc500 msgr2=0x7ffa401035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.881+0000 7ffa4526f640 1 -- 172.21.15.69:0/3434456653 shutdown_connections 2024-08-02T17:58:10.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:10.882+0000 7ffa4526f640 1 -- 172.21.15.69:0/3434456653 wait complete. 2024-08-02T17:58:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:10 smithi083 ceph-mon[37031]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 708 op/s 2024-08-02T17:58:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:10 smithi083 ceph-mon[37031]: osdmap e107: 12 total, 12 up, 12 in 2024-08-02T17:58:11.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:10 smithi077 ceph-mon[37108]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 708 op/s 2024-08-02T17:58:11.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:10 smithi077 ceph-mon[37108]: osdmap e107: 12 total, 12 up, 12 in 2024-08-02T17:58:11.603 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:58:12.123 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:12.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.767+0000 7fe3e6b46640 1 -- 172.21.15.69:0/1775562336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e0100b50 msgr2=0x7fe3e0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.767+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/1775562336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e0100b50 0x7fe3e0100f30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009930 tx=0x7fe3d402a4d0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.771+0000 7fe3e6b46640 1 -- 172.21.15.69:0/1775562336 shutdown_connections 2024-08-02T17:58:12.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.771+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/1775562336 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e01022a0 0x7fe3e01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.771+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/1775562336 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e0102c20 0x7fe3e010d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.773 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.771+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/1775562336 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e0100b50 0x7fe3e0100f30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.774 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.771+0000 7fe3e6b46640 1 -- 172.21.15.69:0/1775562336 >> 172.21.15.69:0/1775562336 conn(0x7fe3e00fc500 msgr2=0x7fe3e00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:12.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.779+0000 7fe3e6b46640 1 -- 172.21.15.69:0/1775562336 shutdown_connections 2024-08-02T17:58:12.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.781+0000 7fe3e6b46640 1 -- 172.21.15.69:0/1775562336 wait complete. 2024-08-02T17:58:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.782+0000 7fe3e6b46640 1 Processor -- start 2024-08-02T17:58:12.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.782+0000 7fe3e6b46640 1 -- start start 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3e6b46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e0100b50 0x7fe3e006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3e6b46640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01022a0 0x7fe3e006daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3e6b46640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e0102c20 0x7fe3e0071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3e48e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e0100b50 0x7fe3e006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3e48e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e0100b50 0x7fe3e006d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43000/0 (socket says 172.21.15.69:43000) 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01022a0 0x7fe3e006daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3dffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01022a0 0x7fe3e006daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43108/0 (socket says 172.21.15.69:43108) 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3e48e2640 1 -- 172.21.15.69:0/3562648113 learned_addr learned my addr 172.21.15.69:0/3562648113 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e006bf10 con 0x7fe3e0102c20 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.783+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e006bd90 con 0x7fe3e01022a0 2024-08-02T17:58:12.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.784+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e006c090 con 0x7fe3e0100b50 2024-08-02T17:58:12.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.784+0000 7fe3dffff640 1 -- 172.21.15.69:0/3562648113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e0100b50 msgr2=0x7fe3e006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.784+0000 7fe3dffff640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e0100b50 0x7fe3e006d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.784+0000 7fe3dffff640 1 -- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e0102c20 msgr2=0x7fe3e0071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.785+0000 7fe3dffff640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e0102c20 0x7fe3e0071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.785+0000 7fe3dffff640 1 -- 172.21.15.69:0/3562648113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e00725f0 con 0x7fe3e01022a0 2024-08-02T17:58:12.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.785+0000 7fe3dffff640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01022a0 0x7fe3e006daf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00afa0 tx=0x7fe3cc00ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:12.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.785+0000 7fe3ddffb640 1 -- 172.21.15.69:0/3562648113 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc013d70 con 0x7fe3e01022a0 2024-08-02T17:58:12.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.786+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e0072880 con 0x7fe3e01022a0 2024-08-02T17:58:12.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.786+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e0069ca0 con 0x7fe3e01022a0 2024-08-02T17:58:12.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.786+0000 7fe3ddffb640 1 -- 172.21.15.69:0/3562648113 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3cc007770 con 0x7fe3e01022a0 2024-08-02T17:58:12.789 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.787+0000 7fe3ddffb640 1 -- 172.21.15.69:0/3562648113 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0039b0 con 0x7fe3e01022a0 2024-08-02T17:58:12.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.790+0000 7fe3e8f44740 1 Processor -- start 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.792+0000 7fe3e8f44740 1 -- start start 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.792+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f0800 0x55f33f6f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.792+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f1120 0x55f33f6fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.792+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f6fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.792+0000 7fe3e8f44740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f33f64da40 con 0x55f33f6f0800 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.792+0000 7fe3e8f44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f33f5e59d0 con 0x55f33f6fb9e0 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.792+0000 7fe3e8f44740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f33f649a00 con 0x55f33f6f1120 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3e48e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f1120 0x55f33f6fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3e48e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f1120 0x55f33f6fb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43002/0 (socket says 172.21.15.69:43002) 2024-08-02T17:58:12.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3e48e2640 1 -- 172.21.15.69:0/2280049349 learned_addr learned my addr 172.21.15.69:0/2280049349 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3ddffb640 1 -- 172.21.15.69:0/3562648113 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe3cc007920 con 0x7fe3e01022a0 2024-08-02T17:58:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3dffff640 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f6fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3e48e2640 1 -- 172.21.15.69:0/2280049349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 msgr2=0x55f33f6fddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f6fddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3e48e2640 1 -- 172.21.15.69:0/2280049349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f0800 msgr2=0x55f33f6f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f0800 0x55f33f6f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.793+0000 7fe3e48e2640 1 -- 172.21.15.69:0/2280049349 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33f5ec650 con 0x55f33f6f1120 2024-08-02T17:58:12.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.794+0000 7fe3dffff640 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f6fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:58:12.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.795+0000 7fe3ddffb640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3d00a2d10 0x7fe3d00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.795+0000 7fe3ddffb640 1 -- 172.21.15.69:0/3562648113 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fe3cc0cf900 con 0x7fe3e01022a0 2024-08-02T17:58:12.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.795+0000 7fe3e50e3640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3d00a2d10 0x7fe3d00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.796+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f1120 0x55f33f6fb4a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005ab0 tx=0x7fe3d40316f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96179ac47707afcd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:12.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.796+0000 7fe3caffd640 1 -- 172.21.15.69:0/2280049349 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4040070 con 0x55f33f6f1120 2024-08-02T17:58:12.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.798+0000 7fe3e50e3640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3d00a2d10 0x7fe3d00a51d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8005c40 tx=0x7fe3d8005bd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:12.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.798+0000 7fe3caffd640 1 -- 172.21.15.69:0/2280049349 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d4005040 con 0x55f33f6f1120 2024-08-02T17:58:12.801 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.798+0000 7fe3caffd640 1 -- 172.21.15.69:0/2280049349 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4035070 con 0x55f33f6f1120 2024-08-02T17:58:12.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.800+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2280049349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f1120 msgr2=0x55f33f6fb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.802 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.800+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f1120 0x55f33f6fb4a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4005ab0 tx=0x7fe3d40316f0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.804+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2280049349 shutdown_connections 2024-08-02T17:58:12.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.804+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f6fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.804+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f1120 0x55f33f6fb4a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.804+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2280049349 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f0800 0x55f33f6f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.804+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2280049349 >> 172.21.15.69:0/2280049349 conn(0x55f33f6ecd90 msgr2=0x55f33f6ed170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:12.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.804+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2280049349 shutdown_connections 2024-08-02T17:58:12.806 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.804+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2280049349 wait complete. 2024-08-02T17:58:12.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.805+0000 7fe3e8f44740 1 -- start start 2024-08-02T17:58:12.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.805+0000 7fe3e8f44740 1 Processor -- start 2024-08-02T17:58:12.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.806+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6f0800 0x55f33f6f88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.806+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 0x55f33f6f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.806+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6fb9e0 0x55f33f6f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.806+0000 7fe3e8f44740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f33f6fe5b0 con 0x55f33f6f1120 2024-08-02T17:58:12.808 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e50e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6fb9e0 0x55f33f6f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e50e3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6fb9e0 0x55f33f6f9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43006/0 (socket says 172.21.15.69:43006) 2024-08-02T17:58:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e50e3640 1 -- 172.21.15.69:0/2991973548 learned_addr learned my addr 172.21.15.69:0/2991973548 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f33f608b20 con 0x55f33f6f0800 2024-08-02T17:58:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f33f6fe730 con 0x55f33f6fb9e0 2024-08-02T17:58:12.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e8f44740 10 client.0 _ll_register_callbacks cb 0x55f33f606bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:12.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e50e3640 1 -- 172.21.15.69:0/2991973548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6f0800 msgr2=0x55f33f6f88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:12.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e50e3640 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6f0800 0x55f33f6f88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e50e3640 1 -- 172.21.15.69:0/2991973548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 msgr2=0x55f33f6f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:12.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e50e3640 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 0x55f33f6f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.807+0000 7fe3e50e3640 1 -- 172.21.15.69:0/2991973548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33f3bf3c0 con 0x55f33f6fb9e0 2024-08-02T17:58:12.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.808+0000 7fe3e50e3640 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6fb9e0 0x55f33f6f9320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8010790 tx=0x7fe3d8010b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2e50fdecd592cb0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:12.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.808+0000 7fe3bffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:12.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.808+0000 7fe3bffff640 1 -- 172.21.15.69:0/2991973548 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8020070 con 0x55f33f6fb9e0 2024-08-02T17:58:12.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.808+0000 7fe3bffff640 1 -- 172.21.15.69:0/2991973548 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fe3d800fb40 con 0x55f33f6fb9e0 2024-08-02T17:58:12.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.808+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d4005040 con 0x55f33f6fb9e0 2024-08-02T17:58:12.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.809+0000 7fe3bffff640 1 -- 172.21.15.69:0/2991973548 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3d800fd40 con 0x55f33f6fb9e0 2024-08-02T17:58:12.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.809+0000 7fe3e8f44740 10 client.27541 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:12.811 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.809+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55f33f70a120 con 0x55f33f6fb9e0 2024-08-02T17:58:12.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.815+0000 7fe3bffff640 1 -- 172.21.15.69:0/2991973548 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3d8025460 con 0x55f33f6fb9e0 2024-08-02T17:58:12.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.815+0000 7fe3e8f44740 10 client.27541 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:12.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.815+0000 7fe3e8f44740 10 client.27541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:12.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.815+0000 7fe3e8f44740 20 client.27541 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:12.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.815+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f33f70b4d0 0x55f33f70d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.815+0000 7fe3e8f44740 4 client.27541 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:12.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.815+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe3d4005040 con 0x55f33f70b4d0 2024-08-02T17:58:12.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.816+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f33f70b4d0 0x55f33f70d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.817+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f33f70b4d0 0x55f33f70d960 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac61a8074d38e508 server_cookie=b06fbd902e503dcb in_seq=0 out_seq=0 2024-08-02T17:58:12.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.817+0000 7fe3bffff640 10 client.27541 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:12.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.817+0000 7fe3bffff640 1 -- 172.21.15.69:0/2991973548 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe3d4005040 con 0x55f33f70b4d0 2024-08-02T17:58:12.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.817+0000 7fe3bffff640 10 client.27541 handle_command_reply: tid=0 2024-08-02T17:58:12.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.817+0000 7fe3e8f44740 1 client.27541 shutdown 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6fb9e0 msgr2=0x55f33f6f9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6fb9e0 0x55f33f6f9320 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8010790 tx=0x7fe3d8010b90 comp rx=0 tx=0).stop 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 shutdown_connections 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 0x55f33f6f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f33f70b4d0 0x55f33f70d960 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3bffff640 0 client.27541 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6fb9e0 0x55f33f6f9320 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/2991973548 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6f0800 0x55f33f6f88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.818+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 >> 172.21.15.69:0/2991973548 conn(0x55f33f6ecd90 msgr2=0x55f33f6f2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:12.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.825+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 shutdown_connections 2024-08-02T17:58:12.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.828+0000 7fe3e8f44740 1 -- 172.21.15.69:0/2991973548 wait complete. 2024-08-02T17:58:12.829 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.828+0000 7fe3e8f44740 20 client.27541 trim_cache size 0 max 16384 2024-08-02T17:58:12.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:12 smithi069 ceph-mon[32746]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-08-02T17:58:12.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.858+0000 7fe3e8f44740 1 Processor -- start 2024-08-02T17:58:12.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.859+0000 7fe3e8f44740 1 -- start start 2024-08-02T17:58:12.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f6f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6f1120 0x55f33f713270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e48e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f6f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e48e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f6f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43018/0 (socket says 172.21.15.69:43018) 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e48e2640 1 -- 172.21.15.69:0/538103377 learned_addr learned my addr 172.21.15.69:0/538103377 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/538103377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6fb9e0 0x55f33f7137b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e8f44740 1 -- 172.21.15.69:0/538103377 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4012980 con 0x55f33f6fb9e0 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e8f44740 1 -- 172.21.15.69:0/538103377 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4010250 con 0x55f33f6f1120 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.860+0000 7fe3e8f44740 1 -- 172.21.15.69:0/538103377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400e0a0 con 0x55f33f6f0800 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.861+0000 7fe3e48e2640 1 -- 172.21.15.69:0/538103377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6f1120 msgr2=0x55f33f713270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.861+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/538103377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6f1120 0x55f33f713270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.861+0000 7fe3e48e2640 1 -- 172.21.15.69:0/538103377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6fb9e0 msgr2=0x55f33f7137b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.861+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/538103377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6fb9e0 0x55f33f7137b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.861+0000 7fe3e48e2640 1 -- 172.21.15.69:0/538103377 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f33f6fb330 con 0x55f33f6f0800 2024-08-02T17:58:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.861+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/538103377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f6f3260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40057c0 tx=0x7fe3d4031a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f653e916248ec6a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:12.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.862+0000 7fe3ca7fc640 1 -- 172.21.15.69:0/538103377 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4040070 con 0x55f33f6f0800 2024-08-02T17:58:12.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.862+0000 7fe3ca7fc640 1 -- 172.21.15.69:0/538103377 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d404a040 con 0x55f33f6f0800 2024-08-02T17:58:12.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.862+0000 7fe3ca7fc640 1 -- 172.21.15.69:0/538103377 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4035040 con 0x55f33f6f0800 2024-08-02T17:58:12.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.864+0000 7fe3e8f44740 1 -- 172.21.15.69:0/538103377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 msgr2=0x55f33f6f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.864+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/538103377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f6f3260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40057c0 tx=0x7fe3d4031a80 comp rx=0 tx=0).stop 2024-08-02T17:58:12.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.866+0000 7fe3e8f44740 1 -- 172.21.15.69:0/538103377 shutdown_connections 2024-08-02T17:58:12.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.866+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/538103377 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6fb9e0 0x55f33f7137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.866+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/538103377 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6f1120 0x55f33f713270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.866+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/538103377 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f6f3260 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.866+0000 7fe3e8f44740 1 -- 172.21.15.69:0/538103377 >> 172.21.15.69:0/538103377 conn(0x55f33f6ecd90 msgr2=0x55f33f6e35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:12.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.870+0000 7fe3e8f44740 1 -- 172.21.15.69:0/538103377 shutdown_connections 2024-08-02T17:58:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.870+0000 7fe3e8f44740 1 -- 172.21.15.69:0/538103377 wait complete. 2024-08-02T17:58:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 1 -- start start 2024-08-02T17:58:12.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 1 Processor -- start 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f703e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 0x55f33f7011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f701710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400e710 con 0x55f33f6f1120 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400fb80 con 0x55f33f6fb9e0 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c400ede0 con 0x55f33f6f0800 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e8f44740 10 client.0 _ll_register_callbacks cb 0x7fe3d8014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e50e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f701710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e48e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f703e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e50e3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f701710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:43128/0 (socket says 172.21.15.69:43128) 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e48e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f703e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43030/0 (socket says 172.21.15.69:43030) 2024-08-02T17:58:12.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.871+0000 7fe3e50e3640 1 -- 172.21.15.69:0/3762117622 learned_addr learned my addr 172.21.15.69:0/3762117622 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:12.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.872+0000 7fe3dffff640 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 0x55f33f7011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.872+0000 7fe3e48e2640 1 -- 172.21.15.69:0/3762117622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 msgr2=0x55f33f701710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:12.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.872+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f701710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.872+0000 7fe3e48e2640 1 -- 172.21.15.69:0/3762117622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 msgr2=0x55f33f7011d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:12.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.872+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 0x55f33f7011d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.872+0000 7fe3e48e2640 1 -- 172.21.15.69:0/3762117622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f33f701c50 con 0x55f33f6f0800 2024-08-02T17:58:12.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.872+0000 7fe3e50e3640 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f701710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:12.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.872+0000 7fe3e48e2640 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f703e80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40057c0 tx=0x7fe3d402a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4ad0a22c6379da3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:12.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.875+0000 7fe3bffff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:12.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.875+0000 7fe3bffff640 1 -- 172.21.15.69:0/3762117622 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4035070 con 0x55f33f6f0800 2024-08-02T17:58:12.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.875+0000 7fe3bffff640 1 -- 172.21.15.69:0/3762117622 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fe3d4004040 con 0x55f33f6f0800 2024-08-02T17:58:12.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.875+0000 7fe3e8f44740 1 -- 172.21.15.69:0/3762117622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3d404a040 con 0x55f33f6f0800 2024-08-02T17:58:12.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.878+0000 7fe3bffff640 1 -- 172.21.15.69:0/3762117622 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe3d4040050 con 0x55f33f6f0800 2024-08-02T17:58:12.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.879+0000 7fe3e8f44740 10 client.27547 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:12.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.879+0000 7fe3e8f44740 1 -- 172.21.15.69:0/3762117622 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f33f6e3f50 con 0x55f33f6f0800 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.880+0000 7fe3bffff640 1 -- 172.21.15.69:0/3762117622 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3d404a040 con 0x55f33f6f0800 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.880+0000 7fe3bffff640 1 -- 172.21.15.69:0/3762117622 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe3d403a040 con 0x55f33f6f0800 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.880+0000 7fe3e8f44740 10 client.27547 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.880+0000 7fe3e8f44740 10 client.27547 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.880+0000 7fe3e8f44740 20 client.27547 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.880+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f33f70b4d0 0x55f33f6e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.880+0000 7fe3e8f44740 4 client.27547 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.880+0000 7fe3e8f44740 1 -- 172.21.15.69:0/3762117622 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f33f65afb0 con 0x55f33f70b4d0 2024-08-02T17:58:12.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.881+0000 7fe3dffff640 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f33f70b4d0 0x55f33f6e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:12.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.884+0000 7fe3dffff640 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f33f70b4d0 0x55f33f6e61f0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31593534ea41c1e server_cookie=7096f5954b503973 in_seq=0 out_seq=0 2024-08-02T17:58:12.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.884+0000 7fe3bffff640 10 client.27547 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:12.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.884+0000 7fe3bffff640 1 -- 172.21.15.69:0/3762117622 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f33f65afb0 con 0x55f33f70b4d0 2024-08-02T17:58:12.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.884+0000 7fe3bffff640 10 client.27547 handle_command_reply: tid=0 2024-08-02T17:58:12.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.884+0000 7fe3e8f44740 1 client.27547 shutdown 2024-08-02T17:58:12.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3e8f44740 1 -- 172.21.15.69:0/3762117622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 msgr2=0x55f33f703e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.889 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f703e80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40057c0 tx=0x7fe3d402a4d0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3e8f44740 1 -- 172.21.15.69:0/3762117622 shutdown_connections 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f33f6f1120 0x55f33f7011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f33f70b4d0 0x55f33f6e61f0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f33f6fb9e0 0x55f33f701710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3e8f44740 1 --2- 172.21.15.69:0/3762117622 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f33f6f0800 0x55f33f703e80 secure :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40057c0 tx=0x7fe3d402a4d0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3e8f44740 1 -- 172.21.15.69:0/3762117622 >> 172.21.15.69:0/3762117622 conn(0x55f33f6ecd90 msgr2=0x55f33f6f9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.887+0000 7fe3bffff640 0 client.27547 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.888+0000 7fe3e8f44740 1 -- 172.21.15.69:0/3762117622 shutdown_connections 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.888+0000 7fe3e8f44740 1 -- 172.21.15.69:0/3762117622 wait complete. 2024-08-02T17:58:12.890 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.888+0000 7fe3e8f44740 20 client.27547 trim_cache size 0 max 16384 2024-08-02T17:58:12.891 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:58:12.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.889+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3d00a2d10 msgr2=0x7fe3d00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.889+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3d00a2d10 0x7fe3d00a51d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8005c40 tx=0x7fe3d8005bd0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.889+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01022a0 msgr2=0x7fe3e006daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:12.892 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.889+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01022a0 0x7fe3e006daf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc00afa0 tx=0x7fe3cc00ce40 comp rx=0 tx=0).stop 2024-08-02T17:58:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.890+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 shutdown_connections 2024-08-02T17:58:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.890+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe3e0102c20 0x7fe3e0071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.890+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3d00a2d10 0x7fe3d00a51d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.890+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe3e01022a0 0x7fe3e006daf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.890+0000 7fe3e6b46640 1 --2- 172.21.15.69:0/3562648113 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e0100b50 0x7fe3e006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:12.893 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.890+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 >> 172.21.15.69:0/3562648113 conn(0x7fe3e00fc500 msgr2=0x7fe3e01044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.890+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 shutdown_connections 2024-08-02T17:58:12.894 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:12.890+0000 7fe3e6b46640 1 -- 172.21.15.69:0/3562648113 wait complete. 2024-08-02T17:58:13.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:12 smithi083 ceph-mon[37031]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-08-02T17:58:13.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:12 smithi077 ceph-mon[37108]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 3.4 MiB/s wr, 307 op/s 2024-08-02T17:58:14.638 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:58:14.639 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:58:14.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 665.0 s, 500.8 tps, lat 199.406 ms stddev 248.729 2024-08-02T17:58:15.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:14 smithi083 ceph-mon[37031]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-08-02T17:58:15.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:14 smithi077 ceph-mon[37108]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-08-02T17:58:15.113 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:14 smithi069 ceph-mon[32746]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 6.9 MiB/s wr, 575 op/s 2024-08-02T17:58:15.163 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:15.895 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.847+0000 7fe413493640 1 -- 172.21.15.69:0/658510730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe40c1147e0 msgr2=0x7fe40c116bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.847+0000 7fe413493640 1 --2- 172.21.15.69:0/658510730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe40c1147e0 0x7fe40c116bd0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe404009ca0 tx=0x7fe40402a950 comp rx=0 tx=0).stop 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.848+0000 7fe413493640 1 -- 172.21.15.69:0/658510730 shutdown_connections 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.848+0000 7fe413493640 1 --2- 172.21.15.69:0/658510730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe40c1147e0 0x7fe40c116bd0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.848+0000 7fe413493640 1 --2- 172.21.15.69:0/658510730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe40c109840 0x7fe40c1142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.848+0000 7fe413493640 1 --2- 172.21.15.69:0/658510730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40c108f20 0x7fe40c109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.848+0000 7fe413493640 1 -- 172.21.15.69:0/658510730 >> 172.21.15.69:0/658510730 conn(0x7fe40c06de70 msgr2=0x7fe40c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.848+0000 7fe413493640 1 -- 172.21.15.69:0/658510730 shutdown_connections 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.848+0000 7fe413493640 1 -- 172.21.15.69:0/658510730 wait complete. 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.848+0000 7fe413493640 1 Processor -- start 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe413493640 1 -- start start 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe413493640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe40c108f20 0x7fe40c1a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.896 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe413493640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe40c109840 0x7fe40c1a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe413493640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40c1147e0 0x7fe40c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe413493640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c11a4b0 con 0x7fe40c109840 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe413493640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c11a330 con 0x7fe40c108f20 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe413493640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c11a630 con 0x7fe40c1147e0 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe41122f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe40c108f20 0x7fe40c1a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe41122f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe40c108f20 0x7fe40c1a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40496/0 (socket says 172.21.15.69:40496) 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe41122f640 1 -- 172.21.15.69:0/3194741309 learned_addr learned my addr 172.21.15.69:0/3194741309 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe410a2e640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe40c109840 0x7fe40c1a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.849+0000 7fe411a30640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40c1147e0 0x7fe40c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe41122f640 1 -- 172.21.15.69:0/3194741309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40c1147e0 msgr2=0x7fe40c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe41122f640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40c1147e0 0x7fe40c1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe41122f640 1 -- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe40c109840 msgr2=0x7fe40c1a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:15.897 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe41122f640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe40c109840 0x7fe40c1a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe41122f640 1 -- 172.21.15.69:0/3194741309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe40c1a0f80 con 0x7fe40c108f20 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe41122f640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe40c108f20 0x7fe40c1a6660 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009f40 tx=0x7fe3fc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe4027fc640 1 -- 172.21.15.69:0/3194741309 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc00cd80 con 0x7fe40c108f20 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe413493640 1 -- 172.21.15.69:0/3194741309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe40c1a1270 con 0x7fe40c108f20 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe4027fc640 1 -- 172.21.15.69:0/3194741309 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3fc005500 con 0x7fe40c108f20 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.850+0000 7fe413493640 1 -- 172.21.15.69:0/3194741309 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe40c10a030 con 0x7fe40c108f20 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.851+0000 7fe4027fc640 1 -- 172.21.15.69:0/3194741309 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc029060 con 0x7fe40c108f20 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.853+0000 7fe415891740 1 Processor -- start 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.853+0000 7fe415891740 1 -- start start 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.853+0000 7fe4027fc640 1 -- 172.21.15.69:0/3194741309 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe3fc0049a0 con 0x7fe40c108f20 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe415891740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828200 0x55a3618285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe415891740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828b20 0x55a361832ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe415891740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe415891740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a3617ebe40 con 0x55a3618333e0 2024-08-02T17:58:15.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe415891740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a36178d7f0 con 0x55a361828200 2024-08-02T17:58:15.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe415891740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a361734fb0 con 0x55a361828b20 2024-08-02T17:58:15.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828200 0x55a3618285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828200 0x55a3618285e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40500/0 (socket says 172.21.15.69:40500) 2024-08-02T17:58:15.899 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe41122f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828b20 0x55a361832ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 -- 172.21.15.69:0/2210536713 learned_addr learned my addr 172.21.15.69:0/2210536713 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 -- 172.21.15.69:0/2210536713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828b20 msgr2=0x55a361832ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828b20 0x55a361832ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe410a2e640 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 -- 172.21.15.69:0/2210536713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 msgr2=0x55a3618357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618357d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 -- 172.21.15.69:0/2210536713 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3614ff6a0 con 0x55a361828200 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe410a2e640 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.854+0000 7fe411a30640 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828200 0x55a3618285e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe404009ca0 tx=0x7fe404009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8e9a9b9e568fed0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:15.900 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.855+0000 7fe3f77fe640 1 -- 172.21.15.69:0/2210536713 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404034040 con 0x55a361828200 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.855+0000 7fe4027fc640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3f80a2cf0 0x7fe3f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.855+0000 7fe3f77fe640 1 -- 172.21.15.69:0/2210536713 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40402fb40 con 0x55a361828200 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.855+0000 7fe3f77fe640 1 -- 172.21.15.69:0/2210536713 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404004460 con 0x55a361828200 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.855+0000 7fe41122f640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3f80a2cf0 0x7fe3f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.855+0000 7fe4027fc640 1 -- 172.21.15.69:0/3194741309 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fe3fc0d3bd0 con 0x7fe40c108f20 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 -- 172.21.15.69:0/2210536713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828200 msgr2=0x55a3618285e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe41122f640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3f80a2cf0 0x7fe3f80a51b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55a3618255f0 tx=0x7fe3fc0d7680 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828200 0x55a3618285e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe404009ca0 tx=0x7fe404009360 comp rx=0 tx=0).stop 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 -- 172.21.15.69:0/2210536713 shutdown_connections 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828b20 0x55a361832ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 --2- 172.21.15.69:0/2210536713 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828200 0x55a3618285e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 -- 172.21.15.69:0/2210536713 >> 172.21.15.69:0/2210536713 conn(0x55a361824790 msgr2=0x55a361824b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:15.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 -- 172.21.15.69:0/2210536713 shutdown_connections 2024-08-02T17:58:15.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 -- 172.21.15.69:0/2210536713 wait complete. 2024-08-02T17:58:15.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.856+0000 7fe415891740 1 -- start start 2024-08-02T17:58:15.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.857+0000 7fe415891740 1 Processor -- start 2024-08-02T17:58:15.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.858+0000 7fe415891740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828200 0x55a361830350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.858+0000 7fe415891740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361830890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.902 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.858+0000 7fe415891740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3618333e0 0x55a361830dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.860+0000 7fe415891740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a361748cb0 con 0x55a361828200 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.860+0000 7fe415891740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a3617897b0 con 0x55a361828b20 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.860+0000 7fe415891740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a361835fb0 con 0x55a3618333e0 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.860+0000 7fe415891740 10 client.0 _ll_register_callbacks cb 0x55a36149cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.860+0000 7fe411a30640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3618333e0 0x55a361830dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.860+0000 7fe411a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828200 0x55a361830350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.860+0000 7fe411a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828200 0x55a361830350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56834/0 (socket says 172.21.15.69:56834) 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.860+0000 7fe411a30640 1 -- 172.21.15.69:0/3638398813 learned_addr learned my addr 172.21.15.69:0/3638398813 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.862+0000 7fe410a2e640 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361830890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.864+0000 7fe411a30640 1 -- 172.21.15.69:0/3638398813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3618333e0 msgr2=0x55a361830dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:15.903 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.864+0000 7fe411a30640 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3618333e0 0x55a361830dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.864+0000 7fe411a30640 1 -- 172.21.15.69:0/3638398813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 msgr2=0x55a361830890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.864+0000 7fe411a30640 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361830890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.864+0000 7fe411a30640 1 -- 172.21.15.69:0/3638398813 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a361747bc0 con 0x55a361828200 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.865+0000 7fe411a30640 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828200 0x55a361830350 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe40402ae70 tx=0x7fe404005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09646c5b1aaec09 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.867+0000 7fe3f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.867+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/3638398813 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40405a060 con 0x55a361828200 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.867+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/3638398813 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fe40404f970 con 0x55a361828200 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.867+0000 7fe415891740 1 -- 172.21.15.69:0/3638398813 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe40402fb40 con 0x55a361828200 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.868+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/3638398813 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe404061050 con 0x55a361828200 2024-08-02T17:58:15.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.868+0000 7fe415891740 10 client.18309 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.868+0000 7fe415891740 1 -- 172.21.15.69:0/3638398813 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a361841e90 con 0x55a361828200 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.868+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/3638398813 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40402fb40 con 0x55a361828200 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.869+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/3638398813 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe404042070 con 0x55a361828200 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.869+0000 7fe415891740 10 client.18309 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.869+0000 7fe415891740 10 client.18309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.869+0000 7fe415891740 20 client.18309 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.869+0000 7fe415891740 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3618432a0 0x55a361845690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.869+0000 7fe415891740 4 client.18309 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.869+0000 7fe415891740 1 -- 172.21.15.69:0/3638398813 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a36179ad60 con 0x55a3618432a0 2024-08-02T17:58:15.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.872+0000 7fe410a2e640 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3618432a0 0x55a361845690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.889+0000 7fe410a2e640 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3618432a0 0x55a361845690 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da3e08a4d46172e server_cookie=41b315e521f9cd2f in_seq=0 out_seq=0 2024-08-02T17:58:15.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.889+0000 7fe3f4ff9640 10 client.18309 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:15.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.894+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/3638398813 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a36179ad60 con 0x55a3618432a0 2024-08-02T17:58:15.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.894+0000 7fe3f4ff9640 10 client.18309 handle_command_reply: tid=0 2024-08-02T17:58:15.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.895+0000 7fe415891740 1 client.18309 shutdown 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.900+0000 7fe415891740 1 -- 172.21.15.69:0/3638398813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828200 msgr2=0x55a361830350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.900+0000 7fe415891740 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828200 0x55a361830350 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe40402ae70 tx=0x7fe404005fd0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.904+0000 7fe415891740 1 -- 172.21.15.69:0/3638398813 shutdown_connections 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.904+0000 7fe415891740 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3618432a0 0x55a361845690 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.904+0000 7fe415891740 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a3618333e0 0x55a361830dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.904+0000 7fe415891740 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361830890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.904+0000 7fe415891740 1 --2- 172.21.15.69:0/3638398813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828200 0x55a361830350 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.904+0000 7fe415891740 1 -- 172.21.15.69:0/3638398813 >> 172.21.15.69:0/3638398813 conn(0x55a361824790 msgr2=0x55a36182a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:15.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.904+0000 7fe3f4ff9640 0 client.18309 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:15.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.908+0000 7fe415891740 1 -- 172.21.15.69:0/3638398813 shutdown_connections 2024-08-02T17:58:15.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.909+0000 7fe415891740 1 -- 172.21.15.69:0/3638398813 wait complete. 2024-08-02T17:58:15.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.909+0000 7fe415891740 20 client.18309 trim_cache size 0 max 16384 2024-08-02T17:58:15.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.946+0000 7fe415891740 1 Processor -- start 2024-08-02T17:58:15.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.946+0000 7fe415891740 1 -- start start 2024-08-02T17:58:15.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.947+0000 7fe415891740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 0x55a361830c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.947+0000 7fe415891740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828b20 0x55a36184af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.947+0000 7fe415891740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3618333e0 0x55a36184b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.947+0000 7fe415891740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1b7fb0 con 0x55a361828b20 2024-08-02T17:58:15.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.947+0000 7fe415891740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1b5880 con 0x55a3618333e0 2024-08-02T17:58:15.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.947+0000 7fe415891740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1b36d0 con 0x55a361828200 2024-08-02T17:58:15.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.948+0000 7fe410a2e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828b20 0x55a36184af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.948+0000 7fe410a2e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828b20 0x55a36184af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56842/0 (socket says 172.21.15.69:56842) 2024-08-02T17:58:15.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.948+0000 7fe410a2e640 1 -- 172.21.15.69:0/1798001119 learned_addr learned my addr 172.21.15.69:0/1798001119 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:15.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.950+0000 7fe411a30640 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 0x55a361830c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.950+0000 7fe411a30640 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3618333e0 0x55a36184b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.954+0000 7fe410a2e640 1 -- 172.21.15.69:0/1798001119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 msgr2=0x55a361830c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:15.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.954+0000 7fe410a2e640 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 0x55a361830c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.954+0000 7fe410a2e640 1 -- 172.21.15.69:0/1798001119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3618333e0 msgr2=0x55a36184b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:15.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.954+0000 7fe410a2e640 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3618333e0 0x55a36184b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.954+0000 7fe410a2e640 1 -- 172.21.15.69:0/1798001119 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36183f0c0 con 0x55a361828b20 2024-08-02T17:58:15.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.956+0000 7fe410a2e640 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828b20 0x55a36184af40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe40c072fe0 tx=0x7fe408004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7c5eb3de6a300a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:15.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.956+0000 7fe3f6ffd640 1 -- 172.21.15.69:0/1798001119 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408003cc0 con 0x55a361828b20 2024-08-02T17:58:15.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.956+0000 7fe3f6ffd640 1 -- 172.21.15.69:0/1798001119 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe408009e30 con 0x55a361828b20 2024-08-02T17:58:15.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.956+0000 7fe3f6ffd640 1 -- 172.21.15.69:0/1798001119 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe40801e430 con 0x55a361828b20 2024-08-02T17:58:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.957+0000 7fe415891740 1 -- 172.21.15.69:0/1798001119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828b20 msgr2=0x55a36184af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.957+0000 7fe415891740 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828b20 0x55a36184af40 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe40c072fe0 tx=0x7fe408004410 comp rx=0 tx=0).stop 2024-08-02T17:58:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.958+0000 7fe415891740 1 -- 172.21.15.69:0/1798001119 shutdown_connections 2024-08-02T17:58:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.958+0000 7fe415891740 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a361828b20 0x55a36184af40 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.958+0000 7fe415891740 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a3618333e0 0x55a36184b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.958+0000 7fe415891740 1 --2- 172.21.15.69:0/1798001119 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 0x55a361830c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.958+0000 7fe415891740 1 -- 172.21.15.69:0/1798001119 >> 172.21.15.69:0/1798001119 conn(0x55a361824790 msgr2=0x55a3618323d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:15.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.963+0000 7fe415891740 1 -- 172.21.15.69:0/1798001119 shutdown_connections 2024-08-02T17:58:15.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.964+0000 7fe415891740 1 -- 172.21.15.69:0/1798001119 wait complete. 2024-08-02T17:58:15.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.964+0000 7fe415891740 1 -- start start 2024-08-02T17:58:15.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.964+0000 7fe415891740 1 Processor -- start 2024-08-02T17:58:15.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.966+0000 7fe415891740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 0x55a361836f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe415891740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361837490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe415891740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe415891740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1b3d40 con 0x55a3618333e0 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe415891740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1b51b0 con 0x55a361828b20 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe415891740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe40c1b4410 con 0x55a361828200 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe410a2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361837490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe415891740 10 client.0 _ll_register_callbacks cb 0x7fe3fc0d7550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe410a2e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361837490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40536/0 (socket says 172.21.15.69:40536) 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe410a2e640 1 -- 172.21.15.69:0/2829985384 learned_addr learned my addr 172.21.15.69:0/2829985384 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe411a30640 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 0x55a361836f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe411a30640 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe410a2e640 1 -- 172.21.15.69:0/2829985384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 msgr2=0x55a361836f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe410a2e640 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 0x55a361836f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe410a2e640 1 -- 172.21.15.69:0/2829985384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 msgr2=0x55a3618379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:15.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe410a2e640 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:15.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.967+0000 7fe410a2e640 1 -- 172.21.15.69:0/2829985384 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3618acf10 con 0x55a361828b20 2024-08-02T17:58:15.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.968+0000 7fe410a2e640 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361837490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55a361828f90 tx=0x7fe408004d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5073df1b8caae992 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:15.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.971+0000 7fe3f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:58:15.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.971+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/2829985384 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe408004800 con 0x55a361828b20 2024-08-02T17:58:15.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.971+0000 7fe415891740 1 -- 172.21.15.69:0/2829985384 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe408009e30 con 0x55a361828b20 2024-08-02T17:58:15.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.971+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/2829985384 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fe408005020 con 0x55a361828b20 2024-08-02T17:58:15.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.971+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/2829985384 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe408039050 con 0x55a361828b20 2024-08-02T17:58:15.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.974+0000 7fe415891740 10 client.27758 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:15.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.974+0000 7fe415891740 1 -- 172.21.15.69:0/2829985384 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a3618ae200 con 0x55a361828b20 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.983+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/2829985384 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe40801e430 con 0x55a361828b20 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.983+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/2829985384 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe408029480 con 0x55a361828b20 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.984+0000 7fe415891740 10 client.27758 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.984+0000 7fe415891740 10 client.27758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.984+0000 7fe415891740 20 client.27758 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.984+0000 7fe415891740 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3618ae800 0x55a3618b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.984+0000 7fe415891740 4 client.27758 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.984+0000 7fe415891740 1 -- 172.21.15.69:0/2829985384 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fe408009e30 con 0x55a3618ae800 2024-08-02T17:58:15.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.984+0000 7fe411a30640 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3618ae800 0x55a3618b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:15.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.989+0000 7fe411a30640 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3618ae800 0x55a3618b0c20 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4f953ee663223a server_cookie=91d0e8ab8d756530 in_seq=0 out_seq=0 2024-08-02T17:58:15.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.991+0000 7fe3f4ff9640 10 client.27758 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:15.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.991+0000 7fe3f4ff9640 1 -- 172.21.15.69:0/2829985384 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fe408009e30 con 0x55a3618ae800 2024-08-02T17:58:15.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.991+0000 7fe3f4ff9640 10 client.27758 handle_command_reply: tid=0 2024-08-02T17:58:15.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.991+0000 7fe415891740 1 client.27758 shutdown 2024-08-02T17:58:15.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.994+0000 7fe415891740 1 -- 172.21.15.69:0/2829985384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 msgr2=0x55a361837490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:15.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.994+0000 7fe415891740 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361837490 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55a361828f90 tx=0x7fe408004d90 comp rx=0 tx=0).stop 2024-08-02T17:58:16.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.999+0000 7fe415891740 1 -- 172.21.15.69:0/2829985384 shutdown_connections 2024-08-02T17:58:16.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.999+0000 7fe415891740 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a3618333e0 0x55a3618379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:16.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.999+0000 7fe415891740 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a3618ae800 0x55a3618b0c20 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:16.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.999+0000 7fe415891740 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a361828b20 0x55a361837490 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:16.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.999+0000 7fe415891740 1 --2- 172.21.15.69:0/2829985384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a361828200 0x55a361836f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:16.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.999+0000 7fe3f4ff9640 0 client.27758 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:16.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:15.999+0000 7fe415891740 1 -- 172.21.15.69:0/2829985384 >> 172.21.15.69:0/2829985384 conn(0x55a361824790 msgr2=0x55a3618230d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:16.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.003+0000 7fe415891740 1 -- 172.21.15.69:0/2829985384 shutdown_connections 2024-08-02T17:58:16.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.003+0000 7fe415891740 1 -- 172.21.15.69:0/2829985384 wait complete. 2024-08-02T17:58:16.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.005+0000 7fe415891740 20 client.27758 trim_cache size 0 max 16384 2024-08-02T17:58:16.008 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:58:16.008 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:58:16.008 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "7ced13ed-b1da-47b0-adb0-73abab5d30ef", 2024-08-02T17:58:16.008 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:58:16.008 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.007+0000 7fe413493640 1 -- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3f80a2cf0 msgr2=0x7fe3f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.007+0000 7fe413493640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3f80a2cf0 0x7fe3f80a51b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55a3618255f0 tx=0x7fe3fc0d7680 comp rx=0 tx=0).stop 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.007+0000 7fe413493640 1 -- 172.21.15.69:0/3194741309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe40c108f20 msgr2=0x7fe40c1a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.007+0000 7fe413493640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe40c108f20 0x7fe40c1a6660 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc009f40 tx=0x7fe3fc00ee80 comp rx=0 tx=0).stop 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.007+0000 7fe413493640 1 -- 172.21.15.69:0/3194741309 shutdown_connections 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.007+0000 7fe413493640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe40c109840 0x7fe40c1a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.007+0000 7fe413493640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe3f80a2cf0 0x7fe3f80a51b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.008+0000 7fe413493640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe40c1147e0 0x7fe40c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.008+0000 7fe413493640 1 --2- 172.21.15.69:0/3194741309 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe40c108f20 0x7fe40c1a6660 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:16.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.008+0000 7fe413493640 1 -- 172.21.15.69:0/3194741309 >> 172.21.15.69:0/3194741309 conn(0x7fe40c06de70 msgr2=0x7fe40c070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:16.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.012+0000 7fe413493640 1 -- 172.21.15.69:0/3194741309 shutdown_connections 2024-08-02T17:58:16.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:16.012+0000 7fe413493640 1 -- 172.21.15.69:0/3194741309 wait complete. 2024-08-02T17:58:16.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:16 smithi069 ceph-mon[32746]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.0 MiB/s wr, 491 op/s 2024-08-02T17:58:16.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:16 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:58:16.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:16 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:58:16.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:16 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:58:16.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:16 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:58:16.969 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:58:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:16 smithi083 ceph-mon[37031]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.0 MiB/s wr, 491 op/s 2024-08-02T17:58:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:16 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:58:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:16 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:58:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:16 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:58:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:16 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:58:17.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:16 smithi077 ceph-mon[37108]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 6.0 MiB/s wr, 491 op/s 2024-08-02T17:58:17.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:16 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:58:17.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:16 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:58:17.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:16 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:58:17.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:16 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:58:17.473 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:18.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 -- 172.21.15.69:0/2482082812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1078b0 msgr2=0x7fdc6c107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 --2- 172.21.15.69:0/2482082812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1078b0 0x7fdc6c107c90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009930 tx=0x7fdc5c02a420 comp rx=0 tx=0).stop 2024-08-02T17:58:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 -- 172.21.15.69:0/2482082812 shutdown_connections 2024-08-02T17:58:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 --2- 172.21.15.69:0/2482082812 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc6c102250 0x7fdc6c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 --2- 172.21.15.69:0/2482082812 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc6c1018b0 0x7fdc6c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 --2- 172.21.15.69:0/2482082812 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c1078b0 0x7fdc6c107c90 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 -- 172.21.15.69:0/2482082812 >> 172.21.15.69:0/2482082812 conn(0x7fdc6c0fd610 msgr2=0x7fdc6c0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 -- 172.21.15.69:0/2482082812 shutdown_connections 2024-08-02T17:58:18.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.066+0000 7fdc715b1640 1 -- 172.21.15.69:0/2482082812 wait complete. 2024-08-02T17:58:18.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc715b1640 1 Processor -- start 2024-08-02T17:58:18.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc715b1640 1 -- start start 2024-08-02T17:58:18.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc715b1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc6c1018b0 0x7fdc6c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc715b1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c102250 0x7fdc6c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc715b1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc6c1078b0 0x7fdc6c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc715b1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c111cb0 con 0x7fdc6c102250 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc715b1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c111b30 con 0x7fdc6c1018b0 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc715b1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c111e30 con 0x7fdc6c1078b0 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc63fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c102250 0x7fdc6c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc63fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c102250 0x7fdc6c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56870/0 (socket says 172.21.15.69:56870) 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc63fff640 1 -- 172.21.15.69:0/1260093660 learned_addr learned my addr 172.21.15.69:0/1260093660 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc6affd640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc6c1018b0 0x7fdc6c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc6c1078b0 0x7fdc6c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc63fff640 1 -- 172.21.15.69:0/1260093660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc6c1078b0 msgr2=0x7fdc6c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc63fff640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc6c1078b0 0x7fdc6c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc63fff640 1 -- 172.21.15.69:0/1260093660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc6c1018b0 msgr2=0x7fdc6c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc63fff640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc6c1018b0 0x7fdc6c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.067+0000 7fdc63fff640 1 -- 172.21.15.69:0/1260093660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6c19d140 con 0x7fdc6c102250 2024-08-02T17:58:18.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.068+0000 7fdc63fff640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c102250 0x7fdc6c1986d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400e9f0 tx=0x7fdc5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:18.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.068+0000 7fdc68ff9640 1 -- 172.21.15.69:0/1260093660 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400cd80 con 0x7fdc6c102250 2024-08-02T17:58:18.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.068+0000 7fdc68ff9640 1 -- 172.21.15.69:0/1260093660 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc54010910 con 0x7fdc6c102250 2024-08-02T17:58:18.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.068+0000 7fdc68ff9640 1 -- 172.21.15.69:0/1260093660 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54018990 con 0x7fdc6c102250 2024-08-02T17:58:18.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.068+0000 7fdc715b1640 1 -- 172.21.15.69:0/1260093660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6c19d430 con 0x7fdc6c102250 2024-08-02T17:58:18.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.068+0000 7fdc715b1640 1 -- 172.21.15.69:0/1260093660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc6c074150 con 0x7fdc6c102250 2024-08-02T17:58:18.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.071+0000 7fdc68ff9640 1 -- 172.21.15.69:0/1260093660 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdc54010ac0 con 0x7fdc6c102250 2024-08-02T17:58:18.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.071+0000 7fdc739af740 1 Processor -- start 2024-08-02T17:58:18.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.071+0000 7fdc739af740 1 -- start start 2024-08-02T17:58:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.072+0000 7fdc68ff9640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdc580a2d70 0x7fdc580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.073+0000 7fdc68ff9640 1 -- 172.21.15.69:0/1260093660 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fdc540cfad0 con 0x7fdc6c102250 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.073+0000 7fdc739af740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.073+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdc580a2d70 0x7fdc580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.073+0000 7fdc739af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 0x55fe009aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.073+0000 7fdc739af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009ad3f0 0x55fe009af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.073+0000 7fdc739af740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fe00907980 con 0x55fe009a2210 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.073+0000 7fdc739af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe0089fa70 con 0x55fe009ad3f0 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.073+0000 7fdc739af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe00903940 con 0x55fe009a2b30 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56880/0 (socket says 172.21.15.69:56880) 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 -- 172.21.15.69:0/2834647488 learned_addr learned my addr 172.21.15.69:0/2834647488 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 0x55fe009aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdc580a2d70 0x7fdc580a5230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc64005fd0 tx=0x7fdc64009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009ad3f0 0x55fe009af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 -- 172.21.15.69:0/2834647488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 msgr2=0x55fe009aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 0x55fe009aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 -- 172.21.15.69:0/2834647488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009ad3f0 msgr2=0x55fe009af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009ad3f0 0x55fe009af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 -- 172.21.15.69:0/2834647488 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe009026e0 con 0x55fe009a2210 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009ad3f0 0x55fe009af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009a25f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c02a940 tx=0x7fdc5c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744051f6848a943e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.074+0000 7fdc617fa640 1 -- 172.21.15.69:0/2834647488 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c040070 con 0x55fe009a2210 2024-08-02T17:58:18.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc617fa640 1 -- 172.21.15.69:0/2834647488 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c031870 con 0x55fe009a2210 2024-08-02T17:58:18.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc617fa640 1 -- 172.21.15.69:0/2834647488 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c035040 con 0x55fe009a2210 2024-08-02T17:58:18.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc739af740 1 -- 172.21.15.69:0/2834647488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 msgr2=0x55fe009a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc739af740 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009a25f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c02a940 tx=0x7fdc5c005bd0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc739af740 1 -- 172.21.15.69:0/2834647488 shutdown_connections 2024-08-02T17:58:18.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc739af740 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009ad3f0 0x55fe009af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc739af740 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 0x55fe009aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc739af740 1 --2- 172.21.15.69:0/2834647488 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009a25f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc739af740 1 -- 172.21.15.69:0/2834647488 >> 172.21.15.69:0/2834647488 conn(0x55fe0099e7a0 msgr2=0x55fe0099eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:18.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.075+0000 7fdc739af740 1 -- 172.21.15.69:0/2834647488 shutdown_connections 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.076+0000 7fdc739af740 1 -- 172.21.15.69:0/2834647488 wait complete. 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.076+0000 7fdc739af740 1 -- start start 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.076+0000 7fdc739af740 1 Processor -- start 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.076+0000 7fdc739af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2210 0x55fe009aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.076+0000 7fdc739af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc739af740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc739af740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fe009affc0 con 0x55fe009ad3f0 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc739af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fe008c2aa0 con 0x55fe009a2b30 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc739af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fe009b0140 con 0x55fe009a2210 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2210 0x55fe009aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2210 0x55fe009aa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:39992/0 (socket says 172.21.15.69:39992) 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 -- 172.21.15.69:0/2459946283 learned_addr learned my addr 172.21.15.69:0/2459946283 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:18.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 -- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 msgr2=0x55fe009aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 -- 172.21.15.69:0/2459946283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 msgr2=0x55fe009aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc739af740 10 client.0 _ll_register_callbacks cb 0x55fe008b1e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 -- 172.21.15.69:0/2459946283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe008a6650 con 0x55fe009a2210 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc739af740 1 -- 172.21.15.69:0/2459946283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c031870 con 0x55fe009a2210 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2210 0x55fe009aa220 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009a50 tx=0x7fdc5c02a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5958602b0cc32fac server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc527fc640 10 client.27577 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc527fc640 1 -- 172.21.15.69:0/2459946283 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c04c020 con 0x55fe009a2210 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.077+0000 7fdc527fc640 1 -- 172.21.15.69:0/2459946283 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fdc5c004980 con 0x55fe009a2210 2024-08-02T17:58:18.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc527fc640 1 -- 172.21.15.69:0/2459946283 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c040070 con 0x55fe009a2210 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc739af740 10 client.27577 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc739af740 1 -- 172.21.15.69:0/2459946283 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55fe006793c0 con 0x55fe009a2210 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc527fc640 1 -- 172.21.15.69:0/2459946283 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdc5c004ba0 con 0x55fe009a2210 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc739af740 10 client.27577 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc739af740 10 client.27577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc739af740 20 client.27577 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc739af740 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe009bca80 0x55fe009bee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc739af740 4 client.27577 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc739af740 1 -- 172.21.15.69:0/2459946283 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe00914ef0 con 0x55fe009bca80 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.078+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe009bca80 0x55fe009bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc527fc640 10 client.27577 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe009bca80 0x55fe009bee70 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c32dcbd8daa1c09 server_cookie=99b1bc6ae0cda120 in_seq=0 out_seq=0 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc527fc640 1 -- 172.21.15.69:0/2459946283 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe00914ef0 con 0x55fe009bca80 2024-08-02T17:58:18.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc527fc640 10 client.27577 handle_command_reply: tid=0 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 client.27577 shutdown 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 -- 172.21.15.69:0/2459946283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2210 msgr2=0x55fe009aa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2210 0x55fe009aa220 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009a50 tx=0x7fdc5c02a420 comp rx=0 tx=0).stop 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 -- 172.21.15.69:0/2459946283 shutdown_connections 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe009bca80 0x55fe009bee70 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 --2- 172.21.15.69:0/2459946283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2210 0x55fe009aa220 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc739af740 1 -- 172.21.15.69:0/2459946283 >> 172.21.15.69:0/2459946283 conn(0x55fe0099e7a0 msgr2=0x55fe009a4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.079+0000 7fdc527fc640 0 client.27577 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.080+0000 7fdc739af740 1 -- 172.21.15.69:0/2459946283 shutdown_connections 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.080+0000 7fdc739af740 1 -- 172.21.15.69:0/2459946283 wait complete. 2024-08-02T17:58:18.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.080+0000 7fdc739af740 20 client.27577 trim_cache size 0 max 16384 2024-08-02T17:58:18.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.098+0000 7fdc739af740 1 Processor -- start 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.098+0000 7fdc739af740 1 -- start start 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc739af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2210 0x55fe009a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc739af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 0x55fe009aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc739af740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009c4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc739af740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c1b3970 con 0x55fe009ad3f0 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc739af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c1b1240 con 0x55fe009a2210 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc739af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c1af090 con 0x55fe009a2b30 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56910/0 (socket says 172.21.15.69:56910) 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 -- 172.21.15.69:0/461832375 learned_addr learned my addr 172.21.15.69:0/461832375 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6affd640 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2210 0x55fe009a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc63fff640 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 0x55fe009aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 -- 172.21.15.69:0/461832375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 msgr2=0x55fe009aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:18.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 0x55fe009aaaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 -- 172.21.15.69:0/461832375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2210 msgr2=0x55fe009a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:18.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2210 0x55fe009a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 -- 172.21.15.69:0/461832375 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe0099f930 con 0x55fe009ad3f0 2024-08-02T17:58:18.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.099+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009c4410 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fdc64003c20 tx=0x7fdc64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86326f0719670238 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:18.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.100+0000 7fdc60ff9640 1 -- 172.21.15.69:0/461832375 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64028630 con 0x55fe009ad3f0 2024-08-02T17:58:18.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.100+0000 7fdc60ff9640 1 -- 172.21.15.69:0/461832375 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc64003e50 con 0x55fe009ad3f0 2024-08-02T17:58:18.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.100+0000 7fdc60ff9640 1 -- 172.21.15.69:0/461832375 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc64027920 con 0x55fe009ad3f0 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.100+0000 7fdc739af740 1 -- 172.21.15.69:0/461832375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 msgr2=0x55fe009c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.100+0000 7fdc739af740 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009c4410 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fdc64003c20 tx=0x7fdc64014000 comp rx=0 tx=0).stop 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 -- 172.21.15.69:0/461832375 shutdown_connections 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009ad3f0 0x55fe009c4410 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009a2b30 0x55fe009aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 --2- 172.21.15.69:0/461832375 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2210 0x55fe009a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 -- 172.21.15.69:0/461832375 >> 172.21.15.69:0/461832375 conn(0x55fe0099e7a0 msgr2=0x55fe009ac190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 -- 172.21.15.69:0/461832375 shutdown_connections 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 -- 172.21.15.69:0/461832375 wait complete. 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 -- start start 2024-08-02T17:58:18.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.101+0000 7fdc739af740 1 Processor -- start 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc739af740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc739af740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009a6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc739af740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009ad3f0 0x55fe009a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc739af740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c1af700 con 0x55fe009a2210 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc739af740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c1b0b70 con 0x55fe009a2b30 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc739af740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6c1afdd0 con 0x55fe009ad3f0 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc739af740 10 client.0 _ll_register_callbacks cb 0x7fdc5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:56924/0 (socket says 172.21.15.69:56924) 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 -- 172.21.15.69:0/2625803761 learned_addr learned my addr 172.21.15.69:0/2625803761 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009a6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6b7fe640 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009ad3f0 0x55fe009a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 -- 172.21.15.69:0/2625803761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009ad3f0 msgr2=0x55fe009a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:18.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009ad3f0 0x55fe009a73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 -- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 msgr2=0x55fe009a6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009a6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 -- 172.21.15.69:0/2625803761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe009a7be0 con 0x55fe009a2210 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc6affd640 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009b5a50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005520 tx=0x7fdc5c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81318e25341255c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.102+0000 7fdc527fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc527fc640 1 -- 172.21.15.69:0/2625803761 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c04c040 con 0x55fe009a2210 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc527fc640 1 -- 172.21.15.69:0/2625803761 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fdc5c03c650 con 0x55fe009a2210 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 1 -- 172.21.15.69:0/2625803761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc64003e50 con 0x55fe009a2210 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc527fc640 1 -- 172.21.15.69:0/2625803761 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c068050 con 0x55fe009a2210 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 10 client.18333 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:18.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 1 -- 172.21.15.69:0/2625803761 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fe00a273e0 con 0x55fe009a2210 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc527fc640 1 -- 172.21.15.69:0/2625803761 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc5c05f070 con 0x55fe009a2210 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc527fc640 1 -- 172.21.15.69:0/2625803761 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fdc5c05a540 con 0x55fe009a2210 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 10 client.18333 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 20 client.18333 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe00a27a40 0x55fe00a29e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 4 client.18333 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc739af740 1 -- 172.21.15.69:0/2625803761 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fdc64003e50 con 0x55fe00a27a40 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.103+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe00a27a40 0x55fe00a29e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc63fff640 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe00a27a40 0x55fe00a29e30 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e3bd8387d061d3 server_cookie=56fd4c17c329a6c4 in_seq=0 out_seq=0 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc527fc640 10 client.18333 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc527fc640 1 -- 172.21.15.69:0/2625803761 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7fdc64003e50 con 0x55fe00a27a40 2024-08-02T17:58:18.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc527fc640 10 client.18333 handle_command_reply: tid=0 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 client.18333 shutdown 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 -- 172.21.15.69:0/2625803761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 msgr2=0x55fe009b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009b5a50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005520 tx=0x7fdc5c0058d0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 -- 172.21.15.69:0/2625803761 shutdown_connections 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fe00a27a40 0x55fe00a29e30 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fe009ad3f0 0x55fe009a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fe009a2b30 0x55fe009a6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 --2- 172.21.15.69:0/2625803761 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fe009a2210 0x55fe009b5a50 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 -- 172.21.15.69:0/2625803761 >> 172.21.15.69:0/2625803761 conn(0x55fe0099e7a0 msgr2=0x55fe009ab860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc527fc640 0 client.18333 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 -- 172.21.15.69:0/2625803761 shutdown_connections 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 1 -- 172.21.15.69:0/2625803761 wait complete. 2024-08-02T17:58:18.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.104+0000 7fdc739af740 20 client.18333 trim_cache size 0 max 16384 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: "7ced13ed-b1da-47b0-adb0-73abab5d30ef": { 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "7ced13ed-b1da-47b0-adb0-73abab5d30ef", 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:18.113 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:58:18.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 -- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdc580a2d70 msgr2=0x7fdc580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdc580a2d70 0x7fdc580a5230 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdc64005fd0 tx=0x7fdc64009500 comp rx=0 tx=0).stop 2024-08-02T17:58:18.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 -- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c102250 msgr2=0x7fdc6c1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c102250 0x7fdc6c1986d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400e9f0 tx=0x7fdc5400eeb0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 -- 172.21.15.69:0/1260093660 shutdown_connections 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fdc6c102250 0x7fdc6c1986d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdc580a2d70 0x7fdc580a5230 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc6c1078b0 0x7fdc6c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 --2- 172.21.15.69:0/1260093660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fdc6c1018b0 0x7fdc6c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 -- 172.21.15.69:0/1260093660 >> 172.21.15.69:0/1260093660 conn(0x7fdc6c0fd610 msgr2=0x7fdc6c0fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 -- 172.21.15.69:0/1260093660 shutdown_connections 2024-08-02T17:58:18.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:18.105+0000 7fdc715b1640 1 -- 172.21.15.69:0/1260093660 wait complete. 2024-08-02T17:58:18.879 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:18 smithi069 ceph-mon[32746]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.7 MiB/s wr, 517 op/s 2024-08-02T17:58:18.880 INFO:tasks.cephfs.filesystem:scrub status for tag:7ced13ed-b1da-47b0-adb0-73abab5d30ef - {'path': '/', 'tag': '7ced13ed-b1da-47b0-adb0-73abab5d30ef', 'options': 'recursive,force'} 2024-08-02T17:58:19.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:18 smithi083 ceph-mon[37031]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.7 MiB/s wr, 517 op/s 2024-08-02T17:58:19.099 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:18 smithi077 ceph-mon[37108]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 5.7 MiB/s wr, 517 op/s 2024-08-02T17:58:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.916+0000 7ff96100b640 1 -- 172.21.15.69:0/2239478708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff95c11ae70 msgr2=0x7ff95c11d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.916+0000 7ff96100b640 1 --2- 172.21.15.69:0/2239478708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff95c11ae70 0x7ff95c11d260 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff954009c10 tx=0x7ff95402a700 comp rx=0 tx=0).stop 2024-08-02T17:58:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.922+0000 7ff96100b640 1 -- 172.21.15.69:0/2239478708 shutdown_connections 2024-08-02T17:58:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.922+0000 7ff96100b640 1 --2- 172.21.15.69:0/2239478708 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff95c109300 0x7ff95c109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.922+0000 7ff96100b640 1 --2- 172.21.15.69:0/2239478708 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff95c11ae70 0x7ff95c11d260 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.922+0000 7ff96100b640 1 --2- 172.21.15.69:0/2239478708 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c108f20 0x7ff95c10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.922+0000 7ff96100b640 1 -- 172.21.15.69:0/2239478708 >> 172.21.15.69:0/2239478708 conn(0x7ff95c06cb20 msgr2=0x7ff95c06cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.930+0000 7ff96100b640 1 -- 172.21.15.69:0/2239478708 shutdown_connections 2024-08-02T17:58:19.932 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.931+0000 7ff96100b640 1 -- 172.21.15.69:0/2239478708 wait complete. 2024-08-02T17:58:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.932+0000 7ff96100b640 1 Processor -- start 2024-08-02T17:58:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.932+0000 7ff96100b640 1 -- start start 2024-08-02T17:58:19.933 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.932+0000 7ff96100b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c108f20 0x7ff95c11a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.932+0000 7ff96100b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff95c109300 0x7ff95c1134d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.932+0000 7ff96100b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff95c11ae70 0x7ff95c113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.932+0000 7ff96100b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c120a10 con 0x7ff95c109300 2024-08-02T17:58:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.932+0000 7ff96100b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c120890 con 0x7ff95c11ae70 2024-08-02T17:58:19.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.932+0000 7ff96100b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c120b90 con 0x7ff95c108f20 2024-08-02T17:58:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.933+0000 7ff95ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c108f20 0x7ff95c11a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.933+0000 7ff95ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c108f20 0x7ff95c11a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40038/0 (socket says 172.21.15.69:40038) 2024-08-02T17:58:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.933+0000 7ff95ad9d640 1 -- 172.21.15.69:0/421005788 learned_addr learned my addr 172.21.15.69:0/421005788 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:19.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.933+0000 7ff95b59e640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff95c11ae70 0x7ff95c113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.934+0000 7ff95a59c640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff95c109300 0x7ff95c1134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.934+0000 7ff95ad9d640 1 -- 172.21.15.69:0/421005788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff95c11ae70 msgr2=0x7ff95c113a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.934+0000 7ff95ad9d640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff95c11ae70 0x7ff95c113a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.934+0000 7ff95ad9d640 1 -- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff95c109300 msgr2=0x7ff95c1134d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.934+0000 7ff95ad9d640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff95c109300 0x7ff95c1134d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.934+0000 7ff95ad9d640 1 -- 172.21.15.69:0/421005788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff95c113fd0 con 0x7ff95c108f20 2024-08-02T17:58:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.935+0000 7ff95a59c640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff95c109300 0x7ff95c1134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:58:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.935+0000 7ff95ad9d640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c108f20 0x7ff95c11a450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00e9c0 tx=0x7ff94c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.935+0000 7ff943fff640 1 -- 172.21.15.69:0/421005788 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c00ccc0 con 0x7ff95c108f20 2024-08-02T17:58:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.935+0000 7ff943fff640 1 -- 172.21.15.69:0/421005788 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff94c005500 con 0x7ff95c108f20 2024-08-02T17:58:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.935+0000 7ff943fff640 1 -- 172.21.15.69:0/421005788 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94c010430 con 0x7ff95c108f20 2024-08-02T17:58:19.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.936+0000 7ff96100b640 1 -- 172.21.15.69:0/421005788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff95c1142c0 con 0x7ff95c108f20 2024-08-02T17:58:19.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.936+0000 7ff96100b640 1 -- 172.21.15.69:0/421005788 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff95c1182a0 con 0x7ff95c108f20 2024-08-02T17:58:19.941 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.939+0000 7ff963d65740 1 Processor -- start 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.939+0000 7ff963d65740 1 -- start start 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff963d65740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x56087602ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff963d65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 0x560876035430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff963d65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 0x560876037d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff963d65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560875f8e9b0 con 0x56087602b0d0 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff963d65740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560875f36800 con 0x56087602a7b0 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff963d65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560875f8a970 con 0x560876035970 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x56087602ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x56087602ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40608/0 (socket says 172.21.15.69:40608) 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 -- 172.21.15.69:0/3792200163 learned_addr learned my addr 172.21.15.69:0/3792200163 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95a59c640 1 --2- 172.21.15.69:0/3792200163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 0x560876037d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 -- 172.21.15.69:0/3792200163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 msgr2=0x560876037d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:19.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 --2- 172.21.15.69:0/3792200163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 0x560876037d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 -- 172.21.15.69:0/3792200163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 msgr2=0x560876035430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:58:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 --2- 172.21.15.69:0/3792200163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 0x560876035430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 -- 172.21.15.69:0/3792200163 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560875ca2210 con 0x56087602a7b0 2024-08-02T17:58:19.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.940+0000 7ff95b59e640 1 --2- 172.21.15.69:0/3792200163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x56087602ab90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff954009be0 tx=0x7ff954030a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb06967f4eee40ec server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.941+0000 7ff9417fa640 1 -- 172.21.15.69:0/3792200163 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954041030 con 0x56087602a7b0 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.941+0000 7ff943fff640 1 -- 172.21.15.69:0/421005788 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff94c005000 con 0x7ff95c108f20 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.941+0000 7ff9417fa640 1 -- 172.21.15.69:0/3792200163 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff954003970 con 0x56087602a7b0 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.941+0000 7ff9417fa640 1 -- 172.21.15.69:0/3792200163 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954036040 con 0x56087602a7b0 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff943fff640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff9440a2e40 0x7ff9440a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 -- 172.21.15.69:0/3792200163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 msgr2=0x56087602ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 --2- 172.21.15.69:0/3792200163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x56087602ab90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff954009be0 tx=0x7ff954030a70 comp rx=0 tx=0).stop 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 -- 172.21.15.69:0/3792200163 shutdown_connections 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 --2- 172.21.15.69:0/3792200163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 0x560876035430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 --2- 172.21.15.69:0/3792200163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 0x560876037d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 --2- 172.21.15.69:0/3792200163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x56087602ab90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 -- 172.21.15.69:0/3792200163 >> 172.21.15.69:0/3792200163 conn(0x560876026d20 msgr2=0x560876027100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff943fff640 1 -- 172.21.15.69:0/421005788 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7ff94c0cfb40 con 0x7ff95c108f20 2024-08-02T17:58:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 -- 172.21.15.69:0/3792200163 shutdown_connections 2024-08-02T17:58:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.942+0000 7ff963d65740 1 -- 172.21.15.69:0/3792200163 wait complete. 2024-08-02T17:58:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.943+0000 7ff963d65740 1 -- start start 2024-08-02T17:58:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.943+0000 7ff963d65740 1 Processor -- start 2024-08-02T17:58:19.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.943+0000 7ff95ad9d640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff9440a2e40 0x7ff9440a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.943+0000 7ff963d65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x5608760328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.943+0000 7ff963d65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 0x560876032de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.943+0000 7ff95ad9d640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff9440a2e40 0x7ff9440a5300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x560876027b80 tx=0x7ff94c009f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.943+0000 7ff95b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x5608760328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x5608760328a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40054/0 (socket says 172.21.15.69:40054) 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95b59e640 1 -- 172.21.15.69:0/3640088316 learned_addr learned my addr 172.21.15.69:0/3640088316 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff963d65740 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876035970 0x560876033320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560876038540 con 0x56087602b0d0 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560875f4a500 con 0x560876035970 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95a59c640 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 0x560876032de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5608760386c0 con 0x56087602a7b0 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff963d65740 10 client.0 _ll_register_callbacks cb 0x560875ca1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95b59e640 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876035970 0x560876033320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95a59c640 1 -- 172.21.15.69:0/3640088316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 msgr2=0x5608760328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95a59c640 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x5608760328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95a59c640 1 -- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876035970 msgr2=0x560876033320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95a59c640 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876035970 0x560876033320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95a59c640 1 -- 172.21.15.69:0/3640088316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560875d6e620 con 0x56087602b0d0 2024-08-02T17:58:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.944+0000 7ff95b59e640 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x5608760328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.946+0000 7ff95a59c640 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 0x560876032de0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff95c114c90 tx=0x7ff95000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ed09e690971953 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.946+0000 7ff93eff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.948+0000 7ff93eff5640 1 -- 172.21.15.69:0/3640088316 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950018020 con 0x56087602b0d0 2024-08-02T17:58:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.948+0000 7ff93eff5640 1 -- 172.21.15.69:0/3640088316 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7ff950014660 con 0x56087602b0d0 2024-08-02T17:58:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.948+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff954003970 con 0x56087602b0d0 2024-08-02T17:58:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.948+0000 7ff93eff5640 1 -- 172.21.15.69:0/3640088316 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff950023050 con 0x56087602b0d0 2024-08-02T17:58:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff963d65740 10 client.18339 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608760441d0 con 0x56087602b0d0 2024-08-02T17:58:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff93eff5640 1 -- 172.21.15.69:0/3640088316 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff954003970 con 0x56087602b0d0 2024-08-02T17:58:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff93eff5640 1 -- 172.21.15.69:0/3640088316 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff950013950 con 0x56087602b0d0 2024-08-02T17:58:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff963d65740 10 client.18339 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff963d65740 10 client.18339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff963d65740 20 client.18339 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff963d65740 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5608760456e0 0x560876047ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff963d65740 4 client.18339 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560875f9c0e0 con 0x5608760456e0 2024-08-02T17:58:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.949+0000 7ff95b59e640 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5608760456e0 0x560876047ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.950+0000 7ff93eff5640 10 client.18339 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.950+0000 7ff95b59e640 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5608760456e0 0x560876047ad0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb88abf9a5acae1 server_cookie=5427c6b62e2fef87 in_seq=0 out_seq=0 2024-08-02T17:58:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.950+0000 7ff93eff5640 1 -- 172.21.15.69:0/3640088316 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560875f9c0e0 con 0x5608760456e0 2024-08-02T17:58:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.950+0000 7ff93eff5640 10 client.18339 handle_command_reply: tid=0 2024-08-02T17:58:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.950+0000 7ff963d65740 1 client.18339 shutdown 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 msgr2=0x560876032de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 0x560876032de0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff95c114c90 tx=0x7ff95000f5b0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 shutdown_connections 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56087602b0d0 0x560876032de0 secure :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff95c114c90 tx=0x7ff95000f5b0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5608760456e0 0x560876047ad0 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff93eff5640 0 client.18339 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876035970 0x560876033320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 --2- 172.21.15.69:0/3640088316 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x5608760328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 >> 172.21.15.69:0/3640088316 conn(0x560876026d20 msgr2=0x56087602ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 shutdown_connections 2024-08-02T17:58:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 1 -- 172.21.15.69:0/3640088316 wait complete. 2024-08-02T17:58:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.951+0000 7ff963d65740 20 client.18339 trim_cache size 0 max 16384 2024-08-02T17:58:19.956 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 670.0 s, 757.8 tps, lat 137.618 ms stddev 183.942 2024-08-02T17:58:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.969+0000 7ff963d65740 1 Processor -- start 2024-08-02T17:58:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.969+0000 7ff963d65740 1 -- start start 2024-08-02T17:58:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff963d65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x560876034700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff963d65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876035970 0x56087604d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff963d65740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876034c40 0x56087604d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.972 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876035970 0x56087604d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff963d65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1c8ea0 con 0x560876035970 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff963d65740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1c6770 con 0x560876034c40 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x560876034700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff963d65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1c45c0 con 0x56087602a7b0 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x560876034700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:40064/0 (socket says 172.21.15.69:40064) 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95b59e640 1 -- 172.21.15.69:0/3211510460 learned_addr learned my addr 172.21.15.69:0/3211510460 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95b59e640 1 --2- 172.21.15.69:0/3211510460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876034c40 0x56087604d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95a59c640 1 -- 172.21.15.69:0/3211510460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 msgr2=0x560876034700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:19.973 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95a59c640 1 --2- 172.21.15.69:0/3211510460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x560876034700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95a59c640 1 -- 172.21.15.69:0/3211510460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876034c40 msgr2=0x56087604d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95a59c640 1 --2- 172.21.15.69:0/3211510460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876034c40 0x56087604d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95a59c640 1 -- 172.21.15.69:0/3211510460 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560876047e40 con 0x560876035970 2024-08-02T17:58:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.970+0000 7ff95a59c640 1 --2- 172.21.15.69:0/3211510460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876035970 0x56087604d3c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x560876033960 tx=0x7ff950004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e496f0f5d0f30413 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff940ff9640 1 -- 172.21.15.69:0/3211510460 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95002c070 con 0x560876035970 2024-08-02T17:58:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff940ff9640 1 -- 172.21.15.69:0/3211510460 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff950013aa0 con 0x560876035970 2024-08-02T17:58:19.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff940ff9640 1 -- 172.21.15.69:0/3211510460 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff950018070 con 0x560876035970 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff963d65740 1 -- 172.21.15.69:0/3211510460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876035970 msgr2=0x56087604d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff963d65740 1 --2- 172.21.15.69:0/3211510460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876035970 0x56087604d3c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x560876033960 tx=0x7ff950004640 comp rx=0 tx=0).stop 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff963d65740 1 -- 172.21.15.69:0/3211510460 shutdown_connections 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff963d65740 1 --2- 172.21.15.69:0/3211510460 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876035970 0x56087604d3c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff963d65740 1 --2- 172.21.15.69:0/3211510460 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560876034c40 0x56087604d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff963d65740 1 --2- 172.21.15.69:0/3211510460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56087602a7b0 0x560876034700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.971+0000 7ff963d65740 1 -- 172.21.15.69:0/3211510460 >> 172.21.15.69:0/3211510460 conn(0x560876026d20 msgr2=0x560876033ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 -- 172.21.15.69:0/3211510460 shutdown_connections 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 -- 172.21.15.69:0/3211510460 wait complete. 2024-08-02T17:58:19.978 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 -- start start 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 Processor -- start 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x560876039e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876034c40 0x56087603a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 0x5608760af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1c4c30 con 0x560876034c40 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1c60a0 con 0x56087602a7b0 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff963d65740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff95c1c5300 con 0x560876035970 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.972+0000 7ff95b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x560876039e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x560876039e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40642/0 (socket says 172.21.15.69:40642) 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95a59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876034c40 0x56087603a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 -- 172.21.15.69:0/1179743394 learned_addr learned my addr 172.21.15.69:0/1179743394 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff963d65740 10 client.0 _ll_register_callbacks cb 0x7ff95000b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:19.979 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 0x5608760af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 -- 172.21.15.69:0/1179743394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 msgr2=0x5608760af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 0x5608760af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 -- 172.21.15.69:0/1179743394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876034c40 msgr2=0x56087603a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876034c40 0x56087603a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 -- 172.21.15.69:0/1179743394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608760af9a0 con 0x56087602a7b0 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff95b59e640 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x560876039e90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff954000f80 tx=0x7ff95403f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa6a7dcce16025f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff93eff5640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff93eff5640 1 -- 172.21.15.69:0/1179743394 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff954036040 con 0x56087602a7b0 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff963d65740 1 -- 172.21.15.69:0/1179743394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff950013aa0 con 0x56087602a7b0 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff93eff5640 1 -- 172.21.15.69:0/1179743394 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7ff954030c70 con 0x56087602a7b0 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff93eff5640 1 -- 172.21.15.69:0/1179743394 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff954041040 con 0x56087602a7b0 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff963d65740 10 client.27791 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:19.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.973+0000 7ff963d65740 1 -- 172.21.15.69:0/1179743394 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5608760b0c70 con 0x56087602a7b0 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff93eff5640 1 -- 172.21.15.69:0/1179743394 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff950013aa0 con 0x56087602a7b0 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff93eff5640 1 -- 172.21.15.69:0/1179743394 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff95403d460 con 0x56087602a7b0 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff963d65740 10 client.27791 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff963d65740 10 client.27791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff963d65740 20 client.27791 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff963d65740 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5608760b0f00 0x5608760b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff963d65740 4 client.27791 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff963d65740 1 -- 172.21.15.69:0/1179743394 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560875f96430 con 0x5608760b0f00 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff95a59c640 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5608760b0f00 0x5608760b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff93eff5640 10 client.27791 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:19.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:19.974+0000 7ff95a59c640 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5608760b0f00 0x5608760b32f0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b114497c91809c4 server_cookie=9c59a97f93a9eaec in_seq=0 out_seq=0 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff93eff5640 1 -- 172.21.15.69:0/1179743394 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560875f96430 con 0x5608760b0f00 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff93eff5640 10 client.27791 handle_command_reply: tid=0 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 client.27791 shutdown 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 -- 172.21.15.69:0/1179743394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 msgr2=0x560876039e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x560876039e90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff954000f80 tx=0x7ff95403f040 comp rx=0 tx=0).stop 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 -- 172.21.15.69:0/1179743394 shutdown_connections 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560876034c40 0x56087603a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5608760b0f00 0x5608760b32f0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff93eff5640 0 client.27791 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560876035970 0x5608760af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 --2- 172.21.15.69:0/1179743394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56087602a7b0 0x560876039e90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 -- 172.21.15.69:0/1179743394 >> 172.21.15.69:0/1179743394 conn(0x560876026d20 msgr2=0x56087602ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.002+0000 7ff963d65740 1 -- 172.21.15.69:0/1179743394 shutdown_connections 2024-08-02T17:58:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff963d65740 1 -- 172.21.15.69:0/1179743394 wait complete. 2024-08-02T17:58:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff963d65740 20 client.27791 trim_cache size 0 max 16384 2024-08-02T17:58:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:58:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:58:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:58:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:58:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 -- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff9440a2e40 msgr2=0x7ff9440a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff9440a2e40 0x7ff9440a5300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x560876027b80 tx=0x7ff94c009f90 comp rx=0 tx=0).stop 2024-08-02T17:58:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 -- 172.21.15.69:0/421005788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c108f20 msgr2=0x7ff95c11a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c108f20 0x7ff95c11a450 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff94c00e9c0 tx=0x7ff94c00ee80 comp rx=0 tx=0).stop 2024-08-02T17:58:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 -- 172.21.15.69:0/421005788 shutdown_connections 2024-08-02T17:58:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff95c109300 0x7ff95c1134d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff9440a2e40 0x7ff9440a5300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff95c11ae70 0x7ff95c113a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 --2- 172.21.15.69:0/421005788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff95c108f20 0x7ff95c11a450 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 -- 172.21.15.69:0/421005788 >> 172.21.15.69:0/421005788 conn(0x7ff95c06cb20 msgr2=0x7ff95c070650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 -- 172.21.15.69:0/421005788 shutdown_connections 2024-08-02T17:58:20.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:20.003+0000 7ff96100b640 1 -- 172.21.15.69:0/421005788 wait complete. 2024-08-02T17:58:20.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:19 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:58:20.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:19 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:58:20.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:19 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:58:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:20 smithi083 ceph-mon[37031]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-08-02T17:58:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:20 smithi077 ceph-mon[37108]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-08-02T17:58:21.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:20 smithi069 ceph-mon[32746]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 6.7 MiB/s wr, 596 op/s 2024-08-02T17:58:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:22 smithi083 ceph-mon[37031]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.8 MiB/s wr, 514 op/s 2024-08-02T17:58:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:22 smithi077 ceph-mon[37108]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.8 MiB/s wr, 514 op/s 2024-08-02T17:58:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:22 smithi069 ceph-mon[32746]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 5.8 MiB/s wr, 514 op/s 2024-08-02T17:58:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 675.0 s, 769.2 tps, lat 126.826 ms stddev 146.354 2024-08-02T17:58:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:24 smithi083 ceph-mon[37031]: pgmap v374: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 8.9 MiB/s wr, 715 op/s 2024-08-02T17:58:25.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:24 smithi077 ceph-mon[37108]: pgmap v374: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 8.9 MiB/s wr, 715 op/s 2024-08-02T17:58:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:24 smithi069 ceph-mon[32746]: pgmap v374: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 8.9 MiB/s wr, 715 op/s 2024-08-02T17:58:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:26 smithi083 ceph-mon[37031]: pgmap v375: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-08-02T17:58:27.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:26 smithi077 ceph-mon[37108]: pgmap v375: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-08-02T17:58:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:26 smithi069 ceph-mon[32746]: pgmap v375: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 6.8 MiB/s wr, 556 op/s 2024-08-02T17:58:29.025 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:28 smithi083 ceph-mon[37031]: pgmap v376: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.4 MiB/s wr, 622 op/s 2024-08-02T17:58:29.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:28 smithi077 ceph-mon[37108]: pgmap v376: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.4 MiB/s wr, 622 op/s 2024-08-02T17:58:29.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:28 smithi069 ceph-mon[32746]: pgmap v376: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 7.4 MiB/s wr, 622 op/s 2024-08-02T17:58:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 680.0 s, 761.8 tps, lat 133.702 ms stddev 160.201 2024-08-02T17:58:31.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:30 smithi083 ceph-mon[37031]: pgmap v377: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.6 MiB/s wr, 658 op/s 2024-08-02T17:58:31.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:30 smithi077 ceph-mon[37108]: pgmap v377: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.6 MiB/s wr, 658 op/s 2024-08-02T17:58:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:30 smithi069 ceph-mon[32746]: pgmap v377: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 8.6 MiB/s wr, 658 op/s 2024-08-02T17:58:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:32 smithi083 ceph-mon[37031]: pgmap v378: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 466 op/s 2024-08-02T17:58:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:32 smithi077 ceph-mon[37108]: pgmap v378: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 466 op/s 2024-08-02T17:58:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:32 smithi069 ceph-mon[32746]: pgmap v378: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.5 MiB/s wr, 466 op/s 2024-08-02T17:58:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.169+0000 7f0bcf05f640 1 -- 172.21.15.69:0/3386078628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 msgr2=0x7f0bc8116690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.169+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/3386078628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 0x7f0bc8116690 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009900 tx=0x7f0bbc032c30 comp rx=0 tx=0).stop 2024-08-02T17:58:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.169+0000 7f0bcf05f640 1 -- 172.21.15.69:0/3386078628 shutdown_connections 2024-08-02T17:58:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.169+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/3386078628 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc8116bd0 0x7f0bc8118fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.169+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/3386078628 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 0x7f0bc8116690 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.169+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/3386078628 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc8108f20 0x7f0bc8109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.169+0000 7f0bcf05f640 1 -- 172.21.15.69:0/3386078628 >> 172.21.15.69:0/3386078628 conn(0x7f0bc806cb20 msgr2=0x7f0bc806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.172+0000 7f0bcf05f640 1 -- 172.21.15.69:0/3386078628 shutdown_connections 2024-08-02T17:58:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.174+0000 7f0bcf05f640 1 -- 172.21.15.69:0/3386078628 wait complete. 2024-08-02T17:58:33.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.175+0000 7f0bcf05f640 1 Processor -- start 2024-08-02T17:58:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.176+0000 7f0bcf05f640 1 -- start start 2024-08-02T17:58:33.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.176+0000 7f0bcf05f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc8108f20 0x7f0bc81a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.176+0000 7f0bcf05f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 0x7f0bc81a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.177+0000 7f0bcf05f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc8116bd0 0x7f0bc819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.177+0000 7f0bcf05f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc810ec60 con 0x7f0bc8108f20 2024-08-02T17:58:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.177+0000 7f0bcd5fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc8116bd0 0x7f0bc819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.177+0000 7f0bcd5fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc8116bd0 0x7f0bc819e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57326/0 (socket says 172.21.15.69:57326) 2024-08-02T17:58:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.177+0000 7f0bcd5fc640 1 -- 172.21.15.69:0/1332282100 learned_addr learned my addr 172.21.15.69:0/1332282100 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:33.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.177+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 0x7f0bc81a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.177+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc810eae0 con 0x7f0bc8109840 2024-08-02T17:58:33.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.177+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc810ede0 con 0x7f0bc8116bd0 2024-08-02T17:58:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.181+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc8108f20 0x7f0bc81a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.181+0000 7f0bc7fff640 1 -- 172.21.15.69:0/1332282100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc8116bd0 msgr2=0x7f0bc819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.181+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc8116bd0 0x7f0bc819e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.181+0000 7f0bc7fff640 1 -- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc8108f20 msgr2=0x7f0bc81a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.181+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc8108f20 0x7f0bc81a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.181+0000 7f0bc7fff640 1 -- 172.21.15.69:0/1332282100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc819f290 con 0x7f0bc8109840 2024-08-02T17:58:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.182+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 0x7f0bc81a4ac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009850 tx=0x7f0bbc004390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.184+0000 7f0bc5ffb640 1 -- 172.21.15.69:0/1332282100 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc04a020 con 0x7f0bc8109840 2024-08-02T17:58:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.184+0000 7f0bc5ffb640 1 -- 172.21.15.69:0/1332282100 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bbc0054e0 con 0x7f0bc8109840 2024-08-02T17:58:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.184+0000 7f0bc5ffb640 1 -- 172.21.15.69:0/1332282100 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc003b80 con 0x7f0bc8109840 2024-08-02T17:58:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.184+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc819f4c0 con 0x7f0bc8109840 2024-08-02T17:58:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.185+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc81ab320 con 0x7f0bc8109840 2024-08-02T17:58:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.189+0000 7f0bd1db7740 1 Processor -- start 2024-08-02T17:58:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.191+0000 7f0bd1db7740 1 -- start start 2024-08-02T17:58:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.191+0000 7f0bc5ffb640 1 -- 172.21.15.69:0/1332282100 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0bbc04d020 con 0x7f0bc8109840 2024-08-02T17:58:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.198+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6aaf3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.198+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab0f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.198+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab11d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.198+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556a6aa579b0 con 0x556a6aaf3670 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.198+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a6a9ff800 con 0x556a6ab0f980 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.198+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a6aa53970 con 0x556a6aaf3f90 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.199+0000 7f0bc5ffb640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bb00a2e80 0x7f0bb00a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.199+0000 7f0bc5ffb640 1 -- 172.21.15.69:0/1332282100 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f0bbc0f8720 con 0x7f0bc8109840 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.200+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab11d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.200+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab11d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:46942/0 (socket says 172.21.15.69:46942) 2024-08-02T17:58:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.200+0000 7f0bc7fff640 1 -- 172.21.15.69:0/1132493164 learned_addr learned my addr 172.21.15.69:0/1132493164 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.200+0000 7f0bc7fff640 1 -- 172.21.15.69:0/1132493164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 msgr2=0x556a6ab0f440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:58:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.200+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1132493164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab0f440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.200+0000 7f0bc7fff640 1 -- 172.21.15.69:0/1132493164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 msgr2=0x556a6aaf3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.200+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1132493164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6aaf3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.200+0000 7f0bc7fff640 1 -- 172.21.15.69:0/1132493164 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a6a76b210 con 0x556a6ab0f980 2024-08-02T17:58:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.201+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/1132493164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab11d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0c0da0 tx=0x7f0bbc003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0ad8cabe139d9b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.201+0000 7f0bcd5fc640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bb00a2e80 0x7f0bb00a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.205+0000 7f0baeffd640 1 -- 172.21.15.69:0/1132493164 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0dd070 con 0x556a6ab0f980 2024-08-02T17:58:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.205+0000 7f0baeffd640 1 -- 172.21.15.69:0/1132493164 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bbc0c5040 con 0x556a6ab0f980 2024-08-02T17:58:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.206+0000 7f0bd1db7740 1 -- 172.21.15.69:0/1132493164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 msgr2=0x556a6ab11d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.206+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/1132493164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab11d70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0c0da0 tx=0x7f0bbc003680 comp rx=0 tx=0).stop 2024-08-02T17:58:33.211 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.207+0000 7f0bcd5fc640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bb00a2e80 0x7f0bb00a5340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00099e0 tx=0x7f0bc0009a50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.212+0000 7f0bd1db7740 1 -- 172.21.15.69:0/1132493164 shutdown_connections 2024-08-02T17:58:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.212+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/1132493164 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab11d70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.212+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/1132493164 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab0f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.212+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/1132493164 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6aaf3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.212+0000 7f0bd1db7740 1 -- 172.21.15.69:0/1132493164 >> 172.21.15.69:0/1132493164 conn(0x556a6aaefd20 msgr2=0x556a6aaf0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:33.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.221+0000 7f0bd1db7740 1 -- 172.21.15.69:0/1132493164 shutdown_connections 2024-08-02T17:58:33.224 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.223+0000 7f0bd1db7740 1 -- 172.21.15.69:0/1132493164 wait complete. 2024-08-02T17:58:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.223+0000 7f0bd1db7740 1 -- start start 2024-08-02T17:58:33.225 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.223+0000 7f0bd1db7740 1 Processor -- start 2024-08-02T17:58:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.226+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab0c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.226+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 0x556a6ab0ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.226+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6ab0f980 0x556a6ab0d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.226+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556a6ab12310 con 0x556a6aaf3670 2024-08-02T17:58:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.226+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a6aa13500 con 0x556a6aaf3f90 2024-08-02T17:58:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.226+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a6ab12490 con 0x556a6ab0f980 2024-08-02T17:58:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.226+0000 7f0bd1db7740 10 client.0 _ll_register_callbacks cb 0x556a6a76aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:33.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.226+0000 7f0bcd5fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6ab0f980 0x556a6ab0d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.228+0000 7f0bcd5fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6ab0f980 0x556a6ab0d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57348/0 (socket says 172.21.15.69:57348) 2024-08-02T17:58:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.228+0000 7f0bcd5fc640 1 -- 172.21.15.69:0/3294799712 learned_addr learned my addr 172.21.15.69:0/3294799712 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.229+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 0x556a6ab0ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.229+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab0c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.231+0000 7f0bcd5fc640 1 -- 172.21.15.69:0/3294799712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 msgr2=0x556a6ab0ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.231+0000 7f0bcd5fc640 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 0x556a6ab0ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.231+0000 7f0bcd5fc640 1 -- 172.21.15.69:0/3294799712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 msgr2=0x556a6ab0c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:33.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.231+0000 7f0bcd5fc640 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab0c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.235 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.231+0000 7f0bcd5fc640 1 -- 172.21.15.69:0/3294799712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a6a837620 con 0x556a6ab0f980 2024-08-02T17:58:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.235+0000 7f0bcd5fc640 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6ab0f980 0x556a6ab0d360 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0014e50 tx=0x7f0bc0009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0684be0fcfd263e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:33.238 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.236+0000 7f0ba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.238+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3294799712 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0024060 con 0x556a6ab0f980 2024-08-02T17:58:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.238+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3294799712 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f0bc001f600 con 0x556a6ab0f980 2024-08-02T17:58:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.240+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3294799712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bbc0c5040 con 0x556a6ab0f980 2024-08-02T17:58:33.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.241+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3294799712 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bc001f800 con 0x556a6ab0f980 2024-08-02T17:58:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.244+0000 7f0bd1db7740 10 client.27604 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:33.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.244+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3294799712 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556a6ab1e010 con 0x556a6ab0f980 2024-08-02T17:58:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.247+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3294799712 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bbc0c5040 con 0x556a6ab0f980 2024-08-02T17:58:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.247+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3294799712 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0bc001e530 con 0x556a6ab0f980 2024-08-02T17:58:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.247+0000 7f0bd1db7740 10 client.27604 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.247+0000 7f0bd1db7740 10 client.27604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.247+0000 7f0bd1db7740 20 client.27604 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.247+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a6ab1f860 0x556a6ab21ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.247+0000 7f0bd1db7740 4 client.27604 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.247+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3294799712 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a6aa650e0 con 0x556a6ab1f860 2024-08-02T17:58:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.250+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a6ab1f860 0x556a6ab21ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.257 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.256+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a6ab1f860 0x556a6ab21ce0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b87a387dde43b34 server_cookie=390924eb2a89ec61 in_seq=0 out_seq=0 2024-08-02T17:58:33.260 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.259+0000 7f0ba7fff640 10 client.27604 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:33.263 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.261+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3294799712 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a6aa650e0 con 0x556a6ab1f860 2024-08-02T17:58:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.261+0000 7f0ba7fff640 10 client.27604 handle_command_reply: tid=0 2024-08-02T17:58:33.264 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.261+0000 7f0bd1db7740 1 client.27604 shutdown 2024-08-02T17:58:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.266+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3294799712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6ab0f980 msgr2=0x556a6ab0d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.270 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.266+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6ab0f980 0x556a6ab0d360 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0014e50 tx=0x7f0bc0009710 comp rx=0 tx=0).stop 2024-08-02T17:58:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.271+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3294799712 shutdown_connections 2024-08-02T17:58:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.271+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a6ab1f860 0x556a6ab21ce0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.271+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6ab0f980 0x556a6ab0d360 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.271+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 0x556a6ab0ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.271+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3294799712 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab0c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.271+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3294799712 >> 172.21.15.69:0/3294799712 conn(0x556a6aaefd20 msgr2=0x556a6aaf5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:33.272 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.271+0000 7f0ba7fff640 0 client.27604 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:33.278 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.277+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3294799712 shutdown_connections 2024-08-02T17:58:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.281+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3294799712 wait complete. 2024-08-02T17:58:33.282 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.281+0000 7f0bd1db7740 20 client.27604 trim_cache size 0 max 16384 2024-08-02T17:58:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.311+0000 7f0bd1db7740 1 Processor -- start 2024-08-02T17:58:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.311+0000 7f0bd1db7740 1 -- start start 2024-08-02T17:58:33.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3670 0x556a6aaea900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 0x556a6ab0d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bccdfb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3670 0x556a6aaea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bccdfb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3670 0x556a6aaea900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57350/0 (socket says 172.21.15.69:57350) 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6ab0f980 0x556a6ab275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81b7e30 con 0x556a6ab0f980 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81b5700 con 0x556a6aaf3f90 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81b3550 con 0x556a6aaf3670 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.312+0000 7f0bccdfb640 1 -- 172.21.15.69:0/2278433764 learned_addr learned my addr 172.21.15.69:0/2278433764 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/2278433764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 0x556a6ab0d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bccdfb640 1 -- 172.21.15.69:0/2278433764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 msgr2=0x556a6ab0d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/2278433764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 0x556a6ab0d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bccdfb640 1 -- 172.21.15.69:0/2278433764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6ab0f980 msgr2=0x556a6ab275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/2278433764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6ab0f980 0x556a6ab275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bccdfb640 1 -- 172.21.15.69:0/2278433764 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a6ab0b870 con 0x556a6aaf3670 2024-08-02T17:58:33.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/2278433764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3670 0x556a6aaea900 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8000c00 tx=0x7f0bb8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c7424d1086aec6f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:33.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bae7fc640 1 -- 172.21.15.69:0/2278433764 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80140d0 con 0x556a6aaf3670 2024-08-02T17:58:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bae7fc640 1 -- 172.21.15.69:0/2278433764 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb8014270 con 0x556a6aaf3670 2024-08-02T17:58:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.313+0000 7f0bae7fc640 1 -- 172.21.15.69:0/2278433764 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8012a10 con 0x556a6aaf3670 2024-08-02T17:58:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.314+0000 7f0bd1db7740 1 -- 172.21.15.69:0/2278433764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3670 msgr2=0x556a6aaea900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.314+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/2278433764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3670 0x556a6aaea900 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8000c00 tx=0x7f0bb8005f90 comp rx=0 tx=0).stop 2024-08-02T17:58:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.315+0000 7f0bd1db7740 1 -- 172.21.15.69:0/2278433764 shutdown_connections 2024-08-02T17:58:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.315+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/2278433764 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6ab0f980 0x556a6ab275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.315+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/2278433764 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6aaf3f90 0x556a6ab0d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.315+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/2278433764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3670 0x556a6aaea900 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.315+0000 7f0bd1db7740 1 -- 172.21.15.69:0/2278433764 >> 172.21.15.69:0/2278433764 conn(0x556a6aaefd20 msgr2=0x556a6ab0e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.315+0000 7f0bd1db7740 1 -- 172.21.15.69:0/2278433764 shutdown_connections 2024-08-02T17:58:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.315+0000 7f0bd1db7740 1 -- 172.21.15.69:0/2278433764 wait complete. 2024-08-02T17:58:33.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.316+0000 7f0bd1db7740 1 -- start start 2024-08-02T17:58:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.316+0000 7f0bd1db7740 1 Processor -- start 2024-08-02T17:58:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab12c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bd1db7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81b3bc0 con 0x556a6aaf3670 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81b5030 con 0x556a6ab0f980 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bd1db7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc81b4290 con 0x556a6aaf3f90 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bd1db7740 10 client.0 _ll_register_callbacks cb 0x7f0bc000af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bccdfb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab12c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bcd5fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bc7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57362/0 (socket says 172.21.15.69:57362) 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bccdfb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab12c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:39286/0 (socket says 172.21.15.69:39286) 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bc7fff640 1 -- 172.21.15.69:0/3942890917 learned_addr learned my addr 172.21.15.69:0/3942890917 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bc7fff640 1 -- 172.21.15.69:0/3942890917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 msgr2=0x556a6ab885b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:33.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab885b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bc7fff640 1 -- 172.21.15.69:0/3942890917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 msgr2=0x556a6ab12c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab12c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.317+0000 7f0bc7fff640 1 -- 172.21.15.69:0/3942890917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a6ab88b70 con 0x556a6aaf3f90 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bc7fff640 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab131a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0c48d0 tx=0x7f0bbc0c2e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6050e8f3be999bed server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0ba7fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3942890917 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0dd040 con 0x556a6aaf3f90 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3942890917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bb8014270 con 0x556a6aaf3f90 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3942890917 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f0bbc0f5ce0 con 0x556a6aaf3f90 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3942890917 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0bbc014050 con 0x556a6aaf3f90 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 10 client.27610 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3942890917 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556a6ab89e80 con 0x556a6aaf3f90 2024-08-02T17:58:33.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3942890917 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bb8014270 con 0x556a6aaf3f90 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3942890917 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0bbc0cd120 con 0x556a6aaf3f90 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 10 client.27610 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 10 client.27610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 20 client.27610 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a6ab8a460 0x556a6ab8c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 4 client.27610 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3942890917 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a6aa5f430 con 0x556a6ab8a460 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.318+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a6ab8a460 0x556a6ab8c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.319+0000 7f0ba7fff640 10 client.27610 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:33.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.319+0000 7f0bccdfb640 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a6ab8a460 0x556a6ab8c880 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c7f3002927f330 server_cookie=fdb7663a305752cc in_seq=0 out_seq=0 2024-08-02T17:58:33.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.333+0000 7f0ba7fff640 1 -- 172.21.15.69:0/3942890917 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a6aa5f430 con 0x556a6ab8a460 2024-08-02T17:58:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.333+0000 7f0ba7fff640 10 client.27610 handle_command_reply: tid=0 2024-08-02T17:58:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.333+0000 7f0bd1db7740 1 client.27610 shutdown 2024-08-02T17:58:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.333+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3942890917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 msgr2=0x556a6ab131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.333+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab131a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0c48d0 tx=0x7f0bbc0c2e50 comp rx=0 tx=0).stop 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3942890917 shutdown_connections 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a6ab8a460 0x556a6ab8c880 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a6ab0f980 0x556a6ab885b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a6aaf3f90 0x556a6ab131a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 1 --2- 172.21.15.69:0/3942890917 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a6aaf3670 0x556a6ab12c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0ba7fff640 0 client.27610 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3942890917 >> 172.21.15.69:0/3942890917 conn(0x556a6aaefd20 msgr2=0x556a6aaee660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3942890917 shutdown_connections 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 1 -- 172.21.15.69:0/3942890917 wait complete. 2024-08-02T17:58:33.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.334+0000 7f0bd1db7740 20 client.27610 trim_cache size 0 max 16384 2024-08-02T17:58:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:58:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:58:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:58:33.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:58:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bb00a2e80 msgr2=0x7f0bb00a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bb00a2e80 0x7f0bb00a5340 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00099e0 tx=0x7f0bc0009a50 comp rx=0 tx=0).stop 2024-08-02T17:58:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 msgr2=0x7f0bc81a4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 0x7f0bc81a4ac0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc009850 tx=0x7f0bbc004390 comp rx=0 tx=0).stop 2024-08-02T17:58:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 shutdown_connections 2024-08-02T17:58:33.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0bb00a2e80 0x7f0bb00a5340 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0bc8116bd0 0x7f0bc819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0bc8109840 0x7f0bc81a4ac0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 --2- 172.21.15.69:0/1332282100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0bc8108f20 0x7f0bc81a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 >> 172.21.15.69:0/1332282100 conn(0x7f0bc806cb20 msgr2=0x7f0bc81170f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 shutdown_connections 2024-08-02T17:58:33.339 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:33.335+0000 7f0bcf05f640 1 -- 172.21.15.69:0/1332282100 wait complete. 2024-08-02T17:58:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 685.0 s, 889.2 tps, lat 112.175 ms stddev 120.079 2024-08-02T17:58:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:34 smithi083 ceph-mon[37031]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 678 op/s 2024-08-02T17:58:35.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:34 smithi077 ceph-mon[37108]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 678 op/s 2024-08-02T17:58:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:34 smithi069 ceph-mon[32746]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.6 MiB/s wr, 678 op/s 2024-08-02T17:58:37.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:36 smithi083 ceph-mon[37031]: pgmap v380: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 MiB/s wr, 479 op/s 2024-08-02T17:58:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:36 smithi077 ceph-mon[37108]: pgmap v380: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 MiB/s wr, 479 op/s 2024-08-02T17:58:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:36 smithi069 ceph-mon[32746]: pgmap v380: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.5 MiB/s wr, 479 op/s 2024-08-02T17:58:39.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:38 smithi083 ceph-mon[37031]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 519 op/s 2024-08-02T17:58:39.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:38 smithi077 ceph-mon[37108]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 519 op/s 2024-08-02T17:58:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:38 smithi069 ceph-mon[32746]: pgmap v381: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.8 MiB/s wr, 519 op/s 2024-08-02T17:58:39.419 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-08-02T17:58:39.419 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:58:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 690.0 s, 658.0 tps, lat 147.665 ms stddev 178.121 2024-08-02T17:58:40.011 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 -- 172.21.15.69:0/3464779547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41007c0 msgr2=0x7f07b4100c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 --2- 172.21.15.69:0/3464779547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41007c0 0x7f07b4100c20 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f07a4009900 tx=0x7f07a402a440 comp rx=0 tx=0).stop 2024-08-02T17:58:40.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 -- 172.21.15.69:0/3464779547 shutdown_connections 2024-08-02T17:58:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 --2- 172.21.15.69:0/3464779547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41007c0 0x7f07b4100c20 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 --2- 172.21.15.69:0/3464779547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07b4101160 0x7f07b406a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 --2- 172.21.15.69:0/3464779547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b41067c0 0x7f07b4106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 -- 172.21.15.69:0/3464779547 >> 172.21.15.69:0/3464779547 conn(0x7f07b40fc500 msgr2=0x7f07b40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 -- 172.21.15.69:0/3464779547 shutdown_connections 2024-08-02T17:58:40.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.620+0000 7f07b950d640 1 -- 172.21.15.69:0/3464779547 wait complete. 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.621+0000 7f07b950d640 1 Processor -- start 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.621+0000 7f07b950d640 1 -- start start 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b950d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b41007c0 0x7f07b41a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b950d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07b4101160 0x7f07b41a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b950d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41067c0 0x7f07b419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b950d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f07b410fb20 con 0x7f07b41067c0 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b950d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f07b410f9a0 con 0x7f07b4101160 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b950d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07b410fca0 con 0x7f07b41007c0 2024-08-02T17:58:40.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41067c0 0x7f07b419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:40.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41067c0 0x7f07b419e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37320/0 (socket says 172.21.15.69:37320) 2024-08-02T17:58:40.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b37fe640 1 -- 172.21.15.69:0/4033902886 learned_addr learned my addr 172.21.15.69:0/4033902886 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:40.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b37fe640 1 -- 172.21.15.69:0/4033902886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b41007c0 msgr2=0x7f07b41a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:40.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b37fe640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b41007c0 0x7f07b41a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:40.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b37fe640 1 -- 172.21.15.69:0/4033902886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07b4101160 msgr2=0x7f07b41a4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:40.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b37fe640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07b4101160 0x7f07b41a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:40.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.622+0000 7f07b37fe640 1 -- 172.21.15.69:0/4033902886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b419eed0 con 0x7f07b41067c0 2024-08-02T17:58:40.625 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.623+0000 7f07b37fe640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41067c0 0x7f07b419e660 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00eaa0 tx=0x7f07ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:40.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.623+0000 7f079bfff640 1 -- 172.21.15.69:0/4033902886 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00cdb0 con 0x7f07b41067c0 2024-08-02T17:58:40.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.623+0000 7f079bfff640 1 -- 172.21.15.69:0/4033902886 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac005020 con 0x7f07b41067c0 2024-08-02T17:58:40.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.623+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b419f1c0 con 0x7f07b41067c0 2024-08-02T17:58:40.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.623+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b41ab510 con 0x7f07b41067c0 2024-08-02T17:58:40.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.623+0000 7f079bfff640 1 -- 172.21.15.69:0/4033902886 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac0116b0 con 0x7f07b41067c0 2024-08-02T17:58:40.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.624+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4101f00 con 0x7f07b41067c0 2024-08-02T17:58:40.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.627+0000 7f079bfff640 1 -- 172.21.15.69:0/4033902886 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f07ac005560 con 0x7f07b41067c0 2024-08-02T17:58:40.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.628+0000 7f079bfff640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f079c0a2f20 0x7f079c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:40.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.628+0000 7f079bfff640 1 -- 172.21.15.69:0/4033902886 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f07ac0cfe90 con 0x7f07b41067c0 2024-08-02T17:58:40.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.628+0000 7f07b2ffd640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f079c0a2f20 0x7f079c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:40.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.628+0000 7f07b2ffd640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f079c0a2f20 0x7f079c0a53e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009710 tx=0x7f07a8006c90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:40.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:40.631+0000 7f079bfff640 1 -- 172.21.15.69:0/4033902886 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f07ac016030 con 0x7f07b41067c0 2024-08-02T17:58:40.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:40 smithi069 ceph-mon[32746]: pgmap v382: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2024-08-02T17:58:41.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.008+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07b406a4c0 con 0x7f07b41067c0 2024-08-02T17:58:41.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.012+0000 7f079bfff640 1 -- 172.21.15.69:0/4033902886 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f07ac09c760 con 0x7f07b41067c0 2024-08-02T17:58:41.013 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:58:41.014 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:58:41.014 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f079c0a2f20 msgr2=0x7f079c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f079c0a2f20 0x7f079c0a53e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f07a8009710 tx=0x7f07a8006c90 comp rx=0 tx=0).stop 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41067c0 msgr2=0x7f07b419e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41067c0 0x7f07b419e660 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00eaa0 tx=0x7f07ac00ef60 comp rx=0 tx=0).stop 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 shutdown_connections 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f07b41067c0 0x7f07b419e660 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f079c0a2f20 0x7f079c0a53e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f07b4101160 0x7f07b41a4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 --2- 172.21.15.69:0/4033902886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07b41007c0 0x7f07b41a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 >> 172.21.15.69:0/4033902886 conn(0x7f07b40fc500 msgr2=0x7f07b40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:41.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 shutdown_connections 2024-08-02T17:58:41.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:41.015+0000 7f07b950d640 1 -- 172.21.15.69:0/4033902886 wait complete. 2024-08-02T17:58:41.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:40 smithi083 ceph-mon[37031]: pgmap v382: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2024-08-02T17:58:41.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:40 smithi077 ceph-mon[37108]: pgmap v382: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.3 MiB/s wr, 647 op/s 2024-08-02T17:58:41.998 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:41 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/4033902886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:42.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:41 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/4033902886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:42.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:41 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/4033902886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:42.383 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-08-02T17:58:42.383 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-08-02T17:58:42.935 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:42 smithi083 ceph-mon[37031]: pgmap v383: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 509 op/s 2024-08-02T17:58:43.069 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:42 smithi069 ceph-mon[32746]: pgmap v383: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 509 op/s 2024-08-02T17:58:43.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:42 smithi077 ceph-mon[37108]: pgmap v383: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.4 MiB/s wr, 509 op/s 2024-08-02T17:58:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 695.0 s, 428.0 tps, lat 229.946 ms stddev 264.895 2024-08-02T17:58:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:44 smithi083 ceph-mon[37031]: pgmap v384: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 900 op/s 2024-08-02T17:58:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:44 smithi077 ceph-mon[37108]: pgmap v384: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 900 op/s 2024-08-02T17:58:45.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:44 smithi069 ceph-mon[32746]: pgmap v384: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 11 MiB/s wr, 900 op/s 2024-08-02T17:58:46.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.456+0000 7f683e14b640 1 -- 172.21.15.69:0/721973904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 msgr2=0x7f68381001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.456+0000 7f683e14b640 1 --2- 172.21.15.69:0/721973904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 0x7f68381001a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f682c009860 tx=0x7f682c02a340 comp rx=0 tx=0).stop 2024-08-02T17:58:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.458+0000 7f683e14b640 1 -- 172.21.15.69:0/721973904 shutdown_connections 2024-08-02T17:58:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.458+0000 7f683e14b640 1 --2- 172.21.15.69:0/721973904 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68381006e0 0x7f683810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.458+0000 7f683e14b640 1 --2- 172.21.15.69:0/721973904 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 0x7f68381001a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.458+0000 7f683e14b640 1 --2- 172.21.15.69:0/721973904 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6838106820 0x7f6838106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.458+0000 7f683e14b640 1 -- 172.21.15.69:0/721973904 >> 172.21.15.69:0/721973904 conn(0x7f6838076e90 msgr2=0x7f68380fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.458+0000 7f683e14b640 1 -- 172.21.15.69:0/721973904 shutdown_connections 2024-08-02T17:58:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.460+0000 7f683e14b640 1 -- 172.21.15.69:0/721973904 wait complete. 2024-08-02T17:58:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f683e14b640 1 Processor -- start 2024-08-02T17:58:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f683e14b640 1 -- start start 2024-08-02T17:58:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f683e14b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 0x7f6838115ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f683e14b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68381006e0 0x7f6838116410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f683e14b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6838106820 0x7f6838110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f683e14b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6838102b10 con 0x7f68381006e0 2024-08-02T17:58:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f683e14b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6838102990 con 0x7f6838106820 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f683e14b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6838102c90 con 0x7f68380ffd40 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f6837fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6838106820 0x7f6838110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f6837fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6838106820 0x7f6838110fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35170/0 (socket says 172.21.15.69:35170) 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f6837fff640 1 -- 172.21.15.69:0/1122819417 learned_addr learned my addr 172.21.15.69:0/1122819417 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.461+0000 7f68377fe640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 0x7f6838115ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.462+0000 7f68377fe640 1 -- 172.21.15.69:0/1122819417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6838106820 msgr2=0x7f6838110fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.462+0000 7f68377fe640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6838106820 0x7f6838110fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.462+0000 7f68377fe640 1 -- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68381006e0 msgr2=0x7f6838116410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.462+0000 7f68377fe640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68381006e0 0x7f6838116410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.462+0000 7f68377fe640 1 -- 172.21.15.69:0/1122819417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68381118a0 con 0x7f68380ffd40 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.462+0000 7f68377fe640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 0x7f6838115ed0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f682800d6d0 tx=0x7f682800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.463+0000 7f6834ff9640 1 -- 172.21.15.69:0/1122819417 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828017070 con 0x7f68380ffd40 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.463+0000 7f683e14b640 1 -- 172.21.15.69:0/1122819417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6838111b90 con 0x7f68380ffd40 2024-08-02T17:58:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.463+0000 7f683e14b640 1 -- 172.21.15.69:0/1122819417 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683811bdc0 con 0x7f68380ffd40 2024-08-02T17:58:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.464+0000 7f6834ff9640 1 -- 172.21.15.69:0/1122819417 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6828010460 con 0x7f68380ffd40 2024-08-02T17:58:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.464+0000 7f6834ff9640 1 -- 172.21.15.69:0/1122819417 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682800f600 con 0x7f68380ffd40 2024-08-02T17:58:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.467+0000 7f6834ff9640 1 -- 172.21.15.69:0/1122819417 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f682801a070 con 0x7f68380ffd40 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.467+0000 7f6840f78740 1 Processor -- start 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.467+0000 7f6840f78740 1 -- start start 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6840f78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc2986b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6840f78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc2991430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6840f78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 0x55ffc2993d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6840f78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ffc28ea9b0 con 0x55ffc2991970 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6840f78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffc2892800 con 0x55ffc29870d0 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6840f78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ffc28e6970 con 0x55ffc29867b0 2024-08-02T17:58:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6837fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc2986b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6837fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc2986b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55300/0 (socket says 172.21.15.69:55300) 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f6837fff640 1 -- 172.21.15.69:0/2586739097 learned_addr learned my addr 172.21.15.69:0/2586739097 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.468+0000 7f68377fe640 1 --2- 172.21.15.69:0/2586739097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc2991430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f6837fff640 1 -- 172.21.15.69:0/2586739097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 msgr2=0x55ffc2991430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f6837fff640 1 --2- 172.21.15.69:0/2586739097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc2991430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f6837fff640 1 -- 172.21.15.69:0/2586739097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 msgr2=0x55ffc2993d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f6837fff640 1 --2- 172.21.15.69:0/2586739097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 0x55ffc2993d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f6837fff640 1 -- 172.21.15.69:0/2586739097 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc25fe210 con 0x55ffc29867b0 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f6837fff640 1 --2- 172.21.15.69:0/2586739097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc2986b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6830007f00 tx=0x7f683000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=267d5bedc647c5ee server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f681dffb640 1 -- 172.21.15.69:0/2586739097 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830010840 con 0x55ffc29867b0 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f681dffb640 1 -- 172.21.15.69:0/2586739097 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68300183f0 con 0x55ffc29867b0 2024-08-02T17:58:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.469+0000 7f681dffb640 1 -- 172.21.15.69:0/2586739097 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830017600 con 0x55ffc29867b0 2024-08-02T17:58:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.470+0000 7f6834ff9640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68200a2f00 0x7f68200a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.470+0000 7f6840f78740 1 -- 172.21.15.69:0/2586739097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 msgr2=0x55ffc2986b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.470+0000 7f6840f78740 1 --2- 172.21.15.69:0/2586739097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc2986b90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6830007f00 tx=0x7f683000f790 comp rx=0 tx=0).stop 2024-08-02T17:58:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f68377fe640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68200a2f00 0x7f68200a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6834ff9640 1 -- 172.21.15.69:0/1122819417 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f68280d0150 con 0x7f68380ffd40 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6840f78740 1 -- 172.21.15.69:0/2586739097 shutdown_connections 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6840f78740 1 --2- 172.21.15.69:0/2586739097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 0x55ffc2993d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6840f78740 1 --2- 172.21.15.69:0/2586739097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc2991430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6840f78740 1 --2- 172.21.15.69:0/2586739097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc2986b90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6840f78740 1 -- 172.21.15.69:0/2586739097 >> 172.21.15.69:0/2586739097 conn(0x55ffc2982d20 msgr2=0x55ffc2983100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6840f78740 1 -- 172.21.15.69:0/2586739097 shutdown_connections 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6840f78740 1 -- 172.21.15.69:0/2586739097 wait complete. 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f68377fe640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68200a2f00 0x7f68200a53c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55ffc2983b80 tx=0x7f68280d3d90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.471+0000 7f6840f78740 1 -- start start 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6840f78740 1 Processor -- start 2024-08-02T17:58:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6840f78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc298e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6840f78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc298edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6837fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc298e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6837fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc298e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55302/0 (socket says 172.21.15.69:55302) 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6836ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc298edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6836ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc298edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35186/0 (socket says 172.21.15.69:35186) 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6840f78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 0x55ffc298f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6837fff640 1 -- 172.21.15.69:0/1623305313 learned_addr learned my addr 172.21.15.69:0/1623305313 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ffc2994540 con 0x55ffc2991970 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ffc28a6500 con 0x55ffc29870d0 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.472+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ffc29946c0 con 0x55ffc29867b0 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6837fff640 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 0x55ffc298f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6837fff640 1 -- 172.21.15.69:0/1623305313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 msgr2=0x55ffc298edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6837fff640 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc298edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6837fff640 1 -- 172.21.15.69:0/1623305313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 msgr2=0x55ffc298f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6837fff640 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 0x55ffc298f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6837fff640 1 -- 172.21.15.69:0/1623305313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc26ca620 con 0x55ffc29867b0 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6836ffd640 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc298edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6837fff640 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc298e870 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f68300078f0 tx=0x7f6830007880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83d67136d82346f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6816ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6840f78740 10 client.0 _ll_register_callbacks cb 0x55ffc25fda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6816ffd640 1 -- 172.21.15.69:0/1623305313 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6830004e50 con 0x55ffc29867b0 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6816ffd640 1 -- 172.21.15.69:0/1623305313 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f6830003970 con 0x55ffc29867b0 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68300183f0 con 0x55ffc29867b0 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6816ffd640 1 -- 172.21.15.69:0/1623305313 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6830032050 con 0x55ffc29867b0 2024-08-02T17:58:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6840f78740 10 client.27622 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.473+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55ffc299ff10 con 0x55ffc29867b0 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6816ffd640 1 -- 172.21.15.69:0/1623305313 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6830003b70 con 0x55ffc29867b0 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6840f78740 10 client.27622 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6840f78740 10 client.27622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6840f78740 20 client.27622 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6840f78740 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ffc29a12c0 0x55ffc29a36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6840f78740 4 client.27622 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f68300183f0 con 0x55ffc29a12c0 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6836ffd640 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ffc29a12c0 0x55ffc29a36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6816ffd640 10 client.27622 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.474+0000 7f6836ffd640 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ffc29a12c0 0x55ffc29a36b0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8454ca6c241a1a80 server_cookie=209e4bc16e864d8b in_seq=0 out_seq=0 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6816ffd640 1 -- 172.21.15.69:0/1623305313 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f68300183f0 con 0x55ffc29a12c0 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6816ffd640 10 client.27622 handle_command_reply: tid=0 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 client.27622 shutdown 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 msgr2=0x55ffc298e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc298e870 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f68300078f0 tx=0x7f6830007880 comp rx=0 tx=0).stop 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 shutdown_connections 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc2991970 0x55ffc298f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ffc29a12c0 0x55ffc29a36b0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc298edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6816ffd640 0 client.27622 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 --2- 172.21.15.69:0/1623305313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29867b0 0x55ffc298e870 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.475+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 >> 172.21.15.69:0/1623305313 conn(0x55ffc2982d20 msgr2=0x55ffc2988a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.476+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 shutdown_connections 2024-08-02T17:58:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.476+0000 7f6840f78740 1 -- 172.21.15.69:0/1623305313 wait complete. 2024-08-02T17:58:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.476+0000 7f6840f78740 20 client.27622 trim_cache size 0 max 16384 2024-08-02T17:58:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.503+0000 7f6840f78740 1 Processor -- start 2024-08-02T17:58:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.504+0000 7f6840f78740 1 -- start start 2024-08-02T17:58:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.504+0000 7f6840f78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 0x55ffc29a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.504+0000 7f6840f78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc29a9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.504+0000 7f6840f78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc2991970 0x55ffc29999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.504+0000 7f6840f78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f68240128b0 con 0x55ffc29867b0 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.504+0000 7f6840f78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6824010180 con 0x55ffc29870d0 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.504+0000 7f6840f78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f682400dfd0 con 0x55ffc2991970 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.507+0000 7f6836ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc29a9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.507+0000 7f6836ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc29a9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35198/0 (socket says 172.21.15.69:35198) 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.507+0000 7f6836ffd640 1 -- 172.21.15.69:0/632796734 learned_addr learned my addr 172.21.15.69:0/632796734 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.507+0000 7f6837fff640 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc2991970 0x55ffc29999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.508+0000 7f6837fff640 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 0x55ffc29a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.511+0000 7f6836ffd640 1 -- 172.21.15.69:0/632796734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc2991970 msgr2=0x55ffc29999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.511+0000 7f6836ffd640 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc2991970 0x55ffc29999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.511+0000 7f6836ffd640 1 -- 172.21.15.69:0/632796734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 msgr2=0x55ffc29a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.511+0000 7f6836ffd640 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 0x55ffc29a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.511+0000 7f6836ffd640 1 -- 172.21.15.69:0/632796734 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffc2981450 con 0x55ffc29870d0 2024-08-02T17:58:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.518+0000 7f6836ffd640 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc29a9360 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f682c035040 tx=0x7f682c0304b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bd2082a24e48a63 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.519+0000 7f681d7fa640 1 -- 172.21.15.69:0/632796734 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c037040 con 0x55ffc29870d0 2024-08-02T17:58:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.519+0000 7f681d7fa640 1 -- 172.21.15.69:0/632796734 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c038040 con 0x55ffc29870d0 2024-08-02T17:58:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.519+0000 7f681d7fa640 1 -- 172.21.15.69:0/632796734 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c005650 con 0x55ffc29870d0 2024-08-02T17:58:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.520+0000 7f6840f78740 1 -- 172.21.15.69:0/632796734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 msgr2=0x55ffc29a9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.520+0000 7f6840f78740 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc29a9360 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f682c035040 tx=0x7f682c0304b0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.526+0000 7f6840f78740 1 -- 172.21.15.69:0/632796734 shutdown_connections 2024-08-02T17:58:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.526+0000 7f6840f78740 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc2991970 0x55ffc29999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.526+0000 7f6840f78740 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc29870d0 0x55ffc29a9360 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.526+0000 7f6840f78740 1 --2- 172.21.15.69:0/632796734 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 0x55ffc29a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.526+0000 7f6840f78740 1 -- 172.21.15.69:0/632796734 >> 172.21.15.69:0/632796734 conn(0x55ffc2982d20 msgr2=0x55ffc29907e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:46.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.531+0000 7f6840f78740 1 -- 172.21.15.69:0/632796734 shutdown_connections 2024-08-02T17:58:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.533+0000 7f6840f78740 1 -- 172.21.15.69:0/632796734 wait complete. 2024-08-02T17:58:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.533+0000 7f6840f78740 1 -- start start 2024-08-02T17:58:46.535 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.533+0000 7f6840f78740 1 Processor -- start 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6840f78740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 0x55ffc2997aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6840f78740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29870d0 0x55ffc2997fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6840f78740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc2991970 0x55ffc2998560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6840f78740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f682400e640 con 0x55ffc29867b0 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6840f78740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f682400fab0 con 0x55ffc2991970 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6836ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29870d0 0x55ffc2997fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6836ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29870d0 0x55ffc2997fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55328/0 (socket says 172.21.15.69:55328) 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6836ffd640 1 -- 172.21.15.69:0/596113199 learned_addr learned my addr 172.21.15.69:0/596113199 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6840f78740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f682400ed10 con 0x55ffc29870d0 2024-08-02T17:58:46.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.534+0000 7f6837fff640 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc2991970 0x55ffc2998560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.535+0000 7f6840f78740 10 client.0 _ll_register_callbacks cb 0x7f68280d3c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.535+0000 7f6836ffd640 1 -- 172.21.15.69:0/596113199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc2991970 msgr2=0x55ffc2998560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.535+0000 7f6836ffd640 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc2991970 0x55ffc2998560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.535+0000 7f6836ffd640 1 -- 172.21.15.69:0/596113199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 msgr2=0x55ffc2997aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.535+0000 7f6836ffd640 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 0x55ffc2997aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.538 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.535+0000 7f6836ffd640 1 -- 172.21.15.69:0/596113199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffc2a0b0f0 con 0x55ffc29870d0 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.537+0000 7f6836ffd640 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29870d0 0x55ffc2997fe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ffc28e6990 tx=0x7f682c030e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fae4244dc48a3c09 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6816ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6816ffd640 1 -- 172.21.15.69:0/596113199 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c04f070 con 0x55ffc29870d0 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 1 -- 172.21.15.69:0/596113199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c038040 con 0x55ffc29870d0 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6816ffd640 1 -- 172.21.15.69:0/596113199 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f682c004020 con 0x55ffc29870d0 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6816ffd640 1 -- 172.21.15.69:0/596113199 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f682c037040 con 0x55ffc29870d0 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 10 client.27628 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 1 -- 172.21.15.69:0/596113199 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ffc2a0c3e0 con 0x55ffc29870d0 2024-08-02T17:58:46.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6816ffd640 1 -- 172.21.15.69:0/596113199 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f682c038040 con 0x55ffc29870d0 2024-08-02T17:58:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6816ffd640 1 -- 172.21.15.69:0/596113199 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f682c03e040 con 0x55ffc29870d0 2024-08-02T17:58:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 10 client.27628 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 10 client.27628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 20 client.27628 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ffc29a12c0 0x55ffc2a0ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 4 client.27628 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:58:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.538+0000 7f6840f78740 1 -- 172.21.15.69:0/596113199 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffc28f80e0 con 0x55ffc29a12c0 2024-08-02T17:58:46.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.539+0000 7f6837fff640 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ffc29a12c0 0x55ffc2a0ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.539+0000 7f6837fff640 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ffc29a12c0 0x55ffc2a0ebb0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f6aafb4acc2d863 server_cookie=168169fabaa6bb3 in_seq=0 out_seq=0 2024-08-02T17:58:46.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.539+0000 7f6816ffd640 10 client.27628 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.708+0000 7f6816ffd640 1 -- 172.21.15.69:0/596113199 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffc28f80e0 con 0x55ffc29a12c0 2024-08-02T17:58:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6816ffd640 10 client.27628 handle_command_reply: tid=0 2024-08-02T17:58:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 client.27628 shutdown 2024-08-02T17:58:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 -- 172.21.15.69:0/596113199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29870d0 msgr2=0x55ffc2997fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29870d0 0x55ffc2997fe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55ffc28e6990 tx=0x7f682c030e80 comp rx=0 tx=0).stop 2024-08-02T17:58:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 -- 172.21.15.69:0/596113199 shutdown_connections 2024-08-02T17:58:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ffc29a12c0 0x55ffc2a0ebb0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ffc2991970 0x55ffc2998560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ffc29870d0 0x55ffc2997fe0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 --2- 172.21.15.69:0/596113199 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ffc29867b0 0x55ffc2997aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 -- 172.21.15.69:0/596113199 >> 172.21.15.69:0/596113199 conn(0x55ffc2982d20 msgr2=0x55ffc2981660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6816ffd640 0 client.27628 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 -- 172.21.15.69:0/596113199 shutdown_connections 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 1 -- 172.21.15.69:0/596113199 wait complete. 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.709+0000 7f6840f78740 20 client.27628 trim_cache size 0 max 16384 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:58:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:58:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:58:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:58:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 -- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68200a2f00 msgr2=0x7f68200a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68200a2f00 0x7f68200a53c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55ffc2983b80 tx=0x7f68280d3d90 comp rx=0 tx=0).stop 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 -- 172.21.15.69:0/1122819417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 msgr2=0x7f6838115ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 0x7f6838115ed0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f682800d6d0 tx=0x7f682800db90 comp rx=0 tx=0).stop 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 -- 172.21.15.69:0/1122819417 shutdown_connections 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68381006e0 0x7f6838116410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68200a2f00 0x7f68200a53c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6838106820 0x7f6838110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 --2- 172.21.15.69:0/1122819417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f68380ffd40 0x7f6838115ed0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 -- 172.21.15.69:0/1122819417 >> 172.21.15.69:0/1122819417 conn(0x7f6838076e90 msgr2=0x7f68381045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 -- 172.21.15.69:0/1122819417 shutdown_connections 2024-08-02T17:58:46.714 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:46.710+0000 7f683e14b640 1 -- 172.21.15.69:0/1122819417 wait complete. 2024-08-02T17:58:47.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:46 smithi083 ceph-mon[37031]: pgmap v385: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 734 op/s 2024-08-02T17:58:47.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:46 smithi077 ceph-mon[37108]: pgmap v385: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 734 op/s 2024-08-02T17:58:47.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:46 smithi069 ceph-mon[32746]: pgmap v385: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 7.8 MiB/s wr, 734 op/s 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.500+0000 7fec52b78640 1 -- 172.21.15.69:0/3379580600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109180 msgr2=0x7fec4c10ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.500+0000 7fec52b78640 1 --2- 172.21.15.69:0/3379580600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109180 0x7fec4c10ff80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec44009ca0 tx=0x7fec4402a8f0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.500+0000 7fec52b78640 1 -- 172.21.15.69:0/3379580600 shutdown_connections 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.500+0000 7fec52b78640 1 --2- 172.21.15.69:0/3379580600 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c11ae70 0x7fec4c11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.500+0000 7fec52b78640 1 --2- 172.21.15.69:0/3379580600 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4c109560 0x7fec4c1099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.500+0000 7fec52b78640 1 --2- 172.21.15.69:0/3379580600 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109180 0x7fec4c10ff80 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec44009ca0 tx=0x7fec4402a8f0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.500+0000 7fec52b78640 1 -- 172.21.15.69:0/3379580600 >> 172.21.15.69:0/3379580600 conn(0x7fec4c06de70 msgr2=0x7fec4c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.501+0000 7fec52b78640 1 -- 172.21.15.69:0/3379580600 shutdown_connections 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.501+0000 7fec52b78640 1 -- 172.21.15.69:0/3379580600 wait complete. 2024-08-02T17:58:47.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec52b78640 1 Processor -- start 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec52b78640 1 -- start start 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec52b78640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109560 0x7fec4c1a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec52b78640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c11ae70 0x7fec4c1a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec52b78640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4c1ab990 0x7fec4c1abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec50914640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109560 0x7fec4c1a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec50914640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109560 0x7fec4c1a70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35218/0 (socket says 172.21.15.69:35218) 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec50914640 1 -- 172.21.15.69:0/3893396182 learned_addr learned my addr 172.21.15.69:0/3893396182 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c120b20 con 0x7fec4c11ae70 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.502+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1209a0 con 0x7fec4c109560 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c120ca0 con 0x7fec4c1ab990 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec51115640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4c1ab990 0x7fec4c1abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec50914640 1 -- 172.21.15.69:0/3893396182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4c1ab990 msgr2=0x7fec4c1abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec50914640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4c1ab990 0x7fec4c1abe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec50914640 1 -- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c11ae70 msgr2=0x7fec4c1a7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec50914640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c11ae70 0x7fec4c1a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec50914640 1 -- 172.21.15.69:0/3893396182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c1ac450 con 0x7fec4c109560 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec50914640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109560 0x7fec4c1a70c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec44005e30 tx=0x7fec440308a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec49ffb640 1 -- 172.21.15.69:0/3893396182 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44030ea0 con 0x7fec4c109560 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.503+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c1ac6e0 con 0x7fec4c109560 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.504+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c1b3ec0 con 0x7fec4c109560 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.504+0000 7fec49ffb640 1 -- 172.21.15.69:0/3893396182 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec4402d070 con 0x7fec4c109560 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.504+0000 7fec49ffb640 1 -- 172.21.15.69:0/3893396182 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440046d0 con 0x7fec4c109560 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.507+0000 7fec49ffb640 1 -- 172.21.15.69:0/3893396182 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fec44003970 con 0x7fec4c109560 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.507+0000 7fec54f76740 1 Processor -- start 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.507+0000 7fec54f76740 1 -- start start 2024-08-02T17:58:47.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.507+0000 7fec54f76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea48510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec54f76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea52dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec54f76740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea55700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec54f76740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5556ce9ada70 con 0x5556cea53310 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec54f76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5556ce945780 con 0x5556cea48130 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec54f76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5556ce9a9a30 con 0x5556cea48a50 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec51115640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea48510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec51115640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea48510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35222/0 (socket says 172.21.15.69:35222) 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec51115640 1 -- 172.21.15.69:0/4203295663 learned_addr learned my addr 172.21.15.69:0/4203295663 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec50914640 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea52dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec4bfff640 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea55700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec49ffb640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec340a2df0 0x7fec340a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec51115640 1 -- 172.21.15.69:0/4203295663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 msgr2=0x5556cea52dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec49ffb640 1 -- 172.21.15.69:0/3893396182 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fec440f0b20 con 0x7fec4c109560 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec51115640 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea52dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec51115640 1 -- 172.21.15.69:0/4203295663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 msgr2=0x5556cea55700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec51115640 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea55700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec51115640 1 -- 172.21.15.69:0/4203295663 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ce71fb90 con 0x5556cea48130 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.508+0000 7fec50914640 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.509+0000 7fec4bfff640 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea55700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.509+0000 7fec51115640 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea48510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00bba0 tx=0x7fec3c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c26b1b91022c1a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.509+0000 7fec51115640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec340a2df0 0x7fec340a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.509+0000 7fec32ffd640 1 -- 172.21.15.69:0/4203295663 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c015070 con 0x5556cea48130 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.509+0000 7fec32ffd640 1 -- 172.21.15.69:0/4203295663 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3c01a940 con 0x5556cea48130 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.509+0000 7fec32ffd640 1 -- 172.21.15.69:0/4203295663 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c011c10 con 0x5556cea48130 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 -- 172.21.15.69:0/4203295663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 msgr2=0x5556cea48510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea48510 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00bba0 tx=0x7fec3c00f730 comp rx=0 tx=0).stop 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec51115640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec340a2df0 0x7fec340a52b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fec3c019000 tx=0x7fec3c011e90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 -- 172.21.15.69:0/4203295663 shutdown_connections 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea55700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea52dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 --2- 172.21.15.69:0/4203295663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea48510 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 -- 172.21.15.69:0/4203295663 >> 172.21.15.69:0/4203295663 conn(0x5556cea446c0 msgr2=0x5556cea44aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 -- 172.21.15.69:0/4203295663 shutdown_connections 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 -- 172.21.15.69:0/4203295663 wait complete. 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.510+0000 7fec54f76740 1 -- start start 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.511+0000 7fec54f76740 1 Processor -- start 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.511+0000 7fec54f76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea50170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.511+0000 7fec54f76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.511+0000 7fec54f76740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea50bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec50914640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea50170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec54f76740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5556cea55f30 con 0x5556cea53310 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec54f76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5556ce968990 con 0x5556cea48130 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec54f76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5556cea560b0 con 0x5556cea48a50 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec54f76740 10 client.0 _ll_register_callbacks cb 0x5556ce6bd090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec50914640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea50170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35234/0 (socket says 172.21.15.69:35234) 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec50914640 1 -- 172.21.15.69:0/1165273203 learned_addr learned my addr 172.21.15.69:0/1165273203 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec50914640 1 -- 172.21.15.69:0/1165273203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 msgr2=0x5556cea506b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec50914640 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea506b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec50914640 1 -- 172.21.15.69:0/1165273203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 msgr2=0x5556cea50bf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec50914640 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea50bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.512+0000 7fec50914640 1 -- 172.21.15.69:0/1165273203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ce9676c0 con 0x5556cea48130 2024-08-02T17:58:47.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec50914640 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea50170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fec440b9160 tx=0x7fec440bcad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a251a39cb255146d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec2bfff640 1 -- 172.21.15.69:0/1165273203 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440bd400 con 0x5556cea48130 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec54f76740 1 -- 172.21.15.69:0/1165273203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3c01a940 con 0x5556cea48130 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec2bfff640 1 -- 172.21.15.69:0/1165273203 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fec44004b40 con 0x5556cea48130 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec2bfff640 1 -- 172.21.15.69:0/1165273203 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec440bcd00 con 0x5556cea48130 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec54f76740 10 client.27839 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec54f76740 1 -- 172.21.15.69:0/1165273203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5556cea72b70 con 0x5556cea48130 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.513+0000 7fec2bfff640 1 -- 172.21.15.69:0/1165273203 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3c01a940 con 0x5556cea48130 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec2bfff640 1 -- 172.21.15.69:0/1165273203 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec4404d070 con 0x5556cea48130 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec54f76740 10 client.27839 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec54f76740 10 client.27839 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:58:47.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec54f76740 20 client.27839 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec54f76740 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556cea741f0 0x5556cea766b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec54f76740 4 client.27839 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec54f76740 1 -- 172.21.15.69:0/1165273203 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556ce9bb1a0 con 0x5556cea741f0 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec4bfff640 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556cea741f0 0x5556cea766b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec4bfff640 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556cea741f0 0x5556cea766b0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29f6cf71ed7ffb4c server_cookie=4ddec34c0055ae71 in_seq=0 out_seq=0 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.514+0000 7fec2bfff640 10 client.27839 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec2bfff640 1 -- 172.21.15.69:0/1165273203 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556ce9bb1a0 con 0x5556cea741f0 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec2bfff640 10 client.27839 handle_command_reply: tid=0 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 client.27839 shutdown 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 -- 172.21.15.69:0/1165273203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 msgr2=0x5556cea50170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea50170 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fec440b9160 tx=0x7fec440bcad0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 -- 172.21.15.69:0/1165273203 shutdown_connections 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea50bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556cea741f0 0x5556cea766b0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 --2- 172.21.15.69:0/1165273203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea50170 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec54f76740 1 -- 172.21.15.69:0/1165273203 >> 172.21.15.69:0/1165273203 conn(0x5556cea446c0 msgr2=0x5556cea4a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.515+0000 7fec2bfff640 0 client.27839 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.516+0000 7fec54f76740 1 -- 172.21.15.69:0/1165273203 shutdown_connections 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.516+0000 7fec54f76740 1 -- 172.21.15.69:0/1165273203 wait complete. 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.516+0000 7fec54f76740 20 client.27839 trim_cache size 0 max 16384 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.537+0000 7fec54f76740 1 Processor -- start 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.537+0000 7fec54f76740 1 -- start start 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec54f76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea4a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec54f76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea7bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec54f76740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea7c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec54f76740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec38012960 con 0x5556cea53310 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec54f76740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec38010230 con 0x5556cea48130 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec54f76740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec3800e080 con 0x5556cea48a50 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec50914640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea4a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea7bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea7bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55358/0 (socket says 172.21.15.69:55358) 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.538+0000 7fec4bfff640 1 -- 172.21.15.69:0/995077787 learned_addr learned my addr 172.21.15.69:0/995077787 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.539+0000 7fec51115640 1 --2- 172.21.15.69:0/995077787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea7c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.539+0000 7fec4bfff640 1 -- 172.21.15.69:0/995077787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 msgr2=0x5556cea4a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.539+0000 7fec4bfff640 1 --2- 172.21.15.69:0/995077787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea4a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.539+0000 7fec4bfff640 1 -- 172.21.15.69:0/995077787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 msgr2=0x5556cea7c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.539+0000 7fec4bfff640 1 --2- 172.21.15.69:0/995077787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea7c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.539+0000 7fec4bfff640 1 -- 172.21.15.69:0/995077787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556cea4bb30 con 0x5556cea48a50 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.541+0000 7fec4bfff640 1 --2- 172.21.15.69:0/995077787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea7bf80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5556cea4fa00 tx=0x7fec4000a2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f5c3c8163a17c68 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.543+0000 7fec327fc640 1 -- 172.21.15.69:0/995077787 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40010ce0 con 0x5556cea48a50 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.543+0000 7fec327fc640 1 -- 172.21.15.69:0/995077787 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec40005050 con 0x5556cea48a50 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.543+0000 7fec327fc640 1 -- 172.21.15.69:0/995077787 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40004ab0 con 0x5556cea48a50 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 -- 172.21.15.69:0/995077787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 msgr2=0x5556cea7bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 --2- 172.21.15.69:0/995077787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea7bf80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x5556cea4fa00 tx=0x7fec4000a2d0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 -- 172.21.15.69:0/995077787 shutdown_connections 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 --2- 172.21.15.69:0/995077787 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea7c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 --2- 172.21.15.69:0/995077787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea7bf80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 --2- 172.21.15.69:0/995077787 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea4a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 -- 172.21.15.69:0/995077787 >> 172.21.15.69:0/995077787 conn(0x5556cea446c0 msgr2=0x5556cea49bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 -- 172.21.15.69:0/995077787 shutdown_connections 2024-08-02T17:58:47.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.562+0000 7fec54f76740 1 -- 172.21.15.69:0/995077787 wait complete. 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.563+0000 7fec54f76740 1 -- start start 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.563+0000 7fec54f76740 1 Processor -- start 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.564+0000 7fec54f76740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea57060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.564+0000 7fec54f76740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.564+0000 7fec50914640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea57060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.564+0000 7fec50914640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea57060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35248/0 (socket says 172.21.15.69:35248) 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.564+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.564+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:55372/0 (socket says 172.21.15.69:55372) 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.564+0000 7fec50914640 1 -- 172.21.15.69:0/594382971 learned_addr learned my addr 172.21.15.69:0/594382971 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.564+0000 7fec54f76740 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea57ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fec3800e6f0 con 0x5556cea53310 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fec3800fb60 con 0x5556cea48130 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec3800edc0 con 0x5556cea48a50 2024-08-02T17:58:47.873 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec54f76740 10 client.0 _ll_register_callbacks cb 0x7fec440f5040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec4bfff640 1 -- 172.21.15.69:0/594382971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 msgr2=0x5556cea57060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec4bfff640 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea57060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec4bfff640 1 -- 172.21.15.69:0/594382971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 msgr2=0x5556cea57ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec4bfff640 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea57ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.565+0000 7fec4bfff640 1 -- 172.21.15.69:0/594382971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ceaddf50 con 0x5556cea48a50 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.570+0000 7fec4bfff640 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea575a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5556cea56290 tx=0x7fec40010e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf11d3015a23941d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.570+0000 7fec2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.572+0000 7fec2bfff640 1 -- 172.21.15.69:0/594382971 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40029060 con 0x5556cea48a50 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.572+0000 7fec2bfff640 1 -- 172.21.15.69:0/594382971 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fec400042e0 con 0x5556cea48a50 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.572+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec40005050 con 0x5556cea48a50 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.574+0000 7fec2bfff640 1 -- 172.21.15.69:0/594382971 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec40020040 con 0x5556cea48a50 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.576+0000 7fec54f76740 10 client.27637 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:47.874 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.576+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5556ceadf150 con 0x5556cea48a50 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.579+0000 7fec2bfff640 1 -- 172.21.15.69:0/594382971 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec400047a0 con 0x5556cea48a50 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.579+0000 7fec2bfff640 1 -- 172.21.15.69:0/594382971 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec40023070 con 0x5556cea48a50 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.580+0000 7fec54f76740 10 client.27637 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.580+0000 7fec54f76740 10 client.27637 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.580+0000 7fec54f76740 20 client.27637 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.580+0000 7fec54f76740 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ceadf780 0x5556ceae1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.580+0000 7fec54f76740 4 client.27637 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.580+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x7fec40005050 con 0x5556ceadf780 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.587+0000 7fec50914640 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ceadf780 0x5556ceae1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:47.875 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.595+0000 7fec50914640 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ceadf780 0x5556ceae1b70 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20f59b9935d4414 server_cookie=8c268643b9d9b550 in_seq=0 out_seq=0 2024-08-02T17:58:47.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.595+0000 7fec2bfff640 10 client.27637 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:47.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.922+0000 7fec2bfff640 1 -- 172.21.15.69:0/594382971 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fec40005050 con 0x5556ceadf780 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.922+0000 7fec2bfff640 10 client.27637 handle_command_reply: tid=0 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.922+0000 7fec54f76740 1 client.27637 shutdown 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.922+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 msgr2=0x5556cea575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.922+0000 7fec54f76740 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea575a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x5556cea56290 tx=0x7fec40010e50 comp rx=0 tx=0).stop 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 shutdown_connections 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5556cea53310 0x5556cea57ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5556ceadf780 0x5556ceae1b70 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5556cea48a50 0x5556cea575a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 1 --2- 172.21.15.69:0/594382971 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5556cea48130 0x5556cea57060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec2bfff640 0 client.27637 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 >> 172.21.15.69:0/594382971 conn(0x5556cea446c0 msgr2=0x5556cea43710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:47.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 shutdown_connections 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 1 -- 172.21.15.69:0/594382971 wait complete. 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec54f76740 20 client.27637 trim_cache size 0 max 16384 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 42, 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stdout: "79977741": { 2024-08-02T17:58:47.926 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 42, 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 130.0, 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:58:47.927 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:47.928 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:47.928 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:47.928 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:47.928 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:47.928 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:58:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec340a2df0 msgr2=0x7fec340a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec340a2df0 0x7fec340a52b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fec3c019000 tx=0x7fec3c011e90 comp rx=0 tx=0).stop 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109560 msgr2=0x7fec4c1a70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109560 0x7fec4c1a70c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec44005e30 tx=0x7fec440308a0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 shutdown_connections 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fec4c11ae70 0x7fec4c1a7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fec340a2df0 0x7fec340a52b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec4c1ab990 0x7fec4c1abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 --2- 172.21.15.69:0/3893396182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fec4c109560 0x7fec4c1a70c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 >> 172.21.15.69:0/3893396182 conn(0x7fec4c06de70 msgr2=0x7fec4c11bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 shutdown_connections 2024-08-02T17:58:47.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:47.923+0000 7fec52b78640 1 -- 172.21.15.69:0/3893396182 wait complete. 2024-08-02T17:58:48.577 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '79977741', quiesce took 9.2 seconds. Will release after: 0.8000000000000007 2024-08-02T17:58:48.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:48 smithi069 ceph-mon[32746]: pgmap v386: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 772 op/s 2024-08-02T17:58:48.881 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:58:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:48 smithi083 ceph-mon[37031]: pgmap v386: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 772 op/s 2024-08-02T17:58:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:48 smithi077 ceph-mon[37108]: pgmap v386: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.2 MiB/s wr, 772 op/s 2024-08-02T17:58:49.139 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:49.378 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:58:49.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 -- 172.21.15.69:0/1233122014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba8102250 msgr2=0x7fcba810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 --2- 172.21.15.69:0/1233122014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba8102250 0x7fcba810e780 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009cd0 tx=0x7fcba002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 -- 172.21.15.69:0/1233122014 shutdown_connections 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 --2- 172.21.15.69:0/1233122014 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba8102250 0x7fcba810e780 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 --2- 172.21.15.69:0/1233122014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba81018b0 0x7fcba8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 --2- 172.21.15.69:0/1233122014 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcba81078b0 0x7fcba8107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 -- 172.21.15.69:0/1233122014 >> 172.21.15.69:0/1233122014 conn(0x7fcba80fd5f0 msgr2=0x7fcba80ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 -- 172.21.15.69:0/1233122014 shutdown_connections 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 -- 172.21.15.69:0/1233122014 wait complete. 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.659+0000 7fcbad032640 1 Processor -- start 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcbad032640 1 -- start start 2024-08-02T17:58:49.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcbad032640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba81018b0 0x7fcba81981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcbad032640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcba8102250 0x7fcba8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcbad032640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba81078b0 0x7fcba819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcbad032640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8111cd0 con 0x7fcba81078b0 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcbad032640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8111b50 con 0x7fcba8102250 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcbad032640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8111e50 con 0x7fcba81018b0 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba81078b0 0x7fcba819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba81078b0 0x7fcba819caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33214/0 (socket says 172.21.15.69:33214) 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 -- 172.21.15.69:0/2444276746 learned_addr learned my addr 172.21.15.69:0/2444276746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba659c640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcba8102250 0x7fcba8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba81018b0 0x7fcba81981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 -- 172.21.15.69:0/2444276746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba81018b0 msgr2=0x7fcba81981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba81018b0 0x7fcba81981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 -- 172.21.15.69:0/2444276746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcba8102250 msgr2=0x7fcba8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcba8102250 0x7fcba8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 -- 172.21.15.69:0/2444276746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba819d180 con 0x7fcba81078b0 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.660+0000 7fcba759e640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba81078b0 0x7fcba819caa0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcba0006010 tx=0x7fcba0003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.661+0000 7fcb8ffff640 1 -- 172.21.15.69:0/2444276746 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0004450 con 0x7fcba81078b0 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.661+0000 7fcb8ffff640 1 -- 172.21.15.69:0/2444276746 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba0003e10 con 0x7fcba81078b0 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.661+0000 7fcbad032640 1 -- 172.21.15.69:0/2444276746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba819d410 con 0x7fcba81078b0 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.661+0000 7fcb8ffff640 1 -- 172.21.15.69:0/2444276746 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0046480 con 0x7fcba81078b0 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.661+0000 7fcbad032640 1 -- 172.21.15.69:0/2444276746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81a4d50 con 0x7fcba81078b0 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcb8ffff640 1 -- 172.21.15.69:0/2444276746 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcba00045f0 con 0x7fcba81078b0 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcbaf430740 1 Processor -- start 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcbaf430740 1 -- start start 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc7465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc750eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc7537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcbaf430740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aadc6ab980 con 0x55aadc746210 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcbaf430740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aadc643a70 con 0x55aadc7513f0 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcbaf430740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aadc6a7940 con 0x55aadc746b30 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcba659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc7465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcb8ffff640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcb900a2cf0 0x7fcb900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.663+0000 7fcba659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc7465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33228/0 (socket says 172.21.15.69:33228) 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba659c640 1 -- 172.21.15.69:0/3893028558 learned_addr learned my addr 172.21.15.69:0/3893028558 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc750eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba759e640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcb900a2cf0 0x7fcb900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba659c640 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc7537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcb8ffff640 1 -- 172.21.15.69:0/2444276746 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fcba002e030 con 0x7fcba81078b0 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba659c640 1 -- 172.21.15.69:0/3893028558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 msgr2=0x55aadc750eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba659c640 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc750eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba659c640 1 -- 172.21.15.69:0/3893028558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 msgr2=0x55aadc7537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba659c640 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc7537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba659c640 1 -- 172.21.15.69:0/3893028558 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aadc6a66e0 con 0x55aadc746210 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc750eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba659c640 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc7465f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c013550 tx=0x7fcb9c00a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550df41d39f506df server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcba759e640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcb900a2cf0 0x7fcb900a51b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcba00bcf60 tx=0x7fcba008ce30 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcb8d7fa640 1 -- 172.21.15.69:0/3893028558 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c019030 con 0x55aadc746210 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcb8d7fa640 1 -- 172.21.15.69:0/3893028558 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c005500 con 0x55aadc746210 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.664+0000 7fcb8d7fa640 1 -- 172.21.15.69:0/3893028558 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00f890 con 0x55aadc746210 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 -- 172.21.15.69:0/3893028558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 msgr2=0x55aadc7465f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc7465f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c013550 tx=0x7fcb9c00a760 comp rx=0 tx=0).stop 2024-08-02T17:58:49.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 -- 172.21.15.69:0/3893028558 shutdown_connections 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc7537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc750eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 --2- 172.21.15.69:0/3893028558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc7465f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 -- 172.21.15.69:0/3893028558 >> 172.21.15.69:0/3893028558 conn(0x55aadc7427a0 msgr2=0x55aadc742b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 -- 172.21.15.69:0/3893028558 shutdown_connections 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 -- 172.21.15.69:0/3893028558 wait complete. 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.665+0000 7fcbaf430740 1 -- start start 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcbaf430740 1 Processor -- start 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc74e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc74e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc74ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcbaf430740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55aadc753fc0 con 0x55aadc746210 2024-08-02T17:58:49.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcbaf430740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55aadc666aa0 con 0x55aadc7513f0 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcbaf430740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55aadc754140 con 0x55aadc746b30 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcbaf430740 10 client.0 _ll_register_callbacks cb 0x55aadc655e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc74e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc74e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33240/0 (socket says 172.21.15.69:33240) 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 -- 172.21.15.69:0/1749751931 learned_addr learned my addr 172.21.15.69:0/1749751931 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc74ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc74e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 -- 172.21.15.69:0/1749751931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 msgr2=0x55aadc74e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc74e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 -- 172.21.15.69:0/1749751931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 msgr2=0x55aadc74ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc74ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 -- 172.21.15.69:0/1749751931 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aadc64a650 con 0x55aadc746210 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcba659c640 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc74e230 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00a7d0 tx=0x7fcb9c004df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22627044501fcda server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:49.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcb8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.666+0000 7fcb8aff5640 1 -- 172.21.15.69:0/1749751931 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c019040 con 0x55aadc746210 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcb8aff5640 1 -- 172.21.15.69:0/1749751931 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fcb9c00f830 con 0x55aadc746210 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 1 -- 172.21.15.69:0/1749751931 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9c005500 con 0x55aadc746210 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcb8aff5640 1 -- 172.21.15.69:0/1749751931 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb9c005320 con 0x55aadc746210 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 10 client.18390 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 1 -- 172.21.15.69:0/1749751931 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aadc41d3c0 con 0x55aadc746210 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcb8aff5640 1 -- 172.21.15.69:0/1749751931 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c005500 con 0x55aadc746210 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcb8aff5640 1 -- 172.21.15.69:0/1749751931 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcb9c01e9b0 con 0x55aadc746210 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 10 client.18390 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 10 client.18390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 20 client.18390 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55aadc760dc0 0x55aadc7631b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 4 client.18390 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:49.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcbaf430740 1 -- 172.21.15.69:0/1749751931 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aadc6b8ef0 con 0x55aadc760dc0 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.667+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55aadc760dc0 0x55aadc7631b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.668+0000 7fcb8aff5640 10 client.18390 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.668+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55aadc760dc0 0x55aadc7631b0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cee83866a40bb1 server_cookie=f5868732d015fbba in_seq=0 out_seq=0 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.668+0000 7fcb8aff5640 1 -- 172.21.15.69:0/1749751931 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aadc6b8ef0 con 0x55aadc760dc0 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.668+0000 7fcb8aff5640 10 client.18390 handle_command_reply: tid=0 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.668+0000 7fcbaf430740 1 client.18390 shutdown 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 -- 172.21.15.69:0/1749751931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 msgr2=0x55aadc74e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc74e230 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c00a7d0 tx=0x7fcb9c004df0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 -- 172.21.15.69:0/1749751931 shutdown_connections 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55aadc760dc0 0x55aadc7631b0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc74ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc74e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 --2- 172.21.15.69:0/1749751931 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc74e230 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcb8aff5640 0 client.18390 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:49.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 -- 172.21.15.69:0/1749751931 >> 172.21.15.69:0/1749751931 conn(0x55aadc7427a0 msgr2=0x55aadc748480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:49.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 -- 172.21.15.69:0/1749751931 shutdown_connections 2024-08-02T17:58:49.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 1 -- 172.21.15.69:0/1749751931 wait complete. 2024-08-02T17:58:49.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.669+0000 7fcbaf430740 20 client.18390 trim_cache size 0 max 16384 2024-08-02T17:58:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcbaf430740 1 Processor -- start 2024-08-02T17:58:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcbaf430740 1 -- start start 2024-08-02T17:58:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc748c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc768940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc768e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcbaf430740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b1830 con 0x55aadc746210 2024-08-02T17:58:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcbaf430740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81af100 con 0x55aadc7513f0 2024-08-02T17:58:49.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcbaf430740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81acf50 con 0x55aadc746b30 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc748c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc748c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33242/0 (socket says 172.21.15.69:33242) 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 -- 172.21.15.69:0/3831515947 learned_addr learned my addr 172.21.15.69:0/3831515947 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 -- 172.21.15.69:0/3831515947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 msgr2=0x55aadc768940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 --2- 172.21.15.69:0/3831515947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc768940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 -- 172.21.15.69:0/3831515947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 msgr2=0x55aadc768e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 --2- 172.21.15.69:0/3831515947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc768e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 -- 172.21.15.69:0/3831515947 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aadc743850 con 0x55aadc746210 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcba659c640 1 --2- 172.21.15.69:0/3831515947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc748c70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c001eb0 tx=0x7fcb9c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd492feb4ab07dfd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.686+0000 7fcb8cff9640 1 -- 172.21.15.69:0/3831515947 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c019040 con 0x55aadc746210 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcb8cff9640 1 -- 172.21.15.69:0/3831515947 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c00a800 con 0x55aadc746210 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcb8cff9640 1 -- 172.21.15.69:0/3831515947 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c01e6e0 con 0x55aadc746210 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 -- 172.21.15.69:0/3831515947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 msgr2=0x55aadc748c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.699 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 --2- 172.21.15.69:0/3831515947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc748c70 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c001eb0 tx=0x7fcb9c004c80 comp rx=0 tx=0).stop 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 -- 172.21.15.69:0/3831515947 shutdown_connections 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 --2- 172.21.15.69:0/3831515947 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc7513f0 0x55aadc768e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 --2- 172.21.15.69:0/3831515947 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc768940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 --2- 172.21.15.69:0/3831515947 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc746210 0x55aadc748c70 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 -- 172.21.15.69:0/3831515947 >> 172.21.15.69:0/3831515947 conn(0x55aadc7427a0 msgr2=0x55aadc7501a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 -- 172.21.15.69:0/3831515947 shutdown_connections 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 -- 172.21.15.69:0/3831515947 wait complete. 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.687+0000 7fcbaf430740 1 -- start start 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcbaf430740 1 Processor -- start 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc746210 0x55aadc759570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc759ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcbaf430740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc7513f0 0x55aadc74a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcbaf430740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81ad5c0 con 0x55aadc7513f0 2024-08-02T17:58:49.700 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcbaf430740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81aea30 con 0x55aadc746210 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcbaf430740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81adc90 con 0x55aadc746b30 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcbaf430740 10 client.0 _ll_register_callbacks cb 0x7fcb980093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc7513f0 0x55aadc74a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc7513f0 0x55aadc74a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33252/0 (socket says 172.21.15.69:33252) 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 -- 172.21.15.69:0/251539515 learned_addr learned my addr 172.21.15.69:0/251539515 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc746210 0x55aadc759570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc759ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 -- 172.21.15.69:0/251539515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 msgr2=0x55aadc759ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc759ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 -- 172.21.15.69:0/251539515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc746210 msgr2=0x55aadc759570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc746210 0x55aadc759570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 -- 172.21.15.69:0/251539515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aadc74ae90 con 0x55aadc7513f0 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc759ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:49.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcba659c640 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc7513f0 0x55aadc74a8a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0058c0 tx=0x7fcb9c00ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c092d4464d148441 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.688+0000 7fcb8aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcb8aff5640 1 -- 172.21.15.69:0/251539515 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c019070 con 0x55aadc7513f0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcb8aff5640 1 -- 172.21.15.69:0/251539515 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7fcb9c01e520 con 0x55aadc7513f0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 1 -- 172.21.15.69:0/251539515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9c00a800 con 0x55aadc7513f0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcb8aff5640 1 -- 172.21.15.69:0/251539515 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcb9c01e720 con 0x55aadc7513f0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 10 client.18396 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 1 -- 172.21.15.69:0/251539515 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55aadc7cb200 con 0x55aadc7513f0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcb8aff5640 1 -- 172.21.15.69:0/251539515 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9c00a800 con 0x55aadc7513f0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcb8aff5640 1 -- 172.21.15.69:0/251539515 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcb9c043750 con 0x55aadc7513f0 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 10 client.18396 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 10 client.18396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 20 client.18396 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55aadc7cb7b0 0x55aadc7cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:49.702 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 4 client.18396 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcbaf430740 1 -- 172.21.15.69:0/251539515 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aadc6b3400 con 0x55aadc7cb7b0 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.689+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55aadc7cb7b0 0x55aadc7cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcba6d9d640 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55aadc7cb7b0 0x55aadc7cdba0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fa519c621ea68fe server_cookie=54cb4c300ac1f373 in_seq=0 out_seq=0 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcb8aff5640 10 client.18396 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcb8aff5640 1 -- 172.21.15.69:0/251539515 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aadc6b3400 con 0x55aadc7cb7b0 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcb8aff5640 10 client.18396 handle_command_reply: tid=0 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 client.18396 shutdown 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 -- 172.21.15.69:0/251539515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc7513f0 msgr2=0x55aadc74a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc7513f0 0x55aadc74a8a0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c0058c0 tx=0x7fcb9c00ac60 comp rx=0 tx=0).stop 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 -- 172.21.15.69:0/251539515 shutdown_connections 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55aadc7513f0 0x55aadc74a8a0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55aadc7cb7b0 0x55aadc7cdba0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55aadc746b30 0x55aadc759ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.703 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcb8aff5640 0 client.18396 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 --2- 172.21.15.69:0/251539515 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55aadc746210 0x55aadc759570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 -- 172.21.15.69:0/251539515 >> 172.21.15.69:0/251539515 conn(0x55aadc7427a0 msgr2=0x55aadc75c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 -- 172.21.15.69:0/251539515 shutdown_connections 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 1 -- 172.21.15.69:0/251539515 wait complete. 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.690+0000 7fcbaf430740 20 client.18396 trim_cache size 0 max 16384 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:58:49.704 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:58:49.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 -- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcb900a2cf0 msgr2=0x7fcb900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcb900a2cf0 0x7fcb900a51b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcba00bcf60 tx=0x7fcba008ce30 comp rx=0 tx=0).stop 2024-08-02T17:58:49.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 -- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba81078b0 msgr2=0x7fcba819caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:49.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba81078b0 0x7fcba819caa0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcba0006010 tx=0x7fcba0003c20 comp rx=0 tx=0).stop 2024-08-02T17:58:49.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 -- 172.21.15.69:0/2444276746 shutdown_connections 2024-08-02T17:58:49.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcba81078b0 0x7fcba819caa0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcb900a2cf0 0x7fcb900a51b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcba8102250 0x7fcba8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 --2- 172.21.15.69:0/2444276746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcba81018b0 0x7fcba81981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:49.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 -- 172.21.15.69:0/2444276746 >> 172.21.15.69:0/2444276746 conn(0x7fcba80fd5f0 msgr2=0x7fcba80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:49.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 -- 172.21.15.69:0/2444276746 shutdown_connections 2024-08-02T17:58:49.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:49.691+0000 7fcbad032640 1 -- 172.21.15.69:0/2444276746 wait complete. 2024-08-02T17:58:49.955 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 700.0 s, 282.9 tps, lat 209.854 ms stddev 254.574 2024-08-02T17:58:50.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:58:50.378 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:58:50.464 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:50.942 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:50 smithi069 ceph-mon[32746]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 917 op/s 2024-08-02T17:58:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:50 smithi083 ceph-mon[37031]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 917 op/s 2024-08-02T17:58:51.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:50 smithi077 ceph-mon[37108]: pgmap v387: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 10 MiB/s wr, 917 op/s 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 -- 172.21.15.69:0/1784874573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f4102c20 msgr2=0x7f02f410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 --2- 172.21.15.69:0/1784874573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f4102c20 0x7f02f410d770 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009ca0 tx=0x7f02ec02a950 comp rx=0 tx=0).stop 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 -- 172.21.15.69:0/1784874573 shutdown_connections 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 --2- 172.21.15.69:0/1784874573 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f4102c20 0x7f02f410d770 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 --2- 172.21.15.69:0/1784874573 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f41022a0 0x7f02f41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 --2- 172.21.15.69:0/1784874573 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4100b50 0x7f02f4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 -- 172.21.15.69:0/1784874573 >> 172.21.15.69:0/1784874573 conn(0x7f02f40fc500 msgr2=0x7f02f40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 -- 172.21.15.69:0/1784874573 shutdown_connections 2024-08-02T17:58:51.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.029+0000 7f02faeec640 1 -- 172.21.15.69:0/1784874573 wait complete. 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02faeec640 1 Processor -- start 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02faeec640 1 -- start start 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02faeec640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4100b50 0x7f02f4198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02faeec640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f41022a0 0x7f02f4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02faeec640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f4102c20 0x7f02f41a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02faeec640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4110cb0 con 0x7f02f41022a0 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02faeec640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4110b30 con 0x7f02f4102c20 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02faeec640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4110e30 con 0x7f02f4100b50 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f41022a0 0x7f02f4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f41022a0 0x7f02f4199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33282/0 (socket says 172.21.15.69:33282) 2024-08-02T17:58:51.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 -- 172.21.15.69:0/1547474522 learned_addr learned my addr 172.21.15.69:0/1547474522 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:51.370 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:58:51.370 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:58:51.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 -- 172.21.15.69:0/1547474522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4100b50 msgr2=0x7f02f4198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:51.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f8c88640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4100b50 0x7f02f4198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:51.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f9489640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f4102c20 0x7f02f41a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:51.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4100b50 0x7f02f4198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 -- 172.21.15.69:0/1547474522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f4102c20 msgr2=0x7f02f41a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f4102c20 0x7f02f41a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 -- 172.21.15.69:0/1547474522 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f41a4480 con 0x7f02f41022a0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f8c88640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4100b50 0x7f02f4198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f9489640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f4102c20 0x7f02f41a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.030+0000 7f02f3fff640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f41022a0 0x7f02f4199400 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c920 tx=0x7f02e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.031+0000 7f02f1ffb640 1 -- 172.21.15.69:0/1547474522 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0007c70 con 0x7f02f41022a0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.031+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f41a4650 con 0x7f02f41022a0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.031+0000 7f02f1ffb640 1 -- 172.21.15.69:0/1547474522 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02e0007e10 con 0x7f02f41022a0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.031+0000 7f02f1ffb640 1 -- 172.21.15.69:0/1547474522 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0018890 con 0x7f02f41022a0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.031+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a4be0 con 0x7f02f41022a0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.032+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f4100f30 con 0x7f02f41022a0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.033+0000 7f02f1ffb640 1 -- 172.21.15.69:0/1547474522 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f02e0018a70 con 0x7f02f41022a0 2024-08-02T17:58:51.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.033+0000 7f02f1ffb640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02e40a2cf0 0x7f02e40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.033+0000 7f02f8c88640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02e40a2cf0 0x7f02e40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.034+0000 7f02f1ffb640 1 -- 172.21.15.69:0/1547474522 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f02e00cf9e0 con 0x7f02f41022a0 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.034+0000 7f02f8c88640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02e40a2cf0 0x7f02e40a51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02e800baa0 tx=0x7f02e8005d90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.038+0000 7f02f1ffb640 1 -- 172.21.15.69:0/1547474522 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f02e0014030 con 0x7f02f41022a0 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.240+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02f406a2d0 con 0x7f02f41022a0 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.241+0000 7f02f1ffb640 1 -- 172.21.15.69:0/1547474522 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f02e009c1f0 con 0x7f02f41022a0 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02e40a2cf0 msgr2=0x7f02e40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02e40a2cf0 0x7f02e40a51b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02e800baa0 tx=0x7f02e8005d90 comp rx=0 tx=0).stop 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f41022a0 msgr2=0x7f02f4199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f41022a0 0x7f02f4199400 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c920 tx=0x7f02e000cde0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 shutdown_connections 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f02f41022a0 0x7f02f4199400 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f02e40a2cf0 0x7f02e40a51b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f02f4102c20 0x7f02f41a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 --2- 172.21.15.69:0/1547474522 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f02f4100b50 0x7f02f4198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:51.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 >> 172.21.15.69:0/1547474522 conn(0x7f02f40fc500 msgr2=0x7f02f4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:51.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 shutdown_connections 2024-08-02T17:58:51.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:51.243+0000 7f02faeec640 1 -- 172.21.15.69:0/1547474522 wait complete. 2024-08-02T17:58:51.981 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:51 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1547474522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:52.009 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 79977741 --release --await' 2024-08-02T17:58:52.009 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 79977741 --release --await 2024-08-02T17:58:52.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:51 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1547474522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:52.055 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:52.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:51 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1547474522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 -- 172.21.15.69:0/3292814558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8718102260 msgr2=0x7f871810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 --2- 172.21.15.69:0/3292814558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8718102260 0x7f871810e790 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8710009cd0 tx=0x7f871002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 -- 172.21.15.69:0/3292814558 shutdown_connections 2024-08-02T17:58:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 --2- 172.21.15.69:0/3292814558 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8718102260 0x7f871810e790 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 --2- 172.21.15.69:0/3292814558 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87181018c0 0x7f8718101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 --2- 172.21.15.69:0/3292814558 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87181078c0 0x7f8718107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.534 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 -- 172.21.15.69:0/3292814558 >> 172.21.15.69:0/3292814558 conn(0x7f87180fd620 msgr2=0x7f87180ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 -- 172.21.15.69:0/3292814558 shutdown_connections 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 -- 172.21.15.69:0/3292814558 wait complete. 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 Processor -- start 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.530+0000 7f871dec4640 1 -- start start 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f871dec4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87181018c0 0x7f8718074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f871dec4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718102260 0x7f8718074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f871dec4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87181078c0 0x7f8718074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f871dec4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8718111cc0 con 0x7f87181018c0 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f871dec4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8718111b40 con 0x7f87181078c0 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f871dec4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8718111e40 con 0x7f8718102260 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87181018c0 0x7f8718074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87181018c0 0x7f8718074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33292/0 (socket says 172.21.15.69:33292) 2024-08-02T17:58:52.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 -- 172.21.15.69:0/1534221126 learned_addr learned my addr 172.21.15.69:0/1534221126 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f8717fff640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87181078c0 0x7f8718074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f870effd640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718102260 0x7f8718074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 -- 172.21.15.69:0/1534221126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718102260 msgr2=0x7f8718074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718102260 0x7f8718074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 -- 172.21.15.69:0/1534221126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87181078c0 msgr2=0x7f8718074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87181078c0 0x7f8718074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 -- 172.21.15.69:0/1534221126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87181a65c0 con 0x7f87181018c0 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f8717fff640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87181078c0 0x7f8718074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87177fe640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87181018c0 0x7f8718074220 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f870800e9f0 tx=0x7f870800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87157fa640 1 -- 172.21.15.69:0/1534221126 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870800cd80 con 0x7f87181018c0 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87181a68b0 con 0x7f87181018c0 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87157fa640 1 -- 172.21.15.69:0/1534221126 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8708010910 con 0x7f87181018c0 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.531+0000 7f87157fa640 1 -- 172.21.15.69:0/1534221126 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87080188f0 con 0x7f87181018c0 2024-08-02T17:58:52.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.532+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87181a6df0 con 0x7f87181018c0 2024-08-02T17:58:52.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.533+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8718101d20 con 0x7f87181018c0 2024-08-02T17:58:52.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.534+0000 7f87157fa640 1 -- 172.21.15.69:0/1534221126 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8708010ac0 con 0x7f87181018c0 2024-08-02T17:58:52.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.534+0000 7f87157fa640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87040a2cf0 0x7f87040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:52.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.535+0000 7f87157fa640 1 -- 172.21.15.69:0/1534221126 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f87080d00d0 con 0x7f87181018c0 2024-08-02T17:58:52.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.535+0000 7f870effd640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87040a2cf0 0x7f87040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:52.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.535+0000 7f870effd640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87040a2cf0 0x7f87040a51b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8700005e10 tx=0x7f8700005da0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:52.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.538+0000 7f87157fa640 1 -- 172.21.15.69:0/1534221126 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f870809c8e0 con 0x7f87181018c0 2024-08-02T17:58:52.573 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:52.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.755+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87181a7050 con 0x7f87181018c0 2024-08-02T17:58:52.757 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.756+0000 7f87157fa640 1 -- 172.21.15.69:0/1534221126 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f87080bae20 con 0x7f87181018c0 2024-08-02T17:58:52.758 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:58:52.758 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:58:52.758 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87040a2cf0 msgr2=0x7f87040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87040a2cf0 0x7f87040a51b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8700005e10 tx=0x7f8700005da0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87181018c0 msgr2=0x7f8718074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87181018c0 0x7f8718074220 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f870800e9f0 tx=0x7f870800eeb0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 shutdown_connections 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f87040a2cf0 0x7f87040a51b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87181078c0 0x7f8718074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8718102260 0x7f8718074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 --2- 172.21.15.69:0/1534221126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f87181018c0 0x7f8718074220 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 >> 172.21.15.69:0/1534221126 conn(0x7f87180fd620 msgr2=0x7f87180fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:52.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 shutdown_connections 2024-08-02T17:58:52.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:52.758+0000 7f871dec4640 1 -- 172.21.15.69:0/1534221126 wait complete. 2024-08-02T17:58:53.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:52 smithi083 ceph-mon[37031]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-08-02T17:58:53.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:52 smithi077 ceph-mon[37108]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-08-02T17:58:53.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:52 smithi069 ceph-mon[32746]: pgmap v388: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 7.4 MiB/s wr, 700 op/s 2024-08-02T17:58:53.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 -- 172.21.15.69:0/297447068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 msgr2=0x7f8f0010d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 --2- 172.21.15.69:0/297447068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 0x7f8f0010d680 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009cd0 tx=0x7f8ef802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 -- 172.21.15.69:0/297447068 shutdown_connections 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 --2- 172.21.15.69:0/297447068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 0x7f8f0010d680 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 --2- 172.21.15.69:0/297447068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f001007b0 0x7f8f00100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 --2- 172.21.15.69:0/297447068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f001067b0 0x7f8f00106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 -- 172.21.15.69:0/297447068 >> 172.21.15.69:0/297447068 conn(0x7f8f000fc4d0 msgr2=0x7f8f000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 -- 172.21.15.69:0/297447068 shutdown_connections 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.228+0000 7f8f059ff640 1 -- 172.21.15.69:0/297447068 wait complete. 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8f059ff640 1 Processor -- start 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8f059ff640 1 -- start start 2024-08-02T17:58:53.240 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8f059ff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f001007b0 0x7f8f00198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8f059ff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 0x7f8f00198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8f059ff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f001067b0 0x7f8f0019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8f059ff640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f00111cd0 con 0x7f8f00101150 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8f059ff640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f00111b50 con 0x7f8f001007b0 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8f059ff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f00111e50 con 0x7f8f001067b0 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.229+0000 7f8efe7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 0x7f8f00198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efe7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 0x7f8f00198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33302/0 (socket says 172.21.15.69:33302) 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efe7fc640 1 -- 172.21.15.69:0/158126633 learned_addr learned my addr 172.21.15.69:0/158126633 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f001067b0 0x7f8f0019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efeffd640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f001007b0 0x7f8f00198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efe7fc640 1 -- 172.21.15.69:0/158126633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f001067b0 msgr2=0x7f8f0019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efe7fc640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f001067b0 0x7f8f0019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efe7fc640 1 -- 172.21.15.69:0/158126633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f001007b0 msgr2=0x7f8f00198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efe7fc640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f001007b0 0x7f8f00198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.241 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efe7fc640 1 -- 172.21.15.69:0/158126633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0019d100 con 0x7f8f00101150 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8efe7fc640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 0x7f8f00198690 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400c890 tx=0x7f8ef400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8f049fd640 1 -- 172.21.15.69:0/158126633 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4004220 con 0x7f8f00101150 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8f049fd640 1 -- 172.21.15.69:0/158126633 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef40043c0 con 0x7f8f00101150 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8f049fd640 1 -- 172.21.15.69:0/158126633 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40049e0 con 0x7f8f00101150 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8f059ff640 1 -- 172.21.15.69:0/158126633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f0019d3f0 con 0x7f8f00101150 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.230+0000 7f8f059ff640 1 -- 172.21.15.69:0/158126633 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f001a4cd0 con 0x7f8f00101150 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f049fd640 1 -- 172.21.15.69:0/158126633 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8ef40079b0 con 0x7f8f00101150 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f07dfd740 1 Processor -- start 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f07dfd740 1 -- start start 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae41d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 0x55d9eae4c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f049fd640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ee80a2d70 0x7f8ee80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.242 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae4ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d9eada7c60 con 0x55d9eae41990 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9ead3fb90 con 0x55d9eae4cb70 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.233+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9eada3c20 con 0x55d9eae422b0 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae41d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8efe7fc640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ee80a2d70 0x7f8ee80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae41d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33304/0 (socket says 172.21.15.69:33304) 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 -- 172.21.15.69:0/736759719 learned_addr learned my addr 172.21.15.69:0/736759719 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8efeffd640 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 0x55d9eae4c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8f049fd640 1 -- 172.21.15.69:0/158126633 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f8ef40cf810 con 0x7f8f00101150 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae4ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 -- 172.21.15.69:0/736759719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 msgr2=0x55d9eae4c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:53.243 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 0x55d9eae4c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 -- 172.21.15.69:0/736759719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 msgr2=0x55d9eae4ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae4ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 -- 172.21.15.69:0/736759719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9eaab7580 con 0x55d9eae41990 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8efe7fc640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ee80a2d70 0x7f8ee80a5230 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4097df0 tx=0x7f8ef40c6470 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae41d70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009a60 tx=0x7f8ef8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b5782daef7c15c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8ee5ffb640 1 -- 172.21.15.69:0/736759719 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8031cb0 con 0x55d9eae41990 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8ee5ffb640 1 -- 172.21.15.69:0/736759719 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef8036040 con 0x55d9eae41990 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.234+0000 7f8ee5ffb640 1 -- 172.21.15.69:0/736759719 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80305a0 con 0x55d9eae41990 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 -- 172.21.15.69:0/736759719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 msgr2=0x55d9eae41d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae41d70 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009a60 tx=0x7f8ef8005fa0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 -- 172.21.15.69:0/736759719 shutdown_connections 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae4ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 0x55d9eae4c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.244 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/736759719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae41d70 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 -- 172.21.15.69:0/736759719 >> 172.21.15.69:0/736759719 conn(0x55d9eae3df20 msgr2=0x55d9eae3e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 -- 172.21.15.69:0/736759719 shutdown_connections 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 -- 172.21.15.69:0/736759719 wait complete. 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.235+0000 7f8f07dfd740 1 -- start start 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8f07dfd740 1 Processor -- start 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae49ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 0x55d9eae4a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae4a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55d9eae4f740 con 0x55d9eae41990 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55d9ead457c0 con 0x55d9eae4cb70 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55d9eae4f8c0 con 0x55d9eae422b0 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8f07dfd740 10 client.0 _ll_register_callbacks cb 0x55d9ead4ef50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:53.245 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae49ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae49ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33312/0 (socket says 172.21.15.69:33312) 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 -- 172.21.15.69:0/1719111575 learned_addr learned my addr 172.21.15.69:0/1719111575 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8efeffd640 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 0x55d9eae4a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 -- 172.21.15.69:0/1719111575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 msgr2=0x55d9eae4a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 0x55d9eae4a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 -- 172.21.15.69:0/1719111575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 msgr2=0x55d9eae4a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae4a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 -- 172.21.15.69:0/1719111575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9eab81960 con 0x55d9eae41990 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae49ad0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000f40 tx=0x7f8ef80317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852f39d5aa414c5f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.236+0000 7f8edeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8edeffd640 1 -- 172.21.15.69:0/1719111575 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8003d80 con 0x55d9eae41990 2024-08-02T17:58:53.246 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8edeffd640 1 -- 172.21.15.69:0/1719111575 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f8ef8003f20 con 0x55d9eae41990 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 1 -- 172.21.15.69:0/1719111575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef8036040 con 0x55d9eae41990 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8edeffd640 1 -- 172.21.15.69:0/1719111575 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ef805c050 con 0x55d9eae41990 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 10 client.18420 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 1 -- 172.21.15.69:0/1719111575 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9eae4f740 con 0x55d9eae41990 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8edeffd640 1 -- 172.21.15.69:0/1719111575 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef8036040 con 0x55d9eae41990 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8edeffd640 1 -- 172.21.15.69:0/1719111575 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef8030070 con 0x55d9eae41990 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 10 client.18420 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 10 client.18420 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 20 client.18420 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d9eae5c980 0x55d9eae5ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 4 client.18420 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.237+0000 7f8f07dfd740 1 -- 172.21.15.69:0/1719111575 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9eadb51d0 con 0x55d9eae5c980 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.238+0000 7f8efeffd640 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d9eae5c980 0x55d9eae5ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.247 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.238+0000 7f8edeffd640 10 client.18420 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:53.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.238+0000 7f8efeffd640 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d9eae5c980 0x55d9eae5ed70 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b8dbb4e1dd241e server_cookie=2d735f41ad18bf5a in_seq=0 out_seq=0 2024-08-02T17:58:53.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8edeffd640 1 -- 172.21.15.69:0/1719111575 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d9eadb51d0 con 0x55d9eae5c980 2024-08-02T17:58:53.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8edeffd640 10 client.18420 handle_command_reply: tid=0 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 client.18420 shutdown 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 -- 172.21.15.69:0/1719111575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 msgr2=0x55d9eae49ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae49ad0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8000f40 tx=0x7f8ef80317a0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 -- 172.21.15.69:0/1719111575 shutdown_connections 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d9eae5c980 0x55d9eae5ed70 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae4a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae422b0 0x55d9eae4a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8edeffd640 0 client.18420 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/1719111575 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae49ad0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 -- 172.21.15.69:0/1719111575 >> 172.21.15.69:0/1719111575 conn(0x55d9eae3df20 msgr2=0x55d9eae43c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 -- 172.21.15.69:0/1719111575 shutdown_connections 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 1 -- 172.21.15.69:0/1719111575 wait complete. 2024-08-02T17:58:53.249 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.239+0000 7f8f07dfd740 20 client.18420 trim_cache size 0 max 16384 2024-08-02T17:58:53.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8f07dfd740 1 Processor -- start 2024-08-02T17:58:53.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8f07dfd740 1 -- start start 2024-08-02T17:58:53.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae41990 0x55d9eae64490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae422b0 0x55d9eae649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae46d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001b1710 con 0x55d9eae422b0 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001aefe0 con 0x55d9eae4cb70 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001ace30 con 0x55d9eae41990 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae422b0 0x55d9eae649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae422b0 0x55d9eae649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33316/0 (socket says 172.21.15.69:33316) 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 -- 172.21.15.69:0/537450871 learned_addr learned my addr 172.21.15.69:0/537450871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/537450871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae41990 0x55d9eae64490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 -- 172.21.15.69:0/537450871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae41990 msgr2=0x55d9eae64490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 --2- 172.21.15.69:0/537450871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae41990 0x55d9eae64490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 -- 172.21.15.69:0/537450871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 msgr2=0x55d9eae46d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 --2- 172.21.15.69:0/537450871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae46d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 -- 172.21.15.69:0/537450871 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9eae417c0 con 0x55d9eae422b0 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.256+0000 7f8efeffd640 1 --2- 172.21.15.69:0/537450871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae422b0 0x55d9eae649d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x55d9eae3ed80 tx=0x7f8ef000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5a5cc14399ab79 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:53.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8ee57fa640 1 -- 172.21.15.69:0/537450871 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef001e600 con 0x55d9eae422b0 2024-08-02T17:58:53.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8ee57fa640 1 -- 172.21.15.69:0/537450871 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef001ec80 con 0x55d9eae422b0 2024-08-02T17:58:53.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8ee57fa640 1 -- 172.21.15.69:0/537450871 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef000f420 con 0x55d9eae422b0 2024-08-02T17:58:53.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 -- 172.21.15.69:0/537450871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae422b0 msgr2=0x55d9eae649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/537450871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae422b0 0x55d9eae649d0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x55d9eae3ed80 tx=0x7f8ef000a250 comp rx=0 tx=0).stop 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 -- 172.21.15.69:0/537450871 shutdown_connections 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/537450871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae422b0 0x55d9eae649d0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/537450871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae4cb70 0x55d9eae46d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/537450871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae41990 0x55d9eae64490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 -- 172.21.15.69:0/537450871 >> 172.21.15.69:0/537450871 conn(0x55d9eae3df20 msgr2=0x55d9eae4bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 -- 172.21.15.69:0/537450871 shutdown_connections 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.257+0000 7f8f07dfd740 1 -- 172.21.15.69:0/537450871 wait complete. 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 1 -- start start 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 1 Processor -- start 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae55340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae422b0 0x55d9eae468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae4cb70 0x55d9eae46ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001ad4a0 con 0x55d9eae41990 2024-08-02T17:58:53.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001ae910 con 0x55d9eae422b0 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001adb70 con 0x55d9eae4cb70 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8f07dfd740 10 client.0 _ll_register_callbacks cb 0x7f8ef8031660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae55340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae55340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33318/0 (socket says 172.21.15.69:33318) 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 -- 172.21.15.69:0/2173949901 learned_addr learned my addr 172.21.15.69:0/2173949901 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae4cb70 0x55d9eae46ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 -- 172.21.15.69:0/2173949901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae4cb70 msgr2=0x55d9eae46ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae4cb70 0x55d9eae46ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 -- 172.21.15.69:0/2173949901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae422b0 msgr2=0x55d9eae468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8efeffd640 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae422b0 0x55d9eae468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae422b0 0x55d9eae468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 -- 172.21.15.69:0/2173949901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9eae476f0 con 0x55d9eae41990 2024-08-02T17:58:53.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8eff7fe640 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae55340 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80043a0 tx=0x7f8ef804b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86772053d42906f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.258+0000 7f8edeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8edeffd640 1 -- 172.21.15.69:0/2173949901 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8009540 con 0x55d9eae41990 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8edeffd640 1 -- 172.21.15.69:0/2173949901 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f8ef8049d40 con 0x55d9eae41990 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 1 -- 172.21.15.69:0/2173949901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ef001ec80 con 0x55d9eae41990 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8edeffd640 1 -- 172.21.15.69:0/2173949901 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8ef8003b80 con 0x55d9eae41990 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 10 client.18426 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 1 -- 172.21.15.69:0/2173949901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9eaec7580 con 0x55d9eae41990 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8edeffd640 1 -- 172.21.15.69:0/2173949901 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ef001ec80 con 0x55d9eae41990 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8edeffd640 1 -- 172.21.15.69:0/2173949901 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8ef8030a60 con 0x55d9eae41990 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 10 client.18426 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 10 client.18426 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 20 client.18426 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:53.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d9eae5c980 0x55d9eaec9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 4 client.18426 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "79977741", "release": true, "await": true}] 2024-08-02T17:58:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.259+0000 7f8f07dfd740 1 -- 172.21.15.69:0/2173949901 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "79977741", "release": true, "await": true}) -- 0x55d9eadaf6e0 con 0x55d9eae5c980 2024-08-02T17:58:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.260+0000 7f8efeffd640 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d9eae5c980 0x55d9eaec9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.260+0000 7f8edeffd640 10 client.18426 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:53.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.260+0000 7f8efeffd640 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d9eae5c980 0x55d9eaec9980 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3601aeea790198e server_cookie=8ba05b2f6b0604ff in_seq=0 out_seq=0 2024-08-02T17:58:53.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.335+0000 7f8edeffd640 1 -- 172.21.15.69:0/2173949901 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d9eadaf6e0 con 0x55d9eae5c980 2024-08-02T17:58:53.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.335+0000 7f8edeffd640 10 client.18426 handle_command_reply: tid=0 2024-08-02T17:58:53.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.337+0000 7f8f07dfd740 1 client.18426 shutdown 2024-08-02T17:58:53.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.340+0000 7f8f07dfd740 1 -- 172.21.15.69:0/2173949901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 msgr2=0x55d9eae55340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.340+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae55340 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80043a0 tx=0x7f8ef804b040 comp rx=0 tx=0).stop 2024-08-02T17:58:53.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8f07dfd740 1 -- 172.21.15.69:0/2173949901 shutdown_connections 2024-08-02T17:58:53.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55d9eae5c980 0x55d9eaec9980 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8edeffd640 0 client.18426 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:53.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55d9eae4cb70 0x55d9eae46ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55d9eae422b0 0x55d9eae468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8f07dfd740 1 --2- 172.21.15.69:0/2173949901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55d9eae41990 0x55d9eae55340 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8f07dfd740 1 -- 172.21.15.69:0/2173949901 >> 172.21.15.69:0/2173949901 conn(0x55d9eae3df20 msgr2=0x55d9eae4b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:53.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8f07dfd740 1 -- 172.21.15.69:0/2173949901 shutdown_connections 2024-08-02T17:58:53.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.341+0000 7f8f07dfd740 1 -- 172.21.15.69:0/2173949901 wait complete. 2024-08-02T17:58:53.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.346+0000 7f8f07dfd740 20 client.18426 trim_cache size 0 max 16384 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 44, 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "79977741": { 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 44, 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:58:53.348 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 130.0, 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:53.349 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:53.350 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:58:53.350 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:58:53.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 -- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ee80a2d70 msgr2=0x7f8ee80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ee80a2d70 0x7f8ee80a5230 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4097df0 tx=0x7f8ef40c6470 comp rx=0 tx=0).stop 2024-08-02T17:58:53.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 -- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 msgr2=0x7f8f00198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:53.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 0x7f8f00198690 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400c890 tx=0x7f8ef400cd50 comp rx=0 tx=0).stop 2024-08-02T17:58:53.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 -- 172.21.15.69:0/158126633 shutdown_connections 2024-08-02T17:58:53.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8f00101150 0x7f8f00198690 secure :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400c890 tx=0x7f8ef400cd50 comp rx=0 tx=0).stop 2024-08-02T17:58:53.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f8ee80a2d70 0x7f8ee80a5230 secure :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4097df0 tx=0x7f8ef40c6470 comp rx=0 tx=0).stop 2024-08-02T17:58:53.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8f001067b0 0x7f8f0019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 --2- 172.21.15.69:0/158126633 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8f001007b0 0x7f8f00198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:53.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.354+0000 7f8f059ff640 1 -- 172.21.15.69:0/158126633 >> 172.21.15.69:0/158126633 conn(0x7f8f000fc4d0 msgr2=0x7f8f000fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:53.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.355+0000 7f8f059ff640 1 -- 172.21.15.69:0/158126633 shutdown_connections 2024-08-02T17:58:53.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:53.355+0000 7f8f059ff640 1 -- 172.21.15.69:0/158126633 wait complete. 2024-08-02T17:58:53.872 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:58:54.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:53 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1534221126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:54.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:53 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1534221126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:54.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:53 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1534221126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:58:54.547 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '79977741', total seconds elapsed: 15.1 2024-08-02T17:58:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:54.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:54 smithi069 ceph-mon[32746]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.5 MiB/s wr, 718 op/s 2024-08-02T17:58:54.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:54 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:58:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 705.0 s, 210.7 tps, lat 684.722 ms stddev 1730.519 2024-08-02T17:58:55.026 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:54 smithi083 ceph-mon[37031]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.5 MiB/s wr, 718 op/s 2024-08-02T17:58:55.027 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:54 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:58:55.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:54 smithi077 ceph-mon[37108]: pgmap v389: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 7.5 MiB/s wr, 718 op/s 2024-08-02T17:58:55.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:54 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:58:55.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.330+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/1354444313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bb410ec60 msgr2=0x7f3bb410cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.330+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/1354444313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bb410ec60 0x7f3bb410cfd0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000af30 tx=0x7f3bb002e640 comp rx=0 tx=0).stop 2024-08-02T17:58:55.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.332+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/1354444313 shutdown_connections 2024-08-02T17:58:55.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.332+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/1354444313 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bb410ec60 0x7f3bb410cfd0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.332+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/1354444313 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb410d6e0 0x7f3bb4115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.332+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/1354444313 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bb410e2b0 0x7f3bb410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.332+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/1354444313 >> 172.21.15.69:0/1354444313 conn(0x7f3bb406de70 msgr2=0x7f3bb406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:55.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.333+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/1354444313 shutdown_connections 2024-08-02T17:58:55.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.336+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/1354444313 wait complete. 2024-08-02T17:58:55.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.337+0000 7f3bbc3c0640 1 Processor -- start 2024-08-02T17:58:55.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.337+0000 7f3bbc3c0640 1 -- start start 2024-08-02T17:58:55.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.337+0000 7f3bbc3c0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bb410d6e0 0x7f3bb41a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bbc3c0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bb410e2b0 0x7f3bb41a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bba15c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bb410d6e0 0x7f3bb41a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bba15c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bb410d6e0 0x7f3bb41a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49676/0 (socket says 172.21.15.69:49676) 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bba15c640 1 -- 172.21.15.69:0/2117340492 learned_addr learned my addr 172.21.15.69:0/2117340492 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb410ec60 0x7f3bb419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4118370 con 0x7f3bb410e2b0 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41181f0 con 0x7f3bb410d6e0 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41184f0 con 0x7f3bb410ec60 2024-08-02T17:58:55.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.338+0000 7f3bba95d640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb410ec60 0x7f3bb419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bba15c640 1 -- 172.21.15.69:0/2117340492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb410ec60 msgr2=0x7f3bb419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bba15c640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb410ec60 0x7f3bb419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bba15c640 1 -- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bb410e2b0 msgr2=0x7f3bb41a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bba15c640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bb410e2b0 0x7f3bb41a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bba15c640 1 -- 172.21.15.69:0/2117340492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb419ef20 con 0x7f3bb410d6e0 2024-08-02T17:58:55.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bba15c640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bb410d6e0 0x7f3bb41a4600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b450 tx=0x7f3ba400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:55.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bab7fe640 1 -- 172.21.15.69:0/2117340492 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4015020 con 0x7f3bb410d6e0 2024-08-02T17:58:55.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb419f210 con 0x7f3bb410d6e0 2024-08-02T17:58:55.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb407a180 con 0x7f3bb410d6e0 2024-08-02T17:58:55.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.339+0000 7f3bab7fe640 1 -- 172.21.15.69:0/2117340492 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba4011d70 con 0x7f3bb410d6e0 2024-08-02T17:58:55.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.340+0000 7f3bab7fe640 1 -- 172.21.15.69:0/2117340492 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400f340 con 0x7f3bb410d6e0 2024-08-02T17:58:55.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.344+0000 7f3bbe7be740 1 Processor -- start 2024-08-02T17:58:55.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.345+0000 7f3bbe7be740 1 -- start start 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.345+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f42c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.345+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42c830 0x561c3f43f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.345+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f42c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f42c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59350/0 (socket says 172.21.15.69:59350) 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f441dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bbe7be740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561c3f391a40 con 0x561c3f43f9e0 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bbe7be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c3f3299d0 con 0x561c3f42c830 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bbe7be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c3f38da00 con 0x561c3f42bf10 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bba95d640 1 -- 172.21.15.69:0/3882398654 learned_addr learned my addr 172.21.15.69:0/3882398654 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bab7fe640 1 -- 172.21.15.69:0/2117340492 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3ba4018020 con 0x7f3bb410d6e0 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bb995b640 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f441dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.346+0000 7f3bba15c640 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42c830 0x561c3f43f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.347+0000 7f3bba95d640 1 -- 172.21.15.69:0/3882398654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42c830 msgr2=0x561c3f43f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:55.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.347+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42c830 0x561c3f43f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.347+0000 7f3bba95d640 1 -- 172.21.15.69:0/3882398654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 msgr2=0x561c3f441dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:55.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.347+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f441dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.347+0000 7f3bba95d640 1 -- 172.21.15.69:0/3882398654 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3f330650 con 0x561c3f42bf10 2024-08-02T17:58:55.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.347+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f42c2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009f20 tx=0x7f3bac00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1acd9954f522fc0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.348+0000 7f3bab7fe640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ba00a2df0 0x7f3ba00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.348+0000 7f3bba15c640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ba00a2df0 0x7f3ba00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.349+0000 7f3bab7fe640 1 -- 172.21.15.69:0/2117340492 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f3ba40cf740 con 0x7f3bb410d6e0 2024-08-02T17:58:55.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.350+0000 7f3ba8dea640 1 -- 172.21.15.69:0/3882398654 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac01a070 con 0x561c3f42bf10 2024-08-02T17:58:55.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.350+0000 7f3ba8dea640 1 -- 172.21.15.69:0/3882398654 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bac0158f0 con 0x561c3f42bf10 2024-08-02T17:58:55.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.350+0000 7f3ba8dea640 1 -- 172.21.15.69:0/3882398654 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00a900 con 0x561c3f42bf10 2024-08-02T17:58:55.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3882398654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 msgr2=0x561c3f42c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f42c2f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009f20 tx=0x7f3bac00c8b0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bba15c640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ba00a2df0 0x7f3ba00a52b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40d57e0 tx=0x7f3ba40d5770 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3882398654 shutdown_connections 2024-08-02T17:58:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f441dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42c830 0x561c3f43f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3882398654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f42c2f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3882398654 >> 172.21.15.69:0/3882398654 conn(0x561c3f428580 msgr2=0x561c3f428960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3882398654 shutdown_connections 2024-08-02T17:58:55.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.351+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3882398654 wait complete. 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 1 -- start start 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 1 Processor -- start 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f43c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f42c830 0x561c3f43ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f43f9e0 0x561c3f43d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561c3f442550 con 0x561c3f42c830 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561c3f34cb20 con 0x561c3f43f9e0 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561c3f4426d0 con 0x561c3f42bf10 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.352+0000 7f3bbe7be740 10 client.0 _ll_register_callbacks cb 0x561c3f34abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.353+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f43c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.353+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f43f9e0 0x561c3f43d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.353+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f43c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59362/0 (socket says 172.21.15.69:59362) 2024-08-02T17:58:55.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.353+0000 7f3bba95d640 1 -- 172.21.15.69:0/1622527353 learned_addr learned my addr 172.21.15.69:0/1622527353 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:55.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.353+0000 7f3bb995b640 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f42c830 0x561c3f43ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.354+0000 7f3bba95d640 1 -- 172.21.15.69:0/1622527353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f43f9e0 msgr2=0x561c3f43d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:55.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.354+0000 7f3bba95d640 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f43f9e0 0x561c3f43d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.354+0000 7f3bba95d640 1 -- 172.21.15.69:0/1622527353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f42c830 msgr2=0x561c3f43ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:55.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.354+0000 7f3bba95d640 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f42c830 0x561c3f43ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.354+0000 7f3bba95d640 1 -- 172.21.15.69:0/1622527353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3f1033c0 con 0x561c3f42bf10 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.354+0000 7f3bba95d640 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f43c940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009fd0 tx=0x7f3bac00a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f5c5901d2630fd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.354+0000 7f3b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.354+0000 7f3b9dffb640 1 -- 172.21.15.69:0/1622527353 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac01a070 con 0x561c3f42bf10 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.355+0000 7f3b9dffb640 1 -- 172.21.15.69:0/1622527353 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f3bac00a6a0 con 0x561c3f42bf10 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.355+0000 7f3bbe7be740 1 -- 172.21.15.69:0/1622527353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bac0158f0 con 0x561c3f42bf10 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.355+0000 7f3b9dffb640 1 -- 172.21.15.69:0/1622527353 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3bac02d050 con 0x561c3f42bf10 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.355+0000 7f3bbe7be740 10 client.27670 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.355+0000 7f3bbe7be740 1 -- 172.21.15.69:0/1622527353 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c3f44e2a0 con 0x561c3f42bf10 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.355+0000 7f3b9dffb640 1 -- 172.21.15.69:0/1622527353 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bac0158f0 con 0x561c3f42bf10 2024-08-02T17:58:55.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.356+0000 7f3b9dffb640 1 -- 172.21.15.69:0/1622527353 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3bac024910 con 0x561c3f42bf10 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.356+0000 7f3bbe7be740 10 client.27670 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.356+0000 7f3bbe7be740 10 client.27670 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.356+0000 7f3bbe7be740 20 client.27670 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.356+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3f44f650 0x561c3f451a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.356+0000 7f3bbe7be740 4 client.27670 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.356+0000 7f3bbe7be740 1 -- 172.21.15.69:0/1622527353 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c3f39efb0 con 0x561c3f44f650 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.356+0000 7f3bb995b640 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3f44f650 0x561c3f451a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.357+0000 7f3bb995b640 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3f44f650 0x561c3f451a40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=229b7f54131b7114 server_cookie=3709726b3e48d154 in_seq=0 out_seq=0 2024-08-02T17:58:55.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.357+0000 7f3b9dffb640 10 client.27670 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:58:55.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.357+0000 7f3b9dffb640 1 -- 172.21.15.69:0/1622527353 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561c3f39efb0 con 0x561c3f44f650 2024-08-02T17:58:55.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.357+0000 7f3b9dffb640 10 client.27670 handle_command_reply: tid=0 2024-08-02T17:58:55.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.357+0000 7f3bbe7be740 1 client.27670 shutdown 2024-08-02T17:58:55.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 -- 172.21.15.69:0/1622527353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 msgr2=0x561c3f43c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f43c940 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009fd0 tx=0x7f3bac00a4a0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 -- 172.21.15.69:0/1622527353 shutdown_connections 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3f44f650 0x561c3f451a40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f42c830 0x561c3f43ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3b9dffb640 0 client.27670 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f43f9e0 0x561c3f43d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/1622527353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42bf10 0x561c3f43c940 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 -- 172.21.15.69:0/1622527353 >> 172.21.15.69:0/1622527353 conn(0x561c3f428580 msgr2=0x561c3f42e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 -- 172.21.15.69:0/1622527353 shutdown_connections 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 1 -- 172.21.15.69:0/1622527353 wait complete. 2024-08-02T17:58:55.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.358+0000 7f3bbe7be740 20 client.27670 trim_cache size 0 max 16384 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bbe7be740 1 Processor -- start 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bbe7be740 1 -- start start 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f43f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 0x561c3f439ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f43a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f43f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f43f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49700/0 (socket says 172.21.15.69:49700) 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bb995b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 0x561c3f439ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bba95d640 1 -- 172.21.15.69:0/3817723825 learned_addr learned my addr 172.21.15.69:0/3817723825 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:55.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3817723825 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3b98012a10 con 0x561c3f43f9e0 2024-08-02T17:58:55.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3817723825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b980102e0 con 0x561c3f42bf10 2024-08-02T17:58:55.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3817723825 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9800e130 con 0x561c3f42c830 2024-08-02T17:58:55.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.379+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3817723825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f43a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.380+0000 7f3bba95d640 1 -- 172.21.15.69:0/3817723825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 msgr2=0x561c3f439ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:55.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.380+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3817723825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 0x561c3f439ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.380+0000 7f3bba95d640 1 -- 172.21.15.69:0/3817723825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 msgr2=0x561c3f43a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:55.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.380+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3817723825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f43a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.380+0000 7f3bba95d640 1 -- 172.21.15.69:0/3817723825 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3f43ae90 con 0x561c3f42bf10 2024-08-02T17:58:55.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.381+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3817723825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f43f0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3bac004a80 tx=0x7f3bac000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c7952c9507176cb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:55.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.381+0000 7f3b9ffff640 1 -- 172.21.15.69:0/3817723825 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac01a040 con 0x561c3f42bf10 2024-08-02T17:58:55.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.381+0000 7f3b9ffff640 1 -- 172.21.15.69:0/3817723825 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bac00cdb0 con 0x561c3f42bf10 2024-08-02T17:58:55.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.381+0000 7f3b9ffff640 1 -- 172.21.15.69:0/3817723825 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac01a040 con 0x561c3f42bf10 2024-08-02T17:58:55.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.382+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3817723825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 msgr2=0x561c3f43f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.382+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3817723825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f43f0a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3bac004a80 tx=0x7f3bac000f40 comp rx=0 tx=0).stop 2024-08-02T17:58:55.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.383+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3817723825 shutdown_connections 2024-08-02T17:58:55.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.383+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3817723825 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f43a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.383+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3817723825 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 0x561c3f439ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.383+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3817723825 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f43f0a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.384+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3817723825 >> 172.21.15.69:0/3817723825 conn(0x561c3f428580 msgr2=0x561c3f43ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.384+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3817723825 shutdown_connections 2024-08-02T17:58:55.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.384+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3817723825 wait complete. 2024-08-02T17:58:55.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.384+0000 7f3bbe7be740 1 -- start start 2024-08-02T17:58:55.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.385+0000 7f3bbe7be740 1 Processor -- start 2024-08-02T17:58:55.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.385+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f451a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.385+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 0x561c3f451f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.385+0000 7f3bbe7be740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f4524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.385+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f451a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.385+0000 7f3bba95d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f451a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49710/0 (socket says 172.21.15.69:49710) 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bba95d640 1 -- 172.21.15.69:0/3904894849 learned_addr learned my addr 172.21.15.69:0/3904894849 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bb995b640 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 0x561c3f451f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9800e7a0 con 0x561c3f43f9e0 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9800fc10 con 0x561c3f42bf10 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9800ee70 con 0x561c3f42c830 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bbe7be740 10 client.0 _ll_register_callbacks cb 0x7f3bac018070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f4524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bba95d640 1 -- 172.21.15.69:0/3904894849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 msgr2=0x561c3f451f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 0x561c3f451f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bba95d640 1 -- 172.21.15.69:0/3904894849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 msgr2=0x561c3f4524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f4524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.386+0000 7f3bba95d640 1 -- 172.21.15.69:0/3904894849 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3f4b81d0 con 0x561c3f42bf10 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.387+0000 7f3bba95d640 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f451a20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009d30 tx=0x7f3bac00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e5c2d6551e166d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.387+0000 7f3b9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.387+0000 7f3b9dffb640 1 -- 172.21.15.69:0/3904894849 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac01a040 con 0x561c3f42bf10 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.387+0000 7f3b9dffb640 1 -- 172.21.15.69:0/3904894849 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f3bac012050 con 0x561c3f42bf10 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.387+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bac00cdb0 con 0x561c3f42bf10 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.387+0000 7f3b9dffb640 1 -- 172.21.15.69:0/3904894849 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3bac045050 con 0x561c3f42bf10 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.387+0000 7f3bbe7be740 10 client.27884 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.387+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c3f4b9470 con 0x561c3f42bf10 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3b9dffb640 1 -- 172.21.15.69:0/3904894849 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bac00cdb0 con 0x561c3f42bf10 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3b9dffb640 1 -- 172.21.15.69:0/3904894849 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3bac00a4a0 con 0x561c3f42bf10 2024-08-02T17:58:55.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3bbe7be740 10 client.27884 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3bbe7be740 10 client.27884 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3bbe7be740 20 client.27884 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3f4b9a70 0x561c3f4bbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3bbe7be740 4 client.27884 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c3f3994c0 con 0x561c3f4b9a70 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.388+0000 7f3bb995b640 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3f4b9a70 0x561c3f4bbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.389+0000 7f3bb995b640 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3f4b9a70 0x561c3f4bbea0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e4427c2200e739c server_cookie=e5087396879b2a12 in_seq=0 out_seq=0 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.389+0000 7f3b9dffb640 10 client.27884 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.389+0000 7f3b9dffb640 1 -- 172.21.15.69:0/3904894849 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c3f3994c0 con 0x561c3f4b9a70 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.389+0000 7f3b9dffb640 10 client.27884 handle_command_reply: tid=0 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.389+0000 7f3bbe7be740 1 client.27884 shutdown 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.389+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 msgr2=0x561c3f451a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.389+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f451a20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009d30 tx=0x7f3bac00c8b0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 shutdown_connections 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561c3f4b9a70 0x561c3f4bbea0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561c3f43f9e0 0x561c3f4524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561c3f42c830 0x561c3f451f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 1 --2- 172.21.15.69:0/3904894849 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561c3f42bf10 0x561c3f451a20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 >> 172.21.15.69:0/3904894849 conn(0x561c3f428580 msgr2=0x561c3f426d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3b9dffb640 0 client.27884 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 shutdown_connections 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 1 -- 172.21.15.69:0/3904894849 wait complete. 2024-08-02T17:58:55.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.390+0000 7f3bbe7be740 20 client.27884 trim_cache size 0 max 16384 2024-08-02T17:58:55.398 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:58:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.392+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ba00a2df0 msgr2=0x7f3ba00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.392+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ba00a2df0 0x7f3ba00a52b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40d57e0 tx=0x7f3ba40d5770 comp rx=0 tx=0).stop 2024-08-02T17:58:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.392+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bb410d6e0 msgr2=0x7f3bb41a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.392+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bb410d6e0 0x7f3bb41a4600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400b450 tx=0x7f3ba400b910 comp rx=0 tx=0).stop 2024-08-02T17:58:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.393+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 shutdown_connections 2024-08-02T17:58:55.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.393+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bb410e2b0 0x7f3bb41a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.393+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3ba00a2df0 0x7f3ba00a52b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.393+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bb410ec60 0x7f3bb419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.393+0000 7f3bbc3c0640 1 --2- 172.21.15.69:0/2117340492 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bb410d6e0 0x7f3bb41a4600 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.393+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 >> 172.21.15.69:0/2117340492 conn(0x7f3bb406de70 msgr2=0x7f3bb410b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.393+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 shutdown_connections 2024-08-02T17:58:55.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:55.393+0000 7f3bbc3c0640 1 -- 172.21.15.69:0/2117340492 wait complete. 2024-08-02T17:58:56.124 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:58:56.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:56 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:56 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.180 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:56 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.181 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:56 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.181 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:56 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.181 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:56 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:56 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:56 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:56 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:56 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:56 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:56 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:56 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:56 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:56 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:56 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:56 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:56 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:56.675 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:57.292 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.286+0000 7f30ac2e4640 1 -- 172.21.15.69:0/3354114445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30a41022a0 msgr2=0x7f30a41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.293 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.286+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/3354114445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30a41022a0 0x7f30a41026e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3094009900 tx=0x7f309402a480 comp rx=0 tx=0).stop 2024-08-02T17:58:57.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.293+0000 7f30ac2e4640 1 -- 172.21.15.69:0/3354114445 shutdown_connections 2024-08-02T17:58:57.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.293+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/3354114445 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30a41022a0 0x7f30a41026e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.293+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/3354114445 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30a4102c20 0x7f30a410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.293+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/3354114445 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4100b50 0x7f30a4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.294 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.293+0000 7f30ac2e4640 1 -- 172.21.15.69:0/3354114445 >> 172.21.15.69:0/3354114445 conn(0x7f30a40fc500 msgr2=0x7f30a40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:57.295 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.294+0000 7f30ac2e4640 1 -- 172.21.15.69:0/3354114445 shutdown_connections 2024-08-02T17:58:57.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.297+0000 7f30ac2e4640 1 -- 172.21.15.69:0/3354114445 wait complete. 2024-08-02T17:58:57.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.297+0000 7f30ac2e4640 1 Processor -- start 2024-08-02T17:58:57.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.305+0000 7f30ac2e4640 1 -- start start 2024-08-02T17:58:57.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.305+0000 7f30ac2e4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4100b50 0x7f30a4073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.305+0000 7f30ac2e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30a41022a0 0x7f30a40739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.305+0000 7f30ac2e4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30a4102c20 0x7f30a406c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.305+0000 7f30ac2e4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30a410fbf0 con 0x7f30a4102c20 2024-08-02T17:58:57.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.305+0000 7f30ac2e4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30a410fa70 con 0x7f30a4100b50 2024-08-02T17:58:57.308 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.305+0000 7f30ac2e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30a410fd70 con 0x7f30a41022a0 2024-08-02T17:58:57.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.310+0000 7f30aa080640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4100b50 0x7f30a4073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.310+0000 7f30aa881640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30a4102c20 0x7f30a406c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.310+0000 7f30aa080640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4100b50 0x7f30a4073490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49736/0 (socket says 172.21.15.69:49736) 2024-08-02T17:58:57.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.310+0000 7f30aa080640 1 -- 172.21.15.69:0/1297086136 learned_addr learned my addr 172.21.15.69:0/1297086136 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:57.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.310+0000 7f30aa080640 1 -- 172.21.15.69:0/1297086136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30a41022a0 msgr2=0x7f30a40739d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.310+0000 7f30aa080640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30a41022a0 0x7f30a40739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.310+0000 7f30aa080640 1 -- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30a4102c20 msgr2=0x7f30a406c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.310+0000 7f30aa080640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30a4102c20 0x7f30a406c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.311+0000 7f30aa080640 1 -- 172.21.15.69:0/1297086136 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a406cd80 con 0x7f30a4100b50 2024-08-02T17:58:57.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.311+0000 7f30aa080640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4100b50 0x7f30a4073490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30a000e9f0 tx=0x7f30a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:57.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.313+0000 7f309b7fe640 1 -- 172.21.15.69:0/1297086136 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a000cd80 con 0x7f30a4100b50 2024-08-02T17:58:57.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.314+0000 7f309b7fe640 1 -- 172.21.15.69:0/1297086136 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a0010910 con 0x7f30a4100b50 2024-08-02T17:58:57.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.314+0000 7f30ac2e4640 1 -- 172.21.15.69:0/1297086136 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a406d070 con 0x7f30a4100b50 2024-08-02T17:58:57.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.314+0000 7f30ac2e4640 1 -- 172.21.15.69:0/1297086136 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a4071190 con 0x7f30a4100b50 2024-08-02T17:58:57.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.314+0000 7f309b7fe640 1 -- 172.21.15.69:0/1297086136 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a0018990 con 0x7f30a4100b50 2024-08-02T17:58:57.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.317+0000 7f309b7fe640 1 -- 172.21.15.69:0/1297086136 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f30a0018b30 con 0x7f30a4100b50 2024-08-02T17:58:57.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.318+0000 7f309b7fe640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30900a2ec0 0x7f30900a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.321+0000 7f30ae6e2740 1 Processor -- start 2024-08-02T17:58:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.321+0000 7f309b7fe640 1 -- 172.21.15.69:0/1297086136 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f30a00d0c70 con 0x7f30a4100b50 2024-08-02T17:58:57.324 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.321+0000 7f30ae6e2740 1 -- start start 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b57910 0x55b1a5b61c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b621d0 0x55b1a5b645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b1a5abca40 con 0x55b1a5b621d0 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b1a5a549d0 con 0x55b1a5b57910 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30aa080640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30aa080640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59388/0 (socket says 172.21.15.69:59388) 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30aa080640 1 -- 172.21.15.69:0/3216061900 learned_addr learned my addr 172.21.15.69:0/3216061900 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:57.325 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.322+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b1a5ab8a00 con 0x55b1a5b56ff0 2024-08-02T17:58:57.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.325+0000 7f30a987f640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30900a2ec0 0x7f30900a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.325+0000 7f30a987f640 1 --2- 172.21.15.69:0/3216061900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b57910 0x55b1a5b61c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.326+0000 7f30a987f640 1 -- 172.21.15.69:0/3216061900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b56ff0 msgr2=0x55b1a5b573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:57.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.326+0000 7f30a987f640 1 --2- 172.21.15.69:0/3216061900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.326+0000 7f30a987f640 1 -- 172.21.15.69:0/3216061900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b621d0 msgr2=0x55b1a5b645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:57.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.326+0000 7f30a987f640 1 --2- 172.21.15.69:0/3216061900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b621d0 0x55b1a5b645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.326+0000 7f30a987f640 1 -- 172.21.15.69:0/3216061900 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a5a5b650 con 0x55b1a5b57910 2024-08-02T17:58:57.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.326+0000 7f30a987f640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30900a2ec0 0x7f30900a5380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f30940314e0 tx=0x7f3094005d50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:57.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.330+0000 7f30a987f640 1 --2- 172.21.15.69:0/3216061900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b57910 0x55b1a5b61c90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f309402aa70 tx=0x7f3094043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f2df013e5240031 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.334+0000 7f3098dea640 1 -- 172.21.15.69:0/3216061900 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094004110 con 0x55b1a5b57910 2024-08-02T17:58:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.334+0000 7f3098dea640 1 -- 172.21.15.69:0/3216061900 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094005500 con 0x55b1a5b57910 2024-08-02T17:58:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.334+0000 7f3098dea640 1 -- 172.21.15.69:0/3216061900 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094004d80 con 0x55b1a5b57910 2024-08-02T17:58:57.341 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.335+0000 7f30ae6e2740 1 -- 172.21.15.69:0/3216061900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b57910 msgr2=0x55b1a5b61c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.335+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/3216061900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b57910 0x55b1a5b61c90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f309402aa70 tx=0x7f3094043000 comp rx=0 tx=0).stop 2024-08-02T17:58:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.339+0000 7f30ae6e2740 1 -- 172.21.15.69:0/3216061900 shutdown_connections 2024-08-02T17:58:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.339+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/3216061900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b621d0 0x55b1a5b645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.339+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/3216061900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b57910 0x55b1a5b61c90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.339+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/3216061900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.339+0000 7f30ae6e2740 1 -- 172.21.15.69:0/3216061900 >> 172.21.15.69:0/3216061900 conn(0x55b1a5b53580 msgr2=0x55b1a5b53960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:57.342 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.340+0000 7f30ae6e2740 1 -- 172.21.15.69:0/3216061900 shutdown_connections 2024-08-02T17:58:57.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.340+0000 7f30ae6e2740 1 -- 172.21.15.69:0/3216061900 wait complete. 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.340+0000 7f30ae6e2740 1 -- start start 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30ae6e2740 1 Processor -- start 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b5f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b1a5b64d40 con 0x55b1a5b56ff0 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b1a5a77b20 con 0x55b1a5b621d0 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b1a5b64ec0 con 0x55b1a5b57910 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30ae6e2740 10 client.0 _ll_register_callbacks cb 0x55b1a5a75bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30aa881640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30aa881640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49758/0 (socket says 172.21.15.69:49758) 2024-08-02T17:58:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.341+0000 7f30aa881640 1 -- 172.21.15.69:0/1797469999 learned_addr learned my addr 172.21.15.69:0/1797469999 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:57.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.342+0000 7f30a987f640 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b5f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.345+0000 7f30aa881640 1 -- 172.21.15.69:0/1797469999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 msgr2=0x55b1a5b5f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.345+0000 7f30aa881640 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b5f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.345+0000 7f30aa881640 1 -- 172.21.15.69:0/1797469999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 msgr2=0x55b1a5b5f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.345+0000 7f30aa881640 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.345+0000 7f30aa881640 1 -- 172.21.15.69:0/1797469999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a582e3c0 con 0x55b1a5b621d0 2024-08-02T17:58:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.349+0000 7f30aa881640 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5fb00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f309c010820 tx=0x7f309c010ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=185ba667f0a5acb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.349+0000 7f308dffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:58:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.350+0000 7f308dffb640 1 -- 172.21.15.69:0/1797469999 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c017070 con 0x55b1a5b621d0 2024-08-02T17:58:57.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.350+0000 7f30ae6e2740 1 -- 172.21.15.69:0/1797469999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3094005500 con 0x55b1a5b621d0 2024-08-02T17:58:57.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.350+0000 7f308dffb640 1 -- 172.21.15.69:0/1797469999 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f309c004d20 con 0x55b1a5b621d0 2024-08-02T17:58:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.355+0000 7f308dffb640 1 -- 172.21.15.69:0/1797469999 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f309c00eac0 con 0x55b1a5b621d0 2024-08-02T17:58:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.356+0000 7f30ae6e2740 10 client.27896 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:57.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.356+0000 7f30ae6e2740 1 -- 172.21.15.69:0/1797469999 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b1a5b929c0 con 0x55b1a5b621d0 2024-08-02T17:58:57.357 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:57 smithi069 ceph-mon[32746]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.3 MiB/s wr, 328 op/s 2024-08-02T17:58:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:57 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:58:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:57 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:58:57.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:57 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.360+0000 7f308dffb640 1 -- 172.21.15.69:0/1797469999 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094005500 con 0x55b1a5b621d0 2024-08-02T17:58:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.360+0000 7f308dffb640 1 -- 172.21.15.69:0/1797469999 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f309c004410 con 0x55b1a5b621d0 2024-08-02T17:58:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.360+0000 7f30ae6e2740 10 client.27896 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.360+0000 7f30ae6e2740 10 client.27896 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:58:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.360+0000 7f30ae6e2740 20 client.27896 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.360+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55b1a5b94350 0x55b1a5b967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.360+0000 7f30ae6e2740 4 client.27896 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:57.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.360+0000 7f30ae6e2740 1 -- 172.21.15.69:0/1797469999 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1a5ac9fb0 con 0x55b1a5b94350 2024-08-02T17:58:57.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.364+0000 7f30aa080640 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55b1a5b94350 0x55b1a5b967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.386+0000 7f308dffb640 10 client.27896 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:58:57.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.386+0000 7f30aa080640 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55b1a5b94350 0x55b1a5b967f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef146fce112282eb server_cookie=1f1de86d4c9599b9 in_seq=0 out_seq=0 2024-08-02T17:58:57.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.391+0000 7f308dffb640 1 -- 172.21.15.69:0/1797469999 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b1a5ac9fb0 con 0x55b1a5b94350 2024-08-02T17:58:57.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.391+0000 7f308dffb640 10 client.27896 handle_command_reply: tid=0 2024-08-02T17:58:57.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.391+0000 7f30ae6e2740 1 client.27896 shutdown 2024-08-02T17:58:57.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.391+0000 7f30ae6e2740 1 -- 172.21.15.69:0/1797469999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b621d0 msgr2=0x55b1a5b5fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.391+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5fb00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f309c010820 tx=0x7f309c010ce0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.397+0000 7f30ae6e2740 1 -- 172.21.15.69:0/1797469999 shutdown_connections 2024-08-02T17:58:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.397+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55b1a5b94350 0x55b1a5b967f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.397+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5fb00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.397+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b5f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.397+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/1797469999 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b5f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.397+0000 7f30ae6e2740 1 -- 172.21.15.69:0/1797469999 >> 172.21.15.69:0/1797469999 conn(0x55b1a5b53580 msgr2=0x55b1a5b592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:57.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.397+0000 7f308dffb640 0 client.27896 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:58:57.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.402+0000 7f30ae6e2740 1 -- 172.21.15.69:0/1797469999 shutdown_connections 2024-08-02T17:58:57.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.406+0000 7f30ae6e2740 1 -- 172.21.15.69:0/1797469999 wait complete. 2024-08-02T17:58:57.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.406+0000 7f30ae6e2740 20 client.27896 trim_cache size 0 max 16384 2024-08-02T17:58:57.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.432+0000 7f30ae6e2740 1 Processor -- start 2024-08-02T17:58:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.432+0000 7f30ae6e2740 1 -- start start 2024-08-02T17:58:57.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b9c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b9c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3088012a50 con 0x55b1a5b621d0 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3088010320 con 0x55b1a5b56ff0 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f308800e170 con 0x55b1a5b57910 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30a987f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b9c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30a987f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b9c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59410/0 (socket says 172.21.15.69:59410) 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30a987f640 1 -- 172.21.15.69:0/850878592 learned_addr learned my addr 172.21.15.69:0/850878592 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30a987f640 1 -- 172.21.15.69:0/850878592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b56ff0 msgr2=0x55b1a5b9c100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30a987f640 1 --2- 172.21.15.69:0/850878592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b9c100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30a987f640 1 -- 172.21.15.69:0/850878592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b621d0 msgr2=0x55b1a5b5d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:57.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30a987f640 1 --2- 172.21.15.69:0/850878592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.433+0000 7f30a987f640 1 -- 172.21.15.69:0/850878592 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1a5b5d5d0 con 0x55b1a5b57910 2024-08-02T17:58:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.435+0000 7f30a987f640 1 --2- 172.21.15.69:0/850878592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b9c640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f309402aa70 tx=0x7f3094005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e4cc1eb49c44a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.436+0000 7f308ffff640 1 -- 172.21.15.69:0/850878592 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094057030 con 0x55b1a5b57910 2024-08-02T17:58:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.436+0000 7f308ffff640 1 -- 172.21.15.69:0/850878592 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094005050 con 0x55b1a5b57910 2024-08-02T17:58:57.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.436+0000 7f308ffff640 1 -- 172.21.15.69:0/850878592 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094004820 con 0x55b1a5b57910 2024-08-02T17:58:57.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.437+0000 7f30ae6e2740 1 -- 172.21.15.69:0/850878592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 msgr2=0x55b1a5b9c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.437+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/850878592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b9c640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f309402aa70 tx=0x7f3094005fd0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.437+0000 7f30ae6e2740 1 -- 172.21.15.69:0/850878592 shutdown_connections 2024-08-02T17:58:57.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.437+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/850878592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b621d0 0x55b1a5b5d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.437+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/850878592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b57910 0x55b1a5b9c640 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f309402aa70 tx=0x7f3094005fd0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.437+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/850878592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b9c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.437+0000 7f30ae6e2740 1 -- 172.21.15.69:0/850878592 >> 172.21.15.69:0/850878592 conn(0x55b1a5b53580 msgr2=0x55b1a5b4e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:57.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.441+0000 7f30ae6e2740 1 -- 172.21.15.69:0/850878592 shutdown_connections 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.445+0000 7f30ae6e2740 1 -- 172.21.15.69:0/850878592 wait complete. 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.445+0000 7f30ae6e2740 1 -- start start 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.445+0000 7f30ae6e2740 1 Processor -- start 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b6a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b621d0 0x55b1a5b6a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30ae6e2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b5b6b0 0x55b1a5b5bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f308800e7e0 con 0x55b1a5b56ff0 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f308800fc50 con 0x55b1a5b5b6b0 2024-08-02T17:58:57.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30ae6e2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f308800eeb0 con 0x55b1a5b621d0 2024-08-02T17:58:57.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30ae6e2740 10 client.0 _ll_register_callbacks cb 0x7f309c0099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:57.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30a987f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b621d0 0x55b1a5b6a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30a987f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b621d0 0x55b1a5b6a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59424/0 (socket says 172.21.15.69:59424) 2024-08-02T17:58:57.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30a987f640 1 -- 172.21.15.69:0/313115103 learned_addr learned my addr 172.21.15.69:0/313115103 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:57.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.446+0000 7f30aa881640 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b5b6b0 0x55b1a5b5bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.448+0000 7f30aa080640 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b6a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.448+0000 7f30a987f640 1 -- 172.21.15.69:0/313115103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b5b6b0 msgr2=0x55b1a5b5bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.448+0000 7f30a987f640 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b5b6b0 0x55b1a5b5bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.448+0000 7f30a987f640 1 -- 172.21.15.69:0/313115103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 msgr2=0x55b1a5b6a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:57.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.448+0000 7f30a987f640 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b6a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.448+0000 7f30a987f640 1 -- 172.21.15.69:0/313115103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1a5b5c070 con 0x55b1a5b621d0 2024-08-02T17:58:57.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.449+0000 7f30a987f640 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b621d0 0x55b1a5b6a6b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f309402a9a0 tx=0x7f3094005430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfcf4370c16fc7ba server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:57.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.449+0000 7f308dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:57.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.450+0000 7f308dffb640 1 -- 172.21.15.69:0/313115103 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094061070 con 0x55b1a5b621d0 2024-08-02T17:58:57.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.451+0000 7f308dffb640 1 -- 172.21.15.69:0/313115103 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f3094004320 con 0x55b1a5b621d0 2024-08-02T17:58:57.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.454+0000 7f30ae6e2740 1 -- 172.21.15.69:0/313115103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3094005050 con 0x55b1a5b621d0 2024-08-02T17:58:57.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.455+0000 7f308dffb640 1 -- 172.21.15.69:0/313115103 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f30940046c0 con 0x55b1a5b621d0 2024-08-02T17:58:57.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.456+0000 7f30ae6e2740 10 client.27691 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:57.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.456+0000 7f30ae6e2740 1 -- 172.21.15.69:0/313115103 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b1a5bfed40 con 0x55b1a5b621d0 2024-08-02T17:58:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.459+0000 7f308dffb640 1 -- 172.21.15.69:0/313115103 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094005050 con 0x55b1a5b621d0 2024-08-02T17:58:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.459+0000 7f308dffb640 1 -- 172.21.15.69:0/313115103 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f309403a040 con 0x55b1a5b621d0 2024-08-02T17:58:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.459+0000 7f30ae6e2740 10 client.27691 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.459+0000 7f30ae6e2740 10 client.27691 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:58:57.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.459+0000 7f30ae6e2740 20 client.27691 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.459+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55b1a5bff2b0 0x55b1a5c016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.459+0000 7f30ae6e2740 4 client.27691 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:58:57.461 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.459+0000 7f30ae6e2740 1 -- 172.21.15.69:0/313115103 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b1a5ac44c0 con 0x55b1a5bff2b0 2024-08-02T17:58:57.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.466+0000 7f30aa080640 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55b1a5bff2b0 0x55b1a5c016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:57.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.469+0000 7f308dffb640 10 client.27691 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:58:57.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.469+0000 7f30aa080640 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55b1a5bff2b0 0x55b1a5c016d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=184bca374e63351e server_cookie=36a90256490dabdf in_seq=0 out_seq=0 2024-08-02T17:58:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.469+0000 7f308dffb640 1 -- 172.21.15.69:0/313115103 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b1a5ac44c0 con 0x55b1a5bff2b0 2024-08-02T17:58:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.469+0000 7f308dffb640 10 client.27691 handle_command_reply: tid=0 2024-08-02T17:58:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.469+0000 7f30ae6e2740 1 client.27691 shutdown 2024-08-02T17:58:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f30ae6e2740 1 -- 172.21.15.69:0/313115103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b621d0 msgr2=0x55b1a5b6a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b621d0 0x55b1a5b6a6b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f309402a9a0 tx=0x7f3094005430 comp rx=0 tx=0).stop 2024-08-02T17:58:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f30ae6e2740 1 -- 172.21.15.69:0/313115103 shutdown_connections 2024-08-02T17:58:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55b1a5bff2b0 0x55b1a5c016d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b1a5b5b6b0 0x55b1a5b5bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b1a5b621d0 0x55b1a5b6a6b0 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f309402a9a0 tx=0x7f3094005430 comp rx=0 tx=0).stop 2024-08-02T17:58:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f30ae6e2740 1 --2- 172.21.15.69:0/313115103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b1a5b56ff0 0x55b1a5b6a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f30ae6e2740 1 -- 172.21.15.69:0/313115103 >> 172.21.15.69:0/313115103 conn(0x55b1a5b53580 msgr2=0x55b1a5b5d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:57.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.470+0000 7f308dffb640 0 client.27691 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:58:57.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.473+0000 7f30ae6e2740 1 -- 172.21.15.69:0/313115103 shutdown_connections 2024-08-02T17:58:57.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.474+0000 7f30ae6e2740 1 -- 172.21.15.69:0/313115103 wait complete. 2024-08-02T17:58:57.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.474+0000 7f30ae6e2740 20 client.27691 trim_cache size 0 max 16384 2024-08-02T17:58:57.476 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 -- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30900a2ec0 msgr2=0x7f30900a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30900a2ec0 0x7f30900a5380 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f30940314e0 tx=0x7f3094005d50 comp rx=0 tx=0).stop 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 -- 172.21.15.69:0/1297086136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4100b50 msgr2=0x7f30a4073490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4100b50 0x7f30a4073490 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f30a000e9f0 tx=0x7f30a000eeb0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 -- 172.21.15.69:0/1297086136 shutdown_connections 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30a4102c20 0x7f30a406c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30900a2ec0 0x7f30900a5380 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30a41022a0 0x7f30a40739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 --2- 172.21.15.69:0/1297086136 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30a4100b50 0x7f30a4073490 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:57.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.475+0000 7f30ac2e4640 1 -- 172.21.15.69:0/1297086136 >> 172.21.15.69:0/1297086136 conn(0x7f30a40fc500 msgr2=0x7f30a4105410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:57.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.476+0000 7f30ac2e4640 1 -- 172.21.15.69:0/1297086136 shutdown_connections 2024-08-02T17:58:57.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:57.476+0000 7f30ac2e4640 1 -- 172.21.15.69:0/1297086136 wait complete. 2024-08-02T17:58:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:57 smithi083 ceph-mon[37031]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.3 MiB/s wr, 328 op/s 2024-08-02T17:58:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:57 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:58:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:57 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:58:57.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:57 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:57.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:57 smithi077 ceph-mon[37108]: pgmap v390: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.3 MiB/s wr, 328 op/s 2024-08-02T17:58:57.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:57 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:58:57.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:57 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:58:57.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:57 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:58:58.227 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:58:58.727 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:58:59.196 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:58:59 smithi069 ceph-mon[32746]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-08-02T17:58:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.348+0000 7efc30b39640 1 -- 172.21.15.69:0/2336991559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c100870 msgr2=0x7efc2c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.348+0000 7efc30b39640 1 --2- 172.21.15.69:0/2336991559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c100870 0x7efc2c100c50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009900 tx=0x7efc1c02a490 comp rx=0 tx=0).stop 2024-08-02T17:58:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.348+0000 7efc30b39640 1 -- 172.21.15.69:0/2336991559 shutdown_connections 2024-08-02T17:58:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.348+0000 7efc30b39640 1 --2- 172.21.15.69:0/2336991559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c105c80 0x7efc2c106110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.348+0000 7efc30b39640 1 --2- 172.21.15.69:0/2336991559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc2c101190 0x7efc2c105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.348+0000 7efc30b39640 1 --2- 172.21.15.69:0/2336991559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c100870 0x7efc2c100c50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.348+0000 7efc30b39640 1 -- 172.21.15.69:0/2336991559 >> 172.21.15.69:0/2336991559 conn(0x7efc2c0fc500 msgr2=0x7efc2c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.351+0000 7efc30b39640 1 -- 172.21.15.69:0/2336991559 shutdown_connections 2024-08-02T17:58:59.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.351+0000 7efc30b39640 1 -- 172.21.15.69:0/2336991559 wait complete. 2024-08-02T17:58:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.352+0000 7efc30b39640 1 Processor -- start 2024-08-02T17:58:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.352+0000 7efc30b39640 1 -- start start 2024-08-02T17:58:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.352+0000 7efc30b39640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc2c100870 0x7efc2c073690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc30b39640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c101190 0x7efc2c073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc2a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc2c100870 0x7efc2c073690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc2a59c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc2c100870 0x7efc2c073690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:49788/0 (socket says 172.21.15.69:49788) 2024-08-02T17:58:59.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc29d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c101190 0x7efc2c073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc29d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c101190 0x7efc2c073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59448/0 (socket says 172.21.15.69:59448) 2024-08-02T17:58:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc29d9b640 1 -- 172.21.15.69:0/3374100907 learned_addr learned my addr 172.21.15.69:0/3374100907 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc30b39640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c105c80 0x7efc2c06c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1182e0 con 0x7efc2c105c80 2024-08-02T17:58:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.353+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c118160 con 0x7efc2c100870 2024-08-02T17:58:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.354+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c118460 con 0x7efc2c101190 2024-08-02T17:58:59.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.354+0000 7efc2ad9d640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c105c80 0x7efc2c06c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.355+0000 7efc2a59c640 1 -- 172.21.15.69:0/3374100907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c101190 msgr2=0x7efc2c073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.355+0000 7efc2a59c640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c101190 0x7efc2c073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.355+0000 7efc2a59c640 1 -- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c105c80 msgr2=0x7efc2c06c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.355+0000 7efc2a59c640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c105c80 0x7efc2c06c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.355+0000 7efc2a59c640 1 -- 172.21.15.69:0/3374100907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2c06d1a0 con 0x7efc2c100870 2024-08-02T17:58:59.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.355+0000 7efc2a59c640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc2c100870 0x7efc2c073690 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efc1c033040 tx=0x7efc1c005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.357+0000 7efc2ad9d640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c105c80 0x7efc2c06c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:58:59.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.358+0000 7efc1b7fe640 1 -- 172.21.15.69:0/3374100907 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c040030 con 0x7efc2c100870 2024-08-02T17:58:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.358+0000 7efc1b7fe640 1 -- 172.21.15.69:0/3374100907 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c03c920 con 0x7efc2c100870 2024-08-02T17:58:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.358+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2c06d430 con 0x7efc2c100870 2024-08-02T17:58:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.358+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2c071390 con 0x7efc2c100870 2024-08-02T17:58:59.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.358+0000 7efc1b7fe640 1 -- 172.21.15.69:0/3374100907 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c031a40 con 0x7efc2c100870 2024-08-02T17:58:59.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.362+0000 7efc32f37740 1 Processor -- start 2024-08-02T17:58:59.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.363+0000 7efc32f37740 1 -- start start 2024-08-02T17:58:59.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.363+0000 7efc32f37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e14081c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.363+0000 7efc32f37740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140826c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.363+0000 7efc32f37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e1408295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.363+0000 7efc32f37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e140781a40 con 0x55e1408271d0 2024-08-02T17:58:59.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.363+0000 7efc32f37740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e1407199d0 con 0x55e14081c910 2024-08-02T17:58:59.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.363+0000 7efc32f37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e14077da00 con 0x55e14081bff0 2024-08-02T17:58:59.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc1b7fe640 1 -- 172.21.15.69:0/3374100907 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7efc1c043030 con 0x7efc2c100870 2024-08-02T17:58:59.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc2ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e14081c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc29d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e1408295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc29d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e1408295c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33854/0 (socket says 172.21.15.69:33854) 2024-08-02T17:58:59.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc29d9b640 1 -- 172.21.15.69:0/749261427 learned_addr learned my addr 172.21.15.69:0/749261427 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc2a59c640 1 --2- 172.21.15.69:0/749261427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140826c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc2a59c640 1 -- 172.21.15.69:0/749261427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 msgr2=0x55e14081c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc2a59c640 1 --2- 172.21.15.69:0/749261427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e14081c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc2a59c640 1 -- 172.21.15.69:0/749261427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 msgr2=0x55e1408295c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc2a59c640 1 --2- 172.21.15.69:0/749261427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e1408295c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.366+0000 7efc2a59c640 1 -- 172.21.15.69:0/749261427 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e140720650 con 0x55e14081c910 2024-08-02T17:58:59.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.367+0000 7efc1b7fe640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efc140a2ec0 0x7efc140a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.368+0000 7efc1b7fe640 1 -- 172.21.15.69:0/3374100907 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7efc1c0efc50 con 0x7efc2c100870 2024-08-02T17:58:59.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.369+0000 7efc2a59c640 1 --2- 172.21.15.69:0/749261427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140826c90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0ecf50 tx=0x7efc1c0f3e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=543d2fa3a913151 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.370+0000 7efc2ad9d640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efc140a2ec0 0x7efc140a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.370+0000 7efc18ff9640 1 -- 172.21.15.69:0/749261427 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c04a070 con 0x55e14081c910 2024-08-02T17:58:59.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.370+0000 7efc18ff9640 1 -- 172.21.15.69:0/749261427 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c0bc420 con 0x55e14081c910 2024-08-02T17:58:59.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.370+0000 7efc18ff9640 1 -- 172.21.15.69:0/749261427 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0bbaf0 con 0x55e14081c910 2024-08-02T17:58:59.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.371+0000 7efc32f37740 1 -- 172.21.15.69:0/749261427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 msgr2=0x55e140826c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.371+0000 7efc32f37740 1 --2- 172.21.15.69:0/749261427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140826c90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0ecf50 tx=0x7efc1c0f3e10 comp rx=0 tx=0).stop 2024-08-02T17:58:59.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.371+0000 7efc32f37740 1 -- 172.21.15.69:0/749261427 shutdown_connections 2024-08-02T17:58:59.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.371+0000 7efc32f37740 1 --2- 172.21.15.69:0/749261427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e1408295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.371+0000 7efc32f37740 1 --2- 172.21.15.69:0/749261427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140826c90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.371+0000 7efc32f37740 1 --2- 172.21.15.69:0/749261427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e14081c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.371+0000 7efc32f37740 1 -- 172.21.15.69:0/749261427 >> 172.21.15.69:0/749261427 conn(0x55e140818580 msgr2=0x55e140818960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.371+0000 7efc32f37740 1 -- 172.21.15.69:0/749261427 shutdown_connections 2024-08-02T17:58:59.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.376+0000 7efc32f37740 1 -- 172.21.15.69:0/749261427 wait complete. 2024-08-02T17:58:59.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.376+0000 7efc32f37740 1 -- start start 2024-08-02T17:58:59.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.376+0000 7efc32f37740 1 Processor -- start 2024-08-02T17:58:59.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.380+0000 7efc2ad9d640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efc140a2ec0 0x7efc140a5380 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc24005fd0 tx=0x7efc24009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.381+0000 7efc32f37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e140826c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.381+0000 7efc32f37740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140823fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.381+0000 7efc32f37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e140824750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.381+0000 7efc32f37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e140829b60 con 0x55e1408271d0 2024-08-02T17:58:59.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.381+0000 7efc32f37740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e14073cb20 con 0x55e14081c910 2024-08-02T17:58:59.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.381+0000 7efc32f37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e140829ce0 con 0x55e14081bff0 2024-08-02T17:58:59.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.381+0000 7efc32f37740 10 client.0 _ll_register_callbacks cb 0x55e14073abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:59.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.384+0000 7efc2a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e140826c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.384+0000 7efc2a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e140826c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59466/0 (socket says 172.21.15.69:59466) 2024-08-02T17:58:59.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.384+0000 7efc2a59c640 1 -- 172.21.15.69:0/812115738 learned_addr learned my addr 172.21.15.69:0/812115738 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.384+0000 7efc29d9b640 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140823fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.385+0000 7efc2a59c640 1 -- 172.21.15.69:0/812115738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 msgr2=0x55e140823fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.385+0000 7efc2a59c640 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140823fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.385+0000 7efc2a59c640 1 -- 172.21.15.69:0/812115738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 msgr2=0x55e140824750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.385+0000 7efc2a59c640 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e140824750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.385+0000 7efc2a59c640 1 -- 172.21.15.69:0/812115738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1404f33c0 con 0x55e14081bff0 2024-08-02T17:58:59.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.387+0000 7efc2a59c640 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e140826c70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc1c02a9b0 tx=0x7efc1c005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77ac4304be18b172 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.387+0000 7efc11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:59.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.391+0000 7efc11ffb640 1 -- 172.21.15.69:0/812115738 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c04a070 con 0x55e14081bff0 2024-08-02T17:58:59.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.391+0000 7efc32f37740 1 -- 172.21.15.69:0/812115738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc1c0bc420 con 0x55e14081bff0 2024-08-02T17:58:59.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.391+0000 7efc11ffb640 1 -- 172.21.15.69:0/812115738 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7efc1c0b8390 con 0x55e14081bff0 2024-08-02T17:58:59.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.392+0000 7efc11ffb640 1 -- 172.21.15.69:0/812115738 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc1c005630 con 0x55e14081bff0 2024-08-02T17:58:59.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.392+0000 7efc32f37740 10 client.27700 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:59.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.392+0000 7efc32f37740 1 -- 172.21.15.69:0/812115738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e140829b60 con 0x55e14081bff0 2024-08-02T17:58:59.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.393+0000 7efc11ffb640 1 -- 172.21.15.69:0/812115738 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c0bc420 con 0x55e14081bff0 2024-08-02T17:58:59.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.393+0000 7efc11ffb640 1 -- 172.21.15.69:0/812115738 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efc1c0c4be0 con 0x55e14081bff0 2024-08-02T17:58:59.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.394+0000 7efc32f37740 10 client.27700 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:59.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.394+0000 7efc32f37740 10 client.27700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:59.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.394+0000 7efc32f37740 20 client.27700 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:59.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.394+0000 7efc32f37740 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e140836860 0x55e140838cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.394+0000 7efc32f37740 4 client.27700 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:59.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.394+0000 7efc32f37740 1 -- 172.21.15.69:0/812115738 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e14078efb0 con 0x55e140836860 2024-08-02T17:58:59.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.395+0000 7efc29d9b640 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e140836860 0x55e140838cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.400+0000 7efc29d9b640 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e140836860 0x55e140838cc0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9059777101fdb946 server_cookie=132932701eb839ae in_seq=0 out_seq=0 2024-08-02T17:58:59.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.400+0000 7efc11ffb640 10 client.27700 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:59.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.401+0000 7efc11ffb640 1 -- 172.21.15.69:0/812115738 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e14078efb0 con 0x55e140836860 2024-08-02T17:58:59.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.401+0000 7efc11ffb640 10 client.27700 handle_command_reply: tid=0 2024-08-02T17:58:59.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.401+0000 7efc32f37740 1 client.27700 shutdown 2024-08-02T17:58:59.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.405+0000 7efc32f37740 1 -- 172.21.15.69:0/812115738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 msgr2=0x55e140826c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.405+0000 7efc32f37740 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e140826c70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc1c02a9b0 tx=0x7efc1c005c90 comp rx=0 tx=0).stop 2024-08-02T17:58:59.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.407+0000 7efc32f37740 1 -- 172.21.15.69:0/812115738 shutdown_connections 2024-08-02T17:58:59.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.407+0000 7efc32f37740 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e140824750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.407+0000 7efc32f37740 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e140836860 0x55e140838cc0 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.407+0000 7efc32f37740 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081c910 0x55e140823fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.407+0000 7efc32f37740 1 --2- 172.21.15.69:0/812115738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081bff0 0x55e140826c70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.407+0000 7efc32f37740 1 -- 172.21.15.69:0/812115738 >> 172.21.15.69:0/812115738 conn(0x55e140818580 msgr2=0x55e14081e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.407+0000 7efc11ffb640 0 client.27700 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:59.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.412+0000 7efc32f37740 1 -- 172.21.15.69:0/812115738 shutdown_connections 2024-08-02T17:58:59.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.413+0000 7efc32f37740 1 -- 172.21.15.69:0/812115738 wait complete. 2024-08-02T17:58:59.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.413+0000 7efc32f37740 20 client.27700 trim_cache size 0 max 16384 2024-08-02T17:58:59.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.438+0000 7efc32f37740 1 Processor -- start 2024-08-02T17:58:59.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.439+0000 7efc32f37740 1 -- start start 2024-08-02T17:58:59.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.439+0000 7efc32f37740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e140823f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.439+0000 7efc32f37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081c910 0x55e14081ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.439+0000 7efc32f37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e140838800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.439+0000 7efc32f37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c012a60 con 0x55e1408271d0 2024-08-02T17:58:59.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.439+0000 7efc32f37740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c010330 con 0x55e14081bff0 2024-08-02T17:58:59.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.439+0000 7efc32f37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c00e180 con 0x55e14081c910 2024-08-02T17:58:59.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.440+0000 7efc2ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e140838800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.440+0000 7efc2ad9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e140838800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33866/0 (socket says 172.21.15.69:33866) 2024-08-02T17:58:59.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.440+0000 7efc2ad9d640 1 -- 172.21.15.69:0/724808507 learned_addr learned my addr 172.21.15.69:0/724808507 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.444+0000 7efc2a59c640 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e140823f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.444+0000 7efc29d9b640 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081c910 0x55e14081ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.446+0000 7efc2a59c640 1 -- 172.21.15.69:0/724808507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081c910 msgr2=0x55e14081ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.446+0000 7efc2a59c640 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081c910 0x55e14081ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.446+0000 7efc2a59c640 1 -- 172.21.15.69:0/724808507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 msgr2=0x55e140838800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.446+0000 7efc2a59c640 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e140838800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.446+0000 7efc2a59c640 1 -- 172.21.15.69:0/724808507 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14081c740 con 0x55e14081bff0 2024-08-02T17:58:59.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.449+0000 7efc2a59c640 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e140823f90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009900 tx=0x7efc1c0f3e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1739258174b67568 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.453+0000 7efc13fff640 1 -- 172.21.15.69:0/724808507 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c04a070 con 0x55e14081bff0 2024-08-02T17:58:59.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.453+0000 7efc13fff640 1 -- 172.21.15.69:0/724808507 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c0f3040 con 0x55e14081bff0 2024-08-02T17:58:59.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.454+0000 7efc13fff640 1 -- 172.21.15.69:0/724808507 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c03cad0 con 0x55e14081bff0 2024-08-02T17:58:59.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.455+0000 7efc32f37740 1 -- 172.21.15.69:0/724808507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 msgr2=0x55e140823f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.455+0000 7efc32f37740 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e140823f90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009900 tx=0x7efc1c0f3e10 comp rx=0 tx=0).stop 2024-08-02T17:58:59.457 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.456+0000 7efc32f37740 1 -- 172.21.15.69:0/724808507 shutdown_connections 2024-08-02T17:58:59.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.456+0000 7efc32f37740 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e1408271d0 0x55e140838800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.456+0000 7efc32f37740 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e14081c910 0x55e14081ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.456+0000 7efc32f37740 1 --2- 172.21.15.69:0/724808507 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e140823f90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.456+0000 7efc32f37740 1 -- 172.21.15.69:0/724808507 >> 172.21.15.69:0/724808507 conn(0x55e140818580 msgr2=0x55e140823880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.457+0000 7efc32f37740 1 -- 172.21.15.69:0/724808507 shutdown_connections 2024-08-02T17:58:59.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.457+0000 7efc32f37740 1 -- 172.21.15.69:0/724808507 wait complete. 2024-08-02T17:58:59.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.457+0000 7efc32f37740 1 -- start start 2024-08-02T17:58:59.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.457+0000 7efc32f37740 1 Processor -- start 2024-08-02T17:58:59.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.457+0000 7efc32f37740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e14082ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc32f37740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e14081c910 0x55e14082f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc32f37740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1408271d0 0x55e140820230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc32f37740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c00e7f0 con 0x55e14081c910 2024-08-02T17:58:59.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc32f37740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c00fc60 con 0x55e14081bff0 2024-08-02T17:58:59.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc32f37740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c00eec0 con 0x55e1408271d0 2024-08-02T17:58:59.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc32f37740 10 client.0 _ll_register_callbacks cb 0x7efc1c0f1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:59.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc2ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1408271d0 0x55e140820230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc2ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1408271d0 0x55e140820230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59476/0 (socket says 172.21.15.69:59476) 2024-08-02T17:58:59.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc2ad9d640 1 -- 172.21.15.69:0/1896145365 learned_addr learned my addr 172.21.15.69:0/1896145365 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.460 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.458+0000 7efc2a59c640 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e14082ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.462+0000 7efc2ad9d640 1 -- 172.21.15.69:0/1896145365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 msgr2=0x55e14082ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.462+0000 7efc2ad9d640 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e14082ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.462+0000 7efc2ad9d640 1 -- 172.21.15.69:0/1896145365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e14081c910 msgr2=0x55e14082f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.463 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.462+0000 7efc2ad9d640 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e14081c910 0x55e14082f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.462+0000 7efc2ad9d640 1 -- 172.21.15.69:0/1896145365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e140821de0 con 0x55e1408271d0 2024-08-02T17:58:59.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.465+0000 7efc2ad9d640 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1408271d0 0x55e140820230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efc24009d00 tx=0x7efc24012c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772eb71fdb3811bf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.467+0000 7efc11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.467+0000 7efc11ffb640 1 -- 172.21.15.69:0/1896145365 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24011cf0 con 0x55e1408271d0 2024-08-02T17:58:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.467+0000 7efc32f37740 1 -- 172.21.15.69:0/1896145365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc1c0f3040 con 0x55e1408271d0 2024-08-02T17:58:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.467+0000 7efc11ffb640 1 -- 172.21.15.69:0/1896145365 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7efc24020430 con 0x55e1408271d0 2024-08-02T17:58:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.467+0000 7efc11ffb640 1 -- 172.21.15.69:0/1896145365 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efc24012e50 con 0x55e1408271d0 2024-08-02T17:58:59.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.468+0000 7efc32f37740 10 client.27706 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.468+0000 7efc32f37740 1 -- 172.21.15.69:0/1896145365 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1408a09f0 con 0x55e1408271d0 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.468+0000 7efc11ffb640 1 -- 172.21.15.69:0/1896145365 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c0f3040 con 0x55e1408271d0 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.469+0000 7efc11ffb640 1 -- 172.21.15.69:0/1896145365 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efc2400f040 con 0x55e1408271d0 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.469+0000 7efc32f37740 10 client.27706 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.469+0000 7efc32f37740 10 client.27706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.469+0000 7efc32f37740 20 client.27706 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.469+0000 7efc32f37740 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1408a1150 0x55e1408a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.469+0000 7efc32f37740 4 client.27706 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.469+0000 7efc32f37740 1 -- 172.21.15.69:0/1896145365 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1407894c0 con 0x55e1408a1150 2024-08-02T17:58:59.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.469+0000 7efc2a59c640 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1408a1150 0x55e1408a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.473+0000 7efc2a59c640 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1408a1150 0x55e1408a3590 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecb7c01c746fe2b0 server_cookie=78b53bf32212b2f3 in_seq=0 out_seq=0 2024-08-02T17:58:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.477+0000 7efc11ffb640 10 client.27706 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:59.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.477+0000 7efc11ffb640 1 -- 172.21.15.69:0/1896145365 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1407894c0 con 0x55e1408a1150 2024-08-02T17:58:59.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.477+0000 7efc11ffb640 10 client.27706 handle_command_reply: tid=0 2024-08-02T17:58:59.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.477+0000 7efc32f37740 1 client.27706 shutdown 2024-08-02T17:58:59.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc32f37740 1 -- 172.21.15.69:0/1896145365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1408271d0 msgr2=0x55e140820230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc32f37740 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1408271d0 0x55e140820230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efc24009d00 tx=0x7efc24012c60 comp rx=0 tx=0).stop 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc32f37740 1 -- 172.21.15.69:0/1896145365 shutdown_connections 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc32f37740 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e14081c910 0x55e14082f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc32f37740 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e1408a1150 0x55e1408a3590 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc32f37740 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e1408271d0 0x55e140820230 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc32f37740 1 --2- 172.21.15.69:0/1896145365 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e14081bff0 0x55e14082ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc32f37740 1 -- 172.21.15.69:0/1896145365 >> 172.21.15.69:0/1896145365 conn(0x55e140818580 msgr2=0x55e140822760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.484+0000 7efc11ffb640 0 client.27706 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.485+0000 7efc32f37740 1 -- 172.21.15.69:0/1896145365 shutdown_connections 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.485+0000 7efc32f37740 1 -- 172.21.15.69:0/1896145365 wait complete. 2024-08-02T17:58:59.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.485+0000 7efc32f37740 20 client.27706 trim_cache size 0 max 16384 2024-08-02T17:58:59.490 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:58:59.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.490+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efc140a2ec0 msgr2=0x7efc140a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.490+0000 7efc30b39640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efc140a2ec0 0x7efc140a5380 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc24005fd0 tx=0x7efc24009500 comp rx=0 tx=0).stop 2024-08-02T17:58:59.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.490+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc2c100870 msgr2=0x7efc2c073690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.490+0000 7efc30b39640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc2c100870 0x7efc2c073690 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efc1c033040 tx=0x7efc1c005230 comp rx=0 tx=0).stop 2024-08-02T17:58:59.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.491+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 shutdown_connections 2024-08-02T17:58:59.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.491+0000 7efc30b39640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7efc2c105c80 0x7efc2c06c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.491+0000 7efc30b39640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7efc140a2ec0 0x7efc140a5380 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.491+0000 7efc30b39640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efc2c101190 0x7efc2c073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.491+0000 7efc30b39640 1 --2- 172.21.15.69:0/3374100907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7efc2c100870 0x7efc2c073690 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.491+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 >> 172.21.15.69:0/3374100907 conn(0x7efc2c0fc500 msgr2=0x7efc2c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.491+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 shutdown_connections 2024-08-02T17:58:59.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:58:59.491+0000 7efc30b39640 1 -- 172.21.15.69:0/3374100907 wait complete. 2024-08-02T17:58:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:58:59 smithi083 ceph-mon[37031]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-08-02T17:58:59.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:58:59 smithi077 ceph-mon[37108]: pgmap v391: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 3.0 MiB/s wr, 300 op/s 2024-08-02T17:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.856+0000 7f65f165f640 1 -- 172.21.15.69:0/2818416074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ec11ae70 msgr2=0x7f65ec11d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.856+0000 7f65f165f640 1 --2- 172.21.15.69:0/2818416074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ec11ae70 0x7f65ec11d260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009cd0 tx=0x7f65e402a910 comp rx=0 tx=0).stop 2024-08-02T17:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.858+0000 7f65f165f640 1 -- 172.21.15.69:0/2818416074 shutdown_connections 2024-08-02T17:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.858+0000 7f65f165f640 1 --2- 172.21.15.69:0/2818416074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ec11ae70 0x7f65ec11d260 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.858+0000 7f65f165f640 1 --2- 172.21.15.69:0/2818416074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65ec109300 0x7f65ec109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.858+0000 7f65f165f640 1 --2- 172.21.15.69:0/2818416074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65ec108f20 0x7f65ec10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.858+0000 7f65f165f640 1 -- 172.21.15.69:0/2818416074 >> 172.21.15.69:0/2818416074 conn(0x7f65ec06cb20 msgr2=0x7f65ec06cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.861+0000 7f65f165f640 1 -- 172.21.15.69:0/2818416074 shutdown_connections 2024-08-02T17:58:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.864+0000 7f65f165f640 1 -- 172.21.15.69:0/2818416074 wait complete. 2024-08-02T17:58:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.865+0000 7f65f165f640 1 Processor -- start 2024-08-02T17:58:59.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65f165f640 1 -- start start 2024-08-02T17:58:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65f165f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ec108f20 0x7f65ec11a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65f165f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65ec109300 0x7f65ec114550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65f165f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65ec11ae70 0x7f65ec114a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65f165f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1110a0 con 0x7f65ec11ae70 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65f165f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec110f20 con 0x7f65ec108f20 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65f165f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec111220 con 0x7f65ec109300 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65eb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65ec11ae70 0x7f65ec114a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65eb7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65ec11ae70 0x7f65ec114a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33896/0 (socket says 172.21.15.69:33896) 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65eb7fe640 1 -- 172.21.15.69:0/944848780 learned_addr learned my addr 172.21.15.69:0/944848780 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.866+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65ec109300 0x7f65ec114550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.867+0000 7f65ea7fc640 1 -- 172.21.15.69:0/944848780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ec108f20 msgr2=0x7f65ec11a4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.867+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ec108f20 0x7f65ec11a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.867+0000 7f65ea7fc640 1 -- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65ec11ae70 msgr2=0x7f65ec114a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.867+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65ec11ae70 0x7f65ec114a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.867+0000 7f65ea7fc640 1 -- 172.21.15.69:0/944848780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec1150d0 con 0x7f65ec109300 2024-08-02T17:58:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.867+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65ec109300 0x7f65ec114550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f65e000b3e0 tx=0x7f65e000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.868+0000 7f65d3fff640 1 -- 172.21.15.69:0/944848780 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0015020 con 0x7f65ec109300 2024-08-02T17:58:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.868+0000 7f65d3fff640 1 -- 172.21.15.69:0/944848780 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e0011d10 con 0x7f65ec109300 2024-08-02T17:58:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.868+0000 7f65d3fff640 1 -- 172.21.15.69:0/944848780 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e000f2e0 con 0x7f65ec109300 2024-08-02T17:58:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.869+0000 7f65f165f640 1 -- 172.21.15.69:0/944848780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec074150 con 0x7f65ec109300 2024-08-02T17:58:59.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.869+0000 7f65f165f640 1 -- 172.21.15.69:0/944848780 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec074650 con 0x7f65ec109300 2024-08-02T17:58:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.875+0000 7f65f43b6740 1 Processor -- start 2024-08-02T17:58:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65f43b6740 1 -- start start 2024-08-02T17:58:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x56078460bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784616430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.880 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 0x560784618d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65f43b6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56078456f9b0 con 0x56078460c0d0 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65f43b6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560784517800 con 0x560784616970 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65f43b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56078456b970 con 0x56078460b7b0 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784616430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x56078460bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784616430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33908/0 (socket says 172.21.15.69:33908) 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eaffd640 1 -- 172.21.15.69:0/2608662161 learned_addr learned my addr 172.21.15.69:0/2608662161 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65d3fff640 1 -- 172.21.15.69:0/944848780 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f65e0018020 con 0x7f65ec109300 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eb7fe640 1 -- 172.21.15.69:0/2608662161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 msgr2=0x560784618d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:58:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eb7fe640 1 --2- 172.21.15.69:0/2608662161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 0x560784618d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eb7fe640 1 -- 172.21.15.69:0/2608662161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 msgr2=0x560784616430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eb7fe640 1 --2- 172.21.15.69:0/2608662161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784616430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.876+0000 7f65eb7fe640 1 -- 172.21.15.69:0/2608662161 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560784283210 con 0x56078460b7b0 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.877+0000 7f65eb7fe640 1 --2- 172.21.15.69:0/2608662161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x56078460bb90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009bf0 tx=0x7f65e4001150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da8aa1581ca65dbe server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.877+0000 7f65d17fa640 1 -- 172.21.15.69:0/2608662161 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4031b80 con 0x56078460b7b0 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.877+0000 7f65d17fa640 1 -- 172.21.15.69:0/2608662161 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4036040 con 0x56078460b7b0 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.877+0000 7f65d17fa640 1 -- 172.21.15.69:0/2608662161 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e4004040 con 0x56078460b7b0 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65d3fff640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65d40c4e90 0x7f65d40c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65d3fff640 1 -- 172.21.15.69:0/944848780 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f65e00cecb0 con 0x7f65ec109300 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65f43b6740 1 -- 172.21.15.69:0/2608662161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 msgr2=0x56078460bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2608662161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x56078460bb90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009bf0 tx=0x7f65e4001150 comp rx=0 tx=0).stop 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65eaffd640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65d40c4e90 0x7f65d40c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65f43b6740 1 -- 172.21.15.69:0/2608662161 shutdown_connections 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2608662161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784616430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2608662161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 0x560784618d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2608662161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x56078460bb90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.878+0000 7f65f43b6740 1 -- 172.21.15.69:0/2608662161 >> 172.21.15.69:0/2608662161 conn(0x560784607d20 msgr2=0x560784608100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.879+0000 7f65f43b6740 1 -- 172.21.15.69:0/2608662161 shutdown_connections 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.879+0000 7f65f43b6740 1 -- 172.21.15.69:0/2608662161 wait complete. 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.879+0000 7f65eaffd640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65d40c4e90 0x7f65d40c7350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x560784608b80 tx=0x7f65dc009190 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.879+0000 7f65f43b6740 1 -- start start 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.879+0000 7f65f43b6740 1 Processor -- start 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x5607846138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784613df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 0x560784614330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65f43b6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560784619540 con 0x56078460c0d0 2024-08-02T17:58:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65f43b6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56078452b500 con 0x560784616970 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65f43b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5607846196c0 con 0x56078460b7b0 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x5607846138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65eaffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x5607846138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59516/0 (socket says 172.21.15.69:59516) 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65eaffd640 1 -- 172.21.15.69:0/2992691722 learned_addr learned my addr 172.21.15.69:0/2992691722 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65f43b6740 10 client.0 _ll_register_callbacks cb 0x560784282a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784613df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.880+0000 7f65eb7fe640 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 0x560784614330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65eaffd640 1 -- 172.21.15.69:0/2992691722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 msgr2=0x560784614330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65eaffd640 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 0x560784614330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65eaffd640 1 -- 172.21.15.69:0/2992691722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 msgr2=0x560784613df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65eaffd640 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784613df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65eaffd640 1 -- 172.21.15.69:0/2992691722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078434f620 con 0x56078460b7b0 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784613df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65eaffd640 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x5607846138b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f65dc015760 tx=0x7f65dc0047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32608223727e7fbc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.881+0000 7f65ceff5640 1 -- 172.21.15.69:0/2992691722 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc023020 con 0x56078460b7b0 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.882+0000 7f65f43b6740 1 -- 172.21.15.69:0/2992691722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e4036040 con 0x56078460b7b0 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.882+0000 7f65ceff5640 1 -- 172.21.15.69:0/2992691722 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f65dc00f040 con 0x56078460b7b0 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.882+0000 7f65ceff5640 1 -- 172.21.15.69:0/2992691722 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65dc023020 con 0x56078460b7b0 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.882+0000 7f65f43b6740 10 client.27715 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.882+0000 7f65f43b6740 1 -- 172.21.15.69:0/2992691722 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560784625240 con 0x56078460b7b0 2024-08-02T17:58:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.882+0000 7f65ceff5640 1 -- 172.21.15.69:0/2992691722 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65e4036040 con 0x56078460b7b0 2024-08-02T17:58:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.883+0000 7f65ceff5640 1 -- 172.21.15.69:0/2992691722 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f65dc030020 con 0x56078460b7b0 2024-08-02T17:58:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.883+0000 7f65f43b6740 10 client.27715 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.883+0000 7f65f43b6740 10 client.27715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:59.894 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.883+0000 7f65f43b6740 20 client.27715 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.883+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560784626700 0x560784628af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.883+0000 7f65f43b6740 4 client.27715 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:58:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.883+0000 7f65f43b6740 1 -- 172.21.15.69:0/2992691722 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56078457d0e0 con 0x560784626700 2024-08-02T17:58:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.884+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560784626700 0x560784628af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.884+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560784626700 0x560784628af0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add30b3b3dc32f8b server_cookie=df83e9a91cb7824 in_seq=0 out_seq=0 2024-08-02T17:58:59.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.884+0000 7f65ceff5640 10 client.27715 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.885+0000 7f65ceff5640 1 -- 172.21.15.69:0/2992691722 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56078457d0e0 con 0x560784626700 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.885+0000 7f65ceff5640 10 client.27715 handle_command_reply: tid=0 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.885+0000 7f65f43b6740 1 client.27715 shutdown 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.886+0000 7f65f43b6740 1 -- 172.21.15.69:0/2992691722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 msgr2=0x5607846138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.886+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x5607846138b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f65dc015760 tx=0x7f65dc0047d0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.886+0000 7f65f43b6740 1 -- 172.21.15.69:0/2992691722 shutdown_connections 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x560784613df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560784626700 0x560784628af0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65ceff5640 0 client.27715 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784616970 0x560784614330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65f43b6740 1 --2- 172.21.15.69:0/2992691722 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56078460b7b0 0x5607846138b0 secure :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f65dc015760 tx=0x7f65dc0047d0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65f43b6740 1 -- 172.21.15.69:0/2992691722 >> 172.21.15.69:0/2992691722 conn(0x560784607d20 msgr2=0x56078460da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65f43b6740 1 -- 172.21.15.69:0/2992691722 shutdown_connections 2024-08-02T17:58:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65f43b6740 1 -- 172.21.15.69:0/2992691722 wait complete. 2024-08-02T17:58:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.887+0000 7f65f43b6740 20 client.27715 trim_cache size 0 max 16384 2024-08-02T17:58:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.912+0000 7f65f43b6740 1 Processor -- start 2024-08-02T17:58:59.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.912+0000 7f65f43b6740 1 -- start start 2024-08-02T17:58:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.912+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x56078460e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784616970 0x560784615710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65eaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x56078460e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65eaffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x56078460e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33938/0 (socket says 172.21.15.69:33938) 2024-08-02T17:58:59.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65eaffd640 1 -- 172.21.15.69:0/1117894662 learned_addr learned my addr 172.21.15.69:0/1117894662 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65f43b6740 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784612980 0x56078462e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784616970 0x560784615710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65f43b6740 1 -- 172.21.15.69:0/1117894662 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec081490 con 0x56078460c0d0 2024-08-02T17:58:59.915 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65f43b6740 1 -- 172.21.15.69:0/1117894662 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec07ed60 con 0x560784612980 2024-08-02T17:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.913+0000 7f65f43b6740 1 -- 172.21.15.69:0/1117894662 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec1152a0 con 0x560784616970 2024-08-02T17:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65eb7fe640 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784612980 0x56078462e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65eaffd640 1 -- 172.21.15.69:0/1117894662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784616970 msgr2=0x560784615710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65eaffd640 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784616970 0x560784615710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65eaffd640 1 -- 172.21.15.69:0/1117894662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784612980 msgr2=0x56078462e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65eaffd640 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784612980 0x56078462e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65eaffd640 1 -- 172.21.15.69:0/1117894662 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560784608dd0 con 0x56078460c0d0 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784616970 0x560784615710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65eb7fe640 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784612980 0x56078462e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.914+0000 7f65eaffd640 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x56078460e220 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0094f0 tx=0x7f65dc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201b909c62f84553 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.915+0000 7f65d0ff9640 1 -- 172.21.15.69:0/1117894662 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc023020 con 0x56078460c0d0 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.915+0000 7f65d0ff9640 1 -- 172.21.15.69:0/1117894662 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65dc0283f0 con 0x56078460c0d0 2024-08-02T17:58:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.915+0000 7f65d0ff9640 1 -- 172.21.15.69:0/1117894662 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc01b040 con 0x56078460c0d0 2024-08-02T17:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 -- 172.21.15.69:0/1117894662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 msgr2=0x56078460e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x56078460e220 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0094f0 tx=0x7f65dc00ee00 comp rx=0 tx=0).stop 2024-08-02T17:58:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 -- 172.21.15.69:0/1117894662 shutdown_connections 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560784612980 0x56078462e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784616970 0x560784615710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 --2- 172.21.15.69:0/1117894662 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56078460c0d0 0x56078460e220 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 -- 172.21.15.69:0/1117894662 >> 172.21.15.69:0/1117894662 conn(0x560784607d20 msgr2=0x560784615000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 -- 172.21.15.69:0/1117894662 shutdown_connections 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.916+0000 7f65f43b6740 1 -- 172.21.15.69:0/1117894662 wait complete. 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 1 -- start start 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 1 Processor -- start 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56078460c0d0 0x56078461e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784612980 0x56078461ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560784616970 0x56078461b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65ea7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784612980 0x56078461ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec07d220 con 0x560784616970 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec07e690 con 0x56078460c0d0 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec07d8f0 con 0x560784612980 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65ea7fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784612980 0x56078461ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59532/0 (socket says 172.21.15.69:59532) 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65ea7fc640 1 -- 172.21.15.69:0/4233507428 learned_addr learned my addr 172.21.15.69:0/4233507428 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65eb7fe640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560784616970 0x56078461b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65f43b6740 10 client.0 _ll_register_callbacks cb 0x7f65dc0096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.917+0000 7f65eaffd640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56078460c0d0 0x56078461e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ea7fc640 1 -- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56078460c0d0 msgr2=0x56078461e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56078460c0d0 0x56078461e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ea7fc640 1 -- 172.21.15.69:0/4233507428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560784616970 msgr2=0x56078461b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560784616970 0x56078461b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ea7fc640 1 -- 172.21.15.69:0/4233507428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56078461be60 con 0x560784612980 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65eaffd640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56078460c0d0 0x56078461e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65eb7fe640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560784616970 0x56078461b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ea7fc640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784612980 0x56078461ee90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f65e009ae60 tx=0x7f65e009adc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a2d8cdfc04269ca server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ceff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ceff5640 1 -- 172.21.15.69:0/4233507428 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0097540 con 0x560784612980 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ceff5640 1 -- 172.21.15.69:0/4233507428 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9568+0+0 (secure 0 0 0) 0x7f65e00cc270 con 0x560784612980 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 1 -- 172.21.15.69:0/4233507428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65dc0283f0 con 0x560784612980 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ceff5640 1 -- 172.21.15.69:0/4233507428 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f65e00ab050 con 0x560784612980 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 10 client.27721 fetch_fsmap learned FSMap version 26 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 1 -- 172.21.15.69:0/4233507428 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560784691c20 con 0x560784612980 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ceff5640 1 -- 172.21.15.69:0/4233507428 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65dc0283f0 con 0x560784612980 2024-08-02T17:58:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65ceff5640 1 -- 172.21.15.69:0/4233507428 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f65e009a320 con 0x560784612980 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 10 client.27721 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 10 client.27721 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 20 client.27721 populate_metadata read hostname 'smithi069' 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5607846923d0 0x5607846947f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 4 client.27721 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.918+0000 7f65f43b6740 1 -- 172.21.15.69:0/4233507428 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560784577430 con 0x5607846923d0 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.919+0000 7f65eaffd640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5607846923d0 0x5607846947f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.919+0000 7f65ceff5640 10 client.27721 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:59.924 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.919+0000 7f65eaffd640 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5607846923d0 0x5607846947f0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d504d04889fcb973 server_cookie=ccd8701777efcfe7 in_seq=0 out_seq=0 2024-08-02T17:58:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 710.0 s, 529.8 tps, lat 188.454 ms stddev 221.812 2024-08-02T17:58:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65ceff5640 1 -- 172.21.15.69:0/4233507428 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560784577430 con 0x5607846923d0 2024-08-02T17:58:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65ceff5640 10 client.27721 handle_command_reply: tid=0 2024-08-02T17:58:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 client.27721 shutdown 2024-08-02T17:58:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 -- 172.21.15.69:0/4233507428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784612980 msgr2=0x56078461ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784612980 0x56078461ee90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f65e009ae60 tx=0x7f65e009adc0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 -- 172.21.15.69:0/4233507428 shutdown_connections 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560784616970 0x56078461b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5607846923d0 0x5607846947f0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560784612980 0x56078461ee90 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f65e009ae60 tx=0x7f65e009adc0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65ceff5640 0 client.27721 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 --2- 172.21.15.69:0/4233507428 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56078460c0d0 0x56078461e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 -- 172.21.15.69:0/4233507428 >> 172.21.15.69:0/4233507428 conn(0x560784607d20 msgr2=0x560784606e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 -- 172.21.15.69:0/4233507428 shutdown_connections 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.957+0000 7f65f43b6740 1 -- 172.21.15.69:0/4233507428 wait complete. 2024-08-02T17:58:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.958+0000 7f65f43b6740 20 client.27721 trim_cache size 0 max 16384 2024-08-02T17:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:58:59.961 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 -- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65d40c4e90 msgr2=0x7f65d40c7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65d40c4e90 0x7f65d40c7350 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x560784608b80 tx=0x7f65dc009190 comp rx=0 tx=0).stop 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 -- 172.21.15.69:0/944848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65ec109300 msgr2=0x7f65ec114550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65ec109300 0x7f65ec114550 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f65e000b3e0 tx=0x7f65e000b8a0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 -- 172.21.15.69:0/944848780 shutdown_connections 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f65ec11ae70 0x7f65ec114a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f65d40c4e90 0x7f65d40c7350 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f65ec109300 0x7f65ec114550 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 --2- 172.21.15.69:0/944848780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f65ec108f20 0x7f65ec11a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 -- 172.21.15.69:0/944848780 >> 172.21.15.69:0/944848780 conn(0x7f65ec06cb20 msgr2=0x7f65ec070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 -- 172.21.15.69:0/944848780 shutdown_connections 2024-08-02T17:58:59.962 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:58:59.959+0000 7f65f165f640 1 -- 172.21.15.69:0/944848780 wait complete. 2024-08-02T17:59:01.206 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:59:01.206 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:59:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:01 smithi083 ceph-mon[37031]: pgmap v392: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-08-02T17:59:01.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:01 smithi077 ceph-mon[37108]: pgmap v392: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-08-02T17:59:01.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:01 smithi069 ceph-mon[32746]: pgmap v392: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.4 MiB/s wr, 413 op/s 2024-08-02T17:59:01.794 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:03.383 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:03 smithi069 ceph-mon[32746]: pgmap v393: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-02T17:59:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:03 smithi083 ceph-mon[37031]: pgmap v393: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-02T17:59:03.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:03 smithi077 ceph-mon[37108]: pgmap v393: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 2.1 MiB/s wr, 207 op/s 2024-08-02T17:59:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 715.0 s, 468.8 tps, lat 213.790 ms stddev 287.124 2024-08-02T17:59:05.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 -- 172.21.15.69:0/4080636618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109560 msgr2=0x7f5fec1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:05.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 --2- 172.21.15.69:0/4080636618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109560 0x7f5fec1099c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc01a910 tx=0x7f5fdc03b470 comp rx=0 tx=0).stop 2024-08-02T17:59:05.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 -- 172.21.15.69:0/4080636618 shutdown_connections 2024-08-02T17:59:05.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 --2- 172.21.15.69:0/4080636618 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109560 0x7f5fec1099c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 --2- 172.21.15.69:0/4080636618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fec11ae70 0x7f5fec11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 --2- 172.21.15.69:0/4080636618 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fec109180 0x7f5fec10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 -- 172.21.15.69:0/4080636618 >> 172.21.15.69:0/4080636618 conn(0x7f5fec06de70 msgr2=0x7f5fec06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:05.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 -- 172.21.15.69:0/4080636618 shutdown_connections 2024-08-02T17:59:05.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.313+0000 7f5ff3408640 1 -- 172.21.15.69:0/4080636618 wait complete. 2024-08-02T17:59:05.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.314+0000 7f5ff3408640 1 Processor -- start 2024-08-02T17:59:05.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.314+0000 7f5ff3408640 1 -- start start 2024-08-02T17:59:05.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.315+0000 7f5ff3408640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109180 0x7f5fec11a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.315+0000 7f5ff3408640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fec109560 0x7f5fec1165d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.317 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.315+0000 7f5ff3408640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fec11ae70 0x7f5fec116b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.315+0000 7f5ff3408640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec113030 con 0x7f5fec109180 2024-08-02T17:59:05.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.315+0000 7f5ff3408640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec112eb0 con 0x7f5fec11ae70 2024-08-02T17:59:05.318 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.315+0000 7f5ff3408640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1131b0 con 0x7f5fec109560 2024-08-02T17:59:05.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.316+0000 7f5ff19a5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fec11ae70 0x7f5fec116b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.317+0000 7f5ff11a4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109180 0x7f5fec11a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.317+0000 7f5ff11a4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109180 0x7f5fec11a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52480/0 (socket says 172.21.15.69:52480) 2024-08-02T17:59:05.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.317+0000 7f5ff11a4640 1 -- 172.21.15.69:0/1035773103 learned_addr learned my addr 172.21.15.69:0/1035773103 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:05.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.324+0000 7f5ff19a5640 1 -- 172.21.15.69:0/1035773103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fec109560 msgr2=0x7f5fec1165d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T17:59:05.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.324+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fec109560 0x7f5fec1165d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.324+0000 7f5ff19a5640 1 -- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109180 msgr2=0x7f5fec11a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:05.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.324+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109180 0x7f5fec11a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.324+0000 7f5ff19a5640 1 -- 172.21.15.69:0/1035773103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec117240 con 0x7f5fec11ae70 2024-08-02T17:59:05.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.328+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fec11ae70 0x7f5fec116b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400ea80 tx=0x7f5fe400ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:05.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.332+0000 7f5fe27fc640 1 -- 172.21.15.69:0/1035773103 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4004b20 con 0x7f5fec11ae70 2024-08-02T17:59:05.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.332+0000 7f5fe27fc640 1 -- 172.21.15.69:0/1035773103 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe4004cc0 con 0x7f5fec11ae70 2024-08-02T17:59:05.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.332+0000 7f5fe27fc640 1 -- 172.21.15.69:0/1035773103 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4010720 con 0x7f5fec11ae70 2024-08-02T17:59:05.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.333+0000 7f5ff3408640 1 -- 172.21.15.69:0/1035773103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec1aaea0 con 0x7f5fec11ae70 2024-08-02T17:59:05.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.333+0000 7f5ff3408640 1 -- 172.21.15.69:0/1035773103 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec1ab240 con 0x7f5fec11ae70 2024-08-02T17:59:05.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.337+0000 7f5ff5806740 1 Processor -- start 2024-08-02T17:59:05.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.340+0000 7f5fe27fc640 1 -- 172.21.15.69:0/1035773103 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5fe40108c0 con 0x7f5fec11ae70 2024-08-02T17:59:05.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.341+0000 7f5fe27fc640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fd80a2e80 0x7f5fd80a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.343 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.341+0000 7f5fe27fc640 1 -- 172.21.15.69:0/1035773103 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f5fe40d0200 con 0x7f5fec11ae70 2024-08-02T17:59:05.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.343+0000 7f5ff5806740 1 -- start start 2024-08-02T17:59:05.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.343+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e3f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.343+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3fb20 0x55f538e49ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.343+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e4c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.343+0000 7f5ff5806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f538e02e40 con 0x55f538e4a3e0 2024-08-02T17:59:05.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.343+0000 7f5ff5806740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f538da47f0 con 0x55f538e3f200 2024-08-02T17:59:05.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.343+0000 7f5ff5806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f538d4bfb0 con 0x55f538e3fb20 2024-08-02T17:59:05.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff09a3640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fd80a2e80 0x7f5fd80a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff19a5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e4c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff19a5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e4c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52482/0 (socket says 172.21.15.69:52482) 2024-08-02T17:59:05.348 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff19a5640 1 -- 172.21.15.69:0/4142585393 learned_addr learned my addr 172.21.15.69:0/4142585393 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff11a4640 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e3f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff09a3640 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3fb20 0x55f538e49ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff19a5640 1 -- 172.21.15.69:0/4142585393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3fb20 msgr2=0x55f538e49ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3fb20 0x55f538e49ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff19a5640 1 -- 172.21.15.69:0/4142585393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 msgr2=0x55f538e3f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e3f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.345+0000 7f5ff19a5640 1 -- 172.21.15.69:0/4142585393 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f538b166a0 con 0x55f538e4a3e0 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.346+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e4c7d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4098c80 tx=0x7f5fe409c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2a2fd506ae2bd6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:05.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.346+0000 7f5ff09a3640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fd80a2e80 0x7f5fd80a5340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc044040 tx=0x7f5fdc01a4c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:05.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.350+0000 7f5fd77fe640 1 -- 172.21.15.69:0/4142585393 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe409c2b0 con 0x55f538e4a3e0 2024-08-02T17:59:05.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.350+0000 7f5fd77fe640 1 -- 172.21.15.69:0/4142585393 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe40a0660 con 0x55f538e4a3e0 2024-08-02T17:59:05.351 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.350+0000 7f5fd77fe640 1 -- 172.21.15.69:0/4142585393 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4010bb0 con 0x55f538e4a3e0 2024-08-02T17:59:05.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.351+0000 7f5ff5806740 1 -- 172.21.15.69:0/4142585393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 msgr2=0x55f538e4c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:05.352 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.351+0000 7f5ff5806740 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e4c7d0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4098c80 tx=0x7f5fe409c1a0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.356+0000 7f5ff5806740 1 -- 172.21.15.69:0/4142585393 shutdown_connections 2024-08-02T17:59:05.357 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.356+0000 7f5ff5806740 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e4c7d0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.356+0000 7f5ff5806740 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3fb20 0x55f538e49ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.356+0000 7f5ff5806740 1 --2- 172.21.15.69:0/4142585393 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e3f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.358 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.356+0000 7f5ff5806740 1 -- 172.21.15.69:0/4142585393 >> 172.21.15.69:0/4142585393 conn(0x55f538e3b790 msgr2=0x55f538e3bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:05.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.361+0000 7f5ff5806740 1 -- 172.21.15.69:0/4142585393 shutdown_connections 2024-08-02T17:59:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.367+0000 7f5ff5806740 1 -- 172.21.15.69:0/4142585393 wait complete. 2024-08-02T17:59:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.367+0000 7f5ff5806740 1 -- start start 2024-08-02T17:59:05.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.367+0000 7f5ff5806740 1 Processor -- start 2024-08-02T17:59:05.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.369+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 0x55f538e47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.369+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e47850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.369+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e47d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.369+0000 7f5ff5806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f538d5fcb0 con 0x55f538e4a3e0 2024-08-02T17:59:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.369+0000 7f5ff5806740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f538da07b0 con 0x55f538e3fb20 2024-08-02T17:59:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.369+0000 7f5ff5806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f538e4cfb0 con 0x55f538e3f200 2024-08-02T17:59:05.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.370+0000 7f5ff5806740 10 client.0 _ll_register_callbacks cb 0x55f538ab3ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:05.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.374+0000 7f5ff09a3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e47850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.375+0000 7f5ff11a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 0x55f538e47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.375+0000 7f5ff19a5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e47d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.375+0000 7f5ff19a5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e47d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52498/0 (socket says 172.21.15.69:52498) 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.375+0000 7f5ff19a5640 1 -- 172.21.15.69:0/3287667500 learned_addr learned my addr 172.21.15.69:0/3287667500 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.376+0000 7f5ff11a4640 1 -- 172.21.15.69:0/3287667500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 msgr2=0x55f538e47850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.376+0000 7f5ff11a4640 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e47850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.376+0000 7f5ff11a4640 1 -- 172.21.15.69:0/3287667500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 msgr2=0x55f538e47d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.376+0000 7f5ff11a4640 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e47d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.376+0000 7f5ff11a4640 1 -- 172.21.15.69:0/3287667500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f538d5ebc0 con 0x55f538e3f200 2024-08-02T17:59:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.381+0000 7f5ff11a4640 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 0x55f538e47310 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8012730 tx=0x7f5fe8012b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a24cccfe371bef17 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:05.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.382+0000 7f5fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:59:05.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.385+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/3287667500 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8004050 con 0x55f538e3f200 2024-08-02T17:59:05.387 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:05 smithi069 ceph-mon[32746]: pgmap v394: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.4 MiB/s wr, 430 op/s 2024-08-02T17:59:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.385+0000 7f5ff5806740 1 -- 172.21.15.69:0/3287667500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe40a0660 con 0x55f538e3f200 2024-08-02T17:59:05.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.386+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/3287667500 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f5fe80041f0 con 0x55f538e3f200 2024-08-02T17:59:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.391+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/3287667500 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5fe8025050 con 0x55f538e3f200 2024-08-02T17:59:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.391+0000 7f5ff5806740 10 client.27727 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:05.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.391+0000 7f5ff5806740 1 -- 172.21.15.69:0/3287667500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f538e58f20 con 0x55f538e3f200 2024-08-02T17:59:05.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.398+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/3287667500 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe40a0660 con 0x55f538e3f200 2024-08-02T17:59:05.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.399+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/3287667500 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5fe8003b00 con 0x55f538e3f200 2024-08-02T17:59:05.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.399+0000 7f5ff5806740 10 client.27727 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:05.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.399+0000 7f5ff5806740 10 client.27727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:05.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.399+0000 7f5ff5806740 20 client.27727 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:05.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.399+0000 7f5ff5806740 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f538e5a820 0x55f538e5cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.399+0000 7f5ff5806740 4 client.27727 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:05.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.399+0000 7f5ff5806740 1 -- 172.21.15.69:0/3287667500 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f538db1d60 con 0x55f538e5a820 2024-08-02T17:59:05.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.400+0000 7f5ff09a3640 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f538e5a820 0x55f538e5cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.401+0000 7f5ff09a3640 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f538e5a820 0x55f538e5cca0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e560b02acac99e server_cookie=72a45a0dbd98efe3 in_seq=0 out_seq=0 2024-08-02T17:59:05.402 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.401+0000 7f5fd4ff9640 10 client.27727 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:05.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.404+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/3287667500 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f538db1d60 con 0x55f538e5a820 2024-08-02T17:59:05.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.404+0000 7f5fd4ff9640 10 client.27727 handle_command_reply: tid=0 2024-08-02T17:59:05.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.404+0000 7f5ff5806740 1 client.27727 shutdown 2024-08-02T17:59:05.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5ff5806740 1 -- 172.21.15.69:0/3287667500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 msgr2=0x55f538e47310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5ff5806740 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 0x55f538e47310 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8012730 tx=0x7f5fe8012b30 comp rx=0 tx=0).stop 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5ff5806740 1 -- 172.21.15.69:0/3287667500 shutdown_connections 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5ff5806740 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e47d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5ff5806740 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f538e5a820 0x55f538e5cca0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5ff5806740 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e47850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5ff5806740 1 --2- 172.21.15.69:0/3287667500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 0x55f538e47310 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5ff5806740 1 -- 172.21.15.69:0/3287667500 >> 172.21.15.69:0/3287667500 conn(0x55f538e3b790 msgr2=0x55f538e41470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.411+0000 7f5fd4ff9640 0 client.27727 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:05.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.412+0000 7f5ff5806740 1 -- 172.21.15.69:0/3287667500 shutdown_connections 2024-08-02T17:59:05.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.416+0000 7f5ff5806740 1 -- 172.21.15.69:0/3287667500 wait complete. 2024-08-02T17:59:05.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.416+0000 7f5ff5806740 20 client.27727 trim_cache size 0 max 16384 2024-08-02T17:59:05.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.449+0000 7f5ff5806740 1 Processor -- start 2024-08-02T17:59:05.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.449+0000 7f5ff5806740 1 -- start start 2024-08-02T17:59:05.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.449+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e47880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.449+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e3fb20 0x55f538e47dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.449+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e4a3e0 0x55f538e625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.449+0000 7f5ff5806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b8040 con 0x55f538e3fb20 2024-08-02T17:59:05.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.449+0000 7f5ff5806740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b5910 con 0x55f538e3f200 2024-08-02T17:59:05.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.449+0000 7f5ff5806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b3780 con 0x55f538e4a3e0 2024-08-02T17:59:05.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.450+0000 7f5ff11a4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e47880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.450+0000 7f5ff11a4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e47880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55484/0 (socket says 172.21.15.69:55484) 2024-08-02T17:59:05.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.450+0000 7f5ff11a4640 1 -- 172.21.15.69:0/2831247054 learned_addr learned my addr 172.21.15.69:0/2831247054 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:05.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.451+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/2831247054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e4a3e0 0x55f538e625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.455+0000 7f5ff19a5640 1 -- 172.21.15.69:0/2831247054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 msgr2=0x55f538e47880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:05.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.455+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/2831247054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e47880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.455+0000 7f5ff19a5640 1 -- 172.21.15.69:0/2831247054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e3fb20 msgr2=0x55f538e47dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:05.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.455+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/2831247054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e3fb20 0x55f538e47dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.456+0000 7f5ff19a5640 1 -- 172.21.15.69:0/2831247054 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f538e4cb40 con 0x55f538e4a3e0 2024-08-02T17:59:05.462 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.459+0000 7f5ff19a5640 1 --2- 172.21.15.69:0/2831247054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e4a3e0 0x55f538e625b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400f010 tx=0x7f5fe4098920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=370027b67a5e99f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:05.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.464+0000 7f5fd6ffd640 1 -- 172.21.15.69:0/2831247054 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe409c2b0 con 0x55f538e4a3e0 2024-08-02T17:59:05.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.464+0000 7f5fd6ffd640 1 -- 172.21.15.69:0/2831247054 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe40a4f20 con 0x55f538e4a3e0 2024-08-02T17:59:05.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.464+0000 7f5fd6ffd640 1 -- 172.21.15.69:0/2831247054 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40a0170 con 0x55f538e4a3e0 2024-08-02T17:59:05.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.465+0000 7f5ff5806740 1 -- 172.21.15.69:0/2831247054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e4a3e0 msgr2=0x55f538e625b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:05.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.465+0000 7f5ff5806740 1 --2- 172.21.15.69:0/2831247054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e4a3e0 0x55f538e625b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400f010 tx=0x7f5fe4098920 comp rx=0 tx=0).stop 2024-08-02T17:59:05.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.466+0000 7f5ff5806740 1 -- 172.21.15.69:0/2831247054 shutdown_connections 2024-08-02T17:59:05.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.466+0000 7f5ff5806740 1 --2- 172.21.15.69:0/2831247054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e3fb20 0x55f538e47dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.466+0000 7f5ff5806740 1 --2- 172.21.15.69:0/2831247054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e4a3e0 0x55f538e625b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.466+0000 7f5ff5806740 1 --2- 172.21.15.69:0/2831247054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3f200 0x55f538e47880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.466+0000 7f5ff5806740 1 -- 172.21.15.69:0/2831247054 >> 172.21.15.69:0/2831247054 conn(0x55f538e3b790 msgr2=0x55f538e49390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:05.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.470+0000 7f5ff5806740 1 -- 172.21.15.69:0/2831247054 shutdown_connections 2024-08-02T17:59:05.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.472+0000 7f5ff5806740 1 -- 172.21.15.69:0/2831247054 wait complete. 2024-08-02T17:59:05.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.472+0000 7f5ff5806740 1 -- start start 2024-08-02T17:59:05.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.472+0000 7f5ff5806740 1 Processor -- start 2024-08-02T17:59:05.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.473+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 0x55f538e52740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.473+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e43f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.473+0000 7f5ff5806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e44490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.473+0000 7f5ff5806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b3dd0 con 0x55f538e4a3e0 2024-08-02T17:59:05.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.473+0000 7f5ff5806740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b5240 con 0x55f538e3fb20 2024-08-02T17:59:05.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.473+0000 7f5ff5806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1b44a0 con 0x55f538e3f200 2024-08-02T17:59:05.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.473+0000 7f5ff5806740 10 client.0 _ll_register_callbacks cb 0x7f5fe8009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:05.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.474+0000 7f5ff09a3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e43f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.474+0000 7f5ff09a3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e43f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55494/0 (socket says 172.21.15.69:55494) 2024-08-02T17:59:05.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.474+0000 7f5ff09a3640 1 -- 172.21.15.69:0/396409753 learned_addr learned my addr 172.21.15.69:0/396409753 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:05.480 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.474+0000 7f5ff09a3640 1 -- 172.21.15.69:0/396409753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 msgr2=0x55f538e52740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:05.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.474+0000 7f5ff09a3640 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 0x55f538e52740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.474+0000 7f5ff09a3640 1 -- 172.21.15.69:0/396409753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 msgr2=0x55f538e44490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:05.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.474+0000 7f5ff09a3640 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e44490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.474+0000 7f5ff09a3640 1 -- 172.21.15.69:0/396409753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f538e44a50 con 0x55f538e3fb20 2024-08-02T17:59:05.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.475+0000 7f5ff09a3640 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e43f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc01a910 tx=0x7f5fdc03b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=360b1b8f4e54f98 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:05.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.479+0000 7f5fd4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:59:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.479+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/396409753 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc06c060 con 0x55f538e3fb20 2024-08-02T17:59:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.479+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/396409753 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f5fdc019040 con 0x55f538e3fb20 2024-08-02T17:59:05.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.480+0000 7f5ff5806740 1 -- 172.21.15.69:0/396409753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe40a4f20 con 0x55f538e3fb20 2024-08-02T17:59:05.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.488+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/396409753 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5fdc040960 con 0x55f538e3fb20 2024-08-02T17:59:05.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.488+0000 7f5ff5806740 10 client.27944 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:05.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.488+0000 7f5ff5806740 1 -- 172.21.15.69:0/396409753 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f538ec5420 con 0x55f538e3fb20 2024-08-02T17:59:05.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.494+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/396409753 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe40a4f20 con 0x55f538e3fb20 2024-08-02T17:59:05.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.494+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/396409753 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5fdc04e040 con 0x55f538e3fb20 2024-08-02T17:59:05.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.494+0000 7f5ff5806740 10 client.27944 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:05.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.494+0000 7f5ff5806740 10 client.27944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:05.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.494+0000 7f5ff5806740 20 client.27944 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:05.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.494+0000 7f5ff5806740 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f538ec5b60 0x55f538ec7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:05.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.494+0000 7f5ff5806740 4 client.27944 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:59:05.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.494+0000 7f5ff5806740 1 -- 172.21.15.69:0/396409753 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f538dac270 con 0x55f538ec5b60 2024-08-02T17:59:05.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.495+0000 7f5ff11a4640 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f538ec5b60 0x55f538ec7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:05.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.506+0000 7f5fd4ff9640 10 client.27944 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:05.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.506+0000 7f5ff11a4640 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f538ec5b60 0x55f538ec7f80 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc295b63cdbc093a server_cookie=23eb3513be28c5e in_seq=0 out_seq=0 2024-08-02T17:59:05.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.510+0000 7f5fd4ff9640 1 -- 172.21.15.69:0/396409753 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f538dac270 con 0x55f538ec5b60 2024-08-02T17:59:05.514 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.510+0000 7f5fd4ff9640 10 client.27944 handle_command_reply: tid=0 2024-08-02T17:59:05.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.510+0000 7f5ff5806740 1 client.27944 shutdown 2024-08-02T17:59:05.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 -- 172.21.15.69:0/396409753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 msgr2=0x55f538e43f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:05.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e43f50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc01a910 tx=0x7f5fdc03b470 comp rx=0 tx=0).stop 2024-08-02T17:59:05.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 -- 172.21.15.69:0/396409753 shutdown_connections 2024-08-02T17:59:05.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f538e4a3e0 0x55f538e44490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f538ec5b60 0x55f538ec7f80 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f538e3fb20 0x55f538e43f50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 --2- 172.21.15.69:0/396409753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f538e3f200 0x55f538e52740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 -- 172.21.15.69:0/396409753 >> 172.21.15.69:0/396409753 conn(0x55f538e3b790 msgr2=0x55f538e3a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5fd4ff9640 0 client.27944 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 -- 172.21.15.69:0/396409753 shutdown_connections 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 1 -- 172.21.15.69:0/396409753 wait complete. 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.514+0000 7f5ff5806740 20 client.27944 trim_cache size 0 max 16384 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:59:05.518 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "3d30ab4a-2e5d-45c5-8fd9-492750208617", 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 -- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fd80a2e80 msgr2=0x7f5fd80a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fd80a2e80 0x7f5fd80a5340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc044040 tx=0x7f5fdc01a4c0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 -- 172.21.15.69:0/1035773103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fec11ae70 msgr2=0x7f5fec116b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fec11ae70 0x7f5fec116b10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400ea80 tx=0x7f5fe400ef40 comp rx=0 tx=0).stop 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 -- 172.21.15.69:0/1035773103 shutdown_connections 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5fd80a2e80 0x7f5fd80a5340 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5fec11ae70 0x7f5fec116b10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5fec109560 0x7f5fec1165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 --2- 172.21.15.69:0/1035773103 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5fec109180 0x7f5fec11a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:05.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.515+0000 7f5ff3408640 1 -- 172.21.15.69:0/1035773103 >> 172.21.15.69:0/1035773103 conn(0x7f5fec06de70 msgr2=0x7f5fec0709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:05 smithi083 ceph-mon[37031]: pgmap v394: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.4 MiB/s wr, 430 op/s 2024-08-02T17:59:05.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.521+0000 7f5ff3408640 1 -- 172.21.15.69:0/1035773103 shutdown_connections 2024-08-02T17:59:05.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:05.521+0000 7f5ff3408640 1 -- 172.21.15.69:0/1035773103 wait complete. 2024-08-02T17:59:05.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:05 smithi077 ceph-mon[37108]: pgmap v394: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.4 MiB/s wr, 430 op/s 2024-08-02T17:59:06.256 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:59:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:06 smithi083 ceph-mon[37031]: osdmap e108: 12 total, 12 up, 12 in 2024-08-02T17:59:06.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:06 smithi077 ceph-mon[37108]: osdmap e108: 12 total, 12 up, 12 in 2024-08-02T17:59:06.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:06 smithi069 ceph-mon[32746]: osdmap e108: 12 total, 12 up, 12 in 2024-08-02T17:59:06.770 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:07.188 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:07 smithi069 ceph-mon[32746]: pgmap v396: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.3 MiB/s wr, 515 op/s 2024-08-02T17:59:07.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:07 smithi083 ceph-mon[37031]: pgmap v396: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.3 MiB/s wr, 515 op/s 2024-08-02T17:59:07.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:07 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:59:07.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:07 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:07.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:07 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:59:07.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:07 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:07.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:07 smithi077 ceph-mon[37108]: pgmap v396: 129 pgs: 129 active+clean; 3.2 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 5.3 MiB/s wr, 515 op/s 2024-08-02T17:59:07.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:07 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:59:07.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:07 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:07.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:07 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:59:07.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:07 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:07.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:07 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:59:07.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:07 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:07.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:07 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:59:07.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:07 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:09.431 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:09 smithi069 ceph-mon[32746]: pgmap v397: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.4 MiB/s wr, 527 op/s 2024-08-02T17:59:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:09 smithi083 ceph-mon[37031]: pgmap v397: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.4 MiB/s wr, 527 op/s 2024-08-02T17:59:09.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:09 smithi077 ceph-mon[37108]: pgmap v397: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.4 MiB/s wr, 527 op/s 2024-08-02T17:59:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 720.0 s, 484.0 tps, lat 204.973 ms stddev 298.421 2024-08-02T17:59:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.023+0000 7f37f11ed640 1 -- 172.21.15.69:0/2077724592 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8031600 con 0x7f37ec1022a0 2024-08-02T17:59:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.024+0000 7f37f4453640 1 -- 172.21.15.69:0/2077724592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 msgr2=0x7f37ec1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.024+0000 7f37f4453640 1 --2- 172.21.15.69:0/2077724592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 0x7f37ec1026e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009860 tx=0x7f37e802a3d0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.024+0000 7f37f4453640 1 -- 172.21.15.69:0/2077724592 shutdown_connections 2024-08-02T17:59:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.024+0000 7f37f4453640 1 --2- 172.21.15.69:0/2077724592 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 0x7f37ec1026e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.024+0000 7f37f4453640 1 --2- 172.21.15.69:0/2077724592 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37ec102c20 0x7f37ec10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.024+0000 7f37f4453640 1 --2- 172.21.15.69:0/2077724592 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec100b50 0x7f37ec100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.025+0000 7f37f4453640 1 -- 172.21.15.69:0/2077724592 >> 172.21.15.69:0/2077724592 conn(0x7f37ec0fc500 msgr2=0x7f37ec0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:10.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.025+0000 7f37f4453640 1 -- 172.21.15.69:0/2077724592 shutdown_connections 2024-08-02T17:59:10.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.025+0000 7f37f4453640 1 -- 172.21.15.69:0/2077724592 wait complete. 2024-08-02T17:59:10.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.026+0000 7f37f4453640 1 Processor -- start 2024-08-02T17:59:10.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.026+0000 7f37f4453640 1 -- start start 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f4453640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37ec100b50 0x7f37ec1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f4453640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 0x7f37ec197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f4453640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec102c20 0x7f37ec19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f4453640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec110d00 con 0x7f37ec1022a0 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f4453640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec110b80 con 0x7f37ec100b50 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f19ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 0x7f37ec197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f21ef640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37ec100b50 0x7f37ec1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f19ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 0x7f37ec197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52526/0 (socket says 172.21.15.69:52526) 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f19ee640 1 -- 172.21.15.69:0/2704741305 learned_addr learned my addr 172.21.15.69:0/2704741305 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:10.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f29f0640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec102c20 0x7f37ec19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.027+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec110e80 con 0x7f37ec102c20 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.028+0000 7f37f29f0640 1 -- 172.21.15.69:0/2704741305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37ec100b50 msgr2=0x7f37ec1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.028+0000 7f37f29f0640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37ec100b50 0x7f37ec1971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.028+0000 7f37f29f0640 1 -- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 msgr2=0x7f37ec197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.028+0000 7f37f29f0640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 0x7f37ec197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.028+0000 7f37f29f0640 1 -- 172.21.15.69:0/2704741305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec19c170 con 0x7f37ec102c20 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.028+0000 7f37f29f0640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec102c20 0x7f37ec19ba90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009fa0 tx=0x7f37e400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.029+0000 7f37e37fe640 1 -- 172.21.15.69:0/2704741305 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4003b70 con 0x7f37ec102c20 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.029+0000 7f37e37fe640 1 -- 172.21.15.69:0/2704741305 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4003d10 con 0x7f37ec102c20 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.029+0000 7f37e37fe640 1 -- 172.21.15.69:0/2704741305 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e40046e0 con 0x7f37ec102c20 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.029+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec19c460 con 0x7f37ec102c20 2024-08-02T17:59:10.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.029+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec19c890 con 0x7f37ec102c20 2024-08-02T17:59:10.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.031+0000 7f37e37fe640 1 -- 172.21.15.69:0/2704741305 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f37e4022020 con 0x7f37ec102c20 2024-08-02T17:59:10.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.032+0000 7f37f6851740 1 Processor -- start 2024-08-02T17:59:10.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.032+0000 7f37f6851740 1 -- start start 2024-08-02T17:59:10.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f6851740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f6851740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f6851740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f6851740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555c29848980 con 0x555c298e3a50 2024-08-02T17:59:10.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f6851740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555c297e0a70 con 0x555c298e3130 2024-08-02T17:59:10.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f6851740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c29844940 con 0x555c298f6c00 2024-08-02T17:59:10.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f19ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f19ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298e3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55520/0 (socket says 172.21.15.69:55520) 2024-08-02T17:59:10.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f19ee640 1 -- 172.21.15.69:0/972462466 learned_addr learned my addr 172.21.15.69:0/972462466 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:10.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.033+0000 7f37f21ef640 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37f19ee640 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37e37fe640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37d80a2e40 0x7f37d80a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37f19ee640 1 -- 172.21.15.69:0/972462466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 msgr2=0x555c298f8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37f19ee640 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37f19ee640 1 -- 172.21.15.69:0/972462466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 msgr2=0x555c298f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37f19ee640 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37f19ee640 1 -- 172.21.15.69:0/972462466 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c298436e0 con 0x555c298e3130 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37e37fe640 1 -- 172.21.15.69:0/2704741305 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f37e40d3960 con 0x7f37ec102c20 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37f29f0640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37d80a2e40 0x7f37d80a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37f19ee640 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298e3510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f37e8005e00 tx=0x7f37e80058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=251d5266bba4ee7f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.034+0000 7f37e0dea640 1 -- 172.21.15.69:0/972462466 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8004230 con 0x555c298e3130 2024-08-02T17:59:10.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37e0dea640 1 -- 172.21.15.69:0/972462466 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8030e80 con 0x555c298e3130 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37e0dea640 1 -- 172.21.15.69:0/972462466 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8003bf0 con 0x555c298e3130 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f6851740 1 -- 172.21.15.69:0/972462466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 msgr2=0x555c298e3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f29f0640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37d80a2e40 0x7f37d80a5300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f37e409f7c0 tx=0x7f37e4000f40 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f6851740 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298e3510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f37e8005e00 tx=0x7f37e80058b0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f6851740 1 -- 172.21.15.69:0/972462466 shutdown_connections 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f6851740 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f6851740 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f6851740 1 --2- 172.21.15.69:0/972462466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298e3510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f6851740 1 -- 172.21.15.69:0/972462466 >> 172.21.15.69:0/972462466 conn(0x555c298df7a0 msgr2=0x555c298dfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.035+0000 7f37f6851740 1 -- 172.21.15.69:0/972462466 shutdown_connections 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 -- 172.21.15.69:0/972462466 wait complete. 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 -- start start 2024-08-02T17:59:10.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 Processor -- start 2024-08-02T17:59:10.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555c298f9760 con 0x555c298e3a50 2024-08-02T17:59:10.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555c29803aa0 con 0x555c298e3130 2024-08-02T17:59:10.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555c298f98e0 con 0x555c298f6c00 2024-08-02T17:59:10.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f6851740 10 client.0 _ll_register_callbacks cb 0x555c297f2e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f21ef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f21ef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52532/0 (socket says 172.21.15.69:52532) 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.036+0000 7f37f21ef640 1 -- 172.21.15.69:0/3602813093 learned_addr learned my addr 172.21.15.69:0/3602813093 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f21ef640 1 -- 172.21.15.69:0/3602813093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 msgr2=0x555c298f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f21ef640 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f21ef640 1 -- 172.21.15.69:0/3602813093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 msgr2=0x555c298f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f21ef640 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f21ef640 1 -- 172.21.15.69:0/3602813093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c297e7650 con 0x555c298e3a50 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f21ef640 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f4070 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00ba50 tx=0x7f37dc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdccc6e346fb0a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:10.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3602813093 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc005020 con 0x555c298e3a50 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3602813093 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f37dc0051c0 con 0x555c298e3a50 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.037+0000 7f37f6851740 1 -- 172.21.15.69:0/3602813093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e8030e80 con 0x555c298e3a50 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3602813093 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37dc005400 con 0x555c298e3a50 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37f6851740 10 client.18498 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37f6851740 1 -- 172.21.15.69:0/3602813093 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c295ba3c0 con 0x555c298e3a50 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3602813093 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8030e80 con 0x555c298e3a50 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3602813093 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37dc01b7d0 con 0x555c298e3a50 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37f6851740 10 client.18498 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37f6851740 10 client.18498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37f6851740 20 client.18498 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37f6851740 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555c29906680 0x555c29908a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37f6851740 4 client.18498 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:10.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.038+0000 7f37f6851740 1 -- 172.21.15.69:0/3602813093 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c29855ef0 con 0x555c29906680 2024-08-02T17:59:10.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.039+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555c29906680 0x555c29908a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.039+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555c29906680 0x555c29908a90 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a97389011a8668d server_cookie=4895813c7c5aa20a in_seq=0 out_seq=0 2024-08-02T17:59:10.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.039+0000 7f37d5ffb640 10 client.18498 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:10.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.040+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3602813093 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555c29855ef0 con 0x555c29906680 2024-08-02T17:59:10.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.040+0000 7f37d5ffb640 10 client.18498 handle_command_reply: tid=0 2024-08-02T17:59:10.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.041+0000 7f37f6851740 1 client.18498 shutdown 2024-08-02T17:59:10.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 -- 172.21.15.69:0/3602813093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 msgr2=0x555c298f4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f4070 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00ba50 tx=0x7f37dc00bf10 comp rx=0 tx=0).stop 2024-08-02T17:59:10.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 -- 172.21.15.69:0/3602813093 shutdown_connections 2024-08-02T17:59:10.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c298f4070 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555c29906680 0x555c29908a90 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 --2- 172.21.15.69:0/3602813093 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 -- 172.21.15.69:0/3602813093 >> 172.21.15.69:0/3602813093 conn(0x555c298df7a0 msgr2=0x555c298e5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:10.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37d5ffb640 0 client.18498 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:10.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 -- 172.21.15.69:0/3602813093 shutdown_connections 2024-08-02T17:59:10.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 1 -- 172.21.15.69:0/3602813093 wait complete. 2024-08-02T17:59:10.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.042+0000 7f37f6851740 20 client.18498 trim_cache size 0 max 16384 2024-08-02T17:59:10.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.074+0000 7f37f6851740 1 Processor -- start 2024-08-02T17:59:10.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.075+0000 7f37f6851740 1 -- start start 2024-08-02T17:59:10.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.075+0000 7f37f6851740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.075+0000 7f37f6851740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c2990e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f19ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f19ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55534/0 (socket says 172.21.15.69:55534) 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f21ef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c2990e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f21ef640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c2990e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52546/0 (socket says 172.21.15.69:52546) 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f19ee640 1 -- 172.21.15.69:0/3507035502 learned_addr learned my addr 172.21.15.69:0/3507035502 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f6851740 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c2990e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f6851740 1 -- 172.21.15.69:0/3507035502 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1af690 con 0x555c298e3a50 2024-08-02T17:59:10.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f6851740 1 -- 172.21.15.69:0/3507035502 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1acf60 con 0x555c298e3130 2024-08-02T17:59:10.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.076+0000 7f37f6851740 1 -- 172.21.15.69:0/3507035502 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1aadd0 con 0x555c298f6c00 2024-08-02T17:59:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.077+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c2990e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.078+0000 7f37f19ee640 1 -- 172.21.15.69:0/3507035502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 msgr2=0x555c2990e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.078+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c2990e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.078+0000 7f37f19ee640 1 -- 172.21.15.69:0/3507035502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 msgr2=0x555c2990e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.078+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c2990e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.078+0000 7f37f19ee640 1 -- 172.21.15.69:0/3507035502 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c298f20a0 con 0x555c298e3130 2024-08-02T17:59:10.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.078+0000 7f37f21ef640 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c2990e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.079+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f5990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37e80094c0 tx=0x7f37e802a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b384bec7d2024325 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.079+0000 7f37d7fff640 1 -- 172.21.15.69:0/3507035502 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8030580 con 0x555c298e3130 2024-08-02T17:59:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.079+0000 7f37d7fff640 1 -- 172.21.15.69:0/3507035502 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8030720 con 0x555c298e3130 2024-08-02T17:59:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.079+0000 7f37d7fff640 1 -- 172.21.15.69:0/3507035502 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e803c680 con 0x555c298e3130 2024-08-02T17:59:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.080+0000 7f37f6851740 1 -- 172.21.15.69:0/3507035502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 msgr2=0x555c298f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.080+0000 7f37f6851740 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f5990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f37e80094c0 tx=0x7f37e802a3d0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.083+0000 7f37f6851740 1 -- 172.21.15.69:0/3507035502 shutdown_connections 2024-08-02T17:59:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.083+0000 7f37f6851740 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3a50 0x555c2990e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.083+0000 7f37f6851740 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c2990e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.083+0000 7f37f6851740 1 --2- 172.21.15.69:0/3507035502 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3130 0x555c298f5990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.083+0000 7f37f6851740 1 -- 172.21.15.69:0/3507035502 >> 172.21.15.69:0/3507035502 conn(0x555c298df7a0 msgr2=0x555c298f5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:10.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.088+0000 7f37f6851740 1 -- 172.21.15.69:0/3507035502 shutdown_connections 2024-08-02T17:59:10.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.088+0000 7f37f6851740 1 -- 172.21.15.69:0/3507035502 wait complete. 2024-08-02T17:59:10.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.088+0000 7f37f6851740 1 -- start start 2024-08-02T17:59:10.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.088+0000 7f37f6851740 1 Processor -- start 2024-08-02T17:59:10.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f6851740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3130 0x555c298f0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f6851740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3a50 0x555c298f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f19ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3130 0x555c298f0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f19ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3130 0x555c298f0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52560/0 (socket says 172.21.15.69:52560) 2024-08-02T17:59:10.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f19ee640 1 -- 172.21.15.69:0/3954388400 learned_addr learned my addr 172.21.15.69:0/3954388400 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:10.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f6851740 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1ab420 con 0x555c298e3130 2024-08-02T17:59:10.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1ac890 con 0x555c298e3a50 2024-08-02T17:59:10.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1abaf0 con 0x555c298f6c00 2024-08-02T17:59:10.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.089+0000 7f37f6851740 10 client.0 _ll_register_callbacks cb 0x7f37dc004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:10.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.093+0000 7f37f21ef640 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3a50 0x555c298f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.093+0000 7f37f21ef640 1 -- 172.21.15.69:0/3954388400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 msgr2=0x555c298f15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:10.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.093+0000 7f37f21ef640 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.093+0000 7f37f21ef640 1 -- 172.21.15.69:0/3954388400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3130 msgr2=0x555c298f0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:10.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.093+0000 7f37f21ef640 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3130 0x555c298f0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.093+0000 7f37f21ef640 1 -- 172.21.15.69:0/3954388400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c29981380 con 0x555c298e3a50 2024-08-02T17:59:10.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.093+0000 7f37f21ef640 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3a50 0x555c298f10a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f37dc000c00 tx=0x7f37dc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5fdcc5c7af3301b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:10.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.093+0000 7f37d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:59:10.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.095+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3954388400 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc029430 con 0x555c298e3a50 2024-08-02T17:59:10.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.095+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3954388400 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f37dc017040 con 0x555c298e3a50 2024-08-02T17:59:10.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.095+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e8030720 con 0x555c298e3a50 2024-08-02T17:59:10.100 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.098+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3954388400 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f37dc0295d0 con 0x555c298e3a50 2024-08-02T17:59:10.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.100+0000 7f37f6851740 10 client.27962 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:10.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.101+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c299825c0 con 0x555c298e3a50 2024-08-02T17:59:10.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.102+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3954388400 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc02d020 con 0x555c298e3a50 2024-08-02T17:59:10.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.103+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3954388400 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f37dc032850 con 0x555c298e3a50 2024-08-02T17:59:10.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.103+0000 7f37f6851740 10 client.27962 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:10.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.103+0000 7f37f6851740 10 client.27962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:10.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.103+0000 7f37f6851740 20 client.27962 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:10.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.103+0000 7f37f6851740 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555c29982cf0 0x555c29985110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:10.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.103+0000 7f37f6851740 4 client.27962 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:59:10.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.103+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f37e8030720 con 0x555c29982cf0 2024-08-02T17:59:10.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.108+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555c29982cf0 0x555c29985110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:10.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.115+0000 7f37d5ffb640 10 client.27962 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:10.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.115+0000 7f37f19ee640 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555c29982cf0 0x555c29985110 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba456e26df729888 server_cookie=c41654a61bc8c39c in_seq=0 out_seq=0 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.124+0000 7f37d5ffb640 1 -- 172.21.15.69:0/3954388400 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f37e8030720 con 0x555c29982cf0 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.124+0000 7f37d5ffb640 10 client.27962 handle_command_reply: tid=0 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.124+0000 7f37f6851740 1 client.27962 shutdown 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3a50 msgr2=0x555c298f10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3a50 0x555c298f10a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f37dc000c00 tx=0x7f37dc005850 comp rx=0 tx=0).stop 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 shutdown_connections 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555c29982cf0 0x555c29985110 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555c298f6c00 0x555c298f15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555c298e3a50 0x555c298f10a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37d5ffb640 0 client.27962 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 --2- 172.21.15.69:0/3954388400 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555c298e3130 0x555c298f0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 >> 172.21.15.69:0/3954388400 conn(0x555c298df7a0 msgr2=0x555c298de900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:10.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 shutdown_connections 2024-08-02T17:59:10.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 1 -- 172.21.15.69:0/3954388400 wait complete. 2024-08-02T17:59:10.128 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.125+0000 7f37f6851740 20 client.27962 trim_cache size 0 max 16384 2024-08-02T17:59:10.128 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:59:10.128 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:59:10.128 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:59:10.128 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:59:10.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.126+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37d80a2e40 msgr2=0x7f37d80a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.126+0000 7f37f4453640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37d80a2e40 0x7f37d80a5300 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f37e409f7c0 tx=0x7f37e4000f40 comp rx=0 tx=0).stop 2024-08-02T17:59:10.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.126+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec102c20 msgr2=0x7f37ec19ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:10.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.126+0000 7f37f4453640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec102c20 0x7f37ec19ba90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009fa0 tx=0x7f37e400c6a0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.128+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 shutdown_connections 2024-08-02T17:59:10.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.128+0000 7f37f4453640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f37ec1022a0 0x7f37ec197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.128+0000 7f37f4453640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f37d80a2e40 0x7f37d80a5300 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.128+0000 7f37f4453640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37ec102c20 0x7f37ec19ba90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.128+0000 7f37f4453640 1 --2- 172.21.15.69:0/2704741305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f37ec100b50 0x7f37ec1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:10.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.128+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 >> 172.21.15.69:0/2704741305 conn(0x7f37ec0fc500 msgr2=0x7f37ec0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:10.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.129+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 shutdown_connections 2024-08-02T17:59:10.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:10.131+0000 7f37f4453640 1 -- 172.21.15.69:0/2704741305 wait complete. 2024-08-02T17:59:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:10 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:59:10.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:10 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:59:10.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:10 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:59:11.422 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:11 smithi083 ceph-mon[37031]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.2 MiB/s wr, 611 op/s 2024-08-02T17:59:11.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:11 smithi077 ceph-mon[37108]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.2 MiB/s wr, 611 op/s 2024-08-02T17:59:11.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:11 smithi069 ceph-mon[32746]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.2 MiB/s wr, 611 op/s 2024-08-02T17:59:12.399 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:59:12.400 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:59:12.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:12 smithi083 ceph-mon[37031]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.2 MiB/s wr, 611 op/s 2024-08-02T17:59:12.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:12 smithi077 ceph-mon[37108]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.2 MiB/s wr, 611 op/s 2024-08-02T17:59:12.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:12 smithi069 ceph-mon[32746]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 5.2 MiB/s wr, 611 op/s 2024-08-02T17:59:12.871 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:13.119 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-02T17:59:13.119 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-02T17:59:13.120 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:59:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.123+0000 7f1612dac640 1 -- 172.21.15.69:0/1895305417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c108090 msgr2=0x7f160c108470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.123+0000 7f1612dac640 1 --2- 172.21.15.69:0/1895305417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c108090 0x7f160c108470 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009900 tx=0x7f15fc02a440 comp rx=0 tx=0).stop 2024-08-02T17:59:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.125+0000 7f1612dac640 1 -- 172.21.15.69:0/1895305417 shutdown_connections 2024-08-02T17:59:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.125+0000 7f1612dac640 1 --2- 172.21.15.69:0/1895305417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f160c10d1e0 0x7f160c114d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.125+0000 7f1612dac640 1 --2- 172.21.15.69:0/1895305417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160c108a40 0x7f160c10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.128 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.125+0000 7f1612dac640 1 --2- 172.21.15.69:0/1895305417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c108090 0x7f160c108470 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.129 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.125+0000 7f1612dac640 1 -- 172.21.15.69:0/1895305417 >> 172.21.15.69:0/1895305417 conn(0x7f160c077800 msgr2=0x7f160c079c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:13.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.129+0000 7f1612dac640 1 -- 172.21.15.69:0/1895305417 shutdown_connections 2024-08-02T17:59:13.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.131+0000 7f1612dac640 1 -- 172.21.15.69:0/1895305417 wait complete. 2024-08-02T17:59:13.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.132+0000 7f1612dac640 1 Processor -- start 2024-08-02T17:59:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.134+0000 7f1612dac640 1 -- start start 2024-08-02T17:59:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.134+0000 7f1612dac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f160c108090 0x7f160c073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.134+0000 7f1612dac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160c10d1e0 0x7f160c073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.134+0000 7f1612dac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c06c7f0 0x7f160c06cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.134+0000 7f1612dac640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f160c118250 con 0x7f160c108090 2024-08-02T17:59:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.134+0000 7f1612dac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1180d0 con 0x7f160c06c7f0 2024-08-02T17:59:13.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.134+0000 7f1612dac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f160c1183d0 con 0x7f160c10d1e0 2024-08-02T17:59:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.135+0000 7f160bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160c10d1e0 0x7f160c073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.135+0000 7f160bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160c10d1e0 0x7f160c073ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:41690/0 (socket says 172.21.15.69:41690) 2024-08-02T17:59:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.135+0000 7f160bfff640 1 -- 172.21.15.69:0/3729416476 learned_addr learned my addr 172.21.15.69:0/3729416476 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.136+0000 7f160bfff640 1 -- 172.21.15.69:0/3729416476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c06c7f0 msgr2=0x7f160c06cca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:59:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.136+0000 7f160bfff640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c06c7f0 0x7f160c06cca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.136+0000 7f160bfff640 1 -- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f160c108090 msgr2=0x7f160c073660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T17:59:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.136+0000 7f160bfff640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f160c108090 0x7f160c073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.136+0000 7f160bfff640 1 -- 172.21.15.69:0/3729416476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f160c071180 con 0x7f160c10d1e0 2024-08-02T17:59:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.136+0000 7f160bfff640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160c10d1e0 0x7f160c073ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f160400eb90 tx=0x7f160400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.137+0000 7f1609ffb640 1 -- 172.21.15.69:0/3729416476 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604010630 con 0x7f160c10d1e0 2024-08-02T17:59:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.137+0000 7f1612dac640 1 -- 172.21.15.69:0/3729416476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f160c071470 con 0x7f160c10d1e0 2024-08-02T17:59:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.137+0000 7f1612dac640 1 -- 172.21.15.69:0/3729416476 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f160c0719c0 con 0x7f160c10d1e0 2024-08-02T17:59:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.137+0000 7f1609ffb640 1 -- 172.21.15.69:0/3729416476 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f160400ee90 con 0x7f160c10d1e0 2024-08-02T17:59:13.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.137+0000 7f1609ffb640 1 -- 172.21.15.69:0/3729416476 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1604018b90 con 0x7f160c10d1e0 2024-08-02T17:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.142+0000 7f1615bd6740 1 Processor -- start 2024-08-02T17:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.142+0000 7f1615bd6740 1 -- start start 2024-08-02T17:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.142+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e77b0 0x5623a57e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.142+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.142+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 0x5623a57f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.142+0000 7f1615bd6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5623a574b9b0 con 0x5623a57f2970 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.142+0000 7f1615bd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5623a56f3800 con 0x5623a57e77b0 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.142+0000 7f1615bd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623a5747970 con 0x5623a57e80d0 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.143+0000 7f1611349640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e77b0 0x5623a57e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.143+0000 7f1611349640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e77b0 0x5623a57e7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55546/0 (socket says 172.21.15.69:55546) 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.143+0000 7f1611349640 1 -- 172.21.15.69:0/2709338380 learned_addr learned my addr 172.21.15.69:0/2709338380 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.144+0000 7f1609ffb640 1 -- 172.21.15.69:0/3729416476 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f160400f3c0 con 0x7f160c10d1e0 2024-08-02T17:59:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.146+0000 7f1610b48640 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.153+0000 7f1609ffb640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f15f80a2f00 0x7f15f80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.158 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.157+0000 7f160bfff640 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 0x5623a57f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.157+0000 7f1610b48640 1 -- 172.21.15.69:0/2709338380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e77b0 msgr2=0x5623a57e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.157+0000 7f1610b48640 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e77b0 0x5623a57e7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.157+0000 7f1610b48640 1 -- 172.21.15.69:0/2709338380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 msgr2=0x5623a57f4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.157+0000 7f1610b48640 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 0x5623a57f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.157+0000 7f1610b48640 1 -- 172.21.15.69:0/2709338380 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a545f210 con 0x5623a57e80d0 2024-08-02T17:59:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.157+0000 7f1609ffb640 1 -- 172.21.15.69:0/3729416476 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f16040cfec0 con 0x7f160c10d1e0 2024-08-02T17:59:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.157+0000 7f1611349640 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e77b0 0x5623a57e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.159+0000 7f1611349640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f15f80a2f00 0x7f15f80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.159+0000 7f1610b48640 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f2430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f15fc02a9b0 tx=0x7f15fc0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0c5425c09fe0bb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.160+0000 7f1602ffd640 1 -- 172.21.15.69:0/2709338380 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc03e030 con 0x5623a57e80d0 2024-08-02T17:59:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.160+0000 7f1602ffd640 1 -- 172.21.15.69:0/2709338380 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc0035f0 con 0x5623a57e80d0 2024-08-02T17:59:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.160+0000 7f1602ffd640 1 -- 172.21.15.69:0/2709338380 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc035040 con 0x5623a57e80d0 2024-08-02T17:59:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.161+0000 7f1615bd6740 1 -- 172.21.15.69:0/2709338380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 msgr2=0x5623a57f2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.161+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f2430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f15fc02a9b0 tx=0x7f15fc0056d0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.163+0000 7f1611349640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f15f80a2f00 0x7f15f80a53c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005fd0 tx=0x7f15f40046c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.163+0000 7f1615bd6740 1 -- 172.21.15.69:0/2709338380 shutdown_connections 2024-08-02T17:59:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.163+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 0x5623a57f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.163+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f2430 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.163+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2709338380 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e77b0 0x5623a57e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.163+0000 7f1615bd6740 1 -- 172.21.15.69:0/2709338380 >> 172.21.15.69:0/2709338380 conn(0x5623a57e3d20 msgr2=0x5623a57e4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.164+0000 7f1615bd6740 1 -- 172.21.15.69:0/2709338380 shutdown_connections 2024-08-02T17:59:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.164+0000 7f1615bd6740 1 -- 172.21.15.69:0/2709338380 wait complete. 2024-08-02T17:59:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.164+0000 7f1615bd6740 1 -- start start 2024-08-02T17:59:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.164+0000 7f1615bd6740 1 Processor -- start 2024-08-02T17:59:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57ef850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e80d0 0x5623a57efd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 0x5623a57f02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1615bd6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5623a57f5540 con 0x5623a57e80d0 2024-08-02T17:59:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1615bd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5623a5707500 con 0x5623a57f2970 2024-08-02T17:59:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1615bd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5623a57f56c0 con 0x5623a57e77b0 2024-08-02T17:59:13.169 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1615bd6740 10 client.0 _ll_register_callbacks cb 0x5623a545ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1610b48640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1610b48640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57ef850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:41708/0 (socket says 172.21.15.69:41708) 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1610b48640 1 -- 172.21.15.69:0/2670633914 learned_addr learned my addr 172.21.15.69:0/2670633914 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.165+0000 7f1611349640 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 0x5623a57f02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.166+0000 7f1610b48640 1 -- 172.21.15.69:0/2670633914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 msgr2=0x5623a57f02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.166+0000 7f1610b48640 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 0x5623a57f02d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.166+0000 7f1610b48640 1 -- 172.21.15.69:0/2670633914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e80d0 msgr2=0x5623a57efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.166+0000 7f1610b48640 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e80d0 0x5623a57efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.166+0000 7f1610b48640 1 -- 172.21.15.69:0/2670633914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a552b620 con 0x5623a57e77b0 2024-08-02T17:59:13.170 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.166+0000 7f1610b48640 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57ef850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009850 tx=0x7f15fc0043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a3ee74dc6adbb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.171+0000 7f15ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:59:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.171+0000 7f15ebfff640 1 -- 172.21.15.69:0/2670633914 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc035070 con 0x5623a57e77b0 2024-08-02T17:59:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.171+0000 7f15ebfff640 1 -- 172.21.15.69:0/2670633914 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f15fc0039a0 con 0x5623a57e77b0 2024-08-02T17:59:13.175 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.173+0000 7f1615bd6740 1 -- 172.21.15.69:0/2670633914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15fc0035f0 con 0x5623a57e77b0 2024-08-02T17:59:13.177 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.175+0000 7f15ebfff640 1 -- 172.21.15.69:0/2670633914 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f15fc03e040 con 0x5623a57e77b0 2024-08-02T17:59:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.177+0000 7f1615bd6740 10 client.27757 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:13.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.177+0000 7f1615bd6740 1 -- 172.21.15.69:0/2670633914 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623a57f5540 con 0x5623a57e77b0 2024-08-02T17:59:13.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.178+0000 7f15ebfff640 1 -- 172.21.15.69:0/2670633914 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc0035f0 con 0x5623a57e77b0 2024-08-02T17:59:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.179+0000 7f15ebfff640 1 -- 172.21.15.69:0/2670633914 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f15fc043410 con 0x5623a57e77b0 2024-08-02T17:59:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.179+0000 7f1615bd6740 10 client.27757 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.179+0000 7f1615bd6740 10 client.27757 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.179+0000 7f1615bd6740 20 client.27757 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.179+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5623a5824350 0x5623a5826740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.179+0000 7f1615bd6740 4 client.27757 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.179+0000 7f1615bd6740 1 -- 172.21.15.69:0/2670633914 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623a57590e0 con 0x5623a5824350 2024-08-02T17:59:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.180+0000 7f160bfff640 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5623a5824350 0x5623a5826740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.180+0000 7f160bfff640 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5623a5824350 0x5623a5826740 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7a5a50eb95be170 server_cookie=d310427654b942be in_seq=0 out_seq=0 2024-08-02T17:59:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.180+0000 7f15ebfff640 10 client.27757 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.185+0000 7f15ebfff640 1 -- 172.21.15.69:0/2670633914 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5623a57590e0 con 0x5623a5824350 2024-08-02T17:59:13.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.185+0000 7f15ebfff640 10 client.27757 handle_command_reply: tid=0 2024-08-02T17:59:13.187 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.186+0000 7f1615bd6740 1 client.27757 shutdown 2024-08-02T17:59:13.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.186+0000 7f1615bd6740 1 -- 172.21.15.69:0/2670633914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 msgr2=0x5623a57ef850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.188 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.186+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57ef850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009850 tx=0x7f15fc0043b0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.187+0000 7f1615bd6740 1 -- 172.21.15.69:0/2670633914 shutdown_connections 2024-08-02T17:59:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.187+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e80d0 0x5623a57efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.187+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5623a5824350 0x5623a5826740 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.187+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 0x5623a57f02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.187+0000 7f1615bd6740 1 --2- 172.21.15.69:0/2670633914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57ef850 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.189 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.187+0000 7f1615bd6740 1 -- 172.21.15.69:0/2670633914 >> 172.21.15.69:0/2670633914 conn(0x5623a57e3d20 msgr2=0x5623a57e2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.187+0000 7f15ebfff640 0 client.27757 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.190+0000 7f1615bd6740 1 -- 172.21.15.69:0/2670633914 shutdown_connections 2024-08-02T17:59:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.195+0000 7f1615bd6740 1 -- 172.21.15.69:0/2670633914 wait complete. 2024-08-02T17:59:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.196+0000 7f1615bd6740 20 client.27757 trim_cache size 0 max 16384 2024-08-02T17:59:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.284+0000 7f1615bd6740 1 Processor -- start 2024-08-02T17:59:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.285+0000 7f1615bd6740 1 -- start start 2024-08-02T17:59:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.285+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e77b0 0x5623a57de900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.285+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.286+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 0x5623a582bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.286+0000 7f160bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.286+0000 7f160bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:41724/0 (socket says 172.21.15.69:41724) 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.286+0000 7f160bfff640 1 -- 172.21.15.69:0/1640066342 learned_addr learned my addr 172.21.15.69:0/1640066342 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.286+0000 7f1611349640 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 0x5623a582bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.286+0000 7f1615bd6740 1 -- 172.21.15.69:0/1640066342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec012b60 con 0x5623a57e77b0 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.286+0000 7f1615bd6740 1 -- 172.21.15.69:0/1640066342 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec010430 con 0x5623a57f2970 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.286+0000 7f1615bd6740 1 -- 172.21.15.69:0/1640066342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec00e2a0 con 0x5623a57e80d0 2024-08-02T17:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.287+0000 7f1610b48640 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e77b0 0x5623a57de900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.292+0000 7f160bfff640 1 -- 172.21.15.69:0/1640066342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 msgr2=0x5623a582bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.292+0000 7f160bfff640 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 0x5623a582bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.292+0000 7f160bfff640 1 -- 172.21.15.69:0/1640066342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e77b0 msgr2=0x5623a57de900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.292+0000 7f160bfff640 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e77b0 0x5623a57de900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.292+0000 7f160bfff640 1 -- 172.21.15.69:0/1640066342 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623a57e7530 con 0x5623a57e80d0 2024-08-02T17:59:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.292+0000 7f160bfff640 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f16b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1604098370 tx=0x7f1604000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=694d492dfccb658e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.294+0000 7f16027fc640 1 -- 172.21.15.69:0/1640066342 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16040a02c0 con 0x5623a57e80d0 2024-08-02T17:59:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.294+0000 7f16027fc640 1 -- 172.21.15.69:0/1640066342 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16040a4b80 con 0x5623a57e80d0 2024-08-02T17:59:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.294+0000 7f16027fc640 1 -- 172.21.15.69:0/1640066342 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16040bab50 con 0x5623a57e80d0 2024-08-02T17:59:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.295+0000 7f1615bd6740 1 -- 172.21.15.69:0/1640066342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 msgr2=0x5623a57f16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.295+0000 7f1615bd6740 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f16b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1604098370 tx=0x7f1604000f40 comp rx=0 tx=0).stop 2024-08-02T17:59:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.301+0000 7f1615bd6740 1 -- 172.21.15.69:0/1640066342 shutdown_connections 2024-08-02T17:59:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.301+0000 7f1615bd6740 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57f2970 0x5623a582bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.301+0000 7f1615bd6740 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e80d0 0x5623a57f16b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.301+0000 7f1615bd6740 1 --2- 172.21.15.69:0/1640066342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57e77b0 0x5623a57de900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.301+0000 7f1615bd6740 1 -- 172.21.15.69:0/1640066342 >> 172.21.15.69:0/1640066342 conn(0x5623a57e3d20 msgr2=0x5623a57f0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.309+0000 7f1615bd6740 1 -- 172.21.15.69:0/1640066342 shutdown_connections 2024-08-02T17:59:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.310+0000 7f1615bd6740 1 -- 172.21.15.69:0/1640066342 wait complete. 2024-08-02T17:59:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.311+0000 7f1615bd6740 1 -- start start 2024-08-02T17:59:13.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.311+0000 7f1615bd6740 1 Processor -- start 2024-08-02T17:59:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.311+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.311+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e80d0 0x5623a57f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.311+0000 7f1615bd6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 0x5623a588dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.311+0000 7f1610b48640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.312+0000 7f1610b48640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:41726/0 (socket says 172.21.15.69:41726) 2024-08-02T17:59:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.312+0000 7f1610b48640 1 -- 172.21.15.69:0/288809596 learned_addr learned my addr 172.21.15.69:0/288809596 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.312+0000 7f1611349640 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 0x5623a588dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.312+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec00e8f0 con 0x5623a57f2970 2024-08-02T17:59:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.312+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec00fd60 con 0x5623a57e80d0 2024-08-02T17:59:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.312+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec00efc0 con 0x5623a57e77b0 2024-08-02T17:59:13.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.312+0000 7f1615bd6740 10 client.0 _ll_register_callbacks cb 0x7f15fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.316+0000 7f1610b48640 1 -- 172.21.15.69:0/288809596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e80d0 msgr2=0x5623a57f6fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:59:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.316+0000 7f1610b48640 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e80d0 0x5623a57f6fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.316+0000 7f1610b48640 1 -- 172.21.15.69:0/288809596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 msgr2=0x5623a588dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.316+0000 7f1610b48640 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 0x5623a588dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.316+0000 7f1610b48640 1 -- 172.21.15.69:0/288809596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623a588e3e0 con 0x5623a57e77b0 2024-08-02T17:59:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.320+0000 7f1610b48640 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57f6aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f15fc001eb0 tx=0x7f15fc044ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8aa7e2b17d8ab594 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.320+0000 7f15ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:59:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.321+0000 7f15ebfff640 1 -- 172.21.15.69:0/288809596 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc035070 con 0x5623a57e77b0 2024-08-02T17:59:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.321+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16040a4b80 con 0x5623a57e77b0 2024-08-02T17:59:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.321+0000 7f15ebfff640 1 -- 172.21.15.69:0/288809596 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f15fc02ad40 con 0x5623a57e77b0 2024-08-02T17:59:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.321+0000 7f15ebfff640 1 -- 172.21.15.69:0/288809596 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f15fc05f050 con 0x5623a57e77b0 2024-08-02T17:59:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.321+0000 7f1615bd6740 10 client.27763 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.321+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5623a588f570 con 0x5623a57e77b0 2024-08-02T17:59:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.322+0000 7f15ebfff640 1 -- 172.21.15.69:0/288809596 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15fc043e50 con 0x5623a57e77b0 2024-08-02T17:59:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.322+0000 7f15ebfff640 1 -- 172.21.15.69:0/288809596 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f15fc004980 con 0x5623a57e77b0 2024-08-02T17:59:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.322+0000 7f1615bd6740 10 client.27763 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.322+0000 7f1615bd6740 10 client.27763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.322+0000 7f1615bd6740 20 client.27763 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.322+0000 7f1615bd6740 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5623a588fcc0 0x5623a5892120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.322+0000 7f1615bd6740 4 client.27763 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:59:13.324 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.322+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5623a5753430 con 0x5623a588fcc0 2024-08-02T17:59:13.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.330+0000 7f160bfff640 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5623a588fcc0 0x5623a5892120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.338+0000 7f160bfff640 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5623a588fcc0 0x5623a5892120 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ec31ac21b4b619 server_cookie=b36e4cf54f49fa3 in_seq=0 out_seq=0 2024-08-02T17:59:13.340 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.339+0000 7f15ebfff640 10 client.27763 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.452+0000 7f15ebfff640 1 -- 172.21.15.69:0/288809596 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5623a5753430 con 0x5623a588fcc0 2024-08-02T17:59:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f15ebfff640 10 client.27763 handle_command_reply: tid=0 2024-08-02T17:59:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 client.27763 shutdown 2024-08-02T17:59:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 msgr2=0x5623a57f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57f6aa0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f15fc001eb0 tx=0x7f15fc044ff0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 shutdown_connections 2024-08-02T17:59:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5623a57f2970 0x5623a588dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5623a588fcc0 0x5623a5892120 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5623a57e80d0 0x5623a57f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 --2- 172.21.15.69:0/288809596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5623a57e77b0 0x5623a57f6aa0 secure :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f15fc001eb0 tx=0x7f15fc044ff0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 >> 172.21.15.69:0/288809596 conn(0x5623a57e3d20 msgr2=0x5623a57f0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.453+0000 7f15ebfff640 0 client.27763 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.454+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 shutdown_connections 2024-08-02T17:59:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.454+0000 7f1615bd6740 1 -- 172.21.15.69:0/288809596 wait complete. 2024-08-02T17:59:13.456 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.454+0000 7f1615bd6740 20 client.27763 trim_cache size 0 max 16384 2024-08-02T17:59:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:59:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:59:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:59:13.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:59:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.455+0000 7f1612dac640 1 -- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f15f80a2f00 msgr2=0x7f15f80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f15f80a2f00 0x7f15f80a53c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005fd0 tx=0x7f15f40046c0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 -- 172.21.15.69:0/3729416476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160c10d1e0 msgr2=0x7f160c073ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160c10d1e0 0x7f160c073ba0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f160400eb90 tx=0x7f160400c6a0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 -- 172.21.15.69:0/3729416476 shutdown_connections 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f15f80a2f00 0x7f15f80a53c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f160c06c7f0 0x7f160c06cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f160c10d1e0 0x7f160c073ba0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 --2- 172.21.15.69:0/3729416476 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f160c108090 0x7f160c073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 -- 172.21.15.69:0/3729416476 >> 172.21.15.69:0/3729416476 conn(0x7f160c077800 msgr2=0x7f160c078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 -- 172.21.15.69:0/3729416476 shutdown_connections 2024-08-02T17:59:13.459 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:13.456+0000 7f1612dac640 1 -- 172.21.15.69:0/3729416476 wait complete. 2024-08-02T17:59:13.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 -- 172.21.15.69:0/1183259193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc113710 msgr2=0x7f2dfc115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 --2- 172.21.15.69:0/1183259193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc113710 0x7f2dfc115ba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2df4009ca0 tx=0x7f2df402a8c0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 -- 172.21.15.69:0/1183259193 shutdown_connections 2024-08-02T17:59:13.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 --2- 172.21.15.69:0/1183259193 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2dfc10abb0 0x7f2dfc1130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 --2- 172.21.15.69:0/1183259193 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc113710 0x7f2dfc115ba0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 --2- 172.21.15.69:0/1183259193 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dfc10a290 0x7f2dfc10a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 -- 172.21.15.69:0/1183259193 >> 172.21.15.69:0/1183259193 conn(0x7f2dfc06e000 msgr2=0x7f2dfc06e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:13.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 -- 172.21.15.69:0/1183259193 shutdown_connections 2024-08-02T17:59:13.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 -- 172.21.15.69:0/1183259193 wait complete. 2024-08-02T17:59:13.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 Processor -- start 2024-08-02T17:59:13.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.482+0000 7f2e03251640 1 -- start start 2024-08-02T17:59:13.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e03251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dfc10a290 0x7f2dfc1a6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e03251640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc10abb0 0x7f2dfc1a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e03251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2dfc113710 0x7f2dfc1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e03251640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc1193e0 con 0x7f2dfc113710 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e03251640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc119260 con 0x7f2dfc10a290 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e03251640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2dfc119560 con 0x7f2dfc10abb0 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e00fed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dfc10a290 0x7f2dfc1a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e00fed640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dfc10a290 0x7f2dfc1a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55590/0 (socket says 172.21.15.69:55590) 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e00fed640 1 -- 172.21.15.69:0/404855425 learned_addr learned my addr 172.21.15.69:0/404855425 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e017ee640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2dfc113710 0x7f2dfc1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2dfbfff640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc10abb0 0x7f2dfc1a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e00fed640 1 -- 172.21.15.69:0/404855425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc10abb0 msgr2=0x7f2dfc1a6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e00fed640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc10abb0 0x7f2dfc1a6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e00fed640 1 -- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2dfc113710 msgr2=0x7f2dfc1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e00fed640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2dfc113710 0x7f2dfc1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.483+0000 7f2e00fed640 1 -- 172.21.15.69:0/404855425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dfc1a0f80 con 0x7f2dfc10a290 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.484+0000 7f2e00fed640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dfc10a290 0x7f2dfc1a6660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9f0 tx=0x7f2de800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.484+0000 7f2df9ffb640 1 -- 172.21.15.69:0/404855425 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de800cd80 con 0x7f2dfc10a290 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.484+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dfc1a1270 con 0x7f2dfc10a290 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.484+0000 7f2df9ffb640 1 -- 172.21.15.69:0/404855425 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2de8010910 con 0x7f2dfc10a290 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.484+0000 7f2df9ffb640 1 -- 172.21.15.69:0/404855425 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8018990 con 0x7f2dfc10a290 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.484+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dfc1ad4c0 con 0x7f2dfc10a290 2024-08-02T17:59:13.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.485+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dfc073830 con 0x7f2dfc10a290 2024-08-02T17:59:13.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.487+0000 7f2df9ffb640 1 -- 172.21.15.69:0/404855425 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2de8010ac0 con 0x7f2dfc10a290 2024-08-02T17:59:13.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.487+0000 7f2df9ffb640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2dec0a2d70 0x7f2dec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:13.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.488+0000 7f2df9ffb640 1 -- 172.21.15.69:0/404855425 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7f2de80cfb30 con 0x7f2dfc10a290 2024-08-02T17:59:13.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.488+0000 7f2dfbfff640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2dec0a2d70 0x7f2dec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:13.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.488+0000 7f2dfbfff640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2dec0a2d70 0x7f2dec0a5230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0730e0 tx=0x7f2df0005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:13.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.492+0000 7f2df9ffb640 1 -- 172.21.15.69:0/404855425 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2de809bc90 con 0x7f2dfc10a290 2024-08-02T17:59:13.584 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:13.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.821+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2dfc10f7a0 con 0x7f2dfc10a290 2024-08-02T17:59:13.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.823+0000 7f2df9ffb640 1 -- 172.21.15.69:0/404855425 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f2de809ff90 con 0x7f2dfc10a290 2024-08-02T17:59:13.825 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:59:13.825 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:59:13.826 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:59:13.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.830+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2dec0a2d70 msgr2=0x7f2dec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.830+0000 7f2e03251640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2dec0a2d70 0x7f2dec0a5230 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0730e0 tx=0x7f2df0005f90 comp rx=0 tx=0).stop 2024-08-02T17:59:13.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.830+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dfc10a290 msgr2=0x7f2dfc1a6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:13.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.830+0000 7f2e03251640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dfc10a290 0x7f2dfc1a6660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2de800e9f0 tx=0x7f2de800eeb0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.831+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 shutdown_connections 2024-08-02T17:59:13.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.831+0000 7f2e03251640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2dfc113710 0x7f2dfc1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.831+0000 7f2e03251640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2dec0a2d70 0x7f2dec0a5230 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.831+0000 7f2e03251640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dfc10abb0 0x7f2dfc1a6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.831+0000 7f2e03251640 1 --2- 172.21.15.69:0/404855425 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2dfc10a290 0x7f2dfc1a6660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:13.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.831+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 >> 172.21.15.69:0/404855425 conn(0x7f2dfc06e000 msgr2=0x7f2dfc1091a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:13.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.836+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 shutdown_connections 2024-08-02T17:59:13.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:13.837+0000 7f2e03251640 1 -- 172.21.15.69:0/404855425 wait complete. 2024-08-02T17:59:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.423+0000 7fd6fe13f640 1 -- 172.21.15.69:0/4227968730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f811ae70 msgr2=0x7fd6f811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:14.425 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.423+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/4227968730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f811ae70 0x7fd6f811d260 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd6f001acb0 tx=0x7fd6f003b8e0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.429+0000 7fd6fe13f640 1 -- 172.21.15.69:0/4227968730 shutdown_connections 2024-08-02T17:59:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.429+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/4227968730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f811ae70 0x7fd6f811d260 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.429+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/4227968730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6f8109560 0x7fd6f81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.429+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/4227968730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6f8109180 0x7fd6f810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.429+0000 7fd6fe13f640 1 -- 172.21.15.69:0/4227968730 >> 172.21.15.69:0/4227968730 conn(0x7fd6f806de70 msgr2=0x7fd6f806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:14.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.432+0000 7fd6fe13f640 1 -- 172.21.15.69:0/4227968730 shutdown_connections 2024-08-02T17:59:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.432+0000 7fd6fe13f640 1 -- 172.21.15.69:0/4227968730 wait complete. 2024-08-02T17:59:14.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.432+0000 7fd6fe13f640 1 Processor -- start 2024-08-02T17:59:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.434+0000 7fd6fe13f640 1 -- start start 2024-08-02T17:59:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.434+0000 7fd6fe13f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f8109180 0x7fd6f811a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.434+0000 7fd6fe13f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6f8109560 0x7fd6f8116520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.434+0000 7fd6fe13f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6f811ae70 0x7fd6f8116a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.434+0000 7fd6fe13f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8113050 con 0x7fd6f8109180 2024-08-02T17:59:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.434+0000 7fd6fe13f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f8112ed0 con 0x7fd6f811ae70 2024-08-02T17:59:14.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.434+0000 7fd6fe13f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81131d0 con 0x7fd6f8109560 2024-08-02T17:59:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.436+0000 7fd6f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f8109180 0x7fd6f811a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.436+0000 7fd6f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f8109180 0x7fd6f811a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52658/0 (socket says 172.21.15.69:52658) 2024-08-02T17:59:14.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.436+0000 7fd6f77fe640 1 -- 172.21.15.69:0/2422566893 learned_addr learned my addr 172.21.15.69:0/2422566893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:14.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.437+0000 7fd6f7fff640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6f811ae70 0x7fd6f8116a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:14.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.438+0000 7fd6f6ffd640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6f8109560 0x7fd6f8116520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:14.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.439+0000 7fd6f6ffd640 1 -- 172.21.15.69:0/2422566893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6f811ae70 msgr2=0x7fd6f8116a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:14.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.439+0000 7fd6f6ffd640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6f811ae70 0x7fd6f8116a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.439+0000 7fd6f6ffd640 1 -- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f8109180 msgr2=0x7fd6f811a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:14.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.439+0000 7fd6f6ffd640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f8109180 0x7fd6f811a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.439+0000 7fd6f6ffd640 1 -- 172.21.15.69:0/2422566893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f8117020 con 0x7fd6f8109560 2024-08-02T17:59:14.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.439+0000 7fd6f6ffd640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6f8109560 0x7fd6f8116520 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00c970 tx=0x7fd6ec00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:14.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.441+0000 7fd6f4ff9640 1 -- 172.21.15.69:0/2422566893 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec007bb0 con 0x7fd6f8109560 2024-08-02T17:59:14.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.441+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f8117310 con 0x7fd6f8109560 2024-08-02T17:59:14.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.441+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f81ab190 con 0x7fd6f8109560 2024-08-02T17:59:14.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.441+0000 7fd6f4ff9640 1 -- 172.21.15.69:0/2422566893 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6ec007d50 con 0x7fd6f8109560 2024-08-02T17:59:14.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.441+0000 7fd6f4ff9640 1 -- 172.21.15.69:0/2422566893 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec00f450 con 0x7fd6f8109560 2024-08-02T17:59:14.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.442+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f8073830 con 0x7fd6f8109560 2024-08-02T17:59:14.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.467+0000 7fd6f4ff9640 1 -- 172.21.15.69:0/2422566893 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd6ec004ed0 con 0x7fd6f8109560 2024-08-02T17:59:14.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.468+0000 7fd6f4ff9640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd6e00a2f00 0x7fd6e00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:14.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.468+0000 7fd6f4ff9640 1 -- 172.21.15.69:0/2422566893 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9656+0+0 (secure 0 0 0) 0x7fd6ec0cf880 con 0x7fd6f8109560 2024-08-02T17:59:14.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.468+0000 7fd6f4ff9640 1 -- 172.21.15.69:0/2422566893 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd6ec014030 con 0x7fd6f8109560 2024-08-02T17:59:14.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.473+0000 7fd6f77fe640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd6e00a2f00 0x7fd6e00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:14.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.489+0000 7fd6f77fe640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd6e00a2f00 0x7fd6e00a53c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009950 tx=0x7fd6e8008040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:14.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:14 smithi077 ceph-mon[37108]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-08-02T17:59:14.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:14 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/404855425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:14.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:14 smithi069 ceph-mon[32746]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-08-02T17:59:14.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:14 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/404855425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:14.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:14 smithi083 ceph-mon[37031]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-08-02T17:59:14.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:14 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/404855425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:14.890 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T17:59:14.898 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.897+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6f8117a50 con 0x7fd6f8109560 2024-08-02T17:59:14.901 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.900+0000 7fd6f4ff9640 1 -- 172.21.15.69:0/2422566893 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fd6ec09c050 con 0x7fd6f8109560 2024-08-02T17:59:14.902 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:59:14.902 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:59:14.903 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:59:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.905+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd6e00a2f00 msgr2=0x7fd6e00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.905+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd6e00a2f00 0x7fd6e00a53c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8009950 tx=0x7fd6e8008040 comp rx=0 tx=0).stop 2024-08-02T17:59:14.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.905+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6f8109560 msgr2=0x7fd6f8116520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:14.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.905+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6f8109560 0x7fd6f8116520 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00c970 tx=0x7fd6ec00ce30 comp rx=0 tx=0).stop 2024-08-02T17:59:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.906+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 shutdown_connections 2024-08-02T17:59:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.906+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd6e00a2f00 0x7fd6e00a53c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.906+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd6f811ae70 0x7fd6f8116a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.906+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd6f8109560 0x7fd6f8116520 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.906+0000 7fd6fe13f640 1 --2- 172.21.15.69:0/2422566893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd6f8109180 0x7fd6f811a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:14.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.906+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 >> 172.21.15.69:0/2422566893 conn(0x7fd6f806de70 msgr2=0x7fd6f8070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:14.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.909+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 shutdown_connections 2024-08-02T17:59:14.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:14.914+0000 7fd6fe13f640 1 -- 172.21.15.69:0/2422566893 wait complete. 2024-08-02T17:59:14.952 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 725.0 s, 624.2 tps, lat 164.437 ms stddev 214.531 2024-08-02T17:59:15.765 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-02T17:59:15.765 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-02T17:59:15.890 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:15 smithi069 ceph-mon[32746]: osdmap e109: 12 total, 12 up, 12 in 2024-08-02T17:59:15.891 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:15 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2422566893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:15.990 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:16.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:15 smithi083 ceph-mon[37031]: osdmap e109: 12 total, 12 up, 12 in 2024-08-02T17:59:16.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:15 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2422566893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:16.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:15 smithi077 ceph-mon[37108]: osdmap e109: 12 total, 12 up, 12 in 2024-08-02T17:59:16.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:15 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2422566893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:17.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:16 smithi083 ceph-mon[37031]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-08-02T17:59:17.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:16 smithi077 ceph-mon[37108]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-08-02T17:59:17.107 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:16 smithi069 ceph-mon[32746]: pgmap v402: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 5.0 MiB/s wr, 585 op/s 2024-08-02T17:59:17.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.588+0000 7fe085553640 1 -- 172.21.15.69:0/3898008614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08011ae70 msgr2=0x7fe08011d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.588+0000 7fe085553640 1 --2- 172.21.15.69:0/3898008614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08011ae70 0x7fe08011d260 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe070009900 tx=0x7fe07002a420 comp rx=0 tx=0).stop 2024-08-02T17:59:17.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.590+0000 7fe085553640 1 -- 172.21.15.69:0/3898008614 shutdown_connections 2024-08-02T17:59:17.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.590+0000 7fe085553640 1 --2- 172.21.15.69:0/3898008614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe08006e760 0x7fe08006eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.590+0000 7fe085553640 1 --2- 172.21.15.69:0/3898008614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08011ae70 0x7fe08011d260 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.590+0000 7fe085553640 1 --2- 172.21.15.69:0/3898008614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08010fec0 0x7fe08006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.590+0000 7fe085553640 1 -- 172.21.15.69:0/3898008614 >> 172.21.15.69:0/3898008614 conn(0x7fe080072eb0 msgr2=0x7fe0800732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:17.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.595+0000 7fe085553640 1 -- 172.21.15.69:0/3898008614 shutdown_connections 2024-08-02T17:59:17.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.596+0000 7fe085553640 1 -- 172.21.15.69:0/3898008614 wait complete. 2024-08-02T17:59:17.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.596+0000 7fe085553640 1 Processor -- start 2024-08-02T17:59:17.599 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.598+0000 7fe085553640 1 -- start start 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.598+0000 7fe085553640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe08006e760 0x7fe08011a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.598+0000 7fe085553640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08010fec0 0x7fe080114760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.598+0000 7fe085553640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08011ae70 0x7fe080114ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.598+0000 7fe085553640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe0801110a0 con 0x7fe08006e760 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.598+0000 7fe085553640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe080110f20 con 0x7fe08011ae70 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.598+0000 7fe085553640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe080111220 con 0x7fe08010fec0 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08010fec0 0x7fe080114760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08010fec0 0x7fe080114760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59308/0 (socket says 172.21.15.69:59308) 2024-08-02T17:59:17.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 -- 172.21.15.69:0/3030631258 learned_addr learned my addr 172.21.15.69:0/3030631258 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 -- 172.21.15.69:0/3030631258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08011ae70 msgr2=0x7fe080114ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08011ae70 0x7fe080114ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 -- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe08006e760 msgr2=0x7fe08011a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe08006e760 0x7fe08011a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 -- 172.21.15.69:0/3030631258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe080115540 con 0x7fe08010fec0 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.599+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08010fec0 0x7fe080114760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe07400b4a0 tx=0x7fe07400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.603+0000 7fe07d7fa640 1 -- 172.21.15.69:0/3030631258 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074014070 con 0x7fe08010fec0 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.603+0000 7fe085553640 1 -- 172.21.15.69:0/3030631258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0801bbc60 con 0x7fe08010fec0 2024-08-02T17:59:17.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.603+0000 7fe085553640 1 -- 172.21.15.69:0/3030631258 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0801bc1a0 con 0x7fe08010fec0 2024-08-02T17:59:17.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.606+0000 7fe088bb3740 1 Processor -- start 2024-08-02T17:59:17.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.608+0000 7fe07d7fa640 1 -- 172.21.15.69:0/3030631258 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe074010af0 con 0x7fe08010fec0 2024-08-02T17:59:17.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.608+0000 7fe07d7fa640 1 -- 172.21.15.69:0/3030631258 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074017c50 con 0x7fe08010fec0 2024-08-02T17:59:17.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.612+0000 7fe088bb3740 1 -- start start 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc05ac0 0x56034dc04290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc047d0 0x56034dc04c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc0f130 0x56034dc11520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe088bb3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56034db69a40 con 0x56034dc05ac0 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe088bb3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56034db019d0 con 0x56034dc047d0 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe088bb3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56034db65a00 con 0x56034dc0f130 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe07d7fa640 1 -- 172.21.15.69:0/3030631258 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe074017070 con 0x7fe08010fec0 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe07f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc0f130 0x56034dc11520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe07f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc0f130 0x56034dc11520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59322/0 (socket says 172.21.15.69:59322) 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.614+0000 7fe07f7fe640 1 -- 172.21.15.69:0/288498756 learned_addr learned my addr 172.21.15.69:0/288498756 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.615+0000 7fe07f7fe640 1 -- 172.21.15.69:0/288498756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc047d0 msgr2=0x56034dc04c10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.615+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/288498756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc047d0 0x56034dc04c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.615+0000 7fe07f7fe640 1 -- 172.21.15.69:0/288498756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc05ac0 msgr2=0x56034dc04290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T17:59:17.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.615+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/288498756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc05ac0 0x56034dc04290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.615+0000 7fe07f7fe640 1 -- 172.21.15.69:0/288498756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56034db08650 con 0x56034dc0f130 2024-08-02T17:59:17.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.615+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/288498756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc0f130 0x56034dc11520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0740cce30 tx=0x7fe0740d3f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565d892ed43e13c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:17.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.615+0000 7fe085553640 1 -- 172.21.15.69:0/288498756 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0740e3070 con 0x56034dc0f130 2024-08-02T17:59:17.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.616+0000 7fe085553640 1 -- 172.21.15.69:0/288498756 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0740d8b30 con 0x56034dc0f130 2024-08-02T17:59:17.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.616+0000 7fe085553640 1 -- 172.21.15.69:0/288498756 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0740dd640 con 0x56034dc0f130 2024-08-02T17:59:17.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.617+0000 7fe088bb3740 1 -- 172.21.15.69:0/288498756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc0f130 msgr2=0x56034dc11520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.617+0000 7fe088bb3740 1 --2- 172.21.15.69:0/288498756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc0f130 0x56034dc11520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe0740cce30 tx=0x7fe0740d3f40 comp rx=0 tx=0).stop 2024-08-02T17:59:17.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.622+0000 7fe07d7fa640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe0680a2f00 0x7fe0680a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.624 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.622+0000 7fe07d7fa640 1 -- 172.21.15.69:0/3030631258 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fe0740cfae0 con 0x7fe08010fec0 2024-08-02T17:59:17.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.624+0000 7fe088bb3740 1 -- 172.21.15.69:0/288498756 shutdown_connections 2024-08-02T17:59:17.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.624+0000 7fe088bb3740 1 --2- 172.21.15.69:0/288498756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc0f130 0x56034dc11520 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.624+0000 7fe088bb3740 1 --2- 172.21.15.69:0/288498756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc047d0 0x56034dc04c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.624+0000 7fe088bb3740 1 --2- 172.21.15.69:0/288498756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc05ac0 0x56034dc04290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.626 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.624+0000 7fe088bb3740 1 -- 172.21.15.69:0/288498756 >> 172.21.15.69:0/288498756 conn(0x56034dc00a20 msgr2=0x56034dc00e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:17.626 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:17.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.625+0000 7fe084d52640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe0680a2f00 0x7fe0680a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.628+0000 7fe088bb3740 1 -- 172.21.15.69:0/288498756 shutdown_connections 2024-08-02T17:59:17.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.628+0000 7fe084d52640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe0680a2f00 0x7fe0680a53c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe070033040 tx=0x7fe0700095f0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:17.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.628+0000 7fe088bb3740 1 -- 172.21.15.69:0/288498756 wait complete. 2024-08-02T17:59:17.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.628+0000 7fe088bb3740 1 -- start start 2024-08-02T17:59:17.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.629+0000 7fe088bb3740 1 Processor -- start 2024-08-02T17:59:17.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.629+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc047d0 0x56034dc0c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.629+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 0x56034dc0c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.629+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 0x56034dc0cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.629+0000 7fe088bb3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56034dc11fa0 con 0x56034dc0f130 2024-08-02T17:59:17.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.629+0000 7fe088bb3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56034db24b20 con 0x56034dc05ac0 2024-08-02T17:59:17.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.629+0000 7fe088bb3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56034dc12120 con 0x56034dc047d0 2024-08-02T17:59:17.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.629+0000 7fe088bb3740 10 client.0 _ll_register_callbacks cb 0x56034db22bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:17.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.631+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc047d0 0x56034dc0c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.631+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc047d0 0x56034dc0c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59336/0 (socket says 172.21.15.69:59336) 2024-08-02T17:59:17.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.631+0000 7fe07ffff640 1 -- 172.21.15.69:0/73796259 learned_addr learned my addr 172.21.15.69:0/73796259 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:17.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.631+0000 7fe084d52640 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 0x56034dc0cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.632+0000 7fe07ffff640 1 -- 172.21.15.69:0/73796259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 msgr2=0x56034dc0c5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:59:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.632+0000 7fe07ffff640 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 0x56034dc0c5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.632+0000 7fe07ffff640 1 -- 172.21.15.69:0/73796259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 msgr2=0x56034dc0cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.632+0000 7fe07ffff640 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 0x56034dc0cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.632+0000 7fe07ffff640 1 -- 172.21.15.69:0/73796259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56034d8db3c0 con 0x56034dc047d0 2024-08-02T17:59:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.632+0000 7fe07ffff640 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc047d0 0x56034dc0c0b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe07800e770 tx=0x7fe07800ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2125dd87754ec352 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.632+0000 7fe05f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:59:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.633+0000 7fe05f7fe640 1 -- 172.21.15.69:0/73796259 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078004d20 con 0x56034dc047d0 2024-08-02T17:59:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.633+0000 7fe088bb3740 1 -- 172.21.15.69:0/73796259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0740d8b30 con 0x56034dc047d0 2024-08-02T17:59:17.636 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.633+0000 7fe05f7fe640 1 -- 172.21.15.69:0/73796259 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fe07800ca40 con 0x56034dc047d0 2024-08-02T17:59:17.637 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.633+0000 7fe05f7fe640 1 -- 172.21.15.69:0/73796259 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe078021050 con 0x56034dc047d0 2024-08-02T17:59:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe088bb3740 10 client.27784 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe088bb3740 1 -- 172.21.15.69:0/73796259 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56034dc1ddb0 con 0x56034dc047d0 2024-08-02T17:59:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe05f7fe640 1 -- 172.21.15.69:0/73796259 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0740d8b30 con 0x56034dc047d0 2024-08-02T17:59:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe05f7fe640 1 -- 172.21.15.69:0/73796259 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe078018470 con 0x56034dc047d0 2024-08-02T17:59:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe088bb3740 10 client.27784 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:17.638 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe088bb3740 10 client.27784 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:59:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe088bb3740 20 client.27784 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe088bb3740 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56034dc1f6c0 0x56034dc21b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe088bb3740 4 client.27784 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:17.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.634+0000 7fe088bb3740 1 -- 172.21.15.69:0/73796259 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56034db76fb0 con 0x56034dc1f6c0 2024-08-02T17:59:17.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.637+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56034dc1f6c0 0x56034dc21b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.638+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56034dc1f6c0 0x56034dc21b60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf4bc81b13823b server_cookie=2369cd4e60a4ea2b in_seq=0 out_seq=0 2024-08-02T17:59:17.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.638+0000 7fe05f7fe640 10 client.27784 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:59:17.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.639+0000 7fe05f7fe640 1 -- 172.21.15.69:0/73796259 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56034db76fb0 con 0x56034dc1f6c0 2024-08-02T17:59:17.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.639+0000 7fe05f7fe640 10 client.27784 handle_command_reply: tid=0 2024-08-02T17:59:17.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 client.27784 shutdown 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 -- 172.21.15.69:0/73796259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc047d0 msgr2=0x56034dc0c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc047d0 0x56034dc0c0b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe07800e770 tx=0x7fe07800ec30 comp rx=0 tx=0).stop 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 -- 172.21.15.69:0/73796259 shutdown_connections 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56034dc1f6c0 0x56034dc21b60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 0x56034dc0cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 0x56034dc0c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 --2- 172.21.15.69:0/73796259 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc047d0 0x56034dc0c0b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe088bb3740 1 -- 172.21.15.69:0/73796259 >> 172.21.15.69:0/73796259 conn(0x56034dc00a20 msgr2=0x56034dc06750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.640+0000 7fe05f7fe640 0 client.27784 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:59:17.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.641+0000 7fe088bb3740 1 -- 172.21.15.69:0/73796259 shutdown_connections 2024-08-02T17:59:17.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.641+0000 7fe088bb3740 1 -- 172.21.15.69:0/73796259 wait complete. 2024-08-02T17:59:17.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.645+0000 7fe088bb3740 20 client.27784 trim_cache size 0 max 16384 2024-08-02T17:59:17.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.684+0000 7fe088bb3740 1 Processor -- start 2024-08-02T17:59:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.688+0000 7fe088bb3740 1 -- start start 2024-08-02T17:59:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.688+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc047d0 0x56034dc0b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.689+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc05ac0 0x56034dc27470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.689+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc0f130 0x56034dc279b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.689+0000 7fe088bb3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c012a40 con 0x56034dc047d0 2024-08-02T17:59:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.689+0000 7fe088bb3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c010310 con 0x56034dc0f130 2024-08-02T17:59:17.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.689+0000 7fe088bb3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c00e160 con 0x56034dc05ac0 2024-08-02T17:59:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.693+0000 7fe07f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc05ac0 0x56034dc27470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.693+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc047d0 0x56034dc0b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.693+0000 7fe07f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc05ac0 0x56034dc27470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59344/0 (socket says 172.21.15.69:59344) 2024-08-02T17:59:17.695 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.693+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc047d0 0x56034dc0b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44488/0 (socket says 172.21.15.69:44488) 2024-08-02T17:59:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.693+0000 7fe07f7fe640 1 -- 172.21.15.69:0/2653460727 learned_addr learned my addr 172.21.15.69:0/2653460727 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.694+0000 7fe084d52640 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc0f130 0x56034dc279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.694+0000 7fe07ffff640 1 -- 172.21.15.69:0/2653460727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc05ac0 msgr2=0x56034dc27470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.694+0000 7fe07ffff640 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc05ac0 0x56034dc27470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.696 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.694+0000 7fe07ffff640 1 -- 172.21.15.69:0/2653460727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc0f130 msgr2=0x56034dc279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.694+0000 7fe07ffff640 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc0f130 0x56034dc279b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.694+0000 7fe07ffff640 1 -- 172.21.15.69:0/2653460727 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56034dc05540 con 0x56034dc047d0 2024-08-02T17:59:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.694+0000 7fe07f7fe640 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc05ac0 0x56034dc27470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:17.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.695+0000 7fe07ffff640 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc047d0 0x56034dc0b970 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fe078018870 tx=0x7fe078019f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109620b01e1fd58a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.698+0000 7fe065ffb640 1 -- 172.21.15.69:0/2653460727 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078029070 con 0x56034dc047d0 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.698+0000 7fe065ffb640 1 -- 172.21.15.69:0/2653460727 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe078018470 con 0x56034dc047d0 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.699+0000 7fe088bb3740 1 -- 172.21.15.69:0/2653460727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc047d0 msgr2=0x56034dc0b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.699+0000 7fe088bb3740 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc047d0 0x56034dc0b970 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fe078018870 tx=0x7fe078019f10 comp rx=0 tx=0).stop 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.699+0000 7fe088bb3740 1 -- 172.21.15.69:0/2653460727 shutdown_connections 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.699+0000 7fe088bb3740 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc0f130 0x56034dc279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.699+0000 7fe088bb3740 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc05ac0 0x56034dc27470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.699+0000 7fe088bb3740 1 --2- 172.21.15.69:0/2653460727 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc047d0 0x56034dc0b970 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.701 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.699+0000 7fe088bb3740 1 -- 172.21.15.69:0/2653460727 >> 172.21.15.69:0/2653460727 conn(0x56034dc00a20 msgr2=0x56034dc07410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.700+0000 7fe088bb3740 1 -- 172.21.15.69:0/2653460727 shutdown_connections 2024-08-02T17:59:17.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.700+0000 7fe088bb3740 1 -- 172.21.15.69:0/2653460727 wait complete. 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.700+0000 7fe088bb3740 1 -- start start 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.701+0000 7fe088bb3740 1 Processor -- start 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.701+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 0x56034dc17cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe088bb3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 0x56034dc12400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 0x56034dc17cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 0x56034dc17cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59716/0 (socket says 172.21.15.69:59716) 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe07ffff640 1 -- 172.21.15.69:0/4248211416 learned_addr learned my addr 172.21.15.69:0/4248211416 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe088bb3740 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc12990 0x56034dc12dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c00e7d0 con 0x56034dc0f130 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe084d52640 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc12990 0x56034dc12dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c00fc40 con 0x56034dc05ac0 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.702+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe06c00eea0 con 0x56034dc12990 2024-08-02T17:59:17.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.703+0000 7fe088bb3740 10 client.0 _ll_register_callbacks cb 0x7fe0780098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.703+0000 7fe084d52640 1 -- 172.21.15.69:0/4248211416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 msgr2=0x56034dc17cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.703+0000 7fe084d52640 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 0x56034dc17cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.703+0000 7fe084d52640 1 -- 172.21.15.69:0/4248211416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 msgr2=0x56034dc12400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.703+0000 7fe084d52640 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 0x56034dc12400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.703+0000 7fe084d52640 1 -- 172.21.15.69:0/4248211416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56034dc88600 con 0x56034dc12990 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.703+0000 7fe084d52640 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc12990 0x56034dc12dd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe07002aa10 tx=0x7fe070009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e0750d9cc9494 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.703+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56034db714c0 con 0x56034dc12990 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.704+0000 7fe05f7fe640 10 client.27790 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:59:17.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.704+0000 7fe05f7fe640 1 -- 172.21.15.69:0/4248211416 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07004f070 con 0x56034dc12990 2024-08-02T17:59:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.704+0000 7fe05f7fe640 1 -- 172.21.15.69:0/4248211416 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fe07003c440 con 0x56034dc12990 2024-08-02T17:59:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.705+0000 7fe05f7fe640 1 -- 172.21.15.69:0/4248211416 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe07005b050 con 0x56034dc12990 2024-08-02T17:59:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.706+0000 7fe088bb3740 10 client.27790 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:17.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.706+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x56034dc89870 con 0x56034dc12990 2024-08-02T17:59:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.710+0000 7fe05f7fe640 1 -- 172.21.15.69:0/4248211416 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe07003c640 con 0x56034dc12990 2024-08-02T17:59:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.711+0000 7fe088bb3740 10 client.27790 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.711+0000 7fe088bb3740 10 client.27790 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T17:59:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.711+0000 7fe088bb3740 20 client.27790 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.711+0000 7fe088bb3740 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56034dc89e10 0x56034dc8c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.711+0000 7fe088bb3740 4 client.27790 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:59:17.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.711+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56034db714c0 con 0x56034dc89e10 2024-08-02T17:59:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.712+0000 7fe07ffff640 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56034dc89e10 0x56034dc8c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:17.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.712+0000 7fe07ffff640 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56034dc89e10 0x56034dc8c2b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=103faf76c932efb8 server_cookie=98db470364bbb9d9 in_seq=0 out_seq=0 2024-08-02T17:59:17.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.712+0000 7fe05f7fe640 10 client.27790 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T17:59:17.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.712+0000 7fe05f7fe640 1 -- 172.21.15.69:0/4248211416 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56034db714c0 con 0x56034dc89e10 2024-08-02T17:59:17.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.712+0000 7fe05f7fe640 10 client.27790 handle_command_reply: tid=0 2024-08-02T17:59:17.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.712+0000 7fe088bb3740 1 client.27790 shutdown 2024-08-02T17:59:17.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc12990 msgr2=0x56034dc12dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe088bb3740 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc12990 0x56034dc12dd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe07002aa10 tx=0x7fe070009260 comp rx=0 tx=0).stop 2024-08-02T17:59:17.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 shutdown_connections 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe088bb3740 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x56034dc89e10 0x56034dc8c2b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe088bb3740 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56034dc0f130 0x56034dc12400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe088bb3740 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56034dc12990 0x56034dc12dd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe05f7fe640 0 client.27790 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe088bb3740 1 --2- 172.21.15.69:0/4248211416 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56034dc05ac0 0x56034dc17cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.714+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 >> 172.21.15.69:0/4248211416 conn(0x56034dc00a20 msgr2=0x56034dc05f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.715+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 shutdown_connections 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.715+0000 7fe088bb3740 1 -- 172.21.15.69:0/4248211416 wait complete. 2024-08-02T17:59:17.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.715+0000 7fe088bb3740 20 client.27790 trim_cache size 0 max 16384 2024-08-02T17:59:17.718 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:59:17.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 -- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe0680a2f00 msgr2=0x7fe0680a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe0680a2f00 0x7fe0680a53c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe070033040 tx=0x7fe0700095f0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 -- 172.21.15.69:0/3030631258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08010fec0 msgr2=0x7fe080114760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:17.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08010fec0 0x7fe080114760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe07400b4a0 tx=0x7fe07400b960 comp rx=0 tx=0).stop 2024-08-02T17:59:17.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 -- 172.21.15.69:0/3030631258 shutdown_connections 2024-08-02T17:59:17.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe0680a2f00 0x7fe0680a53c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe08011ae70 0x7fe080114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe08010fec0 0x7fe080114760 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 --2- 172.21.15.69:0/3030631258 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe08006e760 0x7fe08011a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:17.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 -- 172.21.15.69:0/3030631258 >> 172.21.15.69:0/3030631258 conn(0x7fe080072eb0 msgr2=0x7fe08011b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:17.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 -- 172.21.15.69:0/3030631258 shutdown_connections 2024-08-02T17:59:17.721 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:17.716+0000 7fe066ffd640 1 -- 172.21.15.69:0/3030631258 wait complete. 2024-08-02T17:59:18.812 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:18 smithi069 ceph-mon[32746]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 5.2 MiB/s wr, 592 op/s 2024-08-02T17:59:18.864 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T17:59:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.435+0000 7faf5d3b0640 1 -- 172.21.15.69:0/3619240432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf58105c80 msgr2=0x7faf58106110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.435+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/3619240432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf58105c80 0x7faf58106110 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7faf50009be0 tx=0x7faf5002a790 comp rx=0 tx=0).stop 2024-08-02T17:59:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.436+0000 7faf5d3b0640 1 -- 172.21.15.69:0/3619240432 shutdown_connections 2024-08-02T17:59:18.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.436+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/3619240432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf58101190 0x7faf58105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.436+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/3619240432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf58105c80 0x7faf58106110 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.436+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/3619240432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf58100870 0x7faf58100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.436+0000 7faf5d3b0640 1 -- 172.21.15.69:0/3619240432 >> 172.21.15.69:0/3619240432 conn(0x7faf580fc500 msgr2=0x7faf580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:18.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.437+0000 7faf5d3b0640 1 -- 172.21.15.69:0/3619240432 shutdown_connections 2024-08-02T17:59:18.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.438+0000 7faf5d3b0640 1 -- 172.21.15.69:0/3619240432 wait complete. 2024-08-02T17:59:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.439+0000 7faf5d3b0640 1 Processor -- start 2024-08-02T17:59:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.440+0000 7faf5d3b0640 1 -- start start 2024-08-02T17:59:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.440+0000 7faf5d3b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf58100870 0x7faf58073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.440+0000 7faf5d3b0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf58101190 0x7faf5806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.440+0000 7faf5d3b0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf58105c80 0x7faf5806efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.440+0000 7faf5d3b0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faf5806c390 con 0x7faf58101190 2024-08-02T17:59:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.440+0000 7faf5d3b0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf5806c210 con 0x7faf58105c80 2024-08-02T17:59:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.440+0000 7faf5d3b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf5806c510 con 0x7faf58100870 2024-08-02T17:59:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.444+0000 7faf567fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf58101190 0x7faf5806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.444+0000 7faf567fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf58101190 0x7faf5806eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44502/0 (socket says 172.21.15.69:44502) 2024-08-02T17:59:18.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.444+0000 7faf567fc640 1 -- 172.21.15.69:0/1407433689 learned_addr learned my addr 172.21.15.69:0/1407433689 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:18.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.444+0000 7faf56ffd640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf58100870 0x7faf58073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.444+0000 7faf577fe640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf58105c80 0x7faf5806efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.909 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.444+0000 7faf56ffd640 1 -- 172.21.15.69:0/1407433689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf58105c80 msgr2=0x7faf5806efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.444+0000 7faf56ffd640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf58105c80 0x7faf5806efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.445+0000 7faf56ffd640 1 -- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf58101190 msgr2=0x7faf5806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.445+0000 7faf56ffd640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf58101190 0x7faf5806eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.445+0000 7faf56ffd640 1 -- 172.21.15.69:0/1407433689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5806f870 con 0x7faf58100870 2024-08-02T17:59:18.910 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.445+0000 7faf56ffd640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf58100870 0x7faf58073a10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faf4800e790 tx=0x7faf4800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.446+0000 7faf3ffff640 1 -- 172.21.15.69:0/1407433689 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4800eea0 con 0x7faf58100870 2024-08-02T17:59:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.446+0000 7faf3ffff640 1 -- 172.21.15.69:0/1407433689 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf4800ccc0 con 0x7faf58100870 2024-08-02T17:59:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.446+0000 7faf5d3b0640 1 -- 172.21.15.69:0/1407433689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5810dcd0 con 0x7faf58100870 2024-08-02T17:59:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.446+0000 7faf5d3b0640 1 -- 172.21.15.69:0/1407433689 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf5810e210 con 0x7faf58100870 2024-08-02T17:59:18.911 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.446+0000 7faf3ffff640 1 -- 172.21.15.69:0/1407433689 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48018780 con 0x7faf58100870 2024-08-02T17:59:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.451+0000 7faf3ffff640 1 -- 172.21.15.69:0/1407433689 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faf48018920 con 0x7faf58100870 2024-08-02T17:59:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.455+0000 7faf5f7ae740 1 Processor -- start 2024-08-02T17:59:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.455+0000 7faf5f7ae740 1 -- start start 2024-08-02T17:59:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.455+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3130 0x55759b9c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.455+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3a50 0x55759b9cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.912 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9ce310 0x55759b9d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55759b928a70 con 0x55759b9ce310 2024-08-02T17:59:18.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55759b8c0780 con 0x55759b9c3a50 2024-08-02T17:59:18.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55759b924a30 con 0x55759b9c3130 2024-08-02T17:59:18.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3130 0x55759b9c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.913 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf577fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3130 0x55759b9c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59364/0 (socket says 172.21.15.69:59364) 2024-08-02T17:59:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf577fe640 1 -- 172.21.15.69:0/3307894185 learned_addr learned my addr 172.21.15.69:0/3307894185 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf56ffd640 1 --2- 172.21.15.69:0/3307894185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3a50 0x55759b9cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf577fe640 1 -- 172.21.15.69:0/3307894185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3a50 msgr2=0x55759b9cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf577fe640 1 --2- 172.21.15.69:0/3307894185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3a50 0x55759b9cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.914 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf577fe640 1 -- 172.21.15.69:0/3307894185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9ce310 msgr2=0x55759b9d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf577fe640 1 --2- 172.21.15.69:0/3307894185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9ce310 0x55759b9d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.456+0000 7faf577fe640 1 -- 172.21.15.69:0/3307894185 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55759b69ab90 con 0x55759b9c3130 2024-08-02T17:59:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.458+0000 7faf3ffff640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7faf400a2e80 0x7faf400a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.458+0000 7faf3ffff640 1 -- 172.21.15.69:0/1407433689 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7faf480cfa10 con 0x7faf58100870 2024-08-02T17:59:18.915 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.458+0000 7faf577fe640 1 --2- 172.21.15.69:0/3307894185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3130 0x55759b9c3510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf50033000 tx=0x7faf50030b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d51a12ce712c8645 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:18.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.459+0000 7faf3d7fa640 1 -- 172.21.15.69:0/3307894185 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50040030 con 0x55759b9c3130 2024-08-02T17:59:18.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.459+0000 7faf56ffd640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7faf400a2e80 0x7faf400a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.459+0000 7faf3d7fa640 1 -- 172.21.15.69:0/3307894185 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf50003e50 con 0x55759b9c3130 2024-08-02T17:59:18.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.460+0000 7faf5f7ae740 1 -- 172.21.15.69:0/3307894185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3130 msgr2=0x55759b9c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.460+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/3307894185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3130 0x55759b9c3510 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7faf50033000 tx=0x7faf50030b70 comp rx=0 tx=0).stop 2024-08-02T17:59:18.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.461+0000 7faf56ffd640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7faf400a2e80 0x7faf400a5340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55759b9c0520 tx=0x7faf48009600 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:18.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.462+0000 7faf5f7ae740 1 -- 172.21.15.69:0/3307894185 shutdown_connections 2024-08-02T17:59:18.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.462+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/3307894185 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9ce310 0x55759b9d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.462+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/3307894185 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3a50 0x55759b9cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.462+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/3307894185 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3130 0x55759b9c3510 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.462+0000 7faf5f7ae740 1 -- 172.21.15.69:0/3307894185 >> 172.21.15.69:0/3307894185 conn(0x55759b9bf6c0 msgr2=0x55759b9bfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:18.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.463+0000 7faf5f7ae740 1 -- 172.21.15.69:0/3307894185 shutdown_connections 2024-08-02T17:59:18.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.463+0000 7faf5f7ae740 1 -- 172.21.15.69:0/3307894185 wait complete. 2024-08-02T17:59:18.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.463+0000 7faf5f7ae740 1 -- start start 2024-08-02T17:59:18.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.463+0000 7faf5f7ae740 1 Processor -- start 2024-08-02T17:59:18.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.463+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9cb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.463+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9cb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf577fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf577fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59756/0 (socket says 172.21.15.69:59756) 2024-08-02T17:59:18.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf567fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9cb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf567fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9cb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59380/0 (socket says 172.21.15.69:59380) 2024-08-02T17:59:18.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9ce310 0x55759b9cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55759b9d0f80 con 0x55759b9ce310 2024-08-02T17:59:18.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55759b8e3990 con 0x55759b9c3130 2024-08-02T17:59:18.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55759b9d1100 con 0x55759b9c3a50 2024-08-02T17:59:18.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf5f7ae740 10 client.0 _ll_register_callbacks cb 0x55759b638090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:18.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf577fe640 1 -- 172.21.15.69:0/1294332324 learned_addr learned my addr 172.21.15.69:0/1294332324 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:18.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf567fc640 1 -- 172.21.15.69:0/1294332324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 msgr2=0x55759b9cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:18.920 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf567fc640 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf567fc640 1 -- 172.21.15.69:0/1294332324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9ce310 msgr2=0x55759b9cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf567fc640 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9ce310 0x55759b9cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.464+0000 7faf567fc640 1 -- 172.21.15.69:0/1294332324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55759b8e26c0 con 0x55759b9c3a50 2024-08-02T17:59:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.465+0000 7faf567fc640 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9cb660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55759b9c0a40 tx=0x7faf4c00e8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48be9ca791445022 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:18.921 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.465+0000 7faf3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T17:59:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.465+0000 7faf3aff5640 1 -- 172.21.15.69:0/1294332324 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c017070 con 0x55759b9c3a50 2024-08-02T17:59:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.465+0000 7faf3aff5640 1 -- 172.21.15.69:0/1294332324 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7faf4c013680 con 0x55759b9c3a50 2024-08-02T17:59:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.465+0000 7faf5f7ae740 1 -- 172.21.15.69:0/1294332324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf50003e50 con 0x55759b9c3a50 2024-08-02T17:59:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.465+0000 7faf3aff5640 1 -- 172.21.15.69:0/1294332324 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf4c021050 con 0x55759b9c3a50 2024-08-02T17:59:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.465+0000 7faf5f7ae740 10 client.27802 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:18.922 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.465+0000 7faf5f7ae740 1 -- 172.21.15.69:0/1294332324 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55759b9dccc0 con 0x55759b9c3a50 2024-08-02T17:59:18.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf3aff5640 1 -- 172.21.15.69:0/1294332324 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf50003e50 con 0x55759b9c3a50 2024-08-02T17:59:18.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf3aff5640 1 -- 172.21.15.69:0/1294332324 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faf4c012a60 con 0x55759b9c3a50 2024-08-02T17:59:18.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf5f7ae740 10 client.27802 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:18.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf5f7ae740 10 client.27802 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:59:18.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf5f7ae740 20 client.27802 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:18.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55759b9de120 0x55759b9e05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf5f7ae740 4 client.27802 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf5f7ae740 1 -- 172.21.15.69:0/1294332324 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55759b9361a0 con 0x55759b9de120 2024-08-02T17:59:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.466+0000 7faf577fe640 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55759b9de120 0x55759b9e05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.467+0000 7faf577fe640 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55759b9de120 0x55759b9e05c0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7204edcd7b1ea31a server_cookie=3c6cf78a3a2c2ed1 in_seq=0 out_seq=0 2024-08-02T17:59:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.467+0000 7faf3aff5640 10 client.27802 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:18.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.468+0000 7faf3aff5640 1 -- 172.21.15.69:0/1294332324 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55759b9361a0 con 0x55759b9de120 2024-08-02T17:59:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.468+0000 7faf3aff5640 10 client.27802 handle_command_reply: tid=0 2024-08-02T17:59:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.468+0000 7faf5f7ae740 1 client.27802 shutdown 2024-08-02T17:59:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.468+0000 7faf5f7ae740 1 -- 172.21.15.69:0/1294332324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 msgr2=0x55759b9cb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9cb660 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55759b9c0a40 tx=0x7faf4c00e8e0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 -- 172.21.15.69:0/1294332324 shutdown_connections 2024-08-02T17:59:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9ce310 0x55759b9cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55759b9de120 0x55759b9e05c0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9cb660 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/1294332324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 -- 172.21.15.69:0/1294332324 >> 172.21.15.69:0/1294332324 conn(0x55759b9bf6c0 msgr2=0x55759b9c53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:18.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf3aff5640 0 client.27802 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 -- 172.21.15.69:0/1294332324 shutdown_connections 2024-08-02T17:59:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 1 -- 172.21.15.69:0/1294332324 wait complete. 2024-08-02T17:59:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.469+0000 7faf5f7ae740 20 client.27802 trim_cache size 0 max 16384 2024-08-02T17:59:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.499+0000 7faf5f7ae740 1 Processor -- start 2024-08-02T17:59:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.500+0000 7faf5f7ae740 1 -- start start 2024-08-02T17:59:18.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.501+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.501+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3a50 0x55759b9e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.501+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9ce310 0x55759b9c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.501+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faf44012980 con 0x55759b9c3a50 2024-08-02T17:59:18.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.501+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf44010250 con 0x55759b9c3130 2024-08-02T17:59:18.928 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.501+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf4400e0a0 con 0x55759b9ce310 2024-08-02T17:59:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.503+0000 7faf577fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.503+0000 7faf577fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9e5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:59768/0 (socket says 172.21.15.69:59768) 2024-08-02T17:59:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.503+0000 7faf577fe640 1 -- 172.21.15.69:0/316566840 learned_addr learned my addr 172.21.15.69:0/316566840 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.503+0000 7faf577fe640 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9ce310 0x55759b9c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.503+0000 7faf567fc640 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3a50 0x55759b9e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.505+0000 7faf577fe640 1 -- 172.21.15.69:0/316566840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9ce310 msgr2=0x55759b9c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.505+0000 7faf577fe640 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9ce310 0x55759b9c8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.506+0000 7faf577fe640 1 -- 172.21.15.69:0/316566840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3a50 msgr2=0x55759b9e63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.506+0000 7faf577fe640 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3a50 0x55759b9e63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.506+0000 7faf577fe640 1 -- 172.21.15.69:0/316566840 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55759b9c0770 con 0x55759b9c3130 2024-08-02T17:59:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.508+0000 7faf577fe640 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9e5e90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faf50030b40 tx=0x7faf50033040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bbab74d6fc6f92e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:18.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.510+0000 7faf3cff9640 1 -- 172.21.15.69:0/316566840 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50040040 con 0x55759b9c3130 2024-08-02T17:59:18.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.510+0000 7faf3cff9640 1 -- 172.21.15.69:0/316566840 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf50004720 con 0x55759b9c3130 2024-08-02T17:59:18.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.510+0000 7faf3cff9640 1 -- 172.21.15.69:0/316566840 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf500515e0 con 0x55759b9c3130 2024-08-02T17:59:18.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.511+0000 7faf5f7ae740 1 -- 172.21.15.69:0/316566840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 msgr2=0x55759b9e5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.511+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9e5e90 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7faf50030b40 tx=0x7faf50033040 comp rx=0 tx=0).stop 2024-08-02T17:59:18.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.512+0000 7faf5f7ae740 1 -- 172.21.15.69:0/316566840 shutdown_connections 2024-08-02T17:59:18.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.512+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3a50 0x55759b9e63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.512+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9ce310 0x55759b9c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.512+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/316566840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9c3130 0x55759b9e5e90 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.512+0000 7faf5f7ae740 1 -- 172.21.15.69:0/316566840 >> 172.21.15.69:0/316566840 conn(0x55759b9bf6c0 msgr2=0x55759b9c99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.520+0000 7faf5f7ae740 1 -- 172.21.15.69:0/316566840 shutdown_connections 2024-08-02T17:59:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.523+0000 7faf5f7ae740 1 -- 172.21.15.69:0/316566840 wait complete. 2024-08-02T17:59:18.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 1 -- start start 2024-08-02T17:59:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 1 Processor -- start 2024-08-02T17:59:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3130 0x55759b9d25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9d2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9ce310 0x55759ba47e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7faf4400e710 con 0x55759b9c3130 2024-08-02T17:59:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7faf4400fb80 con 0x55759b9ce310 2024-08-02T17:59:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faf4400ede0 con 0x55759b9c3a50 2024-08-02T17:59:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.524+0000 7faf5f7ae740 10 client.0 _ll_register_callbacks cb 0x7faf4c009410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.525+0000 7faf577fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9ce310 0x55759ba47e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.525+0000 7faf577fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3130 0x55759b9d25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.525+0000 7faf577fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3130 0x55759b9d25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44526/0 (socket says 172.21.15.69:44526) 2024-08-02T17:59:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.525+0000 7faf577fe640 1 -- 172.21.15.69:0/2816627415 learned_addr learned my addr 172.21.15.69:0/2816627415 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf577fe640 1 -- 172.21.15.69:0/2816627415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 msgr2=0x55759b9d2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf577fe640 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9d2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf577fe640 1 -- 172.21.15.69:0/2816627415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9ce310 msgr2=0x55759ba47e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:18.935 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf577fe640 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9ce310 0x55759ba47e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf577fe640 1 -- 172.21.15.69:0/2816627415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55759b9bdc70 con 0x55759b9c3130 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf577fe640 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3130 0x55759b9d25b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7faf50009540 tx=0x7faf50000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426d65e42ec46aa4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf3aff5640 1 -- 172.21.15.69:0/2816627415 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf50040050 con 0x55759b9c3130 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf5f7ae740 1 -- 172.21.15.69:0/2816627415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf50004720 con 0x55759b9c3130 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.526+0000 7faf3aff5640 1 -- 172.21.15.69:0/2816627415 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7faf50003970 con 0x55759b9c3130 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.527+0000 7faf3aff5640 1 -- 172.21.15.69:0/2816627415 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faf50030ed0 con 0x55759b9c3130 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.529+0000 7faf5f7ae740 10 client.18537 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.529+0000 7faf5f7ae740 1 -- 172.21.15.69:0/2816627415 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55759ba6b340 con 0x55759b9c3130 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf3aff5640 1 -- 172.21.15.69:0/2816627415 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf50004720 con 0x55759b9c3130 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf3aff5640 1 -- 172.21.15.69:0/2816627415 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faf500513a0 con 0x55759b9c3130 2024-08-02T17:59:18.936 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf5f7ae740 10 client.18537 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf5f7ae740 10 client.18537 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf5f7ae740 20 client.18537 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55759ba6b8d0 0x55759ba6dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf5f7ae740 4 client.18537 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf5f7ae740 1 -- 172.21.15.69:0/2816627415 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55759b9304f0 con 0x55759ba6b8d0 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.530+0000 7faf567fc640 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55759ba6b8d0 0x55759ba6dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.531+0000 7faf3aff5640 10 client.18537 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.531+0000 7faf567fc640 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55759ba6b8d0 0x55759ba6dcf0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3952f19e064c28 server_cookie=21107e49ff65049b in_seq=0 out_seq=0 2024-08-02T17:59:18.937 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 46, 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "50178048": { 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 46, 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:59:18.938 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 150.0, 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:18.939 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf3aff5640 1 -- 172.21.15.69:0/2816627415 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55759b9304f0 con 0x55759ba6b8d0 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf3aff5640 10 client.18537 handle_command_reply: tid=0 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 client.18537 shutdown 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 -- 172.21.15.69:0/2816627415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3130 msgr2=0x55759b9d25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3130 0x55759b9d25b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7faf50009540 tx=0x7faf50000f40 comp rx=0 tx=0).stop 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 -- 172.21.15.69:0/2816627415 shutdown_connections 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55759ba6b8d0 0x55759ba6dcf0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55759b9ce310 0x55759ba47e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55759b9c3a50 0x55759b9d2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 --2- 172.21.15.69:0/2816627415 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55759b9c3130 0x55759b9d25b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 -- 172.21.15.69:0/2816627415 >> 172.21.15.69:0/2816627415 conn(0x55759b9bf6c0 msgr2=0x55759b9bded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:18.940 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf3aff5640 0 client.18537 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 -- 172.21.15.69:0/2816627415 shutdown_connections 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.882+0000 7faf5f7ae740 1 -- 172.21.15.69:0/2816627415 wait complete. 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5f7ae740 20 client.18537 trim_cache size 0 max 16384 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 -- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7faf400a2e80 msgr2=0x7faf400a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7faf400a2e80 0x7faf400a5340 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55759b9c0520 tx=0x7faf48009600 comp rx=0 tx=0).stop 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 -- 172.21.15.69:0/1407433689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf58100870 msgr2=0x7faf58073a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf58100870 0x7faf58073a10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faf4800e790 tx=0x7faf4800ec50 comp rx=0 tx=0).stop 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 -- 172.21.15.69:0/1407433689 shutdown_connections 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7faf58101190 0x7faf5806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7faf400a2e80 0x7faf400a5340 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7faf58105c80 0x7faf5806efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 --2- 172.21.15.69:0/1407433689 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faf58100870 0x7faf58073a10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.883+0000 7faf5d3b0640 1 -- 172.21.15.69:0/1407433689 >> 172.21.15.69:0/1407433689 conn(0x7faf580fc500 msgr2=0x7faf58103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:18.941 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.884+0000 7faf5d3b0640 1 -- 172.21.15.69:0/1407433689 shutdown_connections 2024-08-02T17:59:18.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:18.884+0000 7faf5d3b0640 1 -- 172.21.15.69:0/1407433689 wait complete. 2024-08-02T17:59:19.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:18 smithi083 ceph-mon[37031]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 5.2 MiB/s wr, 592 op/s 2024-08-02T17:59:19.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:18 smithi077 ceph-mon[37108]: pgmap v403: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 5.2 MiB/s wr, 592 op/s 2024-08-02T17:59:19.559 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '50178048', quiesce took 6.4 seconds. Will release after: 23.6 2024-08-02T17:59:19.614 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:19.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 730.0 s, 386.5 tps, lat 182.185 ms stddev 210.214 2024-08-02T17:59:20.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 -- 172.21.15.69:0/3292465637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41067b0 msgr2=0x7f76c4106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 --2- 172.21.15.69:0/3292465637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41067b0 0x7f76c4106b90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009930 tx=0x7f76b402a420 comp rx=0 tx=0).stop 2024-08-02T17:59:20.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 -- 172.21.15.69:0/3292465637 shutdown_connections 2024-08-02T17:59:20.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 --2- 172.21.15.69:0/3292465637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c4101150 0x7f76c410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 --2- 172.21.15.69:0/3292465637 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76c41007b0 0x7f76c4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 --2- 172.21.15.69:0/3292465637 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41067b0 0x7f76c4106b90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 -- 172.21.15.69:0/3292465637 >> 172.21.15.69:0/3292465637 conn(0x7f76c40fc4d0 msgr2=0x7f76c40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 -- 172.21.15.69:0/3292465637 shutdown_connections 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.073+0000 7f76ca715640 1 -- 172.21.15.69:0/3292465637 wait complete. 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.074+0000 7f76ca715640 1 Processor -- start 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.074+0000 7f76ca715640 1 -- start start 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76ca715640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41007b0 0x7f76c4198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76ca715640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76c4101150 0x7f76c4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76ca715640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c41067b0 0x7f76c419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76ca715640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4111cf0 con 0x7f76c41007b0 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76ca715640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4111b70 con 0x7f76c4101150 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76ca715640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76c4111e70 con 0x7f76c41067b0 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41007b0 0x7f76c4198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41007b0 0x7f76c4198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44534/0 (socket says 172.21.15.69:44534) 2024-08-02T17:59:20.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 -- 172.21.15.69:0/2421031147 learned_addr learned my addr 172.21.15.69:0/2421031147 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c41067b0 0x7f76c419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76bbfff640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76c4101150 0x7f76c4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 -- 172.21.15.69:0/2421031147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c41067b0 msgr2=0x7f76c419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c41067b0 0x7f76c419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 -- 172.21.15.69:0/2421031147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76c4101150 msgr2=0x7f76c4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76c4101150 0x7f76c4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 -- 172.21.15.69:0/2421031147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c419d1e0 con 0x7f76c41007b0 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76bbfff640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76c4101150 0x7f76c4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c3fff640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41007b0 0x7f76c4198230 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009230 tx=0x7f76b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c1ffb640 1 -- 172.21.15.69:0/2421031147 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b403f030 con 0x7f76c41007b0 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c1ffb640 1 -- 172.21.15.69:0/2421031147 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76b4003ad0 con 0x7f76c41007b0 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76ca715640 1 -- 172.21.15.69:0/2421031147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c419d470 con 0x7f76c41007b0 2024-08-02T17:59:20.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76c1ffb640 1 -- 172.21.15.69:0/2421031147 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4035070 con 0x7f76c41007b0 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.075+0000 7f76ca715640 1 -- 172.21.15.69:0/2421031147 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c41a4db0 con 0x7f76c41007b0 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.077+0000 7f76c1ffb640 1 -- 172.21.15.69:0/2421031147 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f76b4042030 con 0x7f76c41007b0 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76ccb13740 1 Processor -- start 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76ccb13740 1 -- start start 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d038ff0 0x55e44d0393d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d039910 0x55e44d043c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d0465c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76ccb13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e44cf9ea40 con 0x55e44d038ff0 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76ccb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e44cf369d0 con 0x55e44d0441d0 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76ccb13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e44cf9aa00 con 0x55e44d039910 2024-08-02T17:59:20.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c1ffb640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f76b00a2d70 0x7f76b00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c8cb2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d038ff0 0x55e44d0393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c8cb2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d038ff0 0x55e44d0393d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44550/0 (socket says 172.21.15.69:44550) 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c8cb2640 1 -- 172.21.15.69:0/1820597446 learned_addr learned my addr 172.21.15.69:0/1820597446 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f76b00a2d70 0x7f76b00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c1ffb640 1 -- 172.21.15.69:0/2421031147 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f76b40efd60 con 0x7f76c41007b0 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c3fff640 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d039910 0x55e44d043c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c8cb2640 1 -- 172.21.15.69:0/1820597446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d039910 msgr2=0x55e44d043c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76bbfff640 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d0465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d039910 0x55e44d043c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c8cb2640 1 -- 172.21.15.69:0/1820597446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 msgr2=0x55e44d0465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.078+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d0465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.079+0000 7f76c8cb2640 1 -- 172.21.15.69:0/1820597446 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44cf3d650 con 0x55e44d038ff0 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.079+0000 7f76bbfff640 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d0465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:20.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.079+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d038ff0 0x55e44d0393d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0144f0 tx=0x7f76bc014e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79d81622cc09e1a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.079+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f76b00a2d70 0x7f76b00a5230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f76bc012760 tx=0x7f76bc022000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.079+0000 7f76ba7fc640 1 -- 172.21.15.69:0/1820597446 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc01ce30 con 0x55e44d038ff0 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.079+0000 7f76ba7fc640 1 -- 172.21.15.69:0/1820597446 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76bc012c70 con 0x55e44d038ff0 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.079+0000 7f76ba7fc640 1 -- 172.21.15.69:0/1820597446 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc013bb0 con 0x55e44d038ff0 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 -- 172.21.15.69:0/1820597446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d038ff0 msgr2=0x55e44d0393d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d038ff0 0x55e44d0393d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0144f0 tx=0x7f76bc014e50 comp rx=0 tx=0).stop 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 -- 172.21.15.69:0/1820597446 shutdown_connections 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d0465c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d039910 0x55e44d043c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 --2- 172.21.15.69:0/1820597446 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d038ff0 0x55e44d0393d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 -- 172.21.15.69:0/1820597446 >> 172.21.15.69:0/1820597446 conn(0x55e44d035580 msgr2=0x55e44d035960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 -- 172.21.15.69:0/1820597446 shutdown_connections 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 -- 172.21.15.69:0/1820597446 wait complete. 2024-08-02T17:59:20.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 -- start start 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 Processor -- start 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.080+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d038ff0 0x55e44d0410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d0415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d0441d0 0x55e44d041b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76ccb13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e44d046da0 con 0x55e44d039910 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76ccb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e44cf59b20 con 0x55e44d038ff0 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76ccb13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e44d046f20 con 0x55e44d0441d0 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76ccb13740 10 client.0 _ll_register_callbacks cb 0x55e44cf57bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d0415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d0415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44556/0 (socket says 172.21.15.69:44556) 2024-08-02T17:59:20.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 -- 172.21.15.69:0/3184819952 learned_addr learned my addr 172.21.15.69:0/3184819952 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76c3fff640 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d038ff0 0x55e44d0410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d0441d0 0x55e44d041b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 -- 172.21.15.69:0/3184819952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d0441d0 msgr2=0x55e44d041b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d0441d0 0x55e44d041b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 -- 172.21.15.69:0/3184819952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d038ff0 msgr2=0x55e44d0410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d038ff0 0x55e44d0410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 -- 172.21.15.69:0/3184819952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44cd103c0 con 0x55e44d039910 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76bbfff640 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d0415f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00d2c0 tx=0x7f76ac00d780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5d5c3d793394ce server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76a37fe640 1 -- 172.21.15.69:0/3184819952 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac016070 con 0x55e44d039910 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76a37fe640 1 -- 172.21.15.69:0/3184819952 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f76ac012870 con 0x55e44d039910 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.081+0000 7f76ccb13740 1 -- 172.21.15.69:0/3184819952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76bc012c70 con 0x55e44d039910 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76a37fe640 1 -- 172.21.15.69:0/3184819952 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f76ac022050 con 0x55e44d039910 2024-08-02T17:59:20.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76ccb13740 10 client.18549 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76ccb13740 1 -- 172.21.15.69:0/3184819952 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e44d052970 con 0x55e44d039910 2024-08-02T17:59:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76a37fe640 1 -- 172.21.15.69:0/3184819952 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76bc012c70 con 0x55e44d039910 2024-08-02T17:59:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76a37fe640 1 -- 172.21.15.69:0/3184819952 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f76ac011bd0 con 0x55e44d039910 2024-08-02T17:59:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76ccb13740 10 client.18549 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76ccb13740 10 client.18549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:59:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76ccb13740 20 client.18549 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:20.112 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e44d053d80 0x55e44d056200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76ccb13740 4 client.18549 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:20.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76ccb13740 1 -- 172.21.15.69:0/3184819952 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e44cfabfb0 con 0x55e44d053d80 2024-08-02T17:59:20.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.082+0000 7f76c3fff640 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e44d053d80 0x55e44d056200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76a37fe640 10 client.18549 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:59:20.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76c3fff640 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e44d053d80 0x55e44d056200 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b6203fff2627bec server_cookie=392e4e8839934ae9 in_seq=0 out_seq=0 2024-08-02T17:59:20.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76a37fe640 1 -- 172.21.15.69:0/3184819952 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e44cfabfb0 con 0x55e44d053d80 2024-08-02T17:59:20.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76a37fe640 10 client.18549 handle_command_reply: tid=0 2024-08-02T17:59:20.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 client.18549 shutdown 2024-08-02T17:59:20.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 -- 172.21.15.69:0/3184819952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 msgr2=0x55e44d0415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d0415f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00d2c0 tx=0x7f76ac00d780 comp rx=0 tx=0).stop 2024-08-02T17:59:20.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 -- 172.21.15.69:0/3184819952 shutdown_connections 2024-08-02T17:59:20.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e44d053d80 0x55e44d056200 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d0415f0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d0441d0 0x55e44d041b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76a37fe640 0 client.18549 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:59:20.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3184819952 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d038ff0 0x55e44d0410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.083+0000 7f76ccb13740 1 -- 172.21.15.69:0/3184819952 >> 172.21.15.69:0/3184819952 conn(0x55e44d035580 msgr2=0x55e44d03b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:20.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.084+0000 7f76ccb13740 1 -- 172.21.15.69:0/3184819952 shutdown_connections 2024-08-02T17:59:20.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.084+0000 7f76ccb13740 1 -- 172.21.15.69:0/3184819952 wait complete. 2024-08-02T17:59:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.084+0000 7f76ccb13740 20 client.18549 trim_cache size 0 max 16384 2024-08-02T17:59:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76ccb13740 1 Processor -- start 2024-08-02T17:59:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76ccb13740 1 -- start start 2024-08-02T17:59:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 0x55e44d03ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d05bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d05c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76ccb13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41af6d0 con 0x55e44d039910 2024-08-02T17:59:20.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76ccb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41acfa0 con 0x55e44d0441d0 2024-08-02T17:59:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76ccb13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41aadf0 con 0x55e44d038ff0 2024-08-02T17:59:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d05bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d05bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44568/0 (socket says 172.21.15.69:44568) 2024-08-02T17:59:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 -- 172.21.15.69:0/4025340219 learned_addr learned my addr 172.21.15.69:0/4025340219 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/4025340219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d05c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 -- 172.21.15.69:0/4025340219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 msgr2=0x55e44d03ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:20.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 --2- 172.21.15.69:0/4025340219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 0x55e44d03ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 -- 172.21.15.69:0/4025340219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 msgr2=0x55e44d05c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 --2- 172.21.15.69:0/4025340219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d05c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 -- 172.21.15.69:0/4025340219 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e44d036630 con 0x55e44d039910 2024-08-02T17:59:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76bbfff640 1 --2- 172.21.15.69:0/4025340219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d05bad0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f76ac001eb0 tx=0x7f76ac006da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f98f4cac77bfbed server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76b9ffb640 1 -- 172.21.15.69:0/4025340219 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac016040 con 0x55e44d039910 2024-08-02T17:59:20.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76b9ffb640 1 -- 172.21.15.69:0/4025340219 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ac02b070 con 0x55e44d039910 2024-08-02T17:59:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.097+0000 7f76b9ffb640 1 -- 172.21.15.69:0/4025340219 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ac01ac20 con 0x55e44d039910 2024-08-02T17:59:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 -- 172.21.15.69:0/4025340219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 msgr2=0x55e44d05bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 --2- 172.21.15.69:0/4025340219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d05bad0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f76ac001eb0 tx=0x7f76ac006da0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 -- 172.21.15.69:0/4025340219 shutdown_connections 2024-08-02T17:59:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 --2- 172.21.15.69:0/4025340219 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d039910 0x55e44d05bad0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 --2- 172.21.15.69:0/4025340219 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d0441d0 0x55e44d05c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 --2- 172.21.15.69:0/4025340219 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 0x55e44d03ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 -- 172.21.15.69:0/4025340219 >> 172.21.15.69:0/4025340219 conn(0x55e44d035580 msgr2=0x55e44d0401a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 -- 172.21.15.69:0/4025340219 shutdown_connections 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 -- 172.21.15.69:0/4025340219 wait complete. 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.098+0000 7f76ccb13740 1 -- start start 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76ccb13740 1 Processor -- start 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 0x55e44d0478b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d039910 0x55e44d047df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76ccb13740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d0441d0 0x55e44d0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76ccb13740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41ab460 con 0x55e44d0441d0 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76ccb13740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41ac8d0 con 0x55e44d039910 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76ccb13740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f76c41abb30 con 0x55e44d038ff0 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76ccb13740 10 client.0 _ll_register_callbacks cb 0x7f76ac00b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d0441d0 0x55e44d0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d0441d0 0x55e44d0b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44582/0 (socket says 172.21.15.69:44582) 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 -- 172.21.15.69:0/3839788335 learned_addr learned my addr 172.21.15.69:0/3839788335 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76bbfff640 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d039910 0x55e44d047df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 -- 172.21.15.69:0/3839788335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 msgr2=0x55e44d0478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c3fff640 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 0x55e44d0478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 0x55e44d0478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 -- 172.21.15.69:0/3839788335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d039910 msgr2=0x55e44d047df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d039910 0x55e44d047df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 -- 172.21.15.69:0/3839788335 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e44d0b7050 con 0x55e44d0441d0 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c3fff640 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 0x55e44d0478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76c8cb2640 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d0441d0 0x55e44d0b6a60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0097e0 tx=0x7f76bc0145f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f3f33e85fea63a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.099+0000 7f76a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76a37fe640 1 -- 172.21.15.69:0/3839788335 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc031020 con 0x55e44d0441d0 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76a37fe640 1 -- 172.21.15.69:0/3839788335 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f76bc012c70 con 0x55e44d0441d0 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 1 -- 172.21.15.69:0/3839788335 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ac02b070 con 0x55e44d0441d0 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76a37fe640 1 -- 172.21.15.69:0/3839788335 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f76bc012e70 con 0x55e44d0441d0 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 10 client.18555 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:20.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 1 -- 172.21.15.69:0/3839788335 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e44d0be3b0 con 0x55e44d0441d0 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76a37fe640 1 -- 172.21.15.69:0/3839788335 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ac02b070 con 0x55e44d0441d0 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76a37fe640 1 -- 172.21.15.69:0/3839788335 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f76bc023040 con 0x55e44d0441d0 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 10 client.18555 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 10 client.18555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 20 client.18555 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e44d0be9e0 0x55e44d0c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 4 client.18555 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76ccb13740 1 -- 172.21.15.69:0/3839788335 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e44cfa64c0 con 0x55e44d0be9e0 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.100+0000 7f76c3fff640 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e44d0be9e0 0x55e44d0c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76a37fe640 10 client.18555 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76c3fff640 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e44d0be9e0 0x55e44d0c0e00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d72a6c9a67d9e15b server_cookie=74843cdcd9606ea5 in_seq=0 out_seq=0 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76a37fe640 1 -- 172.21.15.69:0/3839788335 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e44cfa64c0 con 0x55e44d0be9e0 2024-08-02T17:59:20.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76a37fe640 10 client.18555 handle_command_reply: tid=0 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 client.18555 shutdown 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 -- 172.21.15.69:0/3839788335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d0441d0 msgr2=0x55e44d0b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d0441d0 0x55e44d0b6a60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0097e0 tx=0x7f76bc0145f0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 -- 172.21.15.69:0/3839788335 shutdown_connections 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e44d0be9e0 0x55e44d0c0e00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76a37fe640 0 client.18555 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e44d0441d0 0x55e44d0b6a60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e44d039910 0x55e44d047df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 --2- 172.21.15.69:0/3839788335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e44d038ff0 0x55e44d0478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.101+0000 7f76ccb13740 1 -- 172.21.15.69:0/3839788335 >> 172.21.15.69:0/3839788335 conn(0x55e44d035580 msgr2=0x55e44d041020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ccb13740 1 -- 172.21.15.69:0/3839788335 shutdown_connections 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ccb13740 1 -- 172.21.15.69:0/3839788335 wait complete. 2024-08-02T17:59:20.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ccb13740 20 client.18555 trim_cache size 0 max 16384 2024-08-02T17:59:20.125 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 -- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f76b00a2d70 msgr2=0x7f76b00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f76b00a2d70 0x7f76b00a5230 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f76bc012760 tx=0x7f76bc022000 comp rx=0 tx=0).stop 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 -- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41007b0 msgr2=0x7f76c4198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41007b0 0x7f76c4198230 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009230 tx=0x7f76b4004270 comp rx=0 tx=0).stop 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 -- 172.21.15.69:0/2421031147 shutdown_connections 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f76b00a2d70 0x7f76b00a5230 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76c41067b0 0x7f76c419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f76c4101150 0x7f76c4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 --2- 172.21.15.69:0/2421031147 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f76c41007b0 0x7f76c4198230 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:20.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 -- 172.21.15.69:0/2421031147 >> 172.21.15.69:0/2421031147 conn(0x7f76c40fc4d0 msgr2=0x7f76c40fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:20.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.102+0000 7f76ca715640 1 -- 172.21.15.69:0/2421031147 shutdown_connections 2024-08-02T17:59:20.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:20.103+0000 7f76ca715640 1 -- 172.21.15.69:0/2421031147 wait complete. 2024-08-02T17:59:20.782 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:20 smithi069 ceph-mon[32746]: pgmap v404: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-08-02T17:59:20.815 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T17:59:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:20 smithi083 ceph-mon[37031]: pgmap v404: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-08-02T17:59:21.076 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:20 smithi077 ceph-mon[37108]: pgmap v404: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-08-02T17:59:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 -- 172.21.15.69:0/3102935232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a0102250 msgr2=0x7f33a010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 --2- 172.21.15.69:0/3102935232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a0102250 0x7f33a010e780 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f3398009cd0 tx=0x7f339802a8e0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 -- 172.21.15.69:0/3102935232 shutdown_connections 2024-08-02T17:59:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 --2- 172.21.15.69:0/3102935232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a0102250 0x7f33a010e780 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 --2- 172.21.15.69:0/3102935232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33a01018b0 0x7f33a0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 --2- 172.21.15.69:0/3102935232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a01078b0 0x7f33a0107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 -- 172.21.15.69:0/3102935232 >> 172.21.15.69:0/3102935232 conn(0x7f33a00fd5f0 msgr2=0x7f33a00ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 -- 172.21.15.69:0/3102935232 shutdown_connections 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.539+0000 7f33a6c94640 1 -- 172.21.15.69:0/3102935232 wait complete. 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a6c94640 1 Processor -- start 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a6c94640 1 -- start start 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a6c94640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33a01018b0 0x7f33a01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a6c94640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a0102250 0x7f33a0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a6c94640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a01078b0 0x7f33a019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a6c94640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0111cb0 con 0x7f33a01078b0 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a6c94640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0111b30 con 0x7f33a01018b0 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a6c94640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33a0111e30 con 0x7f33a0102250 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a5231640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a01078b0 0x7f33a019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a5231640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a01078b0 0x7f33a019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44600/0 (socket says 172.21.15.69:44600) 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a5231640 1 -- 172.21.15.69:0/4074318901 learned_addr learned my addr 172.21.15.69:0/4074318901 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f3397fff640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a0102250 0x7f33a0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.540+0000 7f33a4a30640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33a01018b0 0x7f33a01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f33a5231640 1 -- 172.21.15.69:0/4074318901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a0102250 msgr2=0x7f33a0198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f33a5231640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a0102250 0x7f33a0198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f33a5231640 1 -- 172.21.15.69:0/4074318901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33a01018b0 msgr2=0x7f33a01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f33a5231640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33a01018b0 0x7f33a01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f33a5231640 1 -- 172.21.15.69:0/4074318901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a019d080 con 0x7f33a01078b0 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f33a5231640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a01078b0 0x7f33a019c9a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f3398005cc0 tx=0x7f3398003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f3395ffb640 1 -- 172.21.15.69:0/4074318901 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398004040 con 0x7f33a01078b0 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f33a6c94640 1 -- 172.21.15.69:0/4074318901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a019d310 con 0x7f33a01078b0 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f3395ffb640 1 -- 172.21.15.69:0/4074318901 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33980046c0 con 0x7f33a01078b0 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f3395ffb640 1 -- 172.21.15.69:0/4074318901 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398030e70 con 0x7f33a01078b0 2024-08-02T17:59:21.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.541+0000 7f33a6c94640 1 -- 172.21.15.69:0/4074318901 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a01a4c50 con 0x7f33a01078b0 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f3395ffb640 1 -- 172.21.15.69:0/4074318901 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f33980305a0 con 0x7f33a01078b0 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a9092740 1 Processor -- start 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a9092740 1 -- start start 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a9092740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf6c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a9092740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf76c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a9092740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf795c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a9092740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561e5bed1a40 con 0x561e5bf6c910 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a9092740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561e5be699d0 con 0x561e5bf771d0 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a9092740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561e5becda00 con 0x561e5bf6bff0 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f3395ffb640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f33880a2cf0 0x7f33880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a4a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf76c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a4a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf76c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44614/0 (socket says 172.21.15.69:44614) 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.544+0000 7f33a4a30640 1 -- 172.21.15.69:0/2566366157 learned_addr learned my addr 172.21.15.69:0/2566366157 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f3397fff640 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf6c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f3395ffb640 1 -- 172.21.15.69:0/4074318901 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f339802e030 con 0x7f33a01078b0 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f33a5231640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f33880a2cf0 0x7f33880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f3397fff640 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f33a4a30640 1 -- 172.21.15.69:0/2566366157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 msgr2=0x561e5bf6c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f33a4a30640 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf6c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f33a4a30640 1 -- 172.21.15.69:0/2566366157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 msgr2=0x561e5bf795c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f33a4a30640 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf795c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f33a4a30640 1 -- 172.21.15.69:0/2566366157 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5be70650 con 0x561e5bf6c910 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f3397fff640 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f33a4a30640 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf76c90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f339c00ac90 tx=0x7f339c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf84f399e5190a28 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f33a5231640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f33880a2cf0 0x7f33880a51b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f33980b93b0 tx=0x7f3398003f70 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:21.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f3386ffd640 1 -- 172.21.15.69:0/2566366157 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c01d070 con 0x561e5bf6c910 2024-08-02T17:59:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f3386ffd640 1 -- 172.21.15.69:0/2566366157 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c018b20 con 0x561e5bf6c910 2024-08-02T17:59:21.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.545+0000 7f3386ffd640 1 -- 172.21.15.69:0/2566366157 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c010b40 con 0x561e5bf6c910 2024-08-02T17:59:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 -- 172.21.15.69:0/2566366157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 msgr2=0x561e5bf76c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf76c90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f339c00ac90 tx=0x7f339c00f790 comp rx=0 tx=0).stop 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 -- 172.21.15.69:0/2566366157 shutdown_connections 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf76c90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf795c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 --2- 172.21.15.69:0/2566366157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf6c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 -- 172.21.15.69:0/2566366157 >> 172.21.15.69:0/2566366157 conn(0x561e5bf68580 msgr2=0x561e5bf68960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 -- 172.21.15.69:0/2566366157 shutdown_connections 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 -- 172.21.15.69:0/2566366157 wait complete. 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.546+0000 7f33a9092740 1 -- start start 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a9092740 1 Processor -- start 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a9092740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6bff0 0x561e5bf74060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a9092740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf745a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a9092740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf771d0 0x561e5bf74ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a9092740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561e5bf79da0 con 0x561e5bf6bff0 2024-08-02T17:59:21.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a9092740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561e5be8cb20 con 0x561e5bf6c910 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a9092740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561e5bf79f20 con 0x561e5bf771d0 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a9092740 10 client.0 _ll_register_callbacks cb 0x561e5be8abb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f3397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6bff0 0x561e5bf74060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f3397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6bff0 0x561e5bf74060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44622/0 (socket says 172.21.15.69:44622) 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f3397fff640 1 -- 172.21.15.69:0/3428102020 learned_addr learned my addr 172.21.15.69:0/3428102020 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f33a4a30640 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f3397fff640 1 -- 172.21.15.69:0/3428102020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf771d0 msgr2=0x561e5bf74ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f3397fff640 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf771d0 0x561e5bf74ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f3397fff640 1 -- 172.21.15.69:0/3428102020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 msgr2=0x561e5bf745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f3397fff640 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.547+0000 7f3397fff640 1 -- 172.21.15.69:0/3428102020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5bc433c0 con 0x561e5bf6bff0 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f3397fff640 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6bff0 0x561e5bf74060 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3390004360 tx=0x7f33900039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95517f81de0e939 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f337ffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:21.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f337ffff640 1 -- 172.21.15.69:0/3428102020 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390012860 con 0x561e5bf6bff0 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f337ffff640 1 -- 172.21.15.69:0/3428102020 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f3390012a00 con 0x561e5bf6bff0 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 1 -- 172.21.15.69:0/3428102020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339c018b20 con 0x561e5bf6bff0 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f337ffff640 1 -- 172.21.15.69:0/3428102020 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3390012c00 con 0x561e5bf6bff0 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 10 client.18567 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 1 -- 172.21.15.69:0/3428102020 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e5bf85a00 con 0x561e5bf6bff0 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f337ffff640 1 -- 172.21.15.69:0/3428102020 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c018b20 con 0x561e5bf6bff0 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f337ffff640 1 -- 172.21.15.69:0/3428102020 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f339001c020 con 0x561e5bf6bff0 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 10 client.18567 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 10 client.18567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 20 client.18567 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561e5bf86ec0 0x561e5bf892b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 4 client.18567 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:21.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.548+0000 7f33a9092740 1 -- 172.21.15.69:0/3428102020 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e5bedefb0 con 0x561e5bf86ec0 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.549+0000 7f33a4a30640 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561e5bf86ec0 0x561e5bf892b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.549+0000 7f33a4a30640 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561e5bf86ec0 0x561e5bf892b0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220932580b6c3ba6 server_cookie=9c6e428123e9e9f0 in_seq=0 out_seq=0 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.549+0000 7f337ffff640 10 client.18567 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f337ffff640 1 -- 172.21.15.69:0/3428102020 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e5bedefb0 con 0x561e5bf86ec0 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f337ffff640 10 client.18567 handle_command_reply: tid=0 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 client.18567 shutdown 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 -- 172.21.15.69:0/3428102020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6bff0 msgr2=0x561e5bf74060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6bff0 0x561e5bf74060 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f3390004360 tx=0x7f33900039c0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 -- 172.21.15.69:0/3428102020 shutdown_connections 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561e5bf86ec0 0x561e5bf892b0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf771d0 0x561e5bf74ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 --2- 172.21.15.69:0/3428102020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6bff0 0x561e5bf74060 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 -- 172.21.15.69:0/3428102020 >> 172.21.15.69:0/3428102020 conn(0x561e5bf68580 msgr2=0x561e5bf6e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f337ffff640 0 client.18567 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 -- 172.21.15.69:0/3428102020 shutdown_connections 2024-08-02T17:59:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 1 -- 172.21.15.69:0/3428102020 wait complete. 2024-08-02T17:59:21.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.550+0000 7f33a9092740 20 client.18567 trim_cache size 0 max 16384 2024-08-02T17:59:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a9092740 1 Processor -- start 2024-08-02T17:59:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a9092740 1 -- start start 2024-08-02T17:59:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a9092740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf6ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a9092740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf6f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a9092740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf771d0 0x561e5bf6f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a9092740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01b16c0 con 0x561e5bf771d0 2024-08-02T17:59:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a9092740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01aef90 con 0x561e5bf6c910 2024-08-02T17:59:21.570 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a9092740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01acde0 con 0x561e5bf6bff0 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f3397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf771d0 0x561e5bf6f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f3397fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf771d0 0x561e5bf6f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44638/0 (socket says 172.21.15.69:44638) 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f3397fff640 1 -- 172.21.15.69:0/584717663 learned_addr learned my addr 172.21.15.69:0/584717663 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f3397fff640 1 -- 172.21.15.69:0/584717663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 msgr2=0x561e5bf6ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f3397fff640 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf6ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f3397fff640 1 -- 172.21.15.69:0/584717663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 msgr2=0x561e5bf6f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a4a30640 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf6f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f3397fff640 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf6f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f3397fff640 1 -- 172.21.15.69:0/584717663 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5bf79900 con 0x561e5bf771d0 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.563+0000 7f33a4a30640 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf6f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f3397fff640 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf771d0 0x561e5bf6f8b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f339000b620 tx=0x7f3390012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7501d6d7c3dea8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33867fc640 1 -- 172.21.15.69:0/584717663 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33900108b0 con 0x561e5bf771d0 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33867fc640 1 -- 172.21.15.69:0/584717663 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3390016040 con 0x561e5bf771d0 2024-08-02T17:59:21.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33867fc640 1 -- 172.21.15.69:0/584717663 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339001e6a0 con 0x561e5bf771d0 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 -- 172.21.15.69:0/584717663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf771d0 msgr2=0x561e5bf6f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf771d0 0x561e5bf6f8b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f339000b620 tx=0x7f3390012f30 comp rx=0 tx=0).stop 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 -- 172.21.15.69:0/584717663 shutdown_connections 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf771d0 0x561e5bf6f8b0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf6c910 0x561e5bf6f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 --2- 172.21.15.69:0/584717663 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf6ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 -- 172.21.15.69:0/584717663 >> 172.21.15.69:0/584717663 conn(0x561e5bf68580 msgr2=0x561e5bf631b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 -- 172.21.15.69:0/584717663 shutdown_connections 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.564+0000 7f33a9092740 1 -- 172.21.15.69:0/584717663 wait complete. 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 1 -- start start 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 1 Processor -- start 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf7f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf7f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf7a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01ad450 con 0x561e5bf6c910 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01ae8c0 con 0x561e5bf771d0 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33a01adb20 con 0x561e5bf6bff0 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a9092740 10 client.0 _ll_register_callbacks cb 0x7f339000bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a4a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf7f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a4a30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf7f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44648/0 (socket says 172.21.15.69:44648) 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f33a4a30640 1 -- 172.21.15.69:0/2837626209 learned_addr learned my addr 172.21.15.69:0/2837626209 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.565+0000 7f3397fff640 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf7a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f3397fff640 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf7f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a4a30640 1 -- 172.21.15.69:0/2837626209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 msgr2=0x561e5bf7f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a4a30640 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf7f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a4a30640 1 -- 172.21.15.69:0/2837626209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 msgr2=0x561e5bf7a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a4a30640 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf7a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a4a30640 1 -- 172.21.15.69:0/2837626209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e5bf7aa50 con 0x561e5bf6c910 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f3397fff640 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf7f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a4a30640 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf7f930 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f339c00ac90 tx=0x7f339c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9d2a39460c0f5d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f337ffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f337ffff640 1 -- 172.21.15.69:0/2837626209 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c032060 con 0x561e5bf6c910 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f337ffff640 1 -- 172.21.15.69:0/2837626209 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f339c010870 con 0x561e5bf6c910 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a9092740 1 -- 172.21.15.69:0/2837626209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3390016040 con 0x561e5bf6c910 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f337ffff640 1 -- 172.21.15.69:0/2837626209 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f339c039050 con 0x561e5bf6c910 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a9092740 10 client.18573 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f33a9092740 1 -- 172.21.15.69:0/2837626209 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561e5bff06e0 con 0x561e5bf6c910 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.566+0000 7f337ffff640 1 -- 172.21.15.69:0/2837626209 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3390016040 con 0x561e5bf6c910 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f337ffff640 1 -- 172.21.15.69:0/2837626209 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f339c0189b0 con 0x561e5bf6c910 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 10 client.18573 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 10 client.18573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:21.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 20 client.18573 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561e5bff0d00 0x561e5bff3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 4 client.18573 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 1 -- 172.21.15.69:0/2837626209 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e5bed94c0 con 0x561e5bff0d00 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f3397fff640 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561e5bff0d00 0x561e5bff3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f337ffff640 10 client.18573 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f3397fff640 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561e5bff0d00 0x561e5bff3120 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976a5fa5b3b3fed8 server_cookie=2457305a1ef2b097 in_seq=0 out_seq=0 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f337ffff640 1 -- 172.21.15.69:0/2837626209 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e5bed94c0 con 0x561e5bff0d00 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f337ffff640 10 client.18573 handle_command_reply: tid=0 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 1 client.18573 shutdown 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 1 -- 172.21.15.69:0/2837626209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 msgr2=0x561e5bf7f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.567+0000 7f33a9092740 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf7f930 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f339c00ac90 tx=0x7f339c010800 comp rx=0 tx=0).stop 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 1 -- 172.21.15.69:0/2837626209 shutdown_connections 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561e5bf6c910 0x561e5bf7f930 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x561e5bff0d00 0x561e5bff3120 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561e5bf771d0 0x561e5bf7a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 1 --2- 172.21.15.69:0/2837626209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561e5bf6bff0 0x561e5bf7f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f337ffff640 0 client.18573 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:21.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 1 -- 172.21.15.69:0/2837626209 >> 172.21.15.69:0/2837626209 conn(0x561e5bf68580 msgr2=0x561e5bf6f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:21.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 1 -- 172.21.15.69:0/2837626209 shutdown_connections 2024-08-02T17:59:21.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 1 -- 172.21.15.69:0/2837626209 wait complete. 2024-08-02T17:59:21.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a9092740 20 client.18573 trim_cache size 0 max 16384 2024-08-02T17:59:21.576 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T17:59:21.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 -- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f33880a2cf0 msgr2=0x7f33880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f33880a2cf0 0x7f33880a51b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f33980b93b0 tx=0x7f3398003f70 comp rx=0 tx=0).stop 2024-08-02T17:59:21.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 -- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a01078b0 msgr2=0x7f33a019c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:21.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a01078b0 0x7f33a019c9a0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f3398005cc0 tx=0x7f3398003970 comp rx=0 tx=0).stop 2024-08-02T17:59:21.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 -- 172.21.15.69:0/4074318901 shutdown_connections 2024-08-02T17:59:21.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f33a01078b0 0x7f33a019c9a0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f33880a2cf0 0x7f33880a51b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f33a0102250 0x7f33a0198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 --2- 172.21.15.69:0/4074318901 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f33a01018b0 0x7f33a01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:21.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.568+0000 7f33a6c94640 1 -- 172.21.15.69:0/4074318901 >> 172.21.15.69:0/4074318901 conn(0x7f33a00fd5f0 msgr2=0x7f33a00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:21.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.569+0000 7f33a6c94640 1 -- 172.21.15.69:0/4074318901 shutdown_connections 2024-08-02T17:59:21.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:21.569+0000 7f33a6c94640 1 -- 172.21.15.69:0/4074318901 wait complete. 2024-08-02T17:59:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:22 smithi069 ceph-mon[32746]: pgmap v405: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-08-02T17:59:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:22 smithi083 ceph-mon[37031]: pgmap v405: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-08-02T17:59:23.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:22 smithi077 ceph-mon[37108]: pgmap v405: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.4 MiB/s wr, 603 op/s 2024-08-02T17:59:23.588 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T17:59:23.589 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T17:59:23.859 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.351+0000 7f184c0e9640 1 -- 172.21.15.69:0/826424172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844102a60 msgr2=0x7f1844102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.351+0000 7f184c0e9640 1 --2- 172.21.15.69:0/826424172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844102a60 0x7f1844102e40 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f1840009cd0 tx=0x7f184002a8e0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.351+0000 7f184c0e9640 1 -- 172.21.15.69:0/826424172 shutdown_connections 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.351+0000 7f184c0e9640 1 --2- 172.21.15.69:0/826424172 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1844107fd0 0x7f184410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.351+0000 7f184c0e9640 1 --2- 172.21.15.69:0/826424172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1844103410 0x7f18441078c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.351+0000 7f184c0e9640 1 --2- 172.21.15.69:0/826424172 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844102a60 0x7f1844102e40 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.351+0000 7f184c0e9640 1 -- 172.21.15.69:0/826424172 >> 172.21.15.69:0/826424172 conn(0x7f18440fc500 msgr2=0x7f18440fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.351+0000 7f184c0e9640 1 -- 172.21.15.69:0/826424172 shutdown_connections 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 -- 172.21.15.69:0/826424172 wait complete. 2024-08-02T17:59:24.370 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 Processor -- start 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 -- start start 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1844102a60 0x7f18441982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1844103410 0x7f1844198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844107fd0 0x7f184419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111dd0 con 0x7f1844107fd0 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111c50 con 0x7f1844102a60 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184c0e9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1844111f50 con 0x7f1844103410 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184a686640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844107fd0 0x7f184419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184a686640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844107fd0 0x7f184419cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44668/0 (socket says 172.21.15.69:44668) 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f184a686640 1 -- 172.21.15.69:0/2266821067 learned_addr learned my addr 172.21.15.69:0/2266821067 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f1849e85640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1844102a60 0x7f18441982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.352+0000 7f1849684640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1844103410 0x7f1844198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f184a686640 1 -- 172.21.15.69:0/2266821067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1844103410 msgr2=0x7f1844198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f184a686640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1844103410 0x7f1844198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f184a686640 1 -- 172.21.15.69:0/2266821067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1844102a60 msgr2=0x7f18441982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f184a686640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1844102a60 0x7f18441982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f184a686640 1 -- 172.21.15.69:0/2266821067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184419d2a0 con 0x7f1844107fd0 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f184a686640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844107fd0 0x7f184419cbc0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f183c00d750 tx=0x7f183c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f183affd640 1 -- 172.21.15.69:0/2266821067 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c017070 con 0x7f1844107fd0 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f183affd640 1 -- 172.21.15.69:0/2266821067 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f183c005500 con 0x7f1844107fd0 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f184c0e9640 1 -- 172.21.15.69:0/2266821067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184419d590 con 0x7f1844107fd0 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f183affd640 1 -- 172.21.15.69:0/2266821067 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c004d90 con 0x7f1844107fd0 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.353+0000 7f184c0e9640 1 -- 172.21.15.69:0/2266821067 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441a4e70 con 0x7f1844107fd0 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.355+0000 7f183affd640 1 -- 172.21.15.69:0/2266821067 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f183c004f30 con 0x7f1844107fd0 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.355+0000 7f184e4e7740 1 Processor -- start 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f184e4e7740 1 -- start start 2024-08-02T17:59:24.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 0x55cd71ee95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9b20 0x55cd71ef3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f184e4e7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd71eace40 con 0x55cd71ee9b20 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f184e4e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cd71e4e7f0 con 0x55cd71ee9200 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f184e4e7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd71df5fb0 con 0x55cd71ef43e0 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849e85640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9b20 0x55cd71ef3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849e85640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9b20 0x55cd71ef3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44680/0 (socket says 172.21.15.69:44680) 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849e85640 1 -- 172.21.15.69:0/2690559721 learned_addr learned my addr 172.21.15.69:0/2690559721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849684640 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 0x55cd71ee95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849684640 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f183affd640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f18300a2cf0 0x7f18300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849e85640 1 -- 172.21.15.69:0/2690559721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 msgr2=0x55cd71ef67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:24.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849e85640 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849e85640 1 -- 172.21.15.69:0/2690559721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 msgr2=0x55cd71ee95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849e85640 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 0x55cd71ee95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849e85640 1 -- 172.21.15.69:0/2690559721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd71bc06a0 con 0x55cd71ee9b20 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f1849684640 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.356+0000 7f184a686640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f18300a2cf0 0x7f18300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.357+0000 7f183affd640 1 -- 172.21.15.69:0/2266821067 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f183c0cf120 con 0x7f1844107fd0 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.357+0000 7f1849e85640 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9b20 0x55cd71ef3ea0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f1840009cd0 tx=0x7f1840005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a373f1a0942baebf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.357+0000 7f182ffff640 1 -- 172.21.15.69:0/2690559721 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840048070 con 0x55cd71ee9b20 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.357+0000 7f182ffff640 1 -- 172.21.15.69:0/2690559721 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1840036070 con 0x55cd71ee9b20 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.357+0000 7f182ffff640 1 -- 172.21.15.69:0/2690559721 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1840004520 con 0x55cd71ee9b20 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- 172.21.15.69:0/2690559721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9b20 msgr2=0x55cd71ef3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184a686640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f18300a2cf0 0x7f18300a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f183c09afe0 tx=0x7f183c06a210 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9b20 0x55cd71ef3ea0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f1840009cd0 tx=0x7f1840005fa0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- 172.21.15.69:0/2690559721 shutdown_connections 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9b20 0x55cd71ef3ea0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2690559721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 0x55cd71ee95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- 172.21.15.69:0/2690559721 >> 172.21.15.69:0/2690559721 conn(0x55cd71ee5790 msgr2=0x55cd71ee5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- 172.21.15.69:0/2690559721 shutdown_connections 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- 172.21.15.69:0/2690559721 wait complete. 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- start start 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 Processor -- start 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ef1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71ef1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd71e09cb0 con 0x55cd71ee9200 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cd71e4a7b0 con 0x55cd71ee9b20 2024-08-02T17:59:24.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.358+0000 7f184e4e7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd71ef6fb0 con 0x55cd71ef43e0 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f184e4e7740 10 client.0 _ll_register_callbacks cb 0x55cd71b5dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ef1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ef1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44686/0 (socket says 172.21.15.69:44686) 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 -- 172.21.15.69:0/1153421802 learned_addr learned my addr 172.21.15.69:0/1153421802 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849e85640 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71ef1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 -- 172.21.15.69:0/1153421802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 msgr2=0x55cd71ef1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 -- 172.21.15.69:0/1153421802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 msgr2=0x55cd71ef1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71ef1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 -- 172.21.15.69:0/1153421802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd71e08bc0 con 0x55cd71ee9200 2024-08-02T17:59:24.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f1849684640 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ef1310 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f183400bf80 tx=0x7f183400bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5e5cc3d0c1ed1e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:24.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f182d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:24.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f182d7fa640 1 -- 172.21.15.69:0/1153421802 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834011b30 con 0x55cd71ee9200 2024-08-02T17:59:24.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f182d7fa640 1 -- 172.21.15.69:0/1153421802 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f1834011cd0 con 0x55cd71ee9200 2024-08-02T17:59:24.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f184e4e7740 1 -- 172.21.15.69:0/1153421802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1840036070 con 0x55cd71ee9200 2024-08-02T17:59:24.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f182d7fa640 1 -- 172.21.15.69:0/1153421802 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1834004020 con 0x55cd71ee9200 2024-08-02T17:59:24.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f184e4e7740 10 client.18585 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:24.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.359+0000 7f184e4e7740 1 -- 172.21.15.69:0/1153421802 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd71f02f70 con 0x55cd71ee9200 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f182d7fa640 1 -- 172.21.15.69:0/1153421802 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1840036070 con 0x55cd71ee9200 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f182d7fa640 1 -- 172.21.15.69:0/1153421802 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1834020400 con 0x55cd71ee9200 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f184e4e7740 10 client.18585 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f184e4e7740 10 client.18585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f184e4e7740 20 client.18585 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f184e4e7740 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cd71f04430 0x55cd71f06820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f184e4e7740 4 client.18585 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f184e4e7740 1 -- 172.21.15.69:0/1153421802 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd71e5bd60 con 0x55cd71f04430 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f1849e85640 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cd71f04430 0x55cd71f06820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f182d7fa640 10 client.18585 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.360+0000 7f1849e85640 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cd71f04430 0x55cd71f06820 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fffb28c188676d server_cookie=6ee7ab1e1b898aad in_seq=0 out_seq=0 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f182d7fa640 1 -- 172.21.15.69:0/1153421802 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cd71e5bd60 con 0x55cd71f04430 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f182d7fa640 10 client.18585 handle_command_reply: tid=0 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 client.18585 shutdown 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 -- 172.21.15.69:0/1153421802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 msgr2=0x55cd71ef1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ef1310 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f183400bf80 tx=0x7f183400bbb0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 -- 172.21.15.69:0/1153421802 shutdown_connections 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cd71f04430 0x55cd71f06820 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71ef1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71ef1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f182d7fa640 0 client.18585 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:24.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 --2- 172.21.15.69:0/1153421802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ef1310 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.361+0000 7f184e4e7740 1 -- 172.21.15.69:0/1153421802 >> 172.21.15.69:0/1153421802 conn(0x55cd71ee5790 msgr2=0x55cd71ee48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:24.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.362+0000 7f184e4e7740 1 -- 172.21.15.69:0/1153421802 shutdown_connections 2024-08-02T17:59:24.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.362+0000 7f184e4e7740 1 -- 172.21.15.69:0/1153421802 wait complete. 2024-08-02T17:59:24.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.362+0000 7f184e4e7740 20 client.18585 trim_cache size 0 max 16384 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f184e4e7740 1 Processor -- start 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f184e4e7740 1 -- start start 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ee04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71eeb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71f0c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f184e4e7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18441b1780 con 0x55cd71ee9200 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f184e4e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18441af050 con 0x55cd71ee9b20 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f184e4e7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18441acea0 con 0x55cd71ef43e0 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.379+0000 7f1849684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ee04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ee04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44692/0 (socket says 172.21.15.69:44692) 2024-08-02T17:59:24.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849684640 1 -- 172.21.15.69:0/66566582 learned_addr learned my addr 172.21.15.69:0/66566582 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849e85640 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71eeb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849684640 1 -- 172.21.15.69:0/66566582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 msgr2=0x55cd71f0c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849684640 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71f0c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849684640 1 -- 172.21.15.69:0/66566582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 msgr2=0x55cd71eeb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849684640 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71eeb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849684640 1 -- 172.21.15.69:0/66566582 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd71ef02c0 con 0x55cd71ee9200 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849e85640 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71eeb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f1849684640 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ee04e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1834012800 tx=0x7f18340125f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb19a254537d4919 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f182f7fe640 1 -- 172.21.15.69:0/66566582 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834032070 con 0x55cd71ee9200 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f182f7fe640 1 -- 172.21.15.69:0/66566582 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18340110d0 con 0x55cd71ee9200 2024-08-02T17:59:24.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.380+0000 7f182f7fe640 1 -- 172.21.15.69:0/66566582 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834016070 con 0x55cd71ee9200 2024-08-02T17:59:24.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 -- 172.21.15.69:0/66566582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 msgr2=0x55cd71ee04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ee04e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f1834012800 tx=0x7f18340125f0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 -- 172.21.15.69:0/66566582 shutdown_connections 2024-08-02T17:59:24.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ef43e0 0x55cd71f0c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9b20 0x55cd71eeb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.387 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 --2- 172.21.15.69:0/66566582 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ee9200 0x55cd71ee04e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 -- 172.21.15.69:0/66566582 >> 172.21.15.69:0/66566582 conn(0x55cd71ee5790 msgr2=0x55cd71eeaca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 -- 172.21.15.69:0/66566582 shutdown_connections 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 -- 172.21.15.69:0/66566582 wait complete. 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 -- start start 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.381+0000 7f184e4e7740 1 Processor -- start 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 0x55cd71ef7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ee9b20 0x55cd71ef84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f184e4e7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ef43e0 0x55cd71ef89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f184e4e7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f18441ad510 con 0x55cd71ef43e0 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f184e4e7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f18441ae980 con 0x55cd71ee9200 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f184e4e7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18441adbe0 con 0x55cd71ee9b20 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f184e4e7740 10 client.0 _ll_register_callbacks cb 0x7f18340126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ef43e0 0x55cd71ef89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ef43e0 0x55cd71ef89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44696/0 (socket says 172.21.15.69:44696) 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 -- 172.21.15.69:0/2861037583 learned_addr learned my addr 172.21.15.69:0/2861037583 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849e85640 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ee9b20 0x55cd71ef84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 0x55cd71ef7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 -- 172.21.15.69:0/2861037583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ee9b20 msgr2=0x55cd71ef84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ee9b20 0x55cd71ef84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 -- 172.21.15.69:0/2861037583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 msgr2=0x55cd71ef7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 0x55cd71ef7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849684640 1 -- 172.21.15.69:0/2861037583 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd71f6e0f0 con 0x55cd71ef43e0 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.382+0000 7f1849e85640 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ee9b20 0x55cd71ef84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f1849684640 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ef43e0 0x55cd71ef89f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1834020730 tx=0x7f183400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f5eb049a5aeea8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f182d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f182d7fa640 1 -- 172.21.15.69:0/2861037583 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834016030 con 0x55cd71ef43e0 2024-08-02T17:59:24.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f182d7fa640 1 -- 172.21.15.69:0/2861037583 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f1834011d30 con 0x55cd71ef43e0 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 1 -- 172.21.15.69:0/2861037583 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18340110d0 con 0x55cd71ef43e0 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f182d7fa640 1 -- 172.21.15.69:0/2861037583 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1834032040 con 0x55cd71ef43e0 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 10 client.18591 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 1 -- 172.21.15.69:0/2861037583 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cd71f6f3f0 con 0x55cd71ef43e0 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f182d7fa640 1 -- 172.21.15.69:0/2861037583 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18340110d0 con 0x55cd71ef43e0 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f182d7fa640 1 -- 172.21.15.69:0/2861037583 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1834038030 con 0x55cd71ef43e0 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 10 client.18591 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 10 client.18591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 20 client.18591 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cd71f6fa50 0x55cd71f71e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 4 client.18591 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.383+0000 7f184e4e7740 1 -- 172.21.15.69:0/2861037583 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cd71e56270 con 0x55cd71f6fa50 2024-08-02T17:59:24.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.384+0000 7f1849e85640 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cd71f6fa50 0x55cd71f71e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.384+0000 7f182d7fa640 10 client.18591 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.384+0000 7f1849e85640 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cd71f6fa50 0x55cd71f71e40 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb6378f811d4bb7 server_cookie=16249df3236ba88 in_seq=0 out_seq=0 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f182d7fa640 1 -- 172.21.15.69:0/2861037583 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cd71e56270 con 0x55cd71f6fa50 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f182d7fa640 10 client.18591 handle_command_reply: tid=0 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 client.18591 shutdown 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 -- 172.21.15.69:0/2861037583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ef43e0 msgr2=0x55cd71ef89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ef43e0 0x55cd71ef89f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f1834020730 tx=0x7f183400be20 comp rx=0 tx=0).stop 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 -- 172.21.15.69:0/2861037583 shutdown_connections 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd71ef43e0 0x55cd71ef89f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55cd71f6fa50 0x55cd71f71e40 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd71ee9b20 0x55cd71ef84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 --2- 172.21.15.69:0/2861037583 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd71ee9200 0x55cd71ef7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f182d7fa640 0 client.18591 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.385+0000 7f184e4e7740 1 -- 172.21.15.69:0/2861037583 >> 172.21.15.69:0/2861037583 conn(0x55cd71ee5790 msgr2=0x55cd71ee48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184e4e7740 1 -- 172.21.15.69:0/2861037583 shutdown_connections 2024-08-02T17:59:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184e4e7740 1 -- 172.21.15.69:0/2861037583 wait complete. 2024-08-02T17:59:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184e4e7740 20 client.18591 trim_cache size 0 max 16384 2024-08-02T17:59:24.392 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:59:24.392 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T17:59:24.392 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "0fffa170-9e53-4e3c-8008-708b02f1f600", 2024-08-02T17:59:24.392 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T17:59:24.392 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:59:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 -- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f18300a2cf0 msgr2=0x7f18300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f18300a2cf0 0x7f18300a51b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f183c09afe0 tx=0x7f183c06a210 comp rx=0 tx=0).stop 2024-08-02T17:59:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 -- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844107fd0 msgr2=0x7f184419cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844107fd0 0x7f184419cbc0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f183c00d750 tx=0x7f183c00dc10 comp rx=0 tx=0).stop 2024-08-02T17:59:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 -- 172.21.15.69:0/2266821067 shutdown_connections 2024-08-02T17:59:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1844107fd0 0x7f184419cbc0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f18300a2cf0 0x7f18300a51b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1844103410 0x7f1844198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 --2- 172.21.15.69:0/2266821067 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1844102a60 0x7f18441982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.386+0000 7f184c0e9640 1 -- 172.21.15.69:0/2266821067 >> 172.21.15.69:0/2266821067 conn(0x7f18440fc500 msgr2=0x7f1844105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.387+0000 7f184c0e9640 1 -- 172.21.15.69:0/2266821067 shutdown_connections 2024-08-02T17:59:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:24.387+0000 7f184c0e9640 1 -- 172.21.15.69:0/2266821067 wait complete. 2024-08-02T17:59:24.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 735.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:59:24.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:24 smithi069 ceph-mon[32746]: pgmap v406: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-08-02T17:59:24.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:24 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T17:59:24.994 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:24 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:24.995 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:24 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T17:59:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:24 smithi083 ceph-mon[37031]: pgmap v406: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-08-02T17:59:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:24 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T17:59:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:24 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:24 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T17:59:25.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:24 smithi077 ceph-mon[37108]: pgmap v406: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 4.4 MiB/s wr, 435 op/s 2024-08-02T17:59:25.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:24 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T17:59:25.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:24 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:25.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:24 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T17:59:25.105 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:59:25.362 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:25.778 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:25 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 -- 172.21.15.69:0/3763448596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3930074be0 msgr2=0x7f3930112380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 --2- 172.21.15.69:0/3763448596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3930074be0 0x7f3930112380 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f3924009900 tx=0x7f392402a480 comp rx=0 tx=0).stop 2024-08-02T17:59:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 -- 172.21.15.69:0/3763448596 shutdown_connections 2024-08-02T17:59:25.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 --2- 172.21.15.69:0/3763448596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3930074be0 0x7f3930112380 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 --2- 172.21.15.69:0/3763448596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39301128c0 0x7f3930114cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 --2- 172.21.15.69:0/3763448596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f39300742c0 0x7f39300746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 -- 172.21.15.69:0/3763448596 >> 172.21.15.69:0/3763448596 conn(0x7f39300fd5f0 msgr2=0x7f39300ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 -- 172.21.15.69:0/3763448596 shutdown_connections 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.776+0000 7f3937daf640 1 -- 172.21.15.69:0/3763448596 wait complete. 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3937daf640 1 Processor -- start 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3937daf640 1 -- start start 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3937daf640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39300742c0 0x7f39301a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3937daf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930074be0 0x7f39301a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3937daf640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39301128c0 0x7f393019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3937daf640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3930118270 con 0x7f39300742c0 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3937daf640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39301180f0 con 0x7f39301128c0 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3937daf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39301183f0 con 0x7f3930074be0 2024-08-02T17:59:25.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3935b4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39300742c0 0x7f39301a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3935b4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39300742c0 0x7f39301a4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47284/0 (socket says 172.21.15.69:47284) 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f3935b4b640 1 -- 172.21.15.69:0/835311472 learned_addr learned my addr 172.21.15.69:0/835311472 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f393534a640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930074be0 0x7f39301a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.777+0000 7f393634c640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39301128c0 0x7f393019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3935b4b640 1 -- 172.21.15.69:0/835311472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930074be0 msgr2=0x7f39301a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3935b4b640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930074be0 0x7f39301a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3935b4b640 1 -- 172.21.15.69:0/835311472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39301128c0 msgr2=0x7f393019e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3935b4b640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39301128c0 0x7f393019e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3935b4b640 1 -- 172.21.15.69:0/835311472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393019ecf0 con 0x7f39300742c0 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f393634c640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39301128c0 0x7f393019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3935b4b640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39300742c0 0x7f39301a4400 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c920 tx=0x7f392c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3922ffd640 1 -- 172.21.15.69:0/835311472 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c007c70 con 0x7f39300742c0 2024-08-02T17:59:25.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3922ffd640 1 -- 172.21.15.69:0/835311472 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c007e10 con 0x7f39300742c0 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3922ffd640 1 -- 172.21.15.69:0/835311472 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c018890 con 0x7f39300742c0 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3937daf640 1 -- 172.21.15.69:0/835311472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393019efe0 con 0x7f39300742c0 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.778+0000 7f3937daf640 1 -- 172.21.15.69:0/835311472 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39301ab2d0 con 0x7f39300742c0 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.780+0000 7f3922ffd640 1 -- 172.21.15.69:0/835311472 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f392c018a70 con 0x7f39300742c0 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.780+0000 7f393a1ad740 1 Processor -- start 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393a1ad740 1 -- start start 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72210 0x55e9f5b725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393a1ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9f5ad7980 con 0x55e9f5b72b30 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393a1ad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9f5a6fa70 con 0x55e9f5b72210 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393a1ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9f5ad3940 con 0x55e9f5b7d3f0 2024-08-02T17:59:25.857 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3935b4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3935b4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47294/0 (socket says 172.21.15.69:47294) 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3935b4b640 1 -- 172.21.15.69:0/3622856700 learned_addr learned my addr 172.21.15.69:0/3622856700 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393634c640 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72210 0x55e9f5b725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f393534a640 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3922ffd640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f391c0a2cf0 0x7f391c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3935b4b640 1 -- 172.21.15.69:0/3622856700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b7d3f0 msgr2=0x55e9f5b7f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3935b4b640 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3935b4b640 1 -- 172.21.15.69:0/3622856700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72210 msgr2=0x55e9f5b725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3935b4b640 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72210 0x55e9f5b725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.781+0000 7f3935b4b640 1 -- 172.21.15.69:0/3622856700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f5ad26e0 con 0x55e9f5b72b30 2024-08-02T17:59:25.858 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.782+0000 7f393634c640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f391c0a2cf0 0x7f391c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.782+0000 7f3922ffd640 1 -- 172.21.15.69:0/835311472 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f392c0cf9b0 con 0x7f39300742c0 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.782+0000 7f3935b4b640 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7ceb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f392c0ccc60 tx=0x7f392c06eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5165c7fb9cc2db4a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.782+0000 7f391bfff640 1 -- 172.21.15.69:0/3622856700 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c09b600 con 0x55e9f5b72b30 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.782+0000 7f393634c640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f391c0a2cf0 0x7f391c0a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3928005c40 tx=0x7f3928004a40 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.782+0000 7f391bfff640 1 -- 172.21.15.69:0/3622856700 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c09c1c0 con 0x55e9f5b72b30 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.782+0000 7f391bfff640 1 -- 172.21.15.69:0/3622856700 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c09a080 con 0x55e9f5b72b30 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 -- 172.21.15.69:0/3622856700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72b30 msgr2=0x55e9f5b7ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7ceb0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f392c0ccc60 tx=0x7f392c06eb30 comp rx=0 tx=0).stop 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 -- 172.21.15.69:0/3622856700 shutdown_connections 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7ceb0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 --2- 172.21.15.69:0/3622856700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72210 0x55e9f5b725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 -- 172.21.15.69:0/3622856700 >> 172.21.15.69:0/3622856700 conn(0x55e9f5b6e7a0 msgr2=0x55e9f5b6eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:25.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 -- 172.21.15.69:0/3622856700 shutdown_connections 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 -- 172.21.15.69:0/3622856700 wait complete. 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 -- start start 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.783+0000 7f393a1ad740 1 Processor -- start 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b7a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393a1ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e9f5b7ffc0 con 0x55e9f5b72210 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393a1ad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e9f5a92aa0 con 0x55e9f5b7d3f0 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393a1ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e9f5b80140 con 0x55e9f5b72b30 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393a1ad740 10 client.0 _ll_register_callbacks cb 0x55e9f5a81e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b7a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b7a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47306/0 (socket says 172.21.15.69:47306) 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 -- 172.21.15.69:0/2153219516 learned_addr learned my addr 172.21.15.69:0/2153219516 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:25.860 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393634c640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393534a640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 -- 172.21.15.69:0/2153219516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 msgr2=0x55e9f5b7a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 -- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 msgr2=0x55e9f5b7ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 -- 172.21.15.69:0/2153219516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f5a76650 con 0x55e9f5b72210 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393534a640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393634c640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f3935b4b640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b7a290 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f392c09b7c0 tx=0x7f392c00ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf95bcac00d75f6d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f39197fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f39197fa640 1 -- 172.21.15.69:0/2153219516 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c09a270 con 0x55e9f5b72210 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f39197fa640 1 -- 172.21.15.69:0/2153219516 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f392c00fb30 con 0x55e9f5b72210 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.784+0000 7f393a1ad740 1 -- 172.21.15.69:0/2153219516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392c09c1c0 con 0x55e9f5b72210 2024-08-02T17:59:25.861 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f39197fa640 1 -- 172.21.15.69:0/2153219516 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f392c09b420 con 0x55e9f5b72210 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393a1ad740 10 client.18603 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393a1ad740 1 -- 172.21.15.69:0/2153219516 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9f58493c0 con 0x55e9f5b72210 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f39197fa640 1 -- 172.21.15.69:0/2153219516 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c09c1c0 con 0x55e9f5b72210 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f39197fa640 1 -- 172.21.15.69:0/2153219516 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f392c098130 con 0x55e9f5b72210 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393a1ad740 10 client.18603 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393a1ad740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393a1ad740 20 client.18603 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393a1ad740 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e9f5b8cd10 0x55e9f5b8f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393a1ad740 4 client.18603 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393a1ad740 1 -- 172.21.15.69:0/2153219516 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9f5ae4ef0 con 0x55e9f5b8cd10 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.785+0000 7f393534a640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e9f5b8cd10 0x55e9f5b8f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.786+0000 7f393534a640 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e9f5b8cd10 0x55e9f5b8f100 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37105801c7c3325c server_cookie=15c2fc3e91713d8a in_seq=0 out_seq=0 2024-08-02T17:59:25.862 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.786+0000 7f39197fa640 10 client.18603 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.786+0000 7f39197fa640 1 -- 172.21.15.69:0/2153219516 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e9f5ae4ef0 con 0x55e9f5b8cd10 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.786+0000 7f39197fa640 10 client.18603 handle_command_reply: tid=0 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.786+0000 7f393a1ad740 1 client.18603 shutdown 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 -- 172.21.15.69:0/2153219516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 msgr2=0x55e9f5b7a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b7a290 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f392c09b7c0 tx=0x7f392c00ff30 comp rx=0 tx=0).stop 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 -- 172.21.15.69:0/2153219516 shutdown_connections 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e9f5b8cd10 0x55e9f5b8f100 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b7ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b7a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 --2- 172.21.15.69:0/2153219516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b7a290 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.863 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f39197fa640 0 client.18603 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 -- 172.21.15.69:0/2153219516 >> 172.21.15.69:0/2153219516 conn(0x55e9f5b6e7a0 msgr2=0x55e9f5b74480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 -- 172.21.15.69:0/2153219516 shutdown_connections 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 1 -- 172.21.15.69:0/2153219516 wait complete. 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.787+0000 7f393a1ad740 20 client.18603 trim_cache size 0 max 16384 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.804+0000 7f393a1ad740 1 Processor -- start 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.804+0000 7f393a1ad740 1 -- start start 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72210 0x55e9f5b74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72b30 0x55e9f5b774f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b77a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.864 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393a1ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f39301b7e00 con 0x55e9f5b7d3f0 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393a1ad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39301b56d0 con 0x55e9f5b72b30 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393a1ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39301b3520 con 0x55e9f5b72210 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b77a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b77a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47318/0 (socket says 172.21.15.69:47318) 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 -- 172.21.15.69:0/1262205602 learned_addr learned my addr 172.21.15.69:0/1262205602 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f3935b4b640 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72210 0x55e9f5b74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 -- 172.21.15.69:0/1262205602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72210 msgr2=0x55e9f5b74c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72210 0x55e9f5b74c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 -- 172.21.15.69:0/1262205602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72b30 msgr2=0x55e9f5b774f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72b30 0x55e9f5b774f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 -- 172.21.15.69:0/1262205602 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9f5b6e240 con 0x55e9f5b7d3f0 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f3935b4b640 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72210 0x55e9f5b74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:25.865 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f393634c640 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b77a30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f392800adf0 tx=0x7f3928015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3bb7d2c256eb4a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f391b7fe640 1 -- 172.21.15.69:0/1262205602 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392801c630 con 0x55e9f5b7d3f0 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f391b7fe640 1 -- 172.21.15.69:0/1262205602 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392801ccb0 con 0x55e9f5b7d3f0 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.805+0000 7f391b7fe640 1 -- 172.21.15.69:0/1262205602 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392801f400 con 0x55e9f5b7d3f0 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 -- 172.21.15.69:0/1262205602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b7d3f0 msgr2=0x55e9f5b77a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b77a30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f392800adf0 tx=0x7f3928015000 comp rx=0 tx=0).stop 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 -- 172.21.15.69:0/1262205602 shutdown_connections 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b77a30 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b72b30 0x55e9f5b774f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 --2- 172.21.15.69:0/1262205602 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72210 0x55e9f5b74c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 -- 172.21.15.69:0/1262205602 >> 172.21.15.69:0/1262205602 conn(0x55e9f5b6e7a0 msgr2=0x55e9f5b7c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 -- 172.21.15.69:0/1262205602 shutdown_connections 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 -- 172.21.15.69:0/1262205602 wait complete. 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 -- start start 2024-08-02T17:59:25.866 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.806+0000 7f393a1ad740 1 Processor -- start 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b8f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b8fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393a1ad740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b91f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393a1ad740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f39301b3b90 con 0x55e9f5b72210 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393a1ad740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f39301b5000 con 0x55e9f5b7d3f0 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393a1ad740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f39301b4260 con 0x55e9f5b72b30 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393a1ad740 10 client.0 _ll_register_callbacks cb 0x7f392800a510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b8f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b8f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47332/0 (socket says 172.21.15.69:47332) 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 -- 172.21.15.69:0/41946065 learned_addr learned my addr 172.21.15.69:0/41946065 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393534a640 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b8fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f393634c640 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b91f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.867 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 -- 172.21.15.69:0/41946065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 msgr2=0x55e9f5b8fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b8fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 -- 172.21.15.69:0/41946065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 msgr2=0x55e9f5b91f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b91f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 -- 172.21.15.69:0/41946065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9f5b92570 con 0x55e9f5b72210 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f3935b4b640 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b8f940 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f392c0cfdb0 tx=0x7f392c09a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bc4d3f83f720fc server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f39197fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f39197fa640 1 -- 172.21.15.69:0/41946065 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c09b600 con 0x55e9f5b72210 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.807+0000 7f39197fa640 1 -- 172.21.15.69:0/41946065 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f392c00fb30 con 0x55e9f5b72210 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 1 -- 172.21.15.69:0/41946065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392801ccb0 con 0x55e9f5b72210 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f39197fa640 1 -- 172.21.15.69:0/41946065 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f392c097d60 con 0x55e9f5b72210 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 10 client.18609 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 1 -- 172.21.15.69:0/41946065 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9f5bf7150 con 0x55e9f5b72210 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f39197fa640 1 -- 172.21.15.69:0/41946065 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392801ccb0 con 0x55e9f5b72210 2024-08-02T17:59:25.868 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f39197fa640 1 -- 172.21.15.69:0/41946065 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f392c09bd20 con 0x55e9f5b72210 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 10 client.18609 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 20 client.18609 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e9f5bf7620 0x55e9f5bf9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 4 client.18609 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393a1ad740 1 -- 172.21.15.69:0/41946065 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9f5adf400 con 0x55e9f5bf7620 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.808+0000 7f393534a640 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e9f5bf7620 0x55e9f5bf9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f39197fa640 10 client.18609 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393534a640 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e9f5bf7620 0x55e9f5bf9a10 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd7d673a724cfa1 server_cookie=befc2ad13ed06627 in_seq=0 out_seq=0 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f39197fa640 1 -- 172.21.15.69:0/41946065 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e9f5adf400 con 0x55e9f5bf7620 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f39197fa640 10 client.18609 handle_command_reply: tid=0 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 client.18609 shutdown 2024-08-02T17:59:25.869 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 -- 172.21.15.69:0/41946065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 msgr2=0x55e9f5b8f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b8f940 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f392c0cfdb0 tx=0x7f392c09a080 comp rx=0 tx=0).stop 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 -- 172.21.15.69:0/41946065 shutdown_connections 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55e9f5bf7620 0x55e9f5bf9a10 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stdout: "0fffa170-9e53-4e3c-8008-708b02f1f600": { 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "0fffa170-9e53-4e3c-8008-708b02f1f600", 2024-08-02T17:59:25.870 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e9f5b7d3f0 0x55e9f5b91f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e9f5b72b30 0x55e9f5b8fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 --2- 172.21.15.69:0/41946065 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e9f5b72210 0x55e9f5b8f940 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f39197fa640 0 client.18609 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 -- 172.21.15.69:0/41946065 >> 172.21.15.69:0/41946065 conn(0x55e9f5b6e7a0 msgr2=0x55e9f5b7a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:25.871 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 -- 172.21.15.69:0/41946065 shutdown_connections 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 1 -- 172.21.15.69:0/41946065 wait complete. 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.809+0000 7f393a1ad740 20 client.18609 trim_cache size 0 max 16384 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 -- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f391c0a2cf0 msgr2=0x7f391c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f391c0a2cf0 0x7f391c0a51b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3928005c40 tx=0x7f3928004a40 comp rx=0 tx=0).stop 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 -- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39300742c0 msgr2=0x7f39301a4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39300742c0 0x7f39301a4400 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c920 tx=0x7f392c00cde0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 -- 172.21.15.69:0/835311472 shutdown_connections 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f391c0a2cf0 0x7f391c0a51b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f39301128c0 0x7f393019e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3930074be0 0x7f39301a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 --2- 172.21.15.69:0/835311472 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f39300742c0 0x7f39301a4400 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 -- 172.21.15.69:0/835311472 >> 172.21.15.69:0/835311472 conn(0x7f39300fd5f0 msgr2=0x7f39300fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 -- 172.21.15.69:0/835311472 shutdown_connections 2024-08-02T17:59:25.872 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:25.810+0000 7f3937daf640 1 -- 172.21.15.69:0/835311472 wait complete. 2024-08-02T17:59:26.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:25 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:26.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:25 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T17:59:26.563 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 -- 172.21.15.69:0/2159640324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01006d0 msgr2=0x7f26e0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 --2- 172.21.15.69:0/2159640324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01006d0 0x7f26e0100b30 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009930 tx=0x7f26cc02a420 comp rx=0 tx=0).stop 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 -- 172.21.15.69:0/2159640324 shutdown_connections 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 --2- 172.21.15.69:0/2159640324 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01006d0 0x7f26e0100b30 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 --2- 172.21.15.69:0/2159640324 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e0101070 0x7f26e010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 --2- 172.21.15.69:0/2159640324 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e01066e0 0x7f26e0106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 -- 172.21.15.69:0/2159640324 >> 172.21.15.69:0/2159640324 conn(0x7f26e00776d0 msgr2=0x7f26e00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 -- 172.21.15.69:0/2159640324 shutdown_connections 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.560+0000 7f26e6e87640 1 -- 172.21.15.69:0/2159640324 wait complete. 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e6e87640 1 Processor -- start 2024-08-02T17:59:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e6e87640 1 -- start start 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e6e87640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01006d0 0x7f26e00fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e6e87640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e0101070 0x7f26e00fd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e6e87640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01066e0 0x7f26e00fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e6e87640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0110ae0 con 0x7f26e01066e0 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e6e87640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0110960 con 0x7f26e0101070 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e6e87640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0110c60 con 0x7f26e01006d0 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e5424640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01066e0 0x7f26e00fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e5424640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01066e0 0x7f26e00fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47362/0 (socket says 172.21.15.69:47362) 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e5424640 1 -- 172.21.15.69:0/1089474542 learned_addr learned my addr 172.21.15.69:0/1089474542 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26dffff640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e0101070 0x7f26e00fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e4c23640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01006d0 0x7f26e00fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e5424640 1 -- 172.21.15.69:0/1089474542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01006d0 msgr2=0x7f26e00fcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e5424640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01006d0 0x7f26e00fcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e5424640 1 -- 172.21.15.69:0/1089474542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e0101070 msgr2=0x7f26e00fd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e5424640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e0101070 0x7f26e00fd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.561+0000 7f26e5424640 1 -- 172.21.15.69:0/1089474542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e01a6660 con 0x7f26e01066e0 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.562+0000 7f26e5424640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01066e0 0x7f26e00fd590 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f26d800eaa0 tx=0x7f26d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.562+0000 7f26ddffb640 1 -- 172.21.15.69:0/1089474542 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d800cdb0 con 0x7f26e01066e0 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.562+0000 7f26ddffb640 1 -- 172.21.15.69:0/1089474542 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d8005020 con 0x7f26e01066e0 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.562+0000 7f26ddffb640 1 -- 172.21.15.69:0/1089474542 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d80116b0 con 0x7f26e01066e0 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.562+0000 7f26e6e87640 1 -- 172.21.15.69:0/1089474542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e01a6950 con 0x7f26e01066e0 2024-08-02T17:59:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.562+0000 7f26e6e87640 1 -- 172.21.15.69:0/1089474542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e01a6e70 con 0x7f26e01066e0 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.564+0000 7f26ddffb640 1 -- 172.21.15.69:0/1089474542 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f26d8005560 con 0x7f26e01066e0 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.565+0000 7f26e9bd6740 1 Processor -- start 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.565+0000 7f26e9bd6740 1 -- start start 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.565+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1f99b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.565+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.565+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1fa4970 0x558dc1fa6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558dc1efd9b0 con 0x558dc1f9a0d0 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558dc1ea5800 con 0x558dc1f997b0 2024-08-02T17:59:26.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558dc1ef9970 con 0x558dc1fa4970 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26ddffb640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f26d00a2d70 0x7f26d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47376/0 (socket says 172.21.15.69:47376) 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 -- 172.21.15.69:0/2464921262 learned_addr learned my addr 172.21.15.69:0/2464921262 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26dffff640 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1f99b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e5424640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f26d00a2d70 0x7f26d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26dffff640 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1fa4970 0x558dc1fa6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26ddffb640 1 -- 172.21.15.69:0/1089474542 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f26d80cfb30 con 0x7f26e01066e0 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 -- 172.21.15.69:0/2464921262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1fa4970 msgr2=0x558dc1fa6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1fa4970 0x558dc1fa6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 -- 172.21.15.69:0/2464921262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 msgr2=0x558dc1f99b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:26.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1f99b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 -- 172.21.15.69:0/2464921262 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc1c11210 con 0x558dc1f9a0d0 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26dffff640 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1fa4970 0x558dc1fa6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e4c23640 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa4430 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f26d400dc30 tx=0x7f26d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dfcae97b86302d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26e5424640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f26d00a2d70 0x7f26d00a5230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26d8018000 tx=0x7f26d80054d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26caffd640 1 -- 172.21.15.69:0/2464921262 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4011440 con 0x558dc1f9a0d0 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26caffd640 1 -- 172.21.15.69:0/2464921262 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4017070 con 0x558dc1f9a0d0 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.566+0000 7f26caffd640 1 -- 172.21.15.69:0/2464921262 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4010910 con 0x558dc1f9a0d0 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 -- 172.21.15.69:0/2464921262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 msgr2=0x558dc1fa4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa4430 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f26d400dc30 tx=0x7f26d400f730 comp rx=0 tx=0).stop 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 -- 172.21.15.69:0/2464921262 shutdown_connections 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa4430 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1fa4970 0x558dc1fa6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/2464921262 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1f99b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 -- 172.21.15.69:0/2464921262 >> 172.21.15.69:0/2464921262 conn(0x558dc1f95d20 msgr2=0x558dc1f96100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 -- 172.21.15.69:0/2464921262 shutdown_connections 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.567+0000 7f26e9bd6740 1 -- 172.21.15.69:0/2464921262 wait complete. 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 1 -- start start 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 1 Processor -- start 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1fa17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558dc1fa7540 con 0x558dc1fa4970 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558dc1eb9500 con 0x558dc1f997b0 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558dc1fa76c0 con 0x558dc1f9a0d0 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e9bd6740 10 client.0 _ll_register_callbacks cb 0x558dc1c10a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:26.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26dffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26dffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47388/0 (socket says 172.21.15.69:47388) 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26dffff640 1 -- 172.21.15.69:0/53296430 learned_addr learned my addr 172.21.15.69:0/53296430 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26dffff640 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1fa17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26e4c23640 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26dffff640 1 -- 172.21.15.69:0/53296430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 msgr2=0x558dc1fa1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26dffff640 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26dffff640 1 -- 172.21.15.69:0/53296430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 msgr2=0x558dc1fa17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.568+0000 7f26dffff640 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1fa17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26dffff640 1 -- 172.21.15.69:0/53296430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc1cdd620 con 0x558dc1fa4970 2024-08-02T17:59:26.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e4c23640 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26dffff640 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa2260 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02a940 tx=0x7f26cc009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d0297d0169a747 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26bffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26bffff640 1 -- 172.21.15.69:0/53296430 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc035040 con 0x558dc1fa4970 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26bffff640 1 -- 172.21.15.69:0/53296430 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f26cc004460 con 0x558dc1fa4970 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 1 -- 172.21.15.69:0/53296430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d4017070 con 0x558dc1fa4970 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26bffff640 1 -- 172.21.15.69:0/53296430 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26cc052050 con 0x558dc1fa4970 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 10 client.18621 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 1 -- 172.21.15.69:0/53296430 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558dc1fb3130 con 0x558dc1fa4970 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26bffff640 1 -- 172.21.15.69:0/53296430 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d4017070 con 0x558dc1fa4970 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26bffff640 1 -- 172.21.15.69:0/53296430 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f26cc03a040 con 0x558dc1fa4970 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 10 client.18621 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 10 client.18621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:26.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 20 client.18621 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558dc1fb45a0 0x558dc1fb6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 4 client.18621 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.569+0000 7f26e9bd6740 1 -- 172.21.15.69:0/53296430 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dc1f0b0e0 con 0x558dc1fb45a0 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.570+0000 7f26e4c23640 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558dc1fb45a0 0x558dc1fb6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.570+0000 7f26e4c23640 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558dc1fb45a0 0x558dc1fb6990 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeb441933dacf47a server_cookie=cf756f801004204a in_seq=0 out_seq=0 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.570+0000 7f26bffff640 10 client.18621 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26bffff640 1 -- 172.21.15.69:0/53296430 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558dc1f0b0e0 con 0x558dc1fb45a0 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26bffff640 10 client.18621 handle_command_reply: tid=0 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 client.18621 shutdown 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 -- 172.21.15.69:0/53296430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 msgr2=0x558dc1fa2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa2260 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02a940 tx=0x7f26cc009660 comp rx=0 tx=0).stop 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 -- 172.21.15.69:0/53296430 shutdown_connections 2024-08-02T17:59:26.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa2260 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558dc1fb45a0 0x558dc1fb6990 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/53296430 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1fa17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 -- 172.21.15.69:0/53296430 >> 172.21.15.69:0/53296430 conn(0x558dc1f95d20 msgr2=0x558dc1f9ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26bffff640 0 client.18621 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 -- 172.21.15.69:0/53296430 shutdown_connections 2024-08-02T17:59:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 1 -- 172.21.15.69:0/53296430 wait complete. 2024-08-02T17:59:26.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.571+0000 7f26e9bd6740 20 client.18621 trim_cache size 0 max 16384 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.586+0000 7f26e9bd6740 1 Processor -- start 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.586+0000 7f26e9bd6740 1 -- start start 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f997b0 0x558dc1f95130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1f9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1fa4970 0x558dc1fa10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f26e01b3970 con 0x558dc1f9a0d0 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26e01b1240 con 0x558dc1fa4970 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26e01af090 con 0x558dc1f997b0 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1f9c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1f9c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47390/0 (socket says 172.21.15.69:47390) 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 -- 172.21.15.69:0/1301250148 learned_addr learned my addr 172.21.15.69:0/1301250148 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26dffff640 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1fa4970 0x558dc1fa10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 -- 172.21.15.69:0/1301250148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f997b0 msgr2=0x558dc1f95130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f997b0 0x558dc1f95130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 -- 172.21.15.69:0/1301250148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1fa4970 msgr2=0x558dc1fa10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1fa4970 0x558dc1fa10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 -- 172.21.15.69:0/1301250148 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc1fa4290 con 0x558dc1f9a0d0 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26dffff640 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1fa4970 0x558dc1fa10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26e4c23640 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1f9c220 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f26d4010580 tx=0x7f26d401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88652eb734534a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26ca7fc640 1 -- 172.21.15.69:0/1301250148 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4005040 con 0x558dc1f9a0d0 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26ca7fc640 1 -- 172.21.15.69:0/1301250148 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d402d030 con 0x558dc1f9a0d0 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.587+0000 7f26ca7fc640 1 -- 172.21.15.69:0/1301250148 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4010620 con 0x558dc1f9a0d0 2024-08-02T17:59:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1301250148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 msgr2=0x558dc1f9c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1f9c220 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f26d4010580 tx=0x7f26d401f040 comp rx=0 tx=0).stop 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1301250148 shutdown_connections 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1f9a0d0 0x558dc1f9c220 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1fa4970 0x558dc1fa10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1301250148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f997b0 0x558dc1f95130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1301250148 >> 172.21.15.69:0/1301250148 conn(0x558dc1f95d20 msgr2=0x558dc1fa3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1301250148 shutdown_connections 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1301250148 wait complete. 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 -- start start 2024-08-02T17:59:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.588+0000 7f26e9bd6740 1 Processor -- start 2024-08-02T17:59:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1facda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26e9bd6740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f26e01af700 con 0x558dc1fa4970 2024-08-02T17:59:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f26e01b0b70 con 0x558dc1f997b0 2024-08-02T17:59:26.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26e9bd6740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f26e01afdd0 con 0x558dc1f9a0d0 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26e9bd6740 10 client.0 _ll_register_callbacks cb 0x7f26cc0319d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47392/0 (socket says 172.21.15.69:47392) 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 -- 172.21.15.69:0/1626504475 learned_addr learned my addr 172.21.15.69:0/1626504475 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26e4c23640 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 -- 172.21.15.69:0/1626504475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 msgr2=0x558dc1fa80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 -- 172.21.15.69:0/1626504475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 msgr2=0x558dc1facda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1facda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 -- 172.21.15.69:0/1626504475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc1fa8b40 con 0x558dc1fa4970 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26dffff640 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa8600 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02ab30 tx=0x7f26cc009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e346c7b0eaaeed6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26bffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.589+0000 7f26bffff640 1 -- 172.21.15.69:0/1626504475 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26cc035070 con 0x558dc1fa4970 2024-08-02T17:59:26.596 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26bffff640 1 -- 172.21.15.69:0/1626504475 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f26cc004260 con 0x558dc1fa4970 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1626504475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26d402d030 con 0x558dc1fa4970 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26bffff640 1 -- 172.21.15.69:0/1626504475 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f26cc004460 con 0x558dc1fa4970 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 10 client.18627 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1626504475 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558dc201ef70 con 0x558dc1fa4970 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26bffff640 1 -- 172.21.15.69:0/1626504475 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26d402d030 con 0x558dc1fa4970 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26bffff640 1 -- 172.21.15.69:0/1626504475 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f26cc004990 con 0x558dc1fa4970 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 10 client.18627 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 10 client.18627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 20 client.18627 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558dc201f660 0x558dc2021a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 4 client.18627 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:59:26.597 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1626504475 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558dc1f05430 con 0x558dc201f660 2024-08-02T17:59:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.590+0000 7f26e4c23640 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558dc201f660 0x558dc2021a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.591+0000 7f26e4c23640 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558dc201f660 0x558dc2021a80 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b11fa33063a9c065 server_cookie=764da9697025b697 in_seq=0 out_seq=0 2024-08-02T17:59:26.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.591+0000 7f26bffff640 10 client.18627 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26bffff640 1 -- 172.21.15.69:0/1626504475 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558dc1f05430 con 0x558dc201f660 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26bffff640 10 client.18627 handle_command_reply: tid=0 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 client.18627 shutdown 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1626504475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 msgr2=0x558dc1fa8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa8600 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02ab30 tx=0x7f26cc009830 comp rx=0 tx=0).stop 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1626504475 shutdown_connections 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558dc1fa4970 0x558dc1fa8600 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x558dc201f660 0x558dc2021a80 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558dc1f9a0d0 0x558dc1fa80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26bffff640 0 client.18627 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 --2- 172.21.15.69:0/1626504475 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558dc1f997b0 0x558dc1facda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.659+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1626504475 >> 172.21.15.69:0/1626504475 conn(0x558dc1f95d20 msgr2=0x558dc1faf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1626504475 shutdown_connections 2024-08-02T17:59:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e9bd6740 1 -- 172.21.15.69:0/1626504475 wait complete. 2024-08-02T17:59:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e9bd6740 20 client.18627 trim_cache size 0 max 16384 2024-08-02T17:59:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:59:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:59:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:59:26.664 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 -- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f26d00a2d70 msgr2=0x7f26d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f26d00a2d70 0x7f26d00a5230 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f26d8018000 tx=0x7f26d80054d0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 -- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01066e0 msgr2=0x7f26e00fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01066e0 0x7f26e00fd590 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f26d800eaa0 tx=0x7f26d800ef60 comp rx=0 tx=0).stop 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 -- 172.21.15.69:0/1089474542 shutdown_connections 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f26e01066e0 0x7f26e00fd590 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f26d00a2d70 0x7f26d00a5230 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f26e0101070 0x7f26e00fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 --2- 172.21.15.69:0/1089474542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f26e01006d0 0x7f26e00fcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 -- 172.21.15.69:0/1089474542 >> 172.21.15.69:0/1089474542 conn(0x7f26e00776d0 msgr2=0x7f26e0069a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:26.665 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 -- 172.21.15.69:0/1089474542 shutdown_connections 2024-08-02T17:59:26.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:26.660+0000 7f26e6e87640 1 -- 172.21.15.69:0/1089474542 wait complete. 2024-08-02T17:59:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:26 smithi083 ceph-mon[37031]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.0 MiB/s wr, 390 op/s 2024-08-02T17:59:27.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:26 smithi077 ceph-mon[37108]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.0 MiB/s wr, 390 op/s 2024-08-02T17:59:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:26 smithi069 ceph-mon[32746]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 4.0 MiB/s wr, 390 op/s 2024-08-02T17:59:27.295 INFO:tasks.cephfs.filesystem:scrub status for tag:0fffa170-9e53-4e3c-8008-708b02f1f600 - {'path': '/', 'tag': '0fffa170-9e53-4e3c-8008-708b02f1f600', 'options': 'recursive,force'} 2024-08-02T17:59:29.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:28 smithi083 ceph-mon[37031]: pgmap v408: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-08-02T17:59:29.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:28 smithi077 ceph-mon[37108]: pgmap v408: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-08-02T17:59:29.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:28 smithi069 ceph-mon[32746]: pgmap v408: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 3.6 MiB/s wr, 359 op/s 2024-08-02T17:59:29.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 740.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:59:30.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:29 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T17:59:30.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:29 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T17:59:30.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:29 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T17:59:31.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:30 smithi083 ceph-mon[37031]: pgmap v409: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.4 MiB/s wr, 419 op/s 2024-08-02T17:59:31.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:30 smithi077 ceph-mon[37108]: pgmap v409: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.4 MiB/s wr, 419 op/s 2024-08-02T17:59:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:30 smithi069 ceph-mon[32746]: pgmap v409: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 3.4 MiB/s wr, 419 op/s 2024-08-02T17:59:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:32 smithi083 ceph-mon[37031]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 608 KiB/s wr, 169 op/s 2024-08-02T17:59:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:32 smithi077 ceph-mon[37108]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 608 KiB/s wr, 169 op/s 2024-08-02T17:59:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:32 smithi069 ceph-mon[32746]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 608 KiB/s wr, 169 op/s 2024-08-02T17:59:34.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 745.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:59:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:34 smithi083 ceph-mon[37031]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 637 KiB/s wr, 172 op/s 2024-08-02T17:59:35.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:34 smithi077 ceph-mon[37108]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 637 KiB/s wr, 172 op/s 2024-08-02T17:59:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:34 smithi069 ceph-mon[32746]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 637 KiB/s wr, 172 op/s 2024-08-02T17:59:37.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:36 smithi083 ceph-mon[37031]: pgmap v412: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 53 KiB/s wr, 111 op/s 2024-08-02T17:59:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:36 smithi077 ceph-mon[37108]: pgmap v412: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 53 KiB/s wr, 111 op/s 2024-08-02T17:59:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:36 smithi069 ceph-mon[32746]: pgmap v412: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 53 KiB/s wr, 111 op/s 2024-08-02T17:59:39.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:38 smithi083 ceph-mon[37031]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 53 KiB/s wr, 111 op/s 2024-08-02T17:59:39.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:38 smithi077 ceph-mon[37108]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 53 KiB/s wr, 111 op/s 2024-08-02T17:59:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:38 smithi069 ceph-mon[32746]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 53 KiB/s wr, 111 op/s 2024-08-02T17:59:39.762 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 -- 172.21.15.69:0/4153305040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f0107800 msgr2=0x7f64f0107be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/4153305040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f0107800 0x7f64f0107be0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009930 tx=0x7f64e002a420 comp rx=0 tx=0).stop 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 -- 172.21.15.69:0/4153305040 shutdown_connections 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/4153305040 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64f0102190 0x7f64f010e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/4153305040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64f01017f0 0x7f64f0101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/4153305040 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f0107800 0x7f64f0107be0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 -- 172.21.15.69:0/4153305040 >> 172.21.15.69:0/4153305040 conn(0x7f64f00fd510 msgr2=0x7f64f00ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 -- 172.21.15.69:0/4153305040 shutdown_connections 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.760+0000 7f64f4f1b640 1 -- 172.21.15.69:0/4153305040 wait complete. 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.761+0000 7f64f4f1b640 1 Processor -- start 2024-08-02T17:59:39.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.761+0000 7f64f4f1b640 1 -- start start 2024-08-02T17:59:39.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64f4f1b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f01017f0 0x7f64f0074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64f4f1b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64f0102190 0x7f64f0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64f4f1b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64f0107800 0x7f64f0074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64f4f1b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0111c40 con 0x7f64f01017f0 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64f4f1b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0111ac0 con 0x7f64f0107800 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64f4f1b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64f0111dc0 con 0x7f64f0102190 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f01017f0 0x7f64f0074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f01017f0 0x7f64f0074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46472/0 (socket says 172.21.15.69:46472) 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 -- 172.21.15.69:0/3937991547 learned_addr learned my addr 172.21.15.69:0/3937991547 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64eed9d640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64f0107800 0x7f64f0074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64edd9b640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64f0102190 0x7f64f0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 -- 172.21.15.69:0/3937991547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64f0102190 msgr2=0x7f64f0074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.766 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64f0102190 0x7f64f0074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 -- 172.21.15.69:0/3937991547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64f0107800 msgr2=0x7f64f0074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64f0107800 0x7f64f0074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 -- 172.21.15.69:0/3937991547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f01a6650 con 0x7f64f01017f0 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64edd9b640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64f0102190 0x7f64f0074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64ee59c640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f01017f0 0x7f64f0074220 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009230 tx=0x7f64e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64df7fe640 1 -- 172.21.15.69:0/3937991547 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e003f030 con 0x7f64f01017f0 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64df7fe640 1 -- 172.21.15.69:0/3937991547 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e0003ad0 con 0x7f64f01017f0 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64df7fe640 1 -- 172.21.15.69:0/3937991547 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0035070 con 0x7f64f01017f0 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64f4f1b640 1 -- 172.21.15.69:0/3937991547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f01a68e0 con 0x7f64f01017f0 2024-08-02T17:59:39.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.762+0000 7f64f4f1b640 1 -- 172.21.15.69:0/3937991547 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f01a6dd0 con 0x7f64f01017f0 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64df7fe640 1 -- 172.21.15.69:0/3937991547 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f64e0042030 con 0x7f64f01017f0 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64f7c7a740 1 Processor -- start 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64f7c7a740 1 -- start start 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dd6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97dd70d0 0x563a97de1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97de1970 0x563a97de3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563a97d3a9b0 con 0x563a97dd67b0 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563a97ce2800 con 0x563a97dd70d0 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.765+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a97d36970 con 0x563a97de1970 2024-08-02T17:59:39.774 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64df7fe640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f64d80a2cf0 0x7f64d80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dd6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dd6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46486/0 (socket says 172.21.15.69:46486) 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 -- 172.21.15.69:0/668140432 learned_addr learned my addr 172.21.15.69:0/668140432 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64edd9b640 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97de1970 0x563a97de3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f64d80a2cf0 0x7f64d80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64df7fe640 1 -- 172.21.15.69:0/3937991547 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f64e00efd50 con 0x7f64f01017f0 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64ee59c640 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97dd70d0 0x563a97de1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 -- 172.21.15.69:0/668140432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97de1970 msgr2=0x563a97de3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97de1970 0x563a97de3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 -- 172.21.15.69:0/668140432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97dd70d0 msgr2=0x563a97de1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97dd70d0 0x563a97de1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.775 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 -- 172.21.15.69:0/668140432 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a97a4e210 con 0x563a97dd67b0 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64edd9b640 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97de1970 0x563a97de3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dd6b90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f64e8012d00 tx=0x7f64e8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfc3e5aaad60972 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64eed9d640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f64d80a2cf0 0x7f64d80a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64e8010c40 tx=0x7f64e800c000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.766+0000 7f64dcff9640 1 -- 172.21.15.69:0/668140432 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8024070 con 0x563a97dd67b0 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64dcff9640 1 -- 172.21.15.69:0/668140432 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e8009d20 con 0x563a97dd67b0 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64dcff9640 1 -- 172.21.15.69:0/668140432 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8005340 con 0x563a97dd67b0 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 -- 172.21.15.69:0/668140432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 msgr2=0x563a97dd6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dd6b90 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f64e8012d00 tx=0x7f64e8010740 comp rx=0 tx=0).stop 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 -- 172.21.15.69:0/668140432 shutdown_connections 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97de1970 0x563a97de3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97dd70d0 0x563a97de1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/668140432 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dd6b90 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 -- 172.21.15.69:0/668140432 >> 172.21.15.69:0/668140432 conn(0x563a97dd2d20 msgr2=0x563a97dd3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:39.776 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 -- 172.21.15.69:0/668140432 shutdown_connections 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.767+0000 7f64f7c7a740 1 -- 172.21.15.69:0/668140432 wait complete. 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 1 -- start start 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 1 Processor -- start 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dde950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 0x563a97ddee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97ddf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563a97de4540 con 0x563a97dd67b0 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563a97cf6500 con 0x563a97de1970 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a97de46c0 con 0x563a97dd70d0 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64f7c7a740 10 client.0 _ll_register_callbacks cb 0x563a97a4da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64ee59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dde950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64ee59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dde950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46496/0 (socket says 172.21.15.69:46496) 2024-08-02T17:59:39.777 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64ee59c640 1 -- 172.21.15.69:0/1224107564 learned_addr learned my addr 172.21.15.69:0/1224107564 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64ee59c640 1 -- 172.21.15.69:0/1224107564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 msgr2=0x563a97ddee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64ee59c640 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 0x563a97ddee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64ee59c640 1 -- 172.21.15.69:0/1224107564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 msgr2=0x563a97ddf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64ee59c640 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97ddf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.768+0000 7f64ee59c640 1 -- 172.21.15.69:0/1224107564 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a97b1a620 con 0x563a97dd67b0 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64ee59c640 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dde950 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f64e00bbb60 tx=0x7f64e00045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5dc36c5a1cc8a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64d5ffb640 1 -- 172.21.15.69:0/1224107564 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0049050 con 0x563a97dd67b0 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64d5ffb640 1 -- 172.21.15.69:0/1224107564 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f64e00b84d0 con 0x563a97dd67b0 2024-08-02T17:59:39.778 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1224107564 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a97d480e0 con 0x563a97dd67b0 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64d5ffb640 1 -- 172.21.15.69:0/1224107564 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64e00c0210 con 0x563a97dd67b0 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 10 client.18639 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1224107564 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a97df0290 con 0x563a97dd67b0 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64d5ffb640 1 -- 172.21.15.69:0/1224107564 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563a97d480e0 con 0x563a97dd67b0 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64d5ffb640 1 -- 172.21.15.69:0/1224107564 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f64e00bc150 con 0x563a97dd67b0 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 10 client.18639 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 10 client.18639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 20 client.18639 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563a97df15a0 0x563a97df3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 4 client.18639 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.769+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1224107564 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a97d42430 con 0x563a97df15a0 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.770+0000 7f64edd9b640 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563a97df15a0 0x563a97df3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.779 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.770+0000 7f64edd9b640 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563a97df15a0 0x563a97df3a40 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5730e830baf59133 server_cookie=eb412e5f5b96098e in_seq=0 out_seq=0 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.770+0000 7f64d5ffb640 10 client.18639 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64d5ffb640 1 -- 172.21.15.69:0/1224107564 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a97d42430 con 0x563a97df15a0 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64d5ffb640 10 client.18639 handle_command_reply: tid=0 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 client.18639 shutdown 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1224107564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 msgr2=0x563a97dde950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dde950 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f64e00bbb60 tx=0x7f64e00045f0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1224107564 shutdown_connections 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563a97df15a0 0x563a97df3a40 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97ddf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64d5ffb640 0 client.18639 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 0x563a97ddee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1224107564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dde950 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.780 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1224107564 >> 172.21.15.69:0/1224107564 conn(0x563a97dd2d20 msgr2=0x563a97dd8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1224107564 shutdown_connections 2024-08-02T17:59:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1224107564 wait complete. 2024-08-02T17:59:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.771+0000 7f64f7c7a740 20 client.18639 trim_cache size 0 max 16384 2024-08-02T17:59:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64f7c7a740 1 Processor -- start 2024-08-02T17:59:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64f7c7a740 1 -- start start 2024-08-02T17:59:39.792 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd67b0 0x563a97de07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd70d0 0x563a97df9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97df9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01b3930 con 0x563a97dd70d0 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01b1200 con 0x563a97de1970 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01af050 con 0x563a97dd67b0 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd70d0 0x563a97df9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd70d0 0x563a97df9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46504/0 (socket says 172.21.15.69:46504) 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 -- 172.21.15.69:0/1067620124 learned_addr learned my addr 172.21.15.69:0/1067620124 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64ee59c640 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd67b0 0x563a97de07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 -- 172.21.15.69:0/1067620124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd67b0 msgr2=0x563a97de07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd67b0 0x563a97de07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.793 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 -- 172.21.15.69:0/1067620124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 msgr2=0x563a97df9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64eed9d640 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97df9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97df9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 -- 172.21.15.69:0/1067620124 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a97dd65d0 con 0x563a97dd70d0 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64ee59c640 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd67b0 0x563a97de07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64eed9d640 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97df9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64edd9b640 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd70d0 0x563a97df9350 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f64e4000c00 tx=0x7f64e4009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4481d4cc3d6d33d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.787+0000 7f64d7fff640 1 -- 172.21.15.69:0/1067620124 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004220 con 0x563a97dd70d0 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64d7fff640 1 -- 172.21.15.69:0/1067620124 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e40043c0 con 0x563a97dd70d0 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64d7fff640 1 -- 172.21.15.69:0/1067620124 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004e50 con 0x563a97dd70d0 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1067620124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd70d0 msgr2=0x563a97df9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd70d0 0x563a97df9350 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f64e4000c00 tx=0x7f64e4009830 comp rx=0 tx=0).stop 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1067620124 shutdown_connections 2024-08-02T17:59:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd70d0 0x563a97df9350 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97df9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/1067620124 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd67b0 0x563a97de07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1067620124 >> 172.21.15.69:0/1067620124 conn(0x563a97dd2d20 msgr2=0x563a97de00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1067620124 shutdown_connections 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 -- 172.21.15.69:0/1067620124 wait complete. 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 -- start start 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.788+0000 7f64f7c7a740 1 Processor -- start 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dea050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 0x563a97de5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64f7c7a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97de58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01af6c0 con 0x563a97dd67b0 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01b0b30 con 0x563a97de1970 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64f7c7a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f64f01afd90 con 0x563a97dd70d0 2024-08-02T17:59:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64f7c7a740 10 client.0 _ll_register_callbacks cb 0x7f64e00bb9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dea050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dea050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46512/0 (socket says 172.21.15.69:46512) 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 -- 172.21.15.69:0/474310644 learned_addr learned my addr 172.21.15.69:0/474310644 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64edd9b640 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 0x563a97de5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64eed9d640 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97de58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 -- 172.21.15.69:0/474310644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 msgr2=0x563a97de5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 0x563a97de5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 -- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 msgr2=0x563a97de58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97de58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 -- 172.21.15.69:0/474310644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a97de6090 con 0x563a97dd67b0 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64eed9d640 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97de58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64ee59c640 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dea050 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f64e00c03d0 tx=0x7f64e002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4512450250994606 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:39.796 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64d5ffb640 1 -- 172.21.15.69:0/474310644 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0049050 con 0x563a97dd67b0 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64d5ffb640 1 -- 172.21.15.69:0/474310644 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f64e00c4780 con 0x563a97dd67b0 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.789+0000 7f64f7c7a740 1 -- 172.21.15.69:0/474310644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e40043c0 con 0x563a97dd67b0 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64d5ffb640 1 -- 172.21.15.69:0/474310644 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f64e00b8100 con 0x563a97dd67b0 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64f7c7a740 10 client.18645 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64f7c7a740 1 -- 172.21.15.69:0/474310644 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a97de6090 con 0x563a97dd67b0 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64d5ffb640 1 -- 172.21.15.69:0/474310644 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64e40043c0 con 0x563a97dd67b0 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64d5ffb640 1 -- 172.21.15.69:0/474310644 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f64e00c9070 con 0x563a97dd67b0 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64f7c7a740 10 client.18645 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64f7c7a740 10 client.18645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64f7c7a740 20 client.18645 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563a97e5c5c0 0x563a97e5e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64f7c7a740 4 client.18645 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:59:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64f7c7a740 1 -- 172.21.15.69:0/474310644 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a97cbf8e0 con 0x563a97e5c5c0 2024-08-02T17:59:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.790+0000 7f64edd9b640 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563a97e5c5c0 0x563a97e5e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.791+0000 7f64d5ffb640 10 client.18645 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.791+0000 7f64edd9b640 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563a97e5c5c0 0x563a97e5e9b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481432fc2bef0160 server_cookie=1eec32a3ce32bc6 in_seq=0 out_seq=0 2024-08-02T17:59:39.806 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64d5ffb640 1 -- 172.21.15.69:0/474310644 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a97cbf8e0 con 0x563a97e5c5c0 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64d5ffb640 10 client.18645 handle_command_reply: tid=0 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 client.18645 shutdown 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 -- 172.21.15.69:0/474310644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 msgr2=0x563a97dea050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dea050 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f64e00c03d0 tx=0x7f64e002a420 comp rx=0 tx=0).stop 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 -- 172.21.15.69:0/474310644 shutdown_connections 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563a97e5c5c0 0x563a97e5e9b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a97de1970 0x563a97de58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64d5ffb640 0 client.18645 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a97dd70d0 0x563a97de5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 --2- 172.21.15.69:0/474310644 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a97dd67b0 0x563a97dea050 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 -- 172.21.15.69:0/474310644 >> 172.21.15.69:0/474310644 conn(0x563a97dd2d20 msgr2=0x563a97ddf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 -- 172.21.15.69:0/474310644 shutdown_connections 2024-08-02T17:59:39.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 1 -- 172.21.15.69:0/474310644 wait complete. 2024-08-02T17:59:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f7c7a740 20 client.18645 trim_cache size 0 max 16384 2024-08-02T17:59:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:59:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:59:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:59:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f4f1b640 1 -- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f64d80a2cf0 msgr2=0x7f64d80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f64d80a2cf0 0x7f64d80a51b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f64e8010c40 tx=0x7f64e800c000 comp rx=0 tx=0).stop 2024-08-02T17:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.805+0000 7f64f4f1b640 1 -- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f01017f0 msgr2=0x7f64f0074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f01017f0 0x7f64f0074220 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f64e0009230 tx=0x7f64e0004270 comp rx=0 tx=0).stop 2024-08-02T17:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 -- 172.21.15.69:0/3937991547 shutdown_connections 2024-08-02T17:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f64d80a2cf0 0x7f64d80a51b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f64f0107800 0x7f64f0074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f64f0102190 0x7f64f0074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 --2- 172.21.15.69:0/3937991547 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f64f01017f0 0x7f64f0074220 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 -- 172.21.15.69:0/3937991547 >> 172.21.15.69:0/3937991547 conn(0x7f64f00fd510 msgr2=0x7f64f00fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 -- 172.21.15.69:0/3937991547 shutdown_connections 2024-08-02T17:59:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:39.806+0000 7f64f4f1b640 1 -- 172.21.15.69:0/3937991547 wait complete. 2024-08-02T17:59:39.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 750.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:59:41.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:40 smithi083 ceph-mon[37031]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 86 KiB/s wr, 100 op/s 2024-08-02T17:59:41.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:40 smithi077 ceph-mon[37108]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 86 KiB/s wr, 100 op/s 2024-08-02T17:59:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:40 smithi069 ceph-mon[32746]: pgmap v414: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 86 KiB/s wr, 100 op/s 2024-08-02T17:59:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:42 smithi083 ceph-mon[37031]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 6 op/s 2024-08-02T17:59:43.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:42 smithi077 ceph-mon[37108]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 6 op/s 2024-08-02T17:59:43.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:42 smithi069 ceph-mon[32746]: pgmap v415: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 68 KiB/s wr, 6 op/s 2024-08-02T17:59:43.160 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:59:43.428 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:43.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 -- 172.21.15.69:0/1000566894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c98100b50 msgr2=0x7f9c98100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:43.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 --2- 172.21.15.69:0/1000566894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c98100b50 0x7f9c98100f30 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009900 tx=0x7f9c8802a480 comp rx=0 tx=0).stop 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 -- 172.21.15.69:0/1000566894 shutdown_connections 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 --2- 172.21.15.69:0/1000566894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c98102c20 0x7f9c9810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 --2- 172.21.15.69:0/1000566894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c981022a0 0x7f9c981026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 --2- 172.21.15.69:0/1000566894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c98100b50 0x7f9c98100f30 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 -- 172.21.15.69:0/1000566894 >> 172.21.15.69:0/1000566894 conn(0x7f9c980fc500 msgr2=0x7f9c980fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 -- 172.21.15.69:0/1000566894 shutdown_connections 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 -- 172.21.15.69:0/1000566894 wait complete. 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 Processor -- start 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.916+0000 7f9c9d776640 1 -- start start 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c9d776640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c98100b50 0x7f9c981990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:43.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c9d776640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c981022a0 0x7f9c981995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c9d776640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c98102c20 0x7f9c981a4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c9d776640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98110cd0 con 0x7f9c981022a0 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c9d776640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98110b50 con 0x7f9c98100b50 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c9d776640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98110e50 con 0x7f9c98102c20 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c981022a0 0x7f9c981995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c981022a0 0x7f9c981995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46538/0 (socket says 172.21.15.69:46538) 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 -- 172.21.15.69:0/3308256650 learned_addr learned my addr 172.21.15.69:0/3308256650 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c96ffd640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c98100b50 0x7f9c981990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c977fe640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c98102c20 0x7f9c981a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 -- 172.21.15.69:0/3308256650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c98102c20 msgr2=0x7f9c981a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c98102c20 0x7f9c981a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 -- 172.21.15.69:0/3308256650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c98100b50 msgr2=0x7f9c981990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:43.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c98100b50 0x7f9c981990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 -- 172.21.15.69:0/3308256650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c981a4660 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c967fc640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c981022a0 0x7f9c981995e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c890 tx=0x7f9c8000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c8ffff640 1 -- 172.21.15.69:0/3308256650 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80004220 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c8ffff640 1 -- 172.21.15.69:0/3308256650 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c800043c0 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c981a4830 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c8ffff640 1 -- 172.21.15.69:0/3308256650 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c800049e0 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.917+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c981a4d70 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.918+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c98100f30 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.920+0000 7f9c8ffff640 1 -- 172.21.15.69:0/3308256650 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9c800079b0 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.921+0000 7f9c8ffff640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c840a2cf0 0x7f9c840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.921+0000 7f9c96ffd640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c840a2cf0 0x7f9c840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.921+0000 7f9c8ffff640 1 -- 172.21.15.69:0/3308256650 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f9c800cf9a0 con 0x7f9c981022a0 2024-08-02T17:59:43.933 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.921+0000 7f9c96ffd640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c840a2cf0 0x7f9c840a51b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c88005ab0 tx=0x7f9c88005a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:43.934 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:43.924+0000 7f9c8ffff640 1 -- 172.21.15.69:0/3308256650 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9c80014030 con 0x7f9c981022a0 2024-08-02T17:59:44.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.125+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c98108dc0 con 0x7f9c981022a0 2024-08-02T17:59:44.127 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.125+0000 7f9c8ffff640 1 -- 172.21.15.69:0/3308256650 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f9c8009c1b0 con 0x7f9c981022a0 2024-08-02T17:59:44.127 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:59:44.127 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:59:44.128 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:59:44.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.127+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c840a2cf0 msgr2=0x7f9c840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:44.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.127+0000 7f9c9d776640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c840a2cf0 0x7f9c840a51b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c88005ab0 tx=0x7f9c88005a20 comp rx=0 tx=0).stop 2024-08-02T17:59:44.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c981022a0 msgr2=0x7f9c981995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:44.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c981022a0 0x7f9c981995e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c890 tx=0x7f9c8000cd50 comp rx=0 tx=0).stop 2024-08-02T17:59:44.129 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 shutdown_connections 2024-08-02T17:59:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9c981022a0 0x7f9c981995e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9c840a2cf0 0x7f9c840a51b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9c98102c20 0x7f9c981a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 --2- 172.21.15.69:0/3308256650 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9c98100b50 0x7f9c981990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 >> 172.21.15.69:0/3308256650 conn(0x7f9c980fc500 msgr2=0x7f9c980fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 shutdown_connections 2024-08-02T17:59:44.130 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:44.128+0000 7f9c9d776640 1 -- 172.21.15.69:0/3308256650 wait complete. 2024-08-02T17:59:44.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:44 smithi069 ceph-mon[32746]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 99 KiB/s wr, 16 op/s 2024-08-02T17:59:44.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:44 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3308256650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:44.863 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 50178048 --release --await' 2024-08-02T17:59:44.863 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 50178048 --release --await 2024-08-02T17:59:44.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 755.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T17:59:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:44 smithi083 ceph-mon[37031]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 99 KiB/s wr, 16 op/s 2024-08-02T17:59:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:44 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3308256650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:45.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:44 smithi077 ceph-mon[37108]: pgmap v416: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 99 KiB/s wr, 16 op/s 2024-08-02T17:59:45.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:44 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3308256650' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:45.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.564+0000 7fe9834dd640 1 -- 172.21.15.69:0/3871725355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 msgr2=0x7fe97c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.564+0000 7fe9834dd640 1 --2- 172.21.15.69:0/3871725355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 0x7fe97c1026e0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009900 tx=0x7fe96c02a480 comp rx=0 tx=0).stop 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.564+0000 7fe9834dd640 1 -- 172.21.15.69:0/3871725355 shutdown_connections 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.564+0000 7fe9834dd640 1 --2- 172.21.15.69:0/3871725355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 0x7fe97c1026e0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.564+0000 7fe9834dd640 1 --2- 172.21.15.69:0/3871725355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe97c102c20 0x7fe97c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.564+0000 7fe9834dd640 1 --2- 172.21.15.69:0/3871725355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c100b50 0x7fe97c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.564+0000 7fe9834dd640 1 -- 172.21.15.69:0/3871725355 >> 172.21.15.69:0/3871725355 conn(0x7fe97c0fc500 msgr2=0x7fe97c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 -- 172.21.15.69:0/3871725355 shutdown_connections 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 -- 172.21.15.69:0/3871725355 wait complete. 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 Processor -- start 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 -- start start 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe97c100b50 0x7fe97c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 0x7fe97c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c102c20 0x7fe97c19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c110cb0 con 0x7fe97c1022a0 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c110b30 con 0x7fe97c100b50 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.565+0000 7fe9834dd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c110e30 con 0x7fe97c102c20 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 0x7fe97c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 0x7fe97c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52194/0 (socket says 172.21.15.69:52194) 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 -- 172.21.15.69:0/56300893 learned_addr learned my addr 172.21.15.69:0/56300893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe981279640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe97c100b50 0x7fe97c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe981a7a640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c102c20 0x7fe97c19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 -- 172.21.15.69:0/56300893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c102c20 msgr2=0x7fe97c19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c102c20 0x7fe97c19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 -- 172.21.15.69:0/56300893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe97c100b50 msgr2=0x7fe97c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe97c100b50 0x7fe97c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 -- 172.21.15.69:0/56300893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97c19d210 con 0x7fe97c1022a0 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe981a7a640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c102c20 0x7fe97c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe980a78640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 0x7fe97c1987a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe96c006010 tx=0x7fe96c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe9727fc640 1 -- 172.21.15.69:0/56300893 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c03f070 con 0x7fe97c1022a0 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe9727fc640 1 -- 172.21.15.69:0/56300893 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe96c031c30 con 0x7fe97c1022a0 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe9834dd640 1 -- 172.21.15.69:0/56300893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe97c19d4a0 con 0x7fe97c1022a0 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe9727fc640 1 -- 172.21.15.69:0/56300893 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c035030 con 0x7fe97c1022a0 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.566+0000 7fe9834dd640 1 -- 172.21.15.69:0/56300893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe97c1a4de0 con 0x7fe97c1022a0 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9727fc640 1 -- 172.21.15.69:0/56300893 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe96c042070 con 0x7fe97c1022a0 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9858db740 1 Processor -- start 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9858db740 1 -- start start 2024-08-02T17:59:45.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9858db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e030d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9858db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 0x55704e03b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9858db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e03df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9858db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55704df96c60 con 0x55704e030990 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9858db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55704df2eb90 con 0x55704e03bb70 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.569+0000 7fe9858db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55704df92c20 con 0x55704e0312b0 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e030d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e030d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52210/0 (socket says 172.21.15.69:52210) 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 -- 172.21.15.69:0/759588044 learned_addr learned my addr 172.21.15.69:0/759588044 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981279640 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 0x55704e03b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe980a78640 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e03df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.576 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe9727fc640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9680a2d70 0x7fe9680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 -- 172.21.15.69:0/759588044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 msgr2=0x55704e03b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 0x55704e03b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 -- 172.21.15.69:0/759588044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 msgr2=0x55704e03df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e03df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 -- 172.21.15.69:0/759588044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704dca6580 con 0x55704e030990 2024-08-02T17:59:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe980a78640 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e03df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe9727fc640 1 -- 172.21.15.69:0/56300893 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fe96c0f0040 con 0x7fe97c1022a0 2024-08-02T17:59:45.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9680a2d70 0x7fe9680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.570+0000 7fe981a7a640 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e030d70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe974009e40 tx=0x7fe97400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469532683fed190f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9677fe640 1 -- 172.21.15.69:0/759588044 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974021070 con 0x55704e030990 2024-08-02T17:59:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9677fe640 1 -- 172.21.15.69:0/759588044 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97401bde0 con 0x55704e030990 2024-08-02T17:59:45.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9677fe640 1 -- 172.21.15.69:0/759588044 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974010b20 con 0x55704e030990 2024-08-02T17:59:45.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9858db740 1 -- 172.21.15.69:0/759588044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 msgr2=0x55704e030d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe981a7a640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9680a2d70 0x7fe9680a5230 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe97401e000 tx=0x7fe974021000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:45.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9858db740 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e030d70 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fe974009e40 tx=0x7fe97400f730 comp rx=0 tx=0).stop 2024-08-02T17:59:45.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9858db740 1 -- 172.21.15.69:0/759588044 shutdown_connections 2024-08-02T17:59:45.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9858db740 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e03df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9858db740 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 0x55704e03b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9858db740 1 --2- 172.21.15.69:0/759588044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e030d70 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.571+0000 7fe9858db740 1 -- 172.21.15.69:0/759588044 >> 172.21.15.69:0/759588044 conn(0x55704e02cf20 msgr2=0x55704e02d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 -- 172.21.15.69:0/759588044 shutdown_connections 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 -- 172.21.15.69:0/759588044 wait complete. 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 -- start start 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 Processor -- start 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 0x55704e038a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e0312b0 0x55704e038fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e03bb70 0x55704e0394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55704e03e740 con 0x55704e03bb70 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55704df347c0 con 0x55704e0312b0 2024-08-02T17:59:45.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55704e03e8c0 con 0x55704e030990 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.572+0000 7fe9858db740 10 client.0 _ll_register_callbacks cb 0x55704df3df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e03bb70 0x55704e0394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e03bb70 0x55704e0394f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52222/0 (socket says 172.21.15.69:52222) 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 -- 172.21.15.69:0/308465774 learned_addr learned my addr 172.21.15.69:0/308465774 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981279640 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 0x55704e038a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe980a78640 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e0312b0 0x55704e038fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 -- 172.21.15.69:0/308465774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 msgr2=0x55704e038a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 0x55704e038a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 -- 172.21.15.69:0/308465774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e0312b0 msgr2=0x55704e038fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e0312b0 0x55704e038fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 -- 172.21.15.69:0/308465774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704dd70960 con 0x55704e03bb70 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe981a7a640 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e03bb70 0x55704e0394f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe974010880 tx=0x7fe974011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76e117378c282f1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:45.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe964ff9640 1 -- 172.21.15.69:0/308465774 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974033060 con 0x55704e03bb70 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe964ff9640 1 -- 172.21.15.69:0/308465774 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fe97401b880 con 0x55704e03bb70 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.573+0000 7fe9858db740 1 -- 172.21.15.69:0/308465774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe97401bde0 con 0x55704e03bb70 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe964ff9640 1 -- 172.21.15.69:0/308465774 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe97403a050 con 0x55704e03bb70 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe9858db740 10 client.18663 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe9858db740 1 -- 172.21.15.69:0/308465774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55704e04a750 con 0x55704e03bb70 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe964ff9640 1 -- 172.21.15.69:0/308465774 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe97401bde0 con 0x55704e03bb70 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe964ff9640 1 -- 172.21.15.69:0/308465774 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe97402d030 con 0x55704e03bb70 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe9858db740 10 client.18663 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe9858db740 10 client.18663 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe9858db740 20 client.18663 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:45.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe9858db740 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55704e04bc10 0x55704e04e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe9858db740 4 client.18663 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe9858db740 1 -- 172.21.15.69:0/308465774 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55704dfa41d0 con 0x55704e04bc10 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.574+0000 7fe981279640 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55704e04bc10 0x55704e04e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.575+0000 7fe981279640 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55704e04bc10 0x55704e04e000 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a869756d25957272 server_cookie=8eec94eae099ab14 in_seq=0 out_seq=0 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.575+0000 7fe964ff9640 10 client.18663 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.575+0000 7fe964ff9640 1 -- 172.21.15.69:0/308465774 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55704dfa41d0 con 0x55704e04bc10 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.575+0000 7fe964ff9640 10 client.18663 handle_command_reply: tid=0 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.575+0000 7fe9858db740 1 client.18663 shutdown 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.575+0000 7fe9858db740 1 -- 172.21.15.69:0/308465774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e03bb70 msgr2=0x55704e0394f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.575+0000 7fe9858db740 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e03bb70 0x55704e0394f0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fe974010880 tx=0x7fe974011580 comp rx=0 tx=0).stop 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 1 -- 172.21.15.69:0/308465774 shutdown_connections 2024-08-02T17:59:45.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e03bb70 0x55704e0394f0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55704e04bc10 0x55704e04e000 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e0312b0 0x55704e038fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 1 --2- 172.21.15.69:0/308465774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 0x55704e038a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 1 -- 172.21.15.69:0/308465774 >> 172.21.15.69:0/308465774 conn(0x55704e02cf20 msgr2=0x55704e032c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe964ff9640 0 client.18663 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 1 -- 172.21.15.69:0/308465774 shutdown_connections 2024-08-02T17:59:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 1 -- 172.21.15.69:0/308465774 wait complete. 2024-08-02T17:59:45.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.576+0000 7fe9858db740 20 client.18663 trim_cache size 0 max 16384 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe9858db740 1 Processor -- start 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe9858db740 1 -- start start 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe9858db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e0333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe9858db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 0x55704e043ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe9858db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e044430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe9858db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c1b1710 con 0x55704e030990 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe9858db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c1aefe0 con 0x55704e03bb70 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe9858db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c1ace30 con 0x55704e0312b0 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e0333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e0333f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52236/0 (socket says 172.21.15.69:52236) 2024-08-02T17:59:45.596 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 -- 172.21.15.69:0/4280701743 learned_addr learned my addr 172.21.15.69:0/4280701743 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe980a78640 1 --2- 172.21.15.69:0/4280701743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 0x55704e043ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 -- 172.21.15.69:0/4280701743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 msgr2=0x55704e043ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 --2- 172.21.15.69:0/4280701743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 0x55704e043ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 -- 172.21.15.69:0/4280701743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 msgr2=0x55704e044430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 --2- 172.21.15.69:0/4280701743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e044430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 -- 172.21.15.69:0/4280701743 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55704e0479a0 con 0x55704e030990 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe981279640 1 --2- 172.21.15.69:0/4280701743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e0333f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe978004280 tx=0x7fe97800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ec10c41ad1cac0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe966ffd640 1 -- 172.21.15.69:0/4280701743 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978003d20 con 0x55704e030990 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe966ffd640 1 -- 172.21.15.69:0/4280701743 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978003ec0 con 0x55704e030990 2024-08-02T17:59:45.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.594+0000 7fe966ffd640 1 -- 172.21.15.69:0/4280701743 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978011aa0 con 0x55704e030990 2024-08-02T17:59:45.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 -- 172.21.15.69:0/4280701743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 msgr2=0x55704e0333f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 --2- 172.21.15.69:0/4280701743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e0333f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fe978004280 tx=0x7fe97800bcb0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.600 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 -- 172.21.15.69:0/4280701743 shutdown_connections 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 --2- 172.21.15.69:0/4280701743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e044430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 --2- 172.21.15.69:0/4280701743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e0312b0 0x55704e043ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 --2- 172.21.15.69:0/4280701743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e030990 0x55704e0333f0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 -- 172.21.15.69:0/4280701743 >> 172.21.15.69:0/4280701743 conn(0x55704e02cf20 msgr2=0x55704e037b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 -- 172.21.15.69:0/4280701743 shutdown_connections 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 -- 172.21.15.69:0/4280701743 wait complete. 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.595+0000 7fe9858db740 1 -- start start 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe9858db740 1 Processor -- start 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe9858db740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 0x55704e036440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe9858db740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e0312b0 0x55704e034fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe9858db740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e0354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe9858db740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c1ad4a0 con 0x55704e0312b0 2024-08-02T17:59:45.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe9858db740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c1ae910 con 0x55704e03bb70 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe9858db740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe97c1adb70 con 0x55704e030990 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe9858db740 10 client.0 _ll_register_callbacks cb 0x7fe97400ff00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe980a78640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e0312b0 0x55704e034fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe980a78640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e0312b0 0x55704e034fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52246/0 (socket says 172.21.15.69:52246) 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe980a78640 1 -- 172.21.15.69:0/3641085898 learned_addr learned my addr 172.21.15.69:0/3641085898 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe981a7a640 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e0354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe981279640 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 0x55704e036440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe980a78640 1 -- 172.21.15.69:0/3641085898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 msgr2=0x55704e036440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe980a78640 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 0x55704e036440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe980a78640 1 -- 172.21.15.69:0/3641085898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 msgr2=0x55704e0354f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe980a78640 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e0354f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.596+0000 7fe980a78640 1 -- 172.21.15.69:0/3641085898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55704e035b90 con 0x55704e0312b0 2024-08-02T17:59:45.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe980a78640 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e0312b0 0x55704e034fb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005ab0 tx=0x7fe96c0bbc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4313297431eec51 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe964ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe964ff9640 1 -- 172.21.15.69:0/3641085898 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c049040 con 0x55704e0312b0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe964ff9640 1 -- 172.21.15.69:0/3641085898 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fe96c0f2110 con 0x55704e0312b0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 1 -- 172.21.15.69:0/3641085898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe978003ec0 con 0x55704e0312b0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe964ff9640 1 -- 172.21.15.69:0/3641085898 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe96c0bc850 con 0x55704e0312b0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 10 client.18669 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 1 -- 172.21.15.69:0/3641085898 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55704e0ba6e0 con 0x55704e0312b0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe964ff9640 1 -- 172.21.15.69:0/3641085898 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe978003ec0 con 0x55704e0312b0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe964ff9640 1 -- 172.21.15.69:0/3641085898 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fe96c0c0120 con 0x55704e0312b0 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 10 client.18669 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 10 client.18669 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 20 client.18669 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:45.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55704e0bae60 0x55704e0bd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:45.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 4 client.18669 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "50178048", "release": true, "await": true}] 2024-08-02T17:59:45.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe9858db740 1 -- 172.21.15.69:0/3641085898 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "50178048", "release": true, "await": true}) -- 0x55704df9e6e0 con 0x55704e0bae60 2024-08-02T17:59:45.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.597+0000 7fe981279640 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55704e0bae60 0x55704e0bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:45.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.598+0000 7fe981279640 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55704e0bae60 0x55704e0bd280 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d34cdf116aaa6c server_cookie=ad1683160d98e518 in_seq=0 out_seq=0 2024-08-02T17:59:45.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.598+0000 7fe964ff9640 10 client.18669 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe964ff9640 1 -- 172.21.15.69:0/3641085898 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55704df9e6e0 con 0x55704e0bae60 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe964ff9640 10 client.18669 handle_command_reply: tid=0 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 client.18669 shutdown 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 -- 172.21.15.69:0/3641085898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e0312b0 msgr2=0x55704e034fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e0312b0 0x55704e034fb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe96c005ab0 tx=0x7fe96c0bbc90 comp rx=0 tx=0).stop 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 -- 172.21.15.69:0/3641085898 shutdown_connections 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55704e0312b0 0x55704e034fb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55704e0bae60 0x55704e0bd280 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55704e03bb70 0x55704e0354f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 --2- 172.21.15.69:0/3641085898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55704e030990 0x55704e036440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe964ff9640 0 client.18669 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:45.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 -- 172.21.15.69:0/3641085898 >> 172.21.15.69:0/3641085898 conn(0x55704e02cf20 msgr2=0x55704e046990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:45.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 -- 172.21.15.69:0/3641085898 shutdown_connections 2024-08-02T17:59:45.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.655+0000 7fe9858db740 1 -- 172.21.15.69:0/3641085898 wait complete. 2024-08-02T17:59:45.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9858db740 20 client.18669 trim_cache size 0 max 16384 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 48, 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "50178048": { 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 48, 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:59:45.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 150.0, 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:45.661 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:59:45.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 -- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9680a2d70 msgr2=0x7fe9680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9680a2d70 0x7fe9680a5230 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe97401e000 tx=0x7fe974021000 comp rx=0 tx=0).stop 2024-08-02T17:59:45.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 -- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 msgr2=0x7fe97c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:45.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 0x7fe97c1987a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fe96c006010 tx=0x7fe96c003c10 comp rx=0 tx=0).stop 2024-08-02T17:59:45.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 -- 172.21.15.69:0/56300893 shutdown_connections 2024-08-02T17:59:45.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe97c1022a0 0x7fe97c1987a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe9680a2d70 0x7fe9680a5230 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.663 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe97c102c20 0x7fe97c19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.656+0000 7fe9834dd640 1 --2- 172.21.15.69:0/56300893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe97c100b50 0x7fe97c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:45.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.657+0000 7fe9834dd640 1 -- 172.21.15.69:0/56300893 >> 172.21.15.69:0/56300893 conn(0x7fe97c0fc500 msgr2=0x7fe97c069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:45.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.657+0000 7fe9834dd640 1 -- 172.21.15.69:0/56300893 shutdown_connections 2024-08-02T17:59:45.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:45.657+0000 7fe9834dd640 1 -- 172.21.15.69:0/56300893 wait complete. 2024-08-02T17:59:46.565 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '50178048', total seconds elapsed: 33.4 2024-08-02T17:59:46.565 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-02T17:59:46.565 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:59:46.860 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:46 smithi069 ceph-mon[32746]: pgmap v417: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 14 op/s 2024-08-02T17:59:47.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:46 smithi083 ceph-mon[37031]: pgmap v417: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 14 op/s 2024-08-02T17:59:47.054 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:47.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:46 smithi077 ceph-mon[37108]: pgmap v417: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 72 KiB/s wr, 14 op/s 2024-08-02T17:59:47.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.644+0000 7f46aca26640 1 -- 172.21.15.69:0/1519431906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a8100fc0 msgr2=0x7f46a810d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.644+0000 7f46aca26640 1 --2- 172.21.15.69:0/1519431906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a8100fc0 0x7f46a810d4d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f469c01a940 tx=0x7f469c03b490 comp rx=0 tx=0).stop 2024-08-02T17:59:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.644+0000 7f46aca26640 1 -- 172.21.15.69:0/1519431906 shutdown_connections 2024-08-02T17:59:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.644+0000 7f46aca26640 1 --2- 172.21.15.69:0/1519431906 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a8100fc0 0x7f46a810d4d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.644+0000 7f46aca26640 1 --2- 172.21.15.69:0/1519431906 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a8100600 0x7f46a8100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.644+0000 7f46aca26640 1 --2- 172.21.15.69:0/1519431906 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a8106690 0x7f46a8106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.644+0000 7f46aca26640 1 -- 172.21.15.69:0/1519431906 >> 172.21.15.69:0/1519431906 conn(0x7f46a80fc340 msgr2=0x7f46a80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:47.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.645+0000 7f46aca26640 1 -- 172.21.15.69:0/1519431906 shutdown_connections 2024-08-02T17:59:47.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.646+0000 7f46aca26640 1 -- 172.21.15.69:0/1519431906 wait complete. 2024-08-02T17:59:47.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.647+0000 7f46aca26640 1 Processor -- start 2024-08-02T17:59:47.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46aca26640 1 -- start start 2024-08-02T17:59:47.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46aca26640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a8100600 0x7f46a8073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:47.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46aca26640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a8100fc0 0x7f46a806eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:47.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46aca26640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a8106690 0x7f46a806f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:47.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46aca26640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f46a806d200 con 0x7f46a8106690 2024-08-02T17:59:47.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46aca26640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46a806d080 con 0x7f46a8100600 2024-08-02T17:59:47.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46aca26640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46a806d380 con 0x7f46a8100fc0 2024-08-02T17:59:47.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a8100600 0x7f46a8073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:47.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46a7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a8106690 0x7f46a806f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:47.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46a77fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a8100600 0x7f46a8073a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33488/0 (socket says 172.21.15.69:33488) 2024-08-02T17:59:47.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.648+0000 7f46a77fe640 1 -- 172.21.15.69:0/3382039563 learned_addr learned my addr 172.21.15.69:0/3382039563 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:47.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.649+0000 7f46a7fff640 1 -- 172.21.15.69:0/3382039563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a8100fc0 msgr2=0x7f46a806eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:47.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.649+0000 7f46a7fff640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a8100fc0 0x7f46a806eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:47.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.649+0000 7f46a7fff640 1 -- 172.21.15.69:0/3382039563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a8100600 msgr2=0x7f46a8073a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:47.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.649+0000 7f46a7fff640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a8100600 0x7f46a8073a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:47.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.649+0000 7f46a7fff640 1 -- 172.21.15.69:0/3382039563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a806f600 con 0x7f46a8106690 2024-08-02T17:59:47.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.649+0000 7f46a7fff640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a8106690 0x7f46a806f040 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f469c03ba80 tx=0x7f469c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:47.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.651+0000 7f46a4ff9640 1 -- 172.21.15.69:0/3382039563 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c051070 con 0x7f46a8106690 2024-08-02T17:59:47.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.651+0000 7f46a4ff9640 1 -- 172.21.15.69:0/3382039563 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f469c0409f0 con 0x7f46a8106690 2024-08-02T17:59:47.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.651+0000 7f46a4ff9640 1 -- 172.21.15.69:0/3382039563 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469c042810 con 0x7f46a8106690 2024-08-02T17:59:47.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.653+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a806f890 con 0x7f46a8106690 2024-08-02T17:59:47.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.653+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a81a2850 con 0x7f46a8106690 2024-08-02T17:59:47.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.654+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4694005290 con 0x7f46a8106690 2024-08-02T17:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.664+0000 7f46a4ff9640 1 -- 172.21.15.69:0/3382039563 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f469c054070 con 0x7f46a8106690 2024-08-02T17:59:47.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.665+0000 7f46a4ff9640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46900a2f00 0x7f46900a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:47.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.666+0000 7f46a4ff9640 1 -- 172.21.15.69:0/3382039563 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f469c101000 con 0x7f46a8106690 2024-08-02T17:59:47.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.666+0000 7f46a77fe640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46900a2f00 0x7f46900a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.669+0000 7f46a4ff9640 1 -- 172.21.15.69:0/3382039563 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f469c0cd850 con 0x7f46a8106690 2024-08-02T17:59:47.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:47.669+0000 7f46a77fe640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46900a2f00 0x7f46900a53c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4698009a30 tx=0x7f4698009190 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:48.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.060+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4694005580 con 0x7f46a8106690 2024-08-02T17:59:48.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.061+0000 7f46a4ff9640 1 -- 172.21.15.69:0/3382039563 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f469c0cd150 con 0x7f46a8106690 2024-08-02T17:59:48.063 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:59:48.063 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:59:48.063 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46900a2f00 msgr2=0x7f46900a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46900a2f00 0x7f46900a53c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4698009a30 tx=0x7f4698009190 comp rx=0 tx=0).stop 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a8106690 msgr2=0x7f46a806f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a8106690 0x7f46a806f040 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f469c03ba80 tx=0x7f469c005040 comp rx=0 tx=0).stop 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 shutdown_connections 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f46a8106690 0x7f46a806f040 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46900a2f00 0x7f46900a53c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46a8100fc0 0x7f46a806eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 --2- 172.21.15.69:0/3382039563 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f46a8100600 0x7f46a8073a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 >> 172.21.15.69:0/3382039563 conn(0x7f46a80fc340 msgr2=0x7f46a80fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 shutdown_connections 2024-08-02T17:59:48.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:48.064+0000 7f46aca26640 1 -- 172.21.15.69:0/3382039563 wait complete. 2024-08-02T17:59:48.903 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-02T17:59:48.903 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-02T17:59:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:48 smithi083 ceph-mon[37031]: pgmap v418: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 282 KiB/s wr, 26 op/s 2024-08-02T17:59:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:48 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3382039563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:49.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:48 smithi077 ceph-mon[37108]: pgmap v418: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 282 KiB/s wr, 26 op/s 2024-08-02T17:59:49.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:48 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3382039563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:48 smithi069 ceph-mon[32746]: pgmap v418: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 282 KiB/s wr, 26 op/s 2024-08-02T17:59:49.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:48 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3382039563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T17:59:49.399 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:49.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 760.0 s, 396.1 tps, lat 1578.097 ms stddev 5963.306 2024-08-02T17:59:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 -- 172.21.15.69:0/2813706817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 msgr2=0x7f831410e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 --2- 172.21.15.69:0/2813706817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 0x7f831410e770 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f830c009be0 tx=0x7f830c02a6c0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 -- 172.21.15.69:0/2813706817 shutdown_connections 2024-08-02T17:59:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 --2- 172.21.15.69:0/2813706817 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83141018a0 0x7f8314101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 --2- 172.21.15.69:0/2813706817 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 0x7f831410e770 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 --2- 172.21.15.69:0/2813706817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83141078a0 0x7f8314107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 -- 172.21.15.69:0/2813706817 >> 172.21.15.69:0/2813706817 conn(0x7f83140fd5e0 msgr2=0x7f83140ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 -- 172.21.15.69:0/2813706817 shutdown_connections 2024-08-02T17:59:49.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.980+0000 7f831be76640 1 -- 172.21.15.69:0/2813706817 wait complete. 2024-08-02T17:59:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.981+0000 7f831be76640 1 Processor -- start 2024-08-02T17:59:49.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.981+0000 7f831be76640 1 -- start start 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.981+0000 7f831be76640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83141018a0 0x7f8314074260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.981+0000 7f831be76640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 0x7f83140747a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.981+0000 7f831be76640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83141078a0 0x7f8314074ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.982+0000 7f831be76640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8314111ca0 con 0x7f83141018a0 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.982+0000 7f831be76640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8314111b20 con 0x7f8314102240 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.982+0000 7f831be76640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8314111e20 con 0x7f83141078a0 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.982+0000 7f8319c12640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83141018a0 0x7f8314074260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.982+0000 7f8319c12640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83141018a0 0x7f8314074260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52270/0 (socket says 172.21.15.69:52270) 2024-08-02T17:59:49.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.982+0000 7f8319c12640 1 -- 172.21.15.69:0/3720320301 learned_addr learned my addr 172.21.15.69:0/3720320301 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.982+0000 7f8319411640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 0x7f83140747a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.982+0000 7f831a413640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83141078a0 0x7f8314074ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.983+0000 7f8319411640 1 -- 172.21.15.69:0/3720320301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83141078a0 msgr2=0x7f8314074ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.983+0000 7f8319411640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83141078a0 0x7f8314074ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.983+0000 7f8319411640 1 -- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83141018a0 msgr2=0x7f8314074260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.983+0000 7f8319411640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83141018a0 0x7f8314074260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.983+0000 7f8319411640 1 -- 172.21.15.69:0/3720320301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831419e260 con 0x7f8314102240 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.983+0000 7f8319411640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 0x7f83140747a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8304005fd0 tx=0x7f8304030e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.984+0000 7f830affd640 1 -- 172.21.15.69:0/3720320301 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830400cd80 con 0x7f8314102240 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.984+0000 7f830affd640 1 -- 172.21.15.69:0/3720320301 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83040035f0 con 0x7f8314102240 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.985+0000 7f830affd640 1 -- 172.21.15.69:0/3720320301 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304032680 con 0x7f8314102240 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.985+0000 7f831be76640 1 -- 172.21.15.69:0/3720320301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831419e4f0 con 0x7f8314102240 2024-08-02T17:59:49.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.985+0000 7f831be76640 1 -- 172.21.15.69:0/3720320301 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831419e950 con 0x7f8314102240 2024-08-02T17:59:49.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.987+0000 7f830affd640 1 -- 172.21.15.69:0/3720320301 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f83040037a0 con 0x7f8314102240 2024-08-02T17:59:49.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.988+0000 7f830affd640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83000a2e40 0x7f83000a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:49.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.988+0000 7f831e274740 1 Processor -- start 2024-08-02T17:59:49.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.988+0000 7f830affd640 1 -- 172.21.15.69:0/3720320301 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f83040f19d0 con 0x7f8314102240 2024-08-02T17:59:49.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.988+0000 7f8319c12640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83000a2e40 0x7f83000a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:49.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.988+0000 7f831e274740 1 -- start start 2024-08-02T17:59:49.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f831e274740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a60510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:49.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f831e274740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60a50 0x556987a6add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f831e274740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f831e274740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5569879c5a70 con 0x556987a60a50 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f831e274740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55698795d780 con 0x556987a6b310 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f831e274740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569879c1a30 con 0x556987a60130 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319411640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60a50 0x556987a6add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319c12640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83000a2e40 0x7f83000a5300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8310009c50 tx=0x7f8310005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319c12640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a60510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319411640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60a50 0x556987a6add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52272/0 (socket says 172.21.15.69:52272) 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319411640 1 -- 172.21.15.69:0/1696066613 learned_addr learned my addr 172.21.15.69:0/1696066613 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f831a413640 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319411640 1 -- 172.21.15.69:0/1696066613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 msgr2=0x556987a60510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319411640 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a60510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319411640 1 -- 172.21.15.69:0/1696066613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 msgr2=0x556987a6d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319411640 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319411640 1 -- 172.21.15.69:0/1696066613 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556987737b90 con 0x556987a60a50 2024-08-02T17:59:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.989+0000 7f8319c12640 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a60510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T17:59:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.990+0000 7f8319411640 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60a50 0x556987a6add0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8304009fd0 tx=0x7f83040be1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa053b51d2e1ddf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.990+0000 7f82fffff640 1 -- 172.21.15.69:0/1696066613 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83040d5070 con 0x556987a60a50 2024-08-02T17:59:49.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.991+0000 7f82fffff640 1 -- 172.21.15.69:0/1696066613 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83040c1e90 con 0x556987a60a50 2024-08-02T17:59:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.991+0000 7f82fffff640 1 -- 172.21.15.69:0/1696066613 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83040bcf30 con 0x556987a60a50 2024-08-02T17:59:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.992+0000 7f831e274740 1 -- 172.21.15.69:0/1696066613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60a50 msgr2=0x556987a6add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.992+0000 7f831e274740 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60a50 0x556987a6add0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8304009fd0 tx=0x7f83040be1e0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.993+0000 7f831e274740 1 -- 172.21.15.69:0/1696066613 shutdown_connections 2024-08-02T17:59:49.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.993+0000 7f831e274740 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60a50 0x556987a6add0 secure :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f8304009fd0 tx=0x7f83040be1e0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.993+0000 7f831e274740 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.993+0000 7f831e274740 1 --2- 172.21.15.69:0/1696066613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a60510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:49.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.993+0000 7f831e274740 1 -- 172.21.15.69:0/1696066613 >> 172.21.15.69:0/1696066613 conn(0x556987a5c6c0 msgr2=0x556987a5caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:49.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:49.995+0000 7f831e274740 1 -- 172.21.15.69:0/1696066613 shutdown_connections 2024-08-02T17:59:50.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.001+0000 7f831e274740 1 -- 172.21.15.69:0/1696066613 wait complete. 2024-08-02T17:59:50.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.001+0000 7f831e274740 1 -- start start 2024-08-02T17:59:50.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.001+0000 7f831e274740 1 Processor -- start 2024-08-02T17:59:50.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.004+0000 7f831e274740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a60130 0x556987a68370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.004+0000 7f831e274740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a6b310 0x556987a688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.004+0000 7f831e274740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a77350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.004+0000 7f831e274740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556987a6dee0 con 0x556987a68df0 2024-08-02T17:59:50.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.004+0000 7f831e274740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556987980990 con 0x556987a60130 2024-08-02T17:59:50.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.004+0000 7f831e274740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556987a6e060 con 0x556987a6b310 2024-08-02T17:59:50.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.004+0000 7f831e274740 10 client.0 _ll_register_callbacks cb 0x5569876d5090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:50.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.009+0000 7f8319c12640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a60130 0x556987a68370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:50.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.009+0000 7f831a413640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a77350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:50.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.009+0000 7f831a413640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a77350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52278/0 (socket says 172.21.15.69:52278) 2024-08-02T17:59:50.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.009+0000 7f831a413640 1 -- 172.21.15.69:0/2295430427 learned_addr learned my addr 172.21.15.69:0/2295430427 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:50.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.013+0000 7f831a413640 1 -- 172.21.15.69:0/2295430427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a6b310 msgr2=0x556987a688b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:59:50.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.013+0000 7f831a413640 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a6b310 0x556987a688b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.013+0000 7f831a413640 1 -- 172.21.15.69:0/2295430427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a60130 msgr2=0x556987a68370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:50.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.013+0000 7f831a413640 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a60130 0x556987a68370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.013+0000 7f831a413640 1 -- 172.21.15.69:0/2295430427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55698797f6c0 con 0x556987a68df0 2024-08-02T17:59:50.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.013+0000 7f8319c12640 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a60130 0x556987a68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:50.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.017+0000 7f831a413640 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a77350 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f830c009be0 tx=0x7f830c030520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e39c9622eebbaf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:50.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.018+0000 7f82fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:50.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.019+0000 7f82fd7fa640 1 -- 172.21.15.69:0/2295430427 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c042070 con 0x556987a68df0 2024-08-02T17:59:50.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.019+0000 7f82fd7fa640 1 -- 172.21.15.69:0/2295430427 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f830c030c20 con 0x556987a68df0 2024-08-02T17:59:50.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.023+0000 7f831e274740 1 -- 172.21.15.69:0/2295430427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83040c1e90 con 0x556987a68df0 2024-08-02T17:59:50.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.027+0000 7f82fd7fa640 1 -- 172.21.15.69:0/2295430427 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f830c036040 con 0x556987a68df0 2024-08-02T17:59:50.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.027+0000 7f831e274740 10 client.18684 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:50.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.027+0000 7f831e274740 1 -- 172.21.15.69:0/2295430427 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556987a7a0e0 con 0x556987a68df0 2024-08-02T17:59:50.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.029+0000 7f82fd7fa640 1 -- 172.21.15.69:0/2295430427 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83040c1e90 con 0x556987a68df0 2024-08-02T17:59:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.030+0000 7f82fd7fa640 1 -- 172.21.15.69:0/2295430427 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f830c004690 con 0x556987a68df0 2024-08-02T17:59:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.030+0000 7f831e274740 10 client.18684 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.030+0000 7f831e274740 10 client.18684 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:59:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.030+0000 7f831e274740 20 client.18684 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.030+0000 7f831e274740 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556987a7b5a0 0x556987a7d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.030+0000 7f831e274740 4 client.18684 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:50.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.030+0000 7f831e274740 1 -- 172.21.15.69:0/2295430427 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569879d31a0 con 0x556987a7b5a0 2024-08-02T17:59:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.030+0000 7f8319c12640 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556987a7b5a0 0x556987a7d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:50.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.031+0000 7f8319c12640 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556987a7b5a0 0x556987a7d9b0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=437ba8cf492204a7 server_cookie=9289db44e71012cd in_seq=0 out_seq=0 2024-08-02T17:59:50.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.032+0000 7f82fd7fa640 10 client.18684 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:50.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.032+0000 7f82fd7fa640 1 -- 172.21.15.69:0/2295430427 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569879d31a0 con 0x556987a7b5a0 2024-08-02T17:59:50.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.032+0000 7f82fd7fa640 10 client.18684 handle_command_reply: tid=0 2024-08-02T17:59:50.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.032+0000 7f831e274740 1 client.18684 shutdown 2024-08-02T17:59:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.034+0000 7f831e274740 1 -- 172.21.15.69:0/2295430427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 msgr2=0x556987a77350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:50.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.034+0000 7f831e274740 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a77350 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f830c009be0 tx=0x7f830c030520 comp rx=0 tx=0).stop 2024-08-02T17:59:50.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.035+0000 7f831e274740 1 -- 172.21.15.69:0/2295430427 shutdown_connections 2024-08-02T17:59:50.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.035+0000 7f831e274740 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a77350 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.035+0000 7f831e274740 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556987a7b5a0 0x556987a7d9b0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.035+0000 7f831e274740 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a6b310 0x556987a688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.035+0000 7f831e274740 1 --2- 172.21.15.69:0/2295430427 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a60130 0x556987a68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.035+0000 7f82fd7fa640 0 client.18684 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:50.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.035+0000 7f831e274740 1 -- 172.21.15.69:0/2295430427 >> 172.21.15.69:0/2295430427 conn(0x556987a5c6c0 msgr2=0x556987a623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:50.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.039+0000 7f831e274740 1 -- 172.21.15.69:0/2295430427 shutdown_connections 2024-08-02T17:59:50.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.039+0000 7f831e274740 1 -- 172.21.15.69:0/2295430427 wait complete. 2024-08-02T17:59:50.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.039+0000 7f831e274740 20 client.18684 trim_cache size 0 max 16384 2024-08-02T17:59:50.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.072+0000 7f831e274740 1 Processor -- start 2024-08-02T17:59:50.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f831e274740 1 -- start start 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f831e274740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a77140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f831e274740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f831e274740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a83280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f831e274740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83141af440 con 0x556987a68df0 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f831e274740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83141acd10 con 0x556987a6b310 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f831e274740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83141a47c0 con 0x556987a60130 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f8319c12640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a77140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f8319c12640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a77140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:57750/0 (socket says 172.21.15.69:57750) 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.073+0000 7f8319c12640 1 -- 172.21.15.69:0/220765048 learned_addr learned my addr 172.21.15.69:0/220765048 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.074+0000 7f831a413640 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a83280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:50.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.074+0000 7f8319411640 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:50.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.074+0000 7f8319c12640 1 -- 172.21.15.69:0/220765048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 msgr2=0x556987a83280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:50.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.074+0000 7f8319c12640 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a83280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.074+0000 7f8319c12640 1 -- 172.21.15.69:0/220765048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 msgr2=0x556987a6a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:50.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.074+0000 7f8319c12640 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a6a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.075+0000 7f8319c12640 1 -- 172.21.15.69:0/220765048 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556987a63c30 con 0x556987a60130 2024-08-02T17:59:50.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.076+0000 7f8319c12640 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a77140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8310013010 tx=0x7f8310013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a60a8754e24e136e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:50.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.077+0000 7f82ff7fe640 1 -- 172.21.15.69:0/220765048 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831000f890 con 0x556987a60130 2024-08-02T17:59:50.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.077+0000 7f82ff7fe640 1 -- 172.21.15.69:0/220765048 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310029430 con 0x556987a60130 2024-08-02T17:59:50.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.077+0000 7f82ff7fe640 1 -- 172.21.15.69:0/220765048 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8310027660 con 0x556987a60130 2024-08-02T17:59:50.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.078+0000 7f831e274740 1 -- 172.21.15.69:0/220765048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 msgr2=0x556987a77140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:50.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.078+0000 7f831e274740 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a77140 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8310013010 tx=0x7f8310013040 comp rx=0 tx=0).stop 2024-08-02T17:59:50.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.083+0000 7f831e274740 1 -- 172.21.15.69:0/220765048 shutdown_connections 2024-08-02T17:59:50.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.083+0000 7f831e274740 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a68df0 0x556987a6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.083+0000 7f831e274740 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a83280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.083+0000 7f831e274740 1 --2- 172.21.15.69:0/220765048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a60130 0x556987a77140 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.083+0000 7f831e274740 1 -- 172.21.15.69:0/220765048 >> 172.21.15.69:0/220765048 conn(0x556987a5c6c0 msgr2=0x556987a69bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:50.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.085+0000 7f831e274740 1 -- 172.21.15.69:0/220765048 shutdown_connections 2024-08-02T17:59:50.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.085+0000 7f831e274740 1 -- 172.21.15.69:0/220765048 wait complete. 2024-08-02T17:59:50.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 1 -- start start 2024-08-02T17:59:50.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 1 Processor -- start 2024-08-02T17:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60130 0x556987a6f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a68df0 0x556987a6f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f83141ab1d0 con 0x556987a60130 2024-08-02T17:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f83141ac640 con 0x556987a6b310 2024-08-02T17:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83141ab8a0 con 0x556987a68df0 2024-08-02T17:59:50.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.086+0000 7f831e274740 10 client.0 _ll_register_callbacks cb 0x7f8310005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:50.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.087+0000 7f831a413640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:50.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.087+0000 7f831a413640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33540/0 (socket says 172.21.15.69:33540) 2024-08-02T17:59:50.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.087+0000 7f831a413640 1 -- 172.21.15.69:0/1889109854 learned_addr learned my addr 172.21.15.69:0/1889109854 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:50.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.088+0000 7f831a413640 1 -- 172.21.15.69:0/1889109854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a68df0 msgr2=0x556987a6f760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.088+0000 7f831a413640 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a68df0 0x556987a6f760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.088+0000 7f831a413640 1 -- 172.21.15.69:0/1889109854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60130 msgr2=0x556987a6f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.088+0000 7f831a413640 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60130 0x556987a6f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.088+0000 7f831a413640 1 -- 172.21.15.69:0/1889109854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556987af6460 con 0x556987a6b310 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.088+0000 7f831a413640 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6fca0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f830c031db0 tx=0x7f830c031de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce41b0bf3ca5820 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.088+0000 7f82fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f82fd7fa640 1 -- 172.21.15.69:0/1889109854 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c042070 con 0x556987a6b310 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f82fd7fa640 1 -- 172.21.15.69:0/1889109854 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f830c030c30 con 0x556987a6b310 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f831e274740 1 -- 172.21.15.69:0/1889109854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8310029430 con 0x556987a6b310 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f82fd7fa640 1 -- 172.21.15.69:0/1889109854 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f830c060050 con 0x556987a6b310 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f831e274740 10 client.28124 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f831e274740 1 -- 172.21.15.69:0/1889109854 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556987af7720 con 0x556987a6b310 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f82fd7fa640 1 -- 172.21.15.69:0/1889109854 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8310029430 con 0x556987a6b310 2024-08-02T17:59:50.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f82fd7fa640 1 -- 172.21.15.69:0/1889109854 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f830c004300 con 0x556987a6b310 2024-08-02T17:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f831e274740 10 client.28124 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f831e274740 10 client.28124 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T17:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.089+0000 7f831e274740 20 client.28124 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.090+0000 7f831e274740 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556987af7df0 0x556987afa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.090+0000 7f831e274740 4 client.28124 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-02T17:59:50.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.090+0000 7f831e274740 1 -- 172.21.15.69:0/1889109854 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5569879cd4f0 con 0x556987af7df0 2024-08-02T17:59:50.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.090+0000 7f8319c12640 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556987af7df0 0x556987afa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:50.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.091+0000 7f8319c12640 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556987af7df0 0x556987afa270 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7f946b71d44668 server_cookie=75bcc1e19fc830b2 in_seq=0 out_seq=0 2024-08-02T17:59:50.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.091+0000 7f82fd7fa640 10 client.28124 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f82fd7fa640 1 -- 172.21.15.69:0/1889109854 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5569879cd4f0 con 0x556987af7df0 2024-08-02T17:59:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f82fd7fa640 10 client.28124 handle_command_reply: tid=0 2024-08-02T17:59:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 client.28124 shutdown 2024-08-02T17:59:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 -- 172.21.15.69:0/1889109854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 msgr2=0x556987a6fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6fca0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f830c031db0 tx=0x7f830c031de0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 -- 172.21.15.69:0/1889109854 shutdown_connections 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556987af7df0 0x556987afa270 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556987a6b310 0x556987a6fca0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556987a68df0 0x556987a6f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f82fd7fa640 0 client.28124 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 --2- 172.21.15.69:0/1889109854 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556987a60130 0x556987a6f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 -- 172.21.15.69:0/1889109854 >> 172.21.15.69:0/1889109854 conn(0x556987a5c6c0 msgr2=0x556987a69bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 -- 172.21.15.69:0/1889109854 shutdown_connections 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 1 -- 172.21.15.69:0/1889109854 wait complete. 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.392+0000 7f831e274740 20 client.28124 trim_cache size 0 max 16384 2024-08-02T17:59:50.395 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 50, 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "2c6a3888": { 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 50, 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T17:59:50.396 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 150.0, 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T17:59:50.397 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:59:50.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 -- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83000a2e40 msgr2=0x7f83000a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:50.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83000a2e40 0x7f83000a5300 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8310009c50 tx=0x7f8310005f90 comp rx=0 tx=0).stop 2024-08-02T17:59:50.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 -- 172.21.15.69:0/3720320301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 msgr2=0x7f83140747a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:50.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 0x7f83140747a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8304005fd0 tx=0x7f8304030e60 comp rx=0 tx=0).stop 2024-08-02T17:59:50.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 -- 172.21.15.69:0/3720320301 shutdown_connections 2024-08-02T17:59:50.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f83000a2e40 0x7f83000a5300 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83141078a0 0x7f8314074ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.398 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f8314102240 0x7f83140747a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 --2- 172.21.15.69:0/3720320301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f83141018a0 0x7f8314074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:50.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 -- 172.21.15.69:0/3720320301 >> 172.21.15.69:0/3720320301 conn(0x7f83140fd5e0 msgr2=0x7f83140fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:50.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 -- 172.21.15.69:0/3720320301 shutdown_connections 2024-08-02T17:59:50.399 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:50.393+0000 7f831be76640 1 -- 172.21.15.69:0/3720320301 wait complete. 2024-08-02T17:59:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:50 smithi083 ceph-mon[37031]: pgmap v419: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-08-02T17:59:51.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:50 smithi077 ceph-mon[37108]: pgmap v419: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-08-02T17:59:51.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:50 smithi069 ceph-mon[32746]: pgmap v419: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-08-02T17:59:51.264 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2c6a3888', quiesce took 4.7 seconds. Will release after: 25.3 2024-08-02T17:59:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 -- 172.21.15.69:0/1979931043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6554103390 msgr2=0x7f655410b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 --2- 172.21.15.69:0/1979931043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6554103390 0x7f655410b870 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6548009900 tx=0x7f654802a480 comp rx=0 tx=0).stop 2024-08-02T17:59:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 -- 172.21.15.69:0/1979931043 shutdown_connections 2024-08-02T17:59:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 --2- 172.21.15.69:0/1979931043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6554103390 0x7f655410b870 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 --2- 172.21.15.69:0/1979931043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f655410be40 0x7f655410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 --2- 172.21.15.69:0/1979931043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6554102a70 0x7f6554102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 -- 172.21.15.69:0/1979931043 >> 172.21.15.69:0/1979931043 conn(0x7f65540776d0 msgr2=0x7f65540fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 -- 172.21.15.69:0/1979931043 shutdown_connections 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.894+0000 7f655b874640 1 -- 172.21.15.69:0/1979931043 wait complete. 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f655b874640 1 Processor -- start 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f655b874640 1 -- start start 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f655b874640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6554102a70 0x7f65541980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f655b874640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554103390 0x7f6554198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f655b874640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f655410be40 0x7f655419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f655b874640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6554111c20 con 0x7f655410be40 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f655b874640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6554111aa0 con 0x7f6554102a70 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f655b874640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6554111da0 con 0x7f6554103390 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f655410be40 0x7f655419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f655410be40 0x7f655419c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52302/0 (socket says 172.21.15.69:52302) 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 -- 172.21.15.69:0/340918491 learned_addr learned my addr 172.21.15.69:0/340918491 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559610640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6554102a70 0x7f65541980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 -- 172.21.15.69:0/340918491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554103390 msgr2=0x7f6554198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6558e0f640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554103390 0x7f6554198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554103390 0x7f6554198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 -- 172.21.15.69:0/340918491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6554102a70 msgr2=0x7f65541980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6554102a70 0x7f65541980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 -- 172.21.15.69:0/340918491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655419d0c0 con 0x7f655410be40 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559610640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6554102a70 0x7f65541980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f6559e11640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f655410be40 0x7f655419c990 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f654c00eb10 tx=0x7f654c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.895+0000 7f65527fc640 1 -- 172.21.15.69:0/340918491 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c004b20 con 0x7f655410be40 2024-08-02T17:59:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.896+0000 7f65527fc640 1 -- 172.21.15.69:0/340918491 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f654c004cc0 con 0x7f655410be40 2024-08-02T17:59:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.896+0000 7f655b874640 1 -- 172.21.15.69:0/340918491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655419d3b0 con 0x7f655410be40 2024-08-02T17:59:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.896+0000 7f65527fc640 1 -- 172.21.15.69:0/340918491 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c010760 con 0x7f655410be40 2024-08-02T17:59:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.896+0000 7f655b874640 1 -- 172.21.15.69:0/340918491 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65541a4c40 con 0x7f655410be40 2024-08-02T17:59:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.898+0000 7f65527fc640 1 -- 172.21.15.69:0/340918491 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f654c005020 con 0x7f655410be40 2024-08-02T17:59:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.898+0000 7f655e5ca740 1 Processor -- start 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.898+0000 7f655e5ca740 1 -- start start 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.898+0000 7f65527fc640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f653c0a2d70 0x7f653c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b1cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b1d0d0 0x5593f1b27430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b27970 0x5593f1b29d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f655e5ca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5593f1a809b0 con 0x5593f1b27970 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f655e5ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5593f1a28800 con 0x5593f1b1c7b0 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f655e5ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5593f1a7c970 con 0x5593f1b1d0d0 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f65527fc640 1 -- 172.21.15.69:0/340918491 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f654c0cff50 con 0x7f655410be40 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6558e0f640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f653c0a2d70 0x7f653c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b27970 0x5593f1b29d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559610640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b1cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b27970 0x5593f1b29d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52316/0 (socket says 172.21.15.69:52316) 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 -- 172.21.15.69:0/4071345384 learned_addr learned my addr 172.21.15.69:0/4071345384 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 -- 172.21.15.69:0/4071345384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b1d0d0 msgr2=0x5593f1b27430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 --2- 172.21.15.69:0/4071345384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b1d0d0 0x5593f1b27430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 -- 172.21.15.69:0/4071345384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 msgr2=0x5593f1b1cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 --2- 172.21.15.69:0/4071345384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b1cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 -- 172.21.15.69:0/4071345384 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593f1794210 con 0x5593f1b27970 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6559e11640 1 --2- 172.21.15.69:0/4071345384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b27970 0x5593f1b29d60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f654c000f80 tx=0x7f654c09bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc7a21e43abdf1c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f6558e0f640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f653c0a2d70 0x7f653c0a5230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5593f1b19b80 tx=0x7f6548000bc0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f653b7fe640 1 -- 172.21.15.69:0/4071345384 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c09c060 con 0x5593f1b27970 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f653b7fe640 1 -- 172.21.15.69:0/4071345384 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f654c0a0410 con 0x5593f1b27970 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.899+0000 7f653b7fe640 1 -- 172.21.15.69:0/4071345384 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0a00c0 con 0x5593f1b27970 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 -- 172.21.15.69:0/4071345384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b27970 msgr2=0x5593f1b29d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 --2- 172.21.15.69:0/4071345384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b27970 0x5593f1b29d60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f654c000f80 tx=0x7f654c09bef0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 -- 172.21.15.69:0/4071345384 shutdown_connections 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 --2- 172.21.15.69:0/4071345384 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b27970 0x5593f1b29d60 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 --2- 172.21.15.69:0/4071345384 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b1d0d0 0x5593f1b27430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 --2- 172.21.15.69:0/4071345384 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b1cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 -- 172.21.15.69:0/4071345384 >> 172.21.15.69:0/4071345384 conn(0x5593f1b18d20 msgr2=0x5593f1b19100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 -- 172.21.15.69:0/4071345384 shutdown_connections 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 -- 172.21.15.69:0/4071345384 wait complete. 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.900+0000 7f655e5ca740 1 -- start start 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f655e5ca740 1 Processor -- start 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1c7b0 0x5593f1b24840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1d0d0 0x5593f1b24d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 0x5593f1b252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f655e5ca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5593f1b2a540 con 0x5593f1b1c7b0 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f655e5ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5593f1a3c500 con 0x5593f1b1d0d0 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f655e5ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5593f1b2a6c0 con 0x5593f1b27970 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f655e5ca740 10 client.0 _ll_register_callbacks cb 0x5593f1793a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559610640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1c7b0 0x5593f1b24840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559610640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1c7b0 0x5593f1b24840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52332/0 (socket says 172.21.15.69:52332) 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559610640 1 -- 172.21.15.69:0/2433894915 learned_addr learned my addr 172.21.15.69:0/2433894915 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1d0d0 0x5593f1b24d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559610640 1 -- 172.21.15.69:0/2433894915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 msgr2=0x5593f1b252c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559e11640 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 0x5593f1b252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559610640 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 0x5593f1b252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559610640 1 -- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1d0d0 msgr2=0x5593f1b24d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559610640 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1d0d0 0x5593f1b24d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6559610640 1 -- 172.21.15.69:0/2433894915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593f1860620 con 0x5593f1b1c7b0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.901+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1d0d0 0x5593f1b24d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f6559610640 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1c7b0 0x5593f1b24840 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6544012480 tx=0x7f6544012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd664dab5964d19 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f6538ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f6538ff9640 1 -- 172.21.15.69:0/2433894915 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65440035f0 con 0x5593f1b1c7b0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f6538ff9640 1 -- 172.21.15.69:0/2433894915 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f6544003790 con 0x5593f1b1c7b0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 1 -- 172.21.15.69:0/2433894915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f654c0a0410 con 0x5593f1b1c7b0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f6538ff9640 1 -- 172.21.15.69:0/2433894915 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6544003990 con 0x5593f1b1c7b0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 10 client.18699 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 1 -- 172.21.15.69:0/2433894915 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5593f1b36210 con 0x5593f1b1c7b0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f6538ff9640 1 -- 172.21.15.69:0/2433894915 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f654c0a0410 con 0x5593f1b1c7b0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f6538ff9640 1 -- 172.21.15.69:0/2433894915 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6544019070 con 0x5593f1b1c7b0 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 10 client.18699 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 10 client.18699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 20 client.18699 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5593f1b376d0 0x5593f1b39b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 4 client.18699 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.902+0000 7f655e5ca740 1 -- 172.21.15.69:0/2433894915 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593f1a8e0e0 con 0x5593f1b376d0 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.903+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5593f1b376d0 0x5593f1b39b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.903+0000 7f6538ff9640 10 client.18699 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.903+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5593f1b376d0 0x5593f1b39b00 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513a44b7cc035f1b server_cookie=a3d273d20c1ca5ef in_seq=0 out_seq=0 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f6538ff9640 1 -- 172.21.15.69:0/2433894915 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5593f1a8e0e0 con 0x5593f1b376d0 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f6538ff9640 10 client.18699 handle_command_reply: tid=0 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 client.18699 shutdown 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 -- 172.21.15.69:0/2433894915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1c7b0 msgr2=0x5593f1b24840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1c7b0 0x5593f1b24840 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6544012480 tx=0x7f6544012940 comp rx=0 tx=0).stop 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 -- 172.21.15.69:0/2433894915 shutdown_connections 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5593f1b376d0 0x5593f1b39b00 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 0x5593f1b252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1d0d0 0x5593f1b24d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f6538ff9640 0 client.18699 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2433894915 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1c7b0 0x5593f1b24840 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 -- 172.21.15.69:0/2433894915 >> 172.21.15.69:0/2433894915 conn(0x5593f1b18d20 msgr2=0x5593f1b1ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 -- 172.21.15.69:0/2433894915 shutdown_connections 2024-08-02T17:59:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 1 -- 172.21.15.69:0/2433894915 wait complete. 2024-08-02T17:59:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.904+0000 7f655e5ca740 20 client.18699 trim_cache size 0 max 16384 2024-08-02T17:59:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f655e5ca740 1 Processor -- start 2024-08-02T17:59:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f655e5ca740 1 -- start start 2024-08-02T17:59:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b1c7b0 0x5593f1b230f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b3f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b27970 0x5593f1b3f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f655e5ca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65541b1870 con 0x5593f1b1d0d0 2024-08-02T17:59:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f655e5ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65541af140 con 0x5593f1b27970 2024-08-02T17:59:52.925 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f655e5ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65541acf90 con 0x5593f1b1c7b0 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f6558e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b3f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f6558e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b3f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52340/0 (socket says 172.21.15.69:52340) 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.919+0000 7f6558e0f640 1 -- 172.21.15.69:0/2200242744 learned_addr learned my addr 172.21.15.69:0/2200242744 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f6558e0f640 1 -- 172.21.15.69:0/2200242744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b1c7b0 msgr2=0x5593f1b230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b1c7b0 0x5593f1b230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f6558e0f640 1 -- 172.21.15.69:0/2200242744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b27970 msgr2=0x5593f1b3f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f6559e11640 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b27970 0x5593f1b3f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b27970 0x5593f1b3f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f6558e0f640 1 -- 172.21.15.69:0/2200242744 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593f1b19e90 con 0x5593f1b1d0d0 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f6559e11640 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b27970 0x5593f1b3f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b3f460 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f65540ffc30 tx=0x7f6548035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a312de53cb61e8c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f653affd640 1 -- 172.21.15.69:0/2200242744 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548059070 con 0x5593f1b1d0d0 2024-08-02T17:59:52.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f653affd640 1 -- 172.21.15.69:0/2200242744 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6548045040 con 0x5593f1b1d0d0 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f653affd640 1 -- 172.21.15.69:0/2200242744 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548035070 con 0x5593f1b1d0d0 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.920+0000 7f655e5ca740 1 -- 172.21.15.69:0/2200242744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 msgr2=0x5593f1b3f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b3f460 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f65540ffc30 tx=0x7f6548035000 comp rx=0 tx=0).stop 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 -- 172.21.15.69:0/2200242744 shutdown_connections 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b3f460 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b27970 0x5593f1b3f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2200242744 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b1c7b0 0x5593f1b230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 -- 172.21.15.69:0/2200242744 >> 172.21.15.69:0/2200242744 conn(0x5593f1b18d20 msgr2=0x5593f1b267b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 -- 172.21.15.69:0/2200242744 shutdown_connections 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 -- 172.21.15.69:0/2200242744 wait complete. 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 -- start start 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 Processor -- start 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b2f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.927 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b2fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 0x5593f1b2c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f65541ad600 con 0x5593f1b1d0d0 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f65541aea70 con 0x5593f1b1c7b0 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f65541adcd0 con 0x5593f1b27970 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f655e5ca740 10 client.0 _ll_register_callbacks cb 0x7f654400bc00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f6558e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b2fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f6558e0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b2fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52342/0 (socket says 172.21.15.69:52342) 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.921+0000 7f6558e0f640 1 -- 172.21.15.69:0/2524894169 learned_addr learned my addr 172.21.15.69:0/2524894169 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6559e11640 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 0x5593f1b2c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6559610640 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b2f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6558e0f640 1 -- 172.21.15.69:0/2524894169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 msgr2=0x5593f1b2c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 0x5593f1b2c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.928 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6558e0f640 1 -- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 msgr2=0x5593f1b2f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b2f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6558e0f640 1 -- 172.21.15.69:0/2524894169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593f1b2ce90 con 0x5593f1b1d0d0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6559610640 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b2f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6558e0f640 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b2fec0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6548006010 tx=0x7f65480315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3782b10b6c60cc10 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6538ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6538ff9640 1 -- 172.21.15.69:0/2524894169 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548059040 con 0x5593f1b1d0d0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6538ff9640 1 -- 172.21.15.69:0/2524894169 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f6548031990 con 0x5593f1b1d0d0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f655e5ca740 1 -- 172.21.15.69:0/2524894169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6548045040 con 0x5593f1b1d0d0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6538ff9640 1 -- 172.21.15.69:0/2524894169 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f654806b050 con 0x5593f1b1d0d0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f655e5ca740 10 client.18705 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f655e5ca740 1 -- 172.21.15.69:0/2524894169 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5593f1ba2ab0 con 0x5593f1b1d0d0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6538ff9640 1 -- 172.21.15.69:0/2524894169 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6548045040 con 0x5593f1b1d0d0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.922+0000 7f6538ff9640 1 -- 172.21.15.69:0/2524894169 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f654805c070 con 0x5593f1b1d0d0 2024-08-02T17:59:52.929 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f655e5ca740 10 client.18705 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f655e5ca740 10 client.18705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f655e5ca740 20 client.18705 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5593f1b376d0 0x5593f1ba5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f655e5ca740 4 client.18705 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T17:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f655e5ca740 1 -- 172.21.15.69:0/2524894169 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5593f1a88430 con 0x5593f1b376d0 2024-08-02T17:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f6559610640 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5593f1b376d0 0x5593f1ba5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f6538ff9640 10 client.18705 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:52.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.923+0000 7f6559610640 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5593f1b376d0 0x5593f1ba5210 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffcf56fb8b75f161 server_cookie=ccfae35b22e0ac0d in_seq=0 out_seq=0 2024-08-02T17:59:52.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f6538ff9640 1 -- 172.21.15.69:0/2524894169 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5593f1a88430 con 0x5593f1b376d0 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f6538ff9640 10 client.18705 handle_command_reply: tid=0 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 client.18705 shutdown 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 -- 172.21.15.69:0/2524894169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 msgr2=0x5593f1b2fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b2fec0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6548006010 tx=0x7f65480315b0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 -- 172.21.15.69:0/2524894169 shutdown_connections 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5593f1b1d0d0 0x5593f1b2fec0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5593f1b376d0 0x5593f1ba5210 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5593f1b27970 0x5593f1b2c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 --2- 172.21.15.69:0/2524894169 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5593f1b1c7b0 0x5593f1b2f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f6538ff9640 0 client.18705 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.942+0000 7f655e5ca740 1 -- 172.21.15.69:0/2524894169 >> 172.21.15.69:0/2524894169 conn(0x5593f1b18d20 msgr2=0x5593f1b17660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:52.945 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655e5ca740 1 -- 172.21.15.69:0/2524894169 shutdown_connections 2024-08-02T17:59:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655e5ca740 1 -- 172.21.15.69:0/2524894169 wait complete. 2024-08-02T17:59:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655e5ca740 20 client.18705 trim_cache size 0 max 16384 2024-08-02T17:59:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T17:59:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T17:59:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T17:59:52.946 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T17:59:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 -- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f653c0a2d70 msgr2=0x7f653c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f653c0a2d70 0x7f653c0a5230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5593f1b19b80 tx=0x7f6548000bc0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 -- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f655410be40 msgr2=0x7f655419c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f655410be40 0x7f655419c990 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f654c00eb10 tx=0x7f654c00c2d0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 -- 172.21.15.69:0/340918491 shutdown_connections 2024-08-02T17:59:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f655410be40 0x7f655419c990 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.947 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f653c0a2d70 0x7f653c0a5230 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6554103390 0x7f6554198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 --2- 172.21.15.69:0/340918491 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6554102a70 0x7f65541980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 -- 172.21.15.69:0/340918491 >> 172.21.15.69:0/340918491 conn(0x7f65540776d0 msgr2=0x7f65540fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 -- 172.21.15.69:0/340918491 shutdown_connections 2024-08-02T17:59:52.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T17:59:52.943+0000 7f655b874640 1 -- 172.21.15.69:0/340918491 wait complete. 2024-08-02T17:59:53.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:52 smithi083 ceph-mon[37031]: pgmap v420: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-08-02T17:59:53.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:52 smithi077 ceph-mon[37108]: pgmap v420: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-08-02T17:59:53.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:52 smithi069 ceph-mon[32746]: pgmap v420: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-08-02T17:59:54.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 765.0 s, 21.2 tps, lat 241.391 ms stddev 366.424 2024-08-02T17:59:55.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:54 smithi083 ceph-mon[37031]: pgmap v421: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 213 op/s 2024-08-02T17:59:55.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:54 smithi077 ceph-mon[37108]: pgmap v421: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 213 op/s 2024-08-02T17:59:55.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:54 smithi069 ceph-mon[32746]: pgmap v421: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 2.4 MiB/s wr, 213 op/s 2024-08-02T17:59:57.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:56 smithi083 ceph-mon[37031]: pgmap v422: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-08-02T17:59:57.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:56 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:59:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:56 smithi077 ceph-mon[37108]: pgmap v422: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-08-02T17:59:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:56 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:59:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:56 smithi069 ceph-mon[32746]: pgmap v422: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-08-02T17:59:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:56 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T17:59:57.297 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T17:59:57.550 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:57.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:57 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:59:57.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:57 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:59:57.941 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:57 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:59:58.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:57 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:59:58.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:57 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:59:58.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:57 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:59:58.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 -- 172.21.15.69:0/3902161308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf341078b0 msgr2=0x7fcf34107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 --2- 172.21.15.69:0/3902161308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf341078b0 0x7fcf34107c90 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009cd0 tx=0x7fcf2c02a8e0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 -- 172.21.15.69:0/3902161308 shutdown_connections 2024-08-02T17:59:58.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 --2- 172.21.15.69:0/3902161308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf34102250 0x7fcf3410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 --2- 172.21.15.69:0/3902161308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf341018b0 0x7fcf34101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 --2- 172.21.15.69:0/3902161308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf341078b0 0x7fcf34107c90 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 -- 172.21.15.69:0/3902161308 >> 172.21.15.69:0/3902161308 conn(0x7fcf340fd5f0 msgr2=0x7fcf340ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 -- 172.21.15.69:0/3902161308 shutdown_connections 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 -- 172.21.15.69:0/3902161308 wait complete. 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 Processor -- start 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 -- start start 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf341018b0 0x7fcf34074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.023+0000 7fcf3af57640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf34102250 0x7fcf34074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf3af57640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf341078b0 0x7fcf34074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf3af57640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34111c70 con 0x7fcf34102250 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf3af57640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34111af0 con 0x7fcf341018b0 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf3af57640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34111df0 con 0x7fcf341078b0 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf34102250 0x7fcf34074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf34102250 0x7fcf34074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40532/0 (socket says 172.21.15.69:40532) 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 -- 172.21.15.69:0/2360288614 learned_addr learned my addr 172.21.15.69:0/2360288614 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf341018b0 0x7fcf34074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf394f4640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf341078b0 0x7fcf34074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 -- 172.21.15.69:0/2360288614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf341078b0 msgr2=0x7fcf34074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf341078b0 0x7fcf34074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 -- 172.21.15.69:0/2360288614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf341018b0 msgr2=0x7fcf34074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf341018b0 0x7fcf34074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 -- 172.21.15.69:0/2360288614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf341a65f0 con 0x7fcf34102250 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf394f4640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf341078b0 0x7fcf34074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf33fff640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf34102250 0x7fcf34074760 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c920 tx=0x7fcf1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf31ffb640 1 -- 172.21.15.69:0/2360288614 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c007c70 con 0x7fcf34102250 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf31ffb640 1 -- 172.21.15.69:0/2360288614 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1c007e10 con 0x7fcf34102250 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf3af57640 1 -- 172.21.15.69:0/2360288614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf341a68e0 con 0x7fcf34102250 2024-08-02T17:59:58.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf31ffb640 1 -- 172.21.15.69:0/2360288614 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c018890 con 0x7fcf34102250 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.024+0000 7fcf3af57640 1 -- 172.21.15.69:0/2360288614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf341a6e20 con 0x7fcf34102250 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.026+0000 7fcf31ffb640 1 -- 172.21.15.69:0/2360288614 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcf1c018a70 con 0x7fcf34102250 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf3d355740 1 Processor -- start 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf3d355740 1 -- start start 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5bf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf3d355740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b9fb524980 con 0x55b9fb5bfb30 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf3d355740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b9fb4bca70 con 0x55b9fb5ca3f0 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf3d355740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b9fb520940 con 0x55b9fb5bf210 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf31ffb640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcf200a2cf0 0x7fcf200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf38cf3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf38cf3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40548/0 (socket says 172.21.15.69:40548) 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf38cf3640 1 -- 172.21.15.69:0/316487551 learned_addr learned my addr 172.21.15.69:0/316487551 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf394f4640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcf200a2cf0 0x7fcf200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf31ffb640 1 -- 172.21.15.69:0/2360288614 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fcf1c0cf9b0 con 0x7fcf34102250 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.027+0000 7fcf394f4640 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5bf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf33fff640 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf38cf3640 1 -- 172.21.15.69:0/316487551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 msgr2=0x55b9fb5bf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5bf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf38cf3640 1 -- 172.21.15.69:0/316487551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 msgr2=0x55b9fb5cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf38cf3640 1 -- 172.21.15.69:0/316487551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9fb51f6e0 con 0x55b9fb5bfb30 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf394f4640 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5bf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf33fff640 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf394f4640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcf200a2cf0 0x7fcf200a51b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf24005d80 tx=0x7fcf24005d10 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:58.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c9eb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009cd0 tx=0x7fcf2c0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673c116d9449c44a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf2affd640 1 -- 172.21.15.69:0/316487551 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c048070 con 0x55b9fb5bfb30 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf2affd640 1 -- 172.21.15.69:0/316487551 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c036070 con 0x55b9fb5bfb30 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.028+0000 7fcf2affd640 1 -- 172.21.15.69:0/316487551 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004630 con 0x55b9fb5bfb30 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 -- 172.21.15.69:0/316487551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 msgr2=0x55b9fb5c9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c9eb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009cd0 tx=0x7fcf2c0314e0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 -- 172.21.15.69:0/316487551 shutdown_connections 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c9eb0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 --2- 172.21.15.69:0/316487551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5bf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 -- 172.21.15.69:0/316487551 >> 172.21.15.69:0/316487551 conn(0x55b9fb5bb7a0 msgr2=0x55b9fb5bbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 -- 172.21.15.69:0/316487551 shutdown_connections 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 -- 172.21.15.69:0/316487551 wait complete. 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 -- start start 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.029+0000 7fcf3d355740 1 Processor -- start 2024-08-02T17:59:58.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55b9fb5ccfc0 con 0x55b9fb5bfb30 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55b9fb4dfaa0 con 0x55b9fb5bf210 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b9fb5cd140 con 0x55b9fb5ca3f0 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 10 client.0 _ll_register_callbacks cb 0x55b9fb4cee60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40558/0 (socket says 172.21.15.69:40558) 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 -- 172.21.15.69:0/3028897012 learned_addr learned my addr 172.21.15.69:0/3028897012 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf394f4640 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 -- 172.21.15.69:0/3028897012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5ca3f0 msgr2=0x55b9fb5c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:58.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 -- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bf210 msgr2=0x55b9fb5c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 -- 172.21.15.69:0/3028897012 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9fb4c3650 con 0x55b9fb5bfb30 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf33fff640 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c7760 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c09ba50 tx=0x7fcf1c09c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1ca6695370d8fb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf13fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf13fff640 1 -- 172.21.15.69:0/3028897012 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0a4730 con 0x55b9fb5bfb30 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf13fff640 1 -- 172.21.15.69:0/3028897012 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fcf1c018d60 con 0x55b9fb5bfb30 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 1 -- 172.21.15.69:0/3028897012 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2c036070 con 0x55b9fb5bfb30 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf13fff640 1 -- 172.21.15.69:0/3028897012 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf1c00fb30 con 0x55b9fb5bfb30 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 10 client.18717 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.030+0000 7fcf3d355740 1 -- 172.21.15.69:0/3028897012 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9fb2963c0 con 0x55b9fb5bfb30 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf13fff640 1 -- 172.21.15.69:0/3028897012 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c036070 con 0x55b9fb5bfb30 2024-08-02T17:59:58.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf13fff640 1 -- 172.21.15.69:0/3028897012 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf1c0a42a0 con 0x55b9fb5bfb30 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf3d355740 10 client.18717 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf3d355740 10 client.18717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf3d355740 20 client.18717 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf3d355740 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b9fb5d9cf0 0x55b9fb5dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf3d355740 4 client.18717 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf3d355740 1 -- 172.21.15.69:0/3028897012 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9fb531ef0 con 0x55b9fb5d9cf0 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b9fb5d9cf0 0x55b9fb5dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b9fb5d9cf0 0x55b9fb5dc0e0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b338463d9dd0e8 server_cookie=2285fd748b6e87e3 in_seq=0 out_seq=0 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.031+0000 7fcf13fff640 10 client.18717 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf13fff640 1 -- 172.21.15.69:0/3028897012 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b9fb531ef0 con 0x55b9fb5d9cf0 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf13fff640 10 client.18717 handle_command_reply: tid=0 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 client.18717 shutdown 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 -- 172.21.15.69:0/3028897012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 msgr2=0x55b9fb5c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c7760 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c09ba50 tx=0x7fcf1c09c280 comp rx=0 tx=0).stop 2024-08-02T17:59:58.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 -- 172.21.15.69:0/3028897012 shutdown_connections 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5c7760 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b9fb5d9cf0 0x55b9fb5dc0e0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 --2- 172.21.15.69:0/3028897012 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf13fff640 0 client.18717 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 -- 172.21.15.69:0/3028897012 >> 172.21.15.69:0/3028897012 conn(0x55b9fb5bb7a0 msgr2=0x55b9fb5c1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 -- 172.21.15.69:0/3028897012 shutdown_connections 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.032+0000 7fcf3d355740 1 -- 172.21.15.69:0/3028897012 wait complete. 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.033+0000 7fcf3d355740 20 client.18717 trim_cache size 0 max 16384 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.049+0000 7fcf3d355740 1 Processor -- start 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.049+0000 7fcf3d355740 1 -- start start 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.049+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.049+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5e1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.049+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.049+0000 7fcf3d355740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b38e0 con 0x55b9fb5ca3f0 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.049+0000 7fcf3d355740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b11b0 con 0x55b9fb5bfb30 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.049+0000 7fcf3d355740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341af000 con 0x55b9fb5bf210 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5e1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40572/0 (socket says 172.21.15.69:40572) 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf38cf3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 -- 172.21.15.69:0/721930069 learned_addr learned my addr 172.21.15.69:0/721930069 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf33fff640 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5e1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 -- 172.21.15.69:0/721930069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 msgr2=0x55b9fb5c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 -- 172.21.15.69:0/721930069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bfb30 msgr2=0x55b9fb5e1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5e1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 -- 172.21.15.69:0/721930069 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9fb5bb5d0 con 0x55b9fb5ca3f0 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf33fff640 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf394f4640 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5e1ed0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcf24011790 tx=0x7fcf2400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994f094a429c37d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf2a7fc640 1 -- 172.21.15.69:0/721930069 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24023070 con 0x55b9fb5ca3f0 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf2a7fc640 1 -- 172.21.15.69:0/721930069 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2401eb70 con 0x55b9fb5ca3f0 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.050+0000 7fcf2a7fc640 1 -- 172.21.15.69:0/721930069 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24017040 con 0x55b9fb5ca3f0 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- 172.21.15.69:0/721930069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5ca3f0 msgr2=0x55b9fb5e1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5e1ed0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fcf24011790 tx=0x7fcf2400b000 comp rx=0 tx=0).stop 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- 172.21.15.69:0/721930069 shutdown_connections 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5e1ed0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5e1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 --2- 172.21.15.69:0/721930069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- 172.21.15.69:0/721930069 >> 172.21.15.69:0/721930069 conn(0x55b9fb5bb7a0 msgr2=0x55b9fb5c8970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- 172.21.15.69:0/721930069 shutdown_connections 2024-08-02T17:59:58.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- 172.21.15.69:0/721930069 wait complete. 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- start start 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 Processor -- start 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341af670 con 0x55b9fb5bfb30 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341b0ae0 con 0x55b9fb5ca3f0 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341afd40 con 0x55b9fb5bf210 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.051+0000 7fcf3d355740 10 client.0 _ll_register_callbacks cb 0x7fcf1c097d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40576/0 (socket says 172.21.15.69:40576) 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 -- 172.21.15.69:0/2162871197 learned_addr learned my addr 172.21.15.69:0/2162871197 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf394f4640 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 -- 172.21.15.69:0/2162871197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 msgr2=0x55b9fb5d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 -- 172.21.15.69:0/2162871197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 msgr2=0x55b9fb5ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 -- 172.21.15.69:0/2162871197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9fb5cec40 con 0x55b9fb5bfb30 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf33fff640 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5cdeb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00cfd0 tx=0x7fcf1c018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df93e65f4fafe5a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf13fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf13fff640 1 -- 172.21.15.69:0/2162871197 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0a4730 con 0x55b9fb5bfb30 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf13fff640 1 -- 172.21.15.69:0/2162871197 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7fcf1c0a42a0 con 0x55b9fb5bfb30 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf3d355740 1 -- 172.21.15.69:0/2162871197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2401eb70 con 0x55b9fb5bfb30 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf13fff640 1 -- 172.21.15.69:0/2162871197 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf1c09b890 con 0x55b9fb5bfb30 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf3d355740 10 client.18723 fetch_fsmap learned FSMap version 26 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf3d355740 1 -- 172.21.15.69:0/2162871197 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b9fb6449d0 con 0x55b9fb5bfb30 2024-08-02T17:59:58.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.052+0000 7fcf13fff640 1 -- 172.21.15.69:0/2162871197 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2401eb70 con 0x55b9fb5bfb30 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf13fff640 1 -- 172.21.15.69:0/2162871197 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcf1c09afa0 con 0x55b9fb5bfb30 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf3d355740 10 client.18723 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf3d355740 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf3d355740 20 client.18723 populate_metadata read hostname 'smithi069' 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf3d355740 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b9fb644f90 0x55b9fb6473b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf3d355740 4 client.18723 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf3d355740 1 -- 172.21.15.69:0/2162871197 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b9fb52c400 con 0x55b9fb644f90 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b9fb644f90 0x55b9fb6473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf13fff640 10 client.18723 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf38cf3640 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b9fb644f90 0x55b9fb6473b0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e73604e77b987e server_cookie=62f352dd9dbefa45 in_seq=0 out_seq=0 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf13fff640 1 -- 172.21.15.69:0/2162871197 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b9fb52c400 con 0x55b9fb644f90 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf13fff640 10 client.18723 handle_command_reply: tid=0 2024-08-02T17:59:58.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.053+0000 7fcf3d355740 1 client.18723 shutdown 2024-08-02T17:59:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 -- 172.21.15.69:0/2162871197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 msgr2=0x55b9fb5cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5cdeb0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00cfd0 tx=0x7fcf1c018f90 comp rx=0 tx=0).stop 2024-08-02T17:59:58.075 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T17:59:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T17:59:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T17:59:58.075 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T17:59:58.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 -- 172.21.15.69:0/2162871197 shutdown_connections 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55b9fb5bfb30 0x55b9fb5cdeb0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55b9fb644f90 0x55b9fb6473b0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55b9fb5ca3f0 0x55b9fb5ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 --2- 172.21.15.69:0/2162871197 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b9fb5bf210 0x55b9fb5d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 -- 172.21.15.69:0/2162871197 >> 172.21.15.69:0/2162871197 conn(0x55b9fb5bb7a0 msgr2=0x55b9fb5c71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf13fff640 0 client.18723 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 -- 172.21.15.69:0/2162871197 shutdown_connections 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 1 -- 172.21.15.69:0/2162871197 wait complete. 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3d355740 20 client.18723 trim_cache size 0 max 16384 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 -- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcf200a2cf0 msgr2=0x7fcf200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcf200a2cf0 0x7fcf200a51b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcf24005d80 tx=0x7fcf24005d10 comp rx=0 tx=0).stop 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 -- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf34102250 msgr2=0x7fcf34074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf34102250 0x7fcf34074760 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00c920 tx=0x7fcf1c00cde0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 -- 172.21.15.69:0/2360288614 shutdown_connections 2024-08-02T17:59:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fcf34102250 0x7fcf34074760 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fcf200a2cf0 0x7fcf200a51b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf341078b0 0x7fcf34074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 --2- 172.21.15.69:0/2360288614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fcf341018b0 0x7fcf34074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 -- 172.21.15.69:0/2360288614 >> 172.21.15.69:0/2360288614 conn(0x7fcf340fd5f0 msgr2=0x7fcf340fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 -- 172.21.15.69:0/2360288614 shutdown_connections 2024-08-02T17:59:58.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:58.054+0000 7fcf3af57640 1 -- 172.21.15.69:0/2360288614 wait complete. 2024-08-02T17:59:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:57 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T17:59:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:57 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T17:59:58.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:57 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T17:59:58.969 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T17:59:58.970 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T17:59:59.228 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T17:59:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 17:59:59 smithi077 ceph-mon[37108]: pgmap v423: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 218 op/s 2024-08-02T17:59:59.349 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 17:59:59 smithi069 ceph-mon[32746]: pgmap v423: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 218 op/s 2024-08-02T17:59:59.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 17:59:59 smithi083 ceph-mon[37031]: pgmap v423: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 2.4 MiB/s wr, 218 op/s 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/3574803374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f981078b0 msgr2=0x7f3f98107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/3574803374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f981078b0 0x7f3f98107c90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009930 tx=0x7f3f8802a420 comp rx=0 tx=0).stop 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/3574803374 shutdown_connections 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/3574803374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f98102250 0x7f3f9810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/3574803374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f981018b0 0x7f3f98101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/3574803374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f981078b0 0x7f3f98107c90 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/3574803374 >> 172.21.15.69:0/3574803374 conn(0x7f3f980fd5f0 msgr2=0x7f3f980ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/3574803374 shutdown_connections 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/3574803374 wait complete. 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 Processor -- start 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.711+0000 7f3f9d9cb640 1 -- start start 2024-08-02T17:59:59.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9d9cb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f981018b0 0x7f3f98074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9d9cb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102250 0x7f3f98074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9d9cb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f981078b0 0x7f3f98074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9d9cb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98111cf0 con 0x7f3f98102250 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9d9cb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98111b70 con 0x7f3f981078b0 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9d9cb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98111e70 con 0x7f3f981018b0 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102250 0x7f3f98074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102250 0x7f3f98074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40592/0 (socket says 172.21.15.69:40592) 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 -- 172.21.15.69:0/274658878 learned_addr learned my addr 172.21.15.69:0/274658878 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f977fe640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f981078b0 0x7f3f98074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f96ffd640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f981018b0 0x7f3f98074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 -- 172.21.15.69:0/274658878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f981018b0 msgr2=0x7f3f98074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f981018b0 0x7f3f98074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 -- 172.21.15.69:0/274658878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f981078b0 msgr2=0x7f3f98074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f981078b0 0x7f3f98074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 -- 172.21.15.69:0/274658878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f981a6710 con 0x7f3f98102250 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f977fe640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f981078b0 0x7f3f98074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f8ffff640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102250 0x7f3f98074760 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b4a0 tx=0x7f3f8000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9c9c9640 1 -- 172.21.15.69:0/274658878 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80014070 con 0x7f3f98102250 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9c9c9640 1 -- 172.21.15.69:0/274658878 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f80010af0 con 0x7f3f98102250 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.712+0000 7f3f9c9c9640 1 -- 172.21.15.69:0/274658878 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f80017d10 con 0x7f3f98102250 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.713+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981a6a00 con 0x7f3f98102250 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.713+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f981a6f40 con 0x7f3f98102250 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.714+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f98101d10 con 0x7f3f98102250 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.715+0000 7f3f9c9c9640 1 -- 172.21.15.69:0/274658878 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3f80017070 con 0x7f3f98102250 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.716+0000 7f3f9c9c9640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3f840a2cf0 0x7f3f840a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T17:59:59.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.716+0000 7f3f96ffd640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3f840a2cf0 0x7f3f840a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T17:59:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.716+0000 7f3f9c9c9640 1 -- 172.21.15.69:0/274658878 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f3f800ceb70 con 0x7f3f98102250 2024-08-02T17:59:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.716+0000 7f3f96ffd640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3f840a2cf0 0x7f3f840a51b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3f88000c00 tx=0x7f3f88031490 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T17:59:59.736 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.720+0000 7f3f9c9c9640 1 -- 172.21.15.69:0/274658878 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f3f8009b490 con 0x7f3f98102250 2024-08-02T17:59:59.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.921+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f98109e20 con 0x7f3f98102250 2024-08-02T17:59:59.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.922+0000 7f3f9c9c9640 1 -- 172.21.15.69:0/274658878 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f3f8009ad90 con 0x7f3f98102250 2024-08-02T17:59:59.924 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T17:59:59.924 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T17:59:59.924 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3f840a2cf0 msgr2=0x7f3f840a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3f840a2cf0 0x7f3f840a51b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3f88000c00 tx=0x7f3f88031490 comp rx=0 tx=0).stop 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102250 msgr2=0x7f3f98074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102250 0x7f3f98074760 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000b4a0 tx=0x7f3f8000b960 comp rx=0 tx=0).stop 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 shutdown_connections 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3f98102250 0x7f3f98074760 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3f840a2cf0 0x7f3f840a51b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3f981078b0 0x7f3f98074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.926 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 --2- 172.21.15.69:0/274658878 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3f981018b0 0x7f3f98074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T17:59:59.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 >> 172.21.15.69:0/274658878 conn(0x7f3f980fd5f0 msgr2=0x7f3f980fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T17:59:59.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 shutdown_connections 2024-08-02T17:59:59.927 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T17:59:59.924+0000 7f3f9d9cb640 1 -- 172.21.15.69:0/274658878 wait complete. 2024-08-02T17:59:59.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 770.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T18:00:00.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:00 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/274658878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:00.266 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:00 smithi069 ceph-mon[32746]: overall HEALTH_OK 2024-08-02T18:00:00.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:00 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/274658878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:00.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:00 smithi077 ceph-mon[37108]: overall HEALTH_OK 2024-08-02T18:00:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:00 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/274658878' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:00.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:00 smithi083 ceph-mon[37031]: overall HEALTH_OK 2024-08-02T18:00:00.624 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T18:00:00.879 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:01.291 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:01 smithi069 ceph-mon[32746]: pgmap v424: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.2 MiB/s wr, 209 op/s 2024-08-02T18:00:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:01 smithi077 ceph-mon[37108]: pgmap v424: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.2 MiB/s wr, 209 op/s 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/4243556967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 msgr2=0x7f6a4810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/4243556967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 0x7f6a4810d680 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009930 tx=0x7f6a4002a420 comp rx=0 tx=0).stop 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/4243556967 shutdown_connections 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/4243556967 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 0x7f6a4810d680 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/4243556967 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a481007b0 0x7f6a48100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/4243556967 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a481067b0 0x7f6a48106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/4243556967 >> 172.21.15.69:0/4243556967 conn(0x7f6a480fc4d0 msgr2=0x7f6a480fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/4243556967 shutdown_connections 2024-08-02T18:00:01.404 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/4243556967 wait complete. 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.372+0000 7f6a4e2e6640 1 Processor -- start 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4e2e6640 1 -- start start 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4e2e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a481007b0 0x7f6a48198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4e2e6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 0x7f6a48199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4e2e6640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a481067b0 0x7f6a481a4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4e2e6640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48110bb0 con 0x7f6a48101150 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4e2e6640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48110a30 con 0x7f6a481067b0 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4e2e6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48110d30 con 0x7f6a481007b0 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 0x7f6a48199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 0x7f6a48199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40618/0 (socket says 172.21.15.69:40618) 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 -- 172.21.15.69:0/29583161 learned_addr learned my addr 172.21.15.69:0/29583161 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 -- 172.21.15.69:0/29583161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a481007b0 msgr2=0x7f6a48198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a481007b0 0x7f6a48198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4c883640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a481067b0 0x7f6a481a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 -- 172.21.15.69:0/29583161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a481067b0 msgr2=0x7f6a481a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a481067b0 0x7f6a481a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 -- 172.21.15.69:0/29583161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a481a4580 con 0x7f6a48101150 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a4c883640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a481067b0 0x7f6a481a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a477fe640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 0x7f6a48199500 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00d950 tx=0x7f6a3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a457fa640 1 -- 172.21.15.69:0/29583161 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c017070 con 0x7f6a48101150 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.373+0000 7f6a457fa640 1 -- 172.21.15.69:0/29583161 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3c009c30 con 0x7f6a48101150 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.374+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/29583161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a481a4750 con 0x7f6a48101150 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.374+0000 7f6a457fa640 1 -- 172.21.15.69:0/29583161 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c00f6b0 con 0x7f6a48101150 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.374+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/29583161 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a481a4c90 con 0x7f6a48101150 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a457fa640 1 -- 172.21.15.69:0/29583161 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6a3c0109d0 con 0x7f6a48101150 2024-08-02T18:00:01.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a506e4740 1 Processor -- start 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a506e4740 1 -- start start 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d9873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 0x556e3d991c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d9945c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a506e4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556e3d8eca40 con 0x556e3d986ff0 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a506e4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e3d8849d0 con 0x556e3d987910 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a506e4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e3d8e8a00 con 0x556e3d9921d0 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a4c883640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d9873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a4c883640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d9873d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40630/0 (socket says 172.21.15.69:40630) 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.376+0000 7f6a4c883640 1 -- 172.21.15.69:0/1401474226 learned_addr learned my addr 172.21.15.69:0/1401474226 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a457fa640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6a300a2d70 0x7f6a300a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a477fe640 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d9945c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a47fff640 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 0x556e3d991c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a4c883640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6a300a2d70 0x7f6a300a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a4c883640 1 -- 172.21.15.69:0/1401474226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 msgr2=0x556e3d9945c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a4c883640 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d9945c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a457fa640 1 -- 172.21.15.69:0/29583161 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f6a3c0cf500 con 0x7f6a48101150 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a4c883640 1 -- 172.21.15.69:0/1401474226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 msgr2=0x556e3d991c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a4c883640 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 0x556e3d991c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a4c883640 1 -- 172.21.15.69:0/1401474226 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e3d88b650 con 0x556e3d986ff0 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a4c883640 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d9873d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6a4002a940 tx=0x7f6a40031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba68cfd884aaaa7b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a4c883640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6a300a2d70 0x7f6a300a5230 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a40005c40 tx=0x7f6a40046000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a2e7fc640 1 -- 172.21.15.69:0/1401474226 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40035070 con 0x556e3d986ff0 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a2e7fc640 1 -- 172.21.15.69:0/1401474226 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a40005520 con 0x556e3d986ff0 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.377+0000 7f6a2e7fc640 1 -- 172.21.15.69:0/1401474226 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40003850 con 0x556e3d986ff0 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 -- 172.21.15.69:0/1401474226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 msgr2=0x556e3d9873d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d9873d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f6a4002a940 tx=0x7f6a40031980 comp rx=0 tx=0).stop 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 -- 172.21.15.69:0/1401474226 shutdown_connections 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d9945c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 0x556e3d991c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1401474226 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d9873d0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 -- 172.21.15.69:0/1401474226 >> 172.21.15.69:0/1401474226 conn(0x556e3d983580 msgr2=0x556e3d983960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 -- 172.21.15.69:0/1401474226 shutdown_connections 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 -- 172.21.15.69:0/1401474226 wait complete. 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 -- start start 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.378+0000 7f6a506e4740 1 Processor -- start 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d986ff0 0x556e3d98f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d98f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d9921d0 0x556e3d98fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a506e4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556e3d994da0 con 0x556e3d987910 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a506e4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556e3d8a7b20 con 0x556e3d9921d0 2024-08-02T18:00:01.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a506e4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556e3d994f20 con 0x556e3d986ff0 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a506e4740 10 client.0 _ll_register_callbacks cb 0x556e3d8a5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d98f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d98f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40634/0 (socket says 172.21.15.69:40634) 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 -- 172.21.15.69:0/1087250154 learned_addr learned my addr 172.21.15.69:0/1087250154 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a4c883640 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d9921d0 0x556e3d98fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a47fff640 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d986ff0 0x556e3d98f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 -- 172.21.15.69:0/1087250154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d986ff0 msgr2=0x556e3d98f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d986ff0 0x556e3d98f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 -- 172.21.15.69:0/1087250154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d9921d0 msgr2=0x556e3d98fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d9921d0 0x556e3d98fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 -- 172.21.15.69:0/1087250154 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e3d65e3c0 con 0x556e3d987910 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a4c883640 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d9921d0 0x556e3d98fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:01.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a477fe640 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d98f630 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c09b310 tx=0x7f6a3c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb086c51e018bf1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a277fe640 1 -- 172.21.15.69:0/1087250154 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c020040 con 0x556e3d987910 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a277fe640 1 -- 172.21.15.69:0/1087250154 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f6a3c0ccac0 con 0x556e3d987910 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.379+0000 7f6a506e4740 1 -- 172.21.15.69:0/1087250154 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a40005520 con 0x556e3d987910 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a277fe640 1 -- 172.21.15.69:0/1087250154 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a3c0cccc0 con 0x556e3d987910 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a506e4740 10 client.18741 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a506e4740 1 -- 172.21.15.69:0/1087250154 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e3d9a0a40 con 0x556e3d987910 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a277fe640 1 -- 172.21.15.69:0/1087250154 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a40005520 con 0x556e3d987910 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a277fe640 1 -- 172.21.15.69:0/1087250154 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6a3c084b50 con 0x556e3d987910 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a506e4740 10 client.18741 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a506e4740 10 client.18741 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a506e4740 20 client.18741 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x556e3d9a1f00 0x556e3d9a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a506e4740 4 client.18741 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a506e4740 1 -- 172.21.15.69:0/1087250154 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e3d8f9fb0 con 0x556e3d9a1f00 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.380+0000 7f6a47fff640 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x556e3d9a1f00 0x556e3d9a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.381+0000 7f6a277fe640 10 client.18741 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.381+0000 7f6a47fff640 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x556e3d9a1f00 0x556e3d9a42f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7848241caf570b02 server_cookie=e4e6ef59573f77a8 in_seq=0 out_seq=0 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.381+0000 7f6a277fe640 1 -- 172.21.15.69:0/1087250154 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556e3d8f9fb0 con 0x556e3d9a1f00 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.381+0000 7f6a277fe640 10 client.18741 handle_command_reply: tid=0 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.381+0000 7f6a506e4740 1 client.18741 shutdown 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.381+0000 7f6a506e4740 1 -- 172.21.15.69:0/1087250154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 msgr2=0x556e3d98f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.381+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d98f630 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c09b310 tx=0x7f6a3c010d70 comp rx=0 tx=0).stop 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 1 -- 172.21.15.69:0/1087250154 shutdown_connections 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x556e3d9a1f00 0x556e3d9a42f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d98f630 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d9921d0 0x556e3d98fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1087250154 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d986ff0 0x556e3d98f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a277fe640 0 client.18741 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 1 -- 172.21.15.69:0/1087250154 >> 172.21.15.69:0/1087250154 conn(0x556e3d983580 msgr2=0x556e3d9892b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 1 -- 172.21.15.69:0/1087250154 shutdown_connections 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 1 -- 172.21.15.69:0/1087250154 wait complete. 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.382+0000 7f6a506e4740 20 client.18741 trim_cache size 0 max 16384 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a506e4740 1 Processor -- start 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a506e4740 1 -- start start 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d986ff0 0x556e3d9a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d9aa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d98c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a506e4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481b1890 con 0x556e3d987910 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a506e4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481af160 con 0x556e3d986ff0 2024-08-02T18:00:01.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a506e4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481acfb0 con 0x556e3d9921d0 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d9aa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d9aa0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40640/0 (socket says 172.21.15.69:40640) 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 -- 172.21.15.69:0/256241182 learned_addr learned my addr 172.21.15.69:0/256241182 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 -- 172.21.15.69:0/256241182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 msgr2=0x556e3d98c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 --2- 172.21.15.69:0/256241182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d98c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 -- 172.21.15.69:0/256241182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d986ff0 msgr2=0x556e3d9a9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 --2- 172.21.15.69:0/256241182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d986ff0 0x556e3d9a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 -- 172.21.15.69:0/256241182 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e3d9a4660 con 0x556e3d987910 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a477fe640 1 --2- 172.21.15.69:0/256241182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d9aa0e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0978f0 tx=0x7f6a3c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42beda8174fc7759 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a2dffb640 1 -- 172.21.15.69:0/256241182 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0b3070 con 0x556e3d987910 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a2dffb640 1 -- 172.21.15.69:0/256241182 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3c014030 con 0x556e3d987910 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.395+0000 7f6a2dffb640 1 -- 172.21.15.69:0/256241182 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0ccac0 con 0x556e3d987910 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 -- 172.21.15.69:0/256241182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 msgr2=0x556e3d9aa0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.414 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 --2- 172.21.15.69:0/256241182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d9aa0e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c0978f0 tx=0x7f6a3c005f90 comp rx=0 tx=0).stop 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 -- 172.21.15.69:0/256241182 shutdown_connections 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 --2- 172.21.15.69:0/256241182 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d987910 0x556e3d9aa0e0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 --2- 172.21.15.69:0/256241182 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d98c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 --2- 172.21.15.69:0/256241182 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d986ff0 0x556e3d9a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 -- 172.21.15.69:0/256241182 >> 172.21.15.69:0/256241182 conn(0x556e3d983580 msgr2=0x556e3d98d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 -- 172.21.15.69:0/256241182 shutdown_connections 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 -- 172.21.15.69:0/256241182 wait complete. 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 -- start start 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.396+0000 7f6a506e4740 1 Processor -- start 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d99a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 0x556e3d98bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a506e4740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d98c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a506e4740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481ad620 con 0x556e3d986ff0 2024-08-02T18:00:01.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a506e4740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481aea90 con 0x556e3d987910 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a506e4740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6a481adcf0 con 0x556e3d9921d0 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a506e4740 10 client.0 _ll_register_callbacks cb 0x7f6a3c00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d99a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d99a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40650/0 (socket says 172.21.15.69:40650) 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 -- 172.21.15.69:0/1826356946 learned_addr learned my addr 172.21.15.69:0/1826356946 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a477fe640 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 0x556e3d98bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 -- 172.21.15.69:0/1826356946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 msgr2=0x556e3d98c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d98c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 -- 172.21.15.69:0/1826356946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 msgr2=0x556e3d98bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 0x556e3d98bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 -- 172.21.15.69:0/1826356946 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e3d98ccc0 con 0x556e3d986ff0 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a47fff640 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d99a970 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x556e3d9843e0 tx=0x7f6a380128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3244753f930ac3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:01.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a277fe640 1 -- 172.21.15.69:0/1826356946 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3801e020 con 0x556e3d986ff0 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a277fe640 1 -- 172.21.15.69:0/1826356946 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f6a38004050 con 0x556e3d986ff0 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.397+0000 7f6a506e4740 1 -- 172.21.15.69:0/1826356946 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a3c014030 con 0x556e3d986ff0 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a277fe640 1 -- 172.21.15.69:0/1826356946 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6a38029050 con 0x556e3d986ff0 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a506e4740 10 client.18747 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a506e4740 1 -- 172.21.15.69:0/1826356946 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e3da0cc70 con 0x556e3d986ff0 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a277fe640 1 -- 172.21.15.69:0/1826356946 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a3c014030 con 0x556e3d986ff0 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a277fe640 1 -- 172.21.15.69:0/1826356946 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6a3800f040 con 0x556e3d986ff0 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a506e4740 10 client.18747 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a506e4740 10 client.18747 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a506e4740 20 client.18747 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x556e3da0d420 0x556e3da0f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:01.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a506e4740 4 client.18747 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a506e4740 1 -- 172.21.15.69:0/1826356946 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e3d8f44c0 con 0x556e3da0d420 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.398+0000 7f6a477fe640 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x556e3da0d420 0x556e3da0f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.399+0000 7f6a277fe640 10 client.18747 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.399+0000 7f6a477fe640 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x556e3da0d420 0x556e3da0f840 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5c0ec5582b77cf6 server_cookie=ea178fe03972f901 in_seq=0 out_seq=0 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a277fe640 1 -- 172.21.15.69:0/1826356946 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e3d8f44c0 con 0x556e3da0d420 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a277fe640 10 client.18747 handle_command_reply: tid=0 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 client.18747 shutdown 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 -- 172.21.15.69:0/1826356946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 msgr2=0x556e3d99a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d99a970 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x556e3d9843e0 tx=0x7f6a380128e0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 -- 172.21.15.69:0/1826356946 shutdown_connections 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x556e3da0d420 0x556e3da0f840 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556e3d9921d0 0x556e3d98c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556e3d987910 0x556e3d98bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 --2- 172.21.15.69:0/1826356946 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556e3d986ff0 0x556e3d99a970 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 -- 172.21.15.69:0/1826356946 >> 172.21.15.69:0/1826356946 conn(0x556e3d983580 msgr2=0x556e3d988a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:01.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a277fe640 0 client.18747 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:01.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 -- 172.21.15.69:0/1826356946 shutdown_connections 2024-08-02T18:00:01.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 1 -- 172.21.15.69:0/1826356946 wait complete. 2024-08-02T18:00:01.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a506e4740 20 client.18747 trim_cache size 0 max 16384 2024-08-02T18:00:01.419 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:01.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6a300a2d70 msgr2=0x7f6a300a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6a300a2d70 0x7f6a300a5230 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a40005c40 tx=0x7f6a40046000 comp rx=0 tx=0).stop 2024-08-02T18:00:01.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 msgr2=0x7f6a48199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:01.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.400+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 0x7f6a48199500 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00d950 tx=0x7f6a3c00de10 comp rx=0 tx=0).stop 2024-08-02T18:00:01.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.401+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/29583161 shutdown_connections 2024-08-02T18:00:01.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.401+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6a48101150 0x7f6a48199500 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.401+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6a300a2d70 0x7f6a300a5230 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.401+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6a481067b0 0x7f6a481a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.401+0000 7f6a4e2e6640 1 --2- 172.21.15.69:0/29583161 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6a481007b0 0x7f6a48198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:01.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.401+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/29583161 >> 172.21.15.69:0/29583161 conn(0x7f6a480fc4d0 msgr2=0x7f6a480fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:01.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.401+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/29583161 shutdown_connections 2024-08-02T18:00:01.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:01.401+0000 7f6a4e2e6640 1 -- 172.21.15.69:0/29583161 wait complete. 2024-08-02T18:00:01.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:01 smithi083 ceph-mon[37031]: pgmap v424: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.2 MiB/s wr, 209 op/s 2024-08-02T18:00:02.249 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T18:00:02.503 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 -- 172.21.15.69:0/3432177918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30281078b0 msgr2=0x7f3028107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 --2- 172.21.15.69:0/3432177918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30281078b0 0x7f3028107c90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f3018009930 tx=0x7f301802a420 comp rx=0 tx=0).stop 2024-08-02T18:00:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 -- 172.21.15.69:0/3432177918 shutdown_connections 2024-08-02T18:00:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 --2- 172.21.15.69:0/3432177918 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3028102250 0x7f302810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 --2- 172.21.15.69:0/3432177918 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30281018b0 0x7f3028101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 --2- 172.21.15.69:0/3432177918 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f30281078b0 0x7f3028107c90 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 -- 172.21.15.69:0/3432177918 >> 172.21.15.69:0/3432177918 conn(0x7f30280fd5f0 msgr2=0x7f30280ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 -- 172.21.15.69:0/3432177918 shutdown_connections 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.996+0000 7f302d66a640 1 -- 172.21.15.69:0/3432177918 wait complete. 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f302d66a640 1 Processor -- start 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f302d66a640 1 -- start start 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f302d66a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30281018b0 0x7f3028074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f302d66a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028102250 0x7f3028074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f302d66a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30281078b0 0x7f3028074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f302d66a640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3028111d50 con 0x7f3028102250 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f302d66a640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3028111bd0 con 0x7f30281078b0 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f302d66a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3028111ed0 con 0x7f30281018b0 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f301ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028102250 0x7f3028074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f301ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028102250 0x7f3028074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40664/0 (socket says 172.21.15.69:40664) 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f301ffff640 1 -- 172.21.15.69:0/583752794 learned_addr learned my addr 172.21.15.69:0/583752794 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f30277fe640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30281078b0 0x7f3028074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f3026ffd640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30281018b0 0x7f3028074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f301ffff640 1 -- 172.21.15.69:0/583752794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30281018b0 msgr2=0x7f3028074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f301ffff640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30281018b0 0x7f3028074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f301ffff640 1 -- 172.21.15.69:0/583752794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30281078b0 msgr2=0x7f3028074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f301ffff640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30281078b0 0x7f3028074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.997+0000 7f301ffff640 1 -- 172.21.15.69:0/583752794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30281a6810 con 0x7f3028102250 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.998+0000 7f30277fe640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30281078b0 0x7f3028074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.998+0000 7f3026ffd640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30281018b0 0x7f3028074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.998+0000 7f301ffff640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028102250 0x7f3028074760 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f301000b4a0 tx=0x7f301000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.998+0000 7f3024ff9640 1 -- 172.21.15.69:0/583752794 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010014070 con 0x7f3028102250 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.998+0000 7f302d66a640 1 -- 172.21.15.69:0/583752794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30281a6b00 con 0x7f3028102250 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.998+0000 7f3024ff9640 1 -- 172.21.15.69:0/583752794 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3010010af0 con 0x7f3028102250 2024-08-02T18:00:03.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.998+0000 7f3024ff9640 1 -- 172.21.15.69:0/583752794 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010017d10 con 0x7f3028102250 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:02.998+0000 7f302d66a640 1 -- 172.21.15.69:0/583752794 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30281a7040 con 0x7f3028102250 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.000+0000 7f3024ff9640 1 -- 172.21.15.69:0/583752794 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3010017070 con 0x7f3028102250 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.000+0000 7f302fa68740 1 Processor -- start 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.000+0000 7f302fa68740 1 -- start start 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f302fa68740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f302fa68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73dcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f302fa68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73dd1d0 0x55e5a73df5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f302fa68740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e5a7337a40 con 0x55e5a73d1ff0 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f302fa68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e5a72cf9d0 con 0x55e5a73d2910 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f302fa68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5a7333a00 con 0x55e5a73dd1d0 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73d23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40672/0 (socket says 172.21.15.69:40672) 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 -- 172.21.15.69:0/801357101 learned_addr learned my addr 172.21.15.69:0/801357101 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f3024ff9640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30140a2cf0 0x7f30140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f301ffff640 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73dd1d0 0x55e5a73df5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f3026ffd640 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73dcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 -- 172.21.15.69:0/801357101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73dd1d0 msgr2=0x55e5a73df5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73dd1d0 0x55e5a73df5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 -- 172.21.15.69:0/801357101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 msgr2=0x55e5a73dcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73dcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 -- 172.21.15.69:0/801357101 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a72d6650 con 0x55e5a73d1ff0 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f3024ff9640 1 -- 172.21.15.69:0/583752794 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f30100ceb70 con 0x7f3028102250 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30140a2cf0 0x7f30140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.001+0000 7f30277fe640 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73d23d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f302000c370 tx=0x7f302000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbeec01f7f705c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:03.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f301d7fa640 1 -- 172.21.15.69:0/801357101 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020016070 con 0x55e5a73d1ff0 2024-08-02T18:00:03.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f301d7fa640 1 -- 172.21.15.69:0/801357101 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3020011d00 con 0x55e5a73d1ff0 2024-08-02T18:00:03.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f301d7fa640 1 -- 172.21.15.69:0/801357101 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302000fbf0 con 0x55e5a73d1ff0 2024-08-02T18:00:03.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f30277fe640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30140a2cf0 0x7f30140a51b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f302001d000 tx=0x7f302001e000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:03.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f302fa68740 1 -- 172.21.15.69:0/801357101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 msgr2=0x55e5a73d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f302fa68740 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73d23d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f302000c370 tx=0x7f302000c830 comp rx=0 tx=0).stop 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f302fa68740 1 -- 172.21.15.69:0/801357101 shutdown_connections 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f302fa68740 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73dd1d0 0x55e5a73df5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f302fa68740 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73dcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f302fa68740 1 --2- 172.21.15.69:0/801357101 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73d23d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f302fa68740 1 -- 172.21.15.69:0/801357101 >> 172.21.15.69:0/801357101 conn(0x55e5a73ce580 msgr2=0x55e5a73ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.002+0000 7f302fa68740 1 -- 172.21.15.69:0/801357101 shutdown_connections 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 -- 172.21.15.69:0/801357101 wait complete. 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 -- start start 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 Processor -- start 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 0x55e5a73da0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73da5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73dd1d0 0x55e5a73dab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55e5a73dfda0 con 0x55e5a73d2910 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55e5a72f2b20 con 0x55e5a73dd1d0 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e5a73dff20 con 0x55e5a73d1ff0 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f302fa68740 10 client.0 _ll_register_callbacks cb 0x55e5a72f0bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f301ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73da5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f301ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73da5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40676/0 (socket says 172.21.15.69:40676) 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f301ffff640 1 -- 172.21.15.69:0/2111346097 learned_addr learned my addr 172.21.15.69:0/2111346097 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.003+0000 7f3026ffd640 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 0x55e5a73da0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f30277fe640 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73dd1d0 0x55e5a73dab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f301ffff640 1 -- 172.21.15.69:0/2111346097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 msgr2=0x55e5a73da0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f301ffff640 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 0x55e5a73da0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f301ffff640 1 -- 172.21.15.69:0/2111346097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73dd1d0 msgr2=0x55e5a73dab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f301ffff640 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73dd1d0 0x55e5a73dab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f301ffff640 1 -- 172.21.15.69:0/2111346097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a70a93c0 con 0x55e5a73d2910 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f301ffff640 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73da5e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f30100cc020 tx=0x7f301009b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbfe45cbb53b031e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f300e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f300e7fc640 1 -- 172.21.15.69:0/2111346097 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301001e040 con 0x55e5a73d2910 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f300e7fc640 1 -- 172.21.15.69:0/2111346097 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f3010097400 con 0x55e5a73d2910 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f302fa68740 1 -- 172.21.15.69:0/2111346097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3020011d00 con 0x55e5a73d2910 2024-08-02T18:00:03.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f300e7fc640 1 -- 172.21.15.69:0/2111346097 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f301009ad90 con 0x55e5a73d2910 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f302fa68740 10 client.18759 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f302fa68740 1 -- 172.21.15.69:0/2111346097 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5a73eb9a0 con 0x55e5a73d2910 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f300e7fc640 1 -- 172.21.15.69:0/2111346097 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3020011d00 con 0x55e5a73d2910 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f300e7fc640 1 -- 172.21.15.69:0/2111346097 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f301009a300 con 0x55e5a73d2910 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f302fa68740 10 client.18759 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.004+0000 7f302fa68740 10 client.18759 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.005+0000 7f302fa68740 20 client.18759 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.005+0000 7f302fa68740 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e5a73ecdb0 0x55e5a73ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.005+0000 7f302fa68740 4 client.18759 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.005+0000 7f302fa68740 1 -- 172.21.15.69:0/2111346097 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5a7344fb0 con 0x55e5a73ecdb0 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.005+0000 7f3026ffd640 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e5a73ecdb0 0x55e5a73ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.005+0000 7f3026ffd640 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e5a73ecdb0 0x55e5a73ef1a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c0908b33a130bb server_cookie=e0e94826635e05d2 in_seq=0 out_seq=0 2024-08-02T18:00:03.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.005+0000 7f300e7fc640 10 client.18759 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f300e7fc640 1 -- 172.21.15.69:0/2111346097 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e5a7344fb0 con 0x55e5a73ecdb0 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f300e7fc640 10 client.18759 handle_command_reply: tid=0 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 client.18759 shutdown 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 -- 172.21.15.69:0/2111346097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 msgr2=0x55e5a73da5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73da5e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f30100cc020 tx=0x7f301009b550 comp rx=0 tx=0).stop 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 -- 172.21.15.69:0/2111346097 shutdown_connections 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e5a73ecdb0 0x55e5a73ef1a0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73da5e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73dd1d0 0x55e5a73dab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f300e7fc640 0 client.18759 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 --2- 172.21.15.69:0/2111346097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 0x55e5a73da0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 -- 172.21.15.69:0/2111346097 >> 172.21.15.69:0/2111346097 conn(0x55e5a73ce580 msgr2=0x55e5a73d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 -- 172.21.15.69:0/2111346097 shutdown_connections 2024-08-02T18:00:03.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 1 -- 172.21.15.69:0/2111346097 wait complete. 2024-08-02T18:00:03.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.006+0000 7f302fa68740 20 client.18759 trim_cache size 0 max 16384 2024-08-02T18:00:03.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.019+0000 7f302fa68740 1 Processor -- start 2024-08-02T18:00:03.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.019+0000 7f302fa68740 1 -- start start 2024-08-02T18:00:03.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f302fa68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73ce580 0x55e5a73d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f302fa68740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f302fa68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f302fa68740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30281b3b70 con 0x55e5a73d1ff0 2024-08-02T18:00:03.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f302fa68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30281b1440 con 0x55e5a73d2910 2024-08-02T18:00:03.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f302fa68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30281af290 con 0x55e5a73ce580 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73f4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40684/0 (socket says 172.21.15.69:40684) 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 -- 172.21.15.69:0/762955732 learned_addr learned my addr 172.21.15.69:0/762955732 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f3026ffd640 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73ce580 0x55e5a73d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f30277fe640 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 -- 172.21.15.69:0/762955732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73ce580 msgr2=0x55e5a73d9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73ce580 0x55e5a73d9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 -- 172.21.15.69:0/762955732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 msgr2=0x55e5a73f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 -- 172.21.15.69:0/762955732 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a73d85f0 con 0x55e5a73d1ff0 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f30277fe640 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:03.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301ffff640 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73f4a70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3010099040 tx=0x7f30100047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53168df20f2939c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301cff9640 1 -- 172.21.15.69:0/762955732 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010097400 con 0x55e5a73d1ff0 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301cff9640 1 -- 172.21.15.69:0/762955732 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30100cc0a0 con 0x55e5a73d1ff0 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.020+0000 7f301cff9640 1 -- 172.21.15.69:0/762955732 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010099f70 con 0x55e5a73d1ff0 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 -- 172.21.15.69:0/762955732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 msgr2=0x55e5a73f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73f4a70 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3010099040 tx=0x7f30100047d0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 -- 172.21.15.69:0/762955732 shutdown_connections 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d1ff0 0x55e5a73f4a70 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73d2910 0x55e5a73f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 --2- 172.21.15.69:0/762955732 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73ce580 0x55e5a73d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 -- 172.21.15.69:0/762955732 >> 172.21.15.69:0/762955732 conn(0x55e5a73d5a90 msgr2=0x55e5a73cd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 -- 172.21.15.69:0/762955732 shutdown_connections 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 -- 172.21.15.69:0/762955732 wait complete. 2024-08-02T18:00:03.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.021+0000 7f302fa68740 1 -- start start 2024-08-02T18:00:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f302fa68740 1 Processor -- start 2024-08-02T18:00:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f302fa68740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73ce580 0x55e5a73e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f302fa68740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 0x55e5a73e0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f302fa68740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f302fa68740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f30281af900 con 0x55e5a73d2910 2024-08-02T18:00:03.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f302fa68740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f30281b0d70 con 0x55e5a73ce580 2024-08-02T18:00:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f302fa68740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f30281affd0 con 0x55e5a73d1ff0 2024-08-02T18:00:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f302fa68740 10 client.0 _ll_register_callbacks cb 0x7f301009a750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73e0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40698/0 (socket says 172.21.15.69:40698) 2024-08-02T18:00:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 -- 172.21.15.69:0/2453186980 learned_addr learned my addr 172.21.15.69:0/2453186980 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f301ffff640 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 0x55e5a73e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f3026ffd640 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73ce580 0x55e5a73e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 -- 172.21.15.69:0/2453186980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 msgr2=0x55e5a73e0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 0x55e5a73e0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 -- 172.21.15.69:0/2453186980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73ce580 msgr2=0x55e5a73e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73ce580 0x55e5a73e55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 -- 172.21.15.69:0/2453186980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a73e1380 con 0x55e5a73d2910 2024-08-02T18:00:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f30277fe640 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73e0e40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f302000c370 tx=0x7f302000fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3579c938c767878 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.022+0000 7f300e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:03.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f300e7fc640 1 -- 172.21.15.69:0/2453186980 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3020016030 con 0x55e5a73d2910 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f300e7fc640 1 -- 172.21.15.69:0/2453186980 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f3020011a30 con 0x55e5a73d2910 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 1 -- 172.21.15.69:0/2453186980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30100cc0a0 con 0x55e5a73d2910 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f300e7fc640 1 -- 172.21.15.69:0/2453186980 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3020010600 con 0x55e5a73d2910 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 10 client.18765 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 1 -- 172.21.15.69:0/2453186980 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5a74576c0 con 0x55e5a73d2910 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f300e7fc640 1 -- 172.21.15.69:0/2453186980 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30100cc0a0 con 0x55e5a73d2910 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f300e7fc640 1 -- 172.21.15.69:0/2453186980 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f302001a070 con 0x55e5a73d2910 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 10 client.18765 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 10 client.18765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:00:03.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 20 client.18765 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e5a73dd1d0 0x55e5a7459cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 4 client.18765 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f302fa68740 1 -- 172.21.15.69:0/2453186980 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5a733f4c0 con 0x55e5a73dd1d0 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.023+0000 7f3026ffd640 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e5a73dd1d0 0x55e5a7459cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f300e7fc640 10 client.18765 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f3026ffd640 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e5a73dd1d0 0x55e5a7459cb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=600f2673050fa105 server_cookie=8f3488a9c720dd94 in_seq=0 out_seq=0 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f300e7fc640 1 -- 172.21.15.69:0/2453186980 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5a733f4c0 con 0x55e5a73dd1d0 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f300e7fc640 10 client.18765 handle_command_reply: tid=0 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 client.18765 shutdown 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 -- 172.21.15.69:0/2453186980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 msgr2=0x55e5a73e0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73e0e40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f302000c370 tx=0x7f302000fb00 comp rx=0 tx=0).stop 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 -- 172.21.15.69:0/2453186980 shutdown_connections 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55e5a73dd1d0 0x55e5a7459cb0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55e5a73d2910 0x55e5a73e0e40 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e5a73d1ff0 0x55e5a73e0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 --2- 172.21.15.69:0/2453186980 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55e5a73ce580 0x55e5a73e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f300e7fc640 0 client.18765 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 -- 172.21.15.69:0/2453186980 >> 172.21.15.69:0/2453186980 conn(0x55e5a73d5a90 msgr2=0x55e5a73d8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 -- 172.21.15.69:0/2453186980 shutdown_connections 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 1 -- 172.21.15.69:0/2453186980 wait complete. 2024-08-02T18:00:03.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.024+0000 7f302fa68740 20 client.18765 trim_cache size 0 max 16384 2024-08-02T18:00:03.043 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:03.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 -- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30140a2cf0 msgr2=0x7f30140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30140a2cf0 0x7f30140a51b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f302001d000 tx=0x7f302001e000 comp rx=0 tx=0).stop 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 -- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028102250 msgr2=0x7f3028074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028102250 0x7f3028074760 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f301000b4a0 tx=0x7f301000b960 comp rx=0 tx=0).stop 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 -- 172.21.15.69:0/583752794 shutdown_connections 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3028102250 0x7f3028074760 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f30140a2cf0 0x7f30140a51b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f30281078b0 0x7f3028074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 --2- 172.21.15.69:0/583752794 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f30281018b0 0x7f3028074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 -- 172.21.15.69:0/583752794 >> 172.21.15.69:0/583752794 conn(0x7f30280fd5f0 msgr2=0x7f30280fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 -- 172.21.15.69:0/583752794 shutdown_connections 2024-08-02T18:00:03.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:03.025+0000 7f302d66a640 1 -- 172.21.15.69:0/583752794 wait complete. 2024-08-02T18:00:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:03 smithi077 ceph-mon[37108]: pgmap v425: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-08-02T18:00:03.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:03 smithi069 ceph-mon[32746]: pgmap v425: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-08-02T18:00:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:03 smithi083 ceph-mon[37031]: pgmap v425: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-08-02T18:00:03.792 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T18:00:04.048 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:04.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- 172.21.15.69:0/3877897373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1078e0 msgr2=0x7f51dc107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 --2- 172.21.15.69:0/3877897373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1078e0 0x7f51dc107cc0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009930 tx=0x7f51d802a420 comp rx=0 tx=0).stop 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- 172.21.15.69:0/3877897373 shutdown_connections 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 --2- 172.21.15.69:0/3877897373 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51dc102280 0x7f51dc10e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 --2- 172.21.15.69:0/3877897373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51dc1018e0 0x7f51dc101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 --2- 172.21.15.69:0/3877897373 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1078e0 0x7f51dc107cc0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- 172.21.15.69:0/3877897373 >> 172.21.15.69:0/3877897373 conn(0x7f51dc0fd600 msgr2=0x7f51dc0ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- 172.21.15.69:0/3877897373 shutdown_connections 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- 172.21.15.69:0/3877897373 wait complete. 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 Processor -- start 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- start start 2024-08-02T18:00:04.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1018e0 0x7f51dc198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51dc102280 0x7f51dc1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51dc1078e0 0x7f51dc19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc111ca0 con 0x7f51dc1018e0 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc111b20 con 0x7f51dc102280 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.524+0000 7f51e42c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc111e20 con 0x7f51dc1078e0 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1018e0 0x7f51dc198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1018e0 0x7f51dc198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40718/0 (socket says 172.21.15.69:40718) 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 -- 172.21.15.69:0/1451056184 learned_addr learned my addr 172.21.15.69:0/1451056184 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e285e640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51dc1078e0 0x7f51dc19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e185c640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51dc102280 0x7f51dc1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 -- 172.21.15.69:0/1451056184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51dc1078e0 msgr2=0x7f51dc19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51dc1078e0 0x7f51dc19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 -- 172.21.15.69:0/1451056184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51dc102280 msgr2=0x7f51dc1986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51dc102280 0x7f51dc1986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 -- 172.21.15.69:0/1451056184 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc19d110 con 0x7f51dc1018e0 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e185c640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51dc102280 0x7f51dc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e205d640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1018e0 0x7f51dc198160 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009a50 tx=0x7f51d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51d37fe640 1 -- 172.21.15.69:0/1451056184 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d803f030 con 0x7f51dc1018e0 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51d37fe640 1 -- 172.21.15.69:0/1451056184 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d8003ce0 con 0x7f51dc1018e0 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e42c1640 1 -- 172.21.15.69:0/1451056184 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc19d3a0 con 0x7f51dc1018e0 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51d37fe640 1 -- 172.21.15.69:0/1451056184 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d8035070 con 0x7f51dc1018e0 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.525+0000 7f51e42c1640 1 -- 172.21.15.69:0/1451056184 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc1a4ce0 con 0x7f51dc1018e0 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51d37fe640 1 -- 172.21.15.69:0/1451056184 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f51d8042030 con 0x7f51dc1018e0 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e66bf740 1 Processor -- start 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e66bf740 1 -- start start 2024-08-02T18:00:04.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 0x55c2348263d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234830c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2348311d0 0x55c2348335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e66bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c23478ba40 con 0x55c2348311d0 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e66bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2347239d0 con 0x55c234825ff0 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e66bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c234787a00 con 0x55c234826910 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51d37fe640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51c80a2cf0 0x7f51c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e185c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2348311d0 0x55c2348335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e185c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2348311d0 0x55c2348335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40726/0 (socket says 172.21.15.69:40726) 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.528+0000 7f51e185c640 1 -- 172.21.15.69:0/1258292054 learned_addr learned my addr 172.21.15.69:0/1258292054 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e205d640 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234830c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e285e640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51c80a2cf0 0x7f51c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e285e640 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 0x55c2348263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51d37fe640 1 -- 172.21.15.69:0/1451056184 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f51d80eff10 con 0x7f51dc1018e0 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e185c640 1 -- 172.21.15.69:0/1258292054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 msgr2=0x55c234830c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e185c640 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234830c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e185c640 1 -- 172.21.15.69:0/1258292054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 msgr2=0x55c2348263d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e185c640 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 0x55c2348263d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e185c640 1 -- 172.21.15.69:0/1258292054 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c23472a650 con 0x55c2348311d0 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e205d640 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234830c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e185c640 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2348311d0 0x55c2348335c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00bbe0 tx=0x7f51cc00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b2995ede1d636d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51e285e640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51c80a2cf0 0x7f51c80a51b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51d4005cc0 tx=0x7f51d400a340 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51d0dea640 1 -- 172.21.15.69:0/1258292054 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc017620 con 0x55c2348311d0 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51d0dea640 1 -- 172.21.15.69:0/1258292054 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51cc017ca0 con 0x55c2348311d0 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.529+0000 7f51d0dea640 1 -- 172.21.15.69:0/1258292054 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00fd70 con 0x55c2348311d0 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 -- 172.21.15.69:0/1258292054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2348311d0 msgr2=0x55c2348335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2348311d0 0x55c2348335c0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00bbe0 tx=0x7f51cc00d710 comp rx=0 tx=0).stop 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 -- 172.21.15.69:0/1258292054 shutdown_connections 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2348311d0 0x55c2348335c0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234830c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 --2- 172.21.15.69:0/1258292054 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 0x55c2348263d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 -- 172.21.15.69:0/1258292054 >> 172.21.15.69:0/1258292054 conn(0x55c234822580 msgr2=0x55c234822960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 -- 172.21.15.69:0/1258292054 shutdown_connections 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 -- 172.21.15.69:0/1258292054 wait complete. 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 -- start start 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.530+0000 7f51e66bf740 1 Processor -- start 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 0x55c23482e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234826910 0x55c23482e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2348311d0 0x55c23482eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e66bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c234833da0 con 0x55c234826910 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e66bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c234746b20 con 0x55c234825ff0 2024-08-02T18:00:04.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e66bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c234833f20 con 0x55c2348311d0 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e66bf740 10 client.0 _ll_register_callbacks cb 0x55c234744bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234826910 0x55c23482e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234826910 0x55c23482e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40728/0 (socket says 172.21.15.69:40728) 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 -- 172.21.15.69:0/3315801338 learned_addr learned my addr 172.21.15.69:0/3315801338 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e205d640 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 0x55c23482e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e285e640 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2348311d0 0x55c23482eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 -- 172.21.15.69:0/3315801338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2348311d0 msgr2=0x55c23482eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2348311d0 0x55c23482eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 -- 172.21.15.69:0/3315801338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 msgr2=0x55c23482e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 0x55c23482e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 -- 172.21.15.69:0/3315801338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2344fd3c0 con 0x55c234826910 2024-08-02T18:00:04.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e285e640 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2348311d0 0x55c23482eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51e185c640 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234826910 0x55c23482e5c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00b390 tx=0x7f51cc00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51f0ceb9e99fbbb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.531+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3315801338 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc026020 con 0x55c234826910 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3315801338 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f51cc017630 con 0x55c234826910 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 1 -- 172.21.15.69:0/3315801338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51cc017ca0 con 0x55c234826910 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3315801338 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51cc031050 con 0x55c234826910 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 10 client.18777 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 1 -- 172.21.15.69:0/3315801338 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c23483f970 con 0x55c234826910 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3315801338 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51cc017ca0 con 0x55c234826910 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3315801338 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51cc021930 con 0x55c234826910 2024-08-02T18:00:04.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 10 client.18777 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 10 client.18777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 20 client.18777 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c234840dd0 0x55c2348431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 4 client.18777 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e66bf740 1 -- 172.21.15.69:0/3315801338 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c234798fb0 con 0x55c234840dd0 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.532+0000 7f51e205d640 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c234840dd0 0x55c2348431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.533+0000 7f51e205d640 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c234840dd0 0x55c2348431c0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d57fe83a2ae94ea server_cookie=a4d43b82bfcfb17c in_seq=0 out_seq=0 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.533+0000 7f51c5ffb640 10 client.18777 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:04.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.533+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3315801338 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c234798fb0 con 0x55c234840dd0 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.533+0000 7f51c5ffb640 10 client.18777 handle_command_reply: tid=0 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.533+0000 7f51e66bf740 1 client.18777 shutdown 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.533+0000 7f51e66bf740 1 -- 172.21.15.69:0/3315801338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234826910 msgr2=0x55c23482e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.533+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234826910 0x55c23482e5c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00b390 tx=0x7f51cc00bf50 comp rx=0 tx=0).stop 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 1 -- 172.21.15.69:0/3315801338 shutdown_connections 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234826910 0x55c23482e5c0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c234840dd0 0x55c2348431c0 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2348311d0 0x55c23482eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51c5ffb640 0 client.18777 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3315801338 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c234825ff0 0x55c23482e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 1 -- 172.21.15.69:0/3315801338 >> 172.21.15.69:0/3315801338 conn(0x55c234822580 msgr2=0x55c234828260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 1 -- 172.21.15.69:0/3315801338 shutdown_connections 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 1 -- 172.21.15.69:0/3315801338 wait complete. 2024-08-02T18:00:04.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.534+0000 7f51e66bf740 20 client.18777 trim_cache size 0 max 16384 2024-08-02T18:00:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e66bf740 1 Processor -- start 2024-08-02T18:00:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e66bf740 1 -- start start 2024-08-02T18:00:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c234821aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234848820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 0x55c234848d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.549 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e66bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1b1780 con 0x55c234825ff0 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e66bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1af050 con 0x55c2348311d0 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e66bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1acea0 con 0x55c234826910 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c234821aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c234821aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40742/0 (socket says 172.21.15.69:40742) 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 -- 172.21.15.69:0/2195917414 learned_addr learned my addr 172.21.15.69:0/2195917414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e185c640 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234848820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 -- 172.21.15.69:0/2195917414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 msgr2=0x55c234848820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234848820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 -- 172.21.15.69:0/2195917414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 msgr2=0x55c234848d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e285e640 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 0x55c234848d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 0x55c234848d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 -- 172.21.15.69:0/2195917414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c234830b30 con 0x55c234825ff0 2024-08-02T18:00:04.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e185c640 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234848820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e285e640 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 0x55c234848d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.547+0000 7f51e205d640 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c234821aa0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f51d80bbf70 tx=0x7f51d80dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15a05e6950e0f5c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51c7fff640 1 -- 172.21.15.69:0/2195917414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80bbfe0 con 0x55c234825ff0 2024-08-02T18:00:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51c7fff640 1 -- 172.21.15.69:0/2195917414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d80c0390 con 0x55c234825ff0 2024-08-02T18:00:04.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51c7fff640 1 -- 172.21.15.69:0/2195917414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d803b460 con 0x55c234825ff0 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 -- 172.21.15.69:0/2195917414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 msgr2=0x55c234821aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c234821aa0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f51d80bbf70 tx=0x7f51d80dc040 comp rx=0 tx=0).stop 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 -- 172.21.15.69:0/2195917414 shutdown_connections 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 0x55c234848d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234848820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 --2- 172.21.15.69:0/2195917414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c234821aa0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 -- 172.21.15.69:0/2195917414 >> 172.21.15.69:0/2195917414 conn(0x55c234822580 msgr2=0x55c23482fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 -- 172.21.15.69:0/2195917414 shutdown_connections 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.548+0000 7f51e66bf740 1 -- 172.21.15.69:0/2195917414 wait complete. 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 1 -- start start 2024-08-02T18:00:04.555 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 1 Processor -- start 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c2348346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234834c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 0x55c2348a97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1ad510 con 0x55c234825ff0 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1ae980 con 0x55c2348311d0 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc1adbe0 con 0x55c234826910 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e66bf740 10 client.0 _ll_register_callbacks cb 0x7f51cc00ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e205d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c2348346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e205d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c2348346d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:40746/0 (socket says 172.21.15.69:40746) 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e205d640 1 -- 172.21.15.69:0/3404883075 learned_addr learned my addr 172.21.15.69:0/3404883075 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e185c640 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234834c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e205d640 1 -- 172.21.15.69:0/3404883075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 msgr2=0x55c234834c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:04.556 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e205d640 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234834c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e205d640 1 -- 172.21.15.69:0/3404883075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 msgr2=0x55c2348a97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e205d640 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 0x55c2348a97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e205d640 1 -- 172.21.15.69:0/3404883075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2348a9da0 con 0x55c234825ff0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.549+0000 7f51e185c640 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234834c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e205d640 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c2348346d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f51d80b8440 tx=0x7f51d80314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb1254547494f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3404883075 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51d80bbfe0 con 0x55c234825ff0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3404883075 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f51d803bd00 con 0x55c234825ff0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 1 -- 172.21.15.69:0/3404883075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51d80c0390 con 0x55c234825ff0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3404883075 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51d80ba2e0 con 0x55c234825ff0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 10 client.18783 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 1 -- 172.21.15.69:0/3404883075 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2348aaff0 con 0x55c234825ff0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3404883075 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51d80c0390 con 0x55c234825ff0 2024-08-02T18:00:04.557 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3404883075 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f51d80ed490 con 0x55c234825ff0 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 10 client.18783 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 10 client.18783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 20 client.18783 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2348ab580 0x55c2348ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 4 client.18783 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.550+0000 7f51e66bf740 1 -- 172.21.15.69:0/3404883075 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2347934c0 con 0x55c2348ab580 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e185c640 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2348ab580 0x55c2348ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e185c640 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2348ab580 0x55c2348ad970 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cdd00d72f524d1e server_cookie=99605a98a43aafb in_seq=0 out_seq=0 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51c5ffb640 10 client.18783 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51c5ffb640 1 -- 172.21.15.69:0/3404883075 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2347934c0 con 0x55c2348ab580 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51c5ffb640 10 client.18783 handle_command_reply: tid=0 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 client.18783 shutdown 2024-08-02T18:00:04.558 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 -- 172.21.15.69:0/3404883075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 msgr2=0x55c2348346d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c2348346d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f51d80b8440 tx=0x7f51d80314b0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 -- 172.21.15.69:0/3404883075 shutdown_connections 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2348ab580 0x55c2348ad970 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2348311d0 0x55c2348a97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c234826910 0x55c234834c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 --2- 172.21.15.69:0/3404883075 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c234825ff0 0x55c2348346d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 -- 172.21.15.69:0/3404883075 >> 172.21.15.69:0/3404883075 conn(0x55c234822580 msgr2=0x55c23482f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51c5ffb640 0 client.18783 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 -- 172.21.15.69:0/3404883075 shutdown_connections 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 1 -- 172.21.15.69:0/3404883075 wait complete. 2024-08-02T18:00:04.559 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.551+0000 7f51e66bf740 20 client.18783 trim_cache size 0 max 16384 2024-08-02T18:00:04.560 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:04.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 -- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51c80a2cf0 msgr2=0x7f51c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51c80a2cf0 0x7f51c80a51b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f51d4005cc0 tx=0x7f51d400a340 comp rx=0 tx=0).stop 2024-08-02T18:00:04.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 -- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1018e0 msgr2=0x7f51dc198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:04.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1018e0 0x7f51dc198160 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f51d8009a50 tx=0x7f51d8004270 comp rx=0 tx=0).stop 2024-08-02T18:00:04.560 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 -- 172.21.15.69:0/1451056184 shutdown_connections 2024-08-02T18:00:04.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f51c80a2cf0 0x7f51c80a51b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51dc1078e0 0x7f51dc19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f51dc102280 0x7f51dc1986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 --2- 172.21.15.69:0/1451056184 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51dc1018e0 0x7f51dc198160 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:04.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 -- 172.21.15.69:0/1451056184 >> 172.21.15.69:0/1451056184 conn(0x7f51dc0fd600 msgr2=0x7f51dc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:04.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 -- 172.21.15.69:0/1451056184 shutdown_connections 2024-08-02T18:00:04.561 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:04.552+0000 7f51e42c1640 1 -- 172.21.15.69:0/1451056184 wait complete. 2024-08-02T18:00:04.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 775.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T18:00:05.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:05 smithi077 ceph-mon[37108]: pgmap v426: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-08-02T18:00:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:05 smithi069 ceph-mon[32746]: pgmap v426: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-08-02T18:00:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:05 smithi083 ceph-mon[37031]: pgmap v426: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.2 MiB/s wr, 136 op/s 2024-08-02T18:00:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 -- 172.21.15.69:0/2612993559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a01006d0 msgr2=0x7f86a0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 --2- 172.21.15.69:0/2612993559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a01006d0 0x7f86a0100b30 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f8688009930 tx=0x7f868802a420 comp rx=0 tx=0).stop 2024-08-02T18:00:06.047 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 -- 172.21.15.69:0/2612993559 shutdown_connections 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 --2- 172.21.15.69:0/2612993559 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a01006d0 0x7f86a0100b30 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 --2- 172.21.15.69:0/2612993559 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a0101070 0x7f86a010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 --2- 172.21.15.69:0/2612993559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01066e0 0x7f86a0106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 -- 172.21.15.69:0/2612993559 >> 172.21.15.69:0/2612993559 conn(0x7f86a00776d0 msgr2=0x7f86a00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 -- 172.21.15.69:0/2612993559 shutdown_connections 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.043+0000 7f86a4a74640 1 -- 172.21.15.69:0/2612993559 wait complete. 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f86a4a74640 1 Processor -- start 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f86a4a74640 1 -- start start 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f86a4a74640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01006d0 0x7f86a0198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f86a4a74640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101070 0x7f86a0198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f86a4a74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01066e0 0x7f86a019ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f86a4a74640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0111c10 con 0x7f86a0101070 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f86a4a74640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0111a90 con 0x7f86a01006d0 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f86a4a74640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0111d90 con 0x7f86a01066e0 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f869dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101070 0x7f86a0198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f869dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101070 0x7f86a0198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38158/0 (socket says 172.21.15.69:38158) 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.044+0000 7f869dd9b640 1 -- 172.21.15.69:0/3238640774 learned_addr learned my addr 172.21.15.69:0/3238640774 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869ed9d640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01066e0 0x7f86a019ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869e59c640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01006d0 0x7f86a0198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869dd9b640 1 -- 172.21.15.69:0/3238640774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01066e0 msgr2=0x7f86a019ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869dd9b640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01066e0 0x7f86a019ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869dd9b640 1 -- 172.21.15.69:0/3238640774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01006d0 msgr2=0x7f86a0198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.049 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869dd9b640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01006d0 0x7f86a0198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869dd9b640 1 -- 172.21.15.69:0/3238640774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a019d130 con 0x7f86a0101070 2024-08-02T18:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869e59c640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01006d0 0x7f86a0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f869dd9b640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101070 0x7f86a0198680 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8688009230 tx=0x7f8688004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f86977fe640 1 -- 172.21.15.69:0/3238640774 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868803f030 con 0x7f86a0101070 2024-08-02T18:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f86a4a74640 1 -- 172.21.15.69:0/3238640774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a019d3c0 con 0x7f86a0101070 2024-08-02T18:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f86977fe640 1 -- 172.21.15.69:0/3238640774 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8688003ad0 con 0x7f86a0101070 2024-08-02T18:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f86977fe640 1 -- 172.21.15.69:0/3238640774 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688035070 con 0x7f86a0101070 2024-08-02T18:00:06.050 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.045+0000 7f86a4a74640 1 -- 172.21.15.69:0/3238640774 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a01a4cb0 con 0x7f86a0101070 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.048+0000 7f86977fe640 1 -- 172.21.15.69:0/3238640774 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8688042030 con 0x7f86a0101070 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.048+0000 7f86a77c7740 1 Processor -- start 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.048+0000 7f86a77c7740 1 -- start start 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.048+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc7fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.048+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc8a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc8a970 0x55f1ccc8cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f86a77c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccbe39b0 con 0x55f1ccc8a970 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f86a77c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccb8b800 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f86a77c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccbdf970 con 0x55f1ccc800d0 2024-08-02T18:00:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f86977fe640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f868c0a2cf0 0x7f868c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc8a970 0x55f1ccc8cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc8a970 0x55f1ccc8cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38160/0 (socket says 172.21.15.69:38160) 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 -- 172.21.15.69:0/2713098232 learned_addr learned my addr 172.21.15.69:0/2713098232 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869ed9d640 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc7fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869e59c640 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc8a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869ed9d640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f868c0a2cf0 0x7f868c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f86977fe640 1 -- 172.21.15.69:0/3238640774 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f86880efd60 con 0x7f86a0101070 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 -- 172.21.15.69:0/2713098232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 msgr2=0x55f1ccc8a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc8a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 -- 172.21.15.69:0/2713098232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 msgr2=0x55f1ccc7fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc7fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 -- 172.21.15.69:0/2713098232 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1cc8f7210 con 0x55f1ccc8a970 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869e59c640 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc8a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869dd9b640 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc8a970 0x55f1ccc8cd60 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f86880ed010 tx=0x7f868802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8afd9617f151907 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f869ed9d640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f868c0a2cf0 0x7f868c0a51b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f86900118d0 tx=0x7f869000a760 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.049+0000 7f8694ff9640 1 -- 172.21.15.69:0/2713098232 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688049070 con 0x55f1ccc8a970 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f8694ff9640 1 -- 172.21.15.69:0/2713098232 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86880c4ae0 con 0x55f1ccc8a970 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f8694ff9640 1 -- 172.21.15.69:0/2713098232 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86880ed320 con 0x55f1ccc8a970 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 -- 172.21.15.69:0/2713098232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc8a970 msgr2=0x55f1ccc8cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc8a970 0x55f1ccc8cd60 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f86880ed010 tx=0x7f868802a420 comp rx=0 tx=0).stop 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 -- 172.21.15.69:0/2713098232 shutdown_connections 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc8a970 0x55f1ccc8cd60 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc8a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2713098232 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc7fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 -- 172.21.15.69:0/2713098232 >> 172.21.15.69:0/2713098232 conn(0x55f1ccc7bd20 msgr2=0x55f1ccc7c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 -- 172.21.15.69:0/2713098232 shutdown_connections 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.050+0000 7f86a77c7740 1 -- 172.21.15.69:0/2713098232 wait complete. 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 1 -- start start 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 1 Processor -- start 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc87860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc87da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc8a970 0x55f1ccc882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccc8d300 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccb9f500 con 0x55f1ccc8a970 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f1ccc8d480 con 0x55f1ccc800d0 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f86a77c7740 10 client.0 _ll_register_callbacks cb 0x55f1cc8f6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc87860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc87860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38162/0 (socket says 172.21.15.69:38162) 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869e59c640 1 -- 172.21.15.69:0/2584503178 learned_addr learned my addr 172.21.15.69:0/2584503178 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869dd9b640 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc87da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869e59c640 1 -- 172.21.15.69:0/2584503178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 msgr2=0x55f1ccc87da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869e59c640 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc87da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869e59c640 1 -- 172.21.15.69:0/2584503178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc8a970 msgr2=0x55f1ccc882e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869e59c640 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc8a970 0x55f1ccc882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.051+0000 7f869e59c640 1 -- 172.21.15.69:0/2584503178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1cc9c3620 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f869e59c640 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc87860 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8698012730 tx=0x7f8698012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd254afac228ecc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f8685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f8685ffb640 1 -- 172.21.15.69:0/2584503178 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869801f060 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f8685ffb640 1 -- 172.21.15.69:0/2584503178 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f8698010da0 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 1 -- 172.21.15.69:0/2584503178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86880c4ae0 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f8685ffb640 1 -- 172.21.15.69:0/2584503178 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8698026050 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 10 client.18795 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 1 -- 172.21.15.69:0/2584503178 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1ccc98f50 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f8685ffb640 1 -- 172.21.15.69:0/2584503178 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86880c4ae0 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f8685ffb640 1 -- 172.21.15.69:0/2584503178 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8698019030 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 10 client.18795 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 10 client.18795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 20 client.18795 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f1ccc9a410 0x55f1ccc9c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 4 client.18795 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.052+0000 7f86a77c7740 1 -- 172.21.15.69:0/2584503178 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1ccbf10e0 con 0x55f1ccc9a410 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.053+0000 7f869dd9b640 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f1ccc9a410 0x55f1ccc9c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.053+0000 7f8685ffb640 10 client.18795 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.053+0000 7f869dd9b640 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f1ccc9a410 0x55f1ccc9c880 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82db57e191621d74 server_cookie=bfbd4ac1c9fbb3f2 in_seq=0 out_seq=0 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f8685ffb640 1 -- 172.21.15.69:0/2584503178 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f1ccbf10e0 con 0x55f1ccc9a410 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f8685ffb640 10 client.18795 handle_command_reply: tid=0 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 client.18795 shutdown 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 -- 172.21.15.69:0/2584503178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 msgr2=0x55f1ccc87860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc87860 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f8698012730 tx=0x7f8698012bf0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 -- 172.21.15.69:0/2584503178 shutdown_connections 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f1ccc9a410 0x55f1ccc9c880 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc8a970 0x55f1ccc882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc800d0 0x55f1ccc87da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f8685ffb640 0 client.18795 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 --2- 172.21.15.69:0/2584503178 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc87860 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 -- 172.21.15.69:0/2584503178 >> 172.21.15.69:0/2584503178 conn(0x55f1ccc7bd20 msgr2=0x55f1ccc81a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 -- 172.21.15.69:0/2584503178 shutdown_connections 2024-08-02T18:00:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 1 -- 172.21.15.69:0/2584503178 wait complete. 2024-08-02T18:00:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.054+0000 7f86a77c7740 20 client.18795 trim_cache size 0 max 16384 2024-08-02T18:00:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.069+0000 7f86a77c7740 1 Processor -- start 2024-08-02T18:00:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.069+0000 7f86a77c7740 1 -- start start 2024-08-02T18:00:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.069+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.069+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc800d0 0x55f1ccc92ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.069+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 0x55f1ccc92ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.069+0000 7f86a77c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01b17a0 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.069+0000 7f86a77c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01af070 con 0x55f1ccc800d0 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.069+0000 7f86a77c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01acec0 con 0x55f1ccc8a970 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc896c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38166/0 (socket says 172.21.15.69:38166) 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 -- 172.21.15.69:0/1786410105 learned_addr learned my addr 172.21.15.69:0/1786410105 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869ed9d640 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 0x55f1ccc92ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869dd9b640 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc800d0 0x55f1ccc92ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 -- 172.21.15.69:0/1786410105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 msgr2=0x55f1ccc92ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 0x55f1ccc92ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 -- 172.21.15.69:0/1786410105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc800d0 msgr2=0x55f1ccc92ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc800d0 0x55f1ccc92ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 -- 172.21.15.69:0/1786410105 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1ccc7bb50 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869dd9b640 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc800d0 0x55f1ccc92ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f869e59c640 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc896c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8698015040 tx=0x7f8698009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b25e1e0fb8a2e02 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f8687fff640 1 -- 172.21.15.69:0/1786410105 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869802f070 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f8687fff640 1 -- 172.21.15.69:0/1786410105 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869800f7f0 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.070+0000 7f8687fff640 1 -- 172.21.15.69:0/1786410105 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869802a910 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- 172.21.15.69:0/1786410105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 msgr2=0x55f1ccc896c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc896c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f8698015040 tx=0x7f8698009a60 comp rx=0 tx=0).stop 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- 172.21.15.69:0/1786410105 shutdown_connections 2024-08-02T18:00:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 0x55f1ccc92ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc800d0 0x55f1ccc92ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 --2- 172.21.15.69:0/1786410105 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccc896c0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- 172.21.15.69:0/1786410105 >> 172.21.15.69:0/1786410105 conn(0x55f1ccc7bd20 msgr2=0x55f1ccc88fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- 172.21.15.69:0/1786410105 shutdown_connections 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- 172.21.15.69:0/1786410105 wait complete. 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- start start 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 Processor -- start 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccd017b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc800d0 0x55f1ccd01cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 0x55f1cccfd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01ad530 con 0x55f1ccc800d0 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01ae9a0 con 0x55f1ccc7f7b0 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01adc00 con 0x55f1ccc8a970 2024-08-02T18:00:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.071+0000 7f86a77c7740 10 client.0 _ll_register_callbacks cb 0x7f8698009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc800d0 0x55f1ccd01cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc800d0 0x55f1ccd01cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38168/0 (socket says 172.21.15.69:38168) 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 -- 172.21.15.69:0/869591116 learned_addr learned my addr 172.21.15.69:0/869591116 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869ed9d640 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 0x55f1cccfd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869e59c640 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccd017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 -- 172.21.15.69:0/869591116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 msgr2=0x55f1cccfd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 0x55f1cccfd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 -- 172.21.15.69:0/869591116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 msgr2=0x55f1ccd017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccd017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 -- 172.21.15.69:0/869591116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1ccc96560 con 0x55f1ccc800d0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f869dd9b640 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc800d0 0x55f1ccd01cf0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f86880bbe00 tx=0x7f8688005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f266c7dfaa00efe2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f8685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f8685ffb640 1 -- 172.21.15.69:0/869591116 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868809f060 con 0x55f1ccc800d0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f8685ffb640 1 -- 172.21.15.69:0/869591116 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f8688003c80 con 0x55f1ccc800d0 2024-08-02T18:00:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f86a77c7740 1 -- 172.21.15.69:0/869591116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f869800f7f0 con 0x55f1ccc800d0 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f8685ffb640 1 -- 172.21.15.69:0/869591116 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8688049040 con 0x55f1ccc800d0 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f86a77c7740 10 client.18801 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f86a77c7740 1 -- 172.21.15.69:0/869591116 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1ccc96c30 con 0x55f1ccc800d0 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.072+0000 7f8685ffb640 1 -- 172.21.15.69:0/869591116 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f869800f7f0 con 0x55f1ccc800d0 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f8685ffb640 1 -- 172.21.15.69:0/869591116 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f86880ed320 con 0x55f1ccc800d0 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f86a77c7740 10 client.18801 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f86a77c7740 10 client.18801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f86a77c7740 20 client.18801 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f86a77c7740 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f1ccd0a0b0 0x55f1ccd0c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f86a77c7740 4 client.18801 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f86a77c7740 1 -- 172.21.15.69:0/869591116 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1ccbeb430 con 0x55f1ccd0a0b0 2024-08-02T18:00:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f869e59c640 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f1ccd0a0b0 0x55f1ccd0c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f8685ffb640 10 client.18801 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:06.082 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.073+0000 7f869e59c640 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f1ccd0a0b0 0x55f1ccd0c4a0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbfd7b79cd789169 server_cookie=ba1c8572648af6b1 in_seq=0 out_seq=0 2024-08-02T18:00:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.088+0000 7f8685ffb640 1 -- 172.21.15.69:0/869591116 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1ccbeb430 con 0x55f1ccd0a0b0 2024-08-02T18:00:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.088+0000 7f8685ffb640 10 client.18801 handle_command_reply: tid=0 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.088+0000 7f86a77c7740 1 client.18801 shutdown 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.088+0000 7f86a77c7740 1 -- 172.21.15.69:0/869591116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc800d0 msgr2=0x55f1ccd01cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.088+0000 7f86a77c7740 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc800d0 0x55f1ccd01cf0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f86880bbe00 tx=0x7f8688005c00 comp rx=0 tx=0).stop 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.088+0000 7f86a77c7740 1 -- 172.21.15.69:0/869591116 shutdown_connections 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a77c7740 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f1ccc800d0 0x55f1ccd01cf0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a77c7740 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f1ccd0a0b0 0x55f1ccd0c4a0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a77c7740 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f1ccc8a970 0x55f1cccfd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a77c7740 1 --2- 172.21.15.69:0/869591116 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f1ccc7f7b0 0x55f1ccd017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f8685ffb640 0 client.18801 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a77c7740 1 -- 172.21.15.69:0/869591116 >> 172.21.15.69:0/869591116 conn(0x55f1ccc7bd20 msgr2=0x55f1ccc95d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a77c7740 1 -- 172.21.15.69:0/869591116 shutdown_connections 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a77c7740 1 -- 172.21.15.69:0/869591116 wait complete. 2024-08-02T18:00:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a77c7740 20 client.18801 trim_cache size 0 max 16384 2024-08-02T18:00:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:00:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:00:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:00:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 -- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f868c0a2cf0 msgr2=0x7f868c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f868c0a2cf0 0x7f868c0a51b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f86900118d0 tx=0x7f869000a760 comp rx=0 tx=0).stop 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 -- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101070 msgr2=0x7f86a0198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101070 0x7f86a0198680 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8688009230 tx=0x7f8688004270 comp rx=0 tx=0).stop 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 -- 172.21.15.69:0/3238640774 shutdown_connections 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f86a0101070 0x7f86a0198680 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f868c0a2cf0 0x7f868c0a51b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f86a01066e0 0x7f86a019ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 --2- 172.21.15.69:0/3238640774 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f86a01006d0 0x7f86a0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.089+0000 7f86a4a74640 1 -- 172.21.15.69:0/3238640774 >> 172.21.15.69:0/3238640774 conn(0x7f86a00776d0 msgr2=0x7f86a00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.090+0000 7f86a4a74640 1 -- 172.21.15.69:0/3238640774 shutdown_connections 2024-08-02T18:00:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:06.090+0000 7f86a4a74640 1 -- 172.21.15.69:0/3238640774 wait complete. 2024-08-02T18:00:06.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T18:00:06.426 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T18:00:06.684 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:07.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 -- 172.21.15.69:0/45982421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8102ae0 msgr2=0x7f05f8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 --2- 172.21.15.69:0/45982421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8102ae0 0x7f05f8102ec0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009930 tx=0x7f05e802a440 comp rx=0 tx=0).stop 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 -- 172.21.15.69:0/45982421 shutdown_connections 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 --2- 172.21.15.69:0/45982421 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05f8108010 0x7f05f810e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 --2- 172.21.15.69:0/45982421 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f8103490 0x7f05f81078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 --2- 172.21.15.69:0/45982421 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8102ae0 0x7f05f8102ec0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 -- 172.21.15.69:0/45982421 >> 172.21.15.69:0/45982421 conn(0x7f05f80fc500 msgr2=0x7f05f80fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 -- 172.21.15.69:0/45982421 shutdown_connections 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.151+0000 7f05fe4da640 1 -- 172.21.15.69:0/45982421 wait complete. 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.152+0000 7f05fe4da640 1 Processor -- start 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.152+0000 7f05fe4da640 1 -- start start 2024-08-02T18:00:07.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.152+0000 7f05fe4da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f8102ae0 0x7f05f8100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.152+0000 7f05fe4da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8103490 0x7f05f8100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.152+0000 7f05fe4da640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05f8108010 0x7f05f81011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.152+0000 7f05fe4da640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8111ea0 con 0x7f05f8103490 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.152+0000 7f05fe4da640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8111d20 con 0x7f05f8108010 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.152+0000 7f05fe4da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05f8112020 con 0x7f05f8102ae0 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8103490 0x7f05f8100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8103490 0x7f05f8100c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38188/0 (socket says 172.21.15.69:38188) 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 -- 172.21.15.69:0/2603058302 learned_addr learned my addr 172.21.15.69:0/2603058302 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f8102ae0 0x7f05f8100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05fca77640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05f8108010 0x7f05f81011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 -- 172.21.15.69:0/2603058302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f8102ae0 msgr2=0x7f05f8100740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f8102ae0 0x7f05f8100740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 -- 172.21.15.69:0/2603058302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05f8108010 msgr2=0x7f05f81011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05f8108010 0x7f05f81011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 -- 172.21.15.69:0/2603058302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f81a6920 con 0x7f05f8103490 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05fca77640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05f8108010 0x7f05f81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f77fe640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8103490 0x7f05f8100c80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f05ec006fd0 tx=0x7f05ec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f57fa640 1 -- 172.21.15.69:0/2603058302 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011830 con 0x7f05f8103490 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f57fa640 1 -- 172.21.15.69:0/2603058302 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05ec016070 con 0x7f05f8103490 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05f57fa640 1 -- 172.21.15.69:0/2603058302 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec00f2e0 con 0x7f05f8103490 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05fe4da640 1 -- 172.21.15.69:0/2603058302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f81a6c10 con 0x7f05f8103490 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.153+0000 7f05fe4da640 1 -- 172.21.15.69:0/2603058302 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f81a7100 con 0x7f05f8103490 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05f57fa640 1 -- 172.21.15.69:0/2603058302 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f05ec01b460 con 0x7f05f8103490 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f06008d8740 1 Processor -- start 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f06008d8740 1 -- start start 2024-08-02T18:00:07.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f06008d8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f06008d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f06008d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 0x559ebe4c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f06008d8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559ebe479e40 con 0x559ebe4b6200 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f06008d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ebe41b7f0 con 0x559ebe4c13e0 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f06008d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ebe3c2fb0 con 0x559ebe4b6b20 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05fca77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05fca77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38194/0 (socket says 172.21.15.69:38194) 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05fca77640 1 -- 172.21.15.69:0/1864598456 learned_addr learned my addr 172.21.15.69:0/1864598456 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05f7fff640 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05fca77640 1 -- 172.21.15.69:0/1864598456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 msgr2=0x559ebe4c0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:07.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05fca77640 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05fca77640 1 -- 172.21.15.69:0/1864598456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 msgr2=0x559ebe4c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05f77fe640 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 0x559ebe4c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05f57fa640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05e00a2cf0 0x7f05e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05fca77640 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 0x559ebe4c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05fca77640 1 -- 172.21.15.69:0/1864598456 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ebe18d6a0 con 0x559ebe4b6200 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.156+0000 7f05f7fff640 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05e00a2cf0 0x7f05e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f05f57fa640 1 -- 172.21.15.69:0/2603058302 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f05ec0cee30 con 0x7f05f8103490 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f05fca77640 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b65e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c450 tx=0x7f05f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9afdebc8bd3933c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f05de7fc640 1 -- 172.21.15.69:0/1864598456 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0007430 con 0x559ebe4b6200 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f05de7fc640 1 -- 172.21.15.69:0/1864598456 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05f0007ab0 con 0x559ebe4b6200 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f05de7fc640 1 -- 172.21.15.69:0/1864598456 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f000aae0 con 0x559ebe4b6200 2024-08-02T18:00:07.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f06008d8740 1 -- 172.21.15.69:0/1864598456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 msgr2=0x559ebe4b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05e00a2cf0 0x7f05e00a51b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f05e8006010 tx=0x7f05e80058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.157+0000 7f06008d8740 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b65e0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c450 tx=0x7f05f000c910 comp rx=0 tx=0).stop 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 -- 172.21.15.69:0/1864598456 shutdown_connections 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 0x559ebe4c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 --2- 172.21.15.69:0/1864598456 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b65e0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 -- 172.21.15.69:0/1864598456 >> 172.21.15.69:0/1864598456 conn(0x559ebe4b2790 msgr2=0x559ebe4b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 -- 172.21.15.69:0/1864598456 shutdown_connections 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 -- 172.21.15.69:0/1864598456 wait complete. 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 -- start start 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.158+0000 7f06008d8740 1 Processor -- start 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6200 0x559ebe4be230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6b20 0x559ebe4be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4c13e0 0x559ebe4becb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559ebe3d6cb0 con 0x559ebe4c13e0 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559ebe4177b0 con 0x559ebe4b6b20 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559ebe4c3fb0 con 0x559ebe4b6200 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 10 client.0 _ll_register_callbacks cb 0x559ebe12aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4c13e0 0x559ebe4becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4c13e0 0x559ebe4becb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38206/0 (socket says 172.21.15.69:38206) 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 -- 172.21.15.69:0/2812760322 learned_addr learned my addr 172.21.15.69:0/2812760322 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:07.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05f77fe640 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6b20 0x559ebe4be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6200 0x559ebe4be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 -- 172.21.15.69:0/2812760322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6200 msgr2=0x559ebe4be230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6200 0x559ebe4be230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 -- 172.21.15.69:0/2812760322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6b20 msgr2=0x559ebe4be770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6b20 0x559ebe4be770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 -- 172.21.15.69:0/2812760322 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ebe3d5bc0 con 0x559ebe4c13e0 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05fca77640 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4c13e0 0x559ebe4becb0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f05f000ce10 tx=0x7f05f0009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c082653a5fa29e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05d77fe640 1 -- 172.21.15.69:0/2812760322 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0007700 con 0x559ebe4c13e0 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05d77fe640 1 -- 172.21.15.69:0/2812760322 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f05f0012040 con 0x559ebe4c13e0 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 1 -- 172.21.15.69:0/2812760322 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f0007ab0 con 0x559ebe4c13e0 2024-08-02T18:00:07.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f05d77fe640 1 -- 172.21.15.69:0/2812760322 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05f002c050 con 0x559ebe4c13e0 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 10 client.18813 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.159+0000 7f06008d8740 1 -- 172.21.15.69:0/2812760322 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ebe4cfdd0 con 0x559ebe4c13e0 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f05d77fe640 1 -- 172.21.15.69:0/2812760322 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05f0007ab0 con 0x559ebe4c13e0 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f05d77fe640 1 -- 172.21.15.69:0/2812760322 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05f0016830 con 0x559ebe4c13e0 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f06008d8740 10 client.18813 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f06008d8740 10 client.18813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f06008d8740 20 client.18813 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f06008d8740 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559ebe4d1230 0x559ebe4d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f06008d8740 4 client.18813 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f06008d8740 1 -- 172.21.15.69:0/2812760322 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ebe428d60 con 0x559ebe4d1230 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.160+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559ebe4d1230 0x559ebe4d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f05d77fe640 10 client.18813 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:07.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559ebe4d1230 0x559ebe4d3620 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9da5c0cd63bbc87 server_cookie=ceb96c4b24071337 in_seq=0 out_seq=0 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f05d77fe640 1 -- 172.21.15.69:0/2812760322 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559ebe428d60 con 0x559ebe4d1230 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f05d77fe640 10 client.18813 handle_command_reply: tid=0 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 client.18813 shutdown 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 -- 172.21.15.69:0/2812760322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4c13e0 msgr2=0x559ebe4becb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4c13e0 0x559ebe4becb0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f05f000ce10 tx=0x7f05f0009db0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 -- 172.21.15.69:0/2812760322 shutdown_connections 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4c13e0 0x559ebe4becb0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559ebe4d1230 0x559ebe4d3620 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6b20 0x559ebe4be770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 --2- 172.21.15.69:0/2812760322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6200 0x559ebe4be230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f05d77fe640 0 client.18813 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.161+0000 7f06008d8740 1 -- 172.21.15.69:0/2812760322 >> 172.21.15.69:0/2812760322 conn(0x559ebe4b2790 msgr2=0x559ebe4b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.162+0000 7f06008d8740 1 -- 172.21.15.69:0/2812760322 shutdown_connections 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.162+0000 7f06008d8740 1 -- 172.21.15.69:0/2812760322 wait complete. 2024-08-02T18:00:07.179 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.162+0000 7f06008d8740 20 client.18813 trim_cache size 0 max 16384 2024-08-02T18:00:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f06008d8740 1 Processor -- start 2024-08-02T18:00:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f06008d8740 1 -- start start 2024-08-02T18:00:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f06008d8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f06008d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f06008d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 0x559ebe4d8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f06008d8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05f81b3b10 con 0x559ebe4b6200 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f06008d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05f81b13e0 con 0x559ebe4c13e0 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f06008d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05f81af230 con 0x559ebe4b6b20 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f05f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f05f7fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38212/0 (socket says 172.21.15.69:38212) 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f05f7fff640 1 -- 172.21.15.69:0/3830554417 learned_addr learned my addr 172.21.15.69:0/3830554417 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.179+0000 7f05fca77640 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 0x559ebe4d8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05f77fe640 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05f7fff640 1 -- 172.21.15.69:0/3830554417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 msgr2=0x559ebe4c01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05f7fff640 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05f7fff640 1 -- 172.21.15.69:0/3830554417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 msgr2=0x559ebe4d8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05f7fff640 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 0x559ebe4d8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05f7fff640 1 -- 172.21.15.69:0/3830554417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ebe4b3840 con 0x559ebe4b6200 2024-08-02T18:00:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05f7fff640 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b8c60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f05e802a960 tx=0x7f05e803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f29a0392f3aa05 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05ddffb640 1 -- 172.21.15.69:0/3830554417 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e8035070 con 0x559ebe4b6200 2024-08-02T18:00:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05ddffb640 1 -- 172.21.15.69:0/3830554417 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e8003e40 con 0x559ebe4b6200 2024-08-02T18:00:07.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.180+0000 7f05ddffb640 1 -- 172.21.15.69:0/3830554417 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e803c9a0 con 0x559ebe4b6200 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 -- 172.21.15.69:0/3830554417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 msgr2=0x559ebe4b8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b8c60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f05e802a960 tx=0x7f05e803f000 comp rx=0 tx=0).stop 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 -- 172.21.15.69:0/3830554417 shutdown_connections 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4c13e0 0x559ebe4d8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4b6b20 0x559ebe4c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 --2- 172.21.15.69:0/3830554417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6200 0x559ebe4b8c60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 -- 172.21.15.69:0/3830554417 >> 172.21.15.69:0/3830554417 conn(0x559ebe4b2790 msgr2=0x559ebe4bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 -- 172.21.15.69:0/3830554417 shutdown_connections 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 -- 172.21.15.69:0/3830554417 wait complete. 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 -- start start 2024-08-02T18:00:07.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 Processor -- start 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6200 0x559ebe4c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6b20 0x559ebe4c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.181+0000 7f06008d8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4c13e0 0x559ebe4ba860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f06008d8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f05f81af8a0 con 0x559ebe4b6b20 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f06008d8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f05f81b0d10 con 0x559ebe4b6200 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f06008d8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05f81aff70 con 0x559ebe4c13e0 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f06008d8740 10 client.0 _ll_register_callbacks cb 0x7f05f0009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6b20 0x559ebe4c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6b20 0x559ebe4c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38226/0 (socket says 172.21.15.69:38226) 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 -- 172.21.15.69:0/2954140361 learned_addr learned my addr 172.21.15.69:0/2954140361 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6200 0x559ebe4c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05fca77640 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4c13e0 0x559ebe4ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 -- 172.21.15.69:0/2954140361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4c13e0 msgr2=0x559ebe4ba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:07.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4c13e0 0x559ebe4ba860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 -- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6200 msgr2=0x559ebe4c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6200 0x559ebe4c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 -- 172.21.15.69:0/2954140361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ebe4bada0 con 0x559ebe4b6b20 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6200 0x559ebe4c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05f77fe640 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6b20 0x559ebe4c9740 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009fd0 tx=0x7f05ec09ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b19b18b5b490875 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05d77fe640 1 -- 172.21.15.69:0/2954140361 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec09b050 con 0x559ebe4b6b20 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05d77fe640 1 -- 172.21.15.69:0/2954140361 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f05ec09f0c0 con 0x559ebe4b6b20 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f06008d8740 1 -- 172.21.15.69:0/2954140361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05e8003e40 con 0x559ebe4b6b20 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f05d77fe640 1 -- 172.21.15.69:0/2954140361 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05ec09f400 con 0x559ebe4b6b20 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f06008d8740 10 client.18819 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.182+0000 7f06008d8740 1 -- 172.21.15.69:0/2954140361 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559ebe53b3d0 con 0x559ebe4b6b20 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f05d77fe640 1 -- 172.21.15.69:0/2954140361 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e8003e40 con 0x559ebe4b6b20 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f05d77fe640 1 -- 172.21.15.69:0/2954140361 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05ec095bc0 con 0x559ebe4b6b20 2024-08-02T18:00:07.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f06008d8740 10 client.18819 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f06008d8740 10 client.18819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f06008d8740 20 client.18819 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f06008d8740 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559ebe53bb40 0x559ebe53df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f06008d8740 4 client.18819 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f06008d8740 1 -- 172.21.15.69:0/2954140361 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559ebe423270 con 0x559ebe53bb40 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559ebe53bb40 0x559ebe53df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f05f7fff640 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559ebe53bb40 0x559ebe53df60 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12065787749ff7cf server_cookie=1956454cf866cae6 in_seq=0 out_seq=0 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.183+0000 7f05d77fe640 10 client.18819 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.184+0000 7f05d77fe640 1 -- 172.21.15.69:0/2954140361 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559ebe423270 con 0x559ebe53bb40 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.184+0000 7f05d77fe640 10 client.18819 handle_command_reply: tid=0 2024-08-02T18:00:07.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.184+0000 7f06008d8740 1 client.18819 shutdown 2024-08-02T18:00:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 -- 172.21.15.69:0/2954140361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6b20 msgr2=0x559ebe4c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6b20 0x559ebe4c9740 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009fd0 tx=0x7f05ec09ab90 comp rx=0 tx=0).stop 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 -- 172.21.15.69:0/2954140361 shutdown_connections 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559ebe4b6b20 0x559ebe4c9740 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559ebe53bb40 0x559ebe53df60 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f05d77fe640 0 client.18819 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559ebe4c13e0 0x559ebe4ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 --2- 172.21.15.69:0/2954140361 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559ebe4b6200 0x559ebe4c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 -- 172.21.15.69:0/2954140361 >> 172.21.15.69:0/2954140361 conn(0x559ebe4b2790 msgr2=0x559ebe4c02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 -- 172.21.15.69:0/2954140361 shutdown_connections 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 1 -- 172.21.15.69:0/2954140361 wait complete. 2024-08-02T18:00:07.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f06008d8740 20 client.18819 trim_cache size 0 max 16384 2024-08-02T18:00:07.192 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:00:07.192 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T18:00:07.192 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "8dc7a0c6-cd1b-4dd7-8132-70a152cb4349", 2024-08-02T18:00:07.192 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T18:00:07.192 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:00:07.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f05fe4da640 1 -- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05e00a2cf0 msgr2=0x7f05e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.185+0000 7f05fe4da640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05e00a2cf0 0x7f05e00a51b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f05e8006010 tx=0x7f05e80058d0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 -- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8103490 msgr2=0x7f05f8100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8103490 0x7f05f8100c80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f05ec006fd0 tx=0x7f05ec00b8e0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 -- 172.21.15.69:0/2603058302 shutdown_connections 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f05f8103490 0x7f05f8100c80 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f05e00a2cf0 0x7f05e00a51b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f05f8108010 0x7f05f81011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 --2- 172.21.15.69:0/2603058302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05f8102ae0 0x7f05f8100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 -- 172.21.15.69:0/2603058302 >> 172.21.15.69:0/2603058302 conn(0x7f05f80fc500 msgr2=0x7f05f8105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 -- 172.21.15.69:0/2603058302 shutdown_connections 2024-08-02T18:00:07.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:07.186+0000 7f05fe4da640 1 -- 172.21.15.69:0/2603058302 wait complete. 2024-08-02T18:00:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:07 smithi077 ceph-mon[37108]: pgmap v427: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 134 KiB/s wr, 39 op/s 2024-08-02T18:00:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:07 smithi069 ceph-mon[32746]: pgmap v427: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 134 KiB/s wr, 39 op/s 2024-08-02T18:00:07.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:07 smithi083 ceph-mon[37031]: pgmap v427: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 134 KiB/s wr, 39 op/s 2024-08-02T18:00:08.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:08 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T18:00:08.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:08 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:08.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:08 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T18:00:08.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:08 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:08 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T18:00:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:08 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:08 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T18:00:08.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:08 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:08.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:08 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T18:00:08.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:08 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:08.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:08 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T18:00:08.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:08 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:09.204 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T18:00:09.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:09 smithi077 ceph-mon[37108]: pgmap v428: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 29 op/s 2024-08-02T18:00:09.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:09 smithi069 ceph-mon[32746]: pgmap v428: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 29 op/s 2024-08-02T18:00:09.468 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:09 smithi083 ceph-mon[37031]: pgmap v428: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 59 KiB/s wr, 29 op/s 2024-08-02T18:00:09.958 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 780.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T18:00:09.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 -- 172.21.15.69:0/2935480939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b101067b0 msgr2=0x7f7b10106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 --2- 172.21.15.69:0/2935480939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b101067b0 0x7f7b10106b90 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009930 tx=0x7f7b0002a420 comp rx=0 tx=0).stop 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 -- 172.21.15.69:0/2935480939 shutdown_connections 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 --2- 172.21.15.69:0/2935480939 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b10101150 0x7f7b1010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 --2- 172.21.15.69:0/2935480939 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b101007b0 0x7f7b10100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 --2- 172.21.15.69:0/2935480939 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b101067b0 0x7f7b10106b90 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 -- 172.21.15.69:0/2935480939 >> 172.21.15.69:0/2935480939 conn(0x7f7b100fc4d0 msgr2=0x7f7b100fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 -- 172.21.15.69:0/2935480939 shutdown_connections 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.963+0000 7f7b15965640 1 -- 172.21.15.69:0/2935480939 wait complete. 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b15965640 1 Processor -- start 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b15965640 1 -- start start 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b15965640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b101007b0 0x7f7b10102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b15965640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b10101150 0x7f7b101028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b15965640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b101067b0 0x7f7b10102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b15965640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10110bb0 con 0x7f7b10101150 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b15965640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10110a30 con 0x7f7b101067b0 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b15965640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10110d30 con 0x7f7b101007b0 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b10101150 0x7f7b101028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b10101150 0x7f7b101028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38232/0 (socket says 172.21.15.69:38232) 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 -- 172.21.15.69:0/856448982 learned_addr learned my addr 172.21.15.69:0/856448982 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b0effd640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b101007b0 0x7f7b10102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b101067b0 0x7f7b10102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 -- 172.21.15.69:0/856448982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b101007b0 msgr2=0x7f7b10102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b101007b0 0x7f7b10102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 -- 172.21.15.69:0/856448982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b101067b0 msgr2=0x7f7b10102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b101067b0 0x7f7b10102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 -- 172.21.15.69:0/856448982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b101a6620 con 0x7f7b10101150 2024-08-02T18:00:09.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b101067b0 0x7f7b10102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.964+0000 7f7b067fc640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b10101150 0x7f7b101028c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f7af800d950 tx=0x7f7af800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.965+0000 7f7b14963640 1 -- 172.21.15.69:0/856448982 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8017070 con 0x7f7b10101150 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.965+0000 7f7b14963640 1 -- 172.21.15.69:0/856448982 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af8009c30 con 0x7f7b10101150 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.965+0000 7f7b15965640 1 -- 172.21.15.69:0/856448982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b101a6910 con 0x7f7b10101150 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.965+0000 7f7b14963640 1 -- 172.21.15.69:0/856448982 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af800f6b0 con 0x7f7b10101150 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.965+0000 7f7b15965640 1 -- 172.21.15.69:0/856448982 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101a6e50 con 0x7f7b10101150 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b14963640 1 -- 172.21.15.69:0/856448982 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7af80109d0 con 0x7f7b10101150 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b17d63740 1 Processor -- start 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b17d63740 1 -- start start 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x5646128495f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612853eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x5646128567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b17d63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5646127ae980 con 0x5646128543f0 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b17d63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564612746a70 con 0x564612849b30 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.967+0000 7f7b17d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5646127aa940 con 0x564612849210 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x5646128567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x5646128567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38240/0 (socket says 172.21.15.69:38240) 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 -- 172.21.15.69:0/522012355 learned_addr learned my addr 172.21.15.69:0/522012355 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b0effd640 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612853eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x5646128495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b14963640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 -- 172.21.15.69:0/522012355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 msgr2=0x5646128495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x5646128495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 -- 172.21.15.69:0/522012355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 msgr2=0x564612853eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612853eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 -- 172.21.15.69:0/522012355 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646127a96e0 con 0x5646128543f0 2024-08-02T18:00:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b14963640 1 -- 172.21.15.69:0/856448982 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f7af80cf4f0 con 0x7f7b10101150 2024-08-02T18:00:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b067fc640 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x5646128567e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f7af80cc770 tx=0x7f7af806a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffa48c83198ba72 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800ae80 tx=0x7f7b08004a40 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b05ffb640 1 -- 172.21.15.69:0/522012355 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8097040 con 0x5646128543f0 2024-08-02T18:00:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b05ffb640 1 -- 172.21.15.69:0/522012355 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af809bd00 con 0x5646128543f0 2024-08-02T18:00:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.968+0000 7f7b05ffb640 1 -- 172.21.15.69:0/522012355 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8068a00 con 0x5646128543f0 2024-08-02T18:00:09.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 -- 172.21.15.69:0/522012355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 msgr2=0x5646128567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x5646128567e0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f7af80cc770 tx=0x7f7af806a530 comp rx=0 tx=0).stop 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 -- 172.21.15.69:0/522012355 shutdown_connections 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x5646128567e0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612853eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 --2- 172.21.15.69:0/522012355 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x5646128495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 -- 172.21.15.69:0/522012355 >> 172.21.15.69:0/522012355 conn(0x5646128457a0 msgr2=0x564612845b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 -- 172.21.15.69:0/522012355 shutdown_connections 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 -- 172.21.15.69:0/522012355 wait complete. 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.969+0000 7f7b17d63740 1 -- start start 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 1 Processor -- start 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x564612851250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612851790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x564612851cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564612856fc0 con 0x5646128543f0 2024-08-02T18:00:09.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564612769aa0 con 0x564612849b30 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564612857140 con 0x564612849210 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 10 client.0 _ll_register_callbacks cb 0x564612758e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x564612851cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x564612851cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38246/0 (socket says 172.21.15.69:38246) 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 -- 172.21.15.69:0/154118753 learned_addr learned my addr 172.21.15.69:0/154118753 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b067fc640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612851790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0effd640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x564612851250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 -- 172.21.15.69:0/154118753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 msgr2=0x564612851250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x564612851250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 -- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 msgr2=0x564612851790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612851790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 -- 172.21.15.69:0/154118753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56461274d650 con 0x5646128543f0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0effd640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x564612851250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b067fc640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612851790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x564612851cd0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7b08012e20 tx=0x7f7b08004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1547d1e1ea9a88 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7aeeffd640 1 -- 172.21.15.69:0/154118753 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800fd50 con 0x5646128543f0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.970+0000 7f7b17d63740 1 -- 172.21.15.69:0/154118753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af809bd00 con 0x5646128543f0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7aeeffd640 1 -- 172.21.15.69:0/154118753 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f7b08025470 con 0x5646128543f0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7aeeffd640 1 -- 172.21.15.69:0/154118753 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b08025670 con 0x5646128543f0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b17d63740 10 client.18831 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b17d63740 1 -- 172.21.15.69:0/154118753 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646125203c0 con 0x5646128543f0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7aeeffd640 1 -- 172.21.15.69:0/154118753 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af809bd00 con 0x5646128543f0 2024-08-02T18:00:09.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7aeeffd640 1 -- 172.21.15.69:0/154118753 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b0801ba00 con 0x5646128543f0 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b17d63740 10 client.18831 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b17d63740 10 client.18831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b17d63740 20 client.18831 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b17d63740 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564612863e40 0x564612866290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b17d63740 4 client.18831 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b17d63740 1 -- 172.21.15.69:0/154118753 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646127bbef0 con 0x564612863e40 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.971+0000 7f7b0effd640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564612863e40 0x564612866290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.972+0000 7f7aeeffd640 10 client.18831 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.972+0000 7f7b0effd640 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564612863e40 0x564612866290 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbf267ab1f222c2d server_cookie=de3c6032a2b15c31 in_seq=0 out_seq=0 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.972+0000 7f7aeeffd640 1 -- 172.21.15.69:0/154118753 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5646127bbef0 con 0x564612863e40 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.972+0000 7f7aeeffd640 10 client.18831 handle_command_reply: tid=0 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.972+0000 7f7b17d63740 1 client.18831 shutdown 2024-08-02T18:00:09.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 -- 172.21.15.69:0/154118753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 msgr2=0x564612851cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x564612851cd0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f7b08012e20 tx=0x7f7b08004ab0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 -- 172.21.15.69:0/154118753 shutdown_connections 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5646128543f0 0x564612851cd0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564612863e40 0x564612866290 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564612849b30 0x564612851790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 --2- 172.21.15.69:0/154118753 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x564612851250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7aeeffd640 0 client.18831 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 -- 172.21.15.69:0/154118753 >> 172.21.15.69:0/154118753 conn(0x5646128457a0 msgr2=0x56461284b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 -- 172.21.15.69:0/154118753 shutdown_connections 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 1 -- 172.21.15.69:0/154118753 wait complete. 2024-08-02T18:00:09.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.973+0000 7f7b17d63740 20 client.18831 trim_cache size 0 max 16384 2024-08-02T18:00:09.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b17d63740 1 Processor -- start 2024-08-02T18:00:09.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b17d63740 1 -- start start 2024-08-02T18:00:09.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x56461284c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849b30 0x56461286bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 0x56461286c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:09.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b17d63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b3a90 con 0x564612849b30 2024-08-02T18:00:09.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b17d63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b1360 con 0x5646128543f0 2024-08-02T18:00:10.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b17d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101af1b0 con 0x564612849210 2024-08-02T18:00:10.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b067fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849b30 0x56461286bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:10.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b067fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849b30 0x56461286bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38258/0 (socket says 172.21.15.69:38258) 2024-08-02T18:00:10.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b067fc640 1 -- 172.21.15.69:0/3233361128 learned_addr learned my addr 172.21.15.69:0/3233361128 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:10.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b0effd640 1 --2- 172.21.15.69:0/3233361128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x56461284c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:10.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b067fc640 1 -- 172.21.15.69:0/3233361128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 msgr2=0x56461284c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:10.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b067fc640 1 --2- 172.21.15.69:0/3233361128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x56461284c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b067fc640 1 -- 172.21.15.69:0/3233361128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 msgr2=0x56461286c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:10.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b067fc640 1 --2- 172.21.15.69:0/3233361128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 0x56461286c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.990+0000 7f7b067fc640 1 -- 172.21.15.69:0/3233361128 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564612846430 con 0x564612849b30 2024-08-02T18:00:10.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b067fc640 1 --2- 172.21.15.69:0/3233361128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849b30 0x56461286bb60 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7af8010fd0 tx=0x7f7af8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc3ea298775ad19 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:10.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b057fa640 1 -- 172.21.15.69:0/3233361128 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8020070 con 0x564612849b30 2024-08-02T18:00:10.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b057fa640 1 -- 172.21.15.69:0/3233361128 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af8009de0 con 0x564612849b30 2024-08-02T18:00:10.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b057fa640 1 -- 172.21.15.69:0/3233361128 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8097040 con 0x564612849b30 2024-08-02T18:00:10.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 -- 172.21.15.69:0/3233361128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849b30 msgr2=0x56461286bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:10.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 --2- 172.21.15.69:0/3233361128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849b30 0x56461286bb60 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f7af8010fd0 tx=0x7f7af8000bc0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 -- 172.21.15.69:0/3233361128 shutdown_connections 2024-08-02T18:00:10.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 --2- 172.21.15.69:0/3233361128 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849b30 0x56461286bb60 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 --2- 172.21.15.69:0/3233361128 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 0x56461286c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 --2- 172.21.15.69:0/3233361128 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849210 0x56461284c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 -- 172.21.15.69:0/3233361128 >> 172.21.15.69:0/3233361128 conn(0x5646128457a0 msgr2=0x564612850340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:10.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 -- 172.21.15.69:0/3233361128 shutdown_connections 2024-08-02T18:00:10.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.991+0000 7f7b17d63740 1 -- 172.21.15.69:0/3233361128 wait complete. 2024-08-02T18:00:10.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 1 -- start start 2024-08-02T18:00:10.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 1 Processor -- start 2024-08-02T18:00:10.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849210 0x56461285c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:10.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849b30 0x56461285cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:10.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 0x564612857ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:10.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101af820 con 0x564612849210 2024-08-02T18:00:10.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101b0c90 con 0x5646128543f0 2024-08-02T18:00:10.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101afef0 con 0x564612849b30 2024-08-02T18:00:10.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b17d63740 10 client.0 _ll_register_callbacks cb 0x7f7b0800aef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:10.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849210 0x56461285c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:10.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849210 0x56461285c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38270/0 (socket says 172.21.15.69:38270) 2024-08-02T18:00:10.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0effd640 1 -- 172.21.15.69:0/2709510077 learned_addr learned my addr 172.21.15.69:0/2709510077 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:10.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 0x564612857ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:10.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b067fc640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849b30 0x56461285cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:10.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0effd640 1 -- 172.21.15.69:0/2709510077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849b30 msgr2=0x56461285cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:10.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0effd640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849b30 0x56461285cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0effd640 1 -- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 msgr2=0x564612857ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:10.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0effd640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 0x564612857ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0effd640 1 -- 172.21.15.69:0/2709510077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646128582c0 con 0x564612849210 2024-08-02T18:00:10.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b067fc640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849b30 0x56461285cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:10.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.992+0000 7f7b0f7fe640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 0x564612857ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:10.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b0effd640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849210 0x56461285c690 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f7b00031680 tx=0x7f7b00005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf91cd943c927f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7aeeffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7aeeffd640 1 -- 172.21.15.69:0/2709510077 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00049030 con 0x564612849210 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7aeeffd640 1 -- 172.21.15.69:0/2709510077 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9568+0+0 (secure 0 0 0) 0x7f7b00004250 con 0x564612849210 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 1 -- 172.21.15.69:0/2709510077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af8009de0 con 0x564612849210 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7aeeffd640 1 -- 172.21.15.69:0/2709510077 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7b00055050 con 0x564612849210 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 10 client.18837 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 1 -- 172.21.15.69:0/2709510077 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5646128ce6e0 con 0x564612849210 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7aeeffd640 1 -- 172.21.15.69:0/2709510077 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7af8009de0 con 0x564612849210 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7aeeffd640 1 -- 172.21.15.69:0/2709510077 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7b00004a70 con 0x564612849210 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 10 client.18837 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 10 client.18837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 20 client.18837 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:10.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646128cee00 0x5646128d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 4 client.18837 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.993+0000 7f7b17d63740 1 -- 172.21.15.69:0/2709510077 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646127b6400 con 0x5646128cee00 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b067fc640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646128cee00 0x5646128d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b067fc640 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646128cee00 0x5646128d1220 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf03e71636775cd server_cookie=e1c75e07c8c2be65 in_seq=0 out_seq=0 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7aeeffd640 10 client.18837 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7aeeffd640 1 -- 172.21.15.69:0/2709510077 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5646127b6400 con 0x5646128cee00 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7aeeffd640 10 client.18837 handle_command_reply: tid=0 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 client.18837 shutdown 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 -- 172.21.15.69:0/2709510077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849210 msgr2=0x56461285c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849210 0x56461285c690 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f7b00031680 tx=0x7f7b00005c00 comp rx=0 tx=0).stop 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 -- 172.21.15.69:0/2709510077 shutdown_connections 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5646128cee00 0x5646128d1220 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5646128543f0 0x564612857ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564612849b30 0x56461285cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 --2- 172.21.15.69:0/2709510077 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564612849210 0x56461285c690 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 -- 172.21.15.69:0/2709510077 >> 172.21.15.69:0/2709510077 conn(0x5646128457a0 msgr2=0x5646128440e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:10.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7aeeffd640 0 client.18837 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:10.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 -- 172.21.15.69:0/2709510077 shutdown_connections 2024-08-02T18:00:10.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.994+0000 7f7b17d63740 1 -- 172.21.15.69:0/2709510077 wait complete. 2024-08-02T18:00:10.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b17d63740 20 client.18837 trim_cache size 0 max 16384 2024-08-02T18:00:10.011 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:00:10.011 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T18:00:10.011 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T18:00:10.011 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:00:10.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 -- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7afc0a2cf0 msgr2=0x7f7afc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800ae80 tx=0x7f7b08004a40 comp rx=0 tx=0).stop 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 -- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b10101150 msgr2=0x7f7b101028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b10101150 0x7f7b101028c0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f7af800d950 tx=0x7f7af800de10 comp rx=0 tx=0).stop 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 -- 172.21.15.69:0/856448982 shutdown_connections 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7b10101150 0x7f7b101028c0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7afc0a2cf0 0x7f7afc0a51b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7b101067b0 0x7f7b10102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 --2- 172.21.15.69:0/856448982 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7b101007b0 0x7f7b10102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 -- 172.21.15.69:0/856448982 >> 172.21.15.69:0/856448982 conn(0x7f7b100fc4d0 msgr2=0x7f7b1006a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 -- 172.21.15.69:0/856448982 shutdown_connections 2024-08-02T18:00:10.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:09.995+0000 7f7b15965640 1 -- 172.21.15.69:0/856448982 wait complete. 2024-08-02T18:00:10.270 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:10 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T18:00:10.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:10 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T18:00:10.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:10 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T18:00:10.873 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T18:00:10.873 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:00:11.133 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:11 smithi069 ceph-mon[32746]: pgmap v429: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 108 op/s 2024-08-02T18:00:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:11 smithi069 ceph-mon[32746]: osdmap e110: 12 total, 12 up, 12 in 2024-08-02T18:00:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:11 smithi083 ceph-mon[37031]: pgmap v429: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 108 op/s 2024-08-02T18:00:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:11 smithi083 ceph-mon[37031]: osdmap e110: 12 total, 12 up, 12 in 2024-08-02T18:00:11.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:11 smithi077 ceph-mon[37108]: pgmap v429: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 68 KiB/s wr, 108 op/s 2024-08-02T18:00:11.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:11 smithi077 ceph-mon[37108]: osdmap e110: 12 total, 12 up, 12 in 2024-08-02T18:00:11.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.593+0000 7f6f0073d640 1 -- 172.21.15.69:0/4292527842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef8102250 msgr2=0x7f6ef810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:11.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.593+0000 7f6f0073d640 1 --2- 172.21.15.69:0/4292527842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef8102250 0x7f6ef810e780 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009cd0 tx=0x7f6ef002a8e0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.593+0000 7f6f0073d640 1 -- 172.21.15.69:0/4292527842 shutdown_connections 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.593+0000 7f6f0073d640 1 --2- 172.21.15.69:0/4292527842 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef8102250 0x7f6ef810e780 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.593+0000 7f6f0073d640 1 --2- 172.21.15.69:0/4292527842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ef81018b0 0x7f6ef8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.593+0000 7f6f0073d640 1 --2- 172.21.15.69:0/4292527842 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef81078b0 0x7f6ef8107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 -- 172.21.15.69:0/4292527842 >> 172.21.15.69:0/4292527842 conn(0x7f6ef80fd610 msgr2=0x7f6ef80ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 -- 172.21.15.69:0/4292527842 shutdown_connections 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 -- 172.21.15.69:0/4292527842 wait complete. 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 Processor -- start 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 -- start start 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ef81018b0 0x7f6ef81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef8102250 0x7f6ef81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef81078b0 0x7f6ef819ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:11.615 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8111d00 con 0x7f6ef81078b0 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8111b80 con 0x7f6ef8102250 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6f0073d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef8111e80 con 0x7f6ef81018b0 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6efecda640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef81078b0 0x7f6ef819ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6efecda640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef81078b0 0x7f6ef819ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38274/0 (socket says 172.21.15.69:38274) 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.594+0000 7f6efecda640 1 -- 172.21.15.69:0/1363700465 learned_addr learned my addr 172.21.15.69:0/1363700465 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efe4d9640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ef81018b0 0x7f6ef81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efdcd8640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef8102250 0x7f6ef81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efecda640 1 -- 172.21.15.69:0/1363700465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ef81018b0 msgr2=0x7f6ef81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efecda640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ef81018b0 0x7f6ef81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efecda640 1 -- 172.21.15.69:0/1363700465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef8102250 msgr2=0x7f6ef81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efecda640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef8102250 0x7f6ef81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efecda640 1 -- 172.21.15.69:0/1363700465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef819d150 con 0x7f6ef81078b0 2024-08-02T18:00:11.616 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efdcd8640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef8102250 0x7f6ef81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6efecda640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef81078b0 0x7f6ef819ca70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009ca0 tx=0x7f6ef0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6eef7fe640 1 -- 172.21.15.69:0/1363700465 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0004240 con 0x7f6ef81078b0 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6eef7fe640 1 -- 172.21.15.69:0/1363700465 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef00305a0 con 0x7f6ef81078b0 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef819d3e0 con 0x7f6ef81078b0 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6eef7fe640 1 -- 172.21.15.69:0/1363700465 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef003a660 con 0x7f6ef81078b0 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.595+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef81a4d20 con 0x7f6ef81078b0 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.596+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef8101d10 con 0x7f6ef81078b0 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.597+0000 7f6eef7fe640 1 -- 172.21.15.69:0/1363700465 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6ef0030750 con 0x7f6ef81078b0 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.598+0000 7f6eef7fe640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6ee40a2cf0 0x7f6ee40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:11.617 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.598+0000 7f6efe4d9640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6ee40a2cf0 0x7f6ee40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:11.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.598+0000 7f6eef7fe640 1 -- 172.21.15.69:0/1363700465 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f6ef002e030 con 0x7f6ef81078b0 2024-08-02T18:00:11.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.598+0000 7f6efe4d9640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6ee40a2cf0 0x7f6ee40a51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4005d70 tx=0x7f6ef4005d00 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:11.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.601+0000 7f6eef7fe640 1 -- 172.21.15.69:0/1363700465 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6ef00bd510 con 0x7f6ef81078b0 2024-08-02T18:00:11.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.808+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ef81a4f50 con 0x7f6ef81078b0 2024-08-02T18:00:11.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.809+0000 7f6eef7fe640 1 -- 172.21.15.69:0/1363700465 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f6ef00bce10 con 0x7f6ef81078b0 2024-08-02T18:00:11.818 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:00:11.818 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:00:11.819 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T18:00:11.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6ee40a2cf0 msgr2=0x7f6ee40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:11.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6ee40a2cf0 0x7f6ee40a51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4005d70 tx=0x7f6ef4005d00 comp rx=0 tx=0).stop 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef81078b0 msgr2=0x7f6ef819ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef81078b0 0x7f6ef819ca70 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0009ca0 tx=0x7f6ef0003ba0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 shutdown_connections 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ef81078b0 0x7f6ef819ca70 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6ee40a2cf0 0x7f6ee40a51b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ef8102250 0x7f6ef81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 --2- 172.21.15.69:0/1363700465 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ef81018b0 0x7f6ef81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 >> 172.21.15.69:0/1363700465 conn(0x7f6ef80fd610 msgr2=0x7f6ef80fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 shutdown_connections 2024-08-02T18:00:11.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:11.811+0000 7f6f0073d640 1 -- 172.21.15.69:0/1363700465 wait complete. 2024-08-02T18:00:12.422 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:12 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1363700465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:12.499 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:12 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1363700465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:12.578 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:12 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1363700465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:12.608 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T18:00:12.864 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:13.248 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:13 smithi069 ceph-mon[32746]: pgmap v431: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 74 KiB/s wr, 125 op/s 2024-08-02T18:00:13.514 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:13 smithi077 ceph-mon[37108]: pgmap v431: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 74 KiB/s wr, 125 op/s 2024-08-02T18:00:13.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:13 smithi083 ceph-mon[37031]: pgmap v431: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 74 KiB/s wr, 125 op/s 2024-08-02T18:00:14.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 -- 172.21.15.69:0/2755305041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a44101150 msgr2=0x7f4a4410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 --2- 172.21.15.69:0/2755305041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a44101150 0x7f4a4410d680 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009cd0 tx=0x7f4a3c02a8e0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 -- 172.21.15.69:0/2755305041 shutdown_connections 2024-08-02T18:00:14.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 --2- 172.21.15.69:0/2755305041 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a44101150 0x7f4a4410d680 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 --2- 172.21.15.69:0/2755305041 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a441007b0 0x7f4a44100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 --2- 172.21.15.69:0/2755305041 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a441067b0 0x7f4a44106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 -- 172.21.15.69:0/2755305041 >> 172.21.15.69:0/2755305041 conn(0x7f4a440fc4d0 msgr2=0x7f4a440fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 -- 172.21.15.69:0/2755305041 shutdown_connections 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 -- 172.21.15.69:0/2755305041 wait complete. 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.437+0000 7f4a49c46640 1 Processor -- start 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a49c46640 1 -- start start 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a49c46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a441007b0 0x7f4a441980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a49c46640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44101150 0x7f4a44198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a49c46640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a441067b0 0x7f4a4419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a49c46640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44110bb0 con 0x7f4a441067b0 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a49c46640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44110a30 con 0x7f4a44101150 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a49c46640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a44110d30 con 0x7f4a441007b0 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a441067b0 0x7f4a4419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a441067b0 0x7f4a4419c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38296/0 (socket says 172.21.15.69:38296) 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 -- 172.21.15.69:0/3638591099 learned_addr learned my addr 172.21.15.69:0/3638591099 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a437fe640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a441007b0 0x7f4a441980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44101150 0x7f4a44198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 -- 172.21.15.69:0/3638591099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a441007b0 msgr2=0x7f4a441980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a441007b0 0x7f4a441980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 -- 172.21.15.69:0/3638591099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44101150 msgr2=0x7f4a44198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44101150 0x7f4a44198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 -- 172.21.15.69:0/3638591099 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4419d070 con 0x7f4a441067b0 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a441067b0 0x7f4a4419c990 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c005cc0 tx=0x7f4a3c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.438+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3638591099 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c004040 con 0x7f4a441067b0 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.439+0000 7f4a49c46640 1 -- 172.21.15.69:0/3638591099 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4419d300 con 0x7f4a441067b0 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.439+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3638591099 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3c0046c0 con 0x7f4a441067b0 2024-08-02T18:00:14.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.439+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3638591099 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c046480 con 0x7f4a441067b0 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.439+0000 7f4a49c46640 1 -- 172.21.15.69:0/3638591099 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a441a4c40 con 0x7f4a441067b0 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3638591099 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4a3c0305a0 con 0x7f4a441067b0 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a4c044740 1 Processor -- start 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a4c044740 1 -- start start 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17efd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efd910 0x561b17f07c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f0a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a4c044740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561b17e62a40 con 0x561b17f081d0 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a4c044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b17dfa9d0 con 0x561b17efd910 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a4c044740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561b17e5ea00 con 0x561b17efcff0 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f0a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a40ff9640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a2c0a2cf0 0x7f4a2c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f0a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38306/0 (socket says 172.21.15.69:38306) 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a42ffd640 1 -- 172.21.15.69:0/189790218 learned_addr learned my addr 172.21.15.69:0/189790218 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.441+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a2c0a2cf0 0x7f4a2c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17efd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a437fe640 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efd910 0x561b17f07c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a40ff9640 1 -- 172.21.15.69:0/3638591099 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f4a3c02e030 con 0x7f4a441067b0 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a42ffd640 1 -- 172.21.15.69:0/189790218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 msgr2=0x561b17efd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17efd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a42ffd640 1 -- 172.21.15.69:0/189790218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efd910 msgr2=0x561b17f07c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efd910 0x561b17f07c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a42ffd640 1 -- 172.21.15.69:0/189790218 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b17e01650 con 0x561b17f081d0 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f0a5c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f4a38012490 tx=0x7f4a3800a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123c8c9241727ec server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a43fff640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a2c0a2cf0 0x7f4a2c0a51b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0ee8c0 tx=0x7f4a3c0dd040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:14.028 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a29ffb640 1 -- 172.21.15.69:0/189790218 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38018030 con 0x561b17f081d0 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a29ffb640 1 -- 172.21.15.69:0/189790218 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a38005500 con 0x561b17f081d0 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.442+0000 7f4a29ffb640 1 -- 172.21.15.69:0/189790218 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38010810 con 0x561b17f081d0 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- 172.21.15.69:0/189790218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 msgr2=0x561b17f0a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f0a5c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f4a38012490 tx=0x7f4a3800a630 comp rx=0 tx=0).stop 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- 172.21.15.69:0/189790218 shutdown_connections 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f0a5c0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efd910 0x561b17f07c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 --2- 172.21.15.69:0/189790218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17efd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- 172.21.15.69:0/189790218 >> 172.21.15.69:0/189790218 conn(0x561b17ef9580 msgr2=0x561b17ef9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- 172.21.15.69:0/189790218 shutdown_connections 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- 172.21.15.69:0/189790218 wait complete. 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- start start 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 Processor -- start 2024-08-02T18:00:14.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17f05090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17efd910 0x561b17f055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17f081d0 0x561b17f05b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x561b17f0ada0 con 0x561b17efd910 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x561b17e1db20 con 0x561b17f081d0 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.443+0000 7f4a4c044740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x561b17f0af20 con 0x561b17efcff0 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a4c044740 10 client.0 _ll_register_callbacks cb 0x561b17e1bbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17efd910 0x561b17f055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17efd910 0x561b17f055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38318/0 (socket says 172.21.15.69:38318) 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 -- 172.21.15.69:0/2813514866 learned_addr learned my addr 172.21.15.69:0/2813514866 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17f081d0 0x561b17f05b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17f05090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 -- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 msgr2=0x561b17f05090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17f05090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 -- 172.21.15.69:0/2813514866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17f081d0 msgr2=0x561b17f05b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17f081d0 0x561b17f05b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 -- 172.21.15.69:0/2813514866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b17bd43c0 con 0x561b17efd910 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17f05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a437fe640 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17efd910 0x561b17f055d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400b940 tx=0x7f4a3400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5a0f27dbca3a59 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a22ffd640 1 -- 172.21.15.69:0/2813514866 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3401b440 con 0x561b17efd910 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a22ffd640 1 -- 172.21.15.69:0/2813514866 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f4a3401b5e0 con 0x561b17efd910 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a4c044740 1 -- 172.21.15.69:0/2813514866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a38005500 con 0x561b17efd910 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a22ffd640 1 -- 172.21.15.69:0/2813514866 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a34024050 con 0x561b17efd910 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a4c044740 10 client.18855 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.444+0000 7f4a4c044740 1 -- 172.21.15.69:0/2813514866 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b17f16a30 con 0x561b17efd910 2024-08-02T18:00:14.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a22ffd640 1 -- 172.21.15.69:0/2813514866 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a38005500 con 0x561b17efd910 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a22ffd640 1 -- 172.21.15.69:0/2813514866 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a34019020 con 0x561b17efd910 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a4c044740 10 client.18855 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a4c044740 10 client.18855 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a4c044740 20 client.18855 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561b17f17f90 0x561b17f1a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a4c044740 4 client.18855 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a4c044740 1 -- 172.21.15.69:0/2813514866 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b17e6ffb0 con 0x561b17f17f90 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.445+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561b17f17f90 0x561b17f1a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561b17f17f90 0x561b17f1a380 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b238e1309fd352b server_cookie=f5791216e944cd94 in_seq=0 out_seq=0 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a22ffd640 10 client.18855 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a22ffd640 1 -- 172.21.15.69:0/2813514866 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561b17e6ffb0 con 0x561b17f17f90 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a22ffd640 10 client.18855 handle_command_reply: tid=0 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 client.18855 shutdown 2024-08-02T18:00:14.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 -- 172.21.15.69:0/2813514866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17efd910 msgr2=0x561b17f055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17efd910 0x561b17f055d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400b940 tx=0x7f4a3400be00 comp rx=0 tx=0).stop 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 -- 172.21.15.69:0/2813514866 shutdown_connections 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561b17f17f90 0x561b17f1a380 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17efd910 0x561b17f055d0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17f081d0 0x561b17f05b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a22ffd640 0 client.18855 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2813514866 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efcff0 0x561b17f05090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.446+0000 7f4a4c044740 1 -- 172.21.15.69:0/2813514866 >> 172.21.15.69:0/2813514866 conn(0x561b17ef9580 msgr2=0x561b17eff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.447+0000 7f4a4c044740 1 -- 172.21.15.69:0/2813514866 shutdown_connections 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.447+0000 7f4a4c044740 1 -- 172.21.15.69:0/2813514866 wait complete. 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.447+0000 7f4a4c044740 20 client.18855 trim_cache size 0 max 16384 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.459+0000 7f4a4c044740 1 Processor -- start 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.459+0000 7f4a4c044740 1 -- start start 2024-08-02T18:00:14.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.459+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 0x561b17f05960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.459+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f10550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.459+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f10a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.459+0000 7f4a4c044740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441b1730 con 0x561b17f081d0 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.459+0000 7f4a4c044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441af000 con 0x561b17efcff0 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.459+0000 7f4a4c044740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441ace70 con 0x561b17efd910 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f10a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f10a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38326/0 (socket says 172.21.15.69:38326) 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 -- 172.21.15.69:0/2723852192 learned_addr learned my addr 172.21.15.69:0/2723852192 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a437fe640 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f10550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 0x561b17f05960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 -- 172.21.15.69:0/2723852192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 msgr2=0x561b17f10550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f10550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 -- 172.21.15.69:0/2723852192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 msgr2=0x561b17f05960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 0x561b17f05960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 -- 172.21.15.69:0/2723852192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b17f1a6f0 con 0x561b17f081d0 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f10a90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f4a38004be0 tx=0x7f4a3800a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7454077ca058867 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a297fa640 1 -- 172.21.15.69:0/2723852192 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38018040 con 0x561b17f081d0 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a297fa640 1 -- 172.21.15.69:0/2723852192 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a380048d0 con 0x561b17f081d0 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.460+0000 7f4a297fa640 1 -- 172.21.15.69:0/2723852192 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38005350 con 0x561b17f081d0 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 -- 172.21.15.69:0/2723852192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 msgr2=0x561b17f10a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f10a90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f4a38004be0 tx=0x7f4a3800a630 comp rx=0 tx=0).stop 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 -- 172.21.15.69:0/2723852192 shutdown_connections 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f10a90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f10550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 --2- 172.21.15.69:0/2723852192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 0x561b17f05960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 -- 172.21.15.69:0/2723852192 >> 172.21.15.69:0/2723852192 conn(0x561b17ef9580 msgr2=0x561b17f07000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:14.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 -- 172.21.15.69:0/2723852192 shutdown_connections 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 -- 172.21.15.69:0/2723852192 wait complete. 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 -- start start 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.461+0000 7f4a4c044740 1 Processor -- start 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 0x561b17f02680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f02bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f1f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441ad4c0 con 0x561b17f081d0 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441ae930 con 0x561b17efcff0 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a441adb90 con 0x561b17efd910 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 10 client.0 _ll_register_callbacks cb 0x7f4a34004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f1f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f1f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:38330/0 (socket says 172.21.15.69:38330) 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 -- 172.21.15.69:0/3418266301 learned_addr learned my addr 172.21.15.69:0/3418266301 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:14.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a437fe640 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f02bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 0x561b17f02680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 -- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 msgr2=0x561b17f02bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f02bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 -- 172.21.15.69:0/3418266301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 msgr2=0x561b17f02680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 0x561b17f02680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 -- 172.21.15.69:0/3418266301 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b17f1fa50 con 0x561b17f081d0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a437fe640 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f02bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a42ffd640 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f1f220 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4a38012da0 tx=0x7f4a38004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30cbbc4cdc408a9c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3418266301 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38018070 con 0x561b17f081d0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3418266301 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f4a380050d0 con 0x561b17f081d0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 1 -- 172.21.15.69:0/3418266301 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a380048d0 con 0x561b17f081d0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3418266301 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a380052d0 con 0x561b17f081d0 2024-08-02T18:00:14.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 10 client.18861 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.462+0000 7f4a4c044740 1 -- 172.21.15.69:0/3418266301 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561b17f867b0 con 0x561b17f081d0 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3418266301 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a380048d0 con 0x561b17f081d0 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3418266301 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4a3800f040 con 0x561b17f081d0 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a4c044740 10 client.18861 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a4c044740 10 client.18861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a4c044740 20 client.18861 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a4c044740 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561b17f17f90 0x561b17f88cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a4c044740 4 client.18861 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a4c044740 1 -- 172.21.15.69:0/3418266301 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b17e6a4c0 con 0x561b17f17f90 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a437fe640 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561b17f17f90 0x561b17f88cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a22ffd640 10 client.18861 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:14.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.463+0000 7f4a437fe640 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561b17f17f90 0x561b17f88cc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d968a86a66729dc8 server_cookie=47a60c4fae4d8d92 in_seq=0 out_seq=0 2024-08-02T18:00:14.039 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:14.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a22ffd640 1 -- 172.21.15.69:0/3418266301 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b17e6a4c0 con 0x561b17f17f90 2024-08-02T18:00:14.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a22ffd640 10 client.18861 handle_command_reply: tid=0 2024-08-02T18:00:14.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 client.18861 shutdown 2024-08-02T18:00:14.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 -- 172.21.15.69:0/3418266301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 msgr2=0x561b17f1f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f1f220 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4a38012da0 tx=0x7f4a38004f30 comp rx=0 tx=0).stop 2024-08-02T18:00:14.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 -- 172.21.15.69:0/3418266301 shutdown_connections 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x561b17f17f90 0x561b17f88cc0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a22ffd640 0 client.18861 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x561b17f081d0 0x561b17f1f220 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x561b17efd910 0x561b17f02bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 --2- 172.21.15.69:0/3418266301 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x561b17efcff0 0x561b17f02680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 -- 172.21.15.69:0/3418266301 >> 172.21.15.69:0/3418266301 conn(0x561b17ef9580 msgr2=0x561b17f03830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 -- 172.21.15.69:0/3418266301 shutdown_connections 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 1 -- 172.21.15.69:0/3418266301 wait complete. 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a4c044740 20 client.18861 trim_cache size 0 max 16384 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a49c46640 1 -- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a2c0a2cf0 msgr2=0x7f4a2c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a49c46640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a2c0a2cf0 0x7f4a2c0a51b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0ee8c0 tx=0x7f4a3c0dd040 comp rx=0 tx=0).stop 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a49c46640 1 -- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a441067b0 msgr2=0x7f4a4419c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.464+0000 7f4a49c46640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a441067b0 0x7f4a4419c990 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c005cc0 tx=0x7f4a3c003970 comp rx=0 tx=0).stop 2024-08-02T18:00:14.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.465+0000 7f4a49c46640 1 -- 172.21.15.69:0/3638591099 shutdown_connections 2024-08-02T18:00:14.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.465+0000 7f4a49c46640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4a441067b0 0x7f4a4419c990 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.465+0000 7f4a49c46640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4a2c0a2cf0 0x7f4a2c0a51b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.465+0000 7f4a49c46640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4a44101150 0x7f4a44198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.465+0000 7f4a49c46640 1 --2- 172.21.15.69:0/3638591099 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a441007b0 0x7f4a441980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:14.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.465+0000 7f4a49c46640 1 -- 172.21.15.69:0/3638591099 >> 172.21.15.69:0/3638591099 conn(0x7f4a440fc4d0 msgr2=0x7f4a44104560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:14.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.465+0000 7f4a49c46640 1 -- 172.21.15.69:0/3638591099 shutdown_connections 2024-08-02T18:00:14.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:13.465+0000 7f4a49c46640 1 -- 172.21.15.69:0/3638591099 wait complete. 2024-08-02T18:00:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 785.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T18:00:15.346 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:15 smithi077 ceph-mon[37108]: pgmap v432: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 57 KiB/s wr, 121 op/s 2024-08-02T18:00:15.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:15 smithi069 ceph-mon[32746]: pgmap v432: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 57 KiB/s wr, 121 op/s 2024-08-02T18:00:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:15 smithi083 ceph-mon[37031]: pgmap v432: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 57 KiB/s wr, 121 op/s 2024-08-02T18:00:15.874 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T18:00:16.141 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:16.564 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 -- 172.21.15.69:0/1617114767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 msgr2=0x7f63e010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 --2- 172.21.15.69:0/1617114767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 0x7f63e010d770 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f63d8009ca0 tx=0x7f63d802a950 comp rx=0 tx=0).stop 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 -- 172.21.15.69:0/1617114767 shutdown_connections 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 --2- 172.21.15.69:0/1617114767 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 0x7f63e010d770 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 --2- 172.21.15.69:0/1617114767 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63e01022a0 0x7f63e01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 --2- 172.21.15.69:0/1617114767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63e0100b50 0x7f63e0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 -- 172.21.15.69:0/1617114767 >> 172.21.15.69:0/1617114767 conn(0x7f63e00fc500 msgr2=0x7f63e00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 -- 172.21.15.69:0/1617114767 shutdown_connections 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.631+0000 7f63e77d7640 1 -- 172.21.15.69:0/1617114767 wait complete. 2024-08-02T18:00:16.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e77d7640 1 Processor -- start 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e77d7640 1 -- start start 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e77d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63e0100b50 0x7f63e00fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e77d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63e01022a0 0x7f63e00fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e77d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 0x7f63e00fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e77d7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0110cb0 con 0x7f63e0102c20 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e77d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0110b30 con 0x7f63e0100b50 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e77d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63e0110e30 con 0x7f63e01022a0 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e5d74640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 0x7f63e00fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e5d74640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 0x7f63e00fd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37466/0 (socket says 172.21.15.69:37466) 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e5d74640 1 -- 172.21.15.69:0/3232289 learned_addr learned my addr 172.21.15.69:0/3232289 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e5573640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63e0100b50 0x7f63e00fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.632+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63e01022a0 0x7f63e00fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63e5d74640 1 -- 172.21.15.69:0/3232289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63e01022a0 msgr2=0x7f63e00fcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63e5d74640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63e01022a0 0x7f63e00fcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63e5d74640 1 -- 172.21.15.69:0/3232289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63e0100b50 msgr2=0x7f63e00fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63e5d74640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63e0100b50 0x7f63e00fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63e5d74640 1 -- 172.21.15.69:0/3232289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e01a65c0 con 0x7f63e0102c20 2024-08-02T18:00:16.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63e5d74640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 0x7f63e00fd430 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f63d8006010 tx=0x7f63d802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:16.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63d67fc640 1 -- 172.21.15.69:0/3232289 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8034040 con 0x7f63e0102c20 2024-08-02T18:00:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63d67fc640 1 -- 172.21.15.69:0/3232289 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d8003e00 con 0x7f63e0102c20 2024-08-02T18:00:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63e77d7640 1 -- 172.21.15.69:0/3232289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e01a6850 con 0x7f63e0102c20 2024-08-02T18:00:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63d67fc640 1 -- 172.21.15.69:0/3232289 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d8045480 con 0x7f63e0102c20 2024-08-02T18:00:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.633+0000 7f63e77d7640 1 -- 172.21.15.69:0/3232289 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e01a6d30 con 0x7f63e0102c20 2024-08-02T18:00:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63e9bd5740 1 Processor -- start 2024-08-02T18:00:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63d67fc640 1 -- 172.21.15.69:0/3232289 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f63d8037020 con 0x7f63e0102c20 2024-08-02T18:00:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63e9bd5740 1 -- start start 2024-08-02T18:00:16.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb6910 0x55ba67fc0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fc11d0 0x55ba67fc35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ba67f1ba40 con 0x55ba67fb6910 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ba67eb39d0 con 0x55ba67fb5ff0 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.636+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba67f17a00 con 0x55ba67fc11d0 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb6910 0x55ba67fc0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb6910 0x55ba67fc0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37470/0 (socket says 172.21.15.69:37470) 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 -- 172.21.15.69:0/3155609474 learned_addr learned my addr 172.21.15.69:0/3155609474 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fc11d0 0x55ba67fc35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 -- 172.21.15.69:0/3155609474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fc11d0 msgr2=0x55ba67fc35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:16.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fc11d0 0x55ba67fc35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 -- 172.21.15.69:0/3155609474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 msgr2=0x55ba67fb63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63d67fc640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63cc0a2cf0 0x7f63cc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 -- 172.21.15.69:0/3155609474 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba67eba650 con 0x55ba67fb6910 2024-08-02T18:00:16.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5d74640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63cc0a2cf0 0x7f63cc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63e5573640 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb6910 0x55ba67fc0c90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00ad00 tx=0x7f63dc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f622acb4e54bc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63d67fc640 1 -- 172.21.15.69:0/3232289 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f63d80f07f0 con 0x7f63e0102c20 2024-08-02T18:00:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63cb7fe640 1 -- 172.21.15.69:0/3155609474 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc01d070 con 0x55ba67fb6910 2024-08-02T18:00:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.637+0000 7f63cb7fe640 1 -- 172.21.15.69:0/3155609474 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63dc018b20 con 0x55ba67fb6910 2024-08-02T18:00:16.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63cb7fe640 1 -- 172.21.15.69:0/3155609474 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc010c20 con 0x55ba67fb6910 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3155609474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb6910 msgr2=0x55ba67fc0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e5d74640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63cc0a2cf0 0x7f63cc0a51b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f63d80edc90 tx=0x7f63d808e970 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb6910 0x55ba67fc0c90 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00ad00 tx=0x7f63dc00f790 comp rx=0 tx=0).stop 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3155609474 shutdown_connections 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb6910 0x55ba67fc0c90 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fc11d0 0x55ba67fc35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3155609474 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3155609474 >> 172.21.15.69:0/3155609474 conn(0x55ba67fb2580 msgr2=0x55ba67fb2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:16.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3155609474 shutdown_connections 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.638+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3155609474 wait complete. 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 1 -- start start 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 1 Processor -- start 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fbe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fbe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fbeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ba67fc3da0 con 0x55ba67fc11d0 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ba67ed6b20 con 0x55ba67fb5ff0 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ba67fc3f20 con 0x55ba67fb6910 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e9bd5740 10 client.0 _ll_register_callbacks cb 0x55ba67ed4bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e4d72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fbeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e4d72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fbeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37480/0 (socket says 172.21.15.69:37480) 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e4d72640 1 -- 172.21.15.69:0/3891795894 learned_addr learned my addr 172.21.15.69:0/3891795894 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fbe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.639+0000 7f63e5573640 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fbe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e4d72640 1 -- 172.21.15.69:0/3891795894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 msgr2=0x55ba67fbe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fbe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e4d72640 1 -- 172.21.15.69:0/3891795894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 msgr2=0x55ba67fbe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fbe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e4d72640 1 -- 172.21.15.69:0/3891795894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba67c8d3c0 con 0x55ba67fc11d0 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fbeb10 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f63d0011dd0 tx=0x7f63d0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc10efa9a7c387c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:16.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3891795894 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d000fe60 con 0x55ba67fc11d0 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3891795894 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f63d0004020 con 0x55ba67fc11d0 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3891795894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63dc018b20 con 0x55ba67fc11d0 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3891795894 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63d0004220 con 0x55ba67fc11d0 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e9bd5740 10 client.18873 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3891795894 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba67fcf980 con 0x55ba67fc11d0 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.640+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3891795894 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63dc018b20 con 0x55ba67fc11d0 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3891795894 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f63d0004a70 con 0x55ba67fc11d0 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63e9bd5740 10 client.18873 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63e9bd5740 10 client.18873 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63e9bd5740 20 client.18873 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55ba67fd1270 0x55ba67fd36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63e9bd5740 4 client.18873 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3891795894 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba67f28fb0 con 0x55ba67fd1270 2024-08-02T18:00:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63e5573640 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55ba67fd1270 0x55ba67fd36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63e5573640 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55ba67fd1270 0x55ba67fd36d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca6d9ccb1aee93cf server_cookie=9f97d20c52618c5c in_seq=0 out_seq=0 2024-08-02T18:00:16.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.641+0000 7f63c8ff9640 10 client.18873 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3891795894 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ba67f28fb0 con 0x55ba67fd1270 2024-08-02T18:00:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63c8ff9640 10 client.18873 handle_command_reply: tid=0 2024-08-02T18:00:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 client.18873 shutdown 2024-08-02T18:00:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3891795894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 msgr2=0x55ba67fbeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fbeb10 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f63d0011dd0 tx=0x7f63d0013040 comp rx=0 tx=0).stop 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3891795894 shutdown_connections 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55ba67fd1270 0x55ba67fd36d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fbeb10 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fbe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63c8ff9640 0 client.18873 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3891795894 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fbe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3891795894 >> 172.21.15.69:0/3891795894 conn(0x55ba67fb2580 msgr2=0x55ba67fb82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3891795894 shutdown_connections 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3891795894 wait complete. 2024-08-02T18:00:16.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.642+0000 7f63e9bd5740 20 client.18873 trim_cache size 0 max 16384 2024-08-02T18:00:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.655+0000 7f63e9bd5740 1 Processor -- start 2024-08-02T18:00:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.655+0000 7f63e9bd5740 1 -- start start 2024-08-02T18:00:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.655+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fd8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.658 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fd9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01b39c0 con 0x55ba67fc11d0 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01b1290 con 0x55ba67fb5ff0 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01af100 con 0x55ba67fb6910 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fd9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fd9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37494/0 (socket says 172.21.15.69:37494) 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 -- 172.21.15.69:0/2917001784 learned_addr learned my addr 172.21.15.69:0/2917001784 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e5573640 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fd8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 -- 172.21.15.69:0/2917001784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 msgr2=0x55ba67fd8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fd8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.659 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 -- 172.21.15.69:0/2917001784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 msgr2=0x55ba67fb8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 -- 172.21.15.69:0/2917001784 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba67fb6740 con 0x55ba67fc11d0 2024-08-02T18:00:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.656+0000 7f63e4d72640 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fd9520 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f63d0004e10 tx=0x7f63d0005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f09aaeceb7ab6c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63caffd640 1 -- 172.21.15.69:0/2917001784 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d000fe60 con 0x55ba67fc11d0 2024-08-02T18:00:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63caffd640 1 -- 172.21.15.69:0/2917001784 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d0004c10 con 0x55ba67fc11d0 2024-08-02T18:00:16.660 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63caffd640 1 -- 172.21.15.69:0/2917001784 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0003840 con 0x55ba67fc11d0 2024-08-02T18:00:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63e9bd5740 1 -- 172.21.15.69:0/2917001784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 msgr2=0x55ba67fd9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fd9520 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f63d0004e10 tx=0x7f63d0005e40 comp rx=0 tx=0).stop 2024-08-02T18:00:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63e9bd5740 1 -- 172.21.15.69:0/2917001784 shutdown_connections 2024-08-02T18:00:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fc11d0 0x55ba67fd9520 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fd8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/2917001784 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fb5ff0 0x55ba67fb8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.657+0000 7f63e9bd5740 1 -- 172.21.15.69:0/2917001784 >> 172.21.15.69:0/2917001784 conn(0x55ba67fb2580 msgr2=0x55ba67fbc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:16.664 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 -- 172.21.15.69:0/2917001784 shutdown_connections 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 -- 172.21.15.69:0/2917001784 wait complete. 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 -- start start 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 Processor -- start 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb5ff0 0x55ba67fc4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fc4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fc11d0 0x55ba68039fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01af750 con 0x55ba67fb5ff0 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01b0bc0 con 0x55ba67fc11d0 2024-08-02T18:00:16.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63e01afe20 con 0x55ba67fb6910 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e9bd5740 10 client.0 _ll_register_callbacks cb 0x7f63d000a9f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e4d72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb5ff0 0x55ba67fc4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e4d72640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb5ff0 0x55ba67fc4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37510/0 (socket says 172.21.15.69:37510) 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e4d72640 1 -- 172.21.15.69:0/3336265033 learned_addr learned my addr 172.21.15.69:0/3336265033 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.658+0000 7f63e5573640 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fc4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e4d72640 1 -- 172.21.15.69:0/3336265033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 msgr2=0x55ba67fc4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fc4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e4d72640 1 -- 172.21.15.69:0/3336265033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fc11d0 msgr2=0x55ba68039fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fc11d0 0x55ba68039fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e4d72640 1 -- 172.21.15.69:0/3336265033 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6803a5b0 con 0x55ba67fb5ff0 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e4d72640 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb5ff0 0x55ba67fc4680 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f63d0011920 tx=0x7f63d0011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb3235d4ce38d31 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:16.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3336265033 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d00043c0 con 0x55ba67fb5ff0 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3336265033 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f63d0035040 con 0x55ba67fb5ff0 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3336265033 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63d0004c10 con 0x55ba67fb5ff0 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3336265033 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63d0045050 con 0x55ba67fb5ff0 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e9bd5740 10 client.18879 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3336265033 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ba6803b870 con 0x55ba67fb5ff0 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.659+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3336265033 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63d0004c10 con 0x55ba67fb5ff0 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3336265033 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f63d0004fe0 con 0x55ba67fb5ff0 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e9bd5740 10 client.18879 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e9bd5740 10 client.18879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e9bd5740 20 client.18879 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55ba6803bf20 0x55ba6803e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e9bd5740 4 client.18879 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:16.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3336265033 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba67f234c0 con 0x55ba6803bf20 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e5573640 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55ba6803bf20 0x55ba6803e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63c8ff9640 10 client.18879 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e5573640 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55ba6803bf20 0x55ba6803e340 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d6c769e020f90b1 server_cookie=6adaafaacb1cbca2 in_seq=0 out_seq=0 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63c8ff9640 1 -- 172.21.15.69:0/3336265033 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba67f234c0 con 0x55ba6803bf20 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63c8ff9640 10 client.18879 handle_command_reply: tid=0 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.660+0000 7f63e9bd5740 1 client.18879 shutdown 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3336265033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb5ff0 msgr2=0x55ba67fc4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb5ff0 0x55ba67fc4680 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f63d0011920 tx=0x7f63d0011950 comp rx=0 tx=0).stop 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3336265033 shutdown_connections 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55ba6803bf20 0x55ba6803e340 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ba67fc11d0 0x55ba68039fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ba67fb6910 0x55ba67fc4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 --2- 172.21.15.69:0/3336265033 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ba67fb5ff0 0x55ba67fc4680 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3336265033 >> 172.21.15.69:0/3336265033 conn(0x55ba67fb2580 msgr2=0x55ba67fb0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63c8ff9640 0 client.18879 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3336265033 shutdown_connections 2024-08-02T18:00:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 1 -- 172.21.15.69:0/3336265033 wait complete. 2024-08-02T18:00:16.669 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e9bd5740 20 client.18879 trim_cache size 0 max 16384 2024-08-02T18:00:16.669 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e77d7640 1 -- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63cc0a2cf0 msgr2=0x7f63cc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e77d7640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63cc0a2cf0 0x7f63cc0a51b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f63d80edc90 tx=0x7f63d808e970 comp rx=0 tx=0).stop 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e77d7640 1 -- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 msgr2=0x7f63e00fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.661+0000 7f63e77d7640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 0x7f63e00fd430 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f63d8006010 tx=0x7f63d802f6f0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.662+0000 7f63e77d7640 1 -- 172.21.15.69:0/3232289 shutdown_connections 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.662+0000 7f63e77d7640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f63e0102c20 0x7f63e00fd430 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.662+0000 7f63e77d7640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f63cc0a2cf0 0x7f63cc0a51b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.662+0000 7f63e77d7640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f63e01022a0 0x7f63e00fcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.670 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.662+0000 7f63e77d7640 1 --2- 172.21.15.69:0/3232289 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f63e0100b50 0x7f63e00fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.662+0000 7f63e77d7640 1 -- 172.21.15.69:0/3232289 >> 172.21.15.69:0/3232289 conn(0x7f63e00fc500 msgr2=0x7f63e0069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.662+0000 7f63e77d7640 1 -- 172.21.15.69:0/3232289 shutdown_connections 2024-08-02T18:00:16.671 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:16.662+0000 7f63e77d7640 1 -- 172.21.15.69:0/3232289 wait complete. 2024-08-02T18:00:17.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:17 smithi069 ceph-mon[32746]: pgmap v433: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 32 KiB/s wr, 119 op/s 2024-08-02T18:00:17.435 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T18:00:17.469 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:17 smithi083 ceph-mon[37031]: pgmap v433: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 32 KiB/s wr, 119 op/s 2024-08-02T18:00:17.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:17 smithi077 ceph-mon[37108]: pgmap v433: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 32 KiB/s wr, 119 op/s 2024-08-02T18:00:17.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 -- 172.21.15.69:0/2579709126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c1057a0 msgr2=0x7fc27c105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:17.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 --2- 172.21.15.69:0/2579709126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c1057a0 0x7fc27c105b80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009900 tx=0x7fc26c02a410 comp rx=0 tx=0).stop 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 -- 172.21.15.69:0/2579709126 shutdown_connections 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 --2- 172.21.15.69:0/2579709126 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc27c101910 0x7fc27c10d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 --2- 172.21.15.69:0/2579709126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c100f70 0x7fc27c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 --2- 172.21.15.69:0/2579709126 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c1057a0 0x7fc27c105b80 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 -- 172.21.15.69:0/2579709126 >> 172.21.15.69:0/2579709126 conn(0x7fc27c076f40 msgr2=0x7fc27c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 -- 172.21.15.69:0/2579709126 shutdown_connections 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 -- 172.21.15.69:0/2579709126 wait complete. 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.937+0000 7fc283251640 1 Processor -- start 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc283251640 1 -- start start 2024-08-02T18:00:17.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc283251640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c100f70 0x7fc27c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc283251640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc27c101910 0x7fc27c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc283251640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c1057a0 0x7fc27c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc283251640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c111d00 con 0x7fc27c100f70 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc283251640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c111b80 con 0x7fc27c101910 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc283251640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c111e80 con 0x7fc27c1057a0 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c100f70 0x7fc27c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c100f70 0x7fc27c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37528/0 (socket says 172.21.15.69:37528) 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 -- 172.21.15.69:0/115709872 learned_addr learned my addr 172.21.15.69:0/115709872 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:17.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc2817ee640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c1057a0 0x7fc27c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc27bfff640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc27c101910 0x7fc27c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 -- 172.21.15.69:0/115709872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c1057a0 msgr2=0x7fc27c19ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c1057a0 0x7fc27c19ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 -- 172.21.15.69:0/115709872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc27c101910 msgr2=0x7fc27c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc27c101910 0x7fc27c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 -- 172.21.15.69:0/115709872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c19d130 con 0x7fc27c100f70 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc27bfff640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc27c101910 0x7fc27c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc2817ee640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c1057a0 0x7fc27c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.938+0000 7fc280fed640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c100f70 0x7fc27c198180 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc26c006010 tx=0x7fc26c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:17.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.939+0000 7fc279ffb640 1 -- 172.21.15.69:0/115709872 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c03f030 con 0x7fc27c100f70 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.939+0000 7fc279ffb640 1 -- 172.21.15.69:0/115709872 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26c004440 con 0x7fc27c100f70 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.939+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c19d3c0 con 0x7fc27c100f70 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.939+0000 7fc279ffb640 1 -- 172.21.15.69:0/115709872 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c035070 con 0x7fc27c100f70 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.939+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c1a4d00 con 0x7fc27c100f70 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.940+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27c1013d0 con 0x7fc27c100f70 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.941+0000 7fc279ffb640 1 -- 172.21.15.69:0/115709872 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc26c042030 con 0x7fc27c100f70 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.941+0000 7fc279ffb640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2680a2d70 0x7fc2680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.942+0000 7fc27bfff640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2680a2d70 0x7fc2680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:17.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.942+0000 7fc279ffb640 1 -- 172.21.15.69:0/115709872 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7fc26c0f0010 con 0x7fc27c100f70 2024-08-02T18:00:17.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.942+0000 7fc27bfff640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2680a2d70 0x7fc2680a5230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0ff440 tx=0x7fc264005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:17.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:17.945+0000 7fc279ffb640 1 -- 172.21.15.69:0/115709872 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fc26c0bc880 con 0x7fc27c100f70 2024-08-02T18:00:17.989 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:18.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.161+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc27c1994e0 con 0x7fc27c100f70 2024-08-02T18:00:18.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.161+0000 7fc279ffb640 1 -- 172.21.15.69:0/115709872 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fc27c1994e0 con 0x7fc27c100f70 2024-08-02T18:00:18.163 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:00:18.163 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:00:18.163 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T18:00:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2680a2d70 msgr2=0x7fc2680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2680a2d70 0x7fc2680a5230 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0ff440 tx=0x7fc264005f90 comp rx=0 tx=0).stop 2024-08-02T18:00:18.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c100f70 msgr2=0x7fc27c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c100f70 0x7fc27c198180 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc26c006010 tx=0x7fc26c004040 comp rx=0 tx=0).stop 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 shutdown_connections 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fc2680a2d70 0x7fc2680a5230 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc27c1057a0 0x7fc27c19ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fc27c101910 0x7fc27c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 --2- 172.21.15.69:0/115709872 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fc27c100f70 0x7fc27c198180 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.163+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 >> 172.21.15.69:0/115709872 conn(0x7fc27c076f40 msgr2=0x7fc27c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.164+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 shutdown_connections 2024-08-02T18:00:18.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:18.164+0000 7fc283251640 1 -- 172.21.15.69:0/115709872 wait complete. 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 -- 172.21.15.69:0/2834891799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 msgr2=0x7fba0010d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 --2- 172.21.15.69:0/2834891799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 0x7fba0010d5e0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009cd0 tx=0x7fb9f802a8e0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 -- 172.21.15.69:0/2834891799 shutdown_connections 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 --2- 172.21.15.69:0/2834891799 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 0x7fba0010d5e0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 --2- 172.21.15.69:0/2834891799 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00100710 0x7fba00100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 --2- 172.21.15.69:0/2834891799 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba00106720 0x7fba00106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 -- 172.21.15.69:0/2834891799 >> 172.21.15.69:0/2834891799 conn(0x7fba000fc3f0 msgr2=0x7fba000fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 -- 172.21.15.69:0/2834891799 shutdown_connections 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 -- 172.21.15.69:0/2834891799 wait complete. 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 Processor -- start 2024-08-02T18:00:19.196 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.183+0000 7fba07bcb640 1 -- start start 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba07bcb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba00100710 0x7fba00198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba07bcb640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 0x7fba00198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba07bcb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00106720 0x7fba0019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba07bcb640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba00110b60 con 0x7fba001010b0 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba07bcb640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba001109e0 con 0x7fba00100710 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba07bcb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba00110ce0 con 0x7fba00106720 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 0x7fba00198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 0x7fba00198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37554/0 (socket says 172.21.15.69:37554) 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 -- 172.21.15.69:0/829516769 learned_addr learned my addr 172.21.15.69:0/829516769 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05967640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba00100710 0x7fba00198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba06168640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00106720 0x7fba0019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 -- 172.21.15.69:0/829516769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00106720 msgr2=0x7fba0019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00106720 0x7fba0019ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.197 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 -- 172.21.15.69:0/829516769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba00100710 msgr2=0x7fba00198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba00100710 0x7fba00198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 -- 172.21.15.69:0/829516769 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0019d140 con 0x7fba001010b0 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05967640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba00100710 0x7fba00198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba05166640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 0x7fba00198690 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c970 tx=0x7fb9fc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fb9f2ffd640 1 -- 172.21.15.69:0/829516769 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc007bb0 con 0x7fba001010b0 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fb9f2ffd640 1 -- 172.21.15.69:0/829516769 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9fc007d50 con 0x7fba001010b0 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fba07bcb640 1 -- 172.21.15.69:0/829516769 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba0019d430 con 0x7fba001010b0 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.184+0000 7fb9f2ffd640 1 -- 172.21.15.69:0/829516769 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00f450 con 0x7fba001010b0 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.185+0000 7fba07bcb640 1 -- 172.21.15.69:0/829516769 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001a4cc0 con 0x7fba001010b0 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fb9f2ffd640 1 -- 172.21.15.69:0/829516769 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb9fc005020 con 0x7fba001010b0 2024-08-02T18:00:19.198 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba0a9f3740 1 Processor -- start 2024-08-02T18:00:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba0a9f3740 1 -- start start 2024-08-02T18:00:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437ccea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cdd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cddd80 0x55a437ce0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a437c329b0 con 0x55a437cce670 2024-08-02T18:00:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a437bda800 con 0x55a437ccef90 2024-08-02T18:00:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a437c2e970 con 0x55a437cddd80 2024-08-02T18:00:19.202 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.187+0000 7fba06168640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437ccea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437ccea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37566/0 (socket says 172.21.15.69:37566) 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 -- 172.21.15.69:0/1309076699 learned_addr learned my addr 172.21.15.69:0/1309076699 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba05166640 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cddd80 0x55a437ce0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fb9f2ffd640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb9ec0a2cf0 0x7fb9ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba05967640 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cdd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 -- 172.21.15.69:0/1309076699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cddd80 msgr2=0x55a437ce0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cddd80 0x55a437ce0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 -- 172.21.15.69:0/1309076699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 msgr2=0x55a437cdd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cdd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 -- 172.21.15.69:0/1309076699 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a437946210 con 0x55a437cce670 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba05967640 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cdd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fb9f2ffd640 1 -- 172.21.15.69:0/829516769 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7fb9fc0cf760 con 0x7fba001010b0 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb9ec0a2cf0 0x7fb9ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fba06168640 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437ccea50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009a60 tx=0x7fb9f8031430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439a354cf3db1b33 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fb9ebfff640 1 -- 172.21.15.69:0/1309076699 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f804d070 con 0x55a437cce670 2024-08-02T18:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fb9ebfff640 1 -- 172.21.15.69:0/1309076699 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f8043070 con 0x55a437cce670 2024-08-02T18:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.188+0000 7fb9ebfff640 1 -- 172.21.15.69:0/1309076699 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8030cd0 con 0x55a437cce670 2024-08-02T18:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 -- 172.21.15.69:0/1309076699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 msgr2=0x55a437ccea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba06168640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb9ec0a2cf0 0x7fb9ec0a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8046000 tx=0x7fb9f804b000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437ccea50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009a60 tx=0x7fb9f8031430 comp rx=0 tx=0).stop 2024-08-02T18:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 -- 172.21.15.69:0/1309076699 shutdown_connections 2024-08-02T18:00:19.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cddd80 0x55a437ce0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cdd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/1309076699 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437ccea50 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 -- 172.21.15.69:0/1309076699 >> 172.21.15.69:0/1309076699 conn(0x55a437ccad20 msgr2=0x55a437ccb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 -- 172.21.15.69:0/1309076699 shutdown_connections 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 -- 172.21.15.69:0/1309076699 wait complete. 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.189+0000 7fba0a9f3740 1 -- start start 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba0a9f3740 1 Processor -- start 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437cdad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437ccef90 0x55a437cdb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437cddd80 0x55a437cdb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a437ce0950 con 0x55a437cce670 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a437bee500 con 0x55a437cddd80 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a437ce0ad0 con 0x55a437ccef90 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba0a9f3740 10 client.0 _ll_register_callbacks cb 0x55a437945a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437cdad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437cdad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37582/0 (socket says 172.21.15.69:37582) 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 -- 172.21.15.69:0/842401143 learned_addr learned my addr 172.21.15.69:0/842401143 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba06168640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437cddd80 0x55a437cdb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05166640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437ccef90 0x55a437cdb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 -- 172.21.15.69:0/842401143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437ccef90 msgr2=0x55a437cdb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437ccef90 0x55a437cdb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 -- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437cddd80 msgr2=0x55a437cdb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437cddd80 0x55a437cdb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 -- 172.21.15.69:0/842401143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a437a12620 con 0x55a437cce670 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05166640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437ccef90 0x55a437cdb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba06168640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437cddd80 0x55a437cdb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fba05967640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437cdad40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400bf80 tx=0x7fb9f4011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1420847df5331cb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fb9e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fb9e97fa640 1 -- 172.21.15.69:0/842401143 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4012710 con 0x55a437cce670 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.190+0000 7fb9e97fa640 1 -- 172.21.15.69:0/842401143 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7fb9f40128b0 con 0x55a437cce670 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 1 -- 172.21.15.69:0/842401143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f8043070 con 0x55a437cce670 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fb9e97fa640 1 -- 172.21.15.69:0/842401143 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9f4012af0 con 0x55a437cce670 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 10 client.18897 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 1 -- 172.21.15.69:0/842401143 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a437cec6f0 con 0x55a437cce670 2024-08-02T18:00:19.207 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fb9e97fa640 1 -- 172.21.15.69:0/842401143 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f8043070 con 0x55a437cce670 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fb9e97fa640 1 -- 172.21.15.69:0/842401143 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb9f401a3d0 con 0x55a437cce670 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 10 client.18897 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 10 client.18897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 20 client.18897 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a437cedc00 0x55a437cefff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 4 client.18897 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.191+0000 7fba0a9f3740 1 -- 172.21.15.69:0/842401143 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a437c400e0 con 0x55a437cedc00 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.192+0000 7fba05166640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a437cedc00 0x55a437cefff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.192+0000 7fba05166640 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a437cedc00 0x55a437cefff0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7546f6d2b6d31289 server_cookie=1b13f77a8b699656 in_seq=0 out_seq=0 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.192+0000 7fb9e97fa640 10 client.18897 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:19.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fb9e97fa640 1 -- 172.21.15.69:0/842401143 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a437c400e0 con 0x55a437cedc00 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fb9e97fa640 10 client.18897 handle_command_reply: tid=0 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 client.18897 shutdown 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 -- 172.21.15.69:0/842401143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 msgr2=0x55a437cdad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437cdad40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fb9f400bf80 tx=0x7fb9f4011870 comp rx=0 tx=0).stop 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 -- 172.21.15.69:0/842401143 shutdown_connections 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a437cedc00 0x55a437cefff0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437cddd80 0x55a437cdb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fb9e97fa640 0 client.18897 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437ccef90 0x55a437cdb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/842401143 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cce670 0x55a437cdad40 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 -- 172.21.15.69:0/842401143 >> 172.21.15.69:0/842401143 conn(0x55a437ccad20 msgr2=0x55a437cd09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 -- 172.21.15.69:0/842401143 shutdown_connections 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 1 -- 172.21.15.69:0/842401143 wait complete. 2024-08-02T18:00:19.209 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.193+0000 7fba0a9f3740 20 client.18897 trim_cache size 0 max 16384 2024-08-02T18:00:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.222+0000 7fba0a9f3740 1 Processor -- start 2024-08-02T18:00:19.229 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.222+0000 7fba0a9f3740 1 -- start start 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.222+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437cdcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.222+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cf5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.222+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437cf5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.222+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba001af6a0 con 0x55a437cddd80 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.222+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba001acf70 con 0x55a437ccef90 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.222+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba001aade0 con 0x55a437cce670 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437cf5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437cf5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37598/0 (socket says 172.21.15.69:37598) 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 -- 172.21.15.69:0/3307870700 learned_addr learned my addr 172.21.15.69:0/3307870700 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.230 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba05166640 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cf5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba05967640 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437cdcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 -- 172.21.15.69:0/3307870700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 msgr2=0x55a437cdcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437cdcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 -- 172.21.15.69:0/3307870700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 msgr2=0x55a437cf5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cf5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 -- 172.21.15.69:0/3307870700 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a437cd9270 con 0x55a437cddd80 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fba06168640 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437cf5e40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80097b0 tx=0x7fb9f803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94864623a4b6d4a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fb9eb7fe640 1 -- 172.21.15.69:0/3307870700 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f804d030 con 0x55a437cddd80 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fb9eb7fe640 1 -- 172.21.15.69:0/3307870700 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f8043070 con 0x55a437cddd80 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.223+0000 7fb9eb7fe640 1 -- 172.21.15.69:0/3307870700 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80307a0 con 0x55a437cddd80 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 -- 172.21.15.69:0/3307870700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 msgr2=0x55a437cf5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437cf5e40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80097b0 tx=0x7fb9f803c000 comp rx=0 tx=0).stop 2024-08-02T18:00:19.231 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 -- 172.21.15.69:0/3307870700 shutdown_connections 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437cf5e40 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437cf5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/3307870700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437cdcba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 -- 172.21.15.69:0/3307870700 >> 172.21.15.69:0/3307870700 conn(0x55a437ccad20 msgr2=0x55a437cdc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 -- 172.21.15.69:0/3307870700 shutdown_connections 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 -- 172.21.15.69:0/3307870700 wait complete. 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.224+0000 7fba0a9f3740 1 -- start start 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba0a9f3740 1 Processor -- start 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437ce1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437ce2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.232 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba0a9f3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437d578d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fba001ab430 con 0x55a437cddd80 2024-08-02T18:00:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fba001ac8a0 con 0x55a437ccef90 2024-08-02T18:00:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba0a9f3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba001abb00 con 0x55a437cce670 2024-08-02T18:00:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba0a9f3740 10 client.0 _ll_register_callbacks cb 0x7fb9f400b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba06168640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437d578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.233 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba06168640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437d578d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37612/0 (socket says 172.21.15.69:37612) 2024-08-02T18:00:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba06168640 1 -- 172.21.15.69:0/771064669 learned_addr learned my addr 172.21.15.69:0/771064669 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba05166640 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437ce2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.234 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.225+0000 7fba05967640 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437ce1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.236 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id 2c6a3888 --release --await' 2024-08-02T18:00:19.236 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id 2c6a3888 --release --await 2024-08-02T18:00:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba06168640 1 -- 172.21.15.69:0/771064669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 msgr2=0x55a437ce1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba06168640 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437ce1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba06168640 1 -- 172.21.15.69:0/771064669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 msgr2=0x55a437ce2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba06168640 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437ce2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba06168640 1 -- 172.21.15.69:0/771064669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a437d57e10 con 0x55a437cddd80 2024-08-02T18:00:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba05967640 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437ce1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:19.240 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba06168640 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437d578d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009780 tx=0x7fb9f802af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1357f78f9bc280 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fb9e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fb9e97fa640 1 -- 172.21.15.69:0/771064669 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8062020 con 0x55a437cddd80 2024-08-02T18:00:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fb9e97fa640 1 -- 172.21.15.69:0/771064669 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7fb9f8057070 con 0x55a437cddd80 2024-08-02T18:00:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba0a9f3740 1 -- 172.21.15.69:0/771064669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f8043070 con 0x55a437cddd80 2024-08-02T18:00:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fb9e97fa640 1 -- 172.21.15.69:0/771064669 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb9f804d070 con 0x55a437cddd80 2024-08-02T18:00:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba0a9f3740 10 client.18903 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fba0a9f3740 1 -- 172.21.15.69:0/771064669 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a437d58fe0 con 0x55a437cddd80 2024-08-02T18:00:19.241 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fb9e97fa640 1 -- 172.21.15.69:0/771064669 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f8043070 con 0x55a437cddd80 2024-08-02T18:00:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.226+0000 7fb9e97fa640 1 -- 172.21.15.69:0/771064669 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb9f8030070 con 0x55a437cddd80 2024-08-02T18:00:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fba0a9f3740 10 client.18903 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fba0a9f3740 10 client.18903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fba0a9f3740 20 client.18903 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a437cedc00 0x55a437d5b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fba0a9f3740 4 client.18903 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:00:19.242 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fba0a9f3740 1 -- 172.21.15.69:0/771064669 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a437c3a430 con 0x55a437cedc00 2024-08-02T18:00:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fba05967640 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a437cedc00 0x55a437d5b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fba05967640 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a437cedc00 0x55a437d5b260 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5133d98e81522895 server_cookie=7b963ff81c2c6e59 in_seq=0 out_seq=0 2024-08-02T18:00:19.243 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.227+0000 7fb9e97fa640 10 client.18903 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fb9e97fa640 1 -- 172.21.15.69:0/771064669 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a437c3a430 con 0x55a437cedc00 2024-08-02T18:00:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fb9e97fa640 10 client.18903 handle_command_reply: tid=0 2024-08-02T18:00:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 client.18903 shutdown 2024-08-02T18:00:19.246 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 -- 172.21.15.69:0/771064669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 msgr2=0x55a437d578d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437d578d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009780 tx=0x7fb9f802af20 comp rx=0 tx=0).stop 2024-08-02T18:00:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 -- 172.21.15.69:0/771064669 shutdown_connections 2024-08-02T18:00:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a437cddd80 0x55a437d578d0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55a437cedc00 0x55a437d5b260 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a437ccef90 0x55a437ce2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fb9e97fa640 0 client.18903 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:19.247 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 --2- 172.21.15.69:0/771064669 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a437cce670 0x55a437ce1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 -- 172.21.15.69:0/771064669 >> 172.21.15.69:0/771064669 conn(0x55a437ccad20 msgr2=0x55a437cc9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 -- 172.21.15.69:0/771064669 shutdown_connections 2024-08-02T18:00:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.243+0000 7fba0a9f3740 1 -- 172.21.15.69:0/771064669 wait complete. 2024-08-02T18:00:19.248 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba0a9f3740 20 client.18903 trim_cache size 0 max 16384 2024-08-02T18:00:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:00:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:00:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:00:19.249 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:00:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 -- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb9ec0a2cf0 msgr2=0x7fb9ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb9ec0a2cf0 0x7fb9ec0a51b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8046000 tx=0x7fb9f804b000 comp rx=0 tx=0).stop 2024-08-02T18:00:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 -- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 msgr2=0x7fba00198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.250 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 0x7fba00198690 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c970 tx=0x7fb9fc00ce30 comp rx=0 tx=0).stop 2024-08-02T18:00:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 -- 172.21.15.69:0/829516769 shutdown_connections 2024-08-02T18:00:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fba001010b0 0x7fba00198690 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb9ec0a2cf0 0x7fb9ec0a51b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.252 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba00106720 0x7fba0019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 --2- 172.21.15.69:0/829516769 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fba00100710 0x7fba00198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.253 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 -- 172.21.15.69:0/829516769 >> 172.21.15.69:0/829516769 conn(0x7fba000fc3f0 msgr2=0x7fba0006a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 -- 172.21.15.69:0/829516769 shutdown_connections 2024-08-02T18:00:19.254 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:19.244+0000 7fba07bcb640 1 -- 172.21.15.69:0/829516769 wait complete. 2024-08-02T18:00:19.420 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:19 smithi069 ceph-mon[32746]: pgmap v434: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 32 KiB/s wr, 117 op/s 2024-08-02T18:00:19.420 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:19 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/115709872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:19.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 -- 172.21.15.69:0/1795192176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 msgr2=0x7f7ac41026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/1795192176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 0x7f7ac41026e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009900 tx=0x7f7aac02a480 comp rx=0 tx=0).stop 2024-08-02T18:00:19.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 -- 172.21.15.69:0/1795192176 shutdown_connections 2024-08-02T18:00:19.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/1795192176 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 0x7f7ac41026e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/1795192176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ac4102c20 0x7f7ac410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/1795192176 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ac4100b50 0x7f7ac4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 -- 172.21.15.69:0/1795192176 >> 172.21.15.69:0/1795192176 conn(0x7f7ac40fc500 msgr2=0x7f7ac40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.433 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 -- 172.21.15.69:0/1795192176 shutdown_connections 2024-08-02T18:00:19.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.418+0000 7f7ac8c89640 1 -- 172.21.15.69:0/1795192176 wait complete. 2024-08-02T18:00:19.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.419+0000 7f7ac8c89640 1 Processor -- start 2024-08-02T18:00:19.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.419+0000 7f7ac8c89640 1 -- start start 2024-08-02T18:00:19.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.419+0000 7f7ac8c89640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ac4100b50 0x7f7ac41991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.419+0000 7f7ac8c89640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 0x7f7ac4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.419+0000 7f7ac8c89640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ac4102c20 0x7f7ac41a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.419+0000 7f7ac8c89640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac4110d50 con 0x7f7ac41022a0 2024-08-02T18:00:19.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.419+0000 7f7ac8c89640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac4110bd0 con 0x7f7ac4100b50 2024-08-02T18:00:19.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.419+0000 7f7ac8c89640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac4110ed0 con 0x7f7ac4102c20 2024-08-02T18:00:19.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 0x7f7ac4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 0x7f7ac4199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37636/0 (socket says 172.21.15.69:37636) 2024-08-02T18:00:19.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 -- 172.21.15.69:0/556725567 learned_addr learned my addr 172.21.15.69:0/556725567 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7ac259c640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ac4100b50 0x7f7ac41991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7ac2d9d640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ac4102c20 0x7f7ac41a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 -- 172.21.15.69:0/556725567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ac4102c20 msgr2=0x7f7ac41a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.436 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ac4102c20 0x7f7ac41a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 -- 172.21.15.69:0/556725567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ac4100b50 msgr2=0x7f7ac41991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ac4100b50 0x7f7ac41991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 -- 172.21.15.69:0/556725567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac41a47a0 con 0x7f7ac41022a0 2024-08-02T18:00:19.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7ac259c640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ac4100b50 0x7f7ac41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:19.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abbfff640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 0x7f7ac4199720 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7aac006010 tx=0x7f7aac003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.437 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abb7fe640 1 -- 172.21.15.69:0/556725567 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac03f070 con 0x7f7ac41022a0 2024-08-02T18:00:19.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abb7fe640 1 -- 172.21.15.69:0/556725567 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aac031e30 con 0x7f7ac41022a0 2024-08-02T18:00:19.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7ac8c89640 1 -- 172.21.15.69:0/556725567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac41a4970 con 0x7f7ac41022a0 2024-08-02T18:00:19.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7abb7fe640 1 -- 172.21.15.69:0/556725567 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac035030 con 0x7f7ac41022a0 2024-08-02T18:00:19.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.420+0000 7f7ac8c89640 1 -- 172.21.15.69:0/556725567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac41a4dc0 con 0x7f7ac41022a0 2024-08-02T18:00:19.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.423+0000 7f7abb7fe640 1 -- 172.21.15.69:0/556725567 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7aac042070 con 0x7f7ac41022a0 2024-08-02T18:00:19.438 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.423+0000 7f7acb087740 1 Processor -- start 2024-08-02T18:00:19.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.423+0000 7f7abb7fe640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7ab00a2cf0 0x7f7ab00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7ac2d9d640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7ab00a2cf0 0x7f7ab00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7acb087740 1 -- start start 2024-08-02T18:00:19.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7abb7fe640 1 -- 172.21.15.69:0/556725567 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f7aac0f02c0 con 0x7f7ac41022a0 2024-08-02T18:00:19.439 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7acb087740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x5556856613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7acb087740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685661910 0x55568566bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7acb087740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55568566c1d0 0x55568566e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7acb087740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5556855c6a40 con 0x55568566c1d0 2024-08-02T18:00:19.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7acb087740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55568555e9d0 con 0x555685661910 2024-08-02T18:00:19.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7acb087740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5556855c2a00 con 0x555685660ff0 2024-08-02T18:00:19.440 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7ac259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x5556856613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685661910 0x55568566bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7ac259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x5556856613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45196/0 (socket says 172.21.15.69:45196) 2024-08-02T18:00:19.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685661910 0x55568566bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40220/0 (socket says 172.21.15.69:40220) 2024-08-02T18:00:19.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7ac259c640 1 -- 172.21.15.69:0/1870558388 learned_addr learned my addr 172.21.15.69:0/1870558388 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.441 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7ac2d9d640 1 --2- 172.21.15.69:0/1870558388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55568566c1d0 0x55568566e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7ac2d9d640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7ab00a2cf0 0x7f7ab00a51b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009ea0 tx=0x7f7abc009410 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7abbfff640 1 -- 172.21.15.69:0/1870558388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 msgr2=0x5556856613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7abbfff640 1 --2- 172.21.15.69:0/1870558388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x5556856613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7abbfff640 1 -- 172.21.15.69:0/1870558388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55568566c1d0 msgr2=0x55568566e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.442 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7abbfff640 1 --2- 172.21.15.69:0/1870558388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55568566c1d0 0x55568566e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.424+0000 7f7abbfff640 1 -- 172.21.15.69:0/1870558388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555685565650 con 0x555685661910 2024-08-02T18:00:19.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.425+0000 7f7abbfff640 1 --2- 172.21.15.69:0/1870558388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685661910 0x55568566bc90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0ed5a0 tx=0x7f7aac0314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4d473ab9ca26bd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.425+0000 7f7ab8ff9640 1 -- 172.21.15.69:0/1870558388 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac049050 con 0x555685661910 2024-08-02T18:00:19.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.425+0000 7f7ab8ff9640 1 -- 172.21.15.69:0/1870558388 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aac0bc370 con 0x555685661910 2024-08-02T18:00:19.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.425+0000 7f7ab8ff9640 1 -- 172.21.15.69:0/1870558388 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac03b5f0 con 0x555685661910 2024-08-02T18:00:19.443 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 -- 172.21.15.69:0/1870558388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685661910 msgr2=0x55568566bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 --2- 172.21.15.69:0/1870558388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685661910 0x55568566bc90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0ed5a0 tx=0x7f7aac0314a0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 -- 172.21.15.69:0/1870558388 shutdown_connections 2024-08-02T18:00:19.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 --2- 172.21.15.69:0/1870558388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55568566c1d0 0x55568566e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 --2- 172.21.15.69:0/1870558388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685661910 0x55568566bc90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.444 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 --2- 172.21.15.69:0/1870558388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x5556856613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 -- 172.21.15.69:0/1870558388 >> 172.21.15.69:0/1870558388 conn(0x55568565d580 msgr2=0x55568565d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 -- 172.21.15.69:0/1870558388 shutdown_connections 2024-08-02T18:00:19.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 -- 172.21.15.69:0/1870558388 wait complete. 2024-08-02T18:00:19.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 -- start start 2024-08-02T18:00:19.445 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.426+0000 7f7acb087740 1 Processor -- start 2024-08-02T18:00:19.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7acb087740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685660ff0 0x555685669050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7acb087740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685669590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7acb087740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55568566c1d0 0x555685669ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7acb087740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55568566eb60 con 0x555685661910 2024-08-02T18:00:19.446 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7acb087740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555685581b20 con 0x555685660ff0 2024-08-02T18:00:19.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7acb087740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55568566ece0 con 0x55568566c1d0 2024-08-02T18:00:19.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7acb087740 10 client.0 _ll_register_callbacks cb 0x55568557fbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:19.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685660ff0 0x555685669050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685660ff0 0x555685669050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40230/0 (socket says 172.21.15.69:40230) 2024-08-02T18:00:19.447 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac259c640 1 -- 172.21.15.69:0/2539498218 learned_addr learned my addr 172.21.15.69:0/2539498218 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac2d9d640 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55568566c1d0 0x555685669ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac259c640 1 -- 172.21.15.69:0/2539498218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55568566c1d0 msgr2=0x555685669ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac259c640 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55568566c1d0 0x555685669ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.448 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac259c640 1 -- 172.21.15.69:0/2539498218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 msgr2=0x555685669590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7abbfff640 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685669590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac259c640 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685669590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7ac259c640 1 -- 172.21.15.69:0/2539498218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556853383c0 con 0x555685660ff0 2024-08-02T18:00:19.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.427+0000 7f7abbfff640 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685669590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:19.449 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7ac259c640 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685660ff0 0x555685669050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40104d0 tx=0x7f7ab4010990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2423dbcf4bd91f34 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:19.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/2539498218 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4018070 con 0x555685660ff0 2024-08-02T18:00:19.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7acb087740 1 -- 172.21.15.69:0/2539498218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aac0bc370 con 0x555685660ff0 2024-08-02T18:00:19.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/2539498218 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f7ab4014640 con 0x555685660ff0 2024-08-02T18:00:19.450 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/2539498218 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ab4024050 con 0x555685660ff0 2024-08-02T18:00:19.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7acb087740 10 client.28268 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:19.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7acb087740 1 -- 172.21.15.69:0/2539498218 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555685682ec0 con 0x555685660ff0 2024-08-02T18:00:19.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/2539498218 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aac0bc370 con 0x555685660ff0 2024-08-02T18:00:19.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.428+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/2539498218 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7ab4012a30 con 0x555685660ff0 2024-08-02T18:00:19.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7acb087740 10 client.28268 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:19.451 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7acb087740 10 client.28268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:19.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7acb087740 20 client.28268 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:19.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7acb087740 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555685684320 0x5556856867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7acb087740 4 client.28268 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:19.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7acb087740 1 -- 172.21.15.69:0/2539498218 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556855d3fb0 con 0x555685684320 2024-08-02T18:00:19.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7abbfff640 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555685684320 0x5556856867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.452 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7aa9ffb640 10 client.28268 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:19.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.429+0000 7f7abbfff640 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555685684320 0x5556856867b0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa486d88690055d2 server_cookie=af27e76d8cf8af02 in_seq=0 out_seq=0 2024-08-02T18:00:19.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/2539498218 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556855d3fb0 con 0x555685684320 2024-08-02T18:00:19.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7aa9ffb640 10 client.28268 handle_command_reply: tid=0 2024-08-02T18:00:19.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 client.28268 shutdown 2024-08-02T18:00:19.453 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 -- 172.21.15.69:0/2539498218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685660ff0 msgr2=0x555685669050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.454 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685660ff0 0x555685669050 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40104d0 tx=0x7f7ab4010990 comp rx=0 tx=0).stop 2024-08-02T18:00:19.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 -- 172.21.15.69:0/2539498218 shutdown_connections 2024-08-02T18:00:19.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685669590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555685684320 0x5556856867b0 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.464 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7aa9ffb640 0 client.28268 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55568566c1d0 0x555685669ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 --2- 172.21.15.69:0/2539498218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555685660ff0 0x555685669050 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 -- 172.21.15.69:0/2539498218 >> 172.21.15.69:0/2539498218 conn(0x55568565d580 msgr2=0x555685663260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 -- 172.21.15.69:0/2539498218 shutdown_connections 2024-08-02T18:00:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 1 -- 172.21.15.69:0/2539498218 wait complete. 2024-08-02T18:00:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.430+0000 7f7acb087740 20 client.28268 trim_cache size 0 max 16384 2024-08-02T18:00:19.465 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.445+0000 7f7acb087740 1 Processor -- start 2024-08-02T18:00:19.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.445+0000 7f7acb087740 1 -- start start 2024-08-02T18:00:19.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.445+0000 7f7acb087740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x55568566aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.445+0000 7f7acb087740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x55568568c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.445+0000 7f7acb087740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568568c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.445+0000 7f7acb087740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41af710 con 0x555685661910 2024-08-02T18:00:19.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.445+0000 7f7acb087740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41acfe0 con 0x55568566c1d0 2024-08-02T18:00:19.466 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.445+0000 7f7acb087740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41aae50 con 0x555685660ff0 2024-08-02T18:00:19.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x55568568c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x55568568c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37652/0 (socket says 172.21.15.69:37652) 2024-08-02T18:00:19.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 -- 172.21.15.69:0/4206462543 learned_addr learned my addr 172.21.15.69:0/4206462543 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7ac259c640 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x55568566aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7ac2d9d640 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568568c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 -- 172.21.15.69:0/4206462543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 msgr2=0x55568566aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.467 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x55568566aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 -- 172.21.15.69:0/4206462543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 msgr2=0x55568568c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568568c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 -- 172.21.15.69:0/4206462543 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55568566bb00 con 0x555685661910 2024-08-02T18:00:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7abbfff640 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x55568568c080 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0ed600 tx=0x7f7aac03b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4601c214a2d3e5d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7aabfff640 1 -- 172.21.15.69:0/4206462543 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac049070 con 0x555685661910 2024-08-02T18:00:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7aabfff640 1 -- 172.21.15.69:0/4206462543 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aac0c4fe0 con 0x555685661910 2024-08-02T18:00:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.446+0000 7f7aabfff640 1 -- 172.21.15.69:0/4206462543 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0b8b70 con 0x555685661910 2024-08-02T18:00:19.468 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 -- 172.21.15.69:0/4206462543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 msgr2=0x55568568c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x55568568c080 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0ed600 tx=0x7f7aac03b990 comp rx=0 tx=0).stop 2024-08-02T18:00:19.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 -- 172.21.15.69:0/4206462543 shutdown_connections 2024-08-02T18:00:19.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x55568568c080 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568568c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 --2- 172.21.15.69:0/4206462543 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x55568566aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 -- 172.21.15.69:0/4206462543 >> 172.21.15.69:0/4206462543 conn(0x55568565d580 msgr2=0x55568566a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 -- 172.21.15.69:0/4206462543 shutdown_connections 2024-08-02T18:00:19.469 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 -- 172.21.15.69:0/4206462543 wait complete. 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.447+0000 7f7acb087740 1 -- start start 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7acb087740 1 Processor -- start 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7acb087740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x555685665660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7acb087740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685665ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7acb087740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568567b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7acb087740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41ab4a0 con 0x555685661910 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7acb087740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41ac910 con 0x55568566c1d0 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7acb087740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac41abb70 con 0x555685660ff0 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7acb087740 10 client.0 _ll_register_callbacks cb 0x7f7ab4007120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685665ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685665ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37660/0 (socket says 172.21.15.69:37660) 2024-08-02T18:00:19.470 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7ac259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x555685665660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7abbfff640 1 -- 172.21.15.69:0/3947156220 learned_addr learned my addr 172.21.15.69:0/3947156220 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7ac2d9d640 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568567b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7abbfff640 1 -- 172.21.15.69:0/3947156220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 msgr2=0x555685665660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7abbfff640 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x555685665660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7abbfff640 1 -- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 msgr2=0x55568567b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7abbfff640 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568567b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7abbfff640 1 -- 172.21.15.69:0/3947156220 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555685666460 con 0x555685661910 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7ac2d9d640 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568567b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.448+0000 7f7ac259c640 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x555685665660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:19.471 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7abbfff640 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685665ba0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0b8050 tx=0x7f7aac034000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2384fd1b521faa5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/3947156220 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac049050 con 0x555685661910 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/3947156220 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9656+0+0 (secure 0 0 0) 0x7f7aac0ed820 con 0x555685661910 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7acb087740 1 -- 172.21.15.69:0/3947156220 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aac0c4fe0 con 0x555685661910 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/3947156220 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7aac0bc370 con 0x555685661910 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7acb087740 10 client.18918 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7acb087740 1 -- 172.21.15.69:0/3947156220 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55568567cce0 con 0x555685661910 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/3947156220 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aac0c4fe0 con 0x555685661910 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/3947156220 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7aac0c4b50 con 0x555685661910 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7acb087740 10 client.18918 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7acb087740 10 client.18918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:19.472 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7acb087740 20 client.18918 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7acb087740 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555685684320 0x5556856f1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.449+0000 7f7acb087740 4 client.18918 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:19.473 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.450+0000 7f7acb087740 1 -- 172.21.15.69:0/3947156220 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556855ce4c0 con 0x555685684320 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.450+0000 7f7ac259c640 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555685684320 0x5556856f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.450+0000 7f7ac259c640 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555685684320 0x5556856f1060 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418a1ee29fbc1ecc server_cookie=4ecf365c50b863ad in_seq=0 out_seq=0 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.450+0000 7f7aa9ffb640 10 client.18918 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.450+0000 7f7aa9ffb640 1 -- 172.21.15.69:0/3947156220 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556855ce4c0 con 0x555685684320 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.450+0000 7f7aa9ffb640 10 client.18918 handle_command_reply: tid=0 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.450+0000 7f7acb087740 1 client.18918 shutdown 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 -- 172.21.15.69:0/3947156220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 msgr2=0x555685665ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685665ba0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f7aac0b8050 tx=0x7f7aac034000 comp rx=0 tx=0).stop 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 -- 172.21.15.69:0/3947156220 shutdown_connections 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555685661910 0x555685665ba0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555685684320 0x5556856f1060 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.474 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55568566c1d0 0x55568567b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 --2- 172.21.15.69:0/3947156220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555685660ff0 0x555685665660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7aa9ffb640 0 client.18918 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 -- 172.21.15.69:0/3947156220 >> 172.21.15.69:0/3947156220 conn(0x55568565d580 msgr2=0x5556856697a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 -- 172.21.15.69:0/3947156220 shutdown_connections 2024-08-02T18:00:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 1 -- 172.21.15.69:0/3947156220 wait complete. 2024-08-02T18:00:19.475 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7acb087740 20 client.18918 trim_cache size 0 max 16384 2024-08-02T18:00:19.475 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:19.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7ac8c89640 1 -- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7ab00a2cf0 msgr2=0x7f7ab00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7ab00a2cf0 0x7f7ab00a51b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009ea0 tx=0x7f7abc009410 comp rx=0 tx=0).stop 2024-08-02T18:00:19.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.451+0000 7f7ac8c89640 1 -- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 msgr2=0x7f7ac4199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:19.477 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 0x7f7ac4199720 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7aac006010 tx=0x7f7aac003c80 comp rx=0 tx=0).stop 2024-08-02T18:00:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 -- 172.21.15.69:0/556725567 shutdown_connections 2024-08-02T18:00:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7ac41022a0 0x7f7ac4199720 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7ab00a2cf0 0x7f7ab00a51b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ac4102c20 0x7f7ac41a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 --2- 172.21.15.69:0/556725567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7ac4100b50 0x7f7ac41991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 -- 172.21.15.69:0/556725567 >> 172.21.15.69:0/556725567 conn(0x7f7ac40fc500 msgr2=0x7f7ac40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:19.478 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 -- 172.21.15.69:0/556725567 shutdown_connections 2024-08-02T18:00:19.479 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:19.452+0000 7f7ac8c89640 1 -- 172.21.15.69:0/556725567 wait complete. 2024-08-02T18:00:19.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:19 smithi083 ceph-mon[37031]: pgmap v434: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 32 KiB/s wr, 117 op/s 2024-08-02T18:00:19.524 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:19 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/115709872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:19.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:19 smithi077 ceph-mon[37108]: pgmap v434: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 32 KiB/s wr, 117 op/s 2024-08-02T18:00:19.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:19 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/115709872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:19.957 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 790.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-08-02T18:00:20.256 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:20.670 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:20 smithi069 ceph-mon[32746]: pgmap v435: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-08-02T18:00:20.670 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:20 smithi069 ceph-mon[32746]: osdmap e111: 12 total, 12 up, 12 in 2024-08-02T18:00:20.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 -- 172.21.15.69:0/2226182726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 msgr2=0x7f4d801078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2226182726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 0x7f4d801078d0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009930 tx=0x7f4d7402a420 comp rx=0 tx=0).stop 2024-08-02T18:00:20.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 -- 172.21.15.69:0/2226182726 shutdown_connections 2024-08-02T18:00:20.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2226182726 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 0x7f4d801078d0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2226182726 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d80108010 0x7f4d8010e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2226182726 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d80102ae0 0x7f4d80102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 -- 172.21.15.69:0/2226182726 >> 172.21.15.69:0/2226182726 conn(0x7f4d800fc500 msgr2=0x7f4d800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:20.778 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 -- 172.21.15.69:0/2226182726 shutdown_connections 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.759+0000 7f4d863c2640 1 -- 172.21.15.69:0/2226182726 wait complete. 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d863c2640 1 Processor -- start 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d863c2640 1 -- start start 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d863c2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d80102ae0 0x7f4d80100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d863c2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 0x7f4d80100c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d863c2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d80108010 0x7f4d801011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d863c2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80111de0 con 0x7f4d80103490 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d863c2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80111c60 con 0x7f4d80102ae0 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d863c2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d80111f60 con 0x7f4d80108010 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d7f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 0x7f4d80100c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d7f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 0x7f4d80100c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37684/0 (socket says 172.21.15.69:37684) 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d7f7fe640 1 -- 172.21.15.69:0/2981124253 learned_addr learned my addr 172.21.15.69:0/2981124253 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:20.779 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d80102ae0 0x7f4d80100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.760+0000 7f4d8495f640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d80108010 0x7f4d801011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7f7fe640 1 -- 172.21.15.69:0/2981124253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d80108010 msgr2=0x7f4d801011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d80108010 0x7f4d801011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7f7fe640 1 -- 172.21.15.69:0/2981124253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d80102ae0 msgr2=0x7f4d80100740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d80102ae0 0x7f4d80100740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7f7fe640 1 -- 172.21.15.69:0/2981124253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d801a67e0 con 0x7f4d80103490 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d8495f640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d80108010 0x7f4d801011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 0x7f4d80100c80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4d74005940 tx=0x7f4d74003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7d7fa640 1 -- 172.21.15.69:0/2981124253 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7403f070 con 0x7f4d80103490 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d863c2640 1 -- 172.21.15.69:0/2981124253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d801a6a70 con 0x7f4d80103490 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7d7fa640 1 -- 172.21.15.69:0/2981124253 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d74031c30 con 0x7f4d80103490 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d7d7fa640 1 -- 172.21.15.69:0/2981124253 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d74035030 con 0x7f4d80103490 2024-08-02T18:00:20.780 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.761+0000 7f4d863c2640 1 -- 172.21.15.69:0/2981124253 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d801a6f50 con 0x7f4d80103490 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d7d7fa640 1 -- 172.21.15.69:0/2981124253 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4d74042070 con 0x7f4d80103490 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d887c0740 1 Processor -- start 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d887c0740 1 -- start start 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 0x557162232d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571622332b0 0x55716223d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x55716223ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d887c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557162198c60 con 0x5571622332b0 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d887c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557162130b90 con 0x557162232990 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.764+0000 7f4d887c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557162194c20 con 0x55716223db70 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571622332b0 0x55716223d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.781 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571622332b0 0x55716223d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37698/0 (socket says 172.21.15.69:37698) 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 -- 172.21.15.69:0/149562868 learned_addr learned my addr 172.21.15.69:0/149562868 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x55716223ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d8495f640 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 0x557162232d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 -- 172.21.15.69:0/149562868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 msgr2=0x55716223ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x55716223ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 -- 172.21.15.69:0/149562868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 msgr2=0x557162232d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 0x557162232d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 -- 172.21.15.69:0/149562868 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557161ea8580 con 0x5571622332b0 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7d7fa640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4d680a2cf0 0x7f4d680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d8495f640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4d680a2cf0 0x7f4d680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571622332b0 0x55716223d630 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000b760 tx=0x7f4d7000bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3390a77b6d855464 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:20.782 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.765+0000 7f4d7d7fa640 1 -- 172.21.15.69:0/2981124253 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f4d740f0050 con 0x7f4d80103490 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d667fc640 1 -- 172.21.15.69:0/149562868 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70003dd0 con 0x5571622332b0 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d8495f640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4d680a2cf0 0x7f4d680a51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800af00 tx=0x7f4d7800a6b0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d667fc640 1 -- 172.21.15.69:0/149562868 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d70015070 con 0x5571622332b0 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d667fc640 1 -- 172.21.15.69:0/149562868 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70011440 con 0x5571622332b0 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 -- 172.21.15.69:0/149562868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571622332b0 msgr2=0x55716223d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571622332b0 0x55716223d630 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000b760 tx=0x7f4d7000bc20 comp rx=0 tx=0).stop 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 -- 172.21.15.69:0/149562868 shutdown_connections 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5571622332b0 0x55716223d630 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x55716223ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 --2- 172.21.15.69:0/149562868 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 0x557162232d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 -- 172.21.15.69:0/149562868 >> 172.21.15.69:0/149562868 conn(0x55716222ef20 msgr2=0x55716222f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 -- 172.21.15.69:0/149562868 shutdown_connections 2024-08-02T18:00:20.783 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.766+0000 7f4d887c0740 1 -- 172.21.15.69:0/149562868 wait complete. 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 1 -- start start 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 1 Processor -- start 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 0x55716223aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571622332b0 0x55716223af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55716223db70 0x55716223b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557162240740 con 0x55716223db70 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5571621367c0 con 0x557162232990 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5571622408c0 con 0x5571622332b0 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d887c0740 10 client.0 _ll_register_callbacks cb 0x55716213ff50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d8495f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55716223db70 0x55716223b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.784 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d8495f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55716223db70 0x55716223b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37700/0 (socket says 172.21.15.69:37700) 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.767+0000 7f4d8495f640 1 -- 172.21.15.69:0/3237331610 learned_addr learned my addr 172.21.15.69:0/3237331610 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571622332b0 0x55716223af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 0x55716223aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d8495f640 1 -- 172.21.15.69:0/3237331610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571622332b0 msgr2=0x55716223af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d8495f640 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571622332b0 0x55716223af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d8495f640 1 -- 172.21.15.69:0/3237331610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 msgr2=0x55716223aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d8495f640 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 0x55716223aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d8495f640 1 -- 172.21.15.69:0/3237331610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557161f72960 con 0x55716223db70 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d8495f640 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55716223db70 0x55716223b490 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800ee40 tx=0x7f4d78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d58b2c13f1ecb51 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/3237331610 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7801cdf0 con 0x55716223db70 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/3237331610 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f4d78027430 con 0x55716223db70 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d887c0740 1 -- 172.21.15.69:0/3237331610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d70015070 con 0x55716223db70 2024-08-02T18:00:20.785 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/3237331610 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d78030050 con 0x55716223db70 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d887c0740 10 client.18930 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.768+0000 7f4d887c0740 1 -- 172.21.15.69:0/3237331610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55716224c810 con 0x55716223db70 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/3237331610 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d70015070 con 0x55716223db70 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/3237331610 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d7801d810 con 0x55716223db70 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d887c0740 10 client.18930 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d887c0740 10 client.18930 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d887c0740 20 client.18930 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d887c0740 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55716224dcd0 0x5571622500c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d887c0740 4 client.18930 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d887c0740 1 -- 172.21.15.69:0/3237331610 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571621a61d0 con 0x55716224dcd0 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.769+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55716224dcd0 0x5571622500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d5f7fe640 10 client.18930 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:20.786 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55716224dcd0 0x5571622500c0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd845cfac3eb2ed server_cookie=f46938d07e81f14 in_seq=0 out_seq=0 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/3237331610 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5571621a61d0 con 0x55716224dcd0 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d5f7fe640 10 client.18930 handle_command_reply: tid=0 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 client.18930 shutdown 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 -- 172.21.15.69:0/3237331610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55716223db70 msgr2=0x55716223b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55716223db70 0x55716223b490 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800ee40 tx=0x7f4d78004a40 comp rx=0 tx=0).stop 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 -- 172.21.15.69:0/3237331610 shutdown_connections 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55716223db70 0x55716223b490 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55716224dcd0 0x5571622500c0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5571622332b0 0x55716223af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 --2- 172.21.15.69:0/3237331610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557162232990 0x55716223aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d5f7fe640 0 client.18930 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.770+0000 7f4d887c0740 1 -- 172.21.15.69:0/3237331610 >> 172.21.15.69:0/3237331610 conn(0x55716222ef20 msgr2=0x557162234c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.771+0000 7f4d887c0740 1 -- 172.21.15.69:0/3237331610 shutdown_connections 2024-08-02T18:00:20.787 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.771+0000 7f4d887c0740 1 -- 172.21.15.69:0/3237331610 wait complete. 2024-08-02T18:00:20.788 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.771+0000 7f4d887c0740 20 client.18930 trim_cache size 0 max 16384 2024-08-02T18:00:20.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d887c0740 1 Processor -- start 2024-08-02T18:00:20.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d887c0740 1 -- start start 2024-08-02T18:00:20.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x5571622353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162255990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x557162255ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d887c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801b39b0 con 0x557162232990 2024-08-02T18:00:20.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d887c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801b1280 con 0x5571622332b0 2024-08-02T18:00:20.790 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d887c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801af0d0 con 0x55716223db70 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x5571622353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x5571622353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37704/0 (socket says 172.21.15.69:37704) 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 -- 172.21.15.69:0/1487312223 learned_addr learned my addr 172.21.15.69:0/1487312223 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162255990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d8495f640 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x557162255ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 -- 172.21.15.69:0/1487312223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 msgr2=0x557162255ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x557162255ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 -- 172.21.15.69:0/1487312223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 msgr2=0x557162255990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162255990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 -- 172.21.15.69:0/1487312223 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55716222ed50 con 0x557162232990 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.788+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x5571622353f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000b760 tx=0x7f4d700049b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5f6d20746e9031 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.789+0000 7f4d65ffb640 1 -- 172.21.15.69:0/1487312223 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70003dd0 con 0x557162232990 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.789+0000 7f4d65ffb640 1 -- 172.21.15.69:0/1487312223 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d70015040 con 0x557162232990 2024-08-02T18:00:20.791 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.789+0000 7f4d65ffb640 1 -- 172.21.15.69:0/1487312223 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70026430 con 0x557162232990 2024-08-02T18:00:20.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 -- 172.21.15.69:0/1487312223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 msgr2=0x5571622353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.794 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x5571622353f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000b760 tx=0x7f4d700049b0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 -- 172.21.15.69:0/1487312223 shutdown_connections 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x557162255ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162255990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1487312223 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x5571622353f0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 -- 172.21.15.69:0/1487312223 >> 172.21.15.69:0/1487312223 conn(0x55716222ef20 msgr2=0x55716223ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 -- 172.21.15.69:0/1487312223 shutdown_connections 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 -- 172.21.15.69:0/1487312223 wait complete. 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 -- start start 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.790+0000 7f4d887c0740 1 Processor -- start 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x557162237a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162237f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d887c0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x5571622384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d887c0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801af740 con 0x557162232990 2024-08-02T18:00:20.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d887c0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801b0bb0 con 0x5571622332b0 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d887c0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d801afe10 con 0x55716223db70 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d887c0740 10 client.0 _ll_register_callbacks cb 0x7f4d7800eaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x557162237a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x557162237a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37718/0 (socket says 172.21.15.69:37718) 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7ffff640 1 -- 172.21.15.69:0/1902781327 learned_addr learned my addr 172.21.15.69:0/1902781327 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162237f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d8495f640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x5571622384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7ffff640 1 -- 172.21.15.69:0/1902781327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 msgr2=0x5571622384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x5571622384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7ffff640 1 -- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 msgr2=0x557162237f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162237f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7ffff640 1 -- 172.21.15.69:0/1902781327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55716222fbb0 con 0x557162232990 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d8495f640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x5571622384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.791+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162237f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d7ffff640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x557162237a50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4d700039c0 tx=0x7f4d70004880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46155ac22f9ed090 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/1902781327 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7003c060 con 0x557162232990 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/1902781327 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f4d70003bd0 con 0x557162232990 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 1 -- 172.21.15.69:0/1902781327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d70015040 con 0x557162232990 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/1902781327 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d70003dd0 con 0x557162232990 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 10 client.18936 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 1 -- 172.21.15.69:0/1902781327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5571622b8ab0 con 0x557162232990 2024-08-02T18:00:20.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/1902781327 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d70015040 con 0x557162232990 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/1902781327 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4d700268f0 con 0x557162232990 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 10 client.18936 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 10 client.18936 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 20 client.18936 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5571622b8fd0 0x5571622bb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 4 client.18936 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2c6a3888", "release": true, "await": true}] 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.792+0000 7f4d887c0740 1 -- 172.21.15.69:0/1902781327 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2c6a3888", "release": true, "await": true}) -- 0x5571621a06e0 con 0x5571622b8fd0 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.793+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5571622b8fd0 0x5571622bb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.793+0000 7f4d7f7fe640 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5571622b8fd0 0x5571622bb3c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df30405434e14b5a server_cookie=73a77f202479de20 in_seq=0 out_seq=0 2024-08-02T18:00:20.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.793+0000 7f4d5f7fe640 10 client.18936 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d5f7fe640 1 -- 172.21.15.69:0/1902781327 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5571621a06e0 con 0x5571622b8fd0 2024-08-02T18:00:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d5f7fe640 10 client.18936 handle_command_reply: tid=0 2024-08-02T18:00:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 client.18936 shutdown 2024-08-02T18:00:20.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 -- 172.21.15.69:0/1902781327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 msgr2=0x557162237a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x557162237a50 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f4d700039c0 tx=0x7f4d70004880 comp rx=0 tx=0).stop 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 -- 172.21.15.69:0/1902781327 shutdown_connections 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5571622b8fd0 0x5571622bb3c0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55716223db70 0x5571622384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5571622332b0 0x557162237f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 --2- 172.21.15.69:0/1902781327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557162232990 0x557162237a50 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d5f7fe640 0 client.18936 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 -- 172.21.15.69:0/1902781327 >> 172.21.15.69:0/1902781327 conn(0x55716222ef20 msgr2=0x55716222d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 -- 172.21.15.69:0/1902781327 shutdown_connections 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 1 -- 172.21.15.69:0/1902781327 wait complete. 2024-08-02T18:00:20.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.845+0000 7f4d887c0740 20 client.18936 trim_cache size 0 max 16384 2024-08-02T18:00:20.849 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:00:20.849 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 26, 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 52, 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "2c6a3888": { 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 52, 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 150.0, 2024-08-02T18:00:20.850 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:00:20.851 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:00:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.848+0000 7f4d863c2640 1 -- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4d680a2cf0 msgr2=0x7f4d680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.848+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4d680a2cf0 0x7f4d680a51b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800af00 tx=0x7f4d7800a6b0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 -- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 msgr2=0x7f4d80100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 0x7f4d80100c80 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f4d74005940 tx=0x7f4d74003c10 comp rx=0 tx=0).stop 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 -- 172.21.15.69:0/2981124253 shutdown_connections 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4d80103490 0x7f4d80100c80 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4d680a2cf0 0x7f4d680a51b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d80108010 0x7f4d801011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 --2- 172.21.15.69:0/2981124253 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4d80102ae0 0x7f4d80100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 -- 172.21.15.69:0/2981124253 >> 172.21.15.69:0/2981124253 conn(0x7f4d800fc500 msgr2=0x7f4d80105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 -- 172.21.15.69:0/2981124253 shutdown_connections 2024-08-02T18:00:20.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:20.849+0000 7f4d863c2640 1 -- 172.21.15.69:0/2981124253 wait complete. 2024-08-02T18:00:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:20 smithi083 ceph-mon[37031]: pgmap v435: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-08-02T18:00:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:20 smithi083 ceph-mon[37031]: osdmap e111: 12 total, 12 up, 12 in 2024-08-02T18:00:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:20 smithi077 ceph-mon[37108]: pgmap v435: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-08-02T18:00:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:20 smithi077 ceph-mon[37108]: osdmap e111: 12 total, 12 up, 12 in 2024-08-02T18:00:21.170 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T18:00:21.171 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T18:00:21.650 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2c6a3888', total seconds elapsed: 35.1 2024-08-02T18:00:21.768 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:22.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.351+0000 7fbc2f557640 1 -- 172.21.15.69:0/786334233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc2811ae70 msgr2=0x7fbc2811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.351+0000 7fbc2f557640 1 --2- 172.21.15.69:0/786334233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc2811ae70 0x7fbc2811d260 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009900 tx=0x7fbc2402a440 comp rx=0 tx=0).stop 2024-08-02T18:00:22.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.352+0000 7fbc2f557640 1 -- 172.21.15.69:0/786334233 shutdown_connections 2024-08-02T18:00:22.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.352+0000 7fbc2f557640 1 --2- 172.21.15.69:0/786334233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc2811ae70 0x7fbc2811d260 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.352+0000 7fbc2f557640 1 --2- 172.21.15.69:0/786334233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc28109560 0x7fbc281099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.352+0000 7fbc2f557640 1 --2- 172.21.15.69:0/786334233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc28109180 0x7fbc2810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.352+0000 7fbc2f557640 1 -- 172.21.15.69:0/786334233 >> 172.21.15.69:0/786334233 conn(0x7fbc2806de70 msgr2=0x7fbc2806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:22.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.353+0000 7fbc2f557640 1 -- 172.21.15.69:0/786334233 shutdown_connections 2024-08-02T18:00:22.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.353+0000 7fbc2f557640 1 -- 172.21.15.69:0/786334233 wait complete. 2024-08-02T18:00:22.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.353+0000 7fbc2f557640 1 Processor -- start 2024-08-02T18:00:22.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.354+0000 7fbc2f557640 1 -- start start 2024-08-02T18:00:22.359 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.354+0000 7fbc2f557640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc28109180 0x7fbc281a7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.354+0000 7fbc2f557640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc28109560 0x7fbc281a7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.354+0000 7fbc2f557640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc2811ae70 0x7fbc281ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.354+0000 7fbc2f557640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc28120b10 con 0x7fbc28109560 2024-08-02T18:00:22.360 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.354+0000 7fbc2f557640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc28120990 con 0x7fbc2811ae70 2024-08-02T18:00:22.361 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.354+0000 7fbc2f557640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc28120c90 con 0x7fbc28109180 2024-08-02T18:00:22.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.358+0000 7fbc2d2f3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc28109180 0x7fbc281a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.364 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.358+0000 7fbc2daf4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc2811ae70 0x7fbc281ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.358+0000 7fbc2daf4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc2811ae70 0x7fbc281ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40300/0 (socket says 172.21.15.69:40300) 2024-08-02T18:00:22.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.358+0000 7fbc2daf4640 1 -- 172.21.15.69:0/4225686654 learned_addr learned my addr 172.21.15.69:0/4225686654 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:22.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.362+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc28109560 0x7fbc281a7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.364+0000 7fbc2daf4640 1 -- 172.21.15.69:0/4225686654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc28109180 msgr2=0x7fbc281a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.364+0000 7fbc2daf4640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc28109180 0x7fbc281a7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.364+0000 7fbc2daf4640 1 -- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc28109560 msgr2=0x7fbc281a7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.364+0000 7fbc2daf4640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc28109560 0x7fbc281a7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.364+0000 7fbc2daf4640 1 -- 172.21.15.69:0/4225686654 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc281abfc0 con 0x7fbc2811ae70 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.364+0000 7fbc2daf4640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc2811ae70 0x7fbc281ab8e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009850 tx=0x7fbc24005390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.365+0000 7fbc1e7fc640 1 -- 172.21.15.69:0/4225686654 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24040030 con 0x7fbc2811ae70 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.365+0000 7fbc1e7fc640 1 -- 172.21.15.69:0/4225686654 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2403c920 con 0x7fbc2811ae70 2024-08-02T18:00:22.368 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.365+0000 7fbc2f557640 1 -- 172.21.15.69:0/4225686654 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc281ac250 con 0x7fbc2811ae70 2024-08-02T18:00:22.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.365+0000 7fbc2f557640 1 -- 172.21.15.69:0/4225686654 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc281b3b90 con 0x7fbc2811ae70 2024-08-02T18:00:22.369 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.366+0000 7fbc1e7fc640 1 -- 172.21.15.69:0/4225686654 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24031c60 con 0x7fbc2811ae70 2024-08-02T18:00:22.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.373+0000 7fbc31955740 1 Processor -- start 2024-08-02T18:00:22.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.373+0000 7fbc31955740 1 -- start start 2024-08-02T18:00:22.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.374+0000 7fbc1e7fc640 1 -- 172.21.15.69:0/4225686654 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbc24043030 con 0x7fbc2811ae70 2024-08-02T18:00:22.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.374+0000 7fbc31955740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9c200 0x559da0d9c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.374+0000 7fbc31955740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9cb20 0x559da0da6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.375+0000 7fbc31955740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 0x559da0da97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.375+0000 7fbc31955740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559da0d5fe40 con 0x559da0da73e0 2024-08-02T18:00:22.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.375+0000 7fbc31955740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559da0d017f0 con 0x559da0d9cb20 2024-08-02T18:00:22.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.375+0000 7fbc2caf2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9c200 0x559da0d9c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.375+0000 7fbc2caf2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9c200 0x559da0d9c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45294/0 (socket says 172.21.15.69:45294) 2024-08-02T18:00:22.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.375+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3307988370 learned_addr learned my addr 172.21.15.69:0/3307988370 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:22.377 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.375+0000 7fbc31955740 1 -- 172.21.15.69:0/3307988370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559da0ca8fb0 con 0x559da0d9c200 2024-08-02T18:00:22.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.376+0000 7fbc1e7fc640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbc140a2f00 0x7fbc140a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.377+0000 7fbc1e7fc640 1 -- 172.21.15.69:0/4225686654 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fbc240f13c0 con 0x7fbc2811ae70 2024-08-02T18:00:22.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.377+0000 7fbc2daf4640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbc140a2f00 0x7fbc140a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.378+0000 7fbc2daf4640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbc140a2f00 0x7fbc140a53c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc240bd420 tx=0x7fbc2403c000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:22.380 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.379+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 0x559da0da97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.379+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9cb20 0x559da0da6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.379+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3307988370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9cb20 msgr2=0x559da0da6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.379+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9cb20 0x559da0da6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.379+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3307988370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 msgr2=0x559da0da97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.379+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 0x559da0da97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.381 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.379+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3307988370 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da0a736a0 con 0x559da0d9c200 2024-08-02T18:00:22.382 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.380+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9c200 0x559da0d9c5e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x559da0d99b10 tx=0x7fbc180096f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bea6d7647c90f1b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:22.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.381+0000 7fbc137fe640 1 -- 172.21.15.69:0/3307988370 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800f410 con 0x559da0d9c200 2024-08-02T18:00:22.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.381+0000 7fbc137fe640 1 -- 172.21.15.69:0/3307988370 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc1800fa90 con 0x559da0d9c200 2024-08-02T18:00:22.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.382+0000 7fbc137fe640 1 -- 172.21.15.69:0/3307988370 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18011a80 con 0x559da0d9c200 2024-08-02T18:00:22.383 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.382+0000 7fbc31955740 1 -- 172.21.15.69:0/3307988370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9c200 msgr2=0x559da0d9c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.384 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.382+0000 7fbc31955740 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9c200 0x559da0d9c5e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x559da0d99b10 tx=0x7fbc180096f0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.383+0000 7fbc31955740 1 -- 172.21.15.69:0/3307988370 shutdown_connections 2024-08-02T18:00:22.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.383+0000 7fbc31955740 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 0x559da0da97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.383+0000 7fbc31955740 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9cb20 0x559da0da6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.383+0000 7fbc31955740 1 --2- 172.21.15.69:0/3307988370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9c200 0x559da0d9c5e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.385 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.383+0000 7fbc31955740 1 -- 172.21.15.69:0/3307988370 >> 172.21.15.69:0/3307988370 conn(0x559da0d98790 msgr2=0x559da0d98b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:22.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.383+0000 7fbc31955740 1 -- 172.21.15.69:0/3307988370 shutdown_connections 2024-08-02T18:00:22.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.383+0000 7fbc31955740 1 -- 172.21.15.69:0/3307988370 wait complete. 2024-08-02T18:00:22.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.384+0000 7fbc31955740 1 -- start start 2024-08-02T18:00:22.386 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.384+0000 7fbc31955740 1 Processor -- start 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc31955740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0da4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc31955740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc31955740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 0x559da0da4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc2caf2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0da4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc31955740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559da0cbccb0 con 0x559da0da73e0 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc31955740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559da0cfd7b0 con 0x559da0d9c200 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc31955740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559da0daa000 con 0x559da0d9cb20 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc2caf2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0da4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40314/0 (socket says 172.21.15.69:40314) 2024-08-02T18:00:22.390 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3859442484 learned_addr learned my addr 172.21.15.69:0/3859442484 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc31955740 10 client.0 _ll_register_callbacks cb 0x559da0a10ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.385+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3859442484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 msgr2=0x559da0da48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3859442484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 msgr2=0x559da0da4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 0x559da0da4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3859442484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da0cbbbc0 con 0x559da0d9c200 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0da4360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009690 tx=0x7fbc18010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98567b3fca13c84e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc10ff9640 1 -- 172.21.15.69:0/3859442484 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1800f410 con 0x559da0d9c200 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc10ff9640 1 -- 172.21.15.69:0/3859442484 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fbc1800f5b0 con 0x559da0d9c200 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.386+0000 7fbc31955740 1 -- 172.21.15.69:0/3859442484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc1800fa90 con 0x559da0d9c200 2024-08-02T18:00:22.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc10ff9640 1 -- 172.21.15.69:0/3859442484 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc18030050 con 0x559da0d9c200 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc31955740 10 client.28298 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc31955740 1 -- 172.21.15.69:0/3859442484 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559da0db6000 con 0x559da0d9c200 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc10ff9640 1 -- 172.21.15.69:0/3859442484 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc18025030 con 0x559da0d9c200 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc10ff9640 1 -- 172.21.15.69:0/3859442484 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc18003710 con 0x559da0d9c200 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc31955740 10 client.28298 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc31955740 10 client.28298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc31955740 20 client.28298 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.387+0000 7fbc31955740 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559da0db73b0 0x559da0db97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.388+0000 7fbc31955740 4 client.28298 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.388+0000 7fbc31955740 1 -- 172.21.15.69:0/3859442484 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbc1800fa90 con 0x559da0db73b0 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.388+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559da0db73b0 0x559da0db97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.388+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559da0db73b0 0x559da0db97a0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27532e1ceea4d705 server_cookie=7b73073a8bf4cd48 in_seq=0 out_seq=0 2024-08-02T18:00:22.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.388+0000 7fbc10ff9640 10 client.28298 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:22.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.389+0000 7fbc10ff9640 1 -- 172.21.15.69:0/3859442484 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fbc1800fa90 con 0x559da0db73b0 2024-08-02T18:00:22.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.389+0000 7fbc10ff9640 10 client.28298 handle_command_reply: tid=0 2024-08-02T18:00:22.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.389+0000 7fbc31955740 1 client.28298 shutdown 2024-08-02T18:00:22.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.389+0000 7fbc31955740 1 -- 172.21.15.69:0/3859442484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 msgr2=0x559da0da4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.389+0000 7fbc31955740 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0da4360 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009690 tx=0x7fbc18010fd0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.389+0000 7fbc31955740 1 -- 172.21.15.69:0/3859442484 shutdown_connections 2024-08-02T18:00:22.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.389+0000 7fbc31955740 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0da73e0 0x559da0da4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.390+0000 7fbc31955740 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559da0db73b0 0x559da0db97a0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.390+0000 7fbc31955740 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.390+0000 7fbc31955740 1 --2- 172.21.15.69:0/3859442484 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0da4360 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.390+0000 7fbc31955740 1 -- 172.21.15.69:0/3859442484 >> 172.21.15.69:0/3859442484 conn(0x559da0d98790 msgr2=0x559da0d978f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:22.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.390+0000 7fbc10ff9640 0 client.28298 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:22.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.390+0000 7fbc31955740 1 -- 172.21.15.69:0/3859442484 shutdown_connections 2024-08-02T18:00:22.395 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.390+0000 7fbc31955740 1 -- 172.21.15.69:0/3859442484 wait complete. 2024-08-02T18:00:22.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.394+0000 7fbc31955740 20 client.28298 trim_cache size 0 max 16384 2024-08-02T18:00:22.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc31955740 1 Processor -- start 2024-08-02T18:00:22.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc31955740 1 -- start start 2024-08-02T18:00:22.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc31955740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9c200 0x559da0d9e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc31955740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc31955740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0da73e0 0x559da0dbef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc31955740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c012940 con 0x559da0d9c200 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc31955740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c010210 con 0x559da0da73e0 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc31955740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c00e060 con 0x559da0d9cb20 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc2caf2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9c200 0x559da0d9e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc2caf2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9c200 0x559da0d9e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37756/0 (socket says 172.21.15.69:37756) 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3422225824 learned_addr learned my addr 172.21.15.69:0/3422225824 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.419+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0da73e0 0x559da0dbef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3422225824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 msgr2=0x559da0da4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:22.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3422225824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0da73e0 msgr2=0x559da0dbef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0da73e0 0x559da0dbef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc2caf2640 1 -- 172.21.15.69:0/3422225824 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559da0d9c910 con 0x559da0d9c200 2024-08-02T18:00:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9c200 0x559da0d9e360 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009ad0 tx=0x7fbc18009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d1ae6950dcbe01 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc12ffd640 1 -- 172.21.15.69:0/3422225824 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18023040 con 0x559da0d9c200 2024-08-02T18:00:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc12ffd640 1 -- 172.21.15.69:0/3422225824 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc18025070 con 0x559da0d9c200 2024-08-02T18:00:22.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.420+0000 7fbc12ffd640 1 -- 172.21.15.69:0/3422225824 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18011b30 con 0x559da0d9c200 2024-08-02T18:00:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 -- 172.21.15.69:0/3422225824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9c200 msgr2=0x559da0d9e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9c200 0x559da0d9e360 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009ad0 tx=0x7fbc18009b20 comp rx=0 tx=0).stop 2024-08-02T18:00:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 -- 172.21.15.69:0/3422225824 shutdown_connections 2024-08-02T18:00:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0da73e0 0x559da0dbef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.426 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0d9cb20 0x559da0da4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 --2- 172.21.15.69:0/3422225824 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9c200 0x559da0d9e360 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 -- 172.21.15.69:0/3422225824 >> 172.21.15.69:0/3422225824 conn(0x559da0d98790 msgr2=0x559da0d9dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 -- 172.21.15.69:0/3422225824 shutdown_connections 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 -- 172.21.15.69:0/3422225824 wait complete. 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 -- start start 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.421+0000 7fbc31955740 1 Processor -- start 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc31955740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0daa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc31955740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9cb20 0x559da0daa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0daa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0daa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:40322/0 (socket says 172.21.15.69:40322) 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 -- 172.21.15.69:0/216610471 learned_addr learned my addr 172.21.15.69:0/216610471 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc31955740 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0da73e0 0x559da0daad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c00e6d0 con 0x559da0d9cb20 2024-08-02T18:00:22.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c00fb40 con 0x559da0d9c200 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c00eda0 con 0x559da0da73e0 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9cb20 0x559da0daa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc31955740 10 client.0 _ll_register_callbacks cb 0x7fbc18013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 -- 172.21.15.69:0/216610471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0da73e0 msgr2=0x559da0daad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0da73e0 0x559da0daad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 -- 172.21.15.69:0/216610471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9cb20 msgr2=0x559da0daa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9cb20 0x559da0daa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 -- 172.21.15.69:0/216610471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559da0e1ff20 con 0x559da0d9c200 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9cb20 0x559da0daa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc2caf2640 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0daa2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009de0 tx=0x7fbc18003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=561826073e75891 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.422+0000 7fbc10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc10ff9640 1 -- 172.21.15.69:0/216610471 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc18023070 con 0x559da0d9c200 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc18025070 con 0x559da0d9c200 2024-08-02T18:00:22.428 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc10ff9640 1 -- 172.21.15.69:0/216610471 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fbc18011850 con 0x559da0d9c200 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc10ff9640 1 -- 172.21.15.69:0/216610471 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc18009690 con 0x559da0d9c200 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 10 client.28304 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559da0e21220 con 0x559da0d9c200 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc10ff9640 1 -- 172.21.15.69:0/216610471 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc18025070 con 0x559da0d9c200 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc10ff9640 1 -- 172.21.15.69:0/216610471 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fbc1802a870 con 0x559da0d9c200 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 10 client.28304 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 10 client.28304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 20 client.28304 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559da0e218f0 0x559da0e23d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 4 client.28304 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559da0d0ed60 con 0x559da0e218f0 2024-08-02T18:00:22.429 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.423+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559da0e218f0 0x559da0e23d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:22.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.424+0000 7fbc2d2f3640 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559da0e218f0 0x559da0e23d10 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7391e0a08e1eceaf server_cookie=430856d524d963ed in_seq=0 out_seq=0 2024-08-02T18:00:22.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.424+0000 7fbc10ff9640 10 client.28304 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:22.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.425+0000 7fbc10ff9640 1 -- 172.21.15.69:0/216610471 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559da0d0ed60 con 0x559da0e218f0 2024-08-02T18:00:22.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.425+0000 7fbc10ff9640 10 client.28304 handle_command_reply: tid=0 2024-08-02T18:00:22.430 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.425+0000 7fbc31955740 1 client.28304 shutdown 2024-08-02T18:00:22.431 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.425+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 msgr2=0x559da0daa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.425+0000 7fbc31955740 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0daa2f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbc18009de0 tx=0x7fbc18003f80 comp rx=0 tx=0).stop 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 shutdown_connections 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559da0d9cb20 0x559da0daa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559da0e218f0 0x559da0e23d10 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559da0da73e0 0x559da0daad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 1 --2- 172.21.15.69:0/216610471 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559da0d9c200 0x559da0daa2f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc10ff9640 0 client.28304 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 >> 172.21.15.69:0/216610471 conn(0x559da0d98790 msgr2=0x559da0d9e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 shutdown_connections 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 1 -- 172.21.15.69:0/216610471 wait complete. 2024-08-02T18:00:22.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.426+0000 7fbc31955740 20 client.28304 trim_cache size 0 max 16384 2024-08-02T18:00:22.433 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:00:22.433 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T18:00:22.433 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "839cd04f-6480-4582-ad44-26062b4d5960", 2024-08-02T18:00:22.433 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T18:00:22.433 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:00:22.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.427+0000 7fbc2f557640 1 -- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbc140a2f00 msgr2=0x7fbc140a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.434 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.427+0000 7fbc2f557640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbc140a2f00 0x7fbc140a53c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbc240bd420 tx=0x7fbc2403c000 comp rx=0 tx=0).stop 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.427+0000 7fbc2f557640 1 -- 172.21.15.69:0/4225686654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc2811ae70 msgr2=0x7fbc281ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.427+0000 7fbc2f557640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc2811ae70 0x7fbc281ab8e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009850 tx=0x7fbc24005390 comp rx=0 tx=0).stop 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.427+0000 7fbc2f557640 1 -- 172.21.15.69:0/4225686654 shutdown_connections 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.428+0000 7fbc2f557640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fbc28109560 0x7fbc281a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.428+0000 7fbc2f557640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fbc140a2f00 0x7fbc140a53c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.428+0000 7fbc2f557640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fbc2811ae70 0x7fbc281ab8e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.428+0000 7fbc2f557640 1 --2- 172.21.15.69:0/4225686654 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc28109180 0x7fbc281a7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.428+0000 7fbc2f557640 1 -- 172.21.15.69:0/4225686654 >> 172.21.15.69:0/4225686654 conn(0x7fbc2806de70 msgr2=0x7fbc2811ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.428+0000 7fbc2f557640 1 -- 172.21.15.69:0/4225686654 shutdown_connections 2024-08-02T18:00:22.435 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:22.428+0000 7fbc2f557640 1 -- 172.21.15.69:0/4225686654 wait complete. 2024-08-02T18:00:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:22 smithi069 ceph-mon[32746]: pgmap v437: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-08-02T18:00:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:22 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T18:00:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:22 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:22.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:22 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T18:00:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:22 smithi083 ceph-mon[37031]: pgmap v437: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-08-02T18:00:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:22 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T18:00:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:22 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:22 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T18:00:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:22 smithi077 ceph-mon[37108]: pgmap v437: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 38 KiB/s wr, 24 op/s 2024-08-02T18:00:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:22 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T18:00:23.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:22 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:23.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:22 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T18:00:23.209 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T18:00:23.653 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:23.990 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:23 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:24.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:23 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:24.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:23 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:24.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.297+0000 7f5b6e101640 1 -- 172.21.15.69:0/3492985802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6810d260 msgr2=0x7f5b68114de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:24.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.297+0000 7f5b6e101640 1 --2- 172.21.15.69:0/3492985802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6810d260 0x7f5b68114de0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009900 tx=0x7f5b5c02a410 comp rx=0 tx=0).stop 2024-08-02T18:00:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.302+0000 7f5b6e101640 1 -- 172.21.15.69:0/3492985802 shutdown_connections 2024-08-02T18:00:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.302+0000 7f5b6e101640 1 --2- 172.21.15.69:0/3492985802 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b6810d260 0x7f5b68114de0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.302+0000 7f5b6e101640 1 --2- 172.21.15.69:0/3492985802 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108b30 0x7f5b6810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.302+0000 7f5b6e101640 1 --2- 172.21.15.69:0/3492985802 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b68108180 0x7f5b68108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.302+0000 7f5b6e101640 1 -- 172.21.15.69:0/3492985802 >> 172.21.15.69:0/3492985802 conn(0x7f5b680778d0 msgr2=0x7f5b68079cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:24.303 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.302+0000 7f5b6e101640 1 -- 172.21.15.69:0/3492985802 shutdown_connections 2024-08-02T18:00:24.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.302+0000 7f5b6e101640 1 -- 172.21.15.69:0/3492985802 wait complete. 2024-08-02T18:00:24.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.303+0000 7f5b6e101640 1 Processor -- start 2024-08-02T18:00:24.305 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.303+0000 7f5b6e101640 1 -- start start 2024-08-02T18:00:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.303+0000 7f5b6e101640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108180 0x7f5b6807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b6e101640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b68108b30 0x7f5b6807c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b6e101640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b6810d260 0x7f5b68080b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b6e101640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6807ab00 con 0x7f5b6810d260 2024-08-02T18:00:24.306 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b6e101640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6807a980 con 0x7f5b68108b30 2024-08-02T18:00:24.307 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b6e101640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6807ac80 con 0x7f5b68108180 2024-08-02T18:00:24.310 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108180 0x7f5b6807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108180 0x7f5b6807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45322/0 (socket says 172.21.15.69:45322) 2024-08-02T18:00:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b677fe640 1 -- 172.21.15.69:0/163360308 learned_addr learned my addr 172.21.15.69:0/163360308 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b677fe640 1 -- 172.21.15.69:0/163360308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b68108b30 msgr2=0x7f5b6807c800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T18:00:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b677fe640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b68108b30 0x7f5b6807c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.311 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b677fe640 1 -- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b6810d260 msgr2=0x7f5b68080b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T18:00:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b677fe640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b6810d260 0x7f5b68080b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.304+0000 7f5b677fe640 1 -- 172.21.15.69:0/163360308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b68081100 con 0x7f5b68108180 2024-08-02T18:00:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.305+0000 7f5b677fe640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108180 0x7f5b6807c2c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000b4a0 tx=0x7f5b6000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.310+0000 7f5b64ff9640 1 -- 172.21.15.69:0/163360308 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60014070 con 0x7f5b68108180 2024-08-02T18:00:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.310+0000 7f5b6e101640 1 -- 172.21.15.69:0/163360308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b680813f0 con 0x7f5b68108180 2024-08-02T18:00:24.312 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.310+0000 7f5b6e101640 1 -- 172.21.15.69:0/163360308 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b680818d0 con 0x7f5b68108180 2024-08-02T18:00:24.316 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.314+0000 7f5b704ff740 1 Processor -- start 2024-08-02T18:00:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.317+0000 7f5b704ff740 1 -- start start 2024-08-02T18:00:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.317+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba52f7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.317+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba52f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.317+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba53045a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.317+0000 7f5b704ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555ba525c980 con 0x555ba52f7850 2024-08-02T18:00:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.317+0000 7f5b704ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555ba51f4a70 con 0x555ba53021b0 2024-08-02T18:00:24.319 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.317+0000 7f5b704ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ba5258940 con 0x555ba52f8b40 2024-08-02T18:00:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.318+0000 7f5b64ff9640 1 -- 172.21.15.69:0/163360308 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60010af0 con 0x7f5b68108180 2024-08-02T18:00:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.318+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba52f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.318+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba52f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37790/0 (socket says 172.21.15.69:37790) 2024-08-02T18:00:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.318+0000 7f5b677fe640 1 -- 172.21.15.69:0/3988069756 learned_addr learned my addr 172.21.15.69:0/3988069756 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.319+0000 7f5b66ffd640 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba53045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.321 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.319+0000 7f5b677fe640 1 -- 172.21.15.69:0/3988069756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 msgr2=0x555ba52f7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.319+0000 7f5b677fe640 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba52f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.319+0000 7f5b677fe640 1 -- 172.21.15.69:0/3988069756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 msgr2=0x555ba53045a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.319+0000 7f5b64ff9640 1 -- 172.21.15.69:0/163360308 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60017d50 con 0x7f5b68108180 2024-08-02T18:00:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.319+0000 7f5b677fe640 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba53045a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.319+0000 7f5b677fe640 1 -- 172.21.15.69:0/3988069756 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba52576e0 con 0x555ba52f7850 2024-08-02T18:00:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.319+0000 7f5b66ffd640 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba53045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:00:24.322 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.320+0000 7f5b64ff9640 1 -- 172.21.15.69:0/163360308 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5b60017070 con 0x7f5b68108180 2024-08-02T18:00:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.321+0000 7f5b677fe640 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba52f7c90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5b600cced0 tx=0x7f5b600d81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b20d541a4d2d74 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.321+0000 7f5b64ff9640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5b500a2f30 0x7f5b500a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.321+0000 7f5b6e101640 1 -- 172.21.15.69:0/3988069756 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60010610 con 0x555ba52f7850 2024-08-02T18:00:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.321+0000 7f5b6e101640 1 -- 172.21.15.69:0/3988069756 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b600e1070 con 0x555ba52f7850 2024-08-02T18:00:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.321+0000 7f5b67fff640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5b500a2f30 0x7f5b500a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.322+0000 7f5b64ff9640 1 -- 172.21.15.69:0/163360308 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f5b600cfb00 con 0x7f5b68108180 2024-08-02T18:00:24.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.322+0000 7f5b704ff740 1 -- 172.21.15.69:0/3988069756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 msgr2=0x555ba52f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.322+0000 7f5b704ff740 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba52f7c90 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f5b600cced0 tx=0x7f5b600d81e0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.323+0000 7f5b67fff640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5b500a2f30 0x7f5b500a53f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009230 tx=0x7f5b5c005760 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.323+0000 7f5b704ff740 1 -- 172.21.15.69:0/3988069756 shutdown_connections 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.323+0000 7f5b704ff740 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba52f7c90 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.323+0000 7f5b704ff740 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba53045a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.323+0000 7f5b704ff740 1 --2- 172.21.15.69:0/3988069756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba52f7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.323+0000 7f5b704ff740 1 -- 172.21.15.69:0/3988069756 >> 172.21.15.69:0/3988069756 conn(0x555ba52f3aa0 msgr2=0x555ba52f3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.324+0000 7f5b704ff740 1 -- 172.21.15.69:0/3988069756 shutdown_connections 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.324+0000 7f5b704ff740 1 -- 172.21.15.69:0/3988069756 wait complete. 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.324+0000 7f5b704ff740 1 -- start start 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.325+0000 7f5b704ff740 1 Processor -- start 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.325+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f7850 0x555ba52ff0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.325+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba52ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.325+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52ffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.325+0000 7f5b704ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x555ba5305010 con 0x555ba53021b0 2024-08-02T18:00:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.326+0000 7f5b704ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x555ba5217aa0 con 0x555ba52f7850 2024-08-02T18:00:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.326+0000 7f5b704ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x555ba5305190 con 0x555ba52f8b40 2024-08-02T18:00:24.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.326+0000 7f5b704ff740 10 client.0 _ll_register_callbacks cb 0x555ba5206e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.326+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52ffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.326+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52ffb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37806/0 (socket says 172.21.15.69:37806) 2024-08-02T18:00:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.326+0000 7f5b67fff640 1 -- 172.21.15.69:0/112533414 learned_addr learned my addr 172.21.15.69:0/112533414 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:24.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.326+0000 7f5b66ffd640 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba52ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.327+0000 7f5b67fff640 1 -- 172.21.15.69:0/112533414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 msgr2=0x555ba52ff620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.327+0000 7f5b67fff640 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba52ff620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.327+0000 7f5b67fff640 1 -- 172.21.15.69:0/112533414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f7850 msgr2=0x555ba52ff0e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:00:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.327+0000 7f5b67fff640 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f7850 0x555ba52ff0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.327+0000 7f5b67fff640 1 -- 172.21.15.69:0/112533414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba51fb650 con 0x555ba53021b0 2024-08-02T18:00:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.327+0000 7f5b66ffd640 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba52ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.328+0000 7f5b67fff640 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52ffb60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c031980 tx=0x7f5b5c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74ec7b7ccfcb300 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.328+0000 7f5b46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:24.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.328+0000 7f5b46ffd640 1 -- 172.21.15.69:0/112533414 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c04b020 con 0x555ba53021b0 2024-08-02T18:00:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.328+0000 7f5b46ffd640 1 -- 172.21.15.69:0/112533414 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f5b5c004040 con 0x555ba53021b0 2024-08-02T18:00:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.328+0000 7f5b704ff740 1 -- 172.21.15.69:0/112533414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555ba5269ef0 con 0x555ba53021b0 2024-08-02T18:00:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.328+0000 7f5b46ffd640 1 -- 172.21.15.69:0/112533414 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b5c056050 con 0x555ba53021b0 2024-08-02T18:00:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.329+0000 7f5b704ff740 10 client.18957 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.329+0000 7f5b704ff740 1 -- 172.21.15.69:0/112533414 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555ba4fce3c0 con 0x555ba53021b0 2024-08-02T18:00:24.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.329+0000 7f5b46ffd640 1 -- 172.21.15.69:0/112533414 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555ba5269ef0 con 0x555ba53021b0 2024-08-02T18:00:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.329+0000 7f5b46ffd640 1 -- 172.21.15.69:0/112533414 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5b5c047440 con 0x555ba53021b0 2024-08-02T18:00:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.329+0000 7f5b704ff740 10 client.18957 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.329+0000 7f5b704ff740 10 client.18957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.330+0000 7f5b704ff740 20 client.18957 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.330+0000 7f5b704ff740 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555ba5312530 0x555ba53149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.330+0000 7f5b704ff740 4 client.18957 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:24.334 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.330+0000 7f5b704ff740 1 -- 172.21.15.69:0/112533414 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ba5264400 con 0x555ba5312530 2024-08-02T18:00:24.335 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.334+0000 7f5b677fe640 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555ba5312530 0x555ba53149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.337+0000 7f5b677fe640 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555ba5312530 0x555ba53149f0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddf9892edb5a360 server_cookie=52c181cef15dce52 in_seq=0 out_seq=0 2024-08-02T18:00:24.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.337+0000 7f5b46ffd640 10 client.18957 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:24.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.339+0000 7f5b46ffd640 1 -- 172.21.15.69:0/112533414 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ba5264400 con 0x555ba5312530 2024-08-02T18:00:24.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.339+0000 7f5b46ffd640 10 client.18957 handle_command_reply: tid=0 2024-08-02T18:00:24.340 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.339+0000 7f5b704ff740 1 client.18957 shutdown 2024-08-02T18:00:24.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.344+0000 7f5b704ff740 1 -- 172.21.15.69:0/112533414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 msgr2=0x555ba52ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:24.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.344+0000 7f5b704ff740 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52ffb60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c031980 tx=0x7f5b5c009260 comp rx=0 tx=0).stop 2024-08-02T18:00:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.348+0000 7f5b704ff740 1 -- 172.21.15.69:0/112533414 shutdown_connections 2024-08-02T18:00:24.349 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.348+0000 7f5b704ff740 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52ffb60 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.348+0000 7f5b704ff740 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555ba5312530 0x555ba53149f0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.348+0000 7f5b704ff740 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba52ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.348+0000 7f5b46ffd640 0 client.18957 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.348+0000 7f5b704ff740 1 --2- 172.21.15.69:0/112533414 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f7850 0x555ba52ff0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.348+0000 7f5b704ff740 1 -- 172.21.15.69:0/112533414 >> 172.21.15.69:0/112533414 conn(0x555ba52f3aa0 msgr2=0x555ba52f2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:24.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.352+0000 7f5b704ff740 1 -- 172.21.15.69:0/112533414 shutdown_connections 2024-08-02T18:00:24.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.352+0000 7f5b704ff740 1 -- 172.21.15.69:0/112533414 wait complete. 2024-08-02T18:00:24.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.352+0000 7f5b704ff740 20 client.18957 trim_cache size 0 max 16384 2024-08-02T18:00:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.386+0000 7f5b704ff740 1 Processor -- start 2024-08-02T18:00:24.388 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b704ff740 1 -- start start 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba531a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba531a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba52fc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b704ff740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54012a50 con 0x555ba52f7850 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b704ff740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b54010320 con 0x555ba53021b0 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b704ff740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5400e170 con 0x555ba52f8b40 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba531a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba531a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:37810/0 (socket says 172.21.15.69:37810) 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 -- 172.21.15.69:0/2692815245 learned_addr learned my addr 172.21.15.69:0/2692815245 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b67fff640 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba52fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b66ffd640 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba531a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 -- 172.21.15.69:0/2692815245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 msgr2=0x555ba531a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:24.391 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba531a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 -- 172.21.15.69:0/2692815245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 msgr2=0x555ba52fc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba52fc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 -- 172.21.15.69:0/2692815245 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba52f38d0 con 0x555ba52f7850 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.387+0000 7f5b677fe640 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba531a2c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5b60017fd0 tx=0x7f5b6000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2931f70e7a1b49f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.388+0000 7f5b4d7fa640 1 -- 172.21.15.69:0/2692815245 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6001e040 con 0x555ba52f7850 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.388+0000 7f5b4d7fa640 1 -- 172.21.15.69:0/2692815245 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b600e1040 con 0x555ba52f7850 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.388+0000 7f5b4d7fa640 1 -- 172.21.15.69:0/2692815245 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600a0110 con 0x555ba52f7850 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.389+0000 7f5b704ff740 1 -- 172.21.15.69:0/2692815245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 msgr2=0x555ba531a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:24.392 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.389+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba531a2c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f5b60017fd0 tx=0x7f5b6000bf90 comp rx=0 tx=0).stop 2024-08-02T18:00:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.390+0000 7f5b704ff740 1 -- 172.21.15.69:0/2692815245 shutdown_connections 2024-08-02T18:00:24.393 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.390+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba53021b0 0x555ba52fc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.390+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f8b40 0x555ba531a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.390+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2692815245 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba52f7850 0x555ba531a2c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.390+0000 7f5b704ff740 1 -- 172.21.15.69:0/2692815245 >> 172.21.15.69:0/2692815245 conn(0x555ba52f3aa0 msgr2=0x555ba52f2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.390+0000 7f5b704ff740 1 -- 172.21.15.69:0/2692815245 shutdown_connections 2024-08-02T18:00:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.391+0000 7f5b704ff740 1 -- 172.21.15.69:0/2692815245 wait complete. 2024-08-02T18:00:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.391+0000 7f5b704ff740 1 -- start start 2024-08-02T18:00:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.392+0000 7f5b704ff740 1 Processor -- start 2024-08-02T18:00:24.394 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.392+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f7850 0x555ba530a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.392+0000 7f5b704ff740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f8b40 0x555ba52fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.393+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f7850 0x555ba530a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.393+0000 7f5b677fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f7850 0x555ba530a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45344/0 (socket says 172.21.15.69:45344) 2024-08-02T18:00:24.396 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.393+0000 7f5b677fe640 1 -- 172.21.15.69:0/2644514085 learned_addr learned my addr 172.21.15.69:0/2644514085 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.393+0000 7f5b66ffd640 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f8b40 0x555ba52fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.393+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52fbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.393+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5400e7e0 con 0x555ba53021b0 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.394+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5400fc50 con 0x555ba52f8b40 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.394+0000 7f5b67fff640 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52fbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.394+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5400eeb0 con 0x555ba52f7850 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.394+0000 7f5b677fe640 1 -- 172.21.15.69:0/2644514085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f8b40 msgr2=0x555ba52fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.394+0000 7f5b677fe640 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f8b40 0x555ba52fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.394+0000 7f5b677fe640 1 -- 172.21.15.69:0/2644514085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 msgr2=0x555ba52fbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.394+0000 7f5b677fe640 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52fbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.394+0000 7f5b677fe640 1 -- 172.21.15.69:0/2644514085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba52fc530 con 0x555ba52f7850 2024-08-02T18:00:24.397 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.395+0000 7f5b677fe640 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f7850 0x555ba530a900 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5b6001b040 tx=0x7f5b600d3140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b39fa9f42021c1f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.395+0000 7f5b46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.395+0000 7f5b46ffd640 1 -- 172.21.15.69:0/2644514085 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600b9060 con 0x555ba52f7850 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.395+0000 7f5b46ffd640 1 -- 172.21.15.69:0/2644514085 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f5b600d8db0 con 0x555ba52f7850 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.395+0000 7f5b704ff740 10 client.0 _ll_register_callbacks cb 0x7f5b6806b330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.395+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b600e1040 con 0x555ba52f7850 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.396+0000 7f5b46ffd640 1 -- 172.21.15.69:0/2644514085 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5b6001e040 con 0x555ba52f7850 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.396+0000 7f5b704ff740 10 client.28075 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.396+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x555ba537cc40 con 0x555ba52f7850 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.396+0000 7f5b46ffd640 1 -- 172.21.15.69:0/2644514085 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5b600983d0 con 0x555ba52f7850 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.396+0000 7f5b704ff740 10 client.28075 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:24.400 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.396+0000 7f5b704ff740 10 client.28075 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.397+0000 7f5b704ff740 20 client.28075 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.397+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555ba5312530 0x555ba537ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.397+0000 7f5b704ff740 4 client.28075 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.397+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5b600e1040 con 0x555ba5312530 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.397+0000 7f5b66ffd640 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555ba5312530 0x555ba537ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.397+0000 7f5b46ffd640 10 client.28075 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.397+0000 7f5b66ffd640 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555ba5312530 0x555ba537ec50 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328ed31cb35152e5 server_cookie=942e57e98c644989 in_seq=0 out_seq=0 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.398+0000 7f5b46ffd640 1 -- 172.21.15.69:0/2644514085 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f5b600e1040 con 0x555ba5312530 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.398+0000 7f5b46ffd640 10 client.28075 handle_command_reply: tid=0 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.398+0000 7f5b704ff740 1 client.28075 shutdown 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.398+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f7850 msgr2=0x555ba530a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:24.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.398+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f7850 0x555ba530a900 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5b6001b040 tx=0x7f5b600d3140 comp rx=0 tx=0).stop 2024-08-02T18:00:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.402+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 shutdown_connections 2024-08-02T18:00:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.402+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x555ba53021b0 0x555ba52fbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.402+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x555ba5312530 0x555ba537ec50 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.402+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x555ba52f8b40 0x555ba52fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.402+0000 7f5b46ffd640 0 client.28075 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:24.405 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.402+0000 7f5b704ff740 1 --2- 172.21.15.69:0/2644514085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x555ba52f7850 0x555ba530a900 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.402+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 >> 172.21.15.69:0/2644514085 conn(0x555ba52f3aa0 msgr2=0x555ba5300750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:24.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.403+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 shutdown_connections 2024-08-02T18:00:24.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.403+0000 7f5b704ff740 1 -- 172.21.15.69:0/2644514085 wait complete. 2024-08-02T18:00:24.406 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.403+0000 7f5b704ff740 20 client.28075 trim_cache size 0 max 16384 2024-08-02T18:00:24.407 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:00:24.407 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T18:00:24.407 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T18:00:24.408 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:00:24.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 -- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5b500a2f30 msgr2=0x7f5b500a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:24.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5b500a2f30 0x7f5b500a53f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009230 tx=0x7f5b5c005760 comp rx=0 tx=0).stop 2024-08-02T18:00:24.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 -- 172.21.15.69:0/163360308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108180 msgr2=0x7f5b6807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:24.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108180 0x7f5b6807c2c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000b4a0 tx=0x7f5b6000b960 comp rx=0 tx=0).stop 2024-08-02T18:00:24.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 -- 172.21.15.69:0/163360308 shutdown_connections 2024-08-02T18:00:24.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5b6810d260 0x7f5b68080b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5b500a2f30 0x7f5b500a53f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5b68108b30 0x7f5b6807c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 --2- 172.21.15.69:0/163360308 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b68108180 0x7f5b6807c2c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:24.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 -- 172.21.15.69:0/163360308 >> 172.21.15.69:0/163360308 conn(0x7f5b680778d0 msgr2=0x7f5b68078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:24.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 -- 172.21.15.69:0/163360308 shutdown_connections 2024-08-02T18:00:24.411 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:24.404+0000 7f5b4e7fc640 1 -- 172.21.15.69:0/163360308 wait complete. 2024-08-02T18:00:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 795.0 s, 415.0 tps, lat 1685.716 ms stddev 6637.322 2024-08-02T18:00:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:24 smithi083 ceph-mon[37031]: pgmap v438: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 773 KiB/s wr, 101 op/s 2024-08-02T18:00:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:24 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T18:00:25.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:24 smithi077 ceph-mon[37108]: pgmap v438: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 773 KiB/s wr, 101 op/s 2024-08-02T18:00:25.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:24 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T18:00:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:24 smithi069 ceph-mon[32746]: pgmap v438: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 773 KiB/s wr, 101 op/s 2024-08-02T18:00:25.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:24 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T18:00:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:26 smithi083 ceph-mon[37031]: pgmap v439: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 928 KiB/s wr, 144 op/s 2024-08-02T18:00:27.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:26 smithi077 ceph-mon[37108]: pgmap v439: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 928 KiB/s wr, 144 op/s 2024-08-02T18:00:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:26 smithi069 ceph-mon[32746]: pgmap v439: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 928 KiB/s wr, 144 op/s 2024-08-02T18:00:28.366 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T18:00:28.366 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:00:28.892 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:29.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:28 smithi083 ceph-mon[37031]: pgmap v440: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-08-02T18:00:29.062 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:28 smithi069 ceph-mon[32746]: pgmap v440: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-08-02T18:00:29.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:28 smithi077 ceph-mon[37108]: pgmap v440: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 1.1 MiB/s wr, 175 op/s 2024-08-02T18:00:29.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.507+0000 7fa82b57c640 1 -- 172.21.15.69:0/415151320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 msgr2=0x7fa824115220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:29.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.507+0000 7fa82b57c640 1 --2- 172.21.15.69:0/415151320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 0x7fa824115220 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005fd0 tx=0x7fa81c02b430 comp rx=0 tx=0).stop 2024-08-02T18:00:29.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.515+0000 7fa82b57c640 1 -- 172.21.15.69:0/415151320 shutdown_connections 2024-08-02T18:00:29.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.515+0000 7fa82b57c640 1 --2- 172.21.15.69:0/415151320 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 0x7fa824115220 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.515+0000 7fa82b57c640 1 --2- 172.21.15.69:0/415151320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa824109b30 0x7fa82410e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.516 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.515+0000 7fa82b57c640 1 --2- 172.21.15.69:0/415151320 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa824109180 0x7fa824109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.517 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.515+0000 7fa82b57c640 1 -- 172.21.15.69:0/415151320 >> 172.21.15.69:0/415151320 conn(0x7fa82406e000 msgr2=0x7fa82406e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:29.520 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.518+0000 7fa82b57c640 1 -- 172.21.15.69:0/415151320 shutdown_connections 2024-08-02T18:00:29.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.522+0000 7fa82b57c640 1 -- 172.21.15.69:0/415151320 wait complete. 2024-08-02T18:00:29.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.522+0000 7fa82b57c640 1 Processor -- start 2024-08-02T18:00:29.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.523+0000 7fa82b57c640 1 -- start start 2024-08-02T18:00:29.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.523+0000 7fa82b57c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa824109180 0x7fa824112430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:29.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.523+0000 7fa82b57c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa824109b30 0x7fa824112970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:29.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.523+0000 7fa82b57c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 0x7fa824112eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:29.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.523+0000 7fa82b57c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa8241180d0 con 0x7fa824109b30 2024-08-02T18:00:29.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.523+0000 7fa82b57c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa824117f50 con 0x7fa82410e7b0 2024-08-02T18:00:29.525 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.523+0000 7fa82b57c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa824118250 con 0x7fa824109180 2024-08-02T18:00:29.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.524+0000 7fa829b19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 0x7fa824112eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:29.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.524+0000 7fa829b19640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 0x7fa824112eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44342/0 (socket says 172.21.15.69:44342) 2024-08-02T18:00:29.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.524+0000 7fa829b19640 1 -- 172.21.15.69:0/2330930635 learned_addr learned my addr 172.21.15.69:0/2330930635 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:29.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.526+0000 7fa829b19640 1 -- 172.21.15.69:0/2330930635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa824109180 msgr2=0x7fa824112430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T18:00:29.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.526+0000 7fa829b19640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa824109180 0x7fa824112430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.528 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.526+0000 7fa829b19640 1 -- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa824109b30 msgr2=0x7fa824112970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:29.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.526+0000 7fa829b19640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa824109b30 0x7fa824112970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.526+0000 7fa829b19640 1 -- 172.21.15.69:0/2330930635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82411cdb0 con 0x7fa82410e7b0 2024-08-02T18:00:29.531 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.529+0000 7fa829b19640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 0x7fa824112eb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02b950 tx=0x7fa81c00aab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:29.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.531+0000 7fa81a7fc640 1 -- 172.21.15.69:0/2330930635 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c042070 con 0x7fa82410e7b0 2024-08-02T18:00:29.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.531+0000 7fa81a7fc640 1 -- 172.21.15.69:0/2330930635 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa81c003970 con 0x7fa82410e7b0 2024-08-02T18:00:29.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.531+0000 7fa81a7fc640 1 -- 172.21.15.69:0/2330930635 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81c037040 con 0x7fa82410e7b0 2024-08-02T18:00:29.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.531+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82411d040 con 0x7fa82410e7b0 2024-08-02T18:00:29.533 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.531+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82411d530 con 0x7fa82410e7b0 2024-08-02T18:00:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.537+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa808005290 con 0x7fa82410e7b0 2024-08-02T18:00:29.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.537+0000 7fa81a7fc640 1 -- 172.21.15.69:0/2330930635 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa81c045070 con 0x7fa82410e7b0 2024-08-02T18:00:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.538+0000 7fa81a7fc640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8100a2ec0 0x7fa8100a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.538+0000 7fa81a7fc640 1 -- 172.21.15.69:0/2330930635 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fa81c0ef2e0 con 0x7fa82410e7b0 2024-08-02T18:00:29.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.540+0000 7fa829318640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8100a2ec0 0x7fa8100a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:29.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.546+0000 7fa829318640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8100a2ec0 0x7fa8100a5380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa820009670 tx=0x7fa820006bd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:29.553 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.551+0000 7fa81a7fc640 1 -- 172.21.15.69:0/2330930635 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fa81c0bbab0 con 0x7fa82410e7b0 2024-08-02T18:00:29.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.881+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa808005850 con 0x7fa82410e7b0 2024-08-02T18:00:29.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.882+0000 7fa81a7fc640 1 -- 172.21.15.69:0/2330930635 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fa81c0bb3b0 con 0x7fa82410e7b0 2024-08-02T18:00:29.884 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:00:29.884 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:00:29.885 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T18:00:29.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8100a2ec0 msgr2=0x7fa8100a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:29.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8100a2ec0 0x7fa8100a5380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa820009670 tx=0x7fa820006bd0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 msgr2=0x7fa824112eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:29.886 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 0x7fa824112eb0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02b950 tx=0x7fa81c00aab0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 shutdown_connections 2024-08-02T18:00:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa824109b30 0x7fa824112970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa8100a2ec0 0x7fa8100a5380 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa82410e7b0 0x7fa824112eb0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 --2- 172.21.15.69:0/2330930635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa824109180 0x7fa824112430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 >> 172.21.15.69:0/2330930635 conn(0x7fa82406e000 msgr2=0x7fa82410a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 shutdown_connections 2024-08-02T18:00:29.887 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:29.884+0000 7fa82b57c640 1 -- 172.21.15.69:0/2330930635 wait complete. 2024-08-02T18:00:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 800.0 s, 498.4 tps, lat 198.913 ms stddev 245.360 2024-08-02T18:00:31.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:30 smithi083 ceph-mon[37031]: pgmap v441: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.0 MiB/s wr, 405 op/s 2024-08-02T18:00:31.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:30 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2330930635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:31.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:30 smithi077 ceph-mon[37108]: pgmap v441: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.0 MiB/s wr, 405 op/s 2024-08-02T18:00:31.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:30 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2330930635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:30 smithi069 ceph-mon[32746]: pgmap v441: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 3.0 MiB/s wr, 405 op/s 2024-08-02T18:00:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:30 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2330930635' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:32.066 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T18:00:32.386 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2928495396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 msgr2=0x7f4bd0100b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2928495396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 0x7f4bd0100b60 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009930 tx=0x7f4bc402a420 comp rx=0 tx=0).stop 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2928495396 shutdown_connections 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2928495396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 0x7f4bd0100b60 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2928495396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd01010a0 0x7f4bd010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2928495396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd0106710 0x7f4bd0106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2928495396 >> 172.21.15.69:0/2928495396 conn(0x7f4bd00fc3e0 msgr2=0x7f4bd00fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2928495396 shutdown_connections 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.383+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2928495396 wait complete. 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd71ea640 1 Processor -- start 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd71ea640 1 -- start start 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd71ea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 0x7f4bd0199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.387 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd71ea640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd01010a0 0x7f4bd0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd71ea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0106710 0x7f4bd01a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd71ea640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0110b50 con 0x7f4bd0100700 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd71ea640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01109d0 con 0x7f4bd01010a0 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd71ea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0110cd0 con 0x7f4bd0106710 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd4f86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 0x7f4bd0199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd4f86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 0x7f4bd0199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45362/0 (socket says 172.21.15.69:45362) 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd4f86640 1 -- 172.21.15.69:0/2472969736 learned_addr learned my addr 172.21.15.69:0/2472969736 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd5787640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0106710 0x7f4bd01a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bcffff640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd01010a0 0x7f4bd0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd4f86640 1 -- 172.21.15.69:0/2472969736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0106710 msgr2=0x7f4bd01a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0106710 0x7f4bd01a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd4f86640 1 -- 172.21.15.69:0/2472969736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd01010a0 msgr2=0x7f4bd0199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.388 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd01010a0 0x7f4bd0199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.384+0000 7f4bd4f86640 1 -- 172.21.15.69:0/2472969736 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd01a45c0 con 0x7f4bd0100700 2024-08-02T18:00:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.385+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 0x7f4bd0199000 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00e9f0 tx=0x7f4bbc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.385+0000 7f4bcdffb640 1 -- 172.21.15.69:0/2472969736 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc00cd80 con 0x7f4bd0100700 2024-08-02T18:00:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.385+0000 7f4bcdffb640 1 -- 172.21.15.69:0/2472969736 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc010910 con 0x7f4bd0100700 2024-08-02T18:00:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.385+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2472969736 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd01a4790 con 0x7f4bd0100700 2024-08-02T18:00:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.385+0000 7f4bcdffb640 1 -- 172.21.15.69:0/2472969736 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc018990 con 0x7f4bd0100700 2024-08-02T18:00:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.385+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2472969736 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd01a4d00 con 0x7f4bd0100700 2024-08-02T18:00:32.389 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.387+0000 7f4bcdffb640 1 -- 172.21.15.69:0/2472969736 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4bbc010ac0 con 0x7f4bd0100700 2024-08-02T18:00:32.390 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.388+0000 7f4bcdffb640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4bc00a2d70 0x7f4bc00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.389+0000 7f4bcdffb640 1 -- 172.21.15.69:0/2472969736 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f4bbc0cfa30 con 0x7f4bd0100700 2024-08-02T18:00:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.389+0000 7f4bcffff640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4bc00a2d70 0x7f4bc00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.389+0000 7f4bda016740 1 Processor -- start 2024-08-02T18:00:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.389+0000 7f4bda016740 1 -- start start 2024-08-02T18:00:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.389+0000 7f4bcffff640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4bc00a2d70 0x7f4bc00a5230 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009900 tx=0x7f4bc40058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.389+0000 7f4bda016740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c573fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.392 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.389+0000 7f4bda016740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c57400d0 0x55c2c574a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bda016740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c574cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bda016740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2c56a39b0 con 0x55c2c573f7b0 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bda016740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2c564b800 con 0x55c2c574a970 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd4f86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c573fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bda016740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c2c569f970 con 0x55c2c57400d0 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd4f86640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c573fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45372/0 (socket says 172.21.15.69:45372) 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd4f86640 1 -- 172.21.15.69:0/3478982775 learned_addr learned my addr 172.21.15.69:0/3478982775 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd5787640 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c574cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bcffff640 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c57400d0 0x55c2c574a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd4f86640 1 -- 172.21.15.69:0/3478982775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c57400d0 msgr2=0x55c2c574a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:32.393 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c57400d0 0x55c2c574a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd4f86640 1 -- 172.21.15.69:0/3478982775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 msgr2=0x55c2c574cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c574cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd4f86640 1 -- 172.21.15.69:0/3478982775 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c53b7210 con 0x55c2c573f7b0 2024-08-02T18:00:32.394 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.390+0000 7f4bd5787640 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c574cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:32.397 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.391+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c573fb90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc098050 tx=0x7f4bbc09b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f370f8562dbf96b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.391+0000 7f4bbaffd640 1 -- 172.21.15.69:0/3478982775 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0ccff0 con 0x55c2c573f7b0 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.391+0000 7f4bbaffd640 1 -- 172.21.15.69:0/3478982775 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc098260 con 0x55c2c573f7b0 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.391+0000 7f4bbaffd640 1 -- 172.21.15.69:0/3478982775 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc09b2a0 con 0x55c2c573f7b0 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.392+0000 7f4bda016740 1 -- 172.21.15.69:0/3478982775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 msgr2=0x55c2c573fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.392+0000 7f4bda016740 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c573fb90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc098050 tx=0x7f4bbc09b670 comp rx=0 tx=0).stop 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 -- 172.21.15.69:0/3478982775 shutdown_connections 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c574cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c57400d0 0x55c2c574a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 --2- 172.21.15.69:0/3478982775 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c573fb90 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 -- 172.21.15.69:0/3478982775 >> 172.21.15.69:0/3478982775 conn(0x55c2c573bd20 msgr2=0x55c2c573c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 -- 172.21.15.69:0/3478982775 shutdown_connections 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 -- 172.21.15.69:0/3478982775 wait complete. 2024-08-02T18:00:32.398 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 -- start start 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 Processor -- start 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.393+0000 7f4bda016740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c573f7b0 0x55c2c5747950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bda016740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c5747e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bda016740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 0x55c2c57483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bda016740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55c2c574d540 con 0x55c2c57400d0 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bda016740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55c2c565f500 con 0x55c2c573f7b0 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bda016740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55c2c574d6c0 con 0x55c2c574a970 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bda016740 10 client.0 _ll_register_callbacks cb 0x55c2c53b6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c573f7b0 0x55c2c5747950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd5787640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 0x55c2c57483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c573f7b0 0x55c2c5747950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44374/0 (socket says 172.21.15.69:44374) 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 -- 172.21.15.69:0/699239004 learned_addr learned my addr 172.21.15.69:0/699239004 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bcffff640 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c5747e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.399 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 -- 172.21.15.69:0/699239004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 msgr2=0x55c2c57483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 0x55c2c57483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 -- 172.21.15.69:0/699239004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 msgr2=0x55c2c5747e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c5747e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 -- 172.21.15.69:0/699239004 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c5483620 con 0x55c2c573f7b0 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd5787640 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 0x55c2c57483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bcffff640 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c5747e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c573f7b0 0x55c2c5747950 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc098000 tx=0x7f4bbc018d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=438519562071a0b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.394+0000 7f4baffff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4baffff640 1 -- 172.21.15.69:0/699239004 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc0a8070 con 0x55c2c573f7b0 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 1 -- 172.21.15.69:0/699239004 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bbc098260 con 0x55c2c573f7b0 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4baffff640 1 -- 172.21.15.69:0/699239004 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f4bbc09b250 con 0x55c2c573f7b0 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4baffff640 1 -- 172.21.15.69:0/699239004 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4bbc010430 con 0x55c2c573f7b0 2024-08-02T18:00:32.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 10 client.28331 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 1 -- 172.21.15.69:0/699239004 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2c5759290 con 0x55c2c573f7b0 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4baffff640 1 -- 172.21.15.69:0/699239004 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc098260 con 0x55c2c573f7b0 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4baffff640 1 -- 172.21.15.69:0/699239004 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4bbc0a1080 con 0x55c2c573f7b0 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 10 client.28331 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 10 client.28331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 20 client.28331 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c575a5e0 0x55c2c575c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 4 client.28331 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.395+0000 7f4bda016740 1 -- 172.21.15.69:0/699239004 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2c56b10e0 con 0x55c2c575a5e0 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.396+0000 7f4bcffff640 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c575a5e0 0x55c2c575c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.396+0000 7f4bcffff640 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c575a5e0 0x55c2c575c9d0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f307174fe0fe788c server_cookie=a8be282d95899fee in_seq=0 out_seq=0 2024-08-02T18:00:32.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.396+0000 7f4baffff640 10 client.28331 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4baffff640 1 -- 172.21.15.69:0/699239004 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c2c56b10e0 con 0x55c2c575a5e0 2024-08-02T18:00:32.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4baffff640 10 client.28331 handle_command_reply: tid=0 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 client.28331 shutdown 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 -- 172.21.15.69:0/699239004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c573f7b0 msgr2=0x55c2c5747950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c573f7b0 0x55c2c5747950 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc098000 tx=0x7f4bbc018d60 comp rx=0 tx=0).stop 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 -- 172.21.15.69:0/699239004 shutdown_connections 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c5747e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c575a5e0 0x55c2c575c9d0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 0x55c2c57483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4baffff640 0 client.28331 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 --2- 172.21.15.69:0/699239004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c573f7b0 0x55c2c5747950 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 -- 172.21.15.69:0/699239004 >> 172.21.15.69:0/699239004 conn(0x55c2c573bd20 msgr2=0x55c2c5741a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 -- 172.21.15.69:0/699239004 shutdown_connections 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 1 -- 172.21.15.69:0/699239004 wait complete. 2024-08-02T18:00:32.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.397+0000 7f4bda016740 20 client.28331 trim_cache size 0 max 16384 2024-08-02T18:00:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.432+0000 7f4bda016740 1 Processor -- start 2024-08-02T18:00:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.432+0000 7f4bda016740 1 -- start start 2024-08-02T18:00:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bda016740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c575f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bda016740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c57400d0 0x55c2c575ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bda016740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 0x55c2c5744bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bda016740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01af6f0 con 0x55c2c573f7b0 2024-08-02T18:00:32.436 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bda016740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01acfc0 con 0x55c2c57400d0 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bda016740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01aae10 con 0x55c2c574a970 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bcffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c57400d0 0x55c2c575ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bcffff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c57400d0 0x55c2c575ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44382/0 (socket says 172.21.15.69:44382) 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.433+0000 7f4bcffff640 1 -- 172.21.15.69:0/280735206 learned_addr learned my addr 172.21.15.69:0/280735206 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.434+0000 7f4bcffff640 1 -- 172.21.15.69:0/280735206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 msgr2=0x55c2c5744bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.434+0000 7f4bcffff640 1 --2- 172.21.15.69:0/280735206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 0x55c2c5744bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.434+0000 7f4bcffff640 1 -- 172.21.15.69:0/280735206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 msgr2=0x55c2c575f9f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.434+0000 7f4bcffff640 1 --2- 172.21.15.69:0/280735206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c575f9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.434+0000 7f4bcffff640 1 -- 172.21.15.69:0/280735206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2c5735e80 con 0x55c2c57400d0 2024-08-02T18:00:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.435+0000 7f4bcffff640 1 --2- 172.21.15.69:0/280735206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c57400d0 0x55c2c575ff30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4054fd0 tx=0x7f4bc4040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a09a99f18c94ff server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.438+0000 7f4bba7fc640 1 -- 172.21.15.69:0/280735206 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4035040 con 0x55c2c57400d0 2024-08-02T18:00:32.440 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.438+0000 7f4bba7fc640 1 -- 172.21.15.69:0/280735206 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc4003cd0 con 0x55c2c57400d0 2024-08-02T18:00:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.439+0000 7f4bda016740 1 -- 172.21.15.69:0/280735206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c57400d0 msgr2=0x55c2c575ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.439+0000 7f4bda016740 1 --2- 172.21.15.69:0/280735206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c57400d0 0x55c2c575ff30 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4054fd0 tx=0x7f4bc4040000 comp rx=0 tx=0).stop 2024-08-02T18:00:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.447+0000 7f4bda016740 1 -- 172.21.15.69:0/280735206 shutdown_connections 2024-08-02T18:00:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.447+0000 7f4bda016740 1 --2- 172.21.15.69:0/280735206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c574a970 0x55c2c5744bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.447+0000 7f4bda016740 1 --2- 172.21.15.69:0/280735206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c57400d0 0x55c2c575ff30 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.447+0000 7f4bda016740 1 --2- 172.21.15.69:0/280735206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c573f7b0 0x55c2c575f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.451 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.447+0000 7f4bda016740 1 -- 172.21.15.69:0/280735206 >> 172.21.15.69:0/280735206 conn(0x55c2c573bd20 msgr2=0x55c2c57498c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:32.454 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.450+0000 7f4bda016740 1 -- 172.21.15.69:0/280735206 shutdown_connections 2024-08-02T18:00:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.453+0000 7f4bda016740 1 -- 172.21.15.69:0/280735206 wait complete. 2024-08-02T18:00:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.453+0000 7f4bda016740 1 -- start start 2024-08-02T18:00:32.455 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.453+0000 7f4bda016740 1 Processor -- start 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bda016740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c573f7b0 0x55c2c575d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bda016740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c575dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bda016740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c575e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bda016740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01ab480 con 0x55c2c57400d0 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bda016740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01ac8f0 con 0x55c2c574a970 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bda016740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd01abb50 con 0x55c2c573f7b0 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bda016740 10 client.0 _ll_register_callbacks cb 0x7f4bbc09b550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bd5787640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c575e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bd5787640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c575e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44390/0 (socket says 172.21.15.69:44390) 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bd5787640 1 -- 172.21.15.69:0/2318040589 learned_addr learned my addr 172.21.15.69:0/2318040589 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:32.457 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.454+0000 7f4bcffff640 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c575dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.455+0000 7f4bcffff640 1 -- 172.21.15.69:0/2318040589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c573f7b0 msgr2=0x55c2c575d700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:00:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.455+0000 7f4bcffff640 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c573f7b0 0x55c2c575d700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.455+0000 7f4bcffff640 1 -- 172.21.15.69:0/2318040589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 msgr2=0x55c2c575e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.455+0000 7f4bcffff640 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c575e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.455+0000 7f4bcffff640 1 -- 172.21.15.69:0/2318040589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2c57c3f20 con 0x55c2c57400d0 2024-08-02T18:00:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.455+0000 7f4bcffff640 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c575dc40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005940 tx=0x7f4bc4003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b1a62a1962e296 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:32.458 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.455+0000 7f4baffff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:32.460 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.459+0000 7f4baffff640 1 -- 172.21.15.69:0/2318040589 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4035040 con 0x55c2c57400d0 2024-08-02T18:00:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.459+0000 7f4bda016740 1 -- 172.21.15.69:0/2318040589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bc4003cd0 con 0x55c2c57400d0 2024-08-02T18:00:32.461 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.459+0000 7f4baffff640 1 -- 172.21.15.69:0/2318040589 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f4bc4054530 con 0x55c2c57400d0 2024-08-02T18:00:32.465 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.464+0000 7f4baffff640 1 -- 172.21.15.69:0/2318040589 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4bc408d050 con 0x55c2c57400d0 2024-08-02T18:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.466+0000 7f4bda016740 10 client.18978 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:32.468 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.466+0000 7f4bda016740 1 -- 172.21.15.69:0/2318040589 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c2c57c5170 con 0x55c2c57400d0 2024-08-02T18:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.468+0000 7f4baffff640 1 -- 172.21.15.69:0/2318040589 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bc4003cd0 con 0x55c2c57400d0 2024-08-02T18:00:32.469 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.468+0000 7f4baffff640 1 -- 172.21.15.69:0/2318040589 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4bc407f8e0 con 0x55c2c57400d0 2024-08-02T18:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.471+0000 7f4bda016740 10 client.18978 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.471+0000 7f4bda016740 10 client.18978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.471+0000 7f4bda016740 20 client.18978 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.471+0000 7f4bda016740 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c575a5e0 0x55c2c57c7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.471+0000 7f4bda016740 4 client.18978 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:00:32.472 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.471+0000 7f4bda016740 1 -- 172.21.15.69:0/2318040589 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2c56ab430 con 0x55c2c575a5e0 2024-08-02T18:00:32.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.476+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c575a5e0 0x55c2c57c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.489+0000 7f4baffff640 10 client.18978 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.489+0000 7f4bd4f86640 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c575a5e0 0x55c2c57c7880 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9c6520a2f52e8e1 server_cookie=10abdbb3c31af0c9 in_seq=0 out_seq=0 2024-08-02T18:00:32.673 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.734+0000 7f4baffff640 1 -- 172.21.15.69:0/2318040589 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2c56ab430 con 0x55c2c575a5e0 2024-08-02T18:00:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.734+0000 7f4baffff640 10 client.18978 handle_command_reply: tid=0 2024-08-02T18:00:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.734+0000 7f4bda016740 1 client.18978 shutdown 2024-08-02T18:00:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4bda016740 1 -- 172.21.15.69:0/2318040589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 msgr2=0x55c2c575dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4bda016740 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c575dc40 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4005940 tx=0x7f4bc4003f50 comp rx=0 tx=0).stop 2024-08-02T18:00:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4bda016740 1 -- 172.21.15.69:0/2318040589 shutdown_connections 2024-08-02T18:00:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4bda016740 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55c2c57400d0 0x55c2c575dc40 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4bda016740 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55c2c575a5e0 0x55c2c57c7880 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4bda016740 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55c2c574a970 0x55c2c575e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.737 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4bda016740 1 --2- 172.21.15.69:0/2318040589 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55c2c573f7b0 0x55c2c575d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4bda016740 1 -- 172.21.15.69:0/2318040589 >> 172.21.15.69:0/2318040589 conn(0x55c2c573bd20 msgr2=0x55c2c5741210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.735+0000 7f4baffff640 0 client.18978 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.736+0000 7f4bda016740 1 -- 172.21.15.69:0/2318040589 shutdown_connections 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.736+0000 7f4bda016740 1 -- 172.21.15.69:0/2318040589 wait complete. 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.736+0000 7f4bda016740 20 client.18978 trim_cache size 0 max 16384 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:00:32.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:00:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4bc00a2d70 msgr2=0x7f4bc00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4bc00a2d70 0x7f4bc00a5230 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009900 tx=0x7f4bc40058d0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 msgr2=0x7f4bd0199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:32.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 0x7f4bd0199000 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00e9f0 tx=0x7f4bbc00eeb0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2472969736 shutdown_connections 2024-08-02T18:00:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f4bc00a2d70 0x7f4bc00a5230 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bd0106710 0x7f4bd01a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4bd01010a0 0x7f4bd0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 --2- 172.21.15.69:0/2472969736 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4bd0100700 0x7f4bd0199000 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2472969736 >> 172.21.15.69:0/2472969736 conn(0x7f4bd00fc3e0 msgr2=0x7f4bd00fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2472969736 shutdown_connections 2024-08-02T18:00:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:32.737+0000 7f4bd71ea640 1 -- 172.21.15.69:0/2472969736 wait complete. 2024-08-02T18:00:33.004 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:32 smithi069 ceph-mon[32746]: pgmap v442: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-08-02T18:00:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:32 smithi083 ceph-mon[37031]: pgmap v442: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-08-02T18:00:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:32 smithi077 ceph-mon[37108]: pgmap v442: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.6 MiB/s wr, 349 op/s 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.308+0000 7f98b8969640 1 -- 172.21.15.69:0/2327900383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98b4109180 msgr2=0x7f98b4109560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.308+0000 7f98b8969640 1 --2- 172.21.15.69:0/2327900383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98b4109180 0x7f98b4109560 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f98a400b880 tx=0x7f98a4005f90 comp rx=0 tx=0).stop 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.309+0000 7f98b8969640 1 -- 172.21.15.69:0/2327900383 shutdown_connections 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.309+0000 7f98b8969640 1 --2- 172.21.15.69:0/2327900383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98b4109b30 0x7f98b4074000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.309+0000 7f98b8969640 1 --2- 172.21.15.69:0/2327900383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40746c0 0x7f98b407b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.309+0000 7f98b8969640 1 --2- 172.21.15.69:0/2327900383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98b4109180 0x7f98b4109560 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.309+0000 7f98b8969640 1 -- 172.21.15.69:0/2327900383 >> 172.21.15.69:0/2327900383 conn(0x7f98b406de70 msgr2=0x7f98b406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.309+0000 7f98b8969640 1 -- 172.21.15.69:0/2327900383 shutdown_connections 2024-08-02T18:00:33.313 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.309+0000 7f98b8969640 1 -- 172.21.15.69:0/2327900383 wait complete. 2024-08-02T18:00:33.315 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b8969640 1 Processor -- start 2024-08-02T18:00:33.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b8969640 1 -- start start 2024-08-02T18:00:33.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b8969640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40746c0 0x7f98b407abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.326 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b8969640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98b4109b30 0x7f98b4075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b8969640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98b40761a0 0x7f98b4076650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b8969640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f98b407e4b0 con 0x7f98b4109b30 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b8969640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98b407e330 con 0x7f98b40746c0 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b8969640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98b407e630 con 0x7f98b40761a0 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40746c0 0x7f98b407abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40746c0 0x7f98b407abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44420/0 (socket says 172.21.15.69:44420) 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b259c640 1 -- 172.21.15.69:0/3533510388 learned_addr learned my addr 172.21.15.69:0/3533510388 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.310+0000 7f98b2d9d640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98b40761a0 0x7f98b4076650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.327 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.311+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98b4109b30 0x7f98b4075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98b259c640 1 -- 172.21.15.69:0/3533510388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98b40761a0 msgr2=0x7f98b4076650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98b259c640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98b40761a0 0x7f98b4076650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98b259c640 1 -- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98b4109b30 msgr2=0x7f98b4075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98b259c640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98b4109b30 0x7f98b4075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98b259c640 1 -- 172.21.15.69:0/3533510388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b4136fc0 con 0x7f98b40746c0 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98b259c640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40746c0 0x7f98b407abd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f98a4034040 tx=0x7f98a402bdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98a37fe640 1 -- 172.21.15.69:0/3533510388 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4041020 con 0x7f98b40746c0 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98b8969640 1 -- 172.21.15.69:0/3533510388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b4137250 con 0x7f98b40746c0 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.313+0000 7f98b8969640 1 -- 172.21.15.69:0/3533510388 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b4137790 con 0x7f98b40746c0 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.317+0000 7f98bad67740 1 Processor -- start 2024-08-02T18:00:33.328 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.317+0000 7f98bad67740 1 -- start start 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.317+0000 7f98bad67740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d601223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.317+0000 7f98bad67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 0x562d6012cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.317+0000 7f98bad67740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d6012d1d0 0x562d6012f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.317+0000 7f98bad67740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562d60087a40 con 0x562d60121ff0 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.317+0000 7f98bad67740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d6001f9d0 con 0x562d6012d1d0 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.317+0000 7f98bad67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562d60083a00 con 0x562d60122910 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d6012d1d0 0x562d6012f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d6012d1d0 0x562d6012f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44422/0 (socket says 172.21.15.69:44422) 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b1d9b640 1 -- 172.21.15.69:0/166557965 learned_addr learned my addr 172.21.15.69:0/166557965 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b259c640 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 0x562d6012cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b2d9d640 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d601223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b1d9b640 1 -- 172.21.15.69:0/166557965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 msgr2=0x562d6012cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:33.329 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 0x562d6012cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b1d9b640 1 -- 172.21.15.69:0/166557965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 msgr2=0x562d601223d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d601223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.318+0000 7f98b1d9b640 1 -- 172.21.15.69:0/166557965 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d60026650 con 0x562d6012d1d0 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.320+0000 7f98a37fe640 1 -- 172.21.15.69:0/3533510388 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a40035f0 con 0x7f98b40746c0 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.320+0000 7f98a37fe640 1 -- 172.21.15.69:0/3533510388 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a403c6b0 con 0x7f98b40746c0 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.320+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d6012d1d0 0x562d6012f5c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00afd0 tx=0x7f98ac0085b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=327d07f913397010 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.321+0000 7f98a37fe640 1 -- 172.21.15.69:0/3533510388 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f98a4044020 con 0x7f98b40746c0 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.321+0000 7f98a0ff9640 1 -- 172.21.15.69:0/166557965 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac05c020 con 0x562d6012d1d0 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.321+0000 7f98a0ff9640 1 -- 172.21.15.69:0/166557965 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac004520 con 0x562d6012d1d0 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.321+0000 7f98a0ff9640 1 -- 172.21.15.69:0/166557965 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac05e9c0 con 0x562d6012d1d0 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.321+0000 7f98a37fe640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f989c0a2f30 0x7f989c0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.330 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.323+0000 7f98bad67740 1 -- 172.21.15.69:0/166557965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d6012d1d0 msgr2=0x562d6012f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.323+0000 7f98bad67740 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d6012d1d0 0x562d6012f5c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00afd0 tx=0x7f98ac0085b0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.323+0000 7f98a37fe640 1 -- 172.21.15.69:0/3533510388 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f98a40f1110 con 0x7f98b40746c0 2024-08-02T18:00:33.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98b2d9d640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f989c0a2f30 0x7f989c0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 -- 172.21.15.69:0/166557965 shutdown_connections 2024-08-02T18:00:33.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d6012d1d0 0x562d6012f5c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 0x562d6012cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.331 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 --2- 172.21.15.69:0/166557965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d601223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 -- 172.21.15.69:0/166557965 >> 172.21.15.69:0/166557965 conn(0x562d6011e580 msgr2=0x562d6011e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 -- 172.21.15.69:0/166557965 shutdown_connections 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 -- 172.21.15.69:0/166557965 wait complete. 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 -- start start 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.326+0000 7f98bad67740 1 Processor -- start 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98bad67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60121ff0 0x562d6012cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98bad67740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60129fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98bad67740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 0x562d6012a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98b259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60121ff0 0x562d6012cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98b259c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60121ff0 0x562d6012cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:53464/0 (socket says 172.21.15.69:53464) 2024-08-02T18:00:33.332 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60129fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60129fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44436/0 (socket says 172.21.15.69:44436) 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98bad67740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562d6012fda0 con 0x562d6012d1d0 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98bad67740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562d60042b20 con 0x562d60122910 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98bad67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562d6012ff20 con 0x562d60121ff0 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98bad67740 10 client.0 _ll_register_callbacks cb 0x562d60040bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.327+0000 7f98b259c640 1 -- 172.21.15.69:0/2616078730 learned_addr learned my addr 172.21.15.69:0/2616078730 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.328+0000 7f98b1d9b640 1 -- 172.21.15.69:0/2616078730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60121ff0 msgr2=0x562d6012cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.328+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60121ff0 0x562d6012cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.328+0000 7f98b1d9b640 1 -- 172.21.15.69:0/2616078730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 msgr2=0x562d6012a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.328+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 0x562d6012a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.333 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.328+0000 7f98b1d9b640 1 -- 172.21.15.69:0/2616078730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d5fdf93c0 con 0x562d60122910 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60129fe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f98ac05e300 tx=0x7f98ac005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d632d337ece343f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f9899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f98b2d9d640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f989c0a2f30 0x7f989c0a53f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f98a800dd10 tx=0x7f98a800d360 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f9899ffb640 1 -- 172.21.15.69:0/2616078730 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac05c050 con 0x562d60122910 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f98bad67740 1 -- 172.21.15.69:0/2616078730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98ac004520 con 0x562d60122910 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f9899ffb640 1 -- 172.21.15.69:0/2616078730 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f98ac004200 con 0x562d60122910 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f9899ffb640 1 -- 172.21.15.69:0/2616078730 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98ac072050 con 0x562d60122910 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f98bad67740 10 client.28349 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.332+0000 7f98bad67740 1 -- 172.21.15.69:0/2616078730 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d6013b9f0 con 0x562d60122910 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f9899ffb640 1 -- 172.21.15.69:0/2616078730 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac004520 con 0x562d60122910 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f9899ffb640 1 -- 172.21.15.69:0/2616078730 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f98ac05eb10 con 0x562d60122910 2024-08-02T18:00:33.336 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f98bad67740 10 client.28349 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f98bad67740 10 client.28349 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:00:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f98bad67740 20 client.28349 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f98bad67740 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x562d6013d250 0x562d6013f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f98bad67740 4 client.28349 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f98bad67740 1 -- 172.21.15.69:0/2616078730 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d60094fb0 con 0x562d6013d250 2024-08-02T18:00:33.337 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.333+0000 7f98b259c640 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x562d6013d250 0x562d6013f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.334+0000 7f9899ffb640 10 client.28349 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.335+0000 7f98b259c640 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x562d6013d250 0x562d6013f690 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c364aea4007bc052 server_cookie=7ce60eb3d4082464 in_seq=0 out_seq=0 2024-08-02T18:00:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.336+0000 7f9899ffb640 1 -- 172.21.15.69:0/2616078730 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562d60094fb0 con 0x562d6013d250 2024-08-02T18:00:33.338 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.336+0000 7f9899ffb640 10 client.28349 handle_command_reply: tid=0 2024-08-02T18:00:33.339 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.336+0000 7f98bad67740 1 client.28349 shutdown 2024-08-02T18:00:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.339+0000 7f98bad67740 1 -- 172.21.15.69:0/2616078730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 msgr2=0x562d60129fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.339+0000 7f98bad67740 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60129fe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f98ac05e300 tx=0x7f98ac005df0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 1 -- 172.21.15.69:0/2616078730 shutdown_connections 2024-08-02T18:00:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x562d6013d250 0x562d6013f690 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.345 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 0x562d6012a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60129fe0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 1 --2- 172.21.15.69:0/2616078730 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60121ff0 0x562d6012cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 1 -- 172.21.15.69:0/2616078730 >> 172.21.15.69:0/2616078730 conn(0x562d6011e580 msgr2=0x562d601242b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f9899ffb640 0 client.28349 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 1 -- 172.21.15.69:0/2616078730 shutdown_connections 2024-08-02T18:00:33.346 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 1 -- 172.21.15.69:0/2616078730 wait complete. 2024-08-02T18:00:33.347 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.340+0000 7f98bad67740 20 client.28349 trim_cache size 0 max 16384 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.359+0000 7f98bad67740 1 Processor -- start 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.359+0000 7f98bad67740 1 -- start start 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.359+0000 7f98bad67740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60121ff0 0x562d60125370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98bad67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 0x562d601258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98bad67740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 0x562d601288f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98bad67740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9894012a00 con 0x562d6012d1d0 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98bad67740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f98940102d0 con 0x562d60121ff0 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98bad67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f989400e120 con 0x562d60122910 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60121ff0 0x562d60125370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.362 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60121ff0 0x562d60125370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44450/0 (socket says 172.21.15.69:44450) 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 -- 172.21.15.69:0/2915348203 learned_addr learned my addr 172.21.15.69:0/2915348203 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b2d9d640 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 0x562d601288f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 0x562d601258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 -- 172.21.15.69:0/2915348203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 msgr2=0x562d601258b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 0x562d601258b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 -- 172.21.15.69:0/2915348203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 msgr2=0x562d601288f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 0x562d601288f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 -- 172.21.15.69:0/2915348203 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6012aae0 con 0x562d60121ff0 2024-08-02T18:00:33.363 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f98b259c640 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60121ff0 0x562d60125370 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f98a40bd2b0 tx=0x7f98a40f0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e222dae7341b35de server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.360+0000 7f989bfff640 1 -- 172.21.15.69:0/2915348203 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40c1720 con 0x562d60121ff0 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f989bfff640 1 -- 172.21.15.69:0/2915348203 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98a40cf070 con 0x562d60121ff0 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f989bfff640 1 -- 172.21.15.69:0/2915348203 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40bd5d0 con 0x562d60121ff0 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 -- 172.21.15.69:0/2915348203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60121ff0 msgr2=0x562d60125370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60121ff0 0x562d60125370 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f98a40bd2b0 tx=0x7f98a40f0040 comp rx=0 tx=0).stop 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 -- 172.21.15.69:0/2915348203 shutdown_connections 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d6012d1d0 0x562d601288f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d60122910 0x562d601258b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 --2- 172.21.15.69:0/2915348203 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60121ff0 0x562d60125370 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 -- 172.21.15.69:0/2915348203 >> 172.21.15.69:0/2915348203 conn(0x562d6011e580 msgr2=0x562d601191b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:33.365 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 -- 172.21.15.69:0/2915348203 shutdown_connections 2024-08-02T18:00:33.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.361+0000 7f98bad67740 1 -- 172.21.15.69:0/2915348203 wait complete. 2024-08-02T18:00:33.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.362+0000 7f98bad67740 1 -- start start 2024-08-02T18:00:33.366 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.362+0000 7f98bad67740 1 Processor -- start 2024-08-02T18:00:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98bad67740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d60135200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98bad67740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60135740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98bad67740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d6012d1d0 0x562d60126290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98bad67740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f989400e790 con 0x562d60121ff0 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98bad67740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f989400fc00 con 0x562d60122910 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98bad67740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f989400ee60 con 0x562d6012d1d0 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98bad67740 10 client.0 _ll_register_callbacks cb 0x7f98a800d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d60135200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60135740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d60135200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45432/0 (socket says 172.21.15.69:45432) 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b1d9b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60135740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:44466/0 (socket says 172.21.15.69:44466) 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b259c640 1 -- 172.21.15.69:0/3613083074 learned_addr learned my addr 172.21.15.69:0/3613083074 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b2d9d640 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d6012d1d0 0x562d60126290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.372 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b1d9b640 1 -- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d6012d1d0 msgr2=0x562d60126290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d6012d1d0 0x562d60126290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b1d9b640 1 -- 172.21.15.69:0/3613083074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 msgr2=0x562d60135200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d60135200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b1d9b640 1 -- 172.21.15.69:0/3613083074 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d60126850 con 0x562d60122910 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b2d9d640 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d6012d1d0 0x562d60126290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.365+0000 7f98b1d9b640 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60135740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f98ac05e330 tx=0x7f98ac0039e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4116d883819ecfa server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f9899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f9899ffb640 1 -- 172.21.15.69:0/3613083074 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac05c070 con 0x562d60122910 2024-08-02T18:00:33.373 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f9899ffb640 1 -- 172.21.15.69:0/3613083074 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f98ac005bf0 con 0x562d60122910 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f98bad67740 1 -- 172.21.15.69:0/3613083074 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a40cf070 con 0x562d60122910 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f9899ffb640 1 -- 172.21.15.69:0/3613083074 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f98ac05e360 con 0x562d60122910 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f98bad67740 10 client.28355 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f98bad67740 1 -- 172.21.15.69:0/3613083074 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562d60127b50 con 0x562d60122910 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f9899ffb640 1 -- 172.21.15.69:0/3613083074 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98ac00c070 con 0x562d60122910 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f9899ffb640 1 -- 172.21.15.69:0/3613083074 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f98ac004120 con 0x562d60122910 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.366+0000 7f98bad67740 10 client.28355 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f98bad67740 10 client.28355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f98bad67740 20 client.28355 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f98bad67740 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x562d601a6f80 0x562d601a9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:33.374 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f98bad67740 4 client.28355 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f98bad67740 1 -- 172.21.15.69:0/3613083074 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f98a40cf070 con 0x562d601a6f80 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f98b259c640 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x562d601a6f80 0x562d601a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f9899ffb640 10 client.28355 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f98b259c640 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x562d601a6f80 0x562d601a9370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cab729c8d3570de1 server_cookie=94b65a064cc9bd77 in_seq=0 out_seq=0 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f9899ffb640 1 -- 172.21.15.69:0/3613083074 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f98a40cf070 con 0x562d601a6f80 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f9899ffb640 10 client.28355 handle_command_reply: tid=0 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.367+0000 7f98bad67740 1 client.28355 shutdown 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 -- 172.21.15.69:0/3613083074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 msgr2=0x562d60135740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60135740 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f98ac05e330 tx=0x7f98ac0039e0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 -- 172.21.15.69:0/3613083074 shutdown_connections 2024-08-02T18:00:33.375 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x562d601a6f80 0x562d601a9370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562d6012d1d0 0x562d60126290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f9899ffb640 0 client.28355 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562d60122910 0x562d60135740 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 --2- 172.21.15.69:0/3613083074 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562d60121ff0 0x562d60135200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 -- 172.21.15.69:0/3613083074 >> 172.21.15.69:0/3613083074 conn(0x562d6011e580 msgr2=0x562d60125370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 -- 172.21.15.69:0/3613083074 shutdown_connections 2024-08-02T18:00:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 1 -- 172.21.15.69:0/3613083074 wait complete. 2024-08-02T18:00:33.376 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.368+0000 7f98bad67740 20 client.28355 trim_cache size 0 max 16384 2024-08-02T18:00:33.377 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 -- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f989c0a2f30 msgr2=0x7f989c0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f989c0a2f30 0x7f989c0a53f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f98a800dd10 tx=0x7f98a800d360 comp rx=0 tx=0).stop 2024-08-02T18:00:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 -- 172.21.15.69:0/3533510388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40746c0 msgr2=0x7f98b407abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40746c0 0x7f98b407abd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f98a4034040 tx=0x7f98a402bdd0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 -- 172.21.15.69:0/3533510388 shutdown_connections 2024-08-02T18:00:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f98b4109b30 0x7f98b4075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f989c0a2f30 0x7f989c0a53f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.378 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98b40761a0 0x7f98b4076650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 --2- 172.21.15.69:0/3533510388 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f98b40746c0 0x7f98b407abd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:33.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 -- 172.21.15.69:0/3533510388 >> 172.21.15.69:0/3533510388 conn(0x7f98b406de70 msgr2=0x7f98b4071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:33.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 -- 172.21.15.69:0/3533510388 shutdown_connections 2024-08-02T18:00:33.379 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:33.369+0000 7f98b8969640 1 -- 172.21.15.69:0/3533510388 wait complete. 2024-08-02T18:00:34.887 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T18:00:34.929 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:34 smithi069 ceph-mon[32746]: pgmap v443: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-08-02T18:00:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 805.0 s, 506.8 tps, lat 194.634 ms stddev 233.667 2024-08-02T18:00:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:34 smithi083 ceph-mon[37031]: pgmap v443: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-08-02T18:00:35.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:34 smithi077 ceph-mon[37108]: pgmap v443: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 4.3 MiB/s wr, 502 op/s 2024-08-02T18:00:35.353 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:35.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.973+0000 7f7456f2d640 1 -- 172.21.15.69:0/3530318907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7450100fc0 msgr2=0x7f745010d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:35.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.973+0000 7f7456f2d640 1 --2- 172.21.15.69:0/3530318907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7450100fc0 0x7f745010d4d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7448009cd0 tx=0x7f744802a910 comp rx=0 tx=0).stop 2024-08-02T18:00:35.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.973+0000 7f7456f2d640 1 -- 172.21.15.69:0/3530318907 shutdown_connections 2024-08-02T18:00:35.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.973+0000 7f7456f2d640 1 --2- 172.21.15.69:0/3530318907 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7450100fc0 0x7f745010d4d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:35.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.973+0000 7f7456f2d640 1 --2- 172.21.15.69:0/3530318907 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7450100600 0x7f7450100a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:35.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.973+0000 7f7456f2d640 1 --2- 172.21.15.69:0/3530318907 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7450106690 0x7f7450106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:35.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.973+0000 7f7456f2d640 1 -- 172.21.15.69:0/3530318907 >> 172.21.15.69:0/3530318907 conn(0x7f74500fc340 msgr2=0x7f74500fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:35.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.973+0000 7f7456f2d640 1 -- 172.21.15.69:0/3530318907 shutdown_connections 2024-08-02T18:00:35.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.975+0000 7f7456f2d640 1 -- 172.21.15.69:0/3530318907 wait complete. 2024-08-02T18:00:35.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.976+0000 7f7456f2d640 1 Processor -- start 2024-08-02T18:00:35.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.976+0000 7f7456f2d640 1 -- start start 2024-08-02T18:00:35.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f7456f2d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7450100600 0x7f745019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f7456f2d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7450100fc0 0x7f745019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f7456f2d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7450106690 0x7f7450195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f7456f2d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f745010f930 con 0x7f7450100600 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f7456f2d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f745010f7b0 con 0x7f7450106690 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f7456f2d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f745010fab0 con 0x7f7450100fc0 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f745672c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7450106690 0x7f7450195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f745672c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7450106690 0x7f7450195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48396/0 (socket says 172.21.15.69:48396) 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.977+0000 7f745672c640 1 -- 172.21.15.69:0/1339051200 learned_addr learned my addr 172.21.15.69:0/1339051200 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:35.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.978+0000 7f745572a640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7450100fc0 0x7f745019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:35.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.978+0000 7f745672c640 1 -- 172.21.15.69:0/1339051200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7450100fc0 msgr2=0x7f745019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:35.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.978+0000 7f745672c640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7450100fc0 0x7f745019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:35.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.978+0000 7f745672c640 1 -- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7450100600 msgr2=0x7f745019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:35.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.978+0000 7f745672c640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7450100600 0x7f745019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:35.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.978+0000 7f745672c640 1 -- 172.21.15.69:0/1339051200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74501961b0 con 0x7f7450106690 2024-08-02T18:00:35.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.983+0000 7f745672c640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7450106690 0x7f7450195a80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f744802ae30 tx=0x7f7448003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:35.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.984+0000 7f7446ffd640 1 -- 172.21.15.69:0/1339051200 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448040a20 con 0x7f7450106690 2024-08-02T18:00:35.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.984+0000 7f7446ffd640 1 -- 172.21.15.69:0/1339051200 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7448003e30 con 0x7f7450106690 2024-08-02T18:00:35.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.985+0000 7f7446ffd640 1 -- 172.21.15.69:0/1339051200 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448030900 con 0x7f7450106690 2024-08-02T18:00:35.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.986+0000 7f7456f2d640 1 -- 172.21.15.69:0/1339051200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7450196440 con 0x7f7450106690 2024-08-02T18:00:35.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.986+0000 7f7456f2d640 1 -- 172.21.15.69:0/1339051200 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a2790 con 0x7f7450106690 2024-08-02T18:00:35.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.989+0000 7f745a58d740 1 Processor -- start 2024-08-02T18:00:35.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.989+0000 7f745a58d740 1 -- start start 2024-08-02T18:00:35.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745a58d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b215f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:35.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745a58d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215fa10 0x55a1b2172680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:35.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745a58d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b2172bc0 0x55a1b2174fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:35.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745a58d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a1b20c4a40 con 0x55a1b215fa10 2024-08-02T18:00:35.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745a58d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1b205c9d0 con 0x55a1b2172bc0 2024-08-02T18:00:35.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745a58d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1b20c0a00 con 0x55a1b215f0f0 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745572a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b215f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745572a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b215f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:49816/0 (socket says 172.21.15.69:49816) 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745572a640 1 -- 172.21.15.69:0/456910477 learned_addr learned my addr 172.21.15.69:0/456910477 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.990+0000 7f745572a640 1 --2- 172.21.15.69:0/456910477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b2172bc0 0x55a1b2174fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.991+0000 7f745572a640 1 -- 172.21.15.69:0/456910477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b2172bc0 msgr2=0x55a1b2174fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.991+0000 7f745572a640 1 --2- 172.21.15.69:0/456910477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b2172bc0 0x55a1b2174fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.991+0000 7f745572a640 1 -- 172.21.15.69:0/456910477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215fa10 msgr2=0x55a1b2172680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.991+0000 7f745572a640 1 --2- 172.21.15.69:0/456910477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215fa10 0x55a1b2172680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:35.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.991+0000 7f745572a640 1 -- 172.21.15.69:0/456910477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b2063650 con 0x55a1b215f0f0 2024-08-02T18:00:35.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.993+0000 7f7446ffd640 1 -- 172.21.15.69:0/1339051200 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7448040bc0 con 0x7f7450106690 2024-08-02T18:00:35.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.993+0000 7f745572a640 1 --2- 172.21.15.69:0/456910477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b215f4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7440011700 tx=0x7f744000a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75700a3b47a12868 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:35.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.994+0000 7f7446ffd640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f743c0a2d20 0x7f743c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:35.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.994+0000 7f7446ffd640 1 -- 172.21.15.69:0/1339051200 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f74480f0e10 con 0x7f7450106690 2024-08-02T18:00:35.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.997+0000 7f7455f2b640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f743c0a2d20 0x7f743c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.998+0000 7f743bfff640 1 -- 172.21.15.69:0/456910477 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440004020 con 0x55a1b215f0f0 2024-08-02T18:00:36.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.998+0000 7f743bfff640 1 -- 172.21.15.69:0/456910477 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7440005500 con 0x55a1b215f0f0 2024-08-02T18:00:36.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.999+0000 7f745a58d740 1 -- 172.21.15.69:0/456910477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 msgr2=0x55a1b215f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:36.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.999+0000 7f745a58d740 1 --2- 172.21.15.69:0/456910477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b215f4d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7440011700 tx=0x7f744000a680 comp rx=0 tx=0).stop 2024-08-02T18:00:36.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.999+0000 7f745a58d740 1 -- 172.21.15.69:0/456910477 shutdown_connections 2024-08-02T18:00:36.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.999+0000 7f745a58d740 1 --2- 172.21.15.69:0/456910477 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215fa10 0x55a1b2172680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.999+0000 7f745a58d740 1 --2- 172.21.15.69:0/456910477 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b2172bc0 0x55a1b2174fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.999+0000 7f745a58d740 1 --2- 172.21.15.69:0/456910477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b215f4d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:35.999+0000 7f745a58d740 1 -- 172.21.15.69:0/456910477 >> 172.21.15.69:0/456910477 conn(0x55a1b215b760 msgr2=0x55a1b215bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:36.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.007+0000 7f7455f2b640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f743c0a2d20 0x7f743c0a51e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7450196bc0 tx=0x7f744c005c70 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:36.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.012+0000 7f745a58d740 1 -- 172.21.15.69:0/456910477 shutdown_connections 2024-08-02T18:00:36.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.012+0000 7f745a58d740 1 -- 172.21.15.69:0/456910477 wait complete. 2024-08-02T18:00:36.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.013+0000 7f745a58d740 1 -- start start 2024-08-02T18:00:36.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.013+0000 7f745a58d740 1 Processor -- start 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.013+0000 7f745a58d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215f0f0 0x55a1b216fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.013+0000 7f745a58d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b216ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745a58d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b2172bc0 0x55a1b21704b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745a58d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55a1b21757d0 con 0x55a1b215f0f0 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745572a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b216ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745572a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b216ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48424/0 (socket says 172.21.15.69:48424) 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745572a640 1 -- 172.21.15.69:0/2979469112 learned_addr learned my addr 172.21.15.69:0/2979469112 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745672c640 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b2172bc0 0x55a1b21704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55a1b207fb20 con 0x55a1b215fa10 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55a1b2175950 con 0x55a1b2172bc0 2024-08-02T18:00:36.016 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.014+0000 7f745a58d740 10 client.0 _ll_register_callbacks cb 0x55a1b207dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:36.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.015+0000 7f7455f2b640 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215f0f0 0x55a1b216fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.015+0000 7f745572a640 1 -- 172.21.15.69:0/2979469112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b2172bc0 msgr2=0x55a1b21704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:36.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.015+0000 7f745572a640 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b2172bc0 0x55a1b21704b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.015+0000 7f745572a640 1 -- 172.21.15.69:0/2979469112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215f0f0 msgr2=0x55a1b216fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:36.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.015+0000 7f745572a640 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215f0f0 0x55a1b216fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.015+0000 7f745572a640 1 -- 172.21.15.69:0/2979469112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b1e363c0 con 0x55a1b215fa10 2024-08-02T18:00:36.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.016+0000 7f745572a640 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b216ff70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7440004890 tx=0x7f744000ae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c76b98a2343861df server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:36.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.018+0000 7f74397fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:36.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.018+0000 7f74397fa640 1 -- 172.21.15.69:0/2979469112 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744000f9a0 con 0x55a1b215fa10 2024-08-02T18:00:36.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.018+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7440005500 con 0x55a1b215fa10 2024-08-02T18:00:36.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.019+0000 7f74397fa640 1 -- 172.21.15.69:0/2979469112 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f744000fb40 con 0x55a1b215fa10 2024-08-02T18:00:36.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.019+0000 7f74397fa640 1 -- 172.21.15.69:0/2979469112 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f744000fd40 con 0x55a1b215fa10 2024-08-02T18:00:36.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.023+0000 7f745a58d740 10 client.28364 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:36.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.023+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a1b21813f0 con 0x55a1b215fa10 2024-08-02T18:00:36.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.026+0000 7f74397fa640 1 -- 172.21.15.69:0/2979469112 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7440005500 con 0x55a1b215fa10 2024-08-02T18:00:36.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.026+0000 7f74397fa640 1 -- 172.21.15.69:0/2979469112 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7440026070 con 0x55a1b215fa10 2024-08-02T18:00:36.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.027+0000 7f745a58d740 10 client.28364 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:36.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.027+0000 7f745a58d740 10 client.28364 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:00:36.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.027+0000 7f745a58d740 20 client.28364 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:36.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.027+0000 7f745a58d740 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a1b2182940 0x55a1b2184d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.027+0000 7f745a58d740 4 client.28364 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:36.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.027+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b20d1fb0 con 0x55a1b2182940 2024-08-02T18:00:36.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.027+0000 7f7455f2b640 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a1b2182940 0x55a1b2184d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.028+0000 7f7455f2b640 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a1b2182940 0x55a1b2184d70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e343cf5de4ba84c7 server_cookie=86cf7e5525a25965 in_seq=0 out_seq=0 2024-08-02T18:00:36.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.028+0000 7f74397fa640 10 client.28364 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:36.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.031+0000 7f74397fa640 1 -- 172.21.15.69:0/2979469112 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a1b20d1fb0 con 0x55a1b2182940 2024-08-02T18:00:36.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.031+0000 7f74397fa640 10 client.28364 handle_command_reply: tid=0 2024-08-02T18:00:36.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.031+0000 7f745a58d740 1 client.28364 shutdown 2024-08-02T18:00:36.037 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 msgr2=0x55a1b216ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f745a58d740 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b216ff70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7440004890 tx=0x7f744000ae60 comp rx=0 tx=0).stop 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 shutdown_connections 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f745a58d740 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a1b2182940 0x55a1b2184d70 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f745a58d740 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b2172bc0 0x55a1b21704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f745a58d740 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b216ff70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f745a58d740 1 --2- 172.21.15.69:0/2979469112 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b215f0f0 0x55a1b216fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 >> 172.21.15.69:0/2979469112 conn(0x55a1b215b760 msgr2=0x55a1b21613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.035+0000 7f74397fa640 0 client.28364 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.036+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 shutdown_connections 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.036+0000 7f745a58d740 1 -- 172.21.15.69:0/2979469112 wait complete. 2024-08-02T18:00:36.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.036+0000 7f745a58d740 20 client.28364 trim_cache size 0 max 16384 2024-08-02T18:00:36.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.061+0000 7f745a58d740 1 Processor -- start 2024-08-02T18:00:36.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.061+0000 7f745a58d740 1 -- start start 2024-08-02T18:00:36.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.061+0000 7f745a58d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b216ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f745a58d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b218a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f7455f2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b216ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f7455f2b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b216ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:49828/0 (socket says 172.21.15.69:49828) 2024-08-02T18:00:36.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f745572a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b218a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f745572a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b218a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48434/0 (socket says 172.21.15.69:48434) 2024-08-02T18:00:36.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f7455f2b640 1 -- 172.21.15.69:0/3206173992 learned_addr learned my addr 172.21.15.69:0/3206173992 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:36.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f745a58d740 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 0x55a1b218abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f745a58d740 1 -- 172.21.15.69:0/3206173992 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74501af150 con 0x55a1b2172bc0 2024-08-02T18:00:36.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f745a58d740 1 -- 172.21.15.69:0/3206173992 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74501aca20 con 0x55a1b215fa10 2024-08-02T18:00:36.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.062+0000 7f745a58d740 1 -- 172.21.15.69:0/3206173992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74501aa870 con 0x55a1b215f0f0 2024-08-02T18:00:36.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.063+0000 7f745672c640 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 0x55a1b218abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.063+0000 7f7455f2b640 1 -- 172.21.15.69:0/3206173992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 msgr2=0x55a1b218a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:36.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.063+0000 7f7455f2b640 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b218a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.063+0000 7f7455f2b640 1 -- 172.21.15.69:0/3206173992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 msgr2=0x55a1b218abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:36.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.063+0000 7f7455f2b640 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 0x55a1b218abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.063+0000 7f7455f2b640 1 -- 172.21.15.69:0/3206173992 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b2159ce0 con 0x55a1b215f0f0 2024-08-02T18:00:36.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.064+0000 7f7455f2b640 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b216ea10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f744c00cd90 tx=0x7f744c00f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f69779eccb3b54e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:36.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.067+0000 7f743b7fe640 1 -- 172.21.15.69:0/3206173992 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c00f650 con 0x55a1b215f0f0 2024-08-02T18:00:36.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.067+0000 7f743b7fe640 1 -- 172.21.15.69:0/3206173992 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f744c028030 con 0x55a1b215f0f0 2024-08-02T18:00:36.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.067+0000 7f743b7fe640 1 -- 172.21.15.69:0/3206173992 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c02ca00 con 0x55a1b215f0f0 2024-08-02T18:00:36.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.068+0000 7f745a58d740 1 -- 172.21.15.69:0/3206173992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 msgr2=0x55a1b216ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:36.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.068+0000 7f745a58d740 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b216ea10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f744c00cd90 tx=0x7f744c00f000 comp rx=0 tx=0).stop 2024-08-02T18:00:36.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.072+0000 7f745a58d740 1 -- 172.21.15.69:0/3206173992 shutdown_connections 2024-08-02T18:00:36.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.072+0000 7f745a58d740 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 0x55a1b218abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.072+0000 7f745a58d740 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215fa10 0x55a1b218a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.072+0000 7f745a58d740 1 --2- 172.21.15.69:0/3206173992 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215f0f0 0x55a1b216ea10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.072+0000 7f745a58d740 1 -- 172.21.15.69:0/3206173992 >> 172.21.15.69:0/3206173992 conn(0x55a1b215b760 msgr2=0x55a1b216e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:36.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.073+0000 7f745a58d740 1 -- 172.21.15.69:0/3206173992 shutdown_connections 2024-08-02T18:00:36.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.073+0000 7f745a58d740 1 -- 172.21.15.69:0/3206173992 wait complete. 2024-08-02T18:00:36.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.074+0000 7f745a58d740 1 -- start start 2024-08-02T18:00:36.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.074+0000 7f745a58d740 1 Processor -- start 2024-08-02T18:00:36.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.075+0000 7f745a58d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215f0f0 0x55a1b217b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.075+0000 7f745a58d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215fa10 0x55a1b2175bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.075+0000 7f7455f2b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215f0f0 0x55a1b217b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.075+0000 7f7455f2b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215f0f0 0x55a1b217b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48440/0 (socket says 172.21.15.69:48440) 2024-08-02T18:00:36.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.075+0000 7f7455f2b640 1 -- 172.21.15.69:0/395100311 learned_addr learned my addr 172.21.15.69:0/395100311 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:36.077 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.075+0000 7f745572a640 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215fa10 0x55a1b2175bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.075+0000 7f745a58d740 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 0x55a1b2176130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f74501aaee0 con 0x55a1b2172bc0 2024-08-02T18:00:36.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f74501ac350 con 0x55a1b215f0f0 2024-08-02T18:00:36.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f74501ab5b0 con 0x55a1b215fa10 2024-08-02T18:00:36.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745a58d740 10 client.0 _ll_register_callbacks cb 0x7f744000adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:36.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745572a640 1 -- 172.21.15.69:0/395100311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215f0f0 msgr2=0x55a1b217b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:36.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745572a640 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215f0f0 0x55a1b217b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745572a640 1 -- 172.21.15.69:0/395100311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 msgr2=0x55a1b2176130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:36.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745572a640 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 0x55a1b2176130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.076+0000 7f745572a640 1 -- 172.21.15.69:0/395100311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b216a850 con 0x55a1b215fa10 2024-08-02T18:00:36.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.077+0000 7f745572a640 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215fa10 0x55a1b2175bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7440004610 tx=0x7f74400274a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a2bc728379310c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:36.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.077+0000 7f74397fa640 10 client.28123 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:36.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.077+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f744c028030 con 0x55a1b215fa10 2024-08-02T18:00:36.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.078+0000 7f74397fa640 1 -- 172.21.15.69:0/395100311 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440005c20 con 0x55a1b215fa10 2024-08-02T18:00:36.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.078+0000 7f74397fa640 1 -- 172.21.15.69:0/395100311 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f7440005530 con 0x55a1b215fa10 2024-08-02T18:00:36.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.078+0000 7f74397fa640 1 -- 172.21.15.69:0/395100311 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f744000a710 con 0x55a1b215fa10 2024-08-02T18:00:36.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.079+0000 7f745a58d740 10 client.28123 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:36.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.079+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55a1b21ec010 con 0x55a1b215fa10 2024-08-02T18:00:36.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.080+0000 7f74397fa640 1 -- 172.21.15.69:0/395100311 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7440026040 con 0x55a1b215fa10 2024-08-02T18:00:36.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.080+0000 7f745a58d740 10 client.28123 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:36.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.080+0000 7f745a58d740 10 client.28123 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:00:36.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.080+0000 7f745a58d740 20 client.28123 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:36.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.080+0000 7f745a58d740 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a1b21ec4d0 0x55a1b21ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:36.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.080+0000 7f745a58d740 4 client.28123 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:36.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.080+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f744c028030 con 0x55a1b21ec4d0 2024-08-02T18:00:36.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.087+0000 7f7455f2b640 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a1b21ec4d0 0x55a1b21ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.092+0000 7f7455f2b640 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a1b21ec4d0 0x55a1b21ee8f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5e62c79dd4e110a server_cookie=95e620751723a61f in_seq=0 out_seq=0 2024-08-02T18:00:36.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.092+0000 7f74397fa640 10 client.28123 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:36.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.092+0000 7f74397fa640 1 -- 172.21.15.69:0/395100311 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f744c028030 con 0x55a1b21ec4d0 2024-08-02T18:00:36.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.092+0000 7f74397fa640 10 client.28123 handle_command_reply: tid=0 2024-08-02T18:00:36.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.092+0000 7f745a58d740 1 client.28123 shutdown 2024-08-02T18:00:36.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.094+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215fa10 msgr2=0x55a1b2175bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:36.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.094+0000 7f745a58d740 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215fa10 0x55a1b2175bf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7440004610 tx=0x7f74400274a0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.097+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 shutdown_connections 2024-08-02T18:00:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.097+0000 7f745a58d740 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55a1b21ec4d0 0x55a1b21ee8f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.097+0000 7f745a58d740 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55a1b2172bc0 0x55a1b2176130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.097+0000 7f745a58d740 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55a1b215fa10 0x55a1b2175bf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.097+0000 7f745a58d740 1 --2- 172.21.15.69:0/395100311 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55a1b215f0f0 0x55a1b217b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.097+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 >> 172.21.15.69:0/395100311 conn(0x55a1b215b760 msgr2=0x55a1b216eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.097+0000 7f74397fa640 0 client.28123 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:36.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.097+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 shutdown_connections 2024-08-02T18:00:36.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.099+0000 7f745a58d740 1 -- 172.21.15.69:0/395100311 wait complete. 2024-08-02T18:00:36.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.100+0000 7f745a58d740 20 client.28123 trim_cache size 0 max 16384 2024-08-02T18:00:36.103 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:36.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 -- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f743c0a2d20 msgr2=0x7f743c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:36.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f743c0a2d20 0x7f743c0a51e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7450196bc0 tx=0x7f744c005c70 comp rx=0 tx=0).stop 2024-08-02T18:00:36.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 -- 172.21.15.69:0/1339051200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7450106690 msgr2=0x7f7450195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:36.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7450106690 0x7f7450195a80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f744802ae30 tx=0x7f7448003c10 comp rx=0 tx=0).stop 2024-08-02T18:00:36.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 -- 172.21.15.69:0/1339051200 shutdown_connections 2024-08-02T18:00:36.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f743c0a2d20 0x7f743c0a51e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7450106690 0x7f7450195a80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7450100fc0 0x7f745019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 --2- 172.21.15.69:0/1339051200 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7450100600 0x7f745019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:36.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 -- 172.21.15.69:0/1339051200 >> 172.21.15.69:0/1339051200 conn(0x7f74500fc340 msgr2=0x7f74500fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:36.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 -- 172.21.15.69:0/1339051200 shutdown_connections 2024-08-02T18:00:36.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:36.103+0000 7f7456f2d640 1 -- 172.21.15.69:0/1339051200 wait complete. 2024-08-02T18:00:36.838 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:36 smithi069 ceph-mon[32746]: pgmap v444: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-08-02T18:00:36.840 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T18:00:37.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:36 smithi083 ceph-mon[37031]: pgmap v444: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-08-02T18:00:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:36 smithi077 ceph-mon[37108]: pgmap v444: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 4.0 MiB/s wr, 461 op/s 2024-08-02T18:00:37.390 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:38.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 -- 172.21.15.69:0/3828021773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e0105290 msgr2=0x7f67e010ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 --2- 172.21.15.69:0/3828021773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e0105290 0x7f67e010ce10 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f67d8005fd0 tx=0x7f67d802eaa0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 -- 172.21.15.69:0/3828021773 shutdown_connections 2024-08-02T18:00:38.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 --2- 172.21.15.69:0/3828021773 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e0105290 0x7f67e010ce10 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 --2- 172.21.15.69:0/3828021773 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e0100b10 0x7f67e0104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 --2- 172.21.15.69:0/3828021773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67e0100160 0x7f67e0100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 -- 172.21.15.69:0/3828021773 >> 172.21.15.69:0/3828021773 conn(0x7f67e0076f60 msgr2=0x7f67e00fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:38.033 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 -- 172.21.15.69:0/3828021773 shutdown_connections 2024-08-02T18:00:38.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.031+0000 7f67e87e0640 1 -- 172.21.15.69:0/3828021773 wait complete. 2024-08-02T18:00:38.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.033+0000 7f67e87e0640 1 Processor -- start 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.034+0000 7f67e87e0640 1 -- start start 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.034+0000 7f67e87e0640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e0100160 0x7f67e0195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.034+0000 7f67e87e0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67e0100b10 0x7f67e0196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e87e0640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e0105290 0x7f67e019a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e87e0640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67e010fa70 con 0x7f67e0105290 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e87e0640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67e010f8f0 con 0x7f67e0100160 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e87e0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67e010fbf0 con 0x7f67e0100b10 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e0100160 0x7f67e0195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e0100160 0x7f67e0195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48462/0 (socket says 172.21.15.69:48462) 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e5d7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67e0100b10 0x7f67e0196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 -- 172.21.15.69:0/4011564087 learned_addr learned my addr 172.21.15.69:0/4011564087 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 -- 172.21.15.69:0/4011564087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67e0100b10 msgr2=0x7f67e0196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67e0100b10 0x7f67e0196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 -- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e0105290 msgr2=0x7f67e019a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e0105290 0x7f67e019a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 -- 172.21.15.69:0/4011564087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e019aec0 con 0x7f67e0100160 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67e0100b10 0x7f67e0196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:38.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.035+0000 7f67e657c640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e0100160 0x7f67e0195f10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00d950 tx=0x7f67dc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.036+0000 7f67d77fe640 1 -- 172.21.15.69:0/4011564087 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc017070 con 0x7f67e0100160 2024-08-02T18:00:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.036+0000 7f67e87e0640 1 -- 172.21.15.69:0/4011564087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e019b1b0 con 0x7f67e0100160 2024-08-02T18:00:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.036+0000 7f67e87e0640 1 -- 172.21.15.69:0/4011564087 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e01a2a90 con 0x7f67e0100160 2024-08-02T18:00:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.036+0000 7f67d77fe640 1 -- 172.21.15.69:0/4011564087 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67dc009c30 con 0x7f67e0100160 2024-08-02T18:00:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.036+0000 7f67d77fe640 1 -- 172.21.15.69:0/4011564087 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00f6b0 con 0x7f67e0100160 2024-08-02T18:00:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.039+0000 7f67d77fe640 1 -- 172.21.15.69:0/4011564087 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f67dc00f850 con 0x7f67e0100160 2024-08-02T18:00:38.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.039+0000 7f67eabde740 1 Processor -- start 2024-08-02T18:00:38.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.039+0000 7f67eabde740 1 -- start start 2024-08-02T18:00:38.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67eabde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x5631738253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67eabde740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 0x56317382fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67eabde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631738301d0 0x5631738325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67eabde740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56317378aa40 con 0x563173825910 2024-08-02T18:00:38.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67e5d7b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631738301d0 0x5631738325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67e5d7b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631738301d0 0x5631738325c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48464/0 (socket says 172.21.15.69:48464) 2024-08-02T18:00:38.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67e5d7b640 1 -- 172.21.15.69:0/3756391989 learned_addr learned my addr 172.21.15.69:0/3756391989 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:38.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67e6d7d640 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x5631738253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.040+0000 7f67e657c640 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 0x56317382fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67eabde740 1 -- 172.21.15.69:0/3756391989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5631737229d0 con 0x5631738301d0 2024-08-02T18:00:38.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67eabde740 1 -- 172.21.15.69:0/3756391989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563173786a00 con 0x563173824ff0 2024-08-02T18:00:38.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67d77fe640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f67cc0a2d70 0x7f67cc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67e5d7b640 1 -- 172.21.15.69:0/3756391989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 msgr2=0x5631738253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x5631738253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67e5d7b640 1 -- 172.21.15.69:0/3756391989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 msgr2=0x56317382fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 0x56317382fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67e5d7b640 1 -- 172.21.15.69:0/3756391989 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563173729650 con 0x5631738301d0 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67d77fe640 1 -- 172.21.15.69:0/4011564087 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f67dc0cf6c0 con 0x7f67e0100160 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631738301d0 0x5631738325c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009340 tx=0x7f67d0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eda10beb19aa78d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.041+0000 7f67e6d7d640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f67cc0a2d70 0x7f67cc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.042+0000 7f67d4ff9640 1 -- 172.21.15.69:0/3756391989 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0004230 con 0x5631738301d0 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.042+0000 7f67d4ff9640 1 -- 172.21.15.69:0/3756391989 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d00043d0 con 0x5631738301d0 2024-08-02T18:00:38.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.042+0000 7f67d4ff9640 1 -- 172.21.15.69:0/3756391989 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00168d0 con 0x5631738301d0 2024-08-02T18:00:38.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.043+0000 7f67eabde740 1 -- 172.21.15.69:0/3756391989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631738301d0 msgr2=0x5631738325c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.043+0000 7f67eabde740 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631738301d0 0x5631738325c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009340 tx=0x7f67d0005f90 comp rx=0 tx=0).stop 2024-08-02T18:00:38.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.043+0000 7f67e6d7d640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f67cc0a2d70 0x7f67cc0a5230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f67d800a500 tx=0x7f67d803d040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:38.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.047+0000 7f67eabde740 1 -- 172.21.15.69:0/3756391989 shutdown_connections 2024-08-02T18:00:38.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.047+0000 7f67eabde740 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 0x56317382fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.047+0000 7f67eabde740 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5631738301d0 0x5631738325c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.047+0000 7f67eabde740 1 --2- 172.21.15.69:0/3756391989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x5631738253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.047+0000 7f67eabde740 1 -- 172.21.15.69:0/3756391989 >> 172.21.15.69:0/3756391989 conn(0x563173821580 msgr2=0x563173821960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:38.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.055+0000 7f67eabde740 1 -- 172.21.15.69:0/3756391989 shutdown_connections 2024-08-02T18:00:38.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.059+0000 7f67eabde740 1 -- 172.21.15.69:0/3756391989 wait complete. 2024-08-02T18:00:38.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.059+0000 7f67eabde740 1 -- start start 2024-08-02T18:00:38.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.059+0000 7f67eabde740 1 Processor -- start 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.062+0000 7f67eabde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 0x56317382d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.062+0000 7f67eabde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173825910 0x56317382d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.062+0000 7f67eabde740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 0x56317382db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.063+0000 7f67eabde740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563173832da0 con 0x5631738301d0 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.063+0000 7f67e5d7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173825910 0x56317382d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.063+0000 7f67e5d7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173825910 0x56317382d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:49864/0 (socket says 172.21.15.69:49864) 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.063+0000 7f67e5d7b640 1 -- 172.21.15.69:0/1183691005 learned_addr learned my addr 172.21.15.69:0/1183691005 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.063+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563173745b20 con 0x563173824ff0 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.063+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563173832f20 con 0x563173825910 2024-08-02T18:00:38.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.063+0000 7f67eabde740 10 client.0 _ll_register_callbacks cb 0x563173743bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:38.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.065+0000 7f67e6d7d640 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 0x56317382db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.066+0000 7f67e5d7b640 1 -- 172.21.15.69:0/1183691005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 msgr2=0x56317382d080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:00:38.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.066+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 0x56317382d080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.066+0000 7f67e5d7b640 1 -- 172.21.15.69:0/1183691005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 msgr2=0x56317382db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:38.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.066+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 0x56317382db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.066+0000 7f67e5d7b640 1 -- 172.21.15.69:0/1183691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631734fc3c0 con 0x563173825910 2024-08-02T18:00:38.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.070+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173825910 0x56317382d5c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009fc0 tx=0x7f67d0005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4e90c48ccab59a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:38.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.072+0000 7f67c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:38.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.073+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1183691005 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0026070 con 0x563173825910 2024-08-02T18:00:38.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.073+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1183691005 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f67d00046c0 con 0x563173825910 2024-08-02T18:00:38.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.076+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d00043d0 con 0x563173825910 2024-08-02T18:00:38.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.081+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1183691005 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67d0030050 con 0x563173825910 2024-08-02T18:00:38.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.082+0000 7f67eabde740 10 client.28135 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:38.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.082+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563173847210 con 0x563173825910 2024-08-02T18:00:38.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.086+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1183691005 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d0004dc0 con 0x563173825910 2024-08-02T18:00:38.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.086+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1183691005 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f67d00165d0 con 0x563173825910 2024-08-02T18:00:38.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.089+0000 7f67eabde740 10 client.28135 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:38.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.089+0000 7f67eabde740 10 client.28135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:38.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.090+0000 7f67eabde740 20 client.28135 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:38.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.090+0000 7f67eabde740 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5631738486d0 0x56317384aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.090+0000 7f67eabde740 4 client.28135 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:38.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.090+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f67d00043d0 con 0x5631738486d0 2024-08-02T18:00:38.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.090+0000 7f67e657c640 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5631738486d0 0x56317384aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.098+0000 7f67e657c640 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5631738486d0 0x56317384aac0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcbd9d8a96718091 server_cookie=63ead2fc0bd99f94 in_seq=0 out_seq=0 2024-08-02T18:00:38.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.098+0000 7f67c9ffb640 10 client.28135 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.101+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1183691005 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f67d00043d0 con 0x5631738486d0 2024-08-02T18:00:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.101+0000 7f67c9ffb640 10 client.28135 handle_command_reply: tid=0 2024-08-02T18:00:38.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.101+0000 7f67eabde740 1 client.28135 shutdown 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173825910 msgr2=0x56317382d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67eabde740 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173825910 0x56317382d5c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009fc0 tx=0x7f67d0005270 comp rx=0 tx=0).stop 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 shutdown_connections 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67eabde740 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 0x56317382db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67eabde740 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5631738486d0 0x56317384aac0 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67eabde740 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173825910 0x56317382d5c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67eabde740 1 --2- 172.21.15.69:0/1183691005 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 0x56317382d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67c9ffb640 0 client.28135 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:38.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.104+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 >> 172.21.15.69:0/1183691005 conn(0x563173821580 msgr2=0x5631738272b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:38.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.106+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 shutdown_connections 2024-08-02T18:00:38.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.106+0000 7f67eabde740 1 -- 172.21.15.69:0/1183691005 wait complete. 2024-08-02T18:00:38.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.108+0000 7f67eabde740 20 client.28135 trim_cache size 0 max 16384 2024-08-02T18:00:38.141 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.133+0000 7f67eabde740 1 Processor -- start 2024-08-02T18:00:38.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67eabde740 1 -- start start 2024-08-02T18:00:38.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67eabde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 0x56317382d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67eabde740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 0x56317382bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67eabde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631738301d0 0x5631738500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67eabde740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4012a50 con 0x563173825910 2024-08-02T18:00:38.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67eabde740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4010320 con 0x563173824ff0 2024-08-02T18:00:38.143 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67eabde740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67c400e170 con 0x5631738301d0 2024-08-02T18:00:38.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67e657c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 0x56317382d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67e657c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 0x56317382d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48478/0 (socket says 172.21.15.69:48478) 2024-08-02T18:00:38.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.134+0000 7f67e657c640 1 -- 172.21.15.69:0/3653249895 learned_addr learned my addr 172.21.15.69:0/3653249895 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:38.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.135+0000 7f67e6d7d640 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631738301d0 0x5631738500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.135+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 0x56317382bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.135+0000 7f67e6d7d640 1 -- 172.21.15.69:0/3653249895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 msgr2=0x56317382d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:38.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.135+0000 7f67e6d7d640 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 0x56317382d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.135+0000 7f67e6d7d640 1 -- 172.21.15.69:0/3653249895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 msgr2=0x56317382bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:38.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.135+0000 7f67e6d7d640 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 0x56317382bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.135+0000 7f67e6d7d640 1 -- 172.21.15.69:0/3653249895 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56317382b610 con 0x5631738301d0 2024-08-02T18:00:38.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.135+0000 7f67e6d7d640 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631738301d0 0x5631738500f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67d802f090 tx=0x7f67d8043000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4aa37023b5a1e7eb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:38.147 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.137+0000 7f67cbfff640 1 -- 172.21.15.69:0/3653249895 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004040 con 0x5631738301d0 2024-08-02T18:00:38.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.137+0000 7f67cbfff640 1 -- 172.21.15.69:0/3653249895 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d80046c0 con 0x5631738301d0 2024-08-02T18:00:38.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.137+0000 7f67cbfff640 1 -- 172.21.15.69:0/3653249895 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d803aa70 con 0x5631738301d0 2024-08-02T18:00:38.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.138+0000 7f67eabde740 1 -- 172.21.15.69:0/3653249895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631738301d0 msgr2=0x5631738500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.148 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.138+0000 7f67eabde740 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631738301d0 0x5631738500f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f67d802f090 tx=0x7f67d8043000 comp rx=0 tx=0).stop 2024-08-02T18:00:38.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.142+0000 7f67eabde740 1 -- 172.21.15.69:0/3653249895 shutdown_connections 2024-08-02T18:00:38.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.142+0000 7f67eabde740 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563173825910 0x56317382bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.142+0000 7f67eabde740 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5631738301d0 0x5631738500f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.142+0000 7f67eabde740 1 --2- 172.21.15.69:0/3653249895 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173824ff0 0x56317382d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.142+0000 7f67eabde740 1 -- 172.21.15.69:0/3653249895 >> 172.21.15.69:0/3653249895 conn(0x563173821580 msgr2=0x56317382b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:38.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.143+0000 7f67eabde740 1 -- 172.21.15.69:0/3653249895 shutdown_connections 2024-08-02T18:00:38.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.143+0000 7f67eabde740 1 -- 172.21.15.69:0/3653249895 wait complete. 2024-08-02T18:00:38.163 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.143+0000 7f67eabde740 1 -- start start 2024-08-02T18:00:38.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.143+0000 7f67eabde740 1 Processor -- start 2024-08-02T18:00:38.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.144+0000 7f67eabde740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x56317382ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.144+0000 7f67eabde740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173825910 0x563173840a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.144+0000 7f67e657c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x56317382ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.144+0000 7f67e657c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x56317382ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:49892/0 (socket says 172.21.15.69:49892) 2024-08-02T18:00:38.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.144+0000 7f67e657c640 1 -- 172.21.15.69:0/1891420173 learned_addr learned my addr 172.21.15.69:0/1891420173 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:38.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.144+0000 7f67eabde740 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 0x5631738410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.164 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.144+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173825910 0x563173840a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.145+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f67c400e7e0 con 0x5631738301d0 2024-08-02T18:00:38.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.145+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f67c400fc50 con 0x563173825910 2024-08-02T18:00:38.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.145+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f67c400eeb0 con 0x563173824ff0 2024-08-02T18:00:38.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.145+0000 7f67e657c640 1 -- 172.21.15.69:0/1891420173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173825910 msgr2=0x563173840a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:38.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.145+0000 7f67e657c640 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173825910 0x563173840a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.165 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.145+0000 7f67e657c640 1 -- 172.21.15.69:0/1891420173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 msgr2=0x5631738410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:38.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.145+0000 7f67e657c640 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 0x5631738410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67e657c640 1 -- 172.21.15.69:0/1891420173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563173829330 con 0x563173824ff0 2024-08-02T18:00:38.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67eabde740 10 client.0 _ll_register_callbacks cb 0x7f67dc0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:38.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d80046c0 con 0x563173824ff0 2024-08-02T18:00:38.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67e657c640 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x56317382ad50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000c00 tx=0x7f67dc09b660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39fb8e00fc624e10 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:38.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67c9ffb640 10 client.28141 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:38.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1891420173 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc020040 con 0x563173824ff0 2024-08-02T18:00:38.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1891420173 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f67dc0860b0 con 0x563173824ff0 2024-08-02T18:00:38.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1891420173 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f67dc09bed0 con 0x563173824ff0 2024-08-02T18:00:38.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.146+0000 7f67eabde740 10 client.28141 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:38.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.147+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5631738b2240 con 0x563173824ff0 2024-08-02T18:00:38.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.148+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1891420173 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f67dc0d2ce0 con 0x563173824ff0 2024-08-02T18:00:38.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.148+0000 7f67eabde740 10 client.28141 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:38.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.148+0000 7f67eabde740 10 client.28141 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:38.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.148+0000 7f67eabde740 20 client.28141 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:38.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.148+0000 7f67eabde740 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5631738b2880 0x5631738b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:38.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.148+0000 7f67eabde740 4 client.28141 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:38.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.148+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563173797fb0 con 0x5631738b2880 2024-08-02T18:00:38.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.149+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5631738b2880 0x5631738b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:38.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.149+0000 7f67e5d7b640 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5631738b2880 0x5631738b4d40 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4c2951db299a7e server_cookie=873cd69c396a335b in_seq=0 out_seq=0 2024-08-02T18:00:38.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.149+0000 7f67c9ffb640 10 client.28141 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:38.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.149+0000 7f67c9ffb640 1 -- 172.21.15.69:0/1891420173 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563173797fb0 con 0x5631738b2880 2024-08-02T18:00:38.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.149+0000 7f67c9ffb640 10 client.28141 handle_command_reply: tid=0 2024-08-02T18:00:38.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.149+0000 7f67eabde740 1 client.28141 shutdown 2024-08-02T18:00:38.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.152+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 msgr2=0x56317382ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.152+0000 7f67eabde740 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x56317382ad50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f67dc000c00 tx=0x7f67dc09b660 comp rx=0 tx=0).stop 2024-08-02T18:00:38.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.153+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 shutdown_connections 2024-08-02T18:00:38.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.153+0000 7f67eabde740 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5631738301d0 0x5631738410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.153+0000 7f67eabde740 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5631738b2880 0x5631738b4d40 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.153+0000 7f67c9ffb640 0 client.28141 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:38.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.153+0000 7f67eabde740 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563173825910 0x563173840a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.153+0000 7f67eabde740 1 --2- 172.21.15.69:0/1891420173 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563173824ff0 0x56317382ad50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.153+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 >> 172.21.15.69:0/1891420173 conn(0x563173821580 msgr2=0x5631738205b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:38.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.155+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 shutdown_connections 2024-08-02T18:00:38.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.155+0000 7f67eabde740 1 -- 172.21.15.69:0/1891420173 wait complete. 2024-08-02T18:00:38.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.155+0000 7f67eabde740 20 client.28141 trim_cache size 0 max 16384 2024-08-02T18:00:38.173 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:38.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 -- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f67cc0a2d70 msgr2=0x7f67cc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f67cc0a2d70 0x7f67cc0a5230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f67d800a500 tx=0x7f67d803d040 comp rx=0 tx=0).stop 2024-08-02T18:00:38.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 -- 172.21.15.69:0/4011564087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e0100160 msgr2=0x7f67e0195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:38.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e0100160 0x7f67e0195f10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00d950 tx=0x7f67dc00de10 comp rx=0 tx=0).stop 2024-08-02T18:00:38.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 -- 172.21.15.69:0/4011564087 shutdown_connections 2024-08-02T18:00:38.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f67e0105290 0x7f67e019a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f67cc0a2d70 0x7f67cc0a5230 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f67e0100b10 0x7f67e0196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 --2- 172.21.15.69:0/4011564087 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f67e0100160 0x7f67e0195f10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:38.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 -- 172.21.15.69:0/4011564087 >> 172.21.15.69:0/4011564087 conn(0x7f67e0076f60 msgr2=0x7f67e0103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:38.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 -- 172.21.15.69:0/4011564087 shutdown_connections 2024-08-02T18:00:38.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:38.159+0000 7f67e87e0640 1 -- 172.21.15.69:0/4011564087 wait complete. 2024-08-02T18:00:38.975 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:38 smithi069 ceph-mon[32746]: pgmap v445: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.5 MiB/s wr, 464 op/s 2024-08-02T18:00:39.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:38 smithi083 ceph-mon[37031]: pgmap v445: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.5 MiB/s wr, 464 op/s 2024-08-02T18:00:39.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:38 smithi077 ceph-mon[37108]: pgmap v445: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 4.5 MiB/s wr, 464 op/s 2024-08-02T18:00:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 810.0 s, 530.4 tps, lat 189.026 ms stddev 242.393 2024-08-02T18:00:40.021 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T18:00:40.022 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T18:00:40.484 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:41.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:40 smithi083 ceph-mon[37031]: pgmap v446: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-08-02T18:00:41.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:40 smithi077 ceph-mon[37108]: pgmap v446: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-08-02T18:00:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:40 smithi069 ceph-mon[32746]: pgmap v446: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 6.0 MiB/s wr, 612 op/s 2024-08-02T18:00:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:42 smithi083 ceph-mon[37031]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 415 op/s 2024-08-02T18:00:43.038 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:42 smithi069 ceph-mon[32746]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 415 op/s 2024-08-02T18:00:43.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:42 smithi077 ceph-mon[37108]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 4.3 MiB/s wr, 415 op/s 2024-08-02T18:00:43.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 -- 172.21.15.69:0/1176834936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 msgr2=0x7fb4741056b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 --2- 172.21.15.69:0/1176834936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 0x7fb4741056b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb464009900 tx=0x7fb46402a410 comp rx=0 tx=0).stop 2024-08-02T18:00:43.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 -- 172.21.15.69:0/1176834936 shutdown_connections 2024-08-02T18:00:43.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 --2- 172.21.15.69:0/1176834936 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 0x7fb4741056b0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 --2- 172.21.15.69:0/1176834936 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb474105bf0 0x7fb474106070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 --2- 172.21.15.69:0/1176834936 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb4741005f0 0x7fb4741009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 -- 172.21.15.69:0/1176834936 >> 172.21.15.69:0/1176834936 conn(0x7fb4740fc500 msgr2=0x7fb4740fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:43.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 -- 172.21.15.69:0/1176834936 shutdown_connections 2024-08-02T18:00:43.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.164+0000 7fb47947c640 1 -- 172.21.15.69:0/1176834936 wait complete. 2024-08-02T18:00:43.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb47947c640 1 Processor -- start 2024-08-02T18:00:43.167 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb47947c640 1 -- start start 2024-08-02T18:00:43.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb47947c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4741005f0 0x7fb4741a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb47947c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 0x7fb4741a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb47947c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb474105bf0 0x7fb47419e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb47947c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb474118360 con 0x7fb474100f10 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb47947c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741181e0 con 0x7fb474105bf0 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb47947c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741184e0 con 0x7fb4741005f0 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4727fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 0x7fb4741a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4727fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 0x7fb4741a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46124/0 (socket says 172.21.15.69:46124) 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4727fc640 1 -- 172.21.15.69:0/1021001058 learned_addr learned my addr 172.21.15.69:0/1021001058 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4737fe640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb474105bf0 0x7fb47419e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb472ffd640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4741005f0 0x7fb4741a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4727fc640 1 -- 172.21.15.69:0/1021001058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4741005f0 msgr2=0x7fb4741a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4727fc640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4741005f0 0x7fb4741a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4727fc640 1 -- 172.21.15.69:0/1021001058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb474105bf0 msgr2=0x7fb47419e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4727fc640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb474105bf0 0x7fb47419e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4727fc640 1 -- 172.21.15.69:0/1021001058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47419eed0 con 0x7fb474100f10 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.165+0000 7fb4737fe640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb474105bf0 0x7fb47419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.166+0000 7fb4727fc640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 0x7fb4741a4ad0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb464006010 tx=0x7fb464003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.166+0000 7fb46bfff640 1 -- 172.21.15.69:0/1021001058 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46403f070 con 0x7fb474100f10 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.166+0000 7fb46bfff640 1 -- 172.21.15.69:0/1021001058 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb464031c30 con 0x7fb474100f10 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.166+0000 7fb47947c640 1 -- 172.21.15.69:0/1021001058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47419f160 con 0x7fb474100f10 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.166+0000 7fb46bfff640 1 -- 172.21.15.69:0/1021001058 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464035030 con 0x7fb474100f10 2024-08-02T18:00:43.170 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.166+0000 7fb47947c640 1 -- 172.21.15.69:0/1021001058 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4741ab340 con 0x7fb474100f10 2024-08-02T18:00:43.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb47b87a740 1 Processor -- start 2024-08-02T18:00:43.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb47b87a740 1 -- start start 2024-08-02T18:00:43.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb46bfff640 1 -- 172.21.15.69:0/1021001058 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb464042070 con 0x7fb474100f10 2024-08-02T18:00:43.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a40003e0 0x55f5a40027d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb47b87a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3fb8e40 con 0x55f5a40003e0 2024-08-02T18:00:43.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb47b87a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3f5a7f0 con 0x55f5a3ff5200 2024-08-02T18:00:43.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb47b87a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3f01fb0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb472ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.173 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb472ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fffea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:49922/0 (socket says 172.21.15.69:49922) 2024-08-02T18:00:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb472ffd640 1 -- 172.21.15.69:0/3899477989 learned_addr learned my addr 172.21.15.69:0/3899477989 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb4727fc640 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a40003e0 0x55f5a40027d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.169+0000 7fb4737fe640 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.170+0000 7fb472ffd640 1 -- 172.21.15.69:0/3899477989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 msgr2=0x55f5a3ff55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.170+0000 7fb472ffd640 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.170+0000 7fb472ffd640 1 -- 172.21.15.69:0/3899477989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a40003e0 msgr2=0x55f5a40027d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:43.174 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.170+0000 7fb472ffd640 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a40003e0 0x55f5a40027d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.170+0000 7fb472ffd640 1 -- 172.21.15.69:0/3899477989 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a3ccc6a0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.170+0000 7fb4737fe640 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:43.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.170+0000 7fb46bfff640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4580a2cf0 0x7fb4580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.170+0000 7fb46bfff640 1 -- 172.21.15.69:0/1021001058 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fb4640efec0 con 0x7fb474100f10 2024-08-02T18:00:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.171+0000 7fb472ffd640 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fffea0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb460009cf0 tx=0x7fb460011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657524bb45a9ff5e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.171+0000 7fb4697fa640 1 -- 172.21.15.69:0/3899477989 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460004230 con 0x55f5a3ff5b20 2024-08-02T18:00:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.171+0000 7fb4697fa640 1 -- 172.21.15.69:0/3899477989 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4600043d0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.171+0000 7fb4697fa640 1 -- 172.21.15.69:0/3899477989 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460019960 con 0x55f5a3ff5b20 2024-08-02T18:00:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.172+0000 7fb4737fe640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4580a2cf0 0x7fb4580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.172+0000 7fb47b87a740 1 -- 172.21.15.69:0/3899477989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 msgr2=0x55f5a3fffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.182 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.172+0000 7fb47b87a740 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fffea0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb460009cf0 tx=0x7fb460011810 comp rx=0 tx=0).stop 2024-08-02T18:00:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.172+0000 7fb4737fe640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4580a2cf0 0x7fb4580a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fd0 tx=0x7fb46c009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.173+0000 7fb47b87a740 1 -- 172.21.15.69:0/3899477989 shutdown_connections 2024-08-02T18:00:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.173+0000 7fb47b87a740 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a40003e0 0x55f5a40027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.173+0000 7fb47b87a740 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fffea0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.173+0000 7fb47b87a740 1 --2- 172.21.15.69:0/3899477989 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.173+0000 7fb47b87a740 1 -- 172.21.15.69:0/3899477989 >> 172.21.15.69:0/3899477989 conn(0x55f5a3ff1790 msgr2=0x55f5a3ff1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:43.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.173+0000 7fb47b87a740 1 -- 172.21.15.69:0/3899477989 shutdown_connections 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.173+0000 7fb47b87a740 1 -- 172.21.15.69:0/3899477989 wait complete. 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.173+0000 7fb47b87a740 1 -- start start 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb47b87a740 1 Processor -- start 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a3ffd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3ffd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a40003e0 0x55f5a3ffdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb47b87a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3f15cb0 con 0x55f5a3ff5200 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb47b87a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f5a3f567b0 con 0x55f5a40003e0 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb47b87a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f5a4002fb0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a40003e0 0x55f5a3ffdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.184 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4727fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3ffd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb47b87a740 10 client.0 _ll_register_callbacks cb 0x55f5a3c69ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4737fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a40003e0 0x55f5a3ffdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48524/0 (socket says 172.21.15.69:48524) 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4737fe640 1 -- 172.21.15.69:0/4159043195 learned_addr learned my addr 172.21.15.69:0/4159043195 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb472ffd640 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a3ffd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4737fe640 1 -- 172.21.15.69:0/4159043195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 msgr2=0x55f5a3ffd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4737fe640 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3ffd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4737fe640 1 -- 172.21.15.69:0/4159043195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 msgr2=0x55f5a3ffd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4737fe640 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a3ffd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.174+0000 7fb4737fe640 1 -- 172.21.15.69:0/4159043195 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a3f14bc0 con 0x55f5a40003e0 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb4737fe640 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a40003e0 0x55f5a3ffdd00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0120c0 tx=0x7fb46c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e4d0bcb8f4fe23d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb4567fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb4567fc640 1 -- 172.21.15.69:0/4159043195 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c011c10 con 0x55f5a40003e0 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb4567fc640 1 -- 172.21.15.69:0/4159043195 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fb46c011db0 con 0x55f5a40003e0 2024-08-02T18:00:43.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb47b87a740 1 -- 172.21.15.69:0/4159043195 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4600043d0 con 0x55f5a40003e0 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb4567fc640 1 -- 172.21.15.69:0/4159043195 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb46c032050 con 0x55f5a40003e0 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb47b87a740 10 client.28397 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb47b87a740 1 -- 172.21.15.69:0/4159043195 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5a400eee0 con 0x55f5a40003e0 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.175+0000 7fb4567fc640 1 -- 172.21.15.69:0/4159043195 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4600043d0 con 0x55f5a40003e0 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb4567fc640 1 -- 172.21.15.69:0/4159043195 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb46c027020 con 0x55f5a40003e0 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb47b87a740 10 client.28397 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb47b87a740 10 client.28397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb47b87a740 20 client.28397 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb47b87a740 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f5a40103f0 0x55f5a4012860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb47b87a740 4 client.28397 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb47b87a740 1 -- 172.21.15.69:0/4159043195 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5a3f67d60 con 0x55f5a40103f0 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb472ffd640 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f5a40103f0 0x55f5a4012860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.186 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb472ffd640 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f5a40103f0 0x55f5a4012860 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717621f9bf8a7ce5 server_cookie=24f947c4bfc8f586 in_seq=0 out_seq=0 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.176+0000 7fb4567fc640 10 client.28397 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb4567fc640 1 -- 172.21.15.69:0/4159043195 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f5a3f67d60 con 0x55f5a40103f0 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb4567fc640 10 client.28397 handle_command_reply: tid=0 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 client.28397 shutdown 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 -- 172.21.15.69:0/4159043195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a40003e0 msgr2=0x55f5a3ffdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a40003e0 0x55f5a3ffdd00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0120c0 tx=0x7fb46c004ab0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 -- 172.21.15.69:0/4159043195 shutdown_connections 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f5a40103f0 0x55f5a4012860 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a40003e0 0x55f5a3ffdd00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a3ff5b20 0x55f5a3ffd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 --2- 172.21.15.69:0/4159043195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a3ffd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb4567fc640 0 client.28397 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 -- 172.21.15.69:0/4159043195 >> 172.21.15.69:0/4159043195 conn(0x55f5a3ff1790 msgr2=0x55f5a3ff08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:43.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 -- 172.21.15.69:0/4159043195 shutdown_connections 2024-08-02T18:00:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.177+0000 7fb47b87a740 1 -- 172.21.15.69:0/4159043195 wait complete. 2024-08-02T18:00:43.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.178+0000 7fb47b87a740 20 client.28397 trim_cache size 0 max 16384 2024-08-02T18:00:43.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb47b87a740 1 Processor -- start 2024-08-02T18:00:43.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb47b87a740 1 -- start start 2024-08-02T18:00:43.208 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 0x55f5a4018130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb47b87a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741b7dc0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb47b87a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741b5690 con 0x55f5a3ff5200 2024-08-02T18:00:43.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb47b87a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741b34e0 con 0x55f5a40003e0 2024-08-02T18:00:43.209 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb472ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb472ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48526/0 (socket says 172.21.15.69:48526) 2024-08-02T18:00:43.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb4737fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 0x55f5a4018130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb472ffd640 1 -- 172.21.15.69:0/625669913 learned_addr learned my addr 172.21.15.69:0/625669913 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:43.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.198+0000 7fb4727fc640 1 --2- 172.21.15.69:0/625669913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb472ffd640 1 -- 172.21.15.69:0/625669913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 msgr2=0x55f5a4018130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:43.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb472ffd640 1 --2- 172.21.15.69:0/625669913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 0x55f5a4018130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb472ffd640 1 -- 172.21.15.69:0/625669913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5b20 msgr2=0x55f5a3fff1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:43.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb472ffd640 1 --2- 172.21.15.69:0/625669913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fff1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb472ffd640 1 -- 172.21.15.69:0/625669913 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5a3ff5030 con 0x55f5a3ff5200 2024-08-02T18:00:43.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb472ffd640 1 --2- 172.21.15.69:0/625669913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff7c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb4600194d0 tx=0x7fb460019420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bffe7d25b99d711d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:43.211 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb468ff9640 1 -- 172.21.15.69:0/625669913 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460031070 con 0x55f5a3ff5200 2024-08-02T18:00:43.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb468ff9640 1 -- 172.21.15.69:0/625669913 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4600035f0 con 0x55f5a3ff5200 2024-08-02T18:00:43.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.199+0000 7fb468ff9640 1 -- 172.21.15.69:0/625669913 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460019820 con 0x55f5a3ff5200 2024-08-02T18:00:43.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 -- 172.21.15.69:0/625669913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 msgr2=0x55f5a3ff7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 --2- 172.21.15.69:0/625669913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff7c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb4600194d0 tx=0x7fb460019420 comp rx=0 tx=0).stop 2024-08-02T18:00:43.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 -- 172.21.15.69:0/625669913 shutdown_connections 2024-08-02T18:00:43.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 --2- 172.21.15.69:0/625669913 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5b20 0x55f5a3fff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.212 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 --2- 172.21.15.69:0/625669913 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 0x55f5a4018130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 --2- 172.21.15.69:0/625669913 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5200 0x55f5a3ff7c60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 -- 172.21.15.69:0/625669913 >> 172.21.15.69:0/625669913 conn(0x55f5a3ff1790 msgr2=0x55f5a3ffeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:43.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 -- 172.21.15.69:0/625669913 shutdown_connections 2024-08-02T18:00:43.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 -- 172.21.15.69:0/625669913 wait complete. 2024-08-02T18:00:43.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 -- start start 2024-08-02T18:00:43.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.200+0000 7fb47b87a740 1 Processor -- start 2024-08-02T18:00:43.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a40085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5b20 0x55f5a4008b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb47b87a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 0x55f5a4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4727fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5b20 0x55f5a4008b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb472ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a40085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4727fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5b20 0x55f5a4008b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:48528/0 (socket says 172.21.15.69:48528) 2024-08-02T18:00:43.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb472ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a40085d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46158/0 (socket says 172.21.15.69:46158) 2024-08-02T18:00:43.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4727fc640 1 -- 172.21.15.69:0/2957587976 learned_addr learned my addr 172.21.15.69:0/2957587976 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741b3b50 con 0x55f5a3ff5200 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741b4fc0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb4741b4220 con 0x55f5a40003e0 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4737fe640 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 0x55f5a4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb47b87a740 10 client.0 _ll_register_callbacks cb 0x7fb46c012840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4727fc640 1 -- 172.21.15.69:0/2957587976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 msgr2=0x55f5a4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4727fc640 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 0x55f5a4003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4727fc640 1 -- 172.21.15.69:0/2957587976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 msgr2=0x55f5a40085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4727fc640 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a40085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb4727fc640 1 -- 172.21.15.69:0/2957587976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5a4004380 con 0x55f5a3ff5b20 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.201+0000 7fb472ffd640 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a40085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb4727fc640 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5b20 0x55f5a4008b10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4640ed260 tx=0x7fb4640bbbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=504ff70522864f6c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:43.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb4567fc640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb4567fc640 1 -- 172.21.15.69:0/2957587976 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464049040 con 0x55f5a3ff5b20 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb4567fc640 1 -- 172.21.15.69:0/2957587976 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fb4640b86f0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4600035f0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb4567fc640 1 -- 172.21.15.69:0/2957587976 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb4640bc080 con 0x55f5a3ff5b20 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb47b87a740 10 client.28403 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f5a407ad80 con 0x55f5a3ff5b20 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb4567fc640 1 -- 172.21.15.69:0/2957587976 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4600035f0 con 0x55f5a3ff5b20 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.202+0000 7fb4567fc640 1 -- 172.21.15.69:0/2957587976 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb4640f1040 con 0x55f5a3ff5b20 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb47b87a740 10 client.28403 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb47b87a740 10 client.28403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb47b87a740 20 client.28403 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb47b87a740 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f5a40103f0 0x55f5a407d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb47b87a740 4 client.28403 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T18:00:43.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f5a3f62270 con 0x55f5a40103f0 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb472ffd640 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f5a40103f0 0x55f5a407d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb472ffd640 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f5a40103f0 0x55f5a407d570 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e8af947d1c051d server_cookie=8da50c8b67235fa9 in_seq=0 out_seq=0 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.203+0000 7fb4567fc640 10 client.28403 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb4567fc640 1 -- 172.21.15.69:0/2957587976 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f5a3f62270 con 0x55f5a40103f0 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb4567fc640 10 client.28403 handle_command_reply: tid=0 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 client.28403 shutdown 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5b20 msgr2=0x55f5a4008b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5b20 0x55f5a4008b10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb4640ed260 tx=0x7fb4640bbbc0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 shutdown_connections 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f5a40103f0 0x55f5a407d570 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f5a40003e0 0x55f5a4003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f5a3ff5b20 0x55f5a4008b10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 --2- 172.21.15.69:0/2957587976 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f5a3ff5200 0x55f5a40085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 >> 172.21.15.69:0/2957587976 conn(0x55f5a3ff1790 msgr2=0x55f5a3ffda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb4567fc640 0 client.28403 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.204+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 shutdown_connections 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.205+0000 7fb47b87a740 1 -- 172.21.15.69:0/2957587976 wait complete. 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.205+0000 7fb47b87a740 20 client.28403 trim_cache size 0 max 16384 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "22e86948-7309-4251-9980-e96097231d09", 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T18:00:43.218 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.205+0000 7fb47947c640 1 -- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4580a2cf0 msgr2=0x7fb4580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.205+0000 7fb47947c640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4580a2cf0 0x7fb4580a51b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb46c009fd0 tx=0x7fb46c009500 comp rx=0 tx=0).stop 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.205+0000 7fb47947c640 1 -- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 msgr2=0x7fb4741a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.205+0000 7fb47947c640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 0x7fb4741a4ad0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb464006010 tx=0x7fb464003c10 comp rx=0 tx=0).stop 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.206+0000 7fb47947c640 1 -- 172.21.15.69:0/1021001058 shutdown_connections 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.206+0000 7fb47947c640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fb474100f10 0x7fb4741a4ad0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.206+0000 7fb47947c640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fb4580a2cf0 0x7fb4580a51b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.206+0000 7fb47947c640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fb474105bf0 0x7fb47419e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.206+0000 7fb47947c640 1 --2- 172.21.15.69:0/1021001058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb4741005f0 0x7fb4741a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:43.220 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.206+0000 7fb47947c640 1 -- 172.21.15.69:0/1021001058 >> 172.21.15.69:0/1021001058 conn(0x7fb4740fc500 msgr2=0x7fb47410f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:43.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.206+0000 7fb47947c640 1 -- 172.21.15.69:0/1021001058 shutdown_connections 2024-08-02T18:00:43.221 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:43.206+0000 7fb47947c640 1 -- 172.21.15.69:0/1021001058 wait complete. 2024-08-02T18:00:43.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:43 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T18:00:43.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:43 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:43.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:43 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T18:00:43.960 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T18:00:44.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:43 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T18:00:44.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:43 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:44.022 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:43 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T18:00:44.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:43 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T18:00:44.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:43 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:44.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:43 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T18:00:44.405 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:44.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:44 smithi069 ceph-mon[32746]: pgmap v448: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 573 op/s 2024-08-02T18:00:44.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:44 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:44.813 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:44 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T18:00:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 815.0 s, 537.0 tps, lat 184.777 ms stddev 226.238 2024-08-02T18:00:44.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.950+0000 7fa3e66f4640 1 -- 172.21.15.69:0/1004557789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e0109560 msgr2=0x7fa3e01099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:44.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.950+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/1004557789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e0109560 0x7fa3e01099c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4009900 tx=0x7fa3d402a440 comp rx=0 tx=0).stop 2024-08-02T18:00:44.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.958+0000 7fa3e66f4640 1 -- 172.21.15.69:0/1004557789 shutdown_connections 2024-08-02T18:00:44.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.958+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/1004557789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3e011ae70 0x7fa3e011d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:44.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.958+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/1004557789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e0109560 0x7fa3e01099c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:44.963 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.958+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/1004557789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e0109180 0x7fa3e010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:44.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.958+0000 7fa3e66f4640 1 -- 172.21.15.69:0/1004557789 >> 172.21.15.69:0/1004557789 conn(0x7fa3e006de70 msgr2=0x7fa3e006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:44.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.964+0000 7fa3e66f4640 1 -- 172.21.15.69:0/1004557789 shutdown_connections 2024-08-02T18:00:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.965+0000 7fa3e66f4640 1 -- 172.21.15.69:0/1004557789 wait complete. 2024-08-02T18:00:44.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.966+0000 7fa3e66f4640 1 Processor -- start 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.973+0000 7fa3e66f4640 1 -- start start 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.973+0000 7fa3e66f4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e0109180 0x7fa3e01a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.973+0000 7fa3e66f4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3e0109560 0x7fa3e01a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.973+0000 7fa3e66f4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e011ae70 0x7fa3e01ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.973+0000 7fa3e66f4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0120c00 con 0x7fa3e0109560 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.973+0000 7fa3e66f4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0120a80 con 0x7fa3e011ae70 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.973+0000 7fa3e66f4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0120d80 con 0x7fa3e0109180 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.974+0000 7fa3df7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3e0109560 0x7fa3e01a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.974+0000 7fa3df7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3e0109560 0x7fa3e01a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34856/0 (socket says 172.21.15.69:34856) 2024-08-02T18:00:44.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.974+0000 7fa3df7fe640 1 -- 172.21.15.69:0/2143201607 learned_addr learned my addr 172.21.15.69:0/2143201607 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:44.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.974+0000 7fa3e4c91640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e011ae70 0x7fa3e01ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:44.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.979+0000 7fa3dffff640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e0109180 0x7fa3e01a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:44.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.982+0000 7fa3e4c91640 1 -- 172.21.15.69:0/2143201607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e0109180 msgr2=0x7fa3e01a7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:44.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.982+0000 7fa3e4c91640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e0109180 0x7fa3e01a7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:44.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.982+0000 7fa3e4c91640 1 -- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3e0109560 msgr2=0x7fa3e01a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:44.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.982+0000 7fa3e4c91640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3e0109560 0x7fa3e01a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:44.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.982+0000 7fa3e4c91640 1 -- 172.21.15.69:0/2143201607 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e01ac050 con 0x7fa3e011ae70 2024-08-02T18:00:44.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.985+0000 7fa3e4c91640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e011ae70 0x7fa3e01ab950 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800be00 tx=0x7fa3d800bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.987+0000 7fa3dd7fa640 1 -- 172.21.15.69:0/2143201607 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8019070 con 0x7fa3e011ae70 2024-08-02T18:00:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.987+0000 7fa3dd7fa640 1 -- 172.21.15.69:0/2143201607 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d800cd90 con 0x7fa3e011ae70 2024-08-02T18:00:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.987+0000 7fa3dd7fa640 1 -- 172.21.15.69:0/2143201607 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80128b0 con 0x7fa3e011ae70 2024-08-02T18:00:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.990+0000 7fa3e66f4640 1 -- 172.21.15.69:0/2143201607 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01ac340 con 0x7fa3e011ae70 2024-08-02T18:00:44.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.990+0000 7fa3e66f4640 1 -- 172.21.15.69:0/2143201607 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e0110650 con 0x7fa3e011ae70 2024-08-02T18:00:45.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:44.999+0000 7fa3e8af2740 1 Processor -- start 2024-08-02T18:00:45.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.000+0000 7fa3dd7fa640 1 -- 172.21.15.69:0/2143201607 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa3d8012a50 con 0x7fa3e011ae70 2024-08-02T18:00:45.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.001+0000 7fa3dd7fa640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa3c80a2f00 0x7fa3c80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.001+0000 7fa3e8af2740 1 -- start start 2024-08-02T18:00:45.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.002+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9210 0x56130bca95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.002+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9b30 0x56130bcb3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.002+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bcb67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.002+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56130bc0e980 con 0x56130bca9210 2024-08-02T18:00:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.002+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56130bba6a70 con 0x56130bcb43f0 2024-08-02T18:00:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.002+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56130bc0a940 con 0x56130bca9b30 2024-08-02T18:00:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.002+0000 7fa3e4c91640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bcb67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.002+0000 7fa3dd7fa640 1 -- 172.21.15.69:0/2143201607 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fa3d80cedc0 con 0x7fa3e011ae70 2024-08-02T18:00:45.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.007+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9210 0x56130bca95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.007+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9210 0x56130bca95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34868/0 (socket says 172.21.15.69:34868) 2024-08-02T18:00:45.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.007+0000 7fa3dffff640 1 -- 172.21.15.69:0/3600145557 learned_addr learned my addr 172.21.15.69:0/3600145557 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:45.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.009+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/3600145557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9b30 0x56130bcb3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.009+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa3c80a2f00 0x7fa3c80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.011+0000 7fa3df7fe640 1 -- 172.21.15.69:0/3600145557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 msgr2=0x56130bcb67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.011+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/3600145557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bcb67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.011+0000 7fa3df7fe640 1 -- 172.21.15.69:0/3600145557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9210 msgr2=0x56130bca95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.011+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/3600145557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9210 0x56130bca95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.011+0000 7fa3df7fe640 1 -- 172.21.15.69:0/3600145557 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130bc096e0 con 0x56130bca9b30 2024-08-02T18:00:45.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.012+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/3600145557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9b30 0x56130bcb3eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4000c00 tx=0x7fa3d403d710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eedccc2a88508aef server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.016+0000 7fa3c67fc640 1 -- 172.21.15.69:0/3600145557 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4035070 con 0x56130bca9b30 2024-08-02T18:00:45.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.016+0000 7fa3c67fc640 1 -- 172.21.15.69:0/3600145557 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d404f070 con 0x56130bca9b30 2024-08-02T18:00:45.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.016+0000 7fa3c67fc640 1 -- 172.21.15.69:0/3600145557 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d40493e0 con 0x56130bca9b30 2024-08-02T18:00:45.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.016+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa3c80a2f00 0x7fa3c80a53c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4031920 tx=0x7fa3d4044000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.018+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3600145557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9b30 msgr2=0x56130bcb3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.019 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.018+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3600145557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9b30 0x56130bcb3eb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4000c00 tx=0x7fa3d403d710 comp rx=0 tx=0).stop 2024-08-02T18:00:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:44 smithi083 ceph-mon[37031]: pgmap v448: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 573 op/s 2024-08-02T18:00:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:44 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:45.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:44 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T18:00:45.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.024+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3600145557 shutdown_connections 2024-08-02T18:00:45.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.024+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3600145557 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bcb67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.026 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.024+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3600145557 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9b30 0x56130bcb3eb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.024+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3600145557 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9210 0x56130bca95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.024+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3600145557 >> 172.21.15.69:0/3600145557 conn(0x56130bca57a0 msgr2=0x56130bca5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:45.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.025+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3600145557 shutdown_connections 2024-08-02T18:00:45.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.025+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3600145557 wait complete. 2024-08-02T18:00:45.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 1 -- start start 2024-08-02T18:00:45.029 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 1 Processor -- start 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcb1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9b30 0x56130bcb1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bcb1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56130bcb6fc0 con 0x56130bca9b30 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56130bbc9aa0 con 0x56130bcb43f0 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56130bcb7140 con 0x56130bca9210 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.026+0000 7fa3e8af2740 10 client.0 _ll_register_callbacks cb 0x56130bbb8e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.027+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcb1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.027+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcb1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35370/0 (socket says 172.21.15.69:35370) 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.027+0000 7fa3dffff640 1 -- 172.21.15.69:0/979933741 learned_addr learned my addr 172.21.15.69:0/979933741 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.027+0000 7fa3dffff640 1 -- 172.21.15.69:0/979933741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 msgr2=0x56130bcb1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.027+0000 7fa3dffff640 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bcb1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.027+0000 7fa3dffff640 1 -- 172.21.15.69:0/979933741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9b30 msgr2=0x56130bcb1890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:00:45.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.027+0000 7fa3dffff640 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9b30 0x56130bcb1890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.027+0000 7fa3dffff640 1 -- 172.21.15.69:0/979933741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130bbad650 con 0x56130bca9210 2024-08-02T18:00:45.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.028+0000 7fa3dffff640 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcb1350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0011560 tx=0x7fa3d0011a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8c1bd4fcbe417f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.028+0000 7fa3bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:45.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.029+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/979933741 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d000f6d0 con 0x56130bca9210 2024-08-02T18:00:45.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.030+0000 7fa3e8af2740 1 -- 172.21.15.69:0/979933741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d404f070 con 0x56130bca9210 2024-08-02T18:00:45.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.030+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/979933741 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fa3d000f870 con 0x56130bca9210 2024-08-02T18:00:45.036 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.035+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/979933741 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa3d0026050 con 0x56130bca9210 2024-08-02T18:00:45.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.035+0000 7fa3e8af2740 10 client.28168 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:45.038 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.035+0000 7fa3e8af2740 1 -- 172.21.15.69:0/979933741 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56130b9803c0 con 0x56130bca9210 2024-08-02T18:00:45.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.038+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/979933741 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d404f070 con 0x56130bca9210 2024-08-02T18:00:45.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.038+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/979933741 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa3d00133a0 con 0x56130bca9210 2024-08-02T18:00:45.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.038+0000 7fa3e8af2740 10 client.28168 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:45.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.038+0000 7fa3e8af2740 10 client.28168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:45.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.038+0000 7fa3e8af2740 20 client.28168 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:45.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.038+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56130bcc4420 0x56130bcc68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.039+0000 7fa3e8af2740 4 client.28168 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:45.041 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.039+0000 7fa3e8af2740 1 -- 172.21.15.69:0/979933741 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56130bc1bef0 con 0x56130bcc4420 2024-08-02T18:00:45.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.040+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56130bcc4420 0x56130bcc68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.041+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56130bcc4420 0x56130bcc68e0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abbba13d27c18ffa server_cookie=18a04c8a25815a73 in_seq=0 out_seq=0 2024-08-02T18:00:45.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.042+0000 7fa3bf7fe640 10 client.28168 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:45.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.042+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/979933741 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56130bc1bef0 con 0x56130bcc4420 2024-08-02T18:00:45.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.043+0000 7fa3bf7fe640 10 client.28168 handle_command_reply: tid=0 2024-08-02T18:00:45.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.043+0000 7fa3e8af2740 1 client.28168 shutdown 2024-08-02T18:00:45.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.044+0000 7fa3e8af2740 1 -- 172.21.15.69:0/979933741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 msgr2=0x56130bcb1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.044+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcb1350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0011560 tx=0x7fa3d0011a20 comp rx=0 tx=0).stop 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.045+0000 7fa3e8af2740 1 -- 172.21.15.69:0/979933741 shutdown_connections 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.045+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9b30 0x56130bcb1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.045+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56130bcc4420 0x56130bcc68e0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.045+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bcb1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.045+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/979933741 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcb1350 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.045+0000 7fa3e8af2740 1 -- 172.21.15.69:0/979933741 >> 172.21.15.69:0/979933741 conn(0x56130bca57a0 msgr2=0x56130bca4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.045+0000 7fa3bf7fe640 0 client.28168 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.045+0000 7fa3e8af2740 1 -- 172.21.15.69:0/979933741 shutdown_connections 2024-08-02T18:00:45.047 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.046+0000 7fa3e8af2740 1 -- 172.21.15.69:0/979933741 wait complete. 2024-08-02T18:00:45.048 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.047+0000 7fa3e8af2740 20 client.28168 trim_cache size 0 max 16384 2024-08-02T18:00:45.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.079+0000 7fa3e8af2740 1 Processor -- start 2024-08-02T18:00:45.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.079+0000 7fa3e8af2740 1 -- start start 2024-08-02T18:00:45.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.080+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bca5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.080+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9b30 0x56130bcb31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.080+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bccc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.081+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01c8eb0 con 0x56130bca9b30 2024-08-02T18:00:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.081+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bca5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.081+0000 7fa3dffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bca5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35376/0 (socket says 172.21.15.69:35376) 2024-08-02T18:00:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.081+0000 7fa3dffff640 1 -- 172.21.15.69:0/2834812928 learned_addr learned my addr 172.21.15.69:0/2834812928 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.081+0000 7fa3e4c91640 1 --2- 172.21.15.69:0/2834812928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bccc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.081+0000 7fa3e8af2740 1 -- 172.21.15.69:0/2834812928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01c6780 con 0x56130bcb43f0 2024-08-02T18:00:45.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.081+0000 7fa3e8af2740 1 -- 172.21.15.69:0/2834812928 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0118820 con 0x56130bca9210 2024-08-02T18:00:45.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.082+0000 7fa3e4c91640 1 -- 172.21.15.69:0/2834812928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 msgr2=0x56130bca5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.082+0000 7fa3e4c91640 1 --2- 172.21.15.69:0/2834812928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bca5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.082+0000 7fa3e4c91640 1 -- 172.21.15.69:0/2834812928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9b30 msgr2=0x56130bcb31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.082+0000 7fa3e4c91640 1 --2- 172.21.15.69:0/2834812928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9b30 0x56130bcb31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.082+0000 7fa3e4c91640 1 -- 172.21.15.69:0/2834812928 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56130bcb0300 con 0x56130bcb43f0 2024-08-02T18:00:45.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.083+0000 7fa3e4c91640 1 --2- 172.21.15.69:0/2834812928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bccc1f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cc1d0 tx=0x7fa3d8096bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2887a1f721751d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.086+0000 7fa3c5ffb640 1 -- 172.21.15.69:0/2834812928 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80d0030 con 0x56130bcb43f0 2024-08-02T18:00:45.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.087+0000 7fa3c5ffb640 1 -- 172.21.15.69:0/2834812928 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d809b050 con 0x56130bcb43f0 2024-08-02T18:00:45.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.087+0000 7fa3c5ffb640 1 -- 172.21.15.69:0/2834812928 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8012e60 con 0x56130bcb43f0 2024-08-02T18:00:45.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.088+0000 7fa3e8af2740 1 -- 172.21.15.69:0/2834812928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 msgr2=0x56130bccc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.088+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/2834812928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bccc1f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cc1d0 tx=0x7fa3d8096bd0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.088+0000 7fa3e8af2740 1 -- 172.21.15.69:0/2834812928 shutdown_connections 2024-08-02T18:00:45.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.088+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/2834812928 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bca9b30 0x56130bcb31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.088+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/2834812928 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bcb43f0 0x56130bccc1f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.088+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/2834812928 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bca5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.088+0000 7fa3e8af2740 1 -- 172.21.15.69:0/2834812928 >> 172.21.15.69:0/2834812928 conn(0x56130bca57a0 msgr2=0x56130bcb2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:45.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.095+0000 7fa3e8af2740 1 -- 172.21.15.69:0/2834812928 shutdown_connections 2024-08-02T18:00:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:44 smithi077 ceph-mon[37108]: pgmap v448: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.6 MiB/s wr, 573 op/s 2024-08-02T18:00:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:44 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:00:45.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:44 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T18:00:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.098+0000 7fa3e8af2740 1 -- 172.21.15.69:0/2834812928 wait complete. 2024-08-02T18:00:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.098+0000 7fa3e8af2740 1 -- start start 2024-08-02T18:00:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.098+0000 7fa3e8af2740 1 Processor -- start 2024-08-02T18:00:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.098+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcbc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bca9b30 0x56130bcadd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.101 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e8af2740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bcb43f0 0x56130bcae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01c4c60 con 0x56130bcb43f0 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01c60b0 con 0x56130bca9b30 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e8af2740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01c5310 con 0x56130bca9210 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e8af2740 10 client.0 _ll_register_callbacks cb 0x7fa3d00095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e4c91640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bcb43f0 0x56130bcae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e4c91640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bcb43f0 0x56130bcae250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34892/0 (socket says 172.21.15.69:34892) 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.099+0000 7fa3e4c91640 1 -- 172.21.15.69:0/3199762871 learned_addr learned my addr 172.21.15.69:0/3199762871 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.100+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bca9b30 0x56130bcadd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.102 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.100+0000 7fa3dffff640 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcbc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.100+0000 7fa3df7fe640 1 -- 172.21.15.69:0/3199762871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 msgr2=0x56130bcbc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.100+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcbc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.100+0000 7fa3df7fe640 1 -- 172.21.15.69:0/3199762871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bcb43f0 msgr2=0x56130bcae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.100+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bcb43f0 0x56130bcae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.100+0000 7fa3df7fe640 1 -- 172.21.15.69:0/3199762871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56130bcae790 con 0x56130bca9b30 2024-08-02T18:00:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.100+0000 7fa3df7fe640 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bca9b30 0x56130bcadd10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x56130bcb0be0 tx=0x7fa3d4041040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b07925e70187142 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.101+0000 7fa3bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:00:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.101+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/3199762871 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4035040 con 0x56130bca9b30 2024-08-02T18:00:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.101+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/3199762871 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fa3d404a690 con 0x56130bca9b30 2024-08-02T18:00:45.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.101+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3199762871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d809b050 con 0x56130bca9b30 2024-08-02T18:00:45.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.105+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/3199762871 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa3d406e050 con 0x56130bca9b30 2024-08-02T18:00:45.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.108+0000 7fa3e8af2740 10 client.28418 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:45.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.109+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3199762871 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56130bd2eee0 con 0x56130bca9b30 2024-08-02T18:00:45.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.110+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/3199762871 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d809b050 con 0x56130bca9b30 2024-08-02T18:00:45.111 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.110+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/3199762871 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa3d403a040 con 0x56130bca9b30 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.112+0000 7fa3e8af2740 10 client.28418 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.112+0000 7fa3e8af2740 10 client.28418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.112+0000 7fa3e8af2740 20 client.28418 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.112+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56130bd2f170 0x56130bd31560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.112+0000 7fa3e8af2740 4 client.28418 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.112+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3199762871 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56130bc16400 con 0x56130bd2f170 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.112+0000 7fa3dffff640 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56130bd2f170 0x56130bd31560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.114+0000 7fa3dffff640 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56130bd2f170 0x56130bd31560 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0a4960b287a292 server_cookie=fd46d5a275cfbe68 in_seq=0 out_seq=0 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.114+0000 7fa3bf7fe640 10 client.28418 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:45.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.114+0000 7fa3bf7fe640 1 -- 172.21.15.69:0/3199762871 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56130bc16400 con 0x56130bd2f170 2024-08-02T18:00:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.114+0000 7fa3bf7fe640 10 client.28418 handle_command_reply: tid=0 2024-08-02T18:00:45.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.114+0000 7fa3e8af2740 1 client.28418 shutdown 2024-08-02T18:00:45.118 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.117+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bca9b30 msgr2=0x56130bcadd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.117+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bca9b30 0x56130bcadd10 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x56130bcb0be0 tx=0x7fa3d4041040 comp rx=0 tx=0).stop 2024-08-02T18:00:45.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.117+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3199762871 shutdown_connections 2024-08-02T18:00:45.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.117+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56130bcb43f0 0x56130bcae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.117+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56130bd2f170 0x56130bd31560 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.118+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56130bca9b30 0x56130bcadd10 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.118+0000 7fa3e8af2740 1 --2- 172.21.15.69:0/3199762871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56130bca9210 0x56130bcbc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.118+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3199762871 >> 172.21.15.69:0/3199762871 conn(0x56130bca57a0 msgr2=0x56130bcb31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:45.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.118+0000 7fa3bf7fe640 0 client.28418 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.122+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3199762871 shutdown_connections 2024-08-02T18:00:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.122+0000 7fa3e8af2740 1 -- 172.21.15.69:0/3199762871 wait complete. 2024-08-02T18:00:45.123 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.122+0000 7fa3e8af2740 20 client.28418 trim_cache size 0 max 16384 2024-08-02T18:00:45.124 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:00:45.124 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T18:00:45.124 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T18:00:45.124 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 -- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa3c80a2f00 msgr2=0x7fa3c80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa3c80a2f00 0x7fa3c80a53c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4031920 tx=0x7fa3d4044000 comp rx=0 tx=0).stop 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 -- 172.21.15.69:0/2143201607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e011ae70 msgr2=0x7fa3e01ab950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e011ae70 0x7fa3e01ab950 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800be00 tx=0x7fa3d800bef0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 -- 172.21.15.69:0/2143201607 shutdown_connections 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa3e0109560 0x7fa3e01a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa3c80a2f00 0x7fa3c80a53c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa3e011ae70 0x7fa3e01ab950 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.125 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 --2- 172.21.15.69:0/2143201607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3e0109180 0x7fa3e01a7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 -- 172.21.15.69:0/2143201607 >> 172.21.15.69:0/2143201607 conn(0x7fa3e006de70 msgr2=0x7fa3e011b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:45.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 -- 172.21.15.69:0/2143201607 shutdown_connections 2024-08-02T18:00:45.126 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:45.123+0000 7fa3e66f4640 1 -- 172.21.15.69:0/2143201607 wait complete. 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/2166853438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f81010a0 msgr2=0x7fa6f810d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/2166853438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f81010a0 0x7fa6f810d5d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0009cd0 tx=0x7fa6f002a8e0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/2166853438 shutdown_connections 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/2166853438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f81010a0 0x7fa6f810d5d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/2166853438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f8100700 0x7fa6f8100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/2166853438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6f8106710 0x7fa6f8106af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/2166853438 >> 172.21.15.69:0/2166853438 conn(0x7fa6f80fc3e0 msgr2=0x7fa6f80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/2166853438 shutdown_connections 2024-08-02T18:00:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.863+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/2166853438 wait complete. 2024-08-02T18:00:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.864+0000 7fa6ff9d7640 1 Processor -- start 2024-08-02T18:00:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.864+0000 7fa6ff9d7640 1 -- start start 2024-08-02T18:00:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.864+0000 7fa6ff9d7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f8100700 0x7fa6f81a4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.864+0000 7fa6ff9d7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81010a0 0x7fa6f81a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.864+0000 7fa6ff9d7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6f8106710 0x7fa6f819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.864+0000 7fa6ff9d7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8110ba0 con 0x7fa6f8100700 2024-08-02T18:00:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.864+0000 7fa6ff9d7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8110a20 con 0x7fa6f8106710 2024-08-02T18:00:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.864+0000 7fa6ff9d7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f8110d20 con 0x7fa6f81010a0 2024-08-02T18:00:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.865+0000 7fa6fcf72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81010a0 0x7fa6f81a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.865+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f8100700 0x7fa6f81a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.865+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f8100700 0x7fa6f81a4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34912/0 (socket says 172.21.15.69:34912) 2024-08-02T18:00:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.865+0000 7fa6fd773640 1 -- 172.21.15.69:0/1413462607 learned_addr learned my addr 172.21.15.69:0/1413462607 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.865+0000 7fa6fdf74640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6f8106710 0x7fa6f819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6fd773640 1 -- 172.21.15.69:0/1413462607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81010a0 msgr2=0x7fa6f81a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6fd773640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81010a0 0x7fa6f81a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6fd773640 1 -- 172.21.15.69:0/1413462607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6f8106710 msgr2=0x7fa6f819e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6fd773640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6f8106710 0x7fa6f819e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6fd773640 1 -- 172.21.15.69:0/1413462607 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f819f0e0 con 0x7fa6f8100700 2024-08-02T18:00:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6fd773640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f8100700 0x7fa6f81a4410 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400ea60 tx=0x7fa6f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6ee7fc640 1 -- 172.21.15.69:0/1413462607 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f400cde0 con 0x7fa6f8100700 2024-08-02T18:00:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6ee7fc640 1 -- 172.21.15.69:0/1413462607 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f40035f0 con 0x7fa6f8100700 2024-08-02T18:00:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/1413462607 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f819f3d0 con 0x7fa6f8100700 2024-08-02T18:00:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.866+0000 7fa6ee7fc640 1 -- 172.21.15.69:0/1413462607 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4010670 con 0x7fa6f8100700 2024-08-02T18:00:45.869 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.867+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/1413462607 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f81ab3d0 con 0x7fa6f8100700 2024-08-02T18:00:45.870 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.869+0000 7fa6ee7fc640 1 -- 172.21.15.69:0/1413462607 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa6f4010810 con 0x7fa6f8100700 2024-08-02T18:00:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.870+0000 7fa6ee7fc640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa6e40a2d70 0x7fa6e40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.871 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.870+0000 7fa702806740 1 Processor -- start 2024-08-02T18:00:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.870+0000 7fa6ee7fc640 1 -- 172.21.15.69:0/1413462607 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fa6f40cfb50 con 0x7fa6f8100700 2024-08-02T18:00:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.870+0000 7fa702806740 1 -- start start 2024-08-02T18:00:45.872 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.870+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa6e40a2d70 0x7fa6e40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa702806740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0719b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa702806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be071a0d0 0x564be0724430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa702806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be0724970 0x564be0726d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa702806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564be067d9b0 con 0x564be0724970 2024-08-02T18:00:45.873 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa702806740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564be0625800 con 0x564be07197b0 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa6e40a2d70 0x7fa6e40a5230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa6f810daa0 tx=0x7fa6e8005c60 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa702806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564be0679970 con 0x564be071a0d0 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0719b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fdf74640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be0724970 0x564be0726d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0719b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60768/0 (socket says 172.21.15.69:60768) 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 -- 172.21.15.69:0/3737490206 learned_addr learned my addr 172.21.15.69:0/3737490206 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be071a0d0 0x564be0724430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 -- 172.21.15.69:0/3737490206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be071a0d0 msgr2=0x564be0724430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be071a0d0 0x564be0724430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 -- 172.21.15.69:0/3737490206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be0724970 msgr2=0x564be0726d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be0724970 0x564be0726d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 -- 172.21.15.69:0/3737490206 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be0391210 con 0x564be07197b0 2024-08-02T18:00:45.874 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be071a0d0 0x564be0724430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.871+0000 7fa6fd773640 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0719b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4000c00 tx=0x7fa6f409bb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34b230706a81e3f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.872+0000 7fa6e37fe640 1 -- 172.21.15.69:0/3737490206 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40a0010 con 0x564be07197b0 2024-08-02T18:00:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.872+0000 7fa6e37fe640 1 -- 172.21.15.69:0/3737490206 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f40b1030 con 0x564be07197b0 2024-08-02T18:00:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.872+0000 7fa6e37fe640 1 -- 172.21.15.69:0/3737490206 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40982d0 con 0x564be07197b0 2024-08-02T18:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 -- 172.21.15.69:0/3737490206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 msgr2=0x564be0719b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0719b90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4000c00 tx=0x7fa6f409bb60 comp rx=0 tx=0).stop 2024-08-02T18:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 -- 172.21.15.69:0/3737490206 shutdown_connections 2024-08-02T18:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be0724970 0x564be0726d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be071a0d0 0x564be0724430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 --2- 172.21.15.69:0/3737490206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0719b90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 -- 172.21.15.69:0/3737490206 >> 172.21.15.69:0/3737490206 conn(0x564be0715d20 msgr2=0x564be0716100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 -- 172.21.15.69:0/3737490206 shutdown_connections 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 -- 172.21.15.69:0/3737490206 wait complete. 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 -- start start 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.873+0000 7fa702806740 1 Processor -- start 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.874+0000 7fa702806740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0721870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa702806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be0721db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa702806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 0x564be07222f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0721870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0721870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60778/0 (socket says 172.21.15.69:60778) 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa6fd773640 1 -- 172.21.15.69:0/2134067396 learned_addr learned my addr 172.21.15.69:0/2134067396 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be0721db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x564be0727540 con 0x564be071a0d0 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x564be0639500 con 0x564be07197b0 2024-08-02T18:00:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.875+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x564be07276c0 con 0x564be0724970 2024-08-02T18:00:45.888 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.876+0000 7fa6fcf72640 1 -- 172.21.15.69:0/2134067396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 msgr2=0x564be07222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.876+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 0x564be07222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.876+0000 7fa6fcf72640 1 -- 172.21.15.69:0/2134067396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 msgr2=0x564be0721870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.876+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0721870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.876+0000 7fa6fcf72640 1 -- 172.21.15.69:0/2134067396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be045d620 con 0x564be071a0d0 2024-08-02T18:00:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.876+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be0721db0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8005fd0 tx=0x7fa6e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ce7b98acf062d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.877+0000 7fa6e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.877+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/2134067396 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e801cd20 con 0x564be071a0d0 2024-08-02T18:00:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.877+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/2134067396 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fa6e8005020 con 0x564be071a0d0 2024-08-02T18:00:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.877+0000 7fa6e0ff9640 10 client.0 inactive, discarding osd_map(111..111 src has 1..111) 2024-08-02T18:00:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.887+0000 7fa702806740 10 client.0 _ll_register_callbacks cb 0x564be0390a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.887+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f40b1030 con 0x564be071a0d0 2024-08-02T18:00:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.890+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/2134067396 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa6e8005220 con 0x564be071a0d0 2024-08-02T18:00:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.899+0000 7fa702806740 10 client.19050 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:45.900 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.899+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({fsmap=26}) -- 0x564be0733020 con 0x564be071a0d0 2024-08-02T18:00:45.901 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.899+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/2134067396 <== mon.0 v2:172.21.15.69:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa6e80053c0 con 0x564be071a0d0 2024-08-02T18:00:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.903+0000 7fa702806740 10 client.19050 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.903+0000 7fa702806740 10 client.19050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.904+0000 7fa702806740 20 client.19050 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.904+0000 7fa702806740 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564be0734420 0x564be07368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.904+0000 7fa702806740 4 client.19050 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.904+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564be068b0e0 con 0x564be0734420 2024-08-02T18:00:45.913 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.910+0000 7fa6fd773640 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564be0734420 0x564be07368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.917+0000 7fa6fd773640 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564be0734420 0x564be07368e0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4378523312d9278c server_cookie=2679c72cf02044c6 in_seq=0 out_seq=0 2024-08-02T18:00:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.917+0000 7fa6e0ff9640 10 client.19050 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.919+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/2134067396 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564be068b0e0 con 0x564be0734420 2024-08-02T18:00:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.919+0000 7fa6e0ff9640 10 client.19050 handle_command_reply: tid=0 2024-08-02T18:00:45.921 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.920+0000 7fa702806740 1 client.19050 shutdown 2024-08-02T18:00:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.934+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 msgr2=0x564be0721db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:45.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.934+0000 7fa702806740 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be0721db0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8005fd0 tx=0x7fa6e80046c0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.936+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 shutdown_connections 2024-08-02T18:00:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.936+0000 7fa702806740 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be0721db0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.936+0000 7fa702806740 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564be0734420 0x564be07368e0 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.937 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.936+0000 7fa702806740 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 0x564be07222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.936+0000 7fa702806740 1 --2- 172.21.15.69:0/2134067396 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be07197b0 0x564be0721870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.938 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.936+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 >> 172.21.15.69:0/2134067396 conn(0x564be0715d20 msgr2=0x564be071ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:45.940 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.939+0000 7fa6e0ff9640 0 client.19050 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.940+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 shutdown_connections 2024-08-02T18:00:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.940+0000 7fa702806740 1 -- 172.21.15.69:0/2134067396 wait complete. 2024-08-02T18:00:45.944 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.942+0000 7fa702806740 20 client.19050 trim_cache size 0 max 16384 2024-08-02T18:00:45.963 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T18:00:45.964 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.980+0000 7fa702806740 1 Processor -- start 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.980+0000 7fa702806740 1 -- start start 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa702806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be07197b0 0x564be07236d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa702806740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be071a0d0 0x564be073c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be07197b0 0x564be07236d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa702806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 0x564be073c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be07197b0 0x564be07236d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34944/0 (socket says 172.21.15.69:34944) 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa6fd773640 1 -- 172.21.15.69:0/2161261325 learned_addr learned my addr 172.21.15.69:0/2161261325 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa702806740 1 -- 172.21.15.69:0/2161261325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81b7ee0 con 0x564be07197b0 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa702806740 1 -- 172.21.15.69:0/2161261325 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81b57b0 con 0x564be071a0d0 2024-08-02T18:00:45.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa702806740 1 -- 172.21.15.69:0/2161261325 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81b3600 con 0x564be0724970 2024-08-02T18:00:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa6fdf74640 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 0x564be073c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.981+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be071a0d0 0x564be073c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:45.984 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.983+0000 7fa6fd773640 1 -- 172.21.15.69:0/2161261325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 msgr2=0x564be073c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.983+0000 7fa6fd773640 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 0x564be073c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.983+0000 7fa6fd773640 1 -- 172.21.15.69:0/2161261325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be071a0d0 msgr2=0x564be073c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.983+0000 7fa6fd773640 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be071a0d0 0x564be073c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:45.985 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.983+0000 7fa6fd773640 1 -- 172.21.15.69:0/2161261325 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564be0734650 con 0x564be07197b0 2024-08-02T18:00:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:45.994+0000 7fa6fd773640 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be07197b0 0x564be07236d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4098070 tx=0x7fa6f409cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317bff57c1828d7f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.002+0000 7fa6e2ffd640 1 -- 172.21.15.69:0/2161261325 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40b8070 con 0x564be07197b0 2024-08-02T18:00:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.002+0000 7fa6e2ffd640 1 -- 172.21.15.69:0/2161261325 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f4099f60 con 0x564be07197b0 2024-08-02T18:00:46.004 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.002+0000 7fa6e2ffd640 1 -- 172.21.15.69:0/2161261325 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40b4e60 con 0x564be07197b0 2024-08-02T18:00:46.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.003+0000 7fa702806740 1 -- 172.21.15.69:0/2161261325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be07197b0 msgr2=0x564be07236d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.003+0000 7fa702806740 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be07197b0 0x564be07236d0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4098070 tx=0x7fa6f409cf90 comp rx=0 tx=0).stop 2024-08-02T18:00:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.004+0000 7fa702806740 1 -- 172.21.15.69:0/2161261325 shutdown_connections 2024-08-02T18:00:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.004+0000 7fa702806740 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0724970 0x564be073c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.004+0000 7fa702806740 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be071a0d0 0x564be073c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.004+0000 7fa702806740 1 --2- 172.21.15.69:0/2161261325 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be07197b0 0x564be07236d0 secure :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4098070 tx=0x7fa6f409cf90 comp rx=0 tx=0).stop 2024-08-02T18:00:46.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.004+0000 7fa702806740 1 -- 172.21.15.69:0/2161261325 >> 172.21.15.69:0/2161261325 conn(0x564be0715d20 msgr2=0x564be0722fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.005+0000 7fa702806740 1 -- 172.21.15.69:0/2161261325 shutdown_connections 2024-08-02T18:00:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.005+0000 7fa702806740 1 -- 172.21.15.69:0/2161261325 wait complete. 2024-08-02T18:00:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 1 -- start start 2024-08-02T18:00:46.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 1 Processor -- start 2024-08-02T18:00:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be07287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be0724970 0x564be0728ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0729220 0x564be07af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81b3c70 con 0x564be071a0d0 2024-08-02T18:00:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81b50e0 con 0x564be0724970 2024-08-02T18:00:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f81b4340 con 0x564be0729220 2024-08-02T18:00:46.009 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.006+0000 7fa702806740 10 client.0 _ll_register_callbacks cb 0x7fa6e800d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.007+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be07287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.007+0000 7fa6fd773640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be07287a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34960/0 (socket says 172.21.15.69:34960) 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.007+0000 7fa6fd773640 1 -- 172.21.15.69:0/672331567 learned_addr learned my addr 172.21.15.69:0/672331567 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.007+0000 7fa6fd773640 1 -- 172.21.15.69:0/672331567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0729220 msgr2=0x564be07af190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.007+0000 7fa6fd773640 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0729220 0x564be07af190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.007+0000 7fa6fd773640 1 -- 172.21.15.69:0/672331567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be0724970 msgr2=0x564be0728ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.007+0000 7fa6fd773640 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be0724970 0x564be0728ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.007+0000 7fa6fd773640 1 -- 172.21.15.69:0/672331567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564be07af720 con 0x564be071a0d0 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.008+0000 7fa6fd773640 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be07287a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4098630 tx=0x7fa6f4097f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48de1ad4b81cb0c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:46.012 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.008+0000 7fa6e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.008+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/672331567 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40b8040 con 0x564be071a0d0 2024-08-02T18:00:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.008+0000 7fa702806740 1 -- 172.21.15.69:0/672331567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6f4099f60 con 0x564be071a0d0 2024-08-02T18:00:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.008+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/672331567 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fa6f40982d0 con 0x564be071a0d0 2024-08-02T18:00:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.008+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/672331567 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa6f40b4e60 con 0x564be071a0d0 2024-08-02T18:00:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.011+0000 7fa702806740 10 client.19056 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:46.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.011+0000 7fa702806740 1 -- 172.21.15.69:0/672331567 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564be07973a0 con 0x564be071a0d0 2024-08-02T18:00:46.022 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.018+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/672331567 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6f4099f60 con 0x564be071a0d0 2024-08-02T18:00:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.018+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/672331567 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa6f409b0b0 con 0x564be071a0d0 2024-08-02T18:00:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.018+0000 7fa702806740 10 client.19056 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.018+0000 7fa702806740 10 client.19056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.018+0000 7fa702806740 20 client.19056 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.018+0000 7fa702806740 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564be0797a10 0x564be0799e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.018+0000 7fa702806740 4 client.19056 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:00:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.018+0000 7fa702806740 1 -- 172.21.15.69:0/672331567 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564be0685430 con 0x564be0797a10 2024-08-02T18:00:46.023 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.020+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564be0797a10 0x564be0799e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:46.039 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.038+0000 7fa6fcf72640 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564be0797a10 0x564be0799e50 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f478ca9a7e3bbd server_cookie=4386aaf1ee684a2b in_seq=0 out_seq=0 2024-08-02T18:00:46.041 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.039+0000 7fa6e0ff9640 10 client.19056 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa6e0ff9640 1 -- 172.21.15.69:0/672331567 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564be0685430 con 0x564be0797a10 2024-08-02T18:00:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa6e0ff9640 10 client.19056 handle_command_reply: tid=0 2024-08-02T18:00:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 client.19056 shutdown 2024-08-02T18:00:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 -- 172.21.15.69:0/672331567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 msgr2=0x564be07287a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be07287a0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4098630 tx=0x7fa6f4097f00 comp rx=0 tx=0).stop 2024-08-02T18:00:46.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 -- 172.21.15.69:0/672331567 shutdown_connections 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x564be0797a10 0x564be0799e50 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x564be0729220 0x564be07af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x564be0724970 0x564be0728ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa6e0ff9640 0 client.19056 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 --2- 172.21.15.69:0/672331567 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x564be071a0d0 0x564be07287a0 secure :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4098630 tx=0x7fa6f4097f00 comp rx=0 tx=0).stop 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 -- 172.21.15.69:0/672331567 >> 172.21.15.69:0/672331567 conn(0x564be0715d20 msgr2=0x564be071ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 -- 172.21.15.69:0/672331567 shutdown_connections 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 1 -- 172.21.15.69:0/672331567 wait complete. 2024-08-02T18:00:46.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.148+0000 7fa702806740 20 client.19056 trim_cache size 0 max 16384 2024-08-02T18:00:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:00:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:00:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:00:46.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa6e40a2d70 msgr2=0x7fa6e40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa6e40a2d70 0x7fa6e40a5230 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa6f810daa0 tx=0x7fa6e8005c60 comp rx=0 tx=0).stop 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f8100700 msgr2=0x7fa6f81a4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f8100700 0x7fa6f81a4410 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400ea60 tx=0x7fa6f400ef20 comp rx=0 tx=0).stop 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/1413462607 shutdown_connections 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa6e40a2d70 0x7fa6e40a5230 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa6f8106710 0x7fa6f819e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa6f81010a0 0x7fa6f81a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 --2- 172.21.15.69:0/1413462607 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa6f8100700 0x7fa6f81a4410 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.149+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/1413462607 >> 172.21.15.69:0/1413462607 conn(0x7fa6f80fc3e0 msgr2=0x7fa6f806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.150+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/1413462607 shutdown_connections 2024-08-02T18:00:46.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:46.150+0000 7fa6ff9d7640 1 -- 172.21.15.69:0/1413462607 wait complete. 2024-08-02T18:00:46.506 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:47.016 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:46 smithi069 ceph-mon[32746]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-08-02T18:00:47.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:46 smithi083 ceph-mon[37031]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-08-02T18:00:47.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:46 smithi077 ceph-mon[37108]: pgmap v449: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 4.5 MiB/s wr, 463 op/s 2024-08-02T18:00:47.731 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:00:47.731 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 -- 172.21.15.69:0/2744797864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2301018c0 msgr2=0x7fe230101d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 --2- 172.21.15.69:0/2744797864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2301018c0 0x7fe230101d20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fe220009930 tx=0x7fe22002a420 comp rx=0 tx=0).stop 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 -- 172.21.15.69:0/2744797864 shutdown_connections 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 --2- 172.21.15.69:0/2744797864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe2301018c0 0x7fe230101d20 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 --2- 172.21.15.69:0/2744797864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe230102260 0x7fe23010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 --2- 172.21.15.69:0/2744797864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2301078c0 0x7fe230107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 -- 172.21.15.69:0/2744797864 >> 172.21.15.69:0/2744797864 conn(0x7fe2300fd620 msgr2=0x7fe2300ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 -- 172.21.15.69:0/2744797864 shutdown_connections 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.381+0000 7fe238609640 1 -- 172.21.15.69:0/2744797864 wait complete. 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe238609640 1 Processor -- start 2024-08-02T18:00:47.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe238609640 1 -- start start 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe238609640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2301018c0 0x7fe230074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe238609640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe230102260 0x7fe230074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe238609640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2301078c0 0x7fe230074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe238609640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe230111ce0 con 0x7fe230102260 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe238609640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe230111b60 con 0x7fe2301078c0 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe238609640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe230111e60 con 0x7fe2301018c0 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe235ba4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe230102260 0x7fe230074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe235ba4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe230102260 0x7fe230074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34978/0 (socket says 172.21.15.69:34978) 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe235ba4640 1 -- 172.21.15.69:0/212003327 learned_addr learned my addr 172.21.15.69:0/212003327 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe2363a5640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2301018c0 0x7fe230074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.382+0000 7fe236ba6640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2301078c0 0x7fe230074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe235ba4640 1 -- 172.21.15.69:0/212003327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2301018c0 msgr2=0x7fe230074220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe235ba4640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2301018c0 0x7fe230074220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe235ba4640 1 -- 172.21.15.69:0/212003327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2301078c0 msgr2=0x7fe230074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe235ba4640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2301078c0 0x7fe230074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe235ba4640 1 -- 172.21.15.69:0/212003327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2301a6720 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe235ba4640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe230102260 0x7fe230074760 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe220009230 tx=0x7fe220004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe2277fe640 1 -- 172.21.15.69:0/212003327 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22003f030 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe238609640 1 -- 172.21.15.69:0/212003327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2301a69b0 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe2277fe640 1 -- 172.21.15.69:0/212003327 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe220003ad0 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe2277fe640 1 -- 172.21.15.69:0/212003327 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe220035070 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.383+0000 7fe238609640 1 -- 172.21.15.69:0/212003327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2301a6ee0 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.384+0000 7fe238609640 1 -- 172.21.15.69:0/212003327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe230101d20 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.385+0000 7fe2277fe640 1 -- 172.21.15.69:0/212003327 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe220042030 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.386+0000 7fe2277fe640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe21c0a2cf0 0x7fe21c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.386+0000 7fe2277fe640 1 -- 172.21.15.69:0/212003327 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7fe2200efd50 con 0x7fe230102260 2024-08-02T18:00:47.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.386+0000 7fe2363a5640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe21c0a2cf0 0x7fe21c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:47.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.386+0000 7fe2363a5640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe21c0a2cf0 0x7fe21c0a51b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe22c005fd0 tx=0x7fe22c005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:47.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.390+0000 7fe2277fe640 1 -- 172.21.15.69:0/212003327 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe2200bc560 con 0x7fe230102260 2024-08-02T18:00:47.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.726+0000 7fe238609640 1 -- 172.21.15.69:0/212003327 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe230109e50 con 0x7fe230102260 2024-08-02T18:00:47.735 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T18:00:47.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.727+0000 7fe2277fe640 1 -- 172.21.15.69:0/212003327 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7fe2200bbe60 con 0x7fe230102260 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.738+0000 7fe2257fa640 1 -- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe21c0a2cf0 msgr2=0x7fe21c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.738+0000 7fe2257fa640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe21c0a2cf0 0x7fe21c0a51b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe22c005fd0 tx=0x7fe22c005850 comp rx=0 tx=0).stop 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.738+0000 7fe2257fa640 1 -- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe230102260 msgr2=0x7fe230074760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.738+0000 7fe2257fa640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe230102260 0x7fe230074760 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fe220009230 tx=0x7fe220004270 comp rx=0 tx=0).stop 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.739+0000 7fe2257fa640 1 -- 172.21.15.69:0/212003327 shutdown_connections 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.739+0000 7fe2257fa640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe230102260 0x7fe230074760 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.739+0000 7fe2257fa640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe21c0a2cf0 0x7fe21c0a51b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.739+0000 7fe2257fa640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe2301078c0 0x7fe230074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.739+0000 7fe2257fa640 1 --2- 172.21.15.69:0/212003327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe2301018c0 0x7fe230074220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:47.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.739+0000 7fe2257fa640 1 -- 172.21.15.69:0/212003327 >> 172.21.15.69:0/212003327 conn(0x7fe2300fd620 msgr2=0x7fe2300fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:47.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.742+0000 7fe2257fa640 1 -- 172.21.15.69:0/212003327 shutdown_connections 2024-08-02T18:00:47.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:47.743+0000 7fe2257fa640 1 -- 172.21.15.69:0/212003327 wait complete. 2024-08-02T18:00:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:48 smithi083 ceph-mon[37031]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 474 op/s 2024-08-02T18:00:49.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:48 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/212003327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:49.023 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:48 smithi069 ceph-mon[32746]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 474 op/s 2024-08-02T18:00:49.023 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:48 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/212003327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:49.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:48 smithi077 ceph-mon[37108]: pgmap v450: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.9 MiB/s wr, 474 op/s 2024-08-02T18:00:49.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:48 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/212003327' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:00:49.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 820.0 s, 455.4 tps, lat 219.852 ms stddev 312.259 2024-08-02T18:00:50.760 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T18:00:51.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:50 smithi083 ceph-mon[37031]: pgmap v451: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.8 MiB/s wr, 683 op/s 2024-08-02T18:00:51.071 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:50 smithi069 ceph-mon[32746]: pgmap v451: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.8 MiB/s wr, 683 op/s 2024-08-02T18:00:51.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:50 smithi077 ceph-mon[37108]: pgmap v451: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 5.8 MiB/s wr, 683 op/s 2024-08-02T18:00:51.291 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:51.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.941+0000 7f6cda263640 1 -- 172.21.15.69:0/1540704302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cd410e730 msgr2=0x7f6cd410eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:51.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.941+0000 7f6cda263640 1 --2- 172.21.15.69:0/1540704302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cd410e730 0x7f6cd410eb10 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009900 tx=0x7f6cd002a440 comp rx=0 tx=0).stop 2024-08-02T18:00:51.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.941+0000 7f6cda263640 1 -- 172.21.15.69:0/1540704302 shutdown_connections 2024-08-02T18:00:51.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.941+0000 7f6cda263640 1 --2- 172.21.15.69:0/1540704302 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd41098a0 0x7f6cd4115410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.941+0000 7f6cda263640 1 --2- 172.21.15.69:0/1540704302 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd4108f20 0x7f6cd4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.941+0000 7f6cda263640 1 --2- 172.21.15.69:0/1540704302 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cd410e730 0x7f6cd410eb10 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.944 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.941+0000 7f6cda263640 1 -- 172.21.15.69:0/1540704302 >> 172.21.15.69:0/1540704302 conn(0x7f6cd4072eb0 msgr2=0x7f6cd40732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:51.947 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.945+0000 7f6cda263640 1 -- 172.21.15.69:0/1540704302 shutdown_connections 2024-08-02T18:00:51.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.947+0000 7f6cda263640 1 -- 172.21.15.69:0/1540704302 wait complete. 2024-08-02T18:00:51.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cda263640 1 Processor -- start 2024-08-02T18:00:51.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cda263640 1 -- start start 2024-08-02T18:00:51.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cda263640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd4108f20 0x7f6cd41a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cda263640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cd41098a0 0x7f6cd41a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cda263640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd410e730 0x7f6cd4114740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cd9261640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd4108f20 0x7f6cd41a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cd9261640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd4108f20 0x7f6cd41a7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35452/0 (socket says 172.21.15.69:35452) 2024-08-02T18:00:51.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cd9261640 1 -- 172.21.15.69:0/1404617323 learned_addr learned my addr 172.21.15.69:0/1404617323 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:51.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd411a370 con 0x7f6cd41098a0 2024-08-02T18:00:51.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.948+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd411a1f0 con 0x7f6cd410e730 2024-08-02T18:00:51.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.949+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd411a4f0 con 0x7f6cd4108f20 2024-08-02T18:00:51.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.949+0000 7f6cd9a62640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd410e730 0x7f6cd4114740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.953 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.949+0000 7f6cd9261640 1 -- 172.21.15.69:0/1404617323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd410e730 msgr2=0x7f6cd4114740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.949+0000 7f6cd9261640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd410e730 0x7f6cd4114740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.949+0000 7f6cd9261640 1 -- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cd41098a0 msgr2=0x7f6cd41a7ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.949+0000 7f6cd9261640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cd41098a0 0x7f6cd41a7ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.949+0000 7f6cd9261640 1 -- 172.21.15.69:0/1404617323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd4115000 con 0x7f6cd4108f20 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.949+0000 7f6cd9261640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd4108f20 0x7f6cd41a7580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009530 tx=0x7f6cd0003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.950+0000 7f6cca7fc640 1 -- 172.21.15.69:0/1404617323 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd003f070 con 0x7f6cd4108f20 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.950+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd4115290 con 0x7f6cd4108f20 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.950+0000 7f6cca7fc640 1 -- 172.21.15.69:0/1404617323 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd0003e40 con 0x7f6cd4108f20 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.950+0000 7f6cca7fc640 1 -- 172.21.15.69:0/1404617323 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0035030 con 0x7f6cd4108f20 2024-08-02T18:00:51.954 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.950+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd41a3130 con 0x7f6cd4108f20 2024-08-02T18:00:51.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.955+0000 7f6cdd8c3740 1 Processor -- start 2024-08-02T18:00:51.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.955+0000 7f6cdd8c3740 1 -- start start 2024-08-02T18:00:51.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.955+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.955+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b70e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.955+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.955+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557138acba40 con 0x557138b661d0 2024-08-02T18:00:51.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.955+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557138a639d0 con 0x557138b66af0 2024-08-02T18:00:51.958 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.955+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557138ac7a00 con 0x557138b713b0 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cd8a60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cd8a60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b737a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35466/0 (socket says 172.21.15.69:35466) 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cd8a60640 1 -- 172.21.15.69:0/1531413813 learned_addr learned my addr 172.21.15.69:0/1531413813 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cd8a60640 1 -- 172.21.15.69:0/1531413813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 msgr2=0x557138b70e70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/1531413813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b70e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cd8a60640 1 -- 172.21.15.69:0/1531413813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 msgr2=0x557138b665b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/1531413813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cd8a60640 1 -- 172.21.15.69:0/1531413813 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557138a6a650 con 0x557138b713b0 2024-08-02T18:00:51.959 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.956+0000 7f6cca7fc640 1 -- 172.21.15.69:0/1404617323 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6cd0042070 con 0x7f6cd4108f20 2024-08-02T18:00:51.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.957+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/1531413813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b737a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400a920 tx=0x7f6cc400ade0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e89cedd18b2ee92 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:51.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.958+0000 7f6cca7fc640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cc00a2e40 0x7f6cc00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.958+0000 7f6cca7fc640 1 -- 172.21.15.69:0/1404617323 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f6cd00f00b0 con 0x7f6cd4108f20 2024-08-02T18:00:51.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.958+0000 7f6cd9a62640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cc00a2e40 0x7f6cc00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.958+0000 7f6cbf7fe640 1 -- 172.21.15.69:0/1531413813 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400cd80 con 0x557138b713b0 2024-08-02T18:00:51.960 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.959+0000 7f6cbf7fe640 1 -- 172.21.15.69:0/1531413813 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc400f930 con 0x557138b713b0 2024-08-02T18:00:51.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.959+0000 7f6cbf7fe640 1 -- 172.21.15.69:0/1531413813 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4018970 con 0x557138b713b0 2024-08-02T18:00:51.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.959+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/1531413813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 msgr2=0x557138b737a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:51.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.959+0000 7f6cd9a62640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cc00a2e40 0x7f6cc00a5300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009420 tx=0x7f6ccc00d040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:51.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.959+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/1531413813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b737a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400a920 tx=0x7f6cc400ade0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.960+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/1531413813 shutdown_connections 2024-08-02T18:00:51.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.960+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/1531413813 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b737a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.960+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/1531413813 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b70e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.960+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/1531413813 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b665b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.960+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/1531413813 >> 172.21.15.69:0/1531413813 conn(0x557138b62760 msgr2=0x557138b62b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:51.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.960+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/1531413813 shutdown_connections 2024-08-02T18:00:51.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.960+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/1531413813 wait complete. 2024-08-02T18:00:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.960+0000 7f6cdd8c3740 1 -- start start 2024-08-02T18:00:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cdd8c3740 1 Processor -- start 2024-08-02T18:00:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557138b73f80 con 0x557138b661d0 2024-08-02T18:00:51.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557138a86b20 con 0x557138b713b0 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cd9261640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557138b74100 con 0x557138b66af0 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cd9261640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34998/0 (socket says 172.21.15.69:34998) 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cd9261640 1 -- 172.21.15.69:0/540542115 learned_addr learned my addr 172.21.15.69:0/540542115 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cdd8c3740 10 client.0 _ll_register_callbacks cb 0x557138a84bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.961+0000 7f6cd9a62640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cd9261640 1 -- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 msgr2=0x557138b6e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cd9261640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cd9261640 1 -- 172.21.15.69:0/540542115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 msgr2=0x557138b6ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cd9261640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cd9261640 1 -- 172.21.15.69:0/540542115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55713883d3c0 con 0x557138b661d0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cd9a62640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cd9261640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6e2d0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6cd002a9b0 tx=0x7f6cd00bcdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f22f06b0121fa2c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cbcff9640 1 -- 172.21.15.69:0/540542115 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd00c1580 con 0x557138b661d0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cbcff9640 1 -- 172.21.15.69:0/540542115 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f6cd00ed630 con 0x557138b661d0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/540542115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc400f930 con 0x557138b661d0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cbcff9640 1 -- 172.21.15.69:0/540542115 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6cd00ed830 con 0x557138b661d0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cdd8c3740 10 client.19068 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.962+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/540542115 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557138b7fc50 con 0x557138b661d0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cbcff9640 1 -- 172.21.15.69:0/540542115 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cd00c5e40 con 0x557138b661d0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cbcff9640 1 -- 172.21.15.69:0/540542115 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6cd00c59b0 con 0x557138b661d0 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cdd8c3740 10 client.19068 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:51.967 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cdd8c3740 10 client.19068 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:00:51.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cdd8c3740 20 client.19068 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:51.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557138b81040 0x557138b83430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cdd8c3740 4 client.19068 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:51.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/540542115 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6cc400f930 con 0x557138b81040 2024-08-02T18:00:51.968 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.963+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557138b81040 0x557138b83430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.964+0000 7f6cbcff9640 10 client.19068 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.964+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557138b81040 0x557138b83430 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aac94af9df9056a4 server_cookie=20953387447f9ebe in_seq=0 out_seq=0 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cbcff9640 1 -- 172.21.15.69:0/540542115 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f6cc400f930 con 0x557138b81040 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cbcff9640 10 client.19068 handle_command_reply: tid=0 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 client.19068 shutdown 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/540542115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 msgr2=0x557138b6e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6e2d0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6cd002a9b0 tx=0x7f6cd00bcdd0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/540542115 shutdown_connections 2024-08-02T18:00:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557138b81040 0x557138b83430 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cbcff9640 0 client.19068 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:51.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/540542115 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6e2d0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.965+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/540542115 >> 172.21.15.69:0/540542115 conn(0x557138b62760 msgr2=0x557138b68490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:51.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.966+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/540542115 shutdown_connections 2024-08-02T18:00:51.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.966+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/540542115 wait complete. 2024-08-02T18:00:51.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.966+0000 7f6cdd8c3740 20 client.19068 trim_cache size 0 max 16384 2024-08-02T18:00:51.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.980+0000 7f6cdd8c3740 1 Processor -- start 2024-08-02T18:00:51.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.980+0000 7f6cdd8c3740 1 -- start start 2024-08-02T18:00:51.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.980+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.980+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b88d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.980+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b89240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.980+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8012800 con 0x557138b661d0 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.980+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb80100d0 con 0x557138b66af0 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.980+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb800df20 con 0x557138b713b0 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b88d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd9a62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b89240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd9261640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b88d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60882/0 (socket says 172.21.15.69:60882) 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd9a62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b89240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35474/0 (socket says 172.21.15.69:35474) 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 -- 172.21.15.69:0/2451761718 learned_addr learned my addr 172.21.15.69:0/2451761718 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 -- 172.21.15.69:0/2451761718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 msgr2=0x557138b89240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/2451761718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b89240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 -- 172.21.15.69:0/2451761718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 msgr2=0x557138b6db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/2451761718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 -- 172.21.15.69:0/2451761718 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557138b6d280 con 0x557138b66af0 2024-08-02T18:00:51.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/2451761718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b88d00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x557138b63ae0 tx=0x7f6cc4005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6781030cefdc7ce server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cbeffd640 1 -- 172.21.15.69:0/2451761718 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400fd60 con 0x557138b66af0 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cbeffd640 1 -- 172.21.15.69:0/2451761718 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc401e040 con 0x557138b66af0 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.981+0000 7f6cbeffd640 1 -- 172.21.15.69:0/2451761718 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4004660 con 0x557138b66af0 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/2451761718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 msgr2=0x557138b88d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/2451761718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b88d00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x557138b63ae0 tx=0x7f6cc4005eb0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/2451761718 shutdown_connections 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/2451761718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b713b0 0x557138b89240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/2451761718 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b66af0 0x557138b88d00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/2451761718 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b6db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/2451761718 >> 172.21.15.69:0/2451761718 conn(0x557138b62760 msgr2=0x557138b70240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/2451761718 shutdown_connections 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.982+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/2451761718 wait complete. 2024-08-02T18:00:51.993 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 1 -- start start 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 1 Processor -- start 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b79b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb800e590 con 0x557138b661d0 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb800fa00 con 0x557138b713b0 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb800ec60 con 0x557138b66af0 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.983+0000 7f6cdd8c3740 10 client.0 _ll_register_callbacks cb 0x7f6cd00bcfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd9a62640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd8a60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd9a62640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60890/0 (socket says 172.21.15.69:60890) 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd9a62640 1 -- 172.21.15.69:0/3929885676 learned_addr learned my addr 172.21.15.69:0/3929885676 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:51.994 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd8a60640 1 -- 172.21.15.69:0/3929885676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 msgr2=0x557138b6b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd8a60640 1 -- 172.21.15.69:0/3929885676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 msgr2=0x557138b79b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b79b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd8a60640 1 -- 172.21.15.69:0/3929885676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557138b6be10 con 0x557138b66af0 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd9a62640 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cd8a60640 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x557138b5dba0 tx=0x7f6cc400a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19eb07cc42ee42da server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.984+0000 7f6cbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.985+0000 7f6cbcff9640 1 -- 172.21.15.69:0/3929885676 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400fd60 con 0x557138b66af0 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.985+0000 7f6cbcff9640 1 -- 172.21.15.69:0/3929885676 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f6cc400f450 con 0x557138b66af0 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.985+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/3929885676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cc401e040 con 0x557138b66af0 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.985+0000 7f6cbcff9640 1 -- 172.21.15.69:0/3929885676 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6cc4040050 con 0x557138b66af0 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.985+0000 7f6cdd8c3740 10 client.28198 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.985+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/3929885676 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557138bebd30 con 0x557138b66af0 2024-08-02T18:00:51.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.985+0000 7f6cbcff9640 1 -- 172.21.15.69:0/3929885676 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cc401e040 con 0x557138b66af0 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cbcff9640 1 -- 172.21.15.69:0/3929885676 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f6cc4037460 con 0x557138b66af0 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cdd8c3740 10 client.28198 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cdd8c3740 10 client.28198 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cdd8c3740 20 client.28198 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557138b81040 0x557138bee150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cdd8c3740 4 client.28198 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/3929885676 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557138ad8fb0 con 0x557138b81040 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cd9261640 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557138b81040 0x557138bee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.986+0000 7f6cbcff9640 10 client.28198 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cd9261640 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557138b81040 0x557138bee150 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5843cd7035f58b6 server_cookie=4beeb63b1a42328a in_seq=0 out_seq=0 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cbcff9640 1 -- 172.21.15.69:0/3929885676 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557138ad8fb0 con 0x557138b81040 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cbcff9640 10 client.28198 handle_command_reply: tid=0 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 client.28198 shutdown 2024-08-02T18:00:51.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/3929885676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 msgr2=0x557138b6b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6b0a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x557138b5dba0 tx=0x7f6cc400a8b0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/3929885676 shutdown_connections 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x557138b81040 0x557138bee150 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557138b713b0 0x557138b6b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557138b66af0 0x557138b6b0a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 --2- 172.21.15.69:0/3929885676 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557138b661d0 0x557138b79b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.987+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/3929885676 >> 172.21.15.69:0/3929885676 conn(0x557138b62760 msgr2=0x557138b7c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.988+0000 7f6cbcff9640 0 client.28198 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.988+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/3929885676 shutdown_connections 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.988+0000 7f6cdd8c3740 1 -- 172.21.15.69:0/3929885676 wait complete. 2024-08-02T18:00:51.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.988+0000 7f6cdd8c3740 20 client.28198 trim_cache size 0 max 16384 2024-08-02T18:00:51.998 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cc00a2e40 msgr2=0x7f6cc00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cc00a2e40 0x7f6cc00a5300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009420 tx=0x7f6ccc00d040 comp rx=0 tx=0).stop 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd4108f20 msgr2=0x7f6cd41a7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd4108f20 0x7f6cd41a7580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009530 tx=0x7f6cd0003cd0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 shutdown_connections 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6cd41098a0 0x7f6cd41a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cc00a2e40 0x7f6cc00a5300 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6cd410e730 0x7f6cd4114740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 --2- 172.21.15.69:0/1404617323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cd4108f20 0x7f6cd41a7580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 >> 172.21.15.69:0/1404617323 conn(0x7f6cd4072eb0 msgr2=0x7f6cd4116fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 shutdown_connections 2024-08-02T18:00:51.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:51.989+0000 7f6cda263640 1 -- 172.21.15.69:0/1404617323 wait complete. 2024-08-02T18:00:52.957 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T18:00:53.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:52 smithi083 ceph-mon[37031]: pgmap v452: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.1 MiB/s wr, 508 op/s 2024-08-02T18:00:53.021 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:52 smithi069 ceph-mon[32746]: pgmap v452: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.1 MiB/s wr, 508 op/s 2024-08-02T18:00:53.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:52 smithi077 ceph-mon[37108]: pgmap v452: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 4.1 MiB/s wr, 508 op/s 2024-08-02T18:00:53.457 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:54.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.048+0000 7f57a3a37640 1 -- 172.21.15.69:0/1110893782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f579c109560 msgr2=0x7f579c1099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:54.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.048+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1110893782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f579c109560 0x7f579c1099c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f578c009900 tx=0x7f578c02a440 comp rx=0 tx=0).stop 2024-08-02T18:00:54.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.049+0000 7f57a3a37640 1 -- 172.21.15.69:0/1110893782 shutdown_connections 2024-08-02T18:00:54.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.049+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1110893782 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579c11ae70 0x7f579c11d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.049+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1110893782 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f579c109560 0x7f579c1099c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.049+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1110893782 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f579c109180 0x7f579c10ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.049+0000 7f57a3a37640 1 -- 172.21.15.69:0/1110893782 >> 172.21.15.69:0/1110893782 conn(0x7f579c06de70 msgr2=0x7f579c06e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:54.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.049+0000 7f57a3a37640 1 -- 172.21.15.69:0/1110893782 shutdown_connections 2024-08-02T18:00:54.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.049+0000 7f57a3a37640 1 -- 172.21.15.69:0/1110893782 wait complete. 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.050+0000 7f57a3a37640 1 Processor -- start 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.051+0000 7f57a3a37640 1 -- start start 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.051+0000 7f57a3a37640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579c109180 0x7f579c1a7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.051+0000 7f57a3a37640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f579c109560 0x7f579c1a7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.051+0000 7f57a3a37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f579c11ae70 0x7f579c1ab910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.051+0000 7f57a3a37640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f579c120b20 con 0x7f579c109560 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.051+0000 7f57a3a37640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1209a0 con 0x7f579c109180 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.051+0000 7f57a3a37640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f579c120ca0 con 0x7f579c11ae70 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a17d3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579c109180 0x7f579c1a7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a17d3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579c109180 0x7f579c1a7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60896/0 (socket says 172.21.15.69:60896) 2024-08-02T18:00:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a17d3640 1 -- 172.21.15.69:0/1072864418 learned_addr learned my addr 172.21.15.69:0/1072864418 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f579c109560 0x7f579c1a7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a17d3640 1 -- 172.21.15.69:0/1072864418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f579c11ae70 msgr2=0x7f579c1ab910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T18:00:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a17d3640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f579c11ae70 0x7f579c1ab910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a17d3640 1 -- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f579c109560 msgr2=0x7f579c1a7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a17d3640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f579c109560 0x7f579c1a7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.055 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.052+0000 7f57a17d3640 1 -- 172.21.15.69:0/1072864418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c1abff0 con 0x7f579c109180 2024-08-02T18:00:54.056 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.054+0000 7f57a17d3640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579c109180 0x7f579c1a7040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f579800e9f0 tx=0x7f579800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:54.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.056+0000 7f57927fc640 1 -- 172.21.15.69:0/1072864418 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579800cd80 con 0x7f579c109180 2024-08-02T18:00:54.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.056+0000 7f57927fc640 1 -- 172.21.15.69:0/1072864418 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5798010910 con 0x7f579c109180 2024-08-02T18:00:54.057 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.056+0000 7f57927fc640 1 -- 172.21.15.69:0/1072864418 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57980188b0 con 0x7f579c109180 2024-08-02T18:00:54.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.057+0000 7f57a3a37640 1 -- 172.21.15.69:0/1072864418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c1ac2e0 con 0x7f579c109180 2024-08-02T18:00:54.058 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.057+0000 7f57a3a37640 1 -- 172.21.15.69:0/1072864418 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c1ac6e0 con 0x7f579c109180 2024-08-02T18:00:54.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.062+0000 7f57927fc640 1 -- 172.21.15.69:0/1072864418 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5798010ac0 con 0x7f579c109180 2024-08-02T18:00:54.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.063+0000 7f57927fc640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f57880a2e80 0x7f57880a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.063+0000 7f57927fc640 1 -- 172.21.15.69:0/1072864418 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f57980cfe00 con 0x7f579c109180 2024-08-02T18:00:54.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.066+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f57880a2e80 0x7f57880a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.070+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f57880a2e80 0x7f57880a5340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f578c02aa30 tx=0x7f578c005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:54.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.078+0000 7f57a5e35740 1 Processor -- start 2024-08-02T18:00:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.079+0000 7f57a5e35740 1 -- start start 2024-08-02T18:00:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fc53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc5910 0x563a64fcfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fd01d0 0x563a64fd25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a5e35740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563a64f2aa40 con 0x563a64fc5910 2024-08-02T18:00:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a5e35740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563a64ec29d0 con 0x563a64fd01d0 2024-08-02T18:00:54.082 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a5e35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a64f26a00 con 0x563a64fc4ff0 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a17d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fc53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a1fd4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fd01d0 0x563a64fd25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a17d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fc53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35520/0 (socket says 172.21.15.69:35520) 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a17d3640 1 -- 172.21.15.69:0/1944472826 learned_addr learned my addr 172.21.15.69:0/1944472826 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.080+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/1944472826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc5910 0x563a64fcfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.084+0000 7f57a17d3640 1 -- 172.21.15.69:0/1944472826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fd01d0 msgr2=0x563a64fd25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.084+0000 7f57a17d3640 1 --2- 172.21.15.69:0/1944472826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fd01d0 0x563a64fd25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.084+0000 7f57a17d3640 1 -- 172.21.15.69:0/1944472826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc5910 msgr2=0x563a64fcfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.084+0000 7f57a17d3640 1 --2- 172.21.15.69:0/1944472826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc5910 0x563a64fcfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.084+0000 7f57a17d3640 1 -- 172.21.15.69:0/1944472826 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a64ec9650 con 0x563a64fc4ff0 2024-08-02T18:00:54.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.084+0000 7f57a17d3640 1 --2- 172.21.15.69:0/1944472826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fc53d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57980988e0 tx=0x7f579809bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4af1d4b3708ba server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:54.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.085+0000 7f57877fe640 1 -- 172.21.15.69:0/1944472826 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57980b7070 con 0x563a64fc4ff0 2024-08-02T18:00:54.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.085+0000 7f57877fe640 1 -- 172.21.15.69:0/1944472826 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57980a4b80 con 0x563a64fc4ff0 2024-08-02T18:00:54.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.085+0000 7f57877fe640 1 -- 172.21.15.69:0/1944472826 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57980b4020 con 0x563a64fc4ff0 2024-08-02T18:00:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.088+0000 7f57a5e35740 1 -- 172.21.15.69:0/1944472826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 msgr2=0x563a64fc53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:54.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.088+0000 7f57a5e35740 1 --2- 172.21.15.69:0/1944472826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fc53d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57980988e0 tx=0x7f579809bfd0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.092+0000 7f57a5e35740 1 -- 172.21.15.69:0/1944472826 shutdown_connections 2024-08-02T18:00:54.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.092+0000 7f57a5e35740 1 --2- 172.21.15.69:0/1944472826 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc5910 0x563a64fcfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.092+0000 7f57a5e35740 1 --2- 172.21.15.69:0/1944472826 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fd01d0 0x563a64fd25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.092+0000 7f57a5e35740 1 --2- 172.21.15.69:0/1944472826 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fc53d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.092+0000 7f57a5e35740 1 -- 172.21.15.69:0/1944472826 >> 172.21.15.69:0/1944472826 conn(0x563a64fc1580 msgr2=0x563a64fc1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:54.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.092+0000 7f57a5e35740 1 -- 172.21.15.69:0/1944472826 shutdown_connections 2024-08-02T18:00:54.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.092+0000 7f57a5e35740 1 -- 172.21.15.69:0/1944472826 wait complete. 2024-08-02T18:00:54.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.092+0000 7f57a5e35740 1 -- start start 2024-08-02T18:00:54.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.093+0000 7f57a5e35740 1 Processor -- start 2024-08-02T18:00:54.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.093+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc4ff0 0x563a64fcd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.093+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 0x563a64fcd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.093+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fd01d0 0x563a64fcdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.093+0000 7f57a5e35740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x563a64fd2da0 con 0x563a64fc4ff0 2024-08-02T18:00:54.097 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.093+0000 7f57a5e35740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563a64ee5b20 con 0x563a64fc5910 2024-08-02T18:00:54.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.093+0000 7f57a5e35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563a64fd2f20 con 0x563a64fd01d0 2024-08-02T18:00:54.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.093+0000 7f57a5e35740 10 client.0 _ll_register_callbacks cb 0x563a64ee3bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:54.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.094+0000 7f57a1fd4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fd01d0 0x563a64fcdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.094+0000 7f57a1fd4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fd01d0 0x563a64fcdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35532/0 (socket says 172.21.15.69:35532) 2024-08-02T18:00:54.098 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.094+0000 7f57a1fd4640 1 -- 172.21.15.69:0/2010363546 learned_addr learned my addr 172.21.15.69:0/2010363546 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:54.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.096+0000 7f57a17d3640 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc4ff0 0x563a64fcd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.096+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 0x563a64fcd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.097+0000 7f57a1fd4640 1 -- 172.21.15.69:0/2010363546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 msgr2=0x563a64fcd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:54.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.097+0000 7f57a1fd4640 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 0x563a64fcd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.097+0000 7f57a1fd4640 1 -- 172.21.15.69:0/2010363546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc4ff0 msgr2=0x563a64fcd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:54.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.097+0000 7f57a1fd4640 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc4ff0 0x563a64fcd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.097+0000 7f57a1fd4640 1 -- 172.21.15.69:0/2010363546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a64c9c3c0 con 0x563a64fd01d0 2024-08-02T18:00:54.103 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.101+0000 7f57a1fd4640 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fd01d0 0x563a64fcdba0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5794011860 tx=0x7f5794011d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1dcde1d6f8410a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:54.104 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.101+0000 7f5784ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:54.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.105+0000 7f5784ff9640 1 -- 172.21.15.69:0/2010363546 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794003e40 con 0x563a64fd01d0 2024-08-02T18:00:54.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.105+0000 7f5784ff9640 1 -- 172.21.15.69:0/2010363546 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f5794004b20 con 0x563a64fd01d0 2024-08-02T18:00:54.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.105+0000 7f57a5e35740 1 -- 172.21.15.69:0/2010363546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57980a4b80 con 0x563a64fd01d0 2024-08-02T18:00:54.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.105+0000 7f5784ff9640 1 -- 172.21.15.69:0/2010363546 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5794004d20 con 0x563a64fd01d0 2024-08-02T18:00:54.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.105+0000 7f57a5e35740 10 client.28207 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:54.107 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.105+0000 7f57a5e35740 1 -- 172.21.15.69:0/2010363546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a64fdead0 con 0x563a64fd01d0 2024-08-02T18:00:54.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.106+0000 7f5784ff9640 1 -- 172.21.15.69:0/2010363546 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57980a4b80 con 0x563a64fd01d0 2024-08-02T18:00:54.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.106+0000 7f5784ff9640 1 -- 172.21.15.69:0/2010363546 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5794018070 con 0x563a64fd01d0 2024-08-02T18:00:54.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.106+0000 7f57a5e35740 10 client.28207 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:54.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.106+0000 7f57a5e35740 10 client.28207 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:00:54.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.106+0000 7f57a5e35740 20 client.28207 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:54.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.106+0000 7f57a5e35740 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x563a64fe03b0 0x563a64fe2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.106+0000 7f57a5e35740 4 client.28207 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:54.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.106+0000 7f57a5e35740 1 -- 172.21.15.69:0/2010363546 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a64f37fb0 con 0x563a64fe03b0 2024-08-02T18:00:54.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.109+0000 7f57a17d3640 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x563a64fe03b0 0x563a64fe2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.111+0000 7f57a17d3640 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x563a64fe03b0 0x563a64fe2850 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62848e53cf278bef server_cookie=549aded3b4cd495 in_seq=0 out_seq=0 2024-08-02T18:00:54.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.111+0000 7f5784ff9640 10 client.28207 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:54.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.112+0000 7f5784ff9640 1 -- 172.21.15.69:0/2010363546 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a64f37fb0 con 0x563a64fe03b0 2024-08-02T18:00:54.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.112+0000 7f5784ff9640 10 client.28207 handle_command_reply: tid=0 2024-08-02T18:00:54.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.112+0000 7f57a5e35740 1 client.28207 shutdown 2024-08-02T18:00:54.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.113+0000 7f57a5e35740 1 -- 172.21.15.69:0/2010363546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fd01d0 msgr2=0x563a64fcdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:54.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.113+0000 7f57a5e35740 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fd01d0 0x563a64fcdba0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5794011860 tx=0x7f5794011d20 comp rx=0 tx=0).stop 2024-08-02T18:00:54.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 1 -- 172.21.15.69:0/2010363546 shutdown_connections 2024-08-02T18:00:54.115 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x563a64fe03b0 0x563a64fe2850 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fd01d0 0x563a64fcdba0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 0x563a64fcd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 1 --2- 172.21.15.69:0/2010363546 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fc4ff0 0x563a64fcd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 1 -- 172.21.15.69:0/2010363546 >> 172.21.15.69:0/2010363546 conn(0x563a64fc1580 msgr2=0x563a64fc72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f5784ff9640 0 client.28207 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 1 -- 172.21.15.69:0/2010363546 shutdown_connections 2024-08-02T18:00:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 1 -- 172.21.15.69:0/2010363546 wait complete. 2024-08-02T18:00:54.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.114+0000 7f57a5e35740 20 client.28207 trim_cache size 0 max 16384 2024-08-02T18:00:54.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.140+0000 7f57a5e35740 1 Processor -- start 2024-08-02T18:00:54.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.140+0000 7f57a5e35740 1 -- start start 2024-08-02T18:00:54.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.140+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc4ff0 0x563a64fcc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.140+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc5910 0x563a64fc7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.142 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.140+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fe8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.141+0000 7f57a5e35740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f579c1d1790 con 0x563a64fd01d0 2024-08-02T18:00:54.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.141+0000 7f57a17d3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc4ff0 0x563a64fcc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.141+0000 7f57a17d3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc4ff0 0x563a64fcc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60932/0 (socket says 172.21.15.69:60932) 2024-08-02T18:00:54.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.141+0000 7f57a17d3640 1 -- 172.21.15.69:0/796389100 learned_addr learned my addr 172.21.15.69:0/796389100 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:54.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.141+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc5910 0x563a64fc7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.141+0000 7f57a1fd4640 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fe8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.141+0000 7f57a5e35740 1 -- 172.21.15.69:0/796389100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f579c084620 con 0x563a64fc4ff0 2024-08-02T18:00:54.144 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.141+0000 7f57a5e35740 1 -- 172.21.15.69:0/796389100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f579c082470 con 0x563a64fc5910 2024-08-02T18:00:54.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.144+0000 7f57a17d3640 1 -- 172.21.15.69:0/796389100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc5910 msgr2=0x563a64fc7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:54.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.144+0000 7f57a17d3640 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc5910 0x563a64fc7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.145 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.144+0000 7f57a17d3640 1 -- 172.21.15.69:0/796389100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 msgr2=0x563a64fe8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:54.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.144+0000 7f57a17d3640 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fe8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.144+0000 7f57a17d3640 1 -- 172.21.15.69:0/796389100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a64fcb6f0 con 0x563a64fc4ff0 2024-08-02T18:00:54.154 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.152+0000 7f57a17d3640 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc4ff0 0x563a64fcc6e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f57980097a0 tx=0x7f57980097f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c5ab4ea3ccdc107 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:54.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.157+0000 7f5786ffd640 1 -- 172.21.15.69:0/796389100 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57980a8070 con 0x563a64fc4ff0 2024-08-02T18:00:54.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.157+0000 7f5786ffd640 1 -- 172.21.15.69:0/796389100 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57980cd3c0 con 0x563a64fc4ff0 2024-08-02T18:00:54.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.158+0000 7f57a5e35740 1 -- 172.21.15.69:0/796389100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc4ff0 msgr2=0x563a64fcc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:54.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.158+0000 7f57a5e35740 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc4ff0 0x563a64fcc6e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f57980097a0 tx=0x7f57980097f0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.159+0000 7f57a5e35740 1 -- 172.21.15.69:0/796389100 shutdown_connections 2024-08-02T18:00:54.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.159+0000 7f57a5e35740 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fe8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.160 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.159+0000 7f57a5e35740 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc5910 0x563a64fc7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.159+0000 7f57a5e35740 1 --2- 172.21.15.69:0/796389100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc4ff0 0x563a64fcc6e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.161 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.159+0000 7f57a5e35740 1 -- 172.21.15.69:0/796389100 >> 172.21.15.69:0/796389100 conn(0x563a64fc1580 msgr2=0x563a64fcba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.163+0000 7f57a5e35740 1 -- 172.21.15.69:0/796389100 shutdown_connections 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.163+0000 7f57a5e35740 1 -- 172.21.15.69:0/796389100 wait complete. 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.163+0000 7f57a5e35740 1 -- start start 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.164+0000 7f57a5e35740 1 Processor -- start 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.164+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fd8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.164+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 0x563a64fd86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.164+0000 7f57a5e35740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fc9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.164+0000 7f57a5e35740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f579c082ae0 con 0x563a64fd01d0 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.164+0000 7f57a5e35740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f579c083f50 con 0x563a64fc5910 2024-08-02T18:00:54.168 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.164+0000 7f57a5e35740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f579c0831b0 con 0x563a64fc4ff0 2024-08-02T18:00:54.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.164+0000 7f57a5e35740 10 client.0 _ll_register_callbacks cb 0x7f5794009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:54.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.166+0000 7f57a1fd4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fc9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.166+0000 7f57a1fd4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fc9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:35080/0 (socket says 172.21.15.69:35080) 2024-08-02T18:00:54.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.166+0000 7f57a1fd4640 1 -- 172.21.15.69:0/4170540635 learned_addr learned my addr 172.21.15.69:0/4170540635 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:54.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.170+0000 7f57a17d3640 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fd8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.172 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.170+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 0x563a64fd86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.174+0000 7f57a17d3640 1 -- 172.21.15.69:0/4170540635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 msgr2=0x563a64fd86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:54.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.174+0000 7f57a17d3640 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 0x563a64fd86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.174+0000 7f57a17d3640 1 -- 172.21.15.69:0/4170540635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 msgr2=0x563a64fc9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:54.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.174+0000 7f57a17d3640 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fc9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.174+0000 7f57a17d3640 1 -- 172.21.15.69:0/4170540635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a64fd8d70 con 0x563a64fc4ff0 2024-08-02T18:00:54.178 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.177+0000 7f57a17d3640 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fd8170 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f579809bea0 tx=0x7f57980a0340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=183362cb4a690bcd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:54.181 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.178+0000 7f5784ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:54.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.182+0000 7f5784ff9640 1 -- 172.21.15.69:0/4170540635 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57980a8040 con 0x563a64fc4ff0 2024-08-02T18:00:54.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.182+0000 7f5784ff9640 1 -- 172.21.15.69:0/4170540635 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f5798010430 con 0x563a64fc4ff0 2024-08-02T18:00:54.185 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.184+0000 7f57a5e35740 1 -- 172.21.15.69:0/4170540635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57980cd3c0 con 0x563a64fc4ff0 2024-08-02T18:00:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.188+0000 7f5784ff9640 1 -- 172.21.15.69:0/4170540635 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f57980b4040 con 0x563a64fc4ff0 2024-08-02T18:00:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.189+0000 7f57a5e35740 10 client.28213 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:54.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.189+0000 7f57a5e35740 1 -- 172.21.15.69:0/4170540635 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563a6505b6d0 con 0x563a64fc4ff0 2024-08-02T18:00:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.193+0000 7f5784ff9640 1 -- 172.21.15.69:0/4170540635 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57980cd3c0 con 0x563a64fc4ff0 2024-08-02T18:00:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.193+0000 7f5784ff9640 1 -- 172.21.15.69:0/4170540635 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f579809add0 con 0x563a64fc4ff0 2024-08-02T18:00:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.193+0000 7f57a5e35740 10 client.28213 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.193+0000 7f57a5e35740 10 client.28213 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:00:54.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.193+0000 7f57a5e35740 20 client.28213 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.193+0000 7f57a5e35740 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x563a6505b950 0x563a6505dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.193+0000 7f57a5e35740 4 client.28213 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:54.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.193+0000 7f57a5e35740 1 -- 172.21.15.69:0/4170540635 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a64f324c0 con 0x563a6505b950 2024-08-02T18:00:54.200 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.198+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x563a6505b950 0x563a6505dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.205+0000 7f57a0fd2640 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x563a6505b950 0x563a6505dd40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0feb77a8188823f server_cookie=e515593724d6ac04 in_seq=0 out_seq=0 2024-08-02T18:00:54.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.205+0000 7f5784ff9640 10 client.28213 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.206+0000 7f5784ff9640 1 -- 172.21.15.69:0/4170540635 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a64f324c0 con 0x563a6505b950 2024-08-02T18:00:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.206+0000 7f5784ff9640 10 client.28213 handle_command_reply: tid=0 2024-08-02T18:00:54.210 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.208+0000 7f57a5e35740 1 client.28213 shutdown 2024-08-02T18:00:54.213 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 -- 172.21.15.69:0/4170540635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 msgr2=0x563a64fd8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fd8170 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f579809bea0 tx=0x7f57980a0340 comp rx=0 tx=0).stop 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 -- 172.21.15.69:0/4170540635 shutdown_connections 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x563a6505b950 0x563a6505dd40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x563a64fd01d0 0x563a64fc9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563a64fc5910 0x563a64fd86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 --2- 172.21.15.69:0/4170540635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563a64fc4ff0 0x563a64fd8170 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 -- 172.21.15.69:0/4170540635 >> 172.21.15.69:0/4170540635 conn(0x563a64fc1580 msgr2=0x563a64fcd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f5784ff9640 0 client.28213 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 -- 172.21.15.69:0/4170540635 shutdown_connections 2024-08-02T18:00:54.214 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 1 -- 172.21.15.69:0/4170540635 wait complete. 2024-08-02T18:00:54.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.211+0000 7f57a5e35740 20 client.28213 trim_cache size 0 max 16384 2024-08-02T18:00:54.215 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 -- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f57880a2e80 msgr2=0x7f57880a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f57880a2e80 0x7f57880a5340 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f578c02aa30 tx=0x7f578c005850 comp rx=0 tx=0).stop 2024-08-02T18:00:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 -- 172.21.15.69:0/1072864418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579c109180 msgr2=0x7f579c1a7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579c109180 0x7f579c1a7040 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f579800e9f0 tx=0x7f579800eeb0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 -- 172.21.15.69:0/1072864418 shutdown_connections 2024-08-02T18:00:54.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f579c109560 0x7f579c1a7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f57880a2e80 0x7f57880a5340 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f579c11ae70 0x7f579c1ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 --2- 172.21.15.69:0/1072864418 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f579c109180 0x7f579c1a7040 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 -- 172.21.15.69:0/1072864418 >> 172.21.15.69:0/1072864418 conn(0x7f579c06de70 msgr2=0x7f579c11bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.212+0000 7f57a3a37640 1 -- 172.21.15.69:0/1072864418 shutdown_connections 2024-08-02T18:00:54.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:54.213+0000 7f57a3a37640 1 -- 172.21.15.69:0/1072864418 wait complete. 2024-08-02T18:00:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 825.0 s, 554.0 tps, lat 185.984 ms stddev 245.199 2024-08-02T18:00:55.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:54 smithi083 ceph-mon[37031]: pgmap v453: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.3 MiB/s wr, 638 op/s 2024-08-02T18:00:55.026 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:54 smithi069 ceph-mon[32746]: pgmap v453: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.3 MiB/s wr, 638 op/s 2024-08-02T18:00:55.070 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T18:00:55.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:54 smithi077 ceph-mon[37108]: pgmap v453: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 5.3 MiB/s wr, 638 op/s 2024-08-02T18:00:55.594 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:00:56.929 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:56 smithi069 ceph-mon[32746]: pgmap v454: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 549 op/s 2024-08-02T18:00:57.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:56 smithi083 ceph-mon[37031]: pgmap v454: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 549 op/s 2024-08-02T18:00:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:56 smithi077 ceph-mon[37108]: pgmap v454: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 549 op/s 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.822+0000 7f13aa0fa640 1 -- 172.21.15.69:0/3626042383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a411ae70 msgr2=0x7f13a411d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.822+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/3626042383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a411ae70 0x7f13a411d260 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1398009900 tx=0x7f139802a440 comp rx=0 tx=0).stop 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.828+0000 7f13aa0fa640 1 -- 172.21.15.69:0/3626042383 shutdown_connections 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.828+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/3626042383 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a411ae70 0x7f13a411d260 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.828+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/3626042383 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a4109560 0x7f13a41099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.828+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/3626042383 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a4109180 0x7f13a410ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.828+0000 7f13aa0fa640 1 -- 172.21.15.69:0/3626042383 >> 172.21.15.69:0/3626042383 conn(0x7f13a406de70 msgr2=0x7f13a406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.828+0000 7f13aa0fa640 1 -- 172.21.15.69:0/3626042383 shutdown_connections 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.828+0000 7f13aa0fa640 1 -- 172.21.15.69:0/3626042383 wait complete. 2024-08-02T18:00:57.639 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.829+0000 7f13aa0fa640 1 Processor -- start 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.829+0000 7f13aa0fa640 1 -- start start 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.830+0000 7f13aa0fa640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a4109180 0x7f13a41a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.835+0000 7f13aa0fa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a4109560 0x7f13a41a7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.835+0000 7f13aa0fa640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a411ae70 0x7f13a41ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.835+0000 7f13aa0fa640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4120b70 con 0x7f13a4109180 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.835+0000 7f13aa0fa640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13a41209f0 con 0x7f13a4109560 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.835+0000 7f13aa0fa640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13a4120cf0 con 0x7f13a411ae70 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a4109180 0x7f13a41a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a4109180 0x7f13a41a6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33582/0 (socket says 172.21.15.69:33582) 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a37fe640 1 -- 172.21.15.69:0/2313140081 learned_addr learned my addr 172.21.15.69:0/2313140081 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a2ffd640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a4109560 0x7f13a41a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a3fff640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a411ae70 0x7f13a41ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.640 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a2ffd640 1 -- 172.21.15.69:0/2313140081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a411ae70 msgr2=0x7f13a41ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a2ffd640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a411ae70 0x7f13a41ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a2ffd640 1 -- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a4109180 msgr2=0x7f13a41a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a2ffd640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a4109180 0x7f13a41a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.836+0000 7f13a2ffd640 1 -- 172.21.15.69:0/2313140081 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a41abfa0 con 0x7f13a4109560 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.837+0000 7f13a2ffd640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a4109560 0x7f13a41a7530 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f139c00bd70 tx=0x7f139c00be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.839+0000 7f13a0ff9640 1 -- 172.21.15.69:0/2313140081 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c019070 con 0x7f13a4109560 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.839+0000 7f13a0ff9640 1 -- 172.21.15.69:0/2313140081 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f139c014920 con 0x7f13a4109560 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.839+0000 7f13a0ff9640 1 -- 172.21.15.69:0/2313140081 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c0129b0 con 0x7f13a4109560 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.840+0000 7f13aa0fa640 1 -- 172.21.15.69:0/2313140081 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a41ac290 con 0x7f13a4109560 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.840+0000 7f13aa0fa640 1 -- 172.21.15.69:0/2313140081 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a41b3b70 con 0x7f13a4109560 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.845+0000 7f13ac4f8740 1 Processor -- start 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.846+0000 7f13a0ff9640 1 -- 172.21.15.69:0/2313140081 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f139c01c070 con 0x7f13a4109560 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.849+0000 7f13ac4f8740 1 -- start start 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.849+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d37f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.641 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.849+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d389c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.849+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d38c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.849+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559b4d2e4a40 con 0x559b4d38a1d0 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.849+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b4d27c9d0 con 0x559b4d37eff0 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.849+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b4d2e0a00 con 0x559b4d37f910 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.850+0000 7f13a3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d37f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.850+0000 7f13a3fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d37f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:38432/0 (socket says 172.21.15.69:38432) 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.850+0000 7f13a3fff640 1 -- 172.21.15.69:0/1338112220 learned_addr learned my addr 172.21.15.69:0/1338112220 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.851+0000 7f13a37fe640 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d389c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.852+0000 7f13a0ff9640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f138c0a2e80 0x7f138c0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a2ffd640 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d38c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a37fe640 1 -- 172.21.15.69:0/1338112220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 msgr2=0x559b4d37f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:57.642 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a37fe640 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d37f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a37fe640 1 -- 172.21.15.69:0/1338112220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 msgr2=0x559b4d38c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a37fe640 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d38c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a37fe640 1 -- 172.21.15.69:0/1338112220 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4d283650 con 0x559b4d37f910 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a3fff640 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d37f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a3fff640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f138c0a2e80 0x7f138c0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.855+0000 7f13a2ffd640 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d38c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.856+0000 7f13a3fff640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f138c0a2e80 0x7f138c0a5340 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1398005ab0 tx=0x7f1398009260 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.856+0000 7f13a0ff9640 1 -- 172.21.15.69:0/2313140081 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f139c0cecf0 con 0x7f13a4109560 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.856+0000 7f13a37fe640 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d389c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1394007780 tx=0x7f1394007c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fef334fc2688030 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.856+0000 7f1389ffb640 1 -- 172.21.15.69:0/1338112220 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139400d950 con 0x559b4d37f910 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.856+0000 7f1389ffb640 1 -- 172.21.15.69:0/1338112220 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1394018070 con 0x559b4d37f910 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.856+0000 7f1389ffb640 1 -- 172.21.15.69:0/1338112220 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13940123e0 con 0x559b4d37f910 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.858+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1338112220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 msgr2=0x559b4d389c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.643 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.858+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d389c90 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1394007780 tx=0x7f1394007c40 comp rx=0 tx=0).stop 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.859+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1338112220 shutdown_connections 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.859+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d38c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.859+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d389c90 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.859+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1338112220 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d37f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.859+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1338112220 >> 172.21.15.69:0/1338112220 conn(0x559b4d37b580 msgr2=0x559b4d37b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.862+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1338112220 shutdown_connections 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.863+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1338112220 wait complete. 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 1 -- start start 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 1 Processor -- start 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d389c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 0x559b4d386fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d3876e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x559b4d39ddb0 con 0x559b4d37eff0 2024-08-02T18:00:57.644 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x559b4d29fb20 con 0x559b4d37f910 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x559b4d39df30 con 0x559b4d38a1d0 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.864+0000 7f13ac4f8740 10 client.0 _ll_register_callbacks cb 0x559b4d29dbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d389c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d389c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33602/0 (socket says 172.21.15.69:33602) 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a37fe640 1 -- 172.21.15.69:0/1272554263 learned_addr learned my addr 172.21.15.69:0/1272554263 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a3fff640 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d3876e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a3fff640 1 -- 172.21.15.69:0/1272554263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 msgr2=0x559b4d386fa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a3fff640 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 0x559b4d386fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a3fff640 1 -- 172.21.15.69:0/1272554263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 msgr2=0x559b4d389c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a3fff640 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d389c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.865+0000 7f13a3fff640 1 -- 172.21.15.69:0/1272554263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4d0563c0 con 0x559b4d38a1d0 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.866+0000 7f13a3fff640 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d3876e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1398033040 tx=0x7f139802a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eabe8cfedd3f01a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:57.645 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.866+0000 7f1382ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.866+0000 7f1382ffd640 1 -- 172.21.15.69:0/1272554263 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398003b50 con 0x559b4d38a1d0 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.867+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1272554263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1394018070 con 0x559b4d38a1d0 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.867+0000 7f1382ffd640 1 -- 172.21.15.69:0/1272554263 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f1398003cf0 con 0x559b4d38a1d0 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.867+0000 7f1382ffd640 1 -- 172.21.15.69:0/1272554263 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1398003ef0 con 0x559b4d38a1d0 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.868+0000 7f13ac4f8740 10 client.28222 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.868+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1272554263 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b4d3a9900 con 0x559b4d38a1d0 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.869+0000 7f1382ffd640 1 -- 172.21.15.69:0/1272554263 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1394018070 con 0x559b4d38a1d0 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.869+0000 7f1382ffd640 1 -- 172.21.15.69:0/1272554263 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1398031bf0 con 0x559b4d38a1d0 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.869+0000 7f13ac4f8740 10 client.28222 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.869+0000 7f13ac4f8740 10 client.28222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.869+0000 7f13ac4f8740 20 client.28222 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.869+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559b4d3ab260 0x559b4d3ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.869+0000 7f13ac4f8740 4 client.28222 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:57.646 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.869+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1272554263 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b4d2f1fb0 con 0x559b4d3ab260 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.870+0000 7f13a37fe640 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559b4d3ab260 0x559b4d3ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.871+0000 7f13a37fe640 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559b4d3ab260 0x559b4d3ad700 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccb7488cf752d29 server_cookie=192c25545ddd6366 in_seq=0 out_seq=0 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.871+0000 7f1382ffd640 10 client.28222 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f1382ffd640 1 -- 172.21.15.69:0/1272554263 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559b4d2f1fb0 con 0x559b4d3ab260 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f1382ffd640 10 client.28222 handle_command_reply: tid=0 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 client.28222 shutdown 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1272554263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 msgr2=0x559b4d3876e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d3876e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1398033040 tx=0x7f139802a440 comp rx=0 tx=0).stop 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1272554263 shutdown_connections 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559b4d3ab260 0x559b4d3ad700 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d3876e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.647 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 0x559b4d386fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/1272554263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d389c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1272554263 >> 172.21.15.69:0/1272554263 conn(0x559b4d37b580 msgr2=0x559b4d381260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.872+0000 7f1382ffd640 0 client.28222 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.878+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1272554263 shutdown_connections 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.878+0000 7f13ac4f8740 1 -- 172.21.15.69:0/1272554263 wait complete. 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.878+0000 7f13ac4f8740 20 client.28222 trim_cache size 0 max 16384 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13ac4f8740 1 Processor -- start 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13ac4f8740 1 -- start start 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d3a7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d386860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d386da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1390012a50 con 0x559b4d38a1d0 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1390010320 con 0x559b4d37eff0 2024-08-02T18:00:57.648 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f139000e170 con 0x559b4d37f910 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13a3fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d386da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d386860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d3a7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13a2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d386860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43556/0 (socket says 172.21.15.69:43556) 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.903+0000 7f13a2ffd640 1 -- 172.21.15.69:0/3421198094 learned_addr learned my addr 172.21.15.69:0/3421198094 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13a37fe640 1 -- 172.21.15.69:0/3421198094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 msgr2=0x559b4d386860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13a37fe640 1 --2- 172.21.15.69:0/3421198094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d386860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13a37fe640 1 -- 172.21.15.69:0/3421198094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 msgr2=0x559b4d386da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13a37fe640 1 --2- 172.21.15.69:0/3421198094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d386da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13a37fe640 1 -- 172.21.15.69:0/3421198094 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4d37f740 con 0x559b4d37eff0 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13a37fe640 1 --2- 172.21.15.69:0/3421198094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d3a7070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1394006a70 tx=0x7f1394008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b05f73fdd3f54387 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13897fa640 1 -- 172.21.15.69:0/3421198094 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139402d070 con 0x559b4d37eff0 2024-08-02T18:00:57.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13897fa640 1 -- 172.21.15.69:0/3421198094 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1394018050 con 0x559b4d37eff0 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.904+0000 7f13897fa640 1 -- 172.21.15.69:0/3421198094 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394027800 con 0x559b4d37eff0 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3421198094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 msgr2=0x559b4d3a7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3421198094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d3a7070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1394006a70 tx=0x7f1394008040 comp rx=0 tx=0).stop 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3421198094 shutdown_connections 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3421198094 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d38a1d0 0x559b4d386da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3421198094 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d37f910 0x559b4d386860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3421198094 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37eff0 0x559b4d3a7070 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3421198094 >> 172.21.15.69:0/3421198094 conn(0x559b4d37b580 msgr2=0x559b4d382270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3421198094 shutdown_connections 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.905+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3421198094 wait complete. 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 1 -- start start 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 1 Processor -- start 2024-08-02T18:00:57.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d38bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 0x559b4d38a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d38aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f139000e7e0 con 0x559b4d37eff0 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f139000fc50 con 0x559b4d37f910 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f139000eeb0 con 0x559b4d38a1d0 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13ac4f8740 10 client.0 _ll_register_callbacks cb 0x7f1398009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d38bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13a37fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d38bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33616/0 (socket says 172.21.15.69:33616) 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13a37fe640 1 -- 172.21.15.69:0/3385017580 learned_addr learned my addr 172.21.15.69:0/3385017580 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13a3fff640 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d38aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.906+0000 7f13a2ffd640 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 0x559b4d38a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f13a3fff640 1 -- 172.21.15.69:0/3385017580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 msgr2=0x559b4d38a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:57.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f13a3fff640 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 0x559b4d38a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f13a3fff640 1 -- 172.21.15.69:0/3385017580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 msgr2=0x559b4d38bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f13a3fff640 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d38bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f13a3fff640 1 -- 172.21.15.69:0/3385017580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4d38b260 con 0x559b4d38a1d0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f13a3fff640 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d38aaf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1398009870 tx=0x7f1398003740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a97e3ed518a7e39e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f1382ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f1382ffd640 1 -- 172.21.15.69:0/3385017580 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398003d50 con 0x559b4d38a1d0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f1382ffd640 1 -- 172.21.15.69:0/3385017580 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f139803e620 con 0x559b4d38a1d0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.907+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3385017580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1394018050 con 0x559b4d38a1d0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f1382ffd640 1 -- 172.21.15.69:0/3385017580 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f139806a050 con 0x559b4d38a1d0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f13ac4f8740 10 client.28228 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3385017580 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559b4d3a3180 con 0x559b4d38a1d0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f1382ffd640 1 -- 172.21.15.69:0/3385017580 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1394018050 con 0x559b4d38a1d0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f1382ffd640 1 -- 172.21.15.69:0/3385017580 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1398065030 con 0x559b4d38a1d0 2024-08-02T18:00:57.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f13ac4f8740 10 client.28228 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f13ac4f8740 10 client.28228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f13ac4f8740 20 client.28228 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559b4d3a3920 0x559b4d417ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f13ac4f8740 4 client.28228 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.908+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3385017580 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b4d2ec4c0 con 0x559b4d3a3920 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.909+0000 7f13a37fe640 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559b4d3a3920 0x559b4d417ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.909+0000 7f1382ffd640 10 client.28228 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.909+0000 7f13a37fe640 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559b4d3a3920 0x559b4d417ff0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea06a8d90e579bea server_cookie=9dc8f2f7706d4f6f in_seq=0 out_seq=0 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.909+0000 7f1382ffd640 1 -- 172.21.15.69:0/3385017580 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b4d2ec4c0 con 0x559b4d3a3920 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.909+0000 7f1382ffd640 10 client.28228 handle_command_reply: tid=0 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.909+0000 7f13ac4f8740 1 client.28228 shutdown 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3385017580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 msgr2=0x559b4d38aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d38aaf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1398009870 tx=0x7f1398003740 comp rx=0 tx=0).stop 2024-08-02T18:00:57.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3385017580 shutdown_connections 2024-08-02T18:00:57.654 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:00:57.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x559b4d3a3920 0x559b4d417ff0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x559b4d38a1d0 0x559b4d38aaf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x559b4d37f910 0x559b4d38a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 --2- 172.21.15.69:0/3385017580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x559b4d37eff0 0x559b4d38bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3385017580 >> 172.21.15.69:0/3385017580 conn(0x559b4d37b580 msgr2=0x559b4d379db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f1382ffd640 0 client.28228 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3385017580 shutdown_connections 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 1 -- 172.21.15.69:0/3385017580 wait complete. 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.910+0000 7f13ac4f8740 20 client.28228 trim_cache size 0 max 16384 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.911+0000 7f13aa0fa640 1 -- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f138c0a2e80 msgr2=0x7f138c0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.911+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f138c0a2e80 0x7f138c0a5340 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f1398005ab0 tx=0x7f1398009260 comp rx=0 tx=0).stop 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.911+0000 7f13aa0fa640 1 -- 172.21.15.69:0/2313140081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a4109560 msgr2=0x7f13a41a7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.911+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a4109560 0x7f13a41a7530 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f139c00bd70 tx=0x7f139c00be60 comp rx=0 tx=0).stop 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.912+0000 7f13aa0fa640 1 -- 172.21.15.69:0/2313140081 shutdown_connections 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.912+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f138c0a2e80 0x7f138c0a5340 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.912+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13a411ae70 0x7f13a41ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.912+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13a4109560 0x7f13a41a7530 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.912+0000 7f13aa0fa640 1 --2- 172.21.15.69:0/2313140081 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13a4109180 0x7f13a41a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:57.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.912+0000 7f13aa0fa640 1 -- 172.21.15.69:0/2313140081 >> 172.21.15.69:0/2313140081 conn(0x7f13a406de70 msgr2=0x7f13a411bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:57.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.913+0000 7f13aa0fa640 1 -- 172.21.15.69:0/2313140081 shutdown_connections 2024-08-02T18:00:57.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:00:56.913+0000 7f13aa0fa640 1 -- 172.21.15.69:0/2313140081 wait complete. 2024-08-02T18:00:58.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:57 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:00:58.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:57 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:00:58.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:57 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:00:59.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:58 smithi083 ceph-mon[37031]: pgmap v455: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.2 MiB/s wr, 518 op/s 2024-08-02T18:00:59.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:58 smithi077 ceph-mon[37108]: pgmap v455: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.2 MiB/s wr, 518 op/s 2024-08-02T18:00:59.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:58 smithi069 ceph-mon[32746]: pgmap v455: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.2 MiB/s wr, 518 op/s 2024-08-02T18:00:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.328+0000 7f13ebfff640 1 -- 172.21.15.69:0/1073170078 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec030950 con 0x7f13f8109840 2024-08-02T18:00:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.328+0000 7f13ff35c640 1 -- 172.21.15.69:0/1073170078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 msgr2=0x7f13f81142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:59.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.328+0000 7f13ff35c640 1 --2- 172.21.15.69:0/1073170078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 0x7f13f81142a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009860 tx=0x7f13ec02a340 comp rx=0 tx=0).stop 2024-08-02T18:00:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.334+0000 7f13ff35c640 1 -- 172.21.15.69:0/1073170078 shutdown_connections 2024-08-02T18:00:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.334+0000 7f13ff35c640 1 --2- 172.21.15.69:0/1073170078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f81147e0 0x7f13f8116bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.334+0000 7f13ff35c640 1 --2- 172.21.15.69:0/1073170078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 0x7f13f81142a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.334+0000 7f13ff35c640 1 --2- 172.21.15.69:0/1073170078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f8108f20 0x7f13f8109300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.334+0000 7f13ff35c640 1 -- 172.21.15.69:0/1073170078 >> 172.21.15.69:0/1073170078 conn(0x7f13f806cb20 msgr2=0x7f13f806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.340+0000 7f13ff35c640 1 -- 172.21.15.69:0/1073170078 shutdown_connections 2024-08-02T18:00:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.342+0000 7f13ff35c640 1 -- 172.21.15.69:0/1073170078 wait complete. 2024-08-02T18:00:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.343+0000 7f13ff35c640 1 Processor -- start 2024-08-02T18:00:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.343+0000 7f13ff35c640 1 -- start start 2024-08-02T18:00:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.344+0000 7f13ff35c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f8108f20 0x7f13f811e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.344+0000 7f13ff35c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 0x7f13f811ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.344+0000 7f13ff35c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f81147e0 0x7f13f8119840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.344+0000 7f13ff35c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81181d0 con 0x7f13f81147e0 2024-08-02T18:00:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.344+0000 7f13ff35c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8118050 con 0x7f13f8109840 2024-08-02T18:00:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.344+0000 7f13ff35c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8118350 con 0x7f13f8108f20 2024-08-02T18:00:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.345+0000 7f13fc8f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 0x7f13f811ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.345+0000 7f13fc8f7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 0x7f13f811ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:38480/0 (socket says 172.21.15.69:38480) 2024-08-02T18:00:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.345+0000 7f13fc8f7640 1 -- 172.21.15.69:0/4249407206 learned_addr learned my addr 172.21.15.69:0/4249407206 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.345+0000 7f13fd8f9640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f81147e0 0x7f13f8119840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.346+0000 7f13fd8f9640 1 -- 172.21.15.69:0/4249407206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f8108f20 msgr2=0x7f13f811e730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T18:00:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.346+0000 7f13fd8f9640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f8108f20 0x7f13f811e730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.346+0000 7f13fd8f9640 1 -- 172.21.15.69:0/4249407206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 msgr2=0x7f13f811ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.346+0000 7f13fd8f9640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 0x7f13f811ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.346+0000 7f13fd8f9640 1 -- 172.21.15.69:0/4249407206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f8119eb0 con 0x7f13f81147e0 2024-08-02T18:00:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.347+0000 7f13fd8f9640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f81147e0 0x7f13f8119840 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f13f000da10 tx=0x7f13f000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.356+0000 7f13ea7fc640 1 -- 172.21.15.69:0/4249407206 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f000bd70 con 0x7f13f81147e0 2024-08-02T18:00:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.356+0000 7f13ea7fc640 1 -- 172.21.15.69:0/4249407206 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f0003e50 con 0x7f13f81147e0 2024-08-02T18:00:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.356+0000 7f13ea7fc640 1 -- 172.21.15.69:0/4249407206 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f00106a0 con 0x7f13f81147e0 2024-08-02T18:00:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.357+0000 7f13ff35c640 1 -- 172.21.15.69:0/4249407206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f811a1a0 con 0x7f13f81147e0 2024-08-02T18:00:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.357+0000 7f13ff35c640 1 -- 172.21.15.69:0/4249407206 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f81bc3d0 con 0x7f13f81147e0 2024-08-02T18:00:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.360+0000 7f1402184740 1 Processor -- start 2024-08-02T18:00:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.363+0000 7f13ea7fc640 1 -- 172.21.15.69:0/4249407206 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f13f0003970 con 0x7f13f81147e0 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f1402184740 1 -- start start 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f1402184740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ecffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f1402184740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed0a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f1402184740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 0x557a7ed0cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f1402184740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557a7ec639b0 con 0x557a7ed000d0 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f1402184740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a7ec0b800 con 0x557a7ed0a970 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f1402184740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a7ec5f970 con 0x557a7ecff7b0 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f13fc8f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ecffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f13fc8f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ecffb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43592/0 (socket says 172.21.15.69:43592) 2024-08-02T18:00:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f13fc8f7640 1 -- 172.21.15.69:0/1421832100 learned_addr learned my addr 172.21.15.69:0/1421832100 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.364+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed0a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 0x557a7ed0cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13fc8f7640 1 -- 172.21.15.69:0/1421832100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 msgr2=0x557a7ed0cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 0x557a7ed0cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13fc8f7640 1 -- 172.21.15.69:0/1421832100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 msgr2=0x557a7ed0a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed0a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13fc8f7640 1 -- 172.21.15.69:0/1421832100 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7e977210 con 0x557a7ecff7b0 2024-08-02T18:00:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ecffb90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f13ec005d00 tx=0x7f13ec009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84902866168fcb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13e37fe640 1 -- 172.21.15.69:0/1421832100 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec037040 con 0x557a7ecff7b0 2024-08-02T18:00:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13e37fe640 1 -- 172.21.15.69:0/1421832100 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13ec004240 con 0x557a7ecff7b0 2024-08-02T18:00:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.365+0000 7f13e37fe640 1 -- 172.21.15.69:0/1421832100 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0046c0 con 0x557a7ecff7b0 2024-08-02T18:00:59.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.366+0000 7f13ea7fc640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13e40a2eb0 0x7f13e40a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.372 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.366+0000 7f13ea7fc640 1 -- 172.21.15.69:0/4249407206 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f13f00cef60 con 0x7f13f81147e0 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.366+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13e40a2eb0 0x7f13e40a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.366+0000 7f1402184740 1 -- 172.21.15.69:0/1421832100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 msgr2=0x557a7ecffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.366+0000 7f1402184740 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ecffb90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f13ec005d00 tx=0x7f13ec009550 comp rx=0 tx=0).stop 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.367+0000 7f1402184740 1 -- 172.21.15.69:0/1421832100 shutdown_connections 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.367+0000 7f1402184740 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed0a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.367+0000 7f1402184740 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 0x557a7ed0cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.367+0000 7f1402184740 1 --2- 172.21.15.69:0/1421832100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ecffb90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.367+0000 7f1402184740 1 -- 172.21.15.69:0/1421832100 >> 172.21.15.69:0/1421832100 conn(0x557a7ecfbd20 msgr2=0x557a7ecfc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.367+0000 7f1402184740 1 -- 172.21.15.69:0/1421832100 shutdown_connections 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.367+0000 7f1402184740 1 -- 172.21.15.69:0/1421832100 wait complete. 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.367+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13e40a2eb0 0x7f13e40a5370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557a7ecfcb80 tx=0x7f13f4009260 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.368+0000 7f1402184740 1 -- start start 2024-08-02T18:00:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.368+0000 7f1402184740 1 Processor -- start 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f1402184740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ecff7b0 0x557a7ed07940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f1402184740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed000d0 0x557a7ed07e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f1402184740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 0x557a7ed083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f1402184740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557a7ed0d540 con 0x557a7ecff7b0 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f1402184740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x557a7ec1f500 con 0x557a7ed0a970 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f1402184740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x557a7ed0d6c0 con 0x557a7ed000d0 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f1402184740 10 client.0 _ll_register_callbacks cb 0x557a7e976a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f13fd8f9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 0x557a7ed083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f13fd0f8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ecff7b0 0x557a7ed07940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f13fd0f8640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ecff7b0 0x557a7ed07940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33660/0 (socket says 172.21.15.69:33660) 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.369+0000 7f13fd0f8640 1 -- 172.21.15.69:0/964791811 learned_addr learned my addr 172.21.15.69:0/964791811 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.370+0000 7f13fd0f8640 1 -- 172.21.15.69:0/964791811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed000d0 msgr2=0x557a7ed07e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:00:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.370+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed000d0 0x557a7ed07e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.370+0000 7f13fd0f8640 1 -- 172.21.15.69:0/964791811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 msgr2=0x557a7ed083c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.370+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 0x557a7ed083c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.370+0000 7f13fd0f8640 1 -- 172.21.15.69:0/964791811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7ea43620 con 0x557a7ecff7b0 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.370+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ecff7b0 0x557a7ed07940 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f13f4012800 tx=0x7f13f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98c88006c3a4cc0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.370+0000 7f13e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.371+0000 7f13e0ff9640 1 -- 172.21.15.69:0/964791811 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4004220 con 0x557a7ecff7b0 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.371+0000 7f13e0ff9640 1 -- 172.21.15.69:0/964791811 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f13f40043c0 con 0x557a7ecff7b0 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.371+0000 7f1402184740 1 -- 172.21.15.69:0/964791811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ec004240 con 0x557a7ecff7b0 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.371+0000 7f13e0ff9640 1 -- 172.21.15.69:0/964791811 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13f402f050 con 0x557a7ecff7b0 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.371+0000 7f1402184740 10 client.19116 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.371+0000 7f1402184740 1 -- 172.21.15.69:0/964791811 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a7ed19330 con 0x557a7ecff7b0 2024-08-02T18:00:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.372+0000 7f13e0ff9640 1 -- 172.21.15.69:0/964791811 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f400fe30 con 0x557a7ecff7b0 2024-08-02T18:00:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.372+0000 7f13e0ff9640 1 -- 172.21.15.69:0/964791811 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13f401f420 con 0x557a7ecff7b0 2024-08-02T18:00:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.372+0000 7f1402184740 10 client.19116 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.372+0000 7f1402184740 10 client.19116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.372+0000 7f1402184740 20 client.19116 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.372+0000 7f1402184740 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557a7ed1a7f0 0x557a7ed1cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.372+0000 7f1402184740 4 client.19116 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:00:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.372+0000 7f1402184740 1 -- 172.21.15.69:0/964791811 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13ec004240 con 0x557a7ed1a7f0 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.373+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557a7ed1a7f0 0x557a7ed1cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.373+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557a7ed1a7f0 0x557a7ed1cc60 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8be8096c2e0e948 server_cookie=7d7daae394c89fe in_seq=0 out_seq=0 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.373+0000 7f13e0ff9640 10 client.19116 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.375+0000 7f13e0ff9640 1 -- 172.21.15.69:0/964791811 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f13ec004240 con 0x557a7ed1a7f0 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.375+0000 7f13e0ff9640 10 client.19116 handle_command_reply: tid=0 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.375+0000 7f1402184740 1 client.19116 shutdown 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.375+0000 7f1402184740 1 -- 172.21.15.69:0/964791811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ecff7b0 msgr2=0x557a7ed07940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.375+0000 7f1402184740 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ecff7b0 0x557a7ed07940 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f13f4012800 tx=0x7f13f4004730 comp rx=0 tx=0).stop 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 1 -- 172.21.15.69:0/964791811 shutdown_connections 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557a7ed1a7f0 0x557a7ed1cc60 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f13e0ff9640 0 client.19116 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed0a970 0x557a7ed083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed000d0 0x557a7ed07e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 1 --2- 172.21.15.69:0/964791811 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ecff7b0 0x557a7ed07940 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 1 -- 172.21.15.69:0/964791811 >> 172.21.15.69:0/964791811 conn(0x557a7ecfbd20 msgr2=0x557a7ed01a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 1 -- 172.21.15.69:0/964791811 shutdown_connections 2024-08-02T18:00:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 1 -- 172.21.15.69:0/964791811 wait complete. 2024-08-02T18:00:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.376+0000 7f1402184740 20 client.19116 trim_cache size 0 max 16384 2024-08-02T18:00:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.399+0000 7f1402184740 1 Processor -- start 2024-08-02T18:00:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.399+0000 7f1402184740 1 -- start start 2024-08-02T18:00:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.399+0000 7f1402184740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ecff7b0 0x557a7ed02270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.399+0000 7f1402184740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed22570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.399+0000 7f1402184740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed0a970 0x557a7ed22ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f13fd0f8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ecff7b0 0x557a7ed02270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f13fd0f8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ecff7b0 0x557a7ed02270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:38508/0 (socket says 172.21.15.69:38508) 2024-08-02T18:00:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f13fc8f7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed22570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f13fc8f7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed22570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33676/0 (socket says 172.21.15.69:33676) 2024-08-02T18:00:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f13fd0f8640 1 -- 172.21.15.69:0/3534181238 learned_addr learned my addr 172.21.15.69:0/3534181238 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f13fd8f9640 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed0a970 0x557a7ed22ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f1402184740 1 -- 172.21.15.69:0/3534181238 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81c8dc0 con 0x557a7ed000d0 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f1402184740 1 -- 172.21.15.69:0/3534181238 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81c6690 con 0x557a7ecff7b0 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.400+0000 7f1402184740 1 -- 172.21.15.69:0/3534181238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81c44e0 con 0x557a7ed0a970 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13fd8f9640 1 -- 172.21.15.69:0/3534181238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ecff7b0 msgr2=0x557a7ed02270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13fd8f9640 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ecff7b0 0x557a7ed02270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13fd8f9640 1 -- 172.21.15.69:0/3534181238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 msgr2=0x557a7ed22570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13fd8f9640 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed22570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13fd8f9640 1 -- 172.21.15.69:0/3534181238 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a7ed05e70 con 0x557a7ed0a970 2024-08-02T18:00:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed22570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13fd8f9640 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed0a970 0x557a7ed22ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f13f00cc2a0 tx=0x7f13f009b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b37e71af96e0990 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13e2ffd640 1 -- 172.21.15.69:0/3534181238 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f00b3070 con 0x557a7ed0a970 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.401+0000 7f13e2ffd640 1 -- 172.21.15.69:0/3534181238 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f00a3d60 con 0x557a7ed0a970 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f13e2ffd640 1 -- 172.21.15.69:0/3534181238 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f00977f0 con 0x557a7ed0a970 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 -- 172.21.15.69:0/3534181238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed0a970 msgr2=0x557a7ed22ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed0a970 0x557a7ed22ab0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f13f00cc2a0 tx=0x7f13f009b940 comp rx=0 tx=0).stop 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 -- 172.21.15.69:0/3534181238 shutdown_connections 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed000d0 0x557a7ed22570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ed0a970 0x557a7ed22ab0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 --2- 172.21.15.69:0/3534181238 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ecff7b0 0x557a7ed02270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 -- 172.21.15.69:0/3534181238 >> 172.21.15.69:0/3534181238 conn(0x557a7ecfbd20 msgr2=0x557a7ed098b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 -- 172.21.15.69:0/3534181238 shutdown_connections 2024-08-02T18:00:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.402+0000 7f1402184740 1 -- 172.21.15.69:0/3534181238 wait complete. 2024-08-02T18:00:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.403+0000 7f1402184740 1 -- start start 2024-08-02T18:00:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.403+0000 7f1402184740 1 Processor -- start 2024-08-02T18:00:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f1402184740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ed12d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f13fd0f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ed12d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f13fd0f8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ed12d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43620/0 (socket says 172.21.15.69:43620) 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f1402184740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed000d0 0x557a7ed13240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f1402184740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed0a970 0x557a7ed0f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f1402184740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81c4b50 con 0x557a7ed0a970 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f1402184740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81c5fc0 con 0x557a7ed000d0 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f1402184740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f13f81c5220 con 0x557a7ecff7b0 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f1402184740 10 client.0 _ll_register_callbacks cb 0x7f13f4009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f13fd0f8640 1 -- 172.21.15.69:0/3700056061 learned_addr learned my addr 172.21.15.69:0/3700056061 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.404+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed000d0 0x557a7ed13240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.405+0000 7f13fd0f8640 1 -- 172.21.15.69:0/3700056061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed000d0 msgr2=0x557a7ed13240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.405+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed000d0 0x557a7ed13240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.405+0000 7f13fd0f8640 1 -- 172.21.15.69:0/3700056061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed0a970 msgr2=0x557a7ed0f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.405+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed0a970 0x557a7ed0f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.405+0000 7f13fd0f8640 1 -- 172.21.15.69:0/3700056061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a7ed0fb00 con 0x557a7ecff7b0 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.405+0000 7f13fd0f8640 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ed12d00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f13f401f870 tx=0x7f13f401f650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f32777417b7e94d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.405+0000 7f13e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.405+0000 7f13e0ff9640 1 -- 172.21.15.69:0/3700056061 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f401fc70 con 0x557a7ecff7b0 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.406+0000 7f13e0ff9640 1 -- 172.21.15.69:0/3700056061 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f13f4004040 con 0x557a7ecff7b0 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.406+0000 7f1402184740 1 -- 172.21.15.69:0/3700056061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f00a3d60 con 0x557a7ecff7b0 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.406+0000 7f13e0ff9640 1 -- 172.21.15.69:0/3700056061 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f13f4004240 con 0x557a7ecff7b0 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.406+0000 7f1402184740 10 client.28240 fetch_fsmap learned FSMap version 26 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.406+0000 7f1402184740 1 -- 172.21.15.69:0/3700056061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557a7ed861d0 con 0x557a7ecff7b0 2024-08-02T18:00:59.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f13e0ff9640 1 -- 172.21.15.69:0/3700056061 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13f402c050 con 0x557a7ecff7b0 2024-08-02T18:00:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f13e0ff9640 1 -- 172.21.15.69:0/3700056061 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f13f4003890 con 0x557a7ecff7b0 2024-08-02T18:00:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f1402184740 10 client.28240 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:00:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f1402184740 10 client.28240 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:00:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f1402184740 20 client.28240 populate_metadata read hostname 'smithi069' 2024-08-02T18:00:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f1402184740 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557a7ed1a7f0 0x557a7ed888e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:00:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f1402184740 4 client.28240 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:00:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f1402184740 1 -- 172.21.15.69:0/3700056061 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f13f00a3d60 con 0x557a7ed1a7f0 2024-08-02T18:00:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.407+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557a7ed1a7f0 0x557a7ed888e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:00:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.408+0000 7f13fc8f7640 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557a7ed1a7f0 0x557a7ed888e0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc01c4ddc046a5b server_cookie=3590411dcd05e6ea in_seq=0 out_seq=0 2024-08-02T18:00:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.408+0000 7f13e0ff9640 10 client.28240 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f13e0ff9640 1 -- 172.21.15.69:0/3700056061 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f13f00a3d60 con 0x557a7ed1a7f0 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f13e0ff9640 10 client.28240 handle_command_reply: tid=0 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 client.28240 shutdown 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 -- 172.21.15.69:0/3700056061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 msgr2=0x557a7ed12d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ed12d00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f13f401f870 tx=0x7f13f401f650 comp rx=0 tx=0).stop 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 -- 172.21.15.69:0/3700056061 shutdown_connections 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x557a7ed0a970 0x557a7ed0f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x557a7ed1a7f0 0x557a7ed888e0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x557a7ed000d0 0x557a7ed13240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 --2- 172.21.15.69:0/3700056061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x557a7ecff7b0 0x557a7ed12d00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 -- 172.21.15.69:0/3700056061 >> 172.21.15.69:0/3700056061 conn(0x557a7ecfbd20 msgr2=0x557a7ecfa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f13e0ff9640 0 client.28240 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 -- 172.21.15.69:0/3700056061 shutdown_connections 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 1 -- 172.21.15.69:0/3700056061 wait complete. 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.422+0000 7f1402184740 20 client.28240 trim_cache size 0 max 16384 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:00:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:00:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 -- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13e40a2eb0 msgr2=0x7f13e40a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13e40a2eb0 0x7f13e40a5370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557a7ecfcb80 tx=0x7f13f4009260 comp rx=0 tx=0).stop 2024-08-02T18:00:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 -- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f81147e0 msgr2=0x7f13f8119840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:00:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f81147e0 0x7f13f8119840 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f13f000da10 tx=0x7f13f000ded0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 -- 172.21.15.69:0/4249407206 shutdown_connections 2024-08-02T18:00:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f13f81147e0 0x7f13f8119840 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13e40a2eb0 0x7f13e40a5370 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f13f8109840 0x7f13f811ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 --2- 172.21.15.69:0/4249407206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f13f8108f20 0x7f13f811e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:00:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 -- 172.21.15.69:0/4249407206 >> 172.21.15.69:0/4249407206 conn(0x7f13f806cb20 msgr2=0x7f13f8070650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:00:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 -- 172.21.15.69:0/4249407206 shutdown_connections 2024-08-02T18:00:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:00:59.423+0000 7f13ff35c640 1 -- 172.21.15.69:0/4249407206 wait complete. 2024-08-02T18:00:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 830.0 s, 446.0 tps, lat 216.252 ms stddev 260.616 2024-08-02T18:01:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:59 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:01:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:59 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:01:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:00:59 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:01:00.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:59 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:01:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:59 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:01:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:00:59 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:01:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:59 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:01:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:59 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:01:00.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:00:59 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:01:00.366 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T18:01:00.366 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T18:01:00.927 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:01:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:01 smithi083 ceph-mon[37031]: pgmap v456: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.5 MiB/s wr, 652 op/s 2024-08-02T18:01:01.272 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:01 smithi069 ceph-mon[32746]: pgmap v456: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.5 MiB/s wr, 652 op/s 2024-08-02T18:01:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:01 smithi077 ceph-mon[37108]: pgmap v456: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.5 MiB/s wr, 652 op/s 2024-08-02T18:01:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.579+0000 7f9418635640 1 -- 172.21.15.69:0/4204443623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9410109180 msgr2=0x7f941010ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.579+0000 7f9418635640 1 --2- 172.21.15.69:0/4204443623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9410109180 0x7f941010ff80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f940c009860 tx=0x7f940c02a380 comp rx=0 tx=0).stop 2024-08-02T18:01:01.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.580+0000 7f9418635640 1 -- 172.21.15.69:0/4204443623 shutdown_connections 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.580+0000 7f9418635640 1 --2- 172.21.15.69:0/4204443623 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f941011ae70 0x7f941011d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.580+0000 7f9418635640 1 --2- 172.21.15.69:0/4204443623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9410109560 0x7f94101099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.580+0000 7f9418635640 1 --2- 172.21.15.69:0/4204443623 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9410109180 0x7f941010ff80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.580+0000 7f9418635640 1 -- 172.21.15.69:0/4204443623 >> 172.21.15.69:0/4204443623 conn(0x7f941006de70 msgr2=0x7f941006e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.580+0000 7f9418635640 1 -- 172.21.15.69:0/4204443623 shutdown_connections 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.580+0000 7f9418635640 1 -- 172.21.15.69:0/4204443623 wait complete. 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.581+0000 7f9418635640 1 Processor -- start 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.581+0000 7f9418635640 1 -- start start 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.581+0000 7f9418635640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9410109180 0x7f941011a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.581+0000 7f9418635640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9410109560 0x7f94101165b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.581+0000 7f9418635640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f941011ae70 0x7f9410116af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.581+0000 7f9418635640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9410113030 con 0x7f941011ae70 2024-08-02T18:01:01.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.581+0000 7f9418635640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9410112eb0 con 0x7f9410109180 2024-08-02T18:01:01.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.581+0000 7f9418635640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f94101131b0 con 0x7f9410109560 2024-08-02T18:01:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9415bd0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9410109560 0x7f94101165b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9415bd0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9410109560 0x7f94101165b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43640/0 (socket says 172.21.15.69:43640) 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9416bd2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f941011ae70 0x7f9410116af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9415bd0640 1 -- 172.21.15.69:0/683077323 learned_addr learned my addr 172.21.15.69:0/683077323 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f94163d1640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9410109180 0x7f941011a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9415bd0640 1 -- 172.21.15.69:0/683077323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9410109180 msgr2=0x7f941011a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9415bd0640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9410109180 0x7f941011a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9415bd0640 1 -- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f941011ae70 msgr2=0x7f9410116af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9415bd0640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f941011ae70 0x7f9410116af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.584+0000 7f9415bd0640 1 -- 172.21.15.69:0/683077323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9410117220 con 0x7f9410109560 2024-08-02T18:01:01.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.585+0000 7f9415bd0640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9410109560 0x7f94101165b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f940001fa60 tx=0x7f940001ff20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:01.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.591+0000 7f94077fe640 1 -- 172.21.15.69:0/683077323 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940001dc50 con 0x7f9410109560 2024-08-02T18:01:01.592 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.591+0000 7f9418635640 1 -- 172.21.15.69:0/683077323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94101aaea0 con 0x7f9410109560 2024-08-02T18:01:01.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.591+0000 7f9418635640 1 -- 172.21.15.69:0/683077323 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94101ab240 con 0x7f9410109560 2024-08-02T18:01:01.593 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.592+0000 7f94077fe640 1 -- 172.21.15.69:0/683077323 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940001ddf0 con 0x7f9410109560 2024-08-02T18:01:01.595 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.594+0000 7f94077fe640 1 -- 172.21.15.69:0/683077323 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94000216a0 con 0x7f9410109560 2024-08-02T18:01:01.597 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.596+0000 7f94077fe640 1 -- 172.21.15.69:0/683077323 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9400021880 con 0x7f9410109560 2024-08-02T18:01:01.601 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.600+0000 7f941aa33740 1 Processor -- start 2024-08-02T18:01:01.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.601+0000 7f94077fe640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.601+0000 7f94077fe640 1 -- 172.21.15.69:0/683077323 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f94000e1be0 con 0x7f9410109560 2024-08-02T18:01:01.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.601+0000 7f941aa33740 1 -- start start 2024-08-02T18:01:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.602+0000 7f941aa33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6200 0x5581396a65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.602+0000 7f941aa33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6b20 0x5581396b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.602+0000 7f941aa33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.602+0000 7f941aa33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558139669e40 con 0x5581396a6b20 2024-08-02T18:01:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.602+0000 7f941aa33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55813960b7f0 con 0x5581396b13e0 2024-08-02T18:01:01.604 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.602+0000 7f941aa33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581395b2fb0 con 0x5581396a6200 2024-08-02T18:01:01.610 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.605+0000 7f94163d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6200 0x5581396a65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.605+0000 7f94163d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6200 0x5581396a65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43644/0 (socket says 172.21.15.69:43644) 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.605+0000 7f94163d1640 1 -- 172.21.15.69:0/3180596026 learned_addr learned my addr 172.21.15.69:0/3180596026 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.605+0000 7f9416bd2640 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.605+0000 7f9416bd2640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.606+0000 7f9415bd0640 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6b20 0x5581396b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.606+0000 7f94163d1640 1 -- 172.21.15.69:0/3180596026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 msgr2=0x5581396b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.606+0000 7f94163d1640 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.606+0000 7f94163d1640 1 -- 172.21.15.69:0/3180596026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6b20 msgr2=0x5581396b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.606+0000 7f94163d1640 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6b20 0x5581396b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.606+0000 7f94163d1640 1 -- 172.21.15.69:0/3180596026 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55813937d6a0 con 0x5581396a6200 2024-08-02T18:01:01.611 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.608+0000 7f94163d1640 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6200 0x5581396a65e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f940c009780 tx=0x7f940c009370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbfadf3836c13dd8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.609+0000 7f9404ff9640 1 -- 172.21.15.69:0/3180596026 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c037040 con 0x5581396a6200 2024-08-02T18:01:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.609+0000 7f9404ff9640 1 -- 172.21.15.69:0/3180596026 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c0305b0 con 0x5581396a6200 2024-08-02T18:01:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.609+0000 7f9404ff9640 1 -- 172.21.15.69:0/3180596026 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c005500 con 0x5581396a6200 2024-08-02T18:01:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.609+0000 7f9416bd2640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9408009fd0 tx=0x7f9408005bd0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.610+0000 7f941aa33740 1 -- 172.21.15.69:0/3180596026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6200 msgr2=0x5581396a65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.612 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.610+0000 7f941aa33740 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6200 0x5581396a65e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f940c009780 tx=0x7f940c009370 comp rx=0 tx=0).stop 2024-08-02T18:01:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.611+0000 7f941aa33740 1 -- 172.21.15.69:0/3180596026 shutdown_connections 2024-08-02T18:01:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.611+0000 7f941aa33740 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6b20 0x5581396b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.611+0000 7f941aa33740 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.611+0000 7f941aa33740 1 --2- 172.21.15.69:0/3180596026 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6200 0x5581396a65e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.613 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.611+0000 7f941aa33740 1 -- 172.21.15.69:0/3180596026 >> 172.21.15.69:0/3180596026 conn(0x5581396a2790 msgr2=0x5581396a2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:01.614 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.613+0000 7f941aa33740 1 -- 172.21.15.69:0/3180596026 shutdown_connections 2024-08-02T18:01:01.618 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.616+0000 7f941aa33740 1 -- 172.21.15.69:0/3180596026 wait complete. 2024-08-02T18:01:01.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.617+0000 7f941aa33740 1 -- start start 2024-08-02T18:01:01.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.617+0000 7f941aa33740 1 Processor -- start 2024-08-02T18:01:01.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f941aa33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396a6200 0x5581396ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f941aa33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f941aa33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396b13e0 0x5581396aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f941aa33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5581395c6cb0 con 0x5581396b13e0 2024-08-02T18:01:01.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f941aa33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5581396077b0 con 0x5581396a6200 2024-08-02T18:01:01.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f941aa33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5581396b3fb0 con 0x5581396a6b20 2024-08-02T18:01:01.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f941aa33740 10 client.0 _ll_register_callbacks cb 0x55813931aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:01.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f94163d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396a6200 0x5581396ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f94163d1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396a6200 0x5581396ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:38546/0 (socket says 172.21.15.69:38546) 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.619+0000 7f94163d1640 1 -- 172.21.15.69:0/2147768848 learned_addr learned my addr 172.21.15.69:0/2147768848 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f94163d1640 1 -- 172.21.15.69:0/2147768848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 msgr2=0x5581396ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f94163d1640 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f94163d1640 1 -- 172.21.15.69:0/2147768848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396b13e0 msgr2=0x5581396aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f94163d1640 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396b13e0 0x5581396aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f94163d1640 1 -- 172.21.15.69:0/2147768848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581395c5bc0 con 0x5581396a6200 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f94163d1640 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396a6200 0x5581396ae2e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f940c031000 tx=0x7f940c030ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=587e0a02138f4bf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f93f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f93f9ffb640 1 -- 172.21.15.69:0/2147768848 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c049070 con 0x5581396a6200 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f941aa33740 1 -- 172.21.15.69:0/2147768848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f940c0305b0 con 0x5581396a6200 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f93f9ffb640 1 -- 172.21.15.69:0/2147768848 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f940c005590 con 0x5581396a6200 2024-08-02T18:01:01.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.620+0000 7f93f9ffb640 1 -- 172.21.15.69:0/2147768848 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f940c037070 con 0x5581396a6200 2024-08-02T18:01:01.627 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.622+0000 7f941aa33740 10 client.28505 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.622+0000 7f941aa33740 1 -- 172.21.15.69:0/2147768848 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5581396bfee0 con 0x5581396a6200 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.622+0000 7f93f9ffb640 1 -- 172.21.15.69:0/2147768848 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c0305b0 con 0x5581396a6200 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.622+0000 7f93f9ffb640 1 -- 172.21.15.69:0/2147768848 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f940c032760 con 0x5581396a6200 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.623+0000 7f941aa33740 10 client.28505 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.623+0000 7f941aa33740 10 client.28505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.623+0000 7f941aa33740 20 client.28505 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.623+0000 7f941aa33740 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5581396c1230 0x5581396c3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.623+0000 7f941aa33740 4 client.28505 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.623+0000 7f941aa33740 1 -- 172.21.15.69:0/2147768848 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558139618d60 con 0x5581396c1230 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.624+0000 7f9415bd0640 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5581396c1230 0x5581396c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.624+0000 7f93f9ffb640 10 client.28505 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:01.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.625+0000 7f9415bd0640 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5581396c1230 0x5581396c3620 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55f276566aa5d743 server_cookie=28aa14af49c03498 in_seq=0 out_seq=0 2024-08-02T18:01:01.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.626+0000 7f93f9ffb640 1 -- 172.21.15.69:0/2147768848 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558139618d60 con 0x5581396c1230 2024-08-02T18:01:01.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.626+0000 7f93f9ffb640 10 client.28505 handle_command_reply: tid=0 2024-08-02T18:01:01.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.626+0000 7f941aa33740 1 client.28505 shutdown 2024-08-02T18:01:01.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.627+0000 7f941aa33740 1 -- 172.21.15.69:0/2147768848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396a6200 msgr2=0x5581396ae2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.627+0000 7f941aa33740 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396a6200 0x5581396ae2e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f940c031000 tx=0x7f940c030ee0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.629+0000 7f941aa33740 1 -- 172.21.15.69:0/2147768848 shutdown_connections 2024-08-02T18:01:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.629+0000 7f941aa33740 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396b13e0 0x5581396aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.629+0000 7f941aa33740 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5581396c1230 0x5581396c3620 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.629+0000 7f941aa33740 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.629+0000 7f941aa33740 1 --2- 172.21.15.69:0/2147768848 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396a6200 0x5581396ae2e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.629+0000 7f941aa33740 1 -- 172.21.15.69:0/2147768848 >> 172.21.15.69:0/2147768848 conn(0x5581396a2790 msgr2=0x5581396a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.629+0000 7f93f9ffb640 0 client.28505 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.630+0000 7f941aa33740 1 -- 172.21.15.69:0/2147768848 shutdown_connections 2024-08-02T18:01:01.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.630+0000 7f941aa33740 1 -- 172.21.15.69:0/2147768848 wait complete. 2024-08-02T18:01:01.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.634+0000 7f941aa33740 20 client.28505 trim_cache size 0 max 16384 2024-08-02T18:01:01.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.663+0000 7f941aa33740 1 Processor -- start 2024-08-02T18:01:01.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.663+0000 7f941aa33740 1 -- start start 2024-08-02T18:01:01.676 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f941aa33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396a17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f941aa33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f941aa33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f941aa33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93f4012940 con 0x5581396a6200 2024-08-02T18:01:01.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f941aa33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93f4010210 con 0x5581396b13e0 2024-08-02T18:01:01.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f941aa33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93f400e060 con 0x5581396a6b20 2024-08-02T18:01:01.677 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f94163d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396a17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f94163d1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396a17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33748/0 (socket says 172.21.15.69:33748) 2024-08-02T18:01:01.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f94163d1640 1 -- 172.21.15.69:0/4042175631 learned_addr learned my addr 172.21.15.69:0/4042175631 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:01.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f9415bd0640 1 --2- 172.21.15.69:0/4042175631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f94163d1640 1 -- 172.21.15.69:0/4042175631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 msgr2=0x5581396c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:01.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f94163d1640 1 --2- 172.21.15.69:0/4042175631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f94163d1640 1 -- 172.21.15.69:0/4042175631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 msgr2=0x5581396c9130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:01:01.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f94163d1640 1 --2- 172.21.15.69:0/4042175631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396c9130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.664+0000 7f94163d1640 1 -- 172.21.15.69:0/4042175631 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581396af2b0 con 0x5581396a6200 2024-08-02T18:01:01.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.665+0000 7f94163d1640 1 --2- 172.21.15.69:0/4042175631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396a17e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f940c0093b0 tx=0x7f940c02a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6778a72ea6aad31 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:01.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.665+0000 7f93fbfff640 1 -- 172.21.15.69:0/4042175631 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c049070 con 0x5581396a6200 2024-08-02T18:01:01.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.665+0000 7f93fbfff640 1 -- 172.21.15.69:0/4042175631 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c0305b0 con 0x5581396a6200 2024-08-02T18:01:01.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.665+0000 7f93fbfff640 1 -- 172.21.15.69:0/4042175631 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c004940 con 0x5581396a6200 2024-08-02T18:01:01.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 -- 172.21.15.69:0/4042175631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 msgr2=0x5581396a17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 --2- 172.21.15.69:0/4042175631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396a17e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f940c0093b0 tx=0x7f940c02a880 comp rx=0 tx=0).stop 2024-08-02T18:01:01.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 -- 172.21.15.69:0/4042175631 shutdown_connections 2024-08-02T18:01:01.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 --2- 172.21.15.69:0/4042175631 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 --2- 172.21.15.69:0/4042175631 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 --2- 172.21.15.69:0/4042175631 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396a17e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 -- 172.21.15.69:0/4042175631 >> 172.21.15.69:0/4042175631 conn(0x5581396a2790 msgr2=0x5581396a10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:01.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 -- 172.21.15.69:0/4042175631 shutdown_connections 2024-08-02T18:01:01.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 -- 172.21.15.69:0/4042175631 wait complete. 2024-08-02T18:01:01.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.666+0000 7f941aa33740 1 -- start start 2024-08-02T18:01:01.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f941aa33740 1 Processor -- start 2024-08-02T18:01:01.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f941aa33740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396b5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f941aa33740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396b5680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f941aa33740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f941aa33740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f93f400e6d0 con 0x5581396a6200 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f941aa33740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f93f400fb40 con 0x5581396b13e0 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f941aa33740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93f400eda0 con 0x5581396a6b20 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f941aa33740 10 client.0 _ll_register_callbacks cb 0x7f94101105a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f9415bd0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f9415bd0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43674/0 (socket says 172.21.15.69:43674) 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f9415bd0640 1 -- 172.21.15.69:0/1156175419 learned_addr learned my addr 172.21.15.69:0/1156175419 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f94163d1640 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.667+0000 7f9416bd2640 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f9415bd0640 1 -- 172.21.15.69:0/1156175419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 msgr2=0x5581396b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f9415bd0640 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f9415bd0640 1 -- 172.21.15.69:0/1156175419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 msgr2=0x5581396b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f9415bd0640 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f9415bd0640 1 -- 172.21.15.69:0/1156175419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813972ad60 con 0x5581396a6b20 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f9415bd0640 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396b5680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f94000adc40 tx=0x7f940001a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d92ea319a516392 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f93f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f93f9ffb640 1 -- 172.21.15.69:0/1156175419 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94000ca070 con 0x5581396a6b20 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f93f9ffb640 1 -- 172.21.15.69:0/1156175419 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f94000aa320 con 0x5581396a6b20 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f941aa33740 1 -- 172.21.15.69:0/1156175419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f940c0305b0 con 0x5581396a6b20 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f93f9ffb640 1 -- 172.21.15.69:0/1156175419 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f94000b2060 con 0x5581396a6b20 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f941aa33740 10 client.28255 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.668+0000 7f941aa33740 1 -- 172.21.15.69:0/1156175419 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55813972c0c0 con 0x5581396a6b20 2024-08-02T18:01:01.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f93f9ffb640 1 -- 172.21.15.69:0/1156175419 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c0305b0 con 0x5581396a6b20 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f93f9ffb640 1 -- 172.21.15.69:0/1156175419 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f94000b6b70 con 0x5581396a6b20 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f941aa33740 10 client.28255 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f941aa33740 10 client.28255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f941aa33740 20 client.28255 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f941aa33740 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55813972c870 0x55813972ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f941aa33740 4 client.28255 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f941aa33740 1 -- 172.21.15.69:0/1156175419 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558139613270 con 0x55813972c870 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.669+0000 7f94163d1640 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55813972c870 0x55813972ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.670+0000 7f94163d1640 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55813972c870 0x55813972ec90 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc1924483cd86497 server_cookie=a69bb2c2557daa2a in_seq=0 out_seq=0 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.670+0000 7f93f9ffb640 10 client.28255 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f93f9ffb640 1 -- 172.21.15.69:0/1156175419 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558139613270 con 0x55813972c870 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f93f9ffb640 10 client.28255 handle_command_reply: tid=0 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 client.28255 shutdown 2024-08-02T18:01:01.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 -- 172.21.15.69:0/1156175419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 msgr2=0x5581396b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396b5680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f94000adc40 tx=0x7f940001a410 comp rx=0 tx=0).stop 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 -- 172.21.15.69:0/1156175419 shutdown_connections 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55813972c870 0x55813972ec90 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5581396b13e0 0x5581396b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5581396a6b20 0x5581396b5680 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 --2- 172.21.15.69:0/1156175419 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5581396a6200 0x5581396b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 -- 172.21.15.69:0/1156175419 >> 172.21.15.69:0/1156175419 conn(0x5581396a2790 msgr2=0x5581396aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f93f9ffb640 0 client.28255 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 -- 172.21.15.69:0/1156175419 shutdown_connections 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 1 -- 172.21.15.69:0/1156175419 wait complete. 2024-08-02T18:01:01.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.671+0000 7f941aa33740 20 client.28255 trim_cache size 0 max 16384 2024-08-02T18:01:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:01:01.686 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T18:01:01.686 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "b16d0f5d-afec-412f-bba7-19c8b4745429", 2024-08-02T18:01:01.686 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T18:01:01.686 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:01:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 -- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 msgr2=0x7f93fc0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9408009fd0 tx=0x7f9408005bd0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 -- 172.21.15.69:0/683077323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9410109560 msgr2=0x7f94101165b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9410109560 0x7f94101165b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f940001fa60 tx=0x7f940001ff20 comp rx=0 tx=0).stop 2024-08-02T18:01:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 -- 172.21.15.69:0/683077323 shutdown_connections 2024-08-02T18:01:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f941011ae70 0x7f9410116af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f93fc0a2f00 0x7f93fc0a53c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9410109560 0x7f94101165b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 --2- 172.21.15.69:0/683077323 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9410109180 0x7f941011a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:01.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 -- 172.21.15.69:0/683077323 >> 172.21.15.69:0/683077323 conn(0x7f941006de70 msgr2=0x7f94100709b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:01.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 -- 172.21.15.69:0/683077323 shutdown_connections 2024-08-02T18:01:01.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:01.676+0000 7f9418635640 1 -- 172.21.15.69:0/683077323 wait complete. 2024-08-02T18:01:02.424 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T18:01:02.950 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:01:03.100 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:03 smithi069 ceph-mon[32746]: pgmap v457: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 403 op/s 2024-08-02T18:01:03.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:03 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T18:01:03.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:03 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:01:03.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:03 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T18:01:03.101 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:03 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:01:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:03 smithi077 ceph-mon[37108]: pgmap v457: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 403 op/s 2024-08-02T18:01:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:03 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T18:01:03.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:03 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:01:03.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:03 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T18:01:03.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:03 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:01:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:03 smithi083 ceph-mon[37031]: pgmap v457: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 3.9 MiB/s wr, 403 op/s 2024-08-02T18:01:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:03 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T18:01:03.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:03 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:01:03.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:03 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T18:01:03.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:03 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.611+0000 7f3be29be640 1 -- 172.21.15.69:0/3825454050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1078c0 msgr2=0x7f3bdc107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.611+0000 7f3be29be640 1 --2- 172.21.15.69:0/3825454050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1078c0 0x7f3bdc107ca0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc009900 tx=0x7f3bcc02a440 comp rx=0 tx=0).stop 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.616+0000 7f3be29be640 1 -- 172.21.15.69:0/3825454050 shutdown_connections 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.616+0000 7f3be29be640 1 --2- 172.21.15.69:0/3825454050 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bdc102260 0x7f3bdc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.616+0000 7f3be29be640 1 --2- 172.21.15.69:0/3825454050 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc1018c0 0x7f3bdc101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.616+0000 7f3be29be640 1 --2- 172.21.15.69:0/3825454050 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1078c0 0x7f3bdc107ca0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.616+0000 7f3be29be640 1 -- 172.21.15.69:0/3825454050 >> 172.21.15.69:0/3825454050 conn(0x7f3bdc0fd620 msgr2=0x7f3bdc0ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.620+0000 7f3be29be640 1 -- 172.21.15.69:0/3825454050 shutdown_connections 2024-08-02T18:01:03.673 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.621+0000 7f3be29be640 1 -- 172.21.15.69:0/3825454050 wait complete. 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.621+0000 7f3be29be640 1 Processor -- start 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3be29be640 1 -- start start 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3be29be640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1018c0 0x7f3bdc117020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3be29be640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc102260 0x7f3bdc117560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3be29be640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bdc1078c0 0x7f3bdc111170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3be29be640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10fad0 con 0x7f3bdc1078c0 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3be29be640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10f950 con 0x7f3bdc1018c0 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3be29be640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc10fc50 con 0x7f3bdc102260 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3bdbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1018c0 0x7f3bdc117020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.674 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3bdbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1018c0 0x7f3bdc117020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:38574/0 (socket says 172.21.15.69:38574) 2024-08-02T18:01:03.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.625+0000 7f3bdbfff640 1 -- 172.21.15.69:0/2869733086 learned_addr learned my addr 172.21.15.69:0/2869733086 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:03.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.626+0000 7f3bdbfff640 1 -- 172.21.15.69:0/2869733086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc102260 msgr2=0x7f3bdc117560 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T18:01:03.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.626+0000 7f3bdbfff640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc102260 0x7f3bdc117560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.626+0000 7f3bdbfff640 1 -- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bdc1078c0 msgr2=0x7f3bdc111170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:03.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.626+0000 7f3bdbfff640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bdc1078c0 0x7f3bdc111170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.675 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.626+0000 7f3bdbfff640 1 -- 172.21.15.69:0/2869733086 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bdc111a30 con 0x7f3bdc1018c0 2024-08-02T18:01:03.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.626+0000 7f3bdbfff640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1018c0 0x7f3bdc117020 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc000c00 tx=0x7f3bcc003e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:03.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.632+0000 7f3bd97fa640 1 -- 172.21.15.69:0/2869733086 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc040030 con 0x7f3bdc1018c0 2024-08-02T18:01:03.678 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.632+0000 7f3bd97fa640 1 -- 172.21.15.69:0/2869733086 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bcc004210 con 0x7f3bdc1018c0 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.632+0000 7f3bd97fa640 1 -- 172.21.15.69:0/2869733086 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0318e0 con 0x7f3bdc1018c0 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.632+0000 7f3be29be640 1 -- 172.21.15.69:0/2869733086 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bdc111cc0 con 0x7f3bdc1018c0 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.632+0000 7f3be29be640 1 -- 172.21.15.69:0/2869733086 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bdc1b3ab0 con 0x7f3bdc1018c0 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.637+0000 7f3bd97fa640 1 -- 172.21.15.69:0/2869733086 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3bcc043030 con 0x7f3bdc1018c0 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.638+0000 7f3be4dbc740 1 Processor -- start 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.638+0000 7f3bd97fa640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bc40a2f20 0x7f3bc40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.639+0000 7f3bd97fa640 1 -- 172.21.15.69:0/2869733086 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f3bcc0efff0 con 0x7f3bdc1018c0 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be4dbc740 1 -- start start 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 0x560e9e9215f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e92beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e92e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560e9e886980 con 0x560e9e92c3f0 2024-08-02T18:01:03.679 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560e9e81ea70 con 0x560e9e921210 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e9e882940 con 0x560e9e921b30 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be0f5b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e92e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be0f5b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e92e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:33778/0 (socket says 172.21.15.69:33778) 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be0f5b640 1 -- 172.21.15.69:0/3394730192 learned_addr learned my addr 172.21.15.69:0/3394730192 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.640+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bc40a2f20 0x7f3bc40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.641+0000 7f3be0f5b640 1 -- 172.21.15.69:0/3394730192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 msgr2=0x560e9e92beb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.641+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/3394730192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e92beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.641+0000 7f3be0f5b640 1 -- 172.21.15.69:0/3394730192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 msgr2=0x560e9e9215f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.641+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/3394730192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 0x560e9e9215f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.641+0000 7f3be0f5b640 1 -- 172.21.15.69:0/3394730192 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9e8816e0 con 0x560e9e92c3f0 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.642+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/3394730192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e92e7e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4012cb0 tx=0x7f3bd400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8d02cb1f314f3d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.642+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bc40a2f20 0x7f3bc40a53e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005c40 tx=0x7f3bd400c000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:03.680 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.644+0000 7f3bc27fc640 1 -- 172.21.15.69:0/3394730192 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4009a80 con 0x560e9e92c3f0 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.644+0000 7f3bc27fc640 1 -- 172.21.15.69:0/3394730192 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4024070 con 0x560e9e92c3f0 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.644+0000 7f3bc27fc640 1 -- 172.21.15.69:0/3394730192 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd401fe00 con 0x560e9e92c3f0 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.645+0000 7f3be4dbc740 1 -- 172.21.15.69:0/3394730192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 msgr2=0x560e9e92e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.645+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/3394730192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e92e7e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4012cb0 tx=0x7f3bd400f730 comp rx=0 tx=0).stop 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.645+0000 7f3be4dbc740 1 -- 172.21.15.69:0/3394730192 shutdown_connections 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.645+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/3394730192 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e92e7e0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.645+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/3394730192 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e92beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.645+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/3394730192 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 0x560e9e9215f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.645+0000 7f3be4dbc740 1 -- 172.21.15.69:0/3394730192 >> 172.21.15.69:0/3394730192 conn(0x560e9e91d7a0 msgr2=0x560e9e91db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.650+0000 7f3be4dbc740 1 -- 172.21.15.69:0/3394730192 shutdown_connections 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.650+0000 7f3be4dbc740 1 -- 172.21.15.69:0/3394730192 wait complete. 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.650+0000 7f3be4dbc740 1 -- start start 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3be4dbc740 1 Processor -- start 2024-08-02T18:01:03.681 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 0x560e9e929460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e9299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e929ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3bdbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 0x560e9e929460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3bdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e9299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3bdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e9299a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43708/0 (socket says 172.21.15.69:43708) 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3bdbfff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 0x560e9e929460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:38576/0 (socket says 172.21.15.69:38576) 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3bdb7fe640 1 -- 172.21.15.69:0/2629703344 learned_addr learned my addr 172.21.15.69:0/2629703344 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560e9e92ef00 con 0x560e9e92c3f0 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560e9e841aa0 con 0x560e9e921210 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.651+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560e9e92f080 con 0x560e9e921b30 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.652+0000 7f3be4dbc740 10 client.0 _ll_register_callbacks cb 0x560e9e830e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.653+0000 7f3bdb7fe640 1 -- 172.21.15.69:0/2629703344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 msgr2=0x560e9e929460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:03.682 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.653+0000 7f3bdb7fe640 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 0x560e9e929460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.653+0000 7f3bdb7fe640 1 -- 172.21.15.69:0/2629703344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 msgr2=0x560e9e929ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.653+0000 7f3bdb7fe640 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e929ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.653+0000 7f3bdb7fe640 1 -- 172.21.15.69:0/2629703344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9e825650 con 0x560e9e921b30 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.653+0000 7f3bdb7fe640 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e9299a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000af90 tx=0x7f3bd000c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31dadd476880439b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.653+0000 7f3bbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.656+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/2629703344 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd000f860 con 0x560e9e921b30 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.656+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/2629703344 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f3bd000fa00 con 0x560e9e921b30 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.657+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd4024070 con 0x560e9e921b30 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.661+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/2629703344 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3bd000fc00 con 0x560e9e921b30 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.661+0000 7f3be4dbc740 10 client.28258 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.661+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e9e5f83c0 con 0x560e9e921b30 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.662+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/2629703344 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd000fda0 con 0x560e9e921b30 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.663+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/2629703344 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3bd0018600 con 0x560e9e921b30 2024-08-02T18:01:03.683 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.663+0000 7f3be4dbc740 10 client.28258 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.663+0000 7f3be4dbc740 10 client.28258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.663+0000 7f3be4dbc740 20 client.28258 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.663+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560e9e93bf60 0x560e9e93e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.663+0000 7f3be4dbc740 4 client.28258 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.663+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3bd4024070 con 0x560e9e93bf60 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.663+0000 7f3bdbfff640 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560e9e93bf60 0x560e9e93e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.664+0000 7f3bdbfff640 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560e9e93bf60 0x560e9e93e380 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b098859b1c20ba server_cookie=1f856b7fe293f4fc in_seq=0 out_seq=0 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.664+0000 7f3bbb7fe640 10 client.28258 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.670+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/2629703344 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f3bd4024070 con 0x560e9e93bf60 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.670+0000 7f3bbb7fe640 10 client.28258 handle_command_reply: tid=0 2024-08-02T18:01:03.684 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.670+0000 7f3be4dbc740 1 client.28258 shutdown 2024-08-02T18:01:03.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.674+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 msgr2=0x560e9e9299a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.674+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e9299a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000af90 tx=0x7f3bd000c760 comp rx=0 tx=0).stop 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.675+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 shutdown_connections 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.675+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e92c3f0 0x560e9e929ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.675+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560e9e93bf60 0x560e9e93e380 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.675+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e9299a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.675+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/2629703344 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e921210 0x560e9e929460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.675+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 >> 172.21.15.69:0/2629703344 conn(0x560e9e91d7a0 msgr2=0x560e9e9234d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.675+0000 7f3bbb7fe640 0 client.28258 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.675+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 shutdown_connections 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.676+0000 7f3be4dbc740 1 -- 172.21.15.69:0/2629703344 wait complete. 2024-08-02T18:01:03.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.676+0000 7f3be4dbc740 20 client.28258 trim_cache size 0 max 16384 2024-08-02T18:01:03.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.709+0000 7f3be4dbc740 1 Processor -- start 2024-08-02T18:01:03.711 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.710+0000 7f3be4dbc740 1 -- start start 2024-08-02T18:01:03.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.710+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921210 0x560e9e929d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.710+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921b30 0x560e9e922cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.710+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e9231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.710+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1d9d10 con 0x560e9e921b30 2024-08-02T18:01:03.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.710+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1d75e0 con 0x560e9e92c3f0 2024-08-02T18:01:03.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.710+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc0739a0 con 0x560e9e921210 2024-08-02T18:01:03.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.711+0000 7f3be0f5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e9231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.714+0000 7f3be0f5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e9231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:38586/0 (socket says 172.21.15.69:38586) 2024-08-02T18:01:03.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.714+0000 7f3bdbfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921210 0x560e9e929d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.714+0000 7f3be0f5b640 1 -- 172.21.15.69:0/4266715786 learned_addr learned my addr 172.21.15.69:0/4266715786 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.715+0000 7f3bdb7fe640 1 --2- 172.21.15.69:0/4266715786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921b30 0x560e9e922cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.717+0000 7f3be0f5b640 1 -- 172.21.15.69:0/4266715786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921210 msgr2=0x560e9e929d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.717+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/4266715786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921210 0x560e9e929d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.717+0000 7f3be0f5b640 1 -- 172.21.15.69:0/4266715786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921b30 msgr2=0x560e9e922cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.717+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/4266715786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921b30 0x560e9e922cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.717+0000 7f3be0f5b640 1 -- 172.21.15.69:0/4266715786 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9e91e910 con 0x560e9e92c3f0 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.718+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/4266715786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e9231f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40124b0 tx=0x7f3bd4009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6631fcb39ec49775 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.720+0000 7f3bc1ffb640 1 -- 172.21.15.69:0/4266715786 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd402c070 con 0x560e9e92c3f0 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.720+0000 7f3bc1ffb640 1 -- 172.21.15.69:0/4266715786 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4024040 con 0x560e9e92c3f0 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.720+0000 7f3bc1ffb640 1 -- 172.21.15.69:0/4266715786 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4005680 con 0x560e9e92c3f0 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.722+0000 7f3be4dbc740 1 -- 172.21.15.69:0/4266715786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 msgr2=0x560e9e9231f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.722+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/4266715786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e9231f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40124b0 tx=0x7f3bd4009f40 comp rx=0 tx=0).stop 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.723+0000 7f3be4dbc740 1 -- 172.21.15.69:0/4266715786 shutdown_connections 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.723+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/4266715786 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921b30 0x560e9e922cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.723+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/4266715786 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e9231f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.723+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/4266715786 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921210 0x560e9e929d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.723+0000 7f3be4dbc740 1 -- 172.21.15.69:0/4266715786 >> 172.21.15.69:0/4266715786 conn(0x560e9e91d7a0 msgr2=0x560e9e9285b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:03.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.729+0000 7f3be4dbc740 1 -- 172.21.15.69:0/4266715786 shutdown_connections 2024-08-02T18:01:03.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.732+0000 7f3be4dbc740 1 -- 172.21.15.69:0/4266715786 wait complete. 2024-08-02T18:01:03.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.732+0000 7f3be4dbc740 1 -- start start 2024-08-02T18:01:03.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.732+0000 7f3be4dbc740 1 Processor -- start 2024-08-02T18:01:03.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.736+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921210 0x560e9e9344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.736+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e934a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.736+0000 7f3be4dbc740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e925730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.736+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc073f60 con 0x560e9e921210 2024-08-02T18:01:03.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.736+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1d6f10 con 0x560e9e92c3f0 2024-08-02T18:01:03.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.736+0000 7f3be4dbc740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3bdc1d6170 con 0x560e9e921b30 2024-08-02T18:01:03.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.736+0000 7f3be4dbc740 10 client.0 _ll_register_callbacks cb 0x7f3bd0005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:03.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.738+0000 7f3bdb7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e934a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.739+0000 7f3be0f5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e925730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.739+0000 7f3be0f5b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e925730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:38596/0 (socket says 172.21.15.69:38596) 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.739+0000 7f3be0f5b640 1 -- 172.21.15.69:0/370963100 learned_addr learned my addr 172.21.15.69:0/370963100 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.740+0000 7f3be0f5b640 1 -- 172.21.15.69:0/370963100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 msgr2=0x560e9e934a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.740+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e934a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.740+0000 7f3be0f5b640 1 -- 172.21.15.69:0/370963100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921210 msgr2=0x560e9e9344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.740+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921210 0x560e9e9344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.740+0000 7f3be0f5b640 1 -- 172.21.15.69:0/370963100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9e925f30 con 0x560e9e92c3f0 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.740+0000 7f3be0f5b640 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e925730 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40096f0 tx=0x7f3bd40093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f10875110074a95 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.740+0000 7f3bbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.741+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/370963100 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4038070 con 0x560e9e92c3f0 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.741+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/370963100 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9568+0+0 (secure 0 0 0) 0x7f3bd4004f30 con 0x560e9e92c3f0 2024-08-02T18:01:03.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.741+0000 7f3be4dbc740 1 -- 172.21.15.69:0/370963100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bd4024040 con 0x560e9e92c3f0 2024-08-02T18:01:03.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.742+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/370963100 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3bd402c040 con 0x560e9e92c3f0 2024-08-02T18:01:03.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.745+0000 7f3be4dbc740 10 client.28523 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:03.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.745+0000 7f3be4dbc740 1 -- 172.21.15.69:0/370963100 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560e9e927240 con 0x560e9e92c3f0 2024-08-02T18:01:03.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.748+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/370963100 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bd4024040 con 0x560e9e92c3f0 2024-08-02T18:01:03.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.748+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/370963100 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3bd401b070 con 0x560e9e92c3f0 2024-08-02T18:01:03.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.750+0000 7f3be4dbc740 10 client.28523 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:03.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.750+0000 7f3be4dbc740 10 client.28523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:03.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.750+0000 7f3be4dbc740 20 client.28523 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:03.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.750+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560e9e9a6d90 0x560e9e9a9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:03.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.750+0000 7f3be4dbc740 4 client.28523 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T18:01:03.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.750+0000 7f3be4dbc740 1 -- 172.21.15.69:0/370963100 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e9e893ef0 con 0x560e9e9a6d90 2024-08-02T18:01:03.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.754+0000 7f3bdbfff640 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560e9e9a6d90 0x560e9e9a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:03.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.756+0000 7f3bdbfff640 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560e9e9a6d90 0x560e9e9a9180 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b13ff79ef839448 server_cookie=7cff41d9c07d5f7a in_seq=0 out_seq=0 2024-08-02T18:01:03.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.759+0000 7f3bbb7fe640 10 client.28523 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:03.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.759+0000 7f3bbb7fe640 1 -- 172.21.15.69:0/370963100 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560e9e893ef0 con 0x560e9e9a6d90 2024-08-02T18:01:03.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.759+0000 7f3bbb7fe640 10 client.28523 handle_command_reply: tid=0 2024-08-02T18:01:03.762 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.759+0000 7f3be4dbc740 1 client.28523 shutdown 2024-08-02T18:01:03.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.763+0000 7f3be4dbc740 1 -- 172.21.15.69:0/370963100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 msgr2=0x560e9e925730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:03.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.763+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e925730 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40096f0 tx=0x7f3bd40093c0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.766 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.764+0000 7f3be4dbc740 1 -- 172.21.15.69:0/370963100 shutdown_connections 2024-08-02T18:01:03.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.764+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560e9e9a6d90 0x560e9e9a9180 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.764+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560e9e92c3f0 0x560e9e925730 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.764+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560e9e921b30 0x560e9e934a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.765+0000 7f3be4dbc740 1 --2- 172.21.15.69:0/370963100 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560e9e921210 0x560e9e9344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.765+0000 7f3be4dbc740 1 -- 172.21.15.69:0/370963100 >> 172.21.15.69:0/370963100 conn(0x560e9e91d7a0 msgr2=0x560e9e93ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:03.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.765+0000 7f3bbb7fe640 0 client.28523 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:03.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.766+0000 7f3be4dbc740 1 -- 172.21.15.69:0/370963100 shutdown_connections 2024-08-02T18:01:03.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.767+0000 7f3be4dbc740 1 -- 172.21.15.69:0/370963100 wait complete. 2024-08-02T18:01:03.769 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.767+0000 7f3be4dbc740 20 client.28523 trim_cache size 0 max 16384 2024-08-02T18:01:03.769 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:01:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T18:01:03.769 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T18:01:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "b16d0f5d-afec-412f-bba7-19c8b4745429": { 2024-08-02T18:01:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T18:01:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "b16d0f5d-afec-412f-bba7-19c8b4745429", 2024-08-02T18:01:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T18:01:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:01:03.770 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:01:03.770 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.768+0000 7f3be29be640 1 -- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bc40a2f20 msgr2=0x7f3bc40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.768+0000 7f3be29be640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bc40a2f20 0x7f3bc40a53e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4005c40 tx=0x7f3bd400c000 comp rx=0 tx=0).stop 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.768+0000 7f3be29be640 1 -- 172.21.15.69:0/2869733086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1018c0 msgr2=0x7f3bdc117020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.768+0000 7f3be29be640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1018c0 0x7f3bdc117020 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc000c00 tx=0x7f3bcc003e90 comp rx=0 tx=0).stop 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.769+0000 7f3be29be640 1 -- 172.21.15.69:0/2869733086 shutdown_connections 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.769+0000 7f3be29be640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3bdc1078c0 0x7f3bdc111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.769+0000 7f3be29be640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3bc40a2f20 0x7f3bc40a53e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.769+0000 7f3be29be640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3bdc102260 0x7f3bdc117560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.769+0000 7f3be29be640 1 --2- 172.21.15.69:0/2869733086 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3bdc1018c0 0x7f3bdc117020 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc000c00 tx=0x7f3bcc003e90 comp rx=0 tx=0).stop 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.769+0000 7f3be29be640 1 -- 172.21.15.69:0/2869733086 >> 172.21.15.69:0/2869733086 conn(0x7f3bdc0fd620 msgr2=0x7f3bdc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.769+0000 7f3be29be640 1 -- 172.21.15.69:0/2869733086 shutdown_connections 2024-08-02T18:01:03.772 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:03.769+0000 7f3be29be640 1 -- 172.21.15.69:0/2869733086 wait complete. 2024-08-02T18:01:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 835.0 s, 446.8 tps, lat 221.213 ms stddev 259.203 2024-08-02T18:01:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:05 smithi077 ceph-mon[37108]: pgmap v458: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.2 MiB/s wr, 562 op/s 2024-08-02T18:01:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:05 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T18:01:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:05 smithi069 ceph-mon[32746]: pgmap v458: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.2 MiB/s wr, 562 op/s 2024-08-02T18:01:05.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:05 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T18:01:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:05 smithi083 ceph-mon[37031]: pgmap v458: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.2 MiB/s wr, 562 op/s 2024-08-02T18:01:05.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:05 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T18:01:06.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:06 smithi077 ceph-mon[37108]: osdmap e112: 12 total, 12 up, 12 in 2024-08-02T18:01:06.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:06 smithi069 ceph-mon[32746]: osdmap e112: 12 total, 12 up, 12 in 2024-08-02T18:01:06.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:06 smithi083 ceph-mon[37031]: osdmap e112: 12 total, 12 up, 12 in 2024-08-02T18:01:07.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:07 smithi077 ceph-mon[37108]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.8 MiB/s wr, 596 op/s 2024-08-02T18:01:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:07 smithi069 ceph-mon[32746]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.8 MiB/s wr, 596 op/s 2024-08-02T18:01:07.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:07 smithi083 ceph-mon[37031]: pgmap v460: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.8 MiB/s wr, 596 op/s 2024-08-02T18:01:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:09 smithi077 ceph-mon[37108]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 4.9 MiB/s wr, 607 op/s 2024-08-02T18:01:09.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:09 smithi069 ceph-mon[32746]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 4.9 MiB/s wr, 607 op/s 2024-08-02T18:01:09.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:09 smithi083 ceph-mon[37031]: pgmap v461: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 4.9 MiB/s wr, 607 op/s 2024-08-02T18:01:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 840.0 s, 364.4 tps, lat 276.168 ms stddev 326.198 2024-08-02T18:01:10.386 INFO:tasks.cephfs.filesystem:scrub status for tag:b16d0f5d-afec-412f-bba7-19c8b4745429 - {'path': '/', 'tag': 'b16d0f5d-afec-412f-bba7-19c8b4745429', 'options': 'recursive,force'} 2024-08-02T18:01:11.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:11 smithi077 ceph-mon[37108]: pgmap v462: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.9 MiB/s wr, 597 op/s 2024-08-02T18:01:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:11 smithi069 ceph-mon[32746]: pgmap v462: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.9 MiB/s wr, 597 op/s 2024-08-02T18:01:11.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:11 smithi083 ceph-mon[37031]: pgmap v462: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.9 MiB/s wr, 597 op/s 2024-08-02T18:01:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.564+0000 7f12c730b640 1 -- 172.21.15.69:0/3301985218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c0107810 msgr2=0x7f12c0107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.564+0000 7f12c730b640 1 --2- 172.21.15.69:0/3301985218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c0107810 0x7f12c0107bf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009900 tx=0x7f12bc02a410 comp rx=0 tx=0).stop 2024-08-02T18:01:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.564+0000 7f12c730b640 1 -- 172.21.15.69:0/3301985218 shutdown_connections 2024-08-02T18:01:12.566 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.564+0000 7f12c730b640 1 --2- 172.21.15.69:0/3301985218 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12c0101800 0x7f12c0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.564+0000 7f12c730b640 1 --2- 172.21.15.69:0/3301985218 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12c01021a0 0x7f12c010e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.564+0000 7f12c730b640 1 --2- 172.21.15.69:0/3301985218 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c0107810 0x7f12c0107bf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.564+0000 7f12c730b640 1 -- 172.21.15.69:0/3301985218 >> 172.21.15.69:0/3301985218 conn(0x7f12c00fd500 msgr2=0x7f12c00ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.564+0000 7f12c730b640 1 -- 172.21.15.69:0/3301985218 shutdown_connections 2024-08-02T18:01:12.567 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.565+0000 7f12c730b640 1 -- 172.21.15.69:0/3301985218 wait complete. 2024-08-02T18:01:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.566+0000 7f12c730b640 1 Processor -- start 2024-08-02T18:01:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.566+0000 7f12c730b640 1 -- start start 2024-08-02T18:01:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.566+0000 7f12c730b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12c0101800 0x7f12c01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.566+0000 7f12c730b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c01021a0 0x7f12c0198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.566+0000 7f12c730b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12c0107810 0x7f12c019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.566+0000 7f12c50a7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12c0101800 0x7f12c01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.566+0000 7f12c50a7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12c0101800 0x7f12c01981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:41842/0 (socket says 172.21.15.69:41842) 2024-08-02T18:01:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c50a7640 1 -- 172.21.15.69:0/469157509 learned_addr learned my addr 172.21.15.69:0/469157509 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0111c50 con 0x7f12c0107810 2024-08-02T18:01:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0111ad0 con 0x7f12c01021a0 2024-08-02T18:01:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12c0111dd0 con 0x7f12c0101800 2024-08-02T18:01:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c48a6640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c01021a0 0x7f12c0198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c48a6640 1 -- 172.21.15.69:0/469157509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12c0101800 msgr2=0x7f12c01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c48a6640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12c0101800 0x7f12c01981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c48a6640 1 -- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12c0107810 msgr2=0x7f12c019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c48a6640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12c0107810 0x7f12c019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c48a6640 1 -- 172.21.15.69:0/469157509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12c019d1c0 con 0x7f12c01021a0 2024-08-02T18:01:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.567+0000 7f12c48a6640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c01021a0 0x7f12c0198710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12b000e9f0 tx=0x7f12b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.568+0000 7f12b67fc640 1 -- 172.21.15.69:0/469157509 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b000ccc0 con 0x7f12c01021a0 2024-08-02T18:01:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.568+0000 7f12b67fc640 1 -- 172.21.15.69:0/469157509 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b0010910 con 0x7f12c01021a0 2024-08-02T18:01:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.568+0000 7f12b67fc640 1 -- 172.21.15.69:0/469157509 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0018980 con 0x7f12c01021a0 2024-08-02T18:01:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.568+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12c019d4b0 con 0x7f12c01021a0 2024-08-02T18:01:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.568+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12c01a4d40 con 0x7f12c01021a0 2024-08-02T18:01:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.568+0000 7f12c50a7640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12c0101800 0x7f12c01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:01:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.571+0000 7f12ca05e740 1 Processor -- start 2024-08-02T18:01:12.573 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.572+0000 7f12b67fc640 1 -- 172.21.15.69:0/469157509 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f12b0010ac0 con 0x7f12c01021a0 2024-08-02T18:01:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.572+0000 7f12ca05e740 1 -- start start 2024-08-02T18:01:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.573+0000 7f12b67fc640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12ac0a2e40 0x7f12ac0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.573+0000 7f12b67fc640 1 -- 172.21.15.69:0/469157509 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9656+0+0 (secure 0 0 0) 0x7f12b00cfc30 con 0x7f12c01021a0 2024-08-02T18:01:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.574+0000 7f12c58a8640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12ac0a2e40 0x7f12ac0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.574+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 0x55ce013f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.574+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 0x55ce013f0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.574+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0cc0 0x55ce013fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.574+0000 7f12ca05e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ce0134a9b0 con 0x55ce013f0320 2024-08-02T18:01:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.574+0000 7f12ca05e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ce012f2800 con 0x55ce013f0cc0 2024-08-02T18:01:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.574+0000 7f12ca05e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ce01346970 con 0x55ce013f4980 2024-08-02T18:01:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.574+0000 7f12c58a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0cc0 0x55ce013fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.576 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.575+0000 7f12c48a6640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 0x55ce013f0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.575+0000 7f12c58a8640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0cc0 0x55ce013fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39248/0 (socket says 172.21.15.69:39248) 2024-08-02T18:01:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.575+0000 7f12c58a8640 1 -- 172.21.15.69:0/295897745 learned_addr learned my addr 172.21.15.69:0/295897745 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.575+0000 7f12c58a8640 1 -- 172.21.15.69:0/295897745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 msgr2=0x55ce013f4d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:01:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.575+0000 7f12c58a8640 1 --2- 172.21.15.69:0/295897745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 0x55ce013f4d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.575+0000 7f12c58a8640 1 -- 172.21.15.69:0/295897745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 msgr2=0x55ce013f0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.575+0000 7f12c58a8640 1 --2- 172.21.15.69:0/295897745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 0x55ce013f0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.575+0000 7f12c58a8640 1 -- 172.21.15.69:0/295897745 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce0105e210 con 0x55ce013f0cc0 2024-08-02T18:01:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.577+0000 7f12c58a8640 1 --2- 172.21.15.69:0/295897745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0cc0 0x55ce013fc680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12b80099a0 tx=0x7f12b8015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=758d738a5d05065c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.577+0000 7f12c58a8640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12ac0a2e40 0x7f12ac0a5300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005fd0 tx=0x7f12b80127c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.577+0000 7f12ab7fe640 1 -- 172.21.15.69:0/295897745 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8003970 con 0x55ce013f0cc0 2024-08-02T18:01:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.577+0000 7f12ab7fe640 1 -- 172.21.15.69:0/295897745 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b8004020 con 0x55ce013f0cc0 2024-08-02T18:01:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.577+0000 7f12ab7fe640 1 -- 172.21.15.69:0/295897745 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8010680 con 0x55ce013f0cc0 2024-08-02T18:01:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.578+0000 7f12ca05e740 1 -- 172.21.15.69:0/295897745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0cc0 msgr2=0x55ce013fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.578+0000 7f12ca05e740 1 --2- 172.21.15.69:0/295897745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0cc0 0x55ce013fc680 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12b80099a0 tx=0x7f12b8015000 comp rx=0 tx=0).stop 2024-08-02T18:01:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.579+0000 7f12ca05e740 1 -- 172.21.15.69:0/295897745 shutdown_connections 2024-08-02T18:01:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.579+0000 7f12ca05e740 1 --2- 172.21.15.69:0/295897745 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 0x55ce013f0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.579+0000 7f12ca05e740 1 --2- 172.21.15.69:0/295897745 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0cc0 0x55ce013fc680 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.579+0000 7f12ca05e740 1 --2- 172.21.15.69:0/295897745 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 0x55ce013f4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.579+0000 7f12ca05e740 1 -- 172.21.15.69:0/295897745 >> 172.21.15.69:0/295897745 conn(0x55ce013e27d0 msgr2=0x55ce013e2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.580+0000 7f12ca05e740 1 -- 172.21.15.69:0/295897745 shutdown_connections 2024-08-02T18:01:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.580+0000 7f12ca05e740 1 -- 172.21.15.69:0/295897745 wait complete. 2024-08-02T18:01:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.580+0000 7f12ca05e740 1 -- start start 2024-08-02T18:01:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12ca05e740 1 Processor -- start 2024-08-02T18:01:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 0x55ce013f9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013fa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f4980 0x55ce013fa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12ca05e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55ce013fcd70 con 0x55ce013f0320 2024-08-02T18:01:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12ca05e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55ce01306500 con 0x55ce013f4980 2024-08-02T18:01:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12ca05e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55ce013fcef0 con 0x55ce013f0cc0 2024-08-02T18:01:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12ca05e740 10 client.0 _ll_register_callbacks cb 0x55ce0105da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12c48a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12c48a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013fa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:41858/0 (socket says 172.21.15.69:41858) 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12c48a6640 1 -- 172.21.15.69:0/888793779 learned_addr learned my addr 172.21.15.69:0/888793779 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.581+0000 7f12c58a8640 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f4980 0x55ce013fa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12c48a6640 1 -- 172.21.15.69:0/888793779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f4980 msgr2=0x55ce013fa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12c48a6640 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f4980 0x55ce013fa540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12c48a6640 1 -- 172.21.15.69:0/888793779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 msgr2=0x55ce013f9ac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12c50a7640 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 0x55ce013f9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12c48a6640 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 0x55ce013f9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12c48a6640 1 -- 172.21.15.69:0/888793779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0112a620 con 0x55ce013f0cc0 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12c48a6640 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013fa000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f12b000c6a0 tx=0x7f12b009c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d28eee0c03d528 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12a8ff9640 1 -- 172.21.15.69:0/888793779 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b009b2d0 con 0x55ce013f0cc0 2024-08-02T18:01:12.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12ca05e740 1 -- 172.21.15.69:0/888793779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b8004020 con 0x55ce013f0cc0 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12a8ff9640 1 -- 172.21.15.69:0/888793779 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9656+0+0 (secure 0 0 0) 0x7f12b0018b20 con 0x55ce013f0cc0 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.582+0000 7f12a8ff9640 1 -- 172.21.15.69:0/888793779 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12b00a48f0 con 0x55ce013f0cc0 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12ca05e740 10 client.28267 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12ca05e740 1 -- 172.21.15.69:0/888793779 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce013e6b50 con 0x55ce013f0cc0 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12a8ff9640 1 -- 172.21.15.69:0/888793779 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b8004020 con 0x55ce013f0cc0 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12a8ff9640 1 -- 172.21.15.69:0/888793779 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f12b009ab40 con 0x55ce013f0cc0 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12ca05e740 10 client.28267 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12ca05e740 10 client.28267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12ca05e740 20 client.28267 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12ca05e740 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce013e7fc0 0x55ce013ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12ca05e740 4 client.28267 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:12.587 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12ca05e740 1 -- 172.21.15.69:0/888793779 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce013580e0 con 0x55ce013e7fc0 2024-08-02T18:01:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.583+0000 7f12c50a7640 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce013e7fc0 0x55ce013ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.584+0000 7f12a8ff9640 10 client.28267 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:12.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.584+0000 7f12c50a7640 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce013e7fc0 0x55ce013ea480 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ac780631488939 server_cookie=d7b14903119f119a in_seq=0 out_seq=0 2024-08-02T18:01:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.596+0000 7f12a8ff9640 1 -- 172.21.15.69:0/888793779 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ce013580e0 con 0x55ce013e7fc0 2024-08-02T18:01:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.596+0000 7f12a8ff9640 10 client.28267 handle_command_reply: tid=0 2024-08-02T18:01:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.596+0000 7f12ca05e740 1 client.28267 shutdown 2024-08-02T18:01:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.602+0000 7f12ca05e740 1 -- 172.21.15.69:0/888793779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 msgr2=0x55ce013fa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.604 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.602+0000 7f12ca05e740 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013fa000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f12b000c6a0 tx=0x7f12b009c440 comp rx=0 tx=0).stop 2024-08-02T18:01:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.610+0000 7f12ca05e740 1 -- 172.21.15.69:0/888793779 shutdown_connections 2024-08-02T18:01:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.610+0000 7f12ca05e740 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce013e7fc0 0x55ce013ea480 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.610+0000 7f12ca05e740 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f4980 0x55ce013fa540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.610+0000 7f12ca05e740 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013fa000 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.610+0000 7f12ca05e740 1 --2- 172.21.15.69:0/888793779 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0320 0x55ce013f9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.610+0000 7f12ca05e740 1 -- 172.21.15.69:0/888793779 >> 172.21.15.69:0/888793779 conn(0x55ce013e27d0 msgr2=0x55ce013efb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.611+0000 7f12a8ff9640 0 client.28267 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.614+0000 7f12ca05e740 1 -- 172.21.15.69:0/888793779 shutdown_connections 2024-08-02T18:01:12.617 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.616+0000 7f12ca05e740 1 -- 172.21.15.69:0/888793779 wait complete. 2024-08-02T18:01:12.618 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.617+0000 7f12ca05e740 20 client.28267 trim_cache size 0 max 16384 2024-08-02T18:01:12.652 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.651+0000 7f12ca05e740 1 Processor -- start 2024-08-02T18:01:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.654+0000 7f12ca05e740 1 -- start start 2024-08-02T18:01:12.656 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.655+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce013efca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.655+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.655+0000 7f12c50a7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce013efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.657 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.655+0000 7f12c50a7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce013efca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39264/0 (socket says 172.21.15.69:39264) 2024-08-02T18:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12c48a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12c48a6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013f6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:41874/0 (socket says 172.21.15.69:41874) 2024-08-02T18:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12c50a7640 1 -- 172.21.15.69:0/1735619347 learned_addr learned my addr 172.21.15.69:0/1735619347 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12ca05e740 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f4980 0x55ce013f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12ca05e740 1 -- 172.21.15.69:0/1735619347 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01af6b0 con 0x55ce013f4980 2024-08-02T18:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12ca05e740 1 -- 172.21.15.69:0/1735619347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01acf80 con 0x55ce013f0320 2024-08-02T18:01:12.658 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12ca05e740 1 -- 172.21.15.69:0/1735619347 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01aadf0 con 0x55ce013f0cc0 2024-08-02T18:01:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12c50a7640 1 -- 172.21.15.69:0/1735619347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 msgr2=0x55ce013f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12c50a7640 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12c50a7640 1 -- 172.21.15.69:0/1735619347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f4980 msgr2=0x55ce013f7240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:01:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.656+0000 7f12c50a7640 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f4980 0x55ce013f7240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.657+0000 7f12c50a7640 1 -- 172.21.15.69:0/1735619347 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce013e3990 con 0x55ce013f0320 2024-08-02T18:01:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.657+0000 7f12c48a6640 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:01:12.659 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.658+0000 7f12c50a7640 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce013efca0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12bc005880 tx=0x7f12bc0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89a5420f5ad997 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.660+0000 7f12aaffd640 1 -- 172.21.15.69:0/1735619347 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc035020 con 0x55ce013f0320 2024-08-02T18:01:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.660+0000 7f12aaffd640 1 -- 172.21.15.69:0/1735619347 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12bc04a070 con 0x55ce013f0320 2024-08-02T18:01:12.661 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.660+0000 7f12aaffd640 1 -- 172.21.15.69:0/1735619347 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc003850 con 0x55ce013f0320 2024-08-02T18:01:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.661+0000 7f12ca05e740 1 -- 172.21.15.69:0/1735619347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 msgr2=0x55ce013efca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.661+0000 7f12ca05e740 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce013efca0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f12bc005880 tx=0x7f12bc0058d0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.664+0000 7f12ca05e740 1 -- 172.21.15.69:0/1735619347 shutdown_connections 2024-08-02T18:01:12.666 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.664+0000 7f12ca05e740 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f4980 0x55ce013f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.664+0000 7f12ca05e740 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f0cc0 0x55ce013f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.664+0000 7f12ca05e740 1 --2- 172.21.15.69:0/1735619347 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce013efca0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.667 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.664+0000 7f12ca05e740 1 -- 172.21.15.69:0/1735619347 >> 172.21.15.69:0/1735619347 conn(0x55ce013e27d0 msgr2=0x55ce014059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:12.668 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.667+0000 7f12ca05e740 1 -- 172.21.15.69:0/1735619347 shutdown_connections 2024-08-02T18:01:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.667+0000 7f12ca05e740 1 -- 172.21.15.69:0/1735619347 wait complete. 2024-08-02T18:01:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.668+0000 7f12ca05e740 1 -- start start 2024-08-02T18:01:12.669 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.668+0000 7f12ca05e740 1 Processor -- start 2024-08-02T18:01:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.668+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce0140de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.668+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0cc0 0x55ce0140c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.669+0000 7f12ca05e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 0x55ce0140c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.669+0000 7f12ca05e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01ab440 con 0x55ce013f0cc0 2024-08-02T18:01:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.669+0000 7f12ca05e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01ac8b0 con 0x55ce013f0320 2024-08-02T18:01:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.669+0000 7f12ca05e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12c01abb10 con 0x55ce013f4980 2024-08-02T18:01:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.669+0000 7f12ca05e740 10 client.0 _ll_register_callbacks cb 0x7f12b009ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.671+0000 7f12c58a8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 0x55ce0140c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.671+0000 7f12c50a7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce0140de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.671+0000 7f12c50a7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce0140de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:39278/0 (socket says 172.21.15.69:39278) 2024-08-02T18:01:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.671+0000 7f12c50a7640 1 -- 172.21.15.69:0/3502657964 learned_addr learned my addr 172.21.15.69:0/3502657964 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.672+0000 7f12c58a8640 1 -- 172.21.15.69:0/3502657964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 msgr2=0x55ce0140de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.672+0000 7f12c58a8640 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce0140de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.672+0000 7f12c58a8640 1 -- 172.21.15.69:0/3502657964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0cc0 msgr2=0x55ce0140c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.672+0000 7f12c58a8640 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0cc0 0x55ce0140c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.672+0000 7f12c58a8640 1 -- 172.21.15.69:0/3502657964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce0140cc90 con 0x55ce013f4980 2024-08-02T18:01:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.673+0000 7f12c58a8640 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 0x55ce0140c750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f12b8012fd0 tx=0x7f12b8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98ed80763d662e2a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.673+0000 7f12a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.673+0000 7f12a8ff9640 1 -- 172.21.15.69:0/3502657964 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b8044060 con 0x55ce013f4980 2024-08-02T18:01:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.673+0000 7f12a8ff9640 1 -- 172.21.15.69:0/3502657964 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9656+0+0 (secure 0 0 0) 0x7f12b801da10 con 0x55ce013f4980 2024-08-02T18:01:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.673+0000 7f12ca05e740 1 -- 172.21.15.69:0/3502657964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12bc04a070 con 0x55ce013f4980 2024-08-02T18:01:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.674+0000 7f12a8ff9640 1 -- 172.21.15.69:0/3502657964 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f12b80097e0 con 0x55ce013f4980 2024-08-02T18:01:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.674+0000 7f12ca05e740 10 client.28273 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12ca05e740 1 -- 172.21.15.69:0/3502657964 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ce0147e750 con 0x55ce013f4980 2024-08-02T18:01:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12a8ff9640 1 -- 172.21.15.69:0/3502657964 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12b801dc70 con 0x55ce013f4980 2024-08-02T18:01:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12a8ff9640 1 -- 172.21.15.69:0/3502657964 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f12b801b070 con 0x55ce013f4980 2024-08-02T18:01:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12ca05e740 10 client.28273 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12ca05e740 10 client.28273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12ca05e740 20 client.28273 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12ca05e740 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce0147eb40 0x55ce01480f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12ca05e740 4 client.28273 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.675+0000 7f12ca05e740 1 -- 172.21.15.69:0/3502657964 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f12bc04a070 con 0x55ce0147eb40 2024-08-02T18:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.676+0000 7f12c50a7640 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce0147eb40 0x55ce01480f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.677+0000 7f12c50a7640 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce0147eb40 0x55ce01480f60 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a9184f9451d9088 server_cookie=8a0ac144e8048616 in_seq=0 out_seq=0 2024-08-02T18:01:12.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.677+0000 7f12a8ff9640 10 client.28273 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.760+0000 7f12a8ff9640 1 -- 172.21.15.69:0/3502657964 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f12bc04a070 con 0x55ce0147eb40 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.760+0000 7f12a8ff9640 10 client.28273 handle_command_reply: tid=0 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.760+0000 7f12ca05e740 1 client.28273 shutdown 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 -- 172.21.15.69:0/3502657964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 msgr2=0x55ce0140c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 0x55ce0140c750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f12b8012fd0 tx=0x7f12b8009500 comp rx=0 tx=0).stop 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 -- 172.21.15.69:0/3502657964 shutdown_connections 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55ce013f0cc0 0x55ce0140c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55ce0147eb40 0x55ce01480f60 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12a8ff9640 0 client.28273 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55ce013f4980 0x55ce0140c750 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 --2- 172.21.15.69:0/3502657964 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55ce013f0320 0x55ce0140de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 -- 172.21.15.69:0/3502657964 >> 172.21.15.69:0/3502657964 conn(0x55ce013e27d0 msgr2=0x55ce013e1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 -- 172.21.15.69:0/3502657964 shutdown_connections 2024-08-02T18:01:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 1 -- 172.21.15.69:0/3502657964 wait complete. 2024-08-02T18:01:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.761+0000 7f12ca05e740 20 client.28273 trim_cache size 0 max 16384 2024-08-02T18:01:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:01:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:01:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:01:12.765 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12ac0a2e40 msgr2=0x7f12ac0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12ac0a2e40 0x7f12ac0a5300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f12b8005fd0 tx=0x7f12b80127c0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c01021a0 msgr2=0x7f12c0198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c01021a0 0x7f12c0198710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f12b000e9f0 tx=0x7f12b000eeb0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 shutdown_connections 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f12c0107810 0x7f12c019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f12ac0a2e40 0x7f12ac0a5300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f12c01021a0 0x7f12c0198710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.762+0000 7f12c730b640 1 --2- 172.21.15.69:0/469157509 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12c0101800 0x7f12c01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.763+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 >> 172.21.15.69:0/469157509 conn(0x7f12c00fd500 msgr2=0x7f12c00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.763+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 shutdown_connections 2024-08-02T18:01:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:12.763+0000 7f12c730b640 1 -- 172.21.15.69:0/469157509 wait complete. 2024-08-02T18:01:13.323 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:13 smithi083 ceph-mon[37031]: pgmap v463: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.9 MiB/s wr, 597 op/s 2024-08-02T18:01:13.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:13 smithi077 ceph-mon[37108]: pgmap v463: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.9 MiB/s wr, 597 op/s 2024-08-02T18:01:13.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:13 smithi069 ceph-mon[32746]: pgmap v463: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 4.9 MiB/s wr, 597 op/s 2024-08-02T18:01:14.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 845.0 s, 370.0 tps, lat 212.625 ms stddev 322.893 2024-08-02T18:01:15.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:15 smithi077 ceph-mon[37108]: pgmap v464: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.0 MiB/s wr, 771 op/s 2024-08-02T18:01:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:15 smithi077 ceph-mon[37108]: osdmap e113: 12 total, 12 up, 12 in 2024-08-02T18:01:15.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:15 smithi069 ceph-mon[32746]: pgmap v464: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.0 MiB/s wr, 771 op/s 2024-08-02T18:01:15.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:15 smithi069 ceph-mon[32746]: osdmap e113: 12 total, 12 up, 12 in 2024-08-02T18:01:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:15 smithi083 ceph-mon[37031]: pgmap v464: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.0 MiB/s wr, 771 op/s 2024-08-02T18:01:15.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:15 smithi083 ceph-mon[37031]: osdmap e113: 12 total, 12 up, 12 in 2024-08-02T18:01:17.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:17 smithi077 ceph-mon[37108]: pgmap v466: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 956 op/s 2024-08-02T18:01:17.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:17 smithi069 ceph-mon[32746]: pgmap v466: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 956 op/s 2024-08-02T18:01:17.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:17 smithi083 ceph-mon[37031]: pgmap v466: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 956 op/s 2024-08-02T18:01:19.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:19 smithi077 ceph-mon[37108]: pgmap v467: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-08-02T18:01:19.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:19 smithi069 ceph-mon[32746]: pgmap v467: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-08-02T18:01:19.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:19 smithi083 ceph-mon[37031]: pgmap v467: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2024-08-02T18:01:19.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 850.0 s, 512.4 tps, lat 232.449 ms stddev 434.312 2024-08-02T18:01:21.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:21 smithi077 ceph-mon[37108]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2024-08-02T18:01:21.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:21 smithi069 ceph-mon[32746]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2024-08-02T18:01:21.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:21 smithi083 ceph-mon[37031]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 21 MiB/s wr, 1.61k op/s 2024-08-02T18:01:23.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:23 smithi077 ceph-mon[37108]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-08-02T18:01:23.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:23 smithi069 ceph-mon[32746]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-08-02T18:01:23.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:23 smithi083 ceph-mon[37031]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 22 MiB/s wr, 1.67k op/s 2024-08-02T18:01:24.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 855.0 s, 71.2 tps, lat 1408.524 ms stddev 2188.363 2024-08-02T18:01:25.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:25 smithi069 ceph-mon[32746]: pgmap v470: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2024-08-02T18:01:25.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:25 smithi083 ceph-mon[37031]: pgmap v470: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2024-08-02T18:01:25.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:25 smithi077 ceph-mon[37108]: pgmap v470: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 18 MiB/s wr, 1.39k op/s 2024-08-02T18:01:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.919+0000 7f502ed5e640 1 -- 172.21.15.69:0/422152158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106450 msgr2=0x7f5028106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:25.922 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.919+0000 7f502ed5e640 1 --2- 172.21.15.69:0/422152158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106450 0x7f5028106830 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f501c009890 tx=0x7f501c02a370 comp rx=0 tx=0).stop 2024-08-02T18:01:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.924+0000 7f502ed5e640 1 -- 172.21.15.69:0/422152158 shutdown_connections 2024-08-02T18:01:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.924+0000 7f502ed5e640 1 --2- 172.21.15.69:0/422152158 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028112760 0x7f5028114b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.924+0000 7f502ed5e640 1 --2- 172.21.15.69:0/422152158 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5028106d70 0x7f5028112220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.924+0000 7f502ed5e640 1 --2- 172.21.15.69:0/422152158 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106450 0x7f5028106830 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.926 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.924+0000 7f502ed5e640 1 -- 172.21.15.69:0/422152158 >> 172.21.15.69:0/422152158 conn(0x7f5028075be0 msgr2=0x7f5028075ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.929+0000 7f502ed5e640 1 -- 172.21.15.69:0/422152158 shutdown_connections 2024-08-02T18:01:25.934 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.933+0000 7f502ed5e640 1 -- 172.21.15.69:0/422152158 wait complete. 2024-08-02T18:01:25.935 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.933+0000 7f502ed5e640 1 Processor -- start 2024-08-02T18:01:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.934+0000 7f502ed5e640 1 -- start start 2024-08-02T18:01:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.934+0000 7f502ed5e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5028106450 0x7f5028105b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.934+0000 7f502ed5e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106d70 0x7f50280ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.934+0000 7f502ed5e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028112760 0x7f5028100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.934+0000 7f502ed5e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f502806c390 con 0x7f5028112760 2024-08-02T18:01:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.934+0000 7f502ed5e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f502806c210 con 0x7f5028106450 2024-08-02T18:01:25.936 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.934+0000 7f502ed5e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f502806c510 con 0x7f5028106d70 2024-08-02T18:01:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.937+0000 7f5027fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106d70 0x7f50280ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.937+0000 7f5027fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106d70 0x7f50280ffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43328/0 (socket says 172.21.15.69:43328) 2024-08-02T18:01:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.937+0000 7f5027fff640 1 -- 172.21.15.69:0/1155452271 learned_addr learned my addr 172.21.15.69:0/1155452271 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.938+0000 7f502cafa640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5028106450 0x7f5028105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.938+0000 7f502d2fb640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028112760 0x7f5028100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.939+0000 7f5027fff640 1 -- 172.21.15.69:0/1155452271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5028106450 msgr2=0x7f5028105b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.939+0000 7f5027fff640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5028106450 0x7f5028105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.939+0000 7f5027fff640 1 -- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028112760 msgr2=0x7f5028100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.939+0000 7f5027fff640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028112760 0x7f5028100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.939+0000 7f5027fff640 1 -- 172.21.15.69:0/1155452271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5028100980 con 0x7f5028106d70 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.939+0000 7f5027fff640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106d70 0x7f50280ffbd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f501400c890 tx=0x7f501400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.940+0000 7f5025ffb640 1 -- 172.21.15.69:0/1155452271 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5014004220 con 0x7f5028106d70 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.940+0000 7f502ed5e640 1 -- 172.21.15.69:0/1155452271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50281b3570 con 0x7f5028106d70 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.940+0000 7f502ed5e640 1 -- 172.21.15.69:0/1155452271 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50281b3a90 con 0x7f5028106d70 2024-08-02T18:01:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.941+0000 7f5025ffb640 1 -- 172.21.15.69:0/1155452271 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50140043c0 con 0x7f5028106d70 2024-08-02T18:01:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.941+0000 7f5025ffb640 1 -- 172.21.15.69:0/1155452271 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50140049c0 con 0x7f5028106d70 2024-08-02T18:01:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.941+0000 7f502d2fb640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028112760 0x7f5028100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:01:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.946+0000 7f5031b90740 1 Processor -- start 2024-08-02T18:01:25.948 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.947+0000 7f5025ffb640 1 -- 172.21.15.69:0/1155452271 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f50140079b0 con 0x7f5028106d70 2024-08-02T18:01:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.948+0000 7f5025ffb640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50180a2f00 0x7f50180a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.948+0000 7f5025ffb640 1 -- 172.21.15.69:0/1155452271 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f50140cf530 con 0x7f5028106d70 2024-08-02T18:01:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.949+0000 7f502d2fb640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50180a2f00 0x7f50180a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.950+0000 7f502d2fb640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50180a2f00 0x7f50180a53c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5020005e30 tx=0x7f50200092c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5031b90740 1 -- start start 2024-08-02T18:01:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5031b90740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 0x560579a70b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5031b90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5031b90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a7b970 0x560579a7dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5031b90740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5605799d49b0 con 0x560579a707b0 2024-08-02T18:01:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5031b90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56057997c800 con 0x560579a7b970 2024-08-02T18:01:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5031b90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5605799d0970 con 0x560579a710d0 2024-08-02T18:01:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5027fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5027fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43340/0 (socket says 172.21.15.69:43340) 2024-08-02T18:01:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.951+0000 7f5027fff640 1 -- 172.21.15.69:0/2185539244 learned_addr learned my addr 172.21.15.69:0/2185539244 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.952+0000 7f5027fff640 1 -- 172.21.15.69:0/2185539244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a7b970 msgr2=0x560579a7dd60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:01:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.952+0000 7f5027fff640 1 --2- 172.21.15.69:0/2185539244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a7b970 0x560579a7dd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.952+0000 7f5027fff640 1 -- 172.21.15.69:0/2185539244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 msgr2=0x560579a70b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.952+0000 7f5027fff640 1 --2- 172.21.15.69:0/2185539244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 0x560579a70b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.952+0000 7f5027fff640 1 -- 172.21.15.69:0/2185539244 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605796e8210 con 0x560579a710d0 2024-08-02T18:01:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.953+0000 7f5027fff640 1 --2- 172.21.15.69:0/2185539244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7b430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f50140a3d40 tx=0x7f501409be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d872ddbd5be8fb server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.956+0000 7f5012ffd640 1 -- 172.21.15.69:0/2185539244 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501401d070 con 0x560579a710d0 2024-08-02T18:01:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.956+0000 7f5012ffd640 1 -- 172.21.15.69:0/2185539244 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5014005370 con 0x560579a710d0 2024-08-02T18:01:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.957+0000 7f5031b90740 1 -- 172.21.15.69:0/2185539244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 msgr2=0x560579a7b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:25.960 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.957+0000 7f5031b90740 1 --2- 172.21.15.69:0/2185539244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7b430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f50140a3d40 tx=0x7f501409be30 comp rx=0 tx=0).stop 2024-08-02T18:01:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.960+0000 7f5031b90740 1 -- 172.21.15.69:0/2185539244 shutdown_connections 2024-08-02T18:01:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.960+0000 7f5031b90740 1 --2- 172.21.15.69:0/2185539244 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a7b970 0x560579a7dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.960+0000 7f5031b90740 1 --2- 172.21.15.69:0/2185539244 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7b430 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.960+0000 7f5031b90740 1 --2- 172.21.15.69:0/2185539244 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 0x560579a70b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.963 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.960+0000 7f5031b90740 1 -- 172.21.15.69:0/2185539244 >> 172.21.15.69:0/2185539244 conn(0x560579a6cd20 msgr2=0x560579a6d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.967+0000 7f5031b90740 1 -- 172.21.15.69:0/2185539244 shutdown_connections 2024-08-02T18:01:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.972+0000 7f5031b90740 1 -- 172.21.15.69:0/2185539244 wait complete. 2024-08-02T18:01:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.972+0000 7f5031b90740 1 -- start start 2024-08-02T18:01:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.972+0000 7f5031b90740 1 Processor -- start 2024-08-02T18:01:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.973+0000 7f5031b90740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 0x560579a78890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.973+0000 7f5031b90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a710d0 0x560579a78dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.973+0000 7f5031b90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a7b970 0x560579a79310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.973+0000 7f5031b90740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560579a7e590 con 0x560579a707b0 2024-08-02T18:01:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.973+0000 7f5031b90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560579990500 con 0x560579a710d0 2024-08-02T18:01:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.973+0000 7f5031b90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560579a7e710 con 0x560579a7b970 2024-08-02T18:01:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.973+0000 7f5031b90740 10 client.0 _ll_register_callbacks cb 0x5605796e7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.973+0000 7f502d2fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a7b970 0x560579a79310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.974+0000 7f502d2fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a7b970 0x560579a79310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:43350/0 (socket says 172.21.15.69:43350) 2024-08-02T18:01:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.974+0000 7f502d2fb640 1 -- 172.21.15.69:0/3914904596 learned_addr learned my addr 172.21.15.69:0/3914904596 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.974+0000 7f502cafa640 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 0x560579a78890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.975+0000 7f502d2fb640 1 -- 172.21.15.69:0/3914904596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a710d0 msgr2=0x560579a78dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:01:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.975+0000 7f502d2fb640 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a710d0 0x560579a78dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.975+0000 7f502d2fb640 1 -- 172.21.15.69:0/3914904596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 msgr2=0x560579a78890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.975+0000 7f502d2fb640 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 0x560579a78890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.975+0000 7f502d2fb640 1 -- 172.21.15.69:0/3914904596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605797b4620 con 0x560579a7b970 2024-08-02T18:01:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.979+0000 7f502d2fb640 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a7b970 0x560579a79310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5020009530 tx=0x7f5020004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46d5fe497afc6d01 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.979+0000 7f5007fff640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.979+0000 7f5007fff640 1 -- 172.21.15.69:0/3914904596 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502001c420 con 0x560579a7b970 2024-08-02T18:01:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.979+0000 7f5007fff640 1 -- 172.21.15.69:0/3914904596 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f502001c5c0 con 0x560579a7b970 2024-08-02T18:01:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.979+0000 7f5031b90740 1 -- 172.21.15.69:0/3914904596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5014005370 con 0x560579a7b970 2024-08-02T18:01:25.983 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.980+0000 7f5007fff640 1 -- 172.21.15.69:0/3914904596 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f502001c7c0 con 0x560579a7b970 2024-08-02T18:01:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.985+0000 7f5031b90740 10 client.28285 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.985+0000 7f5031b90740 1 -- 172.21.15.69:0/3914904596 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560579a8a350 con 0x560579a7b970 2024-08-02T18:01:25.988 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.987+0000 7f5007fff640 1 -- 172.21.15.69:0/3914904596 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5014005370 con 0x560579a7b970 2024-08-02T18:01:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.988+0000 7f5007fff640 1 -- 172.21.15.69:0/3914904596 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f502000b040 con 0x560579a7b970 2024-08-02T18:01:25.989 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.988+0000 7f5031b90740 10 client.28285 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.988+0000 7f5031b90740 10 client.28285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.988+0000 7f5031b90740 20 client.28285 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.988+0000 7f5031b90740 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560579a8bcb0 0x560579a8e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.988+0000 7f5031b90740 4 client.28285 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:25.990 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.988+0000 7f5031b90740 1 -- 172.21.15.69:0/3914904596 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605799e20e0 con 0x560579a8bcb0 2024-08-02T18:01:25.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.993+0000 7f502cafa640 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560579a8bcb0 0x560579a8e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:25.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.995+0000 7f502cafa640 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560579a8bcb0 0x560579a8e0f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c28c797b1ec8669a server_cookie=401c3332765aaacf in_seq=0 out_seq=0 2024-08-02T18:01:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:25.996+0000 7f5007fff640 10 client.28285 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.000+0000 7f5007fff640 1 -- 172.21.15.69:0/3914904596 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5605799e20e0 con 0x560579a8bcb0 2024-08-02T18:01:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.000+0000 7f5007fff640 10 client.28285 handle_command_reply: tid=0 2024-08-02T18:01:26.002 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.000+0000 7f5031b90740 1 client.28285 shutdown 2024-08-02T18:01:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.004+0000 7f5031b90740 1 -- 172.21.15.69:0/3914904596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a7b970 msgr2=0x560579a79310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.004+0000 7f5031b90740 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a7b970 0x560579a79310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5020009530 tx=0x7f5020004ab0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.005+0000 7f5031b90740 1 -- 172.21.15.69:0/3914904596 shutdown_connections 2024-08-02T18:01:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.005+0000 7f5031b90740 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560579a8bcb0 0x560579a8e0f0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.005+0000 7f5031b90740 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a7b970 0x560579a79310 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.005+0000 7f5031b90740 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a710d0 0x560579a78dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.005+0000 7f5031b90740 1 --2- 172.21.15.69:0/3914904596 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a707b0 0x560579a78890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.005+0000 7f5031b90740 1 -- 172.21.15.69:0/3914904596 >> 172.21.15.69:0/3914904596 conn(0x560579a6cd20 msgr2=0x560579a72a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.005+0000 7f5007fff640 0 client.28285 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.007+0000 7f5031b90740 1 -- 172.21.15.69:0/3914904596 shutdown_connections 2024-08-02T18:01:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.010+0000 7f5031b90740 1 -- 172.21.15.69:0/3914904596 wait complete. 2024-08-02T18:01:26.013 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.012+0000 7f5031b90740 20 client.28285 trim_cache size 0 max 16384 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5031b90740 1 Processor -- start 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5031b90740 1 -- start start 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5031b90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5031b90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a79140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5031b90740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 0x560579a93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5031b90740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f500c012910 con 0x560579a7b970 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5031b90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f500c0101e0 con 0x560579a707b0 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5031b90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f500c00e0e0 con 0x560579a710d0 2024-08-02T18:01:26.042 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f502cafa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f502cafa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a73270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:54546/0 (socket says 172.21.15.69:54546) 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f502cafa640 1 -- 172.21.15.69:0/1242434865 learned_addr learned my addr 172.21.15.69:0/1242434865 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f502d2fb640 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 0x560579a93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.039+0000 7f5027fff640 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a79140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f502cafa640 1 -- 172.21.15.69:0/1242434865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 msgr2=0x560579a79140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f502cafa640 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a79140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f502cafa640 1 -- 172.21.15.69:0/1242434865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 msgr2=0x560579a93a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f502cafa640 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 0x560579a93a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f502cafa640 1 -- 172.21.15.69:0/1242434865 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560579a6cb50 con 0x560579a707b0 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f502cafa640 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a73270 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f501c009890 tx=0x7f501c005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0cf11ac71e8d1db server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f50127fc640 1 -- 172.21.15.69:0/1242434865 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c037040 con 0x560579a707b0 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f50127fc640 1 -- 172.21.15.69:0/1242434865 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501c042070 con 0x560579a707b0 2024-08-02T18:01:26.043 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.040+0000 7f50127fc640 1 -- 172.21.15.69:0/1242434865 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c005480 con 0x560579a707b0 2024-08-02T18:01:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.042+0000 7f5031b90740 1 -- 172.21.15.69:0/1242434865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 msgr2=0x560579a73270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.042+0000 7f5031b90740 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a73270 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f501c009890 tx=0x7f501c005a90 comp rx=0 tx=0).stop 2024-08-02T18:01:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.047+0000 7f5031b90740 1 -- 172.21.15.69:0/1242434865 shutdown_connections 2024-08-02T18:01:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.047+0000 7f5031b90740 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 0x560579a93a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.047+0000 7f5031b90740 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a79140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.047+0000 7f5031b90740 1 --2- 172.21.15.69:0/1242434865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a73270 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.047+0000 7f5031b90740 1 -- 172.21.15.69:0/1242434865 >> 172.21.15.69:0/1242434865 conn(0x560579a6cd20 msgr2=0x560579a7a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.052+0000 7f5031b90740 1 -- 172.21.15.69:0/1242434865 shutdown_connections 2024-08-02T18:01:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.053+0000 7f5031b90740 1 -- 172.21.15.69:0/1242434865 wait complete. 2024-08-02T18:01:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.053+0000 7f5031b90740 1 -- start start 2024-08-02T18:01:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.053+0000 7f5031b90740 1 Processor -- start 2024-08-02T18:01:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.057+0000 7f5031b90740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.057+0000 7f5031b90740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.057+0000 7f5031b90740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 0x560579a7f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.057+0000 7f5031b90740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f500c00e6a0 con 0x560579a7b970 2024-08-02T18:01:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.057+0000 7f5031b90740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f500c00fb10 con 0x560579a707b0 2024-08-02T18:01:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.057+0000 7f5031b90740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f500c00ed70 con 0x560579a710d0 2024-08-02T18:01:26.059 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.057+0000 7f5031b90740 10 client.0 _ll_register_callbacks cb 0x7f5020009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.058+0000 7f502cafa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.058+0000 7f502cafa640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:54560/0 (socket says 172.21.15.69:54560) 2024-08-02T18:01:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.058+0000 7f502cafa640 1 -- 172.21.15.69:0/2449393617 learned_addr learned my addr 172.21.15.69:0/2449393617 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.058+0000 7f502d2fb640 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 0x560579a7f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.058+0000 7f5027fff640 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.061+0000 7f502cafa640 1 -- 172.21.15.69:0/2449393617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 msgr2=0x560579a7f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.061+0000 7f502cafa640 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.061+0000 7f502cafa640 1 -- 172.21.15.69:0/2449393617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 msgr2=0x560579a7f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.061+0000 7f502cafa640 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 0x560579a7f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.061+0000 7f502cafa640 1 -- 172.21.15.69:0/2449393617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560579a6de90 con 0x560579a707b0 2024-08-02T18:01:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.062+0000 7f502cafa640 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a840f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f501c005b60 tx=0x7f501c030a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c42b99e904601210 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.062+0000 7f5007fff640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.066+0000 7f5007fff640 1 -- 172.21.15.69:0/2449393617 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c037040 con 0x560579a707b0 2024-08-02T18:01:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.066+0000 7f5007fff640 1 -- 172.21.15.69:0/2449393617 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f501c030720 con 0x560579a707b0 2024-08-02T18:01:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.067+0000 7f5031b90740 1 -- 172.21.15.69:0/2449393617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f501c042070 con 0x560579a707b0 2024-08-02T18:01:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.070+0000 7f5007fff640 1 -- 172.21.15.69:0/2449393617 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f501c0051c0 con 0x560579a707b0 2024-08-02T18:01:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.071+0000 7f5031b90740 10 client.28547 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.071+0000 7f5031b90740 1 -- 172.21.15.69:0/2449393617 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560579af6770 con 0x560579a707b0 2024-08-02T18:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.076+0000 7f5007fff640 1 -- 172.21.15.69:0/2449393617 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501c042070 con 0x560579a707b0 2024-08-02T18:01:26.077 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.076+0000 7f5007fff640 1 -- 172.21.15.69:0/2449393617 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f501c03d450 con 0x560579a707b0 2024-08-02T18:01:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.079+0000 7f5031b90740 10 client.28547 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.079+0000 7f5031b90740 10 client.28547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.079+0000 7f5031b90740 20 client.28547 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.079+0000 7f5031b90740 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560579af6d90 0x560579af9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.079+0000 7f5031b90740 4 client.28547 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:01:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.079+0000 7f5031b90740 1 -- 172.21.15.69:0/2449393617 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5605799dc430 con 0x560579af6d90 2024-08-02T18:01:26.083 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.080+0000 7f5027fff640 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560579af6d90 0x560579af9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.092+0000 7f5027fff640 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560579af6d90 0x560579af9180 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc7a11a23252f4d server_cookie=54eb7c5c8cb6a5f1 in_seq=0 out_seq=0 2024-08-02T18:01:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.092+0000 7f5007fff640 10 client.28547 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5007fff640 1 -- 172.21.15.69:0/2449393617 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5605799dc430 con 0x560579af6d90 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5007fff640 10 client.28547 handle_command_reply: tid=0 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 client.28547 shutdown 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 -- 172.21.15.69:0/2449393617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 msgr2=0x560579a840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a840f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f501c005b60 tx=0x7f501c030a80 comp rx=0 tx=0).stop 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 -- 172.21.15.69:0/2449393617 shutdown_connections 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560579a7b970 0x560579a7f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560579af6d90 0x560579af9180 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560579a710d0 0x560579a7f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 --2- 172.21.15.69:0/2449393617 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560579a707b0 0x560579a840f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.145+0000 7f5031b90740 1 -- 172.21.15.69:0/2449393617 >> 172.21.15.69:0/2449393617 conn(0x560579a6cd20 msgr2=0x560579a79f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.146+0000 7f5007fff640 0 client.28547 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.146+0000 7f5031b90740 1 -- 172.21.15.69:0/2449393617 shutdown_connections 2024-08-02T18:01:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.146+0000 7f5031b90740 1 -- 172.21.15.69:0/2449393617 wait complete. 2024-08-02T18:01:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.146+0000 7f5031b90740 20 client.28547 trim_cache size 0 max 16384 2024-08-02T18:01:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:01:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:01:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:01:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:01:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 -- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50180a2f00 msgr2=0x7f50180a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50180a2f00 0x7f50180a53c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5020005e30 tx=0x7f50200092c0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 -- 172.21.15.69:0/1155452271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106d70 msgr2=0x7f50280ffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106d70 0x7f50280ffbd0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f501400c890 tx=0x7f501400cd50 comp rx=0 tx=0).stop 2024-08-02T18:01:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 -- 172.21.15.69:0/1155452271 shutdown_connections 2024-08-02T18:01:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5028112760 0x7f5028100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50180a2f00 0x7f50180a53c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5028106d70 0x7f50280ffbd0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 --2- 172.21.15.69:0/1155452271 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5028106450 0x7f5028105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 -- 172.21.15.69:0/1155452271 >> 172.21.15.69:0/1155452271 conn(0x7f5028075be0 msgr2=0x7f50281132a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 -- 172.21.15.69:0/1155452271 shutdown_connections 2024-08-02T18:01:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:26.147+0000 7f502ed5e640 1 -- 172.21.15.69:0/1155452271 wait complete. 2024-08-02T18:01:27.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:27 smithi069 ceph-mon[32746]: pgmap v471: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-08-02T18:01:27.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:27 smithi083 ceph-mon[37031]: pgmap v471: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-08-02T18:01:27.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:27 smithi077 ceph-mon[37108]: pgmap v471: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2024-08-02T18:01:29.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:29 smithi083 ceph-mon[37031]: pgmap v472: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 939 op/s 2024-08-02T18:01:29.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:29 smithi077 ceph-mon[37108]: pgmap v472: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 939 op/s 2024-08-02T18:01:29.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:29 smithi069 ceph-mon[32746]: pgmap v472: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 12 MiB/s wr, 939 op/s 2024-08-02T18:01:29.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 860.0 s, 610.2 tps, lat 164.837 ms stddev 285.500 2024-08-02T18:01:31.522 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:31 smithi083 ceph-mon[37031]: pgmap v473: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-08-02T18:01:31.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:31 smithi077 ceph-mon[37108]: pgmap v473: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-08-02T18:01:31.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:31 smithi069 ceph-mon[32746]: pgmap v473: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 11 MiB/s wr, 918 op/s 2024-08-02T18:01:33.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:33 smithi069 ceph-mon[32746]: pgmap v474: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-08-02T18:01:33.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:33 smithi083 ceph-mon[37031]: pgmap v474: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-08-02T18:01:33.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:33 smithi077 ceph-mon[37108]: pgmap v474: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.3 MiB/s wr, 353 op/s 2024-08-02T18:01:34.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 865.0 s, 541.6 tps, lat 192.905 ms stddev 242.004 2024-08-02T18:01:35.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:35 smithi083 ceph-mon[37031]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-02T18:01:35.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:35 smithi077 ceph-mon[37108]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-02T18:01:35.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:35 smithi069 ceph-mon[32746]: pgmap v475: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-02T18:01:37.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:37 smithi083 ceph-mon[37031]: pgmap v476: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-08-02T18:01:37.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:37 smithi077 ceph-mon[37108]: pgmap v476: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-08-02T18:01:37.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:37 smithi069 ceph-mon[32746]: pgmap v476: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 3.9 MiB/s wr, 424 op/s 2024-08-02T18:01:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.300+0000 7fef1fa9e640 1 -- 172.21.15.69:0/2437713516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1811ae70 msgr2=0x7fef1811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.300+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/2437713516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1811ae70 0x7fef1811d260 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fef1001ace0 tx=0x7fef1003b990 comp rx=0 tx=0).stop 2024-08-02T18:01:39.305 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.303+0000 7fef1fa9e640 1 -- 172.21.15.69:0/2437713516 shutdown_connections 2024-08-02T18:01:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.303+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/2437713516 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef18109300 0x7fef18109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.303+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/2437713516 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef1811ae70 0x7fef1811d260 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.303+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/2437713516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef18108f20 0x7fef1810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.303+0000 7fef1fa9e640 1 -- 172.21.15.69:0/2437713516 >> 172.21.15.69:0/2437713516 conn(0x7fef1806cb20 msgr2=0x7fef1806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.306+0000 7fef1fa9e640 1 -- 172.21.15.69:0/2437713516 shutdown_connections 2024-08-02T18:01:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.309+0000 7fef1fa9e640 1 -- 172.21.15.69:0/2437713516 wait complete. 2024-08-02T18:01:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.309+0000 7fef1fa9e640 1 Processor -- start 2024-08-02T18:01:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.309+0000 7fef1fa9e640 1 -- start start 2024-08-02T18:01:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.310+0000 7fef1fa9e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef18108f20 0x7fef1811a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.310+0000 7fef1fa9e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef18109300 0x7fef18116530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.310+0000 7fef1fa9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef1811ae70 0x7fef18116a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.310+0000 7fef1fa9e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef18113090 con 0x7fef18109300 2024-08-02T18:01:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.310+0000 7fef1fa9e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef18112f10 con 0x7fef18108f20 2024-08-02T18:01:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.310+0000 7fef1fa9e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef18113210 con 0x7fef1811ae70 2024-08-02T18:01:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.313+0000 7fef1e03b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef1811ae70 0x7fef18116a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.314+0000 7fef1d83a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef18108f20 0x7fef1811a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.314+0000 7fef1d83a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef18108f20 0x7fef1811a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55116/0 (socket says 172.21.15.69:55116) 2024-08-02T18:01:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.314+0000 7fef1d83a640 1 -- 172.21.15.69:0/305932577 learned_addr learned my addr 172.21.15.69:0/305932577 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:39.316 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.315+0000 7fef1d83a640 1 -- 172.21.15.69:0/305932577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef1811ae70 msgr2=0x7fef18116a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.315+0000 7fef1d83a640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef1811ae70 0x7fef18116a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.315+0000 7fef1d83a640 1 -- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef18109300 msgr2=0x7fef18116530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.315+0000 7fef1d83a640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef18109300 0x7fef18116530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.317 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.315+0000 7fef1d83a640 1 -- 172.21.15.69:0/305932577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef18117030 con 0x7fef18108f20 2024-08-02T18:01:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.319+0000 7fef1d83a640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef18108f20 0x7fef1811a4a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fef0800e9f0 tx=0x7fef0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.322+0000 7fef16ffd640 1 -- 172.21.15.69:0/305932577 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0800cd80 con 0x7fef18108f20 2024-08-02T18:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.322+0000 7fef1fa9e640 1 -- 172.21.15.69:0/305932577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef18117320 con 0x7fef18108f20 2024-08-02T18:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.322+0000 7fef1fa9e640 1 -- 172.21.15.69:0/305932577 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef181ab0f0 con 0x7fef18108f20 2024-08-02T18:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.322+0000 7fef16ffd640 1 -- 172.21.15.69:0/305932577 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef08010910 con 0x7fef18108f20 2024-08-02T18:01:39.326 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.322+0000 7fef16ffd640 1 -- 172.21.15.69:0/305932577 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08018980 con 0x7fef18108f20 2024-08-02T18:01:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.329+0000 7fef16ffd640 1 -- 172.21.15.69:0/305932577 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef08018b20 con 0x7fef18108f20 2024-08-02T18:01:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.330+0000 7fef16ffd640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef000a2f30 0x7fef000a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.331+0000 7fef16ffd640 1 -- 172.21.15.69:0/305932577 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7fef080cf9c0 con 0x7fef18108f20 2024-08-02T18:01:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.331+0000 7fef227fa740 1 Processor -- start 2024-08-02T18:01:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.333+0000 7fef227fa740 1 -- start start 2024-08-02T18:01:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.333+0000 7fef227fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.334+0000 7fef227fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6dec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.334+0000 7fef1d83a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.334+0000 7fef1d83a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55124/0 (socket says 172.21.15.69:55124) 2024-08-02T18:01:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.334+0000 7fef1d83a640 1 -- 172.21.15.69:0/2658079166 learned_addr learned my addr 172.21.15.69:0/2658079166 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.334+0000 7fef227fa740 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6deed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.334+0000 7fef227fa740 1 -- 172.21.15.69:0/2658079166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5627a6d459b0 con 0x5627a6dec970 2024-08-02T18:01:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.334+0000 7fef227fa740 1 -- 172.21.15.69:0/2658079166 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5627a6ced800 con 0x5627a6de17b0 2024-08-02T18:01:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.334+0000 7fef227fa740 1 -- 172.21.15.69:0/2658079166 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627a6d41970 con 0x5627a6de20d0 2024-08-02T18:01:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.335+0000 7fef1d039640 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6dec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.335+0000 7fef1d039640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef000a2f30 0x7fef000a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.336+0000 7fef1e03b640 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6deed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.344+0000 7fef1e03b640 1 -- 172.21.15.69:0/2658079166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 msgr2=0x5627a6dec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.344+0000 7fef1e03b640 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6dec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.344+0000 7fef1e03b640 1 -- 172.21.15.69:0/2658079166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 msgr2=0x5627a6de1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.344+0000 7fef1e03b640 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.344+0000 7fef1e03b640 1 -- 172.21.15.69:0/2658079166 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a6a59210 con 0x5627a6dec970 2024-08-02T18:01:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.344+0000 7fef1d039640 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6dec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:01:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.349+0000 7fef1e03b640 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6deed60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fef1001acb0 tx=0x7fef10040450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9be9aa61a594e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.350+0000 7fef1d039640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef000a2f30 0x7fef000a53f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0058c0 tx=0x7fef0c005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.354+0000 7feefffff640 1 -- 172.21.15.69:0/2658079166 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10045040 con 0x5627a6dec970 2024-08-02T18:01:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.354+0000 7feefffff640 1 -- 172.21.15.69:0/2658079166 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef10003e50 con 0x5627a6dec970 2024-08-02T18:01:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.354+0000 7feefffff640 1 -- 172.21.15.69:0/2658079166 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10004290 con 0x5627a6dec970 2024-08-02T18:01:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.356+0000 7fef227fa740 1 -- 172.21.15.69:0/2658079166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 msgr2=0x5627a6deed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.357 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.356+0000 7fef227fa740 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6deed60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fef1001acb0 tx=0x7fef10040450 comp rx=0 tx=0).stop 2024-08-02T18:01:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.358+0000 7fef227fa740 1 -- 172.21.15.69:0/2658079166 shutdown_connections 2024-08-02T18:01:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.358+0000 7fef227fa740 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6deed60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.358+0000 7fef227fa740 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6dec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.359 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.358+0000 7fef227fa740 1 --2- 172.21.15.69:0/2658079166 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.360 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.358+0000 7fef227fa740 1 -- 172.21.15.69:0/2658079166 >> 172.21.15.69:0/2658079166 conn(0x5627a6dddd20 msgr2=0x5627a6dde100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:39.362 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.361+0000 7fef227fa740 1 -- 172.21.15.69:0/2658079166 shutdown_connections 2024-08-02T18:01:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.362+0000 7fef227fa740 1 -- 172.21.15.69:0/2658079166 wait complete. 2024-08-02T18:01:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.363+0000 7fef227fa740 1 -- start start 2024-08-02T18:01:39.364 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.363+0000 7fef227fa740 1 Processor -- start 2024-08-02T18:01:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef227fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6de98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef227fa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6de20d0 0x5627a6de9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef227fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6dec970 0x5627a6dea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef227fa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5627a6def540 con 0x5627a6de20d0 2024-08-02T18:01:39.365 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef227fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5627a6d01500 con 0x5627a6dec970 2024-08-02T18:01:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef227fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5627a6def6c0 con 0x5627a6de17b0 2024-08-02T18:01:39.366 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef227fa740 10 client.0 _ll_register_callbacks cb 0x5627a6a58a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef1d83a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6de98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.367 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef1d83a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6de98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:56154/0 (socket says 172.21.15.69:56154) 2024-08-02T18:01:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef1d83a640 1 -- 172.21.15.69:0/668415839 learned_addr learned my addr 172.21.15.69:0/668415839 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.364+0000 7fef1e03b640 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6dec970 0x5627a6dea360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.365+0000 7fef1e03b640 1 -- 172.21.15.69:0/668415839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 msgr2=0x5627a6de98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.365+0000 7fef1e03b640 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6de98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.365+0000 7fef1e03b640 1 -- 172.21.15.69:0/668415839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6de20d0 msgr2=0x5627a6de9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.365+0000 7fef1e03b640 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6de20d0 0x5627a6de9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.365+0000 7fef1e03b640 1 -- 172.21.15.69:0/668415839 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a6b25620 con 0x5627a6dec970 2024-08-02T18:01:39.368 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.365+0000 7fef1e03b640 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6dec970 0x5627a6dea360 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fef10005e30 tx=0x7fef10005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18b72c4c609e4e9e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.366+0000 7feefd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.366+0000 7feefd7fa640 1 -- 172.21.15.69:0/668415839 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10045070 con 0x5627a6dec970 2024-08-02T18:01:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.366+0000 7fef227fa740 1 -- 172.21.15.69:0/668415839 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef10003e50 con 0x5627a6dec970 2024-08-02T18:01:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.366+0000 7feefd7fa640 1 -- 172.21.15.69:0/668415839 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7fef10003b80 con 0x5627a6dec970 2024-08-02T18:01:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.366+0000 7feefd7fa640 1 -- 172.21.15.69:0/668415839 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef10063050 con 0x5627a6dec970 2024-08-02T18:01:39.369 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.367+0000 7fef227fa740 10 client.28559 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:39.370 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.367+0000 7fef227fa740 1 -- 172.21.15.69:0/668415839 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627a6def540 con 0x5627a6dec970 2024-08-02T18:01:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.371+0000 7feefd7fa640 1 -- 172.21.15.69:0/668415839 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef10003e50 con 0x5627a6dec970 2024-08-02T18:01:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.371+0000 7feefd7fa640 1 -- 172.21.15.69:0/668415839 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fef100555e0 con 0x5627a6dec970 2024-08-02T18:01:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.371+0000 7fef227fa740 10 client.28559 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.371+0000 7fef227fa740 10 client.28559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.371+0000 7fef227fa740 20 client.28559 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.371+0000 7fef227fa740 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627a6dfc6f0 0x5627a6dfeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.371+0000 7fef227fa740 4 client.28559 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:39.373 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.371+0000 7fef227fa740 1 -- 172.21.15.69:0/668415839 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627a6d530e0 con 0x5627a6dfc6f0 2024-08-02T18:01:39.374 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.372+0000 7fef1d83a640 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627a6dfc6f0 0x5627a6dfeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.373+0000 7fef1d83a640 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627a6dfc6f0 0x5627a6dfeb70 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51467cda907e704 server_cookie=ee8494804a547571 in_seq=0 out_seq=0 2024-08-02T18:01:39.375 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.374+0000 7feefd7fa640 10 client.28559 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.375+0000 7feefd7fa640 1 -- 172.21.15.69:0/668415839 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5627a6d530e0 con 0x5627a6dfc6f0 2024-08-02T18:01:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.375+0000 7feefd7fa640 10 client.28559 handle_command_reply: tid=0 2024-08-02T18:01:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.375+0000 7fef227fa740 1 client.28559 shutdown 2024-08-02T18:01:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 -- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6dec970 msgr2=0x5627a6dea360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6dec970 0x5627a6dea360 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fef10005e30 tx=0x7fef10005bf0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 -- 172.21.15.69:0/668415839 shutdown_connections 2024-08-02T18:01:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6de20d0 0x5627a6de9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627a6dfc6f0 0x5627a6dfeb70 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6dec970 0x5627a6dea360 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 --2- 172.21.15.69:0/668415839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6de98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 -- 172.21.15.69:0/668415839 >> 172.21.15.69:0/668415839 conn(0x5627a6dddd20 msgr2=0x5627a6de3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7feefd7fa640 0 client.28559 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 -- 172.21.15.69:0/668415839 shutdown_connections 2024-08-02T18:01:39.381 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.378+0000 7fef227fa740 1 -- 172.21.15.69:0/668415839 wait complete. 2024-08-02T18:01:39.382 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.379+0000 7fef227fa740 20 client.28559 trim_cache size 0 max 16384 2024-08-02T18:01:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.408+0000 7fef227fa740 1 Processor -- start 2024-08-02T18:01:39.410 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.408+0000 7fef227fa740 1 -- start start 2024-08-02T18:01:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.408+0000 7fef227fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.408+0000 7fef227fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6e04440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.411 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.408+0000 7fef227fa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e04980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef1d83a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef1d83a640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55154/0 (socket says 172.21.15.69:55154) 2024-08-02T18:01:39.412 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef1d039640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6e04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef1d039640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6e04440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:56168/0 (socket says 172.21.15.69:56168) 2024-08-02T18:01:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef1e03b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e04980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.413 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef1e03b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e04980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43742/0 (socket says 172.21.15.69:43742) 2024-08-02T18:01:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef1d83a640 1 -- 172.21.15.69:0/552325542 learned_addr learned my addr 172.21.15.69:0/552325542 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef227fa740 1 -- 172.21.15.69:0/552325542 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef04012a40 con 0x5627a6dec970 2024-08-02T18:01:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef227fa740 1 -- 172.21.15.69:0/552325542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef04010310 con 0x5627a6de17b0 2024-08-02T18:01:39.414 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.409+0000 7fef227fa740 1 -- 172.21.15.69:0/552325542 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400e160 con 0x5627a6de20d0 2024-08-02T18:01:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.417+0000 7fef1d83a640 1 -- 172.21.15.69:0/552325542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 msgr2=0x5627a6e04440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.417+0000 7fef1d83a640 1 --2- 172.21.15.69:0/552325542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6e04440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.417+0000 7fef1d83a640 1 -- 172.21.15.69:0/552325542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 msgr2=0x5627a6e04980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.417+0000 7fef1d83a640 1 --2- 172.21.15.69:0/552325542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e04980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.417+0000 7fef1d83a640 1 -- 172.21.15.69:0/552325542 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627a6de2cc0 con 0x5627a6de17b0 2024-08-02T18:01:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.423+0000 7fef1d83a640 1 --2- 172.21.15.69:0/552325542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de4220 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fef08009830 tx=0x7fef08009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d75860d8792fe74 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.425+0000 7feeff7fe640 1 -- 172.21.15.69:0/552325542 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef080b3070 con 0x5627a6de17b0 2024-08-02T18:01:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.425+0000 7feeff7fe640 1 -- 172.21.15.69:0/552325542 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef08099d90 con 0x5627a6de17b0 2024-08-02T18:01:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.425+0000 7feeff7fe640 1 -- 172.21.15.69:0/552325542 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0809b140 con 0x5627a6de17b0 2024-08-02T18:01:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.426+0000 7fef227fa740 1 -- 172.21.15.69:0/552325542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 msgr2=0x5627a6de4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.426+0000 7fef227fa740 1 --2- 172.21.15.69:0/552325542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de4220 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fef08009830 tx=0x7fef08009590 comp rx=0 tx=0).stop 2024-08-02T18:01:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.426+0000 7fef227fa740 1 -- 172.21.15.69:0/552325542 shutdown_connections 2024-08-02T18:01:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.426+0000 7fef227fa740 1 --2- 172.21.15.69:0/552325542 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e04980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.426+0000 7fef227fa740 1 --2- 172.21.15.69:0/552325542 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de20d0 0x5627a6e04440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.426+0000 7fef227fa740 1 --2- 172.21.15.69:0/552325542 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de17b0 0x5627a6de4220 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.426+0000 7fef227fa740 1 -- 172.21.15.69:0/552325542 >> 172.21.15.69:0/552325542 conn(0x5627a6dddd20 msgr2=0x5627a6de4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.427+0000 7fef227fa740 1 -- 172.21.15.69:0/552325542 shutdown_connections 2024-08-02T18:01:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.428+0000 7fef227fa740 1 -- 172.21.15.69:0/552325542 wait complete. 2024-08-02T18:01:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.428+0000 7fef227fa740 1 -- start start 2024-08-02T18:01:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.428+0000 7fef227fa740 1 Processor -- start 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef227fa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6defef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef227fa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de20d0 0x5627a6df0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef227fa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e65400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef227fa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400e7d0 con 0x5627a6dec970 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef227fa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400fc40 con 0x5627a6de20d0 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef227fa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef0400eea0 con 0x5627a6de17b0 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef227fa740 10 client.0 _ll_register_callbacks cb 0x7fef0c00daf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef1d039640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de20d0 0x5627a6df0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef1d039640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de20d0 0x5627a6df0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55168/0 (socket says 172.21.15.69:55168) 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef1d039640 1 -- 172.21.15.69:0/3309233326 learned_addr learned my addr 172.21.15.69:0/3309233326 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.429+0000 7fef1d83a640 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6defef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.430+0000 7fef1e03b640 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e65400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.430+0000 7fef1d039640 1 -- 172.21.15.69:0/3309233326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 msgr2=0x5627a6defef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.430+0000 7fef1d039640 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6defef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.430+0000 7fef1d039640 1 -- 172.21.15.69:0/3309233326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 msgr2=0x5627a6e65400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.430+0000 7fef1d039640 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e65400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.430+0000 7fef1d039640 1 -- 172.21.15.69:0/3309233326 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627a6e65a10 con 0x5627a6de20d0 2024-08-02T18:01:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.431+0000 7fef1d039640 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de20d0 0x5627a6df0430 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00d370 tx=0x7fef0c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41781ac1e26a0a2f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.433+0000 7feefd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.433+0000 7feefd7fa640 1 -- 172.21.15.69:0/3309233326 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c005020 con 0x5627a6de20d0 2024-08-02T18:01:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.433+0000 7feefd7fa640 1 -- 172.21.15.69:0/3309233326 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7fef0c0051c0 con 0x5627a6de20d0 2024-08-02T18:01:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.434+0000 7fef227fa740 1 -- 172.21.15.69:0/3309233326 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef08099d90 con 0x5627a6de20d0 2024-08-02T18:01:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.435+0000 7feefd7fa640 1 -- 172.21.15.69:0/3309233326 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fef0c0053c0 con 0x5627a6de20d0 2024-08-02T18:01:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.439+0000 7fef227fa740 10 client.28565 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.439+0000 7fef227fa740 1 -- 172.21.15.69:0/3309233326 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5627a6e66d30 con 0x5627a6de20d0 2024-08-02T18:01:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.440+0000 7feefd7fa640 1 -- 172.21.15.69:0/3309233326 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef08099d90 con 0x5627a6de20d0 2024-08-02T18:01:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.440+0000 7feefd7fa640 1 -- 172.21.15.69:0/3309233326 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fef0c01b040 con 0x5627a6de20d0 2024-08-02T18:01:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.440+0000 7fef227fa740 10 client.28565 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.440+0000 7fef227fa740 10 client.28565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.440+0000 7fef227fa740 20 client.28565 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.440+0000 7fef227fa740 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627a6e67510 0x5627a6e69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.440+0000 7fef227fa740 4 client.28565 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:01:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.440+0000 7fef227fa740 1 -- 172.21.15.69:0/3309233326 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5627a6d4d430 con 0x5627a6e67510 2024-08-02T18:01:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.441+0000 7fef1d83a640 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627a6e67510 0x5627a6e69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.444+0000 7fef1d83a640 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627a6e67510 0x5627a6e69930 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be4cd848d4e6b07e server_cookie=aa3849a381f6af8e in_seq=0 out_seq=0 2024-08-02T18:01:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.444+0000 7feefd7fa640 10 client.28565 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:39.493 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.492+0000 7feefd7fa640 1 -- 172.21.15.69:0/3309233326 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5627a6d4d430 con 0x5627a6e67510 2024-08-02T18:01:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.492+0000 7feefd7fa640 10 client.28565 handle_command_reply: tid=0 2024-08-02T18:01:39.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.492+0000 7fef227fa740 1 client.28565 shutdown 2024-08-02T18:01:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7fef227fa740 1 -- 172.21.15.69:0/3309233326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de20d0 msgr2=0x5627a6df0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7fef227fa740 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de20d0 0x5627a6df0430 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00d370 tx=0x7fef0c0046c0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7fef227fa740 1 -- 172.21.15.69:0/3309233326 shutdown_connections 2024-08-02T18:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7fef227fa740 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5627a6dec970 0x5627a6e65400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7fef227fa740 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5627a6e67510 0x5627a6e69930 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7fef227fa740 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5627a6de20d0 0x5627a6df0430 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7fef227fa740 1 --2- 172.21.15.69:0/3309233326 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5627a6de17b0 0x5627a6defef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7fef227fa740 1 -- 172.21.15.69:0/3309233326 >> 172.21.15.69:0/3309233326 conn(0x5627a6dddd20 msgr2=0x5627a6ddc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.494+0000 7feefd7fa640 0 client.28565 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.495+0000 7fef227fa740 1 -- 172.21.15.69:0/3309233326 shutdown_connections 2024-08-02T18:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.495+0000 7fef227fa740 1 -- 172.21.15.69:0/3309233326 wait complete. 2024-08-02T18:01:39.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.495+0000 7fef227fa740 20 client.28565 trim_cache size 0 max 16384 2024-08-02T18:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 -- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef000a2f30 msgr2=0x7fef000a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.498 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef000a2f30 0x7fef000a53f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0058c0 tx=0x7fef0c005850 comp rx=0 tx=0).stop 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 -- 172.21.15.69:0/305932577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef18108f20 msgr2=0x7fef1811a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef18108f20 0x7fef1811a4a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fef0800e9f0 tx=0x7fef0800eeb0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 -- 172.21.15.69:0/305932577 shutdown_connections 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fef18109300 0x7fef18116530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fef000a2f30 0x7fef000a53f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef1811ae70 0x7fef18116a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 --2- 172.21.15.69:0/305932577 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fef18108f20 0x7fef1811a4a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.497+0000 7fef1fa9e640 1 -- 172.21.15.69:0/305932577 >> 172.21.15.69:0/305932577 conn(0x7fef1806cb20 msgr2=0x7fef18070470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:39.499 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.498+0000 7fef1fa9e640 1 -- 172.21.15.69:0/305932577 shutdown_connections 2024-08-02T18:01:39.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:39.498+0000 7fef1fa9e640 1 -- 172.21.15.69:0/305932577 wait complete. 2024-08-02T18:01:39.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:39 smithi083 ceph-mon[37031]: pgmap v477: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 484 op/s 2024-08-02T18:01:39.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:39 smithi077 ceph-mon[37108]: pgmap v477: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 484 op/s 2024-08-02T18:01:39.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:39 smithi069 ceph-mon[32746]: pgmap v477: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 4.4 MiB/s wr, 484 op/s 2024-08-02T18:01:39.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 870.0 s, 715.6 tps, lat 136.982 ms stddev 161.045 2024-08-02T18:01:40.387 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T18:01:40.953 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:01:41.098 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:40 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e[66162]: *** Caught signal (Segmentation fault) ** 2024-08-02T18:01:41.098 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:40 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e[66162]: in thread 7f698ecc6640 thread_name:MR_Finisher 2024-08-02T18:01:41.098 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:40 smithi077 podman[72938]: 2024-08-02 18:01:40.947634271 +0000 UTC m=+0.022625978 container died c94197eae4353ce0895c8e75f6e88a8e0b1be49bdf2c4393ed14a422b7f87457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-02T18:01:41.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:41 smithi069 ceph-mon[32746]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-08-02T18:01:41.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:41 smithi083 ceph-mon[37031]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-08-02T18:01:41.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 -- 172.21.15.69:0/2637323032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 msgr2=0x7f878010d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 --2- 172.21.15.69:0/2637323032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 0x7f878010d670 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f8778009cd0 tx=0x7f877802a8e0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 -- 172.21.15.69:0/2637323032 shutdown_connections 2024-08-02T18:01:41.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 --2- 172.21.15.69:0/2637323032 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 0x7f878010d670 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 --2- 172.21.15.69:0/2637323032 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87801007a0 0x7f8780100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 --2- 172.21.15.69:0/2637323032 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87801067a0 0x7f8780106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 -- 172.21.15.69:0/2637323032 >> 172.21.15.69:0/2637323032 conn(0x7f87800fc4c0 msgr2=0x7f87800fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:41.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 -- 172.21.15.69:0/2637323032 shutdown_connections 2024-08-02T18:01:41.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.520+0000 7f878879d640 1 -- 172.21.15.69:0/2637323032 wait complete. 2024-08-02T18:01:41.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f878879d640 1 Processor -- start 2024-08-02T18:01:41.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f878879d640 1 -- start start 2024-08-02T18:01:41.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f878879d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87801007a0 0x7f8780197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f878879d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 0x7f87801975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f878879d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87801067a0 0x7f878019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f878879d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f8780110ba0 con 0x7f8780101140 2024-08-02T18:01:41.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f878879d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f8780110a20 con 0x7f87801067a0 2024-08-02T18:01:41.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f878879d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8780110d20 con 0x7f87801007a0 2024-08-02T18:01:41.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f8785d38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 0x7f87801975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f8785d38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 0x7f87801975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43768/0 (socket says 172.21.15.69:43768) 2024-08-02T18:01:41.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f8785d38640 1 -- 172.21.15.69:0/3754375696 learned_addr learned my addr 172.21.15.69:0/3754375696 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:41.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f8786d3a640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87801067a0 0x7f878019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.521+0000 7f8786539640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87801007a0 0x7f8780197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.522+0000 7f8786d3a640 1 -- 172.21.15.69:0/3754375696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87801007a0 msgr2=0x7f8780197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.522+0000 7f8786d3a640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87801007a0 0x7f8780197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.522+0000 7f8786d3a640 1 -- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 msgr2=0x7f87801975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.522+0000 7f8786d3a640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 0x7f87801975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.522+0000 7f8786d3a640 1 -- 172.21.15.69:0/3754375696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878019c020 con 0x7f87801067a0 2024-08-02T18:01:41.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.522+0000 7f8786d3a640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87801067a0 0x7f878019b940 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f877802ae00 tx=0x7f8778031df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:41.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.522+0000 7f87777fe640 1 -- 172.21.15.69:0/3754375696 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778043070 con 0x7f87801067a0 2024-08-02T18:01:41.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.523+0000 7f87777fe640 1 -- 172.21.15.69:0/3754375696 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8778003d80 con 0x7f87801067a0 2024-08-02T18:01:41.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.523+0000 7f87777fe640 1 -- 172.21.15.69:0/3754375696 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778030bd0 con 0x7f87801067a0 2024-08-02T18:01:41.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.524+0000 7f878879d640 1 -- 172.21.15.69:0/3754375696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878019c2b0 con 0x7f87801067a0 2024-08-02T18:01:41.527 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.524+0000 7f878879d640 1 -- 172.21.15.69:0/3754375696 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878019c6b0 con 0x7f87801067a0 2024-08-02T18:01:41.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.527+0000 7f878ab9b740 1 Processor -- start 2024-08-02T18:01:41.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.527+0000 7f87777fe640 1 -- 172.21.15.69:0/3754375696 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8778046070 con 0x7f87801067a0 2024-08-02T18:01:41.529 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f878ab9b740 1 -- start start 2024-08-02T18:01:41.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 0x5582ca1265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca130eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca1313f0 0x5582ca1337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f878ab9b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5582ca08b980 con 0x5582ca1313f0 2024-08-02T18:01:41.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f878ab9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5582ca023a70 con 0x5582ca126b30 2024-08-02T18:01:41.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f878ab9b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5582ca087940 con 0x5582ca126210 2024-08-02T18:01:41.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f8786539640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca130eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f8786539640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca130eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55186/0 (socket says 172.21.15.69:55186) 2024-08-02T18:01:41.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f8785d38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca1313f0 0x5582ca1337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f8786539640 1 -- 172.21.15.69:0/2039467168 learned_addr learned my addr 172.21.15.69:0/2039467168 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:41.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f8785d38640 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 0x5582ca1265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.528+0000 7f8786539640 1 -- 172.21.15.69:0/2039467168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 msgr2=0x5582ca1265f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:41.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8786539640 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 0x5582ca1265f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8786539640 1 -- 172.21.15.69:0/2039467168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca1313f0 msgr2=0x5582ca1337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8786539640 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca1313f0 0x5582ca1337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8786539640 1 -- 172.21.15.69:0/2039467168 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ca0866e0 con 0x5582ca126b30 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8785d38640 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca1313f0 0x5582ca1337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f87777fe640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f876c0b3ed0 0x7f876c0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8786539640 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca130eb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f877c009c00 tx=0x7f877c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ddfe50b14ef3340 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8785d38640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f876c0b3ed0 0x7f876c0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f87777fe640 1 -- 172.21.15.69:0/3754375696 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f877802e030 con 0x7f87801067a0 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8774ff9640 1 -- 172.21.15.69:0/2039467168 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c012450 con 0x5582ca126b30 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8774ff9640 1 -- 172.21.15.69:0/2039467168 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c018070 con 0x5582ca126b30 2024-08-02T18:01:41.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.529+0000 7f8774ff9640 1 -- 172.21.15.69:0/2039467168 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c011940 con 0x5582ca126b30 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 -- 172.21.15.69:0/2039467168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 msgr2=0x5582ca130eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca130eb0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f877c009c00 tx=0x7f877c010740 comp rx=0 tx=0).stop 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f8785d38640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f876c0b3ed0 0x7f876c0b6390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5582ca123b20 tx=0x7f8770009700 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 -- 172.21.15.69:0/2039467168 shutdown_connections 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca1313f0 0x5582ca1337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca130eb0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 --2- 172.21.15.69:0/2039467168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 0x5582ca1265f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 -- 172.21.15.69:0/2039467168 >> 172.21.15.69:0/2039467168 conn(0x5582ca1227a0 msgr2=0x5582ca122b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 -- 172.21.15.69:0/2039467168 shutdown_connections 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.530+0000 7f878ab9b740 1 -- 172.21.15.69:0/2039467168 wait complete. 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 1 -- start start 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 1 Processor -- start 2024-08-02T18:01:41.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126210 0x5582ca12e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca12e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca12ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5582ca133fc0 con 0x5582ca126b30 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5582ca046aa0 con 0x5582ca126210 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5582ca134140 con 0x5582ca1313f0 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f878ab9b740 10 client.0 _ll_register_callbacks cb 0x5582ca035e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f8786539640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126210 0x5582ca12e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f8785d38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca12e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f8786539640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126210 0x5582ca12e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55200/0 (socket says 172.21.15.69:55200) 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f8786d3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca12ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f8785d38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca12e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43790/0 (socket says 172.21.15.69:43790) 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.531+0000 7f8786539640 1 -- 172.21.15.69:0/59572914 learned_addr learned my addr 172.21.15.69:0/59572914 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:41.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8786539640 1 -- 172.21.15.69:0/59572914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 msgr2=0x5582ca12ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8786539640 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca12ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8786539640 1 -- 172.21.15.69:0/59572914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 msgr2=0x5582ca12e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8786539640 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca12e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8786539640 1 -- 172.21.15.69:0/59572914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ca02a650 con 0x5582ca126210 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8786539640 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126210 0x5582ca12e2c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f877c011420 tx=0x7f877c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5211eb84e87417 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8769ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8769ffb640 1 -- 172.21.15.69:0/59572914 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c0054f0 con 0x5582ca126210 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f878ab9b740 1 -- 172.21.15.69:0/59572914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877c018070 con 0x5582ca126210 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.532+0000 7f8769ffb640 1 -- 172.21.15.69:0/59572914 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f877c004030 con 0x5582ca126210 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.533+0000 7f8769ffb640 1 -- 172.21.15.69:0/59572914 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f877c030050 con 0x5582ca126210 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.533+0000 7f878ab9b740 10 client.28577 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.533+0000 7f878ab9b740 1 -- 172.21.15.69:0/59572914 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5582c9dfd3c0 con 0x5582ca126210 2024-08-02T18:01:41.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f8769ffb640 1 -- 172.21.15.69:0/59572914 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c018070 con 0x5582ca126210 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f8769ffb640 1 -- 172.21.15.69:0/59572914 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f877c00f040 con 0x5582ca126210 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f878ab9b740 10 client.28577 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f878ab9b740 10 client.28577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f878ab9b740 20 client.28577 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f878ab9b740 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5582ca140e60 0x5582ca143250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f878ab9b740 4 client.28577 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f878ab9b740 1 -- 172.21.15.69:0/59572914 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ca098ef0 con 0x5582ca140e60 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.534+0000 7f8785d38640 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5582ca140e60 0x5582ca143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.535+0000 7f8785d38640 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5582ca140e60 0x5582ca143250 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de242143337a9c5d server_cookie=928df28b00279f0d in_seq=0 out_seq=0 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.535+0000 7f8769ffb640 10 client.28577 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.535+0000 7f8769ffb640 1 -- 172.21.15.69:0/59572914 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5582ca098ef0 con 0x5582ca140e60 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.535+0000 7f8769ffb640 10 client.28577 handle_command_reply: tid=0 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.535+0000 7f878ab9b740 1 client.28577 shutdown 2024-08-02T18:01:41.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 -- 172.21.15.69:0/59572914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126210 msgr2=0x5582ca12e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126210 0x5582ca12e2c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f877c011420 tx=0x7f877c005270 comp rx=0 tx=0).stop 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 -- 172.21.15.69:0/59572914 shutdown_connections 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca12e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5582ca140e60 0x5582ca143250 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f8769ffb640 0 client.28577 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca12ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 --2- 172.21.15.69:0/59572914 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126210 0x5582ca12e2c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 -- 172.21.15.69:0/59572914 >> 172.21.15.69:0/59572914 conn(0x5582ca1227a0 msgr2=0x5582ca128480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 -- 172.21.15.69:0/59572914 shutdown_connections 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 1 -- 172.21.15.69:0/59572914 wait complete. 2024-08-02T18:01:41.545 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.536+0000 7f878ab9b740 20 client.28577 trim_cache size 0 max 16384 2024-08-02T18:01:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.560+0000 7f878ab9b740 1 Processor -- start 2024-08-02T18:01:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f878ab9b740 1 -- start start 2024-08-02T18:01:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 0x5582ca1489c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca148f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.565 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca1313f0 0x5582ca12fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f878ab9b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87801af580 con 0x5582ca126b30 2024-08-02T18:01:41.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f878ab9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87801ace50 con 0x5582ca1313f0 2024-08-02T18:01:41.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f878ab9b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87801aaca0 con 0x5582ca126210 2024-08-02T18:01:41.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8785d38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.566 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8785d38640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:43800/0 (socket says 172.21.15.69:43800) 2024-08-02T18:01:41.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8785d38640 1 -- 172.21.15.69:0/2360171281 learned_addr learned my addr 172.21.15.69:0/2360171281 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:41.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8786539640 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 0x5582ca1489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8786d3a640 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca1313f0 0x5582ca12fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8785d38640 1 -- 172.21.15.69:0/2360171281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 msgr2=0x5582ca1489c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:41.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8785d38640 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 0x5582ca1489c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.567 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8785d38640 1 -- 172.21.15.69:0/2360171281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca1313f0 msgr2=0x5582ca12fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8785d38640 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca1313f0 0x5582ca12fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.561+0000 7f8785d38640 1 -- 172.21.15.69:0/2360171281 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ca125f90 con 0x5582ca126b30 2024-08-02T18:01:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.562+0000 7f8785d38640 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca148f00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f8770009640 tx=0x7f8770010050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4acd6c28bde51d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.562+0000 7f876bfff640 1 -- 172.21.15.69:0/2360171281 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877002a480 con 0x5582ca126b30 2024-08-02T18:01:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.562+0000 7f876bfff640 1 -- 172.21.15.69:0/2360171281 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877002ab00 con 0x5582ca126b30 2024-08-02T18:01:41.568 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.562+0000 7f876bfff640 1 -- 172.21.15.69:0/2360171281 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877001cba0 con 0x5582ca126b30 2024-08-02T18:01:41.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.563+0000 7f878ab9b740 1 -- 172.21.15.69:0/2360171281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 msgr2=0x5582ca148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.569 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.563+0000 7f878ab9b740 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca148f00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f8770009640 tx=0x7f8770010050 comp rx=0 tx=0).stop 2024-08-02T18:01:41.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.564+0000 7f878ab9b740 1 -- 172.21.15.69:0/2360171281 shutdown_connections 2024-08-02T18:01:41.577 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.564+0000 7f878ab9b740 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126b30 0x5582ca148f00 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.564+0000 7f878ab9b740 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca1313f0 0x5582ca12fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.564+0000 7f878ab9b740 1 --2- 172.21.15.69:0/2360171281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca126210 0x5582ca1489c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.564+0000 7f878ab9b740 1 -- 172.21.15.69:0/2360171281 >> 172.21.15.69:0/2360171281 conn(0x5582ca1227a0 msgr2=0x5582ca12d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:41.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.564+0000 7f878ab9b740 1 -- 172.21.15.69:0/2360171281 shutdown_connections 2024-08-02T18:01:41.578 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.564+0000 7f878ab9b740 1 -- 172.21.15.69:0/2360171281 wait complete. 2024-08-02T18:01:41.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 1 -- start start 2024-08-02T18:01:41.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 1 Processor -- start 2024-08-02T18:01:41.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126210 0x5582ca134730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca134c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.579 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca1a9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f87801ab310 con 0x5582ca126210 2024-08-02T18:01:41.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f87801ac780 con 0x5582ca126b30 2024-08-02T18:01:41.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f87801ab9e0 con 0x5582ca1313f0 2024-08-02T18:01:41.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f878ab9b740 10 client.0 _ll_register_callbacks cb 0x7f877c009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:41.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f8786d3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca1a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.580 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f8786d3a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca1a9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:56236/0 (socket says 172.21.15.69:56236) 2024-08-02T18:01:41.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.565+0000 7f8786d3a640 1 -- 172.21.15.69:0/3935971165 learned_addr learned my addr 172.21.15.69:0/3935971165 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:41.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8786539640 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126210 0x5582ca134730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8785d38640 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8786d3a640 1 -- 172.21.15.69:0/3935971165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 msgr2=0x5582ca134c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:41.581 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8786d3a640 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca134c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8786d3a640 1 -- 172.21.15.69:0/3935971165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126210 msgr2=0x5582ca134730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:41.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8786d3a640 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126210 0x5582ca134730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8786d3a640 1 -- 172.21.15.69:0/3935971165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ca1cbed0 con 0x5582ca1313f0 2024-08-02T18:01:41.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8786d3a640 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca1a9980 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f87780bbc40 tx=0x7f87780bb6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a76c99d6f844a03 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:41.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8769ffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:41.582 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8769ffb640 1 -- 172.21.15.69:0/3935971165 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87780d9060 con 0x5582ca1313f0 2024-08-02T18:01:41.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f878ab9b740 1 -- 172.21.15.69:0/3935971165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877002ab00 con 0x5582ca1313f0 2024-08-02T18:01:41.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.566+0000 7f8769ffb640 1 -- 172.21.15.69:0/3935971165 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f8778030d70 con 0x5582ca1313f0 2024-08-02T18:01:41.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f8769ffb640 1 -- 172.21.15.69:0/3935971165 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f87780c4920 con 0x5582ca1313f0 2024-08-02T18:01:41.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f878ab9b740 10 client.28327 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:41.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f878ab9b740 1 -- 172.21.15.69:0/3935971165 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5582ca1cd170 con 0x5582ca1313f0 2024-08-02T18:01:41.583 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f8769ffb640 1 -- 172.21.15.69:0/3935971165 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877002ab00 con 0x5582ca1313f0 2024-08-02T18:01:41.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f8769ffb640 1 -- 172.21.15.69:0/3935971165 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f8778030070 con 0x5582ca1313f0 2024-08-02T18:01:41.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f878ab9b740 10 client.28327 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:41.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f878ab9b740 10 client.28327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:41.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f878ab9b740 20 client.28327 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:41.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f878ab9b740 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5582ca140e60 0x5582ca1cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:41.584 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f878ab9b740 4 client.28327 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T18:01:41.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f878ab9b740 1 -- 172.21.15.69:0/3935971165 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582ca093400 con 0x5582ca140e60 2024-08-02T18:01:41.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.567+0000 7f8786539640 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5582ca140e60 0x5582ca1cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:41.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f8786539640 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5582ca140e60 0x5582ca1cf890 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b691cc3fe1ac92f6 server_cookie=9be49b898baebdb2 in_seq=0 out_seq=0 2024-08-02T18:01:41.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f8769ffb640 10 client.28327 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:41.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f8769ffb640 1 -- 172.21.15.69:0/3935971165 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582ca093400 con 0x5582ca140e60 2024-08-02T18:01:41.585 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f8769ffb640 10 client.28327 handle_command_reply: tid=0 2024-08-02T18:01:41.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 client.28327 shutdown 2024-08-02T18:01:41.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 -- 172.21.15.69:0/3935971165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 msgr2=0x5582ca1a9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca1a9980 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f87780bbc40 tx=0x7f87780bb6d0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 -- 172.21.15.69:0/3935971165 shutdown_connections 2024-08-02T18:01:41.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5582ca140e60 0x5582ca1cf890 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.586 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5582ca1313f0 0x5582ca1a9980 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5582ca126b30 0x5582ca134c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 --2- 172.21.15.69:0/3935971165 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5582ca126210 0x5582ca134730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f878ab9b740 1 -- 172.21.15.69:0/3935971165 >> 172.21.15.69:0/3935971165 conn(0x5582ca1227a0 msgr2=0x5582ca1210e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:41.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.568+0000 7f8769ffb640 0 client.28327 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:41.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878ab9b740 1 -- 172.21.15.69:0/3935971165 shutdown_connections 2024-08-02T18:01:41.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878ab9b740 1 -- 172.21.15.69:0/3935971165 wait complete. 2024-08-02T18:01:41.587 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878ab9b740 20 client.28327 trim_cache size 0 max 16384 2024-08-02T18:01:41.588 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:01:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T18:01:41.588 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T18:01:41.588 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:01:41.589 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 -- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f876c0b3ed0 msgr2=0x7f876c0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f876c0b3ed0 0x7f876c0b6390 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5582ca123b20 tx=0x7f8770009700 comp rx=0 tx=0).stop 2024-08-02T18:01:41.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 -- 172.21.15.69:0/3754375696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87801067a0 msgr2=0x7f878019b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:41.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87801067a0 0x7f878019b940 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f877802ae00 tx=0x7f8778031df0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 -- 172.21.15.69:0/3754375696 shutdown_connections 2024-08-02T18:01:41.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f8780101140 0x7f87801975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f876c0b3ed0 0x7f876c0b6390 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.590 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f87801067a0 0x7f878019b940 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 --2- 172.21.15.69:0/3754375696 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87801007a0 0x7f8780197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:41.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.569+0000 7f878879d640 1 -- 172.21.15.69:0/3754375696 >> 172.21.15.69:0/3754375696 conn(0x7f87800fc4c0 msgr2=0x7f87800fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:41.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.570+0000 7f878879d640 1 -- 172.21.15.69:0/3754375696 shutdown_connections 2024-08-02T18:01:41.591 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:41.570+0000 7f878879d640 1 -- 172.21.15.69:0/3754375696 wait complete. 2024-08-02T18:01:41.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:41 smithi077 ceph-mon[37108]: pgmap v478: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-08-02T18:01:42.097 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:41 smithi077 podman[72938]: 2024-08-02 18:01:41.725087907 +0000 UTC m=+0.800079610 container remove c94197eae4353ce0895c8e75f6e88a8e0b1be49bdf2c4393ed14a422b7f87457 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-08-02T18:01:42.098 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:41 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-08-02T18:01:42.557 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T18:01:42.558 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:01:42.597 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:42 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-08-02T18:01:42.598 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:42 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.e.service: Consumed 3.606s CPU time. 2024-08-02T18:01:43.128 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:01:43.445 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:43 smithi069 ceph-mon[32746]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-08-02T18:01:43.498 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:43 smithi077 ceph-mon[37108]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-08-02T18:01:43.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:43 smithi083 ceph-mon[37031]: pgmap v479: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.5 MiB/s wr, 572 op/s 2024-08-02T18:01:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 -- 172.21.15.69:0/3263338655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80102c20 msgr2=0x7f5a8010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:43.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 --2- 172.21.15.69:0/3263338655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80102c20 0x7f5a8010d770 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000ba20 tx=0x7f5a7002c170 comp rx=0 tx=0).stop 2024-08-02T18:01:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 -- 172.21.15.69:0/3263338655 shutdown_connections 2024-08-02T18:01:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 --2- 172.21.15.69:0/3263338655 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a801022a0 0x7f5a801026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 --2- 172.21.15.69:0/3263338655 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80102c20 0x7f5a8010d770 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 --2- 172.21.15.69:0/3263338655 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a80100b50 0x7f5a80100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 -- 172.21.15.69:0/3263338655 >> 172.21.15.69:0/3263338655 conn(0x7f5a800fc500 msgr2=0x7f5a800fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 -- 172.21.15.69:0/3263338655 shutdown_connections 2024-08-02T18:01:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.721+0000 7f5a85090640 1 -- 172.21.15.69:0/3263338655 wait complete. 2024-08-02T18:01:43.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.722+0000 7f5a85090640 1 Processor -- start 2024-08-02T18:01:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a85090640 1 -- start start 2024-08-02T18:01:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a85090640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80100b50 0x7f5a8010d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a85090640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a801022a0 0x7f5a8010a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a85090640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a80102c20 0x7f5a8010ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a85090640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80113fc0 con 0x7f5a80102c20 2024-08-02T18:01:43.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a85090640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80113e40 con 0x7f5a801022a0 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a85090640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80114140 con 0x7f5a80100b50 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a7ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80100b50 0x7f5a8010d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a7ed9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80100b50 0x7f5a8010d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:56240/0 (socket says 172.21.15.69:56240) 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a7ed9d640 1 -- 172.21.15.69:0/663893381 learned_addr learned my addr 172.21.15.69:0/663893381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a7f59e640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a80102c20 0x7f5a8010ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.723+0000 7f5a7e59c640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a801022a0 0x7f5a8010a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a7f59e640 1 -- 172.21.15.69:0/663893381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80100b50 msgr2=0x7f5a8010d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a7f59e640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80100b50 0x7f5a8010d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a7f59e640 1 -- 172.21.15.69:0/663893381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a801022a0 msgr2=0x7f5a8010a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a7f59e640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a801022a0 0x7f5a8010a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a7f59e640 1 -- 172.21.15.69:0/663893381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8010b450 con 0x7f5a80102c20 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a7e59c640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a801022a0 0x7f5a8010a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:01:43.727 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a7f59e640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a80102c20 0x7f5a8010ab90 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000e090 tx=0x7f5a70004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:43.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a67fff640 1 -- 172.21.15.69:0/663893381 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70039070 con 0x7f5a80102c20 2024-08-02T18:01:43.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a67fff640 1 -- 172.21.15.69:0/663893381 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a70035e10 con 0x7f5a80102c20 2024-08-02T18:01:43.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a67fff640 1 -- 172.21.15.69:0/663893381 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70009b70 con 0x7f5a80102c20 2024-08-02T18:01:43.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.724+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a801aad00 con 0x7f5a80102c20 2024-08-02T18:01:43.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.727+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a801ab1b0 con 0x7f5a80102c20 2024-08-02T18:01:43.730 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.728+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a80101ee0 con 0x7f5a80102c20 2024-08-02T18:01:43.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.731+0000 7f5a67fff640 1 -- 172.21.15.69:0/663893381 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5a7003d070 con 0x7f5a80102c20 2024-08-02T18:01:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.732+0000 7f5a67fff640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5a680a2e40 0x7f5a680a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.732+0000 7f5a67fff640 1 -- 172.21.15.69:0/663893381 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f5a700ef100 con 0x7f5a80102c20 2024-08-02T18:01:43.733 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.732+0000 7f5a7ed9d640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5a680a2e40 0x7f5a680a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:43.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.733+0000 7f5a7ed9d640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5a680a2e40 0x7f5a680a5300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009fd0 tx=0x7f5a78010040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:43.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:43.736+0000 7f5a67fff640 1 -- 172.21.15.69:0/663893381 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5a700bbaa0 con 0x7f5a80102c20 2024-08-02T18:01:44.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.072+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a80100f30 con 0x7f5a80102c20 2024-08-02T18:01:44.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.077+0000 7f5a67fff640 1 -- 172.21.15.69:0/663893381 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11900 (secure 0 0 0) 0x7f5a700bb3a0 con 0x7f5a80102c20 2024-08-02T18:01:44.079 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:01:44.079 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":26,"btime":"2024-08-02T17:46:25:905423+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T17:46:25.691259+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24413":{"gid":24413,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6835/3410285190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3410285190},{"type":"v1","addr":"172.21.15.77:6835","nonce":3410285190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:01:44.079 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 26 2024-08-02T18:01:44.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.082+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5a680a2e40 msgr2=0x7f5a680a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:44.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.082+0000 7f5a85090640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5a680a2e40 0x7f5a680a5300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5a78009fd0 tx=0x7f5a78010040 comp rx=0 tx=0).stop 2024-08-02T18:01:44.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.082+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a80102c20 msgr2=0x7f5a8010ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:44.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.082+0000 7f5a85090640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a80102c20 0x7f5a8010ab90 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000e090 tx=0x7f5a70004270 comp rx=0 tx=0).stop 2024-08-02T18:01:44.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.085+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 shutdown_connections 2024-08-02T18:01:44.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.085+0000 7f5a85090640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5a80102c20 0x7f5a8010ab90 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:44.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.085+0000 7f5a85090640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5a680a2e40 0x7f5a680a5300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:44.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.085+0000 7f5a85090640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5a801022a0 0x7f5a8010a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:44.089 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.085+0000 7f5a85090640 1 --2- 172.21.15.69:0/663893381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5a80100b50 0x7f5a8010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:44.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.085+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 >> 172.21.15.69:0/663893381 conn(0x7f5a800fc500 msgr2=0x7f5a80104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:44.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.088+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 shutdown_connections 2024-08-02T18:01:44.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:44.088+0000 7f5a85090640 1 -- 172.21.15.69:0/663893381 wait complete. 2024-08-02T18:01:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:44 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/663893381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:01:44.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:44 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/663893381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:01:44.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:44 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/663893381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:01:44.805 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T18:01:44.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 875.0 s, 475.2 tps, lat 210.297 ms stddev 226.868 2024-08-02T18:01:45.320 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:01:45.511 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:45 smithi069 ceph-mon[32746]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.8 MiB/s wr, 606 op/s 2024-08-02T18:01:45.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:45 smithi083 ceph-mon[37031]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.8 MiB/s wr, 606 op/s 2024-08-02T18:01:45.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:45 smithi077 ceph-mon[37108]: pgmap v480: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.8 MiB/s wr, 606 op/s 2024-08-02T18:01:47.526 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:47 smithi083 ceph-mon[37031]: pgmap v481: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.2 MiB/s wr, 458 op/s 2024-08-02T18:01:47.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:47 smithi077 ceph-mon[37108]: pgmap v481: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.2 MiB/s wr, 458 op/s 2024-08-02T18:01:47.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:47 smithi069 ceph-mon[32746]: pgmap v481: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 4.2 MiB/s wr, 458 op/s 2024-08-02T18:01:49.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:49 smithi069 ceph-mon[32746]: pgmap v482: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 530 op/s 2024-08-02T18:01:49.484 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:01:49.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.958+0000 7f92df3e5640 1 -- 172.21.15.69:0/260324717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d811ae70 msgr2=0x7f92d811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.958+0000 7f92df3e5640 1 --2- 172.21.15.69:0/260324717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d811ae70 0x7f92d811d260 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009be0 tx=0x7f92d002a710 comp rx=0 tx=0).stop 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.958+0000 7f92df3e5640 1 -- 172.21.15.69:0/260324717 shutdown_connections 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.958+0000 7f92df3e5640 1 --2- 172.21.15.69:0/260324717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92d8109560 0x7f92d81099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.958+0000 7f92df3e5640 1 --2- 172.21.15.69:0/260324717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d811ae70 0x7f92d811d260 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.958+0000 7f92df3e5640 1 --2- 172.21.15.69:0/260324717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92d8109180 0x7f92d810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.958+0000 7f92df3e5640 1 -- 172.21.15.69:0/260324717 >> 172.21.15.69:0/260324717 conn(0x7f92d806de70 msgr2=0x7f92d806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.966+0000 7f92df3e5640 1 -- 172.21.15.69:0/260324717 shutdown_connections 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.966+0000 7f92df3e5640 1 -- 172.21.15.69:0/260324717 wait complete. 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.967+0000 7f92df3e5640 1 Processor -- start 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.967+0000 7f92df3e5640 1 -- start start 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.967+0000 7f92df3e5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92d8109180 0x7f92d8118400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.967+0000 7f92df3e5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92d8109560 0x7f92d8111490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.967+0000 7f92df3e5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d811ae70 0x7f92d81119d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.967+0000 7f92df3e5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f92d8120ad0 con 0x7f92d8109180 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.967+0000 7f92df3e5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92d8120950 con 0x7f92d811ae70 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.967+0000 7f92df3e5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92d8120c50 con 0x7f92d8109560 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.972+0000 7f92dd181640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92d8109180 0x7f92d8118400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.972+0000 7f92dd181640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92d8109180 0x7f92d8118400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34562/0 (socket says 172.21.15.69:34562) 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.972+0000 7f92dd181640 1 -- 172.21.15.69:0/4002196440 learned_addr learned my addr 172.21.15.69:0/4002196440 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.972+0000 7f92dc980640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92d8109560 0x7f92d8111490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.974+0000 7f92dd181640 1 -- 172.21.15.69:0/4002196440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92d8109560 msgr2=0x7f92d8111490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.974+0000 7f92dd181640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92d8109560 0x7f92d8111490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.974+0000 7f92dd181640 1 -- 172.21.15.69:0/4002196440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d811ae70 msgr2=0x7f92d81119d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.974+0000 7f92dd181640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d811ae70 0x7f92d81119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.974+0000 7f92dd181640 1 -- 172.21.15.69:0/4002196440 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d8111f90 con 0x7f92d8109180 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.975+0000 7f92dd181640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92d8109180 0x7f92d8118400 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9f0 tx=0x7f92d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.976+0000 7f92ce7fc640 1 -- 172.21.15.69:0/4002196440 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d400ccc0 con 0x7f92d8109180 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.976+0000 7f92ce7fc640 1 -- 172.21.15.69:0/4002196440 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92d4010910 con 0x7f92d8109180 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.976+0000 7f92ce7fc640 1 -- 172.21.15.69:0/4002196440 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d4018800 con 0x7f92d8109180 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.976+0000 7f92df3e5640 1 -- 172.21.15.69:0/4002196440 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d8112280 con 0x7f92d8109180 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.977+0000 7f92df3e5640 1 -- 172.21.15.69:0/4002196440 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d8116250 con 0x7f92d8109180 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.981+0000 7f92e17e3740 1 Processor -- start 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.985+0000 7f92e17e3740 1 -- start start 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006ba73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bb1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bb45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92e17e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f006b0ca40 con 0x55f006bb21d0 2024-08-02T18:01:49.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92e17e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f006aa49d0 con 0x55f006ba7910 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92e17e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f006b08a00 con 0x55f006ba6ff0 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92dc980640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bb45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92dc980640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bb45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34564/0 (socket says 172.21.15.69:34564) 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.986+0000 7f92dc980640 1 -- 172.21.15.69:0/117566233 learned_addr learned my addr 172.21.15.69:0/117566233 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.987+0000 7f92ce7fc640 1 -- 172.21.15.69:0/4002196440 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f92d4010ac0 con 0x7f92d8109180 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.988+0000 7f92ce7fc640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f92c40a2f00 0x7f92c40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.988+0000 7f92ce7fc640 1 -- 172.21.15.69:0/4002196440 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f92d40cfb70 con 0x7f92d8109180 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.989+0000 7f92dc980640 1 -- 172.21.15.69:0/117566233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 msgr2=0x55f006ba73d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.989+0000 7f92dc980640 1 --2- 172.21.15.69:0/117566233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006ba73d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.989+0000 7f92dc980640 1 -- 172.21.15.69:0/117566233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 msgr2=0x55f006bb1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.989+0000 7f92dc980640 1 --2- 172.21.15.69:0/117566233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bb1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.989+0000 7f92dc980640 1 -- 172.21.15.69:0/117566233 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f006aab650 con 0x55f006bb21d0 2024-08-02T18:01:49.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.994+0000 7f92dc980640 1 --2- 172.21.15.69:0/117566233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bb45c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009be0 tx=0x7f92c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73986dd9ea480dd5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.995+0000 7f92c37fe640 1 -- 172.21.15.69:0/117566233 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004230 con 0x55f006bb21d0 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.995+0000 7f92c37fe640 1 -- 172.21.15.69:0/117566233 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c8009e50 con 0x55f006bb21d0 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.996+0000 7f92dd982640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f92c40a2f00 0x7f92c40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.996+0000 7f92e17e3740 1 -- 172.21.15.69:0/117566233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 msgr2=0x55f006bb45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:47.996+0000 7f92e17e3740 1 --2- 172.21.15.69:0/117566233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bb45c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009be0 tx=0x7f92c8011810 comp rx=0 tx=0).stop 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.002+0000 7f92e17e3740 1 -- 172.21.15.69:0/117566233 shutdown_connections 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.002+0000 7f92e17e3740 1 --2- 172.21.15.69:0/117566233 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bb45c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.002+0000 7f92e17e3740 1 --2- 172.21.15.69:0/117566233 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bb1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.002+0000 7f92e17e3740 1 --2- 172.21.15.69:0/117566233 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006ba73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.002+0000 7f92e17e3740 1 -- 172.21.15.69:0/117566233 >> 172.21.15.69:0/117566233 conn(0x55f006ba3580 msgr2=0x55f006ba3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.004+0000 7f92e17e3740 1 -- 172.21.15.69:0/117566233 shutdown_connections 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.006+0000 7f92e17e3740 1 -- 172.21.15.69:0/117566233 wait complete. 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.006+0000 7f92e17e3740 1 -- start start 2024-08-02T18:01:49.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.007+0000 7f92e17e3740 1 Processor -- start 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.010+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bb1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.010+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.010+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006baf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.010+0000 7f92e17e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f006bb4df0 con 0x55f006bb21d0 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.010+0000 7f92e17e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f006ac7b20 con 0x55f006ba7910 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.010+0000 7f92e17e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f006bb4f70 con 0x55f006ba6ff0 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.010+0000 7f92e17e3740 10 client.0 _ll_register_callbacks cb 0x55f006ac5bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.011+0000 7f92dd982640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f92c40a2f00 0x7f92c40a53c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009b30 tx=0x7f92d00097e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.011+0000 7f92dc980640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.013+0000 7f92dd982640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006baf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.013+0000 7f92dd982640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006baf6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34568/0 (socket says 172.21.15.69:34568) 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.013+0000 7f92dd982640 1 -- 172.21.15.69:0/3933308327 learned_addr learned my addr 172.21.15.69:0/3933308327 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:49.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.013+0000 7f92dd181640 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bb1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.014+0000 7f92dc980640 1 -- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 msgr2=0x55f006bb1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.014+0000 7f92dc980640 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bb1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.014+0000 7f92dc980640 1 -- 172.21.15.69:0/3933308327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 msgr2=0x55f006baf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.014+0000 7f92dc980640 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006baf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.014+0000 7f92dc980640 1 -- 172.21.15.69:0/3933308327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f00687e3c0 con 0x55f006ba7910 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.014+0000 7f92dd982640 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006baf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.014+0000 7f92dd181640 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bb1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.014+0000 7f92dc980640 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baef60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009bb0 tx=0x7f92c80092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=271e2204ba1ece39 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.015+0000 7f92c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.015+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3933308327 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8027020 con 0x55f006ba7910 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.015+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3933308327 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f92c80052d0 con 0x55f006ba7910 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.015+0000 7f92e17e3740 1 -- 172.21.15.69:0/3933308327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c8009e50 con 0x55f006ba7910 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.018+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3933308327 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f92c80054d0 con 0x55f006ba7910 2024-08-02T18:01:49.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.022+0000 7f92e17e3740 10 client.28595 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.022+0000 7f92e17e3740 1 -- 172.21.15.69:0/3933308327 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f006bc09a0 con 0x55f006ba7910 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.022+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3933308327 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c8009e50 con 0x55f006ba7910 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.024+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3933308327 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f92c8018070 con 0x55f006ba7910 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.024+0000 7f92e17e3740 10 client.28595 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.024+0000 7f92e17e3740 10 client.28595 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.024+0000 7f92e17e3740 20 client.28595 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.024+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55f006bc2220 0x55f006bc4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.024+0000 7f92e17e3740 4 client.28595 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.024+0000 7f92e17e3740 1 -- 172.21.15.69:0/3933308327 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f006b19fb0 con 0x55f006bc2220 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.030+0000 7f92dd181640 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55f006bc2220 0x55f006bc4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.038+0000 7f92dd181640 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55f006bc2220 0x55f006bc4660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d014b5aebf0aee4 server_cookie=1ec474346c40d5e3 in_seq=0 out_seq=0 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.038+0000 7f92c0ff9640 10 client.28595 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:01:49.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.040+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3933308327 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f006b19fb0 con 0x55f006bc2220 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.040+0000 7f92c0ff9640 10 client.28595 handle_command_reply: tid=0 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.042+0000 7f92e17e3740 1 client.28595 shutdown 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.043+0000 7f92e17e3740 1 -- 172.21.15.69:0/3933308327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 msgr2=0x55f006baef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.043+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baef60 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009bb0 tx=0x7f92c80092d0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.044+0000 7f92e17e3740 1 -- 172.21.15.69:0/3933308327 shutdown_connections 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.044+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55f006bc2220 0x55f006bc4660 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.044+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006baf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.044+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baef60 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.044+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3933308327 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bb1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.044+0000 7f92e17e3740 1 -- 172.21.15.69:0/3933308327 >> 172.21.15.69:0/3933308327 conn(0x55f006ba3580 msgr2=0x55f006ba9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.044+0000 7f92c0ff9640 0 client.28595 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.045+0000 7f92e17e3740 1 -- 172.21.15.69:0/3933308327 shutdown_connections 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.045+0000 7f92e17e3740 1 -- 172.21.15.69:0/3933308327 wait complete. 2024-08-02T18:01:49.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.045+0000 7f92e17e3740 20 client.28595 trim_cache size 0 max 16384 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.073+0000 7f92e17e3740 1 Processor -- start 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92e17e3740 1 -- start start 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bc9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92e17e3740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc012800 con 0x55f006bb21d0 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92e17e3740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc0100d0 con 0x55f006ba7910 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92e17e3740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc00dfd0 con 0x55f006ba6ff0 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92dc980640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92dc980640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50374/0 (socket says 172.21.15.69:50374) 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.074+0000 7f92dc980640 1 -- 172.21.15.69:0/866689111 learned_addr learned my addr 172.21.15.69:0/866689111 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:49.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.075+0000 7f92dd982640 1 --2- 172.21.15.69:0/866689111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bc9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.075+0000 7f92dc980640 1 -- 172.21.15.69:0/866689111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 msgr2=0x55f006bae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.075+0000 7f92dc980640 1 --2- 172.21.15.69:0/866689111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.075+0000 7f92dc980640 1 -- 172.21.15.69:0/866689111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 msgr2=0x55f006bc9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.075+0000 7f92dc980640 1 --2- 172.21.15.69:0/866689111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bc9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.075+0000 7f92dc980640 1 -- 172.21.15.69:0/866689111 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f006ba7740 con 0x55f006ba7910 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.078+0000 7f92dc980640 1 --2- 172.21.15.69:0/866689111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baed60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f92c802e040 tx=0x7f92c8012f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2e75336167fb64 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.081+0000 7f92c2ffd640 1 -- 172.21.15.69:0/866689111 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8027040 con 0x55f006ba7910 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.082+0000 7f92c2ffd640 1 -- 172.21.15.69:0/866689111 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c8009e50 con 0x55f006ba7910 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.082+0000 7f92c2ffd640 1 -- 172.21.15.69:0/866689111 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c801a440 con 0x55f006ba7910 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.083+0000 7f92e17e3740 1 -- 172.21.15.69:0/866689111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 msgr2=0x55f006baed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.083+0000 7f92e17e3740 1 --2- 172.21.15.69:0/866689111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baed60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f92c802e040 tx=0x7f92c8012f90 comp rx=0 tx=0).stop 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.086+0000 7f92e17e3740 1 -- 172.21.15.69:0/866689111 shutdown_connections 2024-08-02T18:01:49.495 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.086+0000 7f92e17e3740 1 --2- 172.21.15.69:0/866689111 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006bc9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.086+0000 7f92e17e3740 1 --2- 172.21.15.69:0/866689111 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006baed60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.086+0000 7f92e17e3740 1 --2- 172.21.15.69:0/866689111 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.086+0000 7f92e17e3740 1 -- 172.21.15.69:0/866689111 >> 172.21.15.69:0/866689111 conn(0x55f006ba3580 msgr2=0x55f006bad830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.093+0000 7f92e17e3740 1 -- 172.21.15.69:0/866689111 shutdown_connections 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.093+0000 7f92e17e3740 1 -- 172.21.15.69:0/866689111 wait complete. 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.093+0000 7f92e17e3740 1 -- start start 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.093+0000 7f92e17e3740 1 Processor -- start 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.094+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.094+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.094+0000 7f92e17e3740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006babc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.094+0000 7f92dc980640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.094+0000 7f92dc980640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50380/0 (socket says 172.21.15.69:50380) 2024-08-02T18:01:49.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.094+0000 7f92dc980640 1 -- 172.21.15.69:0/3111427366 learned_addr learned my addr 172.21.15.69:0/3111427366 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.094+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc00e590 con 0x55f006bb21d0 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.094+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc00fa00 con 0x55f006ba7910 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.095+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc00ec60 con 0x55f006ba6ff0 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.095+0000 7f92e17e3740 10 client.0 _ll_register_callbacks cb 0x7f92c8009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.097+0000 7f92dd982640 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006babc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.097+0000 7f92dd181640 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.101+0000 7f92dc980640 1 -- 172.21.15.69:0/3111427366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 msgr2=0x55f006bab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.101+0000 7f92dc980640 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.101+0000 7f92dc980640 1 -- 172.21.15.69:0/3111427366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 msgr2=0x55f006babc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.101+0000 7f92dc980640 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006babc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.101+0000 7f92dc980640 1 -- 172.21.15.69:0/3111427366 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f006bb92c0 con 0x55f006ba7910 2024-08-02T18:01:49.497 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.106+0000 7f92dc980640 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bab710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009000 tx=0x7f92c80099a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=164a9afc1ee0195c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.107+0000 7f92c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.110+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3111427366 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8027040 con 0x55f006ba7910 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.110+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3111427366 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f92c80052d0 con 0x55f006ba7910 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.112+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c8009e50 con 0x55f006ba7910 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.116+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3111427366 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f92c8043050 con 0x55f006ba7910 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.117+0000 7f92e17e3740 10 client.28601 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.117+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f006bba540 con 0x55f006ba7910 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.120+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3111427366 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c8009e50 con 0x55f006ba7910 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.120+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3111427366 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f92c8018070 con 0x55f006ba7910 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.122+0000 7f92e17e3740 10 client.28601 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.122+0000 7f92e17e3740 10 client.28601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.122+0000 7f92e17e3740 20 client.28601 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:49.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.122+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55f006bbac90 0x55f006c2ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.122+0000 7f92e17e3740 4 client.28601 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.122+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f006b144c0 con 0x55f006bbac90 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.123+0000 7f92dd181640 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55f006bbac90 0x55f006c2ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.132+0000 7f92dd181640 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55f006bbac90 0x55f006c2ef50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e65d62c660b1c62 server_cookie=dbbc9a788793c237 in_seq=0 out_seq=0 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.133+0000 7f92c0ff9640 10 client.28601 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.135+0000 7f92c0ff9640 1 -- 172.21.15.69:0/3111427366 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f006b144c0 con 0x55f006bbac90 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.135+0000 7f92c0ff9640 10 client.28601 handle_command_reply: tid=0 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.135+0000 7f92e17e3740 1 client.28601 shutdown 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 msgr2=0x55f006bab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bab710 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009000 tx=0x7f92c80099a0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 shutdown_connections 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x55f006bbac90 0x55f006c2ef50 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f006bb21d0 0x55f006babc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f006ba7910 0x55f006bab710 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 --2- 172.21.15.69:0/3111427366 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f006ba6ff0 0x55f006bab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 >> 172.21.15.69:0/3111427366 conn(0x55f006ba3580 msgr2=0x55f006bad790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92c0ff9640 0 client.28601 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.138+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 shutdown_connections 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.139+0000 7f92e17e3740 1 -- 172.21.15.69:0/3111427366 wait complete. 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.143+0000 7f92e17e3740 20 client.28601 trim_cache size 0 max 16384 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.147+0000 7f92df3e5640 1 -- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f92c40a2f00 msgr2=0x7f92c40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.147+0000 7f92df3e5640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f92c40a2f00 0x7f92c40a53c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009b30 tx=0x7f92d00097e0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.147+0000 7f92df3e5640 1 -- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92d8109180 msgr2=0x7f92d8118400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:49.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.147+0000 7f92df3e5640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92d8109180 0x7f92d8118400 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f92d400e9f0 tx=0x7f92d400edf0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.147+0000 7f92df3e5640 1 -- 172.21.15.69:0/4002196440 shutdown_connections 2024-08-02T18:01:49.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.147+0000 7f92df3e5640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f92c40a2f00 0x7f92c40a53c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.147+0000 7f92df3e5640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f92d811ae70 0x7f92d81119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.147+0000 7f92df3e5640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92d8109560 0x7f92d8111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.148+0000 7f92df3e5640 1 --2- 172.21.15.69:0/4002196440 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f92d8109180 0x7f92d8118400 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:49.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.148+0000 7f92df3e5640 1 -- 172.21.15.69:0/4002196440 >> 172.21.15.69:0/4002196440 conn(0x7f92d806de70 msgr2=0x7f92d811bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:49.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.148+0000 7f92df3e5640 1 -- 172.21.15.69:0/4002196440 shutdown_connections 2024-08-02T18:01:49.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:48.148+0000 7f92df3e5640 1 -- 172.21.15.69:0/4002196440 wait complete. 2024-08-02T18:01:49.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:49 smithi083 ceph-mon[37031]: pgmap v482: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 530 op/s 2024-08-02T18:01:49.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:49 smithi077 ceph-mon[37108]: pgmap v482: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.0 MiB/s wr, 530 op/s 2024-08-02T18:01:49.954 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 880.0 s, 444.4 tps, lat 220.096 ms stddev 282.255 2024-08-02T18:01:51.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:51 smithi083 ceph-mon[37031]: pgmap v483: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-08-02T18:01:51.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:51 smithi077 ceph-mon[37108]: pgmap v483: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-08-02T18:01:51.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:51 smithi069 ceph-mon[32746]: pgmap v483: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 543 op/s 2024-08-02T18:01:51.785 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T18:01:52.299 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:01:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.669+0000 7fccf6651640 1 -- 172.21.15.69:0/1000651163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccf011ae70 msgr2=0x7fccf011d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.669+0000 7fccf6651640 1 --2- 172.21.15.69:0/1000651163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccf011ae70 0x7fccf011d260 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcce801ace0 tx=0x7fcce803b990 comp rx=0 tx=0).stop 2024-08-02T18:01:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.670+0000 7fccf6651640 1 -- 172.21.15.69:0/1000651163 shutdown_connections 2024-08-02T18:01:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.670+0000 7fccf6651640 1 --2- 172.21.15.69:0/1000651163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccf011ae70 0x7fccf011d260 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.670+0000 7fccf6651640 1 --2- 172.21.15.69:0/1000651163 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccf0109300 0x7fccf0109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.670+0000 7fccf6651640 1 --2- 172.21.15.69:0/1000651163 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccf0108f20 0x7fccf010ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.673 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.670+0000 7fccf6651640 1 -- 172.21.15.69:0/1000651163 >> 172.21.15.69:0/1000651163 conn(0x7fccf006cb20 msgr2=0x7fccf006cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.674+0000 7fccf6651640 1 -- 172.21.15.69:0/1000651163 shutdown_connections 2024-08-02T18:01:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.677+0000 7fccf6651640 1 -- 172.21.15.69:0/1000651163 wait complete. 2024-08-02T18:01:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.677+0000 7fccf6651640 1 Processor -- start 2024-08-02T18:01:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.677+0000 7fccf6651640 1 -- start start 2024-08-02T18:01:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.678+0000 7fccf6651640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccf0108f20 0x7fccf011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.678+0000 7fccf6651640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccf0109300 0x7fccf0116710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.678+0000 7fccf6651640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccf011ae70 0x7fccf0116c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.678+0000 7fccf6651640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccf01130e0 con 0x7fccf0108f20 2024-08-02T18:01:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.678+0000 7fccf6651640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0112f60 con 0x7fccf011ae70 2024-08-02T18:01:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.678+0000 7fccf6651640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0113260 con 0x7fccf0109300 2024-08-02T18:01:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.679+0000 7fccf4bee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccf011ae70 0x7fccf0116c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.680+0000 7fccf4bee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccf011ae70 0x7fccf0116c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:50396/0 (socket says 172.21.15.69:50396) 2024-08-02T18:01:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.680+0000 7fccf4bee640 1 -- 172.21.15.69:0/4285118037 learned_addr learned my addr 172.21.15.69:0/4285118037 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.682+0000 7fcceffff640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccf0108f20 0x7fccf011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.682+0000 7fccef7fe640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccf0109300 0x7fccf0116710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.682+0000 7fccf4bee640 1 -- 172.21.15.69:0/4285118037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccf0109300 msgr2=0x7fccf0116710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.682+0000 7fccf4bee640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccf0109300 0x7fccf0116710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.684 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.682+0000 7fccf4bee640 1 -- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccf0108f20 msgr2=0x7fccf011a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.682+0000 7fccf4bee640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccf0108f20 0x7fccf011a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.682+0000 7fccf4bee640 1 -- 172.21.15.69:0/4285118037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf0117510 con 0x7fccf011ae70 2024-08-02T18:01:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.683+0000 7fccf4bee640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccf011ae70 0x7fccf0116c50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcce8003ba0 tx=0x7fcce8003bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.687+0000 7fcced7fa640 1 -- 172.21.15.69:0/4285118037 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8057070 con 0x7fccf011ae70 2024-08-02T18:01:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.688+0000 7fcced7fa640 1 -- 172.21.15.69:0/4285118037 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce8040bf0 con 0x7fccf011ae70 2024-08-02T18:01:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.688+0000 7fcced7fa640 1 -- 172.21.15.69:0/4285118037 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8045040 con 0x7fccf011ae70 2024-08-02T18:01:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.689+0000 7fccf6651640 1 -- 172.21.15.69:0/4285118037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf01aada0 con 0x7fccf011ae70 2024-08-02T18:01:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.689+0000 7fccf6651640 1 -- 172.21.15.69:0/4285118037 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf01ab230 con 0x7fccf011ae70 2024-08-02T18:01:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.692+0000 7fccf93ac740 1 Processor -- start 2024-08-02T18:01:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.692+0000 7fccf93ac740 1 -- start start 2024-08-02T18:01:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.692+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.692+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f301430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.692+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f303d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.692+0000 7fccf93ac740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55776f25a9b0 con 0x55776f2f67b0 2024-08-02T18:01:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.692+0000 7fccf93ac740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55776f202800 con 0x55776f301970 2024-08-02T18:01:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.692+0000 7fccf93ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55776f256970 con 0x55776f2f70d0 2024-08-02T18:01:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.697+0000 7fcceffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f301430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.697+0000 7fcceffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f301430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50018/0 (socket says 172.21.15.69:50018) 2024-08-02T18:01:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.697+0000 7fcceffff640 1 -- 172.21.15.69:0/3450943564 learned_addr learned my addr 172.21.15.69:0/3450943564 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.697+0000 7fccef7fe640 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.697+0000 7fccef7fe640 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f303d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.701+0000 7fcceffff640 1 -- 172.21.15.69:0/3450943564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 msgr2=0x55776f303d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.701+0000 7fcceffff640 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f303d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.701+0000 7fcceffff640 1 -- 172.21.15.69:0/3450943564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 msgr2=0x55776f2f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.701+0000 7fcceffff640 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.701+0000 7fcceffff640 1 -- 172.21.15.69:0/3450943564 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776ef6e210 con 0x55776f2f70d0 2024-08-02T18:01:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.701+0000 7fcceffff640 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f301430 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b820 tx=0x7fcce000bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f7e8cbe7c9be52d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.702+0000 7fccd67fc640 1 -- 172.21.15.69:0/3450943564 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce000d950 con 0x55776f2f70d0 2024-08-02T18:01:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.702+0000 7fccd67fc640 1 -- 172.21.15.69:0/3450943564 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce0019070 con 0x55776f2f70d0 2024-08-02T18:01:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.703+0000 7fccf93ac740 1 -- 172.21.15.69:0/3450943564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 msgr2=0x55776f301430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.707 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.703+0000 7fccf93ac740 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f301430 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b820 tx=0x7fcce000bce0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.708 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.707+0000 7fcced7fa640 1 -- 172.21.15.69:0/4285118037 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcce8048020 con 0x7fccf011ae70 2024-08-02T18:01:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.707+0000 7fccf93ac740 1 -- 172.21.15.69:0/3450943564 shutdown_connections 2024-08-02T18:01:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.707+0000 7fccf93ac740 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f303d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.707+0000 7fccf93ac740 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f301430 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.707+0000 7fccf93ac740 1 --2- 172.21.15.69:0/3450943564 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.707+0000 7fccf93ac740 1 -- 172.21.15.69:0/3450943564 >> 172.21.15.69:0/3450943564 conn(0x55776f2f2d20 msgr2=0x55776f2f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.712+0000 7fcced7fa640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccd80a2dc0 0x7fccd80a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.712+0000 7fcced7fa640 1 -- 172.21.15.69:0/4285118037 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7fcce8100a60 con 0x7fccf011ae70 2024-08-02T18:01:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.712+0000 7fccf93ac740 1 -- 172.21.15.69:0/3450943564 shutdown_connections 2024-08-02T18:01:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.713+0000 7fcceffff640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccd80a2dc0 0x7fccd80a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.714+0000 7fccf93ac740 1 -- 172.21.15.69:0/3450943564 wait complete. 2024-08-02T18:01:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.714+0000 7fccf93ac740 1 -- start start 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.714+0000 7fccf93ac740 1 Processor -- start 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2fe7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 0x55776f2fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f2ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fccf93ac740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55776f304590 con 0x55776f2f67b0 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fccf93ac740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55776f216500 con 0x55776f2f70d0 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fccf93ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55776f304710 con 0x55776f301970 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fccf93ac740 10 client.0 _ll_register_callbacks cb 0x55776ef6da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fcceffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2fe7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fcceffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2fe7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34604/0 (socket says 172.21.15.69:34604) 2024-08-02T18:01:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.715+0000 7fcceffff640 1 -- 172.21.15.69:0/364801660 learned_addr learned my addr 172.21.15.69:0/364801660 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.716+0000 7fccef7fe640 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 0x55776f2fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.717+0000 7fcceffff640 1 -- 172.21.15.69:0/364801660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 msgr2=0x55776f2ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.717+0000 7fcceffff640 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f2ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.717+0000 7fcceffff640 1 -- 172.21.15.69:0/364801660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 msgr2=0x55776f2fece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.717+0000 7fcceffff640 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 0x55776f2fece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.717+0000 7fcceffff640 1 -- 172.21.15.69:0/364801660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55776f03a620 con 0x55776f2f67b0 2024-08-02T18:01:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.719+0000 7fcceffff640 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2fe7a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcce0006e70 tx=0x7fcce000dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa9f811a1134d56 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.721+0000 7fcccf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:01:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.722+0000 7fcceffff640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccd80a2dc0 0x7fccd80a5280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b820 tx=0x7fcce001c000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.726+0000 7fcccf7fe640 1 -- 172.21.15.69:0/364801660 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0004250 con 0x55776f2f67b0 2024-08-02T18:01:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.726+0000 7fccf93ac740 1 -- 172.21.15.69:0/364801660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce0019070 con 0x55776f2f67b0 2024-08-02T18:01:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.726+0000 7fcccf7fe640 1 -- 172.21.15.69:0/364801660 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7fcce00043f0 con 0x55776f2f67b0 2024-08-02T18:01:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.728+0000 7fcccf7fe640 1 -- 172.21.15.69:0/364801660 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcce00045f0 con 0x55776f2f67b0 2024-08-02T18:01:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.729+0000 7fccf93ac740 10 client.19221 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.729+0000 7fccf93ac740 1 -- 172.21.15.69:0/364801660 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55776f332200 con 0x55776f2f67b0 2024-08-02T18:01:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.733+0000 7fcccf7fe640 1 -- 172.21.15.69:0/364801660 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce0019070 con 0x55776f2f67b0 2024-08-02T18:01:52.734 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.733+0000 7fcccf7fe640 1 -- 172.21.15.69:0/364801660 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcce0028070 con 0x55776f2f67b0 2024-08-02T18:01:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.735+0000 7fccf93ac740 10 client.19221 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.735+0000 7fccf93ac740 10 client.19221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.735+0000 7fccf93ac740 20 client.19221 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.735+0000 7fccf93ac740 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55776f333660 0x55776f335a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.735+0000 7fccf93ac740 4 client.19221 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:52.738 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.735+0000 7fccf93ac740 1 -- 172.21.15.69:0/364801660 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55776f2680e0 con 0x55776f333660 2024-08-02T18:01:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.741+0000 7fccef7fe640 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55776f333660 0x55776f335a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.749+0000 7fccef7fe640 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55776f333660 0x55776f335a50 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed40cb65bcd38156 server_cookie=f641f121ed77e890 in_seq=0 out_seq=0 2024-08-02T18:01:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.749+0000 7fcccf7fe640 10 client.19221 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.750+0000 7fcccf7fe640 1 -- 172.21.15.69:0/364801660 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55776f2680e0 con 0x55776f333660 2024-08-02T18:01:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.750+0000 7fcccf7fe640 10 client.19221 handle_command_reply: tid=0 2024-08-02T18:01:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.750+0000 7fccf93ac740 1 client.19221 shutdown 2024-08-02T18:01:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.752+0000 7fccf93ac740 1 -- 172.21.15.69:0/364801660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 msgr2=0x55776f2fe7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.752+0000 7fccf93ac740 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2fe7a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fcce0006e70 tx=0x7fcce000dee0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.756+0000 7fccf93ac740 1 -- 172.21.15.69:0/364801660 shutdown_connections 2024-08-02T18:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.756+0000 7fccf93ac740 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55776f333660 0x55776f335a50 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.756+0000 7fccf93ac740 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f2ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.756+0000 7fccf93ac740 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 0x55776f2fece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.756+0000 7fccf93ac740 1 --2- 172.21.15.69:0/364801660 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f2fe7a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.756+0000 7fccf93ac740 1 -- 172.21.15.69:0/364801660 >> 172.21.15.69:0/364801660 conn(0x55776f2f2d20 msgr2=0x55776f2f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.758 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.757+0000 7fcccf7fe640 0 client.19221 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.762+0000 7fccf93ac740 1 -- 172.21.15.69:0/364801660 shutdown_connections 2024-08-02T18:01:52.763 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.762+0000 7fccf93ac740 1 -- 172.21.15.69:0/364801660 wait complete. 2024-08-02T18:01:52.769 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.763+0000 7fccf93ac740 20 client.19221 trim_cache size 0 max 16384 2024-08-02T18:01:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.796+0000 7fccf93ac740 1 Processor -- start 2024-08-02T18:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.796+0000 7fccf93ac740 1 -- start start 2024-08-02T18:01:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.797+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f3009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fcceffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f3009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 0x55776f2fe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f33b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fccf93ac740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccf0080aa0 con 0x55776f2f67b0 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fccf93ac740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccf007e370 con 0x55776f2f70d0 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fccf93ac740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccf007c1c0 con 0x55776f301970 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fccf4bee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f33b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fccf4bee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f33b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50044/0 (socket says 172.21.15.69:50044) 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.798+0000 7fccf4bee640 1 -- 172.21.15.69:0/2332119231 learned_addr learned my addr 172.21.15.69:0/2332119231 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.799+0000 7fccf4bee640 1 -- 172.21.15.69:0/2332119231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 msgr2=0x55776f2fe060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:01:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.799+0000 7fccf4bee640 1 --2- 172.21.15.69:0/2332119231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 0x55776f2fe060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.799+0000 7fccf4bee640 1 -- 172.21.15.69:0/2332119231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 msgr2=0x55776f3009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.799+0000 7fccf4bee640 1 --2- 172.21.15.69:0/2332119231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f3009c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.799+0000 7fccf4bee640 1 -- 172.21.15.69:0/2332119231 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55776f2f2b50 con 0x55776f301970 2024-08-02T18:01:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.800+0000 7fccf4bee640 1 --2- 172.21.15.69:0/2332119231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f33b190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcce80d5380 tx=0x7fcce80cc780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2afbffaf60da39f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.804+0000 7fccd5ffb640 1 -- 172.21.15.69:0/2332119231 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce803d040 con 0x55776f301970 2024-08-02T18:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.804+0000 7fccd5ffb640 1 -- 172.21.15.69:0/2332119231 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce80cd340 con 0x55776f301970 2024-08-02T18:01:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.804+0000 7fccd5ffb640 1 -- 172.21.15.69:0/2332119231 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80c92b0 con 0x55776f301970 2024-08-02T18:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.805+0000 7fccf93ac740 1 -- 172.21.15.69:0/2332119231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 msgr2=0x55776f33b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.805+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2332119231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f33b190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcce80d5380 tx=0x7fcce80cc780 comp rx=0 tx=0).stop 2024-08-02T18:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.805+0000 7fccf93ac740 1 -- 172.21.15.69:0/2332119231 shutdown_connections 2024-08-02T18:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.805+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2332119231 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f301970 0x55776f33b190 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.805+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2332119231 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f2f70d0 0x55776f2fe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.805+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2332119231 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f3009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.808 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.805+0000 7fccf93ac740 1 -- 172.21.15.69:0/2332119231 >> 172.21.15.69:0/2332119231 conn(0x55776f2f2d20 msgr2=0x55776f2f1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.809+0000 7fccf93ac740 1 -- 172.21.15.69:0/2332119231 shutdown_connections 2024-08-02T18:01:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.812+0000 7fccf93ac740 1 -- 172.21.15.69:0/2332119231 wait complete. 2024-08-02T18:01:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.812+0000 7fccf93ac740 1 -- start start 2024-08-02T18:01:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.813+0000 7fccf93ac740 1 Processor -- start 2024-08-02T18:01:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.813+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f30a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f305540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccf93ac740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f305aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f305540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f305540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50048/0 (socket says 172.21.15.69:50048) 2024-08-02T18:01:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccef7fe640 1 -- 172.21.15.69:0/2040384439 learned_addr learned my addr 172.21.15.69:0/2040384439 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fccf007c830 con 0x55776f2f67b0 2024-08-02T18:01:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fccf007dca0 con 0x55776f301970 2024-08-02T18:01:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fccf007cf00 con 0x55776f2f70d0 2024-08-02T18:01:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccf4bee640 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f305aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.814+0000 7fccf93ac740 10 client.0 _ll_register_callbacks cb 0x7fcce000b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.815+0000 7fccf4bee640 1 -- 172.21.15.69:0/2040384439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 msgr2=0x55776f305540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.815+0000 7fccf4bee640 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f305540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.815+0000 7fccf4bee640 1 -- 172.21.15.69:0/2040384439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 msgr2=0x55776f30a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.815+0000 7fccf4bee640 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f30a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.815+0000 7fccf4bee640 1 -- 172.21.15.69:0/2040384439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55776f3062d0 con 0x55776f301970 2024-08-02T18:01:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.818+0000 7fccf4bee640 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f305aa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcce80c90a0 tx=0x7fcce80cb150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce4896d2462c72c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.818+0000 7fcccf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:01:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.819+0000 7fccef7fe640 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f305540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:01:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.820+0000 7fcccf7fe640 1 -- 172.21.15.69:0/2040384439 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80ea060 con 0x55776f301970 2024-08-02T18:01:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.820+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcce80cd340 con 0x55776f301970 2024-08-02T18:01:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.820+0000 7fcccf7fe640 1 -- 172.21.15.69:0/2040384439 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7fcce80ccea0 con 0x55776f301970 2024-08-02T18:01:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.820+0000 7fcccf7fe640 1 -- 172.21.15.69:0/2040384439 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcce80db020 con 0x55776f301970 2024-08-02T18:01:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.820+0000 7fccf93ac740 10 client.28610 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.820+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55776f39e1e0 con 0x55776f301970 2024-08-02T18:01:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.821+0000 7fcccf7fe640 1 -- 172.21.15.69:0/2040384439 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcce80d0ff0 con 0x55776f301970 2024-08-02T18:01:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.821+0000 7fcccf7fe640 1 -- 172.21.15.69:0/2040384439 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fcce80fdf50 con 0x55776f301970 2024-08-02T18:01:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.823+0000 7fccf93ac740 10 client.28610 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.823+0000 7fccf93ac740 10 client.28610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.823+0000 7fccf93ac740 20 client.28610 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.823+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55776f39e7c0 0x55776f3a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.824+0000 7fccf93ac740 4 client.28610 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:01:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.824+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fcce80cd340 con 0x55776f39e7c0 2024-08-02T18:01:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.824+0000 7fcceffff640 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55776f39e7c0 0x55776f3a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.828+0000 7fcceffff640 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55776f39e7c0 0x55776f3a0be0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17676fcdb21d1535 server_cookie=9aa1e0f308e9c966 in_seq=0 out_seq=0 2024-08-02T18:01:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.828+0000 7fcccf7fe640 10 client.28610 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:52.847 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:52 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-08-02T18:01:52.848 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:52 smithi077 systemd[1]: Stopped Ceph mds.e for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T18:01:52.848 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:52 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.e.service: Consumed 3.606s CPU time. 2024-08-02T18:01:52.848 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:52 smithi077 systemd[1]: Starting Ceph mds.e for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:01:52.916 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.914+0000 7f0fa592d640 1 -- 172.21.15.69:0/374936731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 msgr2=0x7f0fa0069ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.917 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.914+0000 7f0fa592d640 1 --2- 172.21.15.69:0/374936731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 0x7f0fa0069ca0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0f98013490 tx=0x7f0f98032bc0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.918 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.916+0000 7f0fa592d640 1 -- 172.21.15.69:0/374936731 shutdown_connections 2024-08-02T18:01:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.916+0000 7f0fa592d640 1 --2- 172.21.15.69:0/374936731 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 0x7f0fa0069ca0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.916+0000 7f0fa592d640 1 --2- 172.21.15.69:0/374936731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa01022a0 0x7f0fa01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.916+0000 7f0fa592d640 1 --2- 172.21.15.69:0/374936731 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa0100b50 0x7f0fa0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.919 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.916+0000 7f0fa592d640 1 -- 172.21.15.69:0/374936731 >> 172.21.15.69:0/374936731 conn(0x7f0fa00fc500 msgr2=0x7f0fa00fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.923 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.922+0000 7f0fa592d640 1 -- 172.21.15.69:0/374936731 shutdown_connections 2024-08-02T18:01:52.924 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.922+0000 7f0fa592d640 1 -- 172.21.15.69:0/374936731 wait complete. 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.923+0000 7f0fa592d640 1 Processor -- start 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.923+0000 7f0fa592d640 1 -- start start 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.923+0000 7f0fa592d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa0100b50 0x7f0fa01a7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.923+0000 7f0fa592d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa01022a0 0x7f0fa01a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.923+0000 7f0fa592d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 0x7f0fa01ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.923+0000 7f0fa592d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa0120d20 con 0x7f0fa0102c20 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.923+0000 7f0fa592d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa0120ba0 con 0x7f0fa0100b50 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.923+0000 7f0fa592d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa0120ea0 con 0x7f0fa01022a0 2024-08-02T18:01:52.925 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.924+0000 7f0f9f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 0x7f0fa01ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.929 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.927+0000 7f0f9f7fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 0x7f0fa01ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34640/0 (socket says 172.21.15.69:34640) 2024-08-02T18:01:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.927+0000 7f0f9f7fe640 1 -- 172.21.15.69:0/3547601956 learned_addr learned my addr 172.21.15.69:0/3547601956 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.928+0000 7f0f9effd640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa0100b50 0x7f0fa01a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.928+0000 7f0f967fc640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa01022a0 0x7f0fa01a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.930 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.928+0000 7f0f9effd640 1 -- 172.21.15.69:0/3547601956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa01022a0 msgr2=0x7f0fa01a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.928+0000 7f0f9effd640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa01022a0 0x7f0fa01a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.928+0000 7f0f9effd640 1 -- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 msgr2=0x7f0fa01ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.928+0000 7f0f9effd640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 0x7f0fa01ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.928+0000 7f0f9effd640 1 -- 172.21.15.69:0/3547601956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa01ac060 con 0x7f0fa0100b50 2024-08-02T18:01:52.931 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.929+0000 7f0f9effd640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa0100b50 0x7f0fa01a7090 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009fd0 tx=0x7f0f9000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.930+0000 7f0fa492b640 1 -- 172.21.15.69:0/3547601956 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90015020 con 0x7f0fa0100b50 2024-08-02T18:01:52.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.930+0000 7f0fa492b640 1 -- 172.21.15.69:0/3547601956 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f90011d10 con 0x7f0fa0100b50 2024-08-02T18:01:52.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.930+0000 7f0fa492b640 1 -- 172.21.15.69:0/3547601956 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9000f2e0 con 0x7f0fa0100b50 2024-08-02T18:01:52.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.931+0000 7f0fa592d640 1 -- 172.21.15.69:0/3547601956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa01ac350 con 0x7f0fa0100b50 2024-08-02T18:01:52.932 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.931+0000 7f0fa592d640 1 -- 172.21.15.69:0/3547601956 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa006b940 con 0x7f0fa0100b50 2024-08-02T18:01:52.942 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.940+0000 7f0fa492b640 1 -- 172.21.15.69:0/3547601956 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0f90018020 con 0x7f0fa0100b50 2024-08-02T18:01:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.941+0000 7f0fa492b640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f8c0a2e80 0x7f0f8c0a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.943 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.942+0000 7f0fa492b640 1 -- 172.21.15.69:0/3547601956 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f0f900cedb0 con 0x7f0fa0100b50 2024-08-02T18:01:52.945 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.944+0000 7f0f967fc640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f8c0a2e80 0x7f0f8c0a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.949 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.944+0000 7f0fa7d2b740 1 Processor -- start 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.945+0000 7f0fa7d2b740 1 -- start start 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.945+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a0ff0 0x562bad8a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.945+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a1910 0x562bad8abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.945+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad8ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.945+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562bad806a40 con 0x562bad8a1910 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.945+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562bad79e9d0 con 0x562bad8a0ff0 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.945+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bad802a00 con 0x562bad8ac1d0 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.946+0000 7f0f9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad8ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.946+0000 7f0f9f7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad8ae5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50066/0 (socket says 172.21.15.69:50066) 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.946+0000 7f0f9f7fe640 1 -- 172.21.15.69:0/528377865 learned_addr learned my addr 172.21.15.69:0/528377865 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:52.950 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.948+0000 7f0f9effd640 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a0ff0 0x562bad8a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.948+0000 7f0f967fc640 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a1910 0x562bad8abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.948+0000 7f0f9f7fe640 1 -- 172.21.15.69:0/528377865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a0ff0 msgr2=0x562bad8a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.948+0000 7f0f9f7fe640 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a0ff0 0x562bad8a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.948+0000 7f0f9f7fe640 1 -- 172.21.15.69:0/528377865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a1910 msgr2=0x562bad8abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.948+0000 7f0f9f7fe640 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a1910 0x562bad8abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.948+0000 7f0f9f7fe640 1 -- 172.21.15.69:0/528377865 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bad7a5650 con 0x562bad8ac1d0 2024-08-02T18:01:52.951 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.949+0000 7f0f967fc640 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a1910 0x562bad8abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:01:52.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.950+0000 7f0f967fc640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f8c0a2e80 0x7f0f8c0a5340 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006fd0 tx=0x7f0f88008040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.952 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.950+0000 7f0f9f7fe640 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad8ae5c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800fcc0 tx=0x7f0f9800fcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ccc1531c1d922d5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.952+0000 7f0f95ffb640 1 -- 172.21.15.69:0/528377865 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f98041070 con 0x562bad8ac1d0 2024-08-02T18:01:52.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.952+0000 7f0f95ffb640 1 -- 172.21.15.69:0/528377865 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f98034ce0 con 0x562bad8ac1d0 2024-08-02T18:01:52.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.953+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/528377865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 msgr2=0x562bad8ae5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.955 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.953+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad8ae5c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0f9800fcc0 tx=0x7f0f9800fcf0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.956 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.955+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/528377865 shutdown_connections 2024-08-02T18:01:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.955+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a1910 0x562bad8abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.955+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad8ae5c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.955+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/528377865 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a0ff0 0x562bad8a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.957 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.955+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/528377865 >> 172.21.15.69:0/528377865 conn(0x562bad89d580 msgr2=0x562bad89d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.961 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.959+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/528377865 shutdown_connections 2024-08-02T18:01:52.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.959+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/528377865 wait complete. 2024-08-02T18:01:52.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.960+0000 7f0fa7d2b740 1 -- start start 2024-08-02T18:01:52.962 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.960+0000 7f0fa7d2b740 1 Processor -- start 2024-08-02T18:01:52.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8abc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 0x562bad8a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x562bad8aedf0 con 0x562bad8a0ff0 2024-08-02T18:01:52.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x562bad7c1b20 con 0x562bad8ac1d0 2024-08-02T18:01:52.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x562bad8aef70 con 0x562bad8a1910 2024-08-02T18:01:52.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0fa7d2b740 10 client.0 _ll_register_callbacks cb 0x562bad7bfbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:52.964 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0f967fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0f967fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:50080/0 (socket says 172.21.15.69:50080) 2024-08-02T18:01:52.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.961+0000 7f0f967fc640 1 -- 172.21.15.69:0/2461063860 learned_addr learned my addr 172.21.15.69:0/2461063860 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:52.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.962+0000 7f0f967fc640 1 -- 172.21.15.69:0/2461063860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 msgr2=0x562bad8a9740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:01:52.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.962+0000 7f0f967fc640 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 0x562bad8a9740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.962+0000 7f0f967fc640 1 -- 172.21.15.69:0/2461063860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 msgr2=0x562bad8abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:52.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.962+0000 7f0f967fc640 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.965 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.962+0000 7f0f967fc640 1 -- 172.21.15.69:0/2461063860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bad5783c0 con 0x562bad8a1910 2024-08-02T18:01:52.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.963+0000 7f0f967fc640 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a8fb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006d40 tx=0x7f0f88004770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadca5faccbaed0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:52.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.964+0000 7f0f7effd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:52.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.964+0000 7f0f7effd640 1 -- 172.21.15.69:0/2461063860 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8801a050 con 0x562bad8a1910 2024-08-02T18:01:52.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.964+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/2461063860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f98034ce0 con 0x562bad8a1910 2024-08-02T18:01:52.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.965+0000 7f0f7effd640 1 -- 172.21.15.69:0/2461063860 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f0f880125c0 con 0x562bad8a1910 2024-08-02T18:01:52.966 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.965+0000 7f0f7effd640 1 -- 172.21.15.69:0/2461063860 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f88014e40 con 0x562bad8a1910 2024-08-02T18:01:52.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.968+0000 7f0fa7d2b740 10 client.28363 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:52.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.968+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/2461063860 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562bad8ba9e0 con 0x562bad8a1910 2024-08-02T18:01:52.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.968+0000 7f0f7effd640 1 -- 172.21.15.69:0/2461063860 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f98034ce0 con 0x562bad8a1910 2024-08-02T18:01:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.969+0000 7f0f7effd640 1 -- 172.21.15.69:0/2461063860 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f88013070 con 0x562bad8a1910 2024-08-02T18:01:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.969+0000 7f0fa7d2b740 10 client.28363 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.969+0000 7f0fa7d2b740 10 client.28363 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:01:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.969+0000 7f0fa7d2b740 20 client.28363 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.969+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562bad8bc370 0x562bad8be830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:52.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.969+0000 7f0fa7d2b740 4 client.28363 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:52.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.969+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/2461063860 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bad813fb0 con 0x562bad8bc370 2024-08-02T18:01:52.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.969+0000 7f0f9effd640 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562bad8bc370 0x562bad8be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:52.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.971+0000 7f0f9effd640 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562bad8bc370 0x562bad8be830 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5944679fcece366a server_cookie=38426239bb74cfc6 in_seq=0 out_seq=0 2024-08-02T18:01:52.972 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.971+0000 7f0f7effd640 10 client.28363 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:01:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.972+0000 7f0f7effd640 1 -- 172.21.15.69:0/2461063860 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562bad813fb0 con 0x562bad8bc370 2024-08-02T18:01:52.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.972+0000 7f0f7effd640 10 client.28363 handle_command_reply: tid=0 2024-08-02T18:01:52.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.972+0000 7f0fa7d2b740 1 client.28363 shutdown 2024-08-02T18:01:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/2461063860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 msgr2=0x562bad8a8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a8fb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006d40 tx=0x7f0f88004770 comp rx=0 tx=0).stop 2024-08-02T18:01:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/2461063860 shutdown_connections 2024-08-02T18:01:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562bad8bc370 0x562bad8be830 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 0x562bad8a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a8fb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/2461063860 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/2461063860 >> 172.21.15.69:0/2461063860 conn(0x562bad89d580 msgr2=0x562bad8a32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0f7effd640 0 client.28363 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:01:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/2461063860 shutdown_connections 2024-08-02T18:01:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.977+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/2461063860 wait complete. 2024-08-02T18:01:52.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.978+0000 7f0fa7d2b740 20 client.28363 trim_cache size 0 max 16384 2024-08-02T18:01:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fcccf7fe640 1 -- 172.21.15.69:0/2040384439 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fcce80cd340 con 0x55776f39e7c0 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fcccf7fe640 10 client.28610 handle_command_reply: tid=0 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 client.28610 shutdown 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 msgr2=0x55776f305aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f305aa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcce80c90a0 tx=0x7fcce80cb150 comp rx=0 tx=0).stop 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 shutdown_connections 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55776f39e7c0 0x55776f3a0be0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55776f301970 0x55776f305aa0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55776f2f70d0 0x55776f305540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 --2- 172.21.15.69:0/2040384439 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55776f2f67b0 0x55776f30a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fcccf7fe640 0 client.28610 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.989+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 >> 172.21.15.69:0/2040384439 conn(0x55776f2f2d20 msgr2=0x55776f2f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 shutdown_connections 2024-08-02T18:01:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf93ac740 1 -- 172.21.15.69:0/2040384439 wait complete. 2024-08-02T18:01:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf93ac740 20 client.28610 trim_cache size 0 max 16384 2024-08-02T18:01:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:01:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:01:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:01:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:01:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf6651640 1 -- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccd80a2dc0 msgr2=0x7fccd80a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf6651640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccd80a2dc0 0x7fccd80a5280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b820 tx=0x7fcce001c000 comp rx=0 tx=0).stop 2024-08-02T18:01:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf6651640 1 -- 172.21.15.69:0/4285118037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccf011ae70 msgr2=0x7fccf0116c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf6651640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccf011ae70 0x7fccf0116c50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcce8003ba0 tx=0x7fcce8003bd0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf6651640 1 -- 172.21.15.69:0/4285118037 shutdown_connections 2024-08-02T18:01:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.990+0000 7fccf6651640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fccd80a2dc0 0x7fccd80a5280 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.991+0000 7fccf6651640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fccf011ae70 0x7fccf0116c50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.991+0000 7fccf6651640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fccf0109300 0x7fccf0116710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.991+0000 7fccf6651640 1 --2- 172.21.15.69:0/4285118037 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fccf0108f20 0x7fccf011a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.991+0000 7fccf6651640 1 -- 172.21.15.69:0/4285118037 >> 172.21.15.69:0/4285118037 conn(0x7fccf006cb20 msgr2=0x7fccf0070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.991+0000 7fccf6651640 1 -- 172.21.15.69:0/4285118037 shutdown_connections 2024-08-02T18:01:52.996 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:01:52.991+0000 7fccf6651640 1 -- 172.21.15.69:0/4285118037 wait complete. 2024-08-02T18:01:53.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.998+0000 7f0fa7d2b740 1 Processor -- start 2024-08-02T18:01:53.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.998+0000 7f0fa7d2b740 1 -- start start 2024-08-02T18:01:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a1910 0x562bad8a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad89d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80012a70 con 0x562bad8a0ff0 2024-08-02T18:01:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80010340 con 0x562bad8a1910 2024-08-02T18:01:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0fa7d2b740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8000e190 con 0x562bad8ac1d0 2024-08-02T18:01:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:53.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34668/0 (socket says 172.21.15.69:34668) 2024-08-02T18:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f9effd640 1 -- 172.21.15.69:0/642908580 learned_addr learned my addr 172.21.15.69:0/642908580 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f967fc640 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a1910 0x562bad8a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f9f7fe640 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad89d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f967fc640 1 -- 172.21.15.69:0/642908580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 msgr2=0x562bad89d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f967fc640 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad89d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f967fc640 1 -- 172.21.15.69:0/642908580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 msgr2=0x562bad8a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:52.999+0000 7f0f967fc640 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.000+0000 7f0f967fc640 1 -- 172.21.15.69:0/642908580 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bad8beba0 con 0x562bad8a1910 2024-08-02T18:01:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.000+0000 7f0f967fc640 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a1910 0x562bad8a47f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f880061e0 tx=0x7f0f88009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a7c0298fd58efe server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.000+0000 7f0f957fa640 1 -- 172.21.15.69:0/642908580 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88035070 con 0x562bad8a1910 2024-08-02T18:01:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.000+0000 7f0f957fa640 1 -- 172.21.15.69:0/642908580 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f88024070 con 0x562bad8a1910 2024-08-02T18:01:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.000+0000 7f0f957fa640 1 -- 172.21.15.69:0/642908580 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8801a070 con 0x562bad8a1910 2024-08-02T18:01:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/642908580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a1910 msgr2=0x562bad8a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a1910 0x562bad8a47f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0f880061e0 tx=0x7f0f88009710 comp rx=0 tx=0).stop 2024-08-02T18:01:53.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/642908580 shutdown_connections 2024-08-02T18:01:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8ac1d0 0x562bad89d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8a1910 0x562bad8a47f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/642908580 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/642908580 >> 172.21.15.69:0/642908580 conn(0x562bad89d580 msgr2=0x562bad8981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/642908580 shutdown_connections 2024-08-02T18:01:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.001+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/642908580 wait complete. 2024-08-02T18:01:53.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.002+0000 7f0fa7d2b740 1 -- start start 2024-08-02T18:01:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.002+0000 7f0fa7d2b740 1 Processor -- start 2024-08-02T18:01:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.002+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:53.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0fa7d2b740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 0x562bad8a5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:34672/0 (socket says 172.21.15.69:34672) 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 -- 172.21.15.69:0/475969678 learned_addr learned my addr 172.21.15.69:0/475969678 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8000e800 con 0x562bad8a0ff0 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8000fc70 con 0x562bad8ac1d0 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8000eed0 con 0x562bad8a1910 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0fa7d2b740 10 client.0 _ll_register_callbacks cb 0x7f0f88009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f967fc640 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9f7fe640 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 0x562bad8a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 -- 172.21.15.69:0/475969678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 msgr2=0x562bad8a5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:53.007 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 -- 172.21.15.69:0/475969678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 msgr2=0x562bad8a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 0x562bad8a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 -- 172.21.15.69:0/475969678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bad8a5f70 con 0x562bad8a0ff0 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f967fc640 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f9effd640 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a6aa0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0f900a3710 tx=0x7f0f90010ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73576bd69375ae0f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.003+0000 7f0f7effd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0f7effd640 1 -- 172.21.15.69:0/475969678 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900b3070 con 0x562bad8a0ff0 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0f7effd640 1 -- 172.21.15.69:0/475969678 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f0f90010380 con 0x562bad8a0ff0 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f88024070 con 0x562bad8a0ff0 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0f7effd640 1 -- 172.21.15.69:0/475969678 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0f900ac050 con 0x562bad8a0ff0 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 10 client.19236 fetch_fsmap learned FSMap version 26 2024-08-02T18:01:53.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562bad8b4330 con 0x562bad8a0ff0 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0f7effd640 1 -- 172.21.15.69:0/475969678 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f9009f380 con 0x562bad8a0ff0 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0f7effd640 1 -- 172.21.15.69:0/475969678 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f0f90097640 con 0x562bad8a0ff0 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 10 client.19236 fetch_fsmap finished waiting for FSMap version 26 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 10 client.19236 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 20 client.19236 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562bad8b49b0 0x562bad929120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 4 client.19236 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.004+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0f88024070 con 0x562bad8b49b0 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.005+0000 7f0f967fc640 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562bad8b49b0 0x562bad929120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.005+0000 7f0f7effd640 10 client.19236 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:01:53.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.005+0000 7f0f967fc640 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562bad8b49b0 0x562bad929120 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df5441342bbc2c3f server_cookie=4cf46c8dd2356095 in_seq=0 out_seq=0 2024-08-02T18:01:53.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.005+0000 7f0f7effd640 1 -- 172.21.15.69:0/475969678 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0f88024070 con 0x562bad8b49b0 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.005+0000 7f0f7effd640 10 client.19236 handle_command_reply: tid=0 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.005+0000 7f0fa7d2b740 1 client.19236 shutdown 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 msgr2=0x562bad8a6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a6aa0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f0f900a3710 tx=0x7f0f90010ff0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 shutdown_connections 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x562bad8b49b0 0x562bad929120 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x562bad8ac1d0 0x562bad8a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0f7effd640 0 client.19236 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x562bad8a1910 0x562bad8a5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 --2- 172.21.15.69:0/475969678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x562bad8a0ff0 0x562bad8a6aa0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 >> 172.21.15.69:0/475969678 conn(0x562bad89d580 msgr2=0x562bad8a42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 shutdown_connections 2024-08-02T18:01:53.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 1 -- 172.21.15.69:0/475969678 wait complete. 2024-08-02T18:01:53.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa7d2b740 20 client.19236 trim_cache size 0 max 16384 2024-08-02T18:01:53.012 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:01:53.012 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa592d640 1 -- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f8c0a2e80 msgr2=0x7f0f8c0a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa592d640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f8c0a2e80 0x7f0f8c0a5340 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0f88006fd0 tx=0x7f0f88008040 comp rx=0 tx=0).stop 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa592d640 1 -- 172.21.15.69:0/3547601956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa0100b50 msgr2=0x7f0fa01a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.006+0000 7f0fa592d640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa0100b50 0x7f0fa01a7090 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009fd0 tx=0x7f0f9000b870 comp rx=0 tx=0).stop 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.007+0000 7f0fa592d640 1 -- 172.21.15.69:0/3547601956 shutdown_connections 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.007+0000 7f0fa592d640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0fa0102c20 0x7f0fa01ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.007+0000 7f0fa592d640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0f8c0a2e80 0x7f0f8c0a5340 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.007+0000 7f0fa592d640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fa01022a0 0x7f0fa01a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.007+0000 7f0fa592d640 1 --2- 172.21.15.69:0/3547601956 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0fa0100b50 0x7f0fa01a7090 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.007+0000 7f0fa592d640 1 -- 172.21.15.69:0/3547601956 >> 172.21.15.69:0/3547601956 conn(0x7f0fa00fc500 msgr2=0x7f0fa0106cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.007+0000 7f0fa592d640 1 -- 172.21.15.69:0/3547601956 shutdown_connections 2024-08-02T18:01:53.013 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:53.007+0000 7f0fa592d640 1 -- 172.21.15.69:0/3547601956 wait complete. 2024-08-02T18:01:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:53 smithi077 ceph-mon[37108]: pgmap v484: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 MiB/s wr, 480 op/s 2024-08-02T18:01:53.348 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:53 smithi077 podman[73026]: 2024-08-02 18:01:52.925654417 +0000 UTC m=+0.101603635 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:01:53.348 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:53 smithi077 podman[73026]: 2024-08-02 18:01:53.268657023 +0000 UTC m=+0.444606235 container create db41dd57f02febd9e4f588d52743106f158e85ec09fac49d6ebff7c8ded537a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0) 2024-08-02T18:01:53.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:53 smithi069 ceph-mon[32746]: pgmap v484: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 MiB/s wr, 480 op/s 2024-08-02T18:01:53.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:53 smithi083 ceph-mon[37031]: pgmap v484: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.5 MiB/s wr, 480 op/s 2024-08-02T18:01:53.924 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T18:01:54.273 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:53 smithi077 podman[73026]: 2024-08-02 18:01:53.884469783 +0000 UTC m=+1.060419015 container init db41dd57f02febd9e4f588d52743106f158e85ec09fac49d6ebff7c8ded537a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-08-02T18:01:54.274 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:53 smithi077 podman[73026]: 2024-08-02 18:01:53.892955782 +0000 UTC m=+1.068904998 container start db41dd57f02febd9e4f588d52743106f158e85ec09fac49d6ebff7c8ded537a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-02T18:01:54.555 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:01:54.597 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:54 smithi077 bash[73026]: db41dd57f02febd9e4f588d52743106f158e85ec09fac49d6ebff7c8ded537a9 2024-08-02T18:01:54.597 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:54 smithi077 systemd[1]: Started Ceph mds.e for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T18:01:54.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 885.0 s, 680.0 tps, lat 148.859 ms stddev 199.300 2024-08-02T18:01:55.097 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:01:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e[73039]: starting mds.e at 2024-08-02T18:01:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:55 smithi083 ceph-mon[37031]: pgmap v485: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-08-02T18:01:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:55 smithi083 ceph-mon[37031]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-08-02T18:01:55.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:55 smithi083 ceph-mon[37031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-08-02T18:01:55.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:55 smithi077 ceph-mon[37108]: pgmap v485: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-08-02T18:01:55.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:55 smithi077 ceph-mon[37108]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-08-02T18:01:55.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:55 smithi077 ceph-mon[37108]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-08-02T18:01:55.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:55 smithi069 ceph-mon[32746]: pgmap v485: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-08-02T18:01:55.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:55 smithi069 ceph-mon[32746]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-08-02T18:01:55.608 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:55 smithi069 ceph-mon[32746]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2024-08-02T18:01:57.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:56 smithi083 ceph-mon[37031]: pgmap v486: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-02T18:01:57.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:56 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6834/3960673726,v1:172.21.15.77:6835/3960673726] up:boot 2024-08-02T18:01:57.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:56 smithi083 ceph-mon[37031]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T18:01:57.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:56 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T18:01:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:56 smithi077 ceph-mon[37108]: pgmap v486: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-02T18:01:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:56 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6834/3960673726,v1:172.21.15.77:6835/3960673726] up:boot 2024-08-02T18:01:57.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:56 smithi077 ceph-mon[37108]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T18:01:57.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:56 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T18:01:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:56 smithi069 ceph-mon[32746]: pgmap v486: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 4.4 MiB/s wr, 470 op/s 2024-08-02T18:01:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:56 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6834/3960673726,v1:172.21.15.77:6835/3960673726] up:boot 2024-08-02T18:01:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:56 smithi069 ceph-mon[32746]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-08-02T18:01:57.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:56 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-08-02T18:01:58.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:58 smithi069 ceph-mon[32746]: pgmap v487: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-08-02T18:01:59.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:58 smithi083 ceph-mon[37031]: pgmap v487: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-08-02T18:01:59.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:58 smithi077 ceph-mon[37108]: pgmap v487: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5.4 MiB/s wr, 561 op/s 2024-08-02T18:01:59.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 890.0 s, 412.6 tps, lat 247.454 ms stddev 286.509 2024-08-02T18:01:59.972 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 -- 172.21.15.69:0/3278132260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1704102c20 msgr2=0x7f170410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 --2- 172.21.15.69:0/3278132260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1704102c20 0x7f170410d770 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009ca0 tx=0x7f16fc02a950 comp rx=0 tx=0).stop 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 -- 172.21.15.69:0/3278132260 shutdown_connections 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 --2- 172.21.15.69:0/3278132260 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f1704102c20 0x7f170410d770 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 --2- 172.21.15.69:0/3278132260 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17041022a0 0x7f17041026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 --2- 172.21.15.69:0/3278132260 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1704100b50 0x7f1704100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 -- 172.21.15.69:0/3278132260 >> 172.21.15.69:0/3278132260 conn(0x7f17040fc500 msgr2=0x7f17040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 -- 172.21.15.69:0/3278132260 shutdown_connections 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 -- 172.21.15.69:0/3278132260 wait complete. 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.588+0000 7f170957c640 1 Processor -- start 2024-08-02T18:01:59.973 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f170957c640 1 -- start start 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f170957c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1704100b50 0x7f17041981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f170957c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17041022a0 0x7f1704198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f170957c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1704102c20 0x7f170419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f170957c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1704110cb0 con 0x7f17041022a0 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f170957c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1704110b30 con 0x7f1704100b50 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f170957c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1704110e30 con 0x7f1704102c20 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17041022a0 0x7f1704198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17041022a0 0x7f1704198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52904/0 (socket says 172.21.15.69:52904) 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 -- 172.21.15.69:0/1999075780 learned_addr learned my addr 172.21.15.69:0/1999075780 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f1702ffd640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1704100b50 0x7f17041981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17037fe640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1704102c20 0x7f170419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 -- 172.21.15.69:0/1999075780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1704102c20 msgr2=0x7f170419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1704102c20 0x7f170419ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 -- 172.21.15.69:0/1999075780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1704100b50 msgr2=0x7f17041981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1704100b50 0x7f17041981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 -- 172.21.15.69:0/1999075780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170419d170 con 0x7f17041022a0 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.589+0000 7f17027fc640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17041022a0 0x7f1704198700 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e9f0 tx=0x7f16f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.590+0000 7f16ebfff640 1 -- 172.21.15.69:0/1999075780 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f800cd80 con 0x7f17041022a0 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.590+0000 7f170957c640 1 -- 172.21.15.69:0/1999075780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170419d460 con 0x7f17041022a0 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.590+0000 7f16ebfff640 1 -- 172.21.15.69:0/1999075780 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f8010910 con 0x7f17041022a0 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.590+0000 7f16ebfff640 1 -- 172.21.15.69:0/1999075780 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f80188f0 con 0x7f17041022a0 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.590+0000 7f170957c640 1 -- 172.21.15.69:0/1999075780 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041a4d40 con 0x7f17041022a0 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.592+0000 7f16ebfff640 1 -- 172.21.15.69:0/1999075780 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f16f8010ac0 con 0x7f17041022a0 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.592+0000 7f170b97a740 1 Processor -- start 2024-08-02T18:01:59.975 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f16ebfff640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f16ec0a2d70 0x7f16ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f170b97a740 1 -- start start 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f16ebfff640 1 -- 172.21.15.69:0/1999075780 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f16f80cfc50 con 0x7f17041022a0 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f17037fe640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f16ec0a2d70 0x7f16ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f170b97a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8882ff0 0x5569a88833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f170b97a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a888dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f170b97a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 0x5569a88905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f170b97a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5569a87e8a40 con 0x5569a888e1d0 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f170b97a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5569a87809d0 con 0x5569a8882ff0 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f170b97a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569a87e4a00 con 0x5569a8883910 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8882ff0 0x5569a88833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.976 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8882ff0 0x5569a88833d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35438/0 (socket says 172.21.15.69:35438) 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f1702ffd640 1 -- 172.21.15.69:0/3756850693 learned_addr learned my addr 172.21.15.69:0/3756850693 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f17027fc640 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a888dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.593+0000 7f17037fe640 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 0x5569a88905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.594+0000 7f17037fe640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f16ec0a2d70 0x7f16ec0a5230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005e30 tx=0x7f16fc02fa10 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.594+0000 7f17037fe640 1 -- 172.21.15.69:0/3756850693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 msgr2=0x5569a888dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.594+0000 7f17037fe640 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a888dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.594+0000 7f17037fe640 1 -- 172.21.15.69:0/3756850693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8882ff0 msgr2=0x5569a88833d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.594+0000 7f17037fe640 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8882ff0 0x5569a88833d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.594+0000 7f17037fe640 1 -- 172.21.15.69:0/3756850693 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a8787650 con 0x5569a888e1d0 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.595+0000 7f17037fe640 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 0x5569a88905c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005cc0 tx=0x7f16fc048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d0e3a25edfb384 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:59.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.597+0000 7f16e97fa640 1 -- 172.21.15.69:0/3756850693 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc034040 con 0x5569a888e1d0 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.597+0000 7f16e97fa640 1 -- 172.21.15.69:0/3756850693 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc044040 con 0x5569a888e1d0 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f16e97fa640 1 -- 172.21.15.69:0/3756850693 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0306a0 con 0x5569a888e1d0 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 -- 172.21.15.69:0/3756850693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 msgr2=0x5569a88905c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 0x5569a88905c0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005cc0 tx=0x7f16fc048000 comp rx=0 tx=0).stop 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 -- 172.21.15.69:0/3756850693 shutdown_connections 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 0x5569a88905c0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a888dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 --2- 172.21.15.69:0/3756850693 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8882ff0 0x5569a88833d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 -- 172.21.15.69:0/3756850693 >> 172.21.15.69:0/3756850693 conn(0x5569a887f580 msgr2=0x5569a887f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 -- 172.21.15.69:0/3756850693 shutdown_connections 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.598+0000 7f170b97a740 1 -- 172.21.15.69:0/3756850693 wait complete. 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.599+0000 7f170b97a740 1 -- start start 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.599+0000 7f170b97a740 1 Processor -- start 2024-08-02T18:01:59.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.599+0000 7f170b97a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8882ff0 0x5569a88872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.599+0000 7f170b97a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8883910 0x5569a8887820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.599+0000 7f170b97a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 0x5569a8887d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.599+0000 7f170b97a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5569a8890d90 con 0x5569a888e1d0 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.599+0000 7f170b97a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5569a87a3b20 con 0x5569a8883910 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.599+0000 7f170b97a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5569a8890f10 con 0x5569a8882ff0 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8882ff0 0x5569a88872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8882ff0 0x5569a88872e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35558/0 (socket says 172.21.15.69:35558) 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 -- 172.21.15.69:0/4270956343 learned_addr learned my addr 172.21.15.69:0/4270956343 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f17027fc640 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8883910 0x5569a8887820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f170b97a740 10 client.0 _ll_register_callbacks cb 0x5569a87a1bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 -- 172.21.15.69:0/4270956343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8883910 msgr2=0x5569a8887820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8883910 0x5569a8887820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 -- 172.21.15.69:0/4270956343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 msgr2=0x5569a8887d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:59.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 0x5569a8887d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 -- 172.21.15.69:0/4270956343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a855a3c0 con 0x5569a8882ff0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f1702ffd640 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8882ff0 0x5569a88872e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f16f40136c0 tx=0x7f16f4013b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf549596b702a629 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f16e6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.600+0000 7f16e6ff5640 1 -- 172.21.15.69:0/4270956343 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4020060 con 0x5569a8882ff0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.601+0000 7f170b97a740 1 -- 172.21.15.69:0/4270956343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16fc044040 con 0x5569a8882ff0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.601+0000 7f16e6ff5640 1 -- 172.21.15.69:0/4270956343 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f16f401b430 con 0x5569a8882ff0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.601+0000 7f16e6ff5640 1 -- 172.21.15.69:0/4270956343 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16f4027050 con 0x5569a8882ff0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.601+0000 7f170b97a740 10 client.28387 fetch_fsmap learned FSMap version 28 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.601+0000 7f170b97a740 1 -- 172.21.15.69:0/4270956343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5569a889ca80 con 0x5569a8882ff0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.601+0000 7f16e6ff5640 1 -- 172.21.15.69:0/4270956343 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc044040 con 0x5569a8882ff0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.601+0000 7f16e6ff5640 1 -- 172.21.15.69:0/4270956343 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f16f400f690 con 0x5569a8882ff0 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.602+0000 7f170b97a740 10 client.28387 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:01:59.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.602+0000 7f170b97a740 10 client.28387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.602+0000 7f170b97a740 20 client.28387 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.602+0000 7f170b97a740 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5569a889de90 0x5569a88a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.602+0000 7f170b97a740 4 client.28387 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.602+0000 7f170b97a740 1 -- 172.21.15.69:0/4270956343 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569a87f5fb0 con 0x5569a889de90 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.603+0000 7f17027fc640 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5569a889de90 0x5569a88a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.603+0000 7f17027fc640 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5569a889de90 0x5569a88a02e0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e73c557ab7384bd server_cookie=b6a1beb18f5a04a in_seq=0 out_seq=0 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.603+0000 7f16e6ff5640 10 client.28387 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f16e6ff5640 1 -- 172.21.15.69:0/4270956343 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569a87f5fb0 con 0x5569a889de90 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f16e6ff5640 10 client.28387 handle_command_reply: tid=0 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 client.28387 shutdown 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 -- 172.21.15.69:0/4270956343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8882ff0 msgr2=0x5569a88872e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8882ff0 0x5569a88872e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f16f40136c0 tx=0x7f16f4013b80 comp rx=0 tx=0).stop 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 -- 172.21.15.69:0/4270956343 shutdown_connections 2024-08-02T18:01:59.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a888e1d0 0x5569a8887d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5569a889de90 0x5569a88a02e0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f16e6ff5640 0 client.28387 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a8883910 0x5569a8887820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 --2- 172.21.15.69:0/4270956343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8882ff0 0x5569a88872e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 -- 172.21.15.69:0/4270956343 >> 172.21.15.69:0/4270956343 conn(0x5569a887f580 msgr2=0x5569a8885260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.604+0000 7f170b97a740 1 -- 172.21.15.69:0/4270956343 shutdown_connections 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.608+0000 7f170b97a740 1 -- 172.21.15.69:0/4270956343 wait complete. 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.608+0000 7f170b97a740 20 client.28387 trim_cache size 0 max 16384 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.632+0000 7f170b97a740 1 Processor -- start 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.633+0000 7f170b97a740 1 -- start start 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.633+0000 7f170b97a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a8887bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.633+0000 7f170b97a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a887e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.633+0000 7f170b97a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a888e1d0 0x5569a88a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.633+0000 7f170b97a740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17041af590 con 0x5569a887f580 2024-08-02T18:01:59.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.633+0000 7f170b97a740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17041ace60 con 0x5569a888e1d0 2024-08-02T18:01:59.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.633+0000 7f170b97a740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17041aacb0 con 0x5569a8883910 2024-08-02T18:01:59.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.633+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a8887bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.634+0000 7f1702ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a8887bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52936/0 (socket says 172.21.15.69:52936) 2024-08-02T18:01:59.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.634+0000 7f1702ffd640 1 -- 172.21.15.69:0/1312143781 learned_addr learned my addr 172.21.15.69:0/1312143781 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:59.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.634+0000 7f1702ffd640 1 -- 172.21.15.69:0/1312143781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 msgr2=0x5569a887e4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:01:59.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.634+0000 7f1702ffd640 1 --2- 172.21.15.69:0/1312143781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a887e4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.634+0000 7f1702ffd640 1 -- 172.21.15.69:0/1312143781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a888e1d0 msgr2=0x5569a88a5bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:01:59.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.634+0000 7f1702ffd640 1 --2- 172.21.15.69:0/1312143781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a888e1d0 0x5569a88a5bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.634+0000 7f1702ffd640 1 -- 172.21.15.69:0/1312143781 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569a88913a0 con 0x5569a887f580 2024-08-02T18:01:59.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.635+0000 7f1702ffd640 1 --2- 172.21.15.69:0/1312143781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a8887bb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f16f40058c0 tx=0x7f16f4009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50afbc31c01b55d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:59.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.635+0000 7f16e8ff9640 1 -- 172.21.15.69:0/1312143781 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f402f070 con 0x5569a887f580 2024-08-02T18:01:59.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.635+0000 7f16e8ff9640 1 -- 172.21.15.69:0/1312143781 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f4011c90 con 0x5569a887f580 2024-08-02T18:01:59.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.635+0000 7f16e8ff9640 1 -- 172.21.15.69:0/1312143781 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40035f0 con 0x5569a887f580 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.636+0000 7f170b97a740 1 -- 172.21.15.69:0/1312143781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 msgr2=0x5569a8887bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.636+0000 7f170b97a740 1 --2- 172.21.15.69:0/1312143781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a8887bb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f16f40058c0 tx=0x7f16f4009e50 comp rx=0 tx=0).stop 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.640+0000 7f170b97a740 1 -- 172.21.15.69:0/1312143781 shutdown_connections 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.640+0000 7f170b97a740 1 --2- 172.21.15.69:0/1312143781 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a888e1d0 0x5569a88a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.640+0000 7f170b97a740 1 --2- 172.21.15.69:0/1312143781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a887e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.640+0000 7f170b97a740 1 --2- 172.21.15.69:0/1312143781 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a8887bb0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.640+0000 7f170b97a740 1 -- 172.21.15.69:0/1312143781 >> 172.21.15.69:0/1312143781 conn(0x5569a8891fb0 msgr2=0x5569a887dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.640+0000 7f170b97a740 1 -- 172.21.15.69:0/1312143781 shutdown_connections 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.645+0000 7f170b97a740 1 -- 172.21.15.69:0/1312143781 wait complete. 2024-08-02T18:01:59.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.645+0000 7f170b97a740 1 -- start start 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.645+0000 7f170b97a740 1 Processor -- start 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.647+0000 7f170b97a740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a889a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.647+0000 7f170b97a740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a8895a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.647+0000 7f170b97a740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a888e1d0 0x5569a8895f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f17027fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a8895a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f17027fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a8895a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:35584/0 (socket says 172.21.15.69:35584) 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f17027fc640 1 -- 172.21.15.69:0/3930409642 learned_addr learned my addr 172.21.15.69:0/3930409642 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17041ab320 con 0x5569a887f580 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17041ac790 con 0x5569a888e1d0 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17041ab9f0 con 0x5569a8883910 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f170b97a740 10 client.0 _ll_register_callbacks cb 0x7f16f4009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f17027fc640 1 -- 172.21.15.69:0/3930409642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a888e1d0 msgr2=0x5569a8895f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:01:59.988 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f17027fc640 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a888e1d0 0x5569a8895f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f17027fc640 1 -- 172.21.15.69:0/3930409642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 msgr2=0x5569a889a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f17027fc640 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a889a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.648+0000 7f17027fc640 1 -- 172.21.15.69:0/3930409642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569a88965c0 con 0x5569a8883910 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.649+0000 7f17027fc640 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a8895a10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f16f809bd60 tx=0x7f16f809b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb00c2fd5206698 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.649+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f4011c90 con 0x5569a8883910 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.651+0000 7f16e6ff5640 10 client.28390 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.651+0000 7f16e6ff5640 1 -- 172.21.15.69:0/3930409642 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f80cd1d0 con 0x5569a8883910 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.651+0000 7f16e6ff5640 1 -- 172.21.15.69:0/3930409642 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f16f80cd370 con 0x5569a8883910 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.651+0000 7f16e6ff5640 1 -- 172.21.15.69:0/3930409642 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f16f80ba720 con 0x5569a8883910 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.652+0000 7f170b97a740 10 client.28390 fetch_fsmap learned FSMap version 28 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.652+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5569a8908920 con 0x5569a8883910 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.655+0000 7f16e6ff5640 1 -- 172.21.15.69:0/3930409642 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f16f80a4be0 con 0x5569a8883910 2024-08-02T18:01:59.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.655+0000 7f170b97a740 10 client.28390 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.655+0000 7f170b97a740 10 client.28390 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.655+0000 7f170b97a740 20 client.28390 populate_metadata read hostname 'smithi069' 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.655+0000 7f170b97a740 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5569a889de90 0x5569a890b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.655+0000 7f170b97a740 4 client.28390 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.655+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f16f4011c90 con 0x5569a889de90 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.657+0000 7f1702ffd640 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5569a889de90 0x5569a890b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.664+0000 7f1702ffd640 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5569a889de90 0x5569a890b040 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8d0d69f8a5ee43 server_cookie=45ac5aabd6394623 in_seq=0 out_seq=0 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.664+0000 7f16e6ff5640 10 client.28390 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.664+0000 7f16e6ff5640 1 -- 172.21.15.69:0/3930409642 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f16f4011c90 con 0x5569a889de90 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.664+0000 7f16e6ff5640 10 client.28390 handle_command_reply: tid=0 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.664+0000 7f170b97a740 1 client.28390 shutdown 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.670+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 msgr2=0x5569a8895a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.670+0000 7f170b97a740 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a8895a10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f16f809bd60 tx=0x7f16f809b730 comp rx=0 tx=0).stop 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.673+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 shutdown_connections 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.673+0000 7f170b97a740 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5569a889de90 0x5569a890b040 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.673+0000 7f170b97a740 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5569a888e1d0 0x5569a8895f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.673+0000 7f170b97a740 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5569a8883910 0x5569a8895a10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.673+0000 7f170b97a740 1 --2- 172.21.15.69:0/3930409642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5569a887f580 0x5569a889a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.673+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 >> 172.21.15.69:0/3930409642 conn(0x5569a8891fb0 msgr2=0x5569a887e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.673+0000 7f16e6ff5640 0 client.28390 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.678+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 shutdown_connections 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.682+0000 7f170b97a740 1 -- 172.21.15.69:0/3930409642 wait complete. 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.686+0000 7f170b97a740 20 client.28390 trim_cache size 0 max 16384 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.688+0000 7f170957c640 1 -- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f16ec0a2d70 msgr2=0x7f16ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.688+0000 7f170957c640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f16ec0a2d70 0x7f16ec0a5230 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f16fc005e30 tx=0x7f16fc02fa10 comp rx=0 tx=0).stop 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.688+0000 7f170957c640 1 -- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17041022a0 msgr2=0x7f1704198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:01:59.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.688+0000 7f170957c640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17041022a0 0x7f1704198700 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f16f800e9f0 tx=0x7f16f800eeb0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.690+0000 7f170957c640 1 -- 172.21.15.69:0/1999075780 shutdown_connections 2024-08-02T18:01:59.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.690+0000 7f170957c640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17041022a0 0x7f1704198700 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.690+0000 7f170957c640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f16ec0a2d70 0x7f16ec0a5230 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.690+0000 7f170957c640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1704102c20 0x7f170419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.690+0000 7f170957c640 1 --2- 172.21.15.69:0/1999075780 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f1704100b50 0x7f17041981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:01:59.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.690+0000 7f170957c640 1 -- 172.21.15.69:0/1999075780 >> 172.21.15.69:0/1999075780 conn(0x7f17040fc500 msgr2=0x7f1704069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:01:59.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.691+0000 7f170957c640 1 -- 172.21.15.69:0/1999075780 shutdown_connections 2024-08-02T18:01:59.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:01:58.691+0000 7f170957c640 1 -- 172.21.15.69:0/1999075780 wait complete. 2024-08-02T18:01:59.993 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:01:59 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:02:00.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:01:59 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:02:00.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:01:59 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:02:01.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:00 smithi083 ceph-mon[37031]: pgmap v488: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-08-02T18:02:01.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:00 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:02:01.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:00 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:02:01.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:00 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:02:01.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:00 smithi077 ceph-mon[37108]: pgmap v488: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-08-02T18:02:01.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:00 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:02:01.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:00 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:02:01.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:00 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:02:01.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:00 smithi069 ceph-mon[32746]: pgmap v488: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.0 MiB/s wr, 609 op/s 2024-08-02T18:02:01.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:00 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:02:01.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:00 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:02:01.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:00 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:02:01.802 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-02T18:02:01.803 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-02T18:02:02.287 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:02.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.966+0000 7f468ea44640 1 -- 172.21.15.69:0/3637715250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f468811ae70 msgr2=0x7f468811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:02.969 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.966+0000 7f468ea44640 1 --2- 172.21.15.69:0/3637715250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f468811ae70 0x7f468811d260 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f468001a910 tx=0x7f468003b470 comp rx=0 tx=0).stop 2024-08-02T18:02:02.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.967+0000 7f468ea44640 1 -- 172.21.15.69:0/3637715250 shutdown_connections 2024-08-02T18:02:02.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.967+0000 7f468ea44640 1 --2- 172.21.15.69:0/3637715250 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f468811ae70 0x7f468811d260 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:02.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.967+0000 7f468ea44640 1 --2- 172.21.15.69:0/3637715250 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688109560 0x7f46881099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:02.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.967+0000 7f468ea44640 1 --2- 172.21.15.69:0/3637715250 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f4688109180 0x7f468810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:02.970 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.967+0000 7f468ea44640 1 -- 172.21.15.69:0/3637715250 >> 172.21.15.69:0/3637715250 conn(0x7f468806de70 msgr2=0x7f468806e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:02.970 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:02 smithi069 ceph-mon[32746]: pgmap v489: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 543 op/s 2024-08-02T18:02:02.971 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.969+0000 7f468ea44640 1 -- 172.21.15.69:0/3637715250 shutdown_connections 2024-08-02T18:02:02.974 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.972+0000 7f468ea44640 1 -- 172.21.15.69:0/3637715250 wait complete. 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.973+0000 7f468ea44640 1 Processor -- start 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.973+0000 7f468ea44640 1 -- start start 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f468ea44640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688109180 0x7f468811a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f468ea44640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4688109560 0x7f46881165f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f468ea44640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468811ae70 0x7f4688116b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f468ea44640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4688113020 con 0x7f4688109560 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f468ea44640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f4688112ea0 con 0x7f468811ae70 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f468ea44640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46881131a0 con 0x7f4688109180 2024-08-02T18:02:02.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f46877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4688109560 0x7f46881165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:02.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f46877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4688109560 0x7f46881165f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52966/0 (socket says 172.21.15.69:52966) 2024-08-02T18:02:02.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f46877fe640 1 -- 172.21.15.69:0/2712744042 learned_addr learned my addr 172.21.15.69:0/2712744042 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:02.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.974+0000 7f468cfe1640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468811ae70 0x7f4688116b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:02.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.977+0000 7f46877fe640 1 -- 172.21.15.69:0/2712744042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688109180 msgr2=0x7f468811a550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-02T18:02:02.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.977+0000 7f46877fe640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688109180 0x7f468811a550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:02.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.977+0000 7f46877fe640 1 -- 172.21.15.69:0/2712744042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468811ae70 msgr2=0x7f4688116b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:02.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.977+0000 7f46877fe640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468811ae70 0x7f4688116b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:02.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.977+0000 7f46877fe640 1 -- 172.21.15.69:0/2712744042 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4688117260 con 0x7f4688109560 2024-08-02T18:02:02.987 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.986+0000 7f46877fe640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4688109560 0x7f46881165f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4670030a00 tx=0x7f4670030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.987+0000 7f46857fa640 1 -- 172.21.15.69:0/2712744042 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467002ed90 con 0x7f4688109560 2024-08-02T18:02:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.988+0000 7f468ea44640 1 -- 172.21.15.69:0/2712744042 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4688074150 con 0x7f4688109560 2024-08-02T18:02:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.988+0000 7f468ea44640 1 -- 172.21.15.69:0/2712744042 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4688074690 con 0x7f4688109560 2024-08-02T18:02:02.989 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.988+0000 7f46857fa640 1 -- 172.21.15.69:0/2712744042 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4670032920 con 0x7f4688109560 2024-08-02T18:02:02.990 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.988+0000 7f46857fa640 1 -- 172.21.15.69:0/2712744042 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467003a990 con 0x7f4688109560 2024-08-02T18:02:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.992+0000 7f4690e42740 1 Processor -- start 2024-08-02T18:02:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f4690e42740 1 -- start start 2024-08-02T18:02:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f4690e42740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3d200 0x55f374c3d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f4690e42740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 0x55f374c47ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f4690e42740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 0x55f374c4a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f4690e42740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f374c00e40 con 0x55f374c3d200 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f4690e42740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f374ba27f0 con 0x55f374c483e0 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f4690e42740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f374b49fb0 con 0x55f374c3db20 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f468cfe1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3d200 0x55f374c3d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f468cfe1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3d200 0x55f374c3d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52970/0 (socket says 172.21.15.69:52970) 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.993+0000 7f468cfe1640 1 -- 172.21.15.69:0/1423722920 learned_addr learned my addr 172.21.15.69:0/1423722920 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.994+0000 7f468cfe1640 1 -- 172.21.15.69:0/1423722920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 msgr2=0x55f374c47ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.994+0000 7f468cfe1640 1 --2- 172.21.15.69:0/1423722920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 0x55f374c47ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.994+0000 7f468cfe1640 1 -- 172.21.15.69:0/1423722920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 msgr2=0x55f374c4a7d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.994+0000 7f468cfe1640 1 --2- 172.21.15.69:0/1423722920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 0x55f374c4a7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.994+0000 7f468cfe1640 1 -- 172.21.15.69:0/1423722920 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3749146a0 con 0x55f374c3d200 2024-08-02T18:02:03.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.995+0000 7f46857fa640 1 -- 172.21.15.69:0/2712744042 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f467003ab30 con 0x7f4688109560 2024-08-02T18:02:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.996+0000 7f46857fa640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46740a2f00 0x7f46740a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.996+0000 7f46857fa640 1 -- 172.21.15.69:0/2712744042 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f46700f1b60 con 0x7f4688109560 2024-08-02T18:02:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.996+0000 7f4687fff640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46740a2f00 0x7f46740a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.003 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:02.997+0000 7f4687fff640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46740a2f00 0x7f46740a53c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55f374c3a5f0 tx=0x7f467802b230 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:03.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.001+0000 7f468cfe1640 1 --2- 172.21.15.69:0/1423722920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3d200 0x55f374c3d5e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4680003680 tx=0x7f46800036b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05b7dd54bd9624e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:03.004 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.001+0000 7f467e7fc640 1 -- 172.21.15.69:0/1423722920 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680011070 con 0x55f374c3d200 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.001+0000 7f467e7fc640 1 -- 172.21.15.69:0/1423722920 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680005400 con 0x55f374c3d200 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.001+0000 7f467e7fc640 1 -- 172.21.15.69:0/1423722920 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680040770 con 0x55f374c3d200 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.002+0000 7f4690e42740 1 -- 172.21.15.69:0/1423722920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3d200 msgr2=0x55f374c3d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.002+0000 7f4690e42740 1 --2- 172.21.15.69:0/1423722920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3d200 0x55f374c3d5e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4680003680 tx=0x7f46800036b0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.002+0000 7f4690e42740 1 -- 172.21.15.69:0/1423722920 shutdown_connections 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.002+0000 7f4690e42740 1 --2- 172.21.15.69:0/1423722920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 0x55f374c4a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.002+0000 7f4690e42740 1 --2- 172.21.15.69:0/1423722920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 0x55f374c47ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.002+0000 7f4690e42740 1 --2- 172.21.15.69:0/1423722920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3d200 0x55f374c3d5e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.002+0000 7f4690e42740 1 -- 172.21.15.69:0/1423722920 >> 172.21.15.69:0/1423722920 conn(0x55f374c39790 msgr2=0x55f374c39b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:03.005 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.003+0000 7f4690e42740 1 -- 172.21.15.69:0/1423722920 shutdown_connections 2024-08-02T18:02:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.003+0000 7f4690e42740 1 -- 172.21.15.69:0/1423722920 wait complete. 2024-08-02T18:02:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.004+0000 7f4690e42740 1 -- start start 2024-08-02T18:02:03.006 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.004+0000 7f4690e42740 1 Processor -- start 2024-08-02T18:02:03.008 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.005+0000 7f4690e42740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.005+0000 7f4690e42740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 0x55f374c45940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.005+0000 7f4690e42740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c483e0 0x55f374c45e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.005+0000 7f4690e42740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55f374b5dcb0 con 0x55f374c483e0 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.005+0000 7f4690e42740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55f374b9e7b0 con 0x55f374c3d200 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.005+0000 7f4690e42740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f374c4afb0 con 0x55f374c3db20 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.005+0000 7f4690e42740 10 client.0 _ll_register_callbacks cb 0x55f3748b1ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f4687fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f4687fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35474/0 (socket says 172.21.15.69:35474) 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f4687fff640 1 -- 172.21.15.69:0/1847231783 learned_addr learned my addr 172.21.15.69:0/1847231783 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:03.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f468cfe1640 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c483e0 0x55f374c45e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f46877fe640 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 0x55f374c45940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f4687fff640 1 -- 172.21.15.69:0/1847231783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 msgr2=0x55f374c45940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:03.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f4687fff640 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 0x55f374c45940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f4687fff640 1 -- 172.21.15.69:0/1847231783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c483e0 msgr2=0x55f374c45e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:03.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f4687fff640 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c483e0 0x55f374c45e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.006+0000 7f4687fff640 1 -- 172.21.15.69:0/1847231783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f374b5cbc0 con 0x55f374c3d200 2024-08-02T18:02:03.010 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.007+0000 7f4687fff640 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45400 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f467802fe00 tx=0x7f4678004af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf495a67d48366d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:03.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.009+0000 7f46677fe640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:02:03.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.009+0000 7f46677fe640 1 -- 172.21.15.69:0/1847231783 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467802edf0 con 0x55f374c3d200 2024-08-02T18:02:03.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.009+0000 7f4690e42740 1 -- 172.21.15.69:0/1847231783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4680005400 con 0x55f374c3d200 2024-08-02T18:02:03.011 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.009+0000 7f46677fe640 1 -- 172.21.15.69:0/1847231783 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f4678045480 con 0x55f374c3d200 2024-08-02T18:02:03.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.013+0000 7f46677fe640 1 -- 172.21.15.69:0/1847231783 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f467804e050 con 0x55f374c3d200 2024-08-02T18:02:03.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.013+0000 7f4690e42740 10 client.28640 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:03.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.013+0000 7f4690e42740 1 -- 172.21.15.69:0/1847231783 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f374c57080 con 0x55f374c3d200 2024-08-02T18:02:03.015 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.013+0000 7f46677fe640 1 -- 172.21.15.69:0/1847231783 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4680005400 con 0x55f374c3d200 2024-08-02T18:02:03.017 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.015+0000 7f46677fe640 1 -- 172.21.15.69:0/1847231783 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f467803b6b0 con 0x55f374c3d200 2024-08-02T18:02:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.015+0000 7f4690e42740 10 client.28640 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.015+0000 7f4690e42740 10 client.28640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.015+0000 7f4690e42740 20 client.28640 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.015+0000 7f4690e42740 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f374c58930 0x55f374c5ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.015+0000 7f4690e42740 4 client.28640 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.015+0000 7f4690e42740 1 -- 172.21.15.69:0/1847231783 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f374bafd60 con 0x55f374c58930 2024-08-02T18:02:03.018 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.015+0000 7f46877fe640 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f374c58930 0x55f374c5ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:02 smithi083 ceph-mon[37031]: pgmap v489: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 543 op/s 2024-08-02T18:02:03.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.019+0000 7f46877fe640 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f374c58930 0x55f374c5ad50 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c159ea24e86d49 server_cookie=72b12d8fd1899984 in_seq=0 out_seq=0 2024-08-02T18:02:03.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.020+0000 7f46677fe640 10 client.28640 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:03.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.025+0000 7f46677fe640 1 -- 172.21.15.69:0/1847231783 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f374bafd60 con 0x55f374c58930 2024-08-02T18:02:03.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.025+0000 7f46677fe640 10 client.28640 handle_command_reply: tid=0 2024-08-02T18:02:03.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.025+0000 7f4690e42740 1 client.28640 shutdown 2024-08-02T18:02:03.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 -- 172.21.15.69:0/1847231783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 msgr2=0x55f374c45400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:03.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45400 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f467802fe00 tx=0x7f4678004af0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 -- 172.21.15.69:0/1847231783 shutdown_connections 2024-08-02T18:02:03.030 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c483e0 0x55f374c45e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f374c58930 0x55f374c5ad50 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3db20 0x55f374c45940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 --2- 172.21.15.69:0/1847231783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45400 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 -- 172.21.15.69:0/1847231783 >> 172.21.15.69:0/1847231783 conn(0x55f374c39790 msgr2=0x55f374c3f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:03.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f46677fe640 0 client.28640 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:03.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 -- 172.21.15.69:0/1847231783 shutdown_connections 2024-08-02T18:02:03.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 1 -- 172.21.15.69:0/1847231783 wait complete. 2024-08-02T18:02:03.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.026+0000 7f4690e42740 20 client.28640 trim_cache size 0 max 16384 2024-08-02T18:02:03.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.061+0000 7f4690e42740 1 Processor -- start 2024-08-02T18:02:03.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.062+0000 7f4690e42740 1 -- start start 2024-08-02T18:02:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.062+0000 7f4690e42740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.062+0000 7f4690e42740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c60660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.062+0000 7f4690e42740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c483e0 0x55f374c60ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.062+0000 7f4690e42740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f4668012a10 con 0x55f374c3db20 2024-08-02T18:02:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.062+0000 7f4690e42740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f46680102e0 con 0x55f374c3d200 2024-08-02T18:02:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.062+0000 7f4690e42740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f466800e130 con 0x55f374c483e0 2024-08-02T18:02:03.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f46877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c60660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f46877fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c60660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:52994/0 (socket says 172.21.15.69:52994) 2024-08-02T18:02:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f4687fff640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f46877fe640 1 -- 172.21.15.69:0/2734570130 learned_addr learned my addr 172.21.15.69:0/2734570130 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f46877fe640 1 -- 172.21.15.69:0/2734570130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c483e0 msgr2=0x55f374c60ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f46877fe640 1 --2- 172.21.15.69:0/2734570130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c483e0 0x55f374c60ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f46877fe640 1 -- 172.21.15.69:0/2734570130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 msgr2=0x55f374c45ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f46877fe640 1 --2- 172.21.15.69:0/2734570130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.063+0000 7f46877fe640 1 -- 172.21.15.69:0/2734570130 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f374c443f0 con 0x55f374c3db20 2024-08-02T18:02:03.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.064+0000 7f46877fe640 1 --2- 172.21.15.69:0/2734570130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c60660 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f46700eeec0 tx=0x7f467003a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdac721c1240617d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:03.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.067+0000 7f467dffb640 1 -- 172.21.15.69:0/2734570130 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700db070 con 0x55f374c3db20 2024-08-02T18:02:03.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.067+0000 7f467dffb640 1 -- 172.21.15.69:0/2734570130 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46700c68a0 con 0x55f374c3db20 2024-08-02T18:02:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.067+0000 7f467dffb640 1 -- 172.21.15.69:0/2734570130 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46700bd080 con 0x55f374c3db20 2024-08-02T18:02:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.068+0000 7f4690e42740 1 -- 172.21.15.69:0/2734570130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 msgr2=0x55f374c60660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.068+0000 7f4690e42740 1 --2- 172.21.15.69:0/2734570130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c60660 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f46700eeec0 tx=0x7f467003a420 comp rx=0 tx=0).stop 2024-08-02T18:02:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.068+0000 7f4690e42740 1 -- 172.21.15.69:0/2734570130 shutdown_connections 2024-08-02T18:02:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.068+0000 7f4690e42740 1 --2- 172.21.15.69:0/2734570130 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c60660 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.068+0000 7f4690e42740 1 --2- 172.21.15.69:0/2734570130 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c483e0 0x55f374c60ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.068+0000 7f4690e42740 1 --2- 172.21.15.69:0/2734570130 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c3d200 0x55f374c45ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.068+0000 7f4690e42740 1 -- 172.21.15.69:0/2734570130 >> 172.21.15.69:0/2734570130 conn(0x55f374c39790 msgr2=0x55f374c47480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.070+0000 7f4690e42740 1 -- 172.21.15.69:0/2734570130 shutdown_connections 2024-08-02T18:02:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.070+0000 7f4690e42740 1 -- 172.21.15.69:0/2734570130 wait complete. 2024-08-02T18:02:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.070+0000 7f4690e42740 1 -- start start 2024-08-02T18:02:03.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.075+0000 7f4690e42740 1 Processor -- start 2024-08-02T18:02:03.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.075+0000 7f4690e42740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3d200 0x55f374c42510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.075+0000 7f4690e42740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c42a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.078 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.075+0000 7f4690e42740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 0x55f374c42f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.075+0000 7f4690e42740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f466800e7a0 con 0x55f374c3db20 2024-08-02T18:02:03.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.075+0000 7f4690e42740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f466800fc10 con 0x55f374c483e0 2024-08-02T18:02:03.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.075+0000 7f4690e42740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f466800ee70 con 0x55f374c3d200 2024-08-02T18:02:03.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.075+0000 7f4690e42740 10 client.0 _ll_register_callbacks cb 0x7f467802eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:03.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.076+0000 7f468cfe1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 0x55f374c42f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.076+0000 7f468cfe1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 0x55f374c42f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:35494/0 (socket says 172.21.15.69:35494) 2024-08-02T18:02:03.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.076+0000 7f468cfe1640 1 -- 172.21.15.69:0/3505451929 learned_addr learned my addr 172.21.15.69:0/3505451929 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:03.079 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.076+0000 7f4687fff640 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3d200 0x55f374c42510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.080 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.077+0000 7f4687fff640 1 -- 172.21.15.69:0/3505451929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 msgr2=0x55f374c42f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:03.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.077+0000 7f4687fff640 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 0x55f374c42f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.077+0000 7f4687fff640 1 -- 172.21.15.69:0/3505451929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 msgr2=0x55f374c42a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:03.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.077+0000 7f4687fff640 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c42a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.077+0000 7f4687fff640 1 -- 172.21.15.69:0/3505451929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f374c4ab40 con 0x55f374c3d200 2024-08-02T18:02:03.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.077+0000 7f4687fff640 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3d200 0x55f374c42510 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f467802e800 tx=0x7f467802ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f7863eb94068f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:03.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.078+0000 7f46677fe640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:02:03.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.078+0000 7f46677fe640 1 -- 172.21.15.69:0/3505451929 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678003e40 con 0x55f374c3d200 2024-08-02T18:02:03.081 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.078+0000 7f4690e42740 1 -- 172.21.15.69:0/3505451929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46700c68a0 con 0x55f374c3d200 2024-08-02T18:02:03.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.082+0000 7f46677fe640 1 -- 172.21.15.69:0/3505451929 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9568+0+0 (secure 0 0 0) 0x7f467803b3b0 con 0x55f374c3d200 2024-08-02T18:02:03.083 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.082+0000 7f46677fe640 1 -- 172.21.15.69:0/3505451929 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4678061050 con 0x55f374c3d200 2024-08-02T18:02:03.085 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.084+0000 7f4690e42740 10 client.28399 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:03.086 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.084+0000 7f4690e42740 1 -- 172.21.15.69:0/3505451929 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f374cc3830 con 0x55f374c3d200 2024-08-02T18:02:03.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.091+0000 7f46677fe640 1 -- 172.21.15.69:0/3505451929 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46700c68a0 con 0x55f374c3d200 2024-08-02T18:02:03.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.091+0000 7f46677fe640 1 -- 172.21.15.69:0/3505451929 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f4678051040 con 0x55f374c3d200 2024-08-02T18:02:03.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.091+0000 7f4690e42740 10 client.28399 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:03.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.091+0000 7f4690e42740 10 client.28399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:03.094 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.091+0000 7f4690e42740 20 client.28399 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:03.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.091+0000 7f4690e42740 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f374cc3fb0 0x55f374cc63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:03.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.091+0000 7f4690e42740 4 client.28399 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-02T18:02:03.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.091+0000 7f4690e42740 1 -- 172.21.15.69:0/3505451929 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f374baa270 con 0x55f374cc3fb0 2024-08-02T18:02:03.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:02 smithi077 ceph-mon[37108]: pgmap v489: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.5 MiB/s wr, 543 op/s 2024-08-02T18:02:03.099 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.094+0000 7f46877fe640 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f374cc3fb0 0x55f374cc63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.100+0000 7f46877fe640 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f374cc3fb0 0x55f374cc63d0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f86795cacf9bfe9e server_cookie=dd31058606029e1c in_seq=0 out_seq=0 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.101+0000 7f46677fe640 10 client.28399 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.101+0000 7f46677fe640 1 -- 172.21.15.69:0/3505451929 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f374baa270 con 0x55f374cc3fb0 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.101+0000 7f46677fe640 10 client.28399 handle_command_reply: tid=0 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.101+0000 7f4690e42740 1 client.28399 shutdown 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 -- 172.21.15.69:0/3505451929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3d200 msgr2=0x55f374c42510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3d200 0x55f374c42510 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f467802e800 tx=0x7f467802ee80 comp rx=0 tx=0).stop 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 -- 172.21.15.69:0/3505451929 shutdown_connections 2024-08-02T18:02:03.105 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55f374c3db20 0x55f374c42a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55f374cc3fb0 0x55f374cc63d0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55f374c483e0 0x55f374c42f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 --2- 172.21.15.69:0/3505451929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f374c3d200 0x55f374c42510 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 -- 172.21.15.69:0/3505451929 >> 172.21.15.69:0/3505451929 conn(0x55f374c39790 msgr2=0x55f374c3ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f46677fe640 0 client.28399 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 -- 172.21.15.69:0/3505451929 shutdown_connections 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 1 -- 172.21.15.69:0/3505451929 wait complete. 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.102+0000 7f4690e42740 20 client.28399 trim_cache size 0 max 16384 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:03.106 INFO:teuthology.orchestra.run.smithi069.stdout: "return_code": 0, 2024-08-02T18:02:03.107 INFO:teuthology.orchestra.run.smithi069.stdout: "scrub_tag": "58f623b4-5921-4b17-bbd8-520a0f3b45c1", 2024-08-02T18:02:03.107 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": "asynchronous" 2024-08-02T18:02:03.107 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:02:03.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.103+0000 7f468ea44640 1 -- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46740a2f00 msgr2=0x7f46740a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:03.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46740a2f00 0x7f46740a53c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55f374c3a5f0 tx=0x7f467802b230 comp rx=0 tx=0).stop 2024-08-02T18:02:03.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 -- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4688109560 msgr2=0x7f46881165f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:03.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4688109560 0x7f46881165f0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4670030a00 tx=0x7f4670030ec0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 -- 172.21.15.69:0/2712744042 shutdown_connections 2024-08-02T18:02:03.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f4688109560 0x7f46881165f0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.108 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f46740a2f00 0x7f46740a53c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f468811ae70 0x7f4688116b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 --2- 172.21.15.69:0/2712744042 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4688109180 0x7f468811a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:03.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 -- 172.21.15.69:0/2712744042 >> 172.21.15.69:0/2712744042 conn(0x7f468806de70 msgr2=0x7f4688070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:03.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 -- 172.21.15.69:0/2712744042 shutdown_connections 2024-08-02T18:02:03.109 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:03.104+0000 7f468ea44640 1 -- 172.21.15.69:0/2712744042 wait complete. 2024-08-02T18:02:03.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:03 smithi069 ceph-mon[32746]: scrub queued for path: / 2024-08-02T18:02:03.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:03 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:02:03.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:03 smithi069 ceph-mon[32746]: scrub summary: active paths [/] 2024-08-02T18:02:04.016 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T18:02:04.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:03 smithi083 ceph-mon[37031]: scrub queued for path: / 2024-08-02T18:02:04.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:03 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:02:04.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:03 smithi083 ceph-mon[37031]: scrub summary: active paths [/] 2024-08-02T18:02:04.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:03 smithi077 ceph-mon[37108]: scrub queued for path: / 2024-08-02T18:02:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:03 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:02:04.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:03 smithi077 ceph-mon[37108]: scrub summary: active paths [/] 2024-08-02T18:02:04.400 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:04.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:04 smithi069 ceph-mon[32746]: pgmap v490: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-08-02T18:02:04.859 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:04 smithi069 ceph-mon[32746]: scrub summary: idle+waiting paths [/] 2024-08-02T18:02:04.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 895.0 s, 414.8 tps, lat 227.884 ms stddev 293.799 2024-08-02T18:02:05.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.008+0000 7ffae755f640 1 -- 172.21.15.69:0/1519488135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100b10 msgr2=0x7ffae0104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.009 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.008+0000 7ffae755f640 1 --2- 172.21.15.69:0/1519488135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100b10 0x7ffae0104b60 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffad000da00 tx=0x7ffad002c4a0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.012+0000 7ffae755f640 1 -- 172.21.15.69:0/1519488135 shutdown_connections 2024-08-02T18:02:05.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.012+0000 7ffae755f640 1 --2- 172.21.15.69:0/1519488135 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffae0105290 0x7ffae010ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.012+0000 7ffae755f640 1 --2- 172.21.15.69:0/1519488135 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100b10 0x7ffae0104b60 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.012+0000 7ffae755f640 1 --2- 172.21.15.69:0/1519488135 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffae0100160 0x7ffae0100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.014 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.012+0000 7ffae755f640 1 -- 172.21.15.69:0/1519488135 >> 172.21.15.69:0/1519488135 conn(0x7ffae0076f60 msgr2=0x7ffae00fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:05.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:04 smithi083 ceph-mon[37031]: pgmap v490: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-08-02T18:02:05.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:04 smithi083 ceph-mon[37031]: scrub summary: idle+waiting paths [/] 2024-08-02T18:02:05.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.020+0000 7ffae755f640 1 -- 172.21.15.69:0/1519488135 shutdown_connections 2024-08-02T18:02:05.021 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.020+0000 7ffae755f640 1 -- 172.21.15.69:0/1519488135 wait complete. 2024-08-02T18:02:05.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.021+0000 7ffae755f640 1 Processor -- start 2024-08-02T18:02:05.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.021+0000 7ffae755f640 1 -- start start 2024-08-02T18:02:05.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.022+0000 7ffae755f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100160 0x7ffae0196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.022+0000 7ffae755f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffae0100b10 0x7ffae0196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.022+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100160 0x7ffae0196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.022+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100160 0x7ffae0196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55268/0 (socket says 172.21.15.69:55268) 2024-08-02T18:02:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.022+0000 7ffae52fb640 1 -- 172.21.15.69:0/3187134845 learned_addr learned my addr 172.21.15.69:0/3187134845 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.022+0000 7ffae4afa640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffae0100b10 0x7ffae0196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.022+0000 7ffae755f640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffae0105290 0x7ffae019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.022+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffae010fae0 con 0x7ffae0105290 2024-08-02T18:02:05.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.023+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffae010f960 con 0x7ffae0100160 2024-08-02T18:02:05.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.023+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffae010fc60 con 0x7ffae0100b10 2024-08-02T18:02:05.031 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.030+0000 7ffae5afc640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffae0105290 0x7ffae019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.030+0000 7ffae52fb640 1 -- 172.21.15.69:0/3187134845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffae0100b10 msgr2=0x7ffae0196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.030+0000 7ffae52fb640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffae0100b10 0x7ffae0196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.030+0000 7ffae52fb640 1 -- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffae0105290 msgr2=0x7ffae019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.030+0000 7ffae52fb640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffae0105290 0x7ffae019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.032 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.030+0000 7ffae52fb640 1 -- 172.21.15.69:0/3187134845 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae019afb0 con 0x7ffae0100160 2024-08-02T18:02:05.034 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.032+0000 7ffae52fb640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100160 0x7ffae0196000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00d950 tx=0x7ffadc00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:05.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.033+0000 7ffad67fc640 1 -- 172.21.15.69:0/3187134845 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc017030 con 0x7ffae0100160 2024-08-02T18:02:05.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.033+0000 7ffad67fc640 1 -- 172.21.15.69:0/3187134845 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffadc004020 con 0x7ffae0100160 2024-08-02T18:02:05.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.033+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae019b2a0 con 0x7ffae0100160 2024-08-02T18:02:05.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.033+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae01a2b80 con 0x7ffae0100160 2024-08-02T18:02:05.035 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.033+0000 7ffad67fc640 1 -- 172.21.15.69:0/3187134845 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0047a0 con 0x7ffae0100160 2024-08-02T18:02:05.039 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.038+0000 7ffad67fc640 1 -- 172.21.15.69:0/3187134845 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffadc004940 con 0x7ffae0100160 2024-08-02T18:02:05.040 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.039+0000 7ffae995d740 1 Processor -- start 2024-08-02T18:02:05.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.040+0000 7ffae995d740 1 -- start start 2024-08-02T18:02:05.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.040+0000 7ffae995d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.042 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.040+0000 7ffae995d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae995d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3d6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae995d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56157a331980 con 0x56157a3db450 2024-08-02T18:02:05.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae995d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56157a2c9a70 con 0x56157a3dbd70 2024-08-02T18:02:05.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae995d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56157a32d940 con 0x56157a3d6410 2024-08-02T18:02:05.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3d5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3d5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55272/0 (socket says 172.21.15.69:55272) 2024-08-02T18:02:05.043 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae52fb640 1 -- 172.21.15.69:0/334268268 learned_addr learned my addr 172.21.15.69:0/334268268 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:05.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae52fb640 1 -- 172.21.15.69:0/334268268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 msgr2=0x56157a3d6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:05.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae52fb640 1 --2- 172.21.15.69:0/334268268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3d6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae52fb640 1 -- 172.21.15.69:0/334268268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 msgr2=0x56157a3db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:05.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae52fb640 1 --2- 172.21.15.69:0/334268268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.041+0000 7ffae52fb640 1 -- 172.21.15.69:0/334268268 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157a32c6e0 con 0x56157a3dbd70 2024-08-02T18:02:05.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.042+0000 7ffae52fb640 1 --2- 172.21.15.69:0/334268268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3d5da0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0cd4f0 tx=0x7ffadc011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e7c94865009238 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:05.044 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.042+0000 7ffad67fc640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffacc0a2d70 0x7ffacc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.043+0000 7ffacb7fe640 1 -- 172.21.15.69:0/334268268 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0d32f0 con 0x56157a3dbd70 2024-08-02T18:02:05.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.043+0000 7ffacb7fe640 1 -- 172.21.15.69:0/334268268 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffadc0d3490 con 0x56157a3dbd70 2024-08-02T18:02:05.045 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.043+0000 7ffacb7fe640 1 -- 172.21.15.69:0/334268268 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0dfaa0 con 0x56157a3dbd70 2024-08-02T18:02:05.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.043+0000 7ffad67fc640 1 -- 172.21.15.69:0/3187134845 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9656+0+0 (secure 0 0 0) 0x7ffadc0d0330 con 0x7ffae0100160 2024-08-02T18:02:05.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.044+0000 7ffae995d740 1 -- 172.21.15.69:0/334268268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 msgr2=0x56157a3d5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.046 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.044+0000 7ffae995d740 1 --2- 172.21.15.69:0/334268268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3d5da0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ffadc0cd4f0 tx=0x7ffadc011000 comp rx=0 tx=0).stop 2024-08-02T18:02:05.049 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.047+0000 7ffae5afc640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffacc0a2d70 0x7ffacc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.050 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.048+0000 7ffae5afc640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffacc0a2d70 0x7ffacc0a5230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009ea0 tx=0x7ffad8009490 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.051+0000 7ffae995d740 1 -- 172.21.15.69:0/334268268 shutdown_connections 2024-08-02T18:02:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.051+0000 7ffae995d740 1 --2- 172.21.15.69:0/334268268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3d6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.051+0000 7ffae995d740 1 --2- 172.21.15.69:0/334268268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3d5da0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.051+0000 7ffae995d740 1 --2- 172.21.15.69:0/334268268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3db830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.051+0000 7ffae995d740 1 -- 172.21.15.69:0/334268268 >> 172.21.15.69:0/334268268 conn(0x56157a3c8250 msgr2=0x56157a3c8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:05.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.059+0000 7ffae995d740 1 -- 172.21.15.69:0/334268268 shutdown_connections 2024-08-02T18:02:05.060 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.059+0000 7ffae995d740 1 -- 172.21.15.69:0/334268268 wait complete. 2024-08-02T18:02:05.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.060+0000 7ffae995d740 1 -- start start 2024-08-02T18:02:05.061 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.060+0000 7ffae995d740 1 Processor -- start 2024-08-02T18:02:05.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae995d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.062 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae995d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3df340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45760/0 (socket says 172.21.15.69:45760) 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae52fb640 1 -- 172.21.15.69:0/3834681006 learned_addr learned my addr 172.21.15.69:0/3834681006 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae995d740 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3dfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56157a3e2790 con 0x56157a3db450 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56157a2ecaa0 con 0x56157a3dbd70 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.061+0000 7ffae5afc640 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3dfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.062+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56157a3e2910 con 0x56157a3d6410 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.062+0000 7ffae995d740 10 client.0 _ll_register_callbacks cb 0x56157a2dbe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.062+0000 7ffae52fb640 1 -- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 msgr2=0x56157a3dfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:05.064 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.062+0000 7ffae52fb640 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3dfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.062+0000 7ffae52fb640 1 -- 172.21.15.69:0/3834681006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 msgr2=0x56157a3df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:05.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.062+0000 7ffae52fb640 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.062+0000 7ffae52fb640 1 -- 172.21.15.69:0/3834681006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157a2d0650 con 0x56157a3d6410 2024-08-02T18:02:05.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.062+0000 7ffae52fb640 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3df340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009a50 tx=0x7ffadc005200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83ee8b6c344ee417 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:05.065 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.063+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffadc0d3490 con 0x56157a3d6410 2024-08-02T18:02:05.068 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.067+0000 7ffac8ff9640 10 client.28405 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:02:05.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.067+0000 7ffac8ff9640 1 -- 172.21.15.69:0/3834681006 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc098620 con 0x56157a3d6410 2024-08-02T18:02:05.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.067+0000 7ffac8ff9640 1 -- 172.21.15.69:0/3834681006 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9656+0+0 (secure 0 0 0) 0x7ffadc09a680 con 0x56157a3d6410 2024-08-02T18:02:05.069 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.067+0000 7ffac8ff9640 1 -- 172.21.15.69:0/3834681006 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffadc0d32f0 con 0x56157a3d6410 2024-08-02T18:02:05.072 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.071+0000 7ffae995d740 10 client.28405 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:05.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.071+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56157a0a33c0 con 0x56157a3d6410 2024-08-02T18:02:05.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.073+0000 7ffac8ff9640 1 -- 172.21.15.69:0/3834681006 <== mon.1 v2:172.21.15.83:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffadc0989f0 con 0x56157a3d6410 2024-08-02T18:02:05.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.073+0000 7ffae995d740 10 client.28405 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:05.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.073+0000 7ffae995d740 10 client.28405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:05.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.073+0000 7ffae995d740 20 client.28405 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:05.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.073+0000 7ffae995d740 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56157a3ef580 0x56157a3f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.073+0000 7ffae995d740 4 client.28405 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:05.075 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.073+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffadc0d3490 con 0x56157a3ef580 2024-08-02T18:02:05.076 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.074+0000 7ffae4afa640 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56157a3ef580 0x56157a3f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.084 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.083+0000 7ffae4afa640 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56157a3ef580 0x56157a3f1970 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9099154ca228ad7a server_cookie=2bc641c21af5f847 in_seq=0 out_seq=0 2024-08-02T18:02:05.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.083+0000 7ffac8ff9640 10 client.28405 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:05.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.086+0000 7ffac8ff9640 1 -- 172.21.15.69:0/3834681006 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ffadc0d3490 con 0x56157a3ef580 2024-08-02T18:02:05.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.086+0000 7ffac8ff9640 10 client.28405 handle_command_reply: tid=0 2024-08-02T18:02:05.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.087+0000 7ffae995d740 1 client.28405 shutdown 2024-08-02T18:02:05.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.090+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 msgr2=0x56157a3df340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.090+0000 7ffae995d740 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3df340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009a50 tx=0x7ffadc005200 comp rx=0 tx=0).stop 2024-08-02T18:02:05.092 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.091+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 shutdown_connections 2024-08-02T18:02:05.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.091+0000 7ffae995d740 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.091+0000 7ffae995d740 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56157a3ef580 0x56157a3f1970 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.091+0000 7ffae995d740 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3dfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.091+0000 7ffae995d740 1 --2- 172.21.15.69:0/3834681006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3df340 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.091+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 >> 172.21.15.69:0/3834681006 conn(0x56157a3c8250 msgr2=0x56157a3c7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:05.093 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.091+0000 7ffac8ff9640 0 client.28405 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:05.095 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.094+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 shutdown_connections 2024-08-02T18:02:05.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.095+0000 7ffae995d740 1 -- 172.21.15.69:0/3834681006 wait complete. 2024-08-02T18:02:05.096 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.095+0000 7ffae995d740 20 client.28405 trim_cache size 0 max 16384 2024-08-02T18:02:05.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:04 smithi077 ceph-mon[37108]: pgmap v490: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-08-02T18:02:05.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:04 smithi077 ceph-mon[37108]: scrub summary: idle+waiting paths [/] 2024-08-02T18:02:05.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.129+0000 7ffae995d740 1 Processor -- start 2024-08-02T18:02:05.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.129+0000 7ffae995d740 1 -- start start 2024-08-02T18:02:05.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.130+0000 7ffae995d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.131 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.130+0000 7ffae995d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3d5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.130+0000 7ffae995d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.132 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.130+0000 7ffae995d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffac4012b00 con 0x56157a3db450 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.130+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.130+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3e1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45774/0 (socket says 172.21.15.69:45774) 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.130+0000 7ffae52fb640 1 -- 172.21.15.69:0/3837527903 learned_addr learned my addr 172.21.15.69:0/3837527903 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.130+0000 7ffae5afc640 1 --2- 172.21.15.69:0/3837527903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.131+0000 7ffae995d740 1 -- 172.21.15.69:0/3837527903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffac40103d0 con 0x56157a3dbd70 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.131+0000 7ffae995d740 1 -- 172.21.15.69:0/3837527903 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffac400e240 con 0x56157a3d6410 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.131+0000 7ffae5afc640 1 -- 172.21.15.69:0/3837527903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 msgr2=0x56157a3e1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.131+0000 7ffae5afc640 1 --2- 172.21.15.69:0/3837527903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3e1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.133 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.131+0000 7ffae5afc640 1 -- 172.21.15.69:0/3837527903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 msgr2=0x56157a3d5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:05.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.131+0000 7ffae5afc640 1 --2- 172.21.15.69:0/3837527903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3d5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.131+0000 7ffae5afc640 1 -- 172.21.15.69:0/3837527903 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157a3dd890 con 0x56157a3dbd70 2024-08-02T18:02:05.134 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.132+0000 7ffae5afc640 1 --2- 172.21.15.69:0/3837527903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3eac00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffad8004b90 tx=0x7ffad8015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee0946a28dd758c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:05.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.134+0000 7ffacaffd640 1 -- 172.21.15.69:0/3837527903 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8023410 con 0x56157a3dbd70 2024-08-02T18:02:05.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.135+0000 7ffacaffd640 1 -- 172.21.15.69:0/3837527903 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad8028070 con 0x56157a3dbd70 2024-08-02T18:02:05.137 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.135+0000 7ffacaffd640 1 -- 172.21.15.69:0/3837527903 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad80218c0 con 0x56157a3dbd70 2024-08-02T18:02:05.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.136+0000 7ffae995d740 1 -- 172.21.15.69:0/3837527903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 msgr2=0x56157a3eac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.138 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.137+0000 7ffae995d740 1 --2- 172.21.15.69:0/3837527903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3eac00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ffad8004b90 tx=0x7ffad8015000 comp rx=0 tx=0).stop 2024-08-02T18:02:05.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.138+0000 7ffae995d740 1 -- 172.21.15.69:0/3837527903 shutdown_connections 2024-08-02T18:02:05.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.138+0000 7ffae995d740 1 --2- 172.21.15.69:0/3837527903 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3d5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.138+0000 7ffae995d740 1 --2- 172.21.15.69:0/3837527903 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3dbd70 0x56157a3eac00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.138+0000 7ffae995d740 1 --2- 172.21.15.69:0/3837527903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3d6410 0x56157a3e1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.140 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.138+0000 7ffae995d740 1 -- 172.21.15.69:0/3837527903 >> 172.21.15.69:0/3837527903 conn(0x56157a3c8250 msgr2=0x56157a3de430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:05.146 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.145+0000 7ffae995d740 1 -- 172.21.15.69:0/3837527903 shutdown_connections 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.145+0000 7ffae995d740 1 -- 172.21.15.69:0/3837527903 wait complete. 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.146+0000 7ffae995d740 1 -- start start 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.146+0000 7ffae995d740 1 Processor -- start 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae995d740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3d6410 0x56157a3f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae995d740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae995d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3dbd70 0x56157a3f3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae995d740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffac400e890 con 0x56157a3db450 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae995d740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffac400fd00 con 0x56157a3d6410 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae995d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffac400ef60 con 0x56157a3dbd70 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae995d740 10 client.0 _ll_register_callbacks cb 0x7ffadc0d3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:05.150 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3d6410 0x56157a3f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae52fb640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3d6410 0x56157a3f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55304/0 (socket says 172.21.15.69:55304) 2024-08-02T18:02:05.151 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.147+0000 7ffae52fb640 1 -- 172.21.15.69:0/2967642738 learned_addr learned my addr 172.21.15.69:0/2967642738 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:05.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.148+0000 7ffae5afc640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3dbd70 0x56157a3f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.152 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.150+0000 7ffae4afa640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.153+0000 7ffae5afc640 1 -- 172.21.15.69:0/2967642738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3d6410 msgr2=0x56157a3f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:05.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.154+0000 7ffae5afc640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3d6410 0x56157a3f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.154+0000 7ffae5afc640 1 -- 172.21.15.69:0/2967642738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 msgr2=0x56157a3f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:05.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.154+0000 7ffae5afc640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.156 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.154+0000 7ffae5afc640 1 -- 172.21.15.69:0/2967642738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157a46a120 con 0x56157a3dbd70 2024-08-02T18:02:05.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.154+0000 7ffae4afa640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:05.157 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.154+0000 7ffae52fb640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3d6410 0x56157a3f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:05.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.159+0000 7ffae5afc640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3dbd70 0x56157a3f3030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffad8013040 tx=0x7ffad8021cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d97909f94d1afe2f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:05.162 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.160+0000 7ffac8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:02:05.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.165+0000 7ffac8ff9640 1 -- 172.21.15.69:0/2967642738 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8023610 con 0x56157a3dbd70 2024-08-02T18:02:05.166 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.165+0000 7ffac8ff9640 1 -- 172.21.15.69:0/2967642738 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9656+0+0 (secure 0 0 0) 0x7ffad80237b0 con 0x56157a3dbd70 2024-08-02T18:02:05.169 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.168+0000 7ffae995d740 1 -- 172.21.15.69:0/2967642738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad8028070 con 0x56157a3dbd70 2024-08-02T18:02:05.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.169+0000 7ffac8ff9640 1 -- 172.21.15.69:0/2967642738 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffad803c050 con 0x56157a3dbd70 2024-08-02T18:02:05.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.170+0000 7ffae995d740 10 client.28411 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:05.171 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.170+0000 7ffae995d740 1 -- 172.21.15.69:0/2967642738 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56157a46b430 con 0x56157a3dbd70 2024-08-02T18:02:05.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.174+0000 7ffac8ff9640 1 -- 172.21.15.69:0/2967642738 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad8028070 con 0x56157a3dbd70 2024-08-02T18:02:05.175 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.174+0000 7ffac8ff9640 1 -- 172.21.15.69:0/2967642738 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffad80378b0 con 0x56157a3dbd70 2024-08-02T18:02:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.174+0000 7ffae995d740 10 client.28411 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.174+0000 7ffae995d740 10 client.28411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.174+0000 7ffae995d740 20 client.28411 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.174+0000 7ffae995d740 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56157a46ba50 0x56157a46de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:05.176 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.174+0000 7ffae995d740 4 client.28411 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T18:02:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.174+0000 7ffae995d740 1 -- 172.21.15.69:0/2967642738 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56157a33eef0 con 0x56157a46ba50 2024-08-02T18:02:05.177 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.175+0000 7ffae52fb640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56157a46ba50 0x56157a46de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:05.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.181+0000 7ffac8ff9640 10 client.28411 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:05.183 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.181+0000 7ffae52fb640 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56157a46ba50 0x56157a46de40 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c7d63eda66374bd server_cookie=52b54429aa484bee in_seq=0 out_seq=0 2024-08-02T18:02:05.187 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffac8ff9640 1 -- 172.21.15.69:0/2967642738 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56157a33eef0 con 0x56157a46ba50 2024-08-02T18:02:05.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffac8ff9640 10 client.28411 handle_command_reply: tid=0 2024-08-02T18:02:05.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 client.28411 shutdown 2024-08-02T18:02:05.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 -- 172.21.15.69:0/2967642738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3dbd70 msgr2=0x56157a3f3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.188 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3dbd70 0x56157a3f3030 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffad8013040 tx=0x7ffad8021cc0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 -- 172.21.15.69:0/2967642738 shutdown_connections 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56157a3db450 0x56157a3f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56157a46ba50 0x56157a46de40 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56157a3dbd70 0x56157a3f3030 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 --2- 172.21.15.69:0/2967642738 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56157a3d6410 0x56157a3f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffae995d740 1 -- 172.21.15.69:0/2967642738 >> 172.21.15.69:0/2967642738 conn(0x56157a3c8250 msgr2=0x56157a3c70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.186+0000 7ffac8ff9640 0 client.28411 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.187+0000 7ffae995d740 1 -- 172.21.15.69:0/2967642738 shutdown_connections 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.187+0000 7ffae995d740 1 -- 172.21.15.69:0/2967642738 wait complete. 2024-08-02T18:02:05.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.187+0000 7ffae995d740 20 client.28411 trim_cache size 0 max 16384 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": { 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "58f623b4-5921-4b17-bbd8-520a0f3b45c1": { 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "path": "/", 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "tag": "58f623b4-5921-4b17-bbd8-520a0f3b45c1", 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: "options": "recursive,force" 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:05.190 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffacc0a2d70 msgr2=0x7ffacc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffacc0a2d70 0x7ffacc0a5230 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009ea0 tx=0x7ffad8009490 comp rx=0 tx=0).stop 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100160 msgr2=0x7ffae0196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100160 0x7ffae0196000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00d950 tx=0x7ffadc00de10 comp rx=0 tx=0).stop 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 shutdown_connections 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffae0105290 0x7ffae019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffacc0a2d70 0x7ffacc0a5230 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffae0100b10 0x7ffae0196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 --2- 172.21.15.69:0/3187134845 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffae0100160 0x7ffae0196000 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 >> 172.21.15.69:0/3187134845 conn(0x7ffae0076f60 msgr2=0x7ffae0103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:05.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 shutdown_connections 2024-08-02T18:02:05.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:05.188+0000 7ffae755f640 1 -- 172.21.15.69:0/3187134845 wait complete. 2024-08-02T18:02:06.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:05 smithi083 ceph-mon[37031]: osdmap e114: 12 total, 12 up, 12 in 2024-08-02T18:02:06.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:05 smithi077 ceph-mon[37108]: osdmap e114: 12 total, 12 up, 12 in 2024-08-02T18:02:06.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:05 smithi069 ceph-mon[32746]: osdmap e114: 12 total, 12 up, 12 in 2024-08-02T18:02:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.124+0000 7f82d1ffb640 1 -- 172.21.15.69:0/3704028195 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c40319e0 con 0x7f82d41006f0 2024-08-02T18:02:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.125+0000 7f82d936c640 1 -- 172.21.15.69:0/3704028195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d41006f0 msgr2=0x7f82d4100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.125+0000 7f82d936c640 1 --2- 172.21.15.69:0/3704028195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d41006f0 0x7f82d4100b50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009930 tx=0x7f82c402a480 comp rx=0 tx=0).stop 2024-08-02T18:02:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.125+0000 7f82d936c640 1 -- 172.21.15.69:0/3704028195 shutdown_connections 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.125+0000 7f82d936c640 1 --2- 172.21.15.69:0/3704028195 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82d4101090 0x7f82d410d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.125+0000 7f82d936c640 1 --2- 172.21.15.69:0/3704028195 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d41006f0 0x7f82d4100b50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.125+0000 7f82d936c640 1 --2- 172.21.15.69:0/3704028195 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82d4106700 0x7f82d4106ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.125+0000 7f82d936c640 1 -- 172.21.15.69:0/3704028195 >> 172.21.15.69:0/3704028195 conn(0x7f82d40fc430 msgr2=0x7f82d40fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.126+0000 7f82d936c640 1 -- 172.21.15.69:0/3704028195 shutdown_connections 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.126+0000 7f82d936c640 1 -- 172.21.15.69:0/3704028195 wait complete. 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.127+0000 7f82d936c640 1 Processor -- start 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.127+0000 7f82d936c640 1 -- start start 2024-08-02T18:02:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d936c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82d41006f0 0x7f82d4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d936c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82d4101090 0x7f82d4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d936c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d4106700 0x7f82d419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d936c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4110c10 con 0x7f82d41006f0 2024-08-02T18:02:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82d41006f0 0x7f82d4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82d4101090 0x7f82d4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82d41006f0 0x7f82d4197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44230/0 (socket says 172.21.15.69:44230) 2024-08-02T18:02:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82d4101090 0x7f82d4197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55308/0 (socket says 172.21.15.69:55308) 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d2ffd640 1 -- 172.21.15.69:0/1315929043 learned_addr learned my addr 172.21.15.69:0/1315929043 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d37fe640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d4106700 0x7f82d419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4110a90 con 0x7f82d4101090 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.128+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82d4110d90 con 0x7f82d4106700 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.129+0000 7f82d37fe640 1 -- 172.21.15.69:0/1315929043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82d4101090 msgr2=0x7f82d4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.129+0000 7f82d37fe640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82d4101090 0x7f82d4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.129+0000 7f82d37fe640 1 -- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82d41006f0 msgr2=0x7f82d4197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.129+0000 7f82d37fe640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82d41006f0 0x7f82d4197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.129+0000 7f82d37fe640 1 -- 172.21.15.69:0/1315929043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d419c200 con 0x7f82d4106700 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.129+0000 7f82d2ffd640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82d41006f0 0x7f82d4197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.129+0000 7f82d27fc640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82d4101090 0x7f82d4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.130+0000 7f82d37fe640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d4106700 0x7f82d419bad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00be20 tx=0x7f82cc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.130+0000 7f82bbfff640 1 -- 172.21.15.69:0/1315929043 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc019070 con 0x7f82d4106700 2024-08-02T18:02:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.130+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d419c4f0 con 0x7f82d4106700 2024-08-02T18:02:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.130+0000 7f82bbfff640 1 -- 172.21.15.69:0/1315929043 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82cc014920 con 0x7f82d4106700 2024-08-02T18:02:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.130+0000 7f82bbfff640 1 -- 172.21.15.69:0/1315929043 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82cc019070 con 0x7f82d4106700 2024-08-02T18:02:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.130+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d41a3d80 con 0x7f82d4106700 2024-08-02T18:02:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.134+0000 7f82dc19e740 1 Processor -- start 2024-08-02T18:02:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.135+0000 7f82bbfff640 1 -- 172.21.15.69:0/1315929043 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f82cc01c070 con 0x7f82d4106700 2024-08-02T18:02:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.136+0000 7f82bbfff640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f82bc0a30f0 0x7f82bc0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.136+0000 7f82bbfff640 1 -- 172.21.15.69:0/1315929043 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9656+0+0 (secure 0 0 0) 0x7f82cc00cc70 con 0x7f82d4106700 2024-08-02T18:02:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.136+0000 7f82d27fc640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f82bc0a30f0 0x7f82bc0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82dc19e740 1 -- start start 2024-08-02T18:02:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa63420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa5ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa63420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa63420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44234/0 (socket says 172.21.15.69:44234) 2024-08-02T18:02:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa6e210 0x55854fa70600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82dc19e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55854f9be9b0 con 0x55854fa5a5f0 2024-08-02T18:02:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82dc19e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55854f966800 con 0x55854fa5a9d0 2024-08-02T18:02:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82dc19e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55854f9ba970 con 0x55854fa6e210 2024-08-02T18:02:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.137+0000 7f82d2ffd640 1 -- 172.21.15.69:0/423393816 learned_addr learned my addr 172.21.15.69:0/423393816 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d27fc640 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa5ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d37fe640 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa6e210 0x55854fa70600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d27fc640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f82bc0a30f0 0x7f82bc0a55b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f82c4000c00 tx=0x7f82c4004580 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d27fc640 1 -- 172.21.15.69:0/423393816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa6e210 msgr2=0x55854fa70600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d27fc640 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa6e210 0x55854fa70600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d27fc640 1 -- 172.21.15.69:0/423393816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 msgr2=0x55854fa63420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d27fc640 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa63420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d27fc640 1 -- 172.21.15.69:0/423393816 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55854f6d2210 con 0x55854fa5a9d0 2024-08-02T18:02:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.138+0000 7f82d37fe640 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa6e210 0x55854fa70600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.139+0000 7f82d27fc640 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa5ae50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c4047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f0e974471a12f67 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.139+0000 7f82b97fa640 1 -- 172.21.15.69:0/423393816 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c404e070 con 0x55854fa5a9d0 2024-08-02T18:02:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.139+0000 7f82b97fa640 1 -- 172.21.15.69:0/423393816 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c4031cc0 con 0x55854fa5a9d0 2024-08-02T18:02:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.139+0000 7f82b97fa640 1 -- 172.21.15.69:0/423393816 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4035040 con 0x55854fa5a9d0 2024-08-02T18:02:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.140+0000 7f82dc19e740 1 -- 172.21.15.69:0/423393816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 msgr2=0x55854fa5ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.140+0000 7f82dc19e740 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa5ae50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c4047000 comp rx=0 tx=0).stop 2024-08-02T18:02:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.140+0000 7f82dc19e740 1 -- 172.21.15.69:0/423393816 shutdown_connections 2024-08-02T18:02:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.140+0000 7f82dc19e740 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa6e210 0x55854fa70600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.140+0000 7f82dc19e740 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa5ae50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.140+0000 7f82dc19e740 1 --2- 172.21.15.69:0/423393816 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa63420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.140+0000 7f82dc19e740 1 -- 172.21.15.69:0/423393816 >> 172.21.15.69:0/423393816 conn(0x55854fa56d20 msgr2=0x55854fa57100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.141+0000 7f82dc19e740 1 -- 172.21.15.69:0/423393816 shutdown_connections 2024-08-02T18:02:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.141+0000 7f82dc19e740 1 -- 172.21.15.69:0/423393816 wait complete. 2024-08-02T18:02:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.141+0000 7f82dc19e740 1 -- start start 2024-08-02T18:02:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.141+0000 7f82dc19e740 1 Processor -- start 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a9d0 0x55854fa60d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa6e210 0x55854fa61240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82dc19e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55854fa70d20 con 0x55854fa5a9d0 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82dc19e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55854f97a500 con 0x55854fa6e210 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45808/0 (socket says 172.21.15.69:45808) 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82d2ffd640 1 -- 172.21.15.69:0/2047876306 learned_addr learned my addr 172.21.15.69:0/2047876306 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55854fa70ea0 con 0x55854fa5a5f0 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82d27fc640 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a9d0 0x55854fa60d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82dc19e740 10 client.0 _ll_register_callbacks cb 0x55854f6d1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82d27fc640 1 -- 172.21.15.69:0/2047876306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 msgr2=0x55854fa607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82d27fc640 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82d27fc640 1 -- 172.21.15.69:0/2047876306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa6e210 msgr2=0x55854fa61240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.142+0000 7f82d27fc640 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa6e210 0x55854fa61240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.143+0000 7f82d27fc640 1 -- 172.21.15.69:0/2047876306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55854f79e620 con 0x55854fa5a9d0 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.143+0000 7f82d27fc640 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a9d0 0x55854fa60d00 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f82c4005ec0 tx=0x7f82c402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d563f796569b0061 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.143+0000 7f82b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.143+0000 7f82b6ff5640 1 -- 172.21.15.69:0/2047876306 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c404e070 con 0x55854fa5a9d0 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.143+0000 7f82b6ff5640 1 -- 172.21.15.69:0/2047876306 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9656+0+0 (secure 0 0 0) 0x7f82c40315e0 con 0x55854fa5a9d0 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.143+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c4031cc0 con 0x55854fa5a9d0 2024-08-02T18:02:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.144+0000 7f82b6ff5640 1 -- 172.21.15.69:0/2047876306 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82c40317e0 con 0x55854fa5a9d0 2024-08-02T18:02:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.144+0000 7f82dc19e740 10 client.19284 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.144+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55854fa70d20 con 0x55854fa5a9d0 2024-08-02T18:02:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.145+0000 7f82b6ff5640 1 -- 172.21.15.69:0/2047876306 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c4031cc0 con 0x55854fa5a9d0 2024-08-02T18:02:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.146+0000 7f82b6ff5640 1 -- 172.21.15.69:0/2047876306 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f82c403c430 con 0x55854fa5a9d0 2024-08-02T18:02:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.146+0000 7f82dc19e740 10 client.19284 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.146+0000 7f82dc19e740 10 client.19284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.146+0000 7f82dc19e740 20 client.19284 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.146+0000 7f82dc19e740 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55854fa7dda0 0x55854fa80200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.146+0000 7f82dc19e740 4 client.19284 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.146+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55854f9cc0e0 con 0x55854fa7dda0 2024-08-02T18:02:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.146+0000 7f82d2ffd640 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55854fa7dda0 0x55854fa80200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.147+0000 7f82b6ff5640 10 client.19284 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.147+0000 7f82d2ffd640 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55854fa7dda0 0x55854fa80200 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ec79803088f9a1 server_cookie=ef0b24d04d131d1b in_seq=0 out_seq=0 2024-08-02T18:02:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.147+0000 7f82b6ff5640 1 -- 172.21.15.69:0/2047876306 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55854f9cc0e0 con 0x55854fa7dda0 2024-08-02T18:02:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.147+0000 7f82b6ff5640 10 client.19284 handle_command_reply: tid=0 2024-08-02T18:02:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 client.19284 shutdown 2024-08-02T18:02:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a9d0 msgr2=0x55854fa60d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a9d0 0x55854fa60d00 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f82c4005ec0 tx=0x7f82c402a480 comp rx=0 tx=0).stop 2024-08-02T18:02:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 shutdown_connections 2024-08-02T18:02:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a9d0 0x55854fa60d00 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55854fa7dda0 0x55854fa80200 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa6e210 0x55854fa61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 --2- 172.21.15.69:0/2047876306 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 >> 172.21.15.69:0/2047876306 conn(0x55854fa56d20 msgr2=0x55854fa55e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82b6ff5640 0 client.19284 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 shutdown_connections 2024-08-02T18:02:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.148+0000 7f82dc19e740 1 -- 172.21.15.69:0/2047876306 wait complete. 2024-08-02T18:02:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.149+0000 7f82dc19e740 20 client.19284 trim_cache size 0 max 16384 2024-08-02T18:02:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.175+0000 7f82dc19e740 1 Processor -- start 2024-08-02T18:02:06.176 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.175+0000 7f82dc19e740 1 -- start start 2024-08-02T18:02:06.178 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.175+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa5f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa85aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa6e210 0x55854fa85fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82dc19e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82c0012a90 con 0x55854fa6e210 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82dc19e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f82c0010360 con 0x55854fa5a9d0 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82dc19e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82c000e1d0 con 0x55854fa5a5f0 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa5f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa5f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:45812/0 (socket says 172.21.15.69:45812) 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d2ffd640 1 -- 172.21.15.69:0/3171989104 learned_addr learned my addr 172.21.15.69:0/3171989104 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d27fc640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa85aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d37fe640 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa6e210 0x55854fa85fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d27fc640 1 -- 172.21.15.69:0/3171989104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 msgr2=0x55854fa5f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:06.179 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d27fc640 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa5f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d27fc640 1 -- 172.21.15.69:0/3171989104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa6e210 msgr2=0x55854fa85fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d27fc640 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa6e210 0x55854fa85fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d27fc640 1 -- 172.21.15.69:0/3171989104 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55854fa7a630 con 0x55854fa5a9d0 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.176+0000 7f82d37fe640 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa6e210 0x55854fa85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.177+0000 7f82d27fc640 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa85aa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f82c403c9f0 tx=0x7f82c40098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8daa3a770f24fa7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.177+0000 7f82b8ff9640 1 -- 172.21.15.69:0/3171989104 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c404e040 con 0x55854fa5a9d0 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.177+0000 7f82b8ff9640 1 -- 172.21.15.69:0/3171989104 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c4060070 con 0x55854fa5a9d0 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.177+0000 7f82b8ff9640 1 -- 172.21.15.69:0/3171989104 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c4031b80 con 0x55854fa5a9d0 2024-08-02T18:02:06.180 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.178+0000 7f82dc19e740 1 -- 172.21.15.69:0/3171989104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 msgr2=0x55854fa85aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.186 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.183+0000 7f82dc19e740 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa85aa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f82c403c9f0 tx=0x7f82c40098f0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.192+0000 7f82dc19e740 1 -- 172.21.15.69:0/3171989104 shutdown_connections 2024-08-02T18:02:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.192+0000 7f82dc19e740 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa6e210 0x55854fa85fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.192+0000 7f82dc19e740 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa5a9d0 0x55854fa85aa0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.192+0000 7f82dc19e740 1 --2- 172.21.15.69:0/3171989104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a5f0 0x55854fa5f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.192+0000 7f82dc19e740 1 -- 172.21.15.69:0/3171989104 >> 172.21.15.69:0/3171989104 conn(0x55854fa56d20 msgr2=0x55854fa5f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:06.194 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.193+0000 7f82dc19e740 1 -- 172.21.15.69:0/3171989104 shutdown_connections 2024-08-02T18:02:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.198+0000 7f82dc19e740 1 -- 172.21.15.69:0/3171989104 wait complete. 2024-08-02T18:02:06.199 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.198+0000 7f82dc19e740 1 -- start start 2024-08-02T18:02:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.198+0000 7f82dc19e740 1 Processor -- start 2024-08-02T18:02:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.199+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.199+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a9d0 0x55854fa73870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.199+0000 7f82dc19e740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa6e210 0x55854fae6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.199+0000 7f82dc19e740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f82c000e820 con 0x55854fa5a5f0 2024-08-02T18:02:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.199+0000 7f82dc19e740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f82c000fc90 con 0x55854fa6e210 2024-08-02T18:02:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.199+0000 7f82dc19e740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f82c000eef0 con 0x55854fa5a9d0 2024-08-02T18:02:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.200+0000 7f82dc19e740 10 client.0 _ll_register_callbacks cb 0x7f82cc021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.202+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.202+0000 7f82d2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa73330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:44260/0 (socket says 172.21.15.69:44260) 2024-08-02T18:02:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.202+0000 7f82d2ffd640 1 -- 172.21.15.69:0/748570395 learned_addr learned my addr 172.21.15.69:0/748570395 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.202+0000 7f82d27fc640 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a9d0 0x55854fa73870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.203+0000 7f82d2ffd640 1 -- 172.21.15.69:0/748570395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a9d0 msgr2=0x55854fa73870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.203+0000 7f82d2ffd640 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a9d0 0x55854fa73870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.204 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.203+0000 7f82d2ffd640 1 -- 172.21.15.69:0/748570395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa6e210 msgr2=0x55854fae6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.203+0000 7f82d2ffd640 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa6e210 0x55854fae6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.203+0000 7f82d2ffd640 1 -- 172.21.15.69:0/748570395 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55854fae6fb0 con 0x55854fa5a5f0 2024-08-02T18:02:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.204+0000 7f82d2ffd640 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa73330 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f82c8014950 tx=0x7f82c8014a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53f31d17ac60649 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:06.205 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.204+0000 7f82b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.204+0000 7f82b6ff5640 1 -- 172.21.15.69:0/748570395 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c801f620 con 0x55854fa5a5f0 2024-08-02T18:02:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.204+0000 7f82b6ff5640 1 -- 172.21.15.69:0/748570395 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9656+0+0 (secure 0 0 0) 0x7f82c801f7c0 con 0x55854fa5a5f0 2024-08-02T18:02:06.206 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.204+0000 7f82dc19e740 1 -- 172.21.15.69:0/748570395 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c4060070 con 0x55854fa5a5f0 2024-08-02T18:02:06.208 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.207+0000 7f82b6ff5640 1 -- 172.21.15.69:0/748570395 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f82c802d050 con 0x55854fa5a5f0 2024-08-02T18:02:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.210+0000 7f82dc19e740 10 client.19290 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:06.212 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.211+0000 7f82dc19e740 1 -- 172.21.15.69:0/748570395 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55854fae81b0 con 0x55854fa5a5f0 2024-08-02T18:02:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.211+0000 7f82b6ff5640 1 -- 172.21.15.69:0/748570395 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c8023070 con 0x55854fa5a5f0 2024-08-02T18:02:06.213 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.211+0000 7f82b6ff5640 1 -- 172.21.15.69:0/748570395 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f82c801e070 con 0x55854fa5a5f0 2024-08-02T18:02:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.213+0000 7f82dc19e740 10 client.19290 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.213+0000 7f82dc19e740 10 client.19290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.213+0000 7f82dc19e740 20 client.19290 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.213+0000 7f82dc19e740 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55854fae88b0 0x55854faeacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.213+0000 7f82dc19e740 4 client.19290 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:02:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.213+0000 7f82dc19e740 1 -- 172.21.15.69:0/748570395 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f82c4060070 con 0x55854fae88b0 2024-08-02T18:02:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.219+0000 7f82d27fc640 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55854fae88b0 0x55854faeacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.233+0000 7f82d27fc640 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55854fae88b0 0x55854faeacf0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1248a9ce7b80c1 server_cookie=a6cb61bdc9d0f18e in_seq=0 out_seq=0 2024-08-02T18:02:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.233+0000 7f82b6ff5640 10 client.19290 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.301+0000 7f82b6ff5640 1 -- 172.21.15.69:0/748570395 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f82c4060070 con 0x55854fae88b0 2024-08-02T18:02:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.301+0000 7f82b6ff5640 10 client.19290 handle_command_reply: tid=0 2024-08-02T18:02:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.302+0000 7f82dc19e740 1 client.19290 shutdown 2024-08-02T18:02:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.303+0000 7f82dc19e740 1 -- 172.21.15.69:0/748570395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 msgr2=0x55854fa73330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.303+0000 7f82dc19e740 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa73330 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f82c8014950 tx=0x7f82c8014a40 comp rx=0 tx=0).stop 2024-08-02T18:02:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 1 -- 172.21.15.69:0/748570395 shutdown_connections 2024-08-02T18:02:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55854fae88b0 0x55854faeacf0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55854fa6e210 0x55854fae6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55854fa5a9d0 0x55854fa73870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 1 --2- 172.21.15.69:0/748570395 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55854fa5a5f0 0x55854fa73330 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 1 -- 172.21.15.69:0/748570395 >> 172.21.15.69:0/748570395 conn(0x55854fa56d20 msgr2=0x55854fa5f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82b6ff5640 0 client.19290 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 1 -- 172.21.15.69:0/748570395 shutdown_connections 2024-08-02T18:02:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 1 -- 172.21.15.69:0/748570395 wait complete. 2024-08-02T18:02:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.304+0000 7f82dc19e740 20 client.19290 trim_cache size 0 max 16384 2024-08-02T18:02:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:02:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:02:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:02:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:02:06.309 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f82bc0a30f0 msgr2=0x7f82bc0a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f82bc0a30f0 0x7f82bc0a55b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f82c4000c00 tx=0x7f82c4004580 comp rx=0 tx=0).stop 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d4106700 msgr2=0x7f82d419bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d4106700 0x7f82d419bad0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f82cc00be20 tx=0x7f82cc00bf10 comp rx=0 tx=0).stop 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 shutdown_connections 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f82bc0a30f0 0x7f82bc0a55b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f82d4106700 0x7f82d419bad0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f82d4101090 0x7f82d4197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 --2- 172.21.15.69:0/1315929043 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f82d41006f0 0x7f82d4197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.305+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 >> 172.21.15.69:0/1315929043 conn(0x7f82d40fc430 msgr2=0x7f82d40fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.306+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 shutdown_connections 2024-08-02T18:02:06.310 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:06.306+0000 7f82d936c640 1 -- 172.21.15.69:0/1315929043 wait complete. 2024-08-02T18:02:07.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:06 smithi083 ceph-mon[37031]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.9 MiB/s wr, 559 op/s 2024-08-02T18:02:07.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:06 smithi077 ceph-mon[37108]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.9 MiB/s wr, 559 op/s 2024-08-02T18:02:07.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:06 smithi069 ceph-mon[32746]: pgmap v492: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 6.9 MiB/s wr, 559 op/s 2024-08-02T18:02:09.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:08 smithi083 ceph-mon[37031]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.9 MiB/s wr, 585 op/s 2024-08-02T18:02:09.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:08 smithi077 ceph-mon[37108]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.9 MiB/s wr, 585 op/s 2024-08-02T18:02:09.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:08 smithi069 ceph-mon[32746]: pgmap v493: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.9 MiB/s wr, 585 op/s 2024-08-02T18:02:09.939 INFO:tasks.cephfs.filesystem:scrub status for tag:58f623b4-5921-4b17-bbd8-520a0f3b45c1 - {'path': '/', 'tag': '58f623b4-5921-4b17-bbd8-520a0f3b45c1', 'options': 'recursive,force'} 2024-08-02T18:02:09.953 INFO:teuthology.orchestra.run.smithi069.stderr:progress: 900.0 s, 398.0 tps, lat 262.461 ms stddev 350.110 2024-08-02T18:02:10.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:09 smithi083 ceph-mon[37031]: scrub summary: idle 2024-08-02T18:02:10.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:09 smithi077 ceph-mon[37108]: scrub summary: idle 2024-08-02T18:02:10.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:09 smithi069 ceph-mon[32746]: scrub summary: idle 2024-08-02T18:02:10.401 INFO:teuthology.orchestra.run.smithi069.stdout:transaction type: 2024-08-02T18:02:10.401 INFO:teuthology.orchestra.run.smithi069.stdout:scaling factor: 32 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:query mode: simple 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:number of clients: 100 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:number of threads: 4 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:duration: 900 s 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:number of transactions actually processed: 749303 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:latency average = 120.120 ms 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:latency stddev = 885.705 ms 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:tps = 832.144073 (including connections establishing) 2024-08-02T18:02:10.402 INFO:teuthology.orchestra.run.smithi069.stdout:tps = 832.146224 (excluding connections establishing) 2024-08-02T18:02:10.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-08-02T18:02:11.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:10 smithi083 ceph-mon[37031]: pgmap v494: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 6.6 MiB/s wr, 592 op/s 2024-08-02T18:02:11.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:10 smithi077 ceph-mon[37108]: pgmap v494: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 6.6 MiB/s wr, 592 op/s 2024-08-02T18:02:11.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:10 smithi069 ceph-mon[32746]: pgmap v494: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 6.6 MiB/s wr, 592 op/s 2024-08-02T18:02:13.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:12 smithi083 ceph-mon[37031]: pgmap v495: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 6.2 MiB/s wr, 549 op/s 2024-08-02T18:02:13.080 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-02T18:02:13.080 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.95 2024-08-02T18:02:13.080 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:02:13.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:12 smithi077 ceph-mon[37108]: pgmap v495: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 6.2 MiB/s wr, 549 op/s 2024-08-02T18:02:13.110 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:12 smithi069 ceph-mon[32746]: pgmap v495: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 6.2 MiB/s wr, 549 op/s 2024-08-02T18:02:13.771 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:14.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.407+0000 7f134d14f640 1 -- 172.21.15.69:0/3024672387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 msgr2=0x7f134811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:14.410 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.407+0000 7f134d14f640 1 --2- 172.21.15.69:0/3024672387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 0x7f134811d260 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1340009ca0 tx=0x7f134002a8f0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.411+0000 7f134d14f640 1 -- 172.21.15.69:0/3024672387 shutdown_connections 2024-08-02T18:02:14.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.411+0000 7f134d14f640 1 --2- 172.21.15.69:0/3024672387 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134806e760 0x7f134806eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.412 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.411+0000 7f134d14f640 1 --2- 172.21.15.69:0/3024672387 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 0x7f134811d260 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.411+0000 7f134d14f640 1 --2- 172.21.15.69:0/3024672387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f134810fec0 0x7f134806e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.411+0000 7f134d14f640 1 -- 172.21.15.69:0/3024672387 >> 172.21.15.69:0/3024672387 conn(0x7f1348072eb0 msgr2=0x7f13480732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:14.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.411+0000 7f134d14f640 1 -- 172.21.15.69:0/3024672387 shutdown_connections 2024-08-02T18:02:14.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.411+0000 7f134d14f640 1 -- 172.21.15.69:0/3024672387 wait complete. 2024-08-02T18:02:14.413 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.412+0000 7f134d14f640 1 Processor -- start 2024-08-02T18:02:14.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.412+0000 7f134d14f640 1 -- start start 2024-08-02T18:02:14.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.412+0000 7f134d14f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f134806e760 0x7f1348119360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:14.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.412+0000 7f134d14f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134810fec0 0x7f13481198a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:14.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.412+0000 7f134d14f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 0x7f1348112670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:14.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.412+0000 7f134d14f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f1348120a10 con 0x7f134810fec0 2024-08-02T18:02:14.415 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.412+0000 7f134d14f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f1348120890 con 0x7f134811ae70 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.412+0000 7f134d14f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1348120b90 con 0x7f134806e760 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.413+0000 7f134c94e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 0x7f1348112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.413+0000 7f134c94e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 0x7f1348112670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:55364/0 (socket says 172.21.15.69:55364) 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.413+0000 7f134c94e640 1 -- 172.21.15.69:0/3154036411 learned_addr learned my addr 172.21.15.69:0/3154036411 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.413+0000 7f134c94e640 1 -- 172.21.15.69:0/3154036411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f134806e760 msgr2=0x7f1348119360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.413+0000 7f134c94e640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f134806e760 0x7f1348119360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.413+0000 7f134c94e640 1 -- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134810fec0 msgr2=0x7f13481198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.413+0000 7f134c94e640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134810fec0 0x7f13481198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.413+0000 7f134c94e640 1 -- 172.21.15.69:0/3154036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1348112ea0 con 0x7f134811ae70 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.414+0000 7f134c94e640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 0x7f1348112670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f13400097b0 tx=0x7f134002fba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:14.416 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.414+0000 7f13457fa640 1 -- 172.21.15.69:0/3154036411 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340003b70 con 0x7f134811ae70 2024-08-02T18:02:14.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.414+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1348113130 con 0x7f134811ae70 2024-08-02T18:02:14.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.414+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1348117060 con 0x7f134811ae70 2024-08-02T18:02:14.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.414+0000 7f13457fa640 1 -- 172.21.15.69:0/3154036411 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134002d070 con 0x7f134811ae70 2024-08-02T18:02:14.417 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.414+0000 7f13457fa640 1 -- 172.21.15.69:0/3154036411 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1340030850 con 0x7f134811ae70 2024-08-02T18:02:14.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.416+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1348109420 con 0x7f134811ae70 2024-08-02T18:02:14.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.419+0000 7f13457fa640 1 -- 172.21.15.69:0/3154036411 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f13400309f0 con 0x7f134811ae70 2024-08-02T18:02:14.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.420+0000 7f13457fa640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13300a2f00 0x7f13300a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:14.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.420+0000 7f13457fa640 1 -- 172.21.15.69:0/3154036411 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9656+0+0 (secure 0 0 0) 0x7f13400f0f60 con 0x7f134811ae70 2024-08-02T18:02:14.424 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.423+0000 7f1347fff640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13300a2f00 0x7f13300a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:14.427 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.425+0000 7f1347fff640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13300a2f00 0x7f13300a53c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f13380096f0 tx=0x7f1338006c40 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:14.432 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.431+0000 7f13457fa640 1 -- 172.21.15.69:0/3154036411 <== mon.2 v2:172.21.15.77:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f134004a060 con 0x7f134811ae70 2024-08-02T18:02:14.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.824+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1348117380 con 0x7f134811ae70 2024-08-02T18:02:14.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.826+0000 7f13457fa640 1 -- 172.21.15.69:0/3154036411 <== mon.2 v2:172.21.15.77:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11901 (secure 0 0 0) 0x7f13400be800 con 0x7f134811ae70 2024-08-02T18:02:14.828 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:14.828 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"btime":"2024-08-02T18:01:55:674676+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":28375,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3960673726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3960673726},{"type":"v1","addr":"172.21.15.77:6835","nonce":3960673726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T18:01:54.670284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24389":{"gid":24389,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:02:14.828 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-08-02T18:02:14.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.837+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13300a2f00 msgr2=0x7f13300a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:14.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.837+0000 7f134d14f640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13300a2f00 0x7f13300a53c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f13380096f0 tx=0x7f1338006c40 comp rx=0 tx=0).stop 2024-08-02T18:02:14.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.837+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 msgr2=0x7f1348112670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:14.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.837+0000 7f134d14f640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 0x7f1348112670 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f13400097b0 tx=0x7f134002fba0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.839+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 shutdown_connections 2024-08-02T18:02:14.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.839+0000 7f134d14f640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f134810fec0 0x7f13481198a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.839+0000 7f134d14f640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f13300a2f00 0x7f13300a53c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.839+0000 7f134d14f640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f134811ae70 0x7f1348112670 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.839+0000 7f134d14f640 1 --2- 172.21.15.69:0/3154036411 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f134806e760 0x7f1348119360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:14.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.839+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 >> 172.21.15.69:0/3154036411 conn(0x7f1348072eb0 msgr2=0x7f134806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:14.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.840+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 shutdown_connections 2024-08-02T18:02:14.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:14.841+0000 7f134d14f640 1 -- 172.21.15.69:0/3154036411 wait complete. 2024-08-02T18:02:15.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:14 smithi083 ceph-mon[37031]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 7.4 MiB/s wr, 687 op/s 2024-08-02T18:02:15.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:14 smithi083 ceph-mon[37031]: osdmap e115: 12 total, 12 up, 12 in 2024-08-02T18:02:15.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:14 smithi077 ceph-mon[37108]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 7.4 MiB/s wr, 687 op/s 2024-08-02T18:02:15.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:14 smithi077 ceph-mon[37108]: osdmap e115: 12 total, 12 up, 12 in 2024-08-02T18:02:15.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:14 smithi069 ceph-mon[32746]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 7.4 MiB/s wr, 687 op/s 2024-08-02T18:02:15.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:14 smithi069 ceph-mon[32746]: osdmap e115: 12 total, 12 up, 12 in 2024-08-02T18:02:16.055 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:15 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3154036411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:16.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:15 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3154036411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:16.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:15 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3154036411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:17.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:16 smithi077 ceph-mon[37108]: pgmap v498: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 7.0 MiB/s wr, 676 op/s 2024-08-02T18:02:17.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:16 smithi069 ceph-mon[32746]: pgmap v498: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 7.0 MiB/s wr, 676 op/s 2024-08-02T18:02:17.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:16 smithi083 ceph-mon[37031]: pgmap v498: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 7.0 MiB/s wr, 676 op/s 2024-08-02T18:02:19.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:18 smithi077 ceph-mon[37108]: pgmap v499: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2024-08-02T18:02:19.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:18 smithi069 ceph-mon[32746]: pgmap v499: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2024-08-02T18:02:19.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:18 smithi083 ceph-mon[37031]: pgmap v499: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.2 MiB/s wr, 590 op/s 2024-08-02T18:02:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.479+0000 7f5e202da640 1 -- 172.21.15.69:0/1146204942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e1811ae70 msgr2=0x7f5e1811d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:19.482 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.479+0000 7f5e202da640 1 --2- 172.21.15.69:0/1146204942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e1811ae70 0x7f5e1811d260 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009860 tx=0x7f5e1402a360 comp rx=0 tx=0).stop 2024-08-02T18:02:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.487+0000 7f5e202da640 1 -- 172.21.15.69:0/1146204942 shutdown_connections 2024-08-02T18:02:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.487+0000 7f5e202da640 1 --2- 172.21.15.69:0/1146204942 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e1811ae70 0x7f5e1811d260 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.487+0000 7f5e202da640 1 --2- 172.21.15.69:0/1146204942 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e18109300 0x7f5e18109740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.487+0000 7f5e202da640 1 --2- 172.21.15.69:0/1146204942 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e18108f20 0x7f5e1810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.487+0000 7f5e202da640 1 -- 172.21.15.69:0/1146204942 >> 172.21.15.69:0/1146204942 conn(0x7f5e1806cb20 msgr2=0x7f5e1806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:19.496 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.495+0000 7f5e202da640 1 -- 172.21.15.69:0/1146204942 shutdown_connections 2024-08-02T18:02:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.497+0000 7f5e202da640 1 -- 172.21.15.69:0/1146204942 wait complete. 2024-08-02T18:02:19.500 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.498+0000 7f5e202da640 1 Processor -- start 2024-08-02T18:02:19.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.501+0000 7f5e202da640 1 -- start start 2024-08-02T18:02:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.502+0000 7f5e202da640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e18108f20 0x7f5e181a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.502+0000 7f5e202da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e18109300 0x7f5e181a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.502+0000 7f5e202da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1811ae70 0x7f5e181ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.502+0000 7f5e202da640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18120a30 con 0x7f5e18109300 2024-08-02T18:02:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.502+0000 7f5e202da640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5e181208b0 con 0x7f5e18108f20 2024-08-02T18:02:19.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.502+0000 7f5e202da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5e18120bb0 con 0x7f5e1811ae70 2024-08-02T18:02:19.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.504+0000 7f5e1d875640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e18109300 0x7f5e181a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.504+0000 7f5e1d875640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e18109300 0x7f5e181a7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41966/0 (socket says 172.21.15.69:41966) 2024-08-02T18:02:19.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.504+0000 7f5e1d875640 1 -- 172.21.15.69:0/2078372127 learned_addr learned my addr 172.21.15.69:0/2078372127 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.506+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e18108f20 0x7f5e181a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.508+0000 7f5e1e877640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1811ae70 0x7f5e181ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.509+0000 7f5e1e076640 1 -- 172.21.15.69:0/2078372127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1811ae70 msgr2=0x7f5e181ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:19.510 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.509+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1811ae70 0x7f5e181ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.509+0000 7f5e1e076640 1 -- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e18109300 msgr2=0x7f5e181a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.509+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e18109300 0x7f5e181a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.511 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.509+0000 7f5e1e076640 1 -- 172.21.15.69:0/2078372127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e181abed0 con 0x7f5e18108f20 2024-08-02T18:02:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.513+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e18108f20 0x7f5e181a6f10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800b3e0 tx=0x7f5e0800b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.514+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2078372127 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e08015020 con 0x7f5e18108f20 2024-08-02T18:02:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.514+0000 7f5e202da640 1 -- 172.21.15.69:0/2078372127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e181ac1c0 con 0x7f5e18108f20 2024-08-02T18:02:19.515 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.514+0000 7f5e202da640 1 -- 172.21.15.69:0/2078372127 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e18110650 con 0x7f5e18108f20 2024-08-02T18:02:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.516+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2078372127 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e08011d10 con 0x7f5e18108f20 2024-08-02T18:02:19.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.516+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2078372127 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800f2e0 con 0x7f5e18108f20 2024-08-02T18:02:19.519 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.517+0000 7f5e23038740 1 Processor -- start 2024-08-02T18:02:19.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.523+0000 7f5e23038740 1 -- start start 2024-08-02T18:02:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.528+0000 7f5e23038740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x563086847b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.528+0000 7f5e23038740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x563086852430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.528+0000 7f5e23038740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 0x563086854d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.528+0000 7f5e23038740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5630867ab9b0 con 0x5630868477b0 2024-08-02T18:02:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.528+0000 7f5e23038740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563086753800 con 0x5630868480d0 2024-08-02T18:02:19.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.528+0000 7f5e23038740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5630867a7970 con 0x563086852970 2024-08-02T18:02:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.530+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2078372127 <== mon.2 v2:172.21.15.77:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5e08018020 con 0x7f5e18108f20 2024-08-02T18:02:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.530+0000 7f5e1e076640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x563086852430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.530+0000 7f5e1e076640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x563086852430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:41848/0 (socket says 172.21.15.69:41848) 2024-08-02T18:02:19.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.530+0000 7f5e1e076640 1 -- 172.21.15.69:0/3816262145 learned_addr learned my addr 172.21.15.69:0/3816262145 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.531+0000 7f5e1e076640 1 -- 172.21.15.69:0/3816262145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 msgr2=0x563086854d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:19.532 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.531+0000 7f5e1e076640 1 --2- 172.21.15.69:0/3816262145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 0x563086854d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.531+0000 7f5e1e076640 1 -- 172.21.15.69:0/3816262145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 msgr2=0x563086847b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.531+0000 7f5e1e076640 1 --2- 172.21.15.69:0/3816262145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x563086847b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.531+0000 7f5e1e076640 1 -- 172.21.15.69:0/3816262145 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630864bf210 con 0x5630868480d0 2024-08-02T18:02:19.533 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.531+0000 7f5e1e076640 1 --2- 172.21.15.69:0/3816262145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x563086852430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5e080cc0e0 tx=0x7f5e080d77e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aca89c2830b08ea server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.532+0000 7f5e0f7fe640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e040a2e40 0x7f5e040a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.534 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.532+0000 7f5e0f7fe640 1 -- 172.21.15.69:0/2078372127 <== mon.2 v2:172.21.15.77:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f5e080ced90 con 0x7f5e18108f20 2024-08-02T18:02:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.535+0000 7f5e0cbfe640 1 -- 172.21.15.69:0/3816262145 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080de070 con 0x5630868480d0 2024-08-02T18:02:19.536 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.535+0000 7f5e0cbfe640 1 -- 172.21.15.69:0/3816262145 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e080d4ce0 con 0x5630868480d0 2024-08-02T18:02:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.536+0000 7f5e23038740 1 -- 172.21.15.69:0/3816262145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 msgr2=0x563086852430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.536+0000 7f5e23038740 1 --2- 172.21.15.69:0/3816262145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x563086852430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5e080cc0e0 tx=0x7f5e080d77e0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.540+0000 7f5e23038740 1 -- 172.21.15.69:0/3816262145 shutdown_connections 2024-08-02T18:02:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.540+0000 7f5e23038740 1 --2- 172.21.15.69:0/3816262145 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 0x563086854d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.540+0000 7f5e23038740 1 --2- 172.21.15.69:0/3816262145 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x563086852430 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.540+0000 7f5e23038740 1 --2- 172.21.15.69:0/3816262145 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x563086847b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.540+0000 7f5e23038740 1 -- 172.21.15.69:0/3816262145 >> 172.21.15.69:0/3816262145 conn(0x563086843d20 msgr2=0x563086844100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.541+0000 7f5e23038740 1 -- 172.21.15.69:0/3816262145 shutdown_connections 2024-08-02T18:02:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.541+0000 7f5e23038740 1 -- 172.21.15.69:0/3816262145 wait complete. 2024-08-02T18:02:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.541+0000 7f5e23038740 1 -- start start 2024-08-02T18:02:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.541+0000 7f5e23038740 1 Processor -- start 2024-08-02T18:02:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.542+0000 7f5e23038740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308684f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.542+0000 7f5e23038740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x56308684fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.542+0000 7f5e23038740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 0x563086850250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.542+0000 7f5e23038740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5630868553c0 con 0x5630868477b0 2024-08-02T18:02:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.542+0000 7f5e23038740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x563086767500 con 0x5630868480d0 2024-08-02T18:02:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.542+0000 7f5e23038740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x563086855540 con 0x563086852970 2024-08-02T18:02:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.542+0000 7f5e23038740 10 client.0 _ll_register_callbacks cb 0x5630864bea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.543+0000 7f5e1e076640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308684f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.543+0000 7f5e1e076640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308684f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:41992/0 (socket says 172.21.15.69:41992) 2024-08-02T18:02:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.543+0000 7f5e1e076640 1 -- 172.21.15.69:0/2929589381 learned_addr learned my addr 172.21.15.69:0/2929589381 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.543+0000 7f5e1d875640 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x56308684fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.543+0000 7f5e1e877640 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 0x563086850250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.544+0000 7f5e1d875640 1 -- 172.21.15.69:0/2929589381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 msgr2=0x563086850250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.544+0000 7f5e1d875640 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 0x563086850250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.544+0000 7f5e1d875640 1 -- 172.21.15.69:0/2929589381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 msgr2=0x56308684f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.544+0000 7f5e1d875640 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308684f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.546 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.544+0000 7f5e1d875640 1 -- 172.21.15.69:0/2929589381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308658b620 con 0x5630868480d0 2024-08-02T18:02:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.544+0000 7f5e1e877640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e040a2e40 0x7f5e040a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.544+0000 7f5e1d875640 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x56308684fd10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x5630868450a0 tx=0x7f5e10009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fde8e15390a8578 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.546+0000 7f5e01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:02:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.546+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2929589381 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1001d060 con 0x5630868480d0 2024-08-02T18:02:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.546+0000 7f5e23038740 1 -- 172.21.15.69:0/2929589381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e080d4ce0 con 0x5630868480d0 2024-08-02T18:02:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.546+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2929589381 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f5e1000cbb0 con 0x5630868480d0 2024-08-02T18:02:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.551+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2929589381 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e1000cdb0 con 0x5630868480d0 2024-08-02T18:02:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.551+0000 7f5e23038740 10 client.28688 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.551+0000 7f5e23038740 1 -- 172.21.15.69:0/2929589381 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630868611a0 con 0x5630868480d0 2024-08-02T18:02:19.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.554+0000 7f5e1e877640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e040a2e40 0x7f5e040a5300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5e1402a8d0 tx=0x7f5e14005c50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.555+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2929589381 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e080d4ce0 con 0x5630868480d0 2024-08-02T18:02:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.556+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2929589381 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e1000b040 con 0x5630868480d0 2024-08-02T18:02:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.556+0000 7f5e23038740 10 client.28688 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.556+0000 7f5e23038740 10 client.28688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.556+0000 7f5e23038740 20 client.28688 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.556+0000 7f5e23038740 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563086862660 0x563086864a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.556+0000 7f5e23038740 4 client.28688 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:19.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.556+0000 7f5e23038740 1 -- 172.21.15.69:0/2929589381 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630867b90e0 con 0x563086862660 2024-08-02T18:02:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.561+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563086862660 0x563086864a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.570+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563086862660 0x563086864a80 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd7fb47ac0483e30 server_cookie=cb0f4ba78625f5b2 in_seq=0 out_seq=0 2024-08-02T18:02:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.571+0000 7f5e01ffb640 10 client.28688 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.575+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2929589381 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5630867b90e0 con 0x563086862660 2024-08-02T18:02:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.575+0000 7f5e01ffb640 10 client.28688 handle_command_reply: tid=0 2024-08-02T18:02:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.575+0000 7f5e23038740 1 client.28688 shutdown 2024-08-02T18:02:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.580+0000 7f5e23038740 1 -- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 msgr2=0x56308684fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.580+0000 7f5e23038740 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x56308684fd10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x5630868450a0 tx=0x7f5e10009f80 comp rx=0 tx=0).stop 2024-08-02T18:02:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.585+0000 7f5e23038740 1 -- 172.21.15.69:0/2929589381 shutdown_connections 2024-08-02T18:02:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.585+0000 7f5e23038740 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x563086862660 0x563086864a80 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.585+0000 7f5e23038740 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x563086852970 0x563086850250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.585+0000 7f5e23038740 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5630868480d0 0x56308684fd10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.585+0000 7f5e23038740 1 --2- 172.21.15.69:0/2929589381 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308684f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.585+0000 7f5e23038740 1 -- 172.21.15.69:0/2929589381 >> 172.21.15.69:0/2929589381 conn(0x563086843d20 msgr2=0x563086849a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.585+0000 7f5e01ffb640 0 client.28688 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.591+0000 7f5e23038740 1 -- 172.21.15.69:0/2929589381 shutdown_connections 2024-08-02T18:02:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.591+0000 7f5e23038740 1 -- 172.21.15.69:0/2929589381 wait complete. 2024-08-02T18:02:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.594+0000 7f5e23038740 20 client.28688 trim_cache size 0 max 16384 2024-08-02T18:02:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.624+0000 7f5e23038740 1 Processor -- start 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.624+0000 7f5e23038740 1 -- start start 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.624+0000 7f5e23038740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308686a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.624+0000 7f5e23038740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 0x56308686a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.624+0000 7f5e23038740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563086852970 0x563086849210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e23038740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc012860 con 0x5630868477b0 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e23038740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc010130 con 0x563086852970 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e23038740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc00e030 con 0x5630868480d0 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308686a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308686a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42000/0 (socket says 172.21.15.69:42000) 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 -- 172.21.15.69:0/3331130350 learned_addr learned my addr 172.21.15.69:0/3331130350 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1d875640 1 --2- 172.21.15.69:0/3331130350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 0x56308686a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 -- 172.21.15.69:0/3331130350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 msgr2=0x56308686a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 --2- 172.21.15.69:0/3331130350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 0x56308686a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 -- 172.21.15.69:0/3331130350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563086852970 msgr2=0x563086849210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 --2- 172.21.15.69:0/3331130350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563086852970 0x563086849210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 -- 172.21.15.69:0/3331130350 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563086843b50 con 0x5630868477b0 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e1e076640 1 --2- 172.21.15.69:0/3331130350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308686a310 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5e080d7ce0 tx=0x7f5e080bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389c95c9fdea67c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e03fff640 1 -- 172.21.15.69:0/3331130350 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080b5070 con 0x5630868477b0 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e03fff640 1 -- 172.21.15.69:0/3331130350 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e080d2300 con 0x5630868477b0 2024-08-02T18:02:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.625+0000 7f5e03fff640 1 -- 172.21.15.69:0/3331130350 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080de040 con 0x5630868477b0 2024-08-02T18:02:19.631 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 -- 172.21.15.69:0/3331130350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 msgr2=0x56308686a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 --2- 172.21.15.69:0/3331130350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308686a310 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f5e080d7ce0 tx=0x7f5e080bb040 comp rx=0 tx=0).stop 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 -- 172.21.15.69:0/3331130350 shutdown_connections 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 --2- 172.21.15.69:0/3331130350 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563086852970 0x563086849210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 --2- 172.21.15.69:0/3331130350 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 0x56308686a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 --2- 172.21.15.69:0/3331130350 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308686a310 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 -- 172.21.15.69:0/3331130350 >> 172.21.15.69:0/3331130350 conn(0x563086843d20 msgr2=0x563086851740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 -- 172.21.15.69:0/3331130350 shutdown_connections 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 -- 172.21.15.69:0/3331130350 wait complete. 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.626+0000 7f5e23038740 1 -- start start 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e23038740 1 Processor -- start 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e23038740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308685aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e23038740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 0x56308684c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308685aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308685aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42006/0 (socket says 172.21.15.69:42006) 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 -- 172.21.15.69:0/2510178044 learned_addr learned my addr 172.21.15.69:0/2510178044 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e23038740 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563086852970 0x56308684c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc00e5f0 con 0x5630868477b0 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc00fa60 con 0x563086852970 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc00ecc0 con 0x5630868480d0 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e23038740 10 client.0 _ll_register_callbacks cb 0x7f5e100099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1d875640 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 0x56308684c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 -- 172.21.15.69:0/2510178044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 msgr2=0x56308684c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 0x56308684c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 -- 172.21.15.69:0/2510178044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563086852970 msgr2=0x56308684c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563086852970 0x56308684c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 -- 172.21.15.69:0/2510178044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56308684cf50 con 0x5630868477b0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e1e076640 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308685aee0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5e080d7fd0 tx=0x7f5e08010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a6439ba2a96970 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.627+0000 7f5e01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.628+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2510178044 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e080b5070 con 0x5630868477b0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.628+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2510178044 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f5e08099270 con 0x5630868477b0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.628+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56308683efa0 con 0x5630868477b0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.628+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2510178044 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5e080de070 con 0x5630868477b0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.628+0000 7f5e23038740 10 client.19305 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.628+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5630868cd180 con 0x5630868477b0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.628+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2510178044 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56308683efa0 con 0x5630868477b0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.628+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2510178044 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5e080b8030 con 0x5630868477b0 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e23038740 10 client.19305 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e23038740 10 client.19305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e23038740 20 client.19305 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e23038740 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5630868cd400 0x5630868cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e23038740 4 client.19305 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630867b3430 con 0x5630868cd400 2024-08-02T18:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e1d875640 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5630868cd400 0x5630868cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e01ffb640 10 client.19305 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:19.629+0000 7f5e1d875640 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5630868cd400 0x5630868cf7f0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d102d4035d527a65 server_cookie=88d660dd418b2d3 in_seq=0 out_seq=0 2024-08-02T18:02:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:20 smithi077 ceph-mon[37108]: pgmap v500: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-08-02T18:02:21.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:20 smithi069 ceph-mon[32746]: pgmap v500: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-08-02T18:02:21.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:20 smithi083 ceph-mon[37031]: pgmap v500: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-08-02T18:02:22.304 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 147.95 --await-for 45.0' 2024-08-02T18:02:22.304 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 147.95 --await-for 45.0 2024-08-02T18:02:22.914 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:22 smithi077 ceph-mon[37108]: pgmap v501: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-08-02T18:02:23.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:22 smithi069 ceph-mon[32746]: pgmap v501: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-08-02T18:02:23.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:22 smithi083 ceph-mon[37031]: pgmap v501: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.8 MiB/s wr, 660 op/s 2024-08-02T18:02:23.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.829+0000 7fd93a215640 1 -- 172.21.15.69:0/3914082714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934102c20 msgr2=0x7fd93410d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:23.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.829+0000 7fd93a215640 1 --2- 172.21.15.69:0/3914082714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934102c20 0x7fd93410d770 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009cd0 tx=0x7fd92c02a970 comp rx=0 tx=0).stop 2024-08-02T18:02:23.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.830+0000 7fd93a215640 1 -- 172.21.15.69:0/3914082714 shutdown_connections 2024-08-02T18:02:23.832 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.830+0000 7fd93a215640 1 --2- 172.21.15.69:0/3914082714 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9341022a0 0x7fd9341026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.830+0000 7fd93a215640 1 --2- 172.21.15.69:0/3914082714 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934102c20 0x7fd93410d770 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.830+0000 7fd93a215640 1 --2- 172.21.15.69:0/3914082714 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd934100b50 0x7fd934100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.830+0000 7fd93a215640 1 -- 172.21.15.69:0/3914082714 >> 172.21.15.69:0/3914082714 conn(0x7fd9340fc500 msgr2=0x7fd9340fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.830+0000 7fd93a215640 1 -- 172.21.15.69:0/3914082714 shutdown_connections 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.830+0000 7fd93a215640 1 -- 172.21.15.69:0/3914082714 wait complete. 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.830+0000 7fd93a215640 1 Processor -- start 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd93a215640 1 -- start start 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd93a215640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934100b50 0x7fd934196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.833 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd93a215640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9341022a0 0x7fd934196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934100b50 0x7fd934196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934100b50 0x7fd934196050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59642/0 (socket says 172.21.15.69:59642) 2024-08-02T18:02:23.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd9337fe640 1 -- 172.21.15.69:0/402779466 learned_addr learned my addr 172.21.15.69:0/402779466 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:23.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd932ffd640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9341022a0 0x7fd934196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd93a215640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd934102c20 0x7fd93419a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd93410fbf0 con 0x7fd9341022a0 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd93410fa70 con 0x7fd934102c20 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd93410fd70 con 0x7fd934100b50 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.831+0000 7fd933fff640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd934102c20 0x7fd93419a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.832+0000 7fd9337fe640 1 -- 172.21.15.69:0/402779466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd934102c20 msgr2=0x7fd93419a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.832+0000 7fd9337fe640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd934102c20 0x7fd93419a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.832+0000 7fd9337fe640 1 -- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9341022a0 msgr2=0x7fd934196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.832+0000 7fd9337fe640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9341022a0 0x7fd934196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.832+0000 7fd9337fe640 1 -- 172.21.15.69:0/402779466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93419b000 con 0x7fd934100b50 2024-08-02T18:02:23.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.832+0000 7fd9337fe640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934100b50 0x7fd934196050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd92400e9f0 tx=0x7fd92400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.834+0000 7fd930ff9640 1 -- 172.21.15.69:0/402779466 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92400cd80 con 0x7fd934100b50 2024-08-02T18:02:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.834+0000 7fd930ff9640 1 -- 172.21.15.69:0/402779466 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd924005500 con 0x7fd934100b50 2024-08-02T18:02:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.834+0000 7fd930ff9640 1 -- 172.21.15.69:0/402779466 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924010690 con 0x7fd934100b50 2024-08-02T18:02:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.834+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93419b2f0 con 0x7fd934100b50 2024-08-02T18:02:23.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.834+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9341a2bd0 con 0x7fd934100b50 2024-08-02T18:02:23.838 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.837+0000 7fd930ff9640 1 -- 172.21.15.69:0/402779466 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd9240046c0 con 0x7fd934100b50 2024-08-02T18:02:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd93c613740 1 Processor -- start 2024-08-02T18:02:23.839 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd93c613740 1 -- start start 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd93c613740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4c2c0 0x55589cc4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd93c613740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4cbe0 0x55589cc56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd93c613740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc59890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd93c613740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55589cbb1810 con 0x55589cc4c2c0 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd93c613740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55589cb58c70 con 0x55589cc574a0 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd93c613740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55589cbad7d0 con 0x55589cc4cbe0 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd933fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4c2c0 0x55589cc4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd933fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4c2c0 0x55589cc4c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42024/0 (socket says 172.21.15.69:42024) 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd932ffd640 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc59890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd933fff640 1 -- 172.21.15.69:0/3885415642 learned_addr learned my addr 172.21.15.69:0/3885415642 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.838+0000 7fd9337fe640 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4cbe0 0x55589cc56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.839+0000 7fd930ff9640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd91c0a2d70 0x7fd91c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.839+0000 7fd932ffd640 1 -- 172.21.15.69:0/3885415642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4cbe0 msgr2=0x55589cc56f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.839+0000 7fd932ffd640 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4cbe0 0x55589cc56f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.839+0000 7fd932ffd640 1 -- 172.21.15.69:0/3885415642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4c2c0 msgr2=0x55589cc4c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.839+0000 7fd932ffd640 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4c2c0 0x55589cc4c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.839+0000 7fd932ffd640 1 -- 172.21.15.69:0/3885415642 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589c923ba0 con 0x55589cc574a0 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.839+0000 7fd933fff640 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4c2c0 0x55589cc4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:23.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.839+0000 7fd930ff9640 1 -- 172.21.15.69:0/402779466 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fd9240cfd70 con 0x7fd934100b50 2024-08-02T18:02:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.840+0000 7fd932ffd640 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc59890 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd92800d6f0 tx=0x7fd92800dc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98c992aa50540d0d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.840+0000 7fd9337fe640 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4cbe0 0x55589cc56f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:23.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.840+0000 7fd919ffb640 1 -- 172.21.15.69:0/3885415642 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928015260 con 0x55589cc574a0 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.840+0000 7fd919ffb640 1 -- 172.21.15.69:0/3885415642 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd92801a070 con 0x55589cc574a0 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.840+0000 7fd919ffb640 1 -- 172.21.15.69:0/3885415642 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928009dc0 con 0x55589cc574a0 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.840+0000 7fd933fff640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd91c0a2d70 0x7fd91c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 -- 172.21.15.69:0/3885415642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 msgr2=0x55589cc59890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc59890 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd92800d6f0 tx=0x7fd92800dc50 comp rx=0 tx=0).stop 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 -- 172.21.15.69:0/3885415642 shutdown_connections 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc59890 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4cbe0 0x55589cc56f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 --2- 172.21.15.69:0/3885415642 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4c2c0 0x55589cc4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 -- 172.21.15.69:0/3885415642 >> 172.21.15.69:0/3885415642 conn(0x55589cc48850 msgr2=0x55589cc48c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:23.847 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 -- 172.21.15.69:0/3885415642 shutdown_connections 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd933fff640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd91c0a2d70 0x7fd91c0a5230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd92c004af0 tx=0x7fd92c0095c0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.841+0000 7fd93c613740 1 -- 172.21.15.69:0/3885415642 wait complete. 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.842+0000 7fd93c613740 1 -- start start 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.842+0000 7fd93c613740 1 Processor -- start 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.842+0000 7fd93c613740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc54480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.842+0000 7fd93c613740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc549c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.842+0000 7fd93c613740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc54f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.842+0000 7fd93c613740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55589cc59fb0 con 0x55589cc4cbe0 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.842+0000 7fd93c613740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55589cb6c9a0 con 0x55589cc574a0 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.842+0000 7fd93c613740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55589cc5a130 con 0x55589cc4c2c0 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd93c613740 10 client.0 _ll_register_callbacks cb 0x55589cb5ac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.848 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc54480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42040/0 (socket says 172.21.15.69:42040) 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc54480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59658/0 (socket says 172.21.15.69:59658) 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd932ffd640 1 -- 172.21.15.69:0/4225749636 learned_addr learned my addr 172.21.15.69:0/4225749636 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd9337fe640 1 -- 172.21.15.69:0/4225749636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 msgr2=0x55589cc54f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd9337fe640 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc54f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd9337fe640 1 -- 172.21.15.69:0/4225749636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 msgr2=0x55589cc549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd9337fe640 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd9337fe640 1 -- 172.21.15.69:0/4225749636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589cb78220 con 0x55589cc4c2c0 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd932ffd640 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd9337fe640 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc54480 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd9240cd070 tx=0x7fd92409ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ecf5535a08abbdf server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd912ffd640 1 -- 172.21.15.69:0/4225749636 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92409e030 con 0x55589cc4c2c0 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd912ffd640 1 -- 172.21.15.69:0/4225749636 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fd9240cd2f0 con 0x55589cc4c2c0 2024-08-02T18:02:23.849 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.843+0000 7fd93c613740 1 -- 172.21.15.69:0/4225749636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd92801a070 con 0x55589cc4c2c0 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.844+0000 7fd912ffd640 1 -- 172.21.15.69:0/4225749636 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9240cd4f0 con 0x55589cc4c2c0 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.844+0000 7fd93c613740 10 client.28453 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.844+0000 7fd93c613740 1 -- 172.21.15.69:0/4225749636 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55589cc65d20 con 0x55589cc4c2c0 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.844+0000 7fd912ffd640 1 -- 172.21.15.69:0/4225749636 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd92801a070 con 0x55589cc4c2c0 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.844+0000 7fd912ffd640 1 -- 172.21.15.69:0/4225749636 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd9240a4620 con 0x55589cc4c2c0 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.844+0000 7fd93c613740 10 client.28453 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.845+0000 7fd93c613740 10 client.28453 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.845+0000 7fd93c613740 20 client.28453 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.845+0000 7fd93c613740 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55589cc676f0 0x55589cc69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.845+0000 7fd93c613740 4 client.28453 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:23.850 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.845+0000 7fd93c613740 1 -- 172.21.15.69:0/4225749636 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55589cbbef40 con 0x55589cc676f0 2024-08-02T18:02:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.846+0000 7fd932ffd640 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55589cc676f0 0x55589cc69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.846+0000 7fd932ffd640 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55589cc676f0 0x55589cc69bb0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f3ea9ff591cad1f server_cookie=3ae53c1b54063229 in_seq=0 out_seq=0 2024-08-02T18:02:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.846+0000 7fd912ffd640 10 client.28453 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd912ffd640 1 -- 172.21.15.69:0/4225749636 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55589cbbef40 con 0x55589cc676f0 2024-08-02T18:02:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd912ffd640 10 client.28453 handle_command_reply: tid=0 2024-08-02T18:02:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 client.28453 shutdown 2024-08-02T18:02:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 -- 172.21.15.69:0/4225749636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 msgr2=0x55589cc54480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:23.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc54480 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd9240cd070 tx=0x7fd92409ba40 comp rx=0 tx=0).stop 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 -- 172.21.15.69:0/4225749636 shutdown_connections 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55589cc676f0 0x55589cc69bb0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc54f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 --2- 172.21.15.69:0/4225749636 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc54480 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 -- 172.21.15.69:0/4225749636 >> 172.21.15.69:0/4225749636 conn(0x55589cc48850 msgr2=0x55589cc47b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd912ffd640 0 client.28453 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 -- 172.21.15.69:0/4225749636 shutdown_connections 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 1 -- 172.21.15.69:0/4225749636 wait complete. 2024-08-02T18:02:23.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.847+0000 7fd93c613740 20 client.28453 trim_cache size 0 max 16384 2024-08-02T18:02:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd93c613740 1 Processor -- start 2024-08-02T18:02:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd93c613740 1 -- start start 2024-08-02T18:02:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd93c613740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd93c613740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc51c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd93c613740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.876 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:59670/0 (socket says 172.21.15.69:59670) 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc51c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd9337fe640 1 -- 172.21.15.69:0/4061416755 learned_addr learned my addr 172.21.15.69:0/4061416755 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd93c613740 1 -- 172.21.15.69:0/4061416755 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341af690 con 0x55589cc4cbe0 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd93c613740 1 -- 172.21.15.69:0/4061416755 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341acf60 con 0x55589cc574a0 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd93c613740 1 -- 172.21.15.69:0/4061416755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341aadb0 con 0x55589cc4c2c0 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.873+0000 7fd933fff640 1 --2- 172.21.15.69:0/4061416755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9337fe640 1 -- 172.21.15.69:0/4061416755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 msgr2=0x55589cc47310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9337fe640 1 --2- 172.21.15.69:0/4061416755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc47310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9337fe640 1 -- 172.21.15.69:0/4061416755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 msgr2=0x55589cc51c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9337fe640 1 --2- 172.21.15.69:0/4061416755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc51c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9337fe640 1 -- 172.21.15.69:0/4061416755 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55589cc4ca10 con 0x55589cc4c2c0 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9337fe640 1 --2- 172.21.15.69:0/4061416755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc516e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c6a0 tx=0x7fd92409c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981f8d7e1e88c801 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9197fa640 1 -- 172.21.15.69:0/4061416755 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240a01f0 con 0x55589cc4c2c0 2024-08-02T18:02:23.877 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9197fa640 1 -- 172.21.15.69:0/4061416755 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9240d32f0 con 0x55589cc4c2c0 2024-08-02T18:02:23.878 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.874+0000 7fd9197fa640 1 -- 172.21.15.69:0/4061416755 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240cd2f0 con 0x55589cc4c2c0 2024-08-02T18:02:23.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.875+0000 7fd93c613740 1 -- 172.21.15.69:0/4061416755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 msgr2=0x55589cc516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:23.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.875+0000 7fd93c613740 1 --2- 172.21.15.69:0/4061416755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc516e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c6a0 tx=0x7fd92409c680 comp rx=0 tx=0).stop 2024-08-02T18:02:23.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.875+0000 7fd93c613740 1 -- 172.21.15.69:0/4061416755 shutdown_connections 2024-08-02T18:02:23.880 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.875+0000 7fd93c613740 1 --2- 172.21.15.69:0/4061416755 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc51c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.875+0000 7fd93c613740 1 --2- 172.21.15.69:0/4061416755 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.875+0000 7fd93c613740 1 --2- 172.21.15.69:0/4061416755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc4c2c0 0x55589cc516e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.875+0000 7fd93c613740 1 -- 172.21.15.69:0/4061416755 >> 172.21.15.69:0/4061416755 conn(0x55589cc48850 msgr2=0x55589cc4f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.876+0000 7fd93c613740 1 -- 172.21.15.69:0/4061416755 shutdown_connections 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.876+0000 7fd93c613740 1 -- 172.21.15.69:0/4061416755 wait complete. 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.876+0000 7fd93c613740 1 -- start start 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.876+0000 7fd93c613740 1 Processor -- start 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd93c613740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc6f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd93c613740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc6fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd93c613740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc6cb80 0x55589cc6d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc6f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc6f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42064/0 (socket says 172.21.15.69:42064) 2024-08-02T18:02:23.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc6fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341ab420 con 0x55589cc4cbe0 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd9337fe640 1 -- 172.21.15.69:0/2183069852 learned_addr learned my addr 172.21.15.69:0/2183069852 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341ac890 con 0x55589cc574a0 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341abaf0 con 0x55589cc6cb80 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd93c613740 10 client.0 _ll_register_callbacks cb 0x7fd92400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.877+0000 7fd933fff640 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc6cb80 0x55589cc6d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd9337fe640 1 -- 172.21.15.69:0/2183069852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc6cb80 msgr2=0x55589cc6d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd9337fe640 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc6cb80 0x55589cc6d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd9337fe640 1 -- 172.21.15.69:0/2183069852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 msgr2=0x55589cc6fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd9337fe640 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc6fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd9337fe640 1 -- 172.21.15.69:0/2183069852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55589cc6d800 con 0x55589cc4cbe0 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd932ffd640 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc6fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:23.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd9337fe640 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc6f720 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd92400cfd0 tx=0x7fd92409ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937ee09a0f9e1d5a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:23.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:23.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd912ffd640 1 -- 172.21.15.69:0/2183069852 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9240a4ab0 con 0x55589cc4cbe0 2024-08-02T18:02:23.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd912ffd640 1 -- 172.21.15.69:0/2183069852 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fd9240cd2f0 con 0x55589cc4cbe0 2024-08-02T18:02:23.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.878+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55589cbb9290 con 0x55589cc4cbe0 2024-08-02T18:02:23.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd912ffd640 1 -- 172.21.15.69:0/2183069852 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd9240a01f0 con 0x55589cc4cbe0 2024-08-02T18:02:23.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd93c613740 10 client.19323 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:23.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55589ccd23b0 con 0x55589cc4cbe0 2024-08-02T18:02:23.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd912ffd640 1 -- 172.21.15.69:0/2183069852 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd924097b90 con 0x55589cc4cbe0 2024-08-02T18:02:23.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd912ffd640 1 -- 172.21.15.69:0/2183069852 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd92409ad00 con 0x55589cc4cbe0 2024-08-02T18:02:23.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd93c613740 10 client.19323 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:23.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd93c613740 10 client.19323 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:02:23.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd93c613740 20 client.19323 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:23.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd93c613740 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55589ccd28a0 0x55589ccd4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:23.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd93c613740 4 client.19323 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.95, "await_for": 45.0}] 2024-08-02T18:02:23.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.879+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.95, "await_for": 45.0}) -- 0x55589cbb9290 con 0x55589ccd28a0 2024-08-02T18:02:23.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.880+0000 7fd932ffd640 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55589ccd28a0 0x55589ccd4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:23.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.880+0000 7fd912ffd640 10 client.19323 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:23.885 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:23.881+0000 7fd932ffd640 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55589ccd28a0 0x55589ccd4c90 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a528a40b89b4462 server_cookie=c5930a7bdae19559 in_seq=0 out_seq=0 2024-08-02T18:02:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.214+0000 7f5e01ffb640 1 -- 172.21.15.69:0/2510178044 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5630867b3430 con 0x5630868cd400 2024-08-02T18:02:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.214+0000 7f5e01ffb640 10 client.19305 handle_command_reply: tid=0 2024-08-02T18:02:24.216 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.214+0000 7f5e23038740 1 client.19305 shutdown 2024-08-02T18:02:24.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.214+0000 7fd912ffd640 1 -- 172.21.15.69:0/2183069852 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55589cbb9290 con 0x55589ccd28a0 2024-08-02T18:02:24.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.214+0000 7fd912ffd640 10 client.19323 handle_command_reply: tid=0 2024-08-02T18:02:24.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.214+0000 7fd93c613740 1 client.19323 shutdown 2024-08-02T18:02:24.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.214+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 msgr2=0x55589cc6f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:24.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.214+0000 7fd93c613740 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc6f720 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd92400cfd0 tx=0x7fd92409ba40 comp rx=0 tx=0).stop 2024-08-02T18:02:24.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.214+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 shutdown_connections 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.214+0000 7fd93c613740 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55589ccd28a0 0x55589ccd4c90 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.214+0000 7fd93c613740 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55589cc6cb80 0x55589cc6d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93c613740 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55589cc574a0 0x55589cc6fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93c613740 1 --2- 172.21.15.69:0/2183069852 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55589cc4cbe0 0x55589cc6f720 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd912ffd640 0 client.19323 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 >> 172.21.15.69:0/2183069852 conn(0x55589cc48850 msgr2=0x55589cc47310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 shutdown_connections 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93c613740 1 -- 172.21.15.69:0/2183069852 wait complete. 2024-08-02T18:02:24.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93c613740 20 client.19323 trim_cache size 0 max 16384 2024-08-02T18:02:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 msgr2=0x56308685aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308685aee0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f5e080d7fd0 tx=0x7f5e08010f80 comp rx=0 tx=0).stop 2024-08-02T18:02:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 shutdown_connections 2024-08-02T18:02:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5630868cd400 0x5630868cf7f0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x563086852970 0x56308684c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e01ffb640 0 client.19305 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5630868480d0 0x56308684c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 --2- 172.21.15.69:0/2510178044 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5630868477b0 0x56308685aee0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 >> 172.21.15.69:0/2510178044 conn(0x563086843d20 msgr2=0x563086849210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 shutdown_connections 2024-08-02T18:02:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 1 -- 172.21.15.69:0/2510178044 wait complete. 2024-08-02T18:02:24.220 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:24.220 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 27, 2024-08-02T18:02:24.220 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 54, 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "f6c7550a": { 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 54, 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:24.221 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 147.9, 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:24.222 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:02:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.215+0000 7f5e23038740 20 client.19305 trim_cache size 0 max 16384 2024-08-02T18:02:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:02:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:02:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:02:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:02:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd91c0a2d70 msgr2=0x7fd91c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd91c0a2d70 0x7fd91c0a5230 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd92c004af0 tx=0x7fd92c0095c0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934100b50 msgr2=0x7fd934196050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934100b50 0x7fd934196050 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd92400e9f0 tx=0x7fd92400edf0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.224 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 shutdown_connections 2024-08-02T18:02:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd9341022a0 0x7fd934196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd91c0a2d70 0x7fd91c0a5230 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd934102c20 0x7fd93419a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.215+0000 7fd93a215640 1 --2- 172.21.15.69:0/402779466 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd934100b50 0x7fd934196050 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.216+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 >> 172.21.15.69:0/402779466 conn(0x7fd9340fc500 msgr2=0x7fd934104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.216+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 shutdown_connections 2024-08-02T18:02:24.225 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:24.216+0000 7fd93a215640 1 -- 172.21.15.69:0/402779466 wait complete. 2024-08-02T18:02:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 -- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e040a2e40 msgr2=0x7f5e040a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e040a2e40 0x7f5e040a5300 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5e1402a8d0 tx=0x7f5e14005c50 comp rx=0 tx=0).stop 2024-08-02T18:02:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 -- 172.21.15.69:0/2078372127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e18108f20 msgr2=0x7f5e181a6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e18108f20 0x7f5e181a6f10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5e0800b3e0 tx=0x7f5e0800b8a0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 -- 172.21.15.69:0/2078372127 shutdown_connections 2024-08-02T18:02:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f5e18109300 0x7f5e181a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f5e040a2e40 0x7f5e040a5300 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5e1811ae70 0x7f5e181ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 --2- 172.21.15.69:0/2078372127 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5e18108f20 0x7f5e181a6f10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 -- 172.21.15.69:0/2078372127 >> 172.21.15.69:0/2078372127 conn(0x7f5e1806cb20 msgr2=0x7f5e1811b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 -- 172.21.15.69:0/2078372127 shutdown_connections 2024-08-02T18:02:24.228 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:24.216+0000 7f5e202da640 1 -- 172.21.15.69:0/2078372127 wait complete. 2024-08-02T18:02:24.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:24 smithi069 ceph-mon[32746]: pgmap v502: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.7 MiB/s wr, 662 op/s 2024-08-02T18:02:25.096 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f6c7550a', quiesce took 12.0 seconds. Will release after: 15.95 2024-08-02T18:02:25.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:24 smithi077 ceph-mon[37108]: pgmap v502: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.7 MiB/s wr, 662 op/s 2024-08-02T18:02:25.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:24 smithi083 ceph-mon[37031]: pgmap v502: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 5.7 MiB/s wr, 662 op/s 2024-08-02T18:02:27.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:26 smithi077 ceph-mon[37108]: pgmap v503: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 5.3 MiB/s wr, 617 op/s 2024-08-02T18:02:27.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:26 smithi069 ceph-mon[32746]: pgmap v503: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 5.3 MiB/s wr, 617 op/s 2024-08-02T18:02:27.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:26 smithi083 ceph-mon[37031]: pgmap v503: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 5.3 MiB/s wr, 617 op/s 2024-08-02T18:02:29.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:28 smithi077 ceph-mon[37108]: pgmap v504: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-08-02T18:02:29.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:28 smithi069 ceph-mon[32746]: pgmap v504: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-08-02T18:02:29.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:28 smithi083 ceph-mon[37031]: pgmap v504: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-08-02T18:02:31.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:30 smithi077 ceph-mon[37108]: pgmap v505: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.6 MiB/s wr, 532 op/s 2024-08-02T18:02:31.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:30 smithi069 ceph-mon[32746]: pgmap v505: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.6 MiB/s wr, 532 op/s 2024-08-02T18:02:31.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:30 smithi083 ceph-mon[37031]: pgmap v505: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 4.6 MiB/s wr, 532 op/s 2024-08-02T18:02:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:32 smithi069 ceph-mon[32746]: pgmap v506: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-08-02T18:02:33.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:32 smithi083 ceph-mon[37031]: pgmap v506: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-08-02T18:02:33.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:32 smithi077 ceph-mon[37108]: pgmap v506: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 2.6 MiB/s wr, 325 op/s 2024-08-02T18:02:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:34 smithi069 ceph-mon[32746]: pgmap v507: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-08-02T18:02:35.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:34 smithi083 ceph-mon[37031]: pgmap v507: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-08-02T18:02:35.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:34 smithi077 ceph-mon[37108]: pgmap v507: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.6 MiB/s wr, 329 op/s 2024-08-02T18:02:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:36 smithi069 ceph-mon[32746]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 474 KiB/s wr, 64 op/s 2024-08-02T18:02:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:36 smithi083 ceph-mon[37031]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 474 KiB/s wr, 64 op/s 2024-08-02T18:02:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 -- 172.21.15.69:0/3108650836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c41006d0 msgr2=0x7fd4c4100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/3108650836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c41006d0 0x7fd4c4100b30 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009930 tx=0x7fd4c002a420 comp rx=0 tx=0).stop 2024-08-02T18:02:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 -- 172.21.15.69:0/3108650836 shutdown_connections 2024-08-02T18:02:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/3108650836 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c41006d0 0x7fd4c4100b30 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/3108650836 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c4101070 0x7fd4c410d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/3108650836 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4c41066e0 0x7fd4c4106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 -- 172.21.15.69:0/3108650836 >> 172.21.15.69:0/3108650836 conn(0x7fd4c40776d0 msgr2=0x7fd4c40fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 -- 172.21.15.69:0/3108650836 shutdown_connections 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.314+0000 7fd4cbafe640 1 -- 172.21.15.69:0/3108650836 wait complete. 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4cbafe640 1 Processor -- start 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4cbafe640 1 -- start start 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4cbafe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c41006d0 0x7fd4c4197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4cbafe640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101070 0x7fd4c41975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4cbafe640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4c41066e0 0x7fd4c419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4cbafe640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c4110b20 con 0x7fd4c4101070 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4cbafe640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c41109a0 con 0x7fd4c41066e0 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4cbafe640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c4110ca0 con 0x7fd4c41006d0 2024-08-02T18:02:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4c9099640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101070 0x7fd4c41975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4c9099640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101070 0x7fd4c41975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47558/0 (socket says 172.21.15.69:47558) 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4c9099640 1 -- 172.21.15.69:0/463815351 learned_addr learned my addr 172.21.15.69:0/463815351 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4c989a640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c41006d0 0x7fd4c4197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.315+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4c41066e0 0x7fd4c419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4c9099640 1 -- 172.21.15.69:0/463815351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c41006d0 msgr2=0x7fd4c4197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4c9099640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c41006d0 0x7fd4c4197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4c9099640 1 -- 172.21.15.69:0/463815351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4c41066e0 msgr2=0x7fd4c419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4c9099640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4c41066e0 0x7fd4c419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4c9099640 1 -- 172.21.15.69:0/463815351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c419c060 con 0x7fd4c4101070 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4c9099640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101070 0x7fd4c41975b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009230 tx=0x7fd4c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4baffd640 1 -- 172.21.15.69:0/463815351 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c003f030 con 0x7fd4c4101070 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4baffd640 1 -- 172.21.15.69:0/463815351 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4c0003ad0 con 0x7fd4c4101070 2024-08-02T18:02:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4cbafe640 1 -- 172.21.15.69:0/463815351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c419c2f0 con 0x7fd4c4101070 2024-08-02T18:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4baffd640 1 -- 172.21.15.69:0/463815351 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0035070 con 0x7fd4c4101070 2024-08-02T18:02:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.316+0000 7fd4cbafe640 1 -- 172.21.15.69:0/463815351 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c4069740 con 0x7fd4c4101070 2024-08-02T18:02:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.318+0000 7fd4baffd640 1 -- 172.21.15.69:0/463815351 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd4c0042030 con 0x7fd4c4101070 2024-08-02T18:02:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4ce852740 1 Processor -- start 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4ce852740 1 -- start start 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4baffd640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd4b00a2cf0 0x7fd4b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583048106d0 0x558304810ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558304810ff0 0x55830481f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 0x558304822170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4ce852740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5583047749b0 con 0x5583048106d0 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4ce852740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55830471c800 con 0x55830481fd80 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.319+0000 7fd4ce852740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558304770970 con 0x558304810ff0 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd4b00a2cf0 0x7fd4b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4baffd640 1 -- 172.21.15.69:0/463815351 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fd4c00efd50 con 0x7fd4c4101070 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583048106d0 0x558304810ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c9099640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558304810ff0 0x55830481f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583048106d0 0x558304810ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47564/0 (socket says 172.21.15.69:47564) 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 -- 172.21.15.69:0/2266580968 learned_addr learned my addr 172.21.15.69:0/2266580968 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 0x558304822170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 -- 172.21.15.69:0/2266580968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558304810ff0 msgr2=0x55830481f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558304810ff0 0x55830481f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 -- 172.21.15.69:0/2266580968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 msgr2=0x558304822170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 0x558304822170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 -- 172.21.15.69:0/2266580968 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558304488210 con 0x5583048106d0 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c9099640 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558304810ff0 0x55830481f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd4b00a2cf0 0x7fd4b00a51b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005fd0 tx=0x7fd4bc014ab0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4c989a640 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583048106d0 0x558304810ab0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400bc90 tx=0x7fd4b40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79576a014e852da5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4affff640 1 -- 172.21.15.69:0/2266580968 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b401c480 con 0x5583048106d0 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4affff640 1 -- 172.21.15.69:0/2266580968 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b401cb00 con 0x5583048106d0 2024-08-02T18:02:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.320+0000 7fd4affff640 1 -- 172.21.15.69:0/2266580968 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4011b40 con 0x5583048106d0 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 -- 172.21.15.69:0/2266580968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583048106d0 msgr2=0x558304810ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583048106d0 0x558304810ab0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd4b400bc90 tx=0x7fd4b40107a0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 -- 172.21.15.69:0/2266580968 shutdown_connections 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 0x558304822170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x558304810ff0 0x55830481f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 --2- 172.21.15.69:0/2266580968 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5583048106d0 0x558304810ab0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 -- 172.21.15.69:0/2266580968 >> 172.21.15.69:0/2266580968 conn(0x55830480cd20 msgr2=0x55830480d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 -- 172.21.15.69:0/2266580968 shutdown_connections 2024-08-02T18:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.321+0000 7fd4ce852740 1 -- 172.21.15.69:0/2266580968 wait complete. 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 1 -- start start 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 1 Processor -- start 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583048106d0 0x55830481cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x55830481d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55830481fd80 0x55830481d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x558304822950 con 0x558304810ff0 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x558304730500 con 0x5583048106d0 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x558304822ad0 con 0x55830481fd80 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4ce852740 10 client.0 _ll_register_callbacks cb 0x558304487a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4c9099640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x55830481d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4c9099640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x55830481d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47574/0 (socket says 172.21.15.69:47574) 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4c9099640 1 -- 172.21.15.69:0/157063449 learned_addr learned my addr 172.21.15.69:0/157063449 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.322+0000 7fd4c989a640 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583048106d0 0x55830481cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55830481fd80 0x55830481d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4c9099640 1 -- 172.21.15.69:0/157063449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55830481fd80 msgr2=0x55830481d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4c9099640 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55830481fd80 0x55830481d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4c9099640 1 -- 172.21.15.69:0/157063449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583048106d0 msgr2=0x55830481cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4c9099640 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583048106d0 0x55830481cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4c9099640 1 -- 172.21.15.69:0/157063449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558304554620 con 0x558304810ff0 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55830481fd80 0x55830481d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4c9099640 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x55830481d1e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00bbdf0 tx=0x7fd4c00bc560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dcddd3db5b9709e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/157063449 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0049040 con 0x558304810ff0 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/157063449 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fd4c002fdb0 con 0x558304810ff0 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ce852740 1 -- 172.21.15.69:0/157063449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4b401cb00 con 0x558304810ff0 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/157063449 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd4c00bbe60 con 0x558304810ff0 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ce852740 10 client.19335 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ce852740 1 -- 172.21.15.69:0/157063449 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55830482e5d0 con 0x558304810ff0 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/157063449 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b401cb00 con 0x558304810ff0 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/157063449 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd4c004a5e0 con 0x558304810ff0 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ce852740 10 client.19335 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ce852740 10 client.19335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.323+0000 7fd4ce852740 20 client.19335 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.324+0000 7fd4ce852740 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55830482fa80 0x558304831eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.324+0000 7fd4ce852740 4 client.19335 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.324+0000 7fd4ce852740 1 -- 172.21.15.69:0/157063449 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583047820e0 con 0x55830482fa80 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.324+0000 7fd4c989a640 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55830482fa80 0x558304831eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.324+0000 7fd4c989a640 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55830482fa80 0x558304831eb0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769643c3ac1ea48b server_cookie=ff088bd9180c911c in_seq=0 out_seq=0 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.324+0000 7fd4ad7fa640 10 client.19335 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/157063449 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583047820e0 con 0x55830482fa80 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ad7fa640 10 client.19335 handle_command_reply: tid=0 2024-08-02T18:02:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 client.19335 shutdown 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 -- 172.21.15.69:0/157063449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 msgr2=0x55830481d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x55830481d1e0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00bbdf0 tx=0x7fd4c00bc560 comp rx=0 tx=0).stop 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 -- 172.21.15.69:0/157063449 shutdown_connections 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x55830481d1e0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55830482fa80 0x558304831eb0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55830481fd80 0x55830481d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 --2- 172.21.15.69:0/157063449 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5583048106d0 0x55830481cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ad7fa640 0 client.19335 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 -- 172.21.15.69:0/157063449 >> 172.21.15.69:0/157063449 conn(0x55830480cd20 msgr2=0x5583048129e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 -- 172.21.15.69:0/157063449 shutdown_connections 2024-08-02T18:02:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 1 -- 172.21.15.69:0/157063449 wait complete. 2024-08-02T18:02:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.325+0000 7fd4ce852740 20 client.19335 trim_cache size 0 max 16384 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ce852740 1 Processor -- start 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ce852740 1 -- start start 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 0x558304837730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558304810ff0 0x558304837c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55830481fd80 0x558304819ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ce852740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c41af4f0 con 0x55830481fd80 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ce852740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c41acdc0 con 0x558304810ff0 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ce852740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c41aac10 con 0x5583048106d0 2024-08-02T18:02:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55830481fd80 0x558304819ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55830481fd80 0x558304819ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47582/0 (socket says 172.21.15.69:47582) 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 -- 172.21.15.69:0/58523229 learned_addr learned my addr 172.21.15.69:0/58523229 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4c9099640 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558304810ff0 0x558304837c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4c989a640 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 0x558304837730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 -- 172.21.15.69:0/58523229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 msgr2=0x558304837730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 0x558304837730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 -- 172.21.15.69:0/58523229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558304810ff0 msgr2=0x558304837c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558304810ff0 0x558304837c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 -- 172.21.15.69:0/58523229 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583048224b0 con 0x55830481fd80 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4c989a640 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 0x558304837730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.341+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55830481fd80 0x558304819ee0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005f40 tx=0x7fd4bc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43c505cd6924c7c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4af7fe640 1 -- 172.21.15.69:0/58523229 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc025070 con 0x55830481fd80 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4af7fe640 1 -- 172.21.15.69:0/58523229 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc021b00 con 0x55830481fd80 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4af7fe640 1 -- 172.21.15.69:0/58523229 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc013c20 con 0x55830481fd80 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 -- 172.21.15.69:0/58523229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55830481fd80 msgr2=0x558304819ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55830481fd80 0x558304819ee0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005f40 tx=0x7fd4bc01d000 comp rx=0 tx=0).stop 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 -- 172.21.15.69:0/58523229 shutdown_connections 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55830481fd80 0x558304819ee0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x558304810ff0 0x558304837c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 --2- 172.21.15.69:0/58523229 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 0x558304837730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 -- 172.21.15.69:0/58523229 >> 172.21.15.69:0/58523229 conn(0x55830480cd20 msgr2=0x55830481ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 -- 172.21.15.69:0/58523229 shutdown_connections 2024-08-02T18:02:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.342+0000 7fd4ce852740 1 -- 172.21.15.69:0/58523229 wait complete. 2024-08-02T18:02:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 1 -- start start 2024-08-02T18:02:37.347 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 1 Processor -- start 2024-08-02T18:02:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 0x558304827d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x5583048282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 0x5583048146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.349 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c41ab280 con 0x558304810ff0 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c41ac6f0 con 0x55830481fd80 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c41ab950 con 0x5583048106d0 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ce852740 10 client.0 _ll_register_callbacks cb 0x7fd4c0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4c9099640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x5583048282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4c9099640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x5583048282a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47596/0 (socket says 172.21.15.69:47596) 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4c9099640 1 -- 172.21.15.69:0/406546279 learned_addr learned my addr 172.21.15.69:0/406546279 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4ca09b640 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 0x5583048146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4c9099640 1 -- 172.21.15.69:0/406546279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 msgr2=0x558304827d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4c9099640 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 0x558304827d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4c9099640 1 -- 172.21.15.69:0/406546279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 msgr2=0x5583048146f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4c9099640 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 0x5583048146f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.343+0000 7fd4c9099640 1 -- 172.21.15.69:0/406546279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558304817460 con 0x558304810ff0 2024-08-02T18:02:37.350 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4c9099640 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x5583048282a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00dc0d0 tx=0x7fd4c00bc320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c58a344ed2ddc73 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/406546279 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0049040 con 0x558304810ff0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/406546279 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fd4c00b84d0 con 0x558304810ff0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 1 -- 172.21.15.69:0/406546279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4bc021b00 con 0x558304810ff0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/406546279 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd4c004a260 con 0x558304810ff0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 10 client.19341 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 1 -- 172.21.15.69:0/406546279 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55830489f350 con 0x558304810ff0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/406546279 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4bc021b00 con 0x558304810ff0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/406546279 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd4c00c4640 con 0x558304810ff0 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 10 client.19341 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:37.351 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 10 client.19341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 20 client.19341 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55830489f8f0 0x5583048a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 4 client.19341 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:02:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.344+0000 7fd4ce852740 1 -- 172.21.15.69:0/406546279 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55830477c430 con 0x55830489f8f0 2024-08-02T18:02:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.345+0000 7fd4c989a640 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55830489f8f0 0x5583048a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.345+0000 7fd4ad7fa640 10 client.19341 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:37.352 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.345+0000 7fd4c989a640 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55830489f8f0 0x5583048a1d10 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77692f0c577cc1d8 server_cookie=955c38b15daf268 in_seq=0 out_seq=0 2024-08-02T18:02:37.352 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:36 smithi077 ceph-mon[37108]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 474 KiB/s wr, 64 op/s 2024-08-02T18:02:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ad7fa640 1 -- 172.21.15.69:0/406546279 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55830477c430 con 0x55830489f8f0 2024-08-02T18:02:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ad7fa640 10 client.19341 handle_command_reply: tid=0 2024-08-02T18:02:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 client.19341 shutdown 2024-08-02T18:02:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 -- 172.21.15.69:0/406546279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 msgr2=0x5583048282a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x5583048282a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00dc0d0 tx=0x7fd4c00bc320 comp rx=0 tx=0).stop 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 -- 172.21.15.69:0/406546279 shutdown_connections 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x558304810ff0 0x5583048282a0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55830489f8f0 0x5583048a1d10 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ad7fa640 0 client.19341 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55830481fd80 0x5583048146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 --2- 172.21.15.69:0/406546279 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5583048106d0 0x558304827d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 -- 172.21.15.69:0/406546279 >> 172.21.15.69:0/406546279 conn(0x55830480cd20 msgr2=0x55830481e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 -- 172.21.15.69:0/406546279 shutdown_connections 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 1 -- 172.21.15.69:0/406546279 wait complete. 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.374+0000 7fd4ce852740 20 client.19341 trim_cache size 0 max 16384 2024-08-02T18:02:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:02:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:02:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:02:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 -- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd4b00a2cf0 msgr2=0x7fd4b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd4b00a2cf0 0x7fd4b00a51b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc005fd0 tx=0x7fd4bc014ab0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 -- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101070 msgr2=0x7fd4c41975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101070 0x7fd4c41975b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009230 tx=0x7fd4c0004270 comp rx=0 tx=0).stop 2024-08-02T18:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 -- 172.21.15.69:0/463815351 shutdown_connections 2024-08-02T18:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd4c4101070 0x7fd4c41975b0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd4b00a2cf0 0x7fd4b00a51b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd4c41066e0 0x7fd4c419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 --2- 172.21.15.69:0/463815351 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd4c41006d0 0x7fd4c4197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 -- 172.21.15.69:0/463815351 >> 172.21.15.69:0/463815351 conn(0x7fd4c40776d0 msgr2=0x7fd4c41035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 -- 172.21.15.69:0/463815351 shutdown_connections 2024-08-02T18:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:37.375+0000 7fd4cbafe640 1 -- 172.21.15.69:0/463815351 wait complete. 2024-08-02T18:02:39.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:38 smithi069 ceph-mon[32746]: pgmap v509: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 365 KiB/s wr, 50 op/s 2024-08-02T18:02:39.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:38 smithi083 ceph-mon[37031]: pgmap v509: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 365 KiB/s wr, 50 op/s 2024-08-02T18:02:39.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:38 smithi077 ceph-mon[37108]: pgmap v509: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 365 KiB/s wr, 50 op/s 2024-08-02T18:02:39.941 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-02T18:02:40.211 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 -- 172.21.15.69:0/2486022170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f4101150 msgr2=0x7fe4f410d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/2486022170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f4101150 0x7fe4f410d680 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009cd0 tx=0x7fe4ec02a8e0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 -- 172.21.15.69:0/2486022170 shutdown_connections 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/2486022170 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f4101150 0x7fe4f410d680 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/2486022170 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f41007b0 0x7fe4f4100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/2486022170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4f41067b0 0x7fe4f4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 -- 172.21.15.69:0/2486022170 >> 172.21.15.69:0/2486022170 conn(0x7fe4f40fc4d0 msgr2=0x7fe4f40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 -- 172.21.15.69:0/2486022170 shutdown_connections 2024-08-02T18:02:40.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.805+0000 7fe4f99f1640 1 -- 172.21.15.69:0/2486022170 wait complete. 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f99f1640 1 Processor -- start 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f99f1640 1 -- start start 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f99f1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f41007b0 0x7fe4f410e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f99f1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4101150 0x7fe4f410e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f99f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4f41067b0 0x7fe4f410eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f99f1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4111cd0 con 0x7fe4f41007b0 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f99f1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4111b50 con 0x7fe4f4101150 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f99f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f4111e50 con 0x7fe4f41067b0 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f41007b0 0x7fe4f410e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f41007b0 0x7fe4f410e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47612/0 (socket says 172.21.15.69:47612) 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 -- 172.21.15.69:0/1301560399 learned_addr learned my addr 172.21.15.69:0/1301560399 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4101150 0x7fe4f410e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 -- 172.21.15.69:0/1301560399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4f41067b0 msgr2=0x7fe4f410eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4f41067b0 0x7fe4f410eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 -- 172.21.15.69:0/1301560399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4101150 msgr2=0x7fe4f410e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4101150 0x7fe4f410e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 -- 172.21.15.69:0/1301560399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f41a6580 con 0x7fe4f41007b0 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.806+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f41007b0 0x7fe4f410e110 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400c970 tx=0x7fe4e400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.807+0000 7fe4f89ef640 1 -- 172.21.15.69:0/1301560399 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e4007bb0 con 0x7fe4f41007b0 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.807+0000 7fe4f99f1640 1 -- 172.21.15.69:0/1301560399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f41a6870 con 0x7fe4f41007b0 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.807+0000 7fe4f89ef640 1 -- 172.21.15.69:0/1301560399 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e4007d50 con 0x7fe4f41007b0 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.807+0000 7fe4f89ef640 1 -- 172.21.15.69:0/1301560399 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400f450 con 0x7fe4f41007b0 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.807+0000 7fe4f99f1640 1 -- 172.21.15.69:0/1301560399 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f41a6db0 con 0x7fe4f41007b0 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4f89ef640 1 -- 172.21.15.69:0/1301560399 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe4e4005020 con 0x7fe4f41007b0 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4fbdef740 1 Processor -- start 2024-08-02T18:02:40.821 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4fbdef740 1 -- start start 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd6505f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd650b30 0x5638dd65aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd65b3f0 0x5638dd65d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5638dd5b5980 con 0x5638dd65b3f0 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638dd54da70 con 0x5638dd650b30 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5638dd5b1940 con 0x5638dd650210 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4ea7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd65b3f0 0x5638dd65d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4f2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd650b30 0x5638dd65aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4ea7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd65b3f0 0x5638dd65d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47618/0 (socket says 172.21.15.69:47618) 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4f2ffd640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd650b30 0x5638dd65aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:33556/0 (socket says 172.21.15.69:33556) 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/2259538263 learned_addr learned my addr 172.21.15.69:0/2259538263 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.809+0000 7fe4f37fe640 1 --2- 172.21.15.69:0/2259538263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd6505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.822 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4f89ef640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4e00a2cf0 0x7fe4e00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/2259538263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 msgr2=0x5638dd6505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/2259538263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd6505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/2259538263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd650b30 msgr2=0x5638dd65aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/2259538263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd650b30 0x5638dd65aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/2259538263 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638dd5b06e0 con 0x5638dd65b3f0 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4f37fe640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4e00a2cf0 0x7fe4e00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4f89ef640 1 -- 172.21.15.69:0/1301560399 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fe4e40cf680 con 0x7fe4f41007b0 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/2259538263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd65b3f0 0x5638dd65d7e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00cc40 tx=0x7fe4dc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac866532a4d6d7c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4e9ffb640 1 -- 172.21.15.69:0/2259538263 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc014070 con 0x5638dd65b3f0 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4e9ffb640 1 -- 172.21.15.69:0/2259538263 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc0047a0 con 0x5638dd65b3f0 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.810+0000 7fe4e9ffb640 1 -- 172.21.15.69:0/2259538263 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc010790 con 0x5638dd65b3f0 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2259538263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd65b3f0 msgr2=0x5638dd65d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/2259538263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd65b3f0 0x5638dd65d7e0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00cc40 tx=0x7fe4dc00e720 comp rx=0 tx=0).stop 2024-08-02T18:02:40.823 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4f37fe640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4e00a2cf0 0x7fe4e00a51b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009430 tx=0x7fe4ec009360 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2259538263 shutdown_connections 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/2259538263 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd65b3f0 0x5638dd65d7e0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/2259538263 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd650b30 0x5638dd65aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/2259538263 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd6505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2259538263 >> 172.21.15.69:0/2259538263 conn(0x5638dd64c7a0 msgr2=0x5638dd64cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2259538263 shutdown_connections 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2259538263 wait complete. 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- start start 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 Processor -- start 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd658330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd658870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd658db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5638dd65dfc0 con 0x5638dd650b30 2024-08-02T18:02:40.824 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5638dd570aa0 con 0x5638dd65b3f0 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.811+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5638dd65e140 con 0x5638dd650210 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4fbdef740 10 client.0 _ll_register_callbacks cb 0x5638dd55fe60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd658870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd658870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47634/0 (socket says 172.21.15.69:47634) 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/1535257717 learned_addr learned my addr 172.21.15.69:0/1535257717 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd658330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4f37fe640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd658db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/1535257717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 msgr2=0x5638dd658330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd658330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 msgr2=0x5638dd658db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd658db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.825 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/1535257717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638dd554650 con 0x5638dd650b30 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4f37fe640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd658db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd658330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd658870 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005da0 tx=0x7fe4dc005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d0226aa5c92888 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/1535257717 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc014030 con 0x5638dd650b30 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/1535257717 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fe4dc004050 con 0x5638dd650b30 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4fbdef740 1 -- 172.21.15.69:0/1535257717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4dc0047a0 con 0x5638dd650b30 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/1535257717 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4dc005590 con 0x5638dd650b30 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4fbdef740 10 client.19353 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.812+0000 7fe4fbdef740 1 -- 172.21.15.69:0/1535257717 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638dd3273c0 con 0x5638dd650b30 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/1535257717 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc0047a0 con 0x5638dd650b30 2024-08-02T18:02:40.826 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/1535257717 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe4dc003bf0 con 0x5638dd650b30 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4fbdef740 10 client.19353 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4fbdef740 10 client.19353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4fbdef740 20 client.19353 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638dd66ae60 0x5638dd66d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4fbdef740 4 client.19353 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4fbdef740 1 -- 172.21.15.69:0/1535257717 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638dd5c2ef0 con 0x5638dd66ae60 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638dd66ae60 0x5638dd66d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4d2ffd640 10 client.19353 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.813+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638dd66ae60 0x5638dd66d250 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6066f3b48cb1623 server_cookie=e8dab319b8e192db in_seq=0 out_seq=0 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/1535257717 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5638dd5c2ef0 con 0x5638dd66ae60 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4d2ffd640 10 client.19353 handle_command_reply: tid=0 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 client.19353 shutdown 2024-08-02T18:02:40.827 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 -- 172.21.15.69:0/1535257717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 msgr2=0x5638dd658870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd658870 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc005da0 tx=0x7fe4dc005210 comp rx=0 tx=0).stop 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 -- 172.21.15.69:0/1535257717 shutdown_connections 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd658870 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638dd66ae60 0x5638dd66d250 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd658db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4d2ffd640 0 client.19353 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/1535257717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd658330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 -- 172.21.15.69:0/1535257717 >> 172.21.15.69:0/1535257717 conn(0x5638dd64c7a0 msgr2=0x5638dd652480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 -- 172.21.15.69:0/1535257717 shutdown_connections 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.814+0000 7fe4fbdef740 1 -- 172.21.15.69:0/1535257717 wait complete. 2024-08-02T18:02:40.828 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.815+0000 7fe4fbdef740 20 client.19353 trim_cache size 0 max 16384 2024-08-02T18:02:40.834 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4fbdef740 1 Processor -- start 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4fbdef740 1 -- start start 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd672a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd672f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd64b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41b39d0 con 0x5638dd650b30 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41b12a0 con 0x5638dd65b3f0 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41af0f0 con 0x5638dd650210 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4ea7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd672f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4ea7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd672f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47646/0 (socket says 172.21.15.69:47646) 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/2374576273 learned_addr learned my addr 172.21.15.69:0/2374576273 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd672a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.835 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.832+0000 7fe4f37fe640 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd64b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/2374576273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 msgr2=0x5638dd672a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd672a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/2374576273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 msgr2=0x5638dd64b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd64b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/2374576273 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638dd66d7e0 con 0x5638dd650b30 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd672f50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc000c00 tx=0x7fe4dc005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a5a5178d61a89 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4e97fa640 1 -- 172.21.15.69:0/2374576273 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc014030 con 0x5638dd650b30 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4e97fa640 1 -- 172.21.15.69:0/2374576273 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc01c020 con 0x5638dd650b30 2024-08-02T18:02:40.836 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.833+0000 7fe4e97fa640 1 -- 172.21.15.69:0/2374576273 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc003d30 con 0x5638dd650b30 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2374576273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 msgr2=0x5638dd672f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd672f50 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc000c00 tx=0x7fe4dc005280 comp rx=0 tx=0).stop 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2374576273 shutdown_connections 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd672f50 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd64b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/2374576273 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd672a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2374576273 >> 172.21.15.69:0/2374576273 conn(0x5638dd64c7a0 msgr2=0x5638dd652480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2374576273 shutdown_connections 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- 172.21.15.69:0/2374576273 wait complete. 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- start start 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 Processor -- start 2024-08-02T18:02:40.840 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd65e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd65e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd65edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41af760 con 0x5638dd650b30 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41b0bd0 con 0x5638dd65b3f0 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f41afe30 con 0x5638dd650210 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4fbdef740 10 client.0 _ll_register_callbacks cb 0x7fe4e40d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.834+0000 7fe4ea7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd65e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4f2ffd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd65e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4ea7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd65e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47656/0 (socket says 172.21.15.69:47656) 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/3323464859 learned_addr learned my addr 172.21.15.69:0/3323464859 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4f37fe640 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd65edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/3323464859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 msgr2=0x5638dd65e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd65e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.841 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/3323464859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 msgr2=0x5638dd65edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd65edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4ea7fc640 1 -- 172.21.15.69:0/3323464859 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638dd6d3ba0 con 0x5638dd650b30 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4ea7fc640 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd65e890 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0104e0 tx=0x7fe4dc00c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538398d9fb431947 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/3323464859 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc014040 con 0x5638dd650b30 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/3323464859 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fe4dc0108f0 con 0x5638dd650b30 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4fbdef740 1 -- 172.21.15.69:0/3323464859 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4dc01c020 con 0x5638dd650b30 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/3323464859 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4dc042050 con 0x5638dd650b30 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4fbdef740 10 client.19359 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.835+0000 7fe4fbdef740 1 -- 172.21.15.69:0/3323464859 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5638dd6d4eb0 con 0x5638dd650b30 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/3323464859 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc01c020 con 0x5638dd650b30 2024-08-02T18:02:40.842 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/3323464859 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe4dc012560 con 0x5638dd650b30 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4fbdef740 10 client.19359 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4fbdef740 10 client.19359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4fbdef740 20 client.19359 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638dd6d53b0 0x5638dd6d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4fbdef740 4 client.19359 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4fbdef740 1 -- 172.21.15.69:0/3323464859 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5638dd5bd400 con 0x5638dd6d53b0 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638dd6d53b0 0x5638dd6d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4f2ffd640 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638dd6d53b0 0x5638dd6d77d0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=396cd47b4a33280 server_cookie=6cd5d2b8e0a54c5c in_seq=0 out_seq=0 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.836+0000 7fe4d2ffd640 10 client.19359 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4d2ffd640 1 -- 172.21.15.69:0/3323464859 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5638dd5bd400 con 0x5638dd6d53b0 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4d2ffd640 10 client.19359 handle_command_reply: tid=0 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 client.19359 shutdown 2024-08-02T18:02:40.843 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 -- 172.21.15.69:0/3323464859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 msgr2=0x5638dd65e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd65e890 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0104e0 tx=0x7fe4dc00c350 comp rx=0 tx=0).stop 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 -- 172.21.15.69:0/3323464859 shutdown_connections 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5638dd650b30 0x5638dd65e890 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5638dd6d53b0 0x5638dd6d77d0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5638dd65b3f0 0x5638dd65edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 --2- 172.21.15.69:0/3323464859 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5638dd650210 0x5638dd65e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 -- 172.21.15.69:0/3323464859 >> 172.21.15.69:0/3323464859 conn(0x5638dd64c7a0 msgr2=0x5638dd64b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4d2ffd640 0 client.19359 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 -- 172.21.15.69:0/3323464859 shutdown_connections 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 1 -- 172.21.15.69:0/3323464859 wait complete. 2024-08-02T18:02:40.844 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4fbdef740 20 client.19359 trim_cache size 0 max 16384 2024-08-02T18:02:40.845 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "status": "no active scrubs running", 2024-08-02T18:02:40.845 INFO:teuthology.orchestra.run.smithi069.stdout: "scrubs": {} 2024-08-02T18:02:40.845 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:02:40.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 -- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4e00a2cf0 msgr2=0x7fe4e00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4e00a2cf0 0x7fe4e00a51b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009430 tx=0x7fe4ec009360 comp rx=0 tx=0).stop 2024-08-02T18:02:40.845 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 -- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f41007b0 msgr2=0x7fe4f410e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f41007b0 0x7fe4f410e110 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe4e400c970 tx=0x7fe4e400ce30 comp rx=0 tx=0).stop 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 -- 172.21.15.69:0/1301560399 shutdown_connections 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe4e00a2cf0 0x7fe4e00a51b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4f41067b0 0x7fe4f410eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe4f4101150 0x7fe4f410e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 --2- 172.21.15.69:0/1301560399 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe4f41007b0 0x7fe4f410e110 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.837+0000 7fe4f99f1640 1 -- 172.21.15.69:0/1301560399 >> 172.21.15.69:0/1301560399 conn(0x7fe4f40fc4d0 msgr2=0x7fe4f40fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.838+0000 7fe4f99f1640 1 -- 172.21.15.69:0/1301560399 shutdown_connections 2024-08-02T18:02:40.846 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:40.838+0000 7fe4f99f1640 1 -- 172.21.15.69:0/1301560399 wait complete. 2024-08-02T18:02:41.048 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:02:41.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:40 smithi069 ceph-mon[32746]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 346 KiB/s wr, 53 op/s 2024-08-02T18:02:41.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:40 smithi083 ceph-mon[37031]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 346 KiB/s wr, 53 op/s 2024-08-02T18:02:41.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:40 smithi077 ceph-mon[37108]: pgmap v510: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 346 KiB/s wr, 53 op/s 2024-08-02T18:02:41.617 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-02T18:02:41.618 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:02:41.738 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:42.235 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 -- 172.21.15.69:0/3957680920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba8101150 msgr2=0x7f6ba810d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3957680920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba8101150 0x7f6ba810d680 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009cd0 tx=0x7f6ba002a8e0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 -- 172.21.15.69:0/3957680920 shutdown_connections 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3957680920 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba8101150 0x7f6ba810d680 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3957680920 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ba81007b0 0x7f6ba8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3957680920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba81067b0 0x7f6ba8106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 -- 172.21.15.69:0/3957680920 >> 172.21.15.69:0/3957680920 conn(0x7f6ba80fc4d0 msgr2=0x7f6ba80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 -- 172.21.15.69:0/3957680920 shutdown_connections 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.221+0000 7f6bafd73640 1 -- 172.21.15.69:0/3957680920 wait complete. 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bafd73640 1 Processor -- start 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bafd73640 1 -- start start 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bafd73640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba81007b0 0x7f6ba8197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bafd73640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ba8101150 0x7f6ba81975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:42.236 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bafd73640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba81067b0 0x7f6ba819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bafd73640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8110bd0 con 0x7f6ba81007b0 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bafd73640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8110a50 con 0x7f6ba8101150 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bafd73640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba8110d50 con 0x7f6ba81067b0 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba81007b0 0x7f6ba8197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba81007b0 0x7f6ba8197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47672/0 (socket says 172.21.15.69:47672) 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 -- 172.21.15.69:0/3669943455 learned_addr learned my addr 172.21.15.69:0/3669943455 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bad30e640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ba8101150 0x7f6ba81975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bae310640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba81067b0 0x7f6ba819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 -- 172.21.15.69:0/3669943455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba81067b0 msgr2=0x7f6ba819b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba81067b0 0x7f6ba819b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 -- 172.21.15.69:0/3669943455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ba8101150 msgr2=0x7f6ba81975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ba8101150 0x7f6ba81975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 -- 172.21.15.69:0/3669943455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba819c040 con 0x7f6ba81007b0 2024-08-02T18:02:42.237 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6bae310640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba81067b0 0x7f6ba819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.222+0000 7f6badb0f640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba81007b0 0x7f6ba8197090 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400d990 tx=0x7f6ba400de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.223+0000 7f6b9effd640 1 -- 172.21.15.69:0/3669943455 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba401b070 con 0x7f6ba81007b0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.223+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba819c330 con 0x7f6ba81007b0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.223+0000 7f6b9effd640 1 -- 172.21.15.69:0/3669943455 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ba400edf0 con 0x7f6ba81007b0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.223+0000 7f6b9effd640 1 -- 172.21.15.69:0/3669943455 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4014870 con 0x7f6ba81007b0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.223+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba806a4c0 con 0x7f6ba81007b0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.223+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba8101ef0 con 0x7f6ba81007b0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.225+0000 7f6b9effd640 1 -- 172.21.15.69:0/3669943455 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6ba401e070 con 0x7f6ba81007b0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.226+0000 7f6b9effd640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b940a2cf0 0x7f6b940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.226+0000 7f6bad30e640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b940a2cf0 0x7f6b940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.226+0000 7f6b9effd640 1 -- 172.21.15.69:0/3669943455 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f6ba40d1310 con 0x7f6ba81007b0 2024-08-02T18:02:42.238 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.226+0000 7f6bad30e640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b940a2cf0 0x7f6b940a51b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80feac0 tx=0x7f6b9800a2d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:42.239 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.229+0000 7f6b9effd640 1 -- 172.21.15.69:0/3669943455 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6ba409d5e0 con 0x7f6ba81007b0 2024-08-02T18:02:42.301 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:42.455 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.454+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ba8100c10 con 0x7f6ba81007b0 2024-08-02T18:02:42.456 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.454+0000 7f6b9effd640 1 -- 172.21.15.69:0/3669943455 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11901 (secure 0 0 0) 0x7f6ba40a18e0 con 0x7f6ba81007b0 2024-08-02T18:02:42.456 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:42.456 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"btime":"2024-08-02T18:01:55:674676+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":28375,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3960673726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3960673726},{"type":"v1","addr":"172.21.15.77:6835","nonce":3960673726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T18:01:54.670284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24389":{"gid":24389,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:02:42.457 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-08-02T18:02:42.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b940a2cf0 msgr2=0x7f6b940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:42.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b940a2cf0 0x7f6b940a51b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ba80feac0 tx=0x7f6b9800a2d0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba81007b0 msgr2=0x7f6ba8197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:42.458 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba81007b0 0x7f6ba8197090 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f6ba400d990 tx=0x7f6ba400de50 comp rx=0 tx=0).stop 2024-08-02T18:02:42.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 shutdown_connections 2024-08-02T18:02:42.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6b940a2cf0 0x7f6b940a51b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ba81067b0 0x7f6ba819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6ba8101150 0x7f6ba81975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 --2- 172.21.15.69:0/3669943455 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6ba81007b0 0x7f6ba8197090 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:42.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 >> 172.21.15.69:0/3669943455 conn(0x7f6ba80fc4d0 msgr2=0x7f6ba80fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:42.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 shutdown_connections 2024-08-02T18:02:42.459 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:42.457+0000 7f6bafd73640 1 -- 172.21.15.69:0/3669943455 wait complete. 2024-08-02T18:02:43.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 -- 172.21.15.69:0/3071486045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac10bec0 msgr2=0x7ff4ac10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:43.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/3071486045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac10bec0 0x7ff4ac10e340 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009900 tx=0x7ff4a402a480 comp rx=0 tx=0).stop 2024-08-02T18:02:43.051 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 -- 172.21.15.69:0/3071486045 shutdown_connections 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/3071486045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac10bec0 0x7ff4ac10e340 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/3071486045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac103400 0x7ff4ac10b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/3071486045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4ac102ae0 0x7ff4ac102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 -- 172.21.15.69:0/3071486045 >> 172.21.15.69:0/3071486045 conn(0x7ff4ac0fc500 msgr2=0x7ff4ac0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 -- 172.21.15.69:0/3071486045 shutdown_connections 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.037+0000 7ff4b2c68640 1 -- 172.21.15.69:0/3071486045 wait complete. 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4b2c68640 1 Processor -- start 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4b2c68640 1 -- start start 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4b2c68640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4ac102ae0 0x7ff4ac198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4b2c68640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac103400 0x7ff4ac1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4b2c68640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac10bec0 0x7ff4ac19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4b2c68640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac111ca0 con 0x7ff4ac103400 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4b2c68640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac111b20 con 0x7ff4ac10bec0 2024-08-02T18:02:43.052 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4b2c68640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac111e20 con 0x7ff4ac102ae0 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.038+0000 7ff4abfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac103400 0x7ff4ac1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4abfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac103400 0x7ff4ac1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:47678/0 (socket says 172.21.15.69:47678) 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4abfff640 1 -- 172.21.15.69:0/677946614 learned_addr learned my addr 172.21.15.69:0/677946614 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4b0a04640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4ac102ae0 0x7ff4ac198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4b1205640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac10bec0 0x7ff4ac19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4abfff640 1 -- 172.21.15.69:0/677946614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4ac102ae0 msgr2=0x7ff4ac198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4abfff640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4ac102ae0 0x7ff4ac198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4abfff640 1 -- 172.21.15.69:0/677946614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac10bec0 msgr2=0x7ff4ac19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4abfff640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac10bec0 0x7ff4ac19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4abfff640 1 -- 172.21.15.69:0/677946614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac19d030 con 0x7ff4ac103400 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4abfff640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac103400 0x7ff4ac1985c0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000d950 tx=0x7ff4a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4a9ffb640 1 -- 172.21.15.69:0/677946614 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0017070 con 0x7ff4ac103400 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac19d320 con 0x7ff4ac103400 2024-08-02T18:02:43.053 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4a9ffb640 1 -- 172.21.15.69:0/677946614 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4a0009c30 con 0x7ff4ac103400 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4a9ffb640 1 -- 172.21.15.69:0/677946614 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000f6b0 con 0x7ff4ac103400 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.039+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac1a4c00 con 0x7ff4ac103400 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.040+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac102ec0 con 0x7ff4ac103400 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.041+0000 7ff4a9ffb640 1 -- 172.21.15.69:0/677946614 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff4a00109d0 con 0x7ff4ac103400 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.042+0000 7ff4a9ffb640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4940a2cf0 0x7ff4940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.042+0000 7ff4b0a04640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4940a2cf0 0x7ff4940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.042+0000 7ff4a9ffb640 1 -- 172.21.15.69:0/677946614 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7ff4a00cf420 con 0x7ff4ac103400 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.042+0000 7ff4b0a04640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4940a2cf0 0x7ff4940a51b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff49c007ec0 tx=0x7ff49c0075b0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:43.054 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.045+0000 7ff4a9ffb640 1 -- 172.21.15.69:0/677946614 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff4a0009de0 con 0x7ff4ac103400 2024-08-02T18:02:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:42 smithi069 ceph-mon[32746]: pgmap v511: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-08-02T18:02:43.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:42 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3669943455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:43.253 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.252+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4ac1069d0 con 0x7ff4ac103400 2024-08-02T18:02:43.254 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.252+0000 7ff4a9ffb640 1 -- 172.21.15.69:0/677946614 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11901 (secure 0 0 0) 0x7ff4a009bc30 con 0x7ff4ac103400 2024-08-02T18:02:43.254 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:43.254 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"btime":"2024-08-02T18:01:55:674676+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":28375,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3960673726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3960673726},{"type":"v1","addr":"172.21.15.77:6835","nonce":3960673726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T18:01:54.670284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24389":{"gid":24389,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:02:43.255 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4940a2cf0 msgr2=0x7ff4940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4940a2cf0 0x7ff4940a51b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff49c007ec0 tx=0x7ff49c0075b0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac103400 msgr2=0x7ff4ac1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac103400 0x7ff4ac1985c0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7ff4a000d950 tx=0x7ff4a000de10 comp rx=0 tx=0).stop 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 shutdown_connections 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff4ac103400 0x7ff4ac1985c0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4940a2cf0 0x7ff4940a51b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff4ac10bec0 0x7ff4ac19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 --2- 172.21.15.69:0/677946614 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff4ac102ae0 0x7ff4ac198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:43.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.254+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 >> 172.21.15.69:0/677946614 conn(0x7ff4ac0fc500 msgr2=0x7ff4ac0fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:43.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.255+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 shutdown_connections 2024-08-02T18:02:43.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:43.255+0000 7ff4b2c68640 1 -- 172.21.15.69:0/677946614 wait complete. 2024-08-02T18:02:43.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:42 smithi083 ceph-mon[37031]: pgmap v511: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-08-02T18:02:43.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:42 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3669943455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:43.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:42 smithi077 ceph-mon[37108]: pgmap v511: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-08-02T18:02:43.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:42 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3669943455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:43.767 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --set-id f6c7550a --release --await' 2024-08-02T18:02:43.767 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --set-id f6c7550a --release --await 2024-08-02T18:02:44.172 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:43 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/677946614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:44.249 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:43 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/677946614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:44.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:43 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/677946614' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:44.446 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-02T18:02:44.554 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:45.048 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:44 smithi069 ceph-mon[32746]: pgmap v512: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 -- 172.21.15.69:0/1232138671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa154102250 msgr2=0x7fa15410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/1232138671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa154102250 0x7fa15410e780 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009cd0 tx=0x7fa14c02a8e0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 -- 172.21.15.69:0/1232138671 shutdown_connections 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/1232138671 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa154102250 0x7fa15410e780 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/1232138671 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1541018b0 0x7fa154101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/1232138671 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa1541078b0 0x7fa154107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 -- 172.21.15.69:0/1232138671 >> 172.21.15.69:0/1232138671 conn(0x7fa1540fd5f0 msgr2=0x7fa1540ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 -- 172.21.15.69:0/1232138671 shutdown_connections 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.134+0000 7fa15b1b3640 1 -- 172.21.15.69:0/1232138671 wait complete. 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa15b1b3640 1 Processor -- start 2024-08-02T18:02:45.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa15b1b3640 1 -- start start 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa15b1b3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1541018b0 0x7fa1541981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa15b1b3640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa154102250 0x7fa154198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa15b1b3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1541078b0 0x7fa15419caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa15b1b3640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa154111cd0 con 0x7fa1541078b0 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa15b1b3640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa154111b50 con 0x7fa154102250 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa15b1b3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa154111e50 con 0x7fa1541018b0 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1541078b0 0x7fa15419caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1541078b0 0x7fa15419caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45956/0 (socket says 172.21.15.69:45956) 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 -- 172.21.15.69:0/382802926 learned_addr learned my addr 172.21.15.69:0/382802926 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa153fff640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa154102250 0x7fa154198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa158f4f640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1541018b0 0x7fa1541981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 -- 172.21.15.69:0/382802926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1541018b0 msgr2=0x7fa1541981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1541018b0 0x7fa1541981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.257 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 -- 172.21.15.69:0/382802926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa154102250 msgr2=0x7fa154198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa154102250 0x7fa154198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 -- 172.21.15.69:0/382802926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15419d180 con 0x7fa1541078b0 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.135+0000 7fa159750640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1541078b0 0x7fa15419caa0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa14c005cc0 tx=0x7fa14c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.136+0000 7fa151ffb640 1 -- 172.21.15.69:0/382802926 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c004040 con 0x7fa1541078b0 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.136+0000 7fa15b1b3640 1 -- 172.21.15.69:0/382802926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15419d410 con 0x7fa1541078b0 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.136+0000 7fa151ffb640 1 -- 172.21.15.69:0/382802926 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa14c0046c0 con 0x7fa1541078b0 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.136+0000 7fa151ffb640 1 -- 172.21.15.69:0/382802926 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c030e70 con 0x7fa1541078b0 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.136+0000 7fa15b1b3640 1 -- 172.21.15.69:0/382802926 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1541a4d50 con 0x7fa1541078b0 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.138+0000 7fa151ffb640 1 -- 172.21.15.69:0/382802926 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa14c0305a0 con 0x7fa1541078b0 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.138+0000 7fa15d5b1740 1 Processor -- start 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.138+0000 7fa15d5b1740 1 -- start start 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.138+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x5576896ecc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.258 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.138+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ed190 0x5576896fba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 0x5576896fe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x557689652c60 con 0x5576896ec870 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576895eab90 con 0x5576896ed190 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55768964ec20 con 0x5576896fbf80 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x5576896ecc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x5576896ecc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45960/0 (socket says 172.21.15.69:45960) 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 -- 172.21.15.69:0/3405003893 learned_addr learned my addr 172.21.15.69:0/3405003893 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa151ffb640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1400a2cf0 0x7fa1400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 0x5576896fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa158f4f640 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ed190 0x5576896fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa159750640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1400a2cf0 0x7fa1400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 -- 172.21.15.69:0/3405003893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 msgr2=0x5576896fe370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:45.259 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 0x5576896fe370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 -- 172.21.15.69:0/3405003893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ed190 msgr2=0x5576896fba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ed190 0x5576896fba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 -- 172.21.15.69:0/3405003893 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557689362580 con 0x5576896ec870 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa158f4f640 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ed190 0x5576896fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa151ffb640 1 -- 172.21.15.69:0/382802926 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fa14c02e030 con 0x7fa1541078b0 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa153fff640 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x5576896ecc50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa13c006aa0 tx=0x7fa13c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32b50d354e51125 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa14affd640 1 -- 172.21.15.69:0/3405003893 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c017610 con 0x5576896ec870 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa14affd640 1 -- 172.21.15.69:0/3405003893 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa13c01c070 con 0x5576896ec870 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.139+0000 7fa14affd640 1 -- 172.21.15.69:0/3405003893 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c015940 con 0x5576896ec870 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 -- 172.21.15.69:0/3405003893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 msgr2=0x5576896ecc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x5576896ecc50 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fa13c006aa0 tx=0x7fa13c0095d0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa159750640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1400a2cf0 0x7fa1400a51b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0b94a0 tx=0x7fa14c03d040 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:45.260 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 -- 172.21.15.69:0/3405003893 shutdown_connections 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 0x5576896fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ed190 0x5576896fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/3405003893 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x5576896ecc50 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 -- 172.21.15.69:0/3405003893 >> 172.21.15.69:0/3405003893 conn(0x5576896e8f20 msgr2=0x5576896e9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 -- 172.21.15.69:0/3405003893 shutdown_connections 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 -- 172.21.15.69:0/3405003893 wait complete. 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.140+0000 7fa15d5b1740 1 -- start start 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa15d5b1740 1 Processor -- start 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 0x5576896f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 0x5576896f9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896fbf80 0x5576896f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5576896feb50 con 0x5576896fbf80 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5576895f07c0 con 0x5576896ec870 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5576896fecd0 con 0x5576896ed190 2024-08-02T18:02:45.261 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa15d5b1740 10 client.0 _ll_register_callbacks cb 0x5576895f9f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896fbf80 0x5576896f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896fbf80 0x5576896f98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45968/0 (socket says 172.21.15.69:45968) 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 -- 172.21.15.69:0/2864633458 learned_addr learned my addr 172.21.15.69:0/2864633458 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 0x5576896f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 0x5576896f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 -- 172.21.15.69:0/2864633458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 msgr2=0x5576896f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 0x5576896f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 -- 172.21.15.69:0/2864633458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 msgr2=0x5576896f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 0x5576896f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 -- 172.21.15.69:0/2864633458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55768942c960 con 0x5576896fbf80 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa153fff640 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896fbf80 0x5576896f98c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa13c015620 tx=0x7fa13c015520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b001a949ba3b47c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:45.262 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.141+0000 7fa133fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa133fff640 1 -- 172.21.15.69:0/2864633458 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c015e00 con 0x5576896fbf80 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa133fff640 1 -- 172.21.15.69:0/2864633458 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fa13c017410 con 0x5576896fbf80 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2864633458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa13c01c070 con 0x5576896fbf80 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa133fff640 1 -- 172.21.15.69:0/2864633458 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa13c033050 con 0x5576896fbf80 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 10 client.19383 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2864633458 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55768970aba0 con 0x5576896fbf80 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa133fff640 1 -- 172.21.15.69:0/2864633458 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa13c01c070 con 0x5576896fbf80 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa133fff640 1 -- 172.21.15.69:0/2864633458 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa13c021920 con 0x5576896fbf80 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 10 client.19383 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 10 client.19383 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 20 client.19383 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55768970bef0 0x55768970e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 4 client.19383 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:45.263 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2864633458 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576896601d0 con 0x55768970bef0 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.142+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55768970bef0 0x55768970e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.143+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55768970bef0 0x55768970e2e0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e58e3fb225cae84 server_cookie=711aa5a45478cb50 in_seq=0 out_seq=0 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.143+0000 7fa133fff640 10 client.19383 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.143+0000 7fa133fff640 1 -- 172.21.15.69:0/2864633458 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5576896601d0 con 0x55768970bef0 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.143+0000 7fa133fff640 10 client.19383 handle_command_reply: tid=0 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.143+0000 7fa15d5b1740 1 client.19383 shutdown 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2864633458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896fbf80 msgr2=0x5576896f98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896fbf80 0x5576896f98c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fa13c015620 tx=0x7fa13c015520 comp rx=0 tx=0).stop 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2864633458 shutdown_connections 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896fbf80 0x5576896f98c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55768970bef0 0x55768970e2e0 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 0x5576896f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.264 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2864633458 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 0x5576896f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa133fff640 0 client.19383 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2864633458 >> 172.21.15.69:0/2864633458 conn(0x5576896e8f20 msgr2=0x5576896eec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2864633458 shutdown_connections 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2864633458 wait complete. 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.144+0000 7fa15d5b1740 20 client.19383 trim_cache size 0 max 16384 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.160+0000 7fa15d5b1740 1 Processor -- start 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.160+0000 7fa15d5b1740 1 -- start start 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 0x557689713a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ed190 0x557689713f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 0x5576896f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541af510 con 0x5576896ed190 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541acde0 con 0x5576896ec870 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541aac30 con 0x5576896fbf80 2024-08-02T18:02:45.265 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ed190 0x557689713f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ed190 0x557689713f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45976/0 (socket says 172.21.15.69:45976) 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 -- 172.21.15.69:0/2483221459 learned_addr learned my addr 172.21.15.69:0/2483221459 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 -- 172.21.15.69:0/2483221459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 msgr2=0x5576896f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2483221459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 0x5576896f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 -- 172.21.15.69:0/2483221459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 msgr2=0x557689713a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2483221459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 0x557689713a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 -- 172.21.15.69:0/2483221459 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576896e7470 con 0x5576896ed190 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2483221459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ed190 0x557689713f60 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b370 tx=0x7fa144005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cad33982b79bbb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa14a7fc640 1 -- 172.21.15.69:0/2483221459 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa144004230 con 0x5576896ed190 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa14a7fc640 1 -- 172.21.15.69:0/2483221459 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1440043d0 con 0x5576896ed190 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.161+0000 7fa14a7fc640 1 -- 172.21.15.69:0/2483221459 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14401b730 con 0x5576896ed190 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2483221459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ed190 msgr2=0x557689713f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.266 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2483221459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ed190 0x557689713f60 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fa14400b370 tx=0x7fa144005c50 comp rx=0 tx=0).stop 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2483221459 shutdown_connections 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2483221459 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ed190 0x557689713f60 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2483221459 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896fbf80 0x5576896f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2483221459 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896ec870 0x557689713a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2483221459 >> 172.21.15.69:0/2483221459 conn(0x5576896e8f20 msgr2=0x5576896faec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2483221459 shutdown_connections 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2483221459 wait complete. 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 -- start start 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.162+0000 7fa15d5b1740 1 Processor -- start 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x557689703f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 0x5576897044a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896fbf80 0x557689700f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541ab2a0 con 0x5576896ec870 2024-08-02T18:02:45.267 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541ac710 con 0x5576896fbf80 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1541ab970 con 0x5576896ed190 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 10 client.0 _ll_register_callbacks cb 0x7fa13c0157b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x557689703f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x557689703f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:45984/0 (socket says 172.21.15.69:45984) 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 -- 172.21.15.69:0/2834846438 learned_addr learned my addr 172.21.15.69:0/2834846438 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 27, 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T18:02:45.268 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 56, 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "f6c7550a": { 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 56, 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 147.9, 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T18:02:45.269 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RELEASED", 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896fbf80 0x557689700f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 0x5576897044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.270 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 -- 172.21.15.69:0/2834846438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 msgr2=0x5576897044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 0x5576897044a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 -- 172.21.15.69:0/2834846438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896fbf80 msgr2=0x557689700f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896fbf80 0x557689700f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 -- 172.21.15.69:0/2834846438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557689701470 con 0x5576896ec870 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa153fff640 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x557689703f60 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00b010 tx=0x7fa13c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb35656570550d0f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa133fff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa133fff640 1 -- 172.21.15.69:0/2834846438 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c015e00 con 0x5576896ec870 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa133fff640 1 -- 172.21.15.69:0/2834846438 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fa13c021610 con 0x5576896ec870 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2834846438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1440043d0 con 0x5576896ec870 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa133fff640 1 -- 172.21.15.69:0/2834846438 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa13c009b10 con 0x5576896ec870 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 10 client.19389 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.163+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2834846438 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557689777180 con 0x5576896ec870 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa133fff640 1 -- 172.21.15.69:0/2834846438 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1440043d0 con 0x5576896ec870 2024-08-02T18:02:45.271 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa133fff640 1 -- 172.21.15.69:0/2834846438 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa13c020a50 con 0x5576896ec870 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa15d5b1740 10 client.19389 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa15d5b1740 10 client.19389 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa15d5b1740 20 client.19389 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5576897777b0 0x557689779bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa15d5b1740 4 client.19389 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f6c7550a", "release": true, "await": true}] 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2834846438 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f6c7550a", "release": true, "await": true}) -- 0x55768965a6e0 con 0x5576897777b0 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5576897777b0 0x557689779bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa133fff640 10 client.19389 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.164+0000 7fa158f4f640 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5576897777b0 0x557689779bd0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f7eb1128f3c5add server_cookie=21f38fedf30ec946 in_seq=0 out_seq=0 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa133fff640 1 -- 172.21.15.69:0/2834846438 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55768965a6e0 con 0x5576897777b0 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa133fff640 10 client.19389 handle_command_reply: tid=0 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 client.19389 shutdown 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2834846438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 msgr2=0x557689703f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.272 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x557689703f60 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fa13c00b010 tx=0x7fa13c010f90 comp rx=0 tx=0).stop 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2834846438 shutdown_connections 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x5576897777b0 0x557689779bd0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5576896fbf80 0x557689700f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5576896ed190 0x5576897044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 --2- 172.21.15.69:0/2834846438 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5576896ec870 0x557689703f60 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2834846438 >> 172.21.15.69:0/2834846438 conn(0x5576896e8f20 msgr2=0x5576896e7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa133fff640 0 client.19389 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2834846438 shutdown_connections 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 1 -- 172.21.15.69:0/2834846438 wait complete. 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15d5b1740 20 client.19389 trim_cache size 0 max 16384 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 -- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1400a2cf0 msgr2=0x7fa1400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1400a2cf0 0x7fa1400a51b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0b94a0 tx=0x7fa14c03d040 comp rx=0 tx=0).stop 2024-08-02T18:02:45.273 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 -- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1541078b0 msgr2=0x7fa15419caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1541078b0 0x7fa15419caa0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa14c005cc0 tx=0x7fa14c003970 comp rx=0 tx=0).stop 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 -- 172.21.15.69:0/382802926 shutdown_connections 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fa1541078b0 0x7fa15419caa0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fa1400a2cf0 0x7fa1400a51b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fa154102250 0x7fa154198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 --2- 172.21.15.69:0/382802926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1541018b0 0x7fa1541981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 -- 172.21.15.69:0/382802926 >> 172.21.15.69:0/382802926 conn(0x7fa1540fd5f0 msgr2=0x7fa1540fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 -- 172.21.15.69:0/382802926 shutdown_connections 2024-08-02T18:02:45.274 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:45.213+0000 7fa15b1b3640 1 -- 172.21.15.69:0/382802926 wait complete. 2024-08-02T18:02:45.275 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:44 smithi083 ceph-mon[37031]: pgmap v512: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-08-02T18:02:45.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:44 smithi077 ceph-mon[37108]: pgmap v512: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-08-02T18:02:45.353 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-08-02T18:02:45.407 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/: 2024-08-02T18:02:45.407 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.407 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T18:02:45.407 INFO:teuthology.orchestra.run.smithi069.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 .. 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 data 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data: 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:total 36 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 . 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 .. 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 2 17:47 base 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Aug 2 17:46 current_logfiles 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 2 17:46 global 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 log 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_commit_ts 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_dynshmem 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Aug 2 17:46 pg_hba.conf 2024-08-02T18:02:45.408 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Aug 2 17:46 pg_ident.conf 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 18:02 pg_logical 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 pg_multixact 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_notify 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_replslot 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_serial 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_snapshots 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 18:02 pg_stat 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 18:02 pg_stat_tmp 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 18:02 pg_subtrans 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_tblspc 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 pg_twophase 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 PG_VERSION 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 2 18:02 pg_wal 2024-08-02T18:02:45.409 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 pg_xact 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Aug 2 17:46 postgresql.auto.conf 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Aug 2 17:46 postgresql.conf 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Aug 2 17:46 postmaster.opts 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base: 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 2 17:47 . 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 1 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 13493 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Aug 2 17:53 13494 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:47 pgsql_tmp 2024-08-02T18:02:45.410 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/1: 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 . 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 2 17:47 .. 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 112 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 113 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 1247 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1247_fsm 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1247_vm 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 2 17:46 1249 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1249_fsm 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1249_vm 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 2 17:46 1255 2024-08-02T18:02:45.411 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1255_fsm 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1255_vm 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 2 17:46 1259 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1259_fsm 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1259_vm 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 2 17:46 13344 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13344_fsm 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13344_vm 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13346 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13348 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13349_fsm 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349_vm 2024-08-02T18:02:45.412 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13351 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13353 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13354_fsm 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354_vm 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13356 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13358 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13359_fsm 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359_vm 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13361 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13363 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1417 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1418 2024-08-02T18:02:45.413 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 174 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 175 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2187 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2224 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2328 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2336 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2337 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2579 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2600 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2600_fsm 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2600_vm 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2601_fsm 2024-08-02T18:02:45.414 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601_vm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2602 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2602_fsm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2602_vm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2603 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2603_fsm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2603_vm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2604 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2605 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2605_fsm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2605_vm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2606_fsm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606_vm 2024-08-02T18:02:45.415 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2607 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2607_fsm 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2607_vm 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 2 17:46 2608 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2608_fsm 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2608_vm 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 2 17:46 2609 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2609_fsm 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2609_vm 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2610 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2610_fsm 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2610_vm 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2611 2024-08-02T18:02:45.416 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2612_fsm 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612_vm 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2613 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2615_fsm 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615_vm 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2616 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2616_fsm 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2616_vm 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 2 17:46 2617 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2617_fsm 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2617_vm 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 2 17:46 2618 2024-08-02T18:02:45.417 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2618_fsm 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2618_vm 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 2 17:46 2619 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2619_fsm 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2619_vm 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2620 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2650 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2651 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2652 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2653 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2654 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2655 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2656 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2657 2024-08-02T18:02:45.418 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 2 17:46 2658 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 2659 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2660 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2661 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2662 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2663 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2664 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2665 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2666 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2667 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2668 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2669 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2670 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 2 17:46 2673 2024-08-02T18:02:45.419 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 2 17:46 2674 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 2 17:46 2675 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2678 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2679 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2680 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2681 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2682 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2683 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2684 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2685 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2686 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2687 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2688 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2689 2024-08-02T18:02:45.420 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 2 17:46 2690 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 2 17:46 2691 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2692 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2693 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2696 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2699 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2701 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2702 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2703 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2704 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2753 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2753_fsm 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2753_vm 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2754 2024-08-02T18:02:45.421 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2755 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2756 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2757 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2830 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2831 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2832 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2833 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2834 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2835 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2836 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2837 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 2 17:46 2838 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2838_fsm 2024-08-02T18:02:45.422 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2838_vm 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2839 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840_fsm 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2840_vm 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2841 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2995 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2996 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3079_fsm 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079_vm 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3080 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3081 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3085 2024-08-02T18:02:45.423 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3118 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3119 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 2 17:46 3164 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3256 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3257 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3258 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3350 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3351 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3379 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3380 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3381 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3394 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3394_fsm 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3394_vm 2024-08-02T18:02:45.424 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3395 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3429 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3430 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3431 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3433 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3439 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3440 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3455 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 2 17:46 3456 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3456_fsm 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3456_vm 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3466 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3467 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3468 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3501 2024-08-02T18:02:45.425 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3502 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3503 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3534 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3541_fsm 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541_vm 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3542 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3574 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3575 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3576 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3596 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3597 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3598 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3599 2024-08-02T18:02:45.426 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3600_fsm 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600_vm 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3601_fsm 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601_vm 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3602_fsm 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602_vm 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603_fsm 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3603_vm 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3604 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3605 2024-08-02T18:02:45.427 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3606 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3607 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3608 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 3609 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3712 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3764_fsm 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764_vm 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3766 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3767 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3997 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4143 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4144 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4145 2024-08-02T18:02:45.428 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4146 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4147 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4148 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4149 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4150 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4151 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4152 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4153 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4154 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4155 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4156 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4157 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4158 2024-08-02T18:02:45.429 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4159 2024-08-02T18:02:45.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4160 2024-08-02T18:02:45.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4161 2024-08-02T18:02:45.430 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4162 2024-08-02T18:02:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4163 2024-08-02T18:02:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4164 2024-08-02T18:02:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4165 2024-08-02T18:02:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4166 2024-08-02T18:02:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4167 2024-08-02T18:02:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4168 2024-08-02T18:02:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4169 2024-08-02T18:02:45.431 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4170 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4171 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4172 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4173 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4174 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 5002 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 548 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 549 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6102 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6104 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6106 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6110 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6111 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6112 2024-08-02T18:02:45.432 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6113 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6117 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 826 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 827 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 828 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 2 17:46 pg_filenode.map 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 PG_VERSION 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:total 7777 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Aug 2 17:46 . 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 2 17:47 .. 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 112 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 113 2024-08-02T18:02:45.433 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 1247 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1247_fsm 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1247_vm 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 2 17:46 1249 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1249_fsm 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1249_vm 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 2 17:46 1255 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1255_fsm 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1255_vm 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 2 17:46 1259 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1259_fsm 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1259_vm 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 2 17:46 13344 2024-08-02T18:02:45.434 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13344_fsm 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13344_vm 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13346 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13348 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13349_fsm 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349_vm 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13351 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13353 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13354_fsm 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354_vm 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13356 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13358 2024-08-02T18:02:45.435 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13359_fsm 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359_vm 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13361 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13363 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1417 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1418 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 174 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 175 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2187 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2224 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2328 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2336 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2337 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2579 2024-08-02T18:02:45.436 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2600 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2600_fsm 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2600_vm 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2601_fsm 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601_vm 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2602 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2602_fsm 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2602_vm 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2603 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2603_fsm 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2603_vm 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2604 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2605 2024-08-02T18:02:45.437 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2605_fsm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2605_vm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2606_fsm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2606_vm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2607 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2607_fsm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2607_vm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 2 17:46 2608 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2608_fsm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2608_vm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 2 17:46 2609 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2609_fsm 2024-08-02T18:02:45.438 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2609_vm 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2610 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2610_fsm 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2610_vm 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2611 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2612_fsm 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612_vm 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2613 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615 2024-08-02T18:02:45.439 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2615_fsm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615_vm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2616 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2616_fsm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2616_vm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 2 17:46 2617 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2617_fsm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2617_vm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 2 17:46 2618 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2618_fsm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2618_vm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Aug 2 17:46 2619 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2619_fsm 2024-08-02T18:02:45.440 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2619_vm 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2620 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2650 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2651 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2652 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2653 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2654 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2655 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2656 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2657 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 2 17:46 2658 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:46 2659 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2660 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2661 2024-08-02T18:02:45.441 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2662 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2663 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2664 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2665 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2666 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2667 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2668 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2669 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2670 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 2 17:46 2673 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 2 17:46 2674 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 2 17:46 2675 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2678 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2679 2024-08-02T18:02:45.442 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2680 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2681 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2682 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2683 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2684 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2685 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2686 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2687 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2688 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2689 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 2 17:46 2690 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 2 17:46 2691 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2692 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2693 2024-08-02T18:02:45.443 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2696 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2699 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2701 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2702 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2703 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2704 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2753 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2753_fsm 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2753_vm 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2754 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2755 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2756 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2757 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2830 2024-08-02T18:02:45.444 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2831 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2832 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2833 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2834 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2835 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2836 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2837 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 2 17:46 2838 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2838_fsm 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2838_vm 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2839 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840_fsm 2024-08-02T18:02:45.445 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2840_vm 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2841 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2995 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2996 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3079_fsm 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079_vm 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3080 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3081 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3085 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3118 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3119 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 2 17:46 3164 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3256 2024-08-02T18:02:45.446 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3257 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3258 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3350 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3351 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3379 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3380 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3381 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3394 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3394_fsm 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3394_vm 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3395 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3429 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3430 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3431 2024-08-02T18:02:45.447 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3433 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3439 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3440 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3455 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 2 17:46 3456 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3456_fsm 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3456_vm 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3466 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3467 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3468 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3501 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3502 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3503 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3534 2024-08-02T18:02:45.448 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3541_fsm 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541_vm 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3542 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3574 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3575 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3576 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3596 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3597 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3598 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3599 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3600_fsm 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600_vm 2024-08-02T18:02:45.449 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3601_fsm 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601_vm 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3602_fsm 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602_vm 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603_fsm 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3603_vm 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3604 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3605 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3606 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3607 2024-08-02T18:02:45.450 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3608 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 3609 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3712 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3764_fsm 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764_vm 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3766 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3767 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3997 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4143 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4144 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4145 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4146 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4147 2024-08-02T18:02:45.451 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4148 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4149 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4150 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4151 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4152 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4153 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4154 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4155 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4156 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4157 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4158 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4159 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4160 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4161 2024-08-02T18:02:45.452 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4162 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4163 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4164 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4165 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4166 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4167 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4168 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4169 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4170 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4171 2024-08-02T18:02:45.453 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4172 2024-08-02T18:02:45.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4173 2024-08-02T18:02:45.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4174 2024-08-02T18:02:45.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 5002 2024-08-02T18:02:45.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 548 2024-08-02T18:02:45.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 549 2024-08-02T18:02:45.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6102 2024-08-02T18:02:45.455 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6104 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6106 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6110 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6111 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6112 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6113 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6117 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 826 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 827 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 828 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 2 17:46 pg_filenode.map 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 PG_VERSION 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-08-02T18:02:45.456 INFO:teuthology.orchestra.run.smithi069.stdout:total 550638 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Aug 2 17:53 . 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 2 17:47 .. 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 112 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 113 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:47 1247 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1247_fsm 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:47 1247_vm 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Aug 2 17:47 1249 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:47 1249_fsm 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:47 1249_vm 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Aug 2 17:46 1255 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1255_fsm 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1255_vm 2024-08-02T18:02:45.457 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Aug 2 18:02 1259 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1259_fsm 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:47 1259_vm 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Aug 2 17:46 13344 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13344_fsm 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13344_vm 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13346 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13348 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13349_fsm 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13349_vm 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13351 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13353 2024-08-02T18:02:45.458 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13354_fsm 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13354_vm 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13356 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13358 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 13359_fsm 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13359_vm 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 13361 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 13363 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1417 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 1418 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 439672832 Aug 2 18:02 16396 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Aug 2 18:02 16396_fsm 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:54 16396_vm 2024-08-02T18:02:45.459 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1966080 Aug 2 18:02 16397 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 18:02 16397_fsm 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 18:02 16397_vm 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1327104 Aug 2 18:02 16399 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 18:02 16399_fsm 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 18:02 16399_vm 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 18:02 16400 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 2 18:02 16402 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Aug 2 18:02 16404 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40501248 Aug 2 18:02 16406 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 18:02 16406_fsm 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:58 16406_vm 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 174 2024-08-02T18:02:45.460 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 175 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2187 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2224 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2328 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2336 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2337 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:47 2579 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2600 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2600_fsm 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2600_vm 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2601_fsm 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2601_vm 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2602 2024-08-02T18:02:45.461 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2602_fsm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2602_vm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2603 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2603_fsm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2603_vm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2604 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2605 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2605_fsm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2605_vm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:47 2606 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2606_fsm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:47 2606_vm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2607 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2607_fsm 2024-08-02T18:02:45.462 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2607_vm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Aug 2 17:47 2608 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2608_fsm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:47 2608_vm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Aug 2 17:46 2609 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2609_fsm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2609_vm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:47 2610 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2610_fsm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:47 2610_vm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2611 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2612_fsm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2612_vm 2024-08-02T18:02:45.463 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2613 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2615_fsm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2615_vm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2616 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2616_fsm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2616_vm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Aug 2 17:46 2617 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2617_fsm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2617_vm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Aug 2 17:46 2618 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2618_fsm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2618_vm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Aug 2 18:02 2619 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:50 2619_fsm 2024-08-02T18:02:45.464 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:47 2619_vm 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2620 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2650 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2651 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2652 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2653 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2654 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2655 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2656 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2657 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 2 17:47 2658 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Aug 2 17:47 2659 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2660 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2661 2024-08-02T18:02:45.465 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:47 2662 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:47 2663 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:47 2664 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:47 2665 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:47 2666 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:47 2667 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2668 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2669 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2670 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Aug 2 17:47 2673 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Aug 2 17:47 2674 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Aug 2 17:46 2675 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:47 2678 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:47 2679 2024-08-02T18:02:45.466 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2680 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2681 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2682 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2683 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2684 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2685 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2686 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2687 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 2688 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Aug 2 17:46 2689 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Aug 2 17:46 2690 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Aug 2 17:46 2691 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2692 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2693 2024-08-02T18:02:45.467 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:51 2696 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2699 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2701 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2702 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:47 2703 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:47 2704 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2753 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2753_fsm 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2753_vm 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2754 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2755 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2756 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 2757 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2830 2024-08-02T18:02:45.468 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2831 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2832 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2833 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2834 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2835 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2836 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2837 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Aug 2 17:46 2838 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2838_fsm 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2838_vm 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2839 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2840_fsm 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2840_vm 2024-08-02T18:02:45.469 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2841 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2995 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2996 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3079_fsm 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3079_vm 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3080 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3081 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:46 3085 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3118 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3119 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Aug 2 17:46 3164 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3256 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3257 2024-08-02T18:02:45.470 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3258 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3350 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3351 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3379 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3380 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3381 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3394 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3394_fsm 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3394_vm 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3395 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3429 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3430 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3431 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3433 2024-08-02T18:02:45.471 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3439 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3440 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Aug 2 17:47 3455 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Aug 2 17:46 3456 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3456_fsm 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3456_vm 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3466 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3467 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3468 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3501 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3502 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3503 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3534 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541 2024-08-02T18:02:45.472 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3541_fsm 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3541_vm 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3542 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3574 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3575 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3576 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3596 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3597 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3598 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3599 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3600_fsm 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3600_vm 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601 2024-08-02T18:02:45.473 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3601_fsm 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3601_vm 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3602_fsm 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3602_vm 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3603_fsm 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3603_vm 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3604 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3605 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3606 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3607 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3608 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Aug 2 17:46 3609 2024-08-02T18:02:45.474 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3712 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 3764_fsm 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3764_vm 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3766 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 3767 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3997 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4143 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4144 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4145 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4146 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4147 2024-08-02T18:02:45.475 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4148 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4149 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4150 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4151 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4152 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4153 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4154 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4155 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4156 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4157 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4158 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4159 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4160 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4161 2024-08-02T18:02:45.476 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4162 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4163 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4164 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4165 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4166 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4167 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4168 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4169 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4170 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4171 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4172 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4173 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4174 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 5002 2024-08-02T18:02:45.477 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 548 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 549 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6102 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6104 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6106 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6110 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6111 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6112 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6113 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6117 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 826 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 827 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 828 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 2 17:46 pg_filenode.map 2024-08-02T18:02:45.478 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Aug 2 17:46 pg_internal.init 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 17:46 PG_VERSION 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:47 . 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Aug 2 17:47 .. 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/global: 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:total 560 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 2 17:46 . 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1213 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1213_fsm 2024-08-02T18:02:45.479 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1213_vm 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1214 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1214_fsm 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1214_vm 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 1232 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 1233 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1260 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1260_fsm 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1260_vm 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1261 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1261_fsm 2024-08-02T18:02:45.480 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1261_vm 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1262 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 1262_fsm 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 1262_vm 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2396 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Aug 2 17:46 2396_fsm 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2396_vm 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2397 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2671 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2672 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2676 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2677 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2694 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2695 2024-08-02T18:02:45.481 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2697 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Aug 2 17:46 2698 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2846 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2847 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2964 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2965 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 2966 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 2967 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 3592 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 3593 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4060 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4061 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4175 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4176 2024-08-02T18:02:45.482 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4177 2024-08-02T18:02:45.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4178 2024-08-02T18:02:45.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4181 2024-08-02T18:02:45.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4182 2024-08-02T18:02:45.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4183 2024-08-02T18:02:45.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4184 2024-08-02T18:02:45.483 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 4185 2024-08-02T18:02:45.486 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 4186 2024-08-02T18:02:45.486 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6000 2024-08-02T18:02:45.486 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6001 2024-08-02T18:02:45.486 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6002 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 6100 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6114 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 6115 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 18:02 pg_control 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Aug 2 17:46 pg_filenode.map 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Aug 2 17:46 pg_internal.init 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/log: 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:total 2 2024-08-02T18:02:45.487 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1380 Aug 2 18:02 postgresql-Fri.log 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.488 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:total 1 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 18:02 . 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 mappings 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Aug 2 18:02 replorigin_checkpoint 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 snapshots 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 18:02 .. 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.489 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 18:02 .. 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 . 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 members 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 offsets 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-08-02T18:02:45.490 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 .. 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:46 0000 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:total 8 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Aug 2 17:46 .. 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Aug 2 17:52 0000 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.491 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.492 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:total 15 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Aug 2 18:02 . 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Aug 2 18:02 db_0.stat 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Aug 2 18:02 db_13494.stat 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Aug 2 18:02 global.stat 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 18:02 . 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.493 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:total 120 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 18:02 . 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Aug 2 18:02 000B 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-08-02T18:02:45.494 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:total 933888 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 2 18:02 . 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 18:02 000000010000000000000063 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:54 000000010000000000000064 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 000000010000000000000065 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 000000010000000000000066 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:54 000000010000000000000067 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 000000010000000000000068 2024-08-02T18:02:45.495 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 000000010000000000000069 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 00000001000000000000006A 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 00000001000000000000006B 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:55 00000001000000000000006C 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:56 00000001000000000000006D 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 00000001000000000000006E 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 00000001000000000000006F 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:55 000000010000000000000070 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 000000010000000000000071 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:54 000000010000000000000072 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 000000010000000000000073 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 000000010000000000000074 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 000000010000000000000075 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 000000010000000000000076 2024-08-02T18:02:45.496 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 000000010000000000000077 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:56 000000010000000000000078 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 000000010000000000000079 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 00000001000000000000007A 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 00000001000000000000007B 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 00000001000000000000007C 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:54 00000001000000000000007D 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 00000001000000000000007E 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:53 00000001000000000000007F 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:52 000000010000000000000080 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000081 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 000000010000000000000082 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000083 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 000000010000000000000084 2024-08-02T18:02:45.497 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:59 000000010000000000000085 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:59 000000010000000000000086 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000087 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 000000010000000000000088 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 18:01 000000010000000000000089 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 00000001000000000000008A 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 18:01 00000001000000000000008B 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 00000001000000000000008C 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 18:00 00000001000000000000008D 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 00000001000000000000008E 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 00000001000000000000008F 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000090 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 000000010000000000000091 2024-08-02T18:02:45.498 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 18:00 000000010000000000000092 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000093 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000094 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 000000010000000000000095 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000096 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:58 000000010000000000000097 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000098 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 000000010000000000000099 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 17:57 00000001000000000000009A 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Aug 2 18:01 00000001000000000000009B 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 archive_status 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:total 0 2024-08-02T18:02:45.499 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Aug 2 17:46 . 2024-08-02T18:02:45.500 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Aug 2 18:02 .. 2024-08-02T18:02:45.500 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:45.500 INFO:teuthology.orchestra.run.smithi069.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-08-02T18:02:45.500 INFO:teuthology.orchestra.run.smithi069.stdout:total 184 2024-08-02T18:02:45.500 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Aug 2 17:46 . 2024-08-02T18:02:45.500 INFO:teuthology.orchestra.run.smithi069.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Aug 2 18:02 .. 2024-08-02T18:02:45.500 INFO:teuthology.orchestra.run.smithi069.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 Aug 2 18:02 0000 2024-08-02T18:02:45.500 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-08-02T18:02:45.512 INFO:tasks.quiescer:joining Quiescers 2024-08-02T18:02:45.512 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-08-02T18:02:45.513 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:02:46.216 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f6c7550a', total seconds elapsed: 33.1 2024-08-02T18:02:46.216 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.95 2024-08-02T18:02:46.216 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:02:46.236 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:46.261 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 -- 172.21.15.69:0/450597098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 msgr2=0x7f3fd8100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/450597098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 0x7f3fd8100420 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009900 tx=0x7f3fd402a3f0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 -- 172.21.15.69:0/450597098 shutdown_connections 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/450597098 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd8106c60 0x7f3fd810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/450597098 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fd81009f0 0x7f3fd8106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/450597098 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 0x7f3fd8100420 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 -- 172.21.15.69:0/450597098 >> 172.21.15.69:0/450597098 conn(0x7f3fd8076f40 msgr2=0x7f3fd80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 -- 172.21.15.69:0/450597098 shutdown_connections 2024-08-02T18:02:46.977 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 -- 172.21.15.69:0/450597098 wait complete. 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 Processor -- start 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.965+0000 7f3fdfb06640 1 -- start start 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdfb06640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 0x7f3fd8073650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdfb06640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81009f0 0x7f3fd8073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdfb06640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fd8106c60 0x7f3fd806c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdfb06640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8111c80 con 0x7f3fd8100040 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdfb06640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8111b00 con 0x7f3fd8106c60 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdfb06640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3fd8111e00 con 0x7f3fd81009f0 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 0x7f3fd8073650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 0x7f3fd8073650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46002/0 (socket says 172.21.15.69:46002) 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 -- 172.21.15.69:0/1939663754 learned_addr learned my addr 172.21.15.69:0/1939663754 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd0a1640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81009f0 0x7f3fd8073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.978 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fd8106c60 0x7f3fd806c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 -- 172.21.15.69:0/1939663754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81009f0 msgr2=0x7f3fd8073b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81009f0 0x7f3fd8073b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 -- 172.21.15.69:0/1939663754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fd8106c60 msgr2=0x7f3fd806c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fd8106c60 0x7f3fd806c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 -- 172.21.15.69:0/1939663754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fd806d180 con 0x7f3fd8100040 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd0a1640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81009f0 0x7f3fd8073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fd8106c60 0x7f3fd806c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 0x7f3fd8073650 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4005ab0 tx=0x7f3fd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fceffd640 1 -- 172.21.15.69:0/1939663754 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd403f030 con 0x7f3fd8100040 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdfb06640 1 -- 172.21.15.69:0/1939663754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fd806d410 con 0x7f3fd8100040 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fdfb06640 1 -- 172.21.15.69:0/1939663754 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fd80713a0 con 0x7f3fd8100040 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fceffd640 1 -- 172.21.15.69:0/1939663754 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd4003ce0 con 0x7f3fd8100040 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.966+0000 7f3fceffd640 1 -- 172.21.15.69:0/1939663754 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4049060 con 0x7f3fd8100040 2024-08-02T18:02:46.979 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fceffd640 1 -- 172.21.15.69:0/1939663754 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3fd4042030 con 0x7f3fd8100040 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fe1f04740 1 Processor -- start 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fe1f04740 1 -- start start 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceaff0 0x5652f8ceb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8cf5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 0x5652f8cf85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652f8c50a40 con 0x5652f8ceaff0 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652f8be89d0 con 0x5652f8cf61d0 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.969+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5652f8c4ca00 con 0x5652f8ceb910 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceaff0 0x5652f8ceb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fceffd640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3fc40a2cf0 0x7f3fc40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceaff0 0x5652f8ceb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46006/0 (socket says 172.21.15.69:46006) 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 -- 172.21.15.69:0/311285719 learned_addr learned my addr 172.21.15.69:0/311285719 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8cf5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.980 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fdd0a1640 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 0x5652f8cf85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3fc40a2cf0 0x7f3fc40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fceffd640 1 -- 172.21.15.69:0/1939663754 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f3fd40ef760 con 0x7f3fd8100040 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 -- 172.21.15.69:0/311285719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 msgr2=0x5652f8cf5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8cf5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 -- 172.21.15.69:0/311285719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 msgr2=0x5652f8cf85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 0x5652f8cf85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 -- 172.21.15.69:0/311285719 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f8bef650 con 0x5652f8ceaff0 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8cf5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceaff0 0x5652f8ceb3d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00149a0 tx=0x7f3fd0014e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b49b8065c860ac server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fc3fff640 1 -- 172.21.15.69:0/311285719 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0021070 con 0x5652f8ceaff0 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fc3fff640 1 -- 172.21.15.69:0/311285719 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd0013460 con 0x5652f8ceaff0 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.970+0000 7f3fc3fff640 1 -- 172.21.15.69:0/311285719 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0008e20 con 0x5652f8ceaff0 2024-08-02T18:02:46.981 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 -- 172.21.15.69:0/311285719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceaff0 msgr2=0x5652f8ceb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceaff0 0x5652f8ceb3d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00149a0 tx=0x7f3fd0014e60 comp rx=0 tx=0).stop 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3fc40a2cf0 0x7f3fc40a51b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3fd001e000 tx=0x7f3fd0019000 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 -- 172.21.15.69:0/311285719 shutdown_connections 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 0x5652f8cf85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8cf5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/311285719 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceaff0 0x5652f8ceb3d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 -- 172.21.15.69:0/311285719 >> 172.21.15.69:0/311285719 conn(0x5652f8ce7580 msgr2=0x5652f8ce7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 -- 172.21.15.69:0/311285719 shutdown_connections 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 -- 172.21.15.69:0/311285719 wait complete. 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.971+0000 7f3fe1f04740 1 -- start start 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fe1f04740 1 Processor -- start 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 0x5652f8cf2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceb910 0x5652f8cf34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.982 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8cf3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x5652f8cf8da0 con 0x5652f8cf61d0 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x5652f8c0bb20 con 0x5652f8ceb910 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x5652f8cf8f20 con 0x5652f8ceaff0 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fe1f04740 10 client.0 _ll_register_callbacks cb 0x5652f8c09bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fde0a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8cf3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fde0a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8cf3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46018/0 (socket says 172.21.15.69:46018) 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fde0a3640 1 -- 172.21.15.69:0/3203951807 learned_addr learned my addr 172.21.15.69:0/3203951807 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fdd0a1640 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceb910 0x5652f8cf34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 0x5652f8cf2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fde0a3640 1 -- 172.21.15.69:0/3203951807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 msgr2=0x5652f8cf2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:46.983 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 0x5652f8cf2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fde0a3640 1 -- 172.21.15.69:0/3203951807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceb910 msgr2=0x5652f8cf34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceb910 0x5652f8cf34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.972+0000 7f3fde0a3640 1 -- 172.21.15.69:0/3203951807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f89c23c0 con 0x5652f8cf61d0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8cf3a10 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0013de0 tx=0x7f3fd00118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29406683e113dbc5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3203951807 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0033060 con 0x5652f8cf61d0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3203951807 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f3fd0004a00 con 0x5652f8cf61d0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3203951807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd0013460 con 0x5652f8cf61d0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3203951807 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fd003a050 con 0x5652f8cf61d0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 10 client.19401 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3203951807 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652f8d049c0 con 0x5652f8cf61d0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3203951807 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd0013460 con 0x5652f8cf61d0 2024-08-02T18:02:46.984 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3203951807 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3fd002d030 con 0x5652f8cf61d0 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 10 client.19401 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 10 client.19401 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 20 client.19401 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5652f8d05db0 0x5652f8d081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 4 client.19401 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.973+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3203951807 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652f8c5dfb0 con 0x5652f8d05db0 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.974+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5652f8d05db0 0x5652f8d081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.974+0000 7f3fc17fa640 10 client.19401 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.974+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5652f8d05db0 0x5652f8d081f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5073c6d7803919ea server_cookie=e06a8a98e3bbd1e9 in_seq=0 out_seq=0 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3203951807 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5652f8c5dfb0 con 0x5652f8d05db0 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fc17fa640 10 client.19401 handle_command_reply: tid=0 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 client.19401 shutdown 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3203951807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 msgr2=0x5652f8cf3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:46.985 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8cf3a10 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0013de0 tx=0x7f3fd00118e0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3203951807 shutdown_connections 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5652f8d05db0 0x5652f8d081f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8cf3a10 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceb910 0x5652f8cf34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3203951807 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 0x5652f8cf2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fc17fa640 0 client.19401 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3203951807 >> 172.21.15.69:0/3203951807 conn(0x5652f8ce7580 msgr2=0x5652f8ced260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3203951807 shutdown_connections 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3203951807 wait complete. 2024-08-02T18:02:46.986 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.975+0000 7f3fe1f04740 20 client.19401 trim_cache size 0 max 16384 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.988+0000 7f3fe1f04740 1 Processor -- start 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.988+0000 7f3fe1f04740 1 -- start start 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.988+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceaff0 0x5652f8ce64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.988+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8d0dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.988+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8d0e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.988+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc012930 con 0x5652f8cf61d0 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.988+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc010200 con 0x5652f8ceaff0 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.988+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc00e050 con 0x5652f8ceb910 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8d0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8d0e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46028/0 (socket says 172.21.15.69:46028) 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 -- 172.21.15.69:0/2440307346 learned_addr learned my addr 172.21.15.69:0/2440307346 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fdd0a1640 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8d0dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.991 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceaff0 0x5652f8ce64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 -- 172.21.15.69:0/2440307346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 msgr2=0x5652f8d0dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8d0dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 -- 172.21.15.69:0/2440307346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceaff0 msgr2=0x5652f8ce64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceaff0 0x5652f8ce64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 -- 172.21.15.69:0/2440307346 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652f8cf5ae0 con 0x5652f8cf61d0 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8d0e000 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0011830 tx=0x7f3fd0013d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524274d7c901a96a server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fc37fe640 1 -- 172.21.15.69:0/2440307346 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0033050 con 0x5652f8cf61d0 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fc37fe640 1 -- 172.21.15.69:0/2440307346 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd0013e10 con 0x5652f8cf61d0 2024-08-02T18:02:46.992 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.989+0000 7f3fc37fe640 1 -- 172.21.15.69:0/2440307346 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0021040 con 0x5652f8cf61d0 2024-08-02T18:02:46.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 -- 172.21.15.69:0/2440307346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 msgr2=0x5652f8d0e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:46.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8d0e000 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0011830 tx=0x7f3fd0013d70 comp rx=0 tx=0).stop 2024-08-02T18:02:46.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 -- 172.21.15.69:0/2440307346 shutdown_connections 2024-08-02T18:02:46.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8cf61d0 0x5652f8d0e000 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceb910 0x5652f8d0dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/2440307346 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8ceaff0 0x5652f8ce64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 -- 172.21.15.69:0/2440307346 >> 172.21.15.69:0/2440307346 conn(0x5652f8ce7580 msgr2=0x5652f8ce5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:46.995 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 -- 172.21.15.69:0/2440307346 shutdown_connections 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 -- 172.21.15.69:0/2440307346 wait complete. 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 -- start start 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.990+0000 7f3fe1f04740 1 Processor -- start 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 0x5652f8cf6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceb910 0x5652f8cf7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fe1f04740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 0x5652f8cfd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc00e6c0 con 0x5652f8ceb910 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc00fb30 con 0x5652f8cf61d0 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fe1f04740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3fbc00ed90 con 0x5652f8ceaff0 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fe1f04740 10 client.0 _ll_register_callbacks cb 0x7f3fd001f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceb910 0x5652f8cf7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceb910 0x5652f8cf7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46040/0 (socket says 172.21.15.69:46040) 2024-08-02T18:02:46.996 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 -- 172.21.15.69:0/3364550394 learned_addr learned my addr 172.21.15.69:0/3364550394 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 0x5652f8cf6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fde0a3640 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 0x5652f8cfd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 -- 172.21.15.69:0/3364550394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 msgr2=0x5652f8cf6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 0x5652f8cf6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 -- 172.21.15.69:0/3364550394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 msgr2=0x5652f8cfd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 0x5652f8cfd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 -- 172.21.15.69:0/3364550394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652f8cfd740 con 0x5652f8ceb910 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fdd0a1640 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceb910 0x5652f8cf7290 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f3fd806dea0 tx=0x7f3fc800a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a4b11eaa3e2317 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.991+0000 7f3fc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3364550394 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8021070 con 0x5652f8ceb910 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3364550394 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f3fc8005220 con 0x5652f8ceb910 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3364550394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fd0013e10 con 0x5652f8ceb910 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3364550394 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3fc802c050 con 0x5652f8ceb910 2024-08-02T18:02:46.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 10 client.19407 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3364550394 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5652f8cfea90 con 0x5652f8ceb910 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3364550394 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fd0013e10 con 0x5652f8ceb910 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3364550394 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3fc801d540 con 0x5652f8ceb910 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 10 client.19407 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 10 client.19407 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 20 client.19407 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5652f8d70cf0 0x5652f8d73110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 4 client.19407 mds_command: new command op to 24427 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3364550394 --> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5652f8c584c0 con 0x5652f8d70cf0 2024-08-02T18:02:46.998 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.992+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5652f8d70cf0 0x5652f8d73110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:46.998 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:46 smithi069 ceph-mon[32746]: pgmap v513: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-08-02T18:02:46.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fdd8a2640 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5652f8d70cf0 0x5652f8d73110 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b67f3356580ce2e4 server_cookie=ff59742c9655464 in_seq=0 out_seq=0 2024-08-02T18:02:46.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fc17fa640 10 client.19407 ms_handle_connect on v2:172.21.15.83:6838/2418869426 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fc17fa640 1 -- 172.21.15.69:0/3364550394 <== mds.2 v2:172.21.15.83:6838/2418869426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5652f8c584c0 con 0x5652f8d70cf0 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fc17fa640 10 client.19407 handle_command_reply: tid=0 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 client.19407 shutdown 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3364550394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceb910 msgr2=0x5652f8cf7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceb910 0x5652f8cf7290 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f3fd806dea0 tx=0x7f3fc800a6e0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3364550394 shutdown_connections 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.83:6838/2418869426,v1:172.21.15.83:6839/2418869426] conn(0x5652f8d70cf0 0x5652f8d73110 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x5652f8ceb910 0x5652f8cf7290 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x5652f8cf61d0 0x5652f8cfd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fc17fa640 0 client.19407 ms_handle_reset on v2:172.21.15.83:6838/2418869426 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 --2- 172.21.15.69:0/3364550394 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x5652f8ceaff0 0x5652f8cf6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.993+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3364550394 >> 172.21.15.69:0/3364550394 conn(0x5652f8ce7580 msgr2=0x5652f8ce65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3364550394 shutdown_connections 2024-08-02T18:02:47.000 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fe1f04740 1 -- 172.21.15.69:0/3364550394 wait complete. 2024-08-02T18:02:47.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fe1f04740 20 client.19407 trim_cache size 0 max 16384 2024-08-02T18:02:47.001 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:02:47.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 -- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3fc40a2cf0 msgr2=0x7f3fc40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:47.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3fc40a2cf0 0x7f3fc40a51b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3fd001e000 tx=0x7f3fd0019000 comp rx=0 tx=0).stop 2024-08-02T18:02:47.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 -- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 msgr2=0x7f3fd8073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 0x7f3fd8073650 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4005ab0 tx=0x7f3fd40042f0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 -- 172.21.15.69:0/1939663754 shutdown_connections 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f3fc40a2cf0 0x7f3fc40a51b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f3fd8106c60 0x7f3fd806c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3fd81009f0 0x7f3fd8073b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 --2- 172.21.15.69:0/1939663754 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f3fd8100040 0x7f3fd8073650 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 -- 172.21.15.69:0/1939663754 >> 172.21.15.69:0/1939663754 conn(0x7f3fd8076f40 msgr2=0x7f3fd8105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 -- 172.21.15.69:0/1939663754 shutdown_connections 2024-08-02T18:02:47.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:46.994+0000 7f3fdfb06640 1 -- 172.21.15.69:0/1939663754 wait complete. 2024-08-02T18:02:47.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:46 smithi083 ceph-mon[37031]: pgmap v513: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-08-02T18:02:47.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:46 smithi077 ceph-mon[37108]: pgmap v513: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-08-02T18:02:47.676 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-02T18:02:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 -- 172.21.15.69:0/150121678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d10102c20 msgr2=0x7f6d1010d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 --2- 172.21.15.69:0/150121678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d10102c20 0x7f6d1010d770 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009ca0 tx=0x7f6d0802a950 comp rx=0 tx=0).stop 2024-08-02T18:02:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 -- 172.21.15.69:0/150121678 shutdown_connections 2024-08-02T18:02:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 --2- 172.21.15.69:0/150121678 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d10102c20 0x7f6d1010d770 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 --2- 172.21.15.69:0/150121678 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d101022a0 0x7f6d101026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 --2- 172.21.15.69:0/150121678 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d10100b50 0x7f6d10100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 -- 172.21.15.69:0/150121678 >> 172.21.15.69:0/150121678 conn(0x7f6d100fc500 msgr2=0x7f6d100fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:47.881 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 -- 172.21.15.69:0/150121678 shutdown_connections 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.860+0000 7f6d15965640 1 -- 172.21.15.69:0/150121678 wait complete. 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d15965640 1 Processor -- start 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d15965640 1 -- start start 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d15965640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d10100b50 0x7f6d10196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d15965640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d101022a0 0x7f6d10196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d15965640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d10102c20 0x7f6d1019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d15965640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010fbc0 con 0x7f6d101022a0 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d15965640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010fa40 con 0x7f6d10100b50 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d15965640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6d1010fd40 con 0x7f6d10102c20 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d0e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d101022a0 0x7f6d10196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d0e7fc640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d101022a0 0x7f6d10196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46046/0 (socket says 172.21.15.69:46046) 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d0e7fc640 1 -- 172.21.15.69:0/146802078 learned_addr learned my addr 172.21.15.69:0/146802078 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:47.882 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.861+0000 7f6d0f7fe640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d10102c20 0x7f6d1019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d0effd640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d10100b50 0x7f6d10196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d0e7fc640 1 -- 172.21.15.69:0/146802078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d10102c20 msgr2=0x7f6d1019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d0e7fc640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d10102c20 0x7f6d1019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d0e7fc640 1 -- 172.21.15.69:0/146802078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d10100b50 msgr2=0x7f6d10196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d0e7fc640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d10100b50 0x7f6d10196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d0e7fc640 1 -- 172.21.15.69:0/146802078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1019afd0 con 0x7f6d101022a0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d0e7fc640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d101022a0 0x7f6d10196560 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000d770 tx=0x7f6d0000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d14963640 1 -- 172.21.15.69:0/146802078 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00017070 con 0x7f6d101022a0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1019b2c0 con 0x7f6d101022a0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d14963640 1 -- 172.21.15.69:0/146802078 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d00005500 con 0x7f6d101022a0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d14963640 1 -- 172.21.15.69:0/146802078 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00004e00 con 0x7f6d101022a0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.862+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d101a2ba0 con 0x7f6d101022a0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.863+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d10100f30 con 0x7f6d101022a0 2024-08-02T18:02:47.883 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.865+0000 7f6d14963640 1 -- 172.21.15.69:0/146802078 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6d00004fa0 con 0x7f6d101022a0 2024-08-02T18:02:47.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.866+0000 7f6d14963640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cf80a4f00 0x7f6cf80a73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:47.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.866+0000 7f6d0effd640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cf80a4f00 0x7f6cf80a73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:47.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.866+0000 7f6d14963640 1 -- 172.21.15.69:0/146802078 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f6d000cf510 con 0x7f6d101022a0 2024-08-02T18:02:47.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.867+0000 7f6d0effd640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cf80a4f00 0x7f6cf80a73c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d04005d00 tx=0x7f6d0400b410 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:47.884 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:47.870+0000 7f6d14963640 1 -- 172.21.15.69:0/146802078 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f6d0009b770 con 0x7f6d101022a0 2024-08-02T18:02:47.935 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:48.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.108+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d10108dc0 con 0x7f6d101022a0 2024-08-02T18:02:48.110 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.109+0000 7f6d14963640 1 -- 172.21.15.69:0/146802078 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11901 (secure 0 0 0) 0x7f6d0009fa70 con 0x7f6d101022a0 2024-08-02T18:02:48.110 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:48.111 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"btime":"2024-08-02T18:01:55:674676+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":28375,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3960673726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3960673726},{"type":"v1","addr":"172.21.15.77:6835","nonce":3960673726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T18:01:54.670284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24389":{"gid":24389,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:02:48.111 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cf80a4f00 msgr2=0x7f6cf80a73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cf80a4f00 0x7f6cf80a73c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6d04005d00 tx=0x7f6d0400b410 comp rx=0 tx=0).stop 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d101022a0 msgr2=0x7f6d10196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d101022a0 0x7f6d10196560 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000d770 tx=0x7f6d0000dc30 comp rx=0 tx=0).stop 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 shutdown_connections 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6d101022a0 0x7f6d10196560 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6cf80a4f00 0x7f6cf80a73c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6d10102c20 0x7f6d1019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 --2- 172.21.15.69:0/146802078 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6d10100b50 0x7f6d10196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:48.113 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 >> 172.21.15.69:0/146802078 conn(0x7f6d100fc500 msgr2=0x7f6d10106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 shutdown_connections 2024-08-02T18:02:48.114 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:48.111+0000 7f6d15965640 1 -- 172.21.15.69:0/146802078 wait complete. 2024-08-02T18:02:48.913 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db --cancel --all' 2024-08-02T18:02:48.913 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db --cancel --all 2024-08-02T18:02:49.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1739737512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58100b50 msgr2=0x7ffb58100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:49.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1739737512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58100b50 0x7ffb58100f30 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ffb48009900 tx=0x7ffb4802a480 comp rx=0 tx=0).stop 2024-08-02T18:02:49.189 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1739737512 shutdown_connections 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1739737512 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb58102c20 0x7ffb5810d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1739737512 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb581022a0 0x7ffb581026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1739737512 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb58100b50 0x7ffb58100f30 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1739737512 >> 172.21.15.69:0/1739737512 conn(0x7ffb580fc500 msgr2=0x7ffb580fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1739737512 shutdown_connections 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.119+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1739737512 wait complete. 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.120+0000 7ffb5d0a4640 1 Processor -- start 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.120+0000 7ffb5d0a4640 1 -- start start 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.120+0000 7ffb5d0a4640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb58100b50 0x7ffb58197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.120+0000 7ffb5d0a4640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb581022a0 0x7ffb58197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.120+0000 7ffb5d0a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb58102c20 0x7ffb5819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:49.190 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.120+0000 7ffb5d0a4640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffb58110cb0 con 0x7ffb581022a0 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.120+0000 7ffb5d0a4640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb58110b30 con 0x7ffb58100b50 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.120+0000 7ffb5d0a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb58110e30 con 0x7ffb58102c20 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb581022a0 0x7ffb58197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb581022a0 0x7ffb58197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46066/0 (socket says 172.21.15.69:46066) 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 -- 172.21.15.69:0/1146627783 learned_addr learned my addr 172.21.15.69:0/1146627783 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5759e640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb58102c20 0x7ffb5819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb56d9d640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb58100b50 0x7ffb58197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 -- 172.21.15.69:0/1146627783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb58102c20 msgr2=0x7ffb5819ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb58102c20 0x7ffb5819ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 -- 172.21.15.69:0/1146627783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb58100b50 msgr2=0x7ffb58197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb58100b50 0x7ffb58197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 -- 172.21.15.69:0/1146627783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5819c0f0 con 0x7ffb581022a0 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5759e640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb58102c20 0x7ffb5819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:49.191 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb56d9d640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb58100b50 0x7ffb58197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5659c640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb581022a0 0x7ffb58197680 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00e9f0 tx=0x7ffb4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb3ffff640 1 -- 172.21.15.69:0/1146627783 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00cd80 con 0x7ffb581022a0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb5819c3e0 con 0x7ffb581022a0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb3ffff640 1 -- 172.21.15.69:0/1146627783 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb4c005500 con 0x7ffb581022a0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb3ffff640 1 -- 172.21.15.69:0/1146627783 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c010690 con 0x7ffb581022a0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.121+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb581a3cc0 con 0x7ffb581022a0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.122+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb58100f30 con 0x7ffb581022a0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.124+0000 7ffb3ffff640 1 -- 172.21.15.69:0/1146627783 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffb4c0046c0 con 0x7ffb581022a0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.125+0000 7ffb3ffff640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb400a2cf0 0x7ffb400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.125+0000 7ffb56d9d640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb400a2cf0 0x7ffb400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.125+0000 7ffb3ffff640 1 -- 172.21.15.69:0/1146627783 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7ffb4c0d0010 con 0x7ffb581022a0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.125+0000 7ffb56d9d640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb400a2cf0 0x7ffb400a51b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffb48005ab0 tx=0x7ffb48005a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:49.192 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.128+0000 7ffb3ffff640 1 -- 172.21.15.69:0/1146627783 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ffb4c053df0 con 0x7ffb581022a0 2024-08-02T18:02:49.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:48 smithi069 ceph-mon[32746]: pgmap v514: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-08-02T18:02:49.193 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:48 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/146802078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:49.195 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:48 smithi083 ceph-mon[37031]: pgmap v514: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-08-02T18:02:49.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:48 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/146802078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:49.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:48 smithi077 ceph-mon[37108]: pgmap v514: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 27 KiB/s wr, 17 op/s 2024-08-02T18:02:49.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:48 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/146802078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:49.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.348+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb581984a0 con 0x7ffb581022a0 2024-08-02T18:02:49.350 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.349+0000 7ffb3ffff640 1 -- 172.21.15.69:0/1146627783 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11901 (secure 0 0 0) 0x7ffb581984a0 con 0x7ffb581022a0 2024-08-02T18:02:49.351 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:49.351 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"btime":"2024-08-02T18:01:55:674676+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":28375,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3960673726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3960673726},{"type":"v1","addr":"172.21.15.77:6835","nonce":3960673726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T18:01:54.670284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24389":{"gid":24389,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:02:49.351 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-08-02T18:02:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb400a2cf0 msgr2=0x7ffb400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb400a2cf0 0x7ffb400a51b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ffb48005ab0 tx=0x7ffb48005a20 comp rx=0 tx=0).stop 2024-08-02T18:02:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb581022a0 msgr2=0x7ffb58197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb581022a0 0x7ffb58197680 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00e9f0 tx=0x7ffb4c00edf0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 shutdown_connections 2024-08-02T18:02:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb581022a0 0x7ffb58197680 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb400a2cf0 0x7ffb400a51b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb58102c20 0x7ffb5819ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 --2- 172.21.15.69:0/1146627783 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb58100b50 0x7ffb58197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:49.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 >> 172.21.15.69:0/1146627783 conn(0x7ffb580fc500 msgr2=0x7ffb580fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:49.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 shutdown_connections 2024-08-02T18:02:49.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:49.351+0000 7ffb5d0a4640 1 -- 172.21.15.69:0/1146627783 wait complete. 2024-08-02T18:02:50.213 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:49 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1146627783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:50.241 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24452 quiesce db / --timeout 90 --expiration 147.95 --await-for 45.0' 2024-08-02T18:02:50.241 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- timeout 105 ceph tell mds.24452 quiesce db / --timeout 90 --expiration 147.95 --await-for 45.0 2024-08-02T18:02:50.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:49 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1146627783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:50.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:49 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1146627783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:02:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/2386434104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a68107810 msgr2=0x7f7a68107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/2386434104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a68107810 0x7f7a68107bf0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009930 tx=0x7f7a5802a420 comp rx=0 tx=0).stop 2024-08-02T18:02:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/2386434104 shutdown_connections 2024-08-02T18:02:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/2386434104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a681021a0 0x7f7a6810e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/2386434104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a68101800 0x7f7a68101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/2386434104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a68107810 0x7f7a68107bf0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/2386434104 >> 172.21.15.69:0/2386434104 conn(0x7f7a680fd500 msgr2=0x7f7a680ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/2386434104 shutdown_connections 2024-08-02T18:02:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/2386434104 wait complete. 2024-08-02T18:02:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6ce4c640 1 Processor -- start 2024-08-02T18:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6ce4c640 1 -- start start 2024-08-02T18:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6ce4c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a68101800 0x7f7a68074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6ce4c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a681021a0 0x7f7a68074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6ce4c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a68107810 0x7f7a68074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6ce4c640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68111ca0 con 0x7f7a681021a0 2024-08-02T18:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6ce4c640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68111b20 con 0x7f7a68107810 2024-08-02T18:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6ce4c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a68111e20 con 0x7f7a68101800 2024-08-02T18:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a65d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a681021a0 0x7f7a68074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a68101800 0x7f7a68074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a65d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a681021a0 0x7f7a68074760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46110/0 (socket says 172.21.15.69:46110) 2024-08-02T18:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a68101800 0x7f7a68074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47304/0 (socket says 172.21.15.69:47304) 2024-08-02T18:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.469+0000 7f7a6659c640 1 -- 172.21.15.69:0/3733350692 learned_addr learned my addr 172.21.15.69:0/3733350692 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a66d9d640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a68107810 0x7f7a68074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a6659c640 1 -- 172.21.15.69:0/3733350692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a68107810 msgr2=0x7f7a68074ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a6659c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a68107810 0x7f7a68074ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a6659c640 1 -- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a681021a0 msgr2=0x7f7a68074760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a6659c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a681021a0 0x7f7a68074760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a6659c640 1 -- 172.21.15.69:0/3733350692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a681a6690 con 0x7f7a68101800 2024-08-02T18:02:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a681021a0 0x7f7a68074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a6659c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a68101800 0x7f7a68074220 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a5802a940 tx=0x7f7a58003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a577fe640 1 -- 172.21.15.69:0/3733350692 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5803f070 con 0x7f7a68101800 2024-08-02T18:02:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/3733350692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a681a6920 con 0x7f7a68101800 2024-08-02T18:02:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a577fe640 1 -- 172.21.15.69:0/3733350692 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a58031c90 con 0x7f7a68101800 2024-08-02T18:02:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a577fe640 1 -- 172.21.15.69:0/3733350692 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58049060 con 0x7f7a68101800 2024-08-02T18:02:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/3733350692 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a681a6da0 con 0x7f7a68101800 2024-08-02T18:02:50.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 -- 172.21.15.69:0/23168550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c41007b0 msgr2=0x7f59c4100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.481 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/23168550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c41007b0 0x7f59c4100c10 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009930 tx=0x7f59c002a420 comp rx=0 tx=0).stop 2024-08-02T18:02:50.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 -- 172.21.15.69:0/23168550 shutdown_connections 2024-08-02T18:02:50.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/23168550 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c41007b0 0x7f59c4100c10 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/23168550 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c4101150 0x7f59c410d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/23168550 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c41067b0 0x7f59c4106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 -- 172.21.15.69:0/23168550 >> 172.21.15.69:0/23168550 conn(0x7f59c40fc4d0 msgr2=0x7f59c40fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.482 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 -- 172.21.15.69:0/23168550 shutdown_connections 2024-08-02T18:02:50.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 -- 172.21.15.69:0/23168550 wait complete. 2024-08-02T18:02:50.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.466+0000 7f59cb5a1640 1 Processor -- start 2024-08-02T18:02:50.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59cb5a1640 1 -- start start 2024-08-02T18:02:50.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59cb5a1640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41007b0 0x7f59c4198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.483 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59cb5a1640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c4101150 0x7f59c41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59cb5a1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c41067b0 0x7f59c41a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59cb5a1640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4110b70 con 0x7f59c4101150 2024-08-02T18:02:50.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59cb5a1640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41109f0 con 0x7f59c41007b0 2024-08-02T18:02:50.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59cb5a1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4110cf0 con 0x7f59c41067b0 2024-08-02T18:02:50.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c8b3c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c4101150 0x7f59c41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c933d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41007b0 0x7f59c4198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.484 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c8b3c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c4101150 0x7f59c41994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46104/0 (socket says 172.21.15.69:46104) 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c8b3c640 1 -- 172.21.15.69:0/971460673 learned_addr learned my addr 172.21.15.69:0/971460673 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c41067b0 0x7f59c41a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c8b3c640 1 -- 172.21.15.69:0/971460673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c41067b0 msgr2=0x7f59c41a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c41067b0 0x7f59c41a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c8b3c640 1 -- 172.21.15.69:0/971460673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41007b0 msgr2=0x7f59c4198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41007b0 0x7f59c4198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.467+0000 7f59c8b3c640 1 -- 172.21.15.69:0/971460673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c41a4560 con 0x7f59c4101150 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c4101150 0x7f59c41994e0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009230 tx=0x7f59c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f59b67fc640 1 -- 172.21.15.69:0/971460673 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c003f030 con 0x7f59c4101150 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f59b67fc640 1 -- 172.21.15.69:0/971460673 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59c0003ad0 con 0x7f59c4101150 2024-08-02T18:02:50.485 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f59b67fc640 1 -- 172.21.15.69:0/971460673 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0035070 con 0x7f59c4101150 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f59cb5a1640 1 -- 172.21.15.69:0/971460673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c41a4730 con 0x7f59c4101150 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.468+0000 7f59cb5a1640 1 -- 172.21.15.69:0/971460673 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c41a4be0 con 0x7f59c4101150 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.470+0000 7f59b67fc640 1 -- 172.21.15.69:0/971460673 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f59c0042030 con 0x7f59c4101150 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.471+0000 7f59cd99f740 1 Processor -- start 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.471+0000 7f59cd99f740 1 -- start start 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59b67fc640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59b00a2d70 0x7f59b00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6daf4870 0x55cd6daf6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59cd99f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd6da46a40 con 0x55cd6dae2570 2024-08-02T18:02:50.486 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59cd99f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cd6d9de9d0 con 0x55cd6dae13d0 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59cd99f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd6da42a00 con 0x55cd6daf4870 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59b67fc640 1 -- 172.21.15.69:0/971460673 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f59c00ef8d0 con 0x7f59c4101150 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46124/0 (socket says 172.21.15.69:46124) 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 -- 172.21.15.69:0/1934748562 learned_addr learned my addr 172.21.15.69:0/1934748562 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6daf4870 0x55cd6daf6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c933d640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59b00a2d70 0x7f59b00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 -- 172.21.15.69:0/1934748562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6daf4870 msgr2=0x55cd6daf6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6daf4870 0x55cd6daf6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 -- 172.21.15.69:0/1934748562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 msgr2=0x55cd6dae1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.487 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c9b3e640 1 -- 172.21.15.69:0/1934748562 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6d9e5650 con 0x55cd6dae2570 2024-08-02T18:02:50.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.472+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:50.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.473+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae0e90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005c20 tx=0x7f59bc009a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8438ba5343460b server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.473+0000 7f59c933d640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59b00a2d70 0x7f59b00a5230 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f59b800ab60 tx=0x7f59b800a2d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.473+0000 7f59af7fe640 1 -- 172.21.15.69:0/1934748562 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc0260f0 con 0x55cd6dae2570 2024-08-02T18:02:50.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.473+0000 7f59af7fe640 1 -- 172.21.15.69:0/1934748562 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59bc00eb80 con 0x55cd6dae2570 2024-08-02T18:02:50.488 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.473+0000 7f59af7fe640 1 -- 172.21.15.69:0/1934748562 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc005660 con 0x55cd6dae2570 2024-08-02T18:02:50.488 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.474+0000 7f7a577fe640 1 -- 172.21.15.69:0/3733350692 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7a58042070 con 0x7f7a68101800 2024-08-02T18:02:50.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.474+0000 7f59cd99f740 1 -- 172.21.15.69:0/1934748562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 msgr2=0x55cd6dae0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.489 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.474+0000 7f59cd99f740 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae0e90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005c20 tx=0x7f59bc009a80 comp rx=0 tx=0).stop 2024-08-02T18:02:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a6fb9c740 1 Processor -- start 2024-08-02T18:02:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a577fe640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7a500a2cf0 0x7f7a500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a577fe640 1 -- 172.21.15.69:0/3733350692 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f7a580efcc0 con 0x7f7a68101800 2024-08-02T18:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a6fb9c740 1 -- start start 2024-08-02T18:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7a500a2cf0 0x7f7a500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.474+0000 7f59cd99f740 1 -- 172.21.15.69:0/1934748562 shutdown_connections 2024-08-02T18:02:50.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.474+0000 7f59cd99f740 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6daf4870 0x55cd6daf6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.474+0000 7f59cd99f740 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.474+0000 7f59cd99f740 1 --2- 172.21.15.69:0/1934748562 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae0e90 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.474+0000 7f59cd99f740 1 -- 172.21.15.69:0/1934748562 >> 172.21.15.69:0/1934748562 conn(0x55cd6dadd580 msgr2=0x55cd6dadd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f59cd99f740 1 -- 172.21.15.69:0/1934748562 shutdown_connections 2024-08-02T18:02:50.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f59cd99f740 1 -- 172.21.15.69:0/1934748562 wait complete. 2024-08-02T18:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcddb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcde0d0 0x55fa0fce8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fcead60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fa0fc419b0 con 0x55fa0fce8970 2024-08-02T18:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fa0fbe9800 con 0x55fa0fcde0d0 2024-08-02T18:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.475+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa0fc3d970 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a66d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fcead60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f59cd99f740 1 -- start start 2024-08-02T18:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcddb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47324/0 (socket says 172.21.15.69:47324) 2024-08-02T18:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 -- 172.21.15.69:0/2944763494 learned_addr learned my addr 172.21.15.69:0/2944763494 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f59cd99f740 1 Processor -- start 2024-08-02T18:02:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7a500a2cf0 0x7f7a500a51b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a68075cb0 tx=0x7f7a5c00b390 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 -- 172.21.15.69:0/2944763494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcde0d0 msgr2=0x55fa0fce8430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-02T18:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 --2- 172.21.15.69:0/2944763494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcde0d0 0x55fa0fce8430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 -- 172.21.15.69:0/2944763494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 msgr2=0x55fa0fcead60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 --2- 172.21.15.69:0/2944763494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fcead60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 -- 172.21.15.69:0/2944763494 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0f955210 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a66d9d640 1 --2- 172.21.15.69:0/2944763494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fcead60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.476+0000 7f7a6659c640 1 --2- 172.21.15.69:0/2944763494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcddb90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7a580b87a0 tx=0x7f7a58009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb59d462bfa92d0c server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.496 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae13d0 0x55cd6dae4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f7a54ff9640 1 -- 172.21.15.69:0/2944763494 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a580bbdd0 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f7a54ff9640 1 -- 172.21.15.69:0/2944763494 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a580c0180 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f7a54ff9640 1 -- 172.21.15.69:0/2944763494 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a580c45b0 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6dae56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59cd99f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55cd6daf7540 con 0x55cd6dae2570 2024-08-02T18:02:50.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59cd99f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55cd6da01b20 con 0x55cd6daf4870 2024-08-02T18:02:50.499 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59cd99f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55cd6daf76c0 con 0x55cd6dae13d0 2024-08-02T18:02:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59cd99f740 10 client.0 _ll_register_callbacks cb 0x55cd6d9ffbb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c8b3c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c8b3c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46144/0 (socket says 172.21.15.69:46144) 2024-08-02T18:02:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c8b3c640 1 -- 172.21.15.69:0/2624885131 learned_addr learned my addr 172.21.15.69:0/2624885131 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6dae56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.500 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c933d640 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae13d0 0x55cd6dae4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c9b3e640 1 -- 172.21.15.69:0/2624885131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae13d0 msgr2=0x55cd6dae4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae13d0 0x55cd6dae4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c9b3e640 1 -- 172.21.15.69:0/2624885131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 msgr2=0x55cd6dae5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c9b3e640 1 -- 172.21.15.69:0/2624885131 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6d7b83c0 con 0x55cd6daf4870 2024-08-02T18:02:50.501 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.477+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:50.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6dae56c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f59bc02d000 tx=0x7f59bc0050b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5f508b11c4a71b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59acff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:02:50.502 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59acff9640 1 -- 172.21.15.69:0/2624885131 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc035070 con 0x55cd6daf4870 2024-08-02T18:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2944763494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcdd7b0 msgr2=0x55fa0fcddb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2944763494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcddb90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7a580b87a0 tx=0x7f7a58009260 comp rx=0 tx=0).stop 2024-08-02T18:02:50.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59acff9640 1 -- 172.21.15.69:0/2624885131 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f59bc00ed30 con 0x55cd6daf4870 2024-08-02T18:02:50.503 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59cd99f740 1 -- 172.21.15.69:0/2624885131 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59bc00eb80 con 0x55cd6daf4870 2024-08-02T18:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2944763494 shutdown_connections 2024-08-02T18:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2944763494 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fcead60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2944763494 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcde0d0 0x55fa0fce8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2944763494 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcddb90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2944763494 >> 172.21.15.69:0/2944763494 conn(0x55fa0fcd9d20 msgr2=0x55fa0fcda100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.505 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59acff9640 1 -- 172.21.15.69:0/2624885131 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f59bc004a40 con 0x55cd6daf4870 2024-08-02T18:02:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2944763494 shutdown_connections 2024-08-02T18:02:50.506 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59cd99f740 10 client.28793 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:50.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59cd99f740 1 -- 172.21.15.69:0/2624885131 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd6db031e0 con 0x55cd6daf4870 2024-08-02T18:02:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2944763494 wait complete. 2024-08-02T18:02:50.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.478+0000 7f59acff9640 1 -- 172.21.15.69:0/2624885131 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59bc003e50 con 0x55cd6daf4870 2024-08-02T18:02:50.508 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 1 -- start start 2024-08-02T18:02:50.509 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 1 Processor -- start 2024-08-02T18:02:50.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f59acff9640 1 -- 172.21.15.69:0/2624885131 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f59bc00f040 con 0x55cd6daf4870 2024-08-02T18:02:50.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f59cd99f740 10 client.28793 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:50.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f59cd99f740 10 client.28793 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:02:50.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f59cd99f740 20 client.28793 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:50.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55cd6db04530 0x55cd6db06920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f59cd99f740 4 client.28793 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:50.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f59cd99f740 1 -- 172.21.15.69:0/2624885131 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f59bc00eb80 con 0x55cd6db04530 2024-08-02T18:02:50.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f59c933d640 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55cd6db04530 0x55cd6db06920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fce5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fce8970 0x55fa0fce63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x55fa0fceb300 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x55fa0fbfd500 con 0x55fa0fce8970 2024-08-02T18:02:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55fa0fceb480 con 0x55fa0fcde0d0 2024-08-02T18:02:50.513 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a6fb9c740 10 client.0 _ll_register_callbacks cb 0x55fa0f954a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:50.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.479+0000 7f7a66d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fce8970 0x55fa0fce63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a66d9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fce8970 0x55fa0fce63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:60996/0 (socket says 172.21.15.69:60996) 2024-08-02T18:02:50.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a65d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fce5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.514 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a66d9d640 1 -- 172.21.15.69:0/1395049417 learned_addr learned my addr 172.21.15.69:0/1395049417 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f59c933d640 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55cd6db04530 0x55cd6db06920 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=137892d753ff0e6b server_cookie=893684d60fbe4026 in_seq=0 out_seq=0 2024-08-02T18:02:50.515 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f59acff9640 10 client.28793 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:02:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a6659c640 1 -- 172.21.15.69:0/1395049417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 msgr2=0x55fa0fce5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a6659c640 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fce5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a6659c640 1 -- 172.21.15.69:0/1395049417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fce8970 msgr2=0x55fa0fce63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.516 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a6659c640 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fce8970 0x55fa0fce63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a6659c640 1 -- 172.21.15.69:0/1395049417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0fa21620 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a66d9d640 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fce8970 0x55fa0fce63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a6659c640 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce5950 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7a580b87a0 tx=0x7f7a580c4ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16019e1ab05fdd08 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:50.517 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a4dffb640 1 -- 172.21.15.69:0/1395049417 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a580ed280 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f59acff9640 1 -- 172.21.15.69:0/2624885131 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f59bc00eb80 con 0x55cd6db04530 2024-08-02T18:02:50.518 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f59acff9640 10 client.28793 handle_command_reply: tid=0 2024-08-02T18:02:50.518 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a4dffb640 1 -- 172.21.15.69:0/1395049417 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f7a580c45b0 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.519 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f59cd99f740 1 client.28793 shutdown 2024-08-02T18:02:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.480+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1395049417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a580c0180 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.520 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a4dffb640 1 -- 172.21.15.69:0/1395049417 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a580a2050 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 -- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 msgr2=0x55cd6dae56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.521 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6dae56c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f59bc02d000 tx=0x7f59bc0050b0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.521 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a6fb9c740 10 client.19443 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1395049417 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa0fceb300 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.522 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a4dffb640 1 -- 172.21.15.69:0/1395049417 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a580c0180 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 -- 172.21.15.69:0/2624885131 shutdown_connections 2024-08-02T18:02:50.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55cd6db04530 0x55cd6db06920 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.522 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae2570 0x55cd6dae5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6dae56c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2624885131 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae13d0 0x55cd6dae4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 -- 172.21.15.69:0/2624885131 >> 172.21.15.69:0/2624885131 conn(0x55cd6dadd580 msgr2=0x55cd6dae32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.523 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59acff9640 0 client.28793 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:02:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a4dffb640 1 -- 172.21.15.69:0/1395049417 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a580d1030 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.524 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 -- 172.21.15.69:0/2624885131 shutdown_connections 2024-08-02T18:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a6fb9c740 10 client.19443 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a6fb9c740 10 client.19443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a6fb9c740 20 client.19443 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fa0fcf8150 0x55fa0fcfa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a6fb9c740 4 client.19443 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1395049417 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa0fc4f0e0 con 0x55fa0fcf8150 2024-08-02T18:02:50.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 1 -- 172.21.15.69:0/2624885131 wait complete. 2024-08-02T18:02:50.526 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.481+0000 7f59cd99f740 20 client.28793 trim_cache size 0 max 16384 2024-08-02T18:02:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.482+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fa0fcf8150 0x55fa0fcfa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.482+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fa0fcf8150 0x55fa0fcfa540 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d920c54c88729cd2 server_cookie=994e23294c96cf4b in_seq=0 out_seq=0 2024-08-02T18:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.482+0000 7f7a4dffb640 10 client.19443 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a4dffb640 1 -- 172.21.15.69:0/1395049417 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa0fc4f0e0 con 0x55fa0fcf8150 2024-08-02T18:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a4dffb640 10 client.19443 handle_command_reply: tid=0 2024-08-02T18:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 client.19443 shutdown 2024-08-02T18:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1395049417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fcdd7b0 msgr2=0x55fa0fce5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce5950 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7a580b87a0 tx=0x7f7a580c4ab0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1395049417 shutdown_connections 2024-08-02T18:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fa0fcf8150 0x55fa0fcfa540 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fce8970 0x55fa0fce63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fce5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a4dffb640 0 client.19443 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1395049417 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce5950 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1395049417 >> 172.21.15.69:0/1395049417 conn(0x55fa0fcd9d20 msgr2=0x55fa0fcdfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1395049417 shutdown_connections 2024-08-02T18:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1395049417 wait complete. 2024-08-02T18:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.483+0000 7f7a6fb9c740 20 client.19443 trim_cache size 0 max 16384 2024-08-02T18:02:50.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59cd99f740 1 Processor -- start 2024-08-02T18:02:50.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59cd99f740 1 -- start start 2024-08-02T18:02:50.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6dae6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.535 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6daf4870 0x55cd6dae3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59cd99f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59c41a6d30 con 0x55cd6daf4870 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59cd99f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4073940 con 0x55cd6dae13d0 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59cd99f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4071790 con 0x55cd6dae2570 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.496+0000 7f59c8b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6dae6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c8b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6dae6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47350/0 (socket says 172.21.15.69:47350) 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c8b3c640 1 -- 172.21.15.69:0/1289725140 learned_addr learned my addr 172.21.15.69:0/1289725140 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c933d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c8b3c640 1 -- 172.21.15.69:0/1289725140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 msgr2=0x55cd6dae5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/1289725140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c8b3c640 1 -- 172.21.15.69:0/1289725140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6daf4870 msgr2=0x55cd6dae3a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/1289725140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6daf4870 0x55cd6dae3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c8b3c640 1 -- 172.21.15.69:0/1289725140 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd6daf3dd0 con 0x55cd6dae2570 2024-08-02T18:02:50.536 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59c8b3c640 1 --2- 172.21.15.69:0/1289725140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6dae6aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b7e40 tx=0x7f59c00045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=221c2a8b45ea1377 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.497+0000 7f59aeffd640 1 -- 172.21.15.69:0/1289725140 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0049040 con 0x55cd6dae2570 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.498+0000 7f59aeffd640 1 -- 172.21.15.69:0/1289725140 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59c00d4070 con 0x55cd6dae2570 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.498+0000 7f59aeffd640 1 -- 172.21.15.69:0/1289725140 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c00b8050 con 0x55cd6dae2570 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.499+0000 7f59cd99f740 1 -- 172.21.15.69:0/1289725140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 msgr2=0x55cd6dae6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.499+0000 7f59cd99f740 1 --2- 172.21.15.69:0/1289725140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6dae6aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f59c00b7e40 tx=0x7f59c00045f0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.499+0000 7f59cd99f740 1 -- 172.21.15.69:0/1289725140 shutdown_connections 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.499+0000 7f59cd99f740 1 --2- 172.21.15.69:0/1289725140 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6daf4870 0x55cd6dae3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.499+0000 7f59cd99f740 1 --2- 172.21.15.69:0/1289725140 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6dae6aa0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.499+0000 7f59cd99f740 1 --2- 172.21.15.69:0/1289725140 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6dae13d0 0x55cd6dae5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.499+0000 7f59cd99f740 1 -- 172.21.15.69:0/1289725140 >> 172.21.15.69:0/1289725140 conn(0x55cd6dadd580 msgr2=0x55cd6dae6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.499+0000 7f59cd99f740 1 -- 172.21.15.69:0/1289725140 shutdown_connections 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.500+0000 7f59cd99f740 1 -- 172.21.15.69:0/1289725140 wait complete. 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.500+0000 7f59cd99f740 1 -- start start 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.500+0000 7f59cd99f740 1 Processor -- start 2024-08-02T18:02:50.537 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.501+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae13d0 0x55cd6db06210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.501+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6db06750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.501+0000 7f59c933d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae13d0 0x55cd6db06210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.501+0000 7f59c933d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae13d0 0x55cd6db06210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46164/0 (socket says 172.21.15.69:46164) 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.501+0000 7f59cd99f740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6db06c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c8b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6db06750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c8b3c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6db06750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47352/0 (socket says 172.21.15.69:47352) 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59cd99f740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4071e00 con 0x55cd6dae13d0 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59cd99f740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f59c4073270 con 0x55cd6daf4870 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59cd99f740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f59c40724d0 con 0x55cd6dae2570 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6db06c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6db06c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:32788/0 (socket says 172.21.15.69:32788) 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 -- 172.21.15.69:0/2522665641 learned_addr learned my addr 172.21.15.69:0/2522665641 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.538 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59cd99f740 10 client.0 _ll_register_callbacks cb 0x7f59bc0266e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:50.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 -- 172.21.15.69:0/2522665641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 msgr2=0x55cd6db06750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6db06750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 -- 172.21.15.69:0/2522665641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae13d0 msgr2=0x55cd6db06210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae13d0 0x55cd6db06210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 -- 172.21.15.69:0/2522665641 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd6db6ce60 con 0x55cd6daf4870 2024-08-02T18:02:50.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c933d640 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae13d0 0x55cd6db06210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:50.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.502+0000 7f59c9b3e640 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6db06c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005c20 tx=0x7f59bc026c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9875fdea730ccd37 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.539 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59acff9640 10 client.0 ms_handle_connect on v2:172.21.15.77:3300/0 2024-08-02T18:02:50.539 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6fb9c740 1 Processor -- start 2024-08-02T18:02:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6fb9c740 1 -- start start 2024-08-02T18:02:50.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59acff9640 1 -- 172.21.15.69:0/2522665641 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc048060 con 0x55cd6daf4870 2024-08-02T18:02:50.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59acff9640 1 -- 172.21.15.69:0/2522665641 <== mon.2 v2:172.21.15.77:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f59bc004be0 con 0x55cd6daf4870 2024-08-02T18:02:50.540 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59cd99f740 1 -- 172.21.15.69:0/2522665641 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59c00d4070 con 0x55cd6daf4870 2024-08-02T18:02:50.540 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcffda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fd002e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a681b3980 con 0x55fa0fce8970 2024-08-02T18:02:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a681b1250 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.541 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6fb9c740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a681af0a0 con 0x55fa0fcde0d0 2024-08-02T18:02:50.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59acff9640 1 -- 172.21.15.69:0/2522665641 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f59bc035040 con 0x55cd6daf4870 2024-08-02T18:02:50.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59cd99f740 10 client.28802 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:50.541 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59cd99f740 1 -- 172.21.15.69:0/2522665641 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd6db6e040 con 0x55cd6daf4870 2024-08-02T18:02:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:32798/0 (socket says 172.21.15.69:32798) 2024-08-02T18:02:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a6659c640 1 -- 172.21.15.69:0/1896255521 learned_addr learned my addr 172.21.15.69:0/1896255521 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.542 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a66d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fd002e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.542 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59acff9640 1 -- 172.21.15.69:0/2522665641 <== mon.2 v2:172.21.15.77:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59bc004790 con 0x55cd6daf4870 2024-08-02T18:02:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/1896255521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.503+0000 7f59acff9640 1 -- 172.21.15.69:0/2522665641 <== mon.2 v2:172.21.15.77:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f59bc004de0 con 0x55cd6daf4870 2024-08-02T18:02:50.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59cd99f740 10 client.28802 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:50.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59cd99f740 10 client.28802 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-08-02T18:02:50.543 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59cd99f740 20 client.28802 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:50.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55cd6db6e760 0x55cd6db70b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59cd99f740 4 client.28802 mds_command: new command op to 24431 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:02:50.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59cd99f740 1 -- 172.21.15.69:0/2522665641 --> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f59c00d4070 con 0x55cd6db6e760 2024-08-02T18:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a6659c640 1 -- 172.21.15.69:0/1896255521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 msgr2=0x55fa0fcffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a6659c640 1 --2- 172.21.15.69:0/1896255521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcffda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a6659c640 1 -- 172.21.15.69:0/1896255521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 msgr2=0x55fa0fd002e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a6659c640 1 --2- 172.21.15.69:0/1896255521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fd002e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a6659c640 1 -- 172.21.15.69:0/1896255521 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa0fcfab70 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.544 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59c933d640 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55cd6db6e760 0x55cd6db70b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a6659c640 1 --2- 172.21.15.69:0/1896255521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce6200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7a580ba080 tx=0x7f7a58003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba4b00ca3b4aa7d server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a4ffff640 1 -- 172.21.15.69:0/1896255521 <== mon.2 v2:172.21.15.77:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a580c0180 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a4ffff640 1 -- 172.21.15.69:0/1896255521 <== mon.2 v2:172.21.15.77:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a580ed280 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f7a4ffff640 1 -- 172.21.15.69:0/1896255521 <== mon.2 v2:172.21.15.77:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a580c45b0 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59c933d640 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55cd6db6e760 0x55cd6db70b80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cafbf7e09cb0e918 server_cookie=2362b31c3c2fc1b0 in_seq=0 out_seq=0 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.504+0000 7f59acff9640 10 client.28802 ms_handle_connect on v2:172.21.15.77:6836/3778799888 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59acff9640 1 -- 172.21.15.69:0/2522665641 <== mds.1 v2:172.21.15.77:6836/3778799888 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f59c00d4070 con 0x55cd6db6e760 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59acff9640 10 client.28802 handle_command_reply: tid=0 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 client.28802 shutdown 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 -- 172.21.15.69:0/2522665641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 msgr2=0x55cd6db06c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6db06c90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f59bc005c20 tx=0x7f59bc026c80 comp rx=0 tx=0).stop 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 -- 172.21.15.69:0/2522665641 shutdown_connections 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.77:6836/3778799888,v1:172.21.15.77:6837/3778799888] conn(0x55cd6db6e760 0x55cd6db70b80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.546 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59acff9640 0 client.28802 ms_handle_reset on v2:172.21.15.77:6836/3778799888 2024-08-02T18:02:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55cd6daf4870 0x55cd6db06c90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55cd6dae2570 0x55cd6db06750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 --2- 172.21.15.69:0/2522665641 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55cd6dae13d0 0x55cd6db06210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 -- 172.21.15.69:0/2522665641 >> 172.21.15.69:0/2522665641 conn(0x55cd6dadd580 msgr2=0x55cd6db00200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 -- 172.21.15.69:0/2522665641 shutdown_connections 2024-08-02T18:02:50.547 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 1 -- 172.21.15.69:0/2522665641 wait complete. 2024-08-02T18:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1896255521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 msgr2=0x55fa0fce6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1896255521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce6200 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7a580ba080 tx=0x7f7a58003f90 comp rx=0 tx=0).stop 2024-08-02T18:02:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1896255521 shutdown_connections 2024-08-02T18:02:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1896255521 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fd002e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1896255521 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/1896255521 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fce6200 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.548 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1896255521 >> 172.21.15.69:0/1896255521 conn(0x55fa0fcd9d20 msgr2=0x55fa0fce78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.548 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f59cd99f740 20 client.28802 trim_cache size 0 max 16384 2024-08-02T18:02:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1896255521 shutdown_connections 2024-08-02T18:02:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.505+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/1896255521 wait complete. 2024-08-02T18:02:50.549 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:02:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f7a6fb9c740 1 -- start start 2024-08-02T18:02:50.549 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f7a6fb9c740 1 Processor -- start 2024-08-02T18:02:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 -- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59b00a2d70 msgr2=0x7f59b00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59b00a2d70 0x7f59b00a5230 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f59b800ab60 tx=0x7f59b800a2d0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 -- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c4101150 msgr2=0x7f59c41994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.550 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c4101150 0x7f59c41994e0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009230 tx=0x7f59c0004270 comp rx=0 tx=0).stop 2024-08-02T18:02:50.550 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcf06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcf0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 -- 172.21.15.69:0/971460673 shutdown_connections 2024-08-02T18:02:50.551 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f59c4101150 0x7f59c41994e0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.551 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f7a6fb9c740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fce1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f59b00a2d70 0x7f59b00a5230 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f59c41067b0 0x7f59c41a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 --2- 172.21.15.69:0/971460673 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f59c41007b0 0x7f59c4198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 -- 172.21.15.69:0/971460673 >> 172.21.15.69:0/971460673 conn(0x7f59c40fc4d0 msgr2=0x7f59c40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 -- 172.21.15.69:0/971460673 shutdown_connections 2024-08-02T18:02:50.552 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:50.506+0000 7f59cb5a1640 1 -- 172.21.15.69:0/971460673 wait complete. 2024-08-02T18:02:50.554 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcf0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcf0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47374/0 (socket says 172.21.15.69:47374) 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcf06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a6659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcf06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.77:3300/0 says I am v2:172.21.15.69:32804/0 (socket says 172.21.15.69:32804) 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 -- 172.21.15.69:0/2429424610 learned_addr learned my addr 172.21.15.69:0/2429424610 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7a681af710 con 0x55fa0fce8970 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a66d9d640 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fce1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7a681b0b80 con 0x55fa0fcdd7b0 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7a681afde0 con 0x55fa0fcde0d0 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a6fb9c740 10 client.0 _ll_register_callbacks cb 0x7f7a600099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 -- 172.21.15.69:0/2429424610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 msgr2=0x55fa0fcf06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcf06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.555 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 -- 172.21.15.69:0/2429424610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 msgr2=0x55fa0fce1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fce1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 -- 172.21.15.69:0/2429424610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa0fcdadd0 con 0x55fa0fcde0d0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a66d9d640 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fce1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a65d9b640 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcf0c10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c011010 tx=0x7f7a5c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2636c621136d236 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.507+0000 7f7a4dffb640 10 client.0 ms_handle_connect on v2:172.21.15.83:3300/0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.508+0000 7f7a4dffb640 1 -- 172.21.15.69:0/2429424610 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c01edc0 con 0x55fa0fcde0d0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.508+0000 7f7a4dffb640 1 -- 172.21.15.69:0/2429424610 <== mon.1 v2:172.21.15.83:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f7a5c02f410 con 0x55fa0fcde0d0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.508+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a580ed280 con 0x55fa0fcde0d0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.508+0000 7f7a4dffb640 1 -- 172.21.15.69:0/2429424610 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c004020 con 0x55fa0fcde0d0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.508+0000 7f7a6fb9c740 10 client.28552 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.508+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fa0fd623b0 con 0x55fa0fcde0d0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.508+0000 7f7a4dffb640 1 -- 172.21.15.69:0/2429424610 <== mon.1 v2:172.21.15.83:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a580ed280 con 0x55fa0fcde0d0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.508+0000 7f7a4dffb640 1 -- 172.21.15.69:0/2429424610 <== mon.1 v2:172.21.15.83:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a5c0035f0 con 0x55fa0fcde0d0 2024-08-02T18:02:50.556 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a6fb9c740 10 client.28552 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a6fb9c740 10 client.28552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a6fb9c740 20 client.28552 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fa0fd62b10 0x55fa0fd64f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a6fb9c740 4 client.28552 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-02T18:02:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa0fc49430 con 0x55fa0fd62b10 2024-08-02T18:02:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a6659c640 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fa0fd62b10 0x55fa0fd64f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a6659c640 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fa0fd62b10 0x55fa0fd64f30 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9862f468b31b8e server_cookie=6bdfa43f9fb3afc0 in_seq=0 out_seq=0 2024-08-02T18:02:50.557 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.509+0000 7f7a4dffb640 10 client.28552 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:50.557 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a4dffb640 1 -- 172.21.15.69:0/2429424610 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fa0fc49430 con 0x55fa0fd62b10 2024-08-02T18:02:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a4dffb640 10 client.28552 handle_command_reply: tid=0 2024-08-02T18:02:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 client.28552 shutdown 2024-08-02T18:02:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 msgr2=0x55fa0fcf0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.558 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcf0c10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c011010 tx=0x7f7a5c004730 comp rx=0 tx=0).stop 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 shutdown_connections 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x55fa0fce8970 0x55fa0fce1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x55fa0fd62b10 0x55fa0fd64f30 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55fa0fcde0d0 0x55fa0fcf0c10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 --2- 172.21.15.69:0/2429424610 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x55fa0fcdd7b0 0x55fa0fcf06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a4dffb640 0 client.28552 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 >> 172.21.15.69:0/2429424610 conn(0x55fa0fcd9d20 msgr2=0x55fa0fcdf210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 shutdown_connections 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 1 -- 172.21.15.69:0/2429424610 wait complete. 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.546+0000 7f7a6fb9c740 20 client.28552 trim_cache size 0 max 16384 2024-08-02T18:02:50.559 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:{ 2024-08-02T18:02:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "message": "", 2024-08-02T18:02:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout: "return_code": 0 2024-08-02T18:02:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stdout:} 2024-08-02T18:02:50.560 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7a500a2cf0 msgr2=0x7f7a500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7a500a2cf0 0x7f7a500a51b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7a68075cb0 tx=0x7f7a5c00b390 comp rx=0 tx=0).stop 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/3733350692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a68101800 msgr2=0x7f7a68074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a68101800 0x7f7a68074220 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7a5802a940 tx=0x7f7a58003b90 comp rx=0 tx=0).stop 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/3733350692 shutdown_connections 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f7a681021a0 0x7f7a68074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f7a500a2cf0 0x7f7a500a51b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7a68107810 0x7f7a68074ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 --2- 172.21.15.69:0/3733350692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7a68101800 0x7f7a68074220 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/3733350692 >> 172.21.15.69:0/3733350692 conn(0x7f7a680fd500 msgr2=0x7f7a680fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/3733350692 shutdown_connections 2024-08-02T18:02:50.561 INFO:teuthology.task.background_exec.ubuntu@smithi069.front.sepia.ceph.com.smithi069.stderr:2024-08-02T18:02:50.547+0000 7f7a6ce4c640 1 -- 172.21.15.69:0/3733350692 wait complete. 2024-08-02T18:02:51.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:50 smithi083 ceph-mon[37031]: pgmap v515: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-08-02T18:02:51.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:50 smithi077 ceph-mon[37108]: pgmap v515: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-08-02T18:02:51.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:50 smithi069 ceph-mon[32746]: pgmap v515: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-08-02T18:02:51.740 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-02T18:02:51.969 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:52.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 -- 172.21.15.69:0/3111947268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d81067b0 msgr2=0x7f52d8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 --2- 172.21.15.69:0/3111947268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d81067b0 0x7f52d8106b90 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f52c8009930 tx=0x7f52c802a420 comp rx=0 tx=0).stop 2024-08-02T18:02:52.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 -- 172.21.15.69:0/3111947268 shutdown_connections 2024-08-02T18:02:52.620 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 --2- 172.21.15.69:0/3111947268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52d8101150 0x7f52d810d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 --2- 172.21.15.69:0/3111947268 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52d81007b0 0x7f52d8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 --2- 172.21.15.69:0/3111947268 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d81067b0 0x7f52d8106b90 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 -- 172.21.15.69:0/3111947268 >> 172.21.15.69:0/3111947268 conn(0x7f52d80fc4d0 msgr2=0x7f52d80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 -- 172.21.15.69:0/3111947268 shutdown_connections 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.616+0000 7f52dceac640 1 -- 172.21.15.69:0/3111947268 wait complete. 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52dceac640 1 Processor -- start 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52dceac640 1 -- start start 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52dceac640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52d81007b0 0x7f52d8197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52dceac640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d8101150 0x7f52d81975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52dceac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52d81067b0 0x7f52d819b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52dceac640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8110b70 con 0x7f52d8101150 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52dceac640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81109f0 con 0x7f52d81007b0 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52dceac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8110cf0 con 0x7f52d81067b0 2024-08-02T18:02:52.621 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d5d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d8101150 0x7f52d81975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d5d9b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d8101150 0x7f52d81975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46262/0 (socket says 172.21.15.69:46262) 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d5d9b640 1 -- 172.21.15.69:0/1721973004 learned_addr learned my addr 172.21.15.69:0/1721973004 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d659c640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52d81007b0 0x7f52d8197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d5d9b640 1 -- 172.21.15.69:0/1721973004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52d81067b0 msgr2=0x7f52d819b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d5d9b640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52d81067b0 0x7f52d819b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d5d9b640 1 -- 172.21.15.69:0/1721973004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52d81007b0 msgr2=0x7f52d8197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d5d9b640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52d81007b0 0x7f52d8197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.617+0000 7f52d5d9b640 1 -- 172.21.15.69:0/1721973004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d819c040 con 0x7f52d8101150 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.618+0000 7f52d5d9b640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d8101150 0x7f52d81975d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f52c000d950 tx=0x7f52c000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.618+0000 7f52cf7fe640 1 -- 172.21.15.69:0/1721973004 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0017070 con 0x7f52d8101150 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.618+0000 7f52dceac640 1 -- 172.21.15.69:0/1721973004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d819c330 con 0x7f52d8101150 2024-08-02T18:02:52.622 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.618+0000 7f52cf7fe640 1 -- 172.21.15.69:0/1721973004 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c0009c30 con 0x7f52d8101150 2024-08-02T18:02:52.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.618+0000 7f52cf7fe640 1 -- 172.21.15.69:0/1721973004 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c000f6b0 con 0x7f52d8101150 2024-08-02T18:02:52.623 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.618+0000 7f52dceac640 1 -- 172.21.15.69:0/1721973004 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d81a3c10 con 0x7f52d8101150 2024-08-02T18:02:52.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.620+0000 7f52cf7fe640 1 -- 172.21.15.69:0/1721973004 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f52c00109d0 con 0x7f52d8101150 2024-08-02T18:02:52.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.620+0000 7f52df2aa740 1 Processor -- start 2024-08-02T18:02:52.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52df2aa740 1 -- start start 2024-08-02T18:02:52.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.628 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6a80 0x560b955d3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52cf7fe640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52c40a2cf0 0x7f52c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955d36b0 0x560b955d5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52df2aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b9552b810 con 0x560b955c6160 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52df2aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560b954d2c70 con 0x560b955d36b0 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52df2aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b955277d0 con 0x560b955c6a80 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46276/0 (socket says 172.21.15.69:46276) 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 -- 172.21.15.69:0/1204473221 learned_addr learned my addr 172.21.15.69:0/1204473221 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d5d9b640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52c40a2cf0 0x7f52c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d659c640 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6a80 0x560b955d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955d36b0 0x560b955d5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 -- 172.21.15.69:0/1204473221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6a80 msgr2=0x560b955d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.629 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52cf7fe640 1 -- 172.21.15.69:0/1721973004 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f52c00cf4e0 con 0x7f52d8101150 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6a80 0x560b955d3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 -- 172.21.15.69:0/1204473221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955d36b0 msgr2=0x560b955d5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955d36b0 0x560b955d5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.621+0000 7f52d6d9d640 1 -- 172.21.15.69:0/1204473221 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b9529dba0 con 0x560b955c6160 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.622+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955c6540 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009fd0 tx=0x7f52d0009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955ebe47a8c3fbed server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.622+0000 7f52d5d9b640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52c40a2cf0 0x7f52c40a51b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f52c00a3dd0 tx=0x7f52c0097a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.622+0000 7f52ccff9640 1 -- 172.21.15.69:0/1204473221 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d00150f0 con 0x560b955c6160 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.622+0000 7f52ccff9640 1 -- 172.21.15.69:0/1204473221 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0010500 con 0x560b955c6160 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.622+0000 7f52ccff9640 1 -- 172.21.15.69:0/1204473221 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0005220 con 0x560b955c6160 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.622+0000 7f52df2aa740 1 -- 172.21.15.69:0/1204473221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 msgr2=0x560b955c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.622+0000 7f52df2aa740 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955c6540 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009fd0 tx=0x7f52d0009710 comp rx=0 tx=0).stop 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 -- 172.21.15.69:0/1204473221 shutdown_connections 2024-08-02T18:02:52.630 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955d36b0 0x560b955d5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6a80 0x560b955d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 --2- 172.21.15.69:0/1204473221 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955c6540 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 -- 172.21.15.69:0/1204473221 >> 172.21.15.69:0/1204473221 conn(0x560b955c2850 msgr2=0x560b955c2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 -- 172.21.15.69:0/1204473221 shutdown_connections 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 -- 172.21.15.69:0/1204473221 wait complete. 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 -- start start 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 Processor -- start 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 0x560b955d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x560b955d6280 con 0x560b955d36b0 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x560b954e69a0 con 0x560b955c6a80 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x560b955d6400 con 0x560b955c6160 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.623+0000 7f52df2aa740 10 client.0 _ll_register_callbacks cb 0x560b954d4c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:52.631 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46290/0 (socket says 172.21.15.69:46290) 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 -- 172.21.15.69:0/3019631045 learned_addr learned my addr 172.21.15.69:0/3019631045 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 0x560b955d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d659c640 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 -- 172.21.15.69:0/3019631045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 msgr2=0x560b955d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 0x560b955d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 -- 172.21.15.69:0/3019631045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 msgr2=0x560b955d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 -- 172.21.15.69:0/3019631045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b954f2220 con 0x560b955d36b0 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955d1000 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f52d0005230 tx=0x7f52d0017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848f6911b43aa713 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52bdffb640 1 -- 172.21.15.69:0/3019631045 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0003970 con 0x560b955d36b0 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52bdffb640 1 -- 172.21.15.69:0/3019631045 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f52d0003b10 con 0x560b955d36b0 2024-08-02T18:02:52.632 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52df2aa740 1 -- 172.21.15.69:0/3019631045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d0010500 con 0x560b955d36b0 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52bdffb640 1 -- 172.21.15.69:0/3019631045 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52d0031050 con 0x560b955d36b0 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52df2aa740 10 client.19479 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.624+0000 7f52df2aa740 1 -- 172.21.15.69:0/3019631045 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b955e1f50 con 0x560b955d36b0 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52bdffb640 1 -- 172.21.15.69:0/3019631045 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0010500 con 0x560b955d36b0 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52bdffb640 1 -- 172.21.15.69:0/3019631045 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f52d000c070 con 0x560b955d36b0 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52df2aa740 10 client.19479 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52df2aa740 10 client.19479 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52df2aa740 20 client.19479 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52df2aa740 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560b955e3410 0x560b955e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52df2aa740 4 client.19479 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52df2aa740 1 -- 172.21.15.69:0/3019631045 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b95538f40 con 0x560b955e3410 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52d659c640 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560b955e3410 0x560b955e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52bdffb640 10 client.19479 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:52.633 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.625+0000 7f52d659c640 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560b955e3410 0x560b955e5800 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76d5d187c444284 server_cookie=11781c3d832bf5d9 in_seq=0 out_seq=0 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52bdffb640 1 -- 172.21.15.69:0/3019631045 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560b95538f40 con 0x560b955e3410 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52bdffb640 10 client.19479 handle_command_reply: tid=0 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 client.19479 shutdown 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 -- 172.21.15.69:0/3019631045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 msgr2=0x560b955d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955d1000 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f52d0005230 tx=0x7f52d0017040 comp rx=0 tx=0).stop 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 -- 172.21.15.69:0/3019631045 shutdown_connections 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955d1000 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560b955e3410 0x560b955e5800 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 --2- 172.21.15.69:0/3019631045 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 0x560b955d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52bdffb640 0 client.19479 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 -- 172.21.15.69:0/3019631045 >> 172.21.15.69:0/3019631045 conn(0x560b955c2850 msgr2=0x560b955c1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 -- 172.21.15.69:0/3019631045 shutdown_connections 2024-08-02T18:02:52.634 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.626+0000 7f52df2aa740 1 -- 172.21.15.69:0/3019631045 wait complete. 2024-08-02T18:02:52.635 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.627+0000 7f52df2aa740 20 client.19479 trim_cache size 0 max 16384 2024-08-02T18:02:52.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.643+0000 7f52df2aa740 1 Processor -- start 2024-08-02T18:02:52.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.643+0000 7f52df2aa740 1 -- start start 2024-08-02T18:02:52.649 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.643+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.643+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955dba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.643+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955d36b0 0x560b955dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.643+0000 7f52df2aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81af590 con 0x560b955c6160 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.643+0000 7f52df2aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ace60 con 0x560b955c6a80 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.643+0000 7f52df2aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81aacb0 con 0x560b955d36b0 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955d2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46304/0 (socket says 172.21.15.69:46304) 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 -- 172.21.15.69:0/1240525551 learned_addr learned my addr 172.21.15.69:0/1240525551 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955d36b0 0x560b955dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d659c640 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955dba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.650 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 -- 172.21.15.69:0/1240525551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955d36b0 msgr2=0x560b955dbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955d36b0 0x560b955dbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 -- 172.21.15.69:0/1240525551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 msgr2=0x560b955dba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955dba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 -- 172.21.15.69:0/1240525551 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b955c68b0 con 0x560b955c6160 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d659c640 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955d2450 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f52d0005960 tx=0x7f52d0005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a72b0b1b5c999 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52bffff640 1 -- 172.21.15.69:0/1240525551 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d003a070 con 0x560b955c6160 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52bffff640 1 -- 172.21.15.69:0/1240525551 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0005020 con 0x560b955c6160 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.644+0000 7f52bffff640 1 -- 172.21.15.69:0/1240525551 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d00138e0 con 0x560b955c6160 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 -- 172.21.15.69:0/1240525551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 msgr2=0x560b955d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955d2450 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f52d0005960 tx=0x7f52d0005cd0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.651 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 -- 172.21.15.69:0/1240525551 shutdown_connections 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955d36b0 0x560b955dbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955dba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 --2- 172.21.15.69:0/1240525551 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955c6160 0x560b955d2450 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 -- 172.21.15.69:0/1240525551 >> 172.21.15.69:0/1240525551 conn(0x560b955c2850 msgr2=0x560b955d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 -- 172.21.15.69:0/1240525551 shutdown_connections 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 -- 172.21.15.69:0/1240525551 wait complete. 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 -- start start 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 Processor -- start 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 0x560b955cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.645+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52df2aa740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.652 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52df2aa740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ab320 con 0x560b955d36b0 2024-08-02T18:02:52.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52df2aa740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ac790 con 0x560b955c6a80 2024-08-02T18:02:52.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52df2aa740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81ab9f0 con 0x560b955c6160 2024-08-02T18:02:52.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52df2aa740 10 client.0 _ll_register_callbacks cb 0x7f52d0015540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:52.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d659c640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46318/0 (socket says 172.21.15.69:46318) 2024-08-02T18:02:52.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 -- 172.21.15.69:0/2407101950 learned_addr learned my addr 172.21.15.69:0/2407101950 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.653 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 0x560b955cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 -- 172.21.15.69:0/2407101950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 msgr2=0x560b955cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 0x560b955cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 -- 172.21.15.69:0/2407101950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 msgr2=0x560b955ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 -- 172.21.15.69:0/2407101950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b955cdb20 con 0x560b955d36b0 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d659c640 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52d6d9d640 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955cd320 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f52d0004a80 tx=0x7f52d0015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f693c18462c08cdd server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52bdffb640 1 -- 172.21.15.69:0/2407101950 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d003a040 con 0x560b955d36b0 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52bdffb640 1 -- 172.21.15.69:0/2407101950 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f52d001b040 con 0x560b955d36b0 2024-08-02T18:02:52.654 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.646+0000 7f52df2aa740 1 -- 172.21.15.69:0/2407101950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52d0005020 con 0x560b955d36b0 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52bdffb640 1 -- 172.21.15.69:0/2407101950 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f52d0009ce0 con 0x560b955d36b0 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52df2aa740 10 client.19485 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52df2aa740 1 -- 172.21.15.69:0/2407101950 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560b95650370 con 0x560b955d36b0 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52bdffb640 1 -- 172.21.15.69:0/2407101950 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52d0005020 con 0x560b955d36b0 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52bdffb640 1 -- 172.21.15.69:0/2407101950 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f52d0040040 con 0x560b955d36b0 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52df2aa740 10 client.19485 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52df2aa740 10 client.19485 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52df2aa740 20 client.19485 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52df2aa740 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560b95650a90 0x560b95652eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52df2aa740 4 client.19485 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.95, "await_for": 45.0}] 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52df2aa740 1 -- 172.21.15.69:0/2407101950 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.95, "await_for": 45.0}) -- 0x560b955be0d0 con 0x560b95650a90 2024-08-02T18:02:52.655 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.647+0000 7f52d659c640 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560b95650a90 0x560b95652eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.648+0000 7f52bdffb640 10 client.19485 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:52.656 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.648+0000 7f52d659c640 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560b95650a90 0x560b95652eb0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b9d77d38fd848cd server_cookie=ff4ebf0418865cd2 in_seq=0 out_seq=0 2024-08-02T18:02:52.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 -- 172.21.15.69:0/245562789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c10bea0 msgr2=0x7ffb9c10e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/245562789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c10bea0 0x7ffb9c10e340 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009ca0 tx=0x7ffb9402a950 comp rx=0 tx=0).stop 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 -- 172.21.15.69:0/245562789 shutdown_connections 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/245562789 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c10bea0 0x7ffb9c10e340 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/245562789 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb9c103390 0x7ffb9c10b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/245562789 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb9c102a70 0x7ffb9c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 -- 172.21.15.69:0/245562789 >> 172.21.15.69:0/245562789 conn(0x7ffb9c0fc4d0 msgr2=0x7ffb9c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 -- 172.21.15.69:0/245562789 shutdown_connections 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 -- 172.21.15.69:0/245562789 wait complete. 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 Processor -- start 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.217+0000 7ffba2ca2640 1 -- start start 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba2ca2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb9c102a70 0x7ffb9c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.739 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba2ca2640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c103390 0x7ffb9c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba2ca2640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb9c10bea0 0x7ffb9c19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba2ca2640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c111ca0 con 0x7ffb9c103390 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba2ca2640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c111b20 con 0x7ffb9c10bea0 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba2ca2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c111e20 con 0x7ffb9c102a70 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c103390 0x7ffb9c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c103390 0x7ffb9c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46204/0 (socket says 172.21.15.69:46204) 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 -- 172.21.15.69:0/2961562579 learned_addr learned my addr 172.21.15.69:0/2961562579 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb9c102a70 0x7ffb9c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba123f640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb9c10bea0 0x7ffb9c19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 -- 172.21.15.69:0/2961562579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb9c102a70 msgr2=0x7ffb9c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb9c102a70 0x7ffb9c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 -- 172.21.15.69:0/2961562579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb9c10bea0 msgr2=0x7ffb9c19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb9c10bea0 0x7ffb9c19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.740 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 -- 172.21.15.69:0/2961562579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c19d0b0 con 0x7ffb9c103390 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c103390 0x7ffb9c198640 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800e9f0 tx=0x7ffb8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb99ffb640 1 -- 172.21.15.69:0/2961562579 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800ccc0 con 0x7ffb9c103390 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffba2ca2640 1 -- 172.21.15.69:0/2961562579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9c19d3a0 con 0x7ffb9c103390 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb99ffb640 1 -- 172.21.15.69:0/2961562579 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8800ce60 con 0x7ffb9c103390 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.218+0000 7ffb99ffb640 1 -- 172.21.15.69:0/2961562579 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88010690 con 0x7ffb9c103390 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.219+0000 7ffba2ca2640 1 -- 172.21.15.69:0/2961562579 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb9c1a4c80 con 0x7ffb9c103390 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.221+0000 7ffb99ffb640 1 -- 172.21.15.69:0/2961562579 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffb880046c0 con 0x7ffb9c103390 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.221+0000 7ffba50a0740 1 Processor -- start 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.221+0000 7ffba50a0740 1 -- start start 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb99ffb640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b7f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.741 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba50a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556a5b7498c0 con 0x556a5b7e86a0 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba50a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a5b6fb9e0 con 0x556a5b7f3880 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba50a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a5b757540 con 0x556a5b7e8fc0 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba123f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b7f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb99ffb640 1 -- 172.21.15.69:0/2961562579 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7ffb880cfbb0 con 0x7ffb9c103390 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47398/0 (socket says 172.21.15.69:47398) 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb9bfff640 1 -- 172.21.15.69:0/733497303 learned_addr learned my addr 172.21.15.69:0/733497303 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba123f640 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb9bfff640 1 -- 172.21.15.69:0/733497303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 msgr2=0x556a5b7f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b7f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb9bfff640 1 -- 172.21.15.69:0/733497303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 msgr2=0x556a5b7e8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7e8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffb9bfff640 1 -- 172.21.15.69:0/733497303 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a5b6e5640 con 0x556a5b7e8fc0 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba123f640 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.222+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009bd0 tx=0x7ffb90005f90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.223+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f3340 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffb88018000 tx=0x7ffb88091110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf224b526c78624e server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.223+0000 7ffb86ffd640 1 -- 172.21.15.69:0/733497303 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8809c380 con 0x556a5b7e8fc0 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.223+0000 7ffb86ffd640 1 -- 172.21.15.69:0/733497303 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8809bc80 con 0x556a5b7e8fc0 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.223+0000 7ffb86ffd640 1 -- 172.21.15.69:0/733497303 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb880b08d0 con 0x556a5b7e8fc0 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.223+0000 7ffba50a0740 1 -- 172.21.15.69:0/733497303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 msgr2=0x556a5b7f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.223+0000 7ffba50a0740 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f3340 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffb88018000 tx=0x7ffb88091110 comp rx=0 tx=0).stop 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 -- 172.21.15.69:0/733497303 shutdown_connections 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b7f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.743 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f3340 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 --2- 172.21.15.69:0/733497303 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7e8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 -- 172.21.15.69:0/733497303 >> 172.21.15.69:0/733497303 conn(0x556a5b7e4c30 msgr2=0x556a5b7e5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 -- 172.21.15.69:0/733497303 shutdown_connections 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 -- 172.21.15.69:0/733497303 wait complete. 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 -- start start 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 Processor -- start 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.224+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7f3880 0x556a5b7f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba50a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x556a5b74cc10 con 0x556a5b7f3880 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba50a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x556a5b70bf00 con 0x556a5b7e8fc0 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba50a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x556a5b751f30 con 0x556a5b7e86a0 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba50a0740 10 client.0 _ll_register_callbacks cb 0x556a5b45cb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:52.744 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba0a3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba123f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7f3880 0x556a5b7f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba0a3e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.69:47412/0 (socket says 172.21.15.69:47412) 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba0a3e640 1 -- 172.21.15.69:0/2972178374 learned_addr learned my addr 172.21.15.69:0/2972178374 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba123f640 1 -- 172.21.15.69:0/2972178374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 msgr2=0x556a5b7f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba123f640 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba123f640 1 -- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7e8fc0 msgr2=0x556a5b7f04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba123f640 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba123f640 1 -- 172.21.15.69:0/2972178374 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a5b4bf620 con 0x556a5b7f3880 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba123f640 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7f3880 0x556a5b7f0a70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ffb94005cc0 tx=0x7ffb9402fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e8ad3ae5e8a2c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffb7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2972178374 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9404a060 con 0x556a5b7f3880 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba50a0740 1 -- 172.21.15.69:0/2972178374 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb8809bc80 con 0x556a5b7f3880 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2972178374 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7ffb9402f670 con 0x556a5b7f3880 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2972178374 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb94051050 con 0x556a5b7f3880 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.225+0000 7ffba50a0740 10 client.19461 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffba50a0740 1 -- 172.21.15.69:0/2972178374 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a5b8062c0 con 0x556a5b7f3880 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2972178374 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb8809bc80 con 0x556a5b7f3880 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2972178374 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffb94004690 con 0x556a5b7f3880 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffba50a0740 10 client.19461 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffba50a0740 10 client.19461 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffba50a0740 20 client.19461 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a5b807720 0x556a5b809b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffba50a0740 4 client.19461 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffba50a0740 1 -- 172.21.15.69:0/2972178374 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a5b75ae70 con 0x556a5b807720 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.226+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a5b807720 0x556a5b809b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.227+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a5b807720 0x556a5b809b10 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4bf6d2794337a19 server_cookie=982c5e7cb12faec5 in_seq=0 out_seq=0 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.227+0000 7ffb7bfff640 10 client.19461 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.227+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2972178374 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a5b75ae70 con 0x556a5b807720 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.227+0000 7ffb7bfff640 10 client.19461 handle_command_reply: tid=0 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.227+0000 7ffba50a0740 1 client.19461 shutdown 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.227+0000 7ffba50a0740 1 -- 172.21.15.69:0/2972178374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7f3880 msgr2=0x556a5b7f0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.227+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7f3880 0x556a5b7f0a70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ffb94005cc0 tx=0x7ffb9402fe80 comp rx=0 tx=0).stop 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 1 -- 172.21.15.69:0/2972178374 shutdown_connections 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7f3880 0x556a5b7f0a70 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a5b807720 0x556a5b809b10 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7e8fc0 0x556a5b7f04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.747 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2972178374 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffb7bfff640 0 client.19461 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 1 -- 172.21.15.69:0/2972178374 >> 172.21.15.69:0/2972178374 conn(0x556a5b7e4c30 msgr2=0x556a5b7ea960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 1 -- 172.21.15.69:0/2972178374 shutdown_connections 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 1 -- 172.21.15.69:0/2972178374 wait complete. 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.228+0000 7ffba50a0740 20 client.19461 trim_cache size 0 max 16384 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.245+0000 7ffba50a0740 1 Processor -- start 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.245+0000 7ffba50a0740 1 -- start start 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e8fc0 0x556a5b7ef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b80f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffba50a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c1ace00 con 0x556a5b7e8fc0 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffba50a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c1aac50 con 0x556a5b7f3880 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffba50a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c1ac730 con 0x556a5b7e86a0 2024-08-02T18:02:52.748 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e8fc0 0x556a5b7ef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e8fc0 0x556a5b7ef410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46238/0 (socket says 172.21.15.69:46238) 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 -- 172.21.15.69:0/3050853840 learned_addr learned my addr 172.21.15.69:0/3050853840 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffba123f640 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b80f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 -- 172.21.15.69:0/3050853840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 msgr2=0x556a5b7f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 -- 172.21.15.69:0/3050853840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 msgr2=0x556a5b80f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b80f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 -- 172.21.15.69:0/3050853840 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a5b803f90 con 0x556a5b7e8fc0 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e8fc0 0x556a5b7ef410 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ffb88098650 tx=0x7ffb88097fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5af77c8af08bc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.246+0000 7ffb867fc640 1 -- 172.21.15.69:0/3050853840 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8809bc80 con 0x556a5b7e8fc0 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffb867fc640 1 -- 172.21.15.69:0/3050853840 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb88092020 con 0x556a5b7e8fc0 2024-08-02T18:02:52.749 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffb867fc640 1 -- 172.21.15.69:0/3050853840 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb880a4460 con 0x556a5b7e8fc0 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 -- 172.21.15.69:0/3050853840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e8fc0 msgr2=0x556a5b7ef410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e8fc0 0x556a5b7ef410 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ffb88098650 tx=0x7ffb88097fa0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 -- 172.21.15.69:0/3050853840 shutdown_connections 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e8fc0 0x556a5b7ef410 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b80f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 --2- 172.21.15.69:0/3050853840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e86a0 0x556a5b7f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 -- 172.21.15.69:0/3050853840 >> 172.21.15.69:0/3050853840 conn(0x556a5b7e4c30 msgr2=0x556a5b7ebae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 -- 172.21.15.69:0/3050853840 shutdown_connections 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.247+0000 7ffba50a0740 1 -- 172.21.15.69:0/3050853840 wait complete. 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 1 -- start start 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 1 Processor -- start 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.750 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7fc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b8713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c1ab990 con 0x556a5b7e86a0 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c1ab2c0 con 0x556a5b7f3880 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c1ac060 con 0x556a5b7e8fc0 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba50a0740 10 client.0 _ll_register_callbacks cb 0x7ffb88005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba0a3e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba0a3e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46246/0 (socket says 172.21.15.69:46246) 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba0a3e640 1 -- 172.21.15.69:0/2501130911 learned_addr learned my addr 172.21.15.69:0/2501130911 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.248+0000 7ffba123f640 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b8713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba0a3e640 1 -- 172.21.15.69:0/2501130911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 msgr2=0x556a5b7fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba0a3e640 1 -- 172.21.15.69:0/2501130911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 msgr2=0x556a5b8713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b8713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba0a3e640 1 -- 172.21.15.69:0/2501130911 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a5b8718f0 con 0x556a5b7e86a0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba0a3e640 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7fbce0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x556a5b80a350 tx=0x7ffb90004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9c076d06e79e97 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffb7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2501130911 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90035060 con 0x556a5b7e86a0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2501130911 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7ffb900035f0 con 0x556a5b7e86a0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 1 -- 172.21.15.69:0/2501130911 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb88092020 con 0x556a5b7e86a0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2501130911 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffb9003c050 con 0x556a5b7e86a0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 10 client.19467 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 1 -- 172.21.15.69:0/2501130911 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a5b872b80 con 0x556a5b7e86a0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2501130911 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb88092020 con 0x556a5b7e86a0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2501130911 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffb90029940 con 0x556a5b7e86a0 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 10 client.19467 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:52.752 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 10 client.19467 resolve_mds: validated gid 24452 aka daemon mds.b 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 20 client.19467 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a5b873250 0x556a5b875670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 4 client.19467 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.249+0000 7ffba50a0740 1 -- 172.21.15.69:0/2501130911 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x556a5b755380 con 0x556a5b873250 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a5b873250 0x556a5b875670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffb7bfff640 10 client.19467 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffb9bfff640 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a5b873250 0x556a5b875670 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89043fbdc7dabd5d server_cookie=fdf194b306f06b5c in_seq=0 out_seq=0 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffb7bfff640 1 -- 172.21.15.69:0/2501130911 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x556a5b755380 con 0x556a5b873250 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffb7bfff640 10 client.19467 handle_command_reply: tid=0 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffba50a0740 1 client.19467 shutdown 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffba50a0740 1 -- 172.21.15.69:0/2501130911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 msgr2=0x556a5b7fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7fbce0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x556a5b80a350 tx=0x7ffb90004730 comp rx=0 tx=0).stop 2024-08-02T18:02:52.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffba50a0740 1 -- 172.21.15.69:0/2501130911 shutdown_connections 2024-08-02T18:02:52.754 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:52.754 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 27, 2024-08-02T18:02:52.754 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T18:02:52.754 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 56, 2024-08-02T18:02:52.754 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": {} 2024-08-02T18:02:52.754 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.250+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x556a5b873250 0x556a5b875670 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x556a5b7f3880 0x556a5b8713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffb7bfff640 0 client.19467 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x556a5b7e8fc0 0x556a5b7fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba50a0740 1 --2- 172.21.15.69:0/2501130911 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x556a5b7e86a0 0x556a5b7fbce0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba50a0740 1 -- 172.21.15.69:0/2501130911 >> 172.21.15.69:0/2501130911 conn(0x556a5b7e4c30 msgr2=0x556a5b7e3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba50a0740 1 -- 172.21.15.69:0/2501130911 shutdown_connections 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba50a0740 1 -- 172.21.15.69:0/2501130911 wait complete. 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba50a0740 20 client.19467 trim_cache size 0 max 16384 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 -- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb8c0a2cf0 msgr2=0x7ffb8c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.755 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009bd0 tx=0x7ffb90005f90 comp rx=0 tx=0).stop 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 -- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c103390 msgr2=0x7ffb9c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c103390 0x7ffb9c198640 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800e9f0 tx=0x7ffb8800eeb0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 -- 172.21.15.69:0/2961562579 shutdown_connections 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ffb9c103390 0x7ffb9c198640 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ffb8c0a2cf0 0x7ffb8c0a51b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ffb9c10bea0 0x7ffb9c19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 --2- 172.21.15.69:0/2961562579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ffb9c102a70 0x7ffb9c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 -- 172.21.15.69:0/2961562579 >> 172.21.15.69:0/2961562579 conn(0x7ffb9c0fc4d0 msgr2=0x7ffb9c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.251+0000 7ffba2ca2640 1 -- 172.21.15.69:0/2961562579 shutdown_connections 2024-08-02T18:02:52.756 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:52.252+0000 7ffba2ca2640 1 -- 172.21.15.69:0/2961562579 wait complete. 2024-08-02T18:02:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52bdffb640 1 -- 172.21.15.69:0/2407101950 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560b955be0d0 con 0x560b95650a90 2024-08-02T18:02:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52bdffb640 10 client.19485 handle_command_reply: tid=0 2024-08-02T18:02:53.066 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52df2aa740 1 client.19485 shutdown 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52df2aa740 1 -- 172.21.15.69:0/2407101950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 msgr2=0x560b955cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52df2aa740 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955cd320 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f52d0004a80 tx=0x7f52d0015040 comp rx=0 tx=0).stop 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52df2aa740 1 -- 172.21.15.69:0/2407101950 shutdown_connections 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52df2aa740 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x560b955d36b0 0x560b955cd320 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52df2aa740 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x560b95650a90 0x560b95652eb0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.064+0000 7f52df2aa740 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x560b955c6a80 0x560b955ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52bdffb640 0 client.19485 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52df2aa740 1 --2- 172.21.15.69:0/2407101950 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x560b955c6160 0x560b955cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52df2aa740 1 -- 172.21.15.69:0/2407101950 >> 172.21.15.69:0/2407101950 conn(0x560b955c2850 msgr2=0x560b955d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52df2aa740 1 -- 172.21.15.69:0/2407101950 shutdown_connections 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52df2aa740 1 -- 172.21.15.69:0/2407101950 wait complete. 2024-08-02T18:02:53.067 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52df2aa740 20 client.19485 trim_cache size 0 max 16384 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 27, 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "leader": 24452, 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "set_version": 58, 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "sets": { 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "56a819eb": { 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "version": 58, 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "age_ref": 0.0, 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T18:02:53.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "timeout": 90.0, 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "expiration": 147.9, 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "members": { 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "file:/": { 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "excluded": false, 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "state": { 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "QUIESCED", 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: "age": 0.0 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:53.069 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:02:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52dceac640 1 -- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52c40a2cf0 msgr2=0x7f52c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52dceac640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52c40a2cf0 0x7f52c40a51b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f52c00a3dd0 tx=0x7f52c0097a20 comp rx=0 tx=0).stop 2024-08-02T18:02:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52dceac640 1 -- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d8101150 msgr2=0x7f52d81975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.065+0000 7f52dceac640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d8101150 0x7f52d81975d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f52c000d950 tx=0x7f52c000de10 comp rx=0 tx=0).stop 2024-08-02T18:02:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.066+0000 7f52dceac640 1 -- 172.21.15.69:0/1721973004 shutdown_connections 2024-08-02T18:02:53.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.066+0000 7f52dceac640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f52d8101150 0x7f52d81975d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.066+0000 7f52dceac640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f52c40a2cf0 0x7f52c40a51b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.066+0000 7f52dceac640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f52d81067b0 0x7f52d819b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.066+0000 7f52dceac640 1 --2- 172.21.15.69:0/1721973004 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f52d81007b0 0x7f52d8197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.066+0000 7f52dceac640 1 -- 172.21.15.69:0/1721973004 >> 172.21.15.69:0/1721973004 conn(0x7f52d80fc4d0 msgr2=0x7f52d80fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.066+0000 7f52dceac640 1 -- 172.21.15.69:0/1721973004 shutdown_connections 2024-08-02T18:02:53.071 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:53.066+0000 7f52dceac640 1 -- 172.21.15.69:0/1721973004 wait complete. 2024-08-02T18:02:53.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:52 smithi083 ceph-mon[37031]: pgmap v516: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-08-02T18:02:53.295 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:52 smithi069 ceph-mon[32746]: pgmap v516: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-08-02T18:02:53.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:52 smithi077 ceph-mon[37108]: pgmap v516: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 11 op/s 2024-08-02T18:02:53.751 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:54.185 INFO:tasks.quiescer:done joining Quiescers 2024-08-02T18:02:54.186 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-08-02T18:02:54.196 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T18:02:54.205 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-08-02T18:02:54.226 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-08-02T18:02:54.226 INFO:teuthology.task.exec:Executing custom commands... 2024-08-02T18:02:54.226 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T18:02:54.227 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-08-02T18:02:54.273 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-08-02T18:02:54.340 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-02T18:02:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/3329735020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc01006d0 msgr2=0x7f9dc0100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/3329735020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc01006d0 0x7f9dc0100b30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009930 tx=0x7f9db402a420 comp rx=0 tx=0).stop 2024-08-02T18:02:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/3329735020 shutdown_connections 2024-08-02T18:02:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/3329735020 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc01006d0 0x7f9dc0100b30 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/3329735020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc0101070 0x7f9dc010d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/3329735020 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dc01066e0 0x7f9dc0106ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.490 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/3329735020 >> 172.21.15.69:0/3329735020 conn(0x7f9dc00776d0 msgr2=0x7f9dc00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/3329735020 shutdown_connections 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.486+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/3329735020 wait complete. 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dc5c8f640 1 Processor -- start 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dc5c8f640 1 -- start start 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dc5c8f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dc01006d0 0x7f9dc0198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dc5c8f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc0101070 0x7f9dc0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dc5c8f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc01066e0 0x7f9dc019c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dc5c8f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0110b20 con 0x7f9dc0101070 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dc5c8f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01109a0 con 0x7f9dc01006d0 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dc5c8f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0110ca0 con 0x7f9dc01066e0 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dbeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc0101070 0x7f9dc0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dbeffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc0101070 0x7f9dc0198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46332/0 (socket says 172.21.15.69:46332) 2024-08-02T18:02:54.491 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dbeffd640 1 -- 172.21.15.69:0/2742563342 learned_addr learned my addr 172.21.15.69:0/2742563342 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.487+0000 7f9dbffff640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc01066e0 0x7f9dc019c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbf7fe640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dc01006d0 0x7f9dc0198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbeffd640 1 -- 172.21.15.69:0/2742563342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc01066e0 msgr2=0x7f9dc019c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbeffd640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc01066e0 0x7f9dc019c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbeffd640 1 -- 172.21.15.69:0/2742563342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dc01006d0 msgr2=0x7f9dc0198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbeffd640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dc01006d0 0x7f9dc0198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbeffd640 1 -- 172.21.15.69:0/2742563342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc019d100 con 0x7f9dc0101070 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbf7fe640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dc01006d0 0x7f9dc0198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbeffd640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc0101070 0x7f9dc0198650 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009230 tx=0x7f9db4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbcff9640 1 -- 172.21.15.69:0/2742563342 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db403f030 con 0x7f9dc0101070 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc019d390 con 0x7f9dc0101070 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbcff9640 1 -- 172.21.15.69:0/2742563342 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db4003ad0 con 0x7f9dc0101070 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dbcff9640 1 -- 172.21.15.69:0/2742563342 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4035070 con 0x7f9dc0101070 2024-08-02T18:02:54.492 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.488+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc01a4c80 con 0x7f9dc0101070 2024-08-02T18:02:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.489+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc0101e10 con 0x7f9dc0101070 2024-08-02T18:02:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.490+0000 7f9dbcff9640 1 -- 172.21.15.69:0/2742563342 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9db4042030 con 0x7f9dc0101070 2024-08-02T18:02:54.493 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.491+0000 7f9dbcff9640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9da80a2cf0 0x7f9da80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.491+0000 7f9dbf7fe640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9da80a2cf0 0x7f9da80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.491+0000 7f9dbcff9640 1 -- 172.21.15.69:0/2742563342 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f9db40efd60 con 0x7f9dc0101070 2024-08-02T18:02:54.494 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.492+0000 7f9dbf7fe640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9da80a2cf0 0x7f9da80a51b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005fd0 tx=0x7f9db0005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.498 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.496+0000 7f9dbcff9640 1 -- 172.21.15.69:0/2742563342 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f9db40bc570 con 0x7f9dc0101070 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 -- 172.21.15.69:0/2721401528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df81078e0 msgr2=0x7f6df8107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 --2- 172.21.15.69:0/2721401528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df81078e0 0x7f6df8107cc0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009930 tx=0x7f6de002a420 comp rx=0 tx=0).stop 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 -- 172.21.15.69:0/2721401528 shutdown_connections 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 --2- 172.21.15.69:0/2721401528 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df8102280 0x7f6df810e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 --2- 172.21.15.69:0/2721401528 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df81018e0 0x7f6df8101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 --2- 172.21.15.69:0/2721401528 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df81078e0 0x7f6df8107cc0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 -- 172.21.15.69:0/2721401528 >> 172.21.15.69:0/2721401528 conn(0x7f6df80fd600 msgr2=0x7f6df80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 -- 172.21.15.69:0/2721401528 shutdown_connections 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.665+0000 7f6dfd075640 1 -- 172.21.15.69:0/2721401528 wait complete. 2024-08-02T18:02:54.685 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.666+0000 7f6dfd075640 1 Processor -- start 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.666+0000 7f6dfd075640 1 -- start start 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6dfd075640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df81018e0 0x7f6df81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6dfd075640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df8102280 0x7f6df8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6dfd075640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df81078e0 0x7f6df819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6dfd075640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8111ce0 con 0x7f6df8102280 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6dfd075640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8111b60 con 0x7f6df81018e0 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6dfd075640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8111e60 con 0x7f6df81078e0 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df8102280 0x7f6df8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df8102280 0x7f6df8198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46336/0 (socket says 172.21.15.69:46336) 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 -- 172.21.15.69:0/257010362 learned_addr learned my addr 172.21.15.69:0/257010362 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df81018e0 0x7f6df81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df759e640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df81078e0 0x7f6df819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 -- 172.21.15.69:0/257010362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df81078e0 msgr2=0x7f6df819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.686 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df81078e0 0x7f6df819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 -- 172.21.15.69:0/257010362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df81018e0 msgr2=0x7f6df81981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df81018e0 0x7f6df81981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 -- 172.21.15.69:0/257010362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df819d170 con 0x7f6df8102280 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.667+0000 7f6df659c640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df8102280 0x7f6df8198700 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f6de800e9f0 tx=0x7f6de800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.668+0000 7f6deffff640 1 -- 172.21.15.69:0/257010362 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de800cd80 con 0x7f6df8102280 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.668+0000 7f6deffff640 1 -- 172.21.15.69:0/257010362 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de8010910 con 0x7f6df8102280 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.668+0000 7f6dfd075640 1 -- 172.21.15.69:0/257010362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df819d460 con 0x7f6df8102280 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.668+0000 7f6deffff640 1 -- 172.21.15.69:0/257010362 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8018990 con 0x7f6df8102280 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.668+0000 7f6dfd075640 1 -- 172.21.15.69:0/257010362 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df8074150 con 0x7f6df8102280 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.670+0000 7f6deffff640 1 -- 172.21.15.69:0/257010362 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6de8010ac0 con 0x7f6df8102280 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.671+0000 7f6deffff640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6de40a2d70 0x7f6de40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.671+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6de40a2d70 0x7f6de40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.671+0000 7f6deffff640 1 -- 172.21.15.69:0/257010362 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f6de80cfb80 con 0x7f6df8102280 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.671+0000 7f6dff473740 1 Processor -- start 2024-08-02T18:02:54.687 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.671+0000 7f6dff473740 1 -- start start 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.671+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6de40a2d70 0x7f6de40a5230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009900 tx=0x7f6de00058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6dff473740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b6893d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6dff473740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b693c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6dff473740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6965c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6dff473740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56302b5eea40 con 0x56302b688ff0 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6dff473740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56302b5869d0 con 0x56302b689910 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6dff473740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56302b5eaa00 con 0x56302b6941d0 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b6893d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b6893d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46346/0 (socket says 172.21.15.69:46346) 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 -- 172.21.15.69:0/1777352519 learned_addr learned my addr 172.21.15.69:0/1777352519 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df659c640 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b693c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df759e640 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.688 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 -- 172.21.15.69:0/1777352519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 msgr2=0x56302b6965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 -- 172.21.15.69:0/1777352519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 msgr2=0x56302b693c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b693c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 -- 172.21.15.69:0/1777352519 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302b58d650 con 0x56302b688ff0 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df759e640 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df659c640 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b693c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b6893d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f6de0033000 tx=0x7f6de003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91b3e6932d3762f server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.672+0000 7f6ded7fa640 1 -- 172.21.15.69:0/1777352519 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0035040 con 0x56302b688ff0 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6ded7fa640 1 -- 172.21.15.69:0/1777352519 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de004e070 con 0x56302b688ff0 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6ded7fa640 1 -- 172.21.15.69:0/1777352519 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0049a70 con 0x56302b688ff0 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 -- 172.21.15.69:0/1777352519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 msgr2=0x56302b6893d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b6893d0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f6de0033000 tx=0x7f6de003d000 comp rx=0 tx=0).stop 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 -- 172.21.15.69:0/1777352519 shutdown_connections 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.689 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b693c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 --2- 172.21.15.69:0/1777352519 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b6893d0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 -- 172.21.15.69:0/1777352519 >> 172.21.15.69:0/1777352519 conn(0x56302b685580 msgr2=0x56302b685960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 -- 172.21.15.69:0/1777352519 shutdown_connections 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.673+0000 7f6dff473740 1 -- 172.21.15.69:0/1777352519 wait complete. 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 1 -- start start 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 1 Processor -- start 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b691040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b691580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b691ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x56302b696da0 con 0x56302b688ff0 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x56302b5a9b20 con 0x56302b689910 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x56302b696f20 con 0x56302b6941d0 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6dff473740 10 client.0 _ll_register_callbacks cb 0x56302b5a7bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:54.690 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.674+0000 7f6df6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b691040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b691040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46358/0 (socket says 172.21.15.69:46358) 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df6d9d640 1 -- 172.21.15.69:0/4287340899 learned_addr learned my addr 172.21.15.69:0/4287340899 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df759e640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b691ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df659c640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b691580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df6d9d640 1 -- 172.21.15.69:0/4287340899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 msgr2=0x56302b691ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b691ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df6d9d640 1 -- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 msgr2=0x56302b691580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b691580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df6d9d640 1 -- 172.21.15.69:0/4287340899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302b3603c0 con 0x56302b688ff0 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df759e640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b691ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df659c640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b691580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b691040 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f6de0005940 tx=0x7f6de00316c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9751dc16a175972 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.691 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6dde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6dde7fc640 1 -- 172.21.15.69:0/4287340899 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0035070 con 0x56302b688ff0 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6dde7fc640 1 -- 172.21.15.69:0/4287340899 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f6de0049c00 con 0x56302b688ff0 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6dff473740 1 -- 172.21.15.69:0/4287340899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de004e070 con 0x56302b688ff0 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6dde7fc640 1 -- 172.21.15.69:0/4287340899 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6de0064050 con 0x56302b688ff0 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.675+0000 7f6dff473740 10 client.19503 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dff473740 1 -- 172.21.15.69:0/4287340899 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56302b6a2930 con 0x56302b688ff0 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dde7fc640 1 -- 172.21.15.69:0/4287340899 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de004e070 con 0x56302b688ff0 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dde7fc640 1 -- 172.21.15.69:0/4287340899 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6de003c440 con 0x56302b688ff0 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dff473740 10 client.19503 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dff473740 10 client.19503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dff473740 20 client.19503 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dff473740 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56302b6a3c80 0x56302b6a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.692 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dff473740 4 client.19503 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6dff473740 1 -- 172.21.15.69:0/4287340899 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56302b5fbfb0 con 0x56302b6a3c80 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.676+0000 7f6df659c640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56302b6a3c80 0x56302b6a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.677+0000 7f6dde7fc640 10 client.19503 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.677+0000 7f6df659c640 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56302b6a3c80 0x56302b6a6070 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0bf7bd953bd7ffc server_cookie=a32a1215b16cef45 in_seq=0 out_seq=0 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.677+0000 7f6dde7fc640 1 -- 172.21.15.69:0/4287340899 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56302b5fbfb0 con 0x56302b6a3c80 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.677+0000 7f6dde7fc640 10 client.19503 handle_command_reply: tid=0 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.677+0000 7f6dff473740 1 client.19503 shutdown 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 -- 172.21.15.69:0/4287340899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 msgr2=0x56302b691040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b691040 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f6de0005940 tx=0x7f6de00316c0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 -- 172.21.15.69:0/4287340899 shutdown_connections 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56302b6a3c80 0x56302b6a6070 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b691ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b691580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 --2- 172.21.15.69:0/4287340899 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b691040 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 -- 172.21.15.69:0/4287340899 >> 172.21.15.69:0/4287340899 conn(0x56302b685580 msgr2=0x56302b68b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dde7fc640 0 client.19503 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:54.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 -- 172.21.15.69:0/4287340899 shutdown_connections 2024-08-02T18:02:54.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 1 -- 172.21.15.69:0/4287340899 wait complete. 2024-08-02T18:02:54.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.678+0000 7f6dff473740 20 client.19503 trim_cache size 0 max 16384 2024-08-02T18:02:54.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.704+0000 7f6dff473740 1 Processor -- start 2024-08-02T18:02:54.705 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.704+0000 7f6dff473740 1 -- start start 2024-08-02T18:02:54.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.704+0000 7f6dff473740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b68ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.706 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.704+0000 7f6dff473740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b692ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.704+0000 7f6dff473740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.704+0000 7f6dff473740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81b39a0 con 0x56302b688ff0 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.704+0000 7f6dff473740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81b1270 con 0x56302b689910 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.704+0000 7f6dff473740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81af0c0 con 0x56302b6941d0 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b68ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b68ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46364/0 (socket says 172.21.15.69:46364) 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 -- 172.21.15.69:0/176141864 learned_addr learned my addr 172.21.15.69:0/176141864 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df759e640 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df659c640 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b692ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.707 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 -- 172.21.15.69:0/176141864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 msgr2=0x56302b6ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 -- 172.21.15.69:0/176141864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 msgr2=0x56302b692ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b692ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 -- 172.21.15.69:0/176141864 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302b686630 con 0x56302b688ff0 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df759e640 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df659c640 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b692ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b68ba50 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f6de003f040 tx=0x7f6de0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109522f09bfd7dae server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6decff9640 1 -- 172.21.15.69:0/176141864 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0035070 con 0x56302b688ff0 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6decff9640 1 -- 172.21.15.69:0/176141864 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de004e070 con 0x56302b688ff0 2024-08-02T18:02:54.708 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.705+0000 7f6decff9640 1 -- 172.21.15.69:0/176141864 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de0040040 con 0x56302b688ff0 2024-08-02T18:02:54.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.706+0000 7f6dff473740 1 -- 172.21.15.69:0/176141864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 msgr2=0x56302b68ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.706+0000 7f6dff473740 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b68ba50 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f6de003f040 tx=0x7f6de0000bc0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.712 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.706+0000 7f6dff473740 1 -- 172.21.15.69:0/176141864 shutdown_connections 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.706+0000 7f6dff473740 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b6941d0 0x56302b6ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.706+0000 7f6dff473740 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b692ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.706+0000 7f6dff473740 1 --2- 172.21.15.69:0/176141864 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b688ff0 0x56302b68ba50 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.706+0000 7f6dff473740 1 -- 172.21.15.69:0/176141864 >> 172.21.15.69:0/176141864 conn(0x56302b685580 msgr2=0x56302b692790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.706+0000 7f6dff473740 1 -- 172.21.15.69:0/176141864 shutdown_connections 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 -- 172.21.15.69:0/176141864 wait complete. 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 -- start start 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 Processor -- start 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b688ff0 0x56302b69c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b69c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b6941d0 0x56302b697250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.713 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81af730 con 0x56302b6941d0 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81b0ba0 con 0x56302b689910 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.707+0000 7f6dff473740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6df81afe00 con 0x56302b688ff0 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6dff473740 10 client.0 _ll_register_callbacks cb 0x7f6de8097830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b6941d0 0x56302b697250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df6d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b688ff0 0x56302b69c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b6941d0 0x56302b697250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:46366/0 (socket says 172.21.15.69:46366) 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 -- 172.21.15.69:0/213257635 learned_addr learned my addr 172.21.15.69:0/213257635 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df659c640 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b69c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 -- 172.21.15.69:0/213257635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b688ff0 msgr2=0x56302b69c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b688ff0 0x56302b69c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 -- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 msgr2=0x56302b69c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b69c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.714 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 -- 172.21.15.69:0/213257635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56302b697a50 con 0x56302b6941d0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b688ff0 0x56302b69c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df659c640 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b69c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6dde7fc640 10 client.0 ms_handle_connect on v2:172.21.15.69:3300/0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6df759e640 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b6941d0 0x56302b697250 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6df0012a10 tx=0x7f6df0012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4344e109b11234d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.708+0000 7f6dde7fc640 1 -- 172.21.15.69:0/213257635 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0021060 con 0x56302b6941d0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dde7fc640 1 -- 172.21.15.69:0/213257635 <== mon.0 v2:172.21.15.69:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f6df0011dd0 con 0x56302b6941d0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 1 -- 172.21.15.69:0/213257635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de004e070 con 0x56302b6941d0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dde7fc640 1 -- 172.21.15.69:0/213257635 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6df0028050 con 0x56302b6941d0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 10 client.19509 fetch_fsmap learned FSMap version 28 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 1 -- 172.21.15.69:0/213257635 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56302b70d1f0 con 0x56302b6941d0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dde7fc640 1 -- 172.21.15.69:0/213257635 <== mon.0 v2:172.21.15.69:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6de004e070 con 0x56302b6941d0 2024-08-02T18:02:54.715 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dde7fc640 1 -- 172.21.15.69:0/213257635 <== mon.0 v2:172.21.15.69:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6df001b070 con 0x56302b6941d0 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 10 client.19509 fetch_fsmap finished waiting for FSMap version 28 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 10 client.19509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 20 client.19509 populate_metadata read hostname 'smithi069' 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56302b70d970 0x56302b70fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 4 client.19509 mds_command: new command op to 24452 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.709+0000 7f6dff473740 1 -- 172.21.15.69:0/213257635 --> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56302b5f64c0 con 0x56302b70d970 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56302b70d970 0x56302b70fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dde7fc640 10 client.19509 ms_handle_connect on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6df6d9d640 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56302b70d970 0x56302b70fd90 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=258c09410337999a server_cookie=ce56a9a5fde06020 in_seq=0 out_seq=0 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dde7fc640 1 -- 172.21.15.69:0/213257635 <== mds.0 v2:172.21.15.77:6838/1183977027 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56302b5f64c0 con 0x56302b70d970 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dde7fc640 10 client.19509 handle_command_reply: tid=0 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 client.19509 shutdown 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 -- 172.21.15.69:0/213257635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b6941d0 msgr2=0x56302b697250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.716 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b6941d0 0x56302b697250 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f6df0012a10 tx=0x7f6df0012ed0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 -- 172.21.15.69:0/213257635 shutdown_connections 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x56302b6941d0 0x56302b697250 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:6838/1183977027,v1:172.21.15.77:6839/1183977027] conn(0x56302b70d970 0x56302b70fd90 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x56302b689910 0x56302b69c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 --2- 172.21.15.69:0/213257635 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x56302b688ff0 0x56302b69c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.710+0000 7f6dff473740 1 -- 172.21.15.69:0/213257635 >> 172.21.15.69:0/213257635 conn(0x56302b685580 msgr2=0x56302b690000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.711+0000 7f6dde7fc640 0 client.19509 ms_handle_reset on v2:172.21.15.77:6838/1183977027 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.711+0000 7f6dff473740 1 -- 172.21.15.69:0/213257635 shutdown_connections 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.711+0000 7f6dff473740 1 -- 172.21.15.69:0/213257635 wait complete. 2024-08-02T18:02:54.717 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.711+0000 7f6dff473740 20 client.19509 trim_cache size 0 max 16384 2024-08-02T18:02:54.718 INFO:teuthology.orchestra.run.smithi069.stdout:[] 2024-08-02T18:02:54.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.711+0000 7f6dfd075640 1 -- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6de40a2d70 msgr2=0x7f6de40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.711+0000 7f6dfd075640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6de40a2d70 0x7f6de40a5230 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6de0009900 tx=0x7f6de00058d0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.718 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.711+0000 7f6dfd075640 1 -- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df8102280 msgr2=0x7f6df8198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.711+0000 7f6dfd075640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df8102280 0x7f6df8198700 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f6de800e9f0 tx=0x7f6de800eeb0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.712+0000 7f6dfd075640 1 -- 172.21.15.69:0/257010362 shutdown_connections 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.712+0000 7f6dfd075640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f6df8102280 0x7f6df8198700 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.712+0000 7f6dfd075640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f6de40a2d70 0x7f6de40a5230 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.712+0000 7f6dfd075640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6df81078e0 0x7f6df819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.712+0000 7f6dfd075640 1 --2- 172.21.15.69:0/257010362 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f6df81018e0 0x7f6df81981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.712+0000 7f6dfd075640 1 -- 172.21.15.69:0/257010362 >> 172.21.15.69:0/257010362 conn(0x7f6df80fd600 msgr2=0x7f6df80fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.712+0000 7f6dfd075640 1 -- 172.21.15.69:0/257010362 shutdown_connections 2024-08-02T18:02:54.719 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.712+0000 7f6dfd075640 1 -- 172.21.15.69:0/257010362 wait complete. 2024-08-02T18:02:54.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.722+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9dc01033b0 con 0x7f9da80a2cf0 2024-08-02T18:02:54.732 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.730+0000 7f9dbcff9640 1 -- 172.21.15.69:0/2742563342 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f9dc01033b0 con 0x7f9da80a2cf0 2024-08-02T18:02:54.732 INFO:teuthology.orchestra.run.smithi069.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-02T00:00:00", "created": "2024-08-02T17:46:35", "first": "2024-08-02T17:47:00", "last": "2024-08-02T18:02:00", "last_pruned": "2024-08-02T18:02:00", "created_count": 16, "pruned_count": 10, "active": true} 2024-08-02T18:02:54.734 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9da80a2cf0 msgr2=0x7f9da80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9da80a2cf0 0x7f9da80a51b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9db0005fd0 tx=0x7f9db0005850 comp rx=0 tx=0).stop 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc0101070 msgr2=0x7f9dc0198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc0101070 0x7f9dc0198650 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009230 tx=0x7f9db4004270 comp rx=0 tx=0).stop 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 shutdown_connections 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f9dc0101070 0x7f9dc0198650 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f9da80a2cf0 0x7f9da80a51b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc01066e0 0x7f9dc019c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 --2- 172.21.15.69:0/2742563342 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f9dc01006d0 0x7f9dc0198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 >> 172.21.15.69:0/2742563342 conn(0x7f9dc00776d0 msgr2=0x7f9dc0069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 shutdown_connections 2024-08-02T18:02:54.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.733+0000 7f9dc5c8f640 1 -- 172.21.15.69:0/2742563342 wait complete. 2024-08-02T18:02:54.747 DEBUG:teuthology.orchestra.run.smithi069:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-02T18:02:54.852 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 -- 172.21.15.69:0/2656866898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 msgr2=0x7f2c70106af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 --2- 172.21.15.69:0/2656866898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 0x7f2c70106af0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009930 tx=0x7f2c6002a420 comp rx=0 tx=0).stop 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 -- 172.21.15.69:0/2656866898 shutdown_connections 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 --2- 172.21.15.69:0/2656866898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c701010a0 0x7f2c7010d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 --2- 172.21.15.69:0/2656866898 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c70100700 0x7f2c70100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 --2- 172.21.15.69:0/2656866898 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 0x7f2c70106af0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 -- 172.21.15.69:0/2656866898 >> 172.21.15.69:0/2656866898 conn(0x7f2c700fc3e0 msgr2=0x7f2c700fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 -- 172.21.15.69:0/2656866898 shutdown_connections 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.849+0000 7f2c75c30640 1 -- 172.21.15.69:0/2656866898 wait complete. 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c75c30640 1 Processor -- start 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c75c30640 1 -- start start 2024-08-02T18:02:54.853 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c75c30640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c70100700 0x7f2c70198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c75c30640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c701010a0 0x7f2c701985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c75c30640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 0x7f2c7019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c75c30640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70111c40 con 0x7f2c70106710 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c75c30640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70111ac0 con 0x7f2c70100700 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c75c30640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2c70111dc0 con 0x7f2c701010a0 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 0x7f2c7019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 0x7f2c7019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36276/0 (socket says 172.21.15.69:36276) 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6ffff640 1 -- 172.21.15.69:0/3910563167 learned_addr learned my addr 172.21.15.69:0/3910563167 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6f7fe640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c70100700 0x7f2c70198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6effd640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c701010a0 0x7f2c701985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6ffff640 1 -- 172.21.15.69:0/3910563167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c701010a0 msgr2=0x7f2c701985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6ffff640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c701010a0 0x7f2c701985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.854 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6ffff640 1 -- 172.21.15.69:0/3910563167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c70100700 msgr2=0x7f2c70198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6ffff640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c70100700 0x7f2c70198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.850+0000 7f2c6ffff640 1 -- 172.21.15.69:0/3910563167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7019d080 con 0x7f2c70106710 2024-08-02T18:02:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.851+0000 7f2c6ffff640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 0x7f2c7019c950 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800eaa0 tx=0x7f2c6800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.851+0000 7f2c6cff9640 1 -- 172.21.15.69:0/3910563167 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68004d20 con 0x7f2c70106710 2024-08-02T18:02:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.851+0000 7f2c6cff9640 1 -- 172.21.15.69:0/3910563167 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c68005020 con 0x7f2c70106710 2024-08-02T18:02:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.851+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7019d370 con 0x7f2c70106710 2024-08-02T18:02:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.851+0000 7f2c6cff9640 1 -- 172.21.15.69:0/3910563167 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c680116d0 con 0x7f2c70106710 2024-08-02T18:02:54.855 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.851+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c701a4c00 con 0x7f2c70106710 2024-08-02T18:02:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.852+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c70100b60 con 0x7f2c70106710 2024-08-02T18:02:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.853+0000 7f2c6cff9640 1 -- 172.21.15.69:0/3910563167 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2c680051d0 con 0x7f2c70106710 2024-08-02T18:02:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.853+0000 7f2c6cff9640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2c580a2d70 0x7f2c580a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.854+0000 7f2c6f7fe640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2c580a2d70 0x7f2c580a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.854+0000 7f2c6cff9640 1 -- 172.21.15.69:0/3910563167 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f2c680cf940 con 0x7f2c70106710 2024-08-02T18:02:54.856 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.854+0000 7f2c6f7fe640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2c580a2d70 0x7f2c580a5230 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009900 tx=0x7f2c600058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:54.859 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:54.858+0000 7f2c6cff9640 1 -- 172.21.15.69:0/3910563167 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f2c68016030 con 0x7f2c70106710 2024-08-02T18:02:55.063 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.061+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 --> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2c700fd240 con 0x7f2c580a2d70 2024-08-02T18:02:55.070 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.069+0000 7f2c6cff9640 1 -- 172.21.15.69:0/3910563167 <== mgr.15192 v2:172.21.15.69:6800/2335326389 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2c700fd240 con 0x7f2c580a2d70 2024-08-02T18:02:55.071 INFO:teuthology.orchestra.run.smithi069.stdout:/ 1m 6m3h 2024-08-02T18:02:55.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.071+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2c580a2d70 msgr2=0x7f2c580a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.071+0000 7f2c75c30640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2c580a2d70 0x7f2c580a5230 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009900 tx=0x7f2c600058d0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.071+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 msgr2=0x7f2c7019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.073 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.071+0000 7f2c75c30640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 0x7f2c7019c950 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800eaa0 tx=0x7f2c6800eea0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.072+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 shutdown_connections 2024-08-02T18:02:55.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.072+0000 7f2c75c30640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2c70106710 0x7f2c7019c950 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.072+0000 7f2c75c30640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f2c580a2d70 0x7f2c580a5230 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.072+0000 7f2c75c30640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2c701010a0 0x7f2c701985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.072+0000 7f2c75c30640 1 --2- 172.21.15.69:0/3910563167 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2c70100700 0x7f2c70198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.072+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 >> 172.21.15.69:0/3910563167 conn(0x7f2c700fc3e0 msgr2=0x7f2c700fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:55.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.072+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 shutdown_connections 2024-08-02T18:02:55.074 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.072+0000 7f2c75c30640 1 -- 172.21.15.69:0/3910563167 wait complete. 2024-08-02T18:02:55.083 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:54 smithi069 ceph-mon[32746]: pgmap v517: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-08-02T18:02:55.084 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-02T18:02:55.095 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-02T18:02:55.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:54 smithi083 ceph-mon[37031]: pgmap v517: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-08-02T18:02:55.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:54 smithi077 ceph-mon[37108]: pgmap v517: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-08-02T18:02:55.383 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_ceph_ceph-c_26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-02T18:02:55.383 INFO:tasks.fwd_scrub:done joining 2024-08-02T18:02:55.384 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-02T18:02:55.395 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-02T18:02:55.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 -- 172.21.15.69:0/183710036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec106720 msgr2=0x7fadec106b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 --2- 172.21.15.69:0/183710036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec106720 0x7fadec106b00 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7faddc009930 tx=0x7faddc02a420 comp rx=0 tx=0).stop 2024-08-02T18:02:55.507 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 -- 172.21.15.69:0/183710036 shutdown_connections 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 --2- 172.21.15.69:0/183710036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadec1010b0 0x7fadec10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 --2- 172.21.15.69:0/183710036 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadec100710 0x7fadec100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 --2- 172.21.15.69:0/183710036 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec106720 0x7fadec106b00 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 -- 172.21.15.69:0/183710036 >> 172.21.15.69:0/183710036 conn(0x7fadec0fc3f0 msgr2=0x7fadec0fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 -- 172.21.15.69:0/183710036 shutdown_connections 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 -- 172.21.15.69:0/183710036 wait complete. 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 Processor -- start 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.504+0000 7fadf29ee640 1 -- start start 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf29ee640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec100710 0x7fadec198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf29ee640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadec1010b0 0x7fadec198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf29ee640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadec106720 0x7fadec19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf29ee640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fadec111c50 con 0x7fadec100710 2024-08-02T18:02:55.508 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf29ee640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fadec111ad0 con 0x7fadec106720 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf29ee640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fadec111dd0 con 0x7fadec1010b0 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec100710 0x7fadec198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec100710 0x7fadec198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36302/0 (socket says 172.21.15.69:36302) 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 -- 172.21.15.69:0/256349353 learned_addr learned my addr 172.21.15.69:0/256349353 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadeb7fe640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadec1010b0 0x7fadec198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf0f8b640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadec106720 0x7fadec19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 -- 172.21.15.69:0/256349353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadec1010b0 msgr2=0x7fadec198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadec1010b0 0x7fadec198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 -- 172.21.15.69:0/256349353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadec106720 msgr2=0x7fadec19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadec106720 0x7fadec19cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 -- 172.21.15.69:0/256349353 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadec19d410 con 0x7fadec100710 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf0f8b640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadec106720 0x7fadec19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadeb7fe640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadec1010b0 0x7fadec198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:55.509 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadebfff640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec100710 0x7fadec198420 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7faddc009a50 tx=0x7faddc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:55.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fade97fa640 1 -- 172.21.15.69:0/256349353 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc03f030 con 0x7fadec100710 2024-08-02T18:02:55.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fade97fa640 1 -- 172.21.15.69:0/256349353 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faddc003ce0 con 0x7fadec100710 2024-08-02T18:02:55.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.505+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadec19d6a0 con 0x7fadec100710 2024-08-02T18:02:55.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.506+0000 7fade97fa640 1 -- 172.21.15.69:0/256349353 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc035070 con 0x7fadec100710 2024-08-02T18:02:55.510 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.506+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadec1a4f90 con 0x7fadec100710 2024-08-02T18:02:55.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.507+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadec101e50 con 0x7fadec100710 2024-08-02T18:02:55.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.508+0000 7fade97fa640 1 -- 172.21.15.69:0/256349353 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7faddc042030 con 0x7fadec100710 2024-08-02T18:02:55.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.508+0000 7fade97fa640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fadd40a2d00 0x7fadd40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:55.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.508+0000 7fadeb7fe640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fadd40a2d00 0x7fadd40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:55.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.508+0000 7fade97fa640 1 -- 172.21.15.69:0/256349353 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7faddc0efe90 con 0x7fadec100710 2024-08-02T18:02:55.511 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.509+0000 7fadeb7fe640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fadd40a2d00 0x7fadd40a51c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fadec0fe980 tx=0x7fade0005d90 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:55.513 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.512+0000 7fade97fa640 1 -- 172.21.15.69:0/256349353 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7faddc0bc6a0 con 0x7fadec100710 2024-08-02T18:02:55.693 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.691+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadec100b70 con 0x7fadec100710 2024-08-02T18:02:55.694 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.693+0000 7fade97fa640 1 -- 172.21.15.69:0/256349353 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+25257 (secure 0 0 0) 0x7faddc0bbfa0 con 0x7fadec100710 2024-08-02T18:02:55.695 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:55.695 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":115,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T18:02:14.676527+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3271476957":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3594391539":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3033851323":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/2680424375":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/452236630":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4036393791":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6800/452236630":"2024-08-03T17:46:24.900194+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fadd40a2d00 msgr2=0x7fadd40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fadd40a2d00 0x7fadd40a51c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fadec0fe980 tx=0x7fade0005d90 comp rx=0 tx=0).stop 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec100710 msgr2=0x7fadec198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec100710 0x7fadec198420 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7faddc009a50 tx=0x7faddc004270 comp rx=0 tx=0).stop 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 shutdown_connections 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fadd40a2d00 0x7fadd40a51c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fadec106720 0x7fadec19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fadec1010b0 0x7fadec198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 --2- 172.21.15.69:0/256349353 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fadec100710 0x7fadec198420 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 >> 172.21.15.69:0/256349353 conn(0x7fadec0fc3f0 msgr2=0x7fadec0fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:55.697 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 shutdown_connections 2024-08-02T18:02:55.698 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.695+0000 7fadf29ee640 1 -- 172.21.15.69:0/256349353 wait complete. 2024-08-02T18:02:55.708 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-02T17:39:54.984711+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-02T17:44:00.981682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-02T17:44:03.102100+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-02T17:44:09.380545+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-02T18:02:55.709 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 -- 172.21.15.69:0/653394329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 msgr2=0x7fd260101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 --2- 172.21.15.69:0/653394329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 0x7fd260101c00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fd254009930 tx=0x7fd25402a420 comp rx=0 tx=0).stop 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 -- 172.21.15.69:0/653394329 shutdown_connections 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 --2- 172.21.15.69:0/653394329 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 0x7fd260101c00 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 --2- 172.21.15.69:0/653394329 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd260102140 0x7fd26010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 --2- 172.21.15.69:0/653394329 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2601077b0 0x7fd260107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 -- 172.21.15.69:0/653394329 >> 172.21.15.69:0/653394329 conn(0x7fd2600fd4e0 msgr2=0x7fd2600ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 -- 172.21.15.69:0/653394329 shutdown_connections 2024-08-02T18:02:55.813 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.811+0000 7fd26685d640 1 -- 172.21.15.69:0/653394329 wait complete. 2024-08-02T18:02:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd26685d640 1 Processor -- start 2024-08-02T18:02:55.814 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd26685d640 1 -- start start 2024-08-02T18:02:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd26685d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 0x7fd260198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd26685d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd260102140 0x7fd260198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd26685d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2601077b0 0x7fd26019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd26685d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fd260111bf0 con 0x7fd2601017a0 2024-08-02T18:02:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd26685d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fd260111a70 con 0x7fd2601077b0 2024-08-02T18:02:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd26685d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd260111d70 con 0x7fd260102140 2024-08-02T18:02:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 0x7fd260198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:55.816 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 0x7fd260198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36324/0 (socket says 172.21.15.69:36324) 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 -- 172.21.15.69:0/2090021068 learned_addr learned my addr 172.21.15.69:0/2090021068 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25f7fe640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd260102140 0x7fd260198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd264dfa640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2601077b0 0x7fd26019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 -- 172.21.15.69:0/2090021068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd260102140 msgr2=0x7fd260198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd260102140 0x7fd260198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 -- 172.21.15.69:0/2090021068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2601077b0 msgr2=0x7fd26019c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2601077b0 0x7fd26019c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 -- 172.21.15.69:0/2090021068 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26019d0f0 con 0x7fd2601017a0 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25f7fe640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd260102140 0x7fd260198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd264dfa640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2601077b0 0x7fd26019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.812+0000 7fd25ffff640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 0x7fd260198100 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd25000d950 tx=0x7fd25000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.813+0000 7fd25d7fa640 1 -- 172.21.15.69:0/2090021068 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250017070 con 0x7fd2601017a0 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.813+0000 7fd25d7fa640 1 -- 172.21.15.69:0/2090021068 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd250009c30 con 0x7fd2601017a0 2024-08-02T18:02:55.817 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.813+0000 7fd25d7fa640 1 -- 172.21.15.69:0/2090021068 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25000f6b0 con 0x7fd2601017a0 2024-08-02T18:02:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.813+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26019d3e0 con 0x7fd2601017a0 2024-08-02T18:02:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.813+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2601a4c70 con 0x7fd2601017a0 2024-08-02T18:02:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.814+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd260102ee0 con 0x7fd2601017a0 2024-08-02T18:02:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.815+0000 7fd25d7fa640 1 -- 172.21.15.69:0/2090021068 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd2500109d0 con 0x7fd2601017a0 2024-08-02T18:02:55.818 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.815+0000 7fd25d7fa640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd2480a2d80 0x7fd2480a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:55.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.815+0000 7fd25f7fe640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd2480a2d80 0x7fd2480a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:55.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.815+0000 7fd25d7fa640 1 -- 172.21.15.69:0/2090021068 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fd2500cf430 con 0x7fd2601017a0 2024-08-02T18:02:55.819 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.816+0000 7fd25f7fe640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd2480a2d80 0x7fd2480a5240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd254005a20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:55.820 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.819+0000 7fd25d7fa640 1 -- 172.21.15.69:0/2090021068 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fd250009de0 con 0x7fd2601017a0 2024-08-02T18:02:55.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.997+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd260109780 con 0x7fd2601017a0 2024-08-02T18:02:55.999 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.997+0000 7fd25d7fa640 1 -- 172.21.15.69:0/2090021068 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7fd25009bc40 con 0x7fd2601017a0 2024-08-02T18:02:55.999 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 1 2024-08-02T18:02:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd2480a2d80 msgr2=0x7fd2480a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd2480a2d80 0x7fd2480a5240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd254000c00 tx=0x7fd254005a20 comp rx=0 tx=0).stop 2024-08-02T18:02:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 msgr2=0x7fd260198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 0x7fd260198100 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd25000d950 tx=0x7fd25000de10 comp rx=0 tx=0).stop 2024-08-02T18:02:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 shutdown_connections 2024-08-02T18:02:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fd2480a2d80 0x7fd2480a5240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fd2601077b0 0x7fd26019c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.001 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd260102140 0x7fd260198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 --2- 172.21.15.69:0/2090021068 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fd2601017a0 0x7fd260198100 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 >> 172.21.15.69:0/2090021068 conn(0x7fd2600fd4e0 msgr2=0x7fd2600fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:56.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 shutdown_connections 2024-08-02T18:02:56.002 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:55.999+0000 7fd26685d640 1 -- 172.21.15.69:0/2090021068 wait complete. 2024-08-02T18:02:56.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:55 smithi069 ceph-mon[32746]: from='client.19491 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T18:02:56.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:55 smithi069 ceph-mon[32746]: from='client.19515 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T18:02:56.011 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:55 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/256349353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T18:02:56.012 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-02T18:02:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 -- 172.21.15.69:0/3869673370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c10bde0 msgr2=0x7f100c10e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 --2- 172.21.15.69:0/3869673370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c10bde0 0x7f100c10e260 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f1004009900 tx=0x7f100402a480 comp rx=0 tx=0).stop 2024-08-02T18:02:56.116 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 -- 172.21.15.69:0/3869673370 shutdown_connections 2024-08-02T18:02:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 --2- 172.21.15.69:0/3869673370 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c10bde0 0x7f100c10e260 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 --2- 172.21.15.69:0/3869673370 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f100c1032b0 0x7f100c10b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 --2- 172.21.15.69:0/3869673370 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f100c102990 0x7f100c102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 -- 172.21.15.69:0/3869673370 >> 172.21.15.69:0/3869673370 conn(0x7f100c0fc3c0 msgr2=0x7f100c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 -- 172.21.15.69:0/3869673370 shutdown_connections 2024-08-02T18:02:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 -- 172.21.15.69:0/3869673370 wait complete. 2024-08-02T18:02:56.117 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 Processor -- start 2024-08-02T18:02:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.115+0000 7f1011cae640 1 -- start start 2024-08-02T18:02:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1011cae640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f100c102990 0x7f100c198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1011cae640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c1032b0 0x7f100c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1011cae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f100c10bde0 0x7f100c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1011cae640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f100c111bc0 con 0x7f100c1032b0 2024-08-02T18:02:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1011cae640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f100c111a40 con 0x7f100c102990 2024-08-02T18:02:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1011cae640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f100c111d40 con 0x7f100c10bde0 2024-08-02T18:02:56.119 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c1032b0 0x7f100c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c1032b0 0x7f100c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36346/0 (socket says 172.21.15.69:36346) 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 -- 172.21.15.69:0/2612910965 learned_addr learned my addr 172.21.15.69:0/2612910965 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100b7fe640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f100c102990 0x7f100c198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100bfff640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f100c10bde0 0x7f100c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 -- 172.21.15.69:0/2612910965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f100c10bde0 msgr2=0x7f100c19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f100c10bde0 0x7f100c19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 -- 172.21.15.69:0/2612910965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f100c102990 msgr2=0x7f100c198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f100c102990 0x7f100c198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 -- 172.21.15.69:0/2612910965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c19d060 con 0x7f100c1032b0 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100bfff640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f100c10bde0 0x7f100c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100b7fe640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f100c102990 0x7f100c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f100affd640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c1032b0 0x7f100c1985b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f100000d740 tx=0x7f100000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:56.120 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1008ff9640 1 -- 172.21.15.69:0/2612910965 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000017070 con 0x7f100c1032b0 2024-08-02T18:02:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c19d350 con 0x7f100c1032b0 2024-08-02T18:02:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1008ff9640 1 -- 172.21.15.69:0/2612910965 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100000de50 con 0x7f100c1032b0 2024-08-02T18:02:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1008ff9640 1 -- 172.21.15.69:0/2612910965 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100000f620 con 0x7f100c1032b0 2024-08-02T18:02:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.116+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c1a4be0 con 0x7f100c1032b0 2024-08-02T18:02:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.117+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100c102d70 con 0x7f100c1032b0 2024-08-02T18:02:56.121 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.118+0000 7f1008ff9640 1 -- 172.21.15.69:0/2612910965 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1000009c30 con 0x7f100c1032b0 2024-08-02T18:02:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.119+0000 7f1008ff9640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0ff40a2d00 0x7f0ff40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.119+0000 7f100b7fe640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0ff40a2d00 0x7f0ff40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.119+0000 7f1008ff9640 1 -- 172.21.15.69:0/2612910965 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f10000cf490 con 0x7f100c1032b0 2024-08-02T18:02:56.122 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.119+0000 7f100b7fe640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0ff40a2d00 0x7f0ff40a51c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc005fd0 tx=0x7f0ffc009500 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:56.124 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.123+0000 7f1008ff9640 1 -- 172.21.15.69:0/2612910965 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f100009bca0 con 0x7f100c1032b0 2024-08-02T18:02:56.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:55 smithi083 ceph-mon[37031]: from='client.19491 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T18:02:56.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:55 smithi083 ceph-mon[37031]: from='client.19515 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T18:02:56.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:55 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/256349353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T18:02:56.298 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.297+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f100c1068a0 con 0x7f100c1032b0 2024-08-02T18:02:56.299 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.297+0000 7f1008ff9640 1 -- 172.21.15.69:0/2612910965 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7f100009b5a0 con 0x7f100c1032b0 2024-08-02T18:02:56.299 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-08-02T18:02:56.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0ff40a2d00 msgr2=0x7f0ff40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0ff40a2d00 0x7f0ff40a51c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc005fd0 tx=0x7f0ffc009500 comp rx=0 tx=0).stop 2024-08-02T18:02:56.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c1032b0 msgr2=0x7f100c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c1032b0 0x7f100c1985b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f100000d740 tx=0x7f100000dc00 comp rx=0 tx=0).stop 2024-08-02T18:02:56.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 shutdown_connections 2024-08-02T18:02:56.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f100c1032b0 0x7f100c1985b0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f0ff40a2d00 0x7f0ff40a51c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.301 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f100c10bde0 0x7f100c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 --2- 172.21.15.69:0/2612910965 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f100c102990 0x7f100c198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 >> 172.21.15.69:0/2612910965 conn(0x7f100c0fc3c0 msgr2=0x7f100c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:56.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 shutdown_connections 2024-08-02T18:02:56.302 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.299+0000 7f1011cae640 1 -- 172.21.15.69:0/2612910965 wait complete. 2024-08-02T18:02:56.312 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-02T18:02:56.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:55 smithi077 ceph-mon[37108]: from='client.19491 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T18:02:56.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:55 smithi077 ceph-mon[37108]: from='client.19515 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-02T18:02:56.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:55 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/256349353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T18:02:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.414+0000 7fae480da640 1 -- 172.21.15.69:0/2561489594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 msgr2=0x7fae4010d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.414+0000 7fae480da640 1 --2- 172.21.15.69:0/2561489594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 0x7fae4010d5b0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fae38009cd0 tx=0x7fae3802a8e0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.418 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.414+0000 7fae480da640 1 -- 172.21.15.69:0/2561489594 shutdown_connections 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.414+0000 7fae480da640 1 --2- 172.21.15.69:0/2561489594 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 0x7fae4010d5b0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.414+0000 7fae480da640 1 --2- 172.21.15.69:0/2561489594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae401006e0 0x7fae40100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.414+0000 7fae480da640 1 --2- 172.21.15.69:0/2561489594 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401066f0 0x7fae40106ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 -- 172.21.15.69:0/2561489594 >> 172.21.15.69:0/2561489594 conn(0x7fae400fc3e0 msgr2=0x7fae400fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 -- 172.21.15.69:0/2561489594 shutdown_connections 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 -- 172.21.15.69:0/2561489594 wait complete. 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 Processor -- start 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 -- start start 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401006e0 0x7fae40198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 0x7fae401993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae401066f0 0x7fae401a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fae40110b30 con 0x7fae40101080 2024-08-02T18:02:56.419 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fae401109b0 con 0x7fae401006e0 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.415+0000 7fae480da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fae40110cb0 con 0x7fae401066f0 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 0x7fae401993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 0x7fae401993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36372/0 (socket says 172.21.15.69:36372) 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 -- 172.21.15.69:0/1469646746 learned_addr learned my addr 172.21.15.69:0/1469646746 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45e76640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401006e0 0x7fae40198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae46677640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae401066f0 0x7fae401a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 -- 172.21.15.69:0/1469646746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae401066f0 msgr2=0x7fae401a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae401066f0 0x7fae401a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 -- 172.21.15.69:0/1469646746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401006e0 msgr2=0x7fae40198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401006e0 0x7fae40198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 -- 172.21.15.69:0/1469646746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae401a4460 con 0x7fae40101080 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45e76640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401006e0 0x7fae40198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:02:56.420 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae46677640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae401066f0 0x7fae401a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae45675640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 0x7fae401993e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c920 tx=0x7fae3000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae36ffd640 1 -- 172.21.15.69:0/1469646746 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30007c70 con 0x7fae40101080 2024-08-02T18:02:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae36ffd640 1 -- 172.21.15.69:0/1469646746 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae30007e10 con 0x7fae40101080 2024-08-02T18:02:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae36ffd640 1 -- 172.21.15.69:0/1469646746 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30018890 con 0x7fae40101080 2024-08-02T18:02:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae401a4630 con 0x7fae40101080 2024-08-02T18:02:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.416+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae401a4ba0 con 0x7fae40101080 2024-08-02T18:02:56.421 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.417+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae40100b40 con 0x7fae40101080 2024-08-02T18:02:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.418+0000 7fae36ffd640 1 -- 172.21.15.69:0/1469646746 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fae30018a50 con 0x7fae40101080 2024-08-02T18:02:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.419+0000 7fae36ffd640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fae2c0a2d80 0x7fae2c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.419+0000 7fae45e76640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fae2c0a2d80 0x7fae2c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.419+0000 7fae36ffd640 1 -- 172.21.15.69:0/1469646746 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7fae300cfc50 con 0x7fae40101080 2024-08-02T18:02:56.422 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.420+0000 7fae45e76640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fae2c0a2d80 0x7fae2c0a5240 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fae3c005d90 tx=0x7fae3c005d20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:56.423 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.422+0000 7fae36ffd640 1 -- 172.21.15.69:0/1469646746 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fae30014030 con 0x7fae40101080 2024-08-02T18:02:56.602 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.601+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fae401a4e60 con 0x7fae40101080 2024-08-02T18:02:56.603 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.601+0000 7fae36ffd640 1 -- 172.21.15.69:0/1469646746 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7fae3000de20 con 0x7fae40101080 2024-08-02T18:02:56.603 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 64 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fae2c0a2d80 msgr2=0x7fae2c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fae2c0a2d80 0x7fae2c0a5240 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fae3c005d90 tx=0x7fae3c005d20 comp rx=0 tx=0).stop 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 msgr2=0x7fae401993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 0x7fae401993e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c920 tx=0x7fae3000cde0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 shutdown_connections 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fae40101080 0x7fae401993e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fae2c0a2d80 0x7fae2c0a5240 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fae401066f0 0x7fae401a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 --2- 172.21.15.69:0/1469646746 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fae401006e0 0x7fae40198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 >> 172.21.15.69:0/1469646746 conn(0x7fae400fc3e0 msgr2=0x7fae400fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:56.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 shutdown_connections 2024-08-02T18:02:56.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.603+0000 7fae480da640 1 -- 172.21.15.69:0/1469646746 wait complete. 2024-08-02T18:02:56.616 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-02T18:02:56.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 -- 172.21.15.69:0/912234840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 msgr2=0x7f688410b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.722 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 --2- 172.21.15.69:0/912234840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 0x7f688410b7c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f6874009900 tx=0x7f687402a480 comp rx=0 tx=0).stop 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 -- 172.21.15.69:0/912234840 shutdown_connections 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 --2- 172.21.15.69:0/912234840 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 0x7f688410b7c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 --2- 172.21.15.69:0/912234840 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688410bde0 0x7f688410e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 --2- 172.21.15.69:0/912234840 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68841029c0 0x7f6884102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 -- 172.21.15.69:0/912234840 >> 172.21.15.69:0/912234840 conn(0x7f68840776e0 msgr2=0x7f68840fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 -- 172.21.15.69:0/912234840 shutdown_connections 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 -- 172.21.15.69:0/912234840 wait complete. 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 Processor -- start 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.719+0000 7f688ba9d640 1 -- start start 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688ba9d640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68841029c0 0x7f6884100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688ba9d640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 0x7f6884100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688ba9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688410bde0 0x7f6884101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.723 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688ba9d640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f6884111bb0 con 0x7f68841032e0 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688ba9d640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f6884111a30 con 0x7f68841029c0 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688ba9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6884111d30 con 0x7f688410bde0 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 0x7f6884100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 0x7f6884100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36390/0 (socket says 172.21.15.69:36390) 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 -- 172.21.15.69:0/3617487629 learned_addr learned my addr 172.21.15.69:0/3617487629 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889839640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68841029c0 0x7f6884100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688a03a640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688410bde0 0x7f6884101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 -- 172.21.15.69:0/3617487629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688410bde0 msgr2=0x7f6884101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688410bde0 0x7f6884101090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 -- 172.21.15.69:0/3617487629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68841029c0 msgr2=0x7f6884100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68841029c0 0x7f6884100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 -- 172.21.15.69:0/3617487629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68841a6580 con 0x7f68841032e0 2024-08-02T18:02:56.724 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889839640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68841029c0 0x7f6884100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:56.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f6889038640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 0x7f6884100b50 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6874006010 tx=0x7f68740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:56.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f687affd640 1 -- 172.21.15.69:0/3617487629 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687403f030 con 0x7f68841032e0 2024-08-02T18:02:56.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f687affd640 1 -- 172.21.15.69:0/3617487629 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6874003ce0 con 0x7f68841032e0 2024-08-02T18:02:56.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68841a6810 con 0x7f68841032e0 2024-08-02T18:02:56.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f687affd640 1 -- 172.21.15.69:0/3617487629 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874035070 con 0x7f68841032e0 2024-08-02T18:02:56.725 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.720+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68841a6d00 con 0x7f68841032e0 2024-08-02T18:02:56.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.721+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6884102da0 con 0x7f68841032e0 2024-08-02T18:02:56.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.723+0000 7f687affd640 1 -- 172.21.15.69:0/3617487629 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6874042030 con 0x7f68841032e0 2024-08-02T18:02:56.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.724+0000 7f687affd640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68700a2d00 0x7f68700a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:56.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.724+0000 7f6889839640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68700a2d00 0x7f68700a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:56.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.724+0000 7f687affd640 1 -- 172.21.15.69:0/3617487629 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f68740f02b0 con 0x7f68841032e0 2024-08-02T18:02:56.726 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.724+0000 7f6889839640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68700a2d00 0x7f68700a51c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6880005d00 tx=0x7f6880009550 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:56.728 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.727+0000 7f687affd640 1 -- 172.21.15.69:0/3617487629 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f68740bcb80 con 0x7f68841032e0 2024-08-02T18:02:56.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.903+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6884106de0 con 0x7f68841032e0 2024-08-02T18:02:56.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.903+0000 7f687affd640 1 -- 172.21.15.69:0/3617487629 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v115) ==== 103+0+11 (secure 0 0 0) 0x7f68740bc480 con 0x7f68841032e0 2024-08-02T18:02:56.905 INFO:teuthology.orchestra.run.smithi069.stdout:pg_num: 32 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68700a2d00 msgr2=0x7f68700a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68700a2d00 0x7f68700a51c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6880005d00 tx=0x7f6880009550 comp rx=0 tx=0).stop 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 msgr2=0x7f6884100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 0x7f6884100b50 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f6874006010 tx=0x7f68740042f0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 shutdown_connections 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f68841032e0 0x7f6884100b50 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f68700a2d00 0x7f68700a51c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f688410bde0 0x7f6884101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 --2- 172.21.15.69:0/3617487629 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f68841029c0 0x7f6884100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 >> 172.21.15.69:0/3617487629 conn(0x7f68840776e0 msgr2=0x7f68840fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:56.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 shutdown_connections 2024-08-02T18:02:56.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:56.905+0000 7f688ba9d640 1 -- 172.21.15.69:0/3617487629 wait complete. 2024-08-02T18:02:56.918 DEBUG:teuthology.orchestra.run.smithi069:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 -- 172.21.15.69:0/324023121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100530 msgr2=0x7f2538100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 --2- 172.21.15.69:0/324023121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100530 0x7f2538100910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f252800aa10 tx=0x7f252802b750 comp rx=0 tx=0).stop 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 -- 172.21.15.69:0/324023121 shutdown_connections 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 --2- 172.21.15.69:0/324023121 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2538105220 0x7f253810cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 --2- 172.21.15.69:0/324023121 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2538100ee0 0x7f2538104ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 --2- 172.21.15.69:0/324023121 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100530 0x7f2538100910 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 -- 172.21.15.69:0/324023121 >> 172.21.15.69:0/324023121 conn(0x7f2538075bb0 msgr2=0x7f2538075fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 -- 172.21.15.69:0/324023121 shutdown_connections 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.018+0000 7f253dc56640 1 -- 172.21.15.69:0/324023121 wait complete. 2024-08-02T18:02:57.020 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f253dc56640 1 Processor -- start 2024-08-02T18:02:57.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f253dc56640 1 -- start start 2024-08-02T18:02:57.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f253dc56640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2538100530 0x7f2538101b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:57.022 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f253dc56640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100ee0 0x7f25381020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f253dc56640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2538105220 0x7f25381025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f253dc56640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f253810fa40 con 0x7f2538100ee0 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f253dc56640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f253810f8c0 con 0x7f2538100530 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f253dc56640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f253810fbc0 con 0x7f2538105220 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100ee0 0x7f25381020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100ee0 0x7f25381020a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36396/0 (socket says 172.21.15.69:36396) 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 -- 172.21.15.69:0/2117934104 learned_addr learned my addr 172.21.15.69:0/2117934104 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f25377fe640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2538100530 0x7f2538101b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2537fff640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2538105220 0x7f25381025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 -- 172.21.15.69:0/2117934104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2538105220 msgr2=0x7f25381025e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2538105220 0x7f25381025e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 -- 172.21.15.69:0/2117934104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2538100530 msgr2=0x7f2538101b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:57.023 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2538100530 0x7f2538101b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 -- 172.21.15.69:0/2117934104 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25381a66a0 con 0x7f2538100ee0 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f25377fe640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2538100530 0x7f2538101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.019+0000 7f2536ffd640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100ee0 0x7f25381020a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f252c006fd0 tx=0x7f252c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.020+0000 7f2534ff9640 1 -- 172.21.15.69:0/2117934104 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c011830 con 0x7f2538100ee0 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.020+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25381a6990 con 0x7f2538100ee0 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.020+0000 7f2534ff9640 1 -- 172.21.15.69:0/2117934104 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f252c016070 con 0x7f2538100ee0 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.020+0000 7f2534ff9640 1 -- 172.21.15.69:0/2117934104 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c00f2e0 con 0x7f2538100ee0 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.020+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25381a6e90 con 0x7f2538100ee0 2024-08-02T18:02:57.024 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.021+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538100910 con 0x7f2538100ee0 2024-08-02T18:02:57.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.022+0000 7f2534ff9640 1 -- 172.21.15.69:0/2117934104 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f252c0119d0 con 0x7f2538100ee0 2024-08-02T18:02:57.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.022+0000 7f2534ff9640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25200a2d00 0x7f25200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:02:57.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.022+0000 7f2534ff9640 1 -- 172.21.15.69:0/2117934104 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9568+0+0 (secure 0 0 0) 0x7f252c0cf530 con 0x7f2538100ee0 2024-08-02T18:02:57.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.022+0000 7f25377fe640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25200a2d00 0x7f25200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:02:57.025 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.023+0000 7f25377fe640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25200a2d00 0x7f25200a51c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f252800a7d0 tx=0x7f25280305e0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:02:57.027 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.025+0000 7f2534ff9640 1 -- 172.21.15.69:0/2117934104 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f252c09be50 con 0x7f2538100ee0 2024-08-02T18:02:57.248 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.247+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2538103090 con 0x7f2538100ee0 2024-08-02T18:02:57.256 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.254+0000 7f2534ff9640 1 -- 172.21.15.69:0/2117934104 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740901 (secure 0 0 0) 0x7f252c09b750 con 0x7f2538100ee0 2024-08-02T18:02:57.265 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "epoch": 25, 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "active_gid": 15192, 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "active_name": "x", 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addrs": { 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6800", 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2335326389 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.266 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v1", 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:6801", 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2335326389 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: "active_addr": "172.21.15.69:6801/2335326389", 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: "active_change": "2024-08-02T17:46:24.900522+0000", 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: "active_mgr_features": 4540701547738038271, 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: "available": true, 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: "standbys": [ 2024-08-02T18:02:57.267 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24802, 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "z", 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.268 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.270 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.272 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.273 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-08-02T18:02:57.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-08-02T18:02:57.275 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-08-02T18:02:57.276 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-08-02T18:02:57.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-08-02T18:02:57.278 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.279 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-08-02T18:02:57.280 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-08-02T18:02:57.281 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.282 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.283 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-08-02T18:02:57.285 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-08-02T18:02:57.286 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-08-02T18:02:57.288 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-08-02T18:02:57.290 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.291 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.292 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.293 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.294 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.296 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.297 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-08-02T18:02:57.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.301 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-08-02T18:02:57.302 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-02T18:02:57.303 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.305 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.306 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.307 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-08-02T18:02:57.308 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.309 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-08-02T18:02:57.310 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-08-02T18:02:57.311 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-08-02T18:02:57.312 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-08-02T18:02:57.313 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.314 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.315 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.316 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.317 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.318 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.319 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.321 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-08-02T18:02:57.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.323 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.324 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.326 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.327 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.328 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.330 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.331 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-08-02T18:02:57.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-08-02T18:02:57.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.334 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.335 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.336 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-02T18:02:57.337 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.339 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.340 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.341 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.342 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.343 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.344 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.345 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.346 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.347 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.349 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.350 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-08-02T18:02:57.351 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.352 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.352 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.352 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.352 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-08-02T18:02:57.352 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-08-02T18:02:57.353 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.353 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.353 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.354 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.356 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.357 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.359 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.361 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.363 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.364 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.365 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.366 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-08-02T18:02:57.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.368 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.369 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.370 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.372 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.373 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.374 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.375 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.376 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.379 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-02T18:02:57.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-02T18:02:57.381 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-02T18:02:57.382 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.383 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.384 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.385 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.386 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.387 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.391 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.392 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.395 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.398 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.399 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.400 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.401 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.402 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.403 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.404 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.405 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.406 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.407 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.408 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-02T18:02:57.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.411 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.413 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-02T18:02:57.415 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-08-02T18:02:57.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.417 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.418 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.421 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.422 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.423 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.425 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.426 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.427 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.428 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.429 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.430 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.431 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.432 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.433 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.434 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.436 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.437 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.439 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.440 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-08-02T18:02:57.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.442 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.443 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.446 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.449 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.450 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.453 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-02T18:02:57.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.456 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.457 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.458 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.459 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.461 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-08-02T18:02:57.463 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-08-02T18:02:57.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.465 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:57.466 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.469 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.470 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.471 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-08-02T18:02:57.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-08-02T18:02:57.473 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-08-02T18:02:57.474 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.476 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.478 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.479 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.480 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.481 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.482 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.486 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.487 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.488 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.489 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-08-02T18:02:57.491 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.492 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.494 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.495 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.496 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-08-02T18:02:57.497 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.498 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.499 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.500 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.502 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.505 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-08-02T18:02:57.507 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.510 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.511 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.514 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.515 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.518 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.519 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.520 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.524 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.526 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.527 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.529 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.530 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.531 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.532 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.534 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.535 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.536 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.538 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.539 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.540 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.541 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.543 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.544 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.545 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.546 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.547 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.550 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.551 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-08-02T18:02:57.552 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.553 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.557 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.557 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.559 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.562 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-08-02T18:02:57.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.564 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.565 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.566 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.567 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.568 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.569 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.570 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.571 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.572 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.573 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.574 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.575 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.576 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.577 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.578 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-08-02T18:02:57.579 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-08-02T18:02:57.580 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-08-02T18:02:57.581 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.582 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.583 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.584 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.585 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.586 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.587 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-08-02T18:02:57.588 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-08-02T18:02:57.589 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.590 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.591 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.592 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.593 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.594 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.595 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.596 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.597 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.598 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.599 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.600 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.601 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.602 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.603 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.604 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.605 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.606 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.607 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.608 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.609 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.610 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.611 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.612 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.613 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.614 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.615 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.616 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.617 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.618 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.619 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.620 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-08-02T18:02:57.621 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-08-02T18:02:57.622 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.623 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-08-02T18:02:57.624 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-08-02T18:02:57.625 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.626 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.627 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.628 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.629 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.630 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.631 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.632 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.633 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.634 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.635 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.636 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.637 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.638 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.639 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.640 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.641 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.642 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.643 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.644 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.645 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.646 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.647 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.648 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.649 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.650 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.651 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.652 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.653 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.654 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.655 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.656 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.657 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.658 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.659 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.660 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.661 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.662 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-08-02T18:02:57.663 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.664 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.665 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.666 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.667 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.668 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.669 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.670 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.671 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.672 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.673 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.674 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.675 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.676 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.677 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-08-02T18:02:57.678 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.679 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.680 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.681 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.683 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.684 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.685 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.686 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.687 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.688 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.689 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.690 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-02T18:02:57.691 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-08-02T18:02:57.692 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-02T18:02:57.693 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-02T18:02:57.694 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.695 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.696 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.697 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.698 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.699 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.700 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.701 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.702 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.703 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.704 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.705 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.706 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "gid": 24896, 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "y", 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "mgr_features": 4540701547738038271, 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.707 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.708 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.709 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.710 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.711 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.712 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.713 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.714 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-08-02T18:02:57.715 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.716 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.717 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.718 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.719 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.720 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.721 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-08-02T18:02:57.722 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-08-02T18:02:57.723 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.724 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.725 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.726 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.727 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-08-02T18:02:57.728 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-08-02T18:02:57.729 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-02T18:02:57.730 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.731 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.732 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.733 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.734 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.735 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.736 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.737 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.738 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.739 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.740 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.741 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.742 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-08-02T18:02:57.743 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-08-02T18:02:57.744 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-08-02T18:02:57.745 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-08-02T18:02:57.746 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.747 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.748 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-08-02T18:02:57.749 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-02T18:02:57.750 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.751 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.752 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.753 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.754 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.755 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.756 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.757 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-08-02T18:02:57.758 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-08-02T18:02:57.759 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.760 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.761 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-02T18:02:57.762 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.763 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.764 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-02T18:02:57.765 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.766 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.767 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-02T18:02:57.768 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.769 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.770 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.771 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.772 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.773 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.774 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-08-02T18:02:57.775 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.776 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.777 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.778 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.779 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.780 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.781 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.782 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.783 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.784 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.785 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.786 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-08-02T18:02:57.787 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.788 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.789 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.790 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.791 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.792 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.793 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.794 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.795 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.796 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.797 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.798 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-02T18:02:57.799 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-02T18:02:57.800 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.801 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.802 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.803 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.804 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.805 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.806 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.807 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.808 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.809 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.810 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.811 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.812 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.813 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.814 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.815 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.816 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.817 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.818 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.819 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.820 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.821 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.822 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.823 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.824 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.825 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.826 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.827 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.828 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.829 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.830 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-02T18:02:57.831 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.832 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.833 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.834 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.835 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.836 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.837 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.838 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.839 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.840 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.841 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.842 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.843 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.844 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.845 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.846 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.847 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.848 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.849 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.850 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.851 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.852 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-02T18:02:57.853 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-02T18:02:57.854 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-02T18:02:57.855 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-08-02T18:02:57.856 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-08-02T18:02:57.857 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.858 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.859 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.860 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.861 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.862 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.863 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.864 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.865 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.866 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.867 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.868 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-08-02T18:02:57.869 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.870 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.871 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.872 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.873 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.874 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.875 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.876 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.877 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.878 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.879 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.880 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.881 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.882 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.883 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.884 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.885 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-08-02T18:02:57.886 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.887 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-02T18:02:57.888 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.889 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.890 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.891 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.892 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.893 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.894 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.895 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.896 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.897 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.898 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.899 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-08-02T18:02:57.900 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-08-02T18:02:57.901 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.902 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.903 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.904 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.905 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.906 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.907 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.908 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.909 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.910 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.911 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.912 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.913 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.914 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.915 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.916 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.917 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.918 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.919 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.920 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-08-02T18:02:57.921 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.922 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.923 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.924 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.925 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.926 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.927 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.928 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.929 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.930 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.931 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.932 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.933 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.934 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-08-02T18:02:57.935 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.936 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.937 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.938 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.939 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.940 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-08-02T18:02:57.941 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.942 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.943 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-08-02T18:02:57.944 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.945 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.946 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.947 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.948 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.949 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.950 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.951 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.952 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.953 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.954 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.955 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.956 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.957 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.958 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.959 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.960 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.961 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.962 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.963 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-08-02T18:02:57.964 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.965 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.965 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.965 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.966 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.967 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.968 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.969 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.970 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.971 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.972 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.973 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.974 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.975 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.976 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.977 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.978 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.979 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.980 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.981 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.982 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.983 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.984 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.985 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.986 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.987 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.988 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.989 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.990 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.991 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.992 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.993 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.994 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:57.995 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.996 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:57.997 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.998 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:57.999 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:58.000 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.001 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.002 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-02T18:02:58.003 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.004 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.005 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.006 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.007 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.008 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.009 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.010 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.011 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.012 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-08-02T18:02:58.013 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.014 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.015 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.016 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.017 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.018 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.019 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.020 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.021 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.022 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.023 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.024 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-08-02T18:02:58.025 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.026 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.027 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.028 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.029 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.030 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.031 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.032 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.033 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.034 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.035 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.036 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.037 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.038 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.039 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.040 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.041 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.042 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.043 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.044 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.045 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.046 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.047 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.048 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.049 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.050 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.051 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.052 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.053 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.054 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.055 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.056 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-08-02T18:02:58.057 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-08-02T18:02:58.058 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-08-02T18:02:58.059 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.060 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.061 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.062 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.063 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.064 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.065 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.066 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.067 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.068 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.069 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.070 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.071 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.072 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.073 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.074 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.075 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.076 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.077 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.078 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.079 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.080 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.081 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.082 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.083 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.084 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.085 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.086 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.087 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.088 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.089 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.090 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.091 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.092 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-02T18:02:58.093 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.094 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.095 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.096 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-08-02T18:02:58.097 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.098 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.099 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.100 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.101 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.102 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.103 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.104 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.105 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.106 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.107 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.108 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.109 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.110 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.111 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.112 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.113 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.114 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.115 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.116 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.117 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.118 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.119 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.120 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.121 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.122 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-02T18:02:58.123 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-02T18:02:58.124 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.125 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-08-02T18:02:58.126 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.127 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.128 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.129 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.130 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.131 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.132 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.133 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-08-02T18:02:58.134 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.135 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "modules": [ 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard", 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "iostat", 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "nfs", 2024-08-02T18:02:58.136 INFO:teuthology.orchestra.run.smithi069.stdout: "restful", 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "snap_schedule" 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "available_modules": [ 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "alerts", 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.137 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How frequently to reexamine health status", 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.138 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.139 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.140 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.141 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.142 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_destination": { 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_destination", 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.143 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email address to send alerts to", 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_from_name": { 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_from_name", 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.144 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "Ceph", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Email From: name", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_host": { 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_host", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.145 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP server", 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_password": { 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_password", 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.146 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Password to authenticate with", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_port": { 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_port", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "465", 2024-08-02T18:02:58.147 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP port", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_sender": { 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_sender", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.148 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SMTP envelope sender", 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_ssl": { 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_ssl", 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.149 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "smtp_user": { 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "smtp_user", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "User to authenticate as", 2024-08-02T18:02:58.150 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "balancer", 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "active": { 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "active", 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.151 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically balance PGs across cluster", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_time": { 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_time", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.152 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0000", 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "beginning time of day to automatically balance", 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "begin_weekday": { 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "begin_weekday", 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.153 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_max_iterations": { 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_max_iterations", 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:58.154 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "25", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "250", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_metrics": { 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_metrics", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.155 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "pgs,objects,bytes", 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "crush_compat_step": { 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crush_compat_step", 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.156 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.5", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0.001", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "0.999", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "aggressiveness of optimization", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "end_time": { 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_time", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2359", 2024-08-02T18:02:58.157 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ending time of day to automatically balance", 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "end_weekday": { 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "end_weekday", 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:58.158 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "0", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "6", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.159 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.160 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.161 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.162 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.163 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "min_score": { 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_score", 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.164 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "upmap", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "crush-compat", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "none", 2024-08-02T18:02:58.165 INFO:teuthology.orchestra.run.smithi069.stdout: "read", 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap", 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap-read" 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Balancer mode", 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_ids": { 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_ids", 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.166 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.169 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.169 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.169 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.170 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_deviation": { 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_deviation", 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.171 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "upmap_max_optimizations": { 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "upmap_max_optimizations", 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.172 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.173 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm", 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_down_multiplier": { 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_down_multiplier", 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.174 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_refresh_rate": { 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_refresh_rate", 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "20", 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.175 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "agent_starting_port": { 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "agent_starting_port", 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4721", 2024-08-02T18:02:58.176 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_ptrace": { 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_ptrace", 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.177 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_interval": { 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_interval", 2024-08-02T18:02:58.178 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "autotune_memory_target_ratio": { 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "autotune_memory_target_ratio", 2024-08-02T18:02:58.179 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0.7", 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm_log_destination": { 2024-08-02T18:02:58.180 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cephadm_log_destination", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "file", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "file,syslog", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "syslog" 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.181 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "cgroups_split": { 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cgroups_split", 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-02T18:02:58.182 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "config_checks_enabled": { 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_checks_enabled", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.183 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "config_dashboard": { 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_dashboard", 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.184 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_alertmanager": { 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_alertmanager", 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.185 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_base": { 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_base", 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Container image name, without the tag", 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.186 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_elasticsearch": { 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_elasticsearch", 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "elasticsearch container image", 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.187 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_grafana": { 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_grafana", 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_haproxy": { 2024-08-02T18:02:58.188 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_haproxy", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "HAproxy container image", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_agent": { 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_agent", 2024-08-02T18:02:58.189 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger agent container image", 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_collector": { 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_collector", 2024-08-02T18:02:58.190 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger collector container image", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_jaeger_query": { 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_jaeger_query", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.191 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Jaeger query container image", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_keepalived": { 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_keepalived", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-02T18:02:58.192 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Keepalived container image", 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_loki": { 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_loki", 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-02T18:02:58.193 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Loki container image", 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_node_exporter": { 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_node_exporter", 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-08-02T18:02:58.194 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_nvmeof": { 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_nvmeof", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.195 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Nvme-of container image", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_prometheus": { 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_prometheus", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.196 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Prometheus container image", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_promtail": { 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_promtail", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Promtail container image", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.197 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_samba": { 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_samba", 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Samba/SMB container image", 2024-08-02T18:02:58.198 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "container_image_snmp_gateway": { 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_image_snmp_gateway", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "SNMP Gateway container image", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.199 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "container_init": { 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "container_init", 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Run podman/docker with `--init`", 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.200 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "daemon_cache_timeout": { 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "daemon_cache_timeout", 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.201 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "default_cephadm_command_timeout": { 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_cephadm_command_timeout", 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "900", 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.202 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "default_registry": { 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "default_registry", 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "docker.io", 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.203 INFO:teuthology.orchestra.run.smithi069.stdout: "device_cache_timeout": { 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_cache_timeout", 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1800", 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache device inventory", 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.204 INFO:teuthology.orchestra.run.smithi069.stdout: "device_enhanced_scan": { 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_enhanced_scan", 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.205 INFO:teuthology.orchestra.run.smithi069.stdout: "facts_cache_timeout": { 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "facts_cache_timeout", 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "seconds to cache host facts data", 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.206 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "grafana_dashboards_path": { 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "grafana_dashboards_path", 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.207 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "host_check_interval": { 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "host_check_interval", 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to perform a host check", 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.208 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "hw_monitoring": { 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hw_monitoring", 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.209 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "inventory_list_all": { 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "inventory_list_all", 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.210 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.211 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "log_refresh_metadata": { 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_refresh_metadata", 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.212 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.213 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.214 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.215 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.216 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.217 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "*", 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "max_count_per_host": { 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_count_per_host", 2024-08-02T18:02:58.218 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of daemons per service per host", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "max_osd_draining_count": { 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_osd_draining_count", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.219 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "migration_current": { 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "migration_current", 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.220 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "internal - do not modify", 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "mode": { 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mode", 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.221 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "root", 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm-package", 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "root" 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "mode for remote execution of cephadm", 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.222 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "oob_default_addr": { 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "oob_default_addr", 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "169.254.1.1", 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.223 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "prometheus_alerts_path": { 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus_alerts_path", 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.224 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_insecure": { 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_insecure", 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.225 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_password": { 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_password", 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.226 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_url": { 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_url", 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.227 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "registry_username": { 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "registry_username", 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.228 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "secure_monitoring_stack": { 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "secure_monitoring_stack", 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-02T18:02:58.229 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "service_discovery_port": { 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "service_discovery_port", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8765", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "cephadm service discovery port", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.230 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "ssh_config_file": { 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssh_config_file", 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.231 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "use_agent": { 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_agent", 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.232 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "use_repo_digest": { 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "use_repo_digest", 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.233 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_failed_host_check": { 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_failed_host_check", 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if the host check fails", 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.234 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_daemons": { 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_daemons", 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.235 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_on_stray_hosts": { 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_on_stray_hosts", 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.236 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crash", 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.237 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.238 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.239 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.240 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.241 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "retain_interval": { 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retain_interval", 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "31536000", 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.242 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_recent_interval": { 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_recent_interval", 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1209600", 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.243 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dashboard", 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.244 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_HOST": { 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.245 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.246 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_ENABLED": { 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_ENABLED", 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.247 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.248 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "ENABLE_BROWSABLE_API": { 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.249 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.250 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.251 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.252 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.253 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.254 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.255 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.256 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.257 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_PASSWORD": { 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.258 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.259 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_URL": { 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_URL", 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.260 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_API_USERNAME": { 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.261 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.262 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.263 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.264 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.265 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_HOST": { 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.266 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.267 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.268 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.269 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.270 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.271 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.272 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.273 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.274 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_ENABLED": { 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.275 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.276 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.277 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8", 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.278 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "45", 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.279 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ACCESS_KEY": { 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.280 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "admin", 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.281 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SECRET_KEY": { 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.282 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "RGW_API_SSL_VERIFY": { 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.283 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:58.284 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.285 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:58.286 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "cross_origin_url": { 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cross_origin_url", 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.287 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "crt_file": { 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "crt_file", 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.288 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "debug": { 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "debug", 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.289 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Enable/disable debug options", 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "jwt_token_ttl": { 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "jwt_token_ttl", 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.290 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "28800", 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.291 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.292 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.293 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.294 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.295 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.296 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "motd": { 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "motd", 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.297 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "The message of the day", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect_resolve_ip_addr": { 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "redirect_resolve_ip_addr", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.298 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.299 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8080", 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.300 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.301 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl_server_port": { 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl_server_port", 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8443", 2024-08-02T18:02:58.302 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.303 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "redirect", 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "redirect" 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.304 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "url_prefix": { 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url_prefix", 2024-08-02T18:02:58.305 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.306 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_monitoring": { 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_monitoring", 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.307 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "monitor device health metrics", 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.308 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.309 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.310 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.311 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.312 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "mark_out_threshold": { 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mark_out_threshold", 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "2419200", 2024-08-02T18:02:58.313 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "pool_name": { 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pool_name", 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.314 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "device_health_metrics", 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "retention_period": { 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "retention_period", 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.315 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15552000", 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long to retain device health metrics", 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_frequency": { 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_frequency", 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.316 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "self_heal": { 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "self_heal", 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.317 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.318 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how frequently to wake up and check device health", 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "warn_threshold": { 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "warn_threshold", 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.319 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7257600", 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "diskprediction_local", 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.320 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.321 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.322 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.323 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.324 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.325 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "predict_interval": { 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predict_interval", 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "86400", 2024-08-02T18:02:58.326 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "predictor_model": { 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "predictor_model", 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.327 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "prophetstor", 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.328 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "600", 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.329 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "influx", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": false, 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "influxdb python module not found", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "batch_size": { 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "batch_size", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5000", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.330 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "database": { 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "database", 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "ceph", 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.331 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "hostname": { 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "hostname", 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server hostname", 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.332 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "30", 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "5", 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.333 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.334 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.335 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.336 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.337 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.338 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "password": { 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "password", 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "password of InfluxDB server user", 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.339 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "port": { 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "port", 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "8086", 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "InfluxDB server port", 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.340 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "ssl": { 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ssl", 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "false", 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.341 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "threads": { 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threads", 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "32", 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.342 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "username": { 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "username", 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "username of InfluxDB server user", 2024-08-02T18:02:58.343 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "verify_ssl": { 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "verify_ssl", 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "true", 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-02T18:02:58.344 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "insights", 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.345 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.346 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.347 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.348 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.349 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.350 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "iostat", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.351 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.352 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.353 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.354 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.355 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "k8sevents", 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "ceph_event_retention_days": { 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "ceph_event_retention_days", 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "7", 2024-08-02T18:02:58.356 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "config_check_secs": { 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "config_check_secs", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "10", 2024-08-02T18:02:58.357 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.358 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.359 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.360 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.361 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.362 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "localpool", 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "failure_domain": { 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "failure_domain", 2024-08-02T18:02:58.363 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "host", 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "failure domain for any created local pool", 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.364 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.365 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.366 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.367 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.368 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "min_size": { 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "min_size", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.369 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default min_size for any created local pool", 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "num_rep": { 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "num_rep", 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3", 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.370 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default replica count for any created local pool", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_num": { 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_num", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "128", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.371 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "default pg_num for any created local pool", 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "prefix": { 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prefix", 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.372 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "name prefix for any created local pool", 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "subtree": { 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "subtree", 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "rack", 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.373 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mds_autoscaler", 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.374 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.375 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.376 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.377 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.378 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.379 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirroring", 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.380 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.381 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.382 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.383 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.384 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "nfs", 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.385 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.386 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.387 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.388 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.389 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.390 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "fail_fs": { 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "fail_fs", 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.391 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.392 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.393 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.394 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.395 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.396 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator": { 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "orchestrator", 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.397 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "cephadm", 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "rook", 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "test_orchestrator" 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Orchestrator backend", 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.398 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_perf_query", 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.399 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.400 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.401 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.402 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.403 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "osd_support", 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.404 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.405 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.406 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.407 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.408 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.409 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "pg_autoscaler", 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.410 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.411 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.412 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.413 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.414 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.415 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "threshold": { 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "threshold", 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.416 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "3.0", 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1.0", 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "scaling threshold", 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "progress", 2024-08-02T18:02:58.417 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_pg_recovery_event": { 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_pg_recovery_event", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.418 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.419 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.420 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.421 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.422 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.423 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.424 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max_completed_events": { 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_completed_events", 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "50", 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.425 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "number of past completed events to remember", 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "sleep_interval": { 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "sleep_interval", 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "5", 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.426 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "how long the module is going to sleep", 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "prometheus", 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "cache": { 2024-08-02T18:02:58.427 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "cache", 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "exclude_perf_counters": { 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "exclude_perf_counters", 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.429 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.430 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.431 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.432 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.433 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.434 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools": { 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools", 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.435 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "300", 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "scrape_interval": { 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "scrape_interval", 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15.0", 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.437 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "::", 2024-08-02T18:02:58.438 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "9283", 2024-08-02T18:02:58.439 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "stale_cache_strategy": { 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stale_cache_strategy", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "log", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.440 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_behaviour": { 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_behaviour", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "default", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "default", 2024-08-02T18:02:58.441 INFO:teuthology.orchestra.run.smithi069.stdout: "error" 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "standby_error_status_code": { 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "standby_error_status_code", 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "500", 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "400", 2024-08-02T18:02:58.442 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "599", 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.443 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.444 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.445 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.446 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.447 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_snap_create": { 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_snap_create", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10", 2024-08-02T18:02:58.448 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "mirror_snapshot_schedule": { 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "mirror_snapshot_schedule", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.449 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "trash_purge_schedule": { 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "trash_purge_schedule", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.450 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "restful", 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "enable_auth": { 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enable_auth", 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.451 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "key_file": { 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "key_file", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.452 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.453 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.454 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.455 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.456 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.457 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "server_addr": { 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_addr", 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.458 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "server_port": { 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "server_port", 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.459 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rgw", 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.460 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.461 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.462 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.463 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.464 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rook", 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.465 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.466 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.467 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.468 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.469 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.470 INFO:teuthology.orchestra.run.smithi069.stdout: "storage_class": { 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "storage_class", 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "local", 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.471 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "selftest", 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.472 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.473 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.474 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.475 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.476 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "roption1": { 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption1", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.477 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "roption2": { 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "roption2", 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "xyz", 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.478 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption1": { 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption1", 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.479 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption2": { 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption2", 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.480 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption3": { 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption3", 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "float", 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.481 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption4": { 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption4", 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption5": { 2024-08-02T18:02:58.482 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption5", 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption6": { 2024-08-02T18:02:58.483 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption6", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "rwoption7": { 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rwoption7", 2024-08-02T18:02:58.484 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "1", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "42", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "testkey": { 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testkey", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.485 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "testlkey": { 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testlkey", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.486 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "testnewline": { 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "testnewline", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.487 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.488 INFO:teuthology.orchestra.run.smithi069.stdout: "allow_m_granularity": { 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "allow_m_granularity", 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "allow minute scheduled snapshots", 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.489 INFO:teuthology.orchestra.run.smithi069.stdout: "dump_on_update": { 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "dump_on_update", 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "dump database to debug log on update", 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.490 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.491 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.492 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.493 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.494 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.495 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "stats", 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.496 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.497 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.498 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.499 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.500 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.501 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "status", 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.502 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.503 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.504 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.505 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.506 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telegraf", 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.507 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "address": { 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "address", 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.508 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "15", 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.509 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.510 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.511 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.512 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.513 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.514 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "telemetry", 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_basic": { 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_basic", 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.515 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_crash": { 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_crash", 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.516 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_device": { 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_device", 2024-08-02T18:02:58.517 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.518 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_ident": { 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_ident", 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.519 INFO:teuthology.orchestra.run.smithi069.stdout: "channel_perf": { 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "channel_perf", 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.520 INFO:teuthology.orchestra.run.smithi069.stdout: "contact": { 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "contact", 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.521 INFO:teuthology.orchestra.run.smithi069.stdout: "description": { 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "description", 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.522 INFO:teuthology.orchestra.run.smithi069.stdout: "device_url": { 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "device_url", 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.523 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enabled": { 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "enabled", 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.524 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "24", 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "8", 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.525 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "last_opt_revision": { 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "last_opt_revision", 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "1", 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.526 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard": { 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard", 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "leaderboard_description": { 2024-08-02T18:02:58.527 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "leaderboard_description", 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.528 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.529 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.530 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.531 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.532 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.533 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "organization": { 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "organization", 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.534 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "proxy": { 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "proxy", 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.535 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "url": { 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "url", 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.536 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "test_orchestrator", 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.537 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.538 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.539 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.540 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.541 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.542 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.543 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.544 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.545 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.546 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.547 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.548 INFO:teuthology.orchestra.run.smithi069.stdout: "max_concurrent_clones": { 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "max_concurrent_clones", 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "4", 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.549 INFO:teuthology.orchestra.run.smithi069.stdout: "periodic_async_work": { 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "periodic_async_work", 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Periodically check for async work", 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.550 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_delay": { 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_delay", 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "0", 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.551 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "snapshot_clone_no_wait": { 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snapshot_clone_no_wait", 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "True", 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.552 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix", 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "can_run": true, 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "error_string": "", 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "module_options": { 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "discovery_interval": { 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "discovery_interval", 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "uint", 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.553 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "100", 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "identifier": { 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "identifier", 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.554 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "interval": { 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "interval", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "secs", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.555 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "60", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "log_level": { 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_level", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.556 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.557 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster": { 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster", 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.558 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_cluster_level": { 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_cluster_level", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "info", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [ 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "critical", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "debug", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "error", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "info", 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "warning" 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.559 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "log_to_file": { 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "log_to_file", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "bool", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 1, 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "False", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.560 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_host": { 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_host", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.561 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_port": { 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_port", 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "int", 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "10051", 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.562 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "zabbix_sender": { 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "zabbix_sender", 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "str", 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "level": "advanced", 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "flags": 0, 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "min": "", 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "max": "", 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "enum_allowed": [], 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "desc": "", 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "long_desc": "", 2024-08-02T18:02:58.563 INFO:teuthology.orchestra.run.smithi069.stdout: "tags": [], 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "see_also": [] 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "services": { 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "dashboard": "https://172.21.15.69:8443/" 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "always_on_modules": { 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "octopus": [ 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-08-02T18:02:58.564 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "pacific": [ 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-08-02T18:02:58.565 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "quincy": [ 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-08-02T18:02:58.566 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "reef": [ 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: ], 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "squid": [ 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "balancer", 2024-08-02T18:02:58.567 INFO:teuthology.orchestra.run.smithi069.stdout: "crash", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "devicehealth", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "orchestrator", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "pg_autoscaler", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "progress", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "rbd_support", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "status", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "telemetry", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "volumes" 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "last_failure_osd_epoch": 95, 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "active_clients": [ 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "devicehealth", 2024-08-02T18:02:58.568 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1160759356 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "libcephsqlite", 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 683860089 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.569 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "rbd_support", 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1338612681 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-08-02T18:02:58.570 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 561147662 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "snap_schedule", 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 2712380350 2024-08-02T18:02:58.571 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: "name": "volumes", 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: "addrvec": [ 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: { 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: "type": "v2", 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: "addr": "172.21.15.69:0", 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: "nonce": 1096929466 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout: ] 2024-08-02T18:02:58.572 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:02:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25200a2d00 msgr2=0x7f25200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25200a2d00 0x7f25200a51c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f252800a7d0 tx=0x7f25280305e0 comp rx=0 tx=0).stop 2024-08-02T18:02:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100ee0 msgr2=0x7f25381020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:02:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100ee0 0x7f25381020a0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f252c006fd0 tx=0x7f252c00b8e0 comp rx=0 tx=0).stop 2024-08-02T18:02:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 shutdown_connections 2024-08-02T18:02:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f2538100ee0 0x7f25381020a0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f25200a2d00 0x7f25200a51c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:58.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2538105220 0x7f25381025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 --2- 172.21.15.69:0/2117934104 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f2538100530 0x7f2538101b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:02:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 >> 172.21.15.69:0/2117934104 conn(0x7f2538075bb0 msgr2=0x7f25380febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:02:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 shutdown_connections 2024-08-02T18:02:58.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:02:57.260+0000 7f253dc56640 1 -- 172.21.15.69:0/2117934104 wait complete. 2024-08-02T18:02:58.575 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:56 smithi077 ceph-mon[37108]: pgmap v518: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-08-02T18:02:58.575 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:56 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2090021068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.575 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:56 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2612910965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.575 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:56 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1469646746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.575 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:56 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3617487629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.575 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:57 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2117934104' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:56 smithi083 ceph-mon[37031]: pgmap v518: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:56 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2090021068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:56 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2612910965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:56 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1469646746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:56 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3617487629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:57 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2117934104' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:56 smithi069 ceph-mon[32746]: pgmap v518: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:56 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2090021068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:56 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2612910965' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:56 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1469646746' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.576 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:56 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3617487629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-08-02T18:02:58.632 DEBUG:tasks.check_counter:Getting stats from a 2024-08-02T18:02:58.632 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-02T18:02:58.632 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:57 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2117934104' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-08-02T18:02:58.897 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:02:59.175 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:02:58 smithi069 ceph-mon[32746]: pgmap v519: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-08-02T18:02:59.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:02:58 smithi083 ceph-mon[37031]: pgmap v519: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-08-02T18:02:59.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:02:58 smithi077 ceph-mon[37108]: pgmap v519: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 35 KiB/s wr, 18 op/s 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 4, 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 4, 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3024, 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 448, 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.003146252, 2024-08-02T18:02:59.426 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000038794, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002190508, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.003951947, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001317315 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-02T18:02:59.427 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 448 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 316, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 306, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1470368, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 188720, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.051943551, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.021041697, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.041748283, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:02:59.428 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 305, 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.017154524, 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000056244 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1470368, 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 188720 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:02:59.429 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 235, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 480, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 276512, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.016732295, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.011784260, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002136298, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 232, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.015243076, 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000065702 2024-08-02T18:02:59.430 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 480, 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 276512 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-08-02T18:02:59.431 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:02:59.432 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-08-02T18:02:59.433 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148754, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 31, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-08-02T18:02:59.434 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-08-02T18:02:59.435 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 319, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1425807, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 319, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-08-02T18:02:59.436 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 319, 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1425807, 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:02:59.437 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:03:00.294 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-08-02T18:03:00.294 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-08-02T18:03:00.294 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-08-02T18:03:00.294 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-08-02T18:03:00.294 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-08-02T18:03:00.294 DEBUG:tasks.check_counter:Getting stats from d 2024-08-02T18:03:00.295 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-02T18:03:00.552 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:00.968 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:00 smithi069 ceph-mon[32746]: pgmap v520: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 23 op/s 2024-08-02T18:03:00.969 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:00 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:03:01.077 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:03:01.077 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:01.077 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 322, 2024-08-02T18:03:01.077 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 312, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1472000, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 192560, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.050755055, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.020795736, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.041044964, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:01.078 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 311, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.022464996, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000072234 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1472000, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 192560 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 240, 2024-08-02T18:03:01.079 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 480, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 277392, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.017027550, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.012165697, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002114881, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 237, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.020253881, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000085459 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.080 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 480, 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 277392 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 4, 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 4, 2024-08-02T18:03:01.081 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3024, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 448, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.002365525, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000054943, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.002286648, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 3, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.004043111, 2024-08-02T18:03:01.082 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.001347703 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 448 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.083 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:01.084 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:01.085 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:01.086 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148794, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 32, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:01.087 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-08-02T18:03:01.088 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 325, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1426587, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 325, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 325, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1426587, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:01.089 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:01.090 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:01.090 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:03:01.090 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:03:01.090 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:03:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:00 smithi083 ceph-mon[37031]: pgmap v520: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 23 op/s 2024-08-02T18:03:01.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:00 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:03:01.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:00 smithi077 ceph-mon[37108]: pgmap v520: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 23 op/s 2024-08-02T18:03:01.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:00 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-08-02T18:03:01.912 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-02T18:03:01.912 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-02T18:03:01.912 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-08-02T18:03:01.912 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-08-02T18:03:01.912 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-08-02T18:03:01.912 DEBUG:tasks.check_counter:Getting stats from g 2024-08-02T18:03:01.912 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-02T18:03:02.165 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:01 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:03:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:01 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:03:02.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:01 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:03:02.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:01 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:03:02.287 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:01 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:03:02.288 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:01 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:03:02.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:01 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-08-02T18:03:02.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:01 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-08-02T18:03:02.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:01 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' 2024-08-02T18:03:02.671 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:03:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-08-02T18:03:02.671 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-08-02T18:03:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-08-02T18:03:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-08-02T18:03:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-08-02T18:03:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-08-02T18:03:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.001134684, 2024-08-02T18:03:02.672 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-08-02T18:03:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000196539, 2024-08-02T18:03:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:02.673 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:02.674 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-02T18:03:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-08-02T18:03:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:02.675 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 318, 2024-08-02T18:03:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 308, 2024-08-02T18:03:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1470912, 2024-08-02T18:03:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 190000, 2024-08-02T18:03:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-08-02T18:03:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-08-02T18:03:02.676 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.053843664, 2024-08-02T18:03:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.021206531, 2024-08-02T18:03:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.044568320, 2024-08-02T18:03:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 307, 2024-08-02T18:03:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.022382672, 2024-08-02T18:03:02.677 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000072907 2024-08-02T18:03:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:02.678 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1470912, 2024-08-02T18:03:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 190000 2024-08-02T18:03:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:02.679 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 6, 2024-08-02T18:03:02.680 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 238, 2024-08-02T18:03:02.680 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 3344, 2024-08-02T18:03:02.680 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 198576, 2024-08-02T18:03:02.680 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 5, 2024-08-02T18:03:02.680 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-08-02T18:03:02.680 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.018665319, 2024-08-02T18:03:02.681 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.012408983, 2024-08-02T18:03:02.681 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.003869675, 2024-08-02T18:03:02.681 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:02.681 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:02.681 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 236, 2024-08-02T18:03:02.681 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.023917296, 2024-08-02T18:03:02.681 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000101344 2024-08-02T18:03:02.682 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.682 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:02.682 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:02.682 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:02.682 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:02.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.683 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-08-02T18:03:02.683 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 198576 2024-08-02T18:03:02.683 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.683 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-08-02T18:03:02.683 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-08-02T18:03:02.683 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-08-02T18:03:02.684 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.684 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-08-02T18:03:02.684 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:02.684 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:02.684 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:02.684 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:02.684 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:02.685 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:02.685 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:02.685 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:02.685 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:02.685 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:02.685 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:02.686 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:02.686 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:02.686 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:02.686 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:02.686 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:02.686 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:02.687 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:02.687 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:02.687 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:02.687 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:02.687 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:02.687 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:02.688 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:02.688 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:02.688 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:02.688 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:02.688 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:02.689 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:02.689 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:02.689 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:02.689 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-08-02T18:03:02.689 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:02.689 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:02.689 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:02.690 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:02.690 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 148955, 2024-08-02T18:03:02.690 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 30, 2024-08-02T18:03:02.690 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:02.690 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:02.691 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-08-02T18:03:02.691 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-08-02T18:03:02.691 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:02.691 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:02.691 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:02.691 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:02.692 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-08-02T18:03:02.692 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-08-02T18:03:02.692 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:02.692 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:02.692 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:02.692 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-08-02T18:03:02.693 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-08-02T18:03:02.693 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-08-02T18:03:02.693 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:02.693 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-08-02T18:03:02.693 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:02.694 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-08-02T18:03:02.694 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:02.694 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:02.694 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-08-02T18:03:02.694 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-08-02T18:03:02.695 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-08-02T18:03:02.695 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 320, 2024-08-02T18:03:02.695 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1426040, 2024-08-02T18:03:02.695 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:02.695 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 320, 2024-08-02T18:03:02.695 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-08-02T18:03:02.696 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-08-02T18:03:02.696 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 320, 2024-08-02T18:03:02.696 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1426040, 2024-08-02T18:03:02.696 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-08-02T18:03:02.696 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:02.696 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:02.697 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:02.697 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:03:02.697 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:03:02.697 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:03:03.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:02 smithi083 ceph-mon[37031]: pgmap v521: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-08-02T18:03:03.326 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:02 smithi069 ceph-mon[32746]: pgmap v521: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-08-02T18:03:03.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:02 smithi077 ceph-mon[37108]: pgmap v521: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-08-02T18:03:03.362 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-08-02T18:03:03.362 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-08-02T18:03:03.362 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-08-02T18:03:03.362 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-08-02T18:03:03.362 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-08-02T18:03:03.362 DEBUG:tasks.check_counter:Getting stats from j 2024-08-02T18:03:03.362 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-02T18:03:03.615 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:04.152 INFO:teuthology.orchestra.run.smithi069.stdout:{ 2024-08-02T18:03:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 327, 2024-08-02T18:03:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 315, 2024-08-02T18:03:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 1475296, 2024-08-02T18:03:04.152 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 193648, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 4, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 0, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.058620594, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.021399098, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.047502728, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 315, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.027692097, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000087911 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:04.153 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 1475296, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 193648 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 3, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 241, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 480, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 277568, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 6, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 3, 2024-08-02T18:03:04.154 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.020369744, 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.013658257, 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.003319766, 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 238, 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.022909099, 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000096256 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:04.155 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 480, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 277568 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_messages": 0, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages": 2, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_bytes": 0, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_bytes": 0, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_created_connections": 2, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_active_connections": 1, 2024-08-02T18:03:04.156 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_total_time": 0.000475872, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_send_time": 0.000000000, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_recv_time": 0.000297412, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:04.157 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "msgr_send_encrypted_bytes": 0 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "cct": { 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "total_workers": 0, 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "unhealthy_workers": 0 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "mempool": { 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:04.158 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:04.159 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:04.160 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_items": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_bytes": 149785, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_anon_items": 32, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_bytes": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_items": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:04.161 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_bytes": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_items": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "pgmap_items": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_bytes": 928, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "mds_co_items": 2, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_1_items": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:04.162 INFO:teuthology.orchestra.run.smithi069.stdout: "unittest_2_items": 0 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: }, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "val": 0, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "max": 104857600, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "get_started": 0, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "get": 326, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "get_sum": 1426717, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "get_or_fail_success": 326, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "take": 0, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "take_sum": 0, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "put": 326, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "put_sum": 1426717, 2024-08-02T18:03:04.163 INFO:teuthology.orchestra.run.smithi069.stdout: "wait": { 2024-08-02T18:03:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "avgcount": 0, 2024-08-02T18:03:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "sum": 0.000000000, 2024-08-02T18:03:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: "avgtime": 0.000000000 2024-08-02T18:03:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:03:04.164 INFO:teuthology.orchestra.run.smithi069.stdout: } 2024-08-02T18:03:04.164 INFO:teuthology.orchestra.run.smithi069.stdout:} 2024-08-02T18:03:04.861 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-02T18:03:04.861 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-02T18:03:04.861 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-08-02T18:03:04.861 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-08-02T18:03:04.861 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-08-02T18:03:04.861 DEBUG:tasks.check_counter:Getting stats from b 2024-08-02T18:03:04.862 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-02T18:03:05.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:04 smithi069 ceph-mon[32746]: pgmap v522: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 37 op/s 2024-08-02T18:03:05.241 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T18:03:05.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:04 smithi083 ceph-mon[37031]: pgmap v522: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 37 op/s 2024-08-02T18:03:05.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:04 smithi077 ceph-mon[37108]: pgmap v522: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 37 op/s 2024-08-02T18:03:05.827 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-08-02T18:03:05.827 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:05.827 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 15550, 2024-08-02T18:03:05.827 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 14578, 2024-08-02T18:03:05.827 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 7634482, 2024-08-02T18:03:05.827 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 23532794, 2024-08-02T18:03:05.827 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 19, 2024-08-02T18:03:05.827 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.948510044, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.307754404, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.279932881, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.287551424, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 14579, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.196871423, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000082095 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3387, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000607465, 2024-08-02T18:03:05.828 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000179 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 562384, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 172528 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 20845, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 26431, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 10222452, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 32893965, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 30, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 6, 2024-08-02T18:03:05.829 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 1.319879540, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.536699101, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.915951007, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.184312511, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 26430, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 2.572201537, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000097321 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 23698, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.003104120, 2024-08-02T18:03:05.830 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000130 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 5849216, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 10219312 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 16452, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 16978, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 8463677, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 25106870, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 383, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 9, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 1.043396172, 2024-08-02T18:03:05.831 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.336358122, 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.379428128, 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.254653625, 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 16972, 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 1.059069012, 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000062400 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 3661, 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000731030, 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000199 2024-08-02T18:03:05.832 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.833 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 85184, 2024-08-02T18:03:05.833 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 140688 2024-08-02T18:03:05.833 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.833 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 28558, 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 8.868303836, 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000310536 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.834 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 822, 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.037419629, 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000045522 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 4519, 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 4519, 2024-08-02T18:03:05.835 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4519, 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 72.151096698, 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.015966164 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-08-02T18:03:05.836 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 1510, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 7, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 1779, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 492, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 1287, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 1064, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 67, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 67, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 4, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 7296, 2024-08-02T18:03:05.837 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 6130, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 283, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-08-02T18:03:05.838 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 26291, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 1018, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 1537029723, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 6, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 26291, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 907, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 26, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 881, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 25384, 2024-08-02T18:03:05.839 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 3, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 3928, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 4405, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 6717, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 2432, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 157, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 1064 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-08-02T18:03:05.840 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 4, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 4, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 398, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 242, 2024-08-02T18:03:05.841 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 155, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 14, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 14701, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 26, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-08-02T18:03:05.842 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 7, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 8562, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 8515, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 8515, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 47, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-08-02T18:03:05.843 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 59, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 86, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 84, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 84, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 2, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 1, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 1, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 50258952, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 50495767, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4309, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 74.030162040, 2024-08-02T18:03:05.844 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.017180357 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 1723, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 1988, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 265, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 97, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 105, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 8, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 1779, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 2603, 2024-08-02T18:03:05.845 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 824, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 67, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 2774, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 2707, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 78884, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 231712 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 5353, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 2, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 4519, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 367, 2024-08-02T18:03:05.846 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 4, 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1272, 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 58.356810868, 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.045877995 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 329, 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 4.099064580, 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.012459162 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.847 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 155, 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.324311520, 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002092332 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1193, 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 4.002721407, 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.003355173 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-08-02T18:03:05.848 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000131286, 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000131286 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.849 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.850 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 17, 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002473084, 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000145475 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 37, 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.265748961, 2024-08-02T18:03:05.851 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.007182404 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 17, 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.119794109, 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.007046712 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 774, 2024-08-02T18:03:05.852 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.538965441, 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000696337 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 135, 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.155338292, 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001150654 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 377, 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 3.549660565, 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.009415545 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-08-02T18:03:05.853 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.009404033, 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.004702016 2024-08-02T18:03:05.854 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 11, 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.408567516, 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.037142501 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-08-02T18:03:05.855 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.002231089, 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001115544 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.856 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 9, 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.021318736, 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.002368748 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.857 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 188, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.294555211, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.001566783 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.858 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 7, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 5, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 488, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 244, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 981, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-08-02T18:03:05.859 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:05.860 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:05.861 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 849640, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 1351, 2024-08-02T18:03:05.862 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 88, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 1, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9768, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 197, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-08-02T18:03:05.863 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 5592513, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 87911, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 36795, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 53958106, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 37, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 36795, 2024-08-02T18:03:05.864 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 36795, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 588.782215633, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.016001690 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 36795, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 44122 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 36795, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 26442, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 10353, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-08-02T18:03:05.865 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 1406, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 2587, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 125, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 4408, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 378, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 3, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 5, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 1035, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-08-02T18:03:05.866 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 26317, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 3973, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-08-02T18:03:05.867 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-08-02T18:03:05.868 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 115, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 31, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 12, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 12, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 3613, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 0, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 272 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-08-02T18:03:05.869 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 12599 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 24, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 8, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 503, 2024-08-02T18:03:05.870 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 242, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 52843, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 20901515, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 52843, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 52843, 2024-08-02T18:03:05.871 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 20901515, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:05.872 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 36795, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 53985740, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 36795, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 53985740, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:05.873 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 36795, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 36795, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 36795, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 36795, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.874 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 242, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 27634, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 242, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:05.875 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 99, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 27634, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x5560c38ae500": { 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 60, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:05.876 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 8562, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 46301463, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 8562, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 4309, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 46301403, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:05.877 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-08-02T18:03:06.688 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-08-02T18:03:06.688 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-08-02T18:03:06.689 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-08-02T18:03:06.689 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=17 2024-08-02T18:03:06.689 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=11 2024-08-02T18:03:06.689 DEBUG:tasks.check_counter:Getting stats from e 2024-08-02T18:03:06.689 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-02T18:03:06.941 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T18:03:07.197 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:06 smithi077 ceph-mon[37108]: pgmap v523: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 58 KiB/s wr, 28 op/s 2024-08-02T18:03:07.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:06 smithi083 ceph-mon[37031]: pgmap v523: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 58 KiB/s wr, 28 op/s 2024-08-02T18:03:07.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:06 smithi069 ceph-mon[32746]: pgmap v523: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 58 KiB/s wr, 28 op/s 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 30, 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 27, 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 150640, 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 14224, 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 0, 2024-08-02T18:03:07.441 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.007983564, 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.001202660, 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.007189427, 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 27, 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.006746007, 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000249852 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.442 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 150640, 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 14224 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1, 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 18, 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 160, 2024-08-02T18:03:07.443 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 81376, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 3, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.011300126, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000784170, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.010245159, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 16, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000994559, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000062159 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:07.444 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 81376 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 0, 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 2, 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 0, 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 0, 2024-08-02T18:03:07.445 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 2, 2024-08-02T18:03:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 1, 2024-08-02T18:03:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.012208378, 2024-08-02T18:03:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.000000000, 2024-08-02T18:03:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.012101790, 2024-08-02T18:03:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:07.446 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:07.447 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 0, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:07.448 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:07.449 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:07.450 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:07.451 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-08-02T18:03:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 148119, 2024-08-02T18:03:07.452 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 27, 2024-08-02T18:03:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-08-02T18:03:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-08-02T18:03:07.453 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 0, 2024-08-02T18:03:07.454 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 0, 2024-08-02T18:03:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-08-02T18:03:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 928, 2024-08-02T18:03:07.455 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 2, 2024-08-02T18:03:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-08-02T18:03:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-08-02T18:03:07.456 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-08-02T18:03:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:07.457 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 27, 2024-08-02T18:03:07.458 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 144012, 2024-08-02T18:03:07.458 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:07.458 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 27, 2024-08-02T18:03:07.458 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:07.458 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:07.458 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 27, 2024-08-02T18:03:07.459 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 144012, 2024-08-02T18:03:07.459 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:07.459 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:07.459 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:07.459 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:07.460 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:07.460 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:07.460 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-08-02T18:03:08.166 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-08-02T18:03:08.166 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-08-02T18:03:08.166 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-08-02T18:03:08.166 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-08-02T18:03:08.167 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-08-02T18:03:08.167 DEBUG:tasks.check_counter:Getting stats from h 2024-08-02T18:03:08.167 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-02T18:03:08.423 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T18:03:08.950 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-08-02T18:03:08.950 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:08.950 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 1013, 2024-08-02T18:03:08.950 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 1044, 2024-08-02T18:03:08.950 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 262867, 2024-08-02T18:03:08.950 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 704178, 2024-08-02T18:03:08.950 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 23, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.114227983, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.046421882, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.063001250, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.005249844, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1042, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.073118170, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000070170 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1957, 2024-08-02T18:03:08.951 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000394734, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000201 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 4176, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 19792 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 407, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 591, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1538509, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 768846, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 14, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 4, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.067199647, 2024-08-02T18:03:08.952 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.026917212, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.052609463, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000284951, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 590, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.049349413, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000083643 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 12, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000001981, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000165 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1535424, 2024-08-02T18:03:08.953 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 739456 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 75, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 165, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 15650, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 664612, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 54, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.036744573, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.006829311, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.027241752, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000214535, 2024-08-02T18:03:08.954 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 163, 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.020625499, 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000126536 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 131, 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000020254, 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000154 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 7360, 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 143088 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-08-02T18:03:08.955 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 47, 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.001358195, 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000028897 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-08-02T18:03:08.956 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 2, 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000201244, 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000100622 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 9, 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 5, 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 5, 2024-08-02T18:03:08.957 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000810453, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000162090 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 4, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 11, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 11, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 0, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 11, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-08-02T18:03:08.958 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 11, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 1, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 2, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 2, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 41, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 38, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 1, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-08-02T18:03:08.959 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 1, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 1018, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 1537029723, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 6, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-08-02T18:03:08.960 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 2, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 2, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 1 2024-08-02T18:03:08.961 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 2, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-08-02T18:03:08.962 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 14, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 28, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.963 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 8, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 8, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 2, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 2, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 2, 2024-08-02T18:03:08.964 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 4194304, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 4196652, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 0, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 7, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.054596657, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.007799522 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 0 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 14, 2024-08-02T18:03:08.965 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 14, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 0, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 12, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 13, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 1, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 11, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 11, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 0, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 2, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 3, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 1, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 51032, 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 223520 2024-08-02T18:03:08.966 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 10, 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 9, 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 32, 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.967 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 4, 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000722381, 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000180595 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.968 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.969 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 1, 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000088072, 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000088072 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.970 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.971 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-08-02T18:03:08.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.972 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.972 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-08-02T18:03:08.972 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.972 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-08-02T18:03:08.973 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.974 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.975 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.976 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.977 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.978 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 4, 2024-08-02T18:03:08.979 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 2, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 2, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 4, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 1, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 973, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:08.980 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:08.981 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:08.982 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 255303, 2024-08-02T18:03:08.983 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 87, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9768, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 197, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:08.984 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 55576, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 215, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-08-02T18:03:08.985 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 44, 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 12740, 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 33, 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 44, 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 44, 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.281993835, 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.006408950 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.986 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 44, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 66 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 44, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 0, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 44, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 0, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 10, 2024-08-02T18:03:08.987 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 7, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 8, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 6, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 20, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-08-02T18:03:08.988 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-08-02T18:03:08.989 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 115, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-08-02T18:03:08.990 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 25, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 9, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 15, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 6, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 14, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 0, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 1 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 1, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-08-02T18:03:08.991 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 36 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.992 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 1491, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 1658220, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 1491, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 1491, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 1658220, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.993 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:08.994 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 44, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 12740, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 44, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 12740, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:08.995 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 44, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 44, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 44, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 44, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.996 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-08-02T18:03:08.997 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x560098fd4500": { 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 8, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 2348, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 8, 2024-08-02T18:03:08.998 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 7, 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 2348, 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:08.999 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-08-02T18:03:09.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:08 smithi083 ceph-mon[37031]: pgmap v524: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-08-02T18:03:09.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:08 smithi077 ceph-mon[37108]: pgmap v524: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-08-02T18:03:09.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:08 smithi069 ceph-mon[32746]: pgmap v524: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-08-02T18:03:09.730 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-08-02T18:03:09.730 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-08-02T18:03:09.730 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-08-02T18:03:09.730 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-08-02T18:03:09.730 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-08-02T18:03:09.730 DEBUG:tasks.check_counter:Getting stats from k 2024-08-02T18:03:09.731 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-02T18:03:10.036 INFO:teuthology.orchestra.run.smithi077.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.b/config 2024-08-02T18:03:10.635 INFO:teuthology.orchestra.run.smithi077.stdout:{ 2024-08-02T18:03:10.635 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:10.635 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 77, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 79, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 29469, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 24496, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 6, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 2, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.015549525, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.003033563, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.008661194, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.002313571, 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:10.636 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 77, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.024122945, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000313285 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 0 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 181, 2024-08-02T18:03:10.637 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 420, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 4093089, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 367925, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 8, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 5, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.027045833, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.012570097, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.007249217, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.003911312, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 417, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.023238029, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000055726 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.638 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 480, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 311264 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_messages": 342, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages": 330, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_bytes": 1495829, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_bytes": 198052, 2024-08-02T18:03:10.639 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_created_connections": 4, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_active_connections": 3, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_total_time": 0.054078204, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_send_time": 0.019253285, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_recv_time": 0.048779686, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_running_fast_dispatch_time": 0.000112002, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 330, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.021217025, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000064294 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.640 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_recv_encrypted_bytes": 1493936, 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "msgr_send_encrypted_bytes": 197088 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "cct": { 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "total_workers": 1, 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "unhealthy_workers": 0 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-MDSRank": { 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 427, 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.013422656, 2024-08-02T18:03:10.641 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000031434 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: "finisher-PurgeQueue": { 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: "queue_len": 0, 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: "complete_latency": { 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.642 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.643 INFO:teuthology.orchestra.run.smithi077.stdout: "mds": { 2024-08-02T18:03:10.646 INFO:teuthology.orchestra.run.smithi077.stdout: "request": 0, 2024-08-02T18:03:10.646 INFO:teuthology.orchestra.run.smithi077.stdout: "reply": 0, 2024-08-02T18:03:10.646 INFO:teuthology.orchestra.run.smithi077.stdout: "reply_latency": { 2024-08-02T18:03:10.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.646 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.646 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.646 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.646 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_reply": 0, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "forward": 0, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_complete": 0, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_fetch_keys": 0, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_commit": 0, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_split": 0, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_merge": 0, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes": 269, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_top": 170, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_bottom": 99, 2024-08-02T18:03:10.647 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pin_tail": 0, 2024-08-02T18:03:10.648 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_pinned": 33, 2024-08-02T18:03:10.648 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_expired": 0, 2024-08-02T18:03:10.648 INFO:teuthology.orchestra.run.smithi077.stdout: "inodes_with_caps": 0, 2024-08-02T18:03:10.648 INFO:teuthology.orchestra.run.smithi077.stdout: "caps": 0, 2024-08-02T18:03:10.648 INFO:teuthology.orchestra.run.smithi077.stdout: "subtrees": 2, 2024-08-02T18:03:10.648 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse": 0, 2024-08-02T18:03:10.648 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_hit": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_forward": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_discover": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_dir_fetch": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_remote_ino": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "traverse_lock": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "load_cent": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "q": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "exported": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "exported_inodes": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "imported": 0, 2024-08-02T18:03:10.649 INFO:teuthology.orchestra.run.smithi077.stdout: "imported_inodes": 0, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_dir_fetch": 0, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_backtrace_fetch": 0, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "openino_peer_discover": 0, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rfiles": 1018, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rbytes": 1537029723, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "root_rsnaps": 6, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_fetch": 0, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_set_tag": 0, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_backtrace_repaired": 0, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_inotable_repaired": 0, 2024-08-02T18:03:10.650 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_inodes": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dir_base_inodes": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_dirfrag_rstats": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "scrub_file_inodes": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_inode_file_caps": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_revoke": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_grant": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_trunc": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-02T18:03:10.651 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps": 0, 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_caps_dirty": 0, 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_cap_release": 0, 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "process_request_cap_release": 0 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_cache": { 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update": 0, 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_update_receipt": 0, 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_try_discover": 0, 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_send_discover": 0, 2024-08-02T18:03:10.652 INFO:teuthology.orchestra.run.smithi077.stdout: "dir_handle_discover": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_delayed": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "num_strays_enqueuing": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_created": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_enqueued": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_reintegrated": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "strays_migrated": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_processing": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_enqueued": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "num_recovering_prioritized": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_started": 0, 2024-08-02T18:03:10.653 INFO:teuthology.orchestra.run.smithi077.stdout: "recovery_completed": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_path": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_quiesce_inode": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_enqueue_scrub": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_exportdir": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_flush": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragmentdir": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_fragstats": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "ireq_inodestats": 0 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_log": { 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "evlrg": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "evadd": 0, 2024-08-02T18:03:10.654 INFO:teuthology.orchestra.run.smithi077.stdout: "evex": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "evtrm": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "ev": 540, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "evexg": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "evexd": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "segadd": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "segex": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "segtrm": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "seg": 2, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "segmjr": 5, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "segexg": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "segexd": 0, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "expos": 50258952, 2024-08-02T18:03:10.655 INFO:teuthology.orchestra.run.smithi077.stdout: "wrpos": 50534218, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "rdpos": 50512588, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "jlat": { 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "replayed": 540 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_mem": { 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "ino": 155, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "ino+": 173, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "ino-": 18, 2024-08-02T18:03:10.656 INFO:teuthology.orchestra.run.smithi077.stdout: "dir": 46, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "dir+": 46, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "dir-": 0, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "dn": 269, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "dn+": 269, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "dn-": 0, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "cap": 0, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "cap+": 0, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "cap-": 0, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "rss": 51184, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "heap": 215328 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.657 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_server": { 2024-08-02T18:03:10.658 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_client_request": 0, 2024-08-02T18:03:10.658 INFO:teuthology.orchestra.run.smithi077.stdout: "dispatch_server_request": 0, 2024-08-02T18:03:10.658 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_request": 0, 2024-08-02T18:03:10.658 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_client_session": 0, 2024-08-02T18:03:10.658 INFO:teuthology.orchestra.run.smithi077.stdout: "handle_peer_request": 0, 2024-08-02T18:03:10.658 INFO:teuthology.orchestra.run.smithi077.stdout: "req_create_latency": { 2024-08-02T18:03:10.658 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.658 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getattr_latency": { 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getfilelock_latency": { 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.659 INFO:teuthology.orchestra.run.smithi077.stdout: "req_link_latency": { 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookup_latency": { 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookuphash_latency": { 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.660 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupino_latency": { 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupname_latency": { 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupparent_latency": { 2024-08-02T18:03:10.661 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lookupsnap_latency": { 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "req_lssnap_latency": { 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.662 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.663 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.663 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.663 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mkdir_latency": { 2024-08-02T18:03:10.663 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.663 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mknod_latency": { 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "req_mksnap_latency": { 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.664 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: "req_open_latency": { 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: "req_readdir_latency": { 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.665 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rename_latency": { 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "req_renamesnap_latency": { 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "req_snapdiff_latency": { 2024-08-02T18:03:10.666 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmdir_latency": { 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmsnap_latency": { 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.667 INFO:teuthology.orchestra.run.smithi077.stdout: "req_rmxattr_latency": { 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setattr_latency": { 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setdirlayout_latency": { 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.668 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setfilelock_latency": { 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setlayout_latency": { 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.669 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: "req_setxattr_latency": { 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: "req_symlink_latency": { 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.670 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.671 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.671 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.671 INFO:teuthology.orchestra.run.smithi077.stdout: "req_unlink_latency": { 2024-08-02T18:03:10.671 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.671 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.671 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.671 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.672 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_revoke_eviction": 0, 2024-08-02T18:03:10.672 INFO:teuthology.orchestra.run.smithi077.stdout: "cap_acquisition_throttle": 0, 2024-08-02T18:03:10.672 INFO:teuthology.orchestra.run.smithi077.stdout: "req_getvxattr_latency": { 2024-08-02T18:03:10.672 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.672 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.672 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.672 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_sessions": { 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "session_count": 2, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "session_add": 2, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "session_remove": 0, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_open": 0, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "sessions_stale": 0, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "total_load": 0, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "average_load": 0, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "avg_session_uptime": 2235, 2024-08-02T18:03:10.673 INFO:teuthology.orchestra.run.smithi077.stdout: "mdthresh_evicted": 0 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "mempool": { 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:10.674 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:10.675 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:10.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:10.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:10.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:10.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:10.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:10.676 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:10.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:10.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:10.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:10.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:10.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:10.677 INFO:teuthology.orchestra.run.smithi077.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:10.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:10.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_items": 0, 2024-08-02T18:03:10.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:10.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:10.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:10.678 INFO:teuthology.orchestra.run.smithi077.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:10.678 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_bytes": 207609, 2024-08-02T18:03:10.679 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_anon_items": 336, 2024-08-02T18:03:10.679 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:10.679 INFO:teuthology.orchestra.run.smithi077.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:10.679 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_bytes": 0, 2024-08-02T18:03:10.679 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_items": 0, 2024-08-02T18:03:10.679 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:10.679 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:10.679 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:10.680 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:10.680 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_bytes": 9768, 2024-08-02T18:03:10.680 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_items": 197, 2024-08-02T18:03:10.680 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:10.680 INFO:teuthology.orchestra.run.smithi077.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:10.680 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:10.680 INFO:teuthology.orchestra.run.smithi077.stdout: "pgmap_items": 0, 2024-08-02T18:03:10.681 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_bytes": 770902, 2024-08-02T18:03:10.681 INFO:teuthology.orchestra.run.smithi077.stdout: "mds_co_items": 20229, 2024-08-02T18:03:10.681 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:10.681 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_1_items": 0, 2024-08-02T18:03:10.681 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:10.681 INFO:teuthology.orchestra.run.smithi077.stdout: "unittest_2_items": 0 2024-08-02T18:03:10.681 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.682 INFO:teuthology.orchestra.run.smithi077.stdout: "objecter": { 2024-08-02T18:03:10.682 INFO:teuthology.orchestra.run.smithi077.stdout: "op_active": 0, 2024-08-02T18:03:10.682 INFO:teuthology.orchestra.run.smithi077.stdout: "op_laggy": 0, 2024-08-02T18:03:10.682 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send": 258, 2024-08-02T18:03:10.682 INFO:teuthology.orchestra.run.smithi077.stdout: "op_send_bytes": 26, 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: "op_resend": 4, 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: "op_reply": 258, 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: "op_latency": { 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 258, 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 6.164870417, 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.023894846 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: "op_inflight": 0, 2024-08-02T18:03:10.683 INFO:teuthology.orchestra.run.smithi077.stdout: "oplen_avg": { 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 258, 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 259 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "op": 258, 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "op_r": 258, 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "op_w": 0, 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "op_rmw": 0, 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "op_pg": 0, 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_stat": 146, 2024-08-02T18:03:10.684 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_create": 0, 2024-08-02T18:03:10.685 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_read": 109, 2024-08-02T18:03:10.685 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_write": 0, 2024-08-02T18:03:10.685 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writefull": 0, 2024-08-02T18:03:10.685 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_writesame": 0, 2024-08-02T18:03:10.685 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_append": 0, 2024-08-02T18:03:10.685 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_zero": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_truncate": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_delete": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_mapext": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_sparse_read": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_clonerange": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_getxattr": 2, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_setxattr": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_cmpxattr": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_rmxattr": 0, 2024-08-02T18:03:10.686 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_resetxattrs": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_call": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_watch": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_notify": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_src_cmpxattr": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_pgls_filter": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "osdop_other": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_active": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_send": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_resend": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "linger_ping": 0, 2024-08-02T18:03:10.687 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_active": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_send": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "poolop_resend": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_active": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_send": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "poolstat_resend": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_active": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_send": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "statfs_resend": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "command_active": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "command_send": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "command_resend": 0, 2024-08-02T18:03:10.688 INFO:teuthology.orchestra.run.smithi077.stdout: "map_epoch": 116, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "map_full": 0, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "map_inc": 3, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_sessions": 6, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_open": 6, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_session_close": 0, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "osd_laggy": 0, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_wr": 0, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_rd": 2, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_del": 0 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "oft": { 2024-08-02T18:03:10.689 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_objs": 0, 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_kv_pairs": 0, 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_updates": 0, 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "omap_total_removes": 0 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "osd-slow-ops": { 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "slow_ops_count": 0 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "purge_queue": { 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops": 0, 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_ops_high_water": 0, 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing": 0, 2024-08-02T18:03:10.690 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executing_high_water": 0, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed_ops": 0, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_executed": 0, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "pq_item_in_journal": 0 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 596, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 5544459, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 596, 2024-08-02T18:03:10.691 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 596, 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 5544459, 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_bytes": { 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:10.692 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 104857600, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 258, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 4046579, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 258, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 4046579, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.693 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-objecter_ops": { 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 1024, 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:10.694 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 258, 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 258, 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 258, 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 258, 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle": { 2024-08-02T18:03:10.695 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:10.696 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: }, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "throttle-write_buf_throttle-0x561c62dac6e0": { 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "val": 0, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "max": 3758096384, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "get_started": 0, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "get": 0, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "get_sum": 0, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:10.697 INFO:teuthology.orchestra.run.smithi077.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: "take": 0, 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: "take_sum": 0, 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: "put": 0, 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: "put_sum": 0, 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: "wait": { 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: "avgcount": 0, 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: "sum": 0.000000000, 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: "avgtime": 0.000000000 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout: } 2024-08-02T18:03:10.698 INFO:teuthology.orchestra.run.smithi077.stdout:} 2024-08-02T18:03:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:10 smithi083 ceph-mon[37031]: pgmap v525: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 57 KiB/s wr, 29 op/s 2024-08-02T18:03:11.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:10 smithi083 ceph-mon[37031]: osdmap e116: 12 total, 12 up, 12 in 2024-08-02T18:03:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:10 smithi077 ceph-mon[37108]: pgmap v525: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 57 KiB/s wr, 29 op/s 2024-08-02T18:03:11.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:10 smithi077 ceph-mon[37108]: osdmap e116: 12 total, 12 up, 12 in 2024-08-02T18:03:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:10 smithi069 ceph-mon[32746]: pgmap v525: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 57 KiB/s wr, 29 op/s 2024-08-02T18:03:11.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:10 smithi069 ceph-mon[32746]: osdmap e116: 12 total, 12 up, 12 in 2024-08-02T18:03:11.691 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-08-02T18:03:11.691 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-08-02T18:03:11.691 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-08-02T18:03:11.691 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-08-02T18:03:11.692 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-08-02T18:03:11.692 DEBUG:tasks.check_counter:Getting stats from c 2024-08-02T18:03:11.692 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-02T18:03:11.988 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T18:03:12.840 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-02T18:03:12.840 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1447, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1434, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1887776, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 545228, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.166936608, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.054836345, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.075466468, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.038775902, 2024-08-02T18:03:12.841 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1433, 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.090974419, 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000063485 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1521536, 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 198608 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.842 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 9, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 244, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 4060, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 661898, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 4, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.015409497, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.008915113, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.004328614, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000053242, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 242, 2024-08-02T18:03:12.843 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.030021277, 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000124054 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 661200 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.844 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2094, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 2095, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 535777, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 665581, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 5, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.141395984, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.040429797, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.035928796, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.056390131, 2024-08-02T18:03:12.845 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2093, 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.081532966, 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000038955 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-02T18:03:12.846 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5262, 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.132744311, 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000025226 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.847 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.848 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 0, 2024-08-02T18:03:12.849 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 1, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-08-02T18:03:12.850 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-08-02T18:03:12.851 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-02T18:03:12.852 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-08-02T18:03:12.853 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-08-02T18:03:12.854 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-08-02T18:03:12.855 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 8, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 2, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 4196652, 2024-08-02T18:03:12.856 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 4196511, 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 8 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 3, 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 3, 2024-08-02T18:03:12.857 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 1, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 1, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 0, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 0, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 50888, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215328 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.858 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.859 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.860 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.861 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.862 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-08-02T18:03:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-08-02T18:03:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.863 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.867 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.868 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-08-02T18:03:12.869 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-08-02T18:03:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 4, 2024-08-02T18:03:12.871 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 2, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 3, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 2866, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:12.872 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:12.873 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:12.874 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 182097, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 49, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:12.875 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 10008, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 203, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 8464, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 13, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-08-02T18:03:12.876 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 3185, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 3185, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3185, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 29.131317613, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.009146410 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3185, 2024-08-02T18:03:12.877 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3186 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 3185, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 3185, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 2083, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 1100, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-08-02T18:03:12.878 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-08-02T18:03:12.879 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-08-02T18:03:12.880 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 116, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 26, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 4, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 5, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 1, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-08-02T18:03:12.881 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-08-02T18:03:12.882 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3546, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 2087142, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:12.883 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3546, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3546, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2087142, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-02T18:03:12.884 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3185, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 2353, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3185, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2353, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.885 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3185, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3185, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3185, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3185, 2024-08-02T18:03:12.886 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:12.887 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x56387a18c6e0": { 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:12.888 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:12.889 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-02T18:03:13.230 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:12 smithi083 ceph-mon[37031]: pgmap v527: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-08-02T18:03:13.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:12 smithi077 ceph-mon[37108]: pgmap v527: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-08-02T18:03:13.359 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:12 smithi069 ceph-mon[32746]: pgmap v527: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 59 KiB/s wr, 28 op/s 2024-08-02T18:03:14.054 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-02T18:03:14.055 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-08-02T18:03:14.055 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-08-02T18:03:14.055 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-08-02T18:03:14.055 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-08-02T18:03:14.055 DEBUG:tasks.check_counter:Getting stats from f 2024-08-02T18:03:14.055 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-02T18:03:14.324 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T18:03:14.932 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-02T18:03:14.932 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:14.932 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1109, 2024-08-02T18:03:14.932 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1111, 2024-08-02T18:03:14.932 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 294479, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 352726, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.090567718, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.026830296, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.024587828, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.030155704, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1109, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.053127674, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000047905 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.933 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 0 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:14.934 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1011, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1247, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 250596, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 892955, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 7, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.103510458, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.030008865, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.026155786, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.038195581, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1245, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.087855839, 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000070566 2024-08-02T18:03:14.935 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 573360 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1456, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1443, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1890736, 2024-08-02T18:03:14.936 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 548412, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 3, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.168286424, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.054535222, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.078893342, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.039373566, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1442, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.096090215, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000066636 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.937 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1521808, 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 199248 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-08-02T18:03:14.938 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5357, 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.144704259, 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000027012 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.939 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 0, 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 0, 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 0, 2024-08-02T18:03:14.940 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 0, 2024-08-02T18:03:14.941 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 1, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-08-02T18:03:14.942 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-08-02T18:03:14.943 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-08-02T18:03:14.944 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 0 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-08-02T18:03:14.945 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-08-02T18:03:14.946 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 12, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 2, 2024-08-02T18:03:14.947 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 4197714, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 4197573, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 12 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-08-02T18:03:14.948 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 3, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 3, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 1, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 1, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 0, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 0, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 0, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 0, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 0, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 0, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 48144, 2024-08-02T18:03:14.949 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 215328 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 0, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 0, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 0, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 0, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-08-02T18:03:14.950 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-08-02T18:03:14.951 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.952 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.953 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.954 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-08-02T18:03:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-08-02T18:03:14.958 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.959 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.959 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.959 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.959 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-08-02T18:03:14.959 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.959 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.959 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.960 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.961 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.962 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-08-02T18:03:14.963 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.964 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.965 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-08-02T18:03:14.966 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 6, 2024-08-02T18:03:14.967 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 4, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 5, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 0, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 0, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 6978, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:14.968 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:14.969 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:14.970 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:14.970 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:14.970 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:14.970 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:14.970 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:14.970 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:14.970 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:14.971 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:14.971 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:14.971 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:14.971 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:14.971 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:14.971 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:14.971 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-08-02T18:03:14.971 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 190234, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 55, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:14.972 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 10008, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 203, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 8464, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 13, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-08-02T18:03:14.973 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 3212, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 21, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 3, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 3212, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3212, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 10.574751673, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.003292263 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.974 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3212, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 3213 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 3212, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 3212, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 0, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 2101, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 0, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 1109, 2024-08-02T18:03:14.975 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 1, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 0, 2024-08-02T18:03:14.976 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-08-02T18:03:14.977 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-08-02T18:03:14.978 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 116, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 26, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 5, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 6, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 1, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 0, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 2, 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 0 2024-08-02T18:03:14.979 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 0, 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 0 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-08-02T18:03:14.980 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3572, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 2092991, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3572, 2024-08-02T18:03:14.981 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3572, 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2092991, 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.982 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3212, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3415, 2024-08-02T18:03:14.983 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3212, 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3415, 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:14.984 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 3212, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 3212, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3212, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3212, 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:14.985 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:14.986 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x563d1ba606e0": { 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-08-02T18:03:14.987 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:14.988 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.989 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:14.989 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-02T18:03:15.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:14 smithi083 ceph-mon[37031]: pgmap v528: 129 pgs: 129 active+clean; 2.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-08-02T18:03:15.348 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:14 smithi077 ceph-mon[37108]: pgmap v528: 129 pgs: 129 active+clean; 2.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-08-02T18:03:15.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:14 smithi069 ceph-mon[32746]: pgmap v528: 129 pgs: 129 active+clean; 2.6 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 31 KiB/s wr, 10 op/s 2024-08-02T18:03:16.257 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-08-02T18:03:16.257 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-08-02T18:03:16.257 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-08-02T18:03:16.257 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-08-02T18:03:16.258 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-08-02T18:03:16.258 DEBUG:tasks.check_counter:Getting stats from i 2024-08-02T18:03:16.258 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-02T18:03:16.519 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T18:03:17.193 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:16 smithi083 ceph-mon[37031]: pgmap v529: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 2025, 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 1355, 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 517974, 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 647368, 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 9, 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 2, 2024-08-02T18:03:17.343 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.128564981, 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.045101004, 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.092667023, 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.006476312, 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1355, 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.073849143, 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000054501 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3240, 2024-08-02T18:03:17.344 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000580749, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000179 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3376, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 5008 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 1459, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 521, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 2015584, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 324411, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 14, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 5, 2024-08-02T18:03:17.345 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.189154871, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.045826889, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.099152297, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.054737647, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 520, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.043332340, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000083331 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 1165, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000207760, 2024-08-02T18:03:17.346 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000178 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 2009184, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 235232 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 84, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 325, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 18320, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 1306730, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 66, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 7, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.036024584, 2024-08-02T18:03:17.347 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.010300189, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.022186136, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000099379, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 321, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.027059574, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000084297 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 90, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000011032, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000122 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.348 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 9408, 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 670560 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 1, 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-MDSRank": { 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 70, 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.002066683, 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000029524 2024-08-02T18:03:17.349 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "finisher-PurgeQueue": { 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "queue_len": 0, 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "complete_latency": { 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 2, 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000012843, 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000006421 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "mds": { 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "request": 13, 2024-08-02T18:03:17.350 INFO:teuthology.orchestra.run.smithi083.stdout: "reply": 9, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "reply_latency": { 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 9, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.836419469, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.204046607 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_reply": 0, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "forward": 4, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_complete": 0, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_fetch_keys": 0, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_commit": 11, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_split": 0, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_merge": 0, 2024-08-02T18:03:17.351 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes": 14, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_top": 0, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_bottom": 14, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pin_tail": 0, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_pinned": 14, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_expired": 0, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "inodes_with_caps": 1, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "caps": 1, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "subtrees": 2, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse": 51, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_hit": 47, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_forward": 2, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_discover": 1, 2024-08-02T18:03:17.352 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_dir_fetch": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_remote_ino": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "traverse_lock": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "load_cent": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "q": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "exported": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "exported_inodes": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "imported": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "imported_inodes": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_dir_fetch": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_backtrace_fetch": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "openino_peer_discover": 0, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rfiles": 1018, 2024-08-02T18:03:17.353 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rbytes": 1537029723, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "root_rsnaps": 6, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_fetch": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_set_tag": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_backtrace_repaired": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_inotable_repaired": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_inodes": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dir_base_inodes": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_dirfrag_rstats": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "scrub_file_inodes": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_inode_file_caps": 0, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_revoke": 13, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_grant": 7, 2024-08-02T18:03:17.354 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_trunc": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps": 14, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_caps_dirty": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_cap_release": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "process_request_cap_release": 1 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_cache": { 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_update_receipt": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_try_discover": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_send_discover": 2, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "dir_handle_discover": 0, 2024-08-02T18:03:17.355 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_delayed": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "num_strays_enqueuing": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_created": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_enqueued": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_reintegrated": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "strays_migrated": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_processing": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_enqueued": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "num_recovering_prioritized": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_started": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "recovery_completed": 0, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_path": 14, 2024-08-02T18:03:17.356 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_quiesce_inode": 70, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_enqueue_scrub": 0, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_exportdir": 0, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_flush": 0, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragmentdir": 0, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_fragstats": 0, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ireq_inodestats": 0 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_log": { 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "evlrg": 0, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "evadd": 12, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "evex": 0, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "evtrm": 0, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "ev": 12, 2024-08-02T18:03:17.357 INFO:teuthology.orchestra.run.smithi083.stdout: "evexg": 0, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "evexd": 0, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "segadd": 2, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "segex": 0, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "segtrm": 0, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "seg": 2, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "segmjr": 2, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "segexg": 0, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "segexd": 0, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "expos": 4194304, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "wrpos": 4197714, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "rdpos": 0, 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "jlat": { 2024-08-02T18:03:17.358 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 11, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.022704770, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.002064070 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "replayed": 0 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_mem": { 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "ino": 17, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "ino+": 17, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "ino-": 0, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "dir": 15, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "dir+": 16, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "dir-": 1, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "dn": 14, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "dn+": 14, 2024-08-02T18:03:17.359 INFO:teuthology.orchestra.run.smithi083.stdout: "dn-": 0, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "cap": 1, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "cap+": 8, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "cap-": 7, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "rss": 51792, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "heap": 223520 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_server": { 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_client_request": 19, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "dispatch_server_request": 0, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_request": 13, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_client_session": 40, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "handle_peer_request": 2, 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "req_create_latency": { 2024-08-02T18:03:17.360 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getattr_latency": { 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 5, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 1.835858359, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.367171671 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getfilelock_latency": { 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.361 INFO:teuthology.orchestra.run.smithi083.stdout: "req_link_latency": { 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookup_latency": { 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 4, 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000561110, 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000140277 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookuphash_latency": { 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.362 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupino_latency": { 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupname_latency": { 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupparent_latency": { 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.363 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lookupsnap_latency": { 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "req_lssnap_latency": { 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mkdir_latency": { 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.364 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mknod_latency": { 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "req_mksnap_latency": { 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "req_open_latency": { 2024-08-02T18:03:17.365 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "req_readdir_latency": { 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rename_latency": { 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.366 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "req_renamesnap_latency": { 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "req_snapdiff_latency": { 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmdir_latency": { 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.367 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmsnap_latency": { 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "req_rmxattr_latency": { 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setattr_latency": { 2024-08-02T18:03:17.368 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setdirlayout_latency": { 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setfilelock_latency": { 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.369 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setlayout_latency": { 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "req_setxattr_latency": { 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "req_symlink_latency": { 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.370 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "req_unlink_latency": { 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_revoke_eviction": 0, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "cap_acquisition_throttle": 0, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "req_getvxattr_latency": { 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.371 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_sessions": { 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "session_count": 2, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "session_add": 6, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "session_remove": 4, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_open": 2, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "sessions_stale": 0, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "total_load": 8, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "average_load": 2, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "avg_session_uptime": 993, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "mdthresh_evicted": 0 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:17.372 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:17.373 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:17.374 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 255191, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 83, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:17.375 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 10008, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 203, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 67861, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 300, 2024-08-02T18:03:17.376 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "objecter": { 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "op_active": 0, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "op_laggy": 0, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send": 54, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "op_send_bytes": 14624, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "op_resend": 28, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "op_reply": 54, 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "op_latency": { 2024-08-02T18:03:17.377 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 54, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.219994820, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.004073978 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "op_inflight": 0, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "oplen_avg": { 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 54, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 82 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "op": 54, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "op_r": 0, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "op_w": 54, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "op_rmw": 0, 2024-08-02T18:03:17.378 INFO:teuthology.orchestra.run.smithi083.stdout: "op_pg": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_stat": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_create": 10, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_read": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_write": 11, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writefull": 8, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_writesame": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_append": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_zero": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_truncate": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_delete": 6, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_mapext": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_sparse_read": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_clonerange": 0, 2024-08-02T18:03:17.379 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_getxattr": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_setxattr": 20, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_cmpxattr": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_rmxattr": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_resetxattrs": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_call": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_watch": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_notify": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_src_cmpxattr": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_pgls_filter": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "osdop_other": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_active": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_send": 0, 2024-08-02T18:03:17.380 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_resend": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "linger_ping": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_active": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_send": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "poolop_resend": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_active": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_send": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "poolstat_resend": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_active": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_send": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "statfs_resend": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "command_active": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "command_send": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "command_resend": 0, 2024-08-02T18:03:17.381 INFO:teuthology.orchestra.run.smithi083.stdout: "map_epoch": 116, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "map_full": 0, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "map_inc": 26, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_sessions": 8, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_open": 11, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_session_close": 3, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_laggy": 0, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_wr": 25, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_rd": 0, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_del": 2 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "oft": { 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_objs": 1, 2024-08-02T18:03:17.382 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_kv_pairs": 0, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_updates": 0, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "omap_total_removes": 0 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "osd-slow-ops": { 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "slow_ops_count": 72 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "purge_queue": { 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops": 0, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_ops_high_water": 0, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing": 0, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executing_high_water": 0, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed_ops": 0, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_executed": 0, 2024-08-02T18:03:17.383 INFO:teuthology.orchestra.run.smithi083.stdout: "pq_item_in_journal": 0 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 3564, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 2158957, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 3564, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 3564, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 2158957, 2024-08-02T18:03:17.384 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_bytes": { 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 54, 2024-08-02T18:03:17.385 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 14624, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 54, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 14624, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-objecter_ops": { 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 1024, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:17.386 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 54, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 54, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 54, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 54, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.387 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle": { 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.388 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-write_buf_throttle-0x55f5d126c500": { 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 3758096384, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 12, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 3410, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 12, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 11, 2024-08-02T18:03:17.389 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 3410, 2024-08-02T18:03:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.390 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:17.390 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-02T18:03:17.391 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:16 smithi069 ceph-mon[32746]: pgmap v529: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-02T18:03:17.391 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:16 smithi077 ceph-mon[37108]: pgmap v529: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-02T18:03:18.076 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-08-02T18:03:18.076 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-08-02T18:03:18.077 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-08-02T18:03:18.077 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-08-02T18:03:18.077 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-08-02T18:03:18.077 DEBUG:tasks.check_counter:Getting stats from l 2024-08-02T18:03:18.077 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-02T18:03:18.329 INFO:teuthology.orchestra.run.smithi083.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.c/config 2024-08-02T18:03:18.860 INFO:teuthology.orchestra.run.smithi083.stdout:{ 2024-08-02T18:03:18.860 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-0": { 2024-08-02T18:03:18.860 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 4, 2024-08-02T18:03:18.860 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 4, 2024-08-02T18:03:18.860 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 3024, 2024-08-02T18:03:18.860 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 448, 2024-08-02T18:03:18.860 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 4, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 0, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.003140916, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.000030614, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.002289239, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 3, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.004238265, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.001412755 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:18.861 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 448 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-1": { 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 328, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 318, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 1473632, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 196400, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 2, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 1, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.053613200, 2024-08-02T18:03:18.862 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.020314550, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.045578494, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 317, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.035059588, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000110598 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 1473632, 2024-08-02T18:03:18.863 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 196400 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "AsyncMessenger::Worker-2": { 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_messages": 3, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages": 245, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_bytes": 480, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_bytes": 278272, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_created_connections": 6, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_active_connections": 3, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_total_time": 0.014405364, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_send_time": 0.009700082, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_recv_time": 0.002074181, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_messages_queue_lat": { 2024-08-02T18:03:18.864 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 242, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.022470638, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000092853 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_handle_ack_lat": { 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_recv_encrypted_bytes": 480, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "msgr_send_encrypted_bytes": 278272 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "cct": { 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "total_workers": 0, 2024-08-02T18:03:18.865 INFO:teuthology.orchestra.run.smithi083.stdout: "unhealthy_workers": 0 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "mempool": { 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_bytes": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bloom_filter_items": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_bytes": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_alloc_items": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_bytes": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_data_items": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_onode_items": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-02T18:03:18.866 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_meta_items": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_bytes": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_other_items": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_cache_buffer_items": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_bytes": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_extent_items": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_bytes": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_blob_items": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_shared_blob_items": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_inline_bl_items": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_bytes": 0, 2024-08-02T18:03:18.867 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_fsck_items": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_bytes": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_txc_items": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_deferred_items": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_bytes": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluestore_writing_items": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_bytes": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_items": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_bytes": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_reader_items": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_bytes": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "bluefs_file_writer_items": 0, 2024-08-02T18:03:18.868 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_bytes": 147970, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_anon_items": 29, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_bytes": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "buffer_meta_items": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_bytes": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_items": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_bytes": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_mapbl_items": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_bytes": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osd_pglog_items": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_bytes": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_items": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_bytes": 0, 2024-08-02T18:03:18.869 INFO:teuthology.orchestra.run.smithi083.stdout: "osdmap_mapping_items": 0, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_bytes": 0, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "pgmap_items": 0, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_bytes": 928, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "mds_co_items": 2, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_bytes": 0, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_1_items": 0, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_bytes": 0, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "unittest_2_items": 0 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: }, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "val": 0, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "max": 104857600, 2024-08-02T18:03:18.870 INFO:teuthology.orchestra.run.smithi083.stdout: "get_started": 0, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "get": 331, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "get_sum": 1427367, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_fail": 0, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "get_or_fail_success": 331, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "take": 0, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "take_sum": 0, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "put": 331, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "put_sum": 1427367, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "wait": { 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "avgcount": 0, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "sum": 0.000000000, 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: "avgtime": 0.000000000 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:18.871 INFO:teuthology.orchestra.run.smithi083.stdout: } 2024-08-02T18:03:18.872 INFO:teuthology.orchestra.run.smithi083.stdout:} 2024-08-02T18:03:19.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:18 smithi083 ceph-mon[37031]: pgmap v530: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-02T18:03:19.347 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:18 smithi077 ceph-mon[37108]: pgmap v530: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-02T18:03:19.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:18 smithi069 ceph-mon[32746]: pgmap v530: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-08-02T18:03:19.477 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-08-02T18:03:19.477 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-08-02T18:03:19.477 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-08-02T18:03:19.477 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-08-02T18:03:19.477 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-08-02T18:03:19.478 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_53ce1462e129f6eb4071986336534c740fdebd31/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph-c_26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-08-02T18:03:19.478 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-02T18:03:19.491 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-02T18:03:19.491 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T18:03:19.491 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-02T18:03:19.527 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T18:03:19.527 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-02T18:03:19.584 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd blocklist ls 2024-08-02T18:03:19.904 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 -- 172.21.15.69:0/1932607062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504103410 msgr2=0x7ff5041078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 --2- 172.21.15.69:0/1932607062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504103410 0x7ff5041078c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009930 tx=0x7ff4f802a420 comp rx=0 tx=0).stop 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 -- 172.21.15.69:0/1932607062 shutdown_connections 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 --2- 172.21.15.69:0/1932607062 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504103410 0x7ff5041078c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 --2- 172.21.15.69:0/1932607062 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff504107fd0 0x7ff50410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 --2- 172.21.15.69:0/1932607062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff504102a60 0x7ff504102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 -- 172.21.15.69:0/1932607062 >> 172.21.15.69:0/1932607062 conn(0x7ff5040fc500 msgr2=0x7ff5040fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 -- 172.21.15.69:0/1932607062 shutdown_connections 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 -- 172.21.15.69:0/1932607062 wait complete. 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.616+0000 7ff50b82b640 1 Processor -- start 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff50b82b640 1 -- start start 2024-08-02T18:03:20.665 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff50b82b640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504102a60 0x7ff50419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff50b82b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff504103410 0x7ff50419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff50b82b640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff504107fd0 0x7ff50419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff50b82b640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7ff504111dd0 con 0x7ff504102a60 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff50b82b640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7ff504111c50 con 0x7ff504107fd0 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff50b82b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff504111f50 con 0x7ff504103410 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504102a60 0x7ff50419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504102a60 0x7ff50419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36094/0 (socket says 172.21.15.69:36094) 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 -- 172.21.15.69:0/2568139796 learned_addr learned my addr 172.21.15.69:0/2568139796 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff508dc6640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff504103410 0x7ff50419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff509dc8640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff504107fd0 0x7ff50419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 -- 172.21.15.69:0/2568139796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff504103410 msgr2=0x7ff50419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:20.666 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff504103410 0x7ff50419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 -- 172.21.15.69:0/2568139796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff504107fd0 msgr2=0x7ff50419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff504107fd0 0x7ff50419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 -- 172.21.15.69:0/2568139796 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50419f460 con 0x7ff504102a60 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff509dc8640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff504107fd0 0x7ff50419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.617+0000 7ff5095c7640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504102a60 0x7ff50419a4b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400ea80 tx=0x7ff4f400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.618+0000 7ff5027fc640 1 -- 172.21.15.69:0/2568139796 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f400ce40 con 0x7ff504102a60 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.618+0000 7ff5027fc640 1 -- 172.21.15.69:0/2568139796 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4f4005020 con 0x7ff504102a60 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.618+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50419f750 con 0x7ff504102a60 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.618+0000 7ff5027fc640 1 -- 172.21.15.69:0/2568139796 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4011760 con 0x7ff504102a60 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.618+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5041a7030 con 0x7ff504102a60 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.619+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff504102e40 con 0x7ff504102a60 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.620+0000 7ff5027fc640 1 -- 172.21.15.69:0/2568139796 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff4f40051d0 con 0x7ff504102a60 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.621+0000 7ff5027fc640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4ec0a2cf0 0x7ff4ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:20.667 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.621+0000 7ff508dc6640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4ec0a2cf0 0x7ff4ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:20.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.621+0000 7ff5027fc640 1 -- 172.21.15.69:0/2568139796 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9568+0+0 (secure 0 0 0) 0x7ff4f40cfef0 con 0x7ff504102a60 2024-08-02T18:03:20.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.621+0000 7ff508dc6640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4ec0a2cf0 0x7ff4ec0a51b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005940 tx=0x7ff4f80058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:20.668 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.625+0000 7ff5027fc640 1 -- 172.21.15.69:0/2568139796 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7ff4f4016030 con 0x7ff504102a60 2024-08-02T18:03:20.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.805+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff50419b810 con 0x7ff504102a60 2024-08-02T18:03:20.807 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.806+0000 7ff5027fc640 1 -- 172.21.15.69:0/2568139796 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v117) ==== 81+0+1001 (secure 0 0 0) 0x7ff4f409c700 con 0x7ff504102a60 2024-08-02T18:03:20.807 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3271476957 2024-08-03T17:46:24.900194+0000 2024-08-02T18:03:20.807 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3594391539 2024-08-03T17:46:24.900194+0000 2024-08-02T18:03:20.807 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3033851323 2024-08-03T17:46:24.900194+0000 2024-08-02T18:03:20.807 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2680424375 2024-08-03T17:46:24.900194+0000 2024-08-02T18:03:20.807 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4073447426 2024-08-03T17:37:34.617397+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3960560393 2024-08-03T17:37:34.617397+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/3376234785 2024-08-03T17:37:56.685064+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/452236630 2024-08-03T17:46:24.900194+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3960560393 2024-08-03T17:37:34.617397+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6801/3376234785 2024-08-03T17:37:56.685064+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/623492814 2024-08-03T17:37:34.617397+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2152855317 2024-08-03T17:37:56.685064+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2060958760 2024-08-03T17:37:56.685064+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/2814849525 2024-08-03T17:37:56.685064+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/3801376647 2024-08-03T17:37:34.617397+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:0/4036393791 2024-08-03T17:46:24.900194+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stdout:172.21.15.69:6800/452236630 2024-08-03T17:46:24.900194+0000 2024-08-02T18:03:20.808 INFO:teuthology.orchestra.run.smithi069.stderr:listed 17 entries 2024-08-02T18:03:20.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4ec0a2cf0 msgr2=0x7ff4ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:20.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4ec0a2cf0 0x7ff4ec0a51b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8005940 tx=0x7ff4f80058d0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504102a60 msgr2=0x7ff50419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:20.809 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504102a60 0x7ff50419a4b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff4f400ea80 tx=0x7ff4f400ef40 comp rx=0 tx=0).stop 2024-08-02T18:03:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 shutdown_connections 2024-08-02T18:03:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7ff4ec0a2cf0 0x7ff4ec0a51b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7ff504107fd0 0x7ff50419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff504103410 0x7ff50419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 --2- 172.21.15.69:0/2568139796 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7ff504102a60 0x7ff50419a4b0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 >> 172.21.15.69:0/2568139796 conn(0x7ff5040fc500 msgr2=0x7ff504100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 shutdown_connections 2024-08-02T18:03:20.810 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:20.808+0000 7ff50b82b640 1 -- 172.21.15.69:0/2568139796 wait complete. 2024-08-02T18:03:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:20 smithi083 ceph-mon[37031]: pgmap v531: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-08-02T18:03:21.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:20 smithi083 ceph-mon[37031]: osdmap e117: 12 total, 12 up, 12 in 2024-08-02T18:03:21.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:20 smithi077 ceph-mon[37108]: pgmap v531: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-08-02T18:03:21.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:20 smithi077 ceph-mon[37108]: osdmap e117: 12 total, 12 up, 12 in 2024-08-02T18:03:21.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:20 smithi069 ceph-mon[32746]: pgmap v531: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-08-02T18:03:21.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:20 smithi069 ceph-mon[32746]: osdmap e117: 12 total, 12 up, 12 in 2024-08-02T18:03:21.572 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-02T18:03:21.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T18:03:21.573 DEBUG:teuthology.orchestra.run.smithi069:> sudo umount /tmp/cephfs 2024-08-02T18:03:21.655 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T18:03:21.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T18:03:21.655 DEBUG:teuthology.orchestra.run.smithi069:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-08-02T18:03:21.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T18:03:21.704 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-08-02T18:03:21.743 INFO:teuthology.orchestra.run.smithi069.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-08-02T18:03:21.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T18:03:21.743 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-08-02T18:03:21.805 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:21 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2568139796' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T18:03:21.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T18:03:21.809 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-08-02T18:03:21.915 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-02T18:03:21.915 DEBUG:teuthology.orchestra.run.smithi069:> set -ex 2024-08-02T18:03:21.915 DEBUG:teuthology.orchestra.run.smithi069:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-02T18:03:21.931 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056029,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497412k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20673 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248704k,nr_inodes=812176,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/81a623d55dce676b45b5ca7ee164698d05a4a86e91c481db6e525bf3e9543867/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/81a623d55dce676b45b5ca7ee164698d05a4a86e91c481db6e525bf3e9543867/diff,workdir=/var/lib/containers/storage/overlay/81a623d55dce676b45b5ca7ee164698d05a4a86e91c481db6e525bf3e9543867/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/4cff1f63e1cc5ca7a291337cdf387924698ddb088c0ec5c0cfea69c802f38f43/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/4cff1f63e1cc5ca7a291337cdf387924698ddb088c0ec5c0cfea69c802f38f43/diff,workdir=/var/lib/containers/storage/overlay/4cff1f63e1cc5ca7a291337cdf387924698ddb088c0ec5c0cfea69c802f38f43/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248704k,nr_inodes=812176,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/3a7e7c2ca5c66e4577e9957401a93315a5aa82b6771da2534c33bd456675acf2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/3a7e7c2ca5c66e4577e9957401a93315a5aa82b6771da2534c33bd456675acf2/diff,workdir=/var/lib/containers/storage/overlay/3a7e7c2ca5c66e4577e9957401a93315a5aa82b6771da2534c33bd456675acf2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/c5fc831db816f6c2c71c035efb451e15ef1f4adc843de452993a602d6e58bb63/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/c5fc831db816f6c2c71c035efb451e15ef1f4adc843de452993a602d6e58bb63/diff,workdir=/var/lib/containers/storage/overlay/c5fc831db816f6c2c71c035efb451e15ef1f4adc843de452993a602d6e58bb63/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b64259de40e88880ffb156dc64bd2772ec04d136abc2c971788fb7e4b44bc5ed/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/b64259de40e88880ffb156dc64bd2772ec04d136abc2c971788fb7e4b44bc5ed/diff,workdir=/var/lib/containers/storage/overlay/b64259de40e88880ffb156dc64bd2772ec04d136abc2c971788fb7e4b44bc5ed/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9fa56a6f068e1da23284dd4780652605b1c530463b37eccc5cfc0c16b7d6c4d3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/9fa56a6f068e1da23284dd4780652605b1c530463b37eccc5cfc0c16b7d6c4d3/diff,workdir=/var/lib/containers/storage/overlay/9fa56a6f068e1da23284dd4780652605b1c530463b37eccc5cfc0c16b7d6c4d3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/47adc0b3ddf67deac5bb2132eae9aca71130cb8d0d5e397fbfb07970995b7ce7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/47adc0b3ddf67deac5bb2132eae9aca71130cb8d0d5e397fbfb07970995b7ce7/diff,workdir=/var/lib/containers/storage/overlay/47adc0b3ddf67deac5bb2132eae9aca71130cb8d0d5e397fbfb07970995b7ce7/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ba00bbc5a3996b64f736c25ef787e0b9b0a930feb8a10744a1698cb7d48c79ec/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/ba00bbc5a3996b64f736c25ef787e0b9b0a930feb8a10744a1698cb7d48c79ec/diff,workdir=/var/lib/containers/storage/overlay/ba00bbc5a3996b64f736c25ef787e0b9b0a930feb8a10744a1698cb7d48c79ec/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/5e420b9a05075761115aa777cf3e897fd0306b48cdd567ebbeadde6df70cbc59/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/5e420b9a05075761115aa777cf3e897fd0306b48cdd567ebbeadde6df70cbc59/diff,workdir=/var/lib/containers/storage/overlay/5e420b9a05075761115aa777cf3e897fd0306b48cdd567ebbeadde6df70cbc59/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8abd8e7f7ae007568d863875387f1a83376c00bdd0fb2304d084fbaceca2debc/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VWHRYFDM37KBR3KPIHILGURQAX:/var/lib/containers/storage/overlay/l/Y2YCPNWZYTJQUKLNXMCTQ7TXMV,upperdir=/var/lib/containers/storage/overlay/8abd8e7f7ae007568d863875387f1a83376c00bdd0fb2304d084fbaceca2debc/diff,workdir=/var/lib/containers/storage/overlay/8abd8e7f7ae007568d863875387f1a83376c00bdd0fb2304d084fbaceca2debc/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497412k,nr_inodes=819200,mode=755,inode64 0 0 2024-08-02T18:03:21.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T18:03:21.932 DEBUG:teuthology.orchestra.run.smithi069:> ip netns list 2024-08-02T18:03:21.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-02T18:03:21.988 DEBUG:teuthology.orchestra.run.smithi069:> sudo ip link delete ceph-brx 2024-08-02T18:03:22.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:21 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2568139796' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T18:03:22.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:21 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2568139796' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-08-02T18:03:22.142 INFO:teuthology.orchestra.run.smithi069.stderr:Cannot find device "ceph-brx" 2024-08-02T18:03:22.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T18:03:22.144 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-02T18:03:22.156 INFO:tasks.cephadm:Teardown begin 2024-08-02T18:03:22.156 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-02T18:03:22.156 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:03:22.412 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:22.796 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:22 smithi069 ceph-mon[32746]: pgmap v533: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-08-02T18:03:22.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 -- 172.21.15.69:0/940606322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb401018b0 msgr2=0x7feb40101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:22.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 --2- 172.21.15.69:0/940606322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb401018b0 0x7feb40101d10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7feb34009930 tx=0x7feb3402a420 comp rx=0 tx=0).stop 2024-08-02T18:03:22.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 -- 172.21.15.69:0/940606322 shutdown_connections 2024-08-02T18:03:22.904 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 --2- 172.21.15.69:0/940606322 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb401018b0 0x7feb40101d10 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 --2- 172.21.15.69:0/940606322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb40102250 0x7feb4010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 --2- 172.21.15.69:0/940606322 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb401078b0 0x7feb40107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 -- 172.21.15.69:0/940606322 >> 172.21.15.69:0/940606322 conn(0x7feb400fd5f0 msgr2=0x7feb400ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 -- 172.21.15.69:0/940606322 shutdown_connections 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.875+0000 7feb45b4f640 1 -- 172.21.15.69:0/940606322 wait complete. 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb45b4f640 1 Processor -- start 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb45b4f640 1 -- start start 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb45b4f640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb401018b0 0x7feb40198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb45b4f640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102250 0x7feb401986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb45b4f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb401078b0 0x7feb4019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:22.905 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb45b4f640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7feb40111cb0 con 0x7feb40102250 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb45b4f640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7feb40111b30 con 0x7feb401018b0 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb45b4f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb40111e30 con 0x7feb401078b0 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb3effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102250 0x7feb401986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb3effd640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102250 0x7feb401986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36114/0 (socket says 172.21.15.69:36114) 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.876+0000 7feb3effd640 1 -- 172.21.15.69:0/2964310721 learned_addr learned my addr 172.21.15.69:0/2964310721 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3ffff640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb401078b0 0x7feb4019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3f7fe640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb401018b0 0x7feb40198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3effd640 1 -- 172.21.15.69:0/2964310721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb401078b0 msgr2=0x7feb4019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3effd640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb401078b0 0x7feb4019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3effd640 1 -- 172.21.15.69:0/2964310721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb401018b0 msgr2=0x7feb40198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3effd640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb401018b0 0x7feb40198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:22.906 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3effd640 1 -- 172.21.15.69:0/2964310721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4019d130 con 0x7feb40102250 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3ffff640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb401078b0 0x7feb4019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3effd640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102250 0x7feb401986c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7feb34009230 tx=0x7feb34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3cff9640 1 -- 172.21.15.69:0/2964310721 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3403f030 con 0x7feb40102250 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4019d3c0 con 0x7feb40102250 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3cff9640 1 -- 172.21.15.69:0/2964310721 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb34003ad0 con 0x7feb40102250 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb3cff9640 1 -- 172.21.15.69:0/2964310721 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34035070 con 0x7feb40102250 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.877+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb401a4d00 con 0x7feb40102250 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.878+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb40101d10 con 0x7feb40102250 2024-08-02T18:03:22.907 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.879+0000 7feb3cff9640 1 -- 172.21.15.69:0/2964310721 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7feb34042030 con 0x7feb40102250 2024-08-02T18:03:22.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.880+0000 7feb3cff9640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb280a2cf0 0x7feb280a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:22.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.880+0000 7feb3f7fe640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb280a2cf0 0x7feb280a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:22.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.880+0000 7feb3cff9640 1 -- 172.21.15.69:0/2964310721 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9568+0+0 (secure 0 0 0) 0x7feb340efe20 con 0x7feb40102250 2024-08-02T18:03:22.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.881+0000 7feb3f7fe640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb280a2cf0 0x7feb280a51b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7feb30005d90 tx=0x7feb30005d20 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:22.908 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:22.884+0000 7feb3cff9640 1 -- 172.21.15.69:0/2964310721 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7feb340bc630 con 0x7feb40102250 2024-08-02T18:03:23.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:22 smithi083 ceph-mon[37031]: pgmap v533: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-08-02T18:03:23.087 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.086+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb401098b0 con 0x7feb40102250 2024-08-02T18:03:23.088 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.087+0000 7feb3cff9640 1 -- 172.21.15.69:0/2964310721 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11901 (secure 0 0 0) 0x7feb340bbf30 con 0x7feb40102250 2024-08-02T18:03:23.088 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:03:23.089 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"btime":"2024-08-02T18:01:55:674676+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":28375,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3960673726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3960673726},{"type":"v1","addr":"172.21.15.77:6835","nonce":3960673726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T18:01:54.670284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24389":{"gid":24389,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:03:23.089 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-08-02T18:03:23.090 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb280a2cf0 msgr2=0x7feb280a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb280a2cf0 0x7feb280a51b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7feb30005d90 tx=0x7feb30005d20 comp rx=0 tx=0).stop 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102250 msgr2=0x7feb401986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102250 0x7feb401986c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7feb34009230 tx=0x7feb34004270 comp rx=0 tx=0).stop 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 shutdown_connections 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7feb40102250 0x7feb401986c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7feb280a2cf0 0x7feb280a51b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb401078b0 0x7feb4019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 --2- 172.21.15.69:0/2964310721 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7feb401018b0 0x7feb40198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 >> 172.21.15.69:0/2964310721 conn(0x7feb400fd5f0 msgr2=0x7feb400fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 shutdown_connections 2024-08-02T18:03:23.091 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:23.089+0000 7feb45b4f640 1 -- 172.21.15.69:0/2964310721 wait complete. 2024-08-02T18:03:23.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:22 smithi077 ceph-mon[37108]: pgmap v533: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-08-02T18:03:23.784 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:23 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2964310721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:03:23.817 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs dump --format=json 2024-08-02T18:03:24.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:23 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2964310721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:03:24.079 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:24.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:23 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2964310721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:03:24.571 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 -- 172.21.15.69:0/155029743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 msgr2=0x7fe618102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/155029743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 0x7fe618102ef0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe608009900 tx=0x7fe60802a480 comp rx=0 tx=0).stop 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 -- 172.21.15.69:0/155029743 shutdown_connections 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/155029743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe61810beb0 0x7fe61810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/155029743 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe618103430 0x7fe61810b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/155029743 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 0x7fe618102ef0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 -- 172.21.15.69:0/155029743 >> 172.21.15.69:0/155029743 conn(0x7fe6180fc4d0 msgr2=0x7fe6180fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 -- 172.21.15.69:0/155029743 shutdown_connections 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.550+0000 7fe61ff4e640 1 -- 172.21.15.69:0/155029743 wait complete. 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61ff4e640 1 Processor -- start 2024-08-02T18:03:24.572 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61ff4e640 1 -- start start 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61ff4e640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 0x7fe618100770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61ff4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618103430 0x7fe618100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61ff4e640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe61810beb0 0x7fe6181011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61ff4e640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111ce0 con 0x7fe618102b10 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61ff4e640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111b60 con 0x7fe61810beb0 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61ff4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111e60 con 0x7fe618103430 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 0x7fe618100770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 0x7fe618100770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:36132/0 (socket says 172.21.15.69:36132) 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 -- 172.21.15.69:0/2906135886 learned_addr learned my addr 172.21.15.69:0/2906135886 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61e4eb640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe61810beb0 0x7fe6181011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:24.573 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61d4e9640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618103430 0x7fe618100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 -- 172.21.15.69:0/2906135886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618103430 msgr2=0x7fe618100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618103430 0x7fe618100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 -- 172.21.15.69:0/2906135886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe61810beb0 msgr2=0x7fe6181011f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe61810beb0 0x7fe6181011f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 -- 172.21.15.69:0/2906135886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6181a6600 con 0x7fe618102b10 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61e4eb640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe61810beb0 0x7fe6181011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.551+0000 7fe61dcea640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 0x7fe618100770 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe608005ab0 tx=0x7fe6080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.552+0000 7fe60effd640 1 -- 172.21.15.69:0/2906135886 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60803f030 con 0x7fe618102b10 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.552+0000 7fe60effd640 1 -- 172.21.15.69:0/2906135886 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe608003ce0 con 0x7fe618102b10 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.552+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6181a6890 con 0x7fe618102b10 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.552+0000 7fe60effd640 1 -- 172.21.15.69:0/2906135886 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608035070 con 0x7fe618102b10 2024-08-02T18:03:24.574 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.552+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6181a6d70 con 0x7fe618102b10 2024-08-02T18:03:24.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.553+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe618102ef0 con 0x7fe618102b10 2024-08-02T18:03:24.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.554+0000 7fe60effd640 1 -- 172.21.15.69:0/2906135886 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe608042030 con 0x7fe618102b10 2024-08-02T18:03:24.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.554+0000 7fe60effd640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe6040a2cf0 0x7fe6040a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:24.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.555+0000 7fe61d4e9640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe6040a2cf0 0x7fe6040a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:24.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.555+0000 7fe60effd640 1 -- 172.21.15.69:0/2906135886 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9568+0+0 (secure 0 0 0) 0x7fe6080f0450 con 0x7fe618102b10 2024-08-02T18:03:24.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.555+0000 7fe61d4e9640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe6040a2cf0 0x7fe6040a51b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe614005e10 tx=0x7fe614005da0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:24.575 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.558+0000 7fe60effd640 1 -- 172.21.15.69:0/2906135886 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fe6080bcc60 con 0x7fe618102b10 2024-08-02T18:03:24.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.763+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe618106f70 con 0x7fe618102b10 2024-08-02T18:03:24.765 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:24 smithi069 ceph-mon[32746]: pgmap v534: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-02T18:03:24.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.764+0000 7fe60effd640 1 -- 172.21.15.69:0/2906135886 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11901 (secure 0 0 0) 0x7fe6080bc560 con 0x7fe618102b10 2024-08-02T18:03:24.766 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:03:24.766 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":28,"btime":"2024-08-02T18:01:55:674676+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14487,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6835/570537244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6834","nonce":570537244},{"type":"v1","addr":"172.21.15.69:6835","nonce":570537244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14499,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6837/3783033503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6836","nonce":3783033503},{"type":"v1","addr":"172.21.15.69:6837","nonce":3783033503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14553,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6839/1157046912","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6838","nonce":1157046912},{"type":"v1","addr":"172.21.15.69:6839","nonce":1157046912}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.69:6841/251587008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6840","nonce":251587008},{"type":"v1","addr":"172.21.15.69:6841","nonce":251587008}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24382,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6833/2696363098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2696363098},{"type":"v1","addr":"172.21.15.83:6833","nonce":2696363098}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":28375,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.77:6835/3960673726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6834","nonce":3960673726},{"type":"v1","addr":"172.21.15.77:6835","nonce":3960673726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-02T17:44:04.896710+0000","modified":"2024-08-02T18:01:54.670284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24452,"mds_1":24431,"mds_2":24427},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24389":{"gid":24389,"name":"k","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.77:6833/2071376346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6832","nonce":2071376346},{"type":"v1","addr":"172.21.15.77:6833","nonce":2071376346}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24403":{"gid":24403,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6835/3991686236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6834","nonce":3991686236},{"type":"v1","addr":"172.21.15.83:6835","nonce":3991686236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24409":{"gid":24409,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.83:6837/1744999893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6836","nonce":1744999893},{"type":"v1","addr":"172.21.15.83:6837","nonce":1744999893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24427":{"gid":24427,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":17,"addr":"172.21.15.83:6839/2418869426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6838","nonce":2418869426},{"type":"v1","addr":"172.21.15.83:6839","nonce":2418869426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24431":{"gid":24431,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":17,"addr":"172.21.15.77:6837/3778799888","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6836","nonce":3778799888},{"type":"v1","addr":"172.21.15.77:6837","nonce":3778799888}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24452":{"gid":24452,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.77:6839/1183977027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6838","nonce":1183977027},{"type":"v1","addr":"172.21.15.77:6839","nonce":1183977027}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24452,"qdb_cluster":[24452,24431,24427]},"id":1}]} 2024-08-02T18:03:24.766 INFO:teuthology.orchestra.run.smithi069.stderr:dumped fsmap epoch 28 2024-08-02T18:03:24.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe6040a2cf0 msgr2=0x7fe6040a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe6040a2cf0 0x7fe6040a51b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe614005e10 tx=0x7fe614005da0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 msgr2=0x7fe618100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 0x7fe618100770 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fe608005ab0 tx=0x7fe6080042f0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 shutdown_connections 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fe6040a2cf0 0x7fe6040a51b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe61810beb0 0x7fe6181011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe618103430 0x7fe618100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 --2- 172.21.15.69:0/2906135886 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe618102b10 0x7fe618100770 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 >> 172.21.15.69:0/2906135886 conn(0x7fe6180fc4d0 msgr2=0x7fe6180fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 shutdown_connections 2024-08-02T18:03:24.768 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:24.766+0000 7fe61ff4e640 1 -- 172.21.15.69:0/2906135886 wait complete. 2024-08-02T18:03:25.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:24 smithi083 ceph-mon[37031]: pgmap v534: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-02T18:03:25.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:24 smithi077 ceph-mon[37108]: pgmap v534: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-02T18:03:25.799 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd dump --format=json 2024-08-02T18:03:26.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:25 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2906135886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:03:26.062 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:26.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:25 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2906135886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:03:26.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:25 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2906135886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-08-02T18:03:26.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 -- 172.21.15.69:0/2935341183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400102a70 msgr2=0x7f0400102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:26.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 --2- 172.21.15.69:0/2935341183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400102a70 0x7f0400102e50 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f03f4009900 tx=0x7f03f402a480 comp rx=0 tx=0).stop 2024-08-02T18:03:26.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 -- 172.21.15.69:0/2935341183 shutdown_connections 2024-08-02T18:03:26.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 --2- 172.21.15.69:0/2935341183 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f040010bea0 0x7f040010e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 --2- 172.21.15.69:0/2935341183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0400103390 0x7f040010b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 --2- 172.21.15.69:0/2935341183 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400102a70 0x7f0400102e50 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.605 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 -- 172.21.15.69:0/2935341183 >> 172.21.15.69:0/2935341183 conn(0x7f04000fc4d0 msgr2=0x7f04000fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 -- 172.21.15.69:0/2935341183 shutdown_connections 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.582+0000 7f0406eb7640 1 -- 172.21.15.69:0/2935341183 wait complete. 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0406eb7640 1 Processor -- start 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0406eb7640 1 -- start start 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0406eb7640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0400102a70 0x7f04001006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0406eb7640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400103390 0x7f0400100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0406eb7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f040010bea0 0x7f0400101150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0406eb7640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111ce0 con 0x7f0400103390 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0406eb7640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111b60 con 0x7f0400102a70 2024-08-02T18:03:26.606 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0406eb7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111e60 con 0x7f040010bea0 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400103390 0x7f0400100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400103390 0x7f0400100c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42860/0 (socket says 172.21.15.69:42860) 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 -- 172.21.15.69:0/1010041737 learned_addr learned my addr 172.21.15.69:0/1010041737 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0404c53640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0400102a70 0x7f04001006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0405454640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f040010bea0 0x7f0400101150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 -- 172.21.15.69:0/1010041737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f040010bea0 msgr2=0x7f0400101150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f040010bea0 0x7f0400101150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 -- 172.21.15.69:0/1010041737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0400102a70 msgr2=0x7f04001006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0400102a70 0x7f04001006d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 -- 172.21.15.69:0/1010041737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04001a6760 con 0x7f0400103390 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f0404c53640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0400102a70 0x7f04001006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:03:26.607 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.583+0000 7f03fffff640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400103390 0x7f0400100c10 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00b3f0 tx=0x7f03ec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.584+0000 7f03fdffb640 1 -- 172.21.15.69:0/1010041737 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec015020 con 0x7f0400103390 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.584+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04001a6a50 con 0x7f0400103390 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.584+0000 7f03fdffb640 1 -- 172.21.15.69:0/1010041737 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03ec011d10 con 0x7f0400103390 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.584+0000 7f03fdffb640 1 -- 172.21.15.69:0/1010041737 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00f2e0 con 0x7f0400103390 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.584+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04001a6f90 con 0x7f0400103390 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.585+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0400102e50 con 0x7f0400103390 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.586+0000 7f03fdffb640 1 -- 172.21.15.69:0/1010041737 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f03ec018020 con 0x7f0400103390 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.586+0000 7f03fdffb640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03f00a2cf0 0x7f03f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.587+0000 7f0404c53640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03f00a2cf0 0x7f03f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.587+0000 7f03fdffb640 1 -- 172.21.15.69:0/1010041737 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9568+0+0 (secure 0 0 0) 0x7f03ec0cf060 con 0x7f0400103390 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.587+0000 7f0404c53640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03f00a2cf0 0x7f03f00a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f03f4000c00 tx=0x7f03f4032d50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:26.608 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.591+0000 7f03fdffb640 1 -- 172.21.15.69:0/1010041737 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f03ec01f020 con 0x7f0400103390 2024-08-02T18:03:26.760 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.758+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0400106f10 con 0x7f0400103390 2024-08-02T18:03:26.760 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:26 smithi069 ceph-mon[32746]: pgmap v535: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:26.761 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.760+0000 7f03fdffb640 1 -- 172.21.15.69:0/1010041737 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+25257 (secure 0 0 0) 0x7f03ec09b980 con 0x7f0400103390 2024-08-02T18:03:26.762 INFO:teuthology.orchestra.run.smithi069.stdout: 2024-08-02T18:03:26.762 INFO:teuthology.orchestra.run.smithi069.stdout:{"epoch":117,"fsid":"b6b04634-50f5-11ef-bcca-c7b262605968","created":"2024-08-02T17:37:04.618643+0000","modified":"2024-08-02T18:03:19.698365+0000","last_up_change":"2024-08-02T17:43:14.668359+0000","last_in_change":"2024-08-02T17:42:54.852898+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-02T17:39:54.984711+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-02T17:44:00.981682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-02T17:44:03.102100+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-02T17:44:09.380545+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":29,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7fad0068-7f9b-4e76-9c10-6338b2fa7619","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6802","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6803","nonce":3281611290}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6804","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6805","nonce":3281611290}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6808","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6809","nonce":3281611290}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6806","nonce":3281611290},{"type":"v1","addr":"172.21.15.69:6807","nonce":3281611290}]},"public_addr":"172.21.15.69:6803/3281611290","cluster_addr":"172.21.15.69:6805/3281611290","heartbeat_back_addr":"172.21.15.69:6809/3281611290","heartbeat_front_addr":"172.21.15.69:6807/3281611290","state":["exists","up"]},{"osd":1,"uuid":"eb73073e-0fd6-4482-b800-03ef39b9c61e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6800","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6801","nonce":1438822900}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6802","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6803","nonce":1438822900}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6806","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6807","nonce":1438822900}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6804","nonce":1438822900},{"type":"v1","addr":"172.21.15.77:6805","nonce":1438822900}]},"public_addr":"172.21.15.77:6801/1438822900","cluster_addr":"172.21.15.77:6803/1438822900","heartbeat_back_addr":"172.21.15.77:6807/1438822900","heartbeat_front_addr":"172.21.15.77:6805/1438822900","state":["exists","up"]},{"osd":2,"uuid":"941227f1-e5e4-4146-9e0d-4405d1f5271c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6801","nonce":2938396779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6803","nonce":2938396779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6807","nonce":2938396779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2938396779},{"type":"v1","addr":"172.21.15.83:6805","nonce":2938396779}]},"public_addr":"172.21.15.83:6801/2938396779","cluster_addr":"172.21.15.83:6803/2938396779","heartbeat_back_addr":"172.21.15.83:6807/2938396779","heartbeat_front_addr":"172.21.15.83:6805/2938396779","state":["exists","up"]},{"osd":3,"uuid":"9482789a-9684-4d80-90d1-4116f82bcce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6810","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6811","nonce":980470189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6812","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6813","nonce":980470189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6816","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6817","nonce":980470189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6814","nonce":980470189},{"type":"v1","addr":"172.21.15.69:6815","nonce":980470189}]},"public_addr":"172.21.15.69:6811/980470189","cluster_addr":"172.21.15.69:6813/980470189","heartbeat_back_addr":"172.21.15.69:6817/980470189","heartbeat_front_addr":"172.21.15.69:6815/980470189","state":["exists","up"]},{"osd":4,"uuid":"e22a3571-a363-4b52-9b84-725cf9b5fe4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6808","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6809","nonce":64984823}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6810","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6811","nonce":64984823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6814","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6815","nonce":64984823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6812","nonce":64984823},{"type":"v1","addr":"172.21.15.77:6813","nonce":64984823}]},"public_addr":"172.21.15.77:6809/64984823","cluster_addr":"172.21.15.77:6811/64984823","heartbeat_back_addr":"172.21.15.77:6815/64984823","heartbeat_front_addr":"172.21.15.77:6813/64984823","state":["exists","up"]},{"osd":5,"uuid":"c3437683-d3b6-4aa5-9d9f-5fc9d3b5cad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6809","nonce":3536789836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6811","nonce":3536789836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6815","nonce":3536789836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3536789836},{"type":"v1","addr":"172.21.15.83:6813","nonce":3536789836}]},"public_addr":"172.21.15.83:6809/3536789836","cluster_addr":"172.21.15.83:6811/3536789836","heartbeat_back_addr":"172.21.15.83:6815/3536789836","heartbeat_front_addr":"172.21.15.83:6813/3536789836","state":["exists","up"]},{"osd":6,"uuid":"e6f58da4-cae1-4c9d-968f-4ab620039158","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6818","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6819","nonce":2370248409}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6820","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6821","nonce":2370248409}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6824","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6825","nonce":2370248409}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6822","nonce":2370248409},{"type":"v1","addr":"172.21.15.69:6823","nonce":2370248409}]},"public_addr":"172.21.15.69:6819/2370248409","cluster_addr":"172.21.15.69:6821/2370248409","heartbeat_back_addr":"172.21.15.69:6825/2370248409","heartbeat_front_addr":"172.21.15.69:6823/2370248409","state":["exists","up"]},{"osd":7,"uuid":"392b17a2-c105-464b-beaa-4f531883e292","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6816","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6817","nonce":2324182206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6818","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6819","nonce":2324182206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6822","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6823","nonce":2324182206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6820","nonce":2324182206},{"type":"v1","addr":"172.21.15.77:6821","nonce":2324182206}]},"public_addr":"172.21.15.77:6817/2324182206","cluster_addr":"172.21.15.77:6819/2324182206","heartbeat_back_addr":"172.21.15.77:6823/2324182206","heartbeat_front_addr":"172.21.15.77:6821/2324182206","state":["exists","up"]},{"osd":8,"uuid":"92a6e799-6b88-4d10-bc05-8a3b5c43c8be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6817","nonce":1048275845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6819","nonce":1048275845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6823","nonce":1048275845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":1048275845},{"type":"v1","addr":"172.21.15.83:6821","nonce":1048275845}]},"public_addr":"172.21.15.83:6817/1048275845","cluster_addr":"172.21.15.83:6819/1048275845","heartbeat_back_addr":"172.21.15.83:6823/1048275845","heartbeat_front_addr":"172.21.15.83:6821/1048275845","state":["exists","up"]},{"osd":9,"uuid":"2ffe6b5e-7950-4db4-8dc5-6d0351b0a346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6826","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6827","nonce":2821083254}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6828","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6829","nonce":2821083254}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6832","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6833","nonce":2821083254}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.69:6830","nonce":2821083254},{"type":"v1","addr":"172.21.15.69:6831","nonce":2821083254}]},"public_addr":"172.21.15.69:6827/2821083254","cluster_addr":"172.21.15.69:6829/2821083254","heartbeat_back_addr":"172.21.15.69:6833/2821083254","heartbeat_front_addr":"172.21.15.69:6831/2821083254","state":["exists","up"]},{"osd":10,"uuid":"88334ccf-216f-4d58-b9e1-6c173f4897ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6824","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6825","nonce":2744845863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6826","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6827","nonce":2744845863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6830","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6831","nonce":2744845863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.77:6828","nonce":2744845863},{"type":"v1","addr":"172.21.15.77:6829","nonce":2744845863}]},"public_addr":"172.21.15.77:6825/2744845863","cluster_addr":"172.21.15.77:6827/2744845863","heartbeat_back_addr":"172.21.15.77:6831/2744845863","heartbeat_front_addr":"172.21.15.77:6829/2744845863","state":["exists","up"]},{"osd":11,"uuid":"2ac94ce4-56aa-4c4c-8c06-d6e61fe63b5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6825","nonce":697474506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6827","nonce":697474506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6831","nonce":697474506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":697474506},{"type":"v1","addr":"172.21.15.83:6829","nonce":697474506}]},"public_addr":"172.21.15.83:6825/697474506","cluster_addr":"172.21.15.83:6827/697474506","heartbeat_back_addr":"172.21.15.83:6831/697474506","heartbeat_front_addr":"172.21.15.83:6829/697474506","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:10.239717+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:31.252766+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:39:51.709563+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:13.396258+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:35.350013+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:40:57.391430+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:19.559139+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:41:42.023003+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:04.443583+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:27.584642+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:42:49.426156+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-02T17:43:12.122041+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":5,"to":8}]},{"pgid":"4.e","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.69:0/3271476957":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3594391539":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/3033851323":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/2680424375":"2024-08-03T17:46:24.900194+0000","172.21.15.69:0/4073447426":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6800/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:6801/452236630":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6801/3960560393":"2024-08-03T17:37:34.617397+0000","172.21.15.69:6801/3376234785":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/623492814":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/2152855317":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2060958760":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/2814849525":"2024-08-03T17:37:56.685064+0000","172.21.15.69:0/3801376647":"2024-08-03T17:37:34.617397+0000","172.21.15.69:0/4036393791":"2024-08-03T17:46:24.900194+0000","172.21.15.69:6800/452236630":"2024-08-03T17:46:24.900194+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]},{"pool":4,"snaps":[{"begin":16,"length":1},{"begin":29,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-02T18:03:26.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03f00a2cf0 msgr2=0x7f03f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:26.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03f00a2cf0 0x7f03f00a51b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f03f4000c00 tx=0x7f03f4032d50 comp rx=0 tx=0).stop 2024-08-02T18:03:26.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400103390 msgr2=0x7f0400100c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:26.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400103390 0x7f0400100c10 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00b3f0 tx=0x7f03ec00b8b0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 shutdown_connections 2024-08-02T18:03:26.764 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f0400103390 0x7f0400100c10 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f03f00a2cf0 0x7f03f00a51b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f040010bea0 0x7f0400101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 --2- 172.21.15.69:0/1010041737 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f0400102a70 0x7f04001006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:26.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 >> 172.21.15.69:0/1010041737 conn(0x7f04000fc4d0 msgr2=0x7f04000fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:26.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 shutdown_connections 2024-08-02T18:03:26.765 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:26.762+0000 7f0406eb7640 1 -- 172.21.15.69:0/1010041737 wait complete. 2024-08-02T18:03:27.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:26 smithi083 ceph-mon[37031]: pgmap v535: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:27.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:26 smithi077 ceph-mon[37108]: pgmap v535: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:27.466 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs fail cephfs 2024-08-02T18:03:27.724 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:27.978 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:27 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/1010041737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T18:03:28.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:27 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/1010041737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T18:03:28.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:27 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/1010041737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-08-02T18:03:28.193 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.189+0000 7f17d4765640 1 -- 172.21.15.69:0/3738326117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc102c20 msgr2=0x7f17cc10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.189+0000 7f17d4765640 1 --2- 172.21.15.69:0/3738326117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc102c20 0x7f17cc10d770 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009ca0 tx=0x7f17c802a950 comp rx=0 tx=0).stop 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 -- 172.21.15.69:0/3738326117 shutdown_connections 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 --2- 172.21.15.69:0/3738326117 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc102c20 0x7f17cc10d770 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 --2- 172.21.15.69:0/3738326117 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17cc1022a0 0x7f17cc1026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 --2- 172.21.15.69:0/3738326117 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17cc100b50 0x7f17cc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 -- 172.21.15.69:0/3738326117 >> 172.21.15.69:0/3738326117 conn(0x7f17cc0fc500 msgr2=0x7f17cc0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 -- 172.21.15.69:0/3738326117 shutdown_connections 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 -- 172.21.15.69:0/3738326117 wait complete. 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 Processor -- start 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 -- start start 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17cc100b50 0x7f17cc0fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:28.194 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc1022a0 0x7f17cc0fcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17cc102c20 0x7f17cc0fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc110cb0 con 0x7f17cc1022a0 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc110b30 con 0x7f17cc100b50 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d4765640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc110e30 con 0x7f17cc102c20 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.190+0000 7f17d1d00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc1022a0 0x7f17cc0fcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d1d00640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc1022a0 0x7f17cc0fcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42878/0 (socket says 172.21.15.69:42878) 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d1d00640 1 -- 172.21.15.69:0/2324647251 learned_addr learned my addr 172.21.15.69:0/2324647251 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d2d02640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17cc102c20 0x7f17cc0fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d2501640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17cc100b50 0x7f17cc0fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d1d00640 1 -- 172.21.15.69:0/2324647251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17cc102c20 msgr2=0x7f17cc0fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d1d00640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17cc102c20 0x7f17cc0fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d1d00640 1 -- 172.21.15.69:0/2324647251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17cc100b50 msgr2=0x7f17cc0fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d1d00640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17cc100b50 0x7f17cc0fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.195 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d1d00640 1 -- 172.21.15.69:0/2324647251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc1a6670 con 0x7f17cc1022a0 2024-08-02T18:03:28.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d1d00640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc1022a0 0x7f17cc0fcef0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00d950 tx=0x7f17bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:28.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17c37fe640 1 -- 172.21.15.69:0/2324647251 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc017030 con 0x7f17cc1022a0 2024-08-02T18:03:28.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17c37fe640 1 -- 172.21.15.69:0/2324647251 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17bc004020 con 0x7f17cc1022a0 2024-08-02T18:03:28.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17c37fe640 1 -- 172.21.15.69:0/2324647251 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0047a0 con 0x7f17cc1022a0 2024-08-02T18:03:28.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc1a6960 con 0x7f17cc1022a0 2024-08-02T18:03:28.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.191+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1a6ea0 con 0x7f17cc1022a0 2024-08-02T18:03:28.196 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.192+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc100f30 con 0x7f17cc1022a0 2024-08-02T18:03:28.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.193+0000 7f17c37fe640 1 -- 172.21.15.69:0/2324647251 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f17bc0041d0 con 0x7f17cc1022a0 2024-08-02T18:03:28.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.194+0000 7f17c37fe640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f17b80a2cf0 0x7f17b80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:28.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.194+0000 7f17d2501640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f17b80a2cf0 0x7f17b80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:28.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.194+0000 7f17c37fe640 1 -- 172.21.15.69:0/2324647251 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9568+0+0 (secure 0 0 0) 0x7f17bc0cf840 con 0x7f17cc1022a0 2024-08-02T18:03:28.197 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.195+0000 7f17d2501640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f17b80a2cf0 0x7f17b80a51b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005fd0 tx=0x7f17c4005850 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:28.199 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.197+0000 7f17c37fe640 1 -- 172.21.15.69:0/2324647251 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f17bc09c050 con 0x7f17cc1022a0 2024-08-02T18:03:28.401 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.400+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f17cc0fdee0 con 0x7f17cc1022a0 2024-08-02T18:03:28.403 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.402+0000 7f17c37fe640 1 -- 172.21.15.69:0/2324647251 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) ==== 161+0+0 (secure 0 0 0) 0x7f17cc0fdee0 con 0x7f17cc1022a0 2024-08-02T18:03:28.404 INFO:teuthology.orchestra.run.smithi069.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-02T18:03:28.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.405+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f17b80a2cf0 msgr2=0x7f17b80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:28.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f17b80a2cf0 0x7f17b80a51b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005fd0 tx=0x7f17c4005850 comp rx=0 tx=0).stop 2024-08-02T18:03:28.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc1022a0 msgr2=0x7f17cc0fcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:28.407 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc1022a0 0x7f17cc0fcef0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00d950 tx=0x7f17bc00de10 comp rx=0 tx=0).stop 2024-08-02T18:03:28.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 shutdown_connections 2024-08-02T18:03:28.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f17cc1022a0 0x7f17cc0fcef0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f17b80a2cf0 0x7f17b80a51b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17cc102c20 0x7f17cc0fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 --2- 172.21.15.69:0/2324647251 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f17cc100b50 0x7f17cc0fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:28.408 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.406+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 >> 172.21.15.69:0/2324647251 conn(0x7f17cc0fc500 msgr2=0x7f17cc069f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:28.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.407+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 shutdown_connections 2024-08-02T18:03:28.409 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:28.407+0000 7f17d4765640 1 -- 172.21.15.69:0/2324647251 wait complete. 2024-08-02T18:03:28.724 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:28 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a[32742]: 2024-08-02T18:03:28.401+0000 7f13cb02d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-02T18:03:28.730 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h[66464]: ignoring --setuser ceph since I am not root 2024-08-02T18:03:28.730 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h[66464]: ignoring --setgroup ceph since I am not root 2024-08-02T18:03:28.730 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mds[66468]: -- 172.21.15.77:0/2219857732 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b09b97860 con 0x562b09b74800 2024-08-02T18:03:28.730 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h[66464]: starting mds.h at 2024-08-02T18:03:28.730 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k[65849]: ignoring --setuser ceph since I am not root 2024-08-02T18:03:28.730 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k[65849]: ignoring --setgroup ceph since I am not root 2024-08-02T18:03:28.731 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mds[65853]: -- 172.21.15.77:0/2583464387 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ceb9963c20 con 0x55ceb9940800 2024-08-02T18:03:28.731 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k[65849]: starting mds.k at 2024-08-02T18:03:28.731 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b[66767]: ignoring --setuser ceph since I am not root 2024-08-02T18:03:28.731 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b[66767]: ignoring --setgroup ceph since I am not root 2024-08-02T18:03:28.731 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mds[66771]: -- 172.21.15.77:0/1219859269 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c14ab81860 con 0x55c14ab5e800 2024-08-02T18:03:28.731 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b[66767]: starting mds.b at 2024-08-02T18:03:28.771 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f[66697]: ignoring --setuser ceph since I am not root 2024-08-02T18:03:28.771 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f[66697]: ignoring --setgroup ceph since I am not root 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mds[66706]: -- 172.21.15.83:0/650267734 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561709741860 con 0x56170971f000 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f[66697]: starting mds.f at 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mon[37031]: pgmap v536: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2324647251' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mon[37031]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mon[37031]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2324647251' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mon[37031]: osdmap e118: 12 total, 12 up, 12 in 2024-08-02T18:03:28.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mon[37031]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-02T18:03:28.773 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c[67010]: ignoring --setuser ceph since I am not root 2024-08-02T18:03:28.773 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c[67010]: ignoring --setgroup ceph since I am not root 2024-08-02T18:03:28.773 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mds[67014]: -- 172.21.15.83:0/3166751516 <== mon.2 v2:172.21.15.77:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5592e52bd860 con 0x5592e529a800 2024-08-02T18:03:28.773 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c[67010]: starting mds.c at 2024-08-02T18:03:28.773 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i[67315]: ignoring --setuser ceph since I am not root 2024-08-02T18:03:28.773 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i[67315]: ignoring --setgroup ceph since I am not root 2024-08-02T18:03:28.773 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-mds[67319]: -- 172.21.15.83:0/222675164 <== mon.0 v2:172.21.15.69:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563487085860 con 0x563487063000 2024-08-02T18:03:28.773 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:03:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i[67315]: starting mds.i at 2024-08-02T18:03:28.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:28 smithi069 ceph-mon[32746]: pgmap v536: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:28.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:28 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2324647251' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-02T18:03:28.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:28 smithi069 ceph-mon[32746]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-02T18:03:28.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:28 smithi069 ceph-mon[32746]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-02T18:03:28.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:28 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2324647251' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-02T18:03:28.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:28 smithi069 ceph-mon[32746]: osdmap e118: 12 total, 12 up, 12 in 2024-08-02T18:03:28.992 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:28 smithi069 ceph-mon[32746]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-02T18:03:29.021 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-02T18:03:29.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mon[37108]: pgmap v536: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:29.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2324647251' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-08-02T18:03:29.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mon[37108]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-02T18:03:29.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mon[37108]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-02T18:03:29.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2324647251' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-02T18:03:29.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mon[37108]: osdmap e118: 12 total, 12 up, 12 in 2024-08-02T18:03:29.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:28 smithi077 ceph-mon[37108]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-02T18:03:29.276 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:29.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 -- 172.21.15.69:0/3988579403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 msgr2=0x7f7364101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:29.795 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 --2- 172.21.15.69:0/3988579403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 0x7f7364101d10 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f734c009930 tx=0x7f734c02a420 comp rx=0 tx=0).stop 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 -- 172.21.15.69:0/3988579403 shutdown_connections 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 --2- 172.21.15.69:0/3988579403 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 0x7f7364101d10 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 --2- 172.21.15.69:0/3988579403 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f7364102250 0x7f736410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 --2- 172.21.15.69:0/3988579403 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f73641078b0 0x7f7364107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 -- 172.21.15.69:0/3988579403 >> 172.21.15.69:0/3988579403 conn(0x7f73640fd5f0 msgr2=0x7f73640ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 -- 172.21.15.69:0/3988579403 shutdown_connections 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.785+0000 7f7368a59640 1 -- 172.21.15.69:0/3988579403 wait complete. 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7368a59640 1 Processor -- start 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7368a59640 1 -- start start 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7368a59640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 0x7f73641980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7368a59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7364102250 0x7f73641985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:29.796 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7368a59640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73641078b0 0x7f736419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7368a59640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f7364111c70 con 0x7f73641018b0 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7368a59640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f7364111af0 con 0x7f73641078b0 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7368a59640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7364111df0 con 0x7f7364102250 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 0x7f73641980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 0x7f73641980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42896/0 (socket says 172.21.15.69:42896) 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 -- 172.21.15.69:0/3944979900 learned_addr learned my addr 172.21.15.69:0/3944979900 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f735bfff640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7364102250 0x7f73641985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7362d9d640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73641078b0 0x7f736419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 -- 172.21.15.69:0/3944979900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7364102250 msgr2=0x7f73641985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7364102250 0x7f73641985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 -- 172.21.15.69:0/3944979900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73641078b0 msgr2=0x7f736419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73641078b0 0x7f736419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 -- 172.21.15.69:0/3944979900 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f736419d050 con 0x7f73641018b0 2024-08-02T18:03:29.797 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f7362d9d640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73641078b0 0x7f736419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.786+0000 7f736259c640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 0x7f73641980a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f735400c920 tx=0x7f735400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.787+0000 7f735b7fe640 1 -- 172.21.15.69:0/3944979900 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354007c70 con 0x7f73641018b0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.787+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f736419d340 con 0x7f73641018b0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.787+0000 7f735b7fe640 1 -- 172.21.15.69:0/3944979900 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7354007e10 con 0x7f73641018b0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.787+0000 7f735b7fe640 1 -- 172.21.15.69:0/3944979900 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7354018890 con 0x7f73641018b0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.787+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73641a4c20 con 0x7f73641018b0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.788+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7364101d10 con 0x7f73641018b0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.789+0000 7f735b7fe640 1 -- 172.21.15.69:0/3944979900 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7354018a70 con 0x7f73641018b0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.789+0000 7f735b7fe640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73500a2cf0 0x7f73500a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.790+0000 7f735bfff640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73500a2cf0 0x7f73500a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.790+0000 7f735b7fe640 1 -- 172.21.15.69:0/3944979900 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9738+0+0 (secure 0 0 0) 0x7f73540cfa10 con 0x7f73641018b0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.790+0000 7f735bfff640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73500a2cf0 0x7f73500a51b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f734c009900 tx=0x7f734c0058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:29.798 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.794+0000 7f735b7fe640 1 -- 172.21.15.69:0/3944979900 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f7354014030 con 0x7f73641018b0 2024-08-02T18:03:29.997 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:29.995+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f73641a4e50 con 0x7f73641018b0 2024-08-02T18:03:30.735 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.733+0000 7f735b7fe640 1 -- 172.21.15.69:0/3944979900 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f735409c170 con 0x7f73641018b0 2024-08-02T18:03:30.735 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: pgmap v538: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6833/2688033297,v1:172.21.15.77:6837/2688033297] up:boot 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6832/1032759013,v1:172.21.15.77:6836/1032759013] up:boot 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.77:6838/3690508086,v1:172.21.15.77:6839/3690508086] up:boot 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.83:6834/5877019,v1:172.21.15.83:6835/5877019] up:boot 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.83:6837/3184715339,v1:172.21.15.83:6839/3184715339] up:boot 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: mds.? [v2:172.21.15.83:6836/2851910546,v1:172.21.15.83:6838/2851910546] up:boot 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T18:03:30.736 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T18:03:30.737 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:30 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3944979900' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-02T18:03:30.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.735+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73500a2cf0 msgr2=0x7f73500a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:30.737 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.735+0000 7f7368a59640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73500a2cf0 0x7f73500a51b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f734c009900 tx=0x7f734c0058d0 comp rx=0 tx=0).stop 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.735+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 msgr2=0x7f73641980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.735+0000 7f7368a59640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 0x7f73641980a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f735400c920 tx=0x7f735400cde0 comp rx=0 tx=0).stop 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.736+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 shutdown_connections 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.736+0000 7f7368a59640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f73500a2cf0 0x7f73500a51b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.736+0000 7f7368a59640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f73641078b0 0x7f736419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.736+0000 7f7368a59640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7364102250 0x7f73641985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.736+0000 7f7368a59640 1 --2- 172.21.15.69:0/3944979900 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f73641018b0 0x7f73641980a0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.736+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 >> 172.21.15.69:0/3944979900 conn(0x7f73640fd5f0 msgr2=0x7f73640fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.736+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 shutdown_connections 2024-08-02T18:03:30.738 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:30.736+0000 7f7368a59640 1 -- 172.21.15.69:0/3944979900 wait complete. 2024-08-02T18:03:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: pgmap v538: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6833/2688033297,v1:172.21.15.77:6837/2688033297] up:boot 2024-08-02T18:03:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6832/1032759013,v1:172.21.15.77:6836/1032759013] up:boot 2024-08-02T18:03:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.77:6838/3690508086,v1:172.21.15.77:6839/3690508086] up:boot 2024-08-02T18:03:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.83:6834/5877019,v1:172.21.15.83:6835/5877019] up:boot 2024-08-02T18:03:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.83:6837/3184715339,v1:172.21.15.83:6839/3184715339] up:boot 2024-08-02T18:03:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: mds.? [v2:172.21.15.83:6836/2851910546,v1:172.21.15.83:6838/2851910546] up:boot 2024-08-02T18:03:30.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-02T18:03:30.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T18:03:30.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T18:03:30.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T18:03:30.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T18:03:30.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T18:03:30.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T18:03:30.772 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:30 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3944979900' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-02T18:03:30.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: pgmap v538: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6833/2688033297,v1:172.21.15.77:6837/2688033297] up:boot 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6832/1032759013,v1:172.21.15.77:6836/1032759013] up:boot 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.77:6838/3690508086,v1:172.21.15.77:6839/3690508086] up:boot 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.83:6834/5877019,v1:172.21.15.83:6835/5877019] up:boot 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.83:6837/3184715339,v1:172.21.15.83:6839/3184715339] up:boot 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: mds.? [v2:172.21.15.83:6836/2851910546,v1:172.21.15.83:6838/2851910546] up:boot 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-08-02T18:03:30.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: from='mgr.15192 172.21.15.69:0/184759365' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-08-02T18:03:30.849 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:30 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3944979900' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-08-02T18:03:31.403 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-02T18:03:31.699 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:31.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:31 smithi083 ceph-mon[37031]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-02T18:03:31.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:31 smithi083 ceph-mon[37031]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-02T18:03:31.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:31 smithi083 ceph-mon[37031]: Cluster is now healthy 2024-08-02T18:03:31.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:31 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3944979900' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-02T18:03:31.771 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:31 smithi083 ceph-mon[37031]: fsmap 12 up:standby 2024-08-02T18:03:31.820 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:31 smithi069 ceph-mon[32746]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-02T18:03:31.820 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:31 smithi069 ceph-mon[32746]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-02T18:03:31.820 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:31 smithi069 ceph-mon[32746]: Cluster is now healthy 2024-08-02T18:03:31.820 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:31 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3944979900' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-02T18:03:31.820 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:31 smithi069 ceph-mon[32746]: fsmap 12 up:standby 2024-08-02T18:03:31.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:31 smithi077 ceph-mon[37108]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-02T18:03:31.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:31 smithi077 ceph-mon[37108]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-02T18:03:31.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:31 smithi077 ceph-mon[37108]: Cluster is now healthy 2024-08-02T18:03:31.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:31 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3944979900' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-02T18:03:31.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:31 smithi077 ceph-mon[37108]: fsmap 12 up:standby 2024-08-02T18:03:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.211+0000 7fe0072d5640 1 -- 172.21.15.69:0/3324583305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 msgr2=0x7fe000112390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:32.215 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.211+0000 7fe0072d5640 1 --2- 172.21.15.69:0/3324583305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 0x7fe000112390 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009900 tx=0x7fdff402a480 comp rx=0 tx=0).stop 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.211+0000 7fe0072d5640 1 -- 172.21.15.69:0/3324583305 shutdown_connections 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.211+0000 7fe0072d5640 1 --2- 172.21.15.69:0/3324583305 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 0x7fe000112390 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.211+0000 7fe0072d5640 1 --2- 172.21.15.69:0/3324583305 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0001128d0 0x7fe000114cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 --2- 172.21.15.69:0/3324583305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000742c0 0x7fe0000746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 -- 172.21.15.69:0/3324583305 >> 172.21.15.69:0/3324583305 conn(0x7fe0000fd5e0 msgr2=0x7fe0000ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 -- 172.21.15.69:0/3324583305 shutdown_connections 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 -- 172.21.15.69:0/3324583305 wait complete. 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 Processor -- start 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 -- start start 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000742c0 0x7fe0001a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 0x7fe0001a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0001128d0 0x7fe00019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:32.216 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7fe000118330 con 0x7fe000074be0 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001181b0 con 0x7fe0001128d0 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.212+0000 7fe0072d5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe0001184b0 con 0x7fe0000742c0 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 0x7fe0001a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 0x7fe0001a4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42918/0 (socket says 172.21.15.69:42918) 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 -- 172.21.15.69:0/154484513 learned_addr learned my addr 172.21.15.69:0/154484513 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe005872640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0001128d0 0x7fe00019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe005071640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000742c0 0x7fe0001a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 -- 172.21.15.69:0/154484513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000742c0 msgr2=0x7fe0001a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000742c0 0x7fe0001a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 -- 172.21.15.69:0/154484513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0001128d0 msgr2=0x7fe00019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0001128d0 0x7fe00019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.217 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 -- 172.21.15.69:0/154484513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00019eee0 con 0x7fe000074be0 2024-08-02T18:03:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe004870640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 0x7fe0001a4ae0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fdff4006010 tx=0x7fdff40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fdff27fc640 1 -- 172.21.15.69:0/154484513 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff403f030 con 0x7fe000074be0 2024-08-02T18:03:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00019f170 con 0x7fe000074be0 2024-08-02T18:03:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fdff27fc640 1 -- 172.21.15.69:0/154484513 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff4003ce0 con 0x7fe000074be0 2024-08-02T18:03:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fdff27fc640 1 -- 172.21.15.69:0/154484513 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4035070 con 0x7fe000074be0 2024-08-02T18:03:32.218 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.213+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0001ab350 con 0x7fe000074be0 2024-08-02T18:03:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.214+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe000075a00 con 0x7fe000074be0 2024-08-02T18:03:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.215+0000 7fdff27fc640 1 -- 172.21.15.69:0/154484513 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdff4042030 con 0x7fe000074be0 2024-08-02T18:03:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.216+0000 7fdff27fc640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfec0a2cf0 0x7fdfec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.216+0000 7fe005071640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfec0a2cf0 0x7fdfec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.216+0000 7fdff27fc640 1 -- 172.21.15.69:0/154484513 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9738+0+0 (secure 0 0 0) 0x7fdff40efe90 con 0x7fe000074be0 2024-08-02T18:03:32.219 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.217+0000 7fe005071640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfec0a2cf0 0x7fdfec0a51b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdffc005fd0 tx=0x7fdffc00b390 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:32.222 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.220+0000 7fdff27fc640 1 -- 172.21.15.69:0/154484513 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7fdff40bc060 con 0x7fe000074be0 2024-08-02T18:03:32.389 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.386+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe000104d10 con 0x7fe000074be0 2024-08-02T18:03:32.742 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.740+0000 7fdff27fc640 1 -- 172.21.15.69:0/154484513 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) ==== 181+0+0 (secure 0 0 0) 0x7fdff40c0360 con 0x7fe000074be0 2024-08-02T18:03:32.742 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_metadata' removed 2024-08-02T18:03:32.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.743+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfec0a2cf0 msgr2=0x7fdfec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:32.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.743+0000 7fe0072d5640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfec0a2cf0 0x7fdfec0a51b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdffc005fd0 tx=0x7fdffc00b390 comp rx=0 tx=0).stop 2024-08-02T18:03:32.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.743+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 msgr2=0x7fe0001a4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:32.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.743+0000 7fe0072d5640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 0x7fe0001a4ae0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fdff4006010 tx=0x7fdff40042f0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.743+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 shutdown_connections 2024-08-02T18:03:32.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.744+0000 7fe0072d5640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7fe000074be0 0x7fe0001a4ae0 secure :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fdff4006010 tx=0x7fdff40042f0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.745 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.744+0000 7fe0072d5640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7fdfec0a2cf0 0x7fdfec0a51b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.744+0000 7fe0072d5640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7fe0001128d0 0x7fe00019e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.744+0000 7fe0072d5640 1 --2- 172.21.15.69:0/154484513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe0000742c0 0x7fe0001a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:32.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.744+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 >> 172.21.15.69:0/154484513 conn(0x7fe0000fd5e0 msgr2=0x7fe0000fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:32.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.744+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 shutdown_connections 2024-08-02T18:03:32.746 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:32.745+0000 7fe0072d5640 1 -- 172.21.15.69:0/154484513 wait complete. 2024-08-02T18:03:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:32 smithi083 ceph-mon[37031]: pgmap v539: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-02T18:03:33.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:32 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/154484513' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:33.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:32 smithi077 ceph-mon[37108]: pgmap v539: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-02T18:03:33.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:32 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/154484513' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:32 smithi069 ceph-mon[32746]: pgmap v539: 129 pgs: 129 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-02T18:03:33.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:32 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/154484513' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:33.436 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-02T18:03:33.697 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:34.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:33 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/154484513' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:34.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:33 smithi083 ceph-mon[37031]: osdmap e119: 12 total, 12 up, 12 in 2024-08-02T18:03:34.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:33 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/154484513' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:34.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:33 smithi077 ceph-mon[37108]: osdmap e119: 12 total, 12 up, 12 in 2024-08-02T18:03:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:33 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/154484513' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:34.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:33 smithi069 ceph-mon[32746]: osdmap e119: 12 total, 12 up, 12 in 2024-08-02T18:03:34.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 -- 172.21.15.69:0/2821907717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 msgr2=0x7f5108101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:34.203 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 --2- 172.21.15.69:0/2821907717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 0x7f5108101d10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009930 tx=0x7f50f802a420 comp rx=0 tx=0).stop 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 -- 172.21.15.69:0/2821907717 shutdown_connections 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 --2- 172.21.15.69:0/2821907717 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 0x7f5108101d10 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 --2- 172.21.15.69:0/2821907717 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5108102250 0x7f510810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 --2- 172.21.15.69:0/2821907717 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51081078b0 0x7f5108107c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 -- 172.21.15.69:0/2821907717 >> 172.21.15.69:0/2821907717 conn(0x7f51080fd5f0 msgr2=0x7f51080ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 -- 172.21.15.69:0/2821907717 shutdown_connections 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 -- 172.21.15.69:0/2821907717 wait complete. 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.193+0000 7f5110317640 1 Processor -- start 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f5110317640 1 -- start start 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f5110317640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 0x7f51081981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f5110317640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5108102250 0x7f5108198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:34.204 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f5110317640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51081078b0 0x7f510819caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f5110317640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5108111cd0 con 0x7f51081018b0 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f5110317640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5108111b50 con 0x7f5108102250 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f5110317640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5108111e50 con 0x7f51081078b0 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 0x7f51081981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 0x7f51081981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:42942/0 (socket says 172.21.15.69:42942) 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 -- 172.21.15.69:0/3620688269 learned_addr learned my addr 172.21.15.69:0/3620688269 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510d8b2640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5108102250 0x7f5108198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 -- 172.21.15.69:0/3620688269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51081078b0 msgr2=0x7f510819caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51081078b0 0x7f510819caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 -- 172.21.15.69:0/3620688269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5108102250 msgr2=0x7f5108198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5108102250 0x7f5108198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 -- 172.21.15.69:0/3620688269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510819d180 con 0x7f51081018b0 2024-08-02T18:03:34.205 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510d8b2640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5108102250 0x7f5108198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-02T18:03:34.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f510e0b3640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 0x7f51081981d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f510400b4a0 tx=0x7f510400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:34.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.194+0000 7f50ff7fe640 1 -- 172.21.15.69:0/3620688269 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104014070 con 0x7f51081018b0 2024-08-02T18:03:34.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.195+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510819d470 con 0x7f51081018b0 2024-08-02T18:03:34.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.195+0000 7f50ff7fe640 1 -- 172.21.15.69:0/3620688269 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5104010af0 con 0x7f51081018b0 2024-08-02T18:03:34.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.195+0000 7f50ff7fe640 1 -- 172.21.15.69:0/3620688269 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5104017d10 con 0x7f51081018b0 2024-08-02T18:03:34.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.195+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51081a4d50 con 0x7f51081018b0 2024-08-02T18:03:34.206 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.196+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5108101d10 con 0x7f51081018b0 2024-08-02T18:03:34.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.197+0000 7f50ff7fe640 1 -- 172.21.15.69:0/3620688269 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f5104017070 con 0x7f51081018b0 2024-08-02T18:03:34.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.198+0000 7f50ff7fe640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50f40a2cf0 0x7f50f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:34.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.198+0000 7f510d8b2640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50f40a2cf0 0x7f50f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:34.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.198+0000 7f50ff7fe640 1 -- 172.21.15.69:0/3620688269 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9184+0+0 (secure 0 0 0) 0x7f51040cea50 con 0x7f51081018b0 2024-08-02T18:03:34.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.198+0000 7f510d8b2640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50f40a2cf0 0x7f50f40a51b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f50f8000c00 tx=0x7f50f8032d50 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:34.207 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.202+0000 7f50ff7fe640 1 -- 172.21.15.69:0/3620688269 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f510409b4f0 con 0x7f51081018b0 2024-08-02T18:03:34.371 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.369+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f510802d5e0 con 0x7f51081018b0 2024-08-02T18:03:34.751 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.749+0000 7f50ff7fe640 1 -- 172.21.15.69:0/3620688269 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v120) ==== 169+0+0 (secure 0 0 0) 0x7f510409adf0 con 0x7f51081018b0 2024-08-02T18:03:34.751 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data' removed 2024-08-02T18:03:34.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50f40a2cf0 msgr2=0x7f50f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:34.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50f40a2cf0 0x7f50f40a51b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f50f8000c00 tx=0x7f50f8032d50 comp rx=0 tx=0).stop 2024-08-02T18:03:34.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 msgr2=0x7f51081981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:34.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 0x7f51081981d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f510400b4a0 tx=0x7f510400b960 comp rx=0 tx=0).stop 2024-08-02T18:03:34.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 shutdown_connections 2024-08-02T18:03:34.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f50f40a2cf0 0x7f50f40a51b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.753 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f51081078b0 0x7f510819caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5108102250 0x7f5108198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 --2- 172.21.15.69:0/3620688269 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f51081018b0 0x7f51081981d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:34.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.751+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 >> 172.21.15.69:0/3620688269 conn(0x7f51080fd5f0 msgr2=0x7f51080fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:34.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.752+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 shutdown_connections 2024-08-02T18:03:34.754 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:34.752+0000 7f5110317640 1 -- 172.21.15.69:0/3620688269 wait complete. 2024-08-02T18:03:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:34 smithi083 ceph-mon[37031]: pgmap v541: 97 pgs: 97 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T18:03:35.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:34 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3620688269' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:35.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:34 smithi077 ceph-mon[37108]: pgmap v541: 97 pgs: 97 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T18:03:35.098 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:34 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3620688269' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:34 smithi069 ceph-mon[32746]: pgmap v541: 97 pgs: 97 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T18:03:35.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:34 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3620688269' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:35.436 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea shell --fsid b6b04634-50f5-11ef-bcca-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-08-02T18:03:35.690 INFO:teuthology.orchestra.run.smithi069.stderr:Inferring config /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/mon.a/config 2024-08-02T18:03:36.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:35 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/3620688269' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:36.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:35 smithi083 ceph-mon[37031]: osdmap e120: 12 total, 12 up, 12 in 2024-08-02T18:03:36.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:35 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/3620688269' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:36.078 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:35 smithi069 ceph-mon[32746]: osdmap e120: 12 total, 12 up, 12 in 2024-08-02T18:03:36.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:35 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/3620688269' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:36.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:35 smithi077 ceph-mon[37108]: osdmap e120: 12 total, 12 up, 12 in 2024-08-02T18:03:36.353 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.299+0000 7f55580c9640 1 -- 172.21.15.69:0/2117882970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 msgr2=0x7f5550106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.299+0000 7f55580c9640 1 --2- 172.21.15.69:0/2117882970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 0x7f5550106b90 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f554c009930 tx=0x7f554c02a420 comp rx=0 tx=0).stop 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.299+0000 7f55580c9640 1 -- 172.21.15.69:0/2117882970 shutdown_connections 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.299+0000 7f55580c9640 1 --2- 172.21.15.69:0/2117882970 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5550101150 0x7f555010d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.299+0000 7f55580c9640 1 --2- 172.21.15.69:0/2117882970 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55501007b0 0x7f5550100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.299+0000 7f55580c9640 1 --2- 172.21.15.69:0/2117882970 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 0x7f5550106b90 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.299+0000 7f55580c9640 1 -- 172.21.15.69:0/2117882970 >> 172.21.15.69:0/2117882970 conn(0x7f55500fc4d0 msgr2=0x7f55500fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 -- 172.21.15.69:0/2117882970 shutdown_connections 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 -- 172.21.15.69:0/2117882970 wait complete. 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 Processor -- start 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 -- start start 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55501007b0 0x7f5550102380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:36.354 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 --2- >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5550101150 0x7f55501028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 0x7f5550102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 -- --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_getmap magic: 0 -- 0x7f5550110bb0 con 0x7f55501067b0 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 -- --> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] -- mon_getmap magic: 0 -- 0x7f5550110a30 con 0x7f5550101150 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.300+0000 7f55580c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5550110d30 con 0x7f55501007b0 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 0x7f5550102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 --2- >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 0x7f5550102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.69:3300/0 says I am v2:172.21.15.69:51166/0 (socket says 172.21.15.69:51166) 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 -- 172.21.15.69:0/2877089890 learned_addr learned my addr 172.21.15.69:0/2877089890 (peer_addr_for_me v2:172.21.15.69:0/0) 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5555e65640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55501007b0 0x7f5550102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5555664640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5550101150 0x7f55501028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 -- 172.21.15.69:0/2877089890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55501007b0 msgr2=0x7f5550102380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55501007b0 0x7f5550102380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 -- 172.21.15.69:0/2877089890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5550101150 msgr2=0x7f55501028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5550101150 0x7f55501028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.355 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 -- 172.21.15.69:0/2877089890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55501a6660 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5556666640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 0x7f5550102e00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f554800cad0 tx=0x7f554800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5546ffd640 1 -- 172.21.15.69:0/2877089890 <== mon.0 v2:172.21.15.69:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554800ae20 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5546ffd640 1 -- 172.21.15.69:0/2877089890 <== mon.0 v2:172.21.15.69:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5548005020 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55501a6950 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f5546ffd640 1 -- 172.21.15.69:0/2877089890 <== mon.0 v2:172.21.15.69:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554800f6f0 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.301+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55501a6e40 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.302+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5550100c10 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.304+0000 7f5546ffd640 1 -- 172.21.15.69:0/2877089890 <== mon.0 v2:172.21.15.69:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f55480051d0 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.304+0000 7f5546ffd640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f553c0a2cf0 0x7f553c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.304+0000 7f5546ffd640 1 -- 172.21.15.69:0/2877089890 <== mon.0 v2:172.21.15.69:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8625+0+0 (secure 0 0 0) 0x7f55480cf790 con 0x7f55501067b0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.304+0000 7f5555e65640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f553c0a2cf0 0x7f553c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.305+0000 7f5555e65640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f553c0a2cf0 0x7f553c0a51b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f554c009900 tx=0x7f554c0058d0 comp rx=0 tx=0).ready entity=mgr.15192 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-02T18:03:36.356 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.308+0000 7f5546ffd640 1 -- 172.21.15.69:0/2877089890 <== mon.0 v2:172.21.15.69:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187871 (secure 0 0 0) 0x7f5548014030 con 0x7f55501067b0 2024-08-02T18:03:36.476 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.475+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 --> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f55501a7130 con 0x7f55501067b0 2024-08-02T18:03:36.763 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.762+0000 7f5546ffd640 1 -- 172.21.15.69:0/2877089890 <== mon.0 v2:172.21.15.69:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v121) ==== 178+0+0 (secure 0 0 0) 0x7f554809bcf0 con 0x7f55501067b0 2024-08-02T18:03:36.764 INFO:teuthology.orchestra.run.smithi069.stderr:pool 'cephfs_data_ec' removed 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f553c0a2cf0 msgr2=0x7f553c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f553c0a2cf0 0x7f553c0a51b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f554c009900 tx=0x7f554c0058d0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 msgr2=0x7f5550102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 0x7f5550102e00 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f554800cad0 tx=0x7f554800cf90 comp rx=0 tx=0).stop 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 shutdown_connections 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:3300/0,v1:172.21.15.69:6789/0] conn(0x7f55501067b0 0x7f5550102e00 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.69:6800/2335326389,v1:172.21.15.69:6801/2335326389] conn(0x7f553c0a2cf0 0x7f553c0a51b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.77:3300/0,v1:172.21.15.77:6789/0] conn(0x7f5550101150 0x7f55501028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 --2- 172.21.15.69:0/2877089890 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f55501007b0 0x7f5550102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 >> 172.21.15.69:0/2877089890 conn(0x7f55500fc4d0 msgr2=0x7f555006a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 shutdown_connections 2024-08-02T18:03:36.767 INFO:teuthology.orchestra.run.smithi069.stderr:2024-08-02T18:03:36.765+0000 7f55580c9640 1 -- 172.21.15.69:0/2877089890 wait complete. 2024-08-02T18:03:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:36 smithi077 ceph-mon[37108]: pgmap v543: 33 pgs: 33 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T18:03:37.097 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:36 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2877089890' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:36 smithi069 ceph-mon[32746]: pgmap v543: 33 pgs: 33 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T18:03:37.108 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:36 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2877089890' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:36 smithi083 ceph-mon[37031]: pgmap v543: 33 pgs: 33 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-02T18:03:37.271 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:36 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2877089890' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-08-02T18:03:37.451 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-02T18:03:37.503 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-02T18:03:37.547 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-02T18:03:37.586 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-02T18:03:37.586 DEBUG:teuthology.orchestra.run.smithi069:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-02T18:03:37.621 DEBUG:teuthology.orchestra.run.smithi077:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-02T18:03:37.651 DEBUG:teuthology.orchestra.run.smithi083:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-02T18:03:37.674 INFO:tasks.cephadm:Stopping all daemons... 2024-08-02T18:03:37.674 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-02T18:03:37.674 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a 2024-08-02T18:03:37.847 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:37 smithi077 ceph-mon[37108]: from='client.? 172.21.15.69:0/2877089890' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:37.848 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:37 smithi077 ceph-mon[37108]: osdmap e121: 12 total, 12 up, 12 in 2024-08-02T18:03:38.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:37 smithi069 systemd[1]: Stopping Ceph mon.a for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:03:38.014 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:37 smithi069 ceph-mon[32746]: from='client.? 172.21.15.69:0/2877089890' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:38.015 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:37 smithi069 ceph-mon[32746]: osdmap e121: 12 total, 12 up, 12 in 2024-08-02T18:03:38.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:37 smithi083 ceph-mon[37031]: from='client.? 172.21.15.69:0/2877089890' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-02T18:03:38.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:37 smithi083 ceph-mon[37031]: osdmap e121: 12 total, 12 up, 12 in 2024-08-02T18:03:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:38 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a[32742]: 2024-08-02T18:03:38.011+0000 7f13d0838640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:03:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:38 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a[32742]: 2024-08-02T18:03:38.011+0000 7f13d0838640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-02T18:03:38.358 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:38 smithi069 podman[126049]: 2024-08-02 18:03:38.227984954 +0000 UTC m=+0.333254946 container died fe3a259ab4b38fbcaa4e7cded27afcd0823751885d63f84dfd20b811e99e1f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8) 2024-08-02T18:03:38.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:38 smithi069 podman[126049]: 2024-08-02 18:03:38.635846283 +0000 UTC m=+0.741116281 container cleanup fe3a259ab4b38fbcaa4e7cded27afcd0823751885d63f84dfd20b811e99e1f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-02T18:03:38.858 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:38 smithi069 bash[126049]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a 2024-08-02T18:03:39.197 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a.service' 2024-08-02T18:03:39.237 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:38 smithi069 podman[126060]: 2024-08-02 18:03:38.970804149 +0000 UTC m=+0.732014150 container remove fe3a259ab4b38fbcaa4e7cded27afcd0823751885d63f84dfd20b811e99e1f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-02T18:03:39.237 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:39 smithi069 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-02T18:03:39.237 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:39 smithi069 systemd[1]: Stopped Ceph mon.a for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T18:03:39.237 INFO:journalctl@ceph.mon.a.smithi069.stdout:Aug 02 18:03:39 smithi069 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.a.service: Consumed 25.301s CPU time. 2024-08-02T18:03:39.291 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:03:39.291 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-02T18:03:39.291 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-02T18:03:39.291 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.b 2024-08-02T18:03:39.598 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:39 smithi077 systemd[1]: Stopping Ceph mon.b for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:03:40.288 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:39 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-b[37104]: 2024-08-02T18:03:39.894+0000 7f98f35d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:03:40.288 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:39 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-b[37104]: 2024-08-02T18:03:39.894+0000 7f98f35d1640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-02T18:03:40.288 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:39 smithi077 podman[73884]: 2024-08-02 18:03:39.946007207 +0000 UTC m=+0.174211457 container died 9105eb6fade4278f57a21dc5a8517545d57f90524d233437a18453f74d487b15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0) 2024-08-02T18:03:40.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:40 smithi077 podman[73884]: 2024-08-02 18:03:40.386877893 +0000 UTC m=+0.615082140 container cleanup 9105eb6fade4278f57a21dc5a8517545d57f90524d233437a18453f74d487b15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-02T18:03:40.597 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:40 smithi077 bash[73884]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-b 2024-08-02T18:03:41.155 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.b.service' 2024-08-02T18:03:41.176 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:40 smithi077 podman[73898]: 2024-08-02 18:03:40.913601302 +0000 UTC m=+0.970087151 container remove 9105eb6fade4278f57a21dc5a8517545d57f90524d233437a18453f74d487b15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-02T18:03:41.176 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:41 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.b.service: Deactivated successfully. 2024-08-02T18:03:41.176 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:41 smithi077 systemd[1]: Stopped Ceph mon.b for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T18:03:41.176 INFO:journalctl@ceph.mon.b.smithi077.stdout:Aug 02 18:03:41 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.b.service: Consumed 15.742s CPU time. 2024-08-02T18:03:41.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:03:41.212 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-02T18:03:41.212 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-02T18:03:41.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.c 2024-08-02T18:03:41.521 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:41 smithi083 systemd[1]: Stopping Ceph mon.c for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:03:42.361 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:42 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-c[37026]: 2024-08-02T18:03:42.093+0000 7f386249a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:03:42.361 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:42 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-c[37026]: 2024-08-02T18:03:42.093+0000 7f386249a640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-02T18:03:42.663 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:42 smithi083 podman[73922]: 2024-08-02 18:03:42.360804471 +0000 UTC m=+0.703659080 container died f8813bf2428c59769581a26e075798c35ea5e3f6fac6237f51109eded38d2781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-08-02T18:03:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:42 smithi083 podman[73922]: 2024-08-02 18:03:42.805744925 +0000 UTC m=+1.148599507 container cleanup f8813bf2428c59769581a26e075798c35ea5e3f6fac6237f51109eded38d2781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-c, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-02T18:03:43.021 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:42 smithi083 bash[73922]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-c 2024-08-02T18:03:43.669 INFO:journalctl@ceph.mon.c.smithi083.stdout:Aug 02 18:03:43 smithi083 podman[73936]: 2024-08-02 18:03:43.372463301 +0000 UTC m=+1.010790767 container remove f8813bf2428c59769581a26e075798c35ea5e3f6fac6237f51109eded38d2781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2) 2024-08-02T18:03:43.689 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mon.c.service' 2024-08-02T18:03:43.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:03:43.732 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-02T18:03:43.732 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-02T18:03:43.732 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.x 2024-08-02T18:03:44.108 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 18:03:43 smithi069 systemd[1]: Stopping Ceph mgr.x for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:03:46.358 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 18:03:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x[32976]: 2024-08-02T18:03:45.932+0000 7fbb6ddd2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:03:46.359 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 18:03:45 smithi069 podman[126168]: 2024-08-02 18:03:45.974629695 +0000 UTC m=+1.860678875 container died a2cd66d57a9b523e66414b87e6cdc3da67f3ad7c2cbf76b1c5d4d71679d7d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-02T18:03:51.826 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 18:03:51 smithi069 podman[126168]: 2024-08-02 18:03:51.502395531 +0000 UTC m=+7.388444707 container cleanup a2cd66d57a9b523e66414b87e6cdc3da67f3ad7c2cbf76b1c5d4d71679d7d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-02T18:03:51.826 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 18:03:51 smithi069 bash[126168]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x 2024-08-02T18:03:52.098 INFO:journalctl@ceph.mgr.x.smithi069.stdout:Aug 02 18:03:51 smithi069 podman[126182]: 2024-08-02 18:03:51.937813507 +0000 UTC m=+5.961034789 container remove a2cd66d57a9b523e66414b87e6cdc3da67f3ad7c2cbf76b1c5d4d71679d7d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:26c3fb8e197dcf7a49a54d1f4c8a7362ee35a8ea, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-02T18:03:52.209 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.x.service' 2024-08-02T18:03:52.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:03:52.245 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-02T18:03:52.246 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-02T18:03:52.246 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.y 2024-08-02T18:03:52.597 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 18:03:52 smithi077 systemd[1]: Stopping Ceph mgr.y for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:03:52.847 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 18:03:52 smithi077 podman[74019]: 2024-08-02 18:03:52.596167187 +0000 UTC m=+0.146968541 container died 8a98cc974e1cfbe7a5c9249dd7c004bde53ed7586595c93c7c99022ba9d4441b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-08-02T18:03:53.328 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 18:03:53 smithi077 podman[74019]: 2024-08-02 18:03:53.000728603 +0000 UTC m=+0.551530033 container cleanup 8a98cc974e1cfbe7a5c9249dd7c004bde53ed7586595c93c7c99022ba9d4441b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-02T18:03:53.328 INFO:journalctl@ceph.mgr.y.smithi077.stdout:Aug 02 18:03:53 smithi077 bash[74019]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-y 2024-08-02T18:03:53.453 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.y.service' 2024-08-02T18:03:53.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:03:53.490 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-02T18:03:53.490 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-02T18:03:53.491 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.z 2024-08-02T18:03:53.771 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:53 smithi083 systemd[1]: Stopping Ceph mgr.z for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:03:54.170 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:53 smithi083 podman[74056]: 2024-08-02 18:03:53.830262776 +0000 UTC m=+0.143263163 container died b13768607c21e124e8536cea0004f9738c0d1b99f9c36ad9dd19bd6b843ef361 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-02T18:03:54.513 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:54 smithi083 podman[74056]: 2024-08-02 18:03:54.268446206 +0000 UTC m=+0.581446593 container cleanup b13768607c21e124e8536cea0004f9738c0d1b99f9c36ad9dd19bd6b843ef361 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-02T18:03:54.513 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:54 smithi083 bash[74056]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z 2024-08-02T18:03:54.746 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.z.service' 2024-08-02T18:03:54.766 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:54 smithi083 podman[74068]: 2024-08-02 18:03:54.511999493 +0000 UTC m=+0.669301490 container remove b13768607c21e124e8536cea0004f9738c0d1b99f9c36ad9dd19bd6b843ef361 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8) 2024-08-02T18:03:54.766 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:54 smithi083 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-08-02T18:03:54.766 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:54 smithi083 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-08-02T18:03:54.766 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:54 smithi083 systemd[1]: Stopped Ceph mgr.z for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T18:03:54.766 INFO:journalctl@ceph.mgr.z.smithi083.stdout:Aug 02 18:03:54 smithi083 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mgr.z.service: Consumed 11.645s CPU time. 2024-08-02T18:03:54.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:03:54.781 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-02T18:03:54.781 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-02T18:03:54.781 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.0 2024-08-02T18:03:55.108 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:03:54 smithi069 systemd[1]: Stopping Ceph osd.0 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:03:55.608 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:03:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0[44036]: 2024-08-02T18:03:55.131+0000 7fb74dd24640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:03:55.608 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:03:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0[44036]: 2024-08-02T18:03:55.131+0000 7fb74dd24640 -1 osd.0 121 *** Got signal Terminated *** 2024-08-02T18:03:55.608 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:03:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0[44036]: 2024-08-02T18:03:55.131+0000 7fb74dd24640 -1 osd.0 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:04:01.100 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:00 smithi069 podman[126300]: 2024-08-02 18:04:00.810097549 +0000 UTC m=+5.832309453 container died a38ea498500bc8af8ef1821a2da59ffea335a7d42e8d9f286d2a6772741522ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T18:04:01.358 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:01 smithi069 podman[126300]: 2024-08-02 18:04:01.290859414 +0000 UTC m=+6.313071316 container cleanup a38ea498500bc8af8ef1821a2da59ffea335a7d42e8d9f286d2a6772741522ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True) 2024-08-02T18:04:01.358 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:01 smithi069 bash[126300]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0 2024-08-02T18:04:01.858 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:01 smithi069 podman[126319]: 2024-08-02 18:04:01.51772553 +0000 UTC m=+0.668549383 container remove a38ea498500bc8af8ef1821a2da59ffea335a7d42e8d9f286d2a6772741522ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-08-02T18:04:02.213 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:01 smithi069 podman[126406]: 2024-08-02 18:04:01.828206875 +0000 UTC m=+0.017121744 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:04:02.213 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:01 smithi069 podman[126406]: 2024-08-02 18:04:01.95194984 +0000 UTC m=+0.140864689 container create cb2cad3c4b6c61236831b85ee595633f968836e3bd0d56703d51d0dbaa813b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-02T18:04:02.599 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:02 smithi069 podman[126406]: 2024-08-02 18:04:02.212691463 +0000 UTC m=+0.401606328 container init cb2cad3c4b6c61236831b85ee595633f968836e3bd0d56703d51d0dbaa813b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-02T18:04:02.599 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:02 smithi069 podman[126406]: 2024-08-02 18:04:02.216330197 +0000 UTC m=+0.405245048 container start cb2cad3c4b6c61236831b85ee595633f968836e3bd0d56703d51d0dbaa813b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-02T18:04:02.599 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:02 smithi069 podman[126406]: 2024-08-02 18:04:02.363188 +0000 UTC m=+0.552102861 container attach cb2cad3c4b6c61236831b85ee595633f968836e3bd0d56703d51d0dbaa813b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0-deactivate, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-02T18:04:02.599 INFO:journalctl@ceph.osd.0.smithi069.stdout:Aug 02 18:04:02 smithi069 podman[126406]: 2024-08-02 18:04:02.598590622 +0000 UTC m=+0.787505471 container died cb2cad3c4b6c61236831b85ee595633f968836e3bd0d56703d51d0dbaa813b06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-02T18:04:03.264 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.0.service' 2024-08-02T18:04:03.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:04:03.304 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-02T18:04:03.304 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-02T18:04:03.304 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.1 2024-08-02T18:04:03.597 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:03 smithi077 systemd[1]: Stopping Ceph osd.1 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:04:04.097 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:03 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1[41440]: 2024-08-02T18:04:03.630+0000 7f1ccf89a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:04:04.098 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:03 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1[41440]: 2024-08-02T18:04:03.630+0000 7f1ccf89a640 -1 osd.1 121 *** Got signal Terminated *** 2024-08-02T18:04:04.098 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:03 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1[41440]: 2024-08-02T18:04:03.630+0000 7f1ccf89a640 -1 osd.1 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:04:09.616 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:09 smithi077 podman[74146]: 2024-08-02 18:04:09.357989817 +0000 UTC m=+5.862734378 container died 4045e158d4a5f6f5762c4228fe78221fee5ca88c274d0f2d368536ca748b8d0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-02T18:04:10.024 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:09 smithi077 podman[74146]: 2024-08-02 18:04:09.73065287 +0000 UTC m=+6.235397426 container cleanup 4045e158d4a5f6f5762c4228fe78221fee5ca88c274d0f2d368536ca748b8d0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-02T18:04:10.025 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:09 smithi077 bash[74146]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1 2024-08-02T18:04:10.347 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:10 smithi077 podman[74161]: 2024-08-02 18:04:10.024134175 +0000 UTC m=+0.662428152 container remove 4045e158d4a5f6f5762c4228fe78221fee5ca88c274d0f2d368536ca748b8d0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-02T18:04:10.711 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:10 smithi077 podman[74250]: 2024-08-02 18:04:10.250611977 +0000 UTC m=+0.017843094 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:04:10.712 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:10 smithi077 podman[74250]: 2024-08-02 18:04:10.358522258 +0000 UTC m=+0.125753441 container create 9dbb8a03aeaff562015dc090b7370dd5eb77c9b8b3278b56d64359770d43dcf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-08-02T18:04:10.712 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:10 smithi077 podman[74250]: 2024-08-02 18:04:10.594782111 +0000 UTC m=+0.362013221 container init 9dbb8a03aeaff562015dc090b7370dd5eb77c9b8b3278b56d64359770d43dcf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-02T18:04:10.712 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:10 smithi077 podman[74250]: 2024-08-02 18:04:10.597992662 +0000 UTC m=+0.365223779 container start 9dbb8a03aeaff562015dc090b7370dd5eb77c9b8b3278b56d64359770d43dcf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-08-02T18:04:10.712 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:10 smithi077 podman[74250]: 2024-08-02 18:04:10.711095073 +0000 UTC m=+0.478326195 container attach 9dbb8a03aeaff562015dc090b7370dd5eb77c9b8b3278b56d64359770d43dcf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-02T18:04:11.306 INFO:journalctl@ceph.osd.1.smithi077.stdout:Aug 02 18:04:10 smithi077 podman[74250]: 2024-08-02 18:04:10.981898114 +0000 UTC m=+0.749129230 container died 9dbb8a03aeaff562015dc090b7370dd5eb77c9b8b3278b56d64359770d43dcf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T18:04:11.637 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.1.service' 2024-08-02T18:04:11.679 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:04:11.679 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-02T18:04:11.679 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-02T18:04:11.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.2 2024-08-02T18:04:11.998 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:11 smithi083 systemd[1]: Stopping Ceph osd.2 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:04:12.271 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:11 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2[42021]: 2024-08-02T18:04:11.997+0000 7f33b1a1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:04:12.271 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:11 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2[42021]: 2024-08-02T18:04:11.997+0000 7f33b1a1a640 -1 osd.2 121 *** Got signal Terminated *** 2024-08-02T18:04:12.271 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:11 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2[42021]: 2024-08-02T18:04:11.997+0000 7f33b1a1a640 -1 osd.2 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:04:18.000 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:17 smithi083 podman[74180]: 2024-08-02 18:04:17.705881107 +0000 UTC m=+5.837664386 container died 5190e289d3a6e4db69fee786dcb6ed06f261c5bf17e67d857339c6dc6bac80d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-02T18:04:18.271 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:18 smithi083 podman[74180]: 2024-08-02 18:04:18.106564565 +0000 UTC m=+6.238347846 container cleanup 5190e289d3a6e4db69fee786dcb6ed06f261c5bf17e67d857339c6dc6bac80d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-02T18:04:18.271 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:18 smithi083 bash[74180]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2 2024-08-02T18:04:18.770 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:18 smithi083 podman[74199]: 2024-08-02 18:04:18.40859668 +0000 UTC m=+0.676799518 container remove 5190e289d3a6e4db69fee786dcb6ed06f261c5bf17e67d857339c6dc6bac80d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-02T18:04:19.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:18 smithi083 podman[74285]: 2024-08-02 18:04:18.671967741 +0000 UTC m=+0.017127210 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:04:19.021 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:18 smithi083 podman[74285]: 2024-08-02 18:04:18.79284922 +0000 UTC m=+0.138008678 container create b3e4dffe78d66190aa6a4764b96b4f4f727f3acc1c9e2efbd5014b3b0f32f604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T18:04:19.521 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:19 smithi083 podman[74285]: 2024-08-02 18:04:19.045621139 +0000 UTC m=+0.390780609 container init b3e4dffe78d66190aa6a4764b96b4f4f727f3acc1c9e2efbd5014b3b0f32f604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-02T18:04:19.522 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:19 smithi083 podman[74285]: 2024-08-02 18:04:19.048624518 +0000 UTC m=+0.393783977 container start b3e4dffe78d66190aa6a4764b96b4f4f727f3acc1c9e2efbd5014b3b0f32f604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2-deactivate, ceph=True, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-08-02T18:04:19.522 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:19 smithi083 podman[74285]: 2024-08-02 18:04:19.179110749 +0000 UTC m=+0.524270225 container attach b3e4dffe78d66190aa6a4764b96b4f4f727f3acc1c9e2efbd5014b3b0f32f604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-02T18:04:19.815 INFO:journalctl@ceph.osd.2.smithi083.stdout:Aug 02 18:04:19 smithi083 podman[74285]: 2024-08-02 18:04:19.543209062 +0000 UTC m=+0.888368523 container died b3e4dffe78d66190aa6a4764b96b4f4f727f3acc1c9e2efbd5014b3b0f32f604 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-02T18:04:20.161 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.2.service' 2024-08-02T18:04:20.199 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:04:20.199 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-02T18:04:20.199 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-02T18:04:20.199 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.3 2024-08-02T18:04:20.529 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:20 smithi069 systemd[1]: Stopping Ceph osd.3 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:04:20.788 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3[50600]: 2024-08-02T18:04:20.527+0000 7ff5f9063640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:04:20.788 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3[50600]: 2024-08-02T18:04:20.527+0000 7ff5f9063640 -1 osd.3 121 *** Got signal Terminated *** 2024-08-02T18:04:20.788 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:20 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3[50600]: 2024-08-02T18:04:20.527+0000 7ff5f9063640 -1 osd.3 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:04:21.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:21.513+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:22.598 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:22.482+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:23.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:23 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:23.305+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:23.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:23.559+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:23.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:23.412+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:23.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:23.490+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:24.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:24 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:24.309+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:24.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:24.547+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:24.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:24.417+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:24.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:24 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:24.521+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:25.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:25 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:25.278+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:25.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:25.536+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:25.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:25.428+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:25.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:25 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:25.563+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:26.608 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:26 smithi069 podman[126521]: 2024-08-02 18:04:26.34755926 +0000 UTC m=+5.943114460 container died 241748359deba7b7915e49419267af13b25c6d2cf049edfe05f128cd5d94257d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-02T18:04:26.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:26 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:26.236+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:26.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:26.490+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:26.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:26.432+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:26.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:26 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:26.558+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:27.105 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:26 smithi069 podman[126521]: 2024-08-02 18:04:26.886263399 +0000 UTC m=+6.481818589 container cleanup 241748359deba7b7915e49419267af13b25c6d2cf049edfe05f128cd5d94257d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-08-02T18:04:27.105 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:26 smithi069 bash[126521]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3 2024-08-02T18:04:27.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:27 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:27.196+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:27.358 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:27 smithi069 podman[126549]: 2024-08-02 18:04:27.104511863 +0000 UTC m=+0.739073738 container remove 241748359deba7b7915e49419267af13b25c6d2cf049edfe05f128cd5d94257d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-02T18:04:27.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:27.504+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:27.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:27.469+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:27.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:27 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:27.527+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:27.858 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:27 smithi069 podman[126638]: 2024-08-02 18:04:27.456516554 +0000 UTC m=+0.017306610 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:04:27.858 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:27 smithi069 podman[126638]: 2024-08-02 18:04:27.572590343 +0000 UTC m=+0.133380440 container create 50638169da1ab8f15c339cfb7b57dd74efc5afb56a29b16e194cb56f47a08837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-08-02T18:04:28.186 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:27 smithi069 podman[126638]: 2024-08-02 18:04:27.858583356 +0000 UTC m=+0.419373407 container init 50638169da1ab8f15c339cfb7b57dd74efc5afb56a29b16e194cb56f47a08837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3-deactivate, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-02T18:04:28.187 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:27 smithi069 podman[126638]: 2024-08-02 18:04:27.862032918 +0000 UTC m=+0.422822972 container start 50638169da1ab8f15c339cfb7b57dd74efc5afb56a29b16e194cb56f47a08837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-08-02T18:04:28.187 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:27 smithi069 podman[126638]: 2024-08-02 18:04:27.950124727 +0000 UTC m=+0.510914792 container attach 50638169da1ab8f15c339cfb7b57dd74efc5afb56a29b16e194cb56f47a08837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-02T18:04:28.187 INFO:journalctl@ceph.osd.3.smithi069.stdout:Aug 02 18:04:28 smithi069 podman[126638]: 2024-08-02 18:04:28.057874385 +0000 UTC m=+0.618664437 container died 50638169da1ab8f15c339cfb7b57dd74efc5afb56a29b16e194cb56f47a08837 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-3-deactivate, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-02T18:04:28.575 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:28 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:28.185+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:28.706 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.3.service' 2024-08-02T18:04:28.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:04:28.748 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-02T18:04:28.748 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-02T18:04:28.748 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.4 2024-08-02T18:04:28.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:28.494+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:28.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:28.490+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:28.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:28 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:28.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:28.848 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:28 smithi077 systemd[1]: Stopping Ceph osd.4 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:04:29.347 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:29 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4[48295]: 2024-08-02T18:04:29.087+0000 7f4e5befe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:04:29.348 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:29 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4[48295]: 2024-08-02T18:04:29.087+0000 7f4e5befe640 -1 osd.4 121 *** Got signal Terminated *** 2024-08-02T18:04:29.348 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:29 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4[48295]: 2024-08-02T18:04:29.087+0000 7f4e5befe640 -1 osd.4 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:04:29.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:29 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:29.185+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:29.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:29.535+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:29.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:29.514+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:29.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:29 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:29.593+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:30.260 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:29.843+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:30.521 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:30.507+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:30.521 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:30.508+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:30.521 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:30.508+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:30.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:30 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:30.228+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:30.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:30 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:30.201+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:31.024 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:30 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:30.596+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:31.024 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:30 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:30.596+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:31.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:30.890+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:31.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:31 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:31.022+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:31.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:31.196+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:31.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:31 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:31.185+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:31.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:31.465+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:31.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:31.506+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:31.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:31.506+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:31.991 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:31 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:31.640+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:31.991 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:31 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:31.640+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:32.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:31.879+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:32.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:31 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:31.989+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:32.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:32 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:32.228+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:32.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:32 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:32.150+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:32.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:32 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:32.476+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:32.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:32 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:32.460+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:32.772 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:32 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:32.460+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:33.028 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:32 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:32.642+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:33.028 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:32 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:32.642+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:33.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:32 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:32.884+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:33.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:33 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:33.027+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:33.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:33.248+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:33.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:33.172+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:33.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:33 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:33.172+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:33.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:33 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:33.462+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:33.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:33 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:33.505+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:33.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:33 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:33.505+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:34.043 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:33 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4[48295]: 2024-08-02T18:04:33.851+0000 7f4e58514640 -1 osd.4 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.994475+0000 front 2024-08-02T18:04:06.994430+0000 (oldest deadline 2024-08-02T18:04:32.894590+0000) 2024-08-02T18:04:34.043 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:33 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:33.667+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:34.043 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:33 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:33.667+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:34.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:33 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:33.893+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:34.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:34 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:34.041+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:34.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:34 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:34.270+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:34.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:34 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:34.204+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:34.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:34 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:34.204+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:34.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:34 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:34.493+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:34.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:34 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:34.539+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:34.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:34 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:34.539+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:34.994 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:34 smithi077 podman[74365]: 2024-08-02 18:04:34.876936319 +0000 UTC m=+5.937800945 container died 409bfd5fd9414899ae377a11080e8a5d23aa7740cfd52b4940a5ee23127276ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux ) 2024-08-02T18:04:34.995 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:34 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:34.631+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:34.995 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:34 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:34.631+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:35.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:34 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:34.880+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:35.318 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:34 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:34.993+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:35.597 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:35 smithi077 podman[74365]: 2024-08-02 18:04:35.318282339 +0000 UTC m=+6.379146964 container cleanup 409bfd5fd9414899ae377a11080e8a5d23aa7740cfd52b4940a5ee23127276ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-02T18:04:35.598 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:35 smithi077 bash[74365]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4 2024-08-02T18:04:35.598 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:35 smithi077 podman[74381]: 2024-08-02 18:04:35.561347995 +0000 UTC m=+0.683010974 container remove 409bfd5fd9414899ae377a11080e8a5d23aa7740cfd52b4940a5ee23127276ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-08-02T18:04:35.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:35 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:35.235+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:35.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:35 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:35.220+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:35.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:35 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:35.220+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:35.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:35 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:35.515+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:35.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:35 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:35.554+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:35.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:35 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:35.554+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:35.945 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:35 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:35.669+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:35.945 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:35 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:35.669+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:36.249 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:35 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:35.944+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:36.249 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:35 smithi077 podman[74469]: 2024-08-02 18:04:35.885222095 +0000 UTC m=+0.017147144 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:04:36.249 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:36 smithi077 podman[74469]: 2024-08-02 18:04:36.004057941 +0000 UTC m=+0.135982970 container create 01a56fa1b80168558ba68e9330c10b28547ef07b165d75a74f55394cddad0877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4-deactivate, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-02T18:04:36.249 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:36 smithi077 podman[74469]: 2024-08-02 18:04:36.24844904 +0000 UTC m=+0.380374070 container init 01a56fa1b80168558ba68e9330c10b28547ef07b165d75a74f55394cddad0877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4-deactivate, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-02T18:04:36.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:35 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:35.861+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:36.598 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:36 smithi077 podman[74469]: 2024-08-02 18:04:36.251637162 +0000 UTC m=+0.383562201 container start 01a56fa1b80168558ba68e9330c10b28547ef07b165d75a74f55394cddad0877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-02T18:04:36.598 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:36 smithi077 podman[74469]: 2024-08-02 18:04:36.348473255 +0000 UTC m=+0.480398294 container attach 01a56fa1b80168558ba68e9330c10b28547ef07b165d75a74f55394cddad0877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-02T18:04:36.598 INFO:journalctl@ceph.osd.4.smithi077.stdout:Aug 02 18:04:36 smithi077 podman[74469]: 2024-08-02 18:04:36.433799535 +0000 UTC m=+0.565724571 container died 01a56fa1b80168558ba68e9330c10b28547ef07b165d75a74f55394cddad0877 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8) 2024-08-02T18:04:36.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:36.225+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:36.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:36.247+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:36.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:36 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:36.247+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:36.895 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:36 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:36.560+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:36.895 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:36 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:36.550+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:36.895 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:36 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:36.550+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:37.044 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:36 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:36.912+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:37.044 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:36 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:36.696+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:37.045 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:36 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:36.696+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:37.065 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.4.service' 2024-08-02T18:04:37.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:04:37.103 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-02T18:04:37.103 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-02T18:04:37.103 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.5 2024-08-02T18:04:37.151 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:36 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:36.894+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:37.421 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:37 smithi083 systemd[1]: Stopping Ceph osd.5 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:04:37.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:37 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:37.261+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:37.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:37 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:37.256+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:37.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:37 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:37.256+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:37.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:37.584+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:37.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:37.420+0000 7facc5f35640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:04:37.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:37.420+0000 7facc5f35640 -1 osd.5 121 *** Got signal Terminated *** 2024-08-02T18:04:37.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:37.420+0000 7facc5f35640 -1 osd.5 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:04:37.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:37.515+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:37.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:37.515+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:37.936 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:37 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:37.653+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:37.937 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:37 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:37.653+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:38.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:37.942+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:38.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:37 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:37.942+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:38.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:37 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:37.935+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:38.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:38 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:38.264+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:38.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:38 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:38.231+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:38.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:38 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:38.231+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:38.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:38 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:38.231+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:38.984 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:38 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:38.624+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:38.984 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:38 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:38.624+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:38.985 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:38 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:38.624+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:38.992 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:38 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:38.543+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:38.992 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:38 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:38.543+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:38.992 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:38 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:38.536+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:38.992 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:38 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:38.536+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:39.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:38 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:38.990+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:39.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:38 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:38.990+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:39.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:38 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:38.983+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:39.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:39.287+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:39.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:39.287+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:39.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:39.247+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:39.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:39.247+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:39.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:39 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:39.247+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:39.982 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:39 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:39.581+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:39.982 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:39 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:39.581+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:39.982 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:39 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:39.522+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:39.983 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:39 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:39.522+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:39.988 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:39 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:39.603+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:39.988 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:39 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:39.603+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:39.988 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:39 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:39.603+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:40.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:39 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:39.980+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:40.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:39 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:39.980+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:40.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:39 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:39.986+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:40.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:40 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:40.263+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:40.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:40 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:40.263+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:40.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:40 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:40.216+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:40.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:40 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:40.216+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:40.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:40 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:40.216+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:40.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:40 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:40.482+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:40.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:40 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:40.482+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:40.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:40 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:40.537+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:40.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:40 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:40.537+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:40.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:40 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:40.594+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:40.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:40 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:40.594+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:40.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:40 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:40.594+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:41.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:40 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:40.975+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:41.512 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:41.023+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:41.512 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:41.023+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:41.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:41 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:41.288+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:41.609 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:41 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:41.288+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:41.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:41 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:41.173+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:41.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:41 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:41.173+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:41.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:41 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:41.173+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:41.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:41.545+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:41.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:41.545+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:41.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:41.510+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:58.154027+0000 front 2024-08-02T18:03:58.154150+0000 (oldest deadline 2024-08-02T18:04:23.453813+0000) 2024-08-02T18:04:41.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:41.510+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:03.954919+0000 front 2024-08-02T18:04:03.954893+0000 (oldest deadline 2024-08-02T18:04:30.355369+0000) 2024-08-02T18:04:41.771 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5[48718]: 2024-08-02T18:04:41.510+0000 7facbe543640 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.956624+0000 front 2024-08-02T18:04:14.956673+0000 (oldest deadline 2024-08-02T18:04:40.856695+0000) 2024-08-02T18:04:42.004 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:41 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:41.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:42.004 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:41 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:41.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:42.004 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:41 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:41.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:42.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:41.986+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:42.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:41 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:41.987+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:42.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:42 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:42.002+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:42.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:42 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:42.002+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:42.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:42 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:42.174+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:42.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:42 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:42.174+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:42.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:42 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:42.174+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:42.609 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:42 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:42.314+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:42.609 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:42 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:42.314+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:42.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:42 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:42.585+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:42.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:42 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:42.585+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:42.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:42 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:42.585+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:43.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:42 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:42.560+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:43.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:42 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:42.560+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:43.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:43 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:43.014+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:43.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:43 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:43.014+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:43.392 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:43 smithi083 podman[74402]: 2024-08-02 18:04:43.09306135 +0000 UTC m=+5.793339657 container died a13a07b0854e2a6c1c8e202cf97e48b433e92762bc09de276cf37d59624e4f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5, ceph=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-02T18:04:43.392 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:43 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:43.025+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:43.393 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:43 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:43.025+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:43.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:43 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:43.318+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:43.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:43 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:43.318+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:43.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:43 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:43.127+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:43.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:43 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:43.127+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:43.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:43 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:43.127+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:43.748 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:43 smithi083 podman[74402]: 2024-08-02 18:04:43.479121608 +0000 UTC m=+6.179399924 container cleanup a13a07b0854e2a6c1c8e202cf97e48b433e92762bc09de276cf37d59624e4f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T18:04:43.748 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:43 smithi083 bash[74402]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5 2024-08-02T18:04:43.748 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:43 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:43.524+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:43.749 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:43 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:43.524+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:43.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:43 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:43.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:43.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:43 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:43.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:43.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:43 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:43.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:44.021 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:43 smithi083 podman[74420]: 2024-08-02 18:04:43.747423021 +0000 UTC m=+0.642874892 container remove a13a07b0854e2a6c1c8e202cf97e48b433e92762bc09de276cf37d59624e4f00 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-02T18:04:44.021 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:43 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:43.976+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:44.021 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:43 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:43.976+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:44.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:44 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:44.018+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:44.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:44 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:44.018+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:44.352 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:44 smithi083 podman[74508]: 2024-08-02 18:04:43.974659221 +0000 UTC m=+0.017072531 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:04:44.352 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:44 smithi083 podman[74508]: 2024-08-02 18:04:44.098473422 +0000 UTC m=+0.140886733 container create e719a9acf340ec4b0cacbd5acc9e61ad93e50b6741987c458da076ba41ad6c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-02T18:04:44.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:44 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:44.275+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:44.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:44 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:44.275+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:44.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:44 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:44.146+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:44.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:44 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:44.146+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:44.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:44 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:44.146+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:44.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:44 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:44.496+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:44.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:44 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:44.496+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:44.772 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:44 smithi083 podman[74508]: 2024-08-02 18:04:44.351696149 +0000 UTC m=+0.394109461 container init e719a9acf340ec4b0cacbd5acc9e61ad93e50b6741987c458da076ba41ad6c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8) 2024-08-02T18:04:44.772 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:44 smithi083 podman[74508]: 2024-08-02 18:04:44.35491781 +0000 UTC m=+0.397331132 container start e719a9acf340ec4b0cacbd5acc9e61ad93e50b6741987c458da076ba41ad6c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-02T18:04:44.772 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:44 smithi083 podman[74508]: 2024-08-02 18:04:44.451100938 +0000 UTC m=+0.493514259 container attach e719a9acf340ec4b0cacbd5acc9e61ad93e50b6741987c458da076ba41ad6c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-08-02T18:04:44.772 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:44 smithi083 podman[74508]: 2024-08-02 18:04:44.543694684 +0000 UTC m=+0.586108005 container died e719a9acf340ec4b0cacbd5acc9e61ad93e50b6741987c458da076ba41ad6c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-02T18:04:44.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:44 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:44.571+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:44.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:44 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:44.571+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:44.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:44 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:44.571+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:45.155 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:45 smithi083 podman[74531]: 2024-08-02 18:04:45.130297426 +0000 UTC m=+0.580869854 container remove e719a9acf340ec4b0cacbd5acc9e61ad93e50b6741987c458da076ba41ad6c43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-02T18:04:45.155 INFO:journalctl@ceph.osd.5.smithi083.stdout:Aug 02 18:04:45 smithi083 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.5.service: Deactivated successfully. 2024-08-02T18:04:45.155 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:44 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:44.927+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:45.156 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:44 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:44.927+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:45.158 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.5.service' 2024-08-02T18:04:45.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:04:45.196 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-02T18:04:45.197 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-02T18:04:45.197 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.6 2024-08-02T18:04:45.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:44 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:44.986+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:45.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:44 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:44.986+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:45.524 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:45.119+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:45.524 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:45.119+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:45.525 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:45.119+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:45.525 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:45 smithi069 systemd[1]: Stopping Ceph osd.6 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:04:45.525 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:45.289+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:45.525 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:45.289+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:45.525 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:45.522+0000 7f919bc9b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:04:45.525 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:45.522+0000 7f919bc9b640 -1 osd.6 121 *** Got signal Terminated *** 2024-08-02T18:04:45.525 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:45.522+0000 7f919bc9b640 -1 osd.6 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:04:45.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:45 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:45.532+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:45.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:45 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:45.532+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:45.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:45 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:45.532+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:45.885 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:45 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:45.541+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:45.885 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:45 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:45.541+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:46.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:45 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:45.884+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:46.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:45 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:45.884+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:46.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:45 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:45.946+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:46.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:45 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:45.946+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:46.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:46 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:46.324+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:46.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:46 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:46.324+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:46.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:46 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:46.125+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:46.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:46 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:46.125+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:46.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:46 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:46.125+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:46.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:46 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:46.503+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:46.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:46 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:46.503+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:46.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:46 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:46.503+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:46.881 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:46 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:46.537+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:46.881 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:46 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:46.537+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:47.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:46 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:46.878+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:47.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:46 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:46.878+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:47.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:46 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:46.979+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:47.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:46 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:46.979+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:47.555 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:47 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:47.514+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:47.556 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:47 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:47.514+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:47.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:47 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:47.290+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:47.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:47 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:47.290+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:47.608 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:47 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:47.290+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.382385+0000 front 2024-08-02T18:04:22.382562+0000 (oldest deadline 2024-08-02T18:04:47.082504+0000) 2024-08-02T18:04:47.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:47 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:47.112+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:47.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:47 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:47.112+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:47.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:47 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:47.112+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:47.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:47 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:47.528+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:47.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:47 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:47.528+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:47.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:47 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:47.528+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:48.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:47 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:47.919+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:48.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:47 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:47.919+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:48.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:48.007+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:48.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:48.007+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:48.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:48 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:48.092+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:48.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:48 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:48.092+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:48.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:48 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:48.092+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:48.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:48 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:48.092+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:48.359 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:48 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:48.259+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:48.359 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:48 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:48.259+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:48.359 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:48 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:48.259+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.382385+0000 front 2024-08-02T18:04:22.382562+0000 (oldest deadline 2024-08-02T18:04:47.082504+0000) 2024-08-02T18:04:48.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:48 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:48.551+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:48.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:48 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:48.551+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:48.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:48.532+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:48.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:48.532+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:48.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:48.532+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:48.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:48.532+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:49.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:48 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:48.889+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:49.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:48 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:48.889+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:49.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:48.969+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:49.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:48.969+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:49.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:48 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:48.969+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:49.358 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:49.254+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:49.358 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:49.254+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:49.358 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:49.254+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.382385+0000 front 2024-08-02T18:04:22.382562+0000 (oldest deadline 2024-08-02T18:04:47.082504+0000) 2024-08-02T18:04:49.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:49.075+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:49.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:49.075+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:49.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:49.075+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:49.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:49.075+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:49.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:49 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:49.502+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:49.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:49 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:49.502+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:49.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:49.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:49.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:49.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:49.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:49.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:49.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:49.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:50.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:49 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:49.916+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:50.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:49 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:49.916+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:50.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:49 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:49.916+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:50.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:49.972+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:50.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:49.972+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:50.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:49 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:49.972+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:50.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:50 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:50.062+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:50.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:50 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:50.062+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:50.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:50 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:50.062+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:50.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:50 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:50.062+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:50.358 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:50 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:50.260+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.079516+0000 front 2024-08-02T18:04:06.079585+0000 (oldest deadline 2024-08-02T18:04:30.179715+0000) 2024-08-02T18:04:50.359 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:50 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:50.260+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:14.880710+0000 front 2024-08-02T18:04:14.880753+0000 (oldest deadline 2024-08-02T18:04:38.380867+0000) 2024-08-02T18:04:50.359 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:50 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6[57271]: 2024-08-02T18:04:50.260+0000 7f91982b1640 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.382385+0000 front 2024-08-02T18:04:22.382562+0000 (oldest deadline 2024-08-02T18:04:47.082504+0000) 2024-08-02T18:04:50.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:50 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:50.462+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:50.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:50 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:50.462+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:50.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:50 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:50.462+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:50.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:50.506+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:50.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:50.506+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:50.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:50.506+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:50.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:50.506+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:51.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:50 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:50.870+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:51.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:50 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:50.870+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:51.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:50 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:50.870+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:51.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:50.957+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:51.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:50.957+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:51.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:50 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:50.957+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:51.358 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:51 smithi069 podman[126739]: 2024-08-02 18:04:51.286080207 +0000 UTC m=+5.890409358 container died c3738a94ab16628c4c9e86f1366ce13f8674cd310d4637cfecbf087e7c099e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-02T18:04:51.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:51.052+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:51.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:51.052+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:51.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:51.052+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:51.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:51 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:51.052+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:51.618 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:51 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:51.492+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:51.618 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:51 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:51.492+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:51.618 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:51 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:51.492+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:51.618 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:51 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:51.492+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:51.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:51 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:51.496+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:51.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:51 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:51.496+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:51.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:51 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:51.496+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:52.092 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:51 smithi069 podman[126739]: 2024-08-02 18:04:51.823274362 +0000 UTC m=+6.427603499 container cleanup c3738a94ab16628c4c9e86f1366ce13f8674cd310d4637cfecbf087e7c099e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-08-02T18:04:52.092 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:51 smithi069 bash[126739]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6 2024-08-02T18:04:52.092 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:52 smithi069 podman[126766]: 2024-08-02 18:04:52.058077655 +0000 UTC m=+0.765036968 container remove c3738a94ab16628c4c9e86f1366ce13f8674cd310d4637cfecbf087e7c099e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-02T18:04:52.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:51 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:51.871+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:52.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:51 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:51.871+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:52.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:51 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:51.871+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:52.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:51 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:51.938+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:52.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:51 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:51.938+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:52.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:51 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:51.938+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:52.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:52 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:52.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:52.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:52 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:52.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:52.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:52 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:52.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:52.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:52 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:52.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:52.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:52.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:52.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:52.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:52.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:52.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:52.789 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:52 smithi069 podman[126859]: 2024-08-02 18:04:52.385910488 +0000 UTC m=+0.017618355 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:04:52.789 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:52 smithi069 podman[126859]: 2024-08-02 18:04:52.517615363 +0000 UTC m=+0.149323223 container create d952e734cbd73969b7a4d4d7f776fa5fd7998938b14108f6c01ddf0b6831f276 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-02T18:04:52.789 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:52 smithi069 podman[126859]: 2024-08-02 18:04:52.789091241 +0000 UTC m=+0.420799102 container init d952e734cbd73969b7a4d4d7f776fa5fd7998938b14108f6c01ddf0b6831f276 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T18:04:52.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:52 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:52.499+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:52.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:52 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:52.499+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:52.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:52 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:52.499+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:52.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:52 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:52.499+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:53.089 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:52 smithi069 podman[126859]: 2024-08-02 18:04:52.792511994 +0000 UTC m=+0.424219861 container start d952e734cbd73969b7a4d4d7f776fa5fd7998938b14108f6c01ddf0b6831f276 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6-deactivate, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-02T18:04:53.089 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:52 smithi069 podman[126859]: 2024-08-02 18:04:52.864031589 +0000 UTC m=+0.495739456 container attach d952e734cbd73969b7a4d4d7f776fa5fd7998938b14108f6c01ddf0b6831f276 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-02T18:04:53.089 INFO:journalctl@ceph.osd.6.smithi069.stdout:Aug 02 18:04:52 smithi069 podman[126859]: 2024-08-02 18:04:52.980697501 +0000 UTC m=+0.612405372 container died d952e734cbd73969b7a4d4d7f776fa5fd7998938b14108f6c01ddf0b6831f276 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-02T18:04:53.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:52.914+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:53.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:52.914+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:53.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:52 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:52.914+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:53.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:52 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:52.909+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:53.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:52 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:52.909+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:53.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:52 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:52.909+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:53.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:53.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:53.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:53.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:53.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:53.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:53.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:53 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:53.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:53.641 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.6.service' 2024-08-02T18:04:53.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:04:53.682 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-02T18:04:53.683 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-02T18:04:53.683 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.7 2024-08-02T18:04:53.737 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:53.477+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:53.737 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:53.477+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:53.737 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:53.477+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:53.738 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:53.477+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:53.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:53.424+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:53.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:53.424+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:53.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:53.424+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:54.014 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:53 smithi077 systemd[1]: Stopping Ceph osd.7 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:04:54.014 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:53.933+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:54.015 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:53.933+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:54.015 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:53.933+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:54.015 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:54.013+0000 7f2d82c27640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:04:54.274 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:53.888+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:54.274 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:53.888+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:54.274 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:53 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:53.888+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:54.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:54.013+0000 7f2d82c27640 -1 osd.7 121 *** Got signal Terminated *** 2024-08-02T18:04:54.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:54.013+0000 7f2d82c27640 -1 osd.7 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:04:54.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:54 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:54.041+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:54.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:54 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:54.041+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:54.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:54 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:54.041+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:54.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:54 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:54.041+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:54.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:54.446+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:54.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:54.446+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:54.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:54.446+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:54.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:54.497+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:54.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:54.497+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:54.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:54.497+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:54.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:54.497+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:55.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:54.885+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:55.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:54.885+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:55.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:54 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:54.885+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:55.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:54.929+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:55.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:54.929+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:55.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:54 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:54.929+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:55.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:55.046+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:55.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:55.046+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:55.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:55.046+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:55.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:55 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:55.046+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:55.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:55.466+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:55.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:55.466+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:55.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:55.466+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:55.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:55.521+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:55.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:55.521+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:55.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:55.521+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:55.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:55.521+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:56.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:55.913+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:56.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:55.913+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:56.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:55 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:55.913+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:56.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:55.881+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:56.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:55.881+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:56.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:55 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:55.881+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:56.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:56 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:56.094+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:56.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:56 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:56.094+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:56.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:56 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:56.094+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:56.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:56 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:56.094+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:56.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:56.476+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:56.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:56.476+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:56.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:56.476+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:56.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:56.526+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:56.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:56.526+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:56.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:56.526+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:56.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:56.526+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:56.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:56.526+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:04:57.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:56.957+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:57.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:56.957+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:57.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:56 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:56.957+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:57.347 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:56.895+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:57.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:56.895+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:57.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:56 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:56.895+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:57.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:57.083+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:57.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:57.083+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:57.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:57.083+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:57.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:57.083+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:57.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:57 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:57.083+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:04:57.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:57 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:57.481+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:57.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:57 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:57.481+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:57.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:57 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:57.481+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:57.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:57 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:57.481+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:04:57.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:57.487+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:57.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:57.487+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:57.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:57.487+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:57.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:57.487+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:57.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:57.487+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:04:58.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:57 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:57.941+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:58.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:57 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:57.941+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:58.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:57 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:57.941+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:58.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:57 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:57.941+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:04:58.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:57.941+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:58.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:57.941+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:58.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:57.941+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:58.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:57.941+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:32.886842+0000 front 2024-08-02T18:04:32.886903+0000 (oldest deadline 2024-08-02T18:04:57.587083+0000) 2024-08-02T18:04:58.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:58 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:58.095+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:58.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:58 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:58.095+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:58.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:58 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:58.095+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:58.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:58 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:58.095+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:58.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:58 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:58.095+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:04:58.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:58.479+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:58.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:58.479+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:58.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:58.479+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:58.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:58.479+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:04:58.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:58.522+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:58.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:58.522+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:58.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:58.522+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:58.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:58.522+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:58.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:58.522+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:04:59.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:58.918+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:04:59.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:58.918+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:04:59.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:58.918+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:04:59.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:58 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:58.918+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:04:59.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:58.952+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.783634+0000 front 2024-08-02T18:04:07.783670+0000 (oldest deadline 2024-08-02T18:04:30.083797+0000) 2024-08-02T18:04:59.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:58.952+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.984698+0000 front 2024-08-02T18:04:12.984926+0000 (oldest deadline 2024-08-02T18:04:41.785221+0000) 2024-08-02T18:04:59.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:58.952+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.886209+0000 front 2024-08-02T18:04:22.886358+0000 (oldest deadline 2024-08-02T18:04:48.186106+0000) 2024-08-02T18:04:59.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:58 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7[54779]: 2024-08-02T18:04:58.952+0000 7f2d7b235640 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:32.886842+0000 front 2024-08-02T18:04:32.886903+0000 (oldest deadline 2024-08-02T18:04:57.587083+0000) 2024-08-02T18:04:59.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:59 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:59.088+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:04:59.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:59 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:59.088+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:04:59.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:59 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:59.088+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:04:59.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:59 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:59.088+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:04:59.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:04:59 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:04:59.088+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:04:59.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:59 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:59.453+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:04:59.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:59 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:59.453+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:04:59.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:59 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:59.453+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:04:59.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:04:59 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:04:59.453+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:04:59.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:59 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:59.533+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:04:59.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:59 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:59.533+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:04:59.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:59 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:59.533+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:04:59.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:59 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:59.533+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:04:59.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:04:59 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:04:59.533+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:00.097 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:04:59 smithi077 podman[74584]: 2024-08-02 18:04:59.846377639 +0000 UTC m=+5.960699028 container died eea0c5da7b9db8b949a7725cbb5775a523a24b1444347ee4e62377872e2e403a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-02T18:05:00.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:59 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:59.874+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:05:00.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:59 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:59.874+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:05:00.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:59 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:59.874+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:05:00.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:04:59 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:04:59.874+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:05:00.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:00.071+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:00.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:00.071+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:00.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:00.071+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:00.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:00.071+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:00.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:00 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:00.071+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:00.490 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:00 smithi077 podman[74584]: 2024-08-02 18:05:00.230330922 +0000 UTC m=+6.344652369 container cleanup eea0c5da7b9db8b949a7725cbb5775a523a24b1444347ee4e62377872e2e403a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-02T18:05:00.491 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:00 smithi077 bash[74584]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7 2024-08-02T18:05:00.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:00 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:00.455+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:00.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:00 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:00.455+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:00.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:00 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:00.455+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:00.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:00 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:00.455+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:00.791 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:00 smithi077 podman[74604]: 2024-08-02 18:05:00.489862459 +0000 UTC m=+0.642246372 container remove eea0c5da7b9db8b949a7725cbb5775a523a24b1444347ee4e62377872e2e403a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-02T18:05:00.791 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:00 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:00.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:00.791 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:00 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:00.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:00.791 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:00 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:00.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:00.792 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:00 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:00.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:00.792 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:00 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:00.564+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:01.077 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:00 smithi077 podman[74689]: 2024-08-02 18:05:00.790814316 +0000 UTC m=+0.098551603 container create 5c91ca0055dede627b356350cd6250f8689cb958c55fd9355584095336e5dc0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-02T18:05:01.077 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:00 smithi077 podman[74689]: 2024-08-02 18:05:00.709259511 +0000 UTC m=+0.016996762 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:05:01.077 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:01 smithi077 podman[74689]: 2024-08-02 18:05:01.076654838 +0000 UTC m=+0.384392093 container init 5c91ca0055dede627b356350cd6250f8689cb958c55fd9355584095336e5dc0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-02T18:05:01.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:00 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:00.906+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:05:01.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:00 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:00.906+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:05:01.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:00 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:00.906+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:05:01.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:00 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:00.906+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:05:01.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:01 smithi077 podman[74689]: 2024-08-02 18:05:01.079815806 +0000 UTC m=+0.387553045 container start 5c91ca0055dede627b356350cd6250f8689cb958c55fd9355584095336e5dc0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-02T18:05:01.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:01 smithi077 podman[74689]: 2024-08-02 18:05:01.176520359 +0000 UTC m=+0.484257617 container attach 5c91ca0055dede627b356350cd6250f8689cb958c55fd9355584095336e5dc0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-02T18:05:01.348 INFO:journalctl@ceph.osd.7.smithi077.stdout:Aug 02 18:05:01 smithi077 podman[74689]: 2024-08-02 18:05:01.268823538 +0000 UTC m=+0.576560776 container died 5c91ca0055dede627b356350cd6250f8689cb958c55fd9355584095336e5dc0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-02T18:05:01.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:01 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:01.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:01.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:01 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:01.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:01.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:01 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:01.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:01.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:01 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:01.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:01.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:01 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:01.087+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:01.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:01 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:01.474+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:01.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:01 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:01.474+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:01.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:01 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:01.474+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:01.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:01 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:01.474+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:01.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:01 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:01.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:01.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:01 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:01.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:01.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:01 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:01.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:01.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:01 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:01.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:01.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:01 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:01.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:01.940 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.7.service' 2024-08-02T18:05:01.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:01.978 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-02T18:05:01.979 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-02T18:05:01.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.8 2024-08-02T18:05:02.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:01 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:01.902+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:05:02.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:01 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:01.902+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:05:02.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:01 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:01.902+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:05:02.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:01 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:01.902+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:05:02.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:02 smithi083 systemd[1]: Stopping Ceph osd.8 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:02.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:02 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:02.134+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:02.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:02 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:02.134+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:02.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:02 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:02.134+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:02.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:02 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:02.134+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:02.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:02 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:02.134+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:02.771 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:02.315+0000 7f8ce93a2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:02.771 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:02.315+0000 7f8ce93a2640 -1 osd.8 121 *** Got signal Terminated *** 2024-08-02T18:05:02.771 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:02.315+0000 7f8ce93a2640 -1 osd.8 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:05:02.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:02.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:02.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:02.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:02.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:02.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:02.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:02.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:02.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:02 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:02.571+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:02.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:02 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:02.571+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:02.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:02 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:02.571+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:02.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:02 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:02.571+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:02.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:02 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:02.571+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:03.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:02.949+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:05:03.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:02.949+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:05:03.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:02.949+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:05:03.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:02 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:02.949+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:05:03.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:03 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:03.145+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:03.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:03 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:03.145+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:03.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:03 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:03.145+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:03.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:03 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:03.145+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:03.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:03 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:03.145+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:03.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:03 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:03.481+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:03.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:03 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:03.481+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:03.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:03 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:03.481+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:03.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:03 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:03.481+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:04.097 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:03 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:03.602+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:04.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:03 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:03.602+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:04.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:03 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:03.602+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:04.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:03 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:03.602+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:04.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:03 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:03.602+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:04.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:03 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:03.915+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:05:04.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:03 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:03.915+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:05:04.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:03 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:03.915+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:05:04.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:03 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:03.915+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:05:04.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:04 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:04.098+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:04.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:04 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:04.098+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:04.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:04 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:04.098+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:04.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:04 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:04.098+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:04.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:04 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:04.098+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:04.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:04 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:04.461+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:04.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:04 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:04.461+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:04.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:04 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:04.461+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:04.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:04 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:04.461+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:04.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:04 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:04.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:04.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:04 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:04.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:04.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:04 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:04.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:04.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:04 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:04.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:04.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:04 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:04.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:05.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:04 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:04.954+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:05:05.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:04 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:04.954+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:05:05.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:04 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:04.954+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:05:05.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:04 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:04.954+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:05:05.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:05 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:05.129+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:05.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:05 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:05.129+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:05.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:05 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:05.129+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:05.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:05 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:05.129+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:05.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:05 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:05.129+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:05.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:05 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:05.129+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:05.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:05.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:05.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:05.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:05.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:05.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:05.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:05.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:06.097 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:05 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:05.614+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:06.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:05 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:05.614+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:06.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:05 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:05.614+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:06.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:05 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:05.614+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:06.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:05 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:05.614+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:06.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:05.950+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:05:06.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:05.950+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:05:06.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:05.950+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:05:06.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:05.950+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:05:06.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:05 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:05.950+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:42.319003+0000 front 2024-08-02T18:04:42.319047+0000 (oldest deadline 2024-08-02T18:05:05.219257+0000) 2024-08-02T18:05:06.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:06 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:06.135+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:06.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:06 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:06.135+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:06.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:06 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:06.135+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:06.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:06 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:06.135+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:06.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:06 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:06.135+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:06.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:06 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:06.135+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:06.773 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:06.390+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:06.774 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:06.390+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:06.774 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:06.390+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:06.774 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:06.390+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:07.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:06 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:06.621+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:07.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:06 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:06.621+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:07.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:06 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:06.621+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:07.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:06 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:06.621+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:07.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:06 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:06.621+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:07.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:06 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:06.621+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:07.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:06.982+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:04.414424+0000 front 2024-08-02T18:04:04.414646+0000 (oldest deadline 2024-08-02T18:04:29.614875+0000) 2024-08-02T18:05:07.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:06.982+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:13.115742+0000 front 2024-08-02T18:04:13.115717+0000 (oldest deadline 2024-08-02T18:04:37.815698+0000) 2024-08-02T18:05:07.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:06.982+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:20.716498+0000 front 2024-08-02T18:04:20.716333+0000 (oldest deadline 2024-08-02T18:04:48.916957+0000) 2024-08-02T18:05:07.272 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:06.982+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.518389+0000 front 2024-08-02T18:04:33.518203+0000 (oldest deadline 2024-08-02T18:04:57.618150+0000) 2024-08-02T18:05:07.272 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:06 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8[55503]: 2024-08-02T18:05:06.982+0000 7f8ce19b0640 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:42.319003+0000 front 2024-08-02T18:04:42.319047+0000 (oldest deadline 2024-08-02T18:05:05.219257+0000) 2024-08-02T18:05:07.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:07 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:07.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:07.358 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:07 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:07.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:07.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:07 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:07.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:07.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:07 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:07.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:07.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:07 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:07.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:07.359 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:07 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:07.090+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:07.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:07 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:07.416+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:07.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:07 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:07.416+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:07.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:07 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:07.416+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:07.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:07 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:07.416+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:07.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:07 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:07.416+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:07.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:07 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:07.575+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:07.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:07 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:07.575+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:07.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:07 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:07.575+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:07.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:07 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:07.575+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:07.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:07 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:07.575+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:07.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:07 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:07.575+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:08.271 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:07 smithi083 podman[74621]: 2024-08-02 18:05:07.960806076 +0000 UTC m=+5.782369785 container died c71cd84fac4119aa8f401b6ac1632ca13bdfb63b4258f04fa5d9dbad99144080 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-08-02T18:05:08.595 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:08 smithi083 podman[74621]: 2024-08-02 18:05:08.366442049 +0000 UTC m=+6.188005757 container cleanup c71cd84fac4119aa8f401b6ac1632ca13bdfb63b4258f04fa5d9dbad99144080 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-02T18:05:08.595 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:08 smithi083 bash[74621]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8 2024-08-02T18:05:08.595 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:08 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:08.407+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:08.595 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:08 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:08.407+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:08.595 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:08 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:08.407+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:08.596 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:08 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:08.407+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:08.596 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:08 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:08.407+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:08.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:08 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:08.123+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:08.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:08 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:08.123+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:08.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:08 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:08.123+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:08.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:08 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:08.123+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:08.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:08 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:08.123+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:08.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:08 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:08.123+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:08.903 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:08 smithi083 podman[74639]: 2024-08-02 18:05:08.593648646 +0000 UTC m=+0.633689042 container remove c71cd84fac4119aa8f401b6ac1632ca13bdfb63b4258f04fa5d9dbad99144080 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-02T18:05:09.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:08 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:08.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:09.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:08 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:08.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:09.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:08 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:08.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:09.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:08 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:08.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:09.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:08 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:08.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:09.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:08 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:08.608+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:09.234 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:08 smithi083 podman[74728]: 2024-08-02 18:05:08.903058915 +0000 UTC m=+0.110961148 container create 5e6ea6396ed14139abd7e113c64fea361d6bafd5cb036e89fb302b40238a6ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-02T18:05:09.235 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:08 smithi083 podman[74728]: 2024-08-02 18:05:08.809227946 +0000 UTC m=+0.017130152 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:05:09.235 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:09 smithi083 podman[74728]: 2024-08-02 18:05:09.130351965 +0000 UTC m=+0.338254162 container init 5e6ea6396ed14139abd7e113c64fea361d6bafd5cb036e89fb302b40238a6ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8-deactivate, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-02T18:05:09.235 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:09 smithi083 podman[74728]: 2024-08-02 18:05:09.133499782 +0000 UTC m=+0.341401970 container start 5e6ea6396ed14139abd7e113c64fea361d6bafd5cb036e89fb302b40238a6ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-02T18:05:09.235 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:09 smithi083 podman[74728]: 2024-08-02 18:05:09.23041326 +0000 UTC m=+0.438315458 container attach 5e6ea6396ed14139abd7e113c64fea361d6bafd5cb036e89fb302b40238a6ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-08-02T18:05:09.521 INFO:journalctl@ceph.osd.8.smithi083.stdout:Aug 02 18:05:09 smithi083 podman[74728]: 2024-08-02 18:05:09.323573525 +0000 UTC m=+0.531475717 container died 5e6ea6396ed14139abd7e113c64fea361d6bafd5cb036e89fb302b40238a6ddb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-02T18:05:09.522 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:09 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:09.372+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:09.522 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:09 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:09.372+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:09.522 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:09 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:09.372+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:09.522 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:09 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:09.372+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:09.522 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:09 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:09.372+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:09.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:09 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:09.142+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:09.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:09 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:09.142+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:09.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:09 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:09.142+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:09.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:09 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:09.142+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:09.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:09 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:09.142+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:09.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:09 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:09.142+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:09.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:09 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:09.573+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:09.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:09 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:09.573+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:09.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:09 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:09.573+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:09.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:09 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:09.573+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:09.849 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:09 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:09.573+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:09.849 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:09 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:09.573+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:10.052 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.8.service' 2024-08-02T18:05:10.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:10.093 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-02T18:05:10.093 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-02T18:05:10.093 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.9 2024-08-02T18:05:10.429 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.137+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:10.430 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.137+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:10.430 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.137+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:10.430 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.137+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:10.430 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.137+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:10.430 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.137+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:10.430 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 systemd[1]: Stopping Ceph osd.9 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:10.431 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.427+0000 7f87a0b6d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:10.431 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.427+0000 7f87a0b6d640 -1 osd.9 121 *** Got signal Terminated *** 2024-08-02T18:05:10.431 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:10 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:10.427+0000 7f87a0b6d640 -1 osd.9 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:05:10.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:10 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:10.406+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:10.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:10 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:10.406+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:10.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:10 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:10.406+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:10.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:10 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:10.406+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:10.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:10 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:10.406+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:10.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:10 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:10.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:10.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:10 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:10.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:10.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:10 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:10.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:10.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:10 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:10.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:10.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:10 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:10.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:10.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:10 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:10.552+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:11.521 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:11 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:11.363+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:11.521 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:11 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:11.363+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:11.521 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:11 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:11.363+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:11.521 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:11 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:11.363+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:11.521 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:11 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:11.363+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:11.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:11.164+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:11.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:11.164+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:11.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:11.164+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:11.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:11.164+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:11.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:11.164+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:11.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:11 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:11.164+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:11.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:11 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:11.593+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:11.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:11 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:11.593+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:11.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:11 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:11.593+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:11.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:11 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:11.593+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:11.849 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:11 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:11.593+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:11.849 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:11 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:11.593+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:12.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:12 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:12.139+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:12.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:12 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:12.139+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:12.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:12 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:12.139+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:12.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:12 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:12.139+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:12.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:12 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:12.139+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:12.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:12 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:12.139+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:12.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:12 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:12.350+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:12.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:12 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:12.350+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:12.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:12 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:12.350+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:12.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:12 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:12.350+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:12.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:12 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:12.350+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:12.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:12 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:12.350+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:12.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:12 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:12.543+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:12.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:12 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:12.543+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:12.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:12 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:12.543+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:12.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:12 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:12.543+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:12.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:12 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:12.543+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:12.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:12 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:12.543+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:13.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:13.179+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:13.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:13.179+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:13.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:13.179+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:13.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:13.179+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:13.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:13.179+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:13.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:13 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:13.179+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:13.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:13.383+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:13.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:13.383+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:13.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:13.383+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:13.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:13.383+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:13.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:13.383+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:13.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:13.383+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:13.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:13 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:13.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:13.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:13 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:13.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:13.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:13 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:13.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:13.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:13 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:13.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:13.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:13 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:13.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:13.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:13 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:13.570+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:14.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:14.185+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:14.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:14.185+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:14.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:14.185+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:14.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:14.185+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:14.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:14.185+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:14.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:14 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:14.185+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:14.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:14 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:14.377+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:14.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:14 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:14.377+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:14.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:14 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:14.377+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:14.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:14 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:14.377+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:14.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:14 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:14.377+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:14.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:14 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:14.377+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:14.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:14 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:14.579+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:14.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:14 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:14.579+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:14.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:14 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:14.579+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:14.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:14 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:14.579+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:14.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:14 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:14.579+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:14.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:14 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:14.579+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:15.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:15.213+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.702177+0000 front 2024-08-02T18:03:57.702230+0000 (oldest deadline 2024-08-02T18:04:23.002250+0000) 2024-08-02T18:05:15.608 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:15.213+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:07.504384+0000 front 2024-08-02T18:04:07.504474+0000 (oldest deadline 2024-08-02T18:04:32.804282+0000) 2024-08-02T18:05:15.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:15.213+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.905536+0000 front 2024-08-02T18:04:16.905450+0000 (oldest deadline 2024-08-02T18:04:38.005387+0000) 2024-08-02T18:05:15.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:15.213+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:23.206620+0000 front 2024-08-02T18:04:23.206522+0000 (oldest deadline 2024-08-02T18:04:47.306629+0000) 2024-08-02T18:05:15.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:15.213+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.807954+0000 front 2024-08-02T18:04:30.807910+0000 (oldest deadline 2024-08-02T18:04:56.107881+0000) 2024-08-02T18:05:15.609 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:15 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9[64050]: 2024-08-02T18:05:15.213+0000 7f879917b640 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.709563+0000 front 2024-08-02T18:04:40.709499+0000 (oldest deadline 2024-08-02T18:05:04.809299+0000) 2024-08-02T18:05:15.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:15 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:15.420+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:15.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:15 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:15.420+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:15.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:15 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:15.420+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:15.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:15 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:15.420+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:15.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:15 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:15.420+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:15.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:15 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:15.420+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:15.847 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:15 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:15.554+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:15.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:15 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:15.554+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:15.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:15 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:15.554+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:15.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:15 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:15.554+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:15.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:15 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:15.554+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:15.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:15 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:15.554+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:15.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:15 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:15.554+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:16.537 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:16 smithi069 podman[126962]: 2024-08-02 18:05:16.271169175 +0000 UTC m=+5.981910666 container died b0e950af058f559c0ca192121770e01a2d1070ae239d604198a5342e24bd31a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-02T18:05:16.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:16 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:16.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:16.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:16 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:16.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:16.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:16 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:16.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:16.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:16 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:16.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:16.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:16 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:16.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:16.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:16 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:16.414+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:16.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:16 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:16.509+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:16.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:16 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:16.509+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:16.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:16 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:16.509+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:16.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:16 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:16.509+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:16.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:16 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:16.509+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:16.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:16 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:16.509+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:16.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:16 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:16.509+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:16.858 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:16 smithi069 podman[126962]: 2024-08-02 18:05:16.711562702 +0000 UTC m=+6.422304183 container cleanup b0e950af058f559c0ca192121770e01a2d1070ae239d604198a5342e24bd31a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-02T18:05:16.858 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:16 smithi069 bash[126962]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9 2024-08-02T18:05:17.305 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:16 smithi069 podman[126977]: 2024-08-02 18:05:16.904758816 +0000 UTC m=+0.628983467 container remove b0e950af058f559c0ca192121770e01a2d1070ae239d604198a5342e24bd31a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-02T18:05:17.600 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:17 smithi069 podman[127066]: 2024-08-02 18:05:17.305362585 +0000 UTC m=+0.100022033 container create 717f8f545fd0e8551b8f44f977b4c70892c2a5cd96b769a59c6686cdaaeebe34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-02T18:05:17.600 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:17 smithi069 podman[127066]: 2024-08-02 18:05:17.222227313 +0000 UTC m=+0.016886765 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:05:17.600 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:17 smithi069 podman[127066]: 2024-08-02 18:05:17.52438813 +0000 UTC m=+0.319047598 container init 717f8f545fd0e8551b8f44f977b4c70892c2a5cd96b769a59c6686cdaaeebe34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9-deactivate, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-02T18:05:17.600 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:17 smithi069 podman[127066]: 2024-08-02 18:05:17.52776679 +0000 UTC m=+0.322426249 container start 717f8f545fd0e8551b8f44f977b4c70892c2a5cd96b769a59c6686cdaaeebe34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8) 2024-08-02T18:05:17.600 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:17 smithi069 podman[127066]: 2024-08-02 18:05:17.599336732 +0000 UTC m=+0.393996190 container attach 717f8f545fd0e8551b8f44f977b4c70892c2a5cd96b769a59c6686cdaaeebe34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-02T18:05:17.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:17 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:17.422+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:17.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:17 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:17.422+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:17.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:17 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:17.422+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:17.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:17 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:17.422+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:17.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:17 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:17.422+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:17.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:17 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:17.422+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:17.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:17 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:17.508+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:17.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:17 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:17.508+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:17.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:17 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:17.508+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:17.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:17 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:17.508+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:17.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:17 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:17.508+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:17.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:17 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:17.508+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:17.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:17 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:17.508+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:17.858 INFO:journalctl@ceph.osd.9.smithi069.stdout:Aug 02 18:05:17 smithi069 podman[127066]: 2024-08-02 18:05:17.710427241 +0000 UTC m=+0.505086693 container died 717f8f545fd0e8551b8f44f977b4c70892c2a5cd96b769a59c6686cdaaeebe34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-02T18:05:18.372 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.9.service' 2024-08-02T18:05:18.413 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:18.413 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-02T18:05:18.413 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-02T18:05:18.413 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.10 2024-08-02T18:05:18.742 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 systemd[1]: Stopping Ceph osd.10 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:18.742 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.545+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:18.742 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.545+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:18.742 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.545+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:18.743 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.545+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:18.743 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.545+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:18.743 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.545+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:18.743 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.545+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:18.743 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.740+0000 7fda45653640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:18.743 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.740+0000 7fda45653640 -1 osd.10 121 *** Got signal Terminated *** 2024-08-02T18:05:18.743 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:18 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:18.740+0000 7fda45653640 -1 osd.10 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:05:18.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:18 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:18.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:18.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:18 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:18.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:18.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:18 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:18.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:18.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:18 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:18.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:18.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:18 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:18.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:18.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:18 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:18.457+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:19.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:19 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:19.493+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:19.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:19 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:19.493+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:19.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:19 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:19.493+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:19.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:19 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:19.493+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:19.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:19 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:19.493+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:19.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:19 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:19.493+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:19.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:19.563+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:19.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:19.563+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:19.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:19.563+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:19.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:19.563+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:19.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:19.563+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:19.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:19.563+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:19.848 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:19 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:19.563+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:21.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:20.529+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:21.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:20.529+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:21.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:20.529+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:21.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:20.529+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:21.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:20.530+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:21.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:20 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:20.530+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:21.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:20.611+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:21.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:20.611+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:21.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:20.611+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:21.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:20.611+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:21.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:20.611+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:21.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:20.611+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:21.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:20 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:20.611+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:21.999 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:21.617+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:21.999 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:21.617+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:21.999 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:21.617+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:21.999 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:21.617+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:22.000 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:21.617+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:22.000 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:21.617+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:22.000 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:21 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:21.617+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:22.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:21.580+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:22.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:21.580+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:22.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:21.580+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:22.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:21.580+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:22.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:21.580+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:22.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:21.580+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:22.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:21.580+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:23.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:22.563+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:23.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:22.563+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:23.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:22.563+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:23.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:22.563+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:23.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:22.563+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:23.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:22.563+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:23.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:22 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:22.563+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:23.097 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:22.639+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:23.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:22.639+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:23.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:22.639+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:23.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:22.639+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:23.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:22.639+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:23.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:22.639+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:23.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:22 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:22.639+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:24.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:23.559+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:24.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:23.559+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:24.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:23.559+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:24.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:23.559+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:24.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:23.559+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:24.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:23.559+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:24.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:23 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:23.559+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:24.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:23.601+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:55.788170+0000 front 2024-08-02T18:03:55.788202+0000 (oldest deadline 2024-08-02T18:04:21.088367+0000) 2024-08-02T18:05:24.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:23.601+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6804 osd.1 since back 2024-08-02T18:04:06.989365+0000 front 2024-08-02T18:04:06.989388+0000 (oldest deadline 2024-08-02T18:04:30.489452+0000) 2024-08-02T18:05:24.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:23.601+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:16.890742+0000 front 2024-08-02T18:04:16.890769+0000 (oldest deadline 2024-08-02T18:04:37.990808+0000) 2024-08-02T18:05:24.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:23.601+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.091835+0000 front 2024-08-02T18:04:22.091754+0000 (oldest deadline 2024-08-02T18:04:48.492457+0000) 2024-08-02T18:05:24.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:23.601+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:33.093709+0000 front 2024-08-02T18:04:33.093675+0000 (oldest deadline 2024-08-02T18:04:55.993726+0000) 2024-08-02T18:05:24.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:23.601+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:40.094714+0000 front 2024-08-02T18:04:40.094559+0000 (oldest deadline 2024-08-02T18:05:05.894896+0000) 2024-08-02T18:05:24.098 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:23 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10[61301]: 2024-08-02T18:05:23.601+0000 7fda3dc61640 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:45.895357+0000 front 2024-08-02T18:04:45.895467+0000 (oldest deadline 2024-08-02T18:05:15.295822+0000) 2024-08-02T18:05:24.675 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:24 smithi077 podman[74794]: 2024-08-02 18:05:24.387727479 +0000 UTC m=+5.773002196 container died fdcc1f61f18abb23c73f2defdd80df62623d492afce98fd2b694bc714c5832cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-02T18:05:25.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:24.578+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:25.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:24.578+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:25.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:24.578+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:25.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:24.578+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:25.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:24.578+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:25.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:24.578+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:25.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:24 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:24.578+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:25.026 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:24 smithi077 podman[74794]: 2024-08-02 18:05:24.765639217 +0000 UTC m=+6.150913920 container cleanup fdcc1f61f18abb23c73f2defdd80df62623d492afce98fd2b694bc714c5832cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T18:05:25.026 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:24 smithi077 bash[74794]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10 2024-08-02T18:05:25.348 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:25 smithi077 podman[74822]: 2024-08-02 18:05:25.02549887 +0000 UTC m=+0.628164523 container remove fdcc1f61f18abb23c73f2defdd80df62623d492afce98fd2b694bc714c5832cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-02T18:05:25.687 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:25 smithi077 podman[74909]: 2024-08-02 18:05:25.281404066 +0000 UTC m=+0.017543173 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:05:25.687 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:25 smithi077 podman[74909]: 2024-08-02 18:05:25.384555451 +0000 UTC m=+0.120694597 container create 74ffc3bde09fc0d4a5e2d8ec140269c92b1f9676121e55b39d8ffc592f3f85d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-02T18:05:25.687 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:25 smithi077 podman[74909]: 2024-08-02 18:05:25.603381492 +0000 UTC m=+0.339520612 container init 74ffc3bde09fc0d4a5e2d8ec140269c92b1f9676121e55b39d8ffc592f3f85d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-08-02T18:05:25.687 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:25 smithi077 podman[74909]: 2024-08-02 18:05:25.606497252 +0000 UTC m=+0.342636365 container start 74ffc3bde09fc0d4a5e2d8ec140269c92b1f9676121e55b39d8ffc592f3f85d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-02T18:05:25.687 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:25 smithi077 podman[74909]: 2024-08-02 18:05:25.686632472 +0000 UTC m=+0.422771586 container attach 74ffc3bde09fc0d4a5e2d8ec140269c92b1f9676121e55b39d8ffc592f3f85d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10-deactivate, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-08-02T18:05:26.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:25.533+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:26.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:25.533+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:26.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:25.533+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:26.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:25.533+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:26.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:25.533+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:26.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:25.533+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:26.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:25 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:25.533+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:26.081 INFO:journalctl@ceph.osd.10.smithi077.stdout:Aug 02 18:05:25 smithi077 podman[74909]: 2024-08-02 18:05:25.792320227 +0000 UTC m=+0.528459335 container died 74ffc3bde09fc0d4a5e2d8ec140269c92b1f9676121e55b39d8ffc592f3f85d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-02T18:05:26.354 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.10.service' 2024-08-02T18:05:26.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:26.394 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-02T18:05:26.394 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-02T18:05:26.394 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.11 2024-08-02T18:05:26.688 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 systemd[1]: Stopping Ceph osd.11 for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:26.688 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.557+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:26.688 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.557+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:26.688 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.557+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:26.688 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.557+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:26.689 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.557+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:26.689 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.557+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:26.689 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.557+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:26.689 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.686+0000 7f8a185bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:26.689 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.686+0000 7f8a185bc640 -1 osd.11 121 *** Got signal Terminated *** 2024-08-02T18:05:26.689 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:26 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:26.686+0000 7f8a185bc640 -1 osd.11 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-02T18:05:28.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:27.525+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:28.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:27.525+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:28.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:27.525+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:28.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:27.525+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:28.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:27.525+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:28.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:27.525+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:28.022 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:27 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:27.525+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:28.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:28.482+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:28.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:28.482+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:28.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:28.482+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:28.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:28.482+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:28.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:28.482+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:28.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:28.482+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:28.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:28 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:28.482+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:29.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:29.490+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:29.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:29.490+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:29.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:29.490+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:29.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:29.490+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:29.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:29.490+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:29.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:29.490+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:29.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:29 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:29.490+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:30.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:30.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:30.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:30.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:30.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:30.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:30.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:30.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:30.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:30.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:30.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:30.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:30.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:30.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:30.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:30 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:30.518+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6820 osd.8 since back 2024-08-02T18:05:04.573350+0000 front 2024-08-02T18:05:04.573290+0000 (oldest deadline 2024-08-02T18:05:30.473289+0000) 2024-08-02T18:05:31.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:31.478+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6806 osd.0 since back 2024-08-02T18:03:57.261796+0000 front 2024-08-02T18:03:57.261669+0000 (oldest deadline 2024-08-02T18:04:22.561740+0000) 2024-08-02T18:05:31.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:31.478+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6804 osd.2 since back 2024-08-02T18:04:12.463947+0000 front 2024-08-02T18:04:12.463817+0000 (oldest deadline 2024-08-02T18:04:38.364050+0000) 2024-08-02T18:05:31.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:31.478+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6814 osd.3 since back 2024-08-02T18:04:22.866157+0000 front 2024-08-02T18:04:22.866191+0000 (oldest deadline 2024-08-02T18:04:49.866776+0000) 2024-08-02T18:05:31.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:31.478+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6812 osd.4 since back 2024-08-02T18:04:30.868269+0000 front 2024-08-02T18:04:30.868350+0000 (oldest deadline 2024-08-02T18:04:57.268966+0000) 2024-08-02T18:05:31.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:31.478+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6812 osd.5 since back 2024-08-02T18:04:41.969722+0000 front 2024-08-02T18:04:41.969748+0000 (oldest deadline 2024-08-02T18:05:06.669922+0000) 2024-08-02T18:05:31.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:31.478+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.69:6822 osd.6 since back 2024-08-02T18:04:46.670520+0000 front 2024-08-02T18:04:46.670507+0000 (oldest deadline 2024-08-02T18:05:11.370317+0000) 2024-08-02T18:05:31.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:31.478+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.77:6820 osd.7 since back 2024-08-02T18:04:55.872032+0000 front 2024-08-02T18:04:55.871991+0000 (oldest deadline 2024-08-02T18:05:21.171937+0000) 2024-08-02T18:05:31.772 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:31 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11[62166]: 2024-08-02T18:05:31.478+0000 7f8a10bca640 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.83:6820 osd.8 since back 2024-08-02T18:05:04.573350+0000 front 2024-08-02T18:05:04.573290+0000 (oldest deadline 2024-08-02T18:05:30.473289+0000) 2024-08-02T18:05:32.771 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:32 smithi083 podman[74841]: 2024-08-02 18:05:32.497847865 +0000 UTC m=+5.908178908 container died 2ab0b2648abd3a7fe5acfecea842f2575d89761fad70ec1fa0f664d6bb383cd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-08-02T18:05:33.135 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:32 smithi083 podman[74841]: 2024-08-02 18:05:32.886905572 +0000 UTC m=+6.297236618 container cleanup 2ab0b2648abd3a7fe5acfecea842f2575d89761fad70ec1fa0f664d6bb383cd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-02T18:05:33.135 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:32 smithi083 bash[74841]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11 2024-08-02T18:05:33.465 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:33 smithi083 podman[74859]: 2024-08-02 18:05:33.134136947 +0000 UTC m=+0.629980592 container remove 2ab0b2648abd3a7fe5acfecea842f2575d89761fad70ec1fa0f664d6bb383cd9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-02T18:05:33.746 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:33 smithi083 podman[74947]: 2024-08-02 18:05:33.366981368 +0000 UTC m=+0.016684944 image pull 564b849e9656f1ee75485c9661bf7ced18201970c51e80ee530a7cf1e40b1208 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6 2024-08-02T18:05:33.746 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:33 smithi083 podman[74947]: 2024-08-02 18:05:33.468260217 +0000 UTC m=+0.117963775 container create 10741c245442e77a82500d665ecc8aa77daa029028c223b95c8d5408e702aeef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True) 2024-08-02T18:05:33.746 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:33 smithi083 podman[74947]: 2024-08-02 18:05:33.687537907 +0000 UTC m=+0.337241482 container init 10741c245442e77a82500d665ecc8aa77daa029028c223b95c8d5408e702aeef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-02T18:05:33.747 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:33 smithi083 podman[74947]: 2024-08-02 18:05:33.690949248 +0000 UTC m=+0.340652807 container start 10741c245442e77a82500d665ecc8aa77daa029028c223b95c8d5408e702aeef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8) 2024-08-02T18:05:33.747 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:33 smithi083 podman[74947]: 2024-08-02 18:05:33.74563758 +0000 UTC m=+0.395341142 container attach 10741c245442e77a82500d665ecc8aa77daa029028c223b95c8d5408e702aeef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T18:05:34.021 INFO:journalctl@ceph.osd.11.smithi083.stdout:Aug 02 18:05:33 smithi083 podman[74947]: 2024-08-02 18:05:33.876948488 +0000 UTC m=+0.526652056 container died 10741c245442e77a82500d665ecc8aa77daa029028c223b95c8d5408e702aeef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-osd-11-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-08-02T18:05:34.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@osd.11.service' 2024-08-02T18:05:34.512 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:34.513 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-02T18:05:34.513 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-02T18:05:34.513 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.a 2024-08-02T18:05:34.826 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 18:05:34 smithi069 systemd[1]: Stopping Ceph mds.a for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:35.108 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 18:05:34 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a[70697]: 2024-08-02T18:05:34.832+0000 7f9480902640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:35.108 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 18:05:34 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a[70697]: 2024-08-02T18:05:34.832+0000 7f9480902640 -1 mds.a *** got signal Terminated *** 2024-08-02T18:05:39.155 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 18:05:38 smithi069 podman[127168]: 2024-08-02 18:05:38.886379142 +0000 UTC m=+4.176173431 container died 2d961c432555926381f9693337c9a3474933f3680dc06bdc1519a36399b4081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-02T18:05:39.522 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 18:05:39 smithi069 podman[127168]: 2024-08-02 18:05:39.312315325 +0000 UTC m=+4.602109615 container cleanup 2d961c432555926381f9693337c9a3474933f3680dc06bdc1519a36399b4081f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-02T18:05:39.523 INFO:journalctl@ceph.mds.a.smithi069.stdout:Aug 02 18:05:39 smithi069 bash[127168]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-a 2024-08-02T18:05:39.745 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.a.service' 2024-08-02T18:05:39.781 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:39.782 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-02T18:05:39.782 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-02T18:05:39.782 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.d 2024-08-02T18:05:40.108 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 18:05:39 smithi069 systemd[1]: Stopping Ceph mds.d for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:40.608 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 18:05:40 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d[69493]: 2024-08-02T18:05:40.133+0000 7f9068b2e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:40.608 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 18:05:40 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d[69493]: 2024-08-02T18:05:40.133+0000 7f9068b2e640 -1 mds.d *** got signal Terminated *** 2024-08-02T18:05:44.323 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 18:05:44 smithi069 podman[127303]: 2024-08-02 18:05:44.042461727 +0000 UTC m=+4.041317140 container died 89156af5b8463ba83b91ca7115134f50166762162393ad6c133694bb17be52bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-02T18:05:44.609 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 18:05:44 smithi069 podman[127303]: 2024-08-02 18:05:44.508458225 +0000 UTC m=+4.507313703 container cleanup 89156af5b8463ba83b91ca7115134f50166762162393ad6c133694bb17be52bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d, RELEASE=squid-release-26c3fb8, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-02T18:05:44.609 INFO:journalctl@ceph.mds.d.smithi069.stdout:Aug 02 18:05:44 smithi069 bash[127303]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-d 2024-08-02T18:05:44.942 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.d.service' 2024-08-02T18:05:44.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:44.976 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-02T18:05:44.976 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-02T18:05:44.976 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.g 2024-08-02T18:05:45.108 INFO:journalctl@ceph.mds.g.smithi069.stdout:Aug 02 18:05:45 smithi069 systemd[1]: Stopping Ceph mds.g for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:45.608 INFO:journalctl@ceph.mds.g.smithi069.stdout:Aug 02 18:05:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-g[70376]: 2024-08-02T18:05:45.278+0000 7ff2ce11e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:45.608 INFO:journalctl@ceph.mds.g.smithi069.stdout:Aug 02 18:05:45 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-g[70376]: 2024-08-02T18:05:45.278+0000 7ff2ce11e640 -1 mds.g *** got signal Terminated *** 2024-08-02T18:05:48.697 INFO:journalctl@ceph.mds.g.smithi069.stdout:Aug 02 18:05:48 smithi069 podman[127425]: 2024-08-02 18:05:48.4125203 +0000 UTC m=+3.220834298 container died 228cfeb24eb72f8f235da192718142ccb86c3bb17709577617129b3edbc3be07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-02T18:05:49.108 INFO:journalctl@ceph.mds.g.smithi069.stdout:Aug 02 18:05:48 smithi069 podman[127425]: 2024-08-02 18:05:48.862828247 +0000 UTC m=+3.671142278 container cleanup 228cfeb24eb72f8f235da192718142ccb86c3bb17709577617129b3edbc3be07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-02T18:05:49.108 INFO:journalctl@ceph.mds.g.smithi069.stdout:Aug 02 18:05:48 smithi069 bash[127425]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-g 2024-08-02T18:05:49.297 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.g.service' 2024-08-02T18:05:49.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:49.331 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-02T18:05:49.331 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-02T18:05:49.331 DEBUG:teuthology.orchestra.run.smithi069:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.j 2024-08-02T18:05:49.643 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 18:05:49 smithi069 systemd[1]: Stopping Ceph mds.j for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:50.108 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 18:05:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j[69094]: 2024-08-02T18:05:49.641+0000 7f91974e1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:50.108 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 18:05:49 smithi069 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j[69094]: 2024-08-02T18:05:49.641+0000 7f91974e1640 -1 mds.j *** got signal Terminated *** 2024-08-02T18:05:52.952 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 18:05:52 smithi069 podman[127547]: 2024-08-02 18:05:52.674404188 +0000 UTC m=+3.129147230 container died fec88f0b3439c29af92724fe38c2388885ebdb4ab9006e51f1aa3034525cde3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True) 2024-08-02T18:05:53.319 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 18:05:53 smithi069 podman[127547]: 2024-08-02 18:05:53.117336819 +0000 UTC m=+3.572079817 container cleanup fec88f0b3439c29af92724fe38c2388885ebdb4ab9006e51f1aa3034525cde3b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-02T18:05:53.319 INFO:journalctl@ceph.mds.j.smithi069.stdout:Aug 02 18:05:53 smithi069 bash[127547]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-j 2024-08-02T18:05:53.541 DEBUG:teuthology.orchestra.run.smithi069:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.j.service' 2024-08-02T18:05:53.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:53.574 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-02T18:05:53.574 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-02T18:05:53.574 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.b 2024-08-02T18:05:53.883 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:05:53 smithi077 systemd[1]: Stopping Ceph mds.b for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:54.140 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:05:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b[66767]: 2024-08-02T18:05:53.882+0000 7f4418dcc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:54.140 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:05:53 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b[66767]: 2024-08-02T18:05:53.882+0000 7f4418dcc640 -1 mds.b *** got signal Terminated *** 2024-08-02T18:05:56.759 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:05:56 smithi077 podman[75010]: 2024-08-02 18:05:56.473506288 +0000 UTC m=+2.698775406 container died 9e1c5a70448f106f7db84964931dd85acbee9967384809aebc8c8dfc55678052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-08-02T18:05:57.097 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:05:56 smithi077 podman[75010]: 2024-08-02 18:05:56.848930493 +0000 UTC m=+3.074199600 container cleanup 9e1c5a70448f106f7db84964931dd85acbee9967384809aebc8c8dfc55678052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-release-26c3fb8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-02T18:05:57.097 INFO:journalctl@ceph.mds.b.smithi077.stdout:Aug 02 18:05:56 smithi077 bash[75010]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-b 2024-08-02T18:05:57.270 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.b.service' 2024-08-02T18:05:57.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:57.305 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-02T18:05:57.305 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-02T18:05:57.305 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.e 2024-08-02T18:05:57.620 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:05:57 smithi077 systemd[1]: Stopping Ceph mds.e for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:05:58.097 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:05:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e[73039]: 2024-08-02T18:05:57.618+0000 7f2a67e06640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:05:58.097 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:05:57 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e[73039]: 2024-08-02T18:05:57.618+0000 7f2a67e06640 -1 mds.e *** got signal Terminated *** 2024-08-02T18:05:59.047 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:05:58 smithi077 podman[75143]: 2024-08-02 18:05:58.763563484 +0000 UTC m=+1.243523338 container died db41dd57f02febd9e4f588d52743106f158e85ec09fac49d6ebff7c8ded537a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-02T18:05:59.348 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:05:59 smithi077 podman[75143]: 2024-08-02 18:05:59.119509306 +0000 UTC m=+1.599469152 container cleanup db41dd57f02febd9e4f588d52743106f158e85ec09fac49d6ebff7c8ded537a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-02T18:05:59.348 INFO:journalctl@ceph.mds.e.smithi077.stdout:Aug 02 18:05:59 smithi077 bash[75143]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-e 2024-08-02T18:05:59.605 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.e.service' 2024-08-02T18:05:59.638 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:05:59.639 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-02T18:05:59.639 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-02T18:05:59.639 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.h 2024-08-02T18:06:00.097 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:05:59 smithi077 systemd[1]: Stopping Ceph mds.h for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:06:00.098 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:05:59 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h[66464]: 2024-08-02T18:05:59.939+0000 7f033bb25640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:06:00.098 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:05:59 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h[66464]: 2024-08-02T18:05:59.939+0000 7f033bb25640 -1 mds.h *** got signal Terminated *** 2024-08-02T18:06:04.291 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:06:03 smithi077 podman[75262]: 2024-08-02 18:06:03.954615764 +0000 UTC m=+4.098798028 container died 17ca429dd59d2e019838574f387b507ed3bba61bb468c2ebf0a6e171b9290c5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-release-26c3fb8, CEPH_POINT_RELEASE=) 2024-08-02T18:06:04.566 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:06:04 smithi077 podman[75262]: 2024-08-02 18:06:04.38997823 +0000 UTC m=+4.534160510 container cleanup 17ca429dd59d2e019838574f387b507ed3bba61bb468c2ebf0a6e171b9290c5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h, RELEASE=squid-release-26c3fb8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-02T18:06:04.567 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:06:04 smithi077 bash[75262]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h 2024-08-02T18:06:04.801 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.h.service' 2024-08-02T18:06:04.822 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:06:04 smithi077 podman[75275]: 2024-08-02 18:06:04.565994146 +0000 UTC m=+0.614302607 container remove 17ca429dd59d2e019838574f387b507ed3bba61bb468c2ebf0a6e171b9290c5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8) 2024-08-02T18:06:04.822 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:06:04 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.h.service: Deactivated successfully. 2024-08-02T18:06:04.822 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:06:04 smithi077 systemd[1]: Stopped Ceph mds.h for b6b04634-50f5-11ef-bcca-c7b262605968. 2024-08-02T18:06:04.822 INFO:journalctl@ceph.mds.h.smithi077.stdout:Aug 02 18:06:04 smithi077 systemd[1]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.h.service: Consumed 2.277s CPU time. 2024-08-02T18:06:04.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:06:04.833 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-02T18:06:04.833 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-02T18:06:04.833 DEBUG:teuthology.orchestra.run.smithi077:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.k 2024-08-02T18:06:05.097 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:06:04 smithi077 systemd[1]: Stopping Ceph mds.k for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:06:05.597 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:06:05 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k[65849]: 2024-08-02T18:06:05.185+0000 7fbd6d0ac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:06:05.597 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:06:05 smithi077 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k[65849]: 2024-08-02T18:06:05.185+0000 7fbd6d0ac640 -1 mds.k *** got signal Terminated *** 2024-08-02T18:06:08.795 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:06:08 smithi077 podman[75383]: 2024-08-02 18:06:08.463502965 +0000 UTC m=+3.418896361 container died 9c0962489c3bb2b9b1c82805acdf8341d72d42993f57e2ab4297df4420372a70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-02T18:06:09.097 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:06:08 smithi077 podman[75383]: 2024-08-02 18:06:08.876932983 +0000 UTC m=+3.832326377 container cleanup 9c0962489c3bb2b9b1c82805acdf8341d72d42993f57e2ab4297df4420372a70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-02T18:06:09.097 INFO:journalctl@ceph.mds.k.smithi077.stdout:Aug 02 18:06:08 smithi077 bash[75383]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-k 2024-08-02T18:06:09.310 DEBUG:teuthology.orchestra.run.smithi077:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.k.service' 2024-08-02T18:06:09.340 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:06:09.341 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-02T18:06:09.341 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-02T18:06:09.341 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.c 2024-08-02T18:06:09.633 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:06:09 smithi083 systemd[1]: Stopping Ceph mds.c for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:06:09.898 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:06:09 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c[67010]: 2024-08-02T18:06:09.631+0000 7fd098203640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:06:09.899 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:06:09 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c[67010]: 2024-08-02T18:06:09.631+0000 7fd098203640 -1 mds.c *** got signal Terminated *** 2024-08-02T18:06:12.771 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:06:12 smithi083 podman[75050]: 2024-08-02 18:06:12.511176351 +0000 UTC m=+2.977664506 container died 14dc08ae1bf9a07b51ef2542fcf3bc8c71c27a5fbfed81c51d590af975dcc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-02T18:06:13.265 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:06:12 smithi083 podman[75050]: 2024-08-02 18:06:12.857123333 +0000 UTC m=+3.323611498 container cleanup 14dc08ae1bf9a07b51ef2542fcf3bc8c71c27a5fbfed81c51d590af975dcc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-02T18:06:13.266 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:06:12 smithi083 bash[75050]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c 2024-08-02T18:06:13.266 INFO:journalctl@ceph.mds.c.smithi083.stdout:Aug 02 18:06:13 smithi083 podman[75076]: 2024-08-02 18:06:13.041708651 +0000 UTC m=+0.523193062 container remove 14dc08ae1bf9a07b51ef2542fcf3bc8c71c27a5fbfed81c51d590af975dcc37c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-c, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-release-26c3fb8, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-02T18:06:13.283 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.c.service' 2024-08-02T18:06:13.318 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:06:13.318 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-02T18:06:13.318 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-02T18:06:13.319 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.f 2024-08-02T18:06:13.620 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:06:13 smithi083 systemd[1]: Stopping Ceph mds.f for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:06:14.021 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:06:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f[66697]: 2024-08-02T18:06:13.619+0000 7fbc52ab3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:06:14.021 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:06:13 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f[66697]: 2024-08-02T18:06:13.619+0000 7fbc52ab3640 -1 mds.f *** got signal Terminated *** 2024-08-02T18:06:16.771 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:06:16 smithi083 podman[75181]: 2024-08-02 18:06:16.508311767 +0000 UTC m=+2.968967253 container died 51c056c7d810adbad40e2bddd54366451d60472557fe91932125dd85311d6bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-08-02T18:06:17.137 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:06:16 smithi083 podman[75181]: 2024-08-02 18:06:16.860837609 +0000 UTC m=+3.321493084 container cleanup 51c056c7d810adbad40e2bddd54366451d60472557fe91932125dd85311d6bf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, RELEASE=squid-release-26c3fb8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-02T18:06:17.137 INFO:journalctl@ceph.mds.f.smithi083.stdout:Aug 02 18:06:16 smithi083 bash[75181]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-f 2024-08-02T18:06:17.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.f.service' 2024-08-02T18:06:17.328 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:06:17.328 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-02T18:06:17.328 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-02T18:06:17.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.i 2024-08-02T18:06:17.657 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:06:17 smithi083 systemd[1]: Stopping Ceph mds.i for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:06:18.021 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:06:17 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i[67315]: 2024-08-02T18:06:17.656+0000 7f7c1d645640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:06:18.021 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:06:17 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i[67315]: 2024-08-02T18:06:17.656+0000 7f7c1d645640 -1 mds.i *** got signal Terminated *** 2024-08-02T18:06:20.771 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:06:20 smithi083 podman[75302]: 2024-08-02 18:06:20.515263445 +0000 UTC m=+2.966494353 container died 6170b510fee657e08a2d13982abb9619ee77a700a284f7bd1fd170bdcf802d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True) 2024-08-02T18:06:21.271 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:06:20 smithi083 podman[75302]: 2024-08-02 18:06:20.914774496 +0000 UTC m=+3.366005362 container cleanup 6170b510fee657e08a2d13982abb9619ee77a700a284f7bd1fd170bdcf802d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-release-26c3fb8) 2024-08-02T18:06:21.271 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:06:20 smithi083 bash[75302]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i 2024-08-02T18:06:21.271 INFO:journalctl@ceph.mds.i.smithi083.stdout:Aug 02 18:06:21 smithi083 podman[75315]: 2024-08-02 18:06:21.082644377 +0000 UTC m=+0.495378645 container remove 6170b510fee657e08a2d13982abb9619ee77a700a284f7bd1fd170bdcf802d78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-release-26c3fb8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-08-02T18:06:21.295 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.i.service' 2024-08-02T18:06:21.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:06:21.326 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-02T18:06:21.326 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-02T18:06:21.326 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.l 2024-08-02T18:06:21.652 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 18:06:21 smithi083 systemd[1]: Stopping Ceph mds.l for b6b04634-50f5-11ef-bcca-c7b262605968... 2024-08-02T18:06:22.021 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 18:06:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l[66377]: 2024-08-02T18:06:21.650+0000 7f6a12c65640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-02T18:06:22.021 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 18:06:21 smithi083 ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l[66377]: 2024-08-02T18:06:21.650+0000 7f6a12c65640 -1 mds.l *** got signal Terminated *** 2024-08-02T18:06:23.461 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 18:06:23 smithi083 podman[75421]: 2024-08-02 18:06:23.178359074 +0000 UTC m=+1.634141988 container died abe436486d2a7b5b5784aa49a82c10a8902245d6ad55f9dce489f45f9b763d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-release-26c3fb8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-02T18:06:23.771 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 18:06:23 smithi083 podman[75421]: 2024-08-02 18:06:23.55192696 +0000 UTC m=+2.007709878 container cleanup abe436486d2a7b5b5784aa49a82c10a8902245d6ad55f9dce489f45f9b763d49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6365e5c9c60465c6a86a1efc2ae80339db907d187de8bd717e7c6952210feca6, name=ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=squid-release-26c3fb8, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-02T18:06:23.771 INFO:journalctl@ceph.mds.l.smithi083.stdout:Aug 02 18:06:23 smithi083 bash[75421]: ceph-b6b04634-50f5-11ef-bcca-c7b262605968-mds-l 2024-08-02T18:06:23.970 DEBUG:teuthology.orchestra.run.smithi083:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6b04634-50f5-11ef-bcca-c7b262605968@mds.l.service' 2024-08-02T18:06:24.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-02T18:06:24.001 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-02T18:06:24.001 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6b04634-50f5-11ef-bcca-c7b262605968 --force --keep-logs 2024-08-02T18:06:24.207 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T18:06:26.750 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6b04634-50f5-11ef-bcca-c7b262605968 --force --keep-logs 2024-08-02T18:06:26.951 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T18:06:29.561 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6b04634-50f5-11ef-bcca-c7b262605968 --force --keep-logs 2024-08-02T18:06:29.763 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T18:06:32.161 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-02T18:06:32.194 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-02T18:06:32.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-02T18:06:32.251 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-02T18:06:32.253 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi069/crash 2024-08-02T18:06:32.254 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash -- . 2024-08-02T18:06:32.295 INFO:teuthology.orchestra.run.smithi069.stderr:tar: /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash: Cannot open: No such file or directory 2024-08-02T18:06:32.296 INFO:teuthology.orchestra.run.smithi069.stderr:tar: Error is not recoverable: exiting now 2024-08-02T18:06:32.297 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi077/crash 2024-08-02T18:06:32.298 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash -- . 2024-08-02T18:06:32.358 INFO:teuthology.orchestra.run.smithi077.stderr:tar: /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash: Cannot open: No such file or directory 2024-08-02T18:06:32.358 INFO:teuthology.orchestra.run.smithi077.stderr:tar: Error is not recoverable: exiting now 2024-08-02T18:06:32.365 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi083/crash 2024-08-02T18:06:32.366 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash -- . 2024-08-02T18:06:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:tar: /var/lib/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/crash: Cannot open: No such file or directory 2024-08-02T18:06:32.402 INFO:teuthology.orchestra.run.smithi083.stderr:tar: Error is not recoverable: exiting now 2024-08-02T18:06:32.403 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-02T18:06:32.404 DEBUG:teuthology.orchestra.run.smithi069:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-02T18:06:32.498 INFO:tasks.cephadm:Compressing logs... 2024-08-02T18:06:32.498 DEBUG:teuthology.orchestra.run.smithi069:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-02T18:06:32.501 DEBUG:teuthology.orchestra.run.smithi077:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-02T18:06:32.503 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-02T18:06:32.531 INFO:teuthology.orchestra.run.smithi069.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-02T18:06:32.532 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log 2024-08-02T18:06:32.532 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log 2024-08-02T18:06:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.g.log 2024-08-02T18:06:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log: /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.j.log 2024-08-02T18:06:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.3.log 2024-08-02T18:06:32.534 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.0.log 2024-08-02T18:06:32.536 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.x.log 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.a.log 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.6.log 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.a.log 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log: 86.8% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log.gz 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.d.log 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.9.log 2024-08-02T18:06:32.537 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-02T18:06:32.538 INFO:teuthology.orchestra.run.smithi083.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-02T18:06:32.539 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-08-02T18:06:32.539 INFO:teuthology.orchestra.run.smithi069.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 94.5% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.j.log.gz 2024-08-02T18:06:32.546 INFO:teuthology.orchestra.run.smithi069.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-08-02T18:06:32.546 INFO:teuthology.orchestra.run.smithi069.stderr: 94.5% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.a.log.gz 2024-08-02T18:06:32.546 INFO:teuthology.orchestra.run.smithi069.stderr: 94.6% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.d.log.gz 2024-08-02T18:06:32.549 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log 2024-08-02T18:06:32.549 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.2.log 2024-08-02T18:06:32.549 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.f.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.5.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.c.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.c.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.z.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.i.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.11.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.8.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.l.log 2024-08-02T18:06:32.550 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.7.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log: /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.b.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.4.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.1.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.y.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.h.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.10.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log 2024-08-02T18:06:32.551 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.e.log 2024-08-02T18:06:32.552 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log 2024-08-02T18:06:32.552 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.b.log 2024-08-02T18:06:32.552 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.k.log 2024-08-02T18:06:32.552 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-02T18:06:32.608 INFO:teuthology.orchestra.run.smithi077.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 93.2% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.k.log.gz 2024-08-02T18:06:32.626 INFO:teuthology.orchestra.run.smithi069.stderr: 94.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-02T18:06:32.630 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 82.1% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log.gz 2024-08-02T18:06:32.655 INFO:teuthology.orchestra.run.smithi077.stderr: 92.2% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.y.log.gz 2024-08-02T18:06:32.656 INFO:teuthology.orchestra.run.smithi069.stderr: 90.2% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log.gz 2024-08-02T18:06:32.664 INFO:teuthology.orchestra.run.smithi069.stderr: 84.5% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log.gz 2024-08-02T18:06:32.667 INFO:teuthology.orchestra.run.smithi069.stderr: 94.5% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.g.log.gz 2024-08-02T18:06:32.707 INFO:teuthology.orchestra.run.smithi083.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-02T18:06:32.723 INFO:teuthology.orchestra.run.smithi069.stderr: 91.0% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log.gz 2024-08-02T18:06:32.757 INFO:teuthology.orchestra.run.smithi077.stderr: 91.0% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log.gz 2024-08-02T18:06:32.765 INFO:teuthology.orchestra.run.smithi083.stderr: 90.3% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log.gz 2024-08-02T18:06:32.772 INFO:teuthology.orchestra.run.smithi083.stderr: 86.8% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log.gz 2024-08-02T18:06:32.777 INFO:teuthology.orchestra.run.smithi077.stderr: 90.3% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.audit.log.gz 2024-08-02T18:06:32.992 INFO:teuthology.orchestra.run.smithi083.stderr: 94.7% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.c.log.gz 2024-08-02T18:06:33.022 INFO:teuthology.orchestra.run.smithi083.stderr: 91.0% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-volume.log.gz 2024-08-02T18:06:33.034 INFO:teuthology.orchestra.run.smithi083.stderr: 94.4% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.l.log.gz 2024-08-02T18:06:33.100 INFO:teuthology.orchestra.run.smithi077.stderr: 82.1% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.cephadm.log.gz 2024-08-02T18:06:33.267 INFO:teuthology.orchestra.run.smithi083.stderr: 92.3% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.z.log.gz 2024-08-02T18:06:33.407 INFO:teuthology.orchestra.run.smithi077.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-02T18:06:33.710 INFO:teuthology.orchestra.run.smithi077.stderr: 86.8% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph.log.gz 2024-08-02T18:06:33.793 INFO:teuthology.orchestra.run.smithi077.stderr: 92.6% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.h.log.gz 2024-08-02T18:06:33.873 INFO:teuthology.orchestra.run.smithi083.stderr: 94.5% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.f.log.gz 2024-08-02T18:06:34.221 INFO:teuthology.orchestra.run.smithi083.stderr: 92.4% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.i.log.gz 2024-08-02T18:06:37.849 INFO:teuthology.orchestra.run.smithi069.stderr: 90.5% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mgr.x.log.gz 2024-08-02T18:06:39.778 INFO:teuthology.orchestra.run.smithi083.stderr: 91.5% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.c.log.gz 2024-08-02T18:06:42.409 INFO:teuthology.orchestra.run.smithi077.stderr: 91.6% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.b.log.gz 2024-08-02T18:06:43.109 INFO:teuthology.orchestra.run.smithi077.stderr: 94.8% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.b.log.gz 2024-08-02T18:06:57.057 INFO:teuthology.orchestra.run.smithi077.stderr: 94.2% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mds.e.log.gz 2024-08-02T18:07:11.010 INFO:teuthology.orchestra.run.smithi069.stderr: 90.3% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-mon.a.log.gz 2024-08-02T18:08:07.752 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.11.log.gz 2024-08-02T18:11:46.485 INFO:teuthology.orchestra.run.smithi083.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.8.log.gz 2024-08-02T18:11:50.739 INFO:teuthology.orchestra.run.smithi077.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.7.log.gz 2024-08-02T18:13:24.176 INFO:teuthology.orchestra.run.smithi077.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.10.log.gz 2024-08-02T18:14:13.941 INFO:teuthology.orchestra.run.smithi069.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.3.log.gz 2024-08-02T18:15:06.141 INFO:teuthology.orchestra.run.smithi077.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.1.log.gz 2024-08-02T18:16:09.808 INFO:teuthology.orchestra.run.smithi077.stderr: 91.1% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.4.log.gz 2024-08-02T18:16:09.809 INFO:teuthology.orchestra.run.smithi077.stderr: 2024-08-02T18:16:09.809 INFO:teuthology.orchestra.run.smithi077.stderr:real 9m37.292s 2024-08-02T18:16:09.809 INFO:teuthology.orchestra.run.smithi077.stderr:user 6m18.420s 2024-08-02T18:16:09.810 INFO:teuthology.orchestra.run.smithi077.stderr:sys 0m14.909s 2024-08-02T18:16:33.868 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.2.log.gz 2024-08-02T18:16:45.097 INFO:teuthology.orchestra.run.smithi069.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.0.log.gz 2024-08-02T18:17:06.936 INFO:teuthology.orchestra.run.smithi083.stderr: 90.8% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.5.log.gz 2024-08-02T18:17:06.938 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-08-02T18:17:06.938 INFO:teuthology.orchestra.run.smithi083.stderr:real 10m34.421s 2024-08-02T18:17:06.938 INFO:teuthology.orchestra.run.smithi083.stderr:user 6m50.799s 2024-08-02T18:17:06.938 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m15.541s 2024-08-02T18:21:26.228 INFO:teuthology.orchestra.run.smithi069.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.6.log.gz 2024-08-02T18:22:23.034 INFO:teuthology.orchestra.run.smithi069.stderr: 90.9% -- replaced with /var/log/ceph/b6b04634-50f5-11ef-bcca-c7b262605968/ceph-osd.9.log.gz 2024-08-02T18:22:23.036 INFO:teuthology.orchestra.run.smithi069.stderr: 2024-08-02T18:22:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:real 15m50.521s 2024-08-02T18:22:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:user 7m46.480s 2024-08-02T18:22:23.036 INFO:teuthology.orchestra.run.smithi069.stderr:sys 0m18.927s 2024-08-02T18:22:23.036 INFO:tasks.cephadm:Archiving logs... 2024-08-02T18:22:23.038 DEBUG:teuthology.misc:Transferring archived files from smithi069:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi069/log 2024-08-02T18:22:23.039 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-02T18:23:44.238 DEBUG:teuthology.misc:Transferring archived files from smithi077:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi077/log 2024-08-02T18:23:44.239 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-02T18:24:49.782 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi083/log 2024-08-02T18:24:49.784 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-02T18:26:01.043 INFO:tasks.cephadm:Removing cluster... 2024-08-02T18:26:01.044 DEBUG:teuthology.orchestra.run.smithi069:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6b04634-50f5-11ef-bcca-c7b262605968 --force 2024-08-02T18:26:01.252 INFO:teuthology.orchestra.run.smithi069.stdout:Deleting cluster with fsid: b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T18:26:02.064 DEBUG:teuthology.orchestra.run.smithi077:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6b04634-50f5-11ef-bcca-c7b262605968 --force 2024-08-02T18:26:02.292 INFO:teuthology.orchestra.run.smithi077.stdout:Deleting cluster with fsid: b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T18:26:03.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6b04634-50f5-11ef-bcca-c7b262605968 --force 2024-08-02T18:26:03.214 INFO:teuthology.orchestra.run.smithi083.stdout:Deleting cluster with fsid: b6b04634-50f5-11ef-bcca-c7b262605968 2024-08-02T18:26:03.945 INFO:tasks.cephadm:Removing cephadm ... 2024-08-02T18:26:03.945 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-02T18:26:03.965 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-02T18:26:04.236 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-02T18:26:04.252 INFO:tasks.cephadm:Teardown complete 2024-08-02T18:26:04.253 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-02T18:26:04.267 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-02T18:26:04.267 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-02T18:26:04.270 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-02T18:26:04.273 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-02T18:26:04.408 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-02T18:26:04.419 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-02T18:26:04.420 DEBUG:teuthology.orchestra.run.smithi069:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-02T18:26:04.423 DEBUG:teuthology.orchestra.run.smithi077:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-02T18:26:04.425 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-02T18:26:04.437 INFO:teuthology.orchestra.run.smithi069.stderr:bash: line 1: ntpq: command not found 2024-08-02T18:26:04.439 INFO:teuthology.orchestra.run.smithi077.stderr:bash: line 1: ntpq: command not found 2024-08-02T18:26:04.439 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-08-02T18:26:04.458 INFO:teuthology.orchestra.run.smithi077.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-02T18:26:04.458 INFO:teuthology.orchestra.run.smithi077.stdout:=============================================================================== 2024-08-02T18:26:04.458 INFO:teuthology.orchestra.run.smithi077.stdout:^* hv01.front.sepia.ceph.com 3 6 377 49 +912us[ +914us] +/- 57ms 2024-08-02T18:26:04.458 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -1354us[-1354us] +/- 52ms 2024-08-02T18:26:04.458 INFO:teuthology.orchestra.run.smithi077.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +672us[ +673us] +/- 73ms 2024-08-02T18:26:04.458 INFO:teuthology.orchestra.run.smithi077.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T18:26:04.462 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-02T18:26:04.462 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-08-02T18:26:04.462 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +861us[ +872us] +/- 57ms 2024-08-02T18:26:04.462 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 49 -1395us[-1395us] +/- 52ms 2024-08-02T18:26:04.462 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +636us[ +646us] +/- 73ms 2024-08-02T18:26:04.463 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T18:26:04.476 INFO:teuthology.orchestra.run.smithi069.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-02T18:26:04.476 INFO:teuthology.orchestra.run.smithi069.stdout:=============================================================================== 2024-08-02T18:26:04.476 INFO:teuthology.orchestra.run.smithi069.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +887us[ +891us] +/- 57ms 2024-08-02T18:26:04.476 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 52 -1387us[-1383us] +/- 52ms 2024-08-02T18:26:04.476 INFO:teuthology.orchestra.run.smithi069.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +652us[ +656us] +/- 73ms 2024-08-02T18:26:04.476 INFO:teuthology.orchestra.run.smithi069.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-02T18:26:04.477 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-02T18:26:04.495 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-02T18:26:04.496 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-02T18:26:04.513 DEBUG:teuthology.orchestra.run.smithi069:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-02T18:26:04.672 DEBUG:teuthology.orchestra.run.smithi077:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-02T18:26:04.819 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-02T18:26:04.913 DEBUG:teuthology.orchestra.run.smithi069:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-02T18:26:04.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T18:26:04.945 DEBUG:teuthology.orchestra.run.smithi077:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-02T18:26:04.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T18:26:04.976 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-08-02T18:26:05.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T18:26:05.003 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-02T18:26:05.016 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-02T18:26:05.069 INFO:teuthology.task.internal:Duration was 3378.830763 seconds 2024-08-02T18:26:05.069 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-02T18:26:05.081 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-02T18:26:05.081 DEBUG:teuthology.orchestra.run.smithi069:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-02T18:26:05.084 DEBUG:teuthology.orchestra.run.smithi077:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-02T18:26:05.087 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-02T18:26:05.136 INFO:teuthology.orchestra.run.smithi077.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-02T18:26:05.144 INFO:teuthology.orchestra.run.smithi069.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-02T18:26:05.168 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-02T18:26:05.573 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-02T18:26:05.573 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi069.front.sepia.ceph.com 2024-08-02T18:26:05.573 DEBUG:teuthology.orchestra.run.smithi069:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-02T18:26:05.596 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi077.front.sepia.ceph.com 2024-08-02T18:26:05.596 DEBUG:teuthology.orchestra.run.smithi077:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-02T18:26:05.629 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-08-02T18:26:05.629 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-02T18:26:05.668 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-02T18:26:05.668 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-02T18:26:05.671 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-02T18:26:05.673 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-02T18:26:05.737 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-02T18:26:05.737 DEBUG:teuthology.orchestra.run.smithi069:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-02T18:26:05.739 DEBUG:teuthology.orchestra.run.smithi077:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-02T18:26:05.742 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-02T18:26:07.430 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-02T18:26:07.443 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-02T18:26:07.443 DEBUG:teuthology.orchestra.run.smithi069:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-02T18:26:07.480 DEBUG:teuthology.orchestra.run.smithi077:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-02T18:26:07.535 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-02T18:26:07.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-02T18:26:07.577 DEBUG:teuthology.orchestra.run.smithi069:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-02T18:26:07.580 DEBUG:teuthology.orchestra.run.smithi077:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-02T18:26:07.582 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-02T18:26:07.618 INFO:teuthology.orchestra.run.smithi069.stdout:kernel.core_pattern = core 2024-08-02T18:26:07.625 INFO:teuthology.orchestra.run.smithi077.stdout:kernel.core_pattern = core 2024-08-02T18:26:07.630 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-08-02T18:26:07.684 DEBUG:teuthology.orchestra.run.smithi069:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-02T18:26:07.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T18:26:07.699 DEBUG:teuthology.orchestra.run.smithi077:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-02T18:26:07.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T18:26:07.715 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-02T18:26:07.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-02T18:26:07.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-02T18:26:07.752 INFO:teuthology.task.internal:Transferring archived files... 2024-08-02T18:26:07.753 DEBUG:teuthology.misc:Transferring archived files from smithi069:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi069 2024-08-02T18:26:07.753 DEBUG:teuthology.orchestra.run.smithi069:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-02T18:26:07.831 DEBUG:teuthology.misc:Transferring archived files from smithi077:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi077 2024-08-02T18:26:07.831 DEBUG:teuthology.orchestra.run.smithi077:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-02T18:26:07.871 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-08-02_15:32:26-fs-squid-release-distro-default-smithi/7832714/remote/smithi083 2024-08-02T18:26:07.871 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-02T18:26:07.910 INFO:teuthology.task.internal:Removing archive directory... 2024-08-02T18:26:07.911 DEBUG:teuthology.orchestra.run.smithi069:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-02T18:26:07.913 DEBUG:teuthology.orchestra.run.smithi077:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-02T18:26:07.915 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-02T18:26:07.961 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-02T18:26:07.975 INFO:teuthology.task.internal:Not uploading archives. 2024-08-02T18:26:07.975 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-02T18:26:08.038 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-02T18:26:08.038 DEBUG:teuthology.orchestra.run.smithi069:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-02T18:26:08.041 DEBUG:teuthology.orchestra.run.smithi077:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-02T18:26:08.044 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-02T18:26:08.053 INFO:teuthology.orchestra.run.smithi069.stdout: 265339 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 2 18:26 /home/ubuntu/cephtest 2024-08-02T18:26:08.057 INFO:teuthology.orchestra.run.smithi077.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 2 18:26 /home/ubuntu/cephtest 2024-08-02T18:26:08.060 INFO:teuthology.orchestra.run.smithi083.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 2 18:26 /home/ubuntu/cephtest 2024-08-02T18:26:08.061 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-02T18:26:08.077 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-02T18:26:08.113 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 3378.830762863159 failure_reason: 'The following counters failed to be set on mds daemons: {''mds.imported'', ''mds.exported''}' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-08-02T18:26:08.114 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-02T18:26:08.204 INFO:teuthology.run:FAIL